From pje at telecommunity.com Tue Sep 1 00:06:37 2009 From: pje at telecommunity.com (P.J. Eby) Date: Mon, 31 Aug 2009 18:06:37 -0400 Subject: [Python-Dev] how important is setting co_filename for a module being imported to what __file__ is set to? In-Reply-To: References: <20090831215249.A11393A4093@sparrow.telecommunity.com> Message-ID: <20090831220718.44AB53A4093@sparrow.telecommunity.com> At 02:57 PM 8/31/2009 -0700, Brett Cannon wrote: >Ignoring that 'new' is not in Python 3.x (luckily 'types' is), I want >a proper solution that doesn't require reconstructing every code >object that I happen to import. Practicality beats purity. ;-) (Especially if it allows importlib to run on older Pythons.) Also, surely you're not worried about *performance* here? From brett at python.org Tue Sep 1 00:12:43 2009 From: brett at python.org (Brett Cannon) Date: Mon, 31 Aug 2009 15:12:43 -0700 Subject: [Python-Dev] how important is setting co_filename for a module being imported to what __file__ is set to? In-Reply-To: <20090831220718.44AB53A4093@sparrow.telecommunity.com> References: <20090831215249.A11393A4093@sparrow.telecommunity.com> <20090831220718.44AB53A4093@sparrow.telecommunity.com> Message-ID: On Mon, Aug 31, 2009 at 15:06, P.J. Eby wrote: > At 02:57 PM 8/31/2009 -0700, Brett Cannon wrote: >> >> Ignoring that 'new' is not in Python 3.x (luckily 'types' is), I want >> a proper solution that doesn't require reconstructing every code >> object that I happen to import. > > Practicality beats purity. ?;-) ?(Especially if it allows importlib to run > on older Pythons.) > I don't care about making importlib run on older versions of Python before 3.1. And this is a minor enough thing that I am not worried about missing in Python 3.1. > Also, surely you're not worried about *performance* here? I do care about performance to an extent, but that is not the primary motivating factor to wanting to go with the marshal API change. -Brett From greg at krypto.org Tue Sep 1 00:07:49 2009 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 31 Aug 2009 15:07:49 -0700 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> Message-ID: <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> On Mon, Aug 31, 2009 at 2:20 PM, Antoine Pitrou wrote: > Gregory P. Smith krypto.org> writes: > > > > food for thought as noticed by a coworker who has been profiling some hot > code > to optimize a library...If a function does not have a return statement we > return > None. Ironically this makes the foo2 function below faster than the bar2 > function at least as measured using bytecode size > > I would be surprised if this "bytecode size" difference made a significant > difference in runtimes, given that function call cost should dwarf the > cumulated > cost of POP_TOP and LOAD_CONST (two of the simplest opcodes you could > find). > > the attached sample code repeatably shows that it makes a difference though its really not much of one (2-3%). I was just wondering if a bytecode for a superinstruction of the common sequence: 6 POP_TOP 7 LOAD_CONST 0 (None) 10 RETURN_VALUE might be worth it. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: none_speed.py Type: text/x-python Size: 619 bytes Desc: not available URL: From collinw at gmail.com Tue Sep 1 00:28:29 2009 From: collinw at gmail.com (Collin Winter) Date: Mon, 31 Aug 2009 15:28:29 -0700 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> Message-ID: <43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com> On Mon, Aug 31, 2009 at 3:07 PM, Gregory P. Smith wrote: > > > On Mon, Aug 31, 2009 at 2:20 PM, Antoine Pitrou wrote: >> >> Gregory P. Smith krypto.org> writes: >> > >> > food for thought as noticed by a coworker who has been profiling some >> > hot code >> to optimize a library...If a function does not have a return statement we >> return >> None.? Ironically this makes the foo2 function below faster than the bar2 >> function at least as measured using bytecode size >> >> I would be surprised if this "bytecode size" difference made a significant >> difference in runtimes, given that function call cost should dwarf the >> cumulated >> cost of POP_TOP and LOAD_CONST (two of the simplest opcodes you could >> find). >> > > the attached sample code repeatably shows that it makes a difference though > its really not much of one (2-3%). > > I was just wondering if a bytecode for a superinstruction of the common > sequence: > > ???????????? 6 POP_TOP > ????????????? 7 LOAD_CONST?????????????? 0 (None) > ???????????? 10 RETURN_VALUE > > might be worth it. I doubt it. You'd save a bit of stack manipulation, but since this will only appear at the end of a function, I'd be skeptical that this would make any macrobenchmarks (statistically) significantly faster. Collin Winter From benjamin at python.org Tue Sep 1 00:28:39 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 31 Aug 2009 17:28:39 -0500 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> Message-ID: <1afaf6160908311528g14065933k9085bcef9b9ab94@mail.gmail.com> 2009/8/31 Gregory P. Smith : > I was just wondering if a bytecode for a superinstruction of the common > sequence: > > ???????????? 6 POP_TOP > ????????????? 7 LOAD_CONST?????????????? 0 (None) > ???????????? 10 RETURN_VALUE > > might be worth it. We could just utilize RETURN_VALUE's op argument. -- Regards, Benjamin From solipsis at pitrou.net Tue Sep 1 00:29:48 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 31 Aug 2009 22:29:48 +0000 (UTC) Subject: [Python-Dev] default of returning None hurts performance? References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> Message-ID: Gregory P. Smith krypto.org> writes: > > I was just wondering if a bytecode for a superinstruction of the common sequence: > 6 POP_TOP > 7 LOAD_CONST?????????????? 0 (None) > 10 RETURN_VALUE > might be worth it. I think superinstructions in general would be a good thing to experiment, as wpython showed. Direct addressing (via a pseudo register file combining locals and constants) would eliminate many bookkeeping-related opcodes in common bytecode. Regards Antoine. From python at rcn.com Tue Sep 1 00:39:47 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 31 Aug 2009 15:39:47 -0700 Subject: [Python-Dev] default of returning None hurts performance? References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com><52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> <43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com> Message-ID: > I was just wondering if a bytecode for a superinstruction of the common > sequence: > > 6 POP_TOP > 7 LOAD_CONST 0 (None) > 10 RETURN_VALUE > > might be worth it. [Collin Winter] > I doubt it. You'd save a bit of stack manipulation, but since this > will only appear at the end of a function, I'd be skeptical that this > would make any macrobenchmarks (statistically) significantly faster. I concur with Collin. And since it appears only at the end of a function, the optimization doesn't help inner-loops in a function (where most of the time usually spent). Raymond From greg.ewing at canterbury.ac.nz Tue Sep 1 02:01:27 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 01 Sep 2009 12:01:27 +1200 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> Message-ID: <4A9C6457.2060609@canterbury.ac.nz> Antoine Pitrou wrote: > Did your coworker run any timings instead of basing his assumptions on bytecode > size? In any case, what are you suggesting -- that the last value returned by a function call in the body should be the default return value? I don't think the unpredictability that would introduce would be a good idea. -- Greg From greg at krypto.org Tue Sep 1 03:03:12 2009 From: greg at krypto.org (Gregory P. Smith) Date: Mon, 31 Aug 2009 18:03:12 -0700 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <4A9C6457.2060609@canterbury.ac.nz> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> Message-ID: <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> On Mon, Aug 31, 2009 at 5:01 PM, Greg Ewing wrote: > Antoine Pitrou wrote: > > Did your coworker run any timings instead of basing his assumptions on >> bytecode >> size? >> > > In any case, what are you suggesting -- that the last value > returned by a function call in the body should be the > default return value? > > I don't think the unpredictability that would introduce > would be a good idea. > gads no. we're not shell or perl! don't do that :) -------------- next part -------------- An HTML attachment was scrubbed... URL: From s7v7nislands at gmail.com Tue Sep 1 05:47:44 2009 From: s7v7nislands at gmail.com (xiaobing jiang) Date: Tue, 1 Sep 2009 11:47:44 +0800 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? Message-ID: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> hi all: In the svn thunk tree, I find in file Object/funcobject.c, the two functions: sm_init and cm_init have a little different. the cm_init function will check the object is really callable, but the sm_init don't. the code like this: if (!PyCallable_Check(callable)) { PyErr_Format(PyExc_TypeError, "'%s' object is not callable", callable->ob_type->tp_name); return -1; } so when use staticmethod and classmethod on a class variable, there are two different behavior. class Test(object): name = "s7v7nislands" name = classmethod(Test.name) # will raise error. name = staticmethod(Test.name) # will not raise error. My idea is: here, the two functions (or maybe classes) should have the same behavior). so is this a bug or something I missing ? thanks! s7v7nislands From amentajo at msu.edu Tue Sep 1 07:10:35 2009 From: amentajo at msu.edu (Joe Amenta) Date: Tue, 1 Sep 2009 01:10:35 -0400 Subject: [Python-Dev] 3to2 0.1 alpha 1 released In-Reply-To: References: <4dc473a50908261229i3d0f600dq38286edafc076e48@mail.gmail.com> <4A959A89.4000000@v.loewis.de> <4dc473a50908261555o2bc4cdbexff1b2dbabb4bab0@mail.gmail.com> Message-ID: <4dc473a50908312210n249faa32xc01c8adc002fa9ac@mail.gmail.com> Thanks Brett and Sridhar for the info. 3to2's version info has been PEP386-ified, and the latest version can always be found at http://pypi.python.org/pypi/3to2/ (the previous link will now generate an error). --Joe On Mon, Aug 31, 2009 at 3:25 PM, Brett Cannon wrote: > On Mon, Aug 31, 2009 at 12:18, Sridhar > Ratnakumar wrote: > > On Wed, 26 Aug 2009 15:55:54 -0700, Joe Amenta wrote: > > > >> -- 3to2 is now registered with PyPI. Did I do it right? > > > >> http://pypi.python.org/pypi/3to2/0.1%20alpha%201 > > > > Please fix the version number to not contain any whitespace characters. > Also > > set the `version` argument in setup(..) in your setup.py. And then you > may > > want to use the `upload` command to upload the new tarball to PyPI. See > > http://wiki.python.org/moin/Distutils/Tutorial for more details. > > See PEP 386 (http://www.python.org/dev/peps/pep-0386/) for what the > current thinking on version numbers is. > > -Brett > -------------- next part -------------- An HTML attachment was scrubbed... URL: From scott+python-dev at scottdial.com Tue Sep 1 09:52:51 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 01 Sep 2009 03:52:51 -0400 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com><52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> <43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com> Message-ID: <4A9CD2D3.9090806@scottdial.com> Raymond Hettinger wrote: >> I was just wondering if a bytecode for a superinstruction of the common >> sequence: >> >> 6 POP_TOP >> 7 LOAD_CONST 0 (None) >> 10 RETURN_VALUE >> >> might be worth it. > > [Collin Winter] >> I doubt it. You'd save a bit of stack manipulation, but since this >> will only appear at the end of a function, I'd be skeptical that this >> would make any macrobenchmarks (statistically) significantly faster. > > I concur with Collin. And since it appears only at the end of a function, > the optimization doesn't help inner-loops in a function (where most of > the time usually spent). > I fail to understand this crude logic. How often is the inner-loop really going to solely call C code? Any call to Python in an inner-loop is going to suffer this penalty on the order of the number of loop iterations)? -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From ncoghlan at gmail.com Tue Sep 1 12:36:46 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 01 Sep 2009 20:36:46 +1000 Subject: [Python-Dev] how important is setting co_filename for a module being imported to what __file__ is set to? In-Reply-To: <4E5EF4D6-7544-4911-9E6E-607D3525ADD9@acm.org> References: <1afaf6160908310755vc5d3872i2f7be0ce33c09040@mail.gmail.com> <4E5EF4D6-7544-4911-9E6E-607D3525ADD9@acm.org> Message-ID: <4A9CF93E.7040107@gmail.com> Fred Drake wrote: > Every time I've been bitten by the wrong co_filename values (usually > from tracebacks), changing the way marshal creates code objects to use a > values passed in has been the thing that made the most sense to me. > > The feature request that's involved here, getting correct co_filename > values, can be implemented in different ways, sure. This particular > change produces the least impact in the because it *doesn't* change the > mutability of code objects. > > I for one appreciate that, mostly because I'm simply wary of making code > objects mutable in this way having unexpected side effects in some library. "linecache" was the one that immediately popped into mind for me (not saying it is affected, just saying it would be the first place I would like for possible side effects). I think this is one of those situations where something has behaved a certain way for so long that it would be really hard to be confident that we had considered all the possible ramifications of changing it. Modifying marshal to allow Python code to override the stored value instead of making it a free-for-all preserves the post-compile immutability characteristic while still letting Brett make importlib mimic the behaviour of import.c. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Tue Sep 1 12:58:30 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 01 Sep 2009 20:58:30 +1000 Subject: [Python-Dev] runpy.py In-Reply-To: References: <332972a20908230909q67517e53y1e65b9155a9fc0b1@mail.gmail.com> <4A92B9C3.5040701@simplistix.co.uk> <1afaf6160908240907y2bb8fabbud044171de72ae052@mail.gmail.com> <4A939F4F.9060306@simplistix.co.uk> <1afaf6160908250125h4cc16ff7h2c33f1162cd25cca@mail.gmail.com> <4A93A85D.3010108@gmail.com> <4A951C43.8050707@gmail.com> <4A9BBB21.4040504@simplistix.co.uk> <4A9BD1EA.9020204@gmail.com> Message-ID: <4A9CFE56.6020108@gmail.com> Brett Cannon wrote: > On Mon, Aug 31, 2009 at 06:36, Nick Coghlan wrote: >> That said, while actually ditching the C code might cause an argument, >> expanding runpy with Python equivalents of the C level functionality >> (i.e. run script by name, run directory/zipfile by name, '-c' switch, >> and other odds and ends that I'm probably forgetting right now, with all >> associated modifications to sys.argv and the __main__ module attributes) >> should be far less controversial. > > It also has the perk of letting alternative VMs not have to implement > all of that stuff themselves, potentially helping to unify even the > command-line interfaces for all the VMs. I created a tracker item for the idea so I don't forget about it: http://bugs.python.org/issue6816 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From scott+python-ideas at scottdial.com Tue Sep 1 09:51:49 2009 From: scott+python-ideas at scottdial.com (Scott Dial) Date: Tue, 01 Sep 2009 03:51:49 -0400 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com><52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> <43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com> Message-ID: <4A9CD295.2090605@scottdial.com> Raymond Hettinger wrote: >> I was just wondering if a bytecode for a superinstruction of the common >> sequence: >> >> 6 POP_TOP >> 7 LOAD_CONST 0 (None) >> 10 RETURN_VALUE >> >> might be worth it. > > [Collin Winter] >> I doubt it. You'd save a bit of stack manipulation, but since this >> will only appear at the end of a function, I'd be skeptical that this >> would make any macrobenchmarks (statistically) significantly faster. > > I concur with Collin. And since it appears only at the end of a function, > the optimization doesn't help inner-loops in a function (where most of > the time usually spent). > I fail to understand this crude logic. How often is the inner-loop really going to solely call C code? Any call to Python in an inner-loop is going to suffer this penalty on the order of the number of loop iterations)? -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From steve at pearwood.info Tue Sep 1 15:04:50 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 1 Sep 2009 23:04:50 +1000 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <4A9CD295.2090605@scottdial.com> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9CD295.2090605@scottdial.com> Message-ID: <200909012304.51889.steve@pearwood.info> On Tue, 1 Sep 2009 05:51:49 pm Scott Dial wrote: > Raymond Hettinger wrote: > >> I was just wondering if a bytecode for a superinstruction of the > >> common sequence: > >> > >> 6 POP_TOP > >> 7 LOAD_CONST 0 (None) > >> 10 RETURN_VALUE > >> > >> might be worth it. > > > > [Collin Winter] > > > >> I doubt it. You'd save a bit of stack manipulation, but since this > >> will only appear at the end of a function, I'd be skeptical that > >> this would make any macrobenchmarks (statistically) significantly > >> faster. > > > > I concur with Collin. And since it appears only at the end of a > > function, the optimization doesn't help inner-loops in a function > > (where most of the time usually spent). > > I fail to understand this crude logic. How often is the inner-loop > really going to solely call C code? Any call to Python in an > inner-loop is going to suffer this penalty on the order of the number > of loop iterations)? Most functions don't suffer this penalty. Consider the following two functions: def g(x): return x() def h(x): x() Now disassemble: >>> dis.dis(g) 2 0 LOAD_FAST 0 (x) 3 CALL_FUNCTION 0 6 RETURN_VALUE >>> dis.dis(h) 2 0 LOAD_FAST 0 (x) 3 CALL_FUNCTION 0 6 POP_TOP 7 LOAD_CONST 0 (None) 10 RETURN_VALUE The first doesn't suffer any such default "return None" penalty, and so won't gain any performance benefit from optimizing it. It is only the subset of functions which don't explicitly return anything which will see any potential benefit. Let me call such functions "procedures" to avoid confusion with those functions which won't see any benefit. While procedures may see some benefit, it's a trivial amount, probably not worth the extra complexity. According to Gregory's tests, the difference is approximately 2% on a trivial do-nothing function. According to my tests on my PC, I might hope to save somewhat less than 0.1 microsecond per procedure call as an absolute saving. As a relative saving though, it will most likely be insignificant: for comparison's sake, urllib2.Request('http://example.com') takes around 150?s on my machine, and math.sin(1.1) around 90?s. For any procedure which does non-trivial amounts of work, saving 0.1?s is insignificant, no matter how many times it is called inside a loop. -- Steven D'Aprano From solipsis at pitrou.net Tue Sep 1 15:25:06 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 01 Sep 2009 15:25:06 +0200 Subject: [Python-Dev] [OT] implicit return values In-Reply-To: <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net> Message-ID: <1251811506.5631.44.camel@localhost> Le mardi 01 septembre 2009 ? 15:09 +0200, Xavier Morel a ?crit : > "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly. Well, feel free to prefer an unreadable language if you want :) Having implicit return values is certainly not something which follows Python's design principles. Even C abandoned the idea. In any case, this discussion is off-topic for this thread. If you want to discuss the topic further, you can post to python-list or python-ideas (it will most certainly be shot down anyway). From catch-all at masklinn.net Tue Sep 1 15:09:23 2009 From: catch-all at masklinn.net (Xavier Morel) Date: Tue, 1 Sep 2009 15:09:23 +0200 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> Message-ID: <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net> On 1 Sep 2009, at 03:03 , Gregory P. Smith wrote: > On Mon, Aug 31, 2009 at 5:01 PM, Greg Ewing >wrote: > >> Antoine Pitrou wrote: >> >> Did your coworker run any timings instead of basing his assumptions >> on >>> bytecode >>> size? >>> >> >> In any case, what are you suggesting -- that the last value >> returned by a function call in the body should be the >> default return value? >> >> I don't think the unpredictability that would introduce >> would be a good idea. >> > > gads no. we're not shell or perl! don't do that :) "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly. From catch-all at masklinn.net Tue Sep 1 15:12:34 2009 From: catch-all at masklinn.net (Xavier Morel) Date: Tue, 1 Sep 2009 15:12:34 +0200 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <4A9C6457.2060609@canterbury.ac.nz> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> Message-ID: On 1 Sep 2009, at 02:01 , Greg Ewing wrote: > I don't think the unpredictability that would introduce > would be a good idea. I fail to grasp the unpredictability of "the last expression evaluated in the body of a function is its return value". It couldn't work in Python because statements aren't expressions, therefore I think def foo(): if cond: 3 else: 4 would break (given if:else: doesn't return a value, the function couldn't have a return value), but in languages where everything is an expression (where if:else: does return a value) there's nothing unpredictable about it. From benjamin at python.org Tue Sep 1 16:21:05 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 1 Sep 2009 09:21:05 -0500 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> Message-ID: <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> 2009/8/31 xiaobing jiang : > My idea is: here, the two functions (or maybe classes) should have the > same behavior). > so is this a bug or something I missing ? I think they should both not check their arguments in __init__ to allow for duck typing. -- Regards, Benjamin From python at rcn.com Tue Sep 1 17:49:07 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 1 Sep 2009 08:49:07 -0700 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <4A9CD295.2090605@scottdial.com> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com><52dc1c820908311507q75e06b08jd582cbdbba00baab@mail.gmail.com> <43aa6ff70908311528w54ac19cft64cfdc7b2094967a@mail.gmail.com> <4A9CD295.2090605@scottdial.com> Message-ID: > > I fail to understand this crude logic. If a function contains any looping (C or otherwise) or does much in the way of meaningful work, then it's unlikely that the single POP_TOP associated with an implied "return None" is taking much of the total runtime. Raymond From xavier.morel at masklinn.net Tue Sep 1 16:36:28 2009 From: xavier.morel at masklinn.net (Xavier Morel) Date: Tue, 1 Sep 2009 16:36:28 +0200 Subject: [Python-Dev] [OT] implicit return values In-Reply-To: <1251811506.5631.44.camel@localhost> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net> <1251811506.5631.44.camel@localhost> Message-ID: On 1 Sep 2009, at 15:25 , Antoine Pitrou wrote: Le mardi 01 septembre 2009 ? 15:09 +0200, Xavier Morel a ?crit : >> "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly. > > Well, feel free to prefer an unreadable language if you want :) Smalltalk or Haskell are hardly inherently unreadable. And it's not like the Python community never lifts features from such languages, so obviously they do (some at least) things right. > it will most certainly be shot down anyway Yep, so there's not much point in bringing it up there. From mcguire at google.com Tue Sep 1 18:40:17 2009 From: mcguire at google.com (Jake McGuire) Date: Tue, 1 Sep 2009 09:40:17 -0700 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <4A9C6457.2060609@canterbury.ac.nz> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> Message-ID: <77c780b40909010940v4c9bc50bs59cf82742b6409f@mail.gmail.com> On Mon, Aug 31, 2009 at 5:01 PM, Greg Ewing wrote: > Antoine Pitrou wrote: > > Did your coworker run any timings instead of basing his assumptions on >> bytecode >> size? >> > > In any case, what are you suggesting -- that the last value > returned by a function call in the body should be the > default return value? > > I don't think the unpredictability that would introduce > would be a good idea. > Never mind the fact that there is a lot of Python code out there that assumes that functions without an explicit return value, return None. This is pretty fundamental to the Python API. -jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From rnk at mit.edu Tue Sep 1 20:01:35 2009 From: rnk at mit.edu (Reid Kleckner) Date: Tue, 1 Sep 2009 14:01:35 -0400 Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the child process of a fork from a spawned thread Message-ID: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com> Hi all, I'm working on http://bugs.python.org/issue6642 for unladen swallow (because it happens to bite us in a weird way), and Jeff Yasskin told me to ask python-dev what the proper behavior should be at exit from the child process of a fork from a spawned thread. Right now, it seems that there is an assumption that when exiting the process, control will always return through Py_Main, which in turn calls Py_Finalize, to do things like GC and calling atexit handlers. Normally, if you fork a process from the main thread, this assumption will remain true, because main and Py_Main are still at the bottom of the stack in the child process. However, if you fork from a spawned thread, then the root of the stack will be the thread bootstrap routine for the platform's pythreads implementation. On one hand, you may not want to call the user's atexit handlers multiple times from different processes if they have externally visible effects. On the other hand, people seem to assume that Py_Finalize will be called at process exit to do various cleanups. On the third hand, maybe Python could just clear out all the atexit handlers in the child after a fork. So what should the correct behavior be? Thanks, Reid From brett at python.org Tue Sep 1 20:36:13 2009 From: brett at python.org (Brett Cannon) Date: Tue, 1 Sep 2009 11:36:13 -0700 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> Message-ID: On Tue, Sep 1, 2009 at 07:21, Benjamin Peterson wrote: > 2009/8/31 xiaobing jiang : >> My idea is: here, the two functions (or maybe classes) should have the >> same behavior). >> so is this a bug or something I missing ? > > I think they should both not check their arguments in __init__ to > allow for duck typing. But what is the point of wrapping something with classmethod or staticmethod that can't be called? It isn't like it is checking explicitly for a function or method, just that it can be called which seems reasonable to me (unless PyCallable_Check() is as off as callable() was). -Brett From martin at v.loewis.de Tue Sep 1 20:58:20 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 01 Sep 2009 20:58:20 +0200 Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the child process of a fork from a spawned thread In-Reply-To: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com> References: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com> Message-ID: <4A9D6ECC.1030201@v.loewis.de> > On one hand, you may not want to call the user's atexit handlers > multiple times from different processes if they have externally > visible effects. On the other hand, people seem to assume that > Py_Finalize will be called at process exit to do various cleanups. On > the third hand, maybe Python could just clear out all the atexit > handlers in the child after a fork. So what should the correct > behavior be? Standard POSIX fork semantics should be a guidance. IIUC, termination of the last thread is equivalent to calling exit(0) (although return from main() still means that exit is invoked right away, and the return value of main is the exit code - right?). Calling exit means to call all exit handlers. So to match POSIX semantics, we should also call the exit handlers in the child process (along with invoking a final garbage collection etc.) Regards, Martin From rnk at mit.edu Tue Sep 1 21:05:04 2009 From: rnk at mit.edu (Reid Kleckner) Date: Tue, 1 Sep 2009 15:05:04 -0400 Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the child process of a fork from a spawned thread In-Reply-To: <4A9D6ECC.1030201@v.loewis.de> References: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com> <4A9D6ECC.1030201@v.loewis.de> Message-ID: <9a9942200909011205o16734de0sbcea5b9218bacc04@mail.gmail.com> On Tue, Sep 1, 2009 at 2:58 PM, "Martin v. L?wis" wrote: >> On one hand, you may not want to call the user's atexit handlers >> multiple times from different processes if they have externally >> visible effects. ?On the other hand, people seem to assume that >> Py_Finalize will be called at process exit to do various cleanups. ?On >> the third hand, maybe Python could just clear out all the atexit >> handlers in the child after a fork. ?So what should the correct >> behavior be? > > Standard POSIX fork semantics should be a guidance. IIUC, termination > of the last thread is equivalent to calling exit(0) (although return > from main() still means that exit is invoked right away, and the return > value of main is the exit code - right?). Calling exit means to call > all exit handlers. It depends, there is also _exit, which exists solely for the purpose of working around exit handlers being called from a forked child process at exit. Which semantics should Python have? In my opinion, it is more obvious that the user's handlers would be called than not, so I agree with you. Reid From martin at v.loewis.de Tue Sep 1 21:48:41 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 01 Sep 2009 21:48:41 +0200 Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the child process of a fork from a spawned thread In-Reply-To: <9a9942200909011205o16734de0sbcea5b9218bacc04@mail.gmail.com> References: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com> <4A9D6ECC.1030201@v.loewis.de> <9a9942200909011205o16734de0sbcea5b9218bacc04@mail.gmail.com> Message-ID: <4A9D7A99.8040804@v.loewis.de> >> Standard POSIX fork semantics should be a guidance. IIUC, termination >> of the last thread is equivalent to calling exit(0) (although return >> from main() still means that exit is invoked right away, and the return >> value of main is the exit code - right?). Calling exit means to call >> all exit handlers. > > It depends, there is also _exit, which exists solely for the purpose > of working around exit handlers being called from a forked child > process at exit. I don't think so. There are other cases where you don't want to run atexit handlers, eg. in a segfault signal handler. _exit has been there "forever", I believe (of course, so did fork()). > Which semantics should Python have? In my opinion, > it is more obvious that the user's handlers would be called than not, > so I agree with you. POSIX says that ending the last thread is equivalent to exit(0), and this is what Python should also do when the last thread ends. This is independent of somebody calling _exit(), which should have the very effect that calling _exit has (i.e. immediate process termination). Regards, Martin From benjamin at python.org Tue Sep 1 23:54:17 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 1 Sep 2009 16:54:17 -0500 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> Message-ID: <1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com> 2009/9/1 Brett Cannon : > On Tue, Sep 1, 2009 at 07:21, Benjamin Peterson wrote: >> 2009/8/31 xiaobing jiang : >>> My idea is: here, the two functions (or maybe classes) should have the >>> same behavior). >>> so is this a bug or something I missing ? >> >> I think they should both not check their arguments in __init__ to >> allow for duck typing. > > But what is the point of wrapping something with classmethod or > staticmethod that can't be called? It isn't like it is checking > explicitly for a function or method, just that it can be called which > seems reasonable to me (unless PyCallable_Check() is as off as > callable() was). Well, if checking if tp_call is not NULL is as bad as callable, then yes. I don't see any reason to use staticmethod or classmethod with a non-callable, but to be consistent, I would, given the choice between removing code and adding another type check, perfer to remove a type check. -- Regards, Benjamin From greg.ewing at canterbury.ac.nz Wed Sep 2 00:04:46 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 02 Sep 2009 10:04:46 +1200 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> Message-ID: <4A9D9A7E.5080209@canterbury.ac.nz> Xavier Morel wrote: > I fail to grasp the unpredictability of "the last expression evaluated > in the body of a function is its return value". It's unpredictable in the sense that if you're writing a function that's not intended to return a value, you're not thinking about what the last call you make in the function returns, so to a first approximation it's just some random value. I often write code that makes use of the fact that falling off the end of a function returns None. This has been a documented part of the Python language from the beginning, and changing it would break a lot of code for no good reason. -- Greg From greg.ewing at canterbury.ac.nz Wed Sep 2 00:10:25 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 02 Sep 2009 10:10:25 +1200 Subject: [Python-Dev] [OT] implicit return values In-Reply-To: References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net> <1251811506.5631.44.camel@localhost> Message-ID: <4A9D9BD1.8000501@canterbury.ac.nz> > Le mardi 01 septembre 2009 ? 15:09 +0200, Xavier Morel a ?crit : > >>> "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly. IIRC, the default return value of a Smalltalk method is self, not the last thing evaluated. (And no, that's not going to happen in Python either -- the BDFL has rejected similar suggestions on previous occasions.) -- Greg From brett at python.org Wed Sep 2 00:13:20 2009 From: brett at python.org (Brett Cannon) Date: Tue, 1 Sep 2009 15:13:20 -0700 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> <1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com> Message-ID: On Tue, Sep 1, 2009 at 14:54, Benjamin Peterson wrote: > 2009/9/1 Brett Cannon : >> On Tue, Sep 1, 2009 at 07:21, Benjamin Peterson wrote: >>> 2009/8/31 xiaobing jiang : >>>> My idea is: here, the two functions (or maybe classes) should have the >>>> same behavior). >>>> so is this a bug or something I missing ? >>> >>> I think they should both not check their arguments in __init__ to >>> allow for duck typing. >> >> But what is the point of wrapping something with classmethod or >> staticmethod that can't be called? It isn't like it is checking >> explicitly for a function or method, just that it can be called which >> seems reasonable to me (unless PyCallable_Check() is as off as >> callable() was). > > Well, if checking if tp_call is not NULL is as bad as callable, then yes. > > I don't see any reason to use staticmethod or classmethod with a > non-callable, but to be consistent, I would, given the choice between > removing code and adding another type check, perfer to remove a type > check. Fine by me. I guess it will just fail later with a slightly more cryptic exception. -Brett From greg.ewing at canterbury.ac.nz Wed Sep 2 00:15:17 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 02 Sep 2009 10:15:17 +1200 Subject: [Python-Dev] Whether to call Py_Finalize when exiting from the child process of a fork from a spawned thread In-Reply-To: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com> References: <9a9942200909011101o1520ee69h762070ac9bf562ad@mail.gmail.com> Message-ID: <4A9D9CF5.802@canterbury.ac.nz> Reid Kleckner wrote: > On one hand, you may not want to call the user's atexit handlers > multiple times from different processes if they have externally > visible effects. On the other hand, people seem to assume that > Py_Finalize will be called at process exit to do various cleanups. On > the third hand, maybe Python could just clear out all the atexit > handlers in the child after a fork. So what should the correct > behavior be? Seems to me there's no single answer to that, because some kinds of atexit handlers may need to be called in child processes, while others may need *not* to be. Maybe we need a flag when registering an atexit handler to indicate whether it should be kept across forks? -- Greg From greg.ewing at canterbury.ac.nz Wed Sep 2 00:28:06 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 02 Sep 2009 10:28:06 +1200 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> Message-ID: <4A9D9FF6.3090408@canterbury.ac.nz> Brett Cannon wrote: > It isn't like it is checking > explicitly for a function or method, just that it can be called which > seems reasonable to me (unless PyCallable_Check() is as off as > callable() was). I think it just checks that there's something in the tp_call slot, which is reasonable -- if it's empty, there's no way that calling the object could ever succeed, so you might as well fail early. -- Greg From benjamin at python.org Wed Sep 2 00:31:26 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 1 Sep 2009 17:31:26 -0500 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <4A9D9FF6.3090408@canterbury.ac.nz> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> <4A9D9FF6.3090408@canterbury.ac.nz> Message-ID: <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com> 2009/9/1 Greg Ewing : > Brett Cannon wrote: >> >> It isn't like it is checking >> explicitly for a function or method, just that it can be called which >> seems reasonable to me (unless PyCallable_Check() is as off as >> callable() was). > > I think it just checks that there's something in the > tp_call slot, which is reasonable -- if it's empty, > there's no way that calling the object could ever > succeed, so you might as well fail early. It depends on whether you're keeping the "callable" object around or not. Somebody could add a __call__ method later. -- Regards, Benjamin From greg.ewing at canterbury.ac.nz Wed Sep 2 00:36:18 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 02 Sep 2009 10:36:18 +1200 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> <4A9D9FF6.3090408@canterbury.ac.nz> <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com> Message-ID: <4A9DA1E2.8000801@canterbury.ac.nz> Benjamin Peterson wrote: > It depends on whether you're keeping the "callable" object around or > not. Somebody could add a __call__ method later. Good point. Removing the check sounds like the right thing to do, then. -- Greg From python at rcn.com Wed Sep 2 01:33:39 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 1 Sep 2009 16:33:39 -0700 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> <1afaf6160909011454t2e4fa960k74783be550a50f95@mail.gmail.com> Message-ID: On Sep 1, 2009, at 2:54 PM, Benjamin Peterson wrote: > 2009/9/1 Brett Cannon : >> On Tue, Sep 1, 2009 at 07:21, Benjamin >> Peterson wrote: >>> 2009/8/31 xiaobing jiang : >>>> My idea is: here, the two functions (or maybe classes) should >>>> have the >>>> same behavior). >>>> so is this a bug or something I missing ? >>> >>> I think they should both not check their arguments in __init__ to >>> allow for duck typing. >> >> But what is the point of wrapping something with classmethod or >> staticmethod that can't be called? It isn't like it is checking >> explicitly for a function or method, just that it can be called which >> seems reasonable to me (unless PyCallable_Check() is as off as >> callable() was). > > Well, if checking if tp_call is not NULL is as bad as callable, then > yes. > > I don't see any reason to use staticmethod or classmethod with a > non-callable, but to be consistent, I would, given the choice between > removing code and adding another type check, perfer to remove a type > check. > Removing the type check is also my preference. Raymond -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Wed Sep 2 02:50:37 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 01 Sep 2009 20:50:37 -0400 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <4A9DA1E2.8000801@canterbury.ac.nz> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> <4A9D9FF6.3090408@canterbury.ac.nz> <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com> <4A9DA1E2.8000801@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Benjamin Peterson wrote: > >> It depends on whether you're keeping the "callable" object around or >> not. Somebody could add a __call__ method later. > > Good point. Removing the check sounds like the > right thing to do, then. Both classmethod & staticmethod are documented as having a *function* (callable, as I interprete that) as their single argument. Seems reasonable to me. Turning the argument into a function after the fact seems like a really esoteric use case. From pje at telecommunity.com Wed Sep 2 03:27:08 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 01 Sep 2009 21:27:08 -0400 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> <4A9D9FF6.3090408@canterbury.ac.nz> <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com> <4A9DA1E2.8000801@canterbury.ac.nz> Message-ID: <20090902012752.3AFBB3A4093@sparrow.telecommunity.com> At 08:50 PM 9/1/2009 -0400, Terry Reedy wrote: >Greg Ewing wrote: >>Benjamin Peterson wrote: >> >>>It depends on whether you're keeping the "callable" object around or >>>not. Somebody could add a __call__ method later. >>Good point. Removing the check sounds like the >>right thing to do, then. > >Both classmethod & staticmethod are documented as having a >*function* (callable, as I interprete that) as their single >argument. Seems reasonable to me. Turning the argument into a >function after the fact seems like a really esoteric use case. The main use case for staticmethod is to prevent __get__ from being called on an object retrieved from a class or an instance. It just happens that the most common types of objects you'd want to do that on are functions. However, if for some reason you intend to make a *descriptor* available as an attribute (via a class default), then wrapping it with staticmethod is the only easy way to do it. For example, if you're writing a class whose instances have an attribute that holds a "property" instance, and you want to provide a class-level default, the simplest way to do it is to wrap the default property instance with staticmethod, so that it's not treated as a property of the class/instance. (Property instances are of course not callable.) From tjreedy at udel.edu Wed Sep 2 07:53:05 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 02 Sep 2009 01:53:05 -0400 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <20090902012752.3AFBB3A4093@sparrow.telecommunity.com> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> <4A9D9FF6.3090408@canterbury.ac.nz> <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com> <4A9DA1E2.8000801@canterbury.ac.nz> <20090902012752.3AFBB3A4093@sparrow.telecommunity.com> Message-ID: P.J. Eby wrote: > At 08:50 PM 9/1/2009 -0400, Terry Reedy wrote: >> Greg Ewing wrote: >>> Benjamin Peterson wrote: >>> >>>> It depends on whether you're keeping the "callable" object around or >>>> not. Somebody could add a __call__ method later. >>> Good point. Removing the check sounds like the >>> right thing to do, then. >> >> Both classmethod & staticmethod are documented as having a *function* >> (callable, as I interprete that) as their single argument. Seems >> reasonable to me. Turning the argument into a function after the fact >> seems like a really esoteric use case. > > The main use case for staticmethod is to prevent __get__ from being > called on an object retrieved from a class or an instance. It just > happens that the most common types of objects you'd want to do that on > are functions. If so, then it is mis-named and mis-documented, and it seems to me that there is *a* rationale (not necessarily determinative) for the current difference in implementation. > However, if for some reason you intend to make a *descriptor* available > as an attribute (via a class default), then wrapping it with > staticmethod is the only easy way to do it. > > For example, if you're writing a class whose instances have an attribute > that holds a "property" instance, and you want to provide a class-level > default, the simplest way to do it is to wrap the default property > instance with staticmethod, so that it's not treated as a property of > the class/instance. > > (Property instances are of course not callable.) So that it is a 'static object' in some sense, but not a static *method*. Thanks for the clarification and example. tjr From s7v7nislands at gmail.com Wed Sep 2 10:24:30 2009 From: s7v7nislands at gmail.com (xiaobing jiang) Date: Wed, 2 Sep 2009 16:24:30 +0800 Subject: [Python-Dev] why different between staticmethod and classmethod on non-callable object? In-Reply-To: <20090902012752.3AFBB3A4093@sparrow.telecommunity.com> References: <8f43df260908312047u2aa73e97k73dca3468c61423c@mail.gmail.com> <1afaf6160909010721x26587481ubfa6cbe136556e0c@mail.gmail.com> <4A9D9FF6.3090408@canterbury.ac.nz> <1afaf6160909011531n46048314hb6c71b55e17a5522@mail.gmail.com> <4A9DA1E2.8000801@canterbury.ac.nz> <20090902012752.3AFBB3A4093@sparrow.telecommunity.com> Message-ID: <8f43df260909020124u19071822yd35ef884f08da77@mail.gmail.com> the three types: function, classmethod, staticmethod are descriptors. but staticmethod's __get__ return the orignal value, others return object of instancemethod. (from souce in Objects/funcobject.c) so the staticmethod just like a wrap that make the wrapped object 'frozen'. like in your example. is it right? the name and document are mis-understanded. thanks 2009/9/2 P.J. Eby : > At 08:50 PM 9/1/2009 -0400, Terry Reedy wrote: >> >> Greg Ewing wrote: >>> >>> Benjamin Peterson wrote: >>> >>>> It depends on whether you're keeping the "callable" object around or >>>> not. Somebody could add a __call__ method later. >>> >>> Good point. Removing the check sounds like the >>> right thing to do, then. >> >> Both classmethod & staticmethod are documented as having a *function* >> (callable, as I interprete that) as their single argument. Seems reasonable >> to me. Turning the argument into a function after the fact seems like a >> really esoteric use case. > > The main use case for staticmethod is to prevent __get__ from being called > on an object retrieved from a class or an instance. ?It just happens that > the most common types of objects you'd want to do that on are functions. > > However, if for some reason you intend to make a *descriptor* available as > an attribute (via a class default), then wrapping it with staticmethod is > the only easy way to do it. > > For example, if you're writing a class whose instances have an attribute > that holds a "property" instance, and you want to provide a class-level > default, the simplest way to do it is to wrap the default property instance > with staticmethod, so that it's not treated as a property of the > class/instance. > > (Property instances are of course not callable.) > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/s7v7nislands%40gmail.com > From catch-all at masklinn.net Wed Sep 2 11:46:31 2009 From: catch-all at masklinn.net (Xavier Morel) Date: Wed, 2 Sep 2009 11:46:31 +0200 Subject: [Python-Dev] [OT] implicit return values In-Reply-To: <4A9D9BD1.8000501@canterbury.ac.nz> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net> <1251811506.5631.44.camel@localhost> <4A9D9BD1.8000501@canterbury.ac.nz> Message-ID: On 2 Sep 2009, at 00:10 , Greg Ewing wrote: Le mardi 01 septembre 2009 ? 15:09 +0200, Xavier Morel a ?crit : >>>> "We" are not Erlang, Smalltalk, OCaml or Haskell either, sadly. > > IIRC, the default return value of a Smalltalk method is > self, not the last thing evaluated. Methods yes (and that's one of the few Smalltalk design "features" I consider truly dumb, considering it has message cascading), but I was referring to blocks rather than methods and the return value of blocks is the last evaluated expression. > (And no, that's not going to happen in Python either -- > the BDFL has rejected similar suggestions on previous > occasions.) I know. From rob.cliffe at btinternet.com Wed Sep 2 12:15:39 2009 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Wed, 2 Sep 2009 11:15:39 +0100 Subject: [Python-Dev] Decorator syntax Message-ID: Hi All, This is my first post to python-dev so I will briefly introduce myself: My name is Rob Cliffe and I am a commercial programmer living in London, UK. I have some 30 years of programming experience but have only been using Python for a couple of years. First I want to say what a fantastic language Python is. It is THE best language for development in my opinion, and a joy to use. My specific issue: I eventually got my head round decorator syntax and realised that what came after the '@' was (basically) a function that took a function as argument and returned a function as result. However it seems to me unPythonesque (i.e. an exception to Python's normal consistency) that the syntax of what follows the '@' should be restricted to either a single (function) identifier or a single (function) identifier with an argument list. The example I tried, which seems not an unreasonable sort of thing to do, was along the lines of: def deco1(func): def deco2(func): DecoList = [deco1, deco2] @DecoList[0] # NO - CAUSES SYNTAX ERROR def foo(): pass I am sure other guys have their own examples. I am of course not the first person to raise this issue, and I see that Guido has a "gut feeling" against allowing a general expression after the '@'. BUT - a general expression can be "smuggled in" very easily as a function argument: def Identity(x): return x @Identity(DecoList[0]) # THIS WORKS def foo(): pass So - the syntax restriction seems not only inconsistent, but pointless; it doesn't forbid anything, but merely means we have to do it in a slightly convoluted (unPythonesque) way. So please, Guido, will you reconsider? Best wishes Rob Cliffe -------------- next part -------------- An HTML attachment was scrubbed... URL: From cmjohnson.mailinglist at gmail.com Wed Sep 2 12:34:49 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Wed, 2 Sep 2009 00:34:49 -1000 Subject: [Python-Dev] Decorator syntax In-Reply-To: References: Message-ID: <3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com> Crossposting to Python-ideas, I asked for the same change to the grammar a couple months back on python-ideas. See http://mail.python.org/pipermail/python-ideas/2009-February/thread.html#2787 I'm all for it, but you'll have to convince Guido that this won't result in confusing to read code. My own examples, unfortunately did not advance your cause, as Guido explained, "My brain hurts trying to understand all this. I don't think this bodes well as a use case for a proposed feature." :-D The trouble is that I was using lambdas upon lambdas to do all kinds of Ruby block-esque tricks. OTOH, if you come up with some simple, clear use cases though, and I think he might still be persuadable to make a simple change to the grammar. ? Carl Johnson Rob Cliffe wrote: > Hi All, > This is my first post to python-dev so I will briefly introduce myself:? My > name is Rob Cliffe and I am a commercial programmer living in London, UK.? I > have some 30 years of programming experience but have only been using Python > for a couple of years. > First I want to say what a fantastic language Python is.? It is THE best > language for development in my opinion, and a joy to use. > > My specific issue: > I eventually got my head round decorator syntax and realised that what came > after the '@' was (basically) a function that took a function as argument > and returned a function as result. > However it seems to me?unPythonesque (i.e. an exception to Python's normal > consistency) that the syntax of what follows the '@' should be restricted to > either a single (function) identifier or a single (function) identifier with > an argument list. > The example I tried, which seems not an unreasonable sort of thing to do, > was along the lines of: > > def deco1(func): > ??? > def deco2(func): > ??? > > DecoList = [deco1, deco2] > > @DecoList[0]??? # NO - CAUSES SYNTAX ERROR > def foo(): > ??? pass > > I am sure other guys have their own examples. > > I am of course not the first person to raise this issue, and I see that > Guido has a "gut feeling" against allowing a general expression after the > '@'. > > BUT - a general expression can be "smuggled in" very easily as a function > argument: > > def Identity(x): return x > > @Identity(DecoList[0])??? # THIS WORKS > def foo(): > ??? pass > > So - the syntax restriction seems not only?inconsistent, but?pointless; it > doesn't forbid anything, but merely means we have to do it in a slightly > convoluted (unPythonesque) way.? So please, Guido, will you reconsider? > > Best wishes > Rob Cliffe From fuzzyman at gmail.com Wed Sep 2 12:49:31 2009 From: fuzzyman at gmail.com (Michael Foord) Date: Wed, 2 Sep 2009 11:49:31 +0100 Subject: [Python-Dev] [Python-ideas] Decorator syntax In-Reply-To: <3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com> References: <3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com> Message-ID: <6f4025010909020349h31573e63if2c112ba471b8872@mail.gmail.com> I actually encountered this for the first time yesterday and didn't realise that the decorator syntax was limited in this way (I was mentally preparing a blog entry when these emails arrived). What I needed to do was turn a Python function into a .NET event handler in IronPython. The simple case is this: from System import EventHandler @EventHandler def on_event(sender, event): # do stuff... This works fine of course, but then I needed to use the 'typed' form which is like this: @EventHandler[HtmlEventArgs] def on_event(sender, event): # do stuff... I didn't realise this was invalid syntax - nor the neat trick with the identity function to bypass the limitation. Michael 2009/9/2 Carl Johnson > Crossposting to Python-ideas, > > I asked for the same change to the grammar a couple months back on > python-ideas. > > See > http://mail.python.org/pipermail/python-ideas/2009-February/thread.html#2787 > > I'm all for it, but you'll have to convince Guido that this won't > result in confusing to read code. My own examples, unfortunately did > not advance your cause, as Guido explained, "My brain hurts trying to > understand all this. I don't think this bodes well as a use case for a > proposed feature." :-D The trouble is that I was using lambdas upon > lambdas to do all kinds of Ruby block-esque tricks. OTOH, if you come > up with some simple, clear use cases though, and I think he might > still be persuadable to make a simple change to the grammar. > > > ? Carl Johnson > > Rob Cliffe wrote: > > > Hi All, > > This is my first post to python-dev so I will briefly introduce myself: > My > > name is Rob Cliffe and I am a commercial programmer living in London, > UK. I > > have some 30 years of programming experience but have only been using > Python > > for a couple of years. > > First I want to say what a fantastic language Python is. It is THE best > > language for development in my opinion, and a joy to use. > > > > My specific issue: > > I eventually got my head round decorator syntax and realised that what > came > > after the '@' was (basically) a function that took a function as argument > > and returned a function as result. > > However it seems to me unPythonesque (i.e. an exception to Python's > normal > > consistency) that the syntax of what follows the '@' should be restricted > to > > either a single (function) identifier or a single (function) identifier > with > > an argument list. > > The example I tried, which seems not an unreasonable sort of thing to do, > > was along the lines of: > > > > def deco1(func): > > > > def deco2(func): > > > > > > DecoList = [deco1, deco2] > > > > @DecoList[0] # NO - CAUSES SYNTAX ERROR > > def foo(): > > pass > > > > I am sure other guys have their own examples. > > > > I am of course not the first person to raise this issue, and I see that > > Guido has a "gut feeling" against allowing a general expression after the > > '@'. > > > > BUT - a general expression can be "smuggled in" very easily as a function > > argument: > > > > def Identity(x): return x > > > > @Identity(DecoList[0]) # THIS WORKS > > def foo(): > > pass > > > > So - the syntax restriction seems not only inconsistent, but pointless; > it > > doesn't forbid anything, but merely means we have to do it in a slightly > > convoluted (unPythonesque) way. So please, Guido, will you reconsider? > > > > Best wishes > > Rob Cliffe > _______________________________________________ > Python-ideas mailing list > Python-ideas at python.org > http://mail.python.org/mailman/listinfo/python-ideas > -- http://www.ironpythoninaction.com/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From xavier.morel at masklinn.net Wed Sep 2 12:50:33 2009 From: xavier.morel at masklinn.net (Xavier Morel) Date: Wed, 2 Sep 2009 12:50:33 +0200 Subject: [Python-Dev] Decorator syntax In-Reply-To: References: Message-ID: <126B4B7C-38AF-45DF-B5F2-DB146E7146C8@masklinn.net> On 2 Sep 2009, at 12:15 , Rob Cliffe wrote: > > @Identity(DecoList[0]) # THIS WORKS > def foo(): > pass For what it's worth, you don't need an id function, you can simply write @itemgetter(0)(decorators) def foo(): 'whatever' or @decorators.__getitem__(0) def foo(): 'whatever' From ncoghlan at gmail.com Wed Sep 2 13:11:40 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 02 Sep 2009 21:11:40 +1000 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <4A9D9A7E.5080209@canterbury.ac.nz> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <4A9D9A7E.5080209@canterbury.ac.nz> Message-ID: <4A9E52EC.3030809@gmail.com> Greg Ewing wrote: > Xavier Morel wrote: > >> I fail to grasp the unpredictability of "the last expression >> evaluated in the body of a function is its return value". > > It's unpredictable in the sense that if you're writing > a function that's not intended to return a value, you're > not thinking about what the last call you make in the > function returns, so to a first approximation it's just > some random value. > > I often write code that makes use of the fact that falling > off the end of a function returns None. This has been a > documented part of the Python language from the beginning, > and changing it would break a lot of code for no good > reason. It also means adding a debugging message, assertion, or otherwise side-effect free statement can change the return value of the function. Not cool. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From kristjan at ccpgames.com Wed Sep 2 14:07:19 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Wed, 2 Sep 2009 12:07:19 +0000 Subject: [Python-Dev] default of returning None hurts performance? In-Reply-To: <4A9E52EC.3030809@gmail.com> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <4A9D9A7E.5080209@canterbury.ac.nz> <4A9E52EC.3030809@gmail.com> Message-ID: <930F189C8A437347B80DF2C156F7EC7F098F163EC4@exchis.ccp.ad.local> Not advocating a change, merely pointing out that it's how Ruby works. K > -----Original Message----- > From: python-dev-bounces+kristjan=ccpgames.com at python.org > [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf > Of Nick Coghlan > Sent: 2. september 2009 11:12 > To: Greg Ewing > Cc: python-dev at python.org > Subject: Re: [Python-Dev] default of returning None hurts performance? > It also means adding a debugging message, assertion, or otherwise > side-effect free statement can change the return value of the function. > Not cool. > From p.f.moore at gmail.com Wed Sep 2 16:16:40 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 2 Sep 2009 15:16:40 +0100 Subject: [Python-Dev] Setting up a buildbot In-Reply-To: <79990c6b0908221308k7d5c87b9m673ea6183946586c@mail.gmail.com> References: <79990c6b0908221228y46f2f592web53226cc33473c4@mail.gmail.com> <20090822194049.GE19190@nexus.in-nomine.org> <79990c6b0908221308k7d5c87b9m673ea6183946586c@mail.gmail.com> Message-ID: <79990c6b0909020716m4148bd46ie7f19a8ac12ec9fb@mail.gmail.com> 2009/8/22 Paul Moore : > 2009/8/22 Jeroen Ruigrok van der Werven : >> -On [20090822 21:30], Paul Moore (p.f.moore at gmail.com) wrote: >>>I've just had a look on python.org, but couldn't immediately see a >>>pointer to instructions on what the process is to set up a buildbot. >> >> http://wiki.python.org/moin/BuildBot comes to mind. > > Ah, thanks. I'll take a look. I saw on planet Python that the buildbots have currently been shut down. I guess this makes my question fairly irrelevant for the moment, then :-( Paul From foom at fuhm.net Wed Sep 2 16:35:29 2009 From: foom at fuhm.net (James Y Knight) Date: Wed, 2 Sep 2009 10:35:29 -0400 Subject: [Python-Dev] Decorator syntax In-Reply-To: References: Message-ID: On Sep 2, 2009, at 6:15 AM, Rob Cliffe wrote: > So - the syntax restriction seems not only inconsistent, but > pointless; it doesn't forbid anything, but merely means we have to > do it in a slightly convoluted (unPythonesque) way. So please, > Guido, will you reconsider? Indeed, it's a silly inconsistent restriction. When it was first added I too suggested that any expression be allowed after the @, rather than having a uniquely special restricted syntax. I argued from consistency of grammar standpoint. But Guido was not persuaded. Good luck to you. :) Here's some of the more relevant messages from the thread back when the @decorator feature was first introduced: http://mail.python.org/pipermail/python-dev/2004-August/046654.html http://mail.python.org/pipermail/python-dev/2004-August/046659.html http://mail.python.org/pipermail/python-dev/2004-August/046675.html http://mail.python.org/pipermail/python-dev/2004-August/046711.html http://mail.python.org/pipermail/python-dev/2004-August/046741.html http://mail.python.org/pipermail/python-dev/2004-August/046753.html http://mail.python.org/pipermail/python-dev/2004-August/046818.html James From bxs187 at case.edu Wed Sep 2 17:19:11 2009 From: bxs187 at case.edu (Boya Sun) Date: Wed, 2 Sep 2009 11:19:11 -0400 Subject: [Python-Dev] Bugs discovered by researchers at Case Western Reserve University Message-ID: <3efbea50909020819x5aba605fid2a78887894ddccf@mail.gmail.com> Dear Developers, I am a Ph.D student from Case Western Reserve University, specialized at software engineering. Our recent approach analyzes bugs that are being fixed in the issue database, and tries to discover any latent bug instances that are the same as the fixed bug but are left unfixed. We have found some spurious code in your project in this approach, and pointed out these code by comments (sometimes also with patches) to the fixed bugs in the issue DB from which it is discovered as follows: Issue 6817: http://bugs.python.org/issue6817 (A new issue, created following Amaury's comments) Issue 2620: http://bugs.python.org/issue2620 Issue 3139: http://bugs.python.org/issue3139 Issue 5705: http://bugs.python.org/issue5705 We hope that we have discovered some real bugs for you. Any comments or suggestions are GREATLY appreciated, since your opinions are very, very precious to us. Boya -- BOYA SUN Computer Science Division Electrical Engineering & Computer Science Department 513 Olin Building Case Western Reserve University 10900 Euclid Avenue Cleveland, OH 44106 boya.sun at case.edu -------------- next part -------------- An HTML attachment was scrubbed... URL: From hyugaricdeau at gmail.com Wed Sep 2 18:06:34 2009 From: hyugaricdeau at gmail.com (Erik Bray) Date: Wed, 2 Sep 2009 12:06:34 -0400 Subject: [Python-Dev] Decorator syntax In-Reply-To: References: Message-ID: On Wed, Sep 2, 2009 at 10:35 AM, James Y Knight wrote: > On Sep 2, 2009, at 6:15 AM, Rob Cliffe wrote: > >> So - the syntax restriction seems not only inconsistent, but pointless; it >> doesn't forbid anything, but merely means we have to do it in a slightly >> convoluted (unPythonesque) way. ?So please, Guido, will you reconsider? > > Indeed, it's a silly inconsistent restriction. When it was first added I too > suggested that any expression be allowed after the @, rather than having a > uniquely special restricted syntax. I argued from consistency of grammar > standpoint. But Guido was not persuaded. Good luck to you. :) > > Here's some of the more relevant messages from the thread back when the > @decorator feature was first introduced: > http://mail.python.org/pipermail/python-dev/2004-August/046654.html > http://mail.python.org/pipermail/python-dev/2004-August/046659.html > http://mail.python.org/pipermail/python-dev/2004-August/046675.html > http://mail.python.org/pipermail/python-dev/2004-August/046711.html > http://mail.python.org/pipermail/python-dev/2004-August/046741.html > http://mail.python.org/pipermail/python-dev/2004-August/046753.html > http://mail.python.org/pipermail/python-dev/2004-August/046818.html I think Guido may have a point about not allowing any arbitrary expression. But I do think that if it allows calls, it should also at least support the itemgetter syntax, for which there seems to be a demonstrable use case. But that's just adding on another special case, so it might be simpler to allow arbitrary expressions. From eric at trueblade.com Wed Sep 2 18:11:48 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 02 Sep 2009 12:11:48 -0400 Subject: [Python-Dev] Decorator syntax In-Reply-To: References: Message-ID: <4A9E9944.6020900@trueblade.com> Erik Bray wrote: > I think Guido may have a point about not allowing any arbitrary > expression. But I do think that if it allows calls, it should also at > least support the itemgetter syntax, for which there seems to be a > demonstrable use case. But that's just adding on another special > case, so it might be simpler to allow arbitrary expressions. It's not the same issue, of course, but note that str.format's object specification "language" supports item and attribute access. And that's the extent of what it allows. If you wanted to do this for decorators, I'm not sure how easy it would be to restrict the expression inside the brackets, or if you'd even want to. str.format has all of this baked-in, it doesn't use any sort of grammar. Eric From python at mrabarnett.plus.com Wed Sep 2 18:36:54 2009 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 02 Sep 2009 17:36:54 +0100 Subject: [Python-Dev] Decorator syntax In-Reply-To: References: Message-ID: <4A9E9F26.8030608@mrabarnett.plus.com> James Y Knight wrote: > On Sep 2, 2009, at 6:15 AM, Rob Cliffe wrote: > >> So - the syntax restriction seems not only inconsistent, but >> pointless; it doesn't forbid anything, but merely means we have to do >> it in a slightly convoluted (unPythonesque) way. So please, Guido, >> will you reconsider? > > Indeed, it's a silly inconsistent restriction. When it was first added I > too suggested that any expression be allowed after the @, rather than > having a uniquely special restricted syntax. I argued from consistency > of grammar standpoint. But Guido was not persuaded. Good luck to you. :) > [snip] I can see no syntactic reason to restrict what can appear after the @. If someone chooses to abuse it then that's unPythonic, but not illegal. From martin at v.loewis.de Wed Sep 2 20:57:52 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 02 Sep 2009 20:57:52 +0200 Subject: [Python-Dev] Setting up a buildbot In-Reply-To: <79990c6b0909020716m4148bd46ie7f19a8ac12ec9fb@mail.gmail.com> References: <79990c6b0908221228y46f2f592web53226cc33473c4@mail.gmail.com> <20090822194049.GE19190@nexus.in-nomine.org> <79990c6b0908221308k7d5c87b9m673ea6183946586c@mail.gmail.com> <79990c6b0909020716m4148bd46ie7f19a8ac12ec9fb@mail.gmail.com> Message-ID: <4A9EC030.3010605@v.loewis.de> > I saw on planet Python that the buildbots have currently been shut > down. I guess this makes my question fairly irrelevant for the moment, > then :-( That was a misunderstanding. It was only the community buildbots, and Grig Gheorghiu is working on restoring them. Regards, Martin From ideasman42 at gmail.com Wed Sep 2 21:15:08 2009 From: ideasman42 at gmail.com (Campbell Barton) Date: Wed, 2 Sep 2009 12:15:08 -0700 Subject: [Python-Dev] Py_CmpToRich removed, alternatives for comparison? Message-ID: <7c1ab96d0909021215n46e50713v850bd6d5e08fcc8f@mail.gmail.com> For blender we have a number of types defined in the C/API like meshes, lamps, metaballs, nurbs etc that dont make sense with some of richcmp's operations. A problem I have is that in python 3.1 the Py_CmpToRich function is removed. Should we copy Py_CmpToRich into our source tree? Otherwise we have fairly long cmp functions that have Py_CmpToRich inline. For C extension writers what is the suggested method for comparing types where Py_LT, Py_GT Py_GE etc are not useful? Since people sometimes ask why use py3.1, we're doing a rewrite that wont be ready for quite some time. -- - Campbell From brett at python.org Wed Sep 2 22:43:50 2009 From: brett at python.org (Brett Cannon) Date: Wed, 2 Sep 2009 13:43:50 -0700 Subject: [Python-Dev] Bugs discovered by researchers at Case Western Reserve University In-Reply-To: <3efbea50909020819x5aba605fid2a78887894ddccf@mail.gmail.com> References: <3efbea50909020819x5aba605fid2a78887894ddccf@mail.gmail.com> Message-ID: On Wed, Sep 2, 2009 at 08:19, Boya Sun wrote: > Dear Developers, > > I am a Ph.D student from Case Western Reserve University, specialized at > software engineering.? Our recent approach analyzes bugs that are being > fixed in the issue database, and tries to discover any latent bug instances > that are the same as the fixed bug but are left unfixed.? We have found some > spurious code in your project in this approach, and pointed out these code > by comments (sometimes also with patches) to the fixed bugs in the issue DB > from which it is discovered as follows: > > Issue 6817: http://bugs.python.org/issue6817 (A new issue, created following > Amaury's comments) > Issue 2620: http://bugs.python.org/issue2620 > Issue 3139: http://bugs.python.org/issue3139 > Issue 5705: http://bugs.python.org/issue5705 I quickly re-opened 2620 and 5705 so we don't lose track of the fact that new code has been attached. -Brett From brett at python.org Wed Sep 2 22:47:53 2009 From: brett at python.org (Brett Cannon) Date: Wed, 2 Sep 2009 13:47:53 -0700 Subject: [Python-Dev] Decorator syntax In-Reply-To: <3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com> References: <3bdda690909020334m4c81cecdm8f57dce54c2da398@mail.gmail.com> Message-ID: People, please note that the discussion has shifted to python-ideas and further comments should happen over there. Carl did the right thing to shift it there, although cross-posting once the conversation redirection has occurred is not needed. hoping-google-wave-will-have-a-permanent-redirector-for-conversations-ly yrs, Brett On Wed, Sep 2, 2009 at 03:34, Carl Johnson wrote: > Crossposting to Python-ideas, > > I asked for the same change to the grammar a couple months back on > python-ideas. > > See http://mail.python.org/pipermail/python-ideas/2009-February/thread.html#2787 > > I'm all for it, but you'll have to convince Guido that this won't > result in confusing to read code. My own examples, unfortunately did > not advance your cause, as Guido explained, "My brain hurts trying to > understand all this. I don't think this bodes well as a use case for a > proposed feature." :-D The trouble is that I was using lambdas upon > lambdas to do all kinds of Ruby block-esque tricks. OTOH, if you come > up with some simple, clear use cases though, and I think he might > still be persuadable to make a simple change to the grammar. > > > ? Carl Johnson > > Rob Cliffe wrote: > >> Hi All, >> This is my first post to python-dev so I will briefly introduce myself:? My >> name is Rob Cliffe and I am a commercial programmer living in London, UK.? I >> have some 30 years of programming experience but have only been using Python >> for a couple of years. >> First I want to say what a fantastic language Python is.? It is THE best >> language for development in my opinion, and a joy to use. >> >> My specific issue: >> I eventually got my head round decorator syntax and realised that what came >> after the '@' was (basically) a function that took a function as argument >> and returned a function as result. >> However it seems to me?unPythonesque (i.e. an exception to Python's normal >> consistency) that the syntax of what follows the '@' should be restricted to >> either a single (function) identifier or a single (function) identifier with >> an argument list. >> The example I tried, which seems not an unreasonable sort of thing to do, >> was along the lines of: >> >> def deco1(func): >> ??? >> def deco2(func): >> ??? >> >> DecoList = [deco1, deco2] >> >> @DecoList[0]??? # NO - CAUSES SYNTAX ERROR >> def foo(): >> ??? pass >> >> I am sure other guys have their own examples. >> >> I am of course not the first person to raise this issue, and I see that >> Guido has a "gut feeling" against allowing a general expression after the >> '@'. >> >> BUT - a general expression can be "smuggled in" very easily as a function >> argument: >> >> def Identity(x): return x >> >> @Identity(DecoList[0])??? # THIS WORKS >> def foo(): >> ??? pass >> >> So - the syntax restriction seems not only?inconsistent, but?pointless; it >> doesn't forbid anything, but merely means we have to do it in a slightly >> convoluted (unPythonesque) way.? So please, Guido, will you reconsider? >> >> Best wishes >> Rob Cliffe > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org > From benjamin at python.org Thu Sep 3 00:19:46 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 2 Sep 2009 17:19:46 -0500 Subject: [Python-Dev] Py_CmpToRich removed, alternatives for comparison? In-Reply-To: <7c1ab96d0909021215n46e50713v850bd6d5e08fcc8f@mail.gmail.com> References: <7c1ab96d0909021215n46e50713v850bd6d5e08fcc8f@mail.gmail.com> Message-ID: <1afaf6160909021519w35c567a9r36aebf203ae64392@mail.gmail.com> 2009/9/2 Campbell Barton : > For blender we have a number of types defined in the C/API like > meshes, lamps, metaballs, nurbs etc that dont make sense with some of > richcmp's operations. > A problem I have is that in python 3.1 the Py_CmpToRich function is removed. > > Should we copy Py_CmpToRich into our source tree? > Otherwise we have fairly long cmp functions that have Py_CmpToRich inline. > > For C extension writers what is the suggested method for comparing > types where Py_LT, Py_GT Py_GE etc are not useful? Return Py_NotImplemented when you get them. -- Regards, Benjamin From olemis at gmail.com Thu Sep 3 20:14:38 2009 From: olemis at gmail.com (Olemis Lang) Date: Thu, 3 Sep 2009 13:14:38 -0500 Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId Message-ID: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com> I realized that PyPI accepts MyOpenId and tried to login to the site. In the process I get this message : {{{ OpenID provider did not provide your email address }}} I mean, is it mandatory to provide the e-mail address in order to bind a user to an OpenId ? I'm curious : I'd like to know if there's a reason to do that. Thnx in advance ! PS: I know is a little bit OT. Hope you dont mind ... -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From martin at v.loewis.de Thu Sep 3 20:49:48 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 03 Sep 2009 20:49:48 +0200 Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId In-Reply-To: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com> References: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com> Message-ID: <4AA00FCC.5080606@v.loewis.de> Olemis Lang wrote: > I realized that PyPI accepts MyOpenId and tried to login to the site. > In the process I get this message : > > {{{ > OpenID provider did not provide your email address > }}} > > I mean, is it mandatory to provide the e-mail address in order to bind > a user to an OpenId ? > I'm curious : I'd like to know if there's a reason to do that. For an existing user? No, there is no reason. For a new user, yes, we mandate a valid, verified email address. > PS: I know is a little bit OT. Hope you dont mind ... For general PyPI discussions, please use catalog-sig. For PyPI bug reports, use the bug tracker. Regards, Martin From greg.ewing at canterbury.ac.nz Thu Sep 3 23:33:14 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 04 Sep 2009 09:33:14 +1200 Subject: [Python-Dev] [OT] implicit return values In-Reply-To: References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net> <1251811506.5631.44.camel@localhost> <4A9D9BD1.8000501@canterbury.ac.nz> Message-ID: <4AA0361A.8020904@canterbury.ac.nz> Xavier Morel wrote: > Methods yes (and that's one of the few Smalltalk design "features" I > consider truly dumb, considering it has message cascading) Cascading is something different -- it's for sending multiple messages to the *same* receiver. It's not dumb to have both. -- Greg From ben+python at benfinney.id.au Fri Sep 4 02:22:52 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 04 Sep 2009 10:22:52 +1000 Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId References: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com> <4AA00FCC.5080606@v.loewis.de> Message-ID: <87ab1b7e3n.fsf@benfinney.id.au> "Martin v. L?wis" writes: > For general PyPI discussions, please use catalog-sig. For PyPI bug > reports, use the bug tracker. The Python bug tracker ? Or the tracker linked from which leads to ? The Python bug tracker has issues with OpenID (it doesn't allow OpenID login at all, AFAICT), and the latter times out attempting a connection for me. -- \ ?We must become the change we want to see.? ?Mahatma Gandhi | `\ | _o__) | Ben Finney From aahz at pythoncraft.com Fri Sep 4 05:40:53 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 3 Sep 2009 20:40:53 -0700 Subject: [Python-Dev] REMINDER: PyCon 2010: Call for Proposals Message-ID: <20090904034053.GA6855@panix.com> Call for proposals -- PyCon 2010 -- =============================================================== Due date: October 1st, 2009 Want to showcase your skills as a Python Hacker? Want to have hundreds of people see your talk on the subject of your choice? Have some hot button issue you think the community needs to address, or have some package, code or project you simply love talking about? Want to launch your master plan to take over the world with python? PyCon is your platform for getting the word out and teaching something new to hundreds of people, face to face. Previous PyCon conferences have had a broad range of presentations, from reports on academic and commercial projects, tutorials on a broad range of subjects and case studies. All conference speakers are volunteers and come from a myriad of backgrounds. Some are new speakers, some are old speakers. Everyone is welcome so bring your passion and your code! We're looking to you to help us top the previous years of success PyCon has had. PyCon 2010 is looking for proposals to fill the formal presentation tracks. The PyCon conference days will be February 19-22, 2010 in Atlanta, Georgia, preceded by the tutorial days (February 17-18), and followed by four days of development sprints (February 22-25). Online proposal submission is open now! Proposals will be accepted through October 1st, with acceptance notifications coming out on November 15th. For the detailed call for proposals, please see: For videos of talks from previous years - check out: We look forward to seeing you in Atlanta! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Look, it's your affair if you want to play with five people, but don't go calling it doubles." --John Cleese anticipates Usenet From martin at v.loewis.de Fri Sep 4 07:18:48 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 04 Sep 2009 07:18:48 +0200 Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId In-Reply-To: <87ab1b7e3n.fsf@benfinney.id.au> References: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com> <4AA00FCC.5080606@v.loewis.de> <87ab1b7e3n.fsf@benfinney.id.au> Message-ID: <4AA0A338.2070506@v.loewis.de> >> For general PyPI discussions, please use catalog-sig. For PyPI bug >> reports, use the bug tracker. > > The Python bug tracker ? Or the tracker > linked from which leads > to ? The latter. The former is for bugs in Python only. > The Python bug tracker has issues with OpenID (it doesn't allow OpenID > login at all, AFAICT) That alone wouldn't make it unsuitable for reporting problems with PyPI, including problems with OpenID, right? > and the latter times out attempting a connection for me. Please try again; it does work most of the time (if it still doesn't work, check your internet connection). Regards, Martin From ben+python at benfinney.id.au Fri Sep 4 07:53:31 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Fri, 04 Sep 2009 15:53:31 +1000 Subject: [Python-Dev] OT : Cannot login to PyPI using MyOpenId References: <24ea26600909031114s4c8733a5y5c2ec333ceff5340@mail.gmail.com> <4AA00FCC.5080606@v.loewis.de> <87ab1b7e3n.fsf@benfinney.id.au> <4AA0A338.2070506@v.loewis.de> Message-ID: <87ws4f5k84.fsf@benfinney.id.au> "Martin v. L?wis" writes: > > The Python bug tracker ? Or the tracker > > linked from which > > leads to > > ? > > The latter. The former is for bugs in Python only. [?] > > and the latter times out attempting a connection for me. > > Please try again; it does work most of the time (if it still doesn't > work, check your internet connection). My internet connection is fine, I'm able to visit various sites without a problem. That tracker loads fine, but when I select ?Log In?, it takes a very long time and eventually times out. This is repeatable (for me) multiple times over the past 12 hours or so. This is probably a SourceForge problem and nothing you can fix directly, but those wanting to receive bug reports into that tracker might be interested to know. -- \ ?If you can't hear me sometimes, it's because I'm in | `\ parentheses.? ?Steven Wright | _o__) | Ben Finney From catch-all at masklinn.net Fri Sep 4 08:34:44 2009 From: catch-all at masklinn.net (Xavier Morel) Date: Fri, 4 Sep 2009 08:34:44 +0200 Subject: [Python-Dev] [OT] implicit return values In-Reply-To: <4AA0361A.8020904@canterbury.ac.nz> References: <52dc1c820908311412q55efc3cejc62b3f1778c4c92a@mail.gmail.com> <4A9C6457.2060609@canterbury.ac.nz> <52dc1c820908311803s6015296fsa27aa4bf96d4f068@mail.gmail.com> <7412EEF1-5062-41D4-B454-1C53C0B45466@masklinn.net> <1251811506.5631.44.camel@localhost> <4A9D9BD1.8000501@canterbury.ac.nz> <4AA0361A.8020904@canterbury.ac.nz> Message-ID: On 3 Sep 2009, at 23:33 , Greg Ewing wrote: Xavier Morel wrote: > >> Methods yes (and that's one of the few Smalltalk design "features" >> I consider truly dumb, considering it has message cascading) > > Cascading is something different -- it's for sending > multiple messages to the *same* receiver. It's not > dumb to have both. I know what cascading is for. The issue is that with message cascading + the "yourself" message, you *never* need to chain on self (you can just cascade and -- if you end up needing the instance to drop down at the end of the cascade -- send `yourself`). Chaining on self is completely redundant in smalltalk as the purpose of this pattern is *also* to send a sequence of messages to the same receiver (something message cascading already handles & guarantees). Therefore defaulting method to self-chaining is very dumb and serves no purpose whatsoever. It doesn't make the language easier to use, less verbose or more practical. It just wastes return values. From kristjan at ccpgames.com Fri Sep 4 11:19:09 2009 From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=) Date: Fri, 4 Sep 2009 09:19:09 +0000 Subject: [Python-Dev] PyMem_Malloc() vs PyObject_Malloc() Message-ID: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local> I noticed something (in 2.5) yesterday, which may be a feature, but is more likely a bug. In tokenizer.c, tok->encoding is allocated using PyMem_MALLOC(). However, this then gets handed to a node->r_str in parsetok.c, and then released in node.c using PyObject_Free(). Now, by coincidence, PyObject_Free() will default to free() for objects that it doesn't recognize, so this works. But is this documented behavior? The reason I ran into this was that I had redirect the PyMem_* API to a different allocator, but left the PyObject_* one alone. My feeling Is that these two APIs shouldn't be interchangeable. Especially since you can't hand a PyObject_Malloc'd object to PyMem_Free() so the inverse shouldn't be expected to work. Any thoughts? Kristj?n -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Fri Sep 4 11:27:58 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Fri, 04 Sep 2009 11:27:58 +0200 Subject: [Python-Dev] PyMem_Malloc() vs PyObject_Malloc() In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local> References: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local> Message-ID: <4AA0DD9E.9090501@egenix.com> Kristj?n Valur J?nsson wrote: > > I noticed something (in 2.5) yesterday, which may be a feature, but is more likely a bug. > In tokenizer.c, tok->encoding is allocated using PyMem_MALLOC(). > However, this then gets handed to a node->r_str in parsetok.c, and then released in node.c using PyObject_Free(). > > Now, by coincidence, PyObject_Free() will default to free() for objects that it doesn't recognize, so this works. But is this documented behavior? The reason I ran into this was that I had redirect the PyMem_* API to a different allocator, but left the PyObject_* one alone. > > My feeling Is that these two APIs shouldn't be interchangeable. Especially since you can't hand a PyObject_Malloc'd object to PyMem_Free() so the inverse shouldn't be expected to work. > > Any thoughts? This is a bug. Please file a bug report for this. In general, either PyObject_* xor PyMem_* memory API should used. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 04 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ncoghlan at gmail.com Fri Sep 4 11:45:08 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 04 Sep 2009 19:45:08 +1000 Subject: [Python-Dev] PyMem_Malloc() vs PyObject_Malloc() In-Reply-To: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local> References: <930F189C8A437347B80DF2C156F7EC7F098F164308@exchis.ccp.ad.local> Message-ID: <4AA0E1A4.2060906@gmail.com> Kristj?n Valur J?nsson wrote: > My feeling Is that these two APIs shouldn?t be interchangeable. > Especially since you can?t hand a PyObject_Malloc?d object to > PyMem_Free() so the inverse shouldn?t be expected to work. I thought this had officially been deemed illegal for a while, and Google found the reference I was looking for in the What's New for 2.5: """Previously these different families all reduced to the platform's malloc() and free() functions. This meant it didn't matter if you got things wrong and allocated memory with the PyMem function but freed it with the PyObject function. With 2.5's changes to obmalloc, these families now do different things and mismatches will probably result in a segfault. You should carefully test your C extension modules with Python 2.5.""" So either the allocation or the release needs to change here. The behaviour of PyObject_Del when handed a pointer it doesn't recognise is currently undocumented. It may be best to make it officially undefined in order to further discourage people from relying on the implicit delegation to PyMem_Free. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From chris at simplistix.co.uk Fri Sep 4 13:11:46 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 04 Sep 2009 12:11:46 +0100 Subject: [Python-Dev] how to debug httplib slowness In-Reply-To: References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> Message-ID: <4AA0F5F2.6030903@simplistix.co.uk> Guido van Rossum wrote: >>> You might see a pattern. Is this on Windows? >> Well, yes, but I'm not 100%. The problematic machine is a Windows box, but >> there are no non-windows boxes on that network and vpn'ing from one of my >> non-windows boxes slows things down enough that I'm not confident what I'd >> be seeing was indicative of the same problem... > > Time to set up a more conclusive test. Do you have something like curl > or wget available on the same box? Time taken with IE: ~2 seconds Time taken with wget: 2.2 seconds Time taken with Python [1]: 20-30 minutes I did a run of the script through cProfile and got the following: pstats.Stats('download.profile').strip_dirs().sort_stats('time').print_stats(10) 1604545 function calls in 1956.057 CPU seconds ncalls tottime percall cumtime percall filename:lineno(function) 1 1950.767 1950.767 1955.952 1955.952 httplib.py:544(_read_chunked) 85125 1.235 0.000 1.235 0.000 {method 'recv' of '_socket.socket' objects} 85838 1.031 0.000 2.246 0.000 socket.py:313(read) 85838 0.787 0.000 3.386 0.000 httplib.py:601(_safe_read) 42928 0.614 0.000 1.779 0.000 socket.py:373(readline) 128775 0.344 0.000 0.344 0.000 {method 'write' of 'cStringIO.StringO' objects} 200796 0.206 0.000 0.206 0.000 {method 'seek' of 'cStringIO.StringO' objects} 85838 0.179 0.000 0.179 0.000 {min} 128767 0.135 0.000 0.135 0.000 {cStringIO.StringIO} 72735 0.116 0.000 0.116 0.000 {method 'read' of 'cStringIO.StringO' objects} ...which isn't what I was expecting! Am I right in reading this as most of the time is being spent in httplib's HTTPResponse._read_chunked and none of the methods it calls? If so, is there a better way that a bunch of print statements to find where in that method the time is being spent? cheers, Chris [1] Python 2.6.2 on Windows Server 2003 R2 running this script: from base64 import encodestring from httplib import HTTPConnection from datetime import datetime conn = HTTPConnection('servername') headers = {} a = 'Basic '+encodestring('username:password').strip() headers['Authorization']=a t = datetime.now() print t conn.request('GET','/some/big/file',None,headers) print 'request:',datetime.now()-t response = conn.getresponse() print 'response:',datetime.now()-t data = response.read() if len(data)<2000: print data print 'read:',datetime.now()-t -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From hodgestar+pythondev at gmail.com Fri Sep 4 13:28:35 2009 From: hodgestar+pythondev at gmail.com (Simon Cross) Date: Fri, 4 Sep 2009 13:28:35 +0200 Subject: [Python-Dev] how to debug httplib slowness In-Reply-To: <4AA0F5F2.6030903@simplistix.co.uk> References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> Message-ID: On Fri, Sep 4, 2009 at 1:11 PM, Chris Withers wrote: > Am I right in reading this as most of the time is being spent in httplib's > HTTPResponse._read_chunked and none of the methods it calls? > > If so, is there a better way that a bunch of print statements to find where > in that method the time is being spent? Well, since the source for _read_chunked includes the comment # XXX This accumulates chunks by repeated string concatenation, # which is not efficient as the number or size of chunks gets big. you might gain some speed improvement with minimal effort by gathering the read data chunks into a list and then returning "".join(chunks) at the end. Schiavo Simon From solipsis at pitrou.net Fri Sep 4 13:40:32 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 4 Sep 2009 11:40:32 +0000 (UTC) Subject: [Python-Dev] how to debug httplib slowness References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> Message-ID: Simon Cross gmail.com> writes: > > Well, since the source for _read_chunked includes the comment > > # XXX This accumulates chunks by repeated string concatenation, > # which is not efficient as the number or size of chunks gets big. > > you might gain some speed improvement with minimal effort by gathering > the read data chunks into a list and then returning "".join(chunks) at > the end. +1 for trying this. Given differences between platforms in realloc() performance, it might be the reason why it goes unnoticed under Linux but degenerates under Windows. As a sidenote, it is interesting that even an stdlib module makes this mistake and acknowledges it without trying to fix it. From chris at simplistix.co.uk Fri Sep 4 17:02:39 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 04 Sep 2009 16:02:39 +0100 Subject: [Python-Dev] how to debug httplib slowness In-Reply-To: References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> Message-ID: <4AA12C0F.1090305@simplistix.co.uk> Simon Cross wrote: > Well, since the source for _read_chunked includes the comment > > # XXX This accumulates chunks by repeated string concatenation, > # which is not efficient as the number or size of chunks gets big. > > you might gain some speed improvement with minimal effort by gathering > the read data chunks into a list and then returning "".join(chunks) at > the end. True, I'll be trying that and reporting back, but, more interestingly, I did some analysis with wireshark (only 200MB-odd of .pcap logs that was fun ;-) to see the differences in the http conversation and noticed more interestingness... So, httplib does this: GET / HTTP/1.1 Host: Accept-Encoding: identity Authorization: Basic HTTP/1.1 200 OK Date: Fri, 04 Sep 2009 11:44:22 GMT Server: Apache-Coyote/1.1 ContentLength: 116245504 Content-Type: application/vnd.excel Transfer-Encoding: chunked While wget does this: GET / HTTP/1.0 User-Agent: Wget/1.11.4 Accept: */* Host: Connection: Keep-Alive Authorization: Basic HTTP/1.1 200 OK Date: Fri, 04 Sep 2009 11:35:19 GMT Server: Apache-Coyote/1.1 ContentLength: 116245504 Content-Type: application/vnd.excel Connection: close Interesting points: - Apache in this instance responds with HTTP 1.1, even though the wget request was 1.0, is that allowed? - Apache responds with a chunked response only to httplib. Why is that? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From phd at phd.pp.ru Fri Sep 4 17:32:41 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Fri, 4 Sep 2009 19:32:41 +0400 Subject: [Python-Dev] how to debug httplib slowness In-Reply-To: <4AA12C0F.1090305@simplistix.co.uk> References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> <4AA12C0F.1090305@simplistix.co.uk> Message-ID: <20090904153241.GB7342@phd.pp.ru> On Fri, Sep 04, 2009 at 04:02:39PM +0100, Chris Withers wrote: > So, httplib does this: > > GET / HTTP/1.1 [skip] > While wget does this: > > GET / HTTP/1.0 [skip] > - Apache responds with a chunked response only to httplib. Why is that? Probably because wget uses HTTP/1.0? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From scott+python-dev at scottdial.com Fri Sep 4 18:03:16 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Fri, 04 Sep 2009 12:03:16 -0400 Subject: [Python-Dev] how to debug httplib slowness In-Reply-To: <4AA12C0F.1090305@simplistix.co.uk> References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> <4AA12C0F.1090305@simplistix.co.uk> Message-ID: <4AA13A44.3010008@scottdial.com> Chris Withers wrote: > - Apache in this instance responds with HTTP 1.1, even though the wget > request was 1.0, is that allowed? > > - Apache responds with a chunked response only to httplib. Why is that? > I find it very confusing that you say "Apache" since your really want to say "Coyote" which is to say "Tomcat". Http11Processor.java at 1547: if (entityBody && http11 && keepAlive) { outputBuffer.addActiveFilter (outputFilters[Constants.CHUNKED_FILTER]); contentDelimitation = true; headers.addValue( Constants.TRANSFERENCODING).setString(Constants.CHUNKED); } else { outputBuffer.addActiveFilter (outputFilters[Constants.IDENTITY_FILTER]); } So, as Oleg said, it's because httplib talks HTTP/1.1 and wget talks HTTP/1.0. All HTTP/1.1 client must support chunked transfer-encoding, and apparently Tomcat/Coyote defaults to that unless it is either an empty message, not a HTTP/1.1 client, or the request is not to be kept alive ("Connection: close" or no more keep-alive slots on the server). As Simon said, changing this to do ''.join(chunks) is really the best first step to take. -Scott -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From status at bugs.python.org Fri Sep 4 18:08:26 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 4 Sep 2009 18:08:26 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090904160826.85D3C785E8@psf.upfronthosting.co.za> ACTIVITY SUMMARY (08/28/09 - 09/04/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2374 open (+24) / 16285 closed (+18) / 18659 total (+42) Open issues with patches: 939 Average duration of open issues: 661 days. Median duration of open issues: 415 days. Open Issues Breakdown open 2341 (+23) pending 32 ( +1) Issues Created Or Reopened (45) _______________________________ Multiple buffer overflows in unicode processing 09/02/09 http://bugs.python.org/issue2620 reopened brett.cannon patch os.getpwent returns unsigned 32bit value, os.setuid refuses it 09/02/09 http://bugs.python.org/issue5705 reopened brett.cannon patch, 64bit math.log, log10 inconsistency 08/28/09 CLOSED http://bugs.python.org/issue6765 reopened tjreedy invalid print in tkinter\test\test_ttk\test_widgets.py 08/28/09 CLOSED http://bugs.python.org/issue6796 created keithc When Beginning Expression with Lookahead Assertion I get no Matc 08/28/09 CLOSED http://bugs.python.org/issue6797 created jwindle Argument for sys.settrace() callbacks documented incorrectly 08/28/09 http://bugs.python.org/issue6798 created robert.kern mimetypes does not give cannonical extension for guess_extension 08/29/09 http://bugs.python.org/issue6799 created ptarjan os.exec* raises "OSError: [Errno 45] Operation not supported" in 08/29/09 http://bugs.python.org/issue6800 created rnk symmetric_difference_update documentation fix 08/30/09 CLOSED http://bugs.python.org/issue6801 reopened r.david.murray build fails on Snow Leopard 08/29/09 http://bugs.python.org/issue6802 created jmr patch Context manager docs refer to contextlib.contextmanager as conte 08/29/09 CLOSED http://bugs.python.org/issue6803 created dhaffey patch IDLE: Detect Python files even if name doesn't end in .py 08/29/09 http://bugs.python.org/issue6804 created gagenellina patch Should be URL for documentation of current release of Python 3 ( 08/30/09 CLOSED http://bugs.python.org/issue6805 created MLModel test_platform fails under Snow Leopard 08/30/09 CLOSED http://bugs.python.org/issue6806 created brett.cannon easy No such file or directory: 'msisupport.dll' in msi.py 08/30/09 http://bugs.python.org/issue6807 created pds python 3.1 documentation tutorial classes 08/30/09 CLOSED http://bugs.python.org/issue6808 created tom_morse Python string.lstrip bug? 08/31/09 CLOSED http://bugs.python.org/issue6809 created mushywushy add link to the documentation of signal.signal 08/31/09 CLOSED http://bugs.python.org/issue6810 created Yinon add a filename argument to marshal.load* 08/31/09 http://bugs.python.org/issue6811 created brett.cannon Snow Leopard python program fails because _PyType_Modified is mi 08/31/09 http://bugs.python.org/issue6812 created soshea update format() documentation and tutorial 09/01/09 CLOSED http://bugs.python.org/issue6813 created grooverdan patch xrange removal from documentation 09/01/09 CLOSED http://bugs.python.org/issue6814 created grooverdan patch UnicodeDecodeError in os.path.expandvars 09/01/09 http://bugs.python.org/issue6815 created shura_zam Provide CPython command line functionality via runpy module 09/01/09 http://bugs.python.org/issue6816 created ncoghlan char buffer in function posix_getcwdu should not be fix length 09/02/09 http://bugs.python.org/issue6817 created boya patch remove/delete method for zipfile/tarfile objects 09/02/09 http://bugs.python.org/issue6818 created rossmclendon Typo in datamodel.rst ("Custon" -> "Custom") 09/02/09 CLOSED http://bugs.python.org/issue6819 created jon patch Redefinition of HAVE_STRFTIME can cause compiler errors. 09/02/09 http://bugs.python.org/issue6820 created rgpitts incorrect doc for PyBuffer_Release 09/02/09 CLOSED http://bugs.python.org/issue6821 created egreen Error calling .storlines from ftplib 09/02/09 http://bugs.python.org/issue6822 created rdevaughn time.strftime does unnecessary range check 09/02/09 http://bugs.python.org/issue6823 created rshapiro easy help for a module should list supported platforms 09/02/09 http://bugs.python.org/issue6824 created rickysarraf Minor documentation bug with os.path.split 09/02/09 http://bugs.python.org/issue6825 created evaned Add __contains__ to range type 09/02/09 CLOSED http://bugs.python.org/issue6826 created hpesoj deepcopy erroneously doesn't call __setstate__ if __getstate__ r 09/02/09 http://bugs.python.org/issue6827 created shauncutts wrongly highlighted blocks in the Tutorial 09/03/09 CLOSED http://bugs.python.org/issue6828 created gagenellina patch Frendly error message when inheriting from function 09/04/09 http://bugs.python.org/issue6829 reopened techtonik easy Some uniformness in defaultdict 09/03/09 CLOSED http://bugs.python.org/issue6830 created t-kamiya 2to3 assignment division conversion 09/03/09 http://bugs.python.org/issue6831 created highfestiva Outputting unicode crushes when printing to file on Linux 09/03/09 CLOSED http://bugs.python.org/issue6832 created Orlowski incorrect: failed local variable referenced before assignment 09/03/09 CLOSED http://bugs.python.org/issue6833 created paultjuhatwork use different mechanism for pythonw on osx 09/03/09 http://bugs.python.org/issue6834 created ronaldoussoren easy doctest problem with decorated function when decorator is define 09/03/09 http://bugs.python.org/issue6835 created goetzpf Mismatching use of memory APIs 09/04/09 http://bugs.python.org/issue6836 created krisvale patch Mark the compiler package as deprecated 09/04/09 http://bugs.python.org/issue6837 created exarkun Issues Now Closed (45) ______________________ poss. patch for fnmatch.py to add {.htm,html} style globbing 505 days http://bugs.python.org/issue2649 georg.brandl webbrowser.py doesn't properly handle BROWSER env var 500 days http://bugs.python.org/issue2666 georg.brandl patch os.getcwd fails for long path names on linux 490 days http://bugs.python.org/issue2722 boya patch, easy bytearrays are not thread safe 440 days http://bugs.python.org/issue3139 loewis patch math.log(x, 10) gives different result than math.log10(x) 368 days http://bugs.python.org/issue3724 tjreedy patch library.pdf - Section 12.13.2 Connection Objects - example cut o 316 days http://bugs.python.org/issue4160 ezio.melotti Remove Monterey support from configure.in 222 days http://bugs.python.org/issue5047 georg.brandl patch compiler.parse raises SyntaxErrors without line number informati 221 days http://bugs.python.org/issue5064 georg.brandl test_funcattrs truncated during unittest conversion 218 days http://bugs.python.org/issue5101 georg.brandl help related topic doesn't exist 204 days http://bugs.python.org/issue5221 georg.brandl easy BaseCookie.load doesn't create Morsel objects for mappings 200 days http://bugs.python.org/issue5275 georg.brandl add conversion table to time module docs 13 days http://bugs.python.org/issue5365 georg.brandl tarfile normalizes arcname 102 days http://bugs.python.org/issue6054 lars.gustaebel should we include argparse 81 days http://bugs.python.org/issue6247 r.david.murray Added Misc/python.pc to 'distclean' Rule 75 days http://bugs.python.org/issue6297 marketdickinson patch inserting None into sys.modules does not raise ImportError with 36 days http://bugs.python.org/issue6563 brett.cannon No update about automatic numbering of fields in format strings 38 days http://bugs.python.org/issue6579 eric.smith test_telnetlib doesn't test Telnet.write 39 days http://bugs.python.org/issue6582 rwanderley patch, easy optparse parse_args argument references wrong 28 days http://bugs.python.org/issue6638 georg.brandl PyInit_shoddy() in shoddy.c does not return anything on success 13 days http://bugs.python.org/issue6732 georg.brandl threading issue in __builtins__.print 9 days http://bugs.python.org/issue6750 amaury.forgeotdarc patch, needs review Non-existent member 'nb_inplace_divide' in PyNumberMethods 11 days http://bugs.python.org/issue6754 georg.brandl ftplib documentation does not document what the acct parameter i 14 days http://bugs.python.org/issue6756 georg.brandl Marshal's documentation incomplete (Bools) 13 days http://bugs.python.org/issue6757 georg.brandl math.log, log10 inconsistency 3 days http://bugs.python.org/issue6765 georg.brandl Python 2.6 tutorial still recommends using Exception.message att 11 days http://bugs.python.org/issue6777 georg.brandl invalid print in tkinter\test\test_ttk\test_widgets.py 5 days http://bugs.python.org/issue6796 keithc When Beginning Expression with Lookahead Assertion I get no Matc 1 days http://bugs.python.org/issue6797 marketdickinson symmetric_difference_update documentation fix 1 days http://bugs.python.org/issue6801 georg.brandl Context manager docs refer to contextlib.contextmanager as conte 0 days http://bugs.python.org/issue6803 georg.brandl patch Should be URL for documentation of current release of Python 3 ( 0 days http://bugs.python.org/issue6805 georg.brandl test_platform fails under Snow Leopard 5 days http://bugs.python.org/issue6806 brett.cannon easy python 3.1 documentation tutorial classes 2 days http://bugs.python.org/issue6808 georg.brandl Python string.lstrip bug? 0 days http://bugs.python.org/issue6809 marketdickinson add link to the documentation of signal.signal 1 days http://bugs.python.org/issue6810 georg.brandl update format() documentation and tutorial 0 days http://bugs.python.org/issue6813 eric.smith patch xrange removal from documentation 0 days http://bugs.python.org/issue6814 georg.brandl patch Typo in datamodel.rst ("Custon" -> "Custom") 1 days http://bugs.python.org/issue6819 georg.brandl patch incorrect doc for PyBuffer_Release 0 days http://bugs.python.org/issue6821 georg.brandl Add __contains__ to range type 0 days http://bugs.python.org/issue6826 marketdickinson wrongly highlighted blocks in the Tutorial 0 days http://bugs.python.org/issue6828 georg.brandl patch Some uniformness in defaultdict 0 days http://bugs.python.org/issue6830 georg.brandl Outputting unicode crushes when printing to file on Linux 1 days http://bugs.python.org/issue6832 Orlowski incorrect: failed local variable referenced before assignment 0 days http://bugs.python.org/issue6833 marketdickinson Deleted files are reinstalled 1900 days http://bugs.python.org/issue977470 georg.brandl Top Issues Most Discussed (10) ______________________________ 15 build fails on Snow Leopard 6 days open http://bugs.python.org/issue6802 8 Outputting unicode crushes when printing to file on Linux 1 days closed http://bugs.python.org/issue6832 6 remove/delete method for zipfile/tarfile objects 2 days open http://bugs.python.org/issue6818 6 Support TLS SNI extension in ssl module 156 days open http://bugs.python.org/issue5639 6 improve xrange.__contains__ 764 days open http://bugs.python.org/issue1766304 5 symmetric_difference_update documentation fix 1 days closed http://bugs.python.org/issue6801 4 help for a module should list supported platforms 2 days open http://bugs.python.org/issue6824 4 bytearrays are not thread safe 440 days closed http://bugs.python.org/issue3139 4 Add os.link() and os.symlink() and os.path.islink() support for 1054 days open http://bugs.python.org/issue1578269 3 attribute error after non-from import 1876 days open http://bugs.python.org/issue992389 From chris at simplistix.co.uk Fri Sep 4 18:16:18 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 04 Sep 2009 17:16:18 +0100 Subject: [Python-Dev] httplib slowness solved - which branches to merge to? In-Reply-To: References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> Message-ID: <4AA13D52.9060601@simplistix.co.uk> Antoine Pitrou wrote: > Simon Cross gmail.com> writes: >> Well, since the source for _read_chunked includes the comment >> >> # XXX This accumulates chunks by repeated string concatenation, >> # which is not efficient as the number or size of chunks gets big. >> >> you might gain some speed improvement with minimal effort by gathering >> the read data chunks into a list and then returning "".join(chunks) at >> the end. > > +1 for trying this. Given differences between platforms in realloc() > performance, it might be the reason why it goes unnoticed under Linux but > degenerates under Windows. And how! The following change dropped the download time using httplib to 2.3 seconds: http://svn.python.org/view/python/trunk/Lib/httplib.py?r1=74523&r2=74655 > As a sidenote, it is interesting that even an stdlib module makes this mistake > and acknowledges it without trying to fix it. No longer in this case ;-) The fix is applied on the trunk, but the problem still exists on the 2.6 branch, 3.1 branch and 3.2 branch. Which of these should I merge to? I assume all of them? Do I need to update any changelog files or similar to indicate this bug has been fixed? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From solipsis at pitrou.net Fri Sep 4 18:26:03 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 4 Sep 2009 16:26:03 +0000 (UTC) Subject: [Python-Dev] httplib slowness solved - which branches to merge to? References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> <4AA13D52.9060601@simplistix.co.uk> Message-ID: Chris Withers simplistix.co.uk> writes: > > The fix is applied on the trunk, but the problem still exists on the 2.6 > branch, 3.1 branch and 3.2 branch. > > Which of these should I merge to? I assume all of them? The performance problem is sufficiently serious that it should be considered a bug so, yes, you should merge to all remaining branches (3.2, 2.6 and 3.1). > Do I need to update any changelog files or similar to indicate this bug > has been fixed? Yes, add an entry to Misc/NEWS under the "Library" section using the appropriate conventions. Thanks Antoine. From chris at simplistix.co.uk Fri Sep 4 19:05:36 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 04 Sep 2009 18:05:36 +0100 Subject: [Python-Dev] test failure on py3k branch in test___all__ Message-ID: <4AA148E0.9050806@simplistix.co.uk> Hi All, Anyone know what's causing this failure? test test___all__ failed -- Traceback (most recent call last): File "Lib/test/test___all__.py", line 106, in test_all self.check_all("profile") File "Lib/test/test___all__.py", line 23, in check_all exec("from %s import *" % modname, names) File "", line 1, in AttributeError: 'module' object has no attribute 'help' cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From g.brandl at gmx.net Fri Sep 4 19:14:04 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 04 Sep 2009 19:14:04 +0200 Subject: [Python-Dev] test failure on py3k branch in test___all__ In-Reply-To: <4AA148E0.9050806@simplistix.co.uk> References: <4AA148E0.9050806@simplistix.co.uk> Message-ID: Chris Withers schrieb: > Hi All, > > Anyone know what's causing this failure? > > test test___all__ failed -- Traceback (most recent call last): > File "Lib/test/test___all__.py", line 106, in test_all > self.check_all("profile") > File "Lib/test/test___all__.py", line 23, in check_all > exec("from %s import *" % modname, names) > File "", line 1, in > AttributeError: 'module' object has no attribute 'help' My fault -- fixed in r74661. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From guido at python.org Fri Sep 4 20:22:21 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 4 Sep 2009 11:22:21 -0700 Subject: [Python-Dev] how to debug httplib slowness In-Reply-To: References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> Message-ID: On Fri, Sep 4, 2009 at 4:28 AM, Simon Cross wrote: > On Fri, Sep 4, 2009 at 1:11 PM, Chris Withers wrote: >> Am I right in reading this as most of the time is being spent in httplib's >> HTTPResponse._read_chunked and none of the methods it calls? >> >> If so, is there a better way that a bunch of print statements to find where >> in that method the time is being spent? > > Well, since the source for _read_chunked includes the comment > > ? ? ? ?# XXX This accumulates chunks by repeated string concatenation, > ? ? ? ?# which is not efficient as the number or size of chunks gets big. > > you might gain some speed improvement with minimal effort by gathering > the read data chunks into a list and then returning "".join(chunks) at > the end. +1 on trying this. Constructing a 116MB string by concatenating 1KB buffers surely must take forever. (116MB divided by 85125 recv() calls give 1365 byte per chunk, which is awful.) The HTTP/1.0 business looks like a red herring. Also agreed that this is an embarrassment. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From chris at simplistix.co.uk Fri Sep 4 21:04:49 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 04 Sep 2009 20:04:49 +0100 Subject: [Python-Dev] how to debug httplib slowness In-Reply-To: References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> Message-ID: <4AA164D1.3060702@simplistix.co.uk> Guido van Rossum wrote: > +1 on trying this. Constructing a 116MB string by concatenating 1KB > buffers surely must take forever. (116MB divided by 85125 recv() calls > give 1365 byte per chunk, which is awful.) The HTTP/1.0 business looks > like a red herring. > > Also agreed that this is an embarrassment. Embarrassment eradicated ;-) http://bugs.python.org/issue6838 Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From solipsis at pitrou.net Fri Sep 4 21:15:18 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 4 Sep 2009 19:15:18 +0000 (UTC) Subject: [Python-Dev] how to debug httplib slowness References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> Message-ID: Guido van Rossum python.org> writes: > > +1 on trying this. Constructing a 116MB string by concatenating 1KB > buffers surely must take forever. (116MB divided by 85125 recv() calls > give 1365 byte per chunk, which is awful.) It certainly is quite small but perhaps the server tries to stay below the detected MTU? (not that there is necessarily any point in doing so for most HTTP content, IMO... except perhaps when the client does some progressive decoding) From skippy.hammond at gmail.com Sat Sep 5 02:30:16 2009 From: skippy.hammond at gmail.com (Mark Hammond) Date: Sat, 05 Sep 2009 10:30:16 +1000 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <877hwlwmzv.fsf@hbox.dyndns.org> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> Message-ID: <4AA1B118.3030404@gmail.com> On 30/08/2009 9:37 PM, Martin Geisler wrote: > Mark Hammond writes: > >> 1) I've stalled on the 'none:' patch I promised to resurrect. While >> doing this, I re-discovered that the tests for win32text appear to >> check win32 line endings are used by win32text on *all* platforms, not >> just Windows. > > I think it is only Patrick Mezard who knows how to run (parts of) the > test suite on Windows. > >> I asked for advice from Dirkjan who referred me to the mercurual-devel >> list, but my request of slightly over a week ago remains unanswered >> (http://selenic.com/pipermail/mercurial-devel/2009-August/014873.html) >> - >> maybe I just need to be more patient... > > Oh no, that's usually the wrong tactic :-) I've been too busy for real > Mercurial work the last couple of weeks, but you should not feel bad > about poking us if you don't get a reply. Or come to the IRC channel > (#mercurial on irc.freenode.net) where Dirkjan (djc) and myself (mg) > hang out when it's daytime in Europe. To be fair, I did mail Dirkjan directly who referred me to the -develop list, which I did with a CC to him and a private mail asking for some help should the mail fall on deaf ears as I feared it would. There really is only so far I'm willing to poke and prod people when I'm well aware we are all volunteers. >> Further, Martin's comments in this thread indicate he believes a new >> extension will be necessary rather than 'fixing' win32text. If this >> is the direction we take, it may mean the none: patch, which targets >> the implementation of win32text, is no longer necessary anyway. > > I suggested a new extension for two reasons: ... Thanks, and that does indeed sound excellent. However, this is going a fair way beyond the original scope I signed up for. While I was willing to help implement some new features into an existing extension, taking on the design and implementation of an entire new extension is something I'm not willing to undertake. I don't think such an extension should even come from the Python community or it will end up being a python-only extension - or at best, will need to run the gauntlet of 2 bike-shedding sessions from both the Python and hg communities which will waste much time. What is the hope of an EOL extension which meets our requirements coming directly out of the hg community? If that hope is small, where does that leave us? Cheers, Mark From martin at v.loewis.de Sat Sep 5 09:15:57 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Sep 2009 09:15:57 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA1B118.3030404@gmail.com> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> Message-ID: <4AA2102D.3080600@v.loewis.de> > What is the hope of an EOL extension which meets our requirements coming > directly out of the hg community? If that hope is small, where does > that leave us? As before. I'll repost my request for help, and we stay with subversion meanwhile. Perhaps I'll post it to some mercurial list as well. Regards, Martin From p.f.moore at gmail.com Sat Sep 5 11:24:38 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 5 Sep 2009 10:24:38 +0100 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA2102D.3080600@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> Message-ID: <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> 2009/9/5 "Martin v. L?wis" : >> What is the hope of an EOL extension which meets our requirements coming >> directly out of the hg community? ?If that hope is small, where does >> that leave us? > > As before. I'll repost my request for help, and we stay with subversion > meanwhile. > > Perhaps I'll post it to some mercurial list as well. Can anyone (re-) post the specification of the proposed extension, to the level that it is currently defined? I'm willing to make an attempt to put together an extension, on the assumption that it'll be easier to refine an implementation than continue discussing possibilities... Paul. From solipsis at pitrou.net Sat Sep 5 13:44:36 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 5 Sep 2009 11:44:36 +0000 (UTC) Subject: [Python-Dev] Mercurial migration: help needed References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > > What is the hope of an EOL extension which meets our requirements coming > directly out of the hg community? If that hope is small, where does > that leave us? I'm starting to wonder what the problem really is that makes it so Python-specific. If I understood correctly, it's about a couple of files which must be stored using non-Unix line endings, right? (in the PC and PCbuild directories?) These files are hardly modified often and by many people (and even more rarely by non-Windows people), so why not just put a verification hook on the server and let the offending committer repair his mistake manually, if it ever happens? (we can even provide a script to help repairing the EOL mistake, like Tools/reindent.py does for indentation mistakes) From martin at v.loewis.de Sat Sep 5 15:19:43 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Sep 2009 15:19:43 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> Message-ID: <4AA2656F.4040805@v.loewis.de> > I'm starting to wonder what the problem really is that makes it so > Python-specific. If I understood correctly, it's about a couple of files which > must be stored using non-Unix line endings, right? (in the PC and PCbuild > directories?) No. It's about files that must, when checked out on Windows, have CRLF endings, and, when checked out on Unix, have LF endings - i.e. all the .py, .c, .h, and .rst files, plus a couple of others which don't require specific treatment. IOW, it's about the default behavior, and the majority of new files. > These files are hardly modified often and by many people (and even more rarely > by non-Windows people), so why not just put a verification hook on the server > and let the offending committer repair his mistake manually, if it ever happens? > (we can even provide a script to help repairing the EOL mistake, like > Tools/reindent.py does for indentation mistakes) This was Dirkjan's original proposal, and it is the proposal that brings so much heat into the discussion, claiming that it is a problem of minorities (I do understand that you were unaware that "the problem" is really with the many files, not with the few). In addition, a DVCS brings in another problem dimension: when people push their changes, they have *already* committed them - and perhaps not even they, but a contributor from which they had been pulling changes. The bogus change may have been weeks ago, so the subversion solution (of rejecting the commit to happen) doesn't quite work that well for a DVCS. Regards, Martin From solipsis at pitrou.net Sat Sep 5 15:26:07 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 05 Sep 2009 15:26:07 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA2656F.4040805@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> Message-ID: <1252157167.5692.43.camel@localhost> Le samedi 05 septembre 2009 ? 15:19 +0200, "Martin v. L?wis" a ?crit : > No. It's about files that must, when checked out on Windows, have CRLF > endings, and, when checked out on Unix, have LF endings - i.e. all the > ..py, .c, .h, and .rst files, plus a couple of others which don't require > specific treatment. > > IOW, it's about the default behavior, and the majority of new files. Ok, sorry for the misunderstanding and the lost bandwidth. > In addition, a DVCS brings in another problem dimension: when people > push their changes, they have *already* committed them - and perhaps not > even they, but a contributor from which they had been pulling changes. > The bogus change may have been weeks ago, so the subversion solution > (of rejecting the commit to happen) doesn't quite work that well for > a DVCS. I don't think this problem is really serious. If the push fails, you can just commit (locally) a new changeset that repairs the EOL or indentation problems, and push the whole bunch of changesets again (I assume the server-side hook will not examine changesets individually, but only the last of them?). From martin at v.loewis.de Sat Sep 5 16:18:02 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Sep 2009 16:18:02 +0200 Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help needed) In-Reply-To: <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> Message-ID: <4AA2731A.9060001@v.loewis.de> > Can anyone (re-) post the specification of the proposed extension, to > the level that it is currently defined? For reference, here are the original specification, mine and Martin Geisler's: http://mail.python.org/pipermail/python-dev/2009-August/090984.html http://mail.python.org/pipermail/python-dev/2009-August/091453.html Here is my attempt at summarizing it: - name of versioned configuration file (in root of tree): .hgeol - names of conversion modes: native, LF, CRLF In the configuration file, there is a section [patterns] which maps file name patterns to conversion modes, e.g. [patterns] **.txt = native **.py = native **.dsp = CRLF **.bat = CRLF Tools/bgen/README = native Lib/email/test/data/msg_26.txt = CRLF - Martin Geisler also proposes that there is a section [repository] native = I personally feel YAGNI; it should only support LF (adding such a feature later may be considered) Open issues: - name of extension - what should happen if the file on disk doesn't have the "expected" line endings, or mixed line endings? E.g. a file declared as native "should" have CRLF on Windows - what if it doesn't, on commit? My proposal: do what svn does (whatever that is). That's it, AFAICT. Martin Geisler also discussed something that I read as an implementation strategy, by mapping the patterns to into the (apparently existing) encode/decode configuration setting. HTH, Martin P.S. If you decide that you will or will not work on it, please let us know. From stephen at xemacs.org Sat Sep 5 16:59:26 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 05 Sep 2009 23:59:26 +0900 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <1252157167.5692.43.camel@localhost> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> Message-ID: <87ws4dcu9d.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > > In addition, a DVCS brings in another problem dimension: when people > > push their changes, they have *already* committed them - and perhaps not > > even they, but a contributor from which they had been pulling changes. > > The bogus change may have been weeks ago, so the subversion solution > > (of rejecting the commit to happen) doesn't quite work that well for > > a DVCS. git has a nice filter-branch command, which would allow you to automatically repair the problem (it works basically by checking out each changeset and rerecording it with the appropriate commands). I know bzr is growing something similar, so presumably it is or will soon be available in hg. > I don't think this problem is really serious. > If the push fails, you can just commit (locally) a new changeset that > repairs the EOL or indentation problems, and push the whole bunch of > changesets again (I assume the server-side hook will not examine > changesets individually, but only the last of them?). That's not a very good solution. Especially with typical Mercurial workflows[1], it's quite possible that you'll have a number of bogus changesets interleaved with good one. I don't think recording a repair is satisfactory. Footnotes: [1] Note that DVCS means you do *not* have to follow Python workflows in your private branches. From tjreedy at udel.edu Sat Sep 5 16:26:36 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 05 Sep 2009 10:26:36 -0400 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA2656F.4040805@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> I'm starting to wonder what the problem really is that makes it so >> Python-specific. If I understood correctly, it's about a couple of files which >> must be stored using non-Unix line endings, right? (in the PC and PCbuild >> directories?) > > No. It's about files that must, when checked out on Windows, have CRLF > endings, and, when checked out on Unix, have LF endings - i.e. all the > .py, .c, .h, and .rst files, plus a couple of others which don't require > specific treatment. > > IOW, it's about the default behavior, and the majority of new files. FWIW, I had the same impression as Antoine. I am aware that 'stupid'pad requires /r/n, but do IDLE and other editors (on Windows) that people would actually use to create/edit such files? I would personally be willing to install a notepad replacement if needed to quickview such files. If essentially all text files need fixed line endings on Windows, then hg really needs this built in. Has it really not been used much on Windows? tjr From mg at lazybytes.net Sat Sep 5 16:27:28 2009 From: mg at lazybytes.net (Martin Geisler) Date: Sat, 05 Sep 2009 16:27:28 +0200 Subject: [Python-Dev] Mercurial migration: help needed References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> Message-ID: <87ws4dpiun.fsf@hbox.dyndns.org> Antoine Pitrou writes: > Le samedi 05 septembre 2009 ? 15:19 +0200, "Martin v. L?wis" a ?crit : > >> In addition, a DVCS brings in another problem dimension: when people >> push their changes, they have *already* committed them - and perhaps >> not even they, but a contributor from which they had been pulling >> changes. The bogus change may have been weeks ago, so the subversion >> solution (of rejecting the commit to happen) doesn't quite work that >> well for a DVCS. > > I don't think this problem is really serious. If the push fails, you > can just commit (locally) a new changeset that repairs the EOL or > indentation problems, and push the whole bunch of changesets again (I > assume the server-side hook will not examine changesets individually, > but only the last of them?). Yes, the server-side hook will have to work like this in order for people to fix mistakes like you just described. -- Martin Geisler VIFF (Virtual Ideal Functionality Framework) brings easy and efficient SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available URL: From solipsis at pitrou.net Sat Sep 5 17:09:39 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 5 Sep 2009 15:09:39 +0000 (UTC) Subject: [Python-Dev] Mercurial migration: help needed References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> Message-ID: Terry Reedy udel.edu> writes: > > If essentially all text files need fixed line endings on Windows, then > hg really needs this built in. Has it really not been used much on Windows? Mercurial is used by e.g. Mozilla, which is not really known for poor Windows support (chances are many Firefox developers are Windows-based). I wonder whether they have written their own extension, or if they simply rely on their text editors to do the right thing. From gzlist at googlemail.com Sat Sep 5 17:19:18 2009 From: gzlist at googlemail.com (Martin (gzlist)) Date: Sat, 5 Sep 2009 16:19:18 +0100 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> Message-ID: On 05/09/2009, Antoine Pitrou wrote: > Terry Reedy udel.edu> writes: >> >> If essentially all text files need fixed line endings on Windows, then >> hg really needs this built in. Has it really not been used much on >> Windows? > > Mercurial is used by e.g. Mozilla, which is not really known for poor > Windows > support (chances are many Firefox developers are Windows-based). I wonder > whether they have written their own extension, or if they simply rely on > their > text editors to do the right thing. Actually, most Firefox developers are mac based. Mozilla isn't a great example of windows integration, they install half-a-unix-system in order to just build under windows, including msys, python 2.5, mercurial, and xemacs. See: Martin From dirkjan at ochtman.nl Sat Sep 5 17:26:12 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 05 Sep 2009 17:26:12 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> Message-ID: <4AA28314.6020205@ochtman.nl> On 05/09/2009 17:09, Antoine Pitrou wrote: > Mercurial is used by e.g. Mozilla, which is not really known for poor Windows > support (chances are many Firefox developers are Windows-based). I wonder > whether they have written their own extension, or if they simply rely on their > text editors to do the right thing. I'm pretty sure they don't have a specific extension for it. I don't know if many of their developers use the win32text extension, but I would guess not (I have been somewhat involved in Mozilla's migration). Cheers, Dirkjan From dirkjan at ochtman.nl Sat Sep 5 17:27:24 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 05 Sep 2009 17:27:24 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <87ws4dcu9d.fsf@uwakimon.sk.tsukuba.ac.jp> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> <87ws4dcu9d.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4AA2835C.6000002@ochtman.nl> On 05/09/2009 16:59, Stephen J. Turnbull wrote: > git has a nice filter-branch command, which would allow you to > automatically repair the problem (it works basically by checking out > each changeset and rerecording it with the appropriate commands). I > know bzr is growing something similar, so presumably it is or will > soon be available in hg. That means you change hashes on the server side, without human feedback. Let's try not to subvert the immutability design that Mercurial tries to encourage. Cheers, Dirkjan From martin at v.loewis.de Sat Sep 5 18:18:33 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 05 Sep 2009 18:18:33 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <1252157167.5692.43.camel@localhost> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> Message-ID: <4AA28F59.5030805@v.loewis.de> > I don't think this problem is really serious. > If the push fails, you can just commit (locally) a new changeset that > repairs the EOL or indentation problems I would find that unfortunate. It's a fairly irrelevant change, yet it may manage to corrupt the history (hg blame). > and push the whole bunch of > changesets again (I assume the server-side hook will not examine > changesets individually, but only the last of them?). That is for us to decide. I can see arguments either way. But it shouldn't happen often that the server refuses a push; all errors should already be caught on the clients. Regards, Martin From martin at v.loewis.de Sat Sep 5 18:20:16 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Sep 2009 18:20:16 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <87ws4dpiun.fsf@hbox.dyndns.org> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> <87ws4dpiun.fsf@hbox.dyndns.org> Message-ID: <4AA28FC0.9010505@v.loewis.de> >> I don't think this problem is really serious. If the push fails, you >> can just commit (locally) a new changeset that repairs the EOL or >> indentation problems, and push the whole bunch of changesets again (I >> assume the server-side hook will not examine changesets individually, >> but only the last of them?). > > Yes, the server-side hook will have to work like this in order for > people to fix mistakes like you just described. Not necessarily. People could also be required to go back and replay all changes. Regards, Martin From dirkjan at ochtman.nl Sat Sep 5 18:25:36 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 5 Sep 2009 18:25:36 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA28F59.5030805@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> <4AA28F59.5030805@v.loewis.de> Message-ID: On Sat, Sep 5, 2009 at 18:18, "Martin v. L?wis" wrote: > But it shouldn't happen often that the server refuses a push; > all errors should already be caught on the clients. We could just mandate the same hook code as a commit hook. Cheers, Dirkjan From martin at v.loewis.de Sat Sep 5 18:25:46 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Sep 2009 18:25:46 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> Message-ID: <4AA2910A.4070504@v.loewis.de> > FWIW, I had the same impression as Antoine. I am aware that 'stupid'pad > requires /r/n, but do IDLE and other editors (on Windows) that people > would actually use to create/edit such files? I would personally be > willing to install a notepad replacement if needed to quickview such files. Visual Studio will create files with CRLF endings. Please don't talk people out of using Visual Studio for development. More generally: please accept that it is consensus that this *has* to be fixed, and that it *is* a problem on Windows. > If essentially all text files need fixed line endings on Windows, then > hg really needs this built in. Has it really not been used much on Windows? I think that's the case. It's pretty much a Unix-only tool, like most of the other DVCS implementations. FWIW, I tried to check out Mozilla (which is in hg), and the check out would always abort with a timeout. Then I downloaded a bundle that they had produced, and try to unbundle it. It took all night, but was complete the other morning. Trying to update the checkout would again make me run into http timeouts. I tried the same on Linux, and it completed within a few minutes. So I conclude that, from a certain project size on, hg is unusable on Windows, atleast on my office machine, running Windows 7. Regards, Martin From martin at v.loewis.de Sat Sep 5 18:28:00 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 05 Sep 2009 18:28:00 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> <4AA28F59.5030805@v.loewis.de> Message-ID: <4AA29190.90500@v.loewis.de> >> But it shouldn't happen often that the server refuses a push; >> all errors should already be caught on the clients. > > We could just mandate the same hook code as a commit hook. I would be in favor (although, IIUC, "mandate" here would be a social thing, not a technical one). Regards, Martin From dirkjan at ochtman.nl Sat Sep 5 18:38:55 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 05 Sep 2009 18:38:55 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA29190.90500@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> <4AA28F59.5030805@v.loewis.de> <4AA29190.90500@v.loewis.de> Message-ID: <4AA2941F.3020000@ochtman.nl> On 05/09/2009 18:28, "Martin v. L?wis" wrote: > I would be in favor (although, IIUC, "mandate" here would be a > social thing, not a technical one). Right, but that would be the same for the extension. Cheers, Dirkjan From g.brandl at gmx.net Sat Sep 5 18:56:12 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 05 Sep 2009 18:56:12 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA28F59.5030805@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> <4AA28F59.5030805@v.loewis.de> Message-ID: Martin v. L?wis schrieb: >> I don't think this problem is really serious. >> If the push fails, you can just commit (locally) a new changeset that >> repairs the EOL or indentation problems > > I would find that unfortunate. It's a fairly irrelevant change, yet > it may manage to corrupt the history (hg blame). I'm for a per-changeset check as well. In the common case, the client will have the "required" extension, and errors will be caught there. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From dirkjan at ochtman.nl Sat Sep 5 19:02:39 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Sat, 5 Sep 2009 19:02:39 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA2910A.4070504@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> Message-ID: On Sat, Sep 5, 2009 at 18:25, "Martin v. L?wis" wrote: > I think that's the case. It's pretty much a Unix-only tool, like most of > the other DVCS implementations. I know a lot of projects use Mercurial on Windows as well, I'm not aware of any big problems with it. > FWIW, I tried to check out Mozilla (which is in hg), and the check out > would always abort with a timeout. Then I downloaded a bundle that they > had produced, and try to unbundle it. It took all night, but was > complete the other morning. Trying to update the checkout would again > make me run into http timeouts. I tried the same on Linux, and it > completed within a few minutes. So I conclude that, from a certain > project size on, hg is unusable on Windows, atleast on my office > machine, running Windows 7. That sounds pretty bad. By check out, do you mean the clone (getting data over the wire) or the actual check out (setting up a working directory)? I think I've heard of problems with the clone part before, for them. We're actually working on improving clone size, though it also seems to have to do with network reliability. Cheers, Dirkjan From p.f.moore at gmail.com Sat Sep 5 19:26:23 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 5 Sep 2009 18:26:23 +0100 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA2910A.4070504@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> Message-ID: <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com> 2009/9/5 "Martin v. L?wis" : >> FWIW, I had the same impression as Antoine. I am aware that 'stupid'pad >> requires /r/n, but do IDLE and other editors (on Windows) that people >> would actually use to create/edit such files? I would personally be >> willing to install a notepad replacement if needed to quickview such files. > > Visual Studio will create files with CRLF endings. Please don't talk > people out of using Visual Studio for development. More generally: > please accept that it is consensus that this *has* to be fixed, and > that it *is* a problem on Windows. (Disclaimer: I have no problem with accepting that the extension is needed - the following is for clarity, in particular to help me understand how the hook will be used) There are 2 separate questions - (1) what is held in the repository, and (2) what is in the user's workspace. The two clearly interact. Taking (2) first, there are *some* files (very few, I believe) that require specific line endings (CRLF - Visual Studio build files, is my understanding). There are also tools that require fixed line endings for input (notepad, Visual Studio). Finally, tools create new files with certain line endings by default (pretty much guaranteed to be platform-native, I'd expect). The result is that user workspaces *may* (quite probably, will) contain files with a mixture of line endings if care is not taken. As regards (1), I assume that for "text" files, a consistent EOL convention (assumed LF) should be used in the repository. It's not clear to me what should be held in the repo for the files requiring specific line endings - my instinct is that they should be treated as "binary" and stored in the repo with the mandated line endings, and checked out unchanged. So we have the following situation: - Some "binary" files which should never be converted - Some "text" files, which must be held in LF form in the repo My view is that how I store text files in my workspace is entirely up to me (and the capabilities of my tools). So, how files get checked out should not be centrally mandated. (Hmm, that may complicate matters). How files are checked in is crucial, so a setting is required which ensures that each file so marked is converted to LF format on checking - effectively working like universal newline mode. So, the issues: 1. Given that the "problematic" tools (notepad and Visual Studio) are Windows tools, we seem to be back to the idea that this extension is only needed by Windows developers. As I understood the consensus to be that the extension should be for all users, I suspect I've missed something. 2. Allowing text files to be checked out in whatever form the user prefers seems complicated. The alternative would likely be to say test files are checked out in "native" form. That works, but would irritate me as I work on Windows, but prefer strongly to use LF line endings (yes, I realise that makes me an oddball...) I'd put up with it if it was the consensus to do this, of course. 3. Is there a realistic possibility that a user could edit one of the CRLF-requiring files with a tool that converts it to LF? If so, is there a need to trap that programmatically (as opposed to simply accepting that this equates to the individual accidentally breaking the build, no worse or better than checking in a C file with a syntax error)? Is this a fair summary? Paul From martin at v.loewis.de Sat Sep 5 19:37:57 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 05 Sep 2009 19:37:57 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> Message-ID: <4AA2A1F5.7090406@v.loewis.de> > I know a lot of projects use Mercurial on Windows as well, I'm not > aware of any big problems with it. I trust that indeed, there are no big problems for most users. I also trust that the hg developers are, in general, open to incorporating improvements on Windows. I'm still skeptical though whether Mercurial is really usable on Windows. Your statement is a somewhat self-fulfilling prophecy: people who run into big problems early likely aren't going to use it. So lack of reports doesn't really mean there aren't any problems. >> FWIW, I tried to check out Mozilla (which is in hg), and the check out >> would always abort with a timeout. Then I downloaded a bundle that they >> had produced, and try to unbundle it. It took all night, but was >> complete the other morning. Trying to update the checkout would again >> make me run into http timeouts. I tried the same on Linux, and it >> completed within a few minutes. So I conclude that, from a certain >> project size on, hg is unusable on Windows, atleast on my office >> machine, running Windows 7. > > That sounds pretty bad. By check out, do you mean the clone (getting > data over the wire) or the actual check out (setting up a working > directory)? Creating the clone. ISTM that it leaves the http connection open while doing stuff locally (or creates multiple of them, and one times out). It starts cloning, and then, after an hour or so, it reports ABORT, and rolls back, for no apparent reason. > I think I've heard of problems with the clone part before, > for them. We're actually working on improving clone size, though it > also seems to have to do with network reliability. Our institute has generally a really good internet connection; I think hg.mozilla.org does as well. Plus, it worked when doing it on the very same machine on Linux. Regards, Martin From martin at v.loewis.de Sat Sep 5 19:56:01 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Sep 2009 19:56:01 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com> Message-ID: <4AA2A631.3010508@v.loewis.de> > There are 2 separate questions - (1) what is held in the repository, > and (2) what is in the user's workspace. The two clearly interact. [...] > As regards (1), I assume that for "text" files, a consistent EOL > convention (assumed LF) should be used in the repository. Correct. I believe Martin (the other one) proposed to make it configurable. I agree that using LF in the repository is sensible. Wrt. "should": there is the debate whether intermediate revisions can deviate from that requirement. > It's not > clear to me what should be held in the repo for the files requiring > specific line endings - my instinct is that they should be treated as > "binary" and stored in the repo with the mandated line endings, and > checked out unchanged. I would say so, yes. One consequence of that is that if you change your mind in hgeols, you need to commit all files that now fail to conform. This is what happens with svn, and it may be tricky to implement as you need to commit files that didn't change on disk (say you switch from native to CRLF on a Windows checkout). OTOH, even if you do store all text files in LF in the repo, then you would still have the problem for files that go from unspecified eol-style to a specified one. So changing hgeols is tricky, period. > - Some "binary" files which should never be converted > - Some "text" files, which must be held in LF form in the repo > > My view is that how I store text files in my workspace is entirely up > to me (and the capabilities of my tools). So, how files get checked > out should not be centrally mandated. Not by technical means, no. In the developer FAQ, there will be clear directions, and you better follow them - or need to accept the blame if you made a mistake because you ignored them. > So, the issues: > > 1. Given that the "problematic" tools (notepad and Visual Studio) are > Windows tools, we seem to be back to the idea that this extension is > only needed by Windows developers. As I understood the consensus to be > that the extension should be for all users, I suspect I've missed > something. Technically, yes, it is only needed on Windows. The desire to have all users use them come from the wish that problems with the setup will be detected earlier. E.g. if the extension stops working with a new Mercurial version, and all users use them, there is a larger motivation to fix them for all users. Things that the extension can do for you on Unix: - check that the syntax of .hgeols is correct; this may affect Unix users which try to edit it - check that all text files have consistent line endings, and refuse checkin if they don't. This may become relevant if a Unix text editor tries to edit a CRLF file, and doesn't quite detect that. > 2. Allowing text files to be checked out in whatever form the user > prefers seems complicated. The alternative would likely be to say test > files are checked out in "native" form. That works, but would irritate > me as I work on Windows, but prefer strongly to use LF line endings > (yes, I realise that makes me an oddball...) I'd put up with it if it > was the consensus to do this, of course. It's consensus, and it's also what subversion does, and CVS did. > 3. Is there a realistic possibility that a user could edit one of the > CRLF-requiring files with a tool that converts it to LF? If so, is > there a need to trap that programmatically (as opposed to simply > accepting that this equates to the individual accidentally breaking > the build, no worse or better than checking in a C file with a syntax > error)? I think everything you can imagine is also realistic. For the less-realistic cases, it may be better if the commit is refused rather than silently fixing it, since the user operated the system in a surprising way - so he may actually have meant to do it that way. One specific case is recent autoconf, which put a CR character into configure, completely breaking svn's eol handling. Regards, Martin From martin at v.loewis.de Sat Sep 5 20:04:29 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 05 Sep 2009 20:04:29 +0200 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA2A631.3010508@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com> <4AA2A631.3010508@v.loewis.de> Message-ID: <4AA2A82D.2090100@v.loewis.de> >> 2. Allowing text files to be checked out in whatever form the user >> prefers seems complicated. The alternative would likely be to say test >> files are checked out in "native" form. That works, but would irritate >> me as I work on Windows, but prefer strongly to use LF line endings >> (yes, I realise that makes me an oddball...) I'd put up with it if it >> was the consensus to do this, of course. > > It's consensus, and it's also what subversion does, and CVS did. Following up to myself: you might want to make it a feature that "native" is configurable, per user or per repo. It should default to CRLF on Windows, but you might want to set it to LF on your system. In that case, the extension would have just its checking functionality. Regards, Martin From gzlist at googlemail.com Sat Sep 5 20:06:46 2009 From: gzlist at googlemail.com (Martin (gzlist)) Date: Sat, 5 Sep 2009 19:06:46 +0100 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA2A1F5.7090406@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> <4AA2A1F5.7090406@v.loewis.de> Message-ID: On 05/09/2009, "Martin v. L?wis" wrote: > > Creating the clone. ISTM that it leaves the http connection open while > doing stuff locally (or creates multiple of them, and one times out). > > It starts cloning, and then, after an hour or so, it reports ABORT, > and rolls back, for no apparent reason. I have been tracking mozilla-central, and believe this is a problem with the repo, that started some time after Aug 04 - which is the last log entry I have in my clone. I presumed it was just a problem my end so hadn't got round trying to debug it yet. If it is a general problem, the fact it's been around for about a month without being addressed might indicate how well tested most DVCSes are under windows. Martin From mg at lazybytes.net Sat Sep 5 21:34:05 2009 From: mg at lazybytes.net (Martin Geisler) Date: Sat, 05 Sep 2009 21:34:05 +0200 Subject: [Python-Dev] Mercurial migration: help needed References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> <87ws4dpiun.fsf@hbox.dyndns.org> <4AA28FC0.9010505@v.loewis.de> Message-ID: <87pra5p4nm.fsf@hbox.dyndns.org> "Martin v. L?wis" writes: >>> I don't think this problem is really serious. If the push fails, you >>> can just commit (locally) a new changeset that repairs the EOL or >>> indentation problems, and push the whole bunch of changesets again >>> (I assume the server-side hook will not examine changesets >>> individually, but only the last of them?). >> >> Yes, the server-side hook will have to work like this in order for >> people to fix mistakes like you just described. > > Not necessarily. People could also be required to go back and replay > all changes. Replaying changes, i.e., editing history is quite easy as long as you have done no merges. So people can indeed fix their mistakes by cleaning up history as long as they have a linear history. Both mq and histedit are available for this: http://mercurial.selenic.com/wiki/MqExtension http://mercurial.selenic.com/wiki/HisteditExtension The problem comes if a small group have been working together on a new feature and have merged changes in from the mainline while doing so. They will then no longer be able to edit past the most recent merge. -- Martin Geisler VIFF (Virtual Ideal Functionality Framework) brings easy and efficient SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available URL: From brett at python.org Sat Sep 5 22:20:14 2009 From: brett at python.org (Brett Cannon) Date: Sat, 5 Sep 2009 13:20:14 -0700 Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help needed) In-Reply-To: <4AA2731A.9060001@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> <4AA2731A.9060001@v.loewis.de> Message-ID: On Sat, Sep 5, 2009 at 07:18, "Martin v. L?wis" wrote: >> Can anyone (re-) post the specification of the proposed extension, to >> the level that it is currently defined? > > For reference, here are the original specification, mine and Martin > Geisler's: > > http://mail.python.org/pipermail/python-dev/2009-August/090984.html > http://mail.python.org/pipermail/python-dev/2009-August/091453.html > > Here is my attempt at summarizing it: > > - name of versioned configuration file (in root of tree): .hgeol > - names of conversion modes: native, LF, CRLF > In the configuration file, there is a section [patterns] which > maps file name patterns to conversion modes, e.g. > > [patterns] > **.txt = native > **.py = native > **.dsp = CRLF > **.bat = CRLF > Tools/bgen/README = native > Lib/email/test/data/msg_26.txt = CRLF > > - Martin Geisler also proposes that there is a section > [repository] > native = > I personally feel YAGNI; it should only support LF (adding such > a feature later may be considered) Do you mean what native is in the repo or what it should be considered on the user's machine? If it's the former then I actually like it as it means a clone doesn't need to do anything special when 'native' matches what is expected in the repo while a commit still does its EOL validation. I still think we need to have a server-side block which rejects commits that messes up the line-endings so people can fix them. Shouldn't mess up 'blame' as the messed up line-endings should only be from their edits. Otherwise it's just like when Tim used to run reindent.py over the entire repo on occasion. And as mentioned in another email by Paul, it would be nice to let the user specify what they want 'native' to be on their machine if they happen to be a Windows user who prefers LF. > > Open issues: > - name of extension StupidLineEndings =) > - what should happen if the file on disk doesn't have the "expected" > line endings, or mixed line endings? E.g. a file declared as native > "should" have CRLF on Windows - what if it doesn't, on commit? > My proposal: do what svn does (whatever that is). Or refuse the commit with a message and tell the user to fix it (if svn doesn't happen to do that). -Brett From wiggin15 at gmail.com Sat Sep 5 23:26:14 2009 From: wiggin15 at gmail.com (Ender Wiggin) Date: Sun, 6 Sep 2009 00:26:14 +0300 Subject: [Python-Dev] Problems with hex-conversion functions Message-ID: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com> Hello everyone. I see several problems with the two hex-conversion function pairs that Python offers: 1. binascii.hexlify and binascii.unhexlify 2. bytes.fromhex and bytes.hex Problem #1: bytes.hex is not implemented, although it was specified in PEP 358. This means there is no symmetrical function to accompany bytes.fromhex. Problem #2: Both pairs perform the same function, although The Zen Of Python suggests that "There should be one-- and preferably only one --obvious way to do it." I do not understand why PEP 358 specified the bytes function pair although it mentioned the binascii pair... Problem #3: bytes.fromhex may receive spaces in the input string, although binascii.unhexlify may not. I see no good reason for these two functions to have different features. Problem #4: binascii.unhexlify may receive both input types: strings or bytes, whereas bytes.fromhex raises an exception when given a bytes parameter. Again there is no reason for these functions to be different. Problem #5: binascii.hexlify returns a bytes type - although ideally, converting to hex should always return string types and converting from hex should always return bytes. IMO there is no meaning of bytes as an output of hexlify, since the output is a representation of other bytes. This is also the suggested behavior of bytes.hex in PEP 358 Problems #4 and #5 call for a decision about the input and output of the functions being discussed: Option A : Strict input and output unhexlify (and bytes.fromhex) may only receives string and may only return bytes hexlify (and bytes.hex) may only receives bytes and may only return strings Option B : Robust input and strict output unhexlify (and bytes.fromhex) may receive bytes and strings and may only return bytes hexlify (and bytes.hex) may receive bytes or strings and may only return strings Of course we may also consider a third option, which will allow the return type of all functions to be robust (perhaps specified in a keyword argument), but as I wrote in the description of problem #5, I see no sense in that. Note that PEP 3137 describes: "... the more strict definitions of encoding and decoding in Python 3000: encoding always takes a Unicode string and returns a bytes sequence, and decoding always takes a bytes sequence and returns a Unicode string." - suggesting option A. To repeat problems #4 and #5, the current behavior does not match any option: * The return type of binascii.hexlify should be string, and this is not the current behavior. As for the input: * Option A is not the current behavior because binascii.unhexlify may receive both input types. * Option B is not the current behavior because bytes.fromhex does not allow bytes as input. To fix these issues, three changes should be applied: 1. Deprecate bytes.fromhex. This fixes the following problems: #4 (go with option B and remove the function that does not allow bytes input) #2 (the binascii functions will be the only way to "do it") #1 (bytes.hex should not be implemented) 2. In order to keep the functionality that bytes.fromhex has over unhexlify, the latter function should be able to handle spaces in its input (fix #3) 3. binascii.hexlify should return string as its return type (fix #5) -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sun Sep 6 00:06:11 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 06 Sep 2009 00:06:11 +0200 Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help needed) In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> <4AA2731A.9060001@v.loewis.de> Message-ID: <4AA2E0D3.3050206@v.loewis.de> >> - Martin Geisler also proposes that there is a section >> [repository] >> native = >> I personally feel YAGNI; it should only support LF (adding such >> a feature later may be considered) > > Do you mean what native is in the repo or what it should be considered > on the user's machine? The former. > If it's the former then I actually like it as > it means a clone doesn't need to do anything special when 'native' > matches what is expected in the repo while a commit still does its EOL > validation. But the same would be true if the repo format would be always LF: when "native" matches (which would then be on Unix), the extension would *still* have to do nothing but validation. > I still think we need to have a server-side block which > rejects commits that messes up the line-endings so people can fix > them. Certainly. > Shouldn't mess up 'blame' as the messed up line-endings should > only be from their edits. It could be that they had a number of commits that eventually lead to the version that they push; this will also push the intermediate versions. So when you then do a blame, it will tell you that the revision was logged as "fix whitespace", rather than "resolve issue #9743". You are mostly right that the committer name would be the same (except when the committer was pushing some changes pulled from the actual contributor), however, I still see these whitespace-only changes as a complication. Regards, Martin From brett at python.org Sun Sep 6 00:25:05 2009 From: brett at python.org (Brett Cannon) Date: Sat, 5 Sep 2009 15:25:05 -0700 Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help needed) In-Reply-To: <4AA2E0D3.3050206@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> <4AA2731A.9060001@v.loewis.de> <4AA2E0D3.3050206@v.loewis.de> Message-ID: On Sat, Sep 5, 2009 at 15:06, "Martin v. L?wis" wrote: >>> - Martin Geisler also proposes that there is a section >>> [repository] >>> native = >>> I personally feel YAGNI; it should only support LF (adding such >>> a feature later may be considered) >> >> Do you mean what native is in the repo or what it should be considered >> on the user's machine? > > The former. > >> If it's the former then I actually like it as >> it means a clone doesn't need to do anything special when 'native' >> matches what is expected in the repo while a commit still does its EOL >> validation. > > But the same would be true if the repo format would be always LF: > when "native" matches (which would then be on Unix), the extension > would *still* have to do nothing but validation. Right, but I am just thinking about how we specify in .hgeols what the repository is expected to be as this extension might work out nicely for other projects who prefer CLRF as their repo-native line ending. > >> I still think we need to have a server-side block which >> rejects commits that messes up the line-endings so people can fix >> them. > > Certainly. > >> Shouldn't mess up 'blame' as the messed up line-endings should >> only be from their edits. > > It could be that they had a number of commits that eventually lead > to the version that they push; this will also push the intermediate > versions. So when you then do a blame, it will tell you that the > revision was logged as "fix whitespace", rather than "resolve issue > #9743". > Yep. > You are mostly right that the committer name would be the same > (except when the committer was pushing some changes pulled from > the actual contributor), however, I still see these whitespace-only > changes as a complication. It's unfortunate, but I see it as a rare occurrence as it would only happen if someone got sloppy. And it should typically get caught client-side before the commit ever occurs, minimizing the whitespace-only commits even more. -Brett From martin at v.loewis.de Sun Sep 6 00:35:26 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 06 Sep 2009 00:35:26 +0200 Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help needed) In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> <4AA2731A.9060001@v.loewis.de> <4AA2E0D3.3050206@v.loewis.de> Message-ID: <4AA2E7AE.6020308@v.loewis.de> > Right, but I am just thinking about how we specify in .hgeols what the > repository is expected to be as this extension might work out nicely > for other projects who prefer CLRF as their repo-native line ending. This is what I refer to as YAGNI. Subversion has LF as the internal storage, and, IIRC, so does CVS. I don't think there is any precedence for wanting something else - and frankly, I can't see how repository storage would matter. >> You are mostly right that the committer name would be the same >> (except when the committer was pushing some changes pulled from >> the actual contributor), however, I still see these whitespace-only >> changes as a complication. > > It's unfortunate, but I see it as a rare occurrence as it would only > happen if someone got sloppy. And it should typically get caught > client-side before the commit ever occurs, minimizing the > whitespace-only commits even more. It would, of course, be possible to ban them altogether, at the expense of users having to replay changes. Regards, Martin From brett at python.org Sun Sep 6 00:39:50 2009 From: brett at python.org (Brett Cannon) Date: Sat, 5 Sep 2009 15:39:50 -0700 Subject: [Python-Dev] Problems with hex-conversion functions In-Reply-To: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com> References: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com> Message-ID: On Sat, Sep 5, 2009 at 14:26, Ender Wiggin wrote: > Hello everyone. > > I see several problems with the two hex-conversion function pairs that > Python offers: > 1. binascii.hexlify and binascii.unhexlify > 2. bytes.fromhex and bytes.hex > > Problem #1: > bytes.hex is not implemented, although it was specified in PEP 358. Probably an oversight. > This means there is no symmetrical function to accompany bytes.fromhex. > > Problem #2: > Both pairs perform the same function, although The Zen Of Python suggests > that > "There should be one-- and preferably only one --obvious way to do it." > I do not understand why PEP 358 specified the bytes function pair although > it mentioned the binascii pair... > It's nicer to have this kind of functionality on the built-ins than in the standard library. "Practicality beats purity". > Problem #3: > bytes.fromhex may receive spaces in the input string, although > binascii.unhexlify may not. > I see no good reason for these two functions to have different features. > Well, one allows for sloppy input while the other does not. Usually accepting sloppy input but giving strict input is better. > Problem #4: > binascii.unhexlify may receive both input types: strings or bytes, whereas > bytes.fromhex raises an exception when given a bytes parameter. > Again there is no reason for these functions to be different. Well, giving bytes back into bytes seems somewhat silly. That's an error in mixing your strings and bytes. > > Problem #5: > binascii.hexlify returns a bytes type - although ideally, converting to hex > should > always return string types and converting from hex should always return > bytes. > IMO there is no meaning of bytes as an output of hexlify, since the output > is a > representation of other bytes. > This is also the suggested behavior of bytes.hex in PEP 358 > > Problems #4 and #5 call for a decision about the input and output of the > functions being discussed: > > Option A : Strict input and output > unhexlify (and bytes.fromhex) may only receives string and may only return > bytes > hexlify (and bytes.hex) may only receives bytes and may only return strings > > Option B : Robust input and strict output > unhexlify (and bytes.fromhex) may receive bytes and strings and may only > return bytes > hexlify (and bytes.hex) may receive bytes or strings and may only return > strings > > Of course we may also consider a third option, which will allow the return > type of > all functions to be robust (perhaps specified in a keyword argument), but as > I wrote in > the description of problem #5, I see no sense in that. > > Note that PEP 3137 describes: "... the more strict definitions of encoding > and decoding in > Python 3000: encoding always takes a Unicode string and returns a bytes > sequence, and decoding > always takes a bytes sequence and returns a Unicode string." - suggesting > option A. > > To repeat problems #4 and #5, the current behavior does not match any > option: > * The return type of binascii.hexlify should be string, and this is not the > current behavior. > As for the input: > * Option A is not the current behavior because binascii.unhexlify may > receive both input types. > * Option B is not the current behavior because bytes.fromhex does not allow > bytes as input. > > To fix these issues, three changes should be applied: > 1. Deprecate bytes.fromhex. This fixes the following problems: > ?? #4 (go with option B and remove the function that does not allow bytes > input) > ?? #2 (the binascii functions will be the only way to "do it") > ?? #1 (bytes.hex should not be implemented) > 2. In order to keep the functionality that bytes.fromhex has over unhexlify, > ?? the latter function should be able to handle spaces in its input (fix #3) > 3. binascii.hexlify should return string as its return type (fix #5) Or we fix bytes.fromhex(), add bytes.hex() and deprecate binascii.(un)hexlify(). -Brett From nyamatongwe at gmail.com Sun Sep 6 00:56:55 2009 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Sun, 6 Sep 2009 08:56:55 +1000 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> Message-ID: <50862ebd0909051556nf68ca7cm6e5d1bc83f40826a@mail.gmail.com> Dirkjan Ochtman: > I know a lot of projects use Mercurial on Windows as well, I'm not > aware of any big problems with it. If you have a Windows-only project with CRLF files using Mercurial then there is no line end problem as Mercurial preserves the CRLFs for you. Line end problems occur on mixed projects where both Windows and Unix tools are used. Neil From nyamatongwe at gmail.com Sun Sep 6 01:01:49 2009 From: nyamatongwe at gmail.com (Neil Hodgson) Date: Sun, 6 Sep 2009 09:01:49 +1000 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com> Message-ID: <50862ebd0909051601p3a52cb4cv83b2cb121ce6424c@mail.gmail.com> Paul Moore: > 1. Given that the "problematic" tools (notepad and Visual Studio) are > Windows tools, we seem to be back to the idea that this extension is > only needed by Windows developers. As I understood the consensus to be > that the extension should be for all users, I suspect I've missed > something. Some of the problems come from users on Unix checking in files with CRLF line ends that they have received using some other mechanism such as sharing a disk between Windows and Linux. I was going to point to a bad revision in a bzr housed project I work on but launchpad isn't working currently. What happened was that an OS X user committed a set of changes but with all the files having a different line ending to the repository. The result is that it is no longer easy to track changes before that revision. It also makes a check out larger. It would help in such cases for the commit command on Unix to either automatically change any CRLF line ends to LF for text files (but not files with an explicitly specified line end) or to display a warning. Neil From mg at lazybytes.net Sun Sep 6 01:57:13 2009 From: mg at lazybytes.net (Martin Geisler) Date: Sun, 06 Sep 2009 01:57:13 +0200 Subject: [Python-Dev] hgeol extension References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> <4AA2731A.9060001@v.loewis.de> Message-ID: <87y6otndwm.fsf@hbox.dyndns.org> "Martin v. L?wis" writes: >> Can anyone (re-) post the specification of the proposed extension, to >> the level that it is currently defined? > > For reference, here are the original specification, mine and Martin > Geisler's: > > http://mail.python.org/pipermail/python-dev/2009-August/090984.html > http://mail.python.org/pipermail/python-dev/2009-August/091453.html > > Here is my attempt at summarizing it: > > - name of versioned configuration file (in root of tree): .hgeol > - names of conversion modes: native, LF, CRLF > In the configuration file, there is a section [patterns] which > maps file name patterns to conversion modes, e.g. > > [patterns] > **.txt = native > **.py = native > **.dsp = CRLF > **.bat = CRLF > Tools/bgen/README = native > Lib/email/test/data/msg_26.txt = CRLF > > - Martin Geisler also proposes that there is a section > [repository] > native = > I personally feel YAGNI; it should only support LF (adding such > a feature later may be considered) I don't think it's a good idea to store everything in LF in the repository. Unlike Subversion, you cannot expect all interactions to take place through the "eol-filter" we're implementing. Letting people checkout a useful unfiltered clone would be possible if we know the repository native format and convert back to that. Anyway, it's a minor detail. More importantly, I've posted a simple, rough extension that does this here: http://markmail.org/message/yj4so736t4cfdulv I figured it would be better to discuss the design and implementation on mercurial-devel since there are more Mercurial hackers there. I've CC'ed a bunch of people from this thread to "seed" the discussion -- the rest of you on python-devel are hereby invited to join :-) http://selenic.com/mailman/listinfo/mercurial-devel -- Martin Geisler VIFF (Virtual Ideal Functionality Framework) brings easy and efficient SMPC (Secure Multiparty Computation) to Python. See: http://viff.dk/. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 196 bytes Desc: not available URL: From ncoghlan at gmail.com Sun Sep 6 05:51:05 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 06 Sep 2009 13:51:05 +1000 Subject: [Python-Dev] Problems with hex-conversion functions In-Reply-To: References: <8449046c0909051426g2eaa3f94s4c5842c25af17abf@mail.gmail.com> Message-ID: <4AA331A9.70209@gmail.com> Brett Cannon wrote: >> To fix these issues, three changes should be applied: >> 1. Deprecate bytes.fromhex. This fixes the following problems: >> #4 (go with option B and remove the function that does not allow bytes >> input) >> #2 (the binascii functions will be the only way to "do it") >> #1 (bytes.hex should not be implemented) >> 2. In order to keep the functionality that bytes.fromhex has over unhexlify, >> the latter function should be able to handle spaces in its input (fix #3) >> 3. binascii.hexlify should return string as its return type (fix #5) > > Or we fix bytes.fromhex(), add bytes.hex() and deprecate binascii.(un)hexlify(). binascii is the legacy approach here, so if anything was to go, those functions would be it. I'm not sure getting rid of them is worth the hassle though (especially in 2.x). Regarding bytes.hex(), it may be better to modify the builtin hex() function to accept bytes as an input type. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From stephen at xemacs.org Sun Sep 6 09:41:24 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 06 Sep 2009 16:41:24 +0900 Subject: [Python-Dev] hgeol extension (Was: Mercurial migration: help needed) In-Reply-To: <4AA2E7AE.6020308@v.loewis.de> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2102D.3080600@v.loewis.de> <79990c6b0909050224x43c85f5ahf9205c5929ff8947@mail.gmail.com> <4AA2731A.9060001@v.loewis.de> <4AA2E0D3.3050206@v.loewis.de> <4AA2E7AE.6020308@v.loewis.de> Message-ID: <873a70y0yj.fsf@uwakimon.sk.tsukuba.ac.jp> "Martin v. L?wis" writes: > This is what I refer to as YAGNI. Subversion has LF as the internal > storage, and, IIRC, so does CVS. I don't think there is any precedence > for wanting something else - and frankly, I can't see how repository > storage would matter. Well, internally you could use U+2028 LINE SEPARATOR, which would screw up *everybody* if they don't use the converter, since there are probably very few editors that understand U+2028. I've heard that this is what Samba did when converting to Unicode: intead of using UTF-8 they used UTF-16 so that English would be at least as buggy as any other language. Maybe there's somebody who was participating in Samba at that time who knows? From stephen at xemacs.org Sun Sep 6 09:56:08 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 06 Sep 2009 16:56:08 +0900 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <4AA2835C.6000002@ochtman.nl> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <1252157167.5692.43.camel@localhost> <87ws4dcu9d.fsf@uwakimon.sk.tsukuba.ac.jp> <4AA2835C.6000002@ochtman.nl> Message-ID: <871vmky09z.fsf@uwakimon.sk.tsukuba.ac.jp> Dirkjan Ochtman writes: > On 05/09/2009 16:59, Stephen J. Turnbull wrote: > > git has a nice filter-branch command, which would allow you to > > automatically repair the problem (it works basically by checking out > > each changeset and rerecording it with the appropriate commands). I > > know bzr is growing something similar, so presumably it is or will > > soon be available in hg. > > That means you change hashes on the server side, without human feedback. > Let's try not to subvert the immutability design that Mercurial tries to > encourage. No, I mean the server refuses to accept it and the submitter can fix it easily (with mq or histedit as Martin G points out), then resubmit. In any case Mercurial's notion of immutability is unsustainable in practice, as the plethora of extensions which mutate history testifies. From stephen at xemacs.org Sun Sep 6 10:26:05 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 06 Sep 2009 17:26:05 +0900 Subject: [Python-Dev] Mercurial migration: help needed In-Reply-To: <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com> References: <4A8A6256.1030009@v.loewis.de> <4A99E77A.5080603@gmail.com> <877hwlwmzv.fsf@hbox.dyndns.org> <4AA1B118.3030404@gmail.com> <4AA2656F.4040805@v.loewis.de> <4AA2910A.4070504@v.loewis.de> <79990c6b0909051026x1b84361fy2eebcc9f553cec4e@mail.gmail.com> Message-ID: <87zl98wkbm.fsf@uwakimon.sk.tsukuba.ac.jp> Paul Moore writes: > The result is that user workspaces *may* (quite probably, will) > contain files with a mixture of line endings if care is not taken. Yes. Under your "fixed-EOL-files-are-binary" scheme, this is guaranteed for Unix systems. > As regards (1), I assume that for "text" files, a consistent EOL > convention (assumed LF) should be used in the repository. It's not > clear to me what should be held in the repo for the files requiring > specific line endings - my instinct is that they should be treated > as "binary" and stored in the repo with the mandated line endings, > and checked out unchanged. Why? Files that require specific line endings are in general used in platform-specific ways. So checking them out with the platform's normal line ending should work fine. > My view is that how I store text files in my workspace is entirely up > to me (and the capabilities of my tools). So, how files get checked > out should not be centrally mandated. Your tools will be able to work with the native EOL convention, or you wouldn't be able to stand using them. In general the extension should default to checking out with the native convention. If you really want to change that, you can; there's nothing the server can do to mandate what's in your workspaces. The "mandate" here is simply the default extension that Python and/or Mercurial will distribute to help developers avoid having their pushes aborted for incorrect EOLs. > So, the issues: > > 1. Given that the "problematic" tools (notepad and Visual Studio) are > Windows tools, we seem to be back to the idea that this extension is > only needed by Windows developers. As I understood the consensus to be > that the extension should be for all users, I suspect I've missed > something. What you've missed is that developers *of* the Windows port are not necessarily developers *on* Windows. If we treat these as text files and check them out in the native convention everywhere, then it doesn't matter if you edit them on Unix or Windows, when you check them out and build on Windows it Just Works[tm]. I have never heard of a Unix cross-IDE port of Visual Studio.... > 2. Allowing text files to be checked out in whatever form the user > prefers seems complicated. It's not a question of "allow". AIUI, you won't be allowed to push a commit with broken line endings to the public repo. This is too much of a burden to impose given the wayward behavior of existing tools, so an extension will be distributed that does the checking (and any needed conversion) for you. If you don't like that extension, you can change it; it shouldn't be too difficult. Eg: > That works, but would irritate me as I work on Windows, but prefer > strongly to use LF line endings (yes, I realise that makes me an > oddball...) I'd put up with it if it was the consensus to do this, > of course. You don't need to. In that case I would guess that you are at very low risk if you disable the checkout side of the extension. > 3. Is there a realistic possibility that a user could edit one of the > CRLF-requiring files with a tool that converts it to LF? Yes. It happened occasionally in XEmacs's CVS repository, and caused great consternation among Windows testers. > If so, is there a need to trap that programmatically (as opposed to > simply accepting that this equates to the individual accidentally > breaking the build, no worse or better than checking in a C file > with a syntax error)? It's worse. Until the problem is fixed, the people who need a different EOL convention are working with a file with breakage on every line. Furthermore, that breakage may be quite silent, eg, if you use XEmacs to edit on Windows but Visual Studio to build. With Emacs it's easy enough to change -- if you recognize the breakage. From frantzcj at gmail.com Mon Sep 7 18:09:36 2009 From: frantzcj at gmail.com (Chris Frantz) Date: Mon, 7 Sep 2009 11:09:36 -0500 Subject: [Python-Dev] Controlling the cipher list for SSL connections Message-ID: Greetings, I would like to be able to set the cipher list when creating an SSL connection.? It appears that the current SSL module doesn't provide this functionality. The attached patch (against trunk) adds this ability to SSLSocket. Thank you, --Chris PS: Please reply directly to me, as I'm not subscribed to this list. Index: Python-2.7/Lib/ssl.py =================================================================== --- Python-2.7/Lib/ssl.py??? (revision 74703) +++ Python-2.7/Lib/ssl.py??? (working copy) @@ -88,7 +88,7 @@ ????????????????? server_side=False, cert_reqs=CERT_NONE, ????????????????? ssl_version=PROTOCOL_SSLv23, ca_certs=None, ????????????????? do_handshake_on_connect=True, -???????????????? suppress_ragged_eofs=True): +???????????????? suppress_ragged_eofs=True, cipher_list=None): ???????? socket.__init__(self, _sock=sock._sock) ???????? # the initializer for socket trashes the methods (tsk, tsk), so... ???????? self.send = lambda data, flags=0: SSLSocket.send(self, data, flags) @@ -110,7 +110,8 @@ ???????????? # yes, create the SSL object ???????????? self._sslobj = _ssl.sslwrap(self._sock, server_side, ???????????????????????????????????????? keyfile, certfile, -??????????????????????????????????????? cert_reqs, ssl_version, ca_certs) +??????????????????????????????????????? cert_reqs, ssl_version, +??????????????????????????????????????? ca_certs, cipher_list) ???????????? if do_handshake_on_connect: ???????????????? timeout = self.gettimeout() ???????????????? try: Index: Python-2.7/Modules/_ssl.c =================================================================== --- Python-2.7/Modules/_ssl.c??? (revision 74703) +++ Python-2.7/Modules/_ssl.c??? (working copy) @@ -261,7 +261,8 @@ ???? ?????? enum py_ssl_server_or_client socket_type, ???? ?????? enum py_ssl_cert_requirements certreq, ???? ?????? enum py_ssl_version proto_version, -??? ?????? char *cacerts_file) +??? ?????? char *cacerts_file, +??? ?????? char *cipher_list) ?{ ???? PySSLObject *self; ???? char *errstr = NULL; @@ -366,6 +367,9 @@ ???? SSL_CTX_set_verify(self->ctx, verification_mode, ???? ??? ??? ?? NULL); /* set verify lvl */ +??? if (cipher_list) +??? ??? SSL_CTX_set_cipher_list(self->ctx, cipher_list); + ???? PySSL_BEGIN_ALLOW_THREADS ???? self->ssl = SSL_new(self->ctx); /* New ssl struct */ ???? PySSL_END_ALLOW_THREADS @@ -407,14 +411,17 @@ ???? char *key_file = NULL; ???? char *cert_file = NULL; ???? char *cacerts_file = NULL; +??? char *cipher_list = NULL; -??? if (!PyArg_ParseTuple(args, "O!i|zziiz:sslwrap", + +??? if (!PyArg_ParseTuple(args, "O!i|zziizz:sslwrap", ???? ??? ??? ????? PySocketModule.Sock_Type, ???? ??? ??? ????? &Sock, ???? ??? ??? ????? &server_side, ???? ??? ??? ????? &key_file, &cert_file, ???? ??? ??? ????? &verification_mode, &protocol, -??? ??? ??? ????? &cacerts_file)) +??? ??? ??? ????? &cacerts_file, +??? ??? ??? ????? &cipher_list)) ???? ??? return NULL; ???? /* @@ -427,12 +434,12 @@ ???? return (PyObject *) newPySSLObject(Sock, key_file, cert_file, ???? ??? ??? ??? ??? ?? server_side, verification_mode, -??? ??? ??? ??? ??? ?? protocol, cacerts_file); +??? ??? ??? ??? ??? ?? protocol, cacerts_file, cipher_list); ?} ?PyDoc_STRVAR(ssl_doc, ?"sslwrap(socket, server_side, [keyfile, certfile, certs_mode, protocol,\n" -"????????????????????????????? cacertsfile]) -> sslobject"); +"????????????????????????????? cacertsfile, cipherlist]) -> sslobject"); ?/* SSL object methods */ From fuzzyman at voidspace.org.uk Mon Sep 7 18:15:21 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 07 Sep 2009 17:15:21 +0100 Subject: [Python-Dev] Controlling the cipher list for SSL connections In-Reply-To: References: Message-ID: <4AA53199.5090409@voidspace.org.uk> Hello Chris, Can you post your patch to the Python bug tracker please - http://bugs.python.org Patches posted to this list tend to get lost... Thanks Michael Chris Frantz wrote: > Greetings, > > I would like to be able to set the cipher list when creating an SSL > connection. It appears that the current SSL module doesn't provide > this functionality. > > The attached patch (against trunk) adds this ability to SSLSocket. > > Thank you, > --Chris > > PS: Please reply directly to me, as I'm not subscribed to this list. > > Index: Python-2.7/Lib/ssl.py > =================================================================== > --- Python-2.7/Lib/ssl.py (revision 74703) > +++ Python-2.7/Lib/ssl.py (working copy) > @@ -88,7 +88,7 @@ > server_side=False, cert_reqs=CERT_NONE, > ssl_version=PROTOCOL_SSLv23, ca_certs=None, > do_handshake_on_connect=True, > - suppress_ragged_eofs=True): > + suppress_ragged_eofs=True, cipher_list=None): > socket.__init__(self, _sock=sock._sock) > # the initializer for socket trashes the methods (tsk, tsk), so... > self.send = lambda data, flags=0: SSLSocket.send(self, data, flags) > @@ -110,7 +110,8 @@ > # yes, create the SSL object > self._sslobj = _ssl.sslwrap(self._sock, server_side, > keyfile, certfile, > - cert_reqs, ssl_version, ca_certs) > + cert_reqs, ssl_version, > + ca_certs, cipher_list) > if do_handshake_on_connect: > timeout = self.gettimeout() > try: > Index: Python-2.7/Modules/_ssl.c > =================================================================== > --- Python-2.7/Modules/_ssl.c (revision 74703) > +++ Python-2.7/Modules/_ssl.c (working copy) > @@ -261,7 +261,8 @@ > enum py_ssl_server_or_client socket_type, > enum py_ssl_cert_requirements certreq, > enum py_ssl_version proto_version, > - char *cacerts_file) > + char *cacerts_file, > + char *cipher_list) > { > PySSLObject *self; > char *errstr = NULL; > @@ -366,6 +367,9 @@ > SSL_CTX_set_verify(self->ctx, verification_mode, > NULL); /* set verify lvl */ > > + if (cipher_list) > + SSL_CTX_set_cipher_list(self->ctx, cipher_list); > + > PySSL_BEGIN_ALLOW_THREADS > self->ssl = SSL_new(self->ctx); /* New ssl struct */ > PySSL_END_ALLOW_THREADS > @@ -407,14 +411,17 @@ > char *key_file = NULL; > char *cert_file = NULL; > char *cacerts_file = NULL; > + char *cipher_list = NULL; > > - if (!PyArg_ParseTuple(args, "O!i|zziiz:sslwrap", > + > + if (!PyArg_ParseTuple(args, "O!i|zziizz:sslwrap", > PySocketModule.Sock_Type, > &Sock, > &server_side, > &key_file, &cert_file, > &verification_mode, &protocol, > - &cacerts_file)) > + &cacerts_file, > + &cipher_list)) > return NULL; > > /* > @@ -427,12 +434,12 @@ > > return (PyObject *) newPySSLObject(Sock, key_file, cert_file, > server_side, verification_mode, > - protocol, cacerts_file); > + protocol, cacerts_file, cipher_list); > } > > PyDoc_STRVAR(ssl_doc, > "sslwrap(socket, server_side, [keyfile, certfile, certs_mode, protocol,\n" > -" cacertsfile]) -> sslobject"); > +" cacertsfile, cipherlist]) -> sslobject"); > > /* SSL object methods */ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From frantzcj at gmail.com Mon Sep 7 18:32:31 2009 From: frantzcj at gmail.com (Chris Frantz) Date: Mon, 7 Sep 2009 11:32:31 -0500 Subject: [Python-Dev] Controlling the cipher list for SSL connections In-Reply-To: <4AA53199.5090409@voidspace.org.uk> References: <4AA53199.5090409@voidspace.org.uk> Message-ID: Done. Attached to Issue 3597, which is a similar request to mine. Best Regards, --Chris From eric.pruitt at gmail.com Mon Sep 7 20:13:35 2009 From: eric.pruitt at gmail.com (Eric Pruitt) Date: Mon, 7 Sep 2009 14:13:35 -0400 Subject: [Python-Dev] PEP 3145 Message-ID: <171e8a410909071113r7d06399t8029f4b795e5b0aa@mail.gmail.com> Hello all, I have been working on adding asynchronous I/O to the Python subprocess module as part of my Google Summer of Code project. Now that I have finished documenting and pruning the code, I present PEP 3145 for its inclusion into the Python core code. Any and all feedback on the PEP (http://www.python.org/dev/peps/pep-3145/) is appreciated. Eric From eric at trueblade.com Mon Sep 7 23:46:42 2009 From: eric at trueblade.com (Eric Smith) Date: Mon, 07 Sep 2009 17:46:42 -0400 Subject: [Python-Dev] Numeric alignment issue with PEP 3101 Message-ID: <4AA57F42.3050201@trueblade.com> The default string formatting alignment for all types, according to PEP 3101, is left aligned. Issue 6857 (http://bugs.python.org/issue6857) points out that for numeric types (int, float, and decimal, at least), the actual implemented default alignment is right aligned. Mark Dickinson and I agree that for numeric types, right alignment makes much more sense as a default. And that's what %-formatting and str.format() both do. I think the PEP should be modified to say that right alignment is the default for numeric types. Also the documentation at http://docs.python.org/library/string.html#formatstrings should have the same modification. Does anyone disagree? Eric. From greg.ewing at canterbury.ac.nz Tue Sep 8 00:05:33 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 08 Sep 2009 10:05:33 +1200 Subject: [Python-Dev] Numeric alignment issue with PEP 3101 In-Reply-To: <4AA57F42.3050201@trueblade.com> References: <4AA57F42.3050201@trueblade.com> Message-ID: <4AA583AD.1000104@canterbury.ac.nz> Eric Smith wrote: > Mark Dickinson and I agree that for numeric types, right alignment makes > much more sense as a default. And that's what %-formatting and > str.format() both do. Is the new formatting supposed to behave the same way as %-formatting for the same format codes? Because the default for %-formatting is actually right alignment for *all* types, including strings. -- Greg From eric at trueblade.com Tue Sep 8 00:10:04 2009 From: eric at trueblade.com (Eric Smith) Date: Mon, 07 Sep 2009 18:10:04 -0400 Subject: [Python-Dev] Numeric alignment issue with PEP 3101 In-Reply-To: <4AA583AD.1000104@canterbury.ac.nz> References: <4AA57F42.3050201@trueblade.com> <4AA583AD.1000104@canterbury.ac.nz> Message-ID: <4AA584BC.2020001@trueblade.com> Greg Ewing wrote: > Is the new formatting supposed to behave the same way > as %-formatting for the same format codes? Because the > default for %-formatting is actually right alignment > for *all* types, including strings. Hmm, I never noticed that. At this point, I think changing the formatting for any types would break code, so we should just change the documentation to reflect how currently works. Eric. From python at rcn.com Tue Sep 8 02:16:18 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 7 Sep 2009 17:16:18 -0700 Subject: [Python-Dev] Numeric alignment issue with PEP 3101 In-Reply-To: <4AA57F42.3050201@trueblade.com> References: <4AA57F42.3050201@trueblade.com> Message-ID: <6930F234-6E8E-4E78-AE7F-FFE97AF72D5C@rcn.com> I concur. Numbers are naturally right aligned. On Sep 7, 2009, at 2:46 PM, Eric Smith wrote: > The default string formatting alignment for all types, according to > PEP 3101, is left aligned. Issue 6857 (http://bugs.python.org/issue6857 > ) points out that for numeric types (int, float, and decimal, at > least), the actual implemented default alignment is right aligned. > > Mark Dickinson and I agree that for numeric types, right alignment > makes much more sense as a default. And that's what %-formatting and > str.format() both do. > > I think the PEP should be modified to say that right alignment is > the default for numeric types. Also the documentation at http://docs.python.org/library/string.html#formatstrings > should have the same modification. > > Does anyone disagree? > > Eric. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com From dickinsm at gmail.com Tue Sep 8 09:11:21 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 8 Sep 2009 08:11:21 +0100 Subject: [Python-Dev] Numeric alignment issue with PEP 3101 In-Reply-To: <4AA584BC.2020001@trueblade.com> References: <4AA57F42.3050201@trueblade.com> <4AA583AD.1000104@canterbury.ac.nz> <4AA584BC.2020001@trueblade.com> Message-ID: <5c6f2a5d0909080011gd558970r321832de90497c2b@mail.gmail.com> On Mon, Sep 7, 2009 at 11:10 PM, Eric Smith wrote: > Hmm, I never noticed that. At this point, I think changing the formatting > for any types would break code, so we should just change the documentation > to reflect how currently works. I think the alignment for Decimal *does* need to be changed, though. It currently left-aligns by default (my fault: I just blindly followed PEP 3101 without thinking too hard about it). I'd like to fix this for 3.2 and 2.7; I'm not sure whether it's too disruptive to fix it in 3.1 and 2.6. Mark From eric at trueblade.com Tue Sep 8 13:56:53 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 08 Sep 2009 07:56:53 -0400 Subject: [Python-Dev] Numeric alignment issue with PEP 3101 In-Reply-To: <5c6f2a5d0909080011gd558970r321832de90497c2b@mail.gmail.com> References: <4AA57F42.3050201@trueblade.com> <4AA583AD.1000104@canterbury.ac.nz> <4AA584BC.2020001@trueblade.com> <5c6f2a5d0909080011gd558970r321832de90497c2b@mail.gmail.com> Message-ID: <4AA64685.1010803@trueblade.com> Mark Dickinson wrote: > On Mon, Sep 7, 2009 at 11:10 PM, Eric Smith wrote: >> Hmm, I never noticed that. At this point, I think changing the formatting >> for any types would break code, so we should just change the documentation >> to reflect how currently works. > > I think the alignment for Decimal *does* need to be changed, though. It > currently left-aligns by default (my fault: I just blindly followed PEP 3101 > without thinking too hard about it). I'd like to fix this for 3.2 and 2.7; I'm > not sure whether it's too disruptive to fix it in 3.1 and 2.6. Right, not changing "any types" was too strong. I think that the numeric types should all default to right alignment. That means Decimal should change. I agree this should only be in 3.2 and 2.7. In addition, PEP 3101 and the documentation should be change to reflect the default right alignment for numeric types. Shame on you, Mark, for actually reading and implementing the specs! ;) You see that I didn't bother with that. At the time, I think I was trying to get str.format() to exactly agree with %-formatting. From naveen.garg at gmail.com Tue Sep 8 07:42:17 2009 From: naveen.garg at gmail.com (tinku99) Date: Mon, 7 Sep 2009 22:42:17 -0700 (PDT) Subject: [Python-Dev] compiling python2.5 on linux under wine In-Reply-To: References: <5b8d13220901080511x2fe96845l8ad86e70908f0a3e@mail.gmail.com> <49666AF7.9010804@v.loewis.de> Message-ID: <25340242.post@talk.nabble.com> aren't you better off just cross compiling under mingw? You don't even really need to do that... you can just dynamically load the brebuilt windows python libraries in your wine apps... wine is for running windows apps on linux, not for cross compiling... wine itself uses the mingw cross compiler when necessary from my understanding. winelib is for creating elf binaries, not windows binaries... -- View this message in context: http://www.nabble.com/compiling-python2.5-on-linux-under-wine-tp21270106p25340242.html Sent from the Python - python-dev mailing list archive at Nabble.com. From chris at simplistix.co.uk Tue Sep 8 14:18:56 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 08 Sep 2009 13:18:56 +0100 Subject: [Python-Dev] [Distutils] Static metadata using setup.cfg In-Reply-To: <13d2dc789ffc897481edc43e88432a31@preisshare.net> References: <94bdd2610908170034q544bca48g7619589bd3764@mail.gmail.com> <4A8A606D.6090306@taupro.com> <0b8f78d03f5dfdeba987ef233b2ae8eb@preisshare.net> <4A8A8FE9.6070905@trueblade.com> <94bdd2610908180932te51f49byc9f3938c29a0c100@mail.gmail.com> <94bdd2610908250248o46349e76uea66245f5ebd1173@mail.gmail.com> <4A9BBA2B.6010903@simplistix.co.uk> <94bdd2610908310637r17af992al469498e90544c5e0@mail.gmail.com> <4AA4F94C.9080702@simplistix.co.uk> <23CAC1AA-9399-405A-A7E6-64F4C052BEC1@mac.com> <4AA6136A.7080904@simplistix.co.uk> <13d2dc789ffc897481edc43e88432a31@preisshare.net> Message-ID: <4AA64BB0.7090309@simplistix.co.uk> David Lyon wrote: > On Tue, 08 Sep 2009 09:18:50 +0100, Chris Withers > wrote: >> >> If Python had a packaging system *and* used it for the standard library, >> then things like this wouldn't be a problem... >> The setup.cfg could just say "requires sqlite greater than version >> x.y.z", and if it was in the standard library, it would be used unless a >> newer version was needed. > > +1 > > Actually, this was already discussed on this mailing list. Yeah, I know, but I had memories of it be poo-poo'ed on Python-Dev... (CC'ing so they can tell me I'm wrong ;-) ) > I suggested that a "requires" section could easily do this, something > along the lines of: > > [Requires] > stdlib=sqlite>=1.5 Tarek, How are requirements spelled for packages in your current setup.cfg? I really don't see why anything should be different for standard library packages (ie: the stdlib= prefix in David's example). Python distributions should just declare all the versions they come with in the same way that whatever-is-being-built by Tarek can introspect in the same way as any other package... > So the concept of having an if/else test for this is superfluous. Right. >> It would also mean it would be possible to >> release bug fix versions of the standard library packages without having >> to roll a whole python release. > > +1 ...which in turn would mean that the standard library is no longer a place where packages go to die... >> Better yet, since "python" should be a package as far as the packaging >> system is concerned, library versions can just say what versions of >> python they work with. > > +1 - good idea ...and for me, the "python" package should be just another package in the distributions dance, called "python" ;-) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From ziade.tarek at gmail.com Tue Sep 8 14:29:12 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 8 Sep 2009 14:29:12 +0200 Subject: [Python-Dev] [Distutils] Static metadata using setup.cfg In-Reply-To: <4AA64BB0.7090309@simplistix.co.uk> References: <94bdd2610908170034q544bca48g7619589bd3764@mail.gmail.com> <94bdd2610908180932te51f49byc9f3938c29a0c100@mail.gmail.com> <94bdd2610908250248o46349e76uea66245f5ebd1173@mail.gmail.com> <4A9BBA2B.6010903@simplistix.co.uk> <94bdd2610908310637r17af992al469498e90544c5e0@mail.gmail.com> <4AA4F94C.9080702@simplistix.co.uk> <23CAC1AA-9399-405A-A7E6-64F4C052BEC1@mac.com> <4AA6136A.7080904@simplistix.co.uk> <13d2dc789ffc897481edc43e88432a31@preisshare.net> <4AA64BB0.7090309@simplistix.co.uk> Message-ID: <94bdd2610909080529w4b3fc34cy43be7643a8e06391@mail.gmail.com> 2009/9/8 Chris Withers : >> I suggested that a "requires" section could easily do this, something >> along the lines of: >> >> [Requires] >> stdlib=sqlite>=1.5 > > Tarek, How are requirements spelled for packages in your current setup.cfg? Sorry, that's another problem we are dealing with here e.g. How *one* requirement is defined. The .cfg file is just a format that holds values, like you would express them in command line or in arguments. The "How are requirements spelled" part is the changes we will add in PEP 345. The templating part is just here to make these requirements (or anything else) vary, depending on a few environment values, so it can be read without installing the distribution, with a vanilla Python. Please stop cross-posting in other mailing-lists it makes the discussions confusing. there are two discussions, to be talked in distutils-SIG: 1/ static metadata using a setup.cfg.in (the current one) 2/ changes that will occur in PEP 345, to include requirement definitions Regards Tarek -- Tarek Ziad? | http://ziade.org |??????????? From ncoghlan at gmail.com Tue Sep 8 15:00:30 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 08 Sep 2009 23:00:30 +1000 Subject: [Python-Dev] [Distutils] Static metadata using setup.cfg In-Reply-To: <4AA64BB0.7090309@simplistix.co.uk> References: <94bdd2610908170034q544bca48g7619589bd3764@mail.gmail.com> <4A8A606D.6090306@taupro.com> <0b8f78d03f5dfdeba987ef233b2ae8eb@preisshare.net> <4A8A8FE9.6070905@trueblade.com> <94bdd2610908180932te51f49byc9f3938c29a0c100@mail.gmail.com> <94bdd2610908250248o46349e76uea66245f5ebd1173@mail.gmail.com> <4A9BBA2B.6010903@simplistix.co.uk> <94bdd2610908310637r17af992al469498e90544c5e0@mail.gmail.com> <4AA4F94C.9080702@simplistix.co.uk> <23CAC1AA-9399-405A-A7E6-64F4C052BEC1@mac.com> <4AA6136A.7080904@simplistix.co.uk> <13d2dc789ffc897481edc43e88432a31@preisshare.net> <4AA64BB0.7090309@simplistix.co.uk> Message-ID: <4AA6556E.8070902@gmail.com> Chris Withers wrote: > David Lyon wrote: >> On Tue, 08 Sep 2009 09:18:50 +0100, Chris Withers >> >> wrote: >>> >>> If Python had a packaging system *and* used it for the standard >>> library, then things like this wouldn't be a problem... >>> The setup.cfg could just say "requires sqlite greater than version >>> x.y.z", and if it was in the standard library, it would be used >>> unless a newer version was needed. >> >> +1 >> >> Actually, this was already discussed on this mailing list. > > Yeah, I know, but I had memories of it be poo-poo'ed on Python-Dev... > (CC'ing so they can tell me I'm wrong ;-) ) My recollection of the python-dev discussions was that the outcome was "there isn't even a distutils-sig consensus yet, so it's a little early for python-dev to be sticking its oar in". Also, I don't think providing new metadata *describing* the standard library was particularly controversial - it was more radical ideas of actually breaking up the core distro into separate components that met resistance. Third party installers having the power to easily "replace" standard library modules for more than a single application was also a heavily debated feature (since that way lies unintended coupling and application/library installations that break previously installed applications). It's probably also worth mentioning that on many of these topics there is a recurring communications gap that usually becomes painful for the distutils-sig side as core devs re-ask questions that were hashed out on the distutils list long before. So I would suggest being prepared to summarise and provide links to such discussions when this all becomes settled enough to bring back to the whole of python-dev. The back and forth of an archived mailing list discussion can sometimes be more persuasive than the relatively sanitised summaries that end up in a PEP :) That said, Tarek's response suggests that this still isn't ready for a wider python-dev discussion, so I've set followup-to accordingly. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From eric at trueblade.com Tue Sep 8 22:41:56 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 8 Sep 2009 16:41:56 -0400 (EDT) Subject: [Python-Dev] Numeric alignment issue with PEP 3101 In-Reply-To: <4AA64685.1010803@trueblade.com> References: <4AA57F42.3050201@trueblade.com> <4AA583AD.1000104@canterbury.ac.nz> <4AA584BC.2020001@trueblade.com> <5c6f2a5d0909080011gd558970r321832de90497c2b@mail.gmail.com> <4AA64685.1010803@trueblade.com> Message-ID: <50109.63.251.87.214.1252442516.squirrel@mail.trueblade.com> > Mark Dickinson wrote: >> I think the alignment for Decimal *does* need to be changed, though. It >> currently left-aligns by default (my fault: I just blindly followed PEP >> 3101 >> without thinking too hard about it). I'd like to fix this for 3.2 and >> 2.7; I'm >> not sure whether it's too disruptive to fix it in 3.1 and 2.6. I notice that complex has the same problem as the just-fixed Decimal. I'll make those changes. From peter at hda3.com Wed Sep 9 01:36:41 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 8 Sep 2009 16:36:41 -0700 Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library In-Reply-To: <8517e9350908270652w4846dfdal8278ce71e3d238b4@mail.gmail.com> References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> <8517e9350908270652w4846dfdal8278ce71e3d238b4@mail.gmail.com> Message-ID: <8517e9350909081636n29ee5082x51fce2721b1e2791@mail.gmail.com> Howdy, So I've made most of the suggested changes here and put up a release of ipaddr. Do folks here consider this pep is final (enough) for submission or is there more work to be done? Cheers, /peter On Thu, Aug 27, 2009 at 6:52 AM, Peter Moody wrote: > Howdy folks, > > the reference code has been updated per your comments; specifically, > there's no more IP/IPv4/IPv6 factory functions, it's all IPAddress() > and IPNetwork constructors. > > I've submitted a patch to the PEP with updated examples and a lengthy > description of the class inheritance and the benefits from that design > choice. hopefully that should go live soon. > > If there are any more suggestions on the PEP or the code, please let me know. > > Cheers, > /peter > > On Tue, Aug 18, 2009 at 1:00 PM, Peter Moody wrote: >> Howdy folks, >> >> I have a first draft of a PEP for including an IP address manipulation >> library in the python stdlib. It seems like there are a lot of really >> smart folks with some, ahem, strong ideas about what an IP address >> module should and shouldn't be so I wanted to solicit your input on >> this pep. >> >> the pep can be found here: >> >> ?http://www.python.org/dev/peps/pep-3144/ >> >> the code can be found here: >> >> ?http://ipaddr-py.googlecode.com/svn/branches/2.0.x/ >> >> Please let me know if you have any comments (some already coming :) >> >> Cheers, >> /peter >> > From benjamin at python.org Wed Sep 9 04:56:35 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 8 Sep 2009 21:56:35 -0500 Subject: [Python-Dev] PEP 3145 In-Reply-To: <171e8a410909071113r7d06399t8029f4b795e5b0aa@mail.gmail.com> References: <171e8a410909071113r7d06399t8029f4b795e5b0aa@mail.gmail.com> Message-ID: <1afaf6160909081956x262ed97dlc2ba1df99e7bc262@mail.gmail.com> 2009/9/7 Eric Pruitt : > Hello all, > > I have been working on adding asynchronous I/O to the Python > subprocess module as part of my Google Summer of Code project. Now > that I have finished documenting and pruning the code, I present PEP > 3145 for its inclusion into the Python core code. Any and all feedback > on the PEP (http://www.python.org/dev/peps/pep-3145/) is appreciated. Hi Eric, One of the reasons you're not getting many response is that you've not pasted the contents of the PEP in this message. That makes it really easy for people to comment on various sections. BTW, it seems like you were trying to use reST formatting with the text PEP layout. Double backquotes only mean something in reST. -- Regards, Benjamin From eric.pruitt at gmail.com Wed Sep 9 05:52:45 2009 From: eric.pruitt at gmail.com (Eric Pruitt) Date: Tue, 8 Sep 2009 23:52:45 -0400 Subject: [Python-Dev] PEP 3145 (With Contents) Message-ID: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com> Alright, I will re-submit with the contents pasted. I never use double backquotes as I think them rather ugly; that is the work of an editor or some automated program in the chain. Plus, it also messed up my line formatting and now I have lines with one word on them... Anyway, the contents of PEP 3145: PEP: 3145 Title: Asynchronous I/O For subprocess.Popen Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson Type: Standards Track Content-Type: text/plain Created: 04-Aug-2009 Python-Version: 3.2 Abstract: In its present form, the subprocess.Popen implementation is prone to dead-locking and blocking of the parent Python script while waiting on data from the child process. Motivation: A search for "python asynchronous subprocess" will turn up numerous accounts of people wanting to execute a child process and communicate with it from time to time reading only the data that is available instead of blocking to wait for the program to produce data [1] [2] [3]. The current behavior of the subprocess module is that when a user sends or receives data via the stdin, stderr and stdout file objects, dead locks are common and documented [4] [5]. While communicate can be used to alleviate some of the buffering issues, it will still cause the parent process to block while attempting to read data when none is available to be read from the child process. Rationale: There is a documented need for asynchronous, non-blocking functionality in subprocess.Popen [6] [7] [2] [3]. Inclusion of the code would improve the utility of the Python standard library that can be used on Unix based and Windows builds of Python. Practically every I/O object in Python has a file-like wrapper of some sort. Sockets already act as such and for strings there is StringIO. Popen can be made to act like a file by simply using the methods attached the the subprocess.Popen.stderr, stdout and stdin file-like objects. But when using the read and write methods of those options, you do not have the benefit of asynchronous I/O. In the proposed solution the wrapper wraps the asynchronous methods to mimic a file object. Reference Implementation: I have been maintaining a Google Code repository that contains all of my changes including tests and documentation [9] as well as blog detailing the problems I have come across in the development process [10]. I have been working on implementing non-blocking asynchronous I/O in the subprocess.Popen module as well as a wrapper class for subprocess.Popen that makes it so that an executed process can take the place of a file by duplicating all of the methods and attributes that file objects have. There are two base functions that have been added to the subprocess.Popen class: Popen.send and Popen._recv, each with two separate implementations, one for Windows and one for Unix based systems. The Windows implementation uses ctypes to access the functions needed to control pipes in the kernel 32 DLL in an asynchronous manner. On Unix based systems, the Python interface for file control serves the same purpose. The different implementations of Popen.send and Popen._recv have identical arguments to make code that uses these functions work across multiple platforms. When calling the Popen._recv function, it requires the pipe name be passed as an argument so there exists the Popen.recv function that passes selects stdout as the pipe for Popen._recv by default. Popen.recv_err selects stderr as the pipe by default. "Popen.recv" and "Popen.recv_err" are much easier to read and understand than "Popen._recv('stdout' ..." and "Popen._recv('stderr' ..." respectively. Since the Popen._recv function does not wait on data to be produced before returning a value, it may return empty bytes. Popen.asyncread handles this issue by returning all data read over a given time interval. The ProcessIOWrapper class uses the asyncread and asyncwrite functions to allow a process to act like a file so that there are no blocking issues that can arise from using the stdout and stdin file objects produced from a subprocess.Popen call. References: [1] [ python-Feature Requests-1191964 ] asynchronous Subprocess http://mail.python.org/pipermail/python-bugs-list/2006-December/ 036524.html [2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess http://ivory.idyll.org/blog/feb-07/problems-with-subprocess [3] How can I run an external command asynchronously from Python? - Stack Overflow http://stackoverflow.com/questions/636561/how-can-i-run-an-external- command-asynchronously-from-python [4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation http://docs.python.org/library/subprocess.html#subprocess.Popen.wait [5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation http://docs.python.org/library/subprocess.html#subprocess.Popen.kill [6] Issue 1191964: asynchronous Subprocess - Python tracker http://bugs.python.org/issue1191964 [7] Module to allow Asynchronous subprocess use on Windows and Posix platforms - ActiveState Code http://code.activestate.com/recipes/440554/ [8] subprocess.rst - subprocdev - Project Hosting on Google Code http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437 [9] subprocdev - Project Hosting on Google Code http://code.google.com/p/subprocdev [10] Python Subprocess Dev http://subdev.blogspot.com/ Copyright: This P.E.P. is licensed under the Open Publication License; http://www.opencontent.org/openpub/. On Tue, Sep 8, 2009 at 22:56, Benjamin Peterson wrote: > 2009/9/7 Eric Pruitt : >> Hello all, >> >> I have been working on adding asynchronous I/O to the Python >> subprocess module as part of my Google Summer of Code project. Now >> that I have finished documenting and pruning the code, I present PEP >> 3145 for its inclusion into the Python core code. Any and all feedback >> on the PEP (http://www.python.org/dev/peps/pep-3145/) is appreciated. > > Hi Eric, > One of the reasons you're not getting many response is that you've not > pasted the contents of the PEP in this message. That makes it really > easy for people to comment on various sections. > > BTW, it seems like you were trying to use reST formatting with the > text PEP layout. Double backquotes only mean something in reST. > > > -- > Regards, > Benjamin > From barry at python.org Wed Sep 9 15:01:35 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 9 Sep 2009 09:01:35 -0400 Subject: [Python-Dev] Python 2.6.3 Message-ID: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> I had previously wanted to release Python 2.6.3 over the summer, but for various personal reasons, the summer was just too insane. I'd like to reschedule a 2.6.3 release, shooting for final release on 25- September. We should probably do a release candidate, so I'd like to make that on 23-September. Does anybody have objections to that schedule? If not, I'll try to spend some time over the next few days looking at outstanding bugs, and marking release blockers, etc. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From ncoghlan at gmail.com Wed Sep 9 15:29:47 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 09 Sep 2009 23:29:47 +1000 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> Message-ID: <4AA7ADCB.6080101@gmail.com> Barry Warsaw wrote: > I had previously wanted to release Python 2.6.3 over the summer, but for > various personal reasons, the summer was just too insane. I'd like to > reschedule a 2.6.3 release, shooting for final release on 25-September. > > We should probably do a release candidate, so I'd like to make that on > 23-September. > > Does anybody have objections to that schedule? If not, I'll try to > spend some time over the next few days looking at outstanding bugs, and > marking release blockers, etc. 2 days seems a little short (particularly allowing 24 hours or so for the Windows and Mac installers to be produced). Haven't we historically left a week between the RC and actual release for maintenance releases? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From barry at python.org Wed Sep 9 15:52:22 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 9 Sep 2009 09:52:22 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: <4AA7ADCB.6080101@gmail.com> References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> Message-ID: <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> On Sep 9, 2009, at 9:29 AM, Nick Coghlan wrote: > Barry Warsaw wrote: >> I had previously wanted to release Python 2.6.3 over the summer, >> but for >> various personal reasons, the summer was just too insane. I'd like >> to >> reschedule a 2.6.3 release, shooting for final release on 25- >> September. >> >> We should probably do a release candidate, so I'd like to make that >> on >> 23-September. >> >> Does anybody have objections to that schedule? If not, I'll try to >> spend some time over the next few days looking at outstanding bugs, >> and >> marking release blockers, etc. > > 2 days seems a little short (particularly allowing 24 hours or so for > the Windows and Mac installers to be produced). Haven't we > historically > left a week between the RC and actual release for maintenance > releases? Actually, I've rarely done rc's for point releases. JFDI :) I still want to release by the 25th, but I'd be willing to move the rc to Monday the 21st. We're really just trying to avoid a brown bag moment, so that should give us enough time to double check the releases. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From brett at python.org Wed Sep 9 18:14:19 2009 From: brett at python.org (Brett Cannon) Date: Wed, 9 Sep 2009 09:14:19 -0700 Subject: [Python-Dev] PEP 3145 In-Reply-To: <1afaf6160909081956x262ed97dlc2ba1df99e7bc262@mail.gmail.com> References: <171e8a410909071113r7d06399t8029f4b795e5b0aa@mail.gmail.com> <1afaf6160909081956x262ed97dlc2ba1df99e7bc262@mail.gmail.com> Message-ID: On Tue, Sep 8, 2009 at 19:56, Benjamin Peterson wrote: > 2009/9/7 Eric Pruitt : >> Hello all, >> >> I have been working on adding asynchronous I/O to the Python >> subprocess module as part of my Google Summer of Code project. Now >> that I have finished documenting and pruning the code, I present PEP >> 3145 for its inclusion into the Python core code. Any and all feedback >> on the PEP (http://www.python.org/dev/peps/pep-3145/) is appreciated. > > Hi Eric, > One of the reasons you're not getting many response is that you've not > pasted the contents of the PEP in this message. That makes it really > easy for people to comment on various sections. > > BTW, it seems like you were trying to use reST formatting with the > text PEP layout. Double backquotes only mean something in reST. That reST markup is my fault, not Eric. So used to reST PEPs that I just instinctively added it when I check the PEP in. -Brett From thomas at python.org Wed Sep 9 18:07:01 2009 From: thomas at python.org (Thomas Wouters) Date: Wed, 9 Sep 2009 18:07:01 +0200 Subject: [Python-Dev] pthreads, fork, import, and execvp In-Reply-To: <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> References: <446C2211.5060408@v.loewis.de> <446C6FC9.5060205@gmail.com> <446CB69E.4090609@v.loewis.de> <9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com> <3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com> <9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com> <9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com> <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> Message-ID: <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith wrote: > On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters wrote: > > > > So attached (and at http://codereview.appspot.com/96125/show ) is a > > preliminary fix, correcting the problem with os.fork(), os.forkpty() and > > os.fork1(). This doesn't expose a general API for C code to use, for two > > reasons: it's not easy, and I need this fix more than I need the API > change > > :-) (I actually need this fix myself for Python 2.4, but it applies > fairly > > cleanly.) > > This looks good to me. > Anyone else want to take a look at this before I check it in? I updated the patch (in Rietveld) to contain some documentation about the hazards of mixing fork and threads, which is the best we can do at the moment, at least without seriously overhauling the threading APIs (which, granted, is not that bad an idea, considering the mess they're in.) I've now thoroughly tested the patch, and for most platforms it's strictly better. On AIX it *may* behave differently (possibly 'incorrectly' for specific cases) if something other than os.fork() calls the C fork() and calls PyOS_AfterFork(), since on AIX it used to nuke the thread lock. *I* think the new behaviour (not nuking the lock) is the correct thing to do, but since most places that release the import lock don't bother to check if the lock was even held, the old behaviour may have been succesfully masking the bug on AIX systems. Perhaps for the backport to 2.6 (which I think is in order, and also in accordance with the guidelines) I should leave the AIX workaround in? Anyone think it should not be removed from 3.x/2.7? > Your idea of making this an API called a 'fork lock' or something > sounds good (though I think it needs a better name. PyBeforeFork & > PyAfterFork?). The subprocess module, for example, disables garbage > collection before forking and restores it afterwards to avoid > http://bugs.python.org/issue1336. That type of thing could also be > done in such a function. > Unfortunately it's rather hard to make those functions work correctly with the current API -- we can't provide functions you can just use as arguments to pthread_atfork because the global interpreter lock is not re-entrant and we have no way of testing whether the current thread holds the GIL. I also get the creepy-crawlies when I look at the various thread_* implementations and see the horribly unsafe things they do (and also, for instance, the PendingCall stuff in ceval.c :S) Unfortunately there's no good way to fix these things without breaking API compatibility, let alone ABI compatibility. > Related to the above subprocess fork + gc bug.. It'd be nice for > CPython to have code that does the fork+misc twiddling+exec all in one > C call without needing to execute Python code in the child process > prior to the exec(). Most of the inner body of subprocess's > _execute_child() method could be done that way. (with the obvious > exception of the preexec_fn) > > > > > To fix the mutex-across-fork problem correctly we should really acquire > > three locks (the import lock, the GIL and the TLS lock, in that order.) > The > > import lock is re-entrant, and the TLS lock is tightly confined to the > > thread-local-storage lookup functions, but the GIL is neither re-entrant > nor > > inspectable. That means we can't use pthread_atfork (we can't tell > whether > > we have the GIL already or not, right before the fork), nor can we > provide a > > convenient API for extension modules to use, really. The best we can do > is > > provide three functions, pthread_atfork-style: a 'prepare' function, an > > 'after-fork-in-child' function, and an 'after-fork-in-parent' function. > The > > 'prepare' function would start by releasing the GIL, then acquire the > import > > lock, the GIL and the TLS lock in that order. It would also record > > *somewhere* the thread_ident of the current thread. The 'in-parent' > function > > would simply release the TLS lock and the import lock, and the 'in-child' > > would release those locks, call the threading._at_fork() function, and > fix > > up the TLS data, using the recorded thread ident to do lookups. The > > 'in-child' function would replace the current PyOS_AfterFork() function > > (which blindly reinitializes the GIL and the TLS lock, and calls > > threading._at_fork().) > > > > Alternatively we could do what we do now, which is to ignore the fact > that > > thread idents may change by fork() and thus that thread-local data may > > disappear, in which case the 'in-child' function could do a little less > > work. I'm suitably scared and disgusted of the TLS implementation, the > > threading implementations we support (including the pthread one) and the > way > > we blindly type-pun a pthread_t to a long, that I'm not sure I want to > try > > and build something correct and reliable on top of it. > > > > -- > > Thomas Wouters > > > > Hi! I'm a .signature virus! copy me into your .signature file to help me > > spread! > > > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org > > > > > -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: From nad at acm.org Wed Sep 9 19:29:04 2009 From: nad at acm.org (Ned Deily) Date: Wed, 09 Sep 2009 10:29:04 -0700 Subject: [Python-Dev] Python 2.6.3 References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> Message-ID: In article <11A6545D-7204-4F61-B55B-1CC77CB5645E at python.org>, Barry Warsaw wrote: > I still want to release by the 25th, but I'd be willing to move the rc > to Monday the 21st. We're really just trying to avoid a brown bag > moment, so that should give us enough time to double check the releases. The recent release of OS X 10.6 (Snow Leopard) has triggered a fair amount of 2.6 bug tracker activity, since 10.6 now includes 2.6 (2.6.1) and a 64-bit version at that. A number of patches have either just been checked-in over the past couple of weeks or are getting some exposure before check-in. Given the timing and the (appropriate) infrequency of 2.6.x releases, I think it would be unfortunate to push 2.6.3 out the door without ensuring that it works well on 10.6. Therefore, I propose that 2.6.3 should have 10.6 compatibility as a "release goal". Without trying to put Ronald on the spot (too much!), it would be a good idea to get his assessment where things stand wrt 2.6 on 10.6 before setting a final release date. -- Ned Deily, nad at acm.org From greg at krypto.org Wed Sep 9 20:19:42 2009 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 9 Sep 2009 11:19:42 -0700 Subject: [Python-Dev] pthreads, fork, import, and execvp In-Reply-To: <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> References: <446C2211.5060408@v.loewis.de> <446C6FC9.5060205@gmail.com> <446CB69E.4090609@v.loewis.de> <9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com> <3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com> <9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com> <9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com> <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> Message-ID: <52dc1c820909091119x57d7aee6vda213659a2402dea@mail.gmail.com> On Wed, Sep 9, 2009 at 9:07 AM, Thomas Wouters wrote: > > > On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith wrote: >> >> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters wrote: >> > >> > So attached (and at http://codereview.appspot.com/96125/show ) is a >> > preliminary fix, correcting the problem with os.fork(), os.forkpty() and >> > os.fork1(). This doesn't expose a general API for C code to use, for two >> > reasons: it's not easy, and I need this fix more than I need the API >> > change >> > :-) (I actually need this fix myself for Python 2.4, but it applies >> > fairly >> > cleanly.) >> >> This looks good to me. > > Anyone else want to take a look at this before I check it in? I updated the > patch (in Rietveld) to contain some documentation about the hazards of > mixing fork and threads, which is the best we can do at the moment, at least > without seriously overhauling the threading APIs (which, granted, is not > that bad an idea, considering the mess they're in.) I've now thoroughly > tested the patch, and for most platforms it's strictly better. On AIX it > *may* behave differently (possibly 'incorrectly' for specific cases) if > something other than os.fork() calls the C fork() and calls > PyOS_AfterFork(), since on AIX it used to nuke the thread lock. *I* think > the new behaviour (not nuking the lock) is the correct thing to do, but > since most places that release the import lock don't bother to check if the > lock was even held, the old behaviour may have been succesfully masking the > bug on AIX systems. > Perhaps for the backport to 2.6 (which I think is in order, and also in > accordance with the guidelines) I should leave the AIX workaround in? Anyone > think it should not be removed from 3.x/2.7? > >> >> Your idea of making this an API called a 'fork lock' or something >> sounds good (though I think it needs a better name. ?PyBeforeFork & >> PyAfterFork?). ?The subprocess module, for example, disables garbage >> collection before forking and restores it afterwards to avoid >> http://bugs.python.org/issue1336. ?That type of thing could also be >> done in such a function. > > Unfortunately it's rather hard to make those functions work correctly with > the current API -- we can't provide functions you can just use as arguments > to pthread_atfork because the global interpreter lock is not re-entrant and > we have no way of testing whether the current thread holds the GIL. I also > get the creepy-crawlies when I look at the various thread_* implementations > and see the horribly unsafe things they do (and also, for instance, the > PendingCall stuff in ceval.c :S) Unfortunately there's no good way to fix > these things without breaking API compatibility, let alone ABI > compatibility. Take a look at http://code.google.com/p/python-atfork/ which I created to address the general fork+threading with locks held causing deadlocks issue with many standard library modules. Currently it only patches the logging module but I intend to support others. I want to get an atfork mechanism into 2.7/3.2 along with every lock in the standard library making proper use of it. See also http://bugs.python.org/issue6721 I make no attempt to deal with C-level locks, only those acquired from python. It doesn't use pthread_atfork but instead models its behavior after that and wraps os.fork and os.forkpty so that they call the registered atfork methods as appropriate. > >> >> Related to the above subprocess fork + gc bug.. It'd be nice for >> CPython to have code that does the fork+misc twiddling+exec all in one >> C call without needing to execute Python code in the child process >> prior to the exec(). ?Most of the inner body of subprocess's >> _execute_child() method could be done that way. ?(with the obvious >> exception of the preexec_fn) >> >> > >> > To fix the mutex-across-fork problem correctly we should really acquire >> > three locks (the import lock, the GIL and the TLS lock, in that order.) >> > The >> > import lock is re-entrant, and the TLS lock is tightly confined to the >> > thread-local-storage lookup functions, but the GIL is neither re-entrant >> > nor >> > inspectable. That means we can't use pthread_atfork (we can't tell >> > whether >> > we have the GIL already or not, right before the fork), nor can we >> > provide a >> > convenient API for extension modules to use, really. The best we can do >> > is >> > provide three functions, pthread_atfork-style: a 'prepare' function, an >> > 'after-fork-in-child' function, and an 'after-fork-in-parent' function. >> > The >> > 'prepare' function would start by releasing the GIL, then acquire the >> > import >> > lock, the GIL and the TLS lock in that order. It would also record >> > *somewhere* the thread_ident of the current thread. The 'in-parent' >> > function >> > would simply release the TLS lock and the import lock, and the >> > 'in-child' >> > would release those locks, call the threading._at_fork() function, and >> > fix >> > up the TLS data, using the recorded thread ident to do lookups. The >> > 'in-child' function would replace the current PyOS_AfterFork() function >> > (which blindly reinitializes the GIL and the TLS lock, and calls >> > threading._at_fork().) >> > >> > Alternatively we could do what we do now, which is to ignore the fact >> > that >> > thread idents may change by fork() and thus that thread-local data may >> > disappear, in which case the 'in-child' function could do a little less >> > work. I'm suitably scared and disgusted of the TLS implementation, the >> > threading implementations we support (including the pthread one) and the >> > way >> > we blindly type-pun a pthread_t to a long, that I'm not sure I want to >> > try >> > and build something correct and reliable on top of it. >> > >> > -- >> > Thomas Wouters >> > >> > Hi! I'm a .signature virus! copy me into your .signature file to help me >> > spread! >> > >> > _______________________________________________ >> > Python-Dev mailing list >> > Python-Dev at python.org >> > http://mail.python.org/mailman/listinfo/python-dev >> > Unsubscribe: >> > http://mail.python.org/mailman/options/python-dev/greg%40krypto.org >> > >> > > > > > -- > Thomas Wouters > > Hi! I'm a .signature virus! copy me into your .signature file to help me > spread! > From greg at krypto.org Wed Sep 9 20:38:53 2009 From: greg at krypto.org (Gregory P. Smith) Date: Wed, 9 Sep 2009 11:38:53 -0700 Subject: [Python-Dev] pthreads, fork, import, and execvp In-Reply-To: <9e804ac0909091136x57272ab2h67e4bdaf458c0b0a@mail.gmail.com> References: <446CB69E.4090609@v.loewis.de> <9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com> <3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com> <9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com> <9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com> <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> <52dc1c820909091119x57d7aee6vda213659a2402dea@mail.gmail.com> <9e804ac0909091136x57272ab2h67e4bdaf458c0b0a@mail.gmail.com> Message-ID: <52dc1c820909091138q42d66f42s8712ac2c4eebaa9d@mail.gmail.com> On Wed, Sep 9, 2009 at 11:36 AM, Thomas Wouters wrote: > > On Wed, Sep 9, 2009 at 20:19, Gregory P. Smith wrote: >> >> Take a look at http://code.google.com/p/python-atfork/ which I created >> to address the general fork+threading with locks held causing >> deadlocks issue with many standard library modules. ?Currently it only >> patches the logging module but I intend to support others. ?I want to >> get an atfork mechanism into 2.7/3.2 along with every lock in the >> standard library making proper use of it. >> >> See also http://bugs.python.org/issue6721 >> >> I make no attempt to deal with C-level locks, only those acquired from >> python. ?It doesn't use pthread_atfork but instead models its behavior >> after that and wraps os.fork and os.forkpty so that they call the >> registered atfork methods as appropriate. > > Well, sure, the *Python code* side of the problem is fixable, thanks to > Python being so awesome ;-P I'm strictly thinking of applications embedding > Python (or even extending it and calling into code that doesn't consider > Python.) Your python-atfork project looks like a terribly good idea, but it > won't fix the embedding/extending issues (nor is it intended to, right?) yeah I decided to only fix what could obviously be fixed and was causing me pain at the moment. ;) From thomas at python.org Wed Sep 9 20:36:02 2009 From: thomas at python.org (Thomas Wouters) Date: Wed, 9 Sep 2009 20:36:02 +0200 Subject: [Python-Dev] pthreads, fork, import, and execvp In-Reply-To: <52dc1c820909091119x57d7aee6vda213659a2402dea@mail.gmail.com> References: <446C6FC9.5060205@gmail.com> <446CB69E.4090609@v.loewis.de> <9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com> <3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com> <9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com> <9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com> <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> <52dc1c820909091119x57d7aee6vda213659a2402dea@mail.gmail.com> Message-ID: <9e804ac0909091136x57272ab2h67e4bdaf458c0b0a@mail.gmail.com> On Wed, Sep 9, 2009 at 20:19, Gregory P. Smith wrote: > On Wed, Sep 9, 2009 at 9:07 AM, Thomas Wouters wrote: > > > > > > On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith wrote: > >> > >> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters > wrote: > >> > > >> > So attached (and at http://codereview.appspot.com/96125/show ) is a > >> > preliminary fix, correcting the problem with os.fork(), os.forkpty() > and > >> > os.fork1(). This doesn't expose a general API for C code to use, for > two > >> > reasons: it's not easy, and I need this fix more than I need the API > >> > change > >> > :-) (I actually need this fix myself for Python 2.4, but it applies > >> > fairly > >> > cleanly.) > >> > >> This looks good to me. > > > > Anyone else want to take a look at this before I check it in? I updated > the > > patch (in Rietveld) to contain some documentation about the hazards of > > mixing fork and threads, which is the best we can do at the moment, at > least > > without seriously overhauling the threading APIs (which, granted, is not > > that bad an idea, considering the mess they're in.) I've now thoroughly > > tested the patch, and for most platforms it's strictly better. On AIX it > > *may* behave differently (possibly 'incorrectly' for specific cases) if > > something other than os.fork() calls the C fork() and calls > > PyOS_AfterFork(), since on AIX it used to nuke the thread lock. *I* think > > the new behaviour (not nuking the lock) is the correct thing to do, but > > since most places that release the import lock don't bother to check if > the > > lock was even held, the old behaviour may have been succesfully masking > the > > bug on AIX systems. > > Perhaps for the backport to 2.6 (which I think is in order, and also in > > accordance with the guidelines) I should leave the AIX workaround in? > Anyone > > think it should not be removed from 3.x/2.7? > > > >> > >> Your idea of making this an API called a 'fork lock' or something > >> sounds good (though I think it needs a better name. PyBeforeFork & > >> PyAfterFork?). The subprocess module, for example, disables garbage > >> collection before forking and restores it afterwards to avoid > >> http://bugs.python.org/issue1336. That type of thing could also be > >> done in such a function. > > > > Unfortunately it's rather hard to make those functions work correctly > with > > the current API -- we can't provide functions you can just use as > arguments > > to pthread_atfork because the global interpreter lock is not re-entrant > and > > we have no way of testing whether the current thread holds the GIL. I > also > > get the creepy-crawlies when I look at the various thread_* > implementations > > and see the horribly unsafe things they do (and also, for instance, the > > PendingCall stuff in ceval.c :S) Unfortunately there's no good way to fix > > these things without breaking API compatibility, let alone ABI > > compatibility. > > Take a look at http://code.google.com/p/python-atfork/ which I created > to address the general fork+threading with locks held causing > deadlocks issue with many standard library modules. Currently it only > patches the logging module but I intend to support others. I want to > get an atfork mechanism into 2.7/3.2 along with every lock in the > standard library making proper use of it. > > See also http://bugs.python.org/issue6721 > > I make no attempt to deal with C-level locks, only those acquired from > python. It doesn't use pthread_atfork but instead models its behavior > after that and wraps os.fork and os.forkpty so that they call the > registered atfork methods as appropriate. Well, sure, the *Python code* side of the problem is fixable, thanks to Python being so awesome ;-P I'm strictly thinking of applications embedding Python (or even extending it and calling into code that doesn't consider Python.) Your python-atfork project looks like a terribly good idea, but it won't fix the embedding/extending issues (nor is it intended to, right?) -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Wed Sep 9 21:06:28 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 9 Sep 2009 15:06:28 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> Message-ID: <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> On Sep 9, 2009, at 1:29 PM, Ned Deily wrote: > In article <11A6545D-7204-4F61-B55B-1CC77CB5645E at python.org>, > Barry Warsaw wrote: >> I still want to release by the 25th, but I'd be willing to move the >> rc >> to Monday the 21st. We're really just trying to avoid a brown bag >> moment, so that should give us enough time to double check the >> releases. > > The recent release of OS X 10.6 (Snow Leopard) has triggered a fair > amount of 2.6 bug tracker activity, since 10.6 now includes 2.6 > (2.6.1) > and a 64-bit version at that. A number of patches have either just > been checked-in over the past couple of weeks or are getting some > exposure before check-in. Given the timing and the (appropriate) > infrequency of 2.6.x releases, I think it would be unfortunate to push > 2.6.3 out the door without ensuring that it works well on 10.6. > Therefore, I propose that 2.6.3 should have 10.6 compatibility as a > "release goal". > > Without trying to put Ronald on the spot (too much!), it would be a > good > idea to get his assessment where things stand wrt 2.6 on 10.6 before > setting a final release date. I'm hoping that Python won't have any issues building and running on 10.6, but I don't have it yet so I can't personally test it out. How would you quantify "works well"? Do you have any thoughts on tests you'd run other than the standard test suite? If 2.6.3 is shown to pass its test suite on 10.5.x, is that good enough? Are the specific bug fixes necessary for 10.6? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From ronaldoussoren at mac.com Wed Sep 9 20:19:37 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 09 Sep 2009 20:19:37 +0200 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> Message-ID: On 9 Sep, 2009, at 19:29, Ned Deily wrote: > > > Without trying to put Ronald on the spot (too much!), it would be a > good > idea to get his assessment where things stand wrt 2.6 on 10.6 before > setting a final release date. MacOS X 10.6 support should be stable now, except for a critical issue with IDLE: opening a new window hangs IDLE (issue 6864). That said, I haven't scanned the issue tracker for more 10.6 related issues. BTW. I'm fine with a sept. 25th release, that should give us enough time to shake out issues with OSX 10.6 support. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From ncoghlan at gmail.com Wed Sep 9 23:56:44 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 10 Sep 2009 07:56:44 +1000 Subject: [Python-Dev] pthreads, fork, import, and execvp In-Reply-To: <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> References: <446C2211.5060408@v.loewis.de> <446C6FC9.5060205@gmail.com> <446CB69E.4090609@v.loewis.de> <9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com> <3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com> <9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com> <9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com> <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> Message-ID: <4AA8249C.9010707@gmail.com> Thomas Wouters wrote: > Your idea of making this an API called a 'fork lock' or something > sounds good (though I think it needs a better name. PyBeforeFork & > PyAfterFork?). The subprocess module, for example, disables garbage > collection before forking and restores it afterwards to avoid > http://bugs.python.org/issue1336. That type of thing could also be > done in such a function. > > > Unfortunately it's rather hard to make those functions work correctly > with the current API -- we can't provide functions you can just use as > arguments to pthread_atfork because the global interpreter lock is not > re-entrant and we have no way of testing whether the current thread > holds the GIL. I thought "make sure I have the GIL, either by already having it or waiting for it if I don't yet have it" was the entire point of the PyGILState_Ensure() API? [1] Cheers, Nick. [1] http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From zvezdan at zope.com Thu Sep 10 14:47:40 2009 From: zvezdan at zope.com (Zvezdan Petkovic) Date: Thu, 10 Sep 2009 08:47:40 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> Message-ID: On Sep 9, 2009, at 2:19 PM, Ronald Oussoren wrote: > MacOS X 10.6 support should be stable now, except for a critical issue > with IDLE: opening a new window hangs IDLE (issue 6864). > > That said, I haven't scanned the issue tracker for more 10.6 related > issues. I just opened issue 6877 and provided a patch that I use for a long time already. It would be nice to have it in the release though since other people could benefit from it too. It resolves a BusError crash when Python readline module is built with a native Mac OS X editline (emulates readline). The patch fixes the issue and enables use of readline module on Mac OS X. Best regards, Zvezdan From aahz at pythoncraft.com Thu Sep 10 17:11:55 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 10 Sep 2009 08:11:55 -0700 Subject: [Python-Dev] FWD: Front Runner Program Message-ID: <20090910151155.GA13116@panix.com> I'm still no-mail on python-dev, forwarding as FYI ----- Forwarded message from Eric Albrecht ----- From: Eric Albrecht To: "webmaster at python.org" Date: Thu, 10 Sep 2009 10:48:11 -0400 Subject: Front Runner Program Regarding: Windows 7 Compatibility for Python Application. I am trying to contact your company regarding the Microsoft Windows 7 Compatibility Program for the application above. I have not been able to get in touch with the person responsible for this application in your company and this is why I am reaching out to you through the Support Team. This application has been identified as one of the applications Microsoft would like to see supported on Windows 7 and I have been tasked by Microsoft to help answer your questions about Windows 7 application compatibility and help you get your application through the Windows 7 "Green Light" compatibility process. If your application already supports Windows Vista, chances are it will already be compatible with Windows 7 without the need for any code changes. By pledging support for Windows 7 you're application will automatically be listed in the Windows Application Compatibility seen currently by more than 1 million users per month. The registration is extremely simple and just asks a few key questions. Here is the link to Microsoft's ISV Application Compatibility page: www.isvappcompat.com . When you have a moment, I would encourage you to visit the site and complete the process to pledge support for your application on Windows 7 by October 22nd 2009 when Windows 7 is officially released. In addition if you are able to pledge compatibility you'll receive access to a special Windows 7 Partner Marketing Kit that includes a press release with a Microsoft quote, plus customizable marketing templates including; email templates, postcards, web banners, business letter, and copy blocks, all to identify to your customers, or potential customers that your solutions are compatible with Windows 7. If you provide me with a phone number where to get in touch with you, I will call you to answer any questions you may have. Once you register on the ISV Application Compatibility site, I would appreciate it if email me to let me know that you have completed so that I can make a note of it for Microsoft. If you register the application under a different partner or application name please let me know in order to track changes. If there is a new version of the application and there are no plans to support Windows 7 on the older version please register the older version as "No planned Support" on the site as well as the new version with desired Win7 compatibility date. Should you have any questions about this email feel free to call me or send an email to my supervisor at v-mafl at microsoft.com. Best regards, Eric Albrecht 800-508-4291 EXT: 309 eric.albrecht at nichecubed.com ----- End forwarded message ----- -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "To me vi is Zen. To use vi is to practice zen. Every command is a koan. Profound to the user, unintelligible to the uninitiated. You discover truth everytime you use it." --reddy at lion.austin.ibm.com From ctb at msu.edu Thu Sep 10 17:20:35 2009 From: ctb at msu.edu (C. Titus Brown) Date: Thu, 10 Sep 2009 08:20:35 -0700 Subject: [Python-Dev] FWD: Front Runner Program In-Reply-To: <20090910151155.GA13116@panix.com> References: <20090910151155.GA13116@panix.com> Message-ID: <20090910152035.GC26180@idyll.org> On Thu, Sep 10, 2009 at 08:11:55AM -0700, Aahz wrote: -> I'm still no-mail on python-dev, forwarding as FYI -> -> ----- Forwarded message from Eric Albrecht ----- -> -> From: Eric Albrecht -> To: "webmaster at python.org" -> Date: Thu, 10 Sep 2009 10:48:11 -0400 -> Subject: Front Runner Program -> -> Regarding: Windows 7 Compatibility for Python Application. Thanks, Aahz! I don't see a Windows 7 buildbot up here: http://www.python.org/dev/buildbot/all/ but I confess that I'm bad at reading these pages. Has anyone tried compiling either trunk or py3k on Win 7? Would this be useful? My recently acquired* MSDN account has led me to getting XP up and running in a VM, and I would be happy to try other Windows OSes of interest. --titus * acquired courtesy of Snakebite/Trent Nelson. -- C. Titus Brown, ctb at msu.edu From nad at acm.org Thu Sep 10 18:23:08 2009 From: nad at acm.org (Ned Deily) Date: Thu, 10 Sep 2009 09:23:08 -0700 Subject: [Python-Dev] Python 2.6.3 References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> Message-ID: In article <9D506035-7C2D-4929-A134-E88EEB7B7D9E at python.org>, Barry Warsaw wrote: > On Sep 9, 2009, at 1:29 PM, Ned Deily wrote: > > > In article <11A6545D-7204-4F61-B55B-1CC77CB5645E at python.org>, > > Barry Warsaw wrote: > >> I still want to release by the 25th, but I'd be willing to move the > >> rc > >> to Monday the 21st. We're really just trying to avoid a brown bag > >> moment, so that should give us enough time to double check the > >> releases. > > > > The recent release of OS X 10.6 (Snow Leopard) has triggered a fair > > amount of 2.6 bug tracker activity, since 10.6 now includes 2.6 > > (2.6.1) > > and a 64-bit version at that. A number of patches have either just > > been checked-in over the past couple of weeks or are getting some > > exposure before check-in. Given the timing and the (appropriate) > > infrequency of 2.6.x releases, I think it would be unfortunate to push > > 2.6.3 out the door without ensuring that it works well on 10.6. > > Therefore, I propose that 2.6.3 should have 10.6 compatibility as a > > "release goal". > > > > Without trying to put Ronald on the spot (too much!), it would be a > > good > > idea to get his assessment where things stand wrt 2.6 on 10.6 before > > setting a final release date. > > I'm hoping that Python won't have any issues building and running on > 10.6, but I don't have it yet so I can't personally test it out. > > How would you quantify "works well"? Do you have any thoughts on > tests you'd run other than the standard test suite? If 2.6.3 is shown > to pass its test suite on 10.5.x, is that good enough? Are the > specific bug fixes necessary for 10.6? Running the standard test suite on 10.6 and seeing no regressions compared to the same suite on 10.5.x seems a reasonable necessary requirement. We have the resources to do that. Beyond that, as Ronald suggests, I think it important to go through the open issues in the next couple of days and identify and flag any potential release-blockers (besides the IDLE problem already mentioned). One other open issue is 64-bit support in the python.org OS X installer. There have been discussions and requests in the past and, with Apple providing 64-bit out of the box in 10.6, it seems like it's time to provide something on python.org as well. One option: continue to provide a 32-bit only installer for ppc and i386 for 10.3.9 and beyond and add a second installer image with 3-way (ppc, i386, x86_64 but no ppc64) 32/64 for 10.5 and beyond. Ronald, is that your current thinking? -- Ned Deily, nad at acm.org From martin at v.loewis.de Thu Sep 10 18:57:09 2009 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 10 Sep 2009 18:57:09 +0200 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> Message-ID: <4AA92FE5.5060004@v.loewis.de> Barry Warsaw schrieb: > I had previously wanted to release Python 2.6.3 over the summer, but for > various personal reasons, the summer was just too insane. I'd like to > reschedule a 2.6.3 release, shooting for final release on 25-September. I'm travelling that week (as well as the time until then), so I would prefer to make it a week later. Regards, Martin From martin at v.loewis.de Thu Sep 10 19:03:01 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 10 Sep 2009 19:03:01 +0200 Subject: [Python-Dev] FWD: Front Runner Program In-Reply-To: <20090910152035.GC26180@idyll.org> References: <20090910151155.GA13116@panix.com> <20090910152035.GC26180@idyll.org> Message-ID: <4AA93145.2050602@v.loewis.de> > My recently acquired* MSDN account has led me to getting XP up and > running in a VM, and I would be happy to try other Windows OSes of > interest. Without checking the specific program, I think chances are that there are higher requirements for being listed in some Microsoft list than "it works", in particular wrt. the installation procedure. Of course, testing that it works is a good idea independent of any logo certification. Regards, Martin From janssen at parc.com Thu Sep 10 19:09:50 2009 From: janssen at parc.com (Bill Janssen) Date: Thu, 10 Sep 2009 10:09:50 PDT Subject: [Python-Dev] Controlling the cipher list for SSL connections In-Reply-To: References: <4AA53199.5090409@voidspace.org.uk> Message-ID: <699.1252602590@parc.com> Thanks, Chris. Can you explain why you want to set the cipher list explicitly? IMO, it's usually better to select a security scheme (TLS1, or SSLv3, etc.), and let the implementation pick the cipher list. Bill Chris Frantz wrote: > Done. > > Attached to Issue 3597, which is a similar request to mine. > > Best Regards, > --Chris > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com From janssen at parc.com Thu Sep 10 20:14:51 2009 From: janssen at parc.com (Bill Janssen) Date: Thu, 10 Sep 2009 11:14:51 PDT Subject: [Python-Dev] Controlling the cipher list for SSL connections In-Reply-To: References: <4AA53199.5090409@voidspace.org.uk> <699.1252602590@parc.com> Message-ID: <4250.1252606491@parc.com> Chris, OK, seems reasonable. Thanks. In the near term, can you do this with M2Crypto or PyOpenSSL? When I started this update in 2007, we were trying to keep the API simple to avoid confusing people and avoid competition with the two full-fledged toolkits out there. But I don't see any real reason not to extend the API a bit. Bill Chris Frantz wrote: > Bill, > > I agree that it's usually better to let the SSL implementation pick > the ciphers. > > I have a certain device that I'd like to talk to that is running on an > underpowered embedded CPU. When I let OpenSSL pick the ciphers, it > chooses something like EDH-RSA-AES-SHA and takes about 3.5 seconds to > finish the handshake. If I can restrict the cipher list to > RSA-RC4-SHA I can reduce the handshake time to less than a second and > improve the throughput of any bulk data transfer over the connection. > > --Chris > > > > On Thu, Sep 10, 2009 at 12:09 PM, Bill Janssen wrote: > > Thanks, Chris. ?Can you explain why you want to set the cipher list > > explicitly? ?IMO, it's usually better to select a security scheme (TLS1, > > or SSLv3, etc.), and let the implementation pick the cipher list. > > > > Bill > > > > Chris Frantz wrote: > > > >> Done. > >> > >> Attached to Issue 3597, which is a similar request to mine. > >> > >> Best Regards, > >> --Chris > >> _______________________________________________ > >> Python-Dev mailing list > >> Python-Dev at python.org > >> http://mail.python.org/mailman/listinfo/python-dev > >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com > > From htoivonen at spikesource.com Thu Sep 10 20:32:16 2009 From: htoivonen at spikesource.com (Heikki Toivonen) Date: Thu, 10 Sep 2009 11:32:16 -0700 Subject: [Python-Dev] Controlling the cipher list for SSL connections In-Reply-To: <4250.1252606491@parc.com> References: <4AA53199.5090409@voidspace.org.uk> <699.1252602590@parc.com> <4250.1252606491@parc.com> Message-ID: <4AA94630.4040904@spikesource.com> Bill Janssen wrote: > OK, seems reasonable. Thanks. In the near term, can you do this with > M2Crypto or PyOpenSSL? > > When I started this update in 2007, we were trying to keep the API > simple to avoid confusing people and avoid competition with the two > full-fledged toolkits out there. But I don't see any real reason not to > extend the API a bit. Speaking as the M2Crypto maintainer, I don't mind the stdlib competing with M2Crypto/getting better at SSL. In fact, I would actually like to see the stdlib SSL implementation getting good enough so that people would not need M2Crypto for SSL (except maybe in special circumstances). There is much M2Crypto does besides SSL so this wouldn't even obsolete it. One of the main things IMO missing from stdlib SSL implementation is hostname checking by default (with override option), but I know you and I have different opinions on this. I would be happy to provide patches against the stdlib SSL implementation for some things M2Crypto does that the stdlib SSL module is missing if we could agree on the features/design first. Simple is good, but I'd like the defaults to be secure and commonly overridden things to be overrideable. -- Heikki Toivonen -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 252 bytes Desc: OpenPGP digital signature URL: From frantzcj at gmail.com Thu Sep 10 20:26:09 2009 From: frantzcj at gmail.com (Chris Frantz) Date: Thu, 10 Sep 2009 13:26:09 -0500 Subject: [Python-Dev] Controlling the cipher list for SSL connections In-Reply-To: <4250.1252606491@parc.com> References: <4AA53199.5090409@voidspace.org.uk> <699.1252602590@parc.com> <4250.1252606491@parc.com> Message-ID: Bill, For now, using pyOpenSSL is acceptable. I just discovered that the web.py framework wants pyOpenSSL. Since my project is also using web.py, I'll need pyOpenSSL anyway. Thank you, --Chris On Thu, Sep 10, 2009 at 1:14 PM, Bill Janssen wrote: > Chris, > > OK, seems reasonable. ?Thanks. ?In the near term, can you do this with > M2Crypto or PyOpenSSL? > > When I started this update in 2007, we were trying to keep the API > simple to avoid confusing people and avoid competition with the two > full-fledged toolkits out there. ?But I don't see any real reason not to > extend the API a bit. > > Bill > > Chris Frantz wrote: > >> Bill, >> >> I agree that it's usually better to let the SSL implementation pick >> the ciphers. >> >> I have a certain device that I'd like to talk to that is running on an >> underpowered embedded CPU. ? When I let OpenSSL pick the ciphers, it >> chooses something like EDH-RSA-AES-SHA and takes about 3.5 seconds to >> finish ?the handshake. ?If I can restrict the cipher list to >> RSA-RC4-SHA I can reduce the handshake time to less than a second and >> improve the throughput of any bulk data transfer over the connection. >> >> --Chris >> >> >> >> On Thu, Sep 10, 2009 at 12:09 PM, Bill Janssen wrote: >> > Thanks, Chris. ?Can you explain why you want to set the cipher list >> > explicitly? ?IMO, it's usually better to select a security scheme (TLS1, >> > or SSLv3, etc.), and let the implementation pick the cipher list. >> > >> > Bill >> > >> > Chris Frantz wrote: >> > >> >> Done. >> >> >> >> Attached to Issue 3597, which is a similar request to mine. >> >> >> >> Best Regards, >> >> --Chris >> >> _______________________________________________ >> >> Python-Dev mailing list >> >> Python-Dev at python.org >> >> http://mail.python.org/mailman/listinfo/python-dev >> >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com >> > > From frantzcj at gmail.com Thu Sep 10 20:04:22 2009 From: frantzcj at gmail.com (Chris Frantz) Date: Thu, 10 Sep 2009 13:04:22 -0500 Subject: [Python-Dev] Controlling the cipher list for SSL connections In-Reply-To: <699.1252602590@parc.com> References: <4AA53199.5090409@voidspace.org.uk> <699.1252602590@parc.com> Message-ID: Bill, I agree that it's usually better to let the SSL implementation pick the ciphers. I have a certain device that I'd like to talk to that is running on an underpowered embedded CPU. When I let OpenSSL pick the ciphers, it chooses something like EDH-RSA-AES-SHA and takes about 3.5 seconds to finish the handshake. If I can restrict the cipher list to RSA-RC4-SHA I can reduce the handshake time to less than a second and improve the throughput of any bulk data transfer over the connection. --Chris On Thu, Sep 10, 2009 at 12:09 PM, Bill Janssen wrote: > Thanks, Chris. ?Can you explain why you want to set the cipher list > explicitly? ?IMO, it's usually better to select a security scheme (TLS1, > or SSLv3, etc.), and let the implementation pick the cipher list. > > Bill > > Chris Frantz wrote: > >> Done. >> >> Attached to Issue 3597, which is a similar request to mine. >> >> Best Regards, >> --Chris >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com > From janssen at parc.com Thu Sep 10 21:01:12 2009 From: janssen at parc.com (Bill Janssen) Date: Thu, 10 Sep 2009 12:01:12 PDT Subject: [Python-Dev] evolving the SSL module API In-Reply-To: <4AA94630.4040904@spikesource.com> Message-ID: <5335.1252609272@parc.com> Heikki, I'm OK with this, too. would you like to propose an extended API for the SSL module? That would give us a starting point to talk about. This should probably be a PEP, just for the sake of writing things down. As you say, the hostname checking feature seems to me possibly appropriate for some application protocols, though it's made the use of HTTPS as a transport-level protocol unnecessarily confusing and buggy. I don't see putting that into the SSL module as a default, but perhaps a utility function in that module, to check a server-side cert against a hostname, is a good idea. Bill Heikki Toivonen wrote: > Bill Janssen wrote: > > OK, seems reasonable. Thanks. In the near term, can you do this with > > M2Crypto or PyOpenSSL? > > > > When I started this update in 2007, we were trying to keep the API > > simple to avoid confusing people and avoid competition with the two > > full-fledged toolkits out there. But I don't see any real reason not to > > extend the API a bit. > > Speaking as the M2Crypto maintainer, I don't mind the stdlib competing > with M2Crypto/getting better at SSL. In fact, I would actually like to > see the stdlib SSL implementation getting good enough so that people > would not need M2Crypto for SSL (except maybe in special circumstances). > There is much M2Crypto does besides SSL so this wouldn't even obsolete it. > > One of the main things IMO missing from stdlib SSL implementation is > hostname checking by default (with override option), but I know you and > I have different opinions on this. I would be happy to provide patches > against the stdlib SSL implementation for some things M2Crypto does that > the stdlib SSL module is missing if we could agree on the > features/design first. Simple is good, but I'd like the defaults to be > secure and commonly overridden things to be overrideable. > > -- > Heikki Toivonen > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com From jnoller at gmail.com Thu Sep 10 21:17:33 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 10 Sep 2009 15:17:33 -0400 Subject: [Python-Dev] evolving the SSL module API In-Reply-To: <5335.1252609272@parc.com> References: <4AA94630.4040904@spikesource.com> <5335.1252609272@parc.com> Message-ID: <4222a8490909101217v4959e340m4986593863fe196@mail.gmail.com> There's also the patch to httplib that Devin Cook has been working on for SSL enhancements, some of which do name checking. He's got most of a patch completed. On Thu, Sep 10, 2009 at 3:01 PM, Bill Janssen wrote: > Heikki, I'm OK with this, too. ?would you like to propose an extended > API for the SSL module? ?That would give us a starting point to talk > about. > > This should probably be a PEP, just for the sake of writing things down. > > As you say, the hostname checking feature seems to me possibly > appropriate for some application protocols, though it's made the use of > HTTPS as a transport-level protocol unnecessarily confusing and buggy. > I don't see putting that into the SSL module as a default, but perhaps a > utility function in that module, to check a server-side cert against a > hostname, is a good idea. > > Bill > > > Heikki Toivonen wrote: > >> Bill Janssen wrote: >> > OK, seems reasonable. ?Thanks. ?In the near term, can you do this with >> > M2Crypto or PyOpenSSL? >> > >> > When I started this update in 2007, we were trying to keep the API >> > simple to avoid confusing people and avoid competition with the two >> > full-fledged toolkits out there. ?But I don't see any real reason not to >> > extend the API a bit. >> >> Speaking as the M2Crypto maintainer, I don't mind the stdlib competing >> with M2Crypto/getting better at SSL. In fact, I would actually like to >> see the stdlib SSL implementation getting good enough so that people >> would not need M2Crypto for SSL (except maybe in special circumstances). >> There is much M2Crypto does besides SSL so this wouldn't even obsolete it. >> >> One of the main things IMO missing from stdlib SSL implementation is >> hostname checking by default (with override option), but I know you and >> I have different opinions on this. I would be happy to provide patches >> against the stdlib SSL implementation for some things M2Crypto does that >> the stdlib SSL module is missing if we could agree on the >> features/design first. Simple is good, but I'd like the defaults to be >> secure and commonly overridden things to be overrideable. >> >> -- >> ? Heikki Toivonen >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com > From rowen at uw.edu Thu Sep 10 21:55:56 2009 From: rowen at uw.edu (Russell E. Owen) Date: Thu, 10 Sep 2009 12:55:56 -0700 Subject: [Python-Dev] Python 2.6.3 References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> Message-ID: In article , Ronald Oussoren wrote: > On 9 Sep, 2009, at 19:29, Ned Deily wrote: > > > > > > Without trying to put Ronald on the spot (too much!), it would be a > > good > > idea to get his assessment where things stand wrt 2.6 on 10.6 before > > setting a final release date. > > MacOS X 10.6 support should be stable now, except for a critical issue > with IDLE: opening a new window hangs IDLE (issue 6864). > > That said, I haven't scanned the issue tracker for more 10.6 related > issues. > > BTW. I'm fine with a sept. 25th release, that should give us enough time > to shake out issues with OSX 10.6 support. > > Ronald--------------------------------------------------------------------- Some or all earlier Mac binaries of Python 2.6 were not compatible with 3rd party Aqua Tcl/Tk (e.g. ActiveState's versions) -- at least on MacOS X 10.5. I hope that will be fixed with the current release. -- Russell From p.f.moore at gmail.com Thu Sep 10 22:52:16 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 10 Sep 2009 21:52:16 +0100 Subject: [Python-Dev] FWD: Front Runner Program In-Reply-To: <20090910152035.GC26180@idyll.org> References: <20090910151155.GA13116@panix.com> <20090910152035.GC26180@idyll.org> Message-ID: <79990c6b0909101352i20f697b1pab8987643cba0fd8@mail.gmail.com> 2009/9/10 C. Titus Brown : > I don't see a Windows 7 buildbot up here: > > ? ? ? ?http://www.python.org/dev/buildbot/all/ > > but I confess that I'm bad at reading these pages. ?Has anyone tried > compiling either trunk or py3k on Win 7? ?Would this be useful? I plan to do something like this but PC breakage has delayed this. Paul. From devin.c.cook at gmail.com Thu Sep 10 23:00:58 2009 From: devin.c.cook at gmail.com (Devin Cook) Date: Thu, 10 Sep 2009 17:00:58 -0400 Subject: [Python-Dev] evolving the SSL module API In-Reply-To: <4222a8490909101217v4959e340m4986593863fe196@mail.gmail.com> References: <4AA94630.4040904@spikesource.com> <5335.1252609272@parc.com> <4222a8490909101217v4959e340m4986593863fe196@mail.gmail.com> Message-ID: <8847f980909101400y683ac7d6vad673838be832544@mail.gmail.com> Yes, my patch implements hostname checking in httplib (although I haven't had time to do much testing). I also made the documentation changes, but have not yet created any test cases since there really aren't any HTTPS test cases in the test_httplib.py file (which is probably another issue that needs attention). We had talked a month or two back about including hostname checking in the ssl module, but the consensus seemed to be that it doesn't belong there. Personally, I would like to see it make it into the ssl module, as that would mean all the modules that use the ssl module (httplib, etc.) wouldn't have to write their own (and it isn't very straightforward... lots of different RFCs involved). Just my 2 cents. -Devin On Thu, Sep 10, 2009 at 3:17 PM, Jesse Noller wrote: > There's also the patch to httplib that Devin Cook has been working on > for SSL enhancements, some of which do name checking. He's got most of > a patch completed. > > On Thu, Sep 10, 2009 at 3:01 PM, Bill Janssen wrote: >> Heikki, I'm OK with this, too. ?would you like to propose an extended >> API for the SSL module? ?That would give us a starting point to talk >> about. >> >> This should probably be a PEP, just for the sake of writing things down. >> >> As you say, the hostname checking feature seems to me possibly >> appropriate for some application protocols, though it's made the use of >> HTTPS as a transport-level protocol unnecessarily confusing and buggy. >> I don't see putting that into the SSL module as a default, but perhaps a >> utility function in that module, to check a server-side cert against a >> hostname, is a good idea. >> >> Bill >> >> From cs at zip.com.au Fri Sep 11 08:06:58 2009 From: cs at zip.com.au (Cameron Simpson) Date: Fri, 11 Sep 2009 16:06:58 +1000 Subject: [Python-Dev] pthreads, fork, import, and execvp In-Reply-To: <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> References: <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> Message-ID: <20090911060658.GA19749@cskk.homeip.net> On 25Jul2009 10:25, Gregory P. Smith wrote: | On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters wrote: | > So attached (and at http://codereview.appspot.com/96125/show ) is a | > preliminary fix, correcting the problem with os.fork(), os.forkpty() and | > os.fork1(). This doesn't expose a general API for C code to use, for two | > reasons: it's not easy, and I need this fix more than I need the API change | > :-) (I actually need this fix myself for Python 2.4, but it applies fairly | > cleanly.) | | This looks good to me. [...] Where's this stand with respect to the upcoming Python 2.6.3? -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ Life is pleasant. Death is peaceful. It's the transition that's troublesome. - Isaac Asimov From mueller.os at web.De Fri Sep 11 09:51:39 2009 From: mueller.os at web.De (R. =?ISO-8859-1?Q?M=FCller?=) Date: Fri, 11 Sep 2009 09:51:39 +0200 Subject: [Python-Dev] M.L.S.(Multiple Language Support) in Core-Compiler Message-ID: <1252655499.3392.1.camel@Klapprechner> How to adapt the existing M.L.S. from Compilermessages to Core-Compiler-Words. Where to begin, for implemantation of this feature? From thomas at python.org Fri Sep 11 13:09:55 2009 From: thomas at python.org (Thomas Wouters) Date: Fri, 11 Sep 2009 13:09:55 +0200 Subject: [Python-Dev] pthreads, fork, import, and execvp In-Reply-To: <20090911060658.GA19749@cskk.homeip.net> References: <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> <20090911060658.GA19749@cskk.homeip.net> Message-ID: <9e804ac0909110409o5bc8b018icc40766b2aacf3e3@mail.gmail.com> On Fri, Sep 11, 2009 at 08:06, Cameron Simpson wrote: > On 25Jul2009 10:25, Gregory P. Smith wrote: > | On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters > wrote: > | > So attached (and at http://codereview.appspot.com/96125/show ) is a > | > preliminary fix, correcting the problem with os.fork(), os.forkpty() > and > | > os.fork1(). This doesn't expose a general API for C code to use, for > two > | > reasons: it's not easy, and I need this fix more than I need the API > change > | > :-) (I actually need this fix myself for Python 2.4, but it applies > fairly > | > cleanly.) > | > | This looks good to me. > [...] > > Where's this stand with respect to the upcoming Python 2.6.3? Unless anyone speaks up, I'll submit the fix (without the change in semantics on AIX) to release26-maint early next week, so it would be in 2.6.3. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Fri Sep 11 17:29:22 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 11 Sep 2009 11:29:22 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: <4AA92FE5.5060004@v.loewis.de> References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA92FE5.5060004@v.loewis.de> Message-ID: On Sep 10, 2009, at 12:57 PM, Martin v. L?wis wrote: > Barry Warsaw schrieb: >> I had previously wanted to release Python 2.6.3 over the summer, >> but for >> various personal reasons, the summer was just too insane. I'd like >> to >> reschedule a 2.6.3 release, shooting for final release on 25- >> September. > > I'm travelling that week (as well as the time until then), so I would > prefer to make it a week later. Cool, thanks Martin. Let's make it: 2.6.3rc1 on 29-sep 2.6.3 final on 02-oct That gives us an extra day between rc and final. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From status at bugs.python.org Fri Sep 11 18:07:22 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 11 Sep 2009 18:07:22 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090911160722.1820578637@psf.upfronthosting.co.za> ACTIVITY SUMMARY (09/04/09 - 09/11/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2394 open (+32) / 16312 closed (+15) / 18706 total (+47) Open issues with patches: 950 Average duration of open issues: 660 days. Median duration of open issues: 416 days. Open Issues Breakdown open 2361 (+32) pending 32 ( +0) Issues Created Or Reopened (48) _______________________________ should we include argparse 09/10/09 http://bugs.python.org/issue6247 reopened michael.foord httplib's _read_chunked extremely slow for lots of chunks 09/04/09 CLOSED http://bugs.python.org/issue6838 created cjw296 easy zipfile can't extract file 09/04/09 http://bugs.python.org/issue6839 created NewerCookie genericpath: _splitext returns '.ext', '' instead of '', '.ext' 09/04/09 CLOSED http://bugs.python.org/issue6840 created optimix patch A typo in Doc/library/stdtypes.rst 09/05/09 CLOSED http://bugs.python.org/issue6841 created cheeselee patch program run in interpreter, fails at command line 09/05/09 CLOSED http://bugs.python.org/issue6842 created jvanpraag No documentation for the module argument to warnings.filterwarni 09/05/09 CLOSED http://bugs.python.org/issue6843 reopened exarkun BaseException DeprecationError raises inappropriately 09/05/09 http://bugs.python.org/issue6844 created aisaac Restart support in binary upload for ftplib 09/05/09 http://bugs.python.org/issue6845 created pablomouzo patch bytearray.pop() returns negative ints 09/05/09 CLOSED http://bugs.python.org/issue6846 created hagen patch Exception strings for bytearray say "bytes" 09/05/09 CLOSED http://bugs.python.org/issue6847 created hagen patch curses module build failure (ncurses 5.7) 09/06/09 CLOSED http://bugs.python.org/issue6848 created marketdickinson Tutorial changes 09/06/09 http://bugs.python.org/issue6849 created michael.foord easy decimal.py: format_dict 09/06/09 CLOSED http://bugs.python.org/issue6850 created skrah urllib.urlopen crashes in a thread on Snow Leopard 09/07/09 http://bugs.python.org/issue6851 created jweber Unicode IO not working in cgi applet 09/07/09 CLOSED http://bugs.python.org/issue6852 created loveminix system proxy not used for https (on windows) 09/07/09 http://bugs.python.org/issue6853 created ldeller patch UnicodeDecodeError when retrieving binary data from cgi.FieldSto 09/07/09 http://bugs.python.org/issue6854 created loveminix ihooks support for relative imports 09/07/09 http://bugs.python.org/issue6855 created nascheme allow setting uid and gid when creating tar files 09/07/09 http://bugs.python.org/issue6856 created tarek patch float().__format__() default alignment 09/07/09 http://bugs.python.org/issue6857 created skrah This is a python file, apply syntax highlighting 09/07/09 http://bugs.python.org/issue6858 created gert stdint (eg. uint64_t) for ctypes 09/07/09 CLOSED http://bugs.python.org/issue6859 created haypo Inconsistent naming of custom command in setup.py help output 09/08/09 http://bugs.python.org/issue6860 created herzbube bytearray.__new__ doesn't subclass 09/08/09 CLOSED http://bugs.python.org/issue6861 created kaizhu exec(), locals() and local variable access 09/08/09 CLOSED http://bugs.python.org/issue6862 created ooev Wrong linker command if CXX set to "ccache g++" 09/08/09 http://bugs.python.org/issue6863 created perttikellomaki patch IDLE 2.6.1 locks up on Mac OS 10.6 09/08/09 http://bugs.python.org/issue6864 created dpogg1 Refcount error in pwd module 09/08/09 CLOSED http://bugs.python.org/issue6865 created caglar10ur patch TestLoader.loadTestsFromName does not use suiteClass attribute t 09/08/09 http://bugs.python.org/issue6866 created MarkRoddy patch return value of epoll.register 09/09/09 http://bugs.python.org/issue6867 created shaovie Check errno of epoll_ctrl 09/09/09 http://bugs.python.org/issue6868 created shaovie Embedded python crashed on 4th run, if "ctypes" is used 09/09/09 http://bugs.python.org/issue6869 created warlock sybase module for python3.1 09/09/09 CLOSED http://bugs.python.org/issue6870 created pravinpchavan decimal.py: more format issues 09/09/09 http://bugs.python.org/issue6871 created skrah Support system readline on OS X 10.6 09/09/09 http://bugs.python.org/issue6872 created marketdickinson patch posix_lchown: possible overflow of uid, gid 09/09/09 http://bugs.python.org/issue6873 created boya patch sequence method .count() and .index() shoud be in immutable seq 09/10/09 http://bugs.python.org/issue6874 created s7v7nislands add os.close() suggestion to mkstemp documentation 09/10/09 http://bugs.python.org/issue6875 created vincele patch readline documentation example error 09/10/09 http://bugs.python.org/issue6876 created schwarz enable compilation of readline module on Mac OS X 10.5 and 10.6 09/10/09 http://bugs.python.org/issue6877 created zvezdan patch outdated docstring in tkinter.Canvas.coords 09/10/09 http://bugs.python.org/issue6878 created SilentGhost misstatement in example explanation using raise 09/11/09 http://bugs.python.org/issue6879 created bluebloodpole class needs forward reference 09/11/09 http://bugs.python.org/issue6880 created bluebloodpole incorrect signature in doc for PyByteArray_Resize 09/11/09 http://bugs.python.org/issue6881 created egreen uuid creates zombies 09/11/09 http://bugs.python.org/issue6882 created dhg OptionParser.allow_interspersed_args is undocumented 09/11/09 http://bugs.python.org/issue6883 created skip.montanaro Impossible to include file in sdist that starts with 'build' on 09/11/09 http://bugs.python.org/issue6884 created cjw296 Issues Now Closed (32) ______________________ IDLE - configDialog - new layout for key config 659 days http://bugs.python.org/issue1457 amaury.forgeotdarc patch Cmd module doesn't support readline completion on OSX Leopard 492 days http://bugs.python.org/issue2740 ronaldoussoren Cannot build _multiprocessing, math, mmap and readline of Python 320 days http://bugs.python.org/issue4204 zvezdan patch Mac DMG install missing version.plist required by bundlebuilder. 236 days http://bugs.python.org/issue4937 ronaldoussoren PEP 372: OrderedDict 191 days http://bugs.python.org/issue5397 matthieu.labbe patch Darwin framework libpython3.0.a is not a normal static library 172 days http://bugs.python.org/issue5514 ronaldoussoren [HP-UX] ld: Unrecognized argument: +s -L 100 days http://bugs.python.org/issue6163 tarek patch c_char_p return value returns string, not bytes 92 days http://bugs.python.org/issue6239 haypo patch Native (and default) tarfile support for setup.py sdist in distu 82 days http://bugs.python.org/issue6296 tarek Build Applet.app 45 days http://bugs.python.org/issue6552 ronaldoussoren decimal.py: minor issues && usability 11 days http://bugs.python.org/issue6795 marketdickinson build fails on Snow Leopard 8 days http://bugs.python.org/issue6802 blb patch Mark the compiler package as deprecated 1 days http://bugs.python.org/issue6837 georg.brandl httplib's _read_chunked extremely slow for lots of chunks 0 days http://bugs.python.org/issue6838 cjw296 easy genericpath: _splitext returns '.ext', '' instead of '', '.ext' 0 days http://bugs.python.org/issue6840 benjamin.peterson patch A typo in Doc/library/stdtypes.rst 0 days http://bugs.python.org/issue6841 georg.brandl patch program run in interpreter, fails at command line 0 days http://bugs.python.org/issue6842 marketdickinson No documentation for the module argument to warnings.filterwarni 0 days http://bugs.python.org/issue6843 georg.brandl bytearray.pop() returns negative ints 1 days http://bugs.python.org/issue6846 marketdickinson patch Exception strings for bytearray say "bytes" 1 days http://bugs.python.org/issue6847 marketdickinson patch curses module build failure (ncurses 5.7) 0 days http://bugs.python.org/issue6848 marketdickinson decimal.py: format_dict 1 days http://bugs.python.org/issue6850 marketdickinson Unicode IO not working in cgi applet 1 days http://bugs.python.org/issue6852 marketdickinson stdint (eg. uint64_t) for ctypes 0 days http://bugs.python.org/issue6859 haypo bytearray.__new__ doesn't subclass 0 days http://bugs.python.org/issue6861 benjamin.peterson exec(), locals() and local variable access 0 days http://bugs.python.org/issue6862 amaury.forgeotdarc Refcount error in pwd module 1 days http://bugs.python.org/issue6865 benjamin.peterson patch sybase module for python3.1 0 days http://bugs.python.org/issue6870 loewis Packaging without disturbing an existing installation 2314 days http://bugs.python.org/issue734115 ronaldoussoren patch os.path.splitext don't handle unix hidden file correctly 1674 days http://bugs.python.org/issue1115886 optimix size_t warnings on OSX 10.3 1246 days http://bugs.python.org/issue1467201 ronaldoussoren Universal MacPython 2.5.1 installation fails 813 days http://bugs.python.org/issue1738250 ronaldoussoren Top Issues Most Discussed (10) ______________________________ 14 BaseException DeprecationError raises inappropriately 6 days open http://bugs.python.org/issue6844 12 Integer & Long types: Performance improvement of 1.6x to 2x fo 26 days open http://bugs.python.org/issue6713 9 build fails on Snow Leopard 8 days closed http://bugs.python.org/issue6802 9 [HP-UX] ld: Unrecognized argument: +s -L 100 days closed http://bugs.python.org/issue6163 7 Support system readline on OS X 10.6 2 days open http://bugs.python.org/issue6872 6 posix_lchown: possible overflow of uid, gid 2 days open http://bugs.python.org/issue6873 6 zipfile can't extract file 7 days open http://bugs.python.org/issue6839 6 Mark the compiler package as deprecated 1 days closed http://bugs.python.org/issue6837 6 Add os.link() and os.symlink() and os.path.islink() support for 1061 days open http://bugs.python.org/issue1578269 5 decimal.py: more format issues 2 days open http://bugs.python.org/issue6871 From brett at python.org Fri Sep 11 20:59:43 2009 From: brett at python.org (Brett Cannon) Date: Fri, 11 Sep 2009 11:59:43 -0700 Subject: [Python-Dev] M.L.S.(Multiple Language Support) in Core-Compiler In-Reply-To: <1252655499.3392.1.camel@Klapprechner> References: <1252655499.3392.1.camel@Klapprechner> Message-ID: On Fri, Sep 11, 2009 at 00:51, R. wrote: > How to adapt the existing M.L.S. from Compilermessages to > Core-Compiler-Words. > > Where to begin, for implemantation of this feature? Um, what existing multi-language support? CPython doesn't have any such thing. -Brett From solipsis at pitrou.net Fri Sep 11 21:03:40 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 11 Sep 2009 19:03:40 +0000 (UTC) Subject: [Python-Dev] M.L.S.(Multiple Language Support) in Core-Compiler References: <1252655499.3392.1.camel@Klapprechner> Message-ID: Brett Cannon python.org> writes: > > On Fri, Sep 11, 2009 at 00:51, R. web.de> wrote: > > How to adapt the existing M.L.S. from Compilermessages to > > Core-Compiler-Words. > > > > Where to begin, for implemantation of this feature? > > Um, what existing multi-language support? CPython doesn't have any such thing. Well, I suppose the message was aimed at another mailing-list... From kalite.lab at feka.tk Fri Sep 11 13:44:02 2009 From: kalite.lab at feka.tk (Kalite LAB.) Date: Fri, 11 Sep 2009 14:44:02 +0300 Subject: [Python-Dev] About Python Message-ID: Dear authorized, I have a question about Python programming language can help you glad you do. User as a decimal point "." Instead of character "," we can use that character? For example, 10.7 instead of 10,7 Thanks in advance ... Best Regards... Feyyaz OZEN -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Sat Sep 12 11:51:43 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 12 Sep 2009 10:51:43 +0100 Subject: [Python-Dev] About Python In-Reply-To: <200909120946.n8C9krOj012318@mail2.webfaction.com> References: <200909120946.n8C9krOj012318@mail2.webfaction.com> Message-ID: <4AAB6F2F.5020405@voidspace.org.uk> Kalite LAB. wrote: > Dear authorized, > I have a question about Python programming language can help you glad > you do. User as a decimal point "." Instead of character "," we can > use that character? For example, 10.7 instead of 10,7 This mailing list is for the development of Python, not development with Python. For general questions about how to do things the python-list (comp.lang.python.newsgroup) and python-tutor-list are appropriate places to ask. That aside I *think* that Python is always culture invariant in its syntax (you always use the '.' point for floating point literals in source code) and *also* for how the float type converts strings to numbers (float('10.7') and not float('10,7')). I assume there is a way of turning strings to floats honoring the locale, but off the top of my head I don't know what it is. All the best, Michael Foord > > Thanks in advance ... > > Best Regards... > Feyyaz OZEN > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From jacky830 at gmail.com Sat Sep 12 13:04:18 2009 From: jacky830 at gmail.com (Jacky Lam) Date: Sat, 12 Sep 2009 19:04:18 +0800 Subject: [Python-Dev] Anyone knows how to compile pywin32 for 64bit operating system Message-ID: I would like to compile a 64 bit version of pywin. I am currently using MingGW for that, but I fail to do so. Does anyone know how to compile pywin32 for 64bit operating system? In addition, any examples for that? Thanks. Jacky From fuzzyman at voidspace.org.uk Sat Sep 12 13:19:09 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sat, 12 Sep 2009 12:19:09 +0100 Subject: [Python-Dev] Anyone knows how to compile pywin32 for 64bit operating system In-Reply-To: References: Message-ID: <4AAB83AD.6040103@voidspace.org.uk> Jacky Lam wrote: > I would like to compile a 64 bit version of pywin. > I am currently using MingGW for that, but I fail to do so. > Does anyone know how to compile pywin32 for 64bit operating system? > In addition, any examples for that? > > Hello Jacky, This list is for the development of Python and pywin32 is not a standard part of Python itself. pywin32 has its own mailing list where you are more likely to get an answer to your question: http://mail.python.org/mailman/listinfo/python-win32 All the best, Michael Foord > Thanks. > > Jacky > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From tjgolden at gmail.com Sat Sep 12 15:08:33 2009 From: tjgolden at gmail.com (Tim Golden) Date: Sat, 12 Sep 2009 14:08:33 +0100 Subject: [Python-Dev] Anyone knows how to compile pywin32 for 64bit operating system In-Reply-To: <4AAB83AD.6040103@voidspace.org.uk> References: <4AAB83AD.6040103@voidspace.org.uk> Message-ID: <4AAB9D51.9000808@gmail.com> Michael Foord wrote: > pywin32 has its own mailing list where you are more likely to get an > answer to your question: > > http://mail.python.org/mailman/listinfo/python-win32 And, just for accuracy's sake: that list isn't a list for pywin32 stuff as such -- although it's obviously a good place to ask. It's for Python development under Windows in general. pywin32 has its own trackers shown here: http://sourceforge.net/projects/pywin32/support Nonetheless, I certainly second the recommendation to post to python-win32. TJG From dsdale24 at gmail.com Sat Sep 12 15:57:58 2009 From: dsdale24 at gmail.com (Darren Dale) Date: Sat, 12 Sep 2009 09:57:58 -0400 Subject: [Python-Dev] clarification on PEP 3124 status Message-ID: Could somebody please clarify the status of PEP 3124? At http://ftp.python.org/dev/peps/ , it is listed as "under consideration", but http://ftp.python.org/dev/peps/pep-3124/ says it has been deferred. I was reading through the discussion on the python-3000 mailing list archive, and at one point somebody asked for other examples where generic functions are used in the community. The numpy project has a basic generic function mechanism for numpy's ufuncs (regular functions that operate on arrays), where subclasses of numpy.ndarray can define __array_prepare__ (this method will be added in numpy-1.4) and __array_wrap__ methods, which are sort of analogous to @before and @after in PEP 3124 (ndarray subclasses define an __array_priority__ attribute to decide how to dispatch). The numpy approach is not a general solution and is not as flexible as what is described in the PEP, but it can be used by functions that operate on subclasses that implement matrices, masked arrays, arrays with physical units. I would be very interested in seeing a framework for generic functions in the numpy standard library. I think would be more simple and flexible than what we currently have. Is there still interest/motivation for supporting generic functions in the standard library? Darren From dsdale24 at gmail.com Sat Sep 12 16:50:33 2009 From: dsdale24 at gmail.com (Darren Dale) Date: Sat, 12 Sep 2009 10:50:33 -0400 Subject: [Python-Dev] clarification on PEP 3124 status In-Reply-To: References: Message-ID: On Sat, Sep 12, 2009 at 9:57 AM, Darren Dale wrote: > I would be very interested in seeing a framework for generic functions > in the numpy standard library. Sorry, I meant to say "python standard library" From martin at v.loewis.de Sun Sep 13 08:29:42 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 13 Sep 2009 08:29:42 +0200 Subject: [Python-Dev] clarification on PEP 3124 status In-Reply-To: References: Message-ID: <4AAC9156.8070904@v.loewis.de> > Could somebody please clarify the status of PEP 3124? At > http://ftp.python.org/dev/peps/ , it is listed as "under > consideration", but http://ftp.python.org/dev/peps/pep-3124/ says it > has been deferred. This isn't really contradictory. "under consideration" means "in progress": it has neither been accepted or rejected. If Phillip doesn't respond here, you may want to ask him directly. My impression is that it is deferred because nobody is pursuing it actively (including Phillip Eby). It's common for a PEP to be in that state for several years, "deferred" then is an indication that readers shouldn't expect a resolution in short term. That said: my personal feeling is that this PEP is way too large, and should be broken into seperate pieces of functionality that can be considered independently. There is a lot of stuff in it that isn't strictly necessary to provide the feature listed in the rationale. Regards, Martin From dsdale24 at gmail.com Sun Sep 13 16:14:32 2009 From: dsdale24 at gmail.com (Darren Dale) Date: Sun, 13 Sep 2009 10:14:32 -0400 Subject: [Python-Dev] clarification on PEP 3124 status In-Reply-To: <4AAC9156.8070904@v.loewis.de> References: <4AAC9156.8070904@v.loewis.de> Message-ID: Hi Martin, On Sun, Sep 13, 2009 at 2:29 AM, "Martin v. L?wis" wrote: >> Could somebody please clarify the status of PEP 3124? At >> http://ftp.python.org/dev/peps/ , it is listed as "under >> consideration", but http://ftp.python.org/dev/peps/pep-3124/ says it >> has been deferred. > > This isn't really contradictory. "under consideration" means "in > progress": it has neither been accepted or rejected. > > If Phillip doesn't respond here, you may want to ask him directly. > My impression is that it is deferred because nobody is pursuing it > actively (including Phillip Eby). It's common for a PEP to be in that > state for several years, "deferred" then is an indication that readers > shouldn't expect a resolution in short term. That is why I asked, I wondered if it is being actively considered and pursued, or if it had been deferred or worse abandoned. > That said: my personal feeling is that this PEP is way too large, and > should be broken into seperate pieces of functionality that can be > considered independently. There is a lot of stuff in it that isn't > strictly necessary to provide the feature listed in the rationale. It would be nice to have a suitable foundation upon which more elaborate third party dispatchers could build. The potential generic functions have in a project like numpy are pretty exciting. Darren From p.f.moore at gmail.com Sun Sep 13 16:54:20 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sun, 13 Sep 2009 15:54:20 +0100 Subject: [Python-Dev] clarification on PEP 3124 status In-Reply-To: References: <4AAC9156.8070904@v.loewis.de> Message-ID: <79990c6b0909130754r76364b4es8daeec6592684f6e@mail.gmail.com> 2009/9/13 Darren Dale : >> If Phillip doesn't respond here, you may want to ask him directly. >> My impression is that it is deferred because nobody is pursuing it >> actively (including Phillip Eby). It's common for a PEP to be in that >> state for several years, "deferred" then is an indication that readers >> shouldn't expect a resolution in short term. > > That is why I asked, I wondered if it is being actively considered and > pursued, or if it had been deferred or worse abandoned. > >> That said: my personal feeling is that this PEP is way too large, and >> should be broken into seperate pieces of functionality that can be >> considered independently. There is a lot of stuff in it that isn't >> strictly necessary to provide the feature listed in the rationale. > > It would be nice to have a suitable foundation upon which more > elaborate third party dispatchers could build. The potential generic > functions have in a project like numpy are pretty exciting. You may also be interested in http://bugs.python.org/issue5135 which is a (much) simpler attempt to introduce generic functions into the standard library. Generally, these things get stalled because the core developers don't have sufficient interest in the topic to do anything directly, and the arguments in favour aren't compelling enough to make a difference. Maybe the benefits numpy would get would help the case. Paul. From dsdale24 at gmail.com Sun Sep 13 18:27:01 2009 From: dsdale24 at gmail.com (Darren Dale) Date: Sun, 13 Sep 2009 12:27:01 -0400 Subject: [Python-Dev] clarification on PEP 3124 status In-Reply-To: <79990c6b0909130754r76364b4es8daeec6592684f6e@mail.gmail.com> References: <4AAC9156.8070904@v.loewis.de> <79990c6b0909130754r76364b4es8daeec6592684f6e@mail.gmail.com> Message-ID: Hi Paul, On Sun, Sep 13, 2009 at 10:54 AM, Paul Moore wrote: > 2009/9/13 Darren Dale : >>> If Phillip doesn't respond here, you may want to ask him directly. >>> My impression is that it is deferred because nobody is pursuing it >>> actively (including Phillip Eby). It's common for a PEP to be in that >>> state for several years, "deferred" then is an indication that readers >>> shouldn't expect a resolution in short term. >> >> That is why I asked, I wondered if it is being actively considered and >> pursued, or if it had been deferred or worse abandoned. >> >>> That said: my personal feeling is that this PEP is way too large, and >>> should be broken into seperate pieces of functionality that can be >>> considered independently. There is a lot of stuff in it that isn't >>> strictly necessary to provide the feature listed in the rationale. >> >> It would be nice to have a suitable foundation upon which more >> elaborate third party dispatchers could build. The potential generic >> functions have in a project like numpy are pretty exciting. > > You may also be interested in http://bugs.python.org/issue5135 which > is a (much) simpler attempt to introduce generic functions into the > standard library. Thanks for the pointer. I actually read through the discussion there yesterday. I don't think simplegeneric would be especially useful to numpy. For example, multiplying a numpy.array([1,2,3]) with a quantities.Quantity([1,2,3], 'm/s') should produce a new Quantity regardless of the order in which they are provided to numpy.multiply(). Numpy can handle this particular example now, but the mechanisms are a bit convoluted. > Generally, these things get stalled because the core developers don't > have sufficient interest in the topic to do anything directly, and the > arguments in favour aren't compelling enough to make a difference. > Maybe the benefits numpy would get would help the case. I am a relatively new contributor to the numpy project, contributing bug fixes and features (most of which have been related to -- or could benefit from -- generic functions) to better support subclasses like Quantity. Numpy has different kinds of arrays (ndarrays, array scalars, masked arrays, matrices) and supports many different data types (int8, float32, complex64, etc). The ability to dispatch based on the object type (or combinations thereof), or on combinations of data types, or perhaps on the units of quantities, seem like good examples where predicative dispatch would be useful. I am primarily trying to get up to speed to help with the effort to transition numpy to python-3. Perhaps generic functions could help make the numpy source code more accessible and maintainable, so that maybe someday there would even be interest in including numpy or some subset thereof in the standard library. Anyway, it is helpful to me to see where generic functions stand and how they might develop in the standard library as we work on numpy support for python 3. Regards, Darren From pje at telecommunity.com Sun Sep 13 20:28:58 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 13 Sep 2009 14:28:58 -0400 Subject: [Python-Dev] clarification on PEP 3124 status In-Reply-To: <4AAC9156.8070904@v.loewis.de> References: <4AAC9156.8070904@v.loewis.de> Message-ID: <20090913182855.F36BD3A403D@sparrow.telecommunity.com> At 08:29 AM 9/13/2009 +0200, Martin v. L?wis wrote: > > Could somebody please clarify the status of PEP 3124? At > > http://ftp.python.org/dev/peps/ , it is listed as "under > > consideration", but http://ftp.python.org/dev/peps/pep-3124/ says it > > has been deferred. > >This isn't really contradictory. "under consideration" means "in >progress": it has neither been accepted or rejected. > >If Phillip doesn't respond here, you may want to ask him directly. >My impression is that it is deferred because nobody is pursuing it >actively (including Phillip Eby). It's common for a PEP to be in that >state for several years, "deferred" then is an indication that readers >shouldn't expect a resolution in short term. > >That said: my personal feeling is that this PEP is way too large, and >should be broken into seperate pieces of functionality that can be >considered independently. There is a lot of stuff in it that isn't >strictly necessary to provide the feature listed in the rationale. It's deferred because the PEP needs a rewrite, and it isn't high on my priorities at the moment. It's also unlikely the rewrite will happen before PEAK-Rules reaches a non-alpha release status. (See http://ftp.python.org/dev/peps/pep-3124/#implementation-notes .) From aahz at pythoncraft.com Mon Sep 14 16:16:27 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 14 Sep 2009 07:16:27 -0700 Subject: [Python-Dev] FWD: Windows 7 Compatibility Message-ID: <20090914141627.GA29347@panix.com> Still no-mail on python-dev, forwarding this because it has a direct e-mail contact for Microsoft at the bottom. ----- Forwarded message from Joanna Cobb ----- > From: Joanna Cobb > To: "webmaster at python.org" > Date: Mon, 14 Sep 2009 10:05:34 -0400 > Subject: Windows 7 Compatibility > > Regarding: Windows 7 Compatibility for Python Software Foundation - Application: Python > > I am trying to contact your company regarding the Microsoft Windows 7 Compatibility Program for Python. I have not been able to get in touch with the person responsible for this application in your company and this is why I am reaching out to you through the Support Team. This application has been identified as one of the applications Microsoft would like to see supported on Windows 7 and I have been tasked by Microsoft to help answer your questions about Windows 7 application compatibility and help you get your application through the Windows 7 "Green Light" compatibility process. > > If your application already supports Windows Vista, chances are it will already be compatible with Windows 7 without the need for any code changes. By pledging support for Windows 7 you're application will automatically be listed in the Windows Application Compatibility seen currently by more than 1 million users per month. The registration is extremely simple and just asks a few key questions. > > Here is the link to Microsoft's ISV Application Compatibility page: http://partner.microsoft.com/isvappcompat. When you have a moment, I would encourage you to visit the site and complete the process to pledge support for your application on Windows 7 by October 22nd 2009 when Windows 7 is officially released. > > In addition if you are able to pledge compatibility you'll receive access to a special Windows 7 Partner Marketing Kit that includes a press release with a Microsoft quote, plus customizable marketing templates including; email templates, postcards, web banners, business letter, and copy blocks, all to identify to your customers, or potential customers that your solutions are compatible with Windows 7. > > If you provide me with a phone number where to get in touch with you, I will call you to answer any questions you may have. > > Once you register on the ISV Application Compatibility site, I would appreciate it if you would email me to let me know that you have completed so that I can make a note of it for Microsoft. If you register the application under a different partner or application name please let me know in order to track changes. If there is a new version of the application and there are no plans to support Windows 7 on the older version please register the older version as "No planned Support" on the site as well as the new version with desired Win7 compatibility date. > > Should you have any questions about this email feel free to call me or send an email to my supervisor at v-mafl at microsoft.com. > > Best regards, > > Joanna Cobb > Business Development Representative > N(3) > niche cubed > Office: 800.508.4291 Ext. 316 > joanna.cobb at nichecubed.com > www.nichecubed.com > > Confidentiality note: This e-mail, and any attachment to it, contains privileged and confidential information intended only for the use of the individual(s) or entity named in the e-mail. If the reader of the e-mail is not the intended recipient, or the employee or agent responsible for delivering it to the intended recipient, you are hereby notified that reading it is strictly prohibited. If you have received this e-mail in error, please immediately return it to the sender and delete it from your system. Thank you. > ----- End forwarded message ----- -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "It's 106 miles to Chicago. We have a full tank of gas, a half-pack of cigarettes, it's dark, and we're wearing sunglasses." "Hit it." From peter at hda3.com Mon Sep 14 18:44:12 2009 From: peter at hda3.com (Peter Moody) Date: Mon, 14 Sep 2009 09:44:12 -0700 Subject: [Python-Dev] PEP 3144 review. Message-ID: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Folks, Guido, I believe PEP 3144 is ready for your review. When you get a chance, can you take a look/make a pronouncement? Cheers, /peter From guido at python.org Mon Sep 14 18:54:41 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 14 Sep 2009 09:54:41 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: What's the opinion of the other interested party or parties? I don't want a repeat of the events last time, where we had to pull it at the last time because there hadn't been enough discussion. On Mon, Sep 14, 2009 at 9:44 AM, Peter Moody wrote: > Folks, Guido, > > I believe PEP 3144 is ready for your review. ?When you get a chance, > can you take a look/make a pronouncement? > > Cheers, > /peter > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From andrewm at object-craft.com.au Tue Sep 15 04:11:30 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Tue, 15 Sep 2009 12:11:30 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: <20090915021130.4E57A600115@longblack.object-craft.com.au> >I believe PEP 3144 is ready for your review. When you get a chance, >can you take a look/make a pronouncement? In my experience it is common to leave out the masked octets when referring to an IPv4 network (the octets are assumed to be zero), so I don't agree with this behaviour from the reference implementation: >>> ipaddr.IPv4Network('10/8') IPv4Network('0.0.0.10/8') >>> ipaddr.IPv4Network('192.168/16') Traceback (most recent call last): File "", line 1, in File "/usr/src/py/ipaddr/ipaddr.py", line 1246, in __init__ raise IPv4IpValidationError(addr[0]) ipaddr.IPv4IpValidationError: '192.168' is not a valid IPv4 address I also couldn't see an easy way to get from a network address to the containing network. For example: >>> ipaddr.IPv4Network('192.168.1.1/16') IPv4Network('192.168.1.1/16') This is close: >>> ipaddr.IPv4Network('192.168.1.1/16').network IPv4Address('192.168.0.0') What I want is a method that returns: IPv4Network('192.168.0.0/16') I appreciate these requests are somewhat contradictory (one calls for masked octets to be insignificant, the other calls for them to be significant), but they are both valid use cases in my experience. Apologies if these have already been covered in prior discussion - I've tried to keep up, but I haven't been able to give it the attention it deserves. I also note that many methods in the reference implementation are not discussed in the PEP. While I don't consider this a problem for the PEP, anyone reviewing the module for inclusion in the standard lib needs to consider them. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From peter at hda3.com Tue Sep 15 08:02:26 2009 From: peter at hda3.com (Peter Moody) Date: Mon, 14 Sep 2009 23:02:26 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090915021130.4E57A600115@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> Message-ID: <8517e9350909142302qcb2e13dyf857874649d3b4c6@mail.gmail.com> On Mon, Sep 14, 2009 at 7:11 PM, Andrew McNamara wrote: >>I believe PEP 3144 is ready for your review. ?When you get a chance, >>can you take a look/make a pronouncement? > > In my experience it is common to leave out the masked octets when > referring to an IPv4 network (the octets are assumed to be zero), so I > don't agree with this behaviour from the reference implementation: huh, this appears to be a bug. filing an issue and I'll have this fixed before anything is submitted (http://code.google.com/p/ipaddr-py/issues/detail?id=35) > ? ?>>> ipaddr.IPv4Network('10/8') > ? ?IPv4Network('0.0.0.10/8') > ? ?>>> ipaddr.IPv4Network('192.168/16') > ? ?Traceback (most recent call last): > ? ? ?File "", line 1, in > ? ? ?File "/usr/src/py/ipaddr/ipaddr.py", line 1246, in __init__ > ? ? ? ?raise IPv4IpValidationError(addr[0]) > ? ?ipaddr.IPv4IpValidationError: '192.168' is not a valid IPv4 address > > I also couldn't see an easy way to get from a network address to the > containing network. For example: > > ? ?>>> ipaddr.IPv4Network('192.168.1.1/16') > ? ?IPv4Network('192.168.1.1/16') > > This is close: > > ? ?>>> ipaddr.IPv4Network('192.168.1.1/16').network > ? ?IPv4Address('192.168.0.0') > > What I want is a method that returns: > > ? ?IPv4Network('192.168.0.0/16') I can see about adding this. you can currently do: >>> a = ipaddr.IPv4Network('192.168.1.1/16') >>> '%s/%s' % (a.network, a.prefixlen) 192.168.0.0/16 (I do something very similar to this in address_exclude) > I appreciate these requests are somewhat contradictory (one calls > for masked octets to be insignificant, the other calls for them to be > significant), but they are both valid use cases in my experience. > > Apologies if these have already been covered in prior discussion - > I've tried to keep up, but I haven't been able to give it the attention > it deserves. no need to apologize, all comments welcome. > I also note that many methods in the reference implementation are not > discussed in the PEP. While I don't consider this a problem for the PEP, > anyone reviewing the module for inclusion in the standard lib needs to > consider them. yeah, I didn't exactly want the PEP to be the pydoc of ipaddr, but I did want to explain the features and show how I thought they were important to whatever might be accepted by python. I can certainly go into much more detail in the PEP if that's deemed important. Cheers, /peter > -- > Andrew McNamara, Senior Developer, Object Craft > http://www.object-craft.com.au/ > From lie.1296 at gmail.com Tue Sep 15 08:15:48 2009 From: lie.1296 at gmail.com (Lie Ryan) Date: Tue, 15 Sep 2009 16:15:48 +1000 Subject: [Python-Dev] Numeric alignment issue with PEP 3101 In-Reply-To: <6930F234-6E8E-4E78-AE7F-FFE97AF72D5C@rcn.com> References: <4AA57F42.3050201@trueblade.com> <6930F234-6E8E-4E78-AE7F-FFE97AF72D5C@rcn.com> Message-ID: Raymond Hettinger wrote: > I concur. Numbers are naturally right aligned. > > Isn't numbers are "naturally right aligned" because of the Big Endian notations that most mathematicians currently use. Had we been using Little Endian notation, numbers would be naturally left-aligned, wouldn't they? And I sort of disagree with saying it's naturally left- or right- aligned; as numbers are naturally (decimal) dot-aligned (if you use dot to separate the whole and fractional part of your number). From python at rcn.com Tue Sep 15 08:20:35 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 14 Sep 2009 23:20:35 -0700 Subject: [Python-Dev] Numeric alignment issue with PEP 3101 References: <4AA57F42.3050201@trueblade.com><6930F234-6E8E-4E78-AE7F-FFE97AF72D5C@rcn.com> Message-ID: <24D0205DCEB74A7CB21EF07D2F0B3B65@RaymondLaptop1> > And I sort of disagree with saying it's naturally left- or right- > aligned; as numbers are naturally (decimal) dot-aligned (if you use dot > to separate the whole and fractional part of your number). How about, "naturally aligned by place value". Raymond From ronaldoussoren at mac.com Tue Sep 15 15:53:14 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 15 Sep 2009 15:53:14 +0200 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> Message-ID: On 10 Sep, 2009, at 18:23, Ned Deily wrote: > In article <9D506035-7C2D-4929-A134-E88EEB7B7D9E at python.org>, > Barry Warsaw wrote: > >> On Sep 9, 2009, at 1:29 PM, Ned Deily wrote: >> >>> In article <11A6545D-7204-4F61-B55B-1CC77CB5645E at python.org>, >>> Barry Warsaw wrote: >>>> I still want to release by the 25th, but I'd be willing to move the >>>> rc >>>> to Monday the 21st. We're really just trying to avoid a brown bag >>>> moment, so that should give us enough time to double check the >>>> releases. >>> >>> The recent release of OS X 10.6 (Snow Leopard) has triggered a fair >>> amount of 2.6 bug tracker activity, since 10.6 now includes 2.6 >>> (2.6.1) >>> and a 64-bit version at that. A number of patches have either just >>> been checked-in over the past couple of weeks or are getting some >>> exposure before check-in. Given the timing and the (appropriate) >>> infrequency of 2.6.x releases, I think it would be unfortunate to >>> push >>> 2.6.3 out the door without ensuring that it works well on 10.6. >>> Therefore, I propose that 2.6.3 should have 10.6 compatibility as a >>> "release goal". >>> >>> Without trying to put Ronald on the spot (too much!), it would be a >>> good >>> idea to get his assessment where things stand wrt 2.6 on 10.6 before >>> setting a final release date. >> >> I'm hoping that Python won't have any issues building and running on >> 10.6, but I don't have it yet so I can't personally test it out. >> >> How would you quantify "works well"? Do you have any thoughts on >> tests you'd run other than the standard test suite? If 2.6.3 is >> shown >> to pass its test suite on 10.5.x, is that good enough? Are the >> specific bug fixes necessary for 10.6? > > Running the standard test suite on 10.6 and seeing no regressions > compared to the same suite on 10.5.x seems a reasonable necessary > requirement. We have the resources to do that. Beyond that, as > Ronald > suggests, I think it important to go through the open issues in the > next > couple of days and identify and flag any potential release-blockers > (besides the IDLE problem already mentioned). The IDLE issue is IMHO a release blocker, as is issue 6851. > > One other open issue is 64-bit support in the python.org OS X > installer. > There have been discussions and requests in the past and, with Apple > providing 64-bit out of the box in 10.6, it seems like it's time to > provide something on python.org as well. One option: continue to > provide a 32-bit only installer for ppc and i386 for 10.3.9 and beyond > and add a second installer image with 3-way (ppc, i386, x86_64 but no > ppc64) 32/64 for 10.5 and beyond. Ronald, is that your current > thinking? 64-bit support can wait until after 2.6.3 is released. I need time to work out what's needed go create a good installer (and not just running the current build-installer.py script because that includes to much for a binary that doesn't run on 10.3.9). That won't happen before 2.6.3 is released because I'm too thinly stretched even without working on that. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From solipsis at pitrou.net Tue Sep 15 16:28:13 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Sep 2009 14:28:13 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> Message-ID: Andrew McNamara object-craft.com.au> writes: > > >>> ipaddr.IPv4Network('192.168.1.1/16').network > IPv4Address('192.168.0.0') Er, does this mean that taking the `network` attribute from a network object actually gives an address object (not a network)? It looks horribly misleading to me. From rdmurray at bitdance.com Tue Sep 15 16:36:08 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 15 Sep 2009 10:36:08 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> Message-ID: On Tue, 15 Sep 2009 at 14:28, Antoine Pitrou wrote: > Andrew McNamara object-craft.com.au> writes: >> >> >>> ipaddr.IPv4Network('192.168.1.1/16').network >> IPv4Address('192.168.0.0') > > Er, does this mean that taking the `network` attribute from a network object > actually gives an address object (not a network)? > It looks horribly misleading to me. That was my opinion, too. I've always called that number the 'zero' of the network, but other people said that it is usually called the 'network'. I also find it odd to see a 'network' with an IP address of 192.168.1.1. That looks like a host-address-plus-netmask to me, not a network address. --David From eric.pruitt at gmail.com Tue Sep 15 18:25:35 2009 From: eric.pruitt at gmail.com (Eric Pruitt) Date: Tue, 15 Sep 2009 12:25:35 -0400 Subject: [Python-Dev] PEP 3145 (With Contents) In-Reply-To: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com> References: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com> Message-ID: <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com> I'm bumping this PEP again in hopes of getting some feedback. Thanks, Eric On Tue, Sep 8, 2009 at 23:52, Eric Pruitt wrote: > PEP: 3145 > Title: Asynchronous I/O For subprocess.Popen > Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson > Type: Standards Track > Content-Type: text/plain > Created: 04-Aug-2009 > Python-Version: 3.2 > > Abstract: > > ? ?In its present form, the subprocess.Popen implementation is prone to > ? ?dead-locking and blocking of the parent Python script while waiting on data > ? ?from the child process. > > Motivation: > > ? ?A search for "python asynchronous subprocess" will turn up numerous > ? ?accounts of people wanting to execute a child process and communicate with > ? ?it from time to time reading only the data that is available instead of > ? ?blocking to wait for the program to produce data [1] [2] [3]. ?The current > ? ?behavior of the subprocess module is that when a user sends or receives > ? ?data via the stdin, stderr and stdout file objects, dead locks are common > ? ?and documented [4] [5]. ?While communicate can be used to alleviate some of > ? ?the buffering issues, it will still cause the parent process to block while > ? ?attempting to read data when none is available to be read from the child > ? ?process. > > Rationale: > > ? ?There is a documented need for asynchronous, non-blocking functionality in > ? ?subprocess.Popen [6] [7] [2] [3]. ?Inclusion of the code would improve the > ? ?utility of the Python standard library that can be used on Unix based and > ? ?Windows builds of Python. ?Practically every I/O object in Python has a > ? ?file-like wrapper of some sort. ?Sockets already act as such and for > ? ?strings there is StringIO. ?Popen can be made to act like a file by simply > ? ?using the methods attached the the subprocess.Popen.stderr, stdout and > ? ?stdin file-like objects. ?But when using the read and write methods of > ? ?those options, you do not have the benefit of asynchronous I/O. ?In the > ? ?proposed solution the wrapper wraps the asynchronous methods to mimic a > ? ?file object. > > Reference Implementation: > > ? ?I have been maintaining a Google Code repository that contains all of my > ? ?changes including tests and documentation [9] as well as blog detailing > ? ?the problems I have come across in the development process [10]. > > ? ?I have been working on implementing non-blocking asynchronous I/O in the > ? ?subprocess.Popen module as well as a wrapper class for subprocess.Popen > ? ?that makes it so that an executed process can take the place of a file by > ? ?duplicating all of the methods and attributes that file objects have. > > ? ?There are two base functions that have been added to the subprocess.Popen > ? ?class: Popen.send and Popen._recv, each with two separate implementations, > ? ?one for Windows and one for Unix based systems. ?The Windows > ? ?implementation uses ctypes to access the functions needed to control pipes > ? ?in the kernel 32 DLL in an asynchronous manner. ?On Unix based systems, > ? ?the Python interface for file control serves the same purpose. ?The > ? ?different implementations of Popen.send and Popen._recv have identical > ? ?arguments to make code that uses these functions work across multiple > ? ?platforms. > > ? ?When calling the Popen._recv function, it requires the pipe name be > ? ?passed as an argument so there exists the Popen.recv function that passes > ? ?selects stdout as the pipe for Popen._recv by default. ?Popen.recv_err > ? ?selects stderr as the pipe by default. "Popen.recv" and "Popen.recv_err" > ? ?are much easier to read and understand than "Popen._recv('stdout' ..." and > ? ?"Popen._recv('stderr' ..." respectively. > > ? ?Since the Popen._recv function does not wait on data to be produced > ? ?before returning a value, it may return empty bytes. ?Popen.asyncread > ? ?handles this issue by returning all data read over a given time > ? ?interval. > > ? ?The ProcessIOWrapper class uses the asyncread and asyncwrite functions to > ? ?allow a process to act like a file so that there are no blocking issues > ? ?that can arise from using the stdout and stdin file objects produced from > ? ?a subprocess.Popen call. > > > References: > > ? ?[1] [ python-Feature Requests-1191964 ] asynchronous Subprocess > ? ? ? ?http://mail.python.org/pipermail/python-bugs-list/2006-December/ > ? ? ? ? ?036524.html > > ? ?[2] Daily Life in an Ivory Basement : /feb-07/problems-with-subprocess > ? ? ? ?http://ivory.idyll.org/blog/feb-07/problems-with-subprocess > > ? ?[3] How can I run an external command asynchronously from Python? - Stack > ? ? ? ?Overflow > ? ? ? ?http://stackoverflow.com/questions/636561/how-can-i-run-an-external- > ? ? ? ? ?command-asynchronously-from-python > > ? ?[4] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation > ? ? ? ?http://docs.python.org/library/subprocess.html#subprocess.Popen.wait > > ? ?[5] 18.1. subprocess - Subprocess management - Python v2.6.2 documentation > ? ? ? ?http://docs.python.org/library/subprocess.html#subprocess.Popen.kill > > ? ?[6] Issue 1191964: asynchronous Subprocess - Python tracker > ? ? ? ?http://bugs.python.org/issue1191964 > > ? ?[7] Module to allow Asynchronous subprocess use on Windows and Posix > ? ? ? ?platforms - ActiveState Code > ? ? ? ?http://code.activestate.com/recipes/440554/ > > ? ?[8] subprocess.rst - subprocdev - Project Hosting on Google Code > ? ? ? ?http://code.google.com/p/subprocdev/source/browse/doc/subprocess.rst?spec=svn2c925e935cad0166d5da85e37c742d8e7f609de5&r=2c925e935cad0166d5da85e37c742d8e7f609de5#437 > > ? ?[9] subprocdev - Project Hosting on Google Code > ? ? ? ?http://code.google.com/p/subprocdev > > ? ?[10] Python Subprocess Dev > ? ? ? ? http://subdev.blogspot.com/ > > Copyright: > > ? ?This P.E.P. is licensed under the Open Publication License; > ? ?http://www.opencontent.org/openpub/. From phd at phd.pp.ru Tue Sep 15 18:56:54 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 15 Sep 2009 20:56:54 +0400 Subject: [Python-Dev] PEP 3145 (With Contents) In-Reply-To: <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com> References: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com> <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com> Message-ID: <20090915165654.GA27798@phd.pp.ru> On Tue, Sep 15, 2009 at 12:25:35PM -0400, Eric Pruitt wrote: > A search for "python asynchronous subprocess" will turn up numerous > accounts of people IMHO there is no need to refer to a search. It'd be enough to say "There are many people...". > kernel 32 DLL Why not just name it kernel32.dll? Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From scott+python-dev at scottdial.com Tue Sep 15 19:16:06 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 15 Sep 2009 13:16:06 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> Message-ID: <4AAFCBD6.5090104@scottdial.com> R. David Murray wrote: > On Tue, 15 Sep 2009 at 14:28, Antoine Pitrou wrote: >> Andrew McNamara object-craft.com.au> writes: >>> >>> >>> ipaddr.IPv4Network('192.168.1.1/16').network >>> IPv4Address('192.168.0.0') >> >> Er, does this mean that taking the `network` attribute from a network >> object >> actually gives an address object (not a network)? >> It looks horribly misleading to me. > > That was my opinion, too. I've always called that number the 'zero' > of the network, but other people said that it is usually called the > 'network'. > > I also find it odd to see a 'network' with an IP address of 192.168.1.1. > That looks like a host-address-plus-netmask to me, not a network > address. > It just happened that I needed a tool today to calculate the gateway IP for an interface, and I took it as an opportunity to try out this ipaddr module. I'll attempt to relate my experience below... I have to concur with the opinions above. I was very confused by the following error: >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0") ... ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not a valid address (hint, it's probably a network) Because, it *is* a address of a host on a network. I gave in and said: >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0") But then, I was dumbfounded as to how I could get the gateway IP from this IPNetwork object. It took me a while to figure out that you can iterate over IPNetwork instances: >>> gateway = net[1] I was then confused, because: >>> print(type(gateway)) Which sorta blew my mind.. I fully expected to receive an IPNetwork back from that operation. It is unclear to me why the network information gets chucked by that operation. I foresee having to work around that in real applications by doing something obnoxious like: >>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask)) But since I only actually needed the IP address for the gateway, it fulfilled my needs. In the end, I found the names IPNetwork/IPAddress and their instantiations confusing. ISTM that IPNetwork is overloaded and plays two roles of being an IPNetwork and being an IPAddressWithNetwork. And finally, it's unclear to me why iterating over a IPNetwork would not produce a sequence of IPNetwork(/IPAddressWithNetwork) objects. ISTM that if I started with an IPNetwork object, the API should always return IPNetwork objects. If I want just an IPAddress object, then I can always fetch the "ip" attribute to get that. Perhaps there is some compelling conceptual argument as to why this is not correct, but it seems like the API destroys information needlessly. Just my opinion.. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From mcguire at google.com Tue Sep 15 19:16:29 2009 From: mcguire at google.com (Jake McGuire) Date: Tue, 15 Sep 2009 10:16:29 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum wrote: > What's the opinion of the other interested party or parties? I don't > want a repeat of the events last time, where we had to pull it at the > last time because there hadn't been enough discussion. How many other people are using this library? I think it's hard to give really useful feedback on an API without using it for some non-trivial task, but maybe other people don't have this problem. -jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter at hda3.com Tue Sep 15 19:36:13 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 15 Sep 2009 10:36:13 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> Message-ID: <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire wrote: > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum wrote: >> >> What's the opinion of the other interested party or parties? I don't >> want a repeat of the events last time, where we had to pull it at the >> last time because there hadn't been enough discussion. > > How many other people are using this library? ?I think it's hard to give > really useful feedback on an API without using it for some non-trivial task, > but maybe other people don't have this problem. > -jake 188 (check that, 190) people have downloaded the 2.0 release in the last week (numbers publicly available from the code.google.com). I can't tell you how many (if any) have downloaded it via svn. Cheers, /peter From peter at hda3.com Tue Sep 15 19:56:35 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 15 Sep 2009 10:56:35 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AAFCBD6.5090104@scottdial.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> Message-ID: <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> On Tue, Sep 15, 2009 at 10:16 AM, Scott Dial wrote: > R. David Murray wrote: >> On Tue, 15 Sep 2009 at 14:28, Antoine Pitrou wrote: >>> Andrew McNamara object-craft.com.au> writes: >>>> >>>> ? ?>>> ipaddr.IPv4Network('192.168.1.1/16').network >>>> ? ? IPv4Address('192.168.0.0') >>> >>> Er, does this mean that taking the `network` attribute from a network >>> object >>> actually gives an address object (not a network)? >>> It looks horribly misleading to me. >> >> That was my opinion, too. ?I've always called that number the 'zero' >> of the network, but other people said that it is usually called the >> 'network'. >> >> I also find it odd to see a 'network' with an IP address of 192.168.1.1. >> That looks like a host-address-plus-netmask to me, not a network >> address. >> > > It just happened that I needed a tool today to calculate the gateway IP > for an interface, and I took it as an opportunity to try out this ipaddr > module. I'll attempt to relate my experience below... > > I have to concur with the opinions above. I was very confused by the > following error: > >>>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0") > ... > ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not > a valid address (hint, it's probably a network) are you actually getting '98.223.189.24/255.255.240.0' back? that doesn't look right. > Because, it *is* a address of a host on a network. I gave in and said: > >>>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0") > > But then, I was dumbfounded as to how I could get the gateway IP from > this IPNetwork object. It took me a while to figure out that you can > iterate over IPNetwork instances: > >>>> gateway = net[1] > > I was then confused, because: > >>>> print(type(gateway)) > > > Which sorta blew my mind.. I fully expected to receive an IPNetwork back > from that operation. It is unclear to me why the network information > gets chucked by that operation. I foresee having to work around that in > real applications by doing something obnoxious like: > >>>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask)) > > But since I only actually needed the IP address for the gateway, it > fulfilled my needs. > > In the end, I found the names IPNetwork/IPAddress and their > instantiations confusing. ISTM that IPNetwork is overloaded and plays > two roles of being an IPNetwork and being an IPAddressWithNetwork. And > finally, it's unclear to me why iterating over a IPNetwork would not > produce a sequence of IPNetwork(/IPAddressWithNetwork) objects. What you're describing, at least for the networks, is basically what ipaddr was. It seemed, after much heated discussion, that the community felt that ipaddr.IPv4Network('1.1.1.0/24')[0] was actually an individual address. ie, each of the addresses in 1.1.1.0/24 are individual addresses, rather than networks with /32 prefix lengths. having said that, I understand your confusion; I spent a year+ treating addresses exactly as you've mentioned. It didn't seem as though the majority of the community felt the same, though, so ipaddr evolved. Cheers, /peter > ISTM that if I started with an IPNetwork object, the API should always > return IPNetwork objects. If I want just an IPAddress object, then I can > always fetch the "ip" attribute to get that. Perhaps there is some > compelling conceptual argument as to why this is not correct, but it > seems like the API destroys information needlessly. > > Just my opinion.. > > -- > Scott Dial > scott at scottdial.com > scodial at cs.indiana.edu > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From solipsis at pitrou.net Tue Sep 15 20:07:23 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Sep 2009 18:07:23 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> Message-ID: Scott Dial scottdial.com> writes: > > >>> gateway = net[1] > > I was then confused, because: > > >>> print(type(gateway)) > > > Which sorta blew my mind.. I fully expected to receive an IPNetwork back > from that operation. Speaking as a non-network specialist, it actually looks logical to me to be given an address if I iterate over a network (the same way that, if I iterate on a list, I get individual elements, not 1-element sublists). I don't understand why you think the network mask should be part of an address. An address is just an address, it specifies the location of a host, not the configuration of the network surrounding it. If you look at an IP header, the destination address doesn't specify a netmask. From scott+python-dev at scottdial.com Tue Sep 15 20:16:45 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 15 Sep 2009 14:16:45 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> Message-ID: <4AAFDA0D.5040602@scottdial.com> Peter Moody wrote: > On Tue, Sep 15, 2009 at 10:16 AM, Scott Dial > wrote: >> In the end, I found the names IPNetwork/IPAddress and their >> instantiations confusing. ISTM that IPNetwork is overloaded and plays >> two roles of being an IPNetwork and being an IPAddressWithNetwork. And >> finally, it's unclear to me why iterating over a IPNetwork would not >> produce a sequence of IPNetwork(/IPAddressWithNetwork) objects. > > What you're describing, at least for the networks, is basically what > ipaddr was. It seemed, after much heated discussion, that the > community felt that > > ipaddr.IPv4Network('1.1.1.0/24')[0] > > was actually an individual address. ie, each of the addresses in > 1.1.1.0/24 are individual addresses, rather than networks with /32 > prefix lengths. > For clarity, I am going to call the current design "A": ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Address('1.1.1.0') And what itt sounds like what you are describing as the old behavior is this (design "B"): ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Network('1.1.1.0/32') Which is different than what I was trying to describe. I expected (design "C"): ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Network('1.1.1.0/24') My summarization of these designs would be that "B" is wrong. And that "A" is better than "B", certainly. At least "A" is not saying something untrue. However, "C" would be truthful and retains a superset of information that "A" provides (the "ip" attribute of IPNetwork). In other words, I don't see why obtaining a host address would *not* retain the hostmask from the network it was obtained from. I am not disagreeing with it being an individual address. I am disagreeing that IPNetwork itself already does represent individual addresses (hence my aliasing it with IPAddressWithNetwork). And wrt, the logical return would be another IPAddressWithNetwork retaining the same mask. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From solipsis at pitrou.net Tue Sep 15 20:23:20 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Sep 2009 18:23:20 +0000 (UTC) Subject: [Python-Dev] PEP 3145 (With Contents) References: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com> <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com> Message-ID: Hello, I would like to know if your approach is based on Python 2.x or 3.x. Python 3.x has new API provisions, in the I/O layer, for non-blocking I/O and it would be nice if your work could fit in that framework. > > Popen can be made to act like a file by simply > > using the methods attached the the subprocess.Popen.stderr, stdout and > > stdin file-like objects. But when using the read and write methods of > > those options, you do not have the benefit of asynchronous I/O. I'm not sure I understand the latter sentence. Do you imply that, with your work, read() and write() do allow you to benefit from async I/O? If so, how? Another question: what mechanism does it use internally? Is this mechanism accessible from the outside, so that people can e.g. integrate this inside a third-party event loop (Twisted, asyncore or whatever else)? The PEP should probably outline the additional APIs a bit more precisely and formally than it currently does. > > The Windows > > implementation uses ctypes to access the functions needed to control pipes > > in the kernel 32 DLL in an asynchronous manner. Sorry for the naive question (I'm not a Windows specialist), but does the allusion to "kernel32.dll" mean that it doesn't function on 64-bit variants? Thanks for your work, Regards Antoine. From exarkun at twistedmatrix.com Tue Sep 15 20:24:56 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Tue, 15 Sep 2009 18:24:56 -0000 Subject: [Python-Dev] PEP 3145 (With Contents) In-Reply-To: <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com> References: <171e8a410909082052j6f9b81c2i9030c5f9074125c@mail.gmail.com> <171e8a410909150925w5711d955v1376771e29999493@mail.gmail.com> Message-ID: <20090915182456.12215.1594428165.divmod.xquotient.17@localhost.localdomain> On 04:25 pm, eric.pruitt at gmail.com wrote: >I'm bumping this PEP again in hopes of getting some feedback. > >Thanks, >Eric > >On Tue, Sep 8, 2009 at 23:52, Eric Pruitt >wrote: >>PEP: 3145 >>Title: Asynchronous I/O For subprocess.Popen >>Author: (James) Eric Pruitt, Charles R. McCreary, Josiah Carlson >>Type: Standards Track >>Content-Type: text/plain >>Created: 04-Aug-2009 >>Python-Version: 3.2 >> >>Abstract: >> >>? ?In its present form, the subprocess.Popen implementation is prone >>to >>? ?dead-locking and blocking of the parent Python script while waiting >>on data >>? ?from the child process. >> >>Motivation: >> >>? ?A search for "python asynchronous subprocess" will turn up numerous >>? ?accounts of people wanting to execute a child process and >>communicate with >>? ?it from time to time reading only the data that is available >>instead of >>? ?blocking to wait for the program to produce data [1] [2] [3]. ?The >>current >>? ?behavior of the subprocess module is that when a user sends or >>receives >>? ?data via the stdin, stderr and stdout file objects, dead locks are >>common >>? ?and documented [4] [5]. ?While communicate can be used to alleviate >>some of >>? ?the buffering issues, it will still cause the parent process to >>block while >>? ?attempting to read data when none is available to be read from the >>child >>? ?process. >> >>Rationale: >> >>? ?There is a documented need for asynchronous, non-blocking >>functionality in >>? ?subprocess.Popen [6] [7] [2] [3]. ?Inclusion of the code would >>improve the >>? ?utility of the Python standard library that can be used on Unix >>based and >>? ?Windows builds of Python. ?Practically every I/O object in Python >>has a >>? ?file-like wrapper of some sort. ?Sockets already act as such and >>for >>? ?strings there is StringIO. ?Popen can be made to act like a file by >>simply >>? ?using the methods attached the the subprocess.Popen.stderr, stdout >>and >>? ?stdin file-like objects. ?But when using the read and write methods >>of >>? ?those options, you do not have the benefit of asynchronous I/O. ?In >>the >>? ?proposed solution the wrapper wraps the asynchronous methods to >>mimic a >>? ?file object. >> >>Reference Implementation: >> >>? ?I have been maintaining a Google Code repository that contains all >>of my >>? ?changes including tests and documentation [9] as well as blog >>detailing >>? ?the problems I have come across in the development process [10]. >> >>? ?I have been working on implementing non-blocking asynchronous I/O >>in the >>? ?subprocess.Popen module as well as a wrapper class for >>subprocess.Popen >>? ?that makes it so that an executed process can take the place of a >>file by >>? ?duplicating all of the methods and attributes that file objects >>have. "Non-blocking" and "asynchronous" are actually two different things. From the rest of this PEP, I think only a non-blocking API is being introduced. I haven't looked beyond the PEP, though, so I might be missing something. >>? ?There are two base functions that have been added to the >>subprocess.Popen >>? ?class: Popen.send and Popen._recv, each with two separate >>implementations, >>? ?one for Windows and one for Unix based systems. ?The Windows >>? ?implementation uses ctypes to access the functions needed to >>control pipes >>? ?in the kernel 32 DLL in an asynchronous manner. ?On Unix based >>systems, >>? ?the Python interface for file control serves the same purpose. ?The >>? ?different implementations of Popen.send and Popen._recv have >>identical >>? ?arguments to make code that uses these functions work across >>multiple >>? ?platforms. Why does the method for non-blocking read from a pipe start with an "_"? This is the convention (widely used) for a private API. The name also doesn't suggest that this is the non-blocking version of reading. Similarly, the name "send" doesn't suggest that this is the non-blocking version of writing. >>? ?When calling the Popen._recv function, it requires the pipe name be >>? ?passed as an argument so there exists the Popen.recv function that >>passes >>? ?selects stdout as the pipe for Popen._recv by default. >>?Popen.recv_err >>? ?selects stderr as the pipe by default. "Popen.recv" and >>"Popen.recv_err" >>? ?are much easier to read and understand than "Popen._recv('stdout' >>..." and >>? ?"Popen._recv('stderr' ..." respectively. What about reading from other file descriptors? subprocess.Popen allows arbitrary file descriptors to be used. Is there any provision here for reading and writing non-blocking from or to those? >>? ?Since the Popen._recv function does not wait on data to be produced >>? ?before returning a value, it may return empty bytes. >>?Popen.asyncread >>? ?handles this issue by returning all data read over a given time >>? ?interval. Oh. Popen.asyncread? What's that? This is the first time the PEP mentions it. >>? ?The ProcessIOWrapper class uses the asyncread and asyncwrite >>functions to >>? ?allow a process to act like a file so that there are no blocking >>issues >>? ?that can arise from using the stdout and stdin file objects >>produced from >>? ?a subprocess.Popen call. What's the ProcessIOWrapper class? And what's the asyncwrite function? Again, this is the first time it's mentioned. So, to sum up, I think my main comment is that the PEP seems to be missing a significant portion of the details of what it's actually proposing. I suspect that this information is present in the implementation, which I have not looked at, but it probably belongs in the PEP. Jean-Paul From daniel at stutzbachenterprises.com Tue Sep 15 20:32:53 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 15 Sep 2009 13:32:53 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AAFCBD6.5090104@scottdial.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> Message-ID: On Tue, Sep 15, 2009 at 12:16 PM, Scott Dial > wrote: > >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0") > ... > ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not > a valid address (hint, it's probably a network) > > Because, it *is* a address of a host on a network. I gave in and said: > 10.1.2.3 is the address of a host on a network. 255.255.240.0 is a subnet mask. >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0") > > But then, I was dumbfounded as to how I could get the gateway IP from > this IPNetwork object. It took me a while to figure out that you can > iterate over IPNetwork instances: > Any IP address on an IP network could be the gateway, so there is no reliable way to determine the gateway IP from the network address and subnet mask alone. Technicalities aside, I agree with you that IPNetwork appears to be doing double-duty as an IPNetwork type and an IPAddressWithNetwork type, which I find confusing. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Tue Sep 15 20:33:50 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 15 Sep 2009 14:33:50 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AAFDA0D.5040602@scottdial.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> Message-ID: On Tue, 15 Sep 2009 at 14:16, Scott Dial wrote: > In other words, I don't see why obtaining a host address would *not* > retain the hostmask from the network it was obtained from. I am not > disagreeing with it being an individual address. I am disagreeing that > IPNetwork itself already does represent individual addresses (hence my > aliasing it with IPAddressWithNetwork). And wrt, the logical return > would be another IPAddressWithNetwork retaining the same mask. In other other words, maybe we have three data types: IPv4Address IPv4AddressWithMask IPv4Network Where myAddressWithMask.network would return an IPv4Network object, and an IPv4Network object would always have the zero of the network as its representation: x = IPv4AddressWithMask('192.168.1.1/24') x.network == IPv4Network('192.168.1.0/24') x.network[1] == x In this scheme, IPv4Network('192.168.1.1/24') would raise a ValueError. Although you could probably have what I called IPv4AddressWithMask be called IPv4Address, and have what is now IPv4Address just have netmask and network attributes of None. If this were done, I would expect IPv4Network.network to be either an attribute error or return self. --David From mcguire at google.com Tue Sep 15 20:33:16 2009 From: mcguire at google.com (Jake McGuire) Date: Tue, 15 Sep 2009 11:33:16 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> Message-ID: <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody wrote: > On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire wrote: > > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum > wrote: > >> > >> What's the opinion of the other interested party or parties? I don't > >> want a repeat of the events last time, where we had to pull it at the > >> last time because there hadn't been enough discussion. > > > > How many other people are using this library? I think it's hard to give > > really useful feedback on an API without using it for some non-trivial > task, > > but maybe other people don't have this problem. > > -jake > > 188 (check that, 190) people have downloaded the 2.0 release in the > last week (numbers publicly available from the code.google.com). I > can't tell you how many (if any) have downloaded it via svn. > Downloading and using are not the same thing. -jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter at hda3.com Tue Sep 15 20:36:44 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 15 Sep 2009 11:36:44 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> Message-ID: <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire wrote: > On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody wrote: >> >> On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire wrote: >> > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum >> > wrote: >> >> >> >> What's the opinion of the other interested party or parties? I don't >> >> want a repeat of the events last time, where we had to pull it at the >> >> last time because there hadn't been enough discussion. >> > >> > How many other people are using this library? ?I think it's hard to give >> > really useful feedback on an API without using it for some non-trivial >> > task, >> > but maybe other people don't have this problem. >> > -jake >> >> 188 (check that, 190) people have downloaded the 2.0 release in the >> last week (numbers publicly available from the code.google.com). I >> can't tell you how many (if any) have downloaded it via svn. > > Downloading and using are not the same thing. Correct, but there is a strong positive correlation between the two. If you have a better method for determining what you would consider an appropriate level of usage, I'm all ears. > -jake From stephen at xemacs.org Tue Sep 15 20:49:46 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Sep 2009 03:49:46 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> Message-ID: <87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > Speaking as a non-network specialist, it actually looks logical to > me to be given an address if I iterate over a network (the same way > that, if I iterate on a list, I get individual elements, not > 1-element sublists). But if you iterate over a string you get one character strings. So it's an issue of convenience of representation. From stephen at xemacs.org Tue Sep 15 20:54:35 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Sep 2009 03:54:35 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AAFDA0D.5040602@scottdial.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> Message-ID: <87skeo11jo.fsf@uwakimon.sk.tsukuba.ac.jp> Scott Dial writes: > ipaddr.IPv4Network('1.1.1.0/24')[0] == ipaddr.IPv4Network('1.1.1.0/24') So foo returns True? def foo(): a = ipaddr.IPv4Network('1.1.1.0/24') return a[0] == a That seems ... weird. Maybe you committed a typo? From solipsis at pitrou.net Tue Sep 15 20:43:59 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Sep 2009 18:43:59 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> Message-ID: R. David Murray bitdance.com> writes: > > x = IPv4AddressWithMask('192.168.1.1/24') > x.network == IPv4Network('192.168.1.0/24') > x.network[1] == x I don't think we need an IPAddressWithMask which would just complicate the API without any obvious benefit. We just need a factory function which returns a tuple after parsing: >>> addr, net = parse_address_with_mask('192.168.1.1/24') >>> addr == IPv4Address('192.168.1.1') True >>> net == IPv4Network('192.168.1.0/24') True Regards Antoine. From peter at hda3.com Tue Sep 15 20:46:01 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 15 Sep 2009 11:46:01 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com> On Tue, Sep 15, 2009 at 11:49 AM, Stephen J. Turnbull wrote: > Antoine Pitrou writes: > > ?> Speaking as a non-network specialist, it actually looks logical to > ?> me to be given an address if I iterate over a network (the same way > ?> that, if I iterate on a list, I get individual elements, not > ?> 1-element sublists). > > But if you iterate over a string you get one character strings. ?So > it's an issue of convenience of representation. True, but you don't get a string of equal length back. from the PEP: - Treat network elements as lists (in so far as it's possible). (that should probably say "treat networks as lists") you will no be able, with a string, to do something like >>> 'this is my string'[0][1][2][3] but that's what's being suggested here for networks. >>> ipaddr.IPv4Network('1.1.1.1/24')[0][1][2][3]... > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From rdmurray at bitdance.com Tue Sep 15 21:09:08 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 15 Sep 2009 15:09:08 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> Message-ID: On Tue, 15 Sep 2009 at 18:43, Antoine Pitrou wrote: > R. David Murray bitdance.com> writes: >> >> x = IPv4AddressWithMask('192.168.1.1/24') >> x.network == IPv4Network('192.168.1.0/24') >> x.network[1] == x > > I don't think we need an IPAddressWithMask which would just complicate the API > without any obvious benefit. > We just need a factory function which returns a tuple after parsing: > > >>> addr, net = parse_address_with_mask('192.168.1.1/24') > >>> addr == IPv4Address('192.168.1.1') > True > >>> net == IPv4Network('192.168.1.0/24') > True I would find that acceptable but sub-optimal. Most of my use cases (which involve manipulating router and firewall configuration files) would then start by making a little class named AddressWithNetwork to hold the tuple returned by your parse function, with attributes 'ip' and 'network' and a representation that included the netmask. Other people's use cases would look like addr, _ = parse_address... An IPv4Address with 'network' and 'mask' attributes that could be None would also not complicate the API, IMO, and would handle both of these use cases. --David From scott+python-dev at scottdial.com Tue Sep 15 21:13:27 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 15 Sep 2009 15:13:27 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp> <8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com> Message-ID: <4AAFE757.9010503@scottdial.com> Peter Moody wrote: > but that's what's being suggested here for networks. > >>>> ipaddr.IPv4Network('1.1.1.1/24')[0][1][2][3]... This example here solidifies my support of RDM's suggestion of there being 3 types: IPv4Address IPv4AddressWithNetwork (or as he called it: IPv4AddressWithMask) IPv4Network The primary difference between IPv4Network and IPv4AddressWithNetwork would be that IPv4AddressWithNetwork would not support iteration/indexing. The example above makes no sense logically and my original suggestion unknowingly opened that can of worms. If indexing a IPv4Network returned IPv4AddressWithNetwork, then that would remove that oddity. This would also solve the weirdness that Stephen brought up in another branch of this discussion: Stephen J. Turnbull wrote: > Scott Dial writes: > > ipaddr.IPv4Network('1.1.1.0/24')[0] == > > ipaddr.IPv4Network('1.1.1.0/24') > > So foo returns True? > > def foo(): > a = ipaddr.IPv4Network('1.1.1.0/24') > return a[0] == a > > That seems ... weird. Maybe you committed a typo? The root of the weirdness is that my proposition would appear to make IPv4Network play double-duty. However, it already does! That you can instantiate a IPv4Network object with a non-zero host is a clue, and that it is apparently the only way to input a host with a mask is the second clue. If I have an IP (10.2.3.4) and I know the netmask (say, 255.255.0.0), then how do I get the network that it is on? As it stands, the only way I see to do this is to do: >>> net = ipaddr.IPv4Network("10.2.3.4/255.255.0.0") Which is strange, because I didn't input a *network*, I inputted an *address* on a network. Moreover, it's strange because: >>> net[0] == net.ip False If I needed that identity to hold true, then I have to do: >>> net = ipaddr.IPv4Network("%s/%s" % (net[0], net.netmask) Although it is unclear why a "network" has an "ip" attribute. As far as I am concerned, IPv4Network objects should *only* have a net.network. Hopefully at this point, I have made the case that IPv4Network already is playing double-duty as a IPv4Network and IPv4AddressWithNetwork. And that the API would be well-served by splitting that role and that it would be considerably less confusing. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From solipsis at pitrou.net Tue Sep 15 21:20:15 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Sep 2009 19:20:15 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> Message-ID: R. David Murray bitdance.com> writes: > > I would find that acceptable but sub-optimal. Most of my use cases > (which involve manipulating router and firewall configuration files) would > then start by making a little class named AddressWithNetwork to hold the > tuple returned by your parse function, with attributes 'ip' and 'network' > and a representation that included the netmask. If the AddressWithNetwork class has useful functionality of its own, then why not (although I think that conceptually, this functionality should belong to the Network class instead). If, however, the AddressWithNetwork class is only a recipient for a tuple, then it sounds useless. It's like having an AddressWithPort to hold things like "127.0.0.1:8080". Building little featureless recipients like that isn't really idiomatic Python, IMO. > An IPv4Address with 'network' and 'mask' attributes that could be > None would also not complicate the API, IMO, and would handle both > of these use cases. But it would be confusing and conceptually bizarre, because an address (in usually accepted terminology) doesn't have a network and a mask. I think we should keep the API clean and reasonably logical, rather than try to cover all use cases at the expense of weird shortcuts. Regards Antoine. From srittau at jroger.in-berlin.de Tue Sep 15 21:35:13 2009 From: srittau at jroger.in-berlin.de (Sebastian Rittau) Date: Tue, 15 Sep 2009 21:35:13 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AAFCBD6.5090104@scottdial.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> Message-ID: <20090915193513.GA6606@jroger.in-berlin.de> On Tue, Sep 15, 2009 at 01:16:06PM -0400, Scott Dial wrote: > I have to concur with the opinions above. I was very confused by the > following error: > > >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0") > ... > ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not > a valid address (hint, it's probably a network) > > Because, it *is* a address of a host on a network. To me, 10.1.2.3/255.255.240.0 is not a host address, but specifies a network. I.e., 10.1.2.3/255.255.240.0 == 10.1.0.0/255.255.240.0 == 10.1.35.200/20. > >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0") > > But then, I was dumbfounded as to how I could get the gateway IP from > this IPNetwork object. Well, you can't. There is no way to determine a gateway, without querying the network topology. This is clearly outside the scope of this module. The only two known host addresses of a network are the network address (10.1.0.0 in the example you gave) and the broadcast address (10.1.15.255). > It took me a while to figure out that you can > iterate over IPNetwork instances: > > >>> gateway = net[1] > > I was then confused, because: > > >>> print(type(gateway)) > > > Which sorta blew my mind.. I fully expected to receive an IPNetwork back > from that operation. It is unclear to me why the network information > gets chucked by that operation. This makes perfect sense to me. An IP network consists of a list of IP addresses. Returning /32 networks seems kind of pointless to me. > I foresee having to work around that in > real applications by doing something obnoxious like: > > >>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask)) But a gateway is not an IP address plus hostmask. A gateway is just a single IP address. What is the use of adding a hostmask to the gateway IP address or some other IP address inside the network? - Sebastian From rdmurray at bitdance.com Tue Sep 15 21:48:07 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 15 Sep 2009 15:48:07 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> Message-ID: On Tue, 15 Sep 2009 at 19:20, Antoine Pitrou wrote: > R. David Murray bitdance.com> writes: >> I would find that acceptable but sub-optimal. Most of my use cases >> (which involve manipulating router and firewall configuration files) would >> then start by making a little class named AddressWithNetwork to hold the >> tuple returned by your parse function, with attributes 'ip' and 'network' >> and a representation that included the netmask. > > If the AddressWithNetwork class has useful functionality of its own, then why > not (although I think that conceptually, this functionality should belong to the > Network class instead). But it's not a network. It's an address that is explicitly associated with a network. It's useful functionality is parsing/validating an address+mask, rendering as address+mask, and being able to get the associated IP and network objects from it. It's a small class, but useful, IMO. > If, however, the AddressWithNetwork class is only a recipient for a tuple, then > it sounds useless. It's like having an AddressWithPort to hold things like > "127.0.0.1:8080". Building little featureless recipients like that isn't really > idiomatic Python, IMO. I could see myself doing that, too, though I haven't had a reason to yet :) Although I'd probably call it 'IPv4EndPoint' or some such, since it would be representing one endpoint of a traffic flow... >> An IPv4Address with 'network' and 'mask' attributes that could be >> None would also not complicate the API, IMO, and would handle both >> of these use cases. > > But it would be confusing and conceptually bizarre, because an address (in > usually accepted terminology) doesn't have a network and a mask. > I think we should keep the API clean and reasonably logical, rather than try to > cover all use cases at the expense of weird shortcuts. OK. I'll withdrawal that suggestion. If AddressWithMask doesn't get added I won't be heartbroken; it is easy enough to write in the programs where I need it (at least it would be if the parsing/validation is in the library). However, I do not think that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') as valid. That's just too confusing and error prone. Oh, and if people don't like 'zero' as the name for the IPv4Address at the beginning of the network IP range, how about 'network_ip' or 'network_address' instead of just network, to make it clearer that it is an address? --David From solipsis at pitrou.net Tue Sep 15 21:58:31 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 15 Sep 2009 21:58:31 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> Message-ID: <1253044711.5678.56.camel@localhost> Le mardi 15 septembre 2009 ? 15:48 -0400, R. David Murray a ?crit : > > It's useful functionality is parsing/validating an address+mask, rendering > as address+mask, and being able to get the associated IP and network objects > from it. It's a small class, but useful, IMO. If it's only about parsing and validating, then a tuple works just fine. Getting the associated IP objects is obtained from iterating over the network. Getting the associated network objects I don't understand. There is a single network, not a bunch of them. > However, I do not think > that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') > as valid. That's just too confusing and error prone. Indeed, it should throw some kind of ValueError instead. > Oh, and if people don't like 'zero' as the name for the IPv4Address > at the beginning of the network IP range, how about 'network_ip' or > 'network_address' instead of just network, to make it clearer that it > is an address? How about something more explicit about how it's obtained, like 'lower_bound'? From martin at v.loewis.de Tue Sep 15 22:12:46 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 15 Sep 2009 22:12:46 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: <4AAFF53E.8090704@v.loewis.de> > What's the opinion of the other interested party or parties? I don't > want a repeat of the events last time, where we had to pull it at the > last time because there hadn't been enough discussion. I'm slightly interested, but didn't have the time to review the PEP at all yet. I may be able to do so before the end of the year, but can't promise that. In any case, I don't see a reason to hurry - if the PEP is adopted two or three months before the release of 2.7, it will still be in time, IMO. Regards, Martin From tjreedy at udel.edu Tue Sep 15 22:30:42 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 15 Sep 2009 16:30:42 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> Message-ID: Peter Moody wrote: > On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire wrote: >> Downloading and using are not the same thing. > > Correct, but there is a strong positive correlation between the two. > If you have a better method for determining what you would consider an > appropriate level of usage, I'm all ears. You sneak in a bit of code that sends a message to home base "In use from ";-) Kidding aside, there are usage reports, but those tend to be lower if the code satifies than if not. I think 200 indicates fair interest. Let us hope there is more feedback, pro and con. tjr From fetchinson at googlemail.com Tue Sep 15 22:34:28 2009 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Tue, 15 Sep 2009 13:34:28 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> Message-ID: >>> 188 (check that, 190) people have downloaded the 2.0 release in the >>> last week (numbers publicly available from the code.google.com). I >>> can't tell you how many (if any) have downloaded it via svn. >> >> Downloading and using are not the same thing. > > Correct, but there is a strong positive correlation between the two. > If you have a better method for determining what you would consider an > appropriate level of usage, I'm all ears. A good way of determining the level of usage would be pointing to open source projects that are popular in the python community and which incorporate your module. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From peter at hda3.com Tue Sep 15 22:43:26 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 15 Sep 2009 13:43:26 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> Message-ID: <8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com> On Tue, Sep 15, 2009 at 1:34 PM, Daniel Fetchinson wrote: >>>> 188 (check that, 190) people have downloaded the 2.0 release in the >>>> last week (numbers publicly available from the code.google.com). I >>>> can't tell you how many (if any) have downloaded it via svn. >>> >>> Downloading and using are not the same thing. >> >> Correct, but there is a strong positive correlation between the two. >> If you have a better method for determining what you would consider an >> appropriate level of usage, I'm all ears. > > A good way of determining the level of usage would be pointing to open > source projects that are popular in the python community and which > incorporate your module. well, the 2.0 release is still new. codesearch.google.com shows some projects using the 1.x release; hopefully some of those 200 downloaders will put up some publicly indexable python code at some point. > Cheers, > Daniel > > -- > Psss, psss, put it down! - http://www.cafepress.com/putitdown > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From rdmurray at bitdance.com Wed Sep 16 01:11:34 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 15 Sep 2009 19:11:34 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <1253044711.5678.56.camel@localhost> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> Message-ID: On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote: > Le mardi 15 septembre 2009 ?? 15:48 -0400, R. David Murray a ??crit : >> >> It's useful functionality is parsing/validating an address+mask, rendering >> as address+mask, and being able to get the associated IP and network objects >> from it. It's a small class, but useful, IMO. > > If it's only about parsing and validating, then a tuple works just fine. > Getting the associated IP objects is obtained from iterating over the > network. > Getting the associated network objects I don't understand. There is a > single network, not a bunch of them. The plural was for the two attributes. I guess I just like accessing things by name better than accessing them by indexes. I suppose a NamedTuple would work. Anyway, like I said I'm not attached to that class. >> However, I do not think >> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') >> as valid. That's just too confusing and error prone. > > Indeed, it should throw some kind of ValueError instead. Peter, what do you think? >> Oh, and if people don't like 'zero' as the name for the IPv4Address >> at the beginning of the network IP range, how about 'network_ip' or >> 'network_address' instead of just network, to make it clearer that it >> is an address? > > How about something more explicit about how it's obtained, like > 'lower_bound'? Well, when this came up earlier 'first' and 'last' were suggested to replace 'network' and 'broadcast', but that proposal didn't get much traction, the argument being that in normal technical parlance the first IP in the netblock is (apparently, this isn't my personal experience(*)) called the network, and the broadcast is almost invariably the last IP in the netblock (and that does agree with my experience). --David (*) in my experience it is called the "network number", which I gather derives from the pre-CIDR days. "network identifier" also appears to be used. In my experience "network" refers to something that has a netmask, although the netmask may be implicit in its class in the classful routing sense, or through other context. Consider this article, for example: http://compnetworking.about.com/od/workingwithipaddresses/l/aa042900a.htm From scott+python-dev at scottdial.com Wed Sep 16 01:34:09 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 15 Sep 2009 19:34:09 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> Message-ID: <4AB02471.7000701@scottdial.com> R. David Murray wrote: > On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote: >> Le mardi 15 septembre 2009 ? 15:48 -0400, R. David Murray a ?crit : >>> However, I do not think >>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') >>> as valid. That's just too confusing and error prone. >> >> Indeed, it should throw some kind of ValueError instead. > > Peter, what do you think? This change would have to be dependent on Antoine's suggestion of: >>> addr, net = parse_address_with_mask('192.168.1.1/24') >>> addr == IPv4Address('192.168.1.1') True >>> net == IPv4Network('192.168.1.0/24') True Otherwise, I am ok with this change too. It resloves the weird duality of IPv4Network. And as RDM says, anyone who wants the IPv4AddressWithNetwork functionality can just role it themselves. At the moment of iteration, you have access to what network it is and can build your own IPv4AddressWithNetwork objects. net = IPv4Network('10.0.0.0/24') netaddrs = [] for addr in net: netaddrs.append(IPv4AddressWithNetwork(addr, net)) I guess I am ok with this. It seems sub-optimal (why not just return a IPv4AddressWithNetwork to begin with?) but I suppose it is no less efficient since the same objects would be constructed. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From scott+python-dev at scottdial.com Wed Sep 16 01:38:40 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Tue, 15 Sep 2009 19:38:40 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090915193513.GA6606@jroger.in-berlin.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <20090915193513.GA6606@jroger.in-berlin.de> Message-ID: <4AB02580.4060502@scottdial.com> Sebastian Rittau wrote: > On Tue, Sep 15, 2009 at 01:16:06PM -0400, Scott Dial wrote: >>>>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0") >> But then, I was dumbfounded as to how I could get the gateway IP from >> this IPNetwork object. > > Well, you can't. There is no way to determine a gateway, without querying > the network topology. This is clearly outside the scope of this module. > The only two known host addresses of a network are the network address > (10.1.0.0 in the example you gave) and the broadcast address (10.1.15.255). I was not writing a general utility for the world to use. On *my* network it is always the [1]th address used for the gateway. But, I agree in general that is not true. >> Which sorta blew my mind.. I fully expected to receive an IPNetwork back >> from that operation. It is unclear to me why the network information >> gets chucked by that operation. > > This makes perfect sense to me. An IP network consists of a list of IP > addresses. Returning /32 networks seems kind of pointless to me. As I clarified later, I expected it to still be a /24 network, not a /32. /32 makes no sense. > But a gateway is not an IP address plus hostmask. A gateway is just a single > IP address. What is the use of adding a hostmask to the gateway IP address > or some other IP address inside the network? My argument was from a destruction of information standpoint. Why is the network chucked away? Wouldn't it be handy to be able to inspected addr.network to see what network it was on? Perhaps because you are building routing rules and need to actually know that. As it is, you will just have to build your own IPAddressWithNetwork objects. Hopefully I have covered all of you inquiries, but I feel like that a lot of this is covered by many other replies in this thread. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From peter at hda3.com Wed Sep 16 02:05:46 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 15 Sep 2009 17:05:46 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB02471.7000701@scottdial.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> Message-ID: <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> On Tue, Sep 15, 2009 at 4:34 PM, Scott Dial wrote: > R. David Murray wrote: >> On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote: >>> Le mardi 15 septembre 2009 ? 15:48 -0400, R. David Murray a ?crit : >>>> However, I do not think >>>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') >>>> as valid. ?That's just too confusing and error prone. >>> >>> Indeed, it should throw some kind of ValueError instead. >> >> Peter, what do you think? I disagree. It seems overly pedantic to throw a valueerror on this. IPy does (used to do?) this and it's one of the primary reasons I wrote ipaddr. > This change would have to be dependent on Antoine's suggestion of: > > ? ?>>> addr, net = parse_address_with_mask('192.168.1.1/24') > ? ?>>> addr == IPv4Address('192.168.1.1') > ? ?True > ? ?>>> net == IPv4Network('192.168.1.0/24') > ? ?True > > Otherwise, I am ok with this change too. It resloves the weird duality > of IPv4Network. And as RDM says, anyone who wants the > IPv4AddressWithNetwork functionality can just role it themselves. At the > moment of iteration, you have access to what network it is and can build > your own IPv4AddressWithNetwork objects. now, I understand that as the author, I have certain biases, but I'm not having the same difficulty with this duality as others. I really don't even see it as a duality. It seems as though your issue could more easily be solved by having an iterator in BaseNet which returns a new network object with the original prefix length. eg: for addr in ipaddr.IPv4Network('1.1.1.0/24').iterwithnetmask(): # 1.1.1.0/24, 1.1.1.1/24, 1.1.1.2/24... (or better yet, .iterwithprefix) this can be done much more cleanly/easily than adding two new (redundant in my eyes) classes and much less pedantically than raising exceptions w/ IPv?Network() constructor received something other than base network address. Is this sufficient? If not, perhaps someone could help me better understand the confusion over IPv?Network objects. Cheers, /peter > net = IPv4Network('10.0.0.0/24') > netaddrs = [] > for addr in net: > ? ?netaddrs.append(IPv4AddressWithNetwork(addr, net)) > > I guess I am ok with this. It seems sub-optimal (why not just return a > IPv4AddressWithNetwork to begin with?) but I suppose it is no less > efficient since the same objects would be constructed. > > -- > Scott Dial > scott at scottdial.com > scodial at cs.indiana.edu > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From solipsis at pitrou.net Wed Sep 16 02:20:41 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 16 Sep 2009 00:20:41 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > >>>> However, I do not think > >>>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') > >>>> as valid. ?That's just too confusing and error prone. > >>> > >>> Indeed, it should throw some kind of ValueError instead. > >> > >> Peter, what do you think? > > I disagree. It seems overly pedantic to throw a valueerror on this. > IPy does (used to do?) this and it's one of the primary reasons I > wrote ipaddr. Python is not PHP and does not try to be overly "smart" or tolerant when faced with bizarrely-formatted input. I don't see any valid reason for entering a network as "192.168.1.1/24" rather than the canonical "192.168.1.0/24". The former might indicate a typing error or a mental slip, so let's be helpful and signal it to the user. > for addr in ipaddr.IPv4Network('1.1.1.0/24').iterwithnetmask(): > # 1.1.1.0/24, 1.1.1.1/24, 1.1.1.2/24... I don't have any strong feelings about it, although I don't really think it is needed, and a plethoric API is not a good thing (neither clarity-wise nor maintenance-wise). > this can be done much more cleanly/easily than adding two new > (redundant in my eyes) classes and much less pedantically than raising > exceptions w/ IPv?Network() constructor received something other than > base network address. I don't think anybody suggested adding two classes. As for the "pedantry" of thoroughly validating inputs, see above. Regards Antoine. From solipsis at pitrou.net Wed Sep 16 02:33:56 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 16 Sep 2009 00:33:56 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> Message-ID: Antoine Pitrou pitrou.net> writes: > > I don't see any valid reason for entering a network as "192.168.1.1/24" rather > than the canonical "192.168.1.0/24". The former might indicate a typing error > or > a mental slip, so let's be helpful and signal it to the user. Or perhaps there can be an optional "strict=True" (or "strict=False") argument to the constructor / parsing function. Regards Antoine. From eric at trueblade.com Wed Sep 16 03:02:01 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 15 Sep 2009 21:02:01 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> Message-ID: <4AB03909.3090608@trueblade.com> Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >>>>>> However, I do not think >>>>>> that the proposed API should accept, eg, IPv4Network('192.168.1.1/24') >>>>>> as valid. That's just too confusing and error prone. >>>>> Indeed, it should throw some kind of ValueError instead. >>>> Peter, what do you think? >> I disagree. It seems overly pedantic to throw a valueerror on this. >> IPy does (used to do?) this and it's one of the primary reasons I >> wrote ipaddr. > > Python is not PHP and does not try to be overly "smart" or tolerant when faced > with bizarrely-formatted input. > > I don't see any valid reason for entering a network as "192.168.1.1/24" rather > than the canonical "192.168.1.0/24". The former might indicate a typing error or > a mental slip, so let's be helpful and signal it to the user. I completely agree. I don't know of any situation where I'd want a network of "192.168.1.1/24" to be anything other than an error. Eric. From eric at trueblade.com Wed Sep 16 03:02:50 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 15 Sep 2009 21:02:50 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> Message-ID: <4AB0393A.5020906@trueblade.com> Antoine Pitrou wrote: > Antoine Pitrou pitrou.net> writes: >> I don't see any valid reason for entering a network as "192.168.1.1/24" rather >> than the canonical "192.168.1.0/24". The former might indicate a typing error >> or >> a mental slip, so let's be helpful and signal it to the user. > > Or perhaps there can be an optional "strict=True" (or "strict=False") argument > to the constructor / parsing function. If so, I hope that we'd default to strict=True. From peter at hda3.com Wed Sep 16 03:05:26 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 15 Sep 2009 18:05:26 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB03909.3090608@trueblade.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <4AB03909.3090608@trueblade.com> Message-ID: <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith wrote: > Antoine Pitrou wrote: >> >> Peter Moody hda3.com> writes: >>>>>>> >>>>>>> However, I do not think >>>>>>> that the proposed API should accept, eg, >>>>>>> IPv4Network('192.168.1.1/24') >>>>>>> as valid. ?That's just too confusing and error prone. >>>>>> >>>>>> Indeed, it should throw some kind of ValueError instead. >>>>> >>>>> Peter, what do you think? >>> >>> I disagree. It seems overly pedantic to throw a valueerror on this. >>> IPy does (used to do?) this and it's one of the primary reasons I >>> wrote ipaddr. >> >> Python is not PHP and does not try to be overly "smart" or tolerant when >> faced >> with bizarrely-formatted input. >> >> I don't see any valid reason for entering a network as "192.168.1.1/24" >> rather >> than the canonical "192.168.1.0/24". The former might indicate a typing >> error or >> a mental slip, so let's be helpful and signal it to the user. > > I completely agree. I don't know of any situation where I'd want a network > of "192.168.1.1/24" to be anything other than an error. when you're entering the address of your nic. > Eric. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From peter at hda3.com Wed Sep 16 03:08:05 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 15 Sep 2009 18:08:05 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> Message-ID: <8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com> On Tue, Sep 15, 2009 at 5:33 PM, Antoine Pitrou wrote: > Antoine Pitrou pitrou.net> writes: >> >> I don't see any valid reason for entering a network as "192.168.1.1/24" rather >> than the canonical "192.168.1.0/24". The former might indicate a typing error >> or >> a mental slip, so let's be helpful and signal it to the user. > > Or perhaps there can be an optional "strict=True" (or "strict=False") argument > to the constructor / parsing function. I can live w/ a default of strict=False. there are plenty of cases where it's not an error and easy enough ways to check, if the developer is concerned, with or without an option. eg if addr.ip != addr.network: Cheers, /peter > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From amk at amk.ca Wed Sep 16 03:48:10 2009 From: amk at amk.ca (A.M. Kuchling) Date: Tue, 15 Sep 2009 21:48:10 -0400 Subject: [Python-Dev] Python Language Summit #2 in February Message-ID: <20090916014810.GA318@andrew-kuchlings-macbook.local> PyCon 2010 will be February 19-21 2010 in Atlanta, Georgia (US). Van Lindberg, PyCon chair, has approved having another Python Language Summit on Thursday, February 18 2010. The web page for it is The Python Language Summit is an invitation-only event for developers of Python implementations and the standard library, to discuss issues of common concern to all implementors. It will mostly be organized similarly to last year's event. I'm going to drop the initial open discussion, which wasn't very useful, and we'll just have three 1.5-hour discussion sessions. The fourth time slot will be left open for chatting, development, or whatever the attendees want to do. We therefore need to decide what those three sessions should be about. Please discuss on python-dev and hopefully we can arrive at some consensus on topics of reasonably wide current interest. (See http://us.pycon.org/2009/about/summits/language/ for a reminder of last year's topics.) Feel free to raise discussions on other lists such as jython-dev, catalog-sig, or wherever, but please summarize the results here on python-dev; I won't see discussions on other lists. Like last time, invitations will be sent to the committers for Python, Jython, IronPython, PyPy, Pynie, plus a few extra people and projects. If you want to suggest another project or person who should be invited, please send me a private e-mail. Andrew M. Kuchling amk at amk.ca From andrewm at object-craft.com.au Wed Sep 16 04:32:31 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Wed, 16 Sep 2009 12:32:31 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com> Message-ID: <20090916023231.5A00D5AC0D6@longblack.object-craft.com.au> >>> I don't see any valid reason for entering a network as "192.168.1.1/24" >>> rather than the canonical "192.168.1.0/24". The former might indicate a >>> typing error or a mental slip, so let's be helpful and signal it to the >>> user. >> >> Or perhaps there can be an optional "strict=True" (or "strict=False") >> argument to the constructor / parsing function. > >I can live w/ a default of strict=False. there are plenty of cases >where it's not an error and easy enough ways to check, if the >developer is concerned, with or without an option. eg if addr.ip != >addr.network: I agree - there are definitely times when it is not an error, but I don't like the idea of a "strict" flag. I've done a bit of everything - router configs with a national ISP, scripts to manage host configuration, user interfaces, you name it. The way I see it, we need: * Two address classes that describe a single IP end-point - "Address" with no mask and "AddressWithMask" (the later being the current Network class, minus the container-like behaviour). * A "Network" container-like class. Same as the current Network class, but addresses with masked bits would be considered an error. This is along the lines that RDM was suggesting, except that we remove the container behaviour from AddressWithMask. Additionally: * The .network attribute on an AddressWithMask would return a Network instance. * An Address class would not have a .network attribute * Network.__contains__() would accept Network, Address and AddressWithMask. Only Network implements __contains__ - an AddressWithMask can't contain another address, although it's .network can. * Maybe an Address should compare equal with an AddressWithMask if the address is identical and the mask is equivalent to /32? Personally, I don't see a strong use-case for the list-like indexing and iteration behaviour - I think it's enough to implement some basic container behaviour, but I won't object to the iterator and indexing, provided they don't distort the rest of the design (which I fear they are doing now). Iterating or indexing a network should return Address or AddressWithMask instances - if the later, the mask should match the parent network's mask. I'm not particularly wedded to the name "AddressWithMask" - maybe it could be NetworkAddress or MaskedAddress or ? -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From jonrascal at gmail.com Wed Sep 16 04:49:38 2009 From: jonrascal at gmail.com (rascal) Date: Tue, 15 Sep 2009 19:49:38 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <8517e9350909151808x173c81a3p8ff2b5d30c06f021@mail.gmail.com> Message-ID: I'd like to see more clear direction for this. Is it a string manipulation library, or an IP address library? There are ONLY TWO types of IP representations: 1. Network = 32 bit number AND 32 bit mask. That is the binary AND. The mask needs to be preserved, but not the network portion. Any library should return the ANDed result. Whether the number is 192.186.1.1/24 or 192.168.1.0/24, the result after 'and' is the same. Add a strict=false in case people need it for validation, better yet, add a validation helper function, but always return the proper form. 2. A host address is an IP address with no network info. No mask information, ever. What about Broadcast / Network IDs? The "Network-ID or Network Number" and "Broadcast" are meaningless without context. 1. A network number by itself it's just another address. Its use depends on context. When routing, it needs a mask. > route 10.0.0.0 255.0.0.0 192.168.1.1 > When writing firewall ACLs, it is a host. > access-list aclname permit ip any 192.168.1.2 255.255.255.254 > Since both are valid, preference shouldn't be given. Instead add a helper for firstip. 2. Same story with broadcast. Use depends on context. Add the lastip helper. What does iterating over a network mean? Iterating over the set of addresses in a network must return host addresses. If it returns a network, then the address should be ANDed, but that would return the same number for every iteration. Only returning hosts makes sense. We need to set a solid line between IP String manipulation and IP manipulation. If you want an IP with a mask that isn't a network, then you're not doing IP address manipulation, you're doing string manipulation. There are many use cases for it, but it's a separate consideration to pure IP manipulation. If string manipulation is desirable, then add another standard library called ipstring (or a separate class if you want it all jumbled in the same import). This way real network computing can be done without weird quirks of representation. R. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Wed Sep 16 07:30:03 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Sep 2009 14:30:03 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AAFE757.9010503@scottdial.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp> <8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com> <4AAFE757.9010503@scottdial.com> Message-ID: <87pr9r1mp0.fsf@uwakimon.sk.tsukuba.ac.jp> I haven't looked closedly at the reference implementation, and my experience is small. So take this as "POLA to the naive new user of the API" discussion. Scott Dial writes: > This example here solidifies my support of RDM's suggestion of there > being 3 types: > > IPv4Address > IPv4AddressWithNetwork (or as he called it: IPv4AddressWithMask) > IPv4Network IPv4AddressWithMask seems silly to me, implementation-wise. IP networks are conceptually globally unique (and even for unroutable networks they're locally unique), and it seems to me that I want that to be true internally (for example, I once implemented an IP network type that maintains a hostname table). If you need *the* network, you'll have to look it up in a mask_to_network table (at least internally). Rather, don't you want to just give IPv4Address an attribute 'network'? This could then be filled in by the indexing method on IPv4Network. Then deriving a mask from a network is trivial (networks need to know their masks anyway), like so: IPv4Address('1.1.1.1').network.mask giving you the mask if .network is known, else an attribute error if .network is None. But deriving *the* corresponding network from a mask will require keeping a table of networks around. > that it is apparently the only way to input a host with a mask is the > second clue. I don't see why you would want to do that. Wouldn't you rather IPv4Address('1.1.1.1').network_from_mask('255.255.0.0') and IPv4Address('1.1.1.1').set_network_from_mask('255.255.0.0') > If I have an IP (10.2.3.4) and I know the netmask (say, 255.255.0.0), > then how do I get the network that it is on? As it stands, the only way > I see to do this is to do: > > >>> net = ipaddr.IPv4Network("10.2.3.4/255.255.0.0") > > Which is strange, because I didn't input a *network*, I inputted an > *address* on a network. No, you inputted a pair of dotted quads. Dotted quads do triple duty representing host addresses, network addresses, and network masks. Since construction is a conversion anyway, I don't have any problem with constructing a network from that pair so that your test: > >>> net[0] == net.ip returns true. Ie, the implementation can do class IPv4Network(...): def __init__(self, dotted_quad_pair_string): ip, mask = pair_string_to_int_pair(dotted_quad_pair_string) self.mask = mask self.ip = ip & mask > IPv4Network objects should *only* have a net.network. That's a naming issue, though, isn't it? I don't see a use for net.ip != net.network, so you wouldn't have both of them. If you *do* have both for some reason, your test net[0] == net.ip seems to have the semantics "How was I constructed?" and I don't see why you can assert that should be True. > Hopefully at this point, I have made the case that IPv4Network already > is playing double-duty as a IPv4Network and IPv4AddressWithNetwork. At this point, no, you've only made the point that you would try to use it that way. Maybe that's natural for you (and you're obviously more expert than I), but it's not for me. I haven't experienced a need for the operations you're trying to do, but if I needed them, I'd be looking for APIs on IPv4Address and IPv4Network like those I described above, not an IPv4AddressWithNetwork type (ie, I would expect that IPv4Address is implemented as IPv4AddressWithNetwork). From stephen at xemacs.org Wed Sep 16 07:48:58 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 16 Sep 2009 14:48:58 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <8517e9350909151056l6e12df92k41b57d13d646b6ab@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> Message-ID: <87ocpb1lth.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > > An IPv4Address with 'network' and 'mask' attributes that could be > > None would also not complicate the API, IMO, and would handle both > > of these use cases. > > But it would be confusing and conceptually bizarre, because an address (in > usually accepted terminology) doesn't have a network and a mask. It doesn't need a mask attribute because the network knows that. OTOH, it would be useful for an address to be able to cache the network it is a member of, because the object that represents a particular network should be unique. That means a lookup in a table, and possibly constructing a new IPv4Network object to represent the anwser if false, if you are given a netmask and asked to deduce *the* network. And addresses do, implicitly, have a network. Otherwise the're useless!<0.5 wink> From martin at v.loewis.de Wed Sep 16 10:00:21 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 16 Sep 2009 10:00:21 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB0393A.5020906@trueblade.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <4AB0393A.5020906@trueblade.com> Message-ID: <4AB09B15.80705@v.loewis.de> >>> I don't see any valid reason for entering a network as >>> "192.168.1.1/24" rather >>> than the canonical "192.168.1.0/24". The former might indicate a >>> typing error >>> or >>> a mental slip, so let's be helpful and signal it to the user. >> >> Or perhaps there can be an optional "strict=True" (or "strict=False") >> argument >> to the constructor / parsing function. > > If so, I hope that we'd default to strict=True. I don't see the point of enforcing strictness here. There are good use cases: if you know your address is 82.94.164.162, how do you compute what you should spell for 82.94.164.162/27? It's very easy to make a mistake here, and doing the computation in your head is really not something that should be necessary, given that a computer can get it correct easily. Regards, Martin From python-3000 at udmvt.ru Wed Sep 16 10:26:30 2009 From: python-3000 at udmvt.ru (python-3000 at udmvt.ru) Date: Wed, 16 Sep 2009 13:26:30 +0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090915193513.GA6606@jroger.in-berlin.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <20090915193513.GA6606@jroger.in-berlin.de> Message-ID: <20090916082630.GF12810@ruber.office.udmvt.ru> On Tue, Sep 15, 2009 at 09:35:13PM +0200, Sebastian Rittau wrote: > On Tue, Sep 15, 2009 at 01:16:06PM -0400, Scott Dial wrote: > > > I have to concur with the opinions above. I was very confused by the > > following error: > > > > >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0") > > ... > > ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not > > a valid address (hint, it's probably a network) > > > > Because, it *is* a address of a host on a network. > > To me, 10.1.2.3/255.255.240.0 is not a host address, but specifies a > network. I.e., 10.1.2.3/255.255.240.0 == 10.1.0.0/255.255.240.0 == > 10.1.35.200/20. We shouldn't invent another bicycle here. It is a host address, that ALSO specifies a network mask, that is not /32. Mask /32 just may be omitted when written, but assumed, when stored by computer. What you talk about is "address with mask". It is just IP address plus mask information. IP address and mask just written together, one after another. You MAY use mask to extract network part from an address, and have a "set" of IP addresses, or range of IP addresses, or "network" as you say. But you are not forced to do that with "MUST" statement, because you may also use mask and IP address to compute widely used default value for a "broadcast" address, or you may extract "host" part from the whole IP address. There is no semantical difference between IPAddress for hosts and IPAddress for networks, look at 'inet' type of PostgreSQL. Python shouldn't have separate types for that - it is stupid, people will say 'python programmers are lamers'. If you need only one IP address - you write AA.BB.CC.DD[/32] or /255.255.255.255 and the mask value may be omitted here. If you need to specify a larger range of IP addresses, you specify another mask value, that is simple. > > > >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0") > > > > But then, I was dumbfounded as to how I could get the gateway IP from > > this IPNetwork object. > > Well, you can't. There is no way to determine a gateway, without querying > the network topology. This is clearly outside the scope of this module. > The only two known host addresses of a network are the network address > (10.1.0.0 in the example you gave) and the broadcast address (10.1.15.255). > > > It took me a while to figure out that you can > > iterate over IPNetwork instances: > > > > >>> gateway = net[1] > > > > I was then confused, because: > > > > >>> print(type(gateway)) > > > > > > Which sorta blew my mind.. I fully expected to receive an IPNetwork back > > from that operation. It is unclear to me why the network information > > gets chucked by that operation. > > This makes perfect sense to me. An IP network consists of a list of IP > addresses. Returning /32 networks seems kind of pointless to me. Why? What differ IP Host address from IP Network address with mask /32? Don't say "they have different python classes". Well, I know, that an element is not a set containing only that element. But that is about mathematical objects. What difference between /32 IP Network and IP Host for Python? Why IPNetwork can't be used where IPAddress is used? When you only need IP address, you only use address part and ignore mask. Why IPAddress can't be used where IPNetwork is accepted? It is just IPNetwork with /32 mask. It is counterintuitive for me, network administrator. > > > I foresee having to work around that in > > real applications by doing something obnoxious like: > > > > >>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask)) > > But a gateway is not an IP address plus hostmask. A gateway is just a single > IP address. What is the use of adding a hostmask to the gateway IP address > or some other IP address inside the network? Just to specify an address together with a mask, for short. It is widespread. People are used to it. When a gateway address written with a mask, it fully specifies you IP stack configuration, it specifies (when computed as usual): 1. netmask 2. your subnetwork address 3. host address 4. address of default route 5. subnetwork broadcast address 6. I don't know, maybe something else? 7. Oh, yes, it shows you the range of IP addresses, available for your use, if it was your subnetwork. So, what is better to write on network diagrams or other documentation, those five data pieces or just one gateway-with-a-mask? > > - Sebastian > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-3000%40udmvt.ru -- Alexey S. From python-3000 at udmvt.ru Wed Sep 16 11:08:49 2009 From: python-3000 at udmvt.ru (Alexey S) Date: Wed, 16 Sep 2009 14:08:49 +0500 Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library In-Reply-To: References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> <4A8C0267.5020408@wildenhain.de> <8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com> <8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com> <4A8C4264.2040708@trueblade.com> Message-ID: <20090916090849.GH12810@ruber.office.udmvt.ru> On Wed, Aug 19, 2009 at 02:28:38PM -0400, Glyph Lefkowitz wrote: > On Wed, Aug 19, 2009 at 2:20 PM, Eric Smith wrote: > > > > I think using .network and .broadcast are pretty well understood to be the > > [0] and [-1] of the network address block. I don't think we want to start > > creating new terms or access patterns here. > > > > +1 on leaving .network and .broadcast as-is (including returning a > > IPvXAddress object). > > > > -1. I think 'network.number' or 'network.zero' is a lot clearer than > 'network.network'. Maybe '.broadcast' would be okay, as long as it *can* be > adjusted for those unusual, or maybe even only hypothetical, networks where > it is not the [-1]. Real life example: network with a /31 mask. There are only two hosts: 0 and 1 first host configures the other's host as broadcast address and vice versa. NOTE - broadcasts are different here! Everything works, no one ever need to address "network" address, broadcasting works as expected. It works well between two our routers. What is wrong here? It just works for two Linuxes. It emulates point-to-point. Well, some weird soft will not let you to configure that (on Windows?heh)? So just let Python be off the shame list. Another real life examples include /32 networks on PPP. Just a point-to-point. No need for broadcasts and networks, a host just have one IP address and send all traffic via point-to-point link, no addressing is required there. This is a working dialup configuration, it works for me, it works for you, probably. It is not weird, it is common, it is used for PPPoE, for ADSL, for dialup. -- Alexey S. From g.brandl at gmx.net Wed Sep 16 11:53:48 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 16 Sep 2009 11:53:48 +0200 Subject: [Python-Dev] displayhook behavior in pdb Message-ID: May I have a short vote on this issue: http://bugs.python.org/issue6903 In short, pdb (since 2.6) uses a separate displayhook in order to avoid _ being reassigned (which screws up debugging apps that use _ as gettext). In that displayhook, I did not add the suppression of printing None, as it can be confusing to look at variables and get no output: (Pdb) foo 1 (Pdb) bar (Pdb) (You could argue that this is what the "p" command is for though.) Now in Python 3, where print is a function, if you call print in a loop (e.g. to debug a list or dictionary, as it is advertised in the pdb docs under the "alias" command), the output has the printed values interspersed with "None"s. Now, what is the lesser evil? cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From fuzzyman at voidspace.org.uk Wed Sep 16 12:08:01 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Sep 2009 11:08:01 +0100 Subject: [Python-Dev] displayhook behavior in pdb In-Reply-To: References: Message-ID: <4AB0B901.6090807@voidspace.org.uk> Georg Brandl wrote: > May I have a short vote on this issue: > > http://bugs.python.org/issue6903 > > In short, pdb (since 2.6) uses a separate displayhook in order to avoid > _ being reassigned (which screws up debugging apps that use _ as gettext). > In that displayhook, I did not add the suppression of printing None, as > it can be confusing to look at variables and get no output: > > (Pdb) foo > 1 > (Pdb) bar > (Pdb) > > (You could argue that this is what the "p" command is for though.) > > Now in Python 3, where print is a function, if you call print in a loop > (e.g. to debug a list or dictionary, as it is advertised in the pdb docs > under the "alias" command), the output has the printed values interspersed > with "None"s. > > Now, what is the lesser evil? > > IMO not showing the extraneous Nones is preferable. Michael > cheers, > Georg > > -- http://www.ironpythoninaction.com/ From p.f.moore at gmail.com Wed Sep 16 12:48:52 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 16 Sep 2009 11:48:52 +0100 Subject: [Python-Dev] displayhook behavior in pdb In-Reply-To: <4AB0B901.6090807@voidspace.org.uk> References: <4AB0B901.6090807@voidspace.org.uk> Message-ID: <79990c6b0909160348n32edc221v8dc082a0c52eb357@mail.gmail.com> 2009/9/16 Michael Foord : > Georg Brandl wrote: >> >> May I have a short vote on this issue: >> >> ? http://bugs.python.org/issue6903 >> >> In short, pdb (since 2.6) uses a separate displayhook in order to avoid >> _ being reassigned (which screws up debugging apps that use _ as gettext). >> In that displayhook, I did not add the suppression of printing None, as >> it can be confusing to look at variables and get no output: >> >> (Pdb) foo >> 1 >> (Pdb) bar >> (Pdb) >> >> (You could argue that this is what the "p" command is for though.) >> >> Now in Python 3, where print is a function, if you call print in a loop >> (e.g. to debug a list or dictionary, as it is advertised in the pdb docs >> under the "alias" command), the output has the printed values interspersed >> with "None"s. >> >> Now, what is the lesser evil? >> >> > > IMO not showing the extraneous Nones is preferable. I agree (although I don't use pdb, so my view shouldn't be given too much weight...) Paul. From steve at pearwood.info Wed Sep 16 12:52:59 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 16 Sep 2009 20:52:59 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> Message-ID: <200909162053.00354.steve@pearwood.info> I've been skimming emails in this thread, since most of them go over my head and I have no current need for an ipaddress module. But one thing I noticed stands out and needs commenting on: On Wed, 16 Sep 2009 11:05:26 am Peter Moody wrote: > On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith wrote: > > I completely agree. I don't know of any situation where I'd want a > > network of "192.168.1.1/24" to be anything other than an error. > > when you're entering the address of your nic. Eric is talking about a network. Peter replies by talking about an address. Perhaps I'm naive, but surely addresses aren't networks? If Peter's use-case is to treat addresses and networks interchangeably, I can't imagine that could be a good thing. Is it? What do the relevant RFCs say? As an outsider to this argument, and judging from the lack of agreement here, it seems to me that some (many? most?) developers who have need of ipaddress functions are a little unclear about the difference, or lack thereof, between addresses and networks. Is it our aim to pander to such confusion, and have a module which will Just Work for such users, even at the risk of sometimes accepting invalid input. Or is it to have a module which is strict and forces the user to Do The Right Thing, even at the cost of being less easy to use? For what it's worth, it seems to me that it would be better to have a strict module in the standard lib, and leave the DWIM code to third parties. -- Steven D'Aprano From fuzzyman at voidspace.org.uk Wed Sep 16 12:59:57 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Sep 2009 11:59:57 +0100 Subject: [Python-Dev] Python Language Summit #2 in February In-Reply-To: <20090916014810.GA318@andrew-kuchlings-macbook.local> References: <20090916014810.GA318@andrew-kuchlings-macbook.local> Message-ID: <4AB0C52D.5050608@voidspace.org.uk> A.M. Kuchling wrote: > PyCon 2010 will be February 19-21 2010 in Atlanta, Georgia (US). > Van Lindberg, PyCon chair, has approved having another Python Language > Summit on Thursday, February 18 2010. The web page for it is > > > The Python Language Summit is an invitation-only event for developers > of Python implementations and the standard library, to discuss issues > of common concern to all implementors. > > It will mostly be organized similarly to last year's event. I'm going > to drop the initial open discussion, which wasn't very useful, and > we'll just have three 1.5-hour discussion sessions. The fourth time > slot will be left open for chatting, development, or whatever the > attendees want to do. > > We therefore need to decide what those three sessions should be about. > Please discuss on python-dev and hopefully we can arrive at some > consensus on topics of reasonably wide current interest. (See > http://us.pycon.org/2009/about/summits/language/ for a reminder of > last year's topics.) > Given the long discussion on the stdlib-sig it seems like a discussion of the standard library would be useful. Potential topics include (some of which partially overlap each other): * Clarifying the deprecation process of modules including whether we will *ever* remove deprecated modules * Breaking the standard library into a separate development repository (which we decided to do at the last language summit but haven't yet - several developers are still keen to do this) * Deprecating and removing more obsolete / unmaintained modules (I believe a new PEP on this will be coming soon) * The goals of the standard library - some people want a 'dead' standard library (as in dead-stable) others want to see an actively improving standard library * Bringing new best-of-breed modules into the standard library (and what to when they fully or partly duplicate existing functionality) All the best, Michael > Feel free to raise discussions on other lists such as jython-dev, > catalog-sig, or wherever, but please summarize the results here on > python-dev; I won't see discussions on other lists. > > Like last time, invitations will be sent to the committers for Python, > Jython, IronPython, PyPy, Pynie, plus a few extra people and projects. > If you want to suggest another project or person who should be > invited, please send me a private e-mail. > > > Andrew M. Kuchling > amk at amk.ca > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From p.f.moore at gmail.com Wed Sep 16 13:04:07 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 16 Sep 2009 12:04:07 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909162053.00354.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> Message-ID: <79990c6b0909160404p4a1d5b5bu773b815b995f73dd@mail.gmail.com> 2009/9/16 Steven D'Aprano : > I've been skimming emails in this thread, since most of them go over my > head and I have no current need for an ipaddress module. Same here. > As an outsider to this argument, and judging from the lack of agreement > here, it seems to me that some (many? most?) developers who have need > of ipaddress functions are a little unclear about the difference, or > lack thereof, between addresses and networks. Is it our aim to pander > to such confusion, and have a module which will Just Work for such > users, even at the risk of sometimes accepting invalid input. Or is it > to have a module which is strict and forces the user to Do The Right > Thing, even at the cost of being less easy to use? > > For what it's worth, it seems to me that it would be better to have a > strict module in the standard lib, and leave the DWIM code to third > parties. As a non-expert, I find the confusion between network, address, address with mask, etc to be extremely unhelpful. I haven't looked to confirm if it's a confusion that only exists in the discussion, or if it's in the code as well. However, from my POV, I'd rather have a module that made the concepts clear and unambiguous, effectively educating me in the details of correct usage while I used it, rather than one that catered to my muddled thinking and allowed me to remain confused. Of course, the discussion seems to imply that even the experts have a confused view, so maybe I'm being too ambitious here :-) Paul. From ncoghlan at gmail.com Wed Sep 16 13:10:33 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 16 Sep 2009 21:10:33 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909162053.00354.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> Message-ID: <4AB0C7A9.3030001@gmail.com> Steven D'Aprano wrote: > I've been skimming emails in this thread, since most of them go over my > head and I have no current need for an ipaddress module. But one thing > I noticed stands out and needs commenting on: > > On Wed, 16 Sep 2009 11:05:26 am Peter Moody wrote: >> On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith > wrote: >>> I completely agree. I don't know of any situation where I'd want a >>> network of "192.168.1.1/24" to be anything other than an error. >> when you're entering the address of your nic. > > Eric is talking about a network. Peter replies by talking about an > address. Martin explained it better in another part of the thread: > if you know your address is 82.94.164.162, how do you compute > what you should spell for 82.94.164.162/27? Or, to put it another way, given an arbitrary host in a network (e.g. your own machine or the default gateway) and the netmask for that network, calculate the network address. With a "lax" parser on IPNetwork this is a trivial task - just create the network object and then retrieve the network address from it. If, on the other hand, IPNetwork demands that you already know the network address before allowing you to create an IPNetwork object, then you're pretty much out of luck - if all you have to work with are the IP strings then this is actually a tricky calculation. If the default IPNetwork constructor was made more strict, then this functionality would have to be made available another way (probably as an alternate constructor like IPNetwork.from_host_address rather than as a boolean 'strict' option) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Wed Sep 16 13:41:43 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 16 Sep 2009 11:41:43 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090915021130.4E57A600115@longblack.object-craft.com.au> <4AAFCBD6.5090104@scottdial.com> <87tyz411rp.fsf@uwakimon.sk.tsukuba.ac.jp> <8517e9350909151146k77261851i35890b2f1b0e902a@mail.gmail.com> <4AAFE757.9010503@scottdial.com> <87pr9r1mp0.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Stephen J. Turnbull xemacs.org> writes: > > Rather, don't you want to just give IPv4Address an attribute > 'network'? This could then be filled in by the indexing method on > IPv4Network. It doesn't make sense to me. An address, conceptually, doesn't have a "network". If I say "213.5.4.68", it isn't part of a specific network /a priori/. Not only doesn't it make sense to me, but it creates ambiguities. Do two similar addresses, but with different "networks", compare and hash equally? Both answers are unsatisfactory and will create nasty surprises for users. I don't see what's difficult in remembering the network object by yourself if you need it. Python is not Java, it has lightweight syntax, useful datatypes and notations (e.g. tuples). You don't need to have every use case taken care of by a dedicated library API. (to make an analogy: if I take an item from a list, it doesn't hold a reference to the list just in case it might be practical to do so) Regards Antoine. From mcguire at google.com Tue Sep 15 20:44:19 2009 From: mcguire at google.com (Jake McGuire) Date: Tue, 15 Sep 2009 11:44:19 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> Message-ID: <77c780b40909151144u17731a44na7b60e3f9debcd8a@mail.gmail.com> On Tue, Sep 15, 2009 at 11:36 AM, Peter Moody wrote: > On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire wrote: > > On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody wrote: > >> > >> On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire > wrote: > >> > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum > >> > wrote: > >> >> > >> >> What's the opinion of the other interested party or parties? I don't > >> >> want a repeat of the events last time, where we had to pull it at the > >> >> last time because there hadn't been enough discussion. > >> > > >> > How many other people are using this library? I think it's hard to > give > >> > really useful feedback on an API without using it for some non-trivial > >> > task, > >> > but maybe other people don't have this problem. > >> > -jake > >> > >> 188 (check that, 190) people have downloaded the 2.0 release in the > >> last week (numbers publicly available from the code.google.com). I > >> can't tell you how many (if any) have downloaded it via svn. > > > > Downloading and using are not the same thing. > > Correct, but there is a strong positive correlation between the two. > If you have a better method for determining what you would consider an > appropriate level of usage, I'm all ears. > Put something on the project page (or download page if possible) saying "ipaddr is being considered for inclusion in the Python standard library. We want to make sure it meets your needs, but we need you to tell us. If you use ipaddr and like it, please let us know on ip-addr-dev." I dunno, maybe it's too much work. But no one else seems to have an opinion strong enough to share, at least not at this point. -jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From mcguire at google.com Tue Sep 15 20:58:12 2009 From: mcguire at google.com (Jake McGuire) Date: Tue, 15 Sep 2009 11:58:12 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <77c780b40909151144u17731a44na7b60e3f9debcd8a@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> <77c780b40909151144u17731a44na7b60e3f9debcd8a@mail.gmail.com> Message-ID: <77c780b40909151158y17f20b65je41f934de35c0a38@mail.gmail.com> On Tue, Sep 15, 2009 at 11:44 AM, Jake McGuire wrote: > On Tue, Sep 15, 2009 at 11:36 AM, Peter Moody wrote: > >> On Tue, Sep 15, 2009 at 11:33 AM, Jake McGuire >> wrote: >> > On Tue, Sep 15, 2009 at 10:36 AM, Peter Moody wrote: >> >> >> >> On Tue, Sep 15, 2009 at 10:16 AM, Jake McGuire >> wrote: >> >> > On Mon, Sep 14, 2009 at 9:54 AM, Guido van Rossum >> >> > wrote: >> >> >> >> >> >> What's the opinion of the other interested party or parties? I don't >> >> >> want a repeat of the events last time, where we had to pull it at >> the >> >> >> last time because there hadn't been enough discussion. >> >> > >> >> > How many other people are using this library? I think it's hard to >> give >> >> > really useful feedback on an API without using it for some >> non-trivial >> >> > task, >> >> > but maybe other people don't have this problem. >> >> > -jake >> >> >> >> 188 (check that, 190) people have downloaded the 2.0 release in the >> >> last week (numbers publicly available from the code.google.com). I >> >> can't tell you how many (if any) have downloaded it via svn. >> > >> > Downloading and using are not the same thing. >> >> Correct, but there is a strong positive correlation between the two. >> If you have a better method for determining what you would consider an >> appropriate level of usage, I'm all ears. >> > > Put something on the project page (or download page if possible) saying > "ipaddr is being considered for inclusion in the Python standard library. > We want to make sure it meets your needs, but we need you to tell us. If > you use ipaddr and like it, please let us know on ip-addr-dev." > > I dunno, maybe it's too much work. But no one else seems to have an > opinion strong enough to share, at least not at this point. > To clarify: there will always be some group of people happy to tell you that whatever you made is crap and that it should have been done differently. There may be many more people out there who think that what you did is great. But those people probably don't read python-dev so their voices don't get heard when it comes to deciding what to do with PEP 3144. The easiest way to get their voices heard is to ask them to speak up, and the one place you know they visited was the ipaddr page on code.google.com, so you may as well ask them there. -jake -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Sep 16 13:53:28 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 16 Sep 2009 11:53:28 +0000 (UTC) Subject: [Python-Dev] Python Language Summit #2 in February References: <20090916014810.GA318@andrew-kuchlings-macbook.local> <4AB0C52D.5050608@voidspace.org.uk> Message-ID: Michael Foord voidspace.org.uk> writes: > > Given the long discussion on the stdlib-sig it seems like a discussion > of the standard library would be useful. Potential topics include (some > of which partially overlap each other): > [snip] Let me suggest the following additional point: * Clarifying maintainership. What is a maintainer, what can be expected from him and what is *not* expected from him. Whether maintainers are, or are not, "owners". What the relationship of other core developers to maintainers should be, and should not be. Of course, all this will perhaps have been discussed before the summit. Regards Antoine. From fuzzyman at voidspace.org.uk Wed Sep 16 13:58:44 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 16 Sep 2009 12:58:44 +0100 Subject: [Python-Dev] Python Language Summit #2 in February In-Reply-To: References: <20090916014810.GA318@andrew-kuchlings-macbook.local> <4AB0C52D.5050608@voidspace.org.uk> Message-ID: <4AB0D2F4.3080007@voidspace.org.uk> Antoine Pitrou wrote: > Michael Foord voidspace.org.uk> writes: > >> Given the long discussion on the stdlib-sig it seems like a discussion >> of the standard library would be useful. Potential topics include (some >> of which partially overlap each other): >> >> > [snip] > > Let me suggest the following additional point: > > * Clarifying maintainership. What is a maintainer, what can be expected from him > and what is *not* expected from him. Whether maintainers are, or are not, > "owners". What the relationship of other core developers to maintainers should > be, and should not be. > Yup, good addition. > Of course, all this will perhaps have been discussed before the summit. > > When we add discussing the relationship of the other implementations to the standard library I think there is plenty to discuss, even if some of the points are clearer or closer to being settled (or as settled as they can possibly get) by the time of the summit. Michael > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From barry at python.org Wed Sep 16 14:07:00 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 16 Sep 2009 08:07:00 -0400 Subject: [Python-Dev] Python Language Summit #2 in February In-Reply-To: References: <20090916014810.GA318@andrew-kuchlings-macbook.local> <4AB0C52D.5050608@voidspace.org.uk> Message-ID: <9A82D87C-9A95-4338-8661-2655F3DB0896@python.org> On Sep 16, 2009, at 7:53 AM, Antoine Pitrou wrote: > Of course, all this will perhaps have been discussed before the > summit. Sure, but not resolved. :) I do think stdlib evolution should be high on the list of topics. -B -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From exarkun at twistedmatrix.com Wed Sep 16 14:50:08 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 16 Sep 2009 12:50:08 -0000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB0C7A9.3030001@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> Message-ID: <20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain> On 11:10 am, ncoghlan at gmail.com wrote: >Steven D'Aprano wrote: >>I've been skimming emails in this thread, since most of them go over >>my >>head and I have no current need for an ipaddress module. But one thing >>I noticed stands out and needs commenting on: >> >>On Wed, 16 Sep 2009 11:05:26 am Peter Moody wrote: >>>On Tue, Sep 15, 2009 at 6:02 PM, Eric Smith >>wrote: >>>>I completely agree. I don't know of any situation where I'd want a >>>>network of "192.168.1.1/24" to be anything other than an error. >>>when you're entering the address of your nic. >> >>Eric is talking about a network. Peter replies by talking about an >>address. > >Martin explained it better in another part of the thread: >>if you know your address is 82.94.164.162, how do you compute >>what you should spell for 82.94.164.162/27? > >Or, to put it another way, given an arbitrary host in a network (e.g. >your own machine or the default gateway) and the netmask for that >network, calculate the network address. > >With a "lax" parser on IPNetwork this is a trivial task - just create >the network object and then retrieve the network address from it. > >If, on the other hand, IPNetwork demands that you already know the >network address before allowing you to create an IPNetwork object, then >you're pretty much out of luck - if all you have to work with are the >IP >strings then this is actually a tricky calculation. > >If the default IPNetwork constructor was made more strict, then this >functionality would have to be made available another way (probably as >an alternate constructor like IPNetwork.from_host_address rather than >as >a boolean 'strict' option) This seems to be the right solution to me, particularly the use of an alternate constructor rather than an ambiguously named flag. Jean-Paul From barry at python.org Wed Sep 16 14:52:47 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 16 Sep 2009 08:52:47 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> Message-ID: On Sep 15, 2009, at 9:53 AM, Ronald Oussoren wrote: > The IDLE issue is IMHO a release blocker, as is issue 6851. So that leaves 4 release blockers for 2.6.3. Three of these are assigned to Ronald. Ronald are you sure you will have time to fix these by then? The one I'm still uncertain on is the IDLE hang. If this only affects OS X 10.6 and there is no progress on it by the time 2.6.3 is otherwise ready, I'm willing to knock this down in severity. Martin, you have the other showstopper bug for 2.6.3. Do you think you will be able to fix that one in time? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From ncoghlan at gmail.com Wed Sep 16 16:35:31 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 17 Sep 2009 00:35:31 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain> Message-ID: <4AB0F7B3.2090600@gmail.com> exarkun at twistedmatrix.com wrote: > On 11:10 am, ncoghlan at gmail.com wrote: >> If the default IPNetwork constructor was made more strict, then this >> functionality would have to be made available another way (probably as >> an alternate constructor like IPNetwork.from_host_address rather than as >> a boolean 'strict' option) > > This seems to be the right solution to me, particularly the use of an > alternate constructor rather than an ambiguously named flag. Particular since someone pointed out that the "Network" object was remembering which IP address happened to be passed to the constructor (which is "Address with Mask" behaviour, as has been mentioned). That said, I've been trying to refrain from commenting on specifics of the API, as I haven't tried to use it for anything. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From eric at trueblade.com Wed Sep 16 16:42:31 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 16 Sep 2009 10:42:31 -0400 Subject: [Python-Dev] Python Language Summit #2 in February In-Reply-To: <20090916014810.GA318@andrew-kuchlings-macbook.local> References: <20090916014810.GA318@andrew-kuchlings-macbook.local> Message-ID: <4AB0F957.8020300@trueblade.com> A.M. Kuchling wrote: > We therefore need to decide what those three sessions should be about. > Please discuss on python-dev and hopefully we can arrive at some > consensus on topics of reasonably wide current interest. (See > http://us.pycon.org/2009/about/summits/language/ for a reminder of > last year's topics.) I'd like to have a discussion on the future of 2.x, at least as far as core development goes. Supporting both 2.x and 3.x cuts my contributions in half, at best. For example, in order to keep 2.x and 3.x nearly in sync, Mark and I back-ported the API's for short-float-repr to 2.x, even though 2.x itself didn't see any benefit from that work (well, at least not much of a benefit). My initial thought is let's stop after 2.7 (if we even release 2.7). But it all depends on uptake of 3.x, and what stopping core development means to the community. Can we discuss this at the language summit? Eric. From guido at python.org Wed Sep 16 17:00:04 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 16 Sep 2009 08:00:04 -0700 Subject: [Python-Dev] displayhook behavior in pdb In-Reply-To: <79990c6b0909160348n32edc221v8dc082a0c52eb357@mail.gmail.com> References: <4AB0B901.6090807@voidspace.org.uk> <79990c6b0909160348n32edc221v8dc082a0c52eb357@mail.gmail.com> Message-ID: On Wed, Sep 16, 2009 at 3:48 AM, Paul Moore wrote: > 2009/9/16 Michael Foord : > > Georg Brandl wrote: > >> > >> May I have a short vote on this issue: > >> > >> http://bugs.python.org/issue6903 > >> > >> In short, pdb (since 2.6) uses a separate displayhook in order to avoid > >> _ being reassigned (which screws up debugging apps that use _ as > gettext). > >> In that displayhook, I did not add the suppression of printing None, as > >> it can be confusing to look at variables and get no output: > >> > >> (Pdb) foo > >> 1 > >> (Pdb) bar > >> (Pdb) > >> > >> (You could argue that this is what the "p" command is for though.) > >> > >> Now in Python 3, where print is a function, if you call print in a loop > >> (e.g. to debug a list or dictionary, as it is advertised in the pdb docs > >> under the "alias" command), the output has the printed values > interspersed > >> with "None"s. > >> > >> Now, what is the lesser evil? > >> > >> > > > > IMO not showing the extraneous Nones is preferable. > > I agree (although I don't use pdb, so my view shouldn't be given too > much weight...) > I *do* use pdb a lot, and I agree that the new behavior is weird. -- --Guido van Rossum (home page: http://www.python.org/~guido/) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rdmurray at bitdance.com Wed Sep 16 17:25:09 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 16 Sep 2009 11:25:09 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <20090916125008.12215.453222785.divmod.xquotient.31@localhost.localdomain> Message-ID: On Wed, 16 Sep 2009 at 12:50, exarkun at twistedmatrix.com wrote: > On 11:10 am, ncoghlan at gmail.com wrote: >> Or, to put it another way, given an arbitrary host in a network (e.g. >> your own machine or the default gateway) and the netmask for that >> network, calculate the network address. >> >> With a "lax" parser on IPNetwork this is a trivial task - just create >> the network object and then retrieve the network address from it. >> >> If, on the other hand, IPNetwork demands that you already know the >> network address before allowing you to create an IPNetwork object, then >> you're pretty much out of luck - if all you have to work with are the IP >> strings then this is actually a tricky calculation. >> >> If the default IPNetwork constructor was made more strict, then this >> functionality would have to be made available another way (probably as >> an alternate constructor like IPNetwork.from_host_address rather than as >> a boolean 'strict' option) > > This seems to be the right solution to me, particularly the use of an > alternate constructor rather than an ambiguously named flag. +1 --David From rdmurray at bitdance.com Wed Sep 16 17:32:25 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 16 Sep 2009 11:32:25 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <79990c6b0909160404p4a1d5b5bu773b815b995f73dd@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <79990c6b0909160404p4a1d5b5bu773b815b995f73dd@mail.gmail.com> Message-ID: On Wed, 16 Sep 2009 at 12:04, Paul Moore wrote: > Of course, the discussion seems to imply that even the experts have a > confused view, so maybe I'm being too ambitious here :-) Part of the problem, as we discovered in the last go-round on ipaddr, is that there are two types of experts: those who primarily deal with networks and for whom host addresses are just a special case, and those who primarily deal with host addresses, for whom networks are relevant only in certain contexts. I hope we are converging on something that works for both. That should certainly be a goal, since the fact that it didn't was what scuttled inclusion last time. --David From g.brandl at gmx.net Wed Sep 16 18:35:59 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 16 Sep 2009 18:35:59 +0200 Subject: [Python-Dev] displayhook behavior in pdb In-Reply-To: References: <4AB0B901.6090807@voidspace.org.uk> <79990c6b0909160348n32edc221v8dc082a0c52eb357@mail.gmail.com> Message-ID: Guido van Rossum schrieb: > >> Now, what is the lesser evil? > >> > >> > > > > IMO not showing the extraneous Nones is preferable. > > I agree (although I don't use pdb, so my view shouldn't be given too > much weight...) > > > I *do* use pdb a lot, and I agree that the new behavior is weird. Okay, I've fixed it in trunk and will backport soon. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From brett at python.org Wed Sep 16 21:57:28 2009 From: brett at python.org (Brett Cannon) Date: Wed, 16 Sep 2009 12:57:28 -0700 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> Message-ID: On Wed, Sep 16, 2009 at 05:52, Barry Warsaw wrote: > On Sep 15, 2009, at 9:53 AM, Ronald Oussoren wrote: > >> The IDLE issue is IMHO a release blocker, as is issue 6851. > > So that leaves 4 release blockers for 2.6.3. ?Three of these are assigned to > Ronald. ?Ronald are you sure you will have time to fix these by then? ?The > one I'm still uncertain on is the IDLE hang. ?If this only affects OS X 10.6 > and there is no progress on it by the time 2.6.3 is otherwise ready, I'm > willing to knock this down in severity. > > Martin, you have the other showstopper bug for 2.6.3. ?Do you think you will > be able to fix that one in time? That issue doesn't require Martin specifically. I just needed to know if changing the BaseException struct would be bad (Georg answered w/ "yes"). It will probably fall on to me (or Georg if his proposed solution pans out) to get this fixed. -Brett From ronaldoussoren at mac.com Wed Sep 16 22:28:34 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Wed, 16 Sep 2009 22:28:34 +0200 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> Message-ID: On 16 Sep, 2009, at 14:52, Barry Warsaw wrote: > On Sep 15, 2009, at 9:53 AM, Ronald Oussoren wrote: > >> The IDLE issue is IMHO a release blocker, as is issue 6851. > > So that leaves 4 release blockers for 2.6.3. Three of these are > assigned to Ronald. Ronald are you sure you will have time to fix > these by then? The one I'm still uncertain on is the IDLE hang. If > this only affects OS X 10.6 and there is no progress on it by the > time 2.6.3 is otherwise ready, I'm willing to knock this down in > severity. I don't know for sure if I'll be able to fix these issues, I'll basicly just have time to work on this during the weekend. * IDLE hang on OSX 10.6: I don't really know where to start looking, this may have something to do with Tk-Cocoa (which is used on 10.6 and not on earlier releases). There is a patch for Tk-Cocoa support on the tracker, but I haven't had time yet to fully test that. * Compile error for ctypes on 10.6: Should be easy to fix. Apple has released the sources of the copy of libffi included with 10.6.1, I'll use that to work on a patch. * Crash with urllib and threads on OSX 10.6: This probably requires rewriting some code using ctypes in C. The crash seems to occur when CoreFoundation tries to initialise itself after being loaded on a secondairy thread by ctypes. To make matters worse the ctypes type annotations in urllib are not correct for 64-bit code (but the crash happens in 32-bit mode as well) Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From thomas at python.org Wed Sep 16 22:34:36 2009 From: thomas at python.org (Thomas Wouters) Date: Wed, 16 Sep 2009 22:34:36 +0200 Subject: [Python-Dev] pthreads, fork, import, and execvp In-Reply-To: <4AA8249C.9010707@gmail.com> References: <446C6FC9.5060205@gmail.com> <446CB69E.4090609@v.loewis.de> <9e804ac0907161308t19dc5399q2484339b18fc6318@mail.gmail.com> <3d2ce8cb0907201126t2ce8866eq3580aed9e3e6bf2@mail.gmail.com> <9e804ac0907201716v584e3337t3be9a7bf3311ff3b@mail.gmail.com> <9e804ac0907231628vd9de823h95656a02ffc59bc4@mail.gmail.com> <52dc1c820907251025j4db82363yea166be14a012ada@mail.gmail.com> <9e804ac0909090907w3ed3120aw70dd1cfd67d30b56@mail.gmail.com> <4AA8249C.9010707@gmail.com> Message-ID: <9e804ac0909161334x3d5d8569je67fd484c1747c7f@mail.gmail.com> On Wed, Sep 9, 2009 at 23:56, Nick Coghlan wrote: > Thomas Wouters wrote: > > Your idea of making this an API called a 'fork lock' or something > > sounds good (though I think it needs a better name. PyBeforeFork & > > PyAfterFork?). The subprocess module, for example, disables garbage > > collection before forking and restores it afterwards to avoid > > http://bugs.python.org/issue1336. That type of thing could also be > > done in such a function. > > > > > > Unfortunately it's rather hard to make those functions work correctly > > with the current API -- we can't provide functions you can just use as > > arguments to pthread_atfork because the global interpreter lock is not > > re-entrant and we have no way of testing whether the current thread > > holds the GIL. > > I thought "make sure I have the GIL, either by already having it or > waiting for it if I don't yet have it" was the entire point of the > PyGILState_Ensure() API? [1] > Hm, yeah. For some reason I was certain it was inappropriate, back when I was trying to create a pthread_atfork-friendly set of functions. At the time I was also hip-deep in the awfulness of Python/thread*.c and its unsafe punning and unwarranted assumptions, so I may have overreacted. I added this as a feature-request issue ( http://bugs.python.org/issue6923 ) and will look at it some more. In the mean time, I fixed the biggest source of issues by checking in the change to make at least calls to fork() made by Python be safe, also backported to 2.6. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Wed Sep 16 22:35:30 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 16 Sep 2009 20:35:30 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit?: > Folks, Guido, > > I believe PEP 3144 is ready for your review. When you get a chance, can > you take a look/make a pronouncement? Besides what has already been said in the thread, I have a bunch of comments: It should be noted that __len__ doesn't work as expected since python internals has this limited to a 32 bit integer and it would need to be at least 128 bits to work with IPV6. You should clarify what it means: does the result get truncated, or is an error thrown when it can't fit inside an int? 1. all IP addresses and networks, both IPv4 and IPv6. (IPAddrBase) 2. all IP addresses of both versions. (BaseIP) 3. all IP networks of both version. (BaseNet) 4. all IPv4 objects, both addresses and networks. (BaseV4) 5. all IPv6 objects, both addresses and networks. (BaseV6) Should those base classes be exposed publically? It may make life more difficult if an alternate implementation (say, a C accelerator) wants to adopt a different code sharing strategy. If they are not to be part of the public API, their names should be prefixed with an underscore. From peter at hda3.com Wed Sep 16 22:48:06 2009 From: peter at hda3.com (Peter Moody) Date: Wed, 16 Sep 2009 13:48:06 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: <8517e9350909161348h63462089x7743826b91d3689@mail.gmail.com> On Wed, Sep 16, 2009 at 1:35 PM, Antoine Pitrou wrote: > Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit?: >> Folks, Guido, >> >> I believe PEP 3144 is ready for your review. ?When you get a chance, can >> you take a look/make a pronouncement? > > Besides what has already been said in the thread, I have a bunch of > comments: > > ? ?It should be noted > ? ?that __len__ doesn't work as expected since python internals has this > ? ?limited to a 32 bit integer and it would need to be at least 128 bits > to > ? ?work with IPV6. > > You should clarify what it means: does the result get truncated, or is an > error thrown when it can't fit inside an int? I'll clarify this, but it looks like this: >>> len(xrange(0xFFFFFFFFFFFFFFFFFFFFFFFFF)) Traceback (most recent call last): File "", line 1, in OverflowError: long int too large to convert to int >>> basically, __len__(self): can only return an int. At least that's what I recall from the discussion. > ? ?1. all IP addresses and networks, both IPv4 and IPv6. (IPAddrBase) > > ? ?2. all IP addresses of both versions. (BaseIP) > > ? ?3. all IP networks of both version. (BaseNet) > > ? ?4. all IPv4 objects, both addresses and networks. (BaseV4) > > ? ?5. all IPv6 objects, both addresses and networks. (BaseV6) > > Should those base classes be exposed publically? It may make life more > difficult if an alternate implementation (say, a C accelerator) wants to > adopt a different code sharing strategy. > > If they are not to be part of the public API, their names should be > prefixed with an underscore. makes sense. I see no need for them to be public. private they go. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From tjreedy at udel.edu Wed Sep 16 23:50:39 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 16 Sep 2009 17:50:39 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909161348h63462089x7743826b91d3689@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909161348h63462089x7743826b91d3689@mail.gmail.com> Message-ID: Peter Moody wrote: > On Wed, Sep 16, 2009 at 1:35 PM, Antoine Pitrou wrote: >> Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit : >>> Folks, Guido, >>> >>> I believe PEP 3144 is ready for your review. When you get a chance, can >>> you take a look/make a pronouncement? >> Besides what has already been said in the thread, I have a bunch of >> comments: >> >> It should be noted >> that __len__ doesn't work as expected since python internals has this >> limited to a 32 bit integer and it would need to be at least 128 bits >> to >> work with IPV6. >> >> You should clarify what it means: does the result get truncated, or is an >> error thrown when it can't fit inside an int? > > I'll clarify this, but it looks like this: > >>>> len(xrange(0xFFFFFFFFFFFFFFFFFFFFFFFFF)) > Traceback (most recent call last): > File "", line 1, in > OverflowError: long int too large to convert to int > > basically, __len__(self): can only return an int. At least that's what > I recall from the discussion. FWIW, the limitation remains in Py3, with a modified error msg. >>> len(range(0xFFFFFFFFFFFFFFFFFFFFFFFFF)) Traceback (most recent call last): File "", line 1, in len(range(0xFFFFFFFFFFFFFFFFFFFFFFFFF)) OverflowError: Python int too large to convert to C ssize_t From greg.ewing at canterbury.ac.nz Wed Sep 16 23:59:47 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Sep 2009 09:59:47 +1200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB0C7A9.3030001@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> Message-ID: <4AB15FD3.5040804@canterbury.ac.nz> Nick Coghlan wrote: > Or, to put it another way, given an arbitrary host in a network (e.g. > your own machine or the default gateway) and the netmask for that > network, calculate the network address. Some people have claimed that the gateway address of a network isn't necessarily the zero address in that network. If that's true, then you *can't* calculate the network address from a host address and a netmask -- there isn't enough information. Furthermore, an IPNetwork object needs to be able to represent a network address whose address part contains bits that aren't in the mask. -- Greg From eric at trueblade.com Thu Sep 17 00:04:32 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 16 Sep 2009 18:04:32 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. Message-ID: Eric. "Greg Ewing" wrote: >Nick Coghlan wrote: > >> Or, to put it another way, given an arbitrary host in a network (e.g. >> your own machine or the default gateway) and the netmask for that >> network, calculate the network address. > >Some people have claimed that the gateway address of a >network isn't necessarily the zero address in that network. >If that's true, then you *can't* calculate the network >address from a host address and a netmask -- there isn't >enough information. Furthermore, an IPNetwork object >needs to be able to represent a network address whose >address part contains bits that aren't in the mask. I don't see why that would be considered a network address, then. It sounds to me like that's a host address plus a netmask. From cs at zip.com.au Thu Sep 17 00:27:12 2009 From: cs at zip.com.au (Cameron Simpson) Date: Thu, 17 Sep 2009 08:27:12 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AAFCBD6.5090104@scottdial.com> References: <4AAFCBD6.5090104@scottdial.com> Message-ID: <20090916222712.GA2794@cskk.homeip.net> On 15Sep2009 13:16, Scott Dial wrote: | It just happened that I needed a tool today to calculate the gateway IP | for an interface, and I took it as an opportunity to try out this ipaddr | module. I'll attempt to relate my experience below... | | I have to concur with the opinions above. I was very confused by the | following error: | | >>> addr = ipaddr.IPAddress("10.1.2.3/255.255.240.0") | ... | ipaddr.IPAddressIPValidationError: '98.223.189.24/255.255.240.0' is not | a valid address (hint, it's probably a network) It sure looks like a network to me, too. | Because, it *is* a address of a host on a network. I gave in and said: No, it is an IP address plus a network mask, and that notation is generally used to describe a network (base IP of network plus the mask to indicate its range). | >>> net = ipaddr.IPNetwork("10.1.2.3/255.255.240.0") | | But then, I was dumbfounded as to how I could get the gateway IP from | this IPNetwork object. It took me a while to figure out that you can | iterate over IPNetwork instances: | | >>> gateway = net[1] Shrug. It is a common convention for a network's _default_ gateway to be the network's base address + 1. But certainly not necessary. If that convention is also your convention, then good. | I was then confused, because: | | >>> print(type(gateway)) | | | Which sorta blew my mind.. I fully expected to receive an IPNetwork back | from that operation. I have no idea why you would expect that. A gateway is just the IP _address_ of a host that will route packets to other networks. It is inherently an address, and not a network. Remember that a network (well, a host on a network) may have multiple gateways; usually a single "default" gateway, and potentially others for special routes to particular other networks. There's no need to have just one gateway address. Perhaps you are thinking of routing table entries (such as recited by "netstat -r" on a UNIX box). They normally consist of a network specification (base-addr/network-mask) and a gateway _address_. | It is unclear to me why the network information | gets chucked by that operation. I foresee having to work around that in | real applications by doing something obnoxious like: | | >>> actual_gateway = ipaddr.IPNetwork("%s/%s" % (gateway, addr.netmask)) | | But since I only actually needed the IP address for the gateway, it | fulfilled my needs. And that is a clue. You only need the gateway IP address in your needs because a gateway _is_ an IP address. It is quite possible you are thinking the word "gateway", but actually thinking about a "route". | In the end, I found the names IPNetwork/IPAddress and their | instantiations confusing. I really believe that you are confused in this instance, not the module. | ISTM that IPNetwork is overloaded and plays | two roles of being an IPNetwork and being an IPAddressWithNetwork. And | finally, it's unclear to me why iterating over a IPNetwork would not | produce a sequence of IPNetwork(/IPAddressWithNetwork) objects. | | ISTM that if I started with an IPNetwork object, the API should always | return IPNetwork objects. Not for methods that produce addresses! | If I want just an IPAddress object, then I can | always fetch the "ip" attribute to get that. Perhaps there is some | compelling conceptual argument as to why this is not correct, but it | seems like the API destroys information needlessly. I really think you have the wrong mental image of what these terms mean. Nothing in your examples seems surprising, confusing or wrong to me. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ From rdmurray at bitdance.com Thu Sep 17 00:27:28 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 16 Sep 2009 18:27:28 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB15FD3.5040804@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> Message-ID: On Thu, 17 Sep 2009 at 09:59, Greg Ewing wrote: > Nick Coghlan wrote: > >> Or, to put it another way, given an arbitrary host in a network (e.g. >> your own machine or the default gateway) and the netmask for that >> network, calculate the network address. > > Some people have claimed that the gateway address of a > network isn't necessarily the zero address in that network. > If that's true, then you *can't* calculate the network > address from a host address and a netmask -- there isn't > enough information. Furthermore, an IPNetwork object > needs to be able to represent a network address whose > address part contains bits that aren't in the mask. A 'gateway' is an IP (ie: host) in a network to which you route packets for delivery elsewhere. That's a very different thing from the 'network number' (or 'network address' or 'network zero' or 'network identifier' or whatever you want to call it). Any IP in a network can be a gateway IP, and in many networks there is more than one gateway (and thus the need for routing tables...) A network is conventionally represented by an IP address in which the bits corresponding to the one bits in the netmask are set to zero, plus the netmask. That IP with the bits set to zero is the 'network number' or zero for the network, and it is almost never used as the address for a host in the network. Thus it would be very surprising to find it being used as a gateway address. There is no reason why an IPNetwork object should record any bits from the portion of the IP address that are covered by the mask bits. To do otherwise is to conflate Networks with individual IPs, which is what got IPAddr rejected the first time around. (And for those of you who don't know/remember, I was initially an IPAddr booster in its original incarnation, because I come from a network engineering background.) This existence of this kind of confusion is why I voted +1 for the IPNetwork constructor rejecting input with non-zero bits covered by the mask, and a separate constructor method to return the network object representing the network that a given IP address is in when a given mask is applied to it. (Which is easily computed by applying the mask to the IP.) --David From daniel at stutzbachenterprises.com Thu Sep 17 00:31:27 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 16 Sep 2009 17:31:27 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB15FD3.5040804@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> Message-ID: On Wed, Sep 16, 2009 at 4:59 PM, Greg Ewing wrote: > Some people have claimed that the gateway address of a > network isn't necessarily the zero address in that network. > I'll go further: I don't think it's even legal for the gateway address to be the zero address of the network (and I used to program the embedded software in routers for a living :) ). > If that's true, then you *can't* calculate the network > address from a host address and a netmask -- there isn't > enough information. In a router or operating system kernel, an IPv4 address or netmask is stored in a 32-bit unsigned integer. Host addresses, network addresses, and netmasks satisfy the following properties, where & is the bitwise-AND operation: network address & netmask == network address host address & netmask == network address of that host A gateway is just a host that will forward packets. A gateway address has the same properties as a host address and isn't the same as the zero address. Every host has a routing table that contains a list of (network address, gateway, netmask) tuples. To see the list, just run "/sbin/route -n" on any Linux system (and most other Unixes; root is not typically required) or "route print" on a Windows system. To route a packet, the operating system essentially performs the following algorithm: def find_gateway_for_destination(destination_address): for (network_address, gateway, netmask) in list_of_routing_table_entires: if network_address == destination_address & netmask: return gateway raise DestinationUnreachable Furthermore, an IPNetwork object > needs to be able to represent a network address whose > address part contains bits that aren't in the mask. > Network addresses never contain bits that aren't in the mask (due to the rule: "network address & netmask == network address"). -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Thu Sep 17 01:56:25 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Sep 2009 11:56:25 +1200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> Message-ID: <4AB17B29.8050708@canterbury.ac.nz> R. David Murray wrote: > A network is conventionally represented by an IP address in which the > bits corresponding to the one bits in the netmask are set to zero, plus > the netmask. Okay, that's clarified things for me, thanks. In that case, we shouldn't be talking about a "network address" at all, but just a "network", and it makes sense to have 1) A class called IPNetwork that contains an IP number and a mask, with the IP number constrained not to have any ones where the mask has zeroes, and 2) A class called IPAddress which only contains an IP number. It seems that some people would also like to have 3) A class called IPAddressWithMask that contains an IP number and a mask, with no constraints, but I'm not enough of an expert to know whether such a thing would be used often enough to be worth having a special type for it, as opposed to using an (IPNetwork, IPAddress) pair, or attaching a 'network' attribute to an IPAddress, or some other solution when the need arises. -- Greg From andrewm at object-craft.com.au Thu Sep 17 02:30:27 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu, 17 Sep 2009 10:30:27 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB17B29.8050708@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> Message-ID: <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> >R. David Murray wrote: > >> A network is conventionally represented by an IP address in which the >> bits corresponding to the one bits in the netmask are set to zero, plus >> the netmask. > >Okay, that's clarified things for me, thanks. Put another way, an "Address" describes a single end-point and a "Network" describes a set of (contiguous) Addresses. Where things have become confused is that, for practical reasons, it is convenient to have a representation for an Address and it's containing Network (the later can be derived from the Address and a mask). We tried to make the current Network entity do double-duty, but it is just leading to confusion. This is why I proprose there be three entities: * an Address entity (same as the current one) * a Network entity (like now, but requires masked bits to be zero) * an AddressWithMask entity (existing Network, but no container behaviour) There is a school of thought that says we only need a single class that behaves like the current Network entity - end-points are simply represented by an all-ones mask. This is, I think, where we started. But this scheme was rejected. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From andrewm at object-craft.com.au Thu Sep 17 02:41:37 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu, 17 Sep 2009 10:41:37 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> Message-ID: <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> >> Some people have claimed that the gateway address of a >> network isn't necessarily the zero address in that network. It almost never is - conventions vary, but it is often the network address plus one, or the broadcast address minus one. >I'll go further: I don't think it's even legal for the gateway address to be >the zero address of the network (and I used to program the embedded software >in routers for a living :) ). I don't think the RFCs forbid the zero address being used, and "enlightened" network stacks allow it (typically routers) to achieve better utilisation of the limited IPv4 address space (for a /24 or larger, wasting one address out of 255 isn't too bad, but it is now typical to use much smaller nets - right down to /30). >> If that's true, then you *can't* calculate the network >> address from a host address and a netmask -- there isn't >> enough information. You can always calculate the network address from the IP address plus mask - the network address is simply the bits that are not masked. In the olden days, the mask was spelled out in octets (eg 255.255.255.0). But we've moved to a more compact and logical notation where the number of leading significant bits is specified (eg /24). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From peter at hda3.com Thu Sep 17 02:58:25 2009 From: peter at hda3.com (Peter Moody) Date: Wed, 16 Sep 2009 17:58:25 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> Message-ID: <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> On Wed, Sep 16, 2009 at 5:30 PM, Andrew McNamara wrote: >>R. David Murray wrote: >> >>> A network is conventionally represented by an IP address in which the >>> bits corresponding to the one bits in the netmask are set to zero, plus >>> the netmask. >> >>Okay, that's clarified things for me, thanks. > > Put another way, an "Address" describes a single end-point and a "Network" > describes a set of (contiguous) Addresses. > > Where things have become confused is that, for practical reasons, it is > convenient to have a representation for an Address and it's containing > Network (the later can be derived from the Address and a mask). We tried > to make the current Network entity do double-duty, but it is just leading > to confusion. > > This is why I proprose there be three entities: > > ?* an Address entity (same as the current one) > ?* a Network entity (like now, but requires masked bits to be zero) > ?* an AddressWithMask entity (existing Network, but no container behaviour) This proposal actually leads to 6 entities (3 for IPv4 and 3 for IPv6). It's still unclear to me what is gained by pulling AddressWithMask functionality out of the current network classes. It's easy enough for the concerned developer who to check if the entered network address does actually have all of its host bits set to zero. It is not my experience that this behavior is desired so often that having the network classes behave as they do now leads to a great deal of confusion. > There is a school of thought that says we only need a single class > that behaves like the current Network entity - end-points are simply > represented by an all-ones mask. This is, I think, where we started. But > this scheme was rejected. I don't think anything has actually been rejected. > -- > Andrew McNamara, Senior Developer, Object Craft > http://www.object-craft.com.au/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From andrewm at object-craft.com.au Thu Sep 17 03:32:19 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu, 17 Sep 2009 11:32:19 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> Message-ID: <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> >This proposal actually leads to 6 entities (3 for IPv4 and 3 for IPv6). Yes, I know - I was just trying to keep to the point. >It's still unclear to me what is gained by pulling AddressWithMask >functionality out of the current network classes. It's easy enough for >the concerned developer who to check if the entered network address >does actually have all of its host bits set to zero. It is not my >experience that this behavior is desired so often that having the >network classes behave as they do now leads to a great deal of >confusion. I think we're in a painful middle ground now - we should either go back to the idea of a single class (per protocol), or make the distinctions clear (networks are containers and addresses are singletons). Personally, I think I would be happy with a single class (but I suspect that's just my laziness speaking). However, I think the structure and discipline of three classes (per protocol) may actually make the concepts easier to understand for non-experts. A particular case in point - if you want to represent a single IP address with netmask (say an interface), you use a Network class, not an Address class. And the .network attribute returns a Address class! The reason I suggest having the Network class assert that masked bits be zero is two-fold: * it ensures the correct class is being used for the job * it ensures application-user errors are detected as early as possible I also suggest the AddressWithMask classes not have any network/container behaviours for a similar reason. If the developer needs these, the .network attribute is only a lookup away. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From peter at hda3.com Thu Sep 17 04:19:04 2009 From: peter at hda3.com (Peter Moody) Date: Wed, 16 Sep 2009 19:19:04 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> Message-ID: <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> On Wed, Sep 16, 2009 at 6:32 PM, Andrew McNamara wrote: >>This proposal actually leads to 6 entities (3 for IPv4 and 3 for IPv6). > > Yes, I know - I was just trying to keep to the point. > >>It's still unclear to me what is gained by pulling AddressWithMask >>functionality out of the current network classes. It's easy enough for >>the concerned developer who to check if the entered network address >>does actually have all of its host bits set to zero. It is not my >>experience that this behavior is desired so often that having the >>network classes behave as they do now leads to a great deal of >>confusion. > > I think we're in a painful middle ground now - we should either go back > to the idea of a single class (per protocol), or make the distinctions > clear (networks are containers and addresses are singletons). > > Personally, I think I would be happy with a single class (but I suspect > that's just my laziness speaking). However, I think the structure and > discipline of three classes (per protocol) may actually make the concepts > easier to understand for non-experts. I think this is where we disagree. I don't think the added complexity does make it any easier to understand. > A particular case in point - if you want to represent a single IP address > with netmask (say an interface), you use a Network class, not an Address > class. And the .network attribute returns a Address class! Right, and I don't see where the confusion lies. You have an address + netmask. ergo, you have a Network object. The single address that defines the base address (most commonly referred to as the network address) is an Address object. there is no netmask associated with that single address, ergo, it's an Address object. > The reason I suggest having the Network class assert that masked bits be > zero is two-fold: > > ?* it ensures the correct class is being used for the job > ?* it ensures application-user errors are detected as early as possible > > I also suggest the AddressWithMask classes not have any network/container > behaviours for a similar reason. If the developer needs these, the > .network attribute is only a lookup away. the problem I have with this approach is that it seems like a long way to go for a shortcut (of checking if addr.ip != addr.network: raise Error). Cheers, /peter > -- > Andrew McNamara, Senior Developer, Object Craft > http://www.object-craft.com.au/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From greg.ewing at canterbury.ac.nz Thu Sep 17 04:30:28 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Sep 2009 14:30:28 +1200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> Message-ID: <4AB19F44.8010102@canterbury.ac.nz> Andrew McNamara wrote: > I also suggest the AddressWithMask classes not have any network/container > behaviours for a similar reason. If the developer needs these, the > .network attribute is only a lookup away. Another way to approach this would be for the Address object to potentially have a 'network' attribute referencing a Network object. Then there are only two classes, but three use cases are covered: 1) a Network 2) a plain, network-agnostic Address with network == None 3) an Address with an attached Network An Address could be constructed in three ways: Address(ip_number) Address(ip_number, network = ) Address(ip_number, mask = ) # constructs and attaches a suitably-masked Network instance We could also have some_network[n] return an Address referring back to the network object it was obtained from. -- Greg From greg.ewing at canterbury.ac.nz Thu Sep 17 04:48:59 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 17 Sep 2009 14:48:59 +1200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> Message-ID: <4AB1A39B.7090003@canterbury.ac.nz> Peter Moody wrote: > I don't see where the confusion lies. You have an address > + netmask. ergo, you have a Network object. The single address that > defines the base address (most commonly referred to as the network > address) is an Address object. there is no netmask associated with > that single address, ergo, it's an Address object. But if I understand correctly, you *don't* have a network, you have something representing a particular host address within a network, plus information that lets you deduce the network to which it belongs. So calling the object a 'Network' is a misnomer. What's more, I don't see the point of having a 'network' attribute without a mask, because the zero address of a network on its own doesn't define the network. You need the zero address plus a mask to do that. I'm not sure what usefulness the zero address on its own has, but if it's considered useful enough to have an attribute for it, calling it something like 'base_address' would be less confusing. -- Greg From andrewm at object-craft.com.au Thu Sep 17 05:21:26 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu, 17 Sep 2009 13:21:26 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> Message-ID: <20090917032126.66F896006DE@longblack.object-craft.com.au> >> I think we're in a painful middle ground now - we should either go back >> to the idea of a single class (per protocol), or make the distinctions >> clear (networks are containers and addresses are singletons). >> >> Personally, I think I would be happy with a single class (but I suspect >> that's just my laziness speaking). However, I think the structure and >> discipline of three classes (per protocol) may actually make the concepts >> easier to understand for non-experts. > >I think this is where we disagree. I don't think the added complexity >does make it any easier to understand. I argue that we're not actually adding any complexity: yes, we add a class (per protocol), but we then merely relocate functionality to clarify the intended use of the classes. >> A particular case in point - if you want to represent a single IP address >> with netmask (say an interface), you use a Network class, not an Address >> class. And the .network attribute returns a Address class! > >Right, and I don't see where the confusion lies. I suggest you are too close to the implementation to be surprised by it. 8-) >You have an address + netmask. ergo, you have a Network object. In a common use case, however, this instance will not represent a network at all, but an address. It will have container-like behaviour, but it should not (this is a property of networks, not addresses). So the instance will be misnamed and have behaviours that are, at best, misleading. >The single address that defines the base address (most commonly referred >to as the network address) is an Address object. there is no netmask >associated with that single address, ergo, it's an Address object. I would argue that a Network never has a single address - by definition, it has two or more nodes. A .network attribute should return a Network instance. If you want the base address, this probably should be called .base_address or just .address (to parallel the .netmask attribute). >> The reason I suggest having the Network class assert that masked bits be >> zero is two-fold: >> >> * it ensures the correct class is being used for the job >> * it ensures application-user errors are detected as early as possible >> >> I also suggest the AddressWithMask classes not have any network/container >> behaviours for a similar reason. If the developer needs these, the >> .network attribute is only a lookup away. > >the problem I have with this approach is that it seems like a long way >to go for a shortcut (of checking if addr.ip != addr.network: raise >Error). This isn't about shortcuts, but about correctness... having the Network object represent a network, and having Address objects represent end-points, and having errors discovered as early as possible. What I'm arguing here is that singletons should not simultaneously be containers - it's not pythonic, and it leads to ambiguity. The underlying IP concepts don't require it either: an IP address is a singleton, a network is a container, and there is no overlap. Yes, an address may be a member of a network, and having a reference to that network on the address object is valuable, but the address should not behave like a network. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From peter at hda3.com Thu Sep 17 05:26:27 2009 From: peter at hda3.com (Peter Moody) Date: Wed, 16 Sep 2009 20:26:27 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB1A39B.7090003@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> Message-ID: <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> On Wed, Sep 16, 2009 at 7:48 PM, Greg Ewing wrote: > Peter Moody wrote: >> >> I don't see where the confusion lies. ?You have an address >> + netmask. ergo, you have a Network object. ?The single address that >> defines the base address (most commonly referred to as the network >> address) is an Address object. there is no netmask associated with >> that single address, ergo, it's an Address object. > > But if I understand correctly, you *don't* have a network, > you have something representing a particular host address > within a network, plus information that lets you deduce the > network to which it belongs. So calling the object a > 'Network' is a misnomer. > > What's more, I don't see the point of having a 'network' > attribute without a mask, because the zero address of a > network on its own doesn't define the network. You need > the zero address plus a mask to do that. > > I'm not sure what usefulness the zero address on its own > has, but if it's considered useful enough to have an > attribute for it, calling it something like 'base_address' > would be less confusing. the address with all of the hosts bits masked to zero is most commonly referred to as the network address. same as the address with all of the host bits set to one is called the broadcast address. calling it something like base_address or min_address will cause quite a bit more confusion. > > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From andrewm at object-craft.com.au Thu Sep 17 05:27:43 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu, 17 Sep 2009 13:27:43 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB19F44.8010102@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> Message-ID: <20090917032743.A231B6006DE@longblack.object-craft.com.au> >Another way to approach this would be for the Address object to >potentially have a 'network' attribute referencing a Network object. Yes - that's reasonable. >Then there are only two classes, but three use cases are covered: > >1) a Network > >2) a plain, network-agnostic Address with network == None > >3) an Address with an attached Network > >An Address could be constructed in three ways: > > Address(ip_number) > > Address(ip_number, network = ) > > Address(ip_number, mask = ) > # constructs and attaches a suitably-masked Network instance I think you still need to support the common notations: Address('10.0.0.1') # .network == None Address('10.0.0.1/255.255.255.0') Address('10.0.0.1/24') >We could also have some_network[n] return an Address >referring back to the network object it was obtained >from. Yes. (Of course, we're simplifying - there would really be classes for each protocol). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From peter at hda3.com Thu Sep 17 05:36:28 2009 From: peter at hda3.com (Peter Moody) Date: Wed, 16 Sep 2009 20:36:28 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917032126.66F896006DE@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> Message-ID: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara wrote: >>> I think we're in a painful middle ground now - we should either go back >>> to the idea of a single class (per protocol), or make the distinctions >>> clear (networks are containers and addresses are singletons). >>> >>> Personally, I think I would be happy with a single class (but I suspect >>> that's just my laziness speaking). However, I think the structure and >>> discipline of three classes (per protocol) may actually make the concepts >>> easier to understand for non-experts. >> >>I think this is where we disagree. I don't think the added complexity >>does make it any easier to understand. > > I argue that we're not actually adding any complexity: yes, we add > a class (per protocol), but we then merely relocate functionality to > clarify the intended use of the classes. And I argue the moving this functionality to new classes (and adding new restrictions to existing classes) doesn't buy anything in the way of overall functionality of the module or a developer's ability to comprehend intended uses. >>> A particular case in point - if you want to represent a single IP address >>> with netmask (say an interface), you use a Network class, not an Address >>> class. And the .network attribute returns a Address class! >> >>Right, and I don't see where the confusion lies. > > I suggest you are too close to the implementation to be surprised by it. 8-) touche :) >>You have an address + netmask. ergo, you have a Network object. > > In a common use case, however, this instance will not represent a > network at all, but an address. It will have container-like behaviour, > but it should not (this is a property of networks, not addresses). So > the instance will be misnamed and have behaviours that are, at best, > misleading. > >>The single address that defines the base address (most commonly referred >>to as the network address) is an Address object. there is no netmask >>associated with that single address, ergo, it's an Address object. > > I would argue that a Network never has a single address - by definition, > it has two or more nodes. A .network attribute should return a Network > instance. If you want the base address, this probably should be called > .base_address or just .address (to parallel the .netmask attribute). .network is shorthand for network address. are .network_address and .broadcast_address less confusing? I have to say, though, .network/.broadcast are fairly common (IPy uses .net, netaddr and ipv4 use, IIRC .network...) >>> The reason I suggest having the Network class assert that masked bits be >>> zero is two-fold: >>> >>> * it ensures the correct class is being used for the job >>> * it ensures application-user errors are detected as early as possible >>> >>> I also suggest the AddressWithMask classes not have any network/container >>> behaviours for a similar reason. If the developer needs these, the >>> .network attribute is only a lookup away. >> >>the problem I have with this approach is that it seems like a long way >>to go for a shortcut (of checking if addr.ip != addr.network: raise >>Error). > > This isn't about shortcuts, but about correctness... having the Network > object represent a network, and having Address objects represent > end-points, and having errors discovered as early as possible. Then what I don't see is the purpose of your network-only-network-object. essentially identical functionality can be obtained with the module as is w/o the added complexity of new classes. Cheers, /peter > What I'm arguing here is that singletons should not simultaneously be > containers - it's not pythonic, and it leads to ambiguity. The underlying > IP concepts don't require it either: an IP address is a singleton, a > network is a container, and there is no overlap. Yes, an address may be a > member of a network, and having a reference to that network on the address > object is valuable, but the address should not behave like a network. > > -- > Andrew McNamara, Senior Developer, Object Craft > http://www.object-craft.com.au/ > From peter at hda3.com Thu Sep 17 05:41:47 2009 From: peter at hda3.com (Peter Moody) Date: Wed, 16 Sep 2009 20:41:47 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> Message-ID: <8517e9350909162041n1685f80cmab96412fa3e47cb4@mail.gmail.com> On Wed, Sep 16, 2009 at 8:36 PM, Peter Moody wrote: > On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara > wrote: >>>> I think we're in a painful middle ground now - we should either go back >>>> to the idea of a single class (per protocol), or make the distinctions >>>> clear (networks are containers and addresses are singletons). >>>> >>>> Personally, I think I would be happy with a single class (but I suspect >>>> that's just my laziness speaking). However, I think the structure and >>>> discipline of three classes (per protocol) may actually make the concepts >>>> easier to understand for non-experts. >>> >>>I think this is where we disagree. I don't think the added complexity >>>does make it any easier to understand. >> >> I argue that we're not actually adding any complexity: yes, we add >> a class (per protocol), but we then merely relocate functionality to >> clarify the intended use of the classes. > > And I argue the moving this functionality to new classes (and adding > new restrictions to existing classes) doesn't buy anything in the way > of overall functionality of the module or a developer's ability to > comprehend intended uses. > >>>> A particular case in point - if you want to represent a single IP address >>>> with netmask (say an interface), you use a Network class, not an Address >>>> class. And the .network attribute returns a Address class! >>> >>>Right, and I don't see where the confusion lies. >> >> I suggest you are too close to the implementation to be surprised by it. 8-) > > touche :) > >>>You have an address + netmask. ergo, you have a Network object. >> >> In a common use case, however, this instance will not represent a >> network at all, but an address. It will have container-like behaviour, >> but it should not (this is a property of networks, not addresses). So >> the instance will be misnamed and have behaviours that are, at best, >> misleading. >> >>>The single address that defines the base address (most commonly referred >>>to as the network address) is an Address object. there is no netmask >>>associated with that single address, ergo, it's an Address object. >> >> I would argue that a Network never has a single address - by definition, >> it has two or more nodes. A .network attribute should return a Network >> instance. If you want the base address, this probably should be called >> .base_address or just .address (to parallel the .netmask attribute). > > .network is shorthand for network address. are .network_address and > .broadcast_address less confusing? ?I have to say, though, > .network/.broadcast are fairly common (IPy uses .net, netaddr and ipv4 > use, IIRC .network...) > >>>> The reason I suggest having the Network class assert that masked bits be >>>> zero is two-fold: >>>> >>>> * it ensures the correct class is being used for the job >>>> * it ensures application-user errors are detected as early as possible >>>> >>>> I also suggest the AddressWithMask classes not have any network/container >>>> behaviours for a similar reason. If the developer needs these, the >>>> .network attribute is only a lookup away. >>> >>>the problem I have with this approach is that it seems like a long way >>>to go for a shortcut (of checking if addr.ip != addr.network: raise >>>Error). >> >> This isn't about shortcuts, but about correctness... having the Network >> object represent a network, and having Address objects represent >> end-points, and having errors discovered as early as possible. > > Then what I don't see is the purpose of your > network-only-network-object. essentially identical functionality can > be obtained with the module as is w/o the added complexity of new > classes. Since you mentioned correctness; it seems like we're coming back to an option to the IPv?Network().__init__ methods which strictly checks for host bits being masked to zero. this provides, I think, the functionality/error checking you're looking for w/o the need for new classes. Cheers, /peter > Cheers, > /peter > >> What I'm arguing here is that singletons should not simultaneously be >> containers - it's not pythonic, and it leads to ambiguity. The underlying >> IP concepts don't require it either: an IP address is a singleton, a >> network is a container, and there is no overlap. Yes, an address may be a >> member of a network, and having a reference to that network on the address >> object is valuable, but the address should not behave like a network. >> >> -- >> Andrew McNamara, Senior Developer, Object Craft >> http://www.object-craft.com.au/ >> > From scott+python-dev at scottdial.com Thu Sep 17 05:53:12 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Wed, 16 Sep 2009 23:53:12 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> Message-ID: <4AB1B2A8.9050204@scottdial.com> Peter Moody wrote: > On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara > wrote: >>>> I think we're in a painful middle ground now - we should either go back >>>> to the idea of a single class (per protocol), or make the distinctions >>>> clear (networks are containers and addresses are singletons). >>>> >>>> Personally, I think I would be happy with a single class (but I suspect >>>> that's just my laziness speaking). However, I think the structure and >>>> discipline of three classes (per protocol) may actually make the concepts >>>> easier to understand for non-experts. >>> I think this is where we disagree. I don't think the added complexity >>> does make it any easier to understand. >> I argue that we're not actually adding any complexity: yes, we add >> a class (per protocol), but we then merely relocate functionality to >> clarify the intended use of the classes. > > And I argue the moving this functionality to new classes (and adding > new restrictions to existing classes) doesn't buy anything in the way > of overall functionality of the module or a developer's ability to > comprehend intended uses. Speaking as the originator of this thread of discourse, the lack of a 3rd class was exactly the source of my confusion and my inability to communicate my confusion to everyone. I clearly did not understand the intended uses of the IPNetwork type because it was capable of playing two roles that are decidedly different conceptually. So, I must respectfully disagree with you. >>> You have an address + netmask. ergo, you have a Network object. >> In a common use case, however, this instance will not represent a >> network at all, but an address. It will have container-like behaviour, >> but it should not (this is a property of networks, not addresses). So >> the instance will be misnamed and have behaviours that are, at best, >> misleading. This is exactly the confusion and duality that I struggled with. >> This isn't about shortcuts, but about correctness... having the Network >> object represent a network, and having Address objects represent >> end-points, and having errors discovered as early as possible. > > Then what I don't see is the purpose of your > network-only-network-object. essentially identical functionality can > be obtained with the module as is w/o the added complexity of new > classes. The purpose is to avoid conflating IPNetwork with an IPAddress that has a mask. If the IPNetwork didn't accept a non-zero host and instead required a developer to use a helper to construct a IPNetwork with a proper address, then there would be less confusion about what exactly a IPNetwork is meant to represent. As it stands, it's purposes is muddled by accepting host addresses too. I am rather indifferent whether there needs to be a IPAddressWithMask type. If that is needed, then it is rather easy to create a type that does that. And, if it is a common pattern, then it could be added to the module later in life. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From tseaver at palladion.com Thu Sep 17 06:14:12 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 17 Sep 2009 00:14:12 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB1B2A8.9050204@scottdial.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> <4AB1B2A8.9050204@scottdial.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Scott Dial wrote: > The purpose is to avoid conflating IPNetwork with an IPAddress that has > a mask. I'm not confused by anything in this discussion except the repeated harping on the (to me imaginary) concept of "address with a mask". Conceptually: - Addresses don't have masks: they are just 32- (128-) bit integers. - Any given address can belong to a number of networks, based on a given topology. - Masks are also 32- (128-) bit integers, which happen to have the property that their leftmost N bits are all zero and the rest are all one. - One can construct a network from the combination of an address and a mask. Such a network *never* needs to know what the address was before masking. - Networks *do* know their "zero" address, as well as their "last address", and can do "containment" operations on addresses (I'm not clear on the need to do such operations on other networks, rather than just using their zero addresses). At the API level, it is convenient to represent the combination (address, mask) as a single string (e.g., '192.168.1.0/24'): parsing that string should return a network, not an address. Let's keep the "parsing human readable / writable strings" part separate from the clear conceptual map. > If the IPNetwork didn't accept a non-zero host and instead > required a developer to use a helper to construct a IPNetwork with a > proper address, then there would be less confusion about what exactly a > IPNetwork is meant to represent. As it stands, it's purposes is muddled > by accepting host addresses too. I can't imagine why it would be useful to try to infer a network from an address: "In the face of ambiguity, refuse the temptation to guess." > I am rather indifferent whether there needs to be a IPAddressWithMask > type. If that is needed, then it is rather easy to create a type that > does that. And, if it is a common pattern, then it could be added to the > module later in life. Agreed. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFKsbeU+gerLs4ltQ4RAlG0AJ9ZvhuXHsTL2hheW/vlzeMmArs5rgCeLDLS eKWSAdkdv++umepu+nK0/7I= =N7ej -----END PGP SIGNATURE----- From andrewm at object-craft.com.au Thu Sep 17 06:17:45 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu, 17 Sep 2009 14:17:45 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> Message-ID: <20090917041745.C331B6006DE@longblack.object-craft.com.au> >> I argue that we're not actually adding any complexity: yes, we add >> a class (per protocol), but we then merely relocate functionality to >> clarify the intended use of the classes. > >And I argue the moving this functionality to new classes (and adding >new restrictions to existing classes) doesn't buy anything in the way >of overall functionality of the module or a developer's ability to >comprehend intended uses. It's mostly just minor refactoring and renaming, which I think makes things clearer, although I agree this is merely an opinion. I would be interest to hear what others think. To summarise: * an address is a singleton (a network endpoint), with no container behaviour. It may optionally reference it's network (via the .network attribute), .address returns mask-less address. * a network is a container-like object. For consistency, .network should return self and raise an exception if the mask conflicts with the address, .address returns the base address, .mask returns an address object. >> I would argue that a Network never has a single address - by definition, >> it has two or more nodes. A .network attribute should return a Network >> instance. If you want the base address, this probably should be called >> .base_address or just .address (to parallel the .netmask attribute). > >.network is shorthand for network address. are .network_address and >.broadcast_address less confusing? I have to say, though, >.network/.broadcast are fairly common (IPy uses .net, netaddr and ipv4 >use, IIRC .network...) Yes, I understand your motivation, but I still think it's going to be more confusing the way you have it. >> This isn't about shortcuts, but about correctness... having the Network >> object represent a network, and having Address objects represent >> end-points, and having errors discovered as early as possible. > >Then what I don't see is the purpose of your >network-only-network-object. essentially identical functionality can >be obtained with the module as is w/o the added complexity of new >classes. Certainly, I'm not talking about adding functionality. What I am suggesting is that if we wish to have a distinction between networks and addresses, then that distinction should be clear and strong, such that the choice of which to use is obvious, and if the wrong one is used, the error is discovered as early as possible. As the module stands, we have a pair of address-without-mask classes called *Address, and a pair of address-with-mask classes called *Network. So, sometimes when you want to record an *address* you use a class called Network, and that class comes with a behaviours that make no sense in the context of a singleton network end-point (it can't "contain" other addresses, although it's .network can). Sorry if I sound like a cracked record - these are subtle concepts, and my ability to explain what I mean is less than is needed, but we'll get there in the end. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From stephen at xemacs.org Thu Sep 17 08:51:25 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Thu, 17 Sep 2009 15:51:25 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917041745.C331B6006DE@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> <20090917041745.C331B6006DE@longblack.object-craft.com.au> Message-ID: <87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp> Andrew McNamara writes: > As the module stands, we have a pair of address-without-mask classes > called *Address, and a pair of address-with-mask classes called > *Network. So, sometimes when you want to record an *address* you use > a class called Network, and that class comes with a behaviours that > make no sense in the context of a singleton network end-point (it can't > "contain" other addresses, although it's .network can). I'm going to consistently use "address" to mean a singleton and "network" to mean a container in the following. I still don't see why an address-with-mask is useful, except that the network is deducible as {'network': address & mask, 'mask': mask}. Is there *any* other way you would *ever* use that? It seems to me that for some purposes (implementing dig(1), for example), an IPv4Address can contain only the address (ie, a 32-bit integer) as a data attribute, and (with methods for using that attribute) that is the minimal implementation of IPv4Address. However, there are other cases (eg, routing) where it's useful to associate an address with its network, and I don't see much harm in doing so by adding a 'network' attribute to the base class IPv4Address, since addresses are hardly useful except in the context of networks. Of course that attribute is often going to be None (eg, in implementing dig(1) the remote nameserver is unlikely to tell you the netmask). However, when iterating over an IPv4Network, the iterator can automatically fill in the 'network' attribute, and that's fairly cheap. While to me neither the 'network' attribute nor the iterator behavior just described seems amazing useful in the base classes, it seems to me that precisely those behaviors will be reinvented over and over again for derived classes. Furthermore they are natural enough that they won't bother people who don't need them. (That's despite at least one person (IIRC it was Antoine) firmly saying "an IPv4Address should contain exactly one 32-bit int, no more, no less", so I could be wrong.) It seems to me that the only good reason for not having a 'network' attribute that contains an IPv4Network instance or None is efficiency: the space for the attribute and the overhead of filling it in the iterator. I personally can't think of an application that would care (from what I hear, Cisco has no interest in writing its routers' IP stacks in Python, amazingly enough), but in theory ... Finally, I agree that using IPv4Network as address-with-mask is a confusing, undiscoverable abuse. In particular, I think that every time I went a week without using that idiom, I'd get nervous when I saw it again: "Are you *sure* that won't raise an error or silently get the lower bits masked off?! If not now, in the next version?" Obviously all the above applies to IPv6 classes, too. From andrewm at object-craft.com.au Thu Sep 17 09:15:16 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Thu, 17 Sep 2009 17:15:16 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> <20090917041745.C331B6006DE@longblack.object-craft.com.au> <87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <20090917071516.217655AC0D6@longblack.object-craft.com.au> > > As the module stands, we have a pair of address-without-mask classes > > called *Address, and a pair of address-with-mask classes called > > *Network. So, sometimes when you want to record an *address* you use > > a class called Network, and that class comes with a behaviours that > > make no sense in the context of a singleton network end-point (it can't > > "contain" other addresses, although it's .network can). > >I'm going to consistently use "address" to mean a singleton and >"network" to mean a container in the following. Ta. I think it's useful to have a common terminology. >I still don't see why an address-with-mask is useful, except that the >network is deducible as {'network': address & mask, 'mask': mask}. Is >there *any* other way you would *ever* use that? > >It seems to me that for some purposes (implementing dig(1), for >example), an IPv4Address can contain only the address (ie, a 32-bit >integer) as a data attribute, and (with methods for using that >attribute) that is the minimal implementation of IPv4Address. > >However, there are other cases (eg, routing) where it's useful to >associate an address with its network, and I don't see much harm in >doing so by adding a 'network' attribute to the base class >IPv4Address, since addresses are hardly useful except in the context >of networks. Of course that attribute is often going to be None (eg, >in implementing dig(1) the remote nameserver is unlikely to tell you >the netmask). However, when iterating over an IPv4Network, the >iterator can automatically fill in the 'network' attribute, and that's >fairly cheap. Conceptually, you sometimes need a bare address, and other times, you need an address with an associated network (host interface configs, router configs, etc). By AddressWithMask, I really mean AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently, IPv4 and IPv6 addressing allows us to derive the network from the host address combined with the netmask - in other words, we don't have to attach a real Network object to Address objects until the user tries to access it, and then we derive it from the address and mask. >While to me neither the 'network' attribute nor the iterator behavior >just described seems amazing useful in the base classes, it seems to >me that precisely those behaviors will be reinvented over and over >again for derived classes. Furthermore they are natural enough that >they won't bother people who don't need them. (That's despite at >least one person (IIRC it was Antoine) firmly saying "an IPv4Address >should contain exactly one 32-bit int, no more, no less", so I could >be wrong.) If you have a .network attribute on an address object, checking if an address is in the same network as another address becomes: addr_a in addr_b.network As the module stands, you write that as: addr_a in addr_b I don't think the intent is as clear with the later. >It seems to me that the only good reason for not having a >'network' attribute that contains an IPv4Network instance or None is >efficiency: the space for the attribute and the overhead of filling it >in the iterator. I personally can't think of an application that >would care (from what I hear, Cisco has no interest in writing its >routers' IP stacks in Python, amazingly enough), but in theory ... The implementation already lazily creates most things like this. >Finally, I agree that using IPv4Network as address-with-mask is a >confusing, undiscoverable abuse. In particular, I think that every >time I went a week without using that idiom, I'd get nervous when I >saw it again: "Are you *sure* that won't raise an error or silently >get the lower bits masked off?! If not now, in the next version?" Yes. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From p.f.moore at gmail.com Thu Sep 17 09:33:35 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 17 Sep 2009 08:33:35 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> Message-ID: <79990c6b0909170033p2b1897c2s8ad00229a0ce2dcd@mail.gmail.com> 2009/9/17 Peter Moody : > On Wed, Sep 16, 2009 at 8:21 PM, Andrew McNamara > wrote: >>>> I think we're in a painful middle ground now - we should either go back >>>> to the idea of a single class (per protocol), or make the distinctions >>>> clear (networks are containers and addresses are singletons). >>>> >>>> Personally, I think I would be happy with a single class (but I suspect >>>> that's just my laziness speaking). However, I think the structure and >>>> discipline of three classes (per protocol) may actually make the concepts >>>> easier to understand for non-experts. >>> >>>I think this is where we disagree. I don't think the added complexity >>>does make it any easier to understand. >> >> I argue that we're not actually adding any complexity: yes, we add >> a class (per protocol), but we then merely relocate functionality to >> clarify the intended use of the classes. > > And I argue the moving this functionality to new classes (and adding > new restrictions to existing classes) doesn't buy anything in the way > of overall functionality of the module or a developer's ability to > comprehend intended uses. For what it's worth, Andrew's making the distinction clatified things for *me*. Whether I am the target audience for the module (I'm not a network expert, but on the odd occasion I need to manipulate IP addresses in a program, and if the standard library had a module for doing so, I'd reach for it) is up to someone else to decide, but if I am, then my vote is for 3 classes based on the explanations I've seen so far. I don't see what having fewer classes buys me as the end user. Paul. From techtonik at gmail.com Thu Sep 17 12:09:20 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Thu, 17 Sep 2009 13:09:20 +0300 Subject: [Python-Dev] Roundup dependency tree changes Message-ID: Hello, How about allowing all authenticated users update dependency field in Python tracker? http://bugs.python.org/issue?@sort0=dependencies&@sortdir0=on&@sort1=&@group0=priority&@group1=&@columns=title,id,activity,dependencies,status&@pagesize=150&@startwith=0 Looks like developers with access don't have enough time to track issue dependencies. --anatoly t. From solipsis at pitrou.net Thu Sep 17 13:49:57 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 17 Sep 2009 11:49:57 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > the address with all of the hosts bits masked to zero is most commonly > referred to as the network address. same as the address with all of > the host bits set to one is called the broadcast address. calling it > something like base_address or min_address will cause quite a bit more > confusion. Quite a bit less IMO. I'm not a network specialist, but the "network address" of a network sounds quite vague to me, while things like "lower_bound", "first", "first_address", "firstip" are immediately understandable and non-ambiguous. From solipsis at pitrou.net Thu Sep 17 13:58:09 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 17 Sep 2009 11:58:09 +0000 (UTC) Subject: [Python-Dev] conceptual clarity References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> Message-ID: Le Thu, 17 Sep 2009 14:30:28 +1200, Greg Ewing a ?crit?: > > 3) an Address with an attached Network > > An Address could be constructed in three ways: > > Address(ip_number) > > Address(ip_number, network = ) > > Address(ip_number, mask = ) > # constructs and attaches a suitably-masked Network instance > > We could also have some_network[n] return an Address referring back to > the network object it was obtained from. It seems you are uselessly conflating two perfectly distinct concepts: Address and Network. You also haven't addresses the issue of comparing together (and hashing) two addresses with the same IP but pointing to a different network. No answer to this issue seems satisfactory and obviously right. As it is, -1 from me. Either we only keep two concepts (Address and Network), or if we introduce a third one (AddressWithMask, whatever) for added practicality; but we shouldn't blur the line between the two former canonical concepts under the pretext that a platypus-like Address might be helpful in some particular situations. Regards Antoine. From eric at trueblade.com Thu Sep 17 14:14:34 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 17 Sep 2009 08:14:34 -0400 Subject: [Python-Dev] conceptual clarity In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> Message-ID: <4AB2282A.3050902@trueblade.com> Antoine Pitrou wrote: > Le Thu, 17 Sep 2009 14:30:28 +1200, Greg Ewing a ?crit : >> 3) an Address with an attached Network >> >> An Address could be constructed in three ways: >> >> Address(ip_number) >> >> Address(ip_number, network = ) >> >> Address(ip_number, mask = ) >> # constructs and attaches a suitably-masked Network instance >> >> We could also have some_network[n] return an Address referring back to >> the network object it was obtained from. > > It seems you are uselessly conflating two perfectly distinct concepts: > Address and Network. You also haven't addresses the issue of comparing > together (and hashing) two addresses with the same IP but pointing to a > different network. No answer to this issue seems satisfactory and > obviously right. > > As it is, -1 from me. Either we only keep two concepts (Address and > Network), or if we introduce a third one (AddressWithMask, whatever) for > added practicality; but we shouldn't blur the line between the two former > canonical concepts under the pretext that a platypus-like Address might > be helpful in some particular situations. I completely agree with this. By keeping the concepts distinct we can catch mis-uses earlier. If needed, convenience functions (or classes, or whatever) could be layered on top. But the underlying concepts need to be clear. Eric. From ncoghlan at gmail.com Thu Sep 17 14:32:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 17 Sep 2009 22:32:49 +1000 Subject: [Python-Dev] conceptual clarity In-Reply-To: <4AB2282A.3050902@trueblade.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> Message-ID: <4AB22C71.1040205@gmail.com> Eric Smith wrote: > Antoine Pitrou wrote: >> As it is, -1 from me. Either we only keep two concepts (Address and >> Network), or if we introduce a third one (AddressWithMask, >> whatever) for added practicality; but we shouldn't blur the line >> between the two former canonical concepts under the pretext that a >> platypus-like Address might be helpful in some particular >> situations. > > I completely agree with this. By keeping the concepts distinct we can > catch mis-uses earlier. If needed, convenience functions (or > classes, or whatever) could be layered on top. But the underlying > concepts need to be clear. That would be my view as well (this includes getting rid of the current duality of IPNetwork by dropping the ip property, only accepting the network address in the normal constructor and having a separate constructor that allows a network object to be created from an arbitrary host address and a netmask) Is-a-2.x-str-a-sequence-of-ASCII-characters-or-a-chunk-of-binary-data?'ly, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From eric at trueblade.com Thu Sep 17 14:39:57 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 17 Sep 2009 08:39:57 -0400 Subject: [Python-Dev] conceptual clarity In-Reply-To: <4AB22C71.1040205@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> Message-ID: <4AB22E1D.30907@trueblade.com> Nick Coghlan wrote: > Eric Smith wrote: >> Antoine Pitrou wrote: >>> As it is, -1 from me. Either we only keep two concepts (Address and >>> Network), or if we introduce a third one (AddressWithMask, >>> whatever) for added practicality; but we shouldn't blur the line >>> between the two former canonical concepts under the pretext that a >>> platypus-like Address might be helpful in some particular >>> situations. >> I completely agree with this. By keeping the concepts distinct we can >> catch mis-uses earlier. If needed, convenience functions (or >> classes, or whatever) could be layered on top. But the underlying >> concepts need to be clear. > > That would be my view as well (this includes getting rid of the current > duality of IPNetwork by dropping the ip property, only accepting the > network address in the normal constructor and having a separate > constructor that allows a network object to be created from an arbitrary > host address and a netmask) Yes, I think that's the best plan. I could live without AddressWithMask (or whatever name), but it would be a nice convenience. > Is-a-2.x-str-a-sequence-of-ASCII-characters-or-a-chunk-of-binary-data?'ly, LOL! Perfect. Eric. From ncoghlan at gmail.com Thu Sep 17 14:40:47 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 17 Sep 2009 22:40:47 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> Message-ID: <4AB22E4F.40509@gmail.com> Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> the address with all of the hosts bits masked to zero is most commonly >> referred to as the network address. same as the address with all of >> the host bits set to one is called the broadcast address. calling it >> something like base_address or min_address will cause quite a bit more >> confusion. > > Quite a bit less IMO. I'm not a network specialist, but the "network address" Nah, network address is perfectly explicit - it's what you get when you bitwise and any host within that network with the netmask. Where it becomes confusing is that we have a property called "network" that returns an IPAddress object rather than an IPNetwork object. People that are network specialists would grasp that fairly easily, but we try to design standard library APIs so that they're novice friendly as well. And the above situation isn't novice friendly. To be honest, given the indexing behaviour, I'm -1 on the idea of giving the network address or broadcast address attribute names *at all*. Consider: network_address = my_net[0] broadcast_address = my_net[-1] The only way the network address could match the number of characters in the indexing method is if you just called it the network id (i.e. my_net.id). For the broadcast address, there is no name that even comes close to matching the indexing approach for brevity. And since IPNetwork is a container for IPAddress instances, the indexing approach means there is zero ambiguity regarding the return type. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From rdmurray at bitdance.com Thu Sep 17 14:49:29 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 08:49:29 -0400 (EDT) Subject: [Python-Dev] conceptual clarity In-Reply-To: <4AB22C71.1040205@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> Message-ID: On Thu, 17 Sep 2009 at 22:32, Nick Coghlan wrote: > Eric Smith wrote: >> Antoine Pitrou wrote: >>> As it is, -1 from me. Either we only keep two concepts (Address and >>> Network), or if we introduce a third one (AddressWithMask, >>> whatever) for added practicality; but we shouldn't blur the line >>> between the two former canonical concepts under the pretext that a >>> platypus-like Address might be helpful in some particular >>> situations. >> >> I completely agree with this. By keeping the concepts distinct we can >> catch mis-uses earlier. If needed, convenience functions (or >> classes, or whatever) could be layered on top. But the underlying >> concepts need to be clear. > > That would be my view as well (this includes getting rid of the current > duality of IPNetwork by dropping the ip property, only accepting the > network address in the normal constructor and having a separate > constructor that allows a network object to be created from an arbitrary > host address and a netmask) I agree as well. If we cannot get consensus on adding a third (or rather fifth and sixth including IPv6) classes for AddressWithMask (and I am no longer an advocate of doing so), then we leave it out and, as suggested by Scott, wait and see what develops and add it later if there is demand for it. --David From rdmurray at bitdance.com Thu Sep 17 14:58:08 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 08:58:08 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> Message-ID: On Wed, 16 Sep 2009 at 20:26, Peter Moody wrote: > On Wed, Sep 16, 2009 at 7:48 PM, Greg Ewing wrote: >> I'm not sure what usefulness the zero address on its own >> has, but if it's considered useful enough to have an >> attribute for it, calling it something like 'base_address' >> would be less confusing. > > the address with all of the hosts bits masked to zero is most commonly > referred to as the network address. same as the address with all of > the host bits set to one is called the broadcast address. calling it > something like base_address or min_address will cause quite a bit more > confusion. If 'network address' is the most common designation for this number (and it may well be), then 'network.address' and 'network.broadcast' would seem to be the best choices for names. But there is definitely an attraction to the idea (I think it was Nick's?) of dropping these attributes entirely and just using indexing to get the addresses. That will make perfect sense to network professionals, and by putting appropriate words in the documentation will educate people new to IP addressing concepts. The advantage is that it is completely unambiguous: the network address really is the first address in the network range, and the broadcast really is the last, in common usage. Exceptional cases can then be handled by custom subclasses, without having someone who has to handle weird broadcast addresses (for example) submitting a bug request to make '.broadcast' settable. --David From eric at trueblade.com Thu Sep 17 15:25:45 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 17 Sep 2009 09:25:45 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB22E4F.40509@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> Message-ID: <4AB238D9.1020204@trueblade.com> Nick Coghlan wrote: > To be honest, given the indexing behaviour, I'm -1 on the idea of giving > the network address or broadcast address attribute names *at all*. Consider: > > network_address = my_net[0] > broadcast_address = my_net[-1] My only concern with this is a possible performance issue with v6 networks. Would this be implemented such that [-1] doesn't need to iterate through the (possibly large) address space of a v6 network? I think it could (and would) be implemented efficiently, so I'm +1 on not giving these names, but adding this "recipe" to the documentation instead. Eric. From drkjam at gmail.com Thu Sep 17 16:26:16 2009 From: drkjam at gmail.com (DrKJam) Date: Thu, 17 Sep 2009 15:26:16 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> <538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com> Message-ID: <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com> Please can we have the following RFCs added to the references section that cover many of the aspects covered by this PEP? RFC 791 - Internet Protocol RFC 1918 - Address Allocation for Private Internets RFC 3330 - Special-Use IPv4 Addresses RFC 4291 - IPv6 Addressing Architecture RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address Dave M. -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Thu Sep 17 16:32:05 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 17 Sep 2009 14:32:05 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit?: > Folks, Guido, > > I believe PEP 3144 is ready for your review. When you get a chance, can > you take a look/make a pronouncement? I might add that, according to the whole discussion, it seems not all of the API is exposed in the PEP. Would it be possible to cover more of the API in the PEP, to be sure we don't overlook anything? From daniel at stutzbachenterprises.com Thu Sep 17 16:37:56 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Thu, 17 Sep 2009 09:37:56 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> <538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com> <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com> Message-ID: On Thu, Sep 17, 2009 at 9:26 AM, DrKJam wrote: > Please can we have the following RFCs added to the references section that > cover many of the aspects covered by this PEP? > > RFC 791 - Internet Protocol > RFC 1918 - Address Allocation for Private Internets > RFC 3330 - Special-Use IPv4 Addresses > RFC 4291 - IPv6 Addressing Architecture > RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address > +1 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter at hda3.com Thu Sep 17 16:40:47 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 07:40:47 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> <538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com> <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com> Message-ID: <8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com> On Thu, Sep 17, 2009 at 7:26 AM, DrKJam wrote: > Please can we have the following RFCs added to the references section that > cover many of the aspects covered by this PEP? > > RFC 791 - Internet Protocol > RFC 1918 - Address Allocation for Private Internets > RFC 3330 - Special-Use IPv4 Addresses > RFC 4291 - IPv6 Addressing Architecture > RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address Sure, can you give me an idea of what specifically you're looking to the rfc's to clarify? simply adding 5 rfc's to the the PEP wouldn't seem to enhance its readability. > Dave M. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > > From drkjam at gmail.com Thu Sep 17 16:44:04 2009 From: drkjam at gmail.com (DrKJam) Date: Thu, 17 Sep 2009 15:44:04 +0100 Subject: [Python-Dev] Fwd: PEP 3144 review. In-Reply-To: <538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com> Message-ID: <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com> > > Antoine Pitrou wrote: > > Peter Moody hda3.com> writes: > >> the address with all of the hosts bits masked to zero is most commonly > >> referred to as the network address. same as the address with all of > >> the host bits set to one is called the broadcast address. calling it > >> something like base_address or min_address will cause quite a bit more > >> confusion. > > > > Quite a bit less IMO. I'm not a network specialist, but the "network > address" > > Nah, network address is perfectly explicit - it's what you get when you > bitwise and any host within that network with the netmask. > > Where it becomes confusing is that we have a property called "network" > that returns an IPAddress object rather than an IPNetwork object. > > People that are network specialists would grasp that fairly easily, but > we try to design standard library APIs so that they're novice friendly > as well. And the above situation isn't novice friendly. > > To be honest, given the indexing behaviour, I'm -1 on the idea of giving > the network address or broadcast address attribute names *at all*. > Consider: > > network_address = my_net[0] > broadcast_address = my_net[-1] > > The only way the network address could match the number of characters in > the indexing method is if you just called it the network id (i.e. > my_net.id). For the broadcast address, there is no name that even comes > close to matching the indexing approach for brevity. > > And since IPNetwork is a container for IPAddress instances, the indexing > approach means there is zero ambiguity regarding the return type. > > -1 from me. I'm happy to see indexing made available alongside specific properties/methods that expose the network (without mask) and broadcast addresses for a given network object, but not to remove them from the interface entirely in favour of indexing alone. While it seems like a good idea, in practice it just won't work or will at least be sub-optimal and violates the rule of least surprise (for IP libraries anyway). I struggled along with this approach in several early versions of netaddr but had to cave in to pressure from users after repeated questions about where to find the broadcast and network (without mask) addresses! Granted, there are decisions to be made about exactly what the properties/methods should be named to avoid ambiguity, but they are important enough to be given access to in their own right. Details in docstrings help too ;-) 'network' and 'broadcast' are very much the convention used pretty much everywhere (including libraries found in other languages such as Ruby and Perl). IPv6 doesn't support the notion of a broadcast address as part of a CIDR network block at all. AFAIK, it is a perfect legitimate for the last address in an IPv6 block to be used to configure a network interface. The IPv6 network object interface should possibly leave out the broadcast property/method altogether although there are reasons to keep it in for the sake of completeness and API consistency. The pros and cons of this need to be considered. BTW, has anyone considered use of the term *CIDR to refer to an address + mask object? It would certainly be less controversial than *Network which already has too many overlapping meanings elsewhere in the interface? Obviously we'd still have the issue of what to do with the host bits to the right of the supplied mask (keep or discard). This is not a clear cut choice of one or the other as it is entirely based on context. For configuring routes, you would likely always want to discard these bits (or at least Cisco does when adding routes). For configuring a network interface you would most certainly want to keep them! > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/drkjam%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter at hda3.com Thu Sep 17 16:38:08 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 07:38:08 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: <8517e9350909170738j6a5b805fp619006eb0a8bf5bb@mail.gmail.com> On Thu, Sep 17, 2009 at 7:32 AM, Antoine Pitrou wrote: > Le Mon, 14 Sep 2009 09:44:12 -0700, Peter Moody a ?crit?: >> Folks, Guido, >> >> I believe PEP 3144 is ready for your review. ?When you get a chance, can >> you take a look/make a pronouncement? > > I might add that, according to the whole discussion, it seems not all of > the API is exposed in the PEP. Would it be possible to cover more of the > API in the PEP, to be sure we don't overlook anything? sounds good. todo += more api in the pep Cheers, /peter > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From peter at hda3.com Thu Sep 17 16:48:23 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 07:48:23 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB238D9.1020204@trueblade.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> Message-ID: <8517e9350909170748g7ec8b1a0u926e26de6907b494@mail.gmail.com> On Thu, Sep 17, 2009 at 6:25 AM, Eric Smith wrote: > Nick Coghlan wrote: >> >> To be honest, given the indexing behaviour, I'm -1 on the idea of giving >> the network address or broadcast address attribute names *at all*. >> Consider: >> >> ?network_address = my_net[0] >> ?broadcast_address = my_net[-1] > > My only concern with this is a possible performance issue with v6 networks. > Would this be implemented such that [-1] doesn't need to iterate through the > (possibly large) address space of a v6 network? > > I think it could (and would) be implemented efficiently, so I'm +1 on not > giving these names, but adding this "recipe" to the documentation instead. indexing is plenty efficient, but the problem is that these names for these attributes are common to the point of causing confusion if they're omitted. Cheers, /peter > Eric. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From eric at trueblade.com Thu Sep 17 16:54:49 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 17 Sep 2009 10:54:49 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909170748g7ec8b1a0u926e26de6907b494@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> <8517e9350909170748g7ec8b1a0u926e26de6907b494@mail.gmail.com> Message-ID: <4AB24DB9.9050605@trueblade.com> Peter Moody wrote: > indexing is plenty efficient, but the problem is that these names for > these attributes are common to the point of causing confusion if > they're omitted. After thinking about it some more, I'm okay with names for [-1] and [0]. I like .broadcast, and I can live with .network (although it's confusing, since in my view it should return an address, not a network). Eric. From solipsis at pitrou.net Thu Sep 17 16:56:57 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 17 Sep 2009 14:56:57 +0000 (UTC) Subject: [Python-Dev] Fwd: PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com> <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com> Message-ID: DrKJam gmail.com> writes: > > BTW, has anyone considered use of the term *CIDR to refer to an address + mask object? This sounds too obscure for a non-specialist. > Obviously we'd still have the issue of what to do with the host bits to the right of the supplied mask (keep or discard). This is not a clear cut choice of one or the other as it is entirely based on context. For configuring routes, you would likely always want to discard these bits (or at least Cisco does when adding routes). For configuring a network interface you would most certainly want to keep them! When configuring a network interface you are not specifying a network, you are specifying two things: - a host address to which the interface is bound (all bits significant) - a network to which the interface is connected (lower bits non-significant) As someone pointed out, the fact that you are specifying a single string like "10.0.1.7/8" is just a notational convenience. Therefore, it seems clear to me that Network objects should ignore (and perhaps zero out) the host bits at the right of the netmask. When you configure an interface from the aforementioned notational convenience, you should retrieve two objects, a Network, and an Address. Regards Antoine. From rdmurray at bitdance.com Thu Sep 17 17:23:19 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 11:23:19 -0400 (EDT) Subject: [Python-Dev] Fwd: PEP 3144 review. In-Reply-To: <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com> <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com> Message-ID: On Thu, 17 Sep 2009 at 15:44, DrKJam wrote: > Granted, there are decisions to be made about exactly what the > properties/methods should be named to avoid ambiguity, but they are > important enough to be given access to in their own right. Details in > docstrings help too ;-) 'network' and 'broadcast' are very much the > convention used pretty much everywhere (including libraries found in other > languages such as Ruby and Perl). Interesting. Out of curiosity I searched for 'perl ip address' and found Net::IP (http://search.cpan.org/dist/Net-IP/IP.pm). This package appears to use the same object class for both addresses and networks. It has neither a 'network' nor a 'broadcast' attribute; instead it has 'ip' (it isn't clear from the docs if that returns the network address or the IP that was passed in to the constructor) and 'last_ip'. --David From nate at binkert.org Thu Sep 17 17:31:54 2009 From: nate at binkert.org (nathan binkert) Date: Thu, 17 Sep 2009 08:31:54 -0700 Subject: [Python-Dev] re.Scanner and string.Template Message-ID: <217accd40909170831p70b48c51te2273011cb516749@mail.gmail.com> I've been working on code that works a lot like string.Template and in the process, I stumbled on re.Scanner. I have two questions. 1) Is re.Scanner ever going to be official? Can I count on it being in future versions of python? It's been there for a really long time, so I assume so, but something that's undocumented seems that it could be dropped easily. 2) Is there a reason that string.Template doesn't use re.Scanner? My understanding of re.Scanner is that it is exactly what you want for something like string.Template. Thanks, Nate Binkert From benjamin at python.org Thu Sep 17 17:34:07 2009 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 17 Sep 2009 10:34:07 -0500 Subject: [Python-Dev] re.Scanner and string.Template In-Reply-To: <217accd40909170831p70b48c51te2273011cb516749@mail.gmail.com> References: <217accd40909170831p70b48c51te2273011cb516749@mail.gmail.com> Message-ID: <1afaf6160909170834o761694a6i733559cdba81f0f5@mail.gmail.com> 2009/9/17 nathan binkert : > I've been working on code that works a lot like string.Template and in > the process, I stumbled on re.Scanner. ?I have two questions. > 1) Is re.Scanner ever going to be official? ?Can I count on it being > in future versions of python? ?It's been there for a really long time, > so I assume so, but something that's undocumented seems that it could > be dropped easily. See http://bugs.python.org/issue5337. > 2) Is there a reason that string.Template doesn't use re.Scanner? ?My > understanding of re.Scanner is that it is exactly what you want for > something like string.Template. Perhaps because it wasn't documented. :) -- Regards, Benjamin From peter at hda3.com Thu Sep 17 18:16:46 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 09:16:46 -0700 Subject: [Python-Dev] conceptual clarity In-Reply-To: <4AB22C71.1040205@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> Message-ID: <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> On Thu, Sep 17, 2009 at 5:32 AM, Nick Coghlan wrote: > Eric Smith wrote: >> Antoine Pitrou wrote: >>> As it is, -1 from me. Either we only keep two concepts (Address and >>> ?Network), or if we introduce a third one (AddressWithMask, >>> whatever) for added practicality; but we shouldn't blur the line >>> between the two former canonical concepts under the pretext that a >>> platypus-like Address might be helpful in some particular >>> situations. >> >> I completely agree with this. By keeping the concepts distinct we can >> ?catch mis-uses earlier. If needed, convenience functions (or >> classes, or whatever) could be layered on top. But the underlying >> concepts need to be clear. > > That would be my view as well (this includes getting rid of the current > duality of IPNetwork by dropping the ip property, only accepting the > network address in the normal constructor and having a separate > constructor that allows a network object to be created from an arbitrary > ?host address and a netmask) Again, the same error-catching functionality can be obtained through an option to the constructor. network and broadcast attributes can be renamed to .\1_address to alleviate confusion as well. I mentioned before that IPy's insistence on receiving masked out networks was one of the main reasons I wrote ipaddr to begin with. Having ipaddr mimic this behavior would make it significantly less useful. Removing functionality in the name of avoiding confusion doesn't make sense when the same confusion can be alleviated w/o the loss. off to patch the pep and implement some of the non controversial changes. /peter > Is-a-2.x-str-a-sequence-of-ASCII-characters-or-a-chunk-of-binary-data?'ly, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From solipsis at pitrou.net Thu Sep 17 18:26:11 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 17 Sep 2009 16:26:11 +0000 (UTC) Subject: [Python-Dev] conceptual clarity References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > Again, the same error-catching functionality can be obtained through > an option to the constructor. network and broadcast attributes can be > renamed to .\1_address to alleviate confusion as well. My remark was not targetting error-catching of non-zero low bits, it was about the suggestion to have a "network" or "mask" attribute on Address objects. Regards Antoine. From peter at hda3.com Thu Sep 17 18:41:18 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 09:41:18 -0700 Subject: [Python-Dev] conceptual clarity In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> Message-ID: <8517e9350909170941o3f5ae0fdy90668d1df0515ca5@mail.gmail.com> On Thu, Sep 17, 2009 at 9:26 AM, Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> >> Again, the same error-catching functionality can be obtained through >> an option to the constructor. network and broadcast attributes can be >> renamed to .\1_address to alleviate confusion as well. > > My remark was not targetting error-catching of non-zero low bits, it was about > the suggestion to have a "network" or "mask" attribute on Address objects. If I'm reading your reply correctly, then I agree that having a network attribute on an address object doesn't make sense. My remark was targeting Eric's reply of, "By keeping the concepts distinct we can catch mis-uses earlier." Cheers, /peter > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From rdmurray at bitdance.com Thu Sep 17 19:14:59 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 13:14:59 -0400 (EDT) Subject: [Python-Dev] Misc/maintainers.rst Message-ID: I floated a proposal on stdlib-sig to create a file named Misc/maintainers.rst. The purpose of this file is to collect knowledge about who knows which modules well enough to make decision about issues in the tracker when the participants in the issue aren't sure, and to write down the community knowledge about who has special interest and expertise in specific topic areas. This proposal was met with approval and, after a couple of small modifications of the proposal, no dissent; so I've created the skeleton of the file. I've filled in some of the blanks where I had personal knowledge, was told info on IRC, from the two referenced PEPs, and from PEP 11. Feel free to respond with comments about the header text, but more importantly let me know what you know about who (especially if it is you) should be listed for each module, platform, and topic, and feel free to suggest additional topics. My goal is to record the community knowledge. Another topic of discussion that is orthogonal to filling in the table is whether or not to publish it outside the repository. Jesse would like to see it included in the Python Documentation, and Georg has suggested the possibility of creating a separate, sphinx-based, automatically-uploaded document collection in the repository to contain this and related information (Misc/devdocs?). The module list was built from the py3k documentation module index, with the addition of 'pybench' from PEP 291. If there are other modules/tools that are missing, or submodules that should be broken out into separate lines, please let me know. After the initial flurry of updates and comments dies down I will check this in. --David ---------------------------------------------------------------------- Maintainers Index ================= This document cross references Python Modules (first table) and platforms (second table) with the Tracker user names of people who are experts and/or resources for that module or platform. This list is intended to be used by issue submitters, issue triage people, and other issue participants to find people to add to the nosy list or to contact directly by email for help and decisions on feature requests and bug fixes. People on this list may be asked to render final judgement on a feature or bug. If no active maintainer is listed for a given module, then questionable changes should go to python-dev, while any other issues can and should be decided by any committer. The last part of this document is a third table, listing broader topic areas in which various people have expertise. These people can also be contacted for help, opinions, and decisions when issues involve their areas. If a listed maintainer does not respond to requests for comment for an extended period (three weeks or more), they should be marked as inactive in this list by placing the word 'inactive' in parenthesis behind their tracker id. They are of course free to remove that inactive mark at any time. Committers should update this table as their areas of expertise widen. New topics may be added to the third table at will. The existence of this list is not meant to indicate that these people *must* be contacted for decisions; it is, rather, a resource to be used by non-committers to find responsible parties, and by committers who do not feel qualified to make a decision in a particular context. See also `PEP 291`_ and `PEP 360`_ for information about certain modules with special rules. .. _`PEP 291`: http://www.python.org/dev/peps/pep-0291/ .. _`PEP 360`: http://www.python.org/dev/peps/pep-0360/ ================== =========== Module Maintainers ================== =========== __future__ __main__ gvanrossum _dummy_thread brett.cannon _thread abc aifc r.david.murray array ast asynchat josiahcarlson asyncore josiahcarlson atexit audioop base64 bdb binascii binhex bisect builtins bz2 calendar cgi cgitb chunk cmath cmd code codecs codeop collections colorsys compileall configparser contextlib copy copyreg cProfile crypt csv ctypes theller curses datetime dbm decimal difflib dis distutils tarek doctest dummy_threading brett.cannon email barry encodings errno exceptions fcntl filecmp fileinput fnmatch formatter fpectl fractions ftplib functools gc getopt getpass gettext glob grp gzip hashlib heapq hmac html http imaplib imghdr imp importlib brett.cannon inspect io pitrou, benjamin.peterson itertools json keyword lib2to3 benjamin.peterson linecache locale logging vsajip macpath mailbox mailcap marshal math mimetypes mmap modulefinder theller, jvr msilib msvcrt multiprocessing jnoller netrc nis nntplib numbers operator optparse aronacher os ossaudiodev parser pdb pickle pickletools pipes pkgutil platform lemburg plistlib poplib posix pprint pstats pty pwd py_compile pybench lemburg pyclbr pydoc queue quopri random re readline reprlib resource rlcompleter runpy sched select shelve shlex shutil signal site smtpd smtplib sndhdr socket socketserver spwd sqlite3 ssl stat string stringprep struct subprocess astrand (inactive) sunau symbol symtable sys syslog tabnanny tarfile telnetlib tempfile termios test textwrap threading time timeit tkinter gpolo token tokenize trace traceback tty turtle gregorlingl types unicodedata unittest michael.foord urllib uu uuid warnings wave weakref webbrowser georg.brandl winreg winsound wsgiref pje xdrlib xml loewis xml.etree effbot (inactive) xmlrpc loewis zipfile zipimport zlib ================== =========== ================== =========== Platform Maintainer ------------------ ----------- AIX Cygwin jlt63 FreeBSD Linux Mac ronaldoussoren NetBSD1 OS2/EMX aimacintyre Solaris HP-UX ================== =========== ================== =========== Interest Area Maintainers ------------------ ----------- algorithms ast/compiler autoconf bsd buildbots data formats database documentation georg.brandl GUI i18n import machinery brett.cannon io pitrou, benjamin.peterson locale makefiles mathematics marketdickinson, eric.smith memory management networking packaging release management time and dates testing michael.foord threads unicode windows ================== =========== From steve at pearwood.info Thu Sep 17 19:27:33 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Sep 2009 03:27:33 +1000 Subject: [Python-Dev] Fwd: PEP 3144 review. In-Reply-To: <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <538a660a0909170717t30834632q8b221b8654f8e8cf@mail.gmail.com> <538a660a0909170744v695056b1o5ee5e0622e492ba9@mail.gmail.com> Message-ID: <200909180327.34065.steve@pearwood.info> On Fri, 18 Sep 2009 12:44:04 am DrKJam wrote: > IPv6 doesn't support the notion of a broadcast address as part of a > CIDR network block at all. AFAIK, it is a perfect legitimate for the > last address in an IPv6 block to be used to configure a network > interface. The IPv6 network object interface should possibly leave > out the broadcast property/method altogether although there are > reasons to keep it in for the sake of completeness and API > consistency. The pros and cons of this need to be considered. This seems to me to be rather weird: that for the sake of completeness and consistency you're considering to give an object which (allegedly) has no notion of "broadcast address" a broadcast address method. If so, I can see three options. Given an appropriate IPv6 object: obj.broadcast() always returns None. obj.broadcast() always raises an exception. obj.broadcast() always returns something which is not a broadcast address. I would hope the third option isn't being seriously considered! Are there any reasons to consider the first two? -- Steven D'Aprano From rdmurray at bitdance.com Thu Sep 17 19:32:13 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 13:32:13 -0400 (EDT) Subject: [Python-Dev] conceptual clarity In-Reply-To: <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> Message-ID: On Thu, 17 Sep 2009 at 09:16, Peter Moody wrote: > I mentioned before that IPy's insistence on receiving masked out > networks was one of the main reasons I wrote ipaddr to begin with. > Having ipaddr mimic this behavior would make it significantly less > useful. Removing functionality in the name of avoiding confusion > doesn't make sense when the same confusion can be alleviated w/o the > loss. We're suggesting moving that functionality (accepting a non-masked IP plus netmask and returning the corresponding Network object) into a different constructor, not eliminating the functionality. --David From peter at hda3.com Thu Sep 17 19:38:37 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 10:38:37 -0700 Subject: [Python-Dev] conceptual clarity In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> Message-ID: <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com> On Thu, Sep 17, 2009 at 10:32 AM, R. David Murray wrote: > On Thu, 17 Sep 2009 at 09:16, Peter Moody wrote: >> >> I mentioned before that IPy's insistence on receiving masked out >> networks was one of the main reasons I wrote ipaddr to begin with. >> Having ipaddr mimic this behavior would make it significantly less >> useful. Removing functionality in the name of avoiding confusion >> doesn't make sense when the same confusion can be alleviated w/o the >> loss. > > We're suggesting moving that functionality (accepting a non-masked IP > plus netmask and returning the corresponding Network object) into > a different constructor, not eliminating the functionality. Do you mean a new constructor function, ala IPAddress and IPNetwork (so the current classes remain largely the same and the various constructors enforce certain restrictions) or something else? At this point, I'm getting confused with the intermingling of class and constructor names. Cheers, /peter > --David > From g.brandl at gmx.net Thu Sep 17 19:38:50 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 17 Sep 2009 19:38:50 +0200 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: R. David Murray schrieb: > I floated a proposal on stdlib-sig to create a file named > Misc/maintainers.rst. The purpose of this file is to collect knowledge > about who knows which modules well enough to make decision about issues > in the tracker when the participants in the issue aren't sure, and to > write down the community knowledge about who has special interest and > expertise in specific topic areas. > > This proposal was met with approval and, after a couple of small > modifications of the proposal, no dissent; so I've created the skeleton > of the file. I've filled in some of the blanks where I had personal > knowledge, was told info on IRC, from the two referenced PEPs, and > from PEP 11. > > Feel free to respond with comments about the header text, but more > importantly let me know what you know about who (especially if it is you) > should be listed for each module, platform, and topic, and feel free to > suggest additional topics. My goal is to record the community knowledge. One thing I'd like to see in the list are real names of committers, not tracker names. Of course, people looking for people to assign a bug to should not have to search for the tracker name, so I'd like to make another request (that Brett already made when we switched trackers): Could we *please* have tracker names that match the committer names? (This doesn't even need to be done by the individual users, I would volunteer to rename all committer accounts and notify them by email that their name changed. This will also be coordinated with the new names used for Mercurial commits, if a change will be made.) > Another topic of discussion that is orthogonal to filling in the table is > whether or not to publish it outside the repository. Jesse would like to > see it included in the Python Documentation, and Georg has suggested the > possibility of creating a separate, sphinx-based, automatically-uploaded > document collection in the repository to contain this and related > information (Misc/devdocs?). For those who aren't on stdlib-sig, I'd like to elaborate a bit on that: There are quite a few resources for and about Python core development, but they aren't very accessible. For example, there's Misc/developers.txt and the upcoming maintainers.txt. Then there's the dev FAQ, but it's not maintained where developers usually look, but on the website. Etc. So the plan would be to consolidate these into another set of rst docs, having them in the repo, editable by every committer, as well as published somewhere on python.org (devdocs.python.org or somesuch). cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From steve at pearwood.info Thu Sep 17 19:44:31 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Sep 2009 03:44:31 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB22E4F.40509@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB22E4F.40509@gmail.com> Message-ID: <200909180344.31794.steve@pearwood.info> On Thu, 17 Sep 2009 10:40:47 pm Nick Coghlan wrote: > Antoine Pitrou wrote: > > Peter Moody hda3.com> writes: > >> the address with all of the hosts bits masked to zero is most > >> commonly referred to as the network address. same as the address > >> with all of the host bits set to one is called the broadcast > >> address. calling it something like base_address or min_address > >> will cause quite a bit more confusion. > > > > Quite a bit less IMO. I'm not a network specialist, but the > > "network address" > > Nah, network address is perfectly explicit - it's what you get when > you bitwise and any host within that network with the netmask. [soapbox] I really wish people would stop misusing the word "explicit" to mean (apparently) "I know how this is commonly defined". To a non-specialist, "the network address" is ambiguous. There are many addresses in a network, and none of them are the entire network. It's like saying, given a list [2, 4, 8, 12], what's "the list item"? If network specialists have a convention of calling hostadd&netmask "the network address", that's fine. It's jargon which seems illogical to me, but if it's useful and in common use, specialists will expect us to support it. But it's not explicit, it's a definition. > Where it becomes confusing is that we have a property called > "network" that returns an IPAddress object rather than an IPNetwork > object. > > People that are network specialists would grasp that fairly easily, > but we try to design standard library APIs so that they're novice > friendly as well. And the above situation isn't novice friendly. +1000 on this! > To be honest, given the indexing behaviour, I'm -1 on the idea of > giving the network address or broadcast address attribute names *at > all*. Consider: > > network_address = my_net[0] > broadcast_address = my_net[-1] > > The only way the network address could match the number of characters > in the indexing method is if you just called it the network id (i.e. > my_net.id). For the broadcast address, there is no name that even > comes close to matching the indexing approach for brevity. Is brevity so much more important than clarity and discoverability? -- Steven D'Aprano From drkjam at gmail.com Thu Sep 17 19:50:22 2009 From: drkjam at gmail.com (David Moss) Date: Thu, 17 Sep 2009 18:50:22 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> <538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com> <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com> <8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com> Message-ID: On 17 Sep 2009, at 15:40, Peter Moody wrote: > On Thu, Sep 17, 2009 at 7:26 AM, DrKJam wrote: >> Please can we have the following RFCs added to the references >> section that >> cover many of the aspects covered by this PEP? >> >> RFC 791 - Internet Protocol >> RFC 1918 - Address Allocation for Private Internets >> RFC 3330 - Special-Use IPv4 Addresses >> RFC 4291 - IPv6 Addressing Architecture >> RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet >> Address > > Sure, can you give me an idea of what specifically you're looking to > the rfc's to clarify? simply adding 5 rfc's to the the PEP wouldn't > seem to enhance its readability. I would hope that these RFCs form the technical basis upon which much of this library depends. > >> Dave M. >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com >> >> From brett at python.org Thu Sep 17 19:56:54 2009 From: brett at python.org (Brett Cannon) Date: Thu, 17 Sep 2009 10:56:54 -0700 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: On Thu, Sep 17, 2009 at 10:38, Georg Brandl wrote: > R. David Murray schrieb: >> I floated a proposal on stdlib-sig to create a file named >> Misc/maintainers.rst. ?The purpose of this file is to collect knowledge >> about who knows which modules well enough to make decision about issues >> in the tracker when the participants in the issue aren't sure, and to >> write down the community knowledge about who has special interest and >> expertise in specific topic areas. >> >> This proposal was met with approval and, after a couple of small >> modifications of the proposal, no dissent; so I've created the skeleton >> of the file. ?I've filled in some of the blanks where I had personal >> knowledge, was told info on IRC, from the two referenced PEPs, and >> from PEP 11. >> >> Feel free to respond with comments about the header text, but more >> importantly let me know what you know about who (especially if it is you) >> should be listed for each module, platform, and topic, and feel free to >> suggest additional topics. ?My goal is to record the community knowledge. > > One thing I'd like to see in the list are real names of committers, > not tracker names. ?Of course, people looking for people to assign a bug > to should not have to search for the tracker name, so I'd like to make > another request (that Brett already made when we switched trackers): > > ?Could we *please* have tracker names that match the committer names? > > (This doesn't even need to be done by the individual users, I would > volunteer to rename all committer accounts and notify them by email that > their name changed. ?This will also be coordinated with the new names > used for Mercurial commits, if a change will be made.) > >> Another topic of discussion that is orthogonal to filling in the table is >> whether or not to publish it outside the repository. ?Jesse would like to >> see it included in the Python Documentation, and Georg has suggested the >> possibility of creating a separate, sphinx-based, automatically-uploaded >> document collection in the repository to contain this and related >> information (Misc/devdocs?). > > For those who aren't on stdlib-sig, I'd like to elaborate a bit on that: > There are quite a few resources for and about Python core development, > but they aren't very accessible. ?For example, there's Misc/developers.txt > and the upcoming maintainers.txt. ?Then there's the dev FAQ, but it's not > maintained where developers usually look, but on the website. ?Etc. > So the plan would be to consolidate these into another set of rst docs, > having them in the repo, editable by every committer, as well as published > somewhere on python.org (devdocs.python.org or somesuch). dev.python.org would be nice to have, from which we can simply redirect www.python.org/dev/ to dev.python.org. www.python.org/dev/ can then get cleaned up be made simpler to navigate and more obvious for how people can get started. -Brett From brett at python.org Thu Sep 17 19:57:55 2009 From: brett at python.org (Brett Cannon) Date: Thu, 17 Sep 2009 10:57:55 -0700 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: Looks great to me! Only thing missing that I can think of is sticking Eric down as the guy who does str.format(). =) On Thu, Sep 17, 2009 at 10:14, R. David Murray wrote: > I floated a proposal on stdlib-sig to create a file named > Misc/maintainers.rst. ?The purpose of this file is to collect knowledge > about who knows which modules well enough to make decision about issues > in the tracker when the participants in the issue aren't sure, and to > write down the community knowledge about who has special interest and > expertise in specific topic areas. > > This proposal was met with approval and, after a couple of small > modifications of the proposal, no dissent; so I've created the skeleton > of the file. ?I've filled in some of the blanks where I had personal > knowledge, was told info on IRC, from the two referenced PEPs, and > from PEP 11. > > Feel free to respond with comments about the header text, but more > importantly let me know what you know about who (especially if it is you) > should be listed for each module, platform, and topic, and feel free to > suggest additional topics. ?My goal is to record the community knowledge. > > Another topic of discussion that is orthogonal to filling in the table is > whether or not to publish it outside the repository. ?Jesse would like to > see it included in the Python Documentation, and Georg has suggested the > possibility of creating a separate, sphinx-based, automatically-uploaded > document collection in the repository to contain this and related > information (Misc/devdocs?). > > The module list was built from the py3k documentation module index, with > the addition of 'pybench' from PEP 291. ?If there are other modules/tools > that are missing, or submodules that should be broken out into > separate lines, please let me know. > > After the initial flurry of updates and comments dies down I will check > this in. > > --David > > ---------------------------------------------------------------------- > Maintainers Index > ================= > > This document cross references Python Modules (first table) and platforms > (second table) with the Tracker user names of people who are experts > and/or resources for that module or platform. ?This list is intended > to be used by issue submitters, issue triage people, and other issue > participants to find people to add to the nosy list or to contact > directly by email for help and decisions on feature requests and bug > fixes. ?People on this list may be asked to render final judgement on a > feature or bug. ?If no active maintainer is listed for a given module, > then questionable changes should go to python-dev, while any other issues > can and should be decided by any committer. > > The last part of this document is a third table, listing broader topic > areas in which various people have expertise. ?These people can also > be contacted for help, opinions, and decisions when issues involve > their areas. > > If a listed maintainer does not respond to requests for comment for an > extended period (three weeks or more), they should be marked as inactive > in this list by placing the word 'inactive' in parenthesis behind their > tracker id. ?They are of course free to remove that inactive mark at > any time. > > Committers should update this table as their areas of expertise widen. > New topics may be added to the third table at will. > > The existence of this list is not meant to indicate that these people > *must* be contacted for decisions; it is, rather, a resource to be used > by non-committers to find responsible parties, and by committers who do > not feel qualified to make a decision in a particular context. > > See also `PEP 291`_ and `PEP 360`_ for information about certain modules > with special rules. > > .. _`PEP 291`: http://www.python.org/dev/peps/pep-0291/ > .. _`PEP 360`: http://www.python.org/dev/peps/pep-0360/ > > > ================== ?=========== > Module ? ? ? ? ? ? ?Maintainers > ================== ?=========== > __future__ > __main__ ? ? ? ? ? ?gvanrossum > _dummy_thread ? ? ? brett.cannon > _thread > abc > aifc ? ? ? ? ? ? ? ?r.david.murray > array > ast > asynchat ? ? ? ? ? ?josiahcarlson > asyncore ? ? ? ? ? ?josiahcarlson > atexit > audioop > base64 > bdb > binascii > binhex > bisect > builtins > bz2 > calendar > cgi > cgitb > chunk > cmath > cmd > code > codecs > codeop > collections > colorsys > compileall > configparser > contextlib > copy > copyreg > cProfile > crypt > csv > ctypes ? ? ? ? ? ? ?theller > curses > datetime > dbm > decimal > difflib > dis > distutils ? ? ? ? ? tarek > doctest > dummy_threading ? ? brett.cannon > email ? ? ? ? ? ? ? barry > encodings > errno > exceptions > fcntl > filecmp > fileinput > fnmatch > formatter > fpectl > fractions > ftplib > functools > gc > getopt > getpass > gettext > glob > grp > gzip > hashlib > heapq > hmac > html > http > imaplib > imghdr > imp > importlib ? ? ? ? ? brett.cannon > inspect > io ? ? ? ? ? ? ? ? ?pitrou, benjamin.peterson > itertools > json > keyword > lib2to3 ? ? ? ? ? ? benjamin.peterson > linecache > locale > logging ? ? ? ? ? ? vsajip > macpath > mailbox > mailcap > marshal > math > mimetypes > mmap > modulefinder ? ? ? ?theller, jvr > msilib > msvcrt > multiprocessing ? ? jnoller > netrc > nis > nntplib > numbers > operator > optparse ? ? ? ? ? ?aronacher > os > ossaudiodev > parser > pdb > pickle > pickletools > pipes > pkgutil > platform ? ? ? ? ? ?lemburg > plistlib > poplib > posix > pprint > pstats > pty > pwd > py_compile > pybench ? ? ? ? ? ? lemburg > pyclbr > pydoc > queue > quopri > random > re > readline > reprlib > resource > rlcompleter > runpy > sched > select > shelve > shlex > shutil > signal > site > smtpd > smtplib > sndhdr > socket > socketserver > spwd > sqlite3 > ssl > stat > string > stringprep > struct > subprocess ? ? ? ? ?astrand (inactive) > sunau > symbol > symtable > sys > syslog > tabnanny > tarfile > telnetlib > tempfile > termios > test > textwrap > threading > time > timeit > tkinter ? ? ? ? ? ? gpolo > token > tokenize > trace > traceback > tty > turtle ? ? ? ? ? ? ?gregorlingl > types > unicodedata > unittest ? ? ? ? ? ?michael.foord > urllib > uu > uuid > warnings > wave > weakref > webbrowser ? ? ? ? ?georg.brandl > winreg > winsound > wsgiref ? ? ? ? ? ? pje > xdrlib > xml ? ? ? ? ? ? ? ? loewis > xml.etree ? ? ? ? ? effbot (inactive) > xmlrpc ? ? ? ? ? ? ?loewis > zipfile > zipimport > zlib > ================== ?=========== > > > ================== ?=========== > Platform ? ? ? ? ? ?Maintainer > ------------------ ?----------- > AIX > Cygwin ? ? ? ? ? ? ?jlt63 > FreeBSD > Linux > Mac ? ? ? ? ? ? ? ? ronaldoussoren > NetBSD1 > OS2/EMX ? ? ? ? ? ? aimacintyre > Solaris > HP-UX > ================== ?=========== > > > ================== ?=========== > Interest Area ? ? ? Maintainers > ------------------ ?----------- > algorithms > ast/compiler > autoconf > bsd > buildbots > data formats > database > documentation ? ? ? georg.brandl > GUI > i18n > import machinery ? ?brett.cannon > io ? ? ? ? ? ? ? ? ?pitrou, benjamin.peterson > locale > makefiles > mathematics ? ? ? ? marketdickinson, eric.smith > memory management > networking > packaging release management > time and dates > testing ? ? ? ? ? ? michael.foord > threads > unicode windows > ================== ?=========== > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > From peter at hda3.com Thu Sep 17 19:59:22 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 10:59:22 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> <538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com> <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com> <8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com> Message-ID: <8517e9350909171059h2a799786saa199e39b811b115@mail.gmail.com> On Thu, Sep 17, 2009 at 10:50 AM, David Moss wrote: > On 17 Sep 2009, at 15:40, Peter Moody wrote: > >> On Thu, Sep 17, 2009 at 7:26 AM, DrKJam wrote: >>> >>> Please can we have the following RFCs added to the references section >>> that >>> cover many of the aspects covered by this PEP? >>> >>> RFC 791 - Internet Protocol >>> RFC 1918 - Address Allocation for Private Internets >>> RFC 3330 - Special-Use IPv4 Addresses >>> RFC 4291 - IPv6 Addressing Architecture >>> RFC 4632 - Classless Inter-domain Routing (CIDR): The Internet Address >> >> Sure, can you give me an idea of what specifically you're looking to >> the rfc's to clarify? simply adding 5 rfc's to the the PEP wouldn't >> seem to enhance its readability. > > I would hope that these RFCs form the technical basis upon which much of > this library depends. yes, and my question was is there something(s) in particular you wish to see highlighted which will help clarify some confusion you currently have, or do you feel that simply adding 5 rfc's to the references section adds to the overall readability of the PEP? see http://www.python.org/dev/peps/pep-0333/ for an example of what I mean. >> >>> Dave M. >>> >>> _______________________________________________ >>> Python-Dev mailing list >>> Python-Dev at python.org >>> http://mail.python.org/mailman/listinfo/python-dev >>> Unsubscribe: >>> http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com >>> >>> > From brett at python.org Thu Sep 17 19:59:10 2009 From: brett at python.org (Brett Cannon) Date: Thu, 17 Sep 2009 10:59:10 -0700 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: On Thu, Sep 17, 2009 at 10:38, Georg Brandl wrote: > R. David Murray schrieb: >> I floated a proposal on stdlib-sig to create a file named >> Misc/maintainers.rst. ?The purpose of this file is to collect knowledge >> about who knows which modules well enough to make decision about issues >> in the tracker when the participants in the issue aren't sure, and to >> write down the community knowledge about who has special interest and >> expertise in specific topic areas. >> >> This proposal was met with approval and, after a couple of small >> modifications of the proposal, no dissent; so I've created the skeleton >> of the file. ?I've filled in some of the blanks where I had personal >> knowledge, was told info on IRC, from the two referenced PEPs, and >> from PEP 11. >> >> Feel free to respond with comments about the header text, but more >> importantly let me know what you know about who (especially if it is you) >> should be listed for each module, platform, and topic, and feel free to >> suggest additional topics. ?My goal is to record the community knowledge. > > One thing I'd like to see in the list are real names of committers, > not tracker names. ?Of course, people looking for people to assign a bug > to should not have to search for the tracker name, so I'd like to make > another request (that Brett already made when we switched trackers): > > ?Could we *please* have tracker names that match the committer names? > > (This doesn't even need to be done by the individual users, I would > volunteer to rename all committer accounts and notify them by email that > their name changed. ?This will also be coordinated with the new names > used for Mercurial commits, if a change will be made.) +1 from me. It would be nice to not have to look up what tracker name someone uses since I know everyone's committer name. -Brett From rdmurray at bitdance.com Thu Sep 17 20:04:11 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 14:04:11 -0400 (EDT) Subject: [Python-Dev] conceptual clarity In-Reply-To: <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com> Message-ID: On Thu, 17 Sep 2009 at 10:38, Peter Moody wrote: > On Thu, Sep 17, 2009 at 10:32 AM, R. David Murray wrote: >> On Thu, 17 Sep 2009 at 09:16, Peter Moody wrote: >>> >>> I mentioned before that IPy's insistence on receiving masked out >>> networks was one of the main reasons I wrote ipaddr to begin with. >>> Having ipaddr mimic this behavior would make it significantly less >>> useful. Removing functionality in the name of avoiding confusion >>> doesn't make sense when the same confusion can be alleviated w/o the >>> loss. >> >> We're suggesting moving that functionality (accepting a non-masked IP >> plus netmask and returning the corresponding Network object) into >> a different constructor, not eliminating the functionality. > > Do you mean a new constructor function, ala IPAddress and IPNetwork > (so the current classes remain largely the same and the various > constructors enforce certain restrictions) or something else? > > At this point, I'm getting confused with the intermingling of class > and constructor names. I mean, eg, IPv4Network.fromHostIP('192.168.1.1/24'). Or some such name. Having another constructor that took an IPv4Address object and a mask would probably be a good idea as well. These could be exposed as appropriately named module level functions instead/as well, of course. I would have IPv4Address itself be strict, and thus the new constructors would compute the network address and call the regular IPv4Address constructor.(*) --David (*) If that's the part you object to (not recording the input IP in the Network object), then clearly you see a need to have an AddressWithMask object of _some_ sort.... From rdmurray at bitdance.com Thu Sep 17 20:08:12 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 14:08:12 -0400 (EDT) Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: On Thu, 17 Sep 2009 at 10:59, Brett Cannon wrote: > On Thu, Sep 17, 2009 at 10:38, Georg Brandl wrote: >> ??Could we *please* have tracker names that match the committer names? >> >> (This doesn't even need to be done by the individual users, I would >> volunteer to rename all committer accounts and notify them by email that >> their name changed. ??This will also be coordinated with the new names >> used for Mercurial commits, if a change will be made.) > > +1 from me. It would be nice to not have to look up what tracker name > someone uses since I know everyone's committer name. +1 from me as well. And speaking of names, people should feel free to email me maintainer info for the table as real names. Whether or not Georg's proposal goes through I'll make the table reflect the correct tracker ids. --David From fetchinson at googlemail.com Thu Sep 17 20:10:51 2009 From: fetchinson at googlemail.com (Daniel Fetchinson) Date: Thu, 17 Sep 2009 11:10:51 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> <8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com> Message-ID: >>>>> 188 (check that, 190) people have downloaded the 2.0 release in the >>>>> last week (numbers publicly available from the code.google.com). I >>>>> can't tell you how many (if any) have downloaded it via svn. >>>> >>>> Downloading and using are not the same thing. >>> >>> Correct, but there is a strong positive correlation between the two. >>> If you have a better method for determining what you would consider an >>> appropriate level of usage, I'm all ears. >> >> A good way of determining the level of usage would be pointing to open >> source projects that are popular in the python community and which >> incorporate your module. > > well, the 2.0 release is still new. codesearch.google.com shows some > projects using the 1.x release; hopefully some of those 200 > downloaders will put up some publicly indexable python code at some > point. I think one first needs to wait until this happens, I meana large user base is formed, before a meaningful discussion can be done on whether to include it in the stdlib or not. The long and largely academic thread here I think illustrates this point. Without a large user base it's up to anybody's gut feelings what is 'right' and what 'feels wrong'. Cheers, Daniel -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From steve at pearwood.info Thu Sep 17 20:26:41 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Sep 2009 04:26:41 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917071516.217655AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp> <20090917071516.217655AC0D6@longblack.object-craft.com.au> Message-ID: <200909180426.41808.steve@pearwood.info> On Thu, 17 Sep 2009 05:15:16 pm Andrew McNamara wrote: > Conceptually, you sometimes need a bare address, and other times, > you need an address with an associated network (host interface > configs, router configs, etc). By AddressWithMask, I really mean > AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently, > IPv4 and IPv6 addressing allows us to derive the network from the > host address combined with the netmask - in other words, we don't > have to attach a real Network object to Address objects until the > user tries to access it, and then we derive it from the address and > mask. To clarify: when you say "derive the network" are you talking about the network (which is a container) or the network address = host_address & netmask (which is not a container)? I think you're referring to the later. If there's need for address+netmask, does it need to be a separate class? Perhaps Address objects could simply have a netmask property, defaulting to None. If you need an "address with mask" object, you create an Address and set the mask: addr = Address(...) addr.netmask = "255.255.255.0" The caller is responsible for ensuring the netmask has been set before trying to use it: network_address = addr & addr.netmask > If you have a .network attribute on an address object, checking if an > address is in the same network as another address becomes: > > addr_a in addr_b.network > > As the module stands, you write that as: > > addr_a in addr_b > > I don't think the intent is as clear with the later. I would find the later completely unclear and disturbing -- how can one address contain another address? -- Steven D'Aprano From amk at amk.ca Thu Sep 17 20:29:44 2009 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 17 Sep 2009 14:29:44 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909171059h2a799786saa199e39b811b115@mail.gmail.com> References: <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> <538a660a0909170719h1a0a4b31i10dee074a43260eb@mail.gmail.com> <538a660a0909170726g471e164dq9b44b55c6523f20d@mail.gmail.com> <8517e9350909170740kf2c1d46t2b118c6835fbefce@mail.gmail.com> <8517e9350909171059h2a799786saa199e39b811b115@mail.gmail.com> Message-ID: <20090917182944.GA13389@amk-desktop.matrixgroup.net> On Thu, Sep 17, 2009 at 10:59:22AM -0700, Peter Moody wrote: > currently have, or do you feel that simply adding 5 rfc's to the > references section adds to the overall readability of the PEP? I would list them simply because it's not obvious which RFC specifies the format of IP addresses or how they're allocated. e.g. if I search for "rfc ip address format", most of the hits are for RFC 3587, "IPv6 Global Unicast Address Format", which doesn't actually explain IPv6 address syntax. --amk From rdmurray at bitdance.com Thu Sep 17 20:35:16 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 14:35:16 -0400 (EDT) Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: On Thu, 17 Sep 2009 at 10:57, Brett Cannon wrote: > Looks great to me! Only thing missing that I can think of is sticking > Eric down as the guy who does str.format(). =) OK, I've added that one to the last table ;) --David From rdmurray at bitdance.com Thu Sep 17 20:40:38 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 14:40:38 -0400 (EDT) Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: On Thu, 17 Sep 2009 at 14:08, R. David Murray wrote: > On Thu, 17 Sep 2009 at 10:59, Brett Cannon wrote: >> On Thu, Sep 17, 2009 at 10:38, Georg Brandl wrote: >> > ??Could we *please* have tracker names that match the committer names? >> > >> > (This doesn't even need to be done by the individual users, I would >> > volunteer to rename all committer accounts and notify them by email that >> > their name changed. ??This will also be coordinated with the new names >> > used for Mercurial commits, if a change will be made.) >> >> +1 from me. It would be nice to not have to look up what tracker name >> someone uses since I know everyone's committer name. > > +1 from me as well. And speaking of names, people should feel free to > email me maintainer info for the table as real names. Whether or > not Georg's proposal goes through I'll make the table reflect the > correct tracker ids. By the way, in case anyone isn't aware of it (Mark Dickinson and I weren't initially), it is _really easy_ to change your tracker id if you want to do it yourself. Just go in 'your details' and change your login name. It'll change everywhere in the tracker, since roundup tracks accounts by an internal number, not the login name. --David From eric at trueblade.com Thu Sep 17 20:43:13 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 17 Sep 2009 14:43:13 -0400 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: <4AB28341.5090908@trueblade.com> R. David Murray wrote: > On Thu, 17 Sep 2009 at 10:57, Brett Cannon wrote: >> Looks great to me! Only thing missing that I can think of is sticking >> Eric down as the guy who does str.format(). =) > > OK, I've added that one to the last table ;) Awesome! I get to spend even more time on formatting strings! I kid. I'm happy to be the string formatting maintainer. From amk at amk.ca Thu Sep 17 20:44:10 2009 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 17 Sep 2009 14:44:10 -0400 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: <20090917184410.GB13389@amk-desktop.matrixgroup.net> On Thu, Sep 17, 2009 at 07:38:50PM +0200, Georg Brandl wrote: > So the plan would be to consolidate these into another set of rst docs, > having them in the repo, editable by every committer, as well as published > somewhere on python.org (devdocs.python.org or somesuch). +1. Should we do something similar with the FAQs at http://www.python.org/doc/faq/ ? --amk From steve at pearwood.info Thu Sep 17 20:47:35 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Sep 2009 04:47:35 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> Message-ID: <200909180447.36475.steve@pearwood.info> On Thu, 17 Sep 2009 10:41:37 am Andrew McNamara wrote: > In the olden days, the mask was spelled out in octets (eg > 255.255.255.0). But we've moved to a more compact and logical > notation where the number of leading significant bits is specified > (eg /24). I hope you're not suggesting the older notation be unsupported? I would expect to be able to use a mask like 255.255.255.192 without having to count bits myself. -- Steven D'Aprano From p.f.moore at gmail.com Thu Sep 17 21:11:51 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Thu, 17 Sep 2009 20:11:51 +0100 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: <79990c6b0909171211q697442f3n1948b946ffe31708@mail.gmail.com> 2009/9/17 R. David Murray : > Feel free to respond with comments about the header text, but more > importantly let me know what you know about who (especially if it is you) > should be listed for each module, platform, and topic, and feel free to > suggest additional topics. ?My goal is to record the community knowledge. A couple more: runpy - Nick Coghlan itertools, collections - Raymond Hettinger (obviously, if I've misrepresented their interest, they should speak up!) Paul. From fdrake at acm.org Thu Sep 17 20:19:03 2009 From: fdrake at acm.org (Fred Drake) Date: Thu, 17 Sep 2009 14:19:03 -0400 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: On Thu, Sep 17, 2009 at 10:38, Georg Brandl wrote: > So the plan would be to consolidate these into another set of rst > docs, > having them in the repo, editable by every committer, as well as > published > somewhere on python.org (devdocs.python.org or somesuch). On Sep 17, 2009, at 1:56 PM, Brett Cannon wrote: > dev.python.org would be nice to have, from which we can simply > redirect www.python.org/dev/ to dev.python.org. www.python.org/dev/ > can then get cleaned up be made simpler to navigate and more obvious > for how people can get started. Many years ago, we decided to add docs.python.org with the "current" version of the documentation, so people would be able to find the docs more easily. Since then, we've had problems with keeping docs.python.org and www.python.org/doc/ in sync, and with different styles being applied to the sites. One of the reasons www.python.org/doc/ was considered less discoverable was the about of only-sometimes-interesting information there; docs.python.org contains only "current" docs (for some vague notion of current and only, given that dev builds and both Python 2 and Python 3 versions). Some claimed that having "docs" at the front of the URL helped, though I don't recall why. I don't know whether users consider the docs more discoverable than they used to; if anyone can provide information about that (now sure what indicator would make even sense), that might be informative. The overload problem is one we *don't* have for developer documentation; the difficulty is in discovering what exists at all. +1 on moving the developer docs to subversion +0 on exposing them online -0 on adding another domain name -Fred -- Fred Drake From mcguire at google.com Thu Sep 17 20:58:29 2009 From: mcguire at google.com (Jake McGuire) Date: Thu, 17 Sep 2009 11:58:29 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> <8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com> Message-ID: <77c780b40909171158u6117e47djf18508acc6e1bc18@mail.gmail.com> On Thursday, September 17, 2009, Daniel Fetchinson wrote: >>>>>> 188 (check that, 190) people have downloaded the 2.0 release in the >>>>>> last week (numbers publicly available from the http://code.google.com). I >>>>>> can't tell you how many (if any) have downloaded it via svn. >>>>> >>>>> Downloading and using are not the same thing. >>>> >>>> Correct, but there is a strong positive correlation between the two. >>>> If you have a better method for determining what you would consider an >>>> appropriate level of usage, I'm all ears. >>> >>> A good way of determining the level of usage would be pointing to open >>> source projects that are popular in the python community and which >>> incorporate your module. >> >> well, the 2.0 release is still new. http://codesearch.google.com shows some >> projects using the 1.x release; hopefully some of those 200 >> downloaders will put up some publicly indexable python code at some >> point. > > I think one first needs to wait until this happens, I meana large user > base is formed, before a meaningful discussion can be done on whether > to include it in the stdlib or not. The long and largely academic > thread here I think illustrates this point. Without a large user base > it's up to anybody's gut feelings what is 'right' and what 'feels > wrong'. +1000 -jake From rdmurray at bitdance.com Thu Sep 17 21:15:41 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 15:15:41 -0400 (EDT) Subject: [Python-Dev] maintainers.rst committed Message-ID: I decided to commit the draft of maintainers.rst in case people would rather update it themselves. I'm happy to continue collecting updates and applying them as well. --David From jnoller at gmail.com Thu Sep 17 21:40:23 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 17 Sep 2009 15:40:23 -0400 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: <4222a8490909171240q4d7e1c81xb134c510af8acc54@mail.gmail.com> On Thu, Sep 17, 2009 at 1:38 PM, Georg Brandl wrote: > R. David Murray schrieb: >> I floated a proposal on stdlib-sig to create a file named >> Misc/maintainers.rst. ?The purpose of this file is to collect knowledge >> about who knows which modules well enough to make decision about issues >> in the tracker when the participants in the issue aren't sure, and to >> write down the community knowledge about who has special interest and >> expertise in specific topic areas. >> >> This proposal was met with approval and, after a couple of small >> modifications of the proposal, no dissent; so I've created the skeleton >> of the file. ?I've filled in some of the blanks where I had personal >> knowledge, was told info on IRC, from the two referenced PEPs, and >> from PEP 11. >> >> Feel free to respond with comments about the header text, but more >> importantly let me know what you know about who (especially if it is you) >> should be listed for each module, platform, and topic, and feel free to >> suggest additional topics. ?My goal is to record the community knowledge. > > One thing I'd like to see in the list are real names of committers, > not tracker names. ?Of course, people looking for people to assign a bug > to should not have to search for the tracker name, so I'd like to make > another request (that Brett already made when we switched trackers): > > ?Could we *please* have tracker names that match the committer names? > > (This doesn't even need to be done by the individual users, I would > volunteer to rename all committer accounts and notify them by email that > their name changed. ?This will also be coordinated with the new names > used for Mercurial commits, if a change will be made.) > >> Another topic of discussion that is orthogonal to filling in the table is >> whether or not to publish it outside the repository. ?Jesse would like to >> see it included in the Python Documentation, and Georg has suggested the >> possibility of creating a separate, sphinx-based, automatically-uploaded >> document collection in the repository to contain this and related >> information (Misc/devdocs?). > > For those who aren't on stdlib-sig, I'd like to elaborate a bit on that: > There are quite a few resources for and about Python core development, > but they aren't very accessible. ?For example, there's Misc/developers.txt > and the upcoming maintainers.txt. ?Then there's the dev FAQ, but it's not > maintained where developers usually look, but on the website. ?Etc. > So the plan would be to consolidate these into another set of rst docs, > having them in the repo, editable by every committer, as well as published > somewhere on python.org (devdocs.python.org or somesuch). > > cheers, > Georg > +1 to getting them in The One True Format (ReST) and up and online, and then porting all the dev docs into The One True Format. From turnbull at sk.tsukuba.ac.jp Thu Sep 17 21:56:43 2009 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 18 Sep 2009 04:56:43 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917071516.217655AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> <20090917041745.C331B6006DE@longblack.object-craft.com.au> <87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp> <20090917071516.217655AC0D6@longblack.object-craft.com.au> Message-ID: <878wgd1h1g.fsf@uwakimon.sk.tsukuba.ac.jp> Andrew McNamara writes: > Conceptually, you sometimes need a bare address, and other times, > you need an address with an associated network (host interface > configs, router configs, etc). By AddressWithMask, I really mean > AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently, > IPv4 and IPv6 addressing allows us to derive the network from the > host That's precisely the claim I deny. Yes, you can derive the network *address* from a generic address and a mask. You cannot derive the *network* that way! Eg, suppose in a routing application you have class IPv4NetworkWithGateway(IPv4Network): ... oops. You want *the* IPv4Network instance that has all the gateway info, not just *any* IPv4Network instance that happens to have the same network address and mask. Address + mask is *not* sufficient to derive the desired IPv4Network instance. I grant that sometimes all you really care about is that the network address and the mask match; constructing on the fly would be sufficient. However, many of the applications I can think of attach additional attributes to IPv4Networks, and really do want "the real thing" vs. a constructed imitation. From python at rcn.com Thu Sep 17 22:13:51 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 17 Sep 2009 13:13:51 -0700 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: <91ECB2DF-7C76-4896-B547-D73D5E71A0B5@rcn.com> On Sep 17, 2009, at 10:59 AM, Brett Cannon wrote: > On Thu, Sep 17, 2009 at 10:38, Georg Brandl wrote: >> R. David Murray schrieb: >>> I floated a proposal on stdlib-sig to create a file named >>> Misc/maintainers.rst. The purpose of this file is to collect >>> knowledge >>> about who knows which modules well enough to make decision about >>> issues >>> in the tracker when the participants in the issue aren't sure, and >>> to >>> write down the community knowledge about who has special interest >>> and >>> expertise in specific topic areas. >>> >>> This proposal was met with approval and, after a couple of small >>> modifications of the proposal, no dissent; so I've created the >>> skeleton >>> of the file. I've filled in some of the blanks where I had personal >>> knowledge, was told info on IRC, from the two referenced PEPs, and >>> from PEP 11. >>> >>> Feel free to respond with comments about the header text, but more >>> importantly let me know what you know about who (especially if it >>> is you) >>> should be listed for each module, platform, and topic, and feel >>> free to >>> suggest additional topics. My goal is to record the community >>> knowledge. >> >> One thing I'd like to see in the list are real names of committers, >> not tracker names. Of course, people looking for people to assign >> a bug >> to should not have to search for the tracker name, so I'd like to >> make >> another request (that Brett already made when we switched trackers): >> >> Could we *please* have tracker names that match the committer names? >> >> (This doesn't even need to be done by the individual users, I would >> volunteer to rename all committer accounts and notify them by email >> that >> their name changed. This will also be coordinated with the new names >> used for Mercurial commits, if a change will be made.) > > +1 from me. It would be nice to not have to look up what tracker name > someone uses since I know everyone's committer name. > +1 from me also. This has long been a source of irritation. Raymond From g.brandl at gmx.net Thu Sep 17 23:32:25 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Thu, 17 Sep 2009 23:32:25 +0200 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: <20090917184410.GB13389@amk-desktop.matrixgroup.net> References: <20090917184410.GB13389@amk-desktop.matrixgroup.net> Message-ID: A.M. Kuchling schrieb: > On Thu, Sep 17, 2009 at 07:38:50PM +0200, Georg Brandl wrote: >> So the plan would be to consolidate these into another set of rst docs, >> having them in the repo, editable by every committer, as well as published >> somewhere on python.org (devdocs.python.org or somesuch). > > +1. > > Should we do something similar with the FAQs at > http://www.python.org/doc/faq/ ? They would then go into the main docs, I guess? OK, so there seems to be some consensus on both the committer name and the dev docs proposal. I'll start working on both by the end of next week, when I have more contiguous time to spare. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ncoghlan at gmail.com Thu Sep 17 23:45:14 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 18 Sep 2009 07:45:14 +1000 Subject: [Python-Dev] conceptual clarity In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com> Message-ID: <4AB2ADEA.40601@gmail.com> R. David Murray wrote: > I would have IPv4Address itself be strict, and thus the new constructors > would compute the network address and call the regular IPv4Address > constructor.(*) s/Address/Network/ in this paragraph :) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Sep 17 23:48:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 18 Sep 2009 07:48:09 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB24DB9.9050605@trueblade.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> <8517e9350909170748g7ec8b1a0u926e26de6907b494@mail.gmail.com> <4AB24DB9.9050605@trueblade.com> Message-ID: <4AB2AE99.8070006@gmail.com> Eric Smith wrote: > Peter Moody wrote: > >> indexing is plenty efficient, but the problem is that these names for >> these attributes are common to the point of causing confusion if >> they're omitted. > > After thinking about it some more, I'm okay with names for [-1] and [0]. > I like .broadcast, and I can live with .network (although it's > confusing, since in my view it should return an address, not a network). And in fact *does* return an Address - the same address that my_net[0] returns. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Sep 17 23:58:56 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 18 Sep 2009 07:58:56 +1000 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: <79990c6b0909171211q697442f3n1948b946ffe31708@mail.gmail.com> References: <79990c6b0909171211q697442f3n1948b946ffe31708@mail.gmail.com> Message-ID: <4AB2B120.5060202@gmail.com> Paul Moore wrote: > runpy - Nick Coghlan And for the general interest table, the import system and the AST compiler are the two main areas that I know better than most. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From greg.ewing at canterbury.ac.nz Fri Sep 18 00:27:32 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 18 Sep 2009 10:27:32 +1200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> Message-ID: <4AB2B7D4.8090107@canterbury.ac.nz> Peter Moody wrote: > the address with all of the hosts bits masked to zero is most commonly > referred to as the network address. Then call the attribute 'network_address', not just 'network'. -- Greg From janssen at parc.com Fri Sep 18 00:33:30 2009 From: janssen at parc.com (Bill Janssen) Date: Thu, 17 Sep 2009 15:33:30 PDT Subject: [Python-Dev] how to debug httplib slowness In-Reply-To: <20090904153241.GB7342@phd.pp.ru> References: <4A82A20A.1070704@simplistix.co.uk> <4A82E120.4030105@simplistix.co.uk> <4AA0F5F2.6030903@simplistix.co.uk> <4AA12C0F.1090305@simplistix.co.uk> <20090904153241.GB7342@phd.pp.ru> Message-ID: <3746.1253226810@parc.com> Oleg Broytmann wrote: > On Fri, Sep 04, 2009 at 04:02:39PM +0100, Chris Withers wrote: > > So, httplib does this: > > > > GET / HTTP/1.1 > [skip] > > While wget does this: > > > > GET / HTTP/1.0 > [skip] > > - Apache responds with a chunked response only to httplib. Why is that? > > Probably because wget uses HTTP/1.0? Yep. Not allowed to respond chunked to 1.0 requests. Bill From rdmurray at bitdance.com Fri Sep 18 00:41:16 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 18:41:16 -0400 (EDT) Subject: [Python-Dev] conceptual clarity In-Reply-To: <4AB2ADEA.40601@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com> <4AB2ADEA.40601@gmail.com> Message-ID: On Fri, 18 Sep 2009 at 07:45, Nick Coghlan wrote: > R. David Murray wrote: >> I would have IPv4Address itself be strict, and thus the new constructors >> would compute the network address and call the regular IPv4Address >> constructor.(*) > > s/Address/Network/ in this paragraph :) Ah, yes, sorry for the slip, and thanks for catching it. --David From peter at hda3.com Fri Sep 18 00:50:16 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 15:50:16 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB2B7D4.8090107@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB2B7D4.8090107@canterbury.ac.nz> Message-ID: <8517e9350909171550y4033bbc4l6beaae10104cf7d7@mail.gmail.com> On Thu, Sep 17, 2009 at 3:27 PM, Greg Ewing wrote: > Peter Moody wrote: > >> the address with all of the hosts bits masked to zero is most commonly >> referred to as the network address. > > Then call the attribute 'network_address', not just 'network'. from an earlier email: Again, the same error-catching functionality can be obtained through an option to the constructor. network and broadcast attributes can be renamed to .\1_address to alleviate confusion as well. > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From amk at amk.ca Fri Sep 18 01:26:52 2009 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 17 Sep 2009 19:26:52 -0400 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: <20090917184410.GB13389@amk-desktop.matrixgroup.net> Message-ID: <20090917232652.GA1730@andrew-kuchlings-macbook.local> On Thu, Sep 17, 2009 at 11:32:25PM +0200, Georg Brandl wrote: > > Should we do something similar with the FAQs at > > http://www.python.org/doc/faq/ ? > > They would then go into the main docs, I guess? That would make sense. However, we already have seven different FAQs there, so I don't think they should all be added as peers of the Tutorial, Library Reference, etc. Maybe there should be a single 'FAQ' document, with the existing faqs as subsections. Another option might be to put them on an entirely new site such as faq.python.org, and then configure www.python.org/doc/faq/ to proxy for it. Should I look at implementing the python.org as an output style for Sphinx? --amk From rdmurray at bitdance.com Fri Sep 18 01:27:23 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 19:27:23 -0400 (EDT) Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: On Thu, 17 Sep 2009 at 14:19, Fred Drake wrote: > One of the reasons www.python.org/doc/ was considered less discoverable was > the about of only-sometimes-interesting information there; docs.python.org > contains only "current" docs (for some vague notion of current and only, > given that dev builds and both Python 2 and Python 3 versions). Some claimed > that having "docs" at the front of the URL helped, though I don't recall why. It's anecdotal, but I can tell you that currently I often type 'http://dev.python.org', get a not found, smack my head, and retype the url. I don't have that problem with docs, obviously :) --David From srittau at jroger.in-berlin.de Fri Sep 18 02:24:23 2009 From: srittau at jroger.in-berlin.de (Sebastian Rittau) Date: Fri, 18 Sep 2009 02:24:23 +0200 Subject: [Python-Dev] conceptual clarity In-Reply-To: References: <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com> Message-ID: <20090918002423.GB7534@jroger.in-berlin.de> On Thu, Sep 17, 2009 at 02:04:11PM -0400, R. David Murray wrote: > I mean, eg, IPv4Network.fromHostIP('192.168.1.1/24'). I'd actually suggest to use >>> net, host = parse_network_and_host("192.168.111.33/24") (IPv4Network('192.168.111.0/24'), IPv4Address('192.168.111.33')) >>> I think this helps the use case of the short network+gateway notation, while keeping the concepts of network and host address cleanly separate. > I would have IPv4Address itself be strict, and thus the new constructors > would compute the network address and call the regular IPv4Address > constructor.(*) Then the regular IPv4Network constructor could be strict and can be used for validation. - Sebastian From andrewm at object-craft.com.au Fri Sep 18 02:49:49 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri, 18 Sep 2009 10:49:49 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909180447.36475.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> <200909180447.36475.steve@pearwood.info> Message-ID: <20090918004949.8DE1D5AC0D6@longblack.object-craft.com.au> >On Thu, 17 Sep 2009 10:41:37 am Andrew McNamara wrote: >> In the olden days, the mask was spelled out in octets (eg >> 255.255.255.0). But we've moved to a more compact and logical >> notation where the number of leading significant bits is specified >> (eg /24). > >I hope you're not suggesting the older notation be unsupported? I would >expect to be able to use a mask like 255.255.255.192 without having to >count bits myself. No, of course not - I was just explaining the relationship between the two notations for people who may not have been aware. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From andrewm at object-craft.com.au Fri Sep 18 02:57:41 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri, 18 Sep 2009 10:57:41 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909180426.41808.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <87eiq612tu.fsf@uwakimon.sk.tsukuba.ac.jp> <20090917071516.217655AC0D6@longblack.object-craft.com.au> <200909180426.41808.steve@pearwood.info> Message-ID: <20090918005742.0B2A95AC0D6@longblack.object-craft.com.au> >> Conceptually, you sometimes need a bare address, and other times, >> you need an address with an associated network (host interface >> configs, router configs, etc). By AddressWithMask, I really mean >> AddressWithEnoughInformationToDeriveNetworkWhenNeeded. Conveniently, >> IPv4 and IPv6 addressing allows us to derive the network from the >> host address combined with the netmask - in other words, we don't >> have to attach a real Network object to Address objects until the >> user tries to access it, and then we derive it from the address and >> mask. > >To clarify: when you say "derive the network" are you talking about the >network (which is a container) or the network address = host_address & >netmask (which is not a container)? I think you're referring to the >later. I mean a Network object which is a container (which, by definition, has a network address + mask). >If there's need for address+netmask, does it need to be a separate >class? Perhaps Address objects could simply have a netmask property, >defaulting to None. If you need an "address with mask" object, you >create an Address and set the mask: > >addr = Address(...) >addr.netmask = "255.255.255.0" Greg Ewing suggested this yesterday - I'm neutral on whether it's done this way or as a separate class. The implementation may be somewhat cleaner if it's a separate class, however. >> If you have a .network attribute on an address object, checking if an >> address is in the same network as another address becomes: >> >> addr_a in addr_b.network >> >> As the module stands, you write that as: >> >> addr_a in addr_b >> >> I don't think the intent is as clear with the later. > >I would find the later completely unclear and disturbing -- how can one >address contain another address? Yes - that's how it works now, and I can only see it resulting in confusion and bugs for no advantage. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From andrewm at object-craft.com.au Fri Sep 18 03:04:46 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri, 18 Sep 2009 11:04:46 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909180344.31794.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB22E4F.40509@gmail.com> <200909180344.31794.steve@pearwood.info> Message-ID: <20090918010446.50EB15AC0D6@longblack.object-craft.com.au> >To a non-specialist, "the network address" is ambiguous. There are many >addresses in a network, and none of them are the entire network. It's >like saying, given a list [2, 4, 8, 12], what's "the list item"? A "network address" is an IP address and mask, but I understand your confusion - we're mixing terminology from disperate domains. In my postings, I have tried to refer to Network (a containter) and Address (an item). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From jnoller at gmail.com Fri Sep 18 03:13:17 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 17 Sep 2009 21:13:17 -0400 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: <20090917232652.GA1730@andrew-kuchlings-macbook.local> References: <20090917184410.GB13389@amk-desktop.matrixgroup.net> <20090917232652.GA1730@andrew-kuchlings-macbook.local> Message-ID: <4222a8490909171813i37ed0ee0i72e42223f3610130@mail.gmail.com> On Thu, Sep 17, 2009 at 7:26 PM, A.M. Kuchling wrote: > On Thu, Sep 17, 2009 at 11:32:25PM +0200, Georg Brandl wrote: >> > Should we do something similar with the FAQs at >> > http://www.python.org/doc/faq/ ? >> >> They would then go into the main docs, I guess? > > That would make sense. ?However, we already have seven different FAQs > there, so I don't think they should all be added as peers of the > Tutorial, Library Reference, etc. ?Maybe there should be a single > 'FAQ' document, with the existing faqs as subsections. > > Another option might be to put them on an entirely new site such as > faq.python.org, and then configure www.python.org/doc/faq/ to proxy > for it. > > Should I look at implementing the python.org as an output style for > Sphinx? > > --amk > I'm definitely pro dev.python.org or python.org/dev From andrewm at object-craft.com.au Fri Sep 18 03:17:25 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri, 18 Sep 2009 11:17:25 +1000 Subject: [Python-Dev] conceptual clarity In-Reply-To: <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> Message-ID: <20090918011725.604DE5AC0D6@longblack.object-craft.com.au> >off to patch the pep and implement some of the non controversial changes. It might be a good idea to add some use-cases to the PEP. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From andrewm at object-craft.com.au Fri Sep 18 03:41:32 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri, 18 Sep 2009 11:41:32 +1000 Subject: [Python-Dev] conceptual clarity In-Reply-To: <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> Message-ID: <20090918014132.1D2985AC0D6@longblack.object-craft.com.au> >Again, the same error-catching functionality can be obtained through >an option to the constructor. network and broadcast attributes can be >renamed to .\1_address to alleviate confusion as well. > >I mentioned before that IPy's insistence on receiving masked out >networks was one of the main reasons I wrote ipaddr to begin with. >Having ipaddr mimic this behavior would make it significantly less >useful. Removing functionality in the name of avoiding confusion >doesn't make sense when the same confusion can be alleviated w/o the >loss. The issue is bigger than error checking - I'm maintaining that a distinction between an Address (singleton, item) and a Network (Container) is useful and should be embraced. The current implementation has already partially gone this route, but hasn't completed the transition, and this does not give the structure to users that it could - there's an obligation on modules in the standard library to provide leadership and clarity without being dictatorial. They are essentially silent mentors. So, while I am not suggesting we build a bondage and discipline machine, I am suggesting that partitioning the functionality differently will result in a better outcome all round. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From rdmurray at bitdance.com Fri Sep 18 04:25:24 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 22:25:24 -0400 (EDT) Subject: [Python-Dev] conceptual clarity In-Reply-To: <20090918002423.GB7534@jroger.in-berlin.de> References: <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <8517e9350909171038l2aee898bg91c57c25b12b44b1@mail.gmail.com> <20090918002423.GB7534@jroger.in-berlin.de> Message-ID: On Fri, 18 Sep 2009 at 02:24, Sebastian Rittau wrote: > On Thu, Sep 17, 2009 at 02:04:11PM -0400, R. David Murray wrote: > >> I mean, eg, IPv4Network.fromHostIP('192.168.1.1/24'). > > I'd actually suggest to use > > >>> net, host = parse_network_and_host("192.168.111.33/24") > (IPv4Network('192.168.111.0/24'), IPv4Address('192.168.111.33')) > >>> > > I think this helps the use case of the short network+gateway notation, > while keeping the concepts of network and host address cleanly separate. > > Then the regular IPv4Network constructor could be strict and can be used > for validation. I'd be happy with this solution, which was also suggested earlier by someone else (Antoine?). --David From rdmurray at bitdance.com Fri Sep 18 04:30:24 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 17 Sep 2009 22:30:24 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090918010446.50EB15AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB22E4F.40509@gmail.com> <200909180344.31794.steve@pearwood.info> <20090918010446.50EB15AC0D6@longblack.object-craft.com.au> Message-ID: On Fri, 18 Sep 2009 at 11:04, Andrew McNamara wrote: >[attribution lost; apparently Steven D'Aprano given the CC] >> To a non-specialist, "the network address" is ambiguous. There are many >> addresses in a network, and none of them are the entire network. It's >> like saying, given a list [2, 4, 8, 12], what's "the list item"? > > A "network address" is an IP address and mask, but I understand your > confusion - we're mixing terminology from disperate domains. In my > postings, I have tried to refer to Network (a containter) and Address > (an item). Apparently not, in many people's vocabulary. The 'network address' is used to designate the IP address whose bits corresponding to the one bits in the mask have been set to zero (ie: the first IP address in the network range). It is interesting how this item seems to lead to the greatest amount of confusion on everyone's part, and I'm guessing it is because the common terminology and usage blurs the line between addresses and networks. And that's what we are trying to make clear(er) through a well structured API. --David From steve at pearwood.info Fri Sep 18 05:18:00 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Sep 2009 13:18:00 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090918010446.50EB15AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909180344.31794.steve@pearwood.info> <20090918010446.50EB15AC0D6@longblack.object-craft.com.au> Message-ID: <200909181318.00733.steve@pearwood.info> On Fri, 18 Sep 2009 11:04:46 am Andrew McNamara wrote: > >To a non-specialist, "the network address" is ambiguous. There are > > many addresses in a network, and none of them are the entire > > network. It's like saying, given a list [2, 4, 8, 12], what's "the > > list item"? > > A "network address" is an IP address and mask, but I understand your > confusion - we're mixing terminology from disperate domains. In my > postings, I have tried to refer to Network (a containter) and Address > (an item). So to clarify, how many different things which need to be handled are there? Items: 1 IP address -- a 32 bit (IPv4) or 128 bit (IPv6) number 2 Netmask -- a bit mask of the form 111..100..0 3 Network address -- the lowest address in a network, and equal to (defined by?) the bitwise-AND of any address in the network with the network's netmask 4 Host address -- the part of the IP address that is not masked by the netmask 5 Broadcast address -- the highest address in a IPv4 network Containers: 6 Network -- a range of IP address Have I missed anything or got anything wrong? -- Steven D'Aprano From steve at pearwood.info Fri Sep 18 05:19:44 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Sep 2009 13:19:44 +1000 Subject: [Python-Dev] conceptual clarity In-Reply-To: <20090918014132.1D2985AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <20090918014132.1D2985AC0D6@longblack.object-craft.com.au> Message-ID: <200909181319.45083.steve@pearwood.info> On Fri, 18 Sep 2009 11:41:32 am Andrew McNamara wrote: > The issue is bigger than error checking - I'm maintaining that a > distinction between an Address (singleton, item) and a Network > (Container) is useful and should be embraced. The current > implementation has already partially gone this route, but hasn't > completed the transition, and this does not give the structure to > users that it could - there's an obligation on modules in the > standard library to provide leadership and clarity without being > dictatorial. They are essentially silent mentors. > > So, while I am not suggesting we build a bondage and discipline > machine, I am suggesting that partitioning the functionality > differently will result in a better outcome all round. As one of those non-expert users, can I give a big THANK YOU to Andrew for this. -- Steven D'Aprano From peter at hda3.com Fri Sep 18 05:29:21 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 20:29:21 -0700 Subject: [Python-Dev] conceptual clarity In-Reply-To: <20090918011725.604DE5AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <20090918011725.604DE5AC0D6@longblack.object-craft.com.au> Message-ID: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara wrote: >>off to patch the pep and implement some of the non controversial changes. > > It might be a good idea to add some use-cases to the PEP. There are several use-cases in the PEP already. The problem is, for every use-case where one can show that the existing implementation is confusing, I can come up with a use-case showing where the existing implementation makes more sense than anything proposed. Cheers, /peter > -- > Andrew McNamara, Senior Developer, Object Craft > http://www.object-craft.com.au/ > From andrewm at object-craft.com.au Fri Sep 18 05:42:03 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri, 18 Sep 2009 13:42:03 +1000 Subject: [Python-Dev] conceptual clarity In-Reply-To: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <20090918011725.604DE5AC0D6@longblack.object-craft.com.au> <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> Message-ID: <20090918034203.E16915AC0D6@longblack.object-craft.com.au> >> It might be a good idea to add some use-cases to the PEP. > >There are several use-cases in the PEP already. Maybe the use-cases deserve their own section in the PEP, or better yet, be pulled up into the Motivation section. >The problem is, for every use-case where one can show that the >existing implementation is confusing, I can come up with a use-case >showing where the existing implementation makes more sense than >anything proposed. Uh, I don't think that is the intention of use-cases - they're there to inform the design, rather than to show how a specific implementation can be used. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From peter at hda3.com Fri Sep 18 05:42:27 2009 From: peter at hda3.com (Peter Moody) Date: Thu, 17 Sep 2009 20:42:27 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <77c780b40909151016n4675292craab781d115723383@mail.gmail.com> <8517e9350909151036o44746c96p53d115d1814d78c4@mail.gmail.com> <77c780b40909151133l58bcf38cs62076e1c0111b04e@mail.gmail.com> <8517e9350909151136w29d81355ma71a74d68c1282f5@mail.gmail.com> <8517e9350909151343q4eb419d1v35ad5468bc2798a8@mail.gmail.com> Message-ID: <8517e9350909172042h21166d4bka8e1eb009ac1bf6d@mail.gmail.com> On Thu, Sep 17, 2009 at 11:10 AM, Daniel Fetchinson wrote: >>>>>> 188 (check that, 190) people have downloaded the 2.0 release in the >>>>>> last week (numbers publicly available from the code.google.com). I >>>>>> can't tell you how many (if any) have downloaded it via svn. >>>>> >>>>> Downloading and using are not the same thing. >>>> >>>> Correct, but there is a strong positive correlation between the two. >>>> If you have a better method for determining what you would consider an >>>> appropriate level of usage, I'm all ears. >>> >>> A good way of determining the level of usage would be pointing to open >>> source projects that are popular in the python community and which >>> incorporate your module. >> >> well, the 2.0 release is still new. codesearch.google.com shows some >> projects using the 1.x release; hopefully some of those 200 >> downloaders will put up some publicly indexable python code at some >> point. > > I think one first needs to wait until this happens, I meana large user > base is formed, before a meaningful discussion can be done on whether > to include it in the stdlib or not. The long and largely academic > thread here I think illustrates this point. Without a large user base > it's up to anybody's gut feelings what is 'right' and what 'feels > wrong'. I like the sound of this. I'm going to continue to work on the PEP and the reference implementation, but as Martin said earlier, there's no reason to hurry. any discussion on ipaddr and/or PEP 3144 can continue on ipaddr-py-dev at googlegroups.com in the meantime (past spam unfortunately means that you have to subscribe in order to post, apologies, but all are welcome). Hopefully, the continued work will help iron any remaining confusion of the API. Cheers, /peter > Cheers, > Daniel > > > -- > Psss, psss, put it down! - http://www.cafepress.com/putitdown > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From steve at pearwood.info Fri Sep 18 05:45:11 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Sep 2009 13:45:11 +1000 Subject: [Python-Dev] conceptual clarity In-Reply-To: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090918011725.604DE5AC0D6@longblack.object-craft.com.au> <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> Message-ID: <200909181345.11638.steve@pearwood.info> On Fri, 18 Sep 2009 01:29:21 pm Peter Moody wrote: > On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara > > wrote: > >>off to patch the pep and implement some of the non controversial > >> changes. > > > > It might be a good idea to add some use-cases to the PEP. > > There are several use-cases in the PEP already. > > The problem is, for every use-case where one can show that the > existing implementation is confusing, I can come up with a use-case > showing where the existing implementation makes more sense than > anything proposed. Please excuse my skepticism, but I find that hard to believe. Given the current API, to test if an address is in the same network as another address, you write: addr_a in addr_b Can you please give a use-case where that makes more sense than the suggested alternative: addr_a in addr_b.network -- Steven D'Aprano From andrewm at object-craft.com.au Fri Sep 18 06:26:19 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Fri, 18 Sep 2009 14:26:19 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909181318.00733.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909180344.31794.steve@pearwood.info> <20090918010446.50EB15AC0D6@longblack.object-craft.com.au> <200909181318.00733.steve@pearwood.info> Message-ID: <20090918042620.0D3B85AC0D6@longblack.object-craft.com.au> >On Fri, 18 Sep 2009 11:04:46 am Andrew McNamara wrote: >> >To a non-specialist, "the network address" is ambiguous. There are >> > many addresses in a network, and none of them are the entire >> > network. It's like saying, given a list [2, 4, 8, 12], what's "the >> > list item"? >> >> A "network address" is an IP address and mask, but I understand your >> confusion - we're mixing terminology from disperate domains. In my >> postings, I have tried to refer to Network (a containter) and Address >> (an item). > >So to clarify, how many different things which need to be handled are >there? > >Items: >1 IP address -- a 32 bit (IPv4) or 128 bit (IPv6) number Yes. >2 Netmask -- a bit mask of the form 111..100..0 I don't think there's much to be gained by exposing a Netmask object, although other objects might have a .netmask property returning an IPAddress instance. Where we expose a netmask, it should be as an Address instance (or maybe a subclass with additional restrictions). >3 Network address -- the lowest address in a network, and equal > to (defined by?) the bitwise-AND of any address in the network > with the network's netmask This idea of a "network address" being simply an IP address is in error - a network address was always an address and a mask, however in the days prior to CIDR, the mask was implicitly specified by the class of the network. >4 Host address -- the part of the IP address that is not masked > by the netmask Well, yes, but I don't think we need an entity representing that. >5 Broadcast address -- the highest address in a IPv4 network Yes, but again, we don't need an entity - as with the netmask, when exposed, it should just be an Address instance (or subclass thereof). >Containers: >6 Network -- a range of IP address Yes, although not an arbitrary or discontinuous range of address. Really, I think we just need two entities (per protocol): Address (& maybe AddressWithMask) * If no mask is specified, this is just the IP address. * If a mask is specified, then it gains a .network property returning a Network instance. It probably should also have a .netmask property containing an Address instance. Network * Has an IP address with netmask * for consistency's sake, masked address bits are not allowed * behaves like a read-only container wrt Addresses So, you want to represent an interface on your host: >>> if_addr = IPv4Address('10.0.0.1/24') from this, you could get: >>> if_addr.address IPv4Address('10.0.0.1') >>> if_addr.netmask IPv4Address('255.255.255.0') >>> if_addr.broadcast IPv4Address('10.0.0.255') >>> if_addr.network IPV4Network('10.0.0.0/24') you might also have an address for the default gateway: >>> router_addr = IPv4Address('10.0.0.254/24') >>> router_addr in if_addr.network True or: >>> router_addr = IPv4Address('10.0.0.254') >>> router_addr in if_addr.network True Or maybe you've subneted your LAN: >>> IPV4Network('10.0.0.0/24') in IPv4Network('10.0.0.0/8') True >>> IPV4Network('10.0.1.0/24') in IPv4Network('10.0.0.0/8') True but: >>> IPV4Network('10.0.0.0/8') in IPv4Network('10.0.0.0/24') False This suggests the natural behaviour if the Address mask doesn't fit in the network: >>> IPv4Address('10.0.0.254/8') in IPv4Network('10.0.0.0/24') False -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From ncoghlan at gmail.com Fri Sep 18 11:11:21 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 18 Sep 2009 19:11:21 +1000 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: <4AB34EB9.4080306@gmail.com> Fred Drake wrote: > On Thu, Sep 17, 2009 at 10:38, Georg Brandl wrote: >> So the plan would be to consolidate these into another set of rst docs, >> having them in the repo, editable by every committer, as well as >> published >> somewhere on python.org (devdocs.python.org or somesuch). > > On Sep 17, 2009, at 1:56 PM, Brett Cannon wrote: >> dev.python.org would be nice to have, from which we can simply >> redirect www.python.org/dev/ to dev.python.org. www.python.org/dev/ >> can then get cleaned up be made simpler to navigate and more obvious >> for how people can get started. > > Many years ago, we decided to add docs.python.org with the "current" > version of the documentation, so people would be able to find the docs > more easily. Since then, we've had problems with keeping > docs.python.org and www.python.org/doc/ in sync, and with different > styles being applied to the sites. We can avoid that problem by having the URLs just be alternative names for the same thing (as Brett suggested), rather than having them be actually different sites (as is the case for docs.python.org and www.python.org/doc). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From p.f.moore at gmail.com Fri Sep 18 12:37:32 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Fri, 18 Sep 2009 11:37:32 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB22E4F.40509@gmail.com> <200909180344.31794.steve@pearwood.info> <20090918010446.50EB15AC0D6@longblack.object-craft.com.au> Message-ID: <79990c6b0909180337x536b0be2xa183bb3cb56f8d3d@mail.gmail.com> 2009/9/18 R. David Murray : > On Fri, 18 Sep 2009 at 11:04, Andrew McNamara wrote: >> >> [attribution lost; apparently Steven D'Aprano given the CC] >>> >>> To a non-specialist, "the network address" is ambiguous. There are many >>> addresses in a network, and none of them are the entire network. It's >>> like saying, given a list [2, 4, 8, 12], what's "the list item"? >> >> A "network address" is an IP address and mask, but I understand your >> confusion - we're mixing terminology from disperate domains. In my >> postings, I have tried to refer to Network (a containter) and Address >> (an item). > > Apparently not, in many people's vocabulary. ?The 'network address' > is used to designate the IP address whose bits corresponding to the one > bits in the mask have been set to zero (ie: the first IP address in the > network range). > > It is interesting how this item seems to lead to the greatest amount of > confusion on everyone's part, and I'm guessing it is because the common > terminology and usage blurs the line between addresses and networks. > And that's what we are trying to make clear(er) through a well structured > API. How non-expert do you want to get? My immediate reaction is that the network address of my PC is 157.215.187.94 - it's what comes up ad "IP address" when I type ipconfig. I understand why that's "wrong", and I see why the definitions above are "better", but that doesn't affect my instinct. Suggestion - just don't use the term "network address" anywhere in the library, the PEP, or its documentation. The term seems too loaded with contradictory meanings to be useful. If the concept behind it (one of the many concepts - among our concepts...) is useful, then maybe coin a new term specific to the module. Paul. From rdmurray at bitdance.com Fri Sep 18 15:24:32 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Fri, 18 Sep 2009 09:24:32 -0400 (EDT) Subject: [Python-Dev] conceptual clarity In-Reply-To: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <20090918011725.604DE5AC0D6@longblack.object-craft.com.au> <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> Message-ID: On Thu, 17 Sep 2009 at 20:29, Peter Moody wrote: > On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara > wrote: >>> off to patch the pep and implement some of the non controversial changes. >> >> It might be a good idea to add some use-cases to the PEP. > > There are several use-cases in the PEP already. > > The problem is, for every use-case where one can show that the > existing implementation is confusing, I can come up with a use-case > showing where the existing implementation makes more sense than > anything proposed. Then all of those use cases should be in the PEP, so we can try to find the best refinement we can of the API for handling all of them. --David From status at bugs.python.org Fri Sep 18 18:07:12 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 18 Sep 2009 18:07:12 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090918160712.81C0F7863C@psf.upfronthosting.co.za> ACTIVITY SUMMARY (09/11/09 - 09/18/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2396 open (+27) / 16366 closed (+29) / 18762 total (+56) Open issues with patches: 953 Average duration of open issues: 664 days. Median duration of open issues: 420 days. Open Issues Breakdown open 2363 (+27) pending 32 ( +0) Issues Created Or Reopened (56) _______________________________ pdb documentation shows running as script using "python" not "py 09/11/09 CLOSED http://bugs.python.org/issue6885 created MLModel cgi.py runs python, not python3 09/11/09 http://bugs.python.org/issue6886 created MLModel executables in lib use /usr/bin/env python, not python3 09/11/09 http://bugs.python.org/issue6887 created MLModel pdb alias command with no arguments is broken 09/11/09 CLOSED http://bugs.python.org/issue6888 created MLModel ??Python???s default encoding is the ???ascii??? encoding.?? 09/11/09 CLOSED http://bugs.python.org/issue6889 created flox IOError has no __unicode__ method - and loses information 09/11/09 CLOSED http://bugs.python.org/issue6890 created michael.foord 26backport small typo in ??unicode.rst??: ??the Uniode APIs should be used? 09/11/09 CLOSED http://bugs.python.org/issue6891 created flox optparser example in optparse documentation is broken 09/12/09 CLOSED http://bugs.python.org/issue6892 created MLModel defaultdict example in py3 doc should mention the new Counter cl 09/12/09 http://bugs.python.org/issue6893 created ezio.melotti urllib2 doesn't respect "no_proxy" environment (python2.6.2) 09/12/09 http://bugs.python.org/issue6894 created xelnor patch locale._parse_localename fails when localename does not contain 09/12/09 http://bugs.python.org/issue6895 created santhosh.thottingal patch Intermittent failures in test_mailbox 09/12/09 http://bugs.python.org/issue6896 created ezio.melotti imaplib fails during login 09/12/09 http://bugs.python.org/issue6897 created debatem1 Unicode Error 09/13/09 CLOSED http://bugs.python.org/issue6898 created vitvn Base.replace breaks tree 09/13/09 http://bugs.python.org/issue6899 created loewis patch Sub-optimal "Locate" button behaviour in Windows CHM file 09/13/09 http://bugs.python.org/issue6900 created tom_seddon Runaway programs often become unresponsive to CTRL-C 09/13/09 CLOSED http://bugs.python.org/issue6901 created bernie Built-in types format incorrectly with 0 padding. 09/13/09 http://bugs.python.org/issue6902 created eric.smith pdb - print in for iteration prints None after printing 09/14/09 CLOSED http://bugs.python.org/issue6903 created MLModel Broken URL for pyqt 09/14/09 CLOSED http://bugs.python.org/issue6904 created thijs inspect.getargspec(print) fails 09/14/09 CLOSED http://bugs.python.org/issue6905 created ubershmekel Tkinter sets an unicode environment variable on win32 09/14/09 http://bugs.python.org/issue6906 created dotz xmlrpclib.make_connection() is not thread safe 09/14/09 CLOSED http://bugs.python.org/issue6907 created haypo Minor markup error in hashlib docs 09/14/09 CLOSED http://bugs.python.org/issue6908 created lehmannro patch python 3.1 - filecmp.cmp exception based on file name 09/14/09 CLOSED http://bugs.python.org/issue6909 created jdeifik 1-char typo in language reference doc of import 09/14/09 CLOSED http://bugs.python.org/issue6910 created MLModel Document changes in asynchat 09/14/09 http://bugs.python.org/issue6911 created lehmannro patch Add 'with' block support to Tools/Scripts/pindent.py 09/14/09 CLOSED http://bugs.python.org/issue6912 created tjreedy Py_SetPythonHome incorrectly documented py3.x (rev 74792) 09/15/09 CLOSED http://bugs.python.org/issue6913 created ideasman42 patch Py_SetPythonHome, undocumented behavoir 09/15/09 CLOSED http://bugs.python.org/issue6914 created ideasman42 os.listdir inconsistenly releases the GIL on win32 09/15/09 http://bugs.python.org/issue6915 created rfk patch Remove deprecated items from asynchat 09/15/09 http://bugs.python.org/issue6916 created lehmannro patch ".et_folder" instead of "get_folder" in mailbox documentation 09/15/09 CLOSED http://bugs.python.org/issue6917 created dauerbaustelle ctypes compilation error on SnowLeopard 09/15/09 http://bugs.python.org/issue6918 created ronaldoussoren Link CRT Statically 09/16/09 CLOSED http://bugs.python.org/issue6919 created hankdane IDLE 3 crashes on Ctrl+Space with "'utf8' codec can't decode byt 09/16/09 CLOSED http://bugs.python.org/issue6920 created Yinon recursion wierdness ... related to variable scoping? 09/16/09 CLOSED http://bugs.python.org/issue6921 created mrdiskodave Interpreter hangs up while trying to decode invalid utf32 stream 09/16/09 CLOSED http://bugs.python.org/issue6922 created mwizard patch Need pthread_atfork-like functionality in CPython 09/16/09 http://bugs.python.org/issue6923 created twouters struct.unpack weird behavior with "bi" (byte then integer) 09/16/09 CLOSED http://bugs.python.org/issue6924 created Manux Doc for locals and vars 09/16/09 http://bugs.python.org/issue6925 created tjreedy socket module missing IPPROTO_IPV6, IPPROTO_IPV4 09/16/09 http://bugs.python.org/issue6926 created jaraco Metaclass doc (Ref 3.3.3) errors 09/16/09 http://bugs.python.org/issue6927 created tjreedy Doc: Class statements and metaclass = xxx 09/16/09 http://bugs.python.org/issue6928 created tjreedy Confusion between "write" method of rowiobase and rawfileio 09/17/09 CLOSED http://bugs.python.org/issue6929 created pakal [PATCH] PyUnicode_DecodeUTF16 docs wrong (byteorder param) 09/17/09 http://bugs.python.org/issue6930 created JonFoster patch dreadful performance in difflib: ndiff and HtmlDiff 09/17/09 http://bugs.python.org/issue6931 created heidar.rafn Open shelves fail when Python exits 09/17/09 CLOSED http://bugs.python.org/issue6932 created lehmannro patch Threading issue with Tkinter Frame.insert 09/17/09 http://bugs.python.org/issue6933 created moraygrieve [PATCH] postflight.framework script (from the Mac OS X .dmg inst 09/17/09 http://bugs.python.org/issue6934 created slavi patch Version updates needed in tutorial 09/17/09 CLOSED http://bugs.python.org/issue6935 created MLModel Import needed to quit Python? 09/17/09 CLOSED http://bugs.python.org/issue6936 created MLModel multiprocessing lock on OS X Snow Leopard dumps core 09/18/09 http://bugs.python.org/issue6937 created aaditya Incorrect multiprocessing log string in managers.py (python 2.6. 09/18/09 CLOSED http://bugs.python.org/issue6938 created ftofficer patch shadows around the io truncate() semantics 09/18/09 http://bugs.python.org/issue6939 created pakal cannot compute sizeof wchar_t on OS X 10.5.7, Python 2.6.2 09/18/09 http://bugs.python.org/issue6940 created rajarshi Issues Now Closed (60) ______________________ xmlrpclib closes connection after each call 580 days http://bugs.python.org/issue2076 haypo Error when printing an exception containing a Unicode string 535 days http://bugs.python.org/issue2517 ezio.melotti patch Windows installer should add Python and Scripts directories to t 396 days http://bugs.python.org/issue3561 srid Issue with RotatingFileHandler logging handler on Windows 99 days http://bugs.python.org/issue4749 gagenellina Add description of special case to "Assignment statements" secti 169 days http://bugs.python.org/issue5621 georg.brandl patch test_(zipfile|zipimport|gzip|distutils|sqlite) fail if zlib is n 121 days http://bugs.python.org/issue6026 ezio.melotti Add "intel" universal architecture on OSX 99 days http://bugs.python.org/issue6245 ronaldoussoren Bug in hashlib 93 days http://bugs.python.org/issue6281 Eloff Error in tutorial section 7.2 88 days http://bugs.python.org/issue6301 michael.foord Tkinter cannot find *64 bit* Tcl/Tk on Mac OS X 69 days http://bugs.python.org/issue6441 ronaldoussoren patch Make inf be almost equal to inf 50 days http://bugs.python.org/issue6567 michael.foord unittest test discovery improvements 51 days http://bugs.python.org/issue6568 michael.foord List the __future__ features in a table 50 days http://bugs.python.org/issue6574 lehmannro patch, easy Profiler doesn't print usage (indexError instead) 41 days http://bugs.python.org/issue6635 doko patch sys._getframe is not available on all Python implementations 28 days http://bugs.python.org/issue6712 michael.foord Python as zip package 25 days http://bugs.python.org/issue6767 senthil.kumaran 2to3 assignment division conversion 9 days http://bugs.python.org/issue6831 benjamin.peterson BaseException DeprecationError raises inappropriately 11 days http://bugs.python.org/issue6844 georg.brandl patch Tutorial changes 7 days http://bugs.python.org/issue6849 michael.foord easy allow setting uid and gid when creating tar files 5 days http://bugs.python.org/issue6856 lars.gustaebel patch TestLoader.loadTestsFromName does not use suiteClass attribute t 5 days http://bugs.python.org/issue6866 michael.foord patch sequence method .count() and .index() shoud be in immutable seq 2 days http://bugs.python.org/issue6874 tjreedy add os.close() suggestion to mkstemp documentation 6 days http://bugs.python.org/issue6875 georg.brandl patch readline documentation example error 6 days http://bugs.python.org/issue6876 georg.brandl misstatement in example explanation using raise 5 days http://bugs.python.org/issue6879 ezio.melotti class needs forward reference 5 days http://bugs.python.org/issue6880 georg.brandl pdb documentation shows running as script using "python" not "py 5 days http://bugs.python.org/issue6885 georg.brandl pdb alias command with no arguments is broken 0 days http://bugs.python.org/issue6888 benjamin.peterson ??Python???s default encoding is the ???ascii??? encoding.?? 0 days http://bugs.python.org/issue6889 benjamin.peterson IOError has no __unicode__ method - and loses information 5 days http://bugs.python.org/issue6890 georg.brandl 26backport small typo in ??unicode.rst??: ??the Uniode APIs should be used? 5 days http://bugs.python.org/issue6891 georg.brandl optparser example in optparse documentation is broken 4 days http://bugs.python.org/issue6892 georg.brandl Unicode Error 0 days http://bugs.python.org/issue6898 ezio.melotti Runaway programs often become unresponsive to CTRL-C 0 days http://bugs.python.org/issue6901 bernie pdb - print in for iteration prints None after printing 3 days http://bugs.python.org/issue6903 georg.brandl Broken URL for pyqt 0 days http://bugs.python.org/issue6904 ezio.melotti inspect.getargspec(print) fails 4 days http://bugs.python.org/issue6905 georg.brandl xmlrpclib.make_connection() is not thread safe 1 days http://bugs.python.org/issue6907 krisvale Minor markup error in hashlib docs 0 days http://bugs.python.org/issue6908 georg.brandl patch python 3.1 - filecmp.cmp exception based on file name 0 days http://bugs.python.org/issue6909 r.david.murray 1-char typo in language reference doc of import 1 days http://bugs.python.org/issue6910 ezio.melotti Add 'with' block support to Tools/Scripts/pindent.py 3 days http://bugs.python.org/issue6912 georg.brandl Py_SetPythonHome incorrectly documented py3.x (rev 74792) 0 days http://bugs.python.org/issue6913 benjamin.peterson patch Py_SetPythonHome, undocumented behavoir 0 days http://bugs.python.org/issue6914 benjamin.peterson ".et_folder" instead of "get_folder" in mailbox documentation 0 days http://bugs.python.org/issue6917 ezio.melotti Link CRT Statically 2 days http://bugs.python.org/issue6919 hankdane IDLE 3 crashes on Ctrl+Space with "'utf8' codec can't decode byt 0 days http://bugs.python.org/issue6920 ezio.melotti recursion wierdness ... related to variable scoping? 0 days http://bugs.python.org/issue6921 mrdiskodave Interpreter hangs up while trying to decode invalid utf32 stream 1 days http://bugs.python.org/issue6922 georg.brandl patch struct.unpack weird behavior with "bi" (byte then integer) 0 days http://bugs.python.org/issue6924 mark.dickinson Confusion between "write" method of rowiobase and rawfileio 0 days http://bugs.python.org/issue6929 pitrou Open shelves fail when Python exits 0 days http://bugs.python.org/issue6932 georg.brandl patch Version updates needed in tutorial 0 days http://bugs.python.org/issue6935 georg.brandl Import needed to quit Python? 0 days http://bugs.python.org/issue6936 georg.brandl Incorrect multiprocessing log string in managers.py (python 2.6. 0 days http://bugs.python.org/issue6938 georg.brandl patch distutils ignores configure's --includedir 1840 days http://bugs.python.org/issue1019715 senthil.kumaran os.path.splitext don't handle unix hidden file correctly 1684 days http://bugs.python.org/issue1115886 senthil.kumaran Function-level import in os triggering an threaded import deadlo 104 days http://bugs.python.org/issue1590864 twouters patch xmlrpclib ServerProxy uses old httplib interface 1010 days http://bugs.python.org/issue1613573 techtonik patch Make xmlrpc use HTTP/1.1 and keepalive 772 days http://bugs.python.org/issue1767370 haypo patch Top Issues Most Discussed (10) ______________________________ 14 should we include argparse 8 days open http://bugs.python.org/issue6247 10 BaseException DeprecationError raises inappropriately 11 days closed http://bugs.python.org/issue6844 9 uuid creates zombies 7 days open http://bugs.python.org/issue6882 9 xmlrpclib ServerProxy uses old httplib interface 1010 days closed http://bugs.python.org/issue1613573 7 multiprocessing lock on OS X Snow Leopard dumps core 0 days open http://bugs.python.org/issue6937 7 expose setresuid 63 days open http://bugs.python.org/issue6508 7 encode and decode should accept 'errors' as a keyword argument 93 days open http://bugs.python.org/issue6300 5 Confusion between "write" method of rowiobase and rawfileio 0 days closed http://bugs.python.org/issue6929 5 Link CRT Statically 2 days closed http://bugs.python.org/issue6919 5 IDLE 2.6.1 locks up on Mac OS 10.6 10 days open http://bugs.python.org/issue6864 From theller at ctypes.org Fri Sep 18 19:13:36 2009 From: theller at ctypes.org (Thomas Heller) Date: Fri, 18 Sep 2009 19:13:36 +0200 Subject: [Python-Dev] OS X buildbot Message-ID: I have updated the OS X buildbot to Snow Leopard. -- Thomas From chambon.pascal at gmail.com Fri Sep 18 21:17:29 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Fri, 18 Sep 2009 21:17:29 +0200 Subject: [Python-Dev] Fuzziness in io module specs Message-ID: <4AB3DCC9.5050209@gmail.com> Hello everyone I'm currently working on a reimplementation of io.FileIO, which would allow cross-platform file range locking and all kinds of other safety features ; however I'm slightly stuck due to some specification fuzziness in the IO docs. CF http://bugs.python.org/issue6939 The main points that annoy me at the moment : - it is unclear what truncate() methods do with the file pointer, and even if the current implementation simply moves it to the truncation point, it's very contrary to the standard way of doing under unix, where the file pointer is normally left unchanged. Shouldn't we specify that the file pointer remains unmoved, and fix the _fileio module accordingly ? - exceptions are not always specified, and even if most of them are IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if we try to wrap a wrong file descriptor when instanciating a new FileIO). This might lead to bad program crashes if some people don't "refuse the temptation to guess" and only get prepared to catch IOErrors - the doc sometimes says that when we receive an empty string from a read() operation, without exceptions, it means the file is empty. However, with the current implementation, if we call file.read(0), we simply receive "", even though it doesn't mean that we're at EOF. Shouldn't we avoid this (rare, I admit) ambiguity on the return value, by preventing read(0) ? Or at least, note in the doc that (we receive an empty string) <-> (the file is at EOF OR we called read with 0 as parameter) ? Are there some arguments that I don't know, which lead to this or that particular implementation choice ? I'd strongly advocate very detailled specifications, letting no room for cross-platform subtilities (that's also a strong goal of my reimplemntation), since that new IO system (which saved me a lot of coding time, by the way) should become the base of many programs. So wouldn't it be a godo idea to write some kind of mini-pep, just to fix the corner cases of the current IO documentation ? I might handle it, if no more-knowledgeable people feels like it. Regards, Pascal From guido at python.org Fri Sep 18 21:32:14 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 18 Sep 2009 12:32:14 -0700 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <4AB3DCC9.5050209@gmail.com> References: <4AB3DCC9.5050209@gmail.com> Message-ID: Why not propose an update to the existing PEP to clarify the vaguenesses? On Fri, Sep 18, 2009 at 12:17 PM, Pascal Chambon wrote: > Hello everyone > > I'm currently working on a reimplementation of io.FileIO, which would allow > cross-platform file range locking and all kinds of other safety features ; > however I'm slightly stuck due to some specification fuzziness in the IO > docs. > CF ? ? http://bugs.python.org/issue6939 > > The main points that annoy me at the moment : > - it is unclear what truncate() methods do with the file pointer, and even > if the current implementation simply moves it to the truncation point, it's > very contrary to the standard way of doing under unix, where the file > pointer is normally left unchanged. Shouldn't we specify that the file > pointer remains unmoved, and fix the _fileio module accordingly ? > - exceptions are not always specified, and even if most of them are > IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if we > try to wrap a wrong file descriptor when instanciating a new FileIO). This > might lead to bad program crashes if some people don't "refuse the > temptation to guess" and only get prepared to catch IOErrors > - the doc sometimes says that when we receive an empty string from a read() > operation, without exceptions, it means the file is empty. However, with the > current implementation, if we call file.read(0), we simply receive "", even > though it doesn't mean that we're at EOF. Shouldn't we avoid this (rare, I > admit) ambiguity on the return value, by preventing read(0) ? Or at least, > note in the doc that (we receive an empty string) <-> (the file is at EOF OR > we called read with 0 as parameter) ? > > Are there some arguments that I don't know, which lead to this or that > particular implementation choice ? > I'd strongly advocate very detailled specifications, letting no room for > cross-platform subtilities (that's also a strong goal of my > reimplemntation), since that new IO system (which saved me a lot of coding > time, by the way) should become the base of many programs. > > So wouldn't it be a godo idea to write some kind of mini-pep, just to fix > the corner cases of the current IO documentation ? I might handle it, if no > more-knowledgeable people feels like it. > > Regards, > Pascal > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From python at mrabarnett.plus.com Fri Sep 18 21:55:00 2009 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 18 Sep 2009 20:55:00 +0100 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <4AB3DCC9.5050209@gmail.com> References: <4AB3DCC9.5050209@gmail.com> Message-ID: <4AB3E594.5090907@mrabarnett.plus.com> [Oops! Hit Send to soon] Pascal Chambon wrote: > Hello everyone > > I'm currently working on a reimplementation of io.FileIO, which would > allow cross-platform file range locking and all kinds of other safety > features ; however I'm slightly stuck due to some specification > fuzziness in the IO docs. > CF http://bugs.python.org/issue6939 > > The main points that annoy me at the moment : > - it is unclear what truncate() methods do with the file pointer, and > even if the current implementation simply moves it to the truncation > point, it's very contrary to the standard way of doing under unix, where > the file pointer is normally left unchanged. Shouldn't we specify that > the file pointer remains unmoved, and fix the _fileio module accordingly ? I think that this should be an invariant: 0 <= file pointer <= file size so the file pointer might sometimes have to be moved. > - exceptions are not always specified, and even if most of them are > IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if > we try to wrap a wrong file descriptor when instanciating a new FileIO). > This might lead to bad program crashes if some people don't "refuse the > temptation to guess" and only get prepared to catch IOErrors > - the doc sometimes says that when we receive an empty string from a > read() operation, without exceptions, it means the file is empty. > However, with the current implementation, if we call file.read(0), we > simply receive "", even though it doesn't mean that we're at EOF. > Shouldn't we avoid this (rare, I admit) ambiguity on the return value, > by preventing read(0) ? Or at least, note in the doc that (we receive an > empty string) <-> (the file is at EOF OR we called read with 0 as > parameter) ? > If you ask for 0 bytes then you get 0 bytes. > Are there some arguments that I don't know, which lead to this or that > particular implementation choice ? > I'd strongly advocate very detailled specifications, letting no room for > cross-platform subtilities (that's also a strong goal of my > reimplemntation), since that new IO system (which saved me a lot of > coding time, by the way) should become the base of many programs. > > So wouldn't it be a godo idea to write some kind of mini-pep, just to > fix the corner cases of the current IO documentation ? I might handle > it, if no more-knowledgeable people feels like it. > As for the question of whether 'truncate' should be able to lengthen a file, the method name suggests no; if the method name were 'resize', for example, then maybe yes, zeroing the new bytes for security. From mrabarnett at mrabarnett.plus.com Fri Sep 18 21:50:33 2009 From: mrabarnett at mrabarnett.plus.com (Matthew Barnett) Date: Fri, 18 Sep 2009 20:50:33 +0100 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <4AB3DCC9.5050209@gmail.com> References: <4AB3DCC9.5050209@gmail.com> Message-ID: <4AB3E489.3010506@mrabarnett.plus.com> Pascal Chambon wrote: > Hello everyone > > I'm currently working on a reimplementation of io.FileIO, which would > allow cross-platform file range locking and all kinds of other safety > features ; however I'm slightly stuck due to some specification > fuzziness in the IO docs. > CF http://bugs.python.org/issue6939 > > The main points that annoy me at the moment : > - it is unclear what truncate() methods do with the file pointer, and > even if the current implementation simply moves it to the truncation > point, it's very contrary to the standard way of doing under unix, where > the file pointer is normally left unchanged. Shouldn't we specify that > the file pointer remains unmoved, and fix the _fileio module accordingly ? I think that this should be an invariant: 0 <= file pointer <= file size so the file pointer might sometimes have to be moved. > - exceptions are not always specified, and even if most of them are > IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if > we try to wrap a wrong file descriptor when instanciating a new FileIO). > This might lead to bad program crashes if some people don't "refuse the > temptation to guess" and only get prepared to catch IOErrors > - the doc sometimes says that when we receive an empty string from a > read() operation, without exceptions, it means the file is empty. > However, with the current implementation, if we call file.read(0), we > simply receive "", even though it doesn't mean that we're at EOF. > Shouldn't we avoid this (rare, I admit) ambiguity on the return value, > by preventing read(0) ? Or at least, note in the doc that (we receive an > empty string) <-> (the file is at EOF OR we called read with 0 as > parameter) ? > If you ask for 0 bytes then you get 0 bytes. > Are there some arguments that I don't know, which lead to this or that > particular implementation choice ? > I'd strongly advocate very detailled specifications, letting no room for > cross-platform subtilities (that's also a strong goal of my > reimplemntation), since that new IO system (which saved me a lot of > coding time, by the way) should become the base of many programs. > > So wouldn't it be a godo idea to write some kind of mini-pep, just to > fix the corner cases of the current IO documentation ? I might handle > it, if no more-knowledgeable people feels like it. > From foom at fuhm.net Fri Sep 18 22:08:23 2009 From: foom at fuhm.net (James Y Knight) Date: Fri, 18 Sep 2009 16:08:23 -0400 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <4AB3E594.5090907@mrabarnett.plus.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> Message-ID: <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> On Sep 18, 2009, at 3:55 PM, MRAB wrote: > I think that this should be an invariant: > > 0 <= file pointer <= file size > > so the file pointer might sometimes have to be moved. > As for the question of whether 'truncate' should be able to lengthen a > file, the method name suggests no; if the method name were 'resize', > for > example, then maybe yes, zeroing the new bytes for security. Why are you just making things up? There is a *vast* amount of precedent for how file operations should work. Python should follow that precedent and do like POSIX unless there's a compelling reason not to. Quoting: If fildes refers to a regular file, the ftruncate() function shall cause the size of the file to be truncated to length. If the size of the file previously exceeded length, the extra data shall no longer be available to reads on the file. If the file previously was smaller than this size, ftruncate() shall either increase the size of the file or fail. XSI-conformant systems shall increase the size of the file. If the file size is increased, the extended area shall appear as if it were zero-filled. The value of the seek pointer shall not be modified by a call to ftruncate(). James From daniel at stutzbachenterprises.com Fri Sep 18 22:15:28 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Fri, 18 Sep 2009 15:15:28 -0500 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <4AB3DCC9.5050209@gmail.com> References: <4AB3DCC9.5050209@gmail.com> Message-ID: On Fri, Sep 18, 2009 at 2:17 PM, Pascal Chambon wrote: > - it is unclear what truncate() methods do with the file pointer, and even > if the current implementation simply moves it to the truncation point, it's > very contrary to the standard way of doing under unix, where the file > pointer is normally left unchanged. Shouldn't we specify that the file > pointer remains unmoved, and fix the _fileio module accordingly ? > +1 on having consistent, documented behavior (regardless of what that behavior is :) ). > - exceptions are not always specified, and even if most of them are > IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if we > try to wrap a wrong file descriptor when instanciating a new FileIO). This > might lead to bad program crashes if some people don't "refuse the > temptation to guess" and only get prepared to catch IOErrors > I'd wager that you may also get a WindowsError in some cases, on Windows systems. Part of the reason for having several different, but similar, exceptions is that they may contain operating system specific error codes and the type of exception helps the programmer figure out how to interpret those codes. I'm not really clear on when IOError is preferred over OSError, but I know that WindowsError necessarily uses a completely different error numbering system. The careful programmer should catch EnvironmentError, which is the base class of all of these different kinds of related errors. +1 on documenting that the methods may raise an EnvironmentError. > - the doc sometimes says that when we receive an empty string from a read() > operation, without exceptions, it means the file is empty. However, with the > current implementation, if we call file.read(0), we simply receive "", even > though it doesn't mean that we're at EOF. Shouldn't we avoid this (rare, I > admit) ambiguity on the return value, by preventing read(0) ? Or at least, > note in the doc that (we receive an empty string) <-> (the file is at EOF OR > we called read with 0 as parameter) ? > Some programs may rely on read(0) and the behavior matches the behavior of POSIX, so I'm -1 on changing the behavior. +1 on documenting the exception to the rule. > Are there some arguments that I don't know, which lead to this or that > particular implementation choice ? > The original I/O PEP and some of the code was put together during a sprint at PyCon 2007. Our primary goal was to get a decent first cut of the new I/O system put together quickly. For nitty-gritty details and corner-cases like these, there's a good chance that the current undocumented behavior is simply an accident of implementation. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Fri Sep 18 22:24:52 2009 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 18 Sep 2009 21:24:52 +0100 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> Message-ID: <4AB3EC94.3020606@mrabarnett.plus.com> James Y Knight wrote: > > On Sep 18, 2009, at 3:55 PM, MRAB wrote: > >> I think that this should be an invariant: >> >> 0 <= file pointer <= file size >> >> so the file pointer might sometimes have to be moved. > >> As for the question of whether 'truncate' should be able to lengthen a >> file, the method name suggests no; if the method name were 'resize', for >> example, then maybe yes, zeroing the new bytes for security. > > > Why are you just making things up? There is a *vast* amount of precedent > for how file operations should work. Python should follow that precedent > and do like POSIX unless there's a compelling reason not to. Quoting: > > If fildes refers to a regular file, the ftruncate() > function shall cause the size of the file to be truncated to > length. If the size of the file previously exceeded length, the > extra data shall no longer be available to reads on the > file. If the file previously was smaller than this size, > ftruncate() shall either increase the size of the file or > fail. XSI-conformant systems shall increase the size of the > file. If the file size is increased, the extended area > shall appear as if it were zero-filled. The value of the seek > pointer shall not be modified by a call to ftruncate(). > "making things up"? I'm just expressing an opinion! From solipsis at pitrou.net Fri Sep 18 22:38:51 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 18 Sep 2009 20:38:51 +0000 (UTC) Subject: [Python-Dev] Fuzziness in io module specs References: <4AB3DCC9.5050209@gmail.com> Message-ID: Le Fri, 18 Sep 2009 21:17:29 +0200, Pascal Chambon a ?crit?: Hello, First, thanks for experimenting with this. (as a sidenote, we lack real-world testing of non-blocking features, perhaps you want to take a look) > I'm currently working on a reimplementation of io.FileIO, which would > allow cross-platform file range locking and all kinds of other safety > features ; Hmm, do you *have* to reimplement it in order to add these new features, or is it just a personal preference? > - it is unclear what truncate() methods do with the file pointer, and > even if the current implementation simply moves it to the truncation > point, it's very contrary to the standard way of doing under unix, where > the file pointer is normally left unchanged. Shouldn't we specify that > the file pointer remains unmoved, and fix the _fileio module accordingly ? Well, first Python and its IO library are cross-platform, so the behaviour is not always identical to POSIX behaviour, especially where Windows and Unix have different specs. Second, now that 3.1 is in the wild, we should be reluctant to change the behaviour just to make it more conformant to what POSIX people can expect. What might be convincing would be an actual use case where POSIX- like behaviour would be significantly more useful than the current on. > - exceptions are not always specified, and even if most of them are > IOErrors, weirdly, in some cases, an OSError is raised instead (ie, if > we try to wrap a wrong file descriptor when instanciating a new FileIO). This is not different than with 2.x here. If you want to trap both OSError and IOError, use EnvironmentError (which is the common base class for both). I agree it is slightly annoying and not well-defined, however. Also, Python can hardly determine by itself whether an error is caused by IO problems or an OS-level malfunction, so the distinction is a bit fallacious. > However, with the current implementation, if we call file.read(0), we > simply receive "", even though it doesn't mean that we're at EOF. > Shouldn't we avoid this (rare, I admit) ambiguity on the return value, > by preventing read(0) ? Well, if you are asking for 0 bytes, it returns 0 bytes. It's not that ambiguous, and it helps avoid special-casing the 0 case :) > So wouldn't it be a godo idea to write some kind of mini-pep, just to > fix the corner cases of the current IO documentation ? Improvements, either to the docs or to the implementation, are always welcome. I think you already know where to post them! Regards Antoine. From solipsis at pitrou.net Fri Sep 18 22:40:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 18 Sep 2009 20:40:14 +0000 (UTC) Subject: [Python-Dev] POSIX [Fuzziness in io module specs] References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> Message-ID: James Y Knight fuhm.net> writes: > > Why are you just making things up? There is a *vast* amount of > precedent for how file operations should work. Python should follow > that precedent and do like POSIX unless there's a compelling reason > not to. Actually, Python is cross-platform and therefore does not necessarily follow POSIX behaviour, especially when it is desired to hide inconsistencies between different platform. (I do agree, of course, that the IO APIs are quite heavily inspired by the POSIX APIs) Regards Antoine. From stephen at xemacs.org Sat Sep 19 02:19:53 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 19 Sep 2009 09:19:53 +0900 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> Message-ID: <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > James Y Knight fuhm.net> writes: > > > > Why are you just making things up? There is a *vast* amount of > > precedent for how file operations should work. Python should follow > > that precedent and do like POSIX unless there's a compelling reason > > not to. > > Actually, Python is cross-platform and therefore does not > necessarily follow POSIX behaviour, especially when it is desired > to hide inconsistencies between different platform. That's what *James* said, except that I prefer his standard, because I believe POSIX documentation to be more accessible to a variety of Python developers than other system's, and it's better documented: rationales are included, history is available, etc. From solipsis at pitrou.net Sat Sep 19 02:58:08 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 19 Sep 2009 00:58:08 +0000 (UTC) Subject: [Python-Dev] POSIX [Fuzziness in io module specs] References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: Le Sat, 19 Sep 2009 09:19:53 +0900, Stephen J. Turnbull a ?crit?: > Antoine Pitrou writes: > > James Y Knight fuhm.net> writes: > > > > > > Why are you just making things up? There is a *vast* amount of > > > precedent for how file operations should work. Python should follow > > > that precedent and do like POSIX unless there's a compelling reason > > > not to. > > > > Actually, Python is cross-platform and therefore does not necessarily > > follow POSIX behaviour, especially when it is desired to hide > > inconsistencies between different platform. > > That's what *James* said, except that I prefer his standard, I don't believe that POSIX compliance is a sufficient argument to ask someone to shut up in the discussion of a cross-platform API. Which is more or less what James' answer was trying to do. So, no, not exactly the same thing that I said. > I > believe POSIX documentation to be more accessible to a variety of Python > developers than other system's, and it's better documented: rationales > are included, history is available, etc. I'm not sure that's true. Various Unix/Linux man pages are readily available on the Internet, but they regard specific implementations, which often depart from the spec in one way or another. POSIX specs themselves don't seem to be easily reachable; you might even have to pay for them. cheers Antoine. From foom at fuhm.net Sat Sep 19 03:51:11 2009 From: foom at fuhm.net (James Y Knight) Date: Fri, 18 Sep 2009 21:51:11 -0400 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: On Sep 18, 2009, at 8:58 PM, Antoine Pitrou wrote: > I'm not sure that's true. Various Unix/Linux man pages are readily > available on the Internet, but they regard specific implementations, > which often depart from the spec in one way or another. POSIX specs > themselves don't seem to be easily reachable; you might even have to > pay > for them. The POSIX specs are quite easily accessible, without payment. I got my quote by doing: man 3p ftruncate I had previously done: apt-get install manpages-posix-dev to install the posix manpages. That package contains the POSIX standard as of 2003. Which is good enough for most uses. It seems to be available here, if you don't have a debian system: http://www.kernel.org/pub/linux/docs/man-pages/man-pages-posix/ There's also a webpage, containing the official POSIX 2008 standard: http://www.opengroup.org/onlinepubs/9699919799/ And to navigate to ftruncate from there, click "System Interfaces" in the left pane, "System Interfaces" in the bottom pane, and then "ftruncate" in the bottom pane. James From greg.ewing at canterbury.ac.nz Sat Sep 19 07:35:48 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 19 Sep 2009 17:35:48 +1200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB238D9.1020204@trueblade.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <4AB1A39B.7090003@canterbury.ac.nz> <8517e9350909162026s52d95309w1fc9716309a9dca5@mail.gmail.com> <4AB22E4F.40509@gmail.com> <4AB238D9.1020204@trueblade.com> Message-ID: <4AB46DB4.2010902@canterbury.ac.nz> Eric Smith wrote: > My only concern with this is a possible performance issue with v6 > networks. Would this be implemented such that [-1] doesn't need to > iterate through the (possibly large) address space of a v6 network? I'm not familiar with v6, but if netmasks work the same way as they do in v4, then there's no need to iterate over anything -- it's just a matter of turning on all the low bits of the address. -- Greg From stephen at xemacs.org Sat Sep 19 09:53:00 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 19 Sep 2009 16:53:00 +0900 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <87ocp7z7z7.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > I don't believe that POSIX compliance is a sufficient argument to ask > someone to shut up in the discussion of a cross-platform API. Which is > more or less what James' answer was trying to do. No, as I read it, James said, "when there's precedent, even a standard, don't make stuff up". He then referred to the POSIX standard, which I assume means that's a standard he likes. But he didn't say it had to be POSIX, or else shut up. He said, "respect precedent" (and implied that ignorance is no excuse, I guess, which is a little harsh on the Internet). And I think he'd agree to weaken his dictum to add "and if there's good reason to vary from the standard, either quote other standards or give rationale referring to why you want to vary from the standard." > I'm not sure that's true [that POSIX is readily available]. [...] > POSIX specs themselves don't seem to be easily reachable; you might > even have to pay for them. Five minutes with Google gives http://standards.ieee.org/catalog/olis/arch_posix.html (IEEE members only) http://www.unix.org/version3/ (registration but no fee required) http://www.opengroup.org/onlinepubs/000095399/toc.htm (title page for above, lets you sneak past the registration, and has higher Googlefu) Older versions also seem to be readily available. From chambon.pascal at gmail.com Sat Sep 19 09:46:58 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sat, 19 Sep 2009 09:46:58 +0200 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB489C9.7000908@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> Message-ID: <4AB48C72.8060603@gmail.com> @pitrou: non-blocking IO in python ? which ones are you thinking about ? I have currently no plan to work on asynchronous IO like win32's readFileEx() etc. (too many troubles for the benefit), however I'd be interested by getting non-blocking operations on IPC pipes (I've crossed several people in trouble with that, having a process never end on some OSes because they couldn't stop threads blocked on pipes). This reimplementation is actually necessary to get file locking, because advanced win32 operations only work on real file handles, not the handles that are underlying the C API layer. Furthermore, some interesting features (like O_EXCL | O_CREAT) are not possible with the current io implementations. So well, reimplementation required ^^ Else, allright, I'll try to summarize the various points in a PEP-update. Concerning the "truncate" method however, on second thought I feel we might take distance from Posix API for naming, precisely since it's anyway too "platform-specific" (windows knows nothing about Posix, and even common unix-like systems modify it in a way or another - several systems don't zero-fill files when extending them). When seeing "truncate", in my opinion, most people will think "it's only to reduce file size" (for beginners), or will immediately get in mind all the tips of posix-like systems (for more experienced developers). Shouldn't we, like other cross-platform APIs, use a more unambiguous notion, like "setLength" (java) or "resize" (Qt) ? And let the filepointer untouched, simply because there are no reasons to move it, especially when extending the file (yep, on windows we're forced to move the pointer, but it's easy to fix) ? If it's too late to modify the IO API, too bad, but I don't feel comfortable with the "truncate" word. And I don't like the fact that we move the filepointer to prevent it from exceeding the file size, whereas on the other hand we can seek() anywhere without getting exceptions (and so, set the filepointer past the end of file). Having 0 <= filepointer <= EOF is OK to me, but then we have to enforce it for all functions, not just truncate. Concerning exceptions, which one is raised is not so important to me, as long as it's well documented and not tricky (eg. WindowsErrors are OK to me, because they subclass OSError, so most cross-platform programs wont even have to know about them). I had the feeling that IOErrors were for operations on file streams (opening, writing/reading, closing...), whereas OSErrors were for manipulations on filesystems (renaming, linking, stating...) and processes. This semantic would be perfect for me, and it's already 95% here, we would just have to fix some unwelcomed OSErrors exceptions in the iomodule. Isn't that worth it ? It'd simplify programmers' job a lot, and allow a more subtle treatment of exceptions (if everyone just catches Environment errors, without being sure of which subcless is actually raised, we miss the point of IOError and OSError). Regards, Pascal > > > > James Y Knight a ?crit : >> >> On Sep 18, 2009, at 8:58 PM, Antoine Pitrou wrote: >>> I'm not sure that's true. Various Unix/Linux man pages are readily >>> available on the Internet, but they regard specific implementations, >>> which often depart from the spec in one way or another. POSIX specs >>> themselves don't seem to be easily reachable; you might even have to >>> pay >>> for them. >> >> >> The POSIX specs are quite easily accessible, without payment. >> >> I got my quote by doing: >> man 3p ftruncate >> >> I had previously done: >> apt-get install manpages-posix-dev >> to install the posix manpages. That package contains the POSIX >> standard as of 2003. Which is good enough for most uses. It seems to >> be available here, if you don't have a debian system: >> http://www.kernel.org/pub/linux/docs/man-pages/man-pages-posix/ >> >> There's also a webpage, containing the official POSIX 2008 standard: >> http://www.opengroup.org/onlinepubs/9699919799/ >> >> And to navigate to ftruncate from there, click "System Interfaces" in >> the left pane, "System Interfaces" in the bottom pane, and then >> "ftruncate" in the bottom pane. >> >> James >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/firephoenix%40wanadoo.fr >> >> >> > > From dave.l.harrison at gmail.com Sat Sep 19 09:59:34 2009 From: dave.l.harrison at gmail.com (David Harrison) Date: Sat, 19 Sep 2009 17:59:34 +1000 Subject: [Python-Dev] conceptual clarity In-Reply-To: <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <4AB19F44.8010102@canterbury.ac.nz> <4AB2282A.3050902@trueblade.com> <4AB22C71.1040205@gmail.com> <8517e9350909170916k70b45504wbd4e47e9422d286e@mail.gmail.com> <20090918011725.604DE5AC0D6@longblack.object-craft.com.au> <8517e9350909172029s190ab8a6xcf23830a939b6bbf@mail.gmail.com> Message-ID: 2009/9/18 Peter Moody : > On Thu, Sep 17, 2009 at 6:17 PM, Andrew McNamara > wrote: >>>off to patch the pep and implement some of the non controversial changes. >> >> It might be a good idea to add some use-cases to the PEP. > > There are several use-cases in the PEP already. > > The problem is, for every use-case where one can show that the > existing implementation is confusing, I can come up with a use-case > showing where the existing implementation makes more sense than > anything proposed. > > Cheers, > /peter For any given use-case there is bound to be an opposing one, but the goal is to satisfy the most common or relevant use-cases that we all experience - can't please all the people all the time etc. I think that alot of the confusion and disagreements surrounding the API are due to the absence of use-cases in the PEP - we can all probably agree an IP address module has intrinsic merit, but how we expect to use it may vary significantly. As a result the discussion seems to be driven by the example implementation, instead of the by the use-cases that make the module worthwhile being added - I would normally think it would be the other way round. Without a clear set of expectations and use-cases, it would also be hard to propose an alternate implementation. From daniel at stutzbachenterprises.com Sat Sep 19 11:24:29 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 19 Sep 2009 04:24:29 -0500 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB48C72.8060603@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com> Message-ID: On Sat, Sep 19, 2009 at 2:46 AM, Pascal Chambon wrote: > This reimplementation is actually necessary to get file locking, because > advanced win32 operations only work on real file handles, not the handles > that are underlying the C API layer. Furthermore, some interesting features > (like O_EXCL | O_CREAT) are not possible with the current io > implementations. So well, reimplementation required ^^ > > Concerning exceptions, which one is raised is not so important to me, as > long as it's well documented and not tricky (eg. WindowsErrors are OK to me, > because they subclass OSError, so most cross-platform programs wont even > have to know about them). > If you use real Windows file handles (instead of the POSIX-ish Windows API), won't you need to return WindowsErrors? > I had the feeling that IOErrors were for operations on file streams > (opening, writing/reading, closing...), whereas OSErrors were for > manipulations on filesystems (renaming, linking, stating...) and processes. > If that were documented and a firm rule, that would certainly be great. It's not too hard to find counterexamples in the current codebase. Also, I'm not sure how one could avoid needing to raise WindowsError in some cases. Maybe someone with more knowledge of the history of IOError vs. OSError could chime in. Python 2.6: >>> os.write(f.fileno(), 'blah') Traceback (most recent call last): File "", line 1, in OSError: [Errno 9] Bad file descriptor >>> f.write('blah') Traceback (most recent call last): File "", line 1, in IOError: [Errno 9] Bad file descriptor -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From chambon.pascal at gmail.com Sat Sep 19 12:31:15 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sat, 19 Sep 2009 12:31:15 +0200 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com> Message-ID: <4AB4B2F3.8060207@gmail.com> Good example with "os.write(f.fileno(), 'blah')" - and you obtain the same error if you try to open an io.FileIo by providing a file descriptor instead of a file name as first argument. This would really deserve an unification. Actually, since Windows Error Codes concern any possible error (IO, file permissions, memory problems...), I thought the best would be to convert them to the most appropriate python standard exception, only defaulting to WindowsError (i.e, OSError's hierarchy) when no other exception type matches. So at the moment, I use a decorator to automatically convert all errors on stream operations into IOErrors. Error codes are not the same as unix ones indeed, but I don't know if it's really important (imo, most people just want to know if the operation was successful, I don't know if many developers scan error codes to act accordingly). For IOError types that really matter (eg. file already locked, buffer full), the easiest is actually to use subclasses of IOError (the io module already does that, even though I'll maybe have to create new exceptions for errors like "file already exists" or "file already locked by another process") Regards, Pascal Daniel Stutzbach a ?crit : > On Sat, Sep 19, 2009 at 2:46 AM, Pascal Chambon > > wrote: > > This reimplementation is actually necessary to get file locking, > because advanced win32 operations only work on real file handles, > not the handles that are underlying the C API layer. Furthermore, > some interesting features (like O_EXCL | O_CREAT) are not possible > with the current io implementations. So well, reimplementation > required ^^ > > > > Concerning exceptions, which one is raised is not so important to > me, as long as it's well documented and not tricky (eg. > WindowsErrors are OK to me, because they subclass OSError, so most > cross-platform programs wont even have to know about them). > > > If you use real Windows file handles (instead of the POSIX-ish Windows > API), won't you need to return WindowsErrors? > > > I had the feeling that IOErrors were for operations on file > streams (opening, writing/reading, closing...), whereas OSErrors > were for manipulations on filesystems (renaming, linking, > stating...) and processes. > > > If that were documented and a firm rule, that would certainly be > great. It's not too hard to find counterexamples in the current > codebase. Also, I'm not sure how one could avoid needing to raise > WindowsError in some cases. > > Maybe someone with more knowledge of the history of IOError vs. > OSError could chime in. > > Python 2.6: > > >>> os.write(f.fileno(), 'blah') > Traceback (most recent call last): > File "", line 1, in > OSError: [Errno 9] Bad file descriptor > >>> f.write('blah') > Traceback (most recent call last): > File "", line 1, in > IOError: [Errno 9] Bad file descriptor > > -- > Daniel Stutzbach, Ph.D. > President, Stutzbach Enterprises, LLC > > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/firephoenix%40wanadoo.fr > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel at stutzbachenterprises.com Sat Sep 19 12:37:07 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 19 Sep 2009 05:37:07 -0500 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB4B2F3.8060207@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com> <4AB4B2F3.8060207@gmail.com> Message-ID: On Sat, Sep 19, 2009 at 5:31 AM, Pascal Chambon wrote: > Actually, since Windows Error Codes concern any possible error (IO, file > permissions, memory problems...), I thought the best would be to convert > them to the most appropriate python standard exception, only defaulting to > WindowsError (i.e, OSError's hierarchy) when no other exception type > matches. So at the moment, I use a decorator to automatically convert all > errors on stream operations into IOErrors. Error codes are not the same as > unix ones indeed, but I don't know if it's really important (imo, most > people just want to know if the operation was successful, I don't know if > many developers scan error codes to act accordingly). > I don't often need to check the error code at runtime but seeing the corresponding message is often critical for debugging. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Sat Sep 19 12:52:28 2009 From: cs at zip.com.au (Cameron Simpson) Date: Sat, 19 Sep 2009 20:52:28 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AB2AE99.8070006@gmail.com> References: <4AB2AE99.8070006@gmail.com> Message-ID: <20090919105228.GA10341@cskk.homeip.net> On 18Sep2009 07:48, Nick Coghlan wrote: | Eric Smith wrote: | > Peter Moody wrote: | >> indexing is plenty efficient, but the problem is that these names for | >> these attributes are common to the point of causing confusion if | >> they're omitted. | > | > After thinking about it some more, I'm okay with names for [-1] and [0]. | > I like .broadcast, and I can live with .network (although it's | > confusing, since in my view it should return an address, not a network). | | And in fact *does* return an Address - the same address that my_net[0] | returns. Yes, I think Eric was complaining about the name being "network", since we have Network objects and this isn't one. -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ From eric at trueblade.com Sat Sep 19 12:54:36 2009 From: eric at trueblade.com (Eric Smith) Date: Sat, 19 Sep 2009 06:54:36 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090919105228.GA10341@cskk.homeip.net> References: <4AB2AE99.8070006@gmail.com> <20090919105228.GA10341@cskk.homeip.net> Message-ID: <4AB4B86C.3010308@trueblade.com> Cameron Simpson wrote: > On 18Sep2009 07:48, Nick Coghlan wrote: > | Eric Smith wrote: > | > Peter Moody wrote: > | >> indexing is plenty efficient, but the problem is that these names for > | >> these attributes are common to the point of causing confusion if > | >> they're omitted. > | > > | > After thinking about it some more, I'm okay with names for [-1] and [0]. > | > I like .broadcast, and I can live with .network (although it's > | > confusing, since in my view it should return an address, not a network). > | > | And in fact *does* return an Address - the same address that my_net[0] > | returns. > > Yes, I think Eric was complaining about the name being "network", since > we have Network objects and this isn't one. Right. thing_commonly_referred_to_as_network_but_is_an_address. From pythoniks at gmail.com Sat Sep 19 09:35:37 2009 From: pythoniks at gmail.com (Pascal Chambon) Date: Sat, 19 Sep 2009 09:35:37 +0200 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4AB489C9.7000908@gmail.com> @pitrou: non-blocking IO in python ? which ones are you thinking about ? I have currently no plan to work on asynchronous IO like win32's readFileEx() etc. (too many troubles for the benefit), however I'd be interested by getting non-blocking operations on IPC pipes (I've crossed several people in trouble with that, having a process never end on some OSes because they couldn't stop threads blocked on pipes). This reimplementation is actually necessary to get file locking, because advanced win32 operations only work on real file handles, not the handles that are underlying the C API layer. Furthermore, some interesting features (like O_EXCL | O_CREAT) are not possible with the current io implementations. So well, reimplementation required ^^ Else, allright, I'll try to summarize the various points in a PEP-update. Concerning the "truncate" method however, on second thought I feel we might take distance from Posix API for naming, precisely since it's anyway too "platform-specific" (windows knows nothing about Posix, and even common unix-like systems modify it in a way or another - several systems don't zero-fill files when extending them). When seeing "truncate", in my opinion, most people will think "it's only to reduce file size" (for beginners), or will immediately get in mind all the tips of posix-like systems (for more experienced developers). Shouldn't we, like other cross-platform APIs, use a more unambiguous notion, like "setLength" (java) or "resize" (Qt) ? And let the filepointer untouched, simply because there are no reasons to move it, especially when extending the file (yep, on windows we're forced to move the pointer, but it's easy to fix) ? If it's too late to modify the IO API, too bad, but I don't feel comfortable with the "truncate" word. And I don't like the fact that we move the filepointer to prevent it from exceeding the file size, whereas on the other hand we can seek() anywhere without getting exceptions (and so, set the filepointer past the end of file). Having 0 <= filepointer <= EOF is OK to me, but then we have to enforce it for all functions, not just truncate. Concerning exceptions, which one is raised is not so important to me, as long as it's well documented and not tricky (eg. WindowsErrors are OK to me, because they subclass OSError, so most cross-platform programs wont even have to know about them). I had the feeling that IOErrors were for operations on file streams (opening, writing/reading, closing...), whereas OSErrors were for manipulations on filesystems (renaming, linking, stating...) and processes. This semantic would be perfect for me, and it's already 95% here, we would just have to fix some unwelcomed OSErrors exceptions in the iomodule. Isn't that worth it ? It'd simplify programmers' job a lot, and allow a more subtle treatment of exceptions (if everyone just catches Environment errors, without being sure of which subcless is actually raised, we miss the point of IOError and OSError). Regards, Pascal James Y Knight a ?crit : > > On Sep 18, 2009, at 8:58 PM, Antoine Pitrou wrote: >> I'm not sure that's true. Various Unix/Linux man pages are readily >> available on the Internet, but they regard specific implementations, >> which often depart from the spec in one way or another. POSIX specs >> themselves don't seem to be easily reachable; you might even have to pay >> for them. > > > The POSIX specs are quite easily accessible, without payment. > > I got my quote by doing: > man 3p ftruncate > > I had previously done: > apt-get install manpages-posix-dev > to install the posix manpages. That package contains the POSIX > standard as of 2003. Which is good enough for most uses. It seems to > be available here, if you don't have a debian system: > http://www.kernel.org/pub/linux/docs/man-pages/man-pages-posix/ > > There's also a webpage, containing the official POSIX 2008 standard: > http://www.opengroup.org/onlinepubs/9699919799/ > > And to navigate to ftruncate from there, click "System Interfaces" in > the left pane, "System Interfaces" in the bottom pane, and then > "ftruncate" in the bottom pane. > > James > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/firephoenix%40wanadoo.fr > > > From martin at v.loewis.de Sat Sep 19 13:09:40 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 19 Sep 2009 13:09:40 +0200 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <4AB4BBF4.4070509@v.loewis.de> >> I >> believe POSIX documentation to be more accessible to a variety of Python >> developers than other system's, and it's better documented: rationales >> are included, history is available, etc. > > I'm not sure that's true. Various Unix/Linux man pages are readily > available on the Internet, but they regard specific implementations, > which often depart from the spec in one way or another. POSIX specs > themselves don't seem to be easily reachable; you might even have to pay > for them. Then please rest assured that it actually *is* true: - the Linux man pages are often a literal copy of the POSIX man pages, so when you look at a Linux man page, there is a good chance that it either is the wording of the POSIX spec, or points out what details are specific to POSIX. - The Open Group publishes POSIX free of charge, and these days also free of registration. - these days, specific implementations typically do *not* deviate from POSIX. Some provide additional features, but in a way that does not harm compatibility. Regards, Martin From rdmurray at bitdance.com Sat Sep 19 14:47:57 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sat, 19 Sep 2009 08:47:57 -0400 (EDT) Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB4B2F3.8060207@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com> <4AB4B2F3.8060207@gmail.com> Message-ID: On Sat, 19 Sep 2009 at 12:31, Pascal Chambon wrote: > stream operations into IOErrors. Error codes are not the same as unix ones > indeed, but I don't know if it's really important (imo, most people just want > to know if the operation was successful, I don't know if many developers scan > error codes to act accordingly). For IOError types that really matter (eg. Doesn't matter if it isn't very many, I think, just that it can be done. But I suspect it is fairly common. I know I have inspected OSError codes (though I can't remember if I've done it for file operations). --David From ncoghlan at gmail.com Sat Sep 19 15:01:23 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 19 Sep 2009 23:01:23 +1000 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB4B2F3.8060207@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com> <4AB4B2F3.8060207@gmail.com> Message-ID: <4AB4D623.8050405@gmail.com> Pascal Chambon wrote: > (imo, most people just want to know if the operation was successful, I > don't know if many developers scan error codes to act accordingly). And as a user of applications written by those developers, it is a practice I detest with a passion. Debugging environmental problems is painful enough without further encouraging applications that lie to their users as to what has actually gone wrong. For example, a file not existing, a file being locked by another process, and the user not having write permissions to the file are problems that demand very different responses from the user. Applications that give the same error message for all three problems are far too common. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Sat Sep 19 19:04:13 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 19 Sep 2009 17:04:13 +0000 (UTC) Subject: [Python-Dev] POSIX [Fuzziness in io module specs] References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> Message-ID: Hello, Pascal Chambon gmail.com> writes: > @pitrou: non-blocking IO in python ? which ones are you thinking about ? I was talking about the existing support for non-blocking IO in the FileIO class (look up EAGAIN in fileio.c), as well as in the Buffered* objects. > If it's too late to modify the IO API, too bad, but I don't feel > comfortable with the "truncate" word. It's certainly too late to modify the IO API only for naming purposes. > And I don't like the fact that we > move the filepointer to prevent it from exceeding the file size, I don't see what you mean: >>> with open('foobar', 'wb') as f: ... f.truncate(0) ... 0 >>> os.stat('foobar').st_size 0 >>> with open('foobar', 'wb') as f: ... f.truncate(16) ... f.tell() ... 16 16 >>> os.stat('foobar').st_size 16 > I had the feeling that IOErrors were for operations on file streams > (opening, writing/reading, closing...), whereas OSErrors were for > manipulations on filesystems (renaming, linking, stating...) and > processes. Ok, but the distinction is certainly fuzzy in many cases. I have no problem with trying to change the corner cases you mention, though. Regards Antoine. From chambon.pascal at gmail.com Sat Sep 19 19:43:31 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sat, 19 Sep 2009 19:43:31 +0200 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> Message-ID: <4AB51843.1030603@gmail.com> Antoine Pitrou a ?crit : > Hello, > > Pascal Chambon gmail.com> writes: > >> @pitrou: non-blocking IO in python ? which ones are you thinking about ? >> > > I was talking about the existing support for non-blocking IO in the FileIO class > (look up EAGAIN in fileio.c), as well as in the Buffered* objects. > > Allright, I'll check that EAGAIN stuff, that I hadn't even noticed :) >> And I don't like the fact that we >> move the filepointer to prevent it from exceeding the file size, >> > > I don't see what you mean: > > Well the sample code you showed is not shocking, but I'd like to have a coherency the with file.seek(), because if truncate() prevents out-of-bound file pointer, other methods should do the same as well (and raise IOError when seeking out of file bounds). >> I had the feeling that IOErrors were for operations on file streams >> (opening, writing/reading, closing...), whereas OSErrors were for >> manipulations on filesystems (renaming, linking, stating...) and >> processes. >> > > Ok, but the distinction is certainly fuzzy in many cases. I have no problem with > trying to change the corner cases you mention, though. > The case which could be problematic there is the file opening, because it can involve problems at all levels of the OS (filesystem not existing, permission problems, file locking...), so we should keep it in the "EnvironmentError" area. But as soon as a file is open, I guess only IOErrors can be involved (no space left, range locked etc), so enforcing all this to raise IOError would be OK I think. -------------- next part -------------- An HTML attachment was scrubbed... URL: From tmm at fastmail.fm Sat Sep 19 20:51:33 2009 From: tmm at fastmail.fm (qwavel) Date: Sat, 19 Sep 2009 11:51:33 -0700 (PDT) Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> Message-ID: <25524412.post@talk.nabble.com> Barry Warsaw wrote: > > On Sep 15, 2009, at 9:53 AM, Ronald Oussoren wrote: > >> The IDLE issue is IMHO a release blocker, as is issue 6851. > > So that leaves 4 release blockers for 2.6.3. > What about this bug: http://bugs.python.org/issue3890 It appears to me that the SSL module is broken in the 2.6.x line on all platforms in one of its most common uses (non-blocking). It also seems that the problem and solution are well understood, so the solution would make a good candidate for 2.6.3? Tom. -- View this message in context: http://www.nabble.com/Python-2.6.3-tp25364517p25524412.html Sent from the Python - python-dev mailing list archive at Nabble.com. From janssen at parc.com Sat Sep 19 22:40:59 2009 From: janssen at parc.com (Bill Janssen) Date: Sat, 19 Sep 2009 13:40:59 PDT Subject: [Python-Dev] Python 2.6.3 In-Reply-To: <25524412.post@talk.nabble.com> References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> <25524412.post@talk.nabble.com> Message-ID: <18464.1253392859@parc.com> qwavel wrote: > What about this bug: > http://bugs.python.org/issue3890 > It appears to me that the SSL module is broken in the 2.6.x line on all > platforms in one of its most common uses (non-blocking). It also seems that > the problem and solution are well understood, so the solution would make a > good candidate for 2.6.3? > Tom. I might quibble about "most common uses", but I agree this would be a good idea. Bill From steve at pearwood.info Sat Sep 19 23:55:03 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 20 Sep 2009 07:55:03 +1000 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB4B2F3.8060207@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB4B2F3.8060207@gmail.com> Message-ID: <200909200755.04172.steve@pearwood.info> On Sat, 19 Sep 2009 08:31:15 pm Pascal Chambon wrote: > Error codes are not the > same as unix ones indeed, but I don't know if it's really important > (imo, most people just want to know if the operation was successful, > I don't know if many developers scan error codes to act accordingly). I do. Please don't throw away useful information. -- Steven D'Aprano From greg.ewing at canterbury.ac.nz Sun Sep 20 01:43:13 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 20 Sep 2009 11:43:13 +1200 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB48C72.8060603@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com> Message-ID: <4AB56C91.2080005@canterbury.ac.nz> Pascal Chambon wrote: > > And let the > filepointer untouched, simply because there are no reasons to move it, On some systems it may not be possible for the file pointer to be positioned past the end of the file (I think Classic MacOS was like that). > I had the feeling that IOErrors were for operations on file streams > (opening, writing/reading, closing...), whereas OSErrors were for > manipulations on filesystems (renaming, linking, stating...) I always thought the distinction was that IOError contains a C library errno value, whereas OSError contains an OS-specific error code. So system calls that aren't part of the C stdlib need to use OSError, at least on some platforms. I don't see that file errors vs. everything else is a very useful distinction to make when catching exceptions. I almost always end up catching EnvironmentError to make sure I get both, particularly when working cross-platform. What we could do with is better platform-independent ways of distinguishing particular error conditions, such as file not found, out of space, etc., either using subclasses of IOError or mapping error codes to a set of platform-independent ones. -- Greg From steve at pearwood.info Sun Sep 20 02:07:11 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 20 Sep 2009 10:07:11 +1000 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> Message-ID: <200909201007.11923.steve@pearwood.info> On Sat, 19 Sep 2009 06:08:23 am James Y Knight wrote: > On Sep 18, 2009, at 3:55 PM, MRAB wrote: > > I think that this should be an invariant: > > > > 0 <= file pointer <= file size > > > > so the file pointer might sometimes have to be moved. > > > > As for the question of whether 'truncate' should be able to > > lengthen a file, the method name suggests no; if the method name > > were 'resize', for > > example, then maybe yes, zeroing the new bytes for security. > > Why are you just making things up? Well, why not? The POSIX standard wasn't derived from the physical laws of the universe, somebody -- or more likely, some committee -- made them up. It's not like we are forced to accept MRAB's suggestions, but surely he's allowed to make them? > There is a *vast* amount of > precedent for how file operations should work. Python should follow > that precedent and do like POSIX unless there's a compelling reason > not to. Quoting: > > If fildes refers to a regular file, the ftruncate() > function shall cause the size of the file to be truncated to > length. If the size of the file previously exceeded length, > the extra data shall no longer be available to reads on the > file. If the file previously was smaller than this size, > ftruncate() shall either increase the size of the file or > fail. XSI-conformant systems shall increase the size of the > file. If the file size is increased, the extended area > shall appear as if it were zero-filled. The value of the seek > pointer shall not be modified by a call to ftruncate(). Standard or not, it has a seriously misleading name. Truncate means "to cut off", and is a standard English word and mathematical term. In English and mathematics, truncating something can NEVER lead it to increase in size, the object being truncated can only decrease or remain the same size. Using "truncate" to mean "increase in size" makes about as much sense as having a list method called "remove" used to insert items. I can't imagine what the committee who approved this were thinking. Java's FileChannel object uses truncate to only shrink a file, never increase it. To increase it, you have to explicitly write to the file. http://java.sun.com/j2se/1.4.2/docs/api/java/nio/channels/FileChannel.html#truncate(long) In any case, some platforms don't allow the file pointer to exceed the size of the file, so it's not clear that the POSIX standard is appropriate for Python. -- Steven D'Aprano From greg.ewing at canterbury.ac.nz Sun Sep 20 02:23:20 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 20 Sep 2009 12:23:20 +1200 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB4D623.8050405@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com> <4AB4B2F3.8060207@gmail.com> <4AB4D623.8050405@gmail.com> Message-ID: <4AB575F8.9040107@canterbury.ac.nz> Nick Coghlan wrote: > For example, a file not existing, a file being locked by another > process, and the user not having write permissions to the file are > problems that demand very different responses from the user. You can display an error-specific message without having to inspect the error code, e.g. try: something_with_file(path) except EnvironmentError, e: report_error("Couldn't do that with %s: %s" % (path, e)) This is a pattern I use a lot, and it seems to work pretty well. -- Greg From greg.ewing at canterbury.ac.nz Sun Sep 20 03:11:53 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 20 Sep 2009 13:11:53 +1200 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <200909201007.11923.steve@pearwood.info> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> Message-ID: <4AB58159.3040303@canterbury.ac.nz> Steven D'Aprano wrote: > Using "truncate" to mean "increase in size" makes about as much sense as > having a list method called "remove" used to insert items. I can't > imagine what the committee who approved this were thinking. I expect the reason is historical. Some time back in the early days of Unix, someone wanted a way of chopping back files, so they added a truncate() system call. Then someone else noticed that it would happily accept an argument greater than the existing length, and it seemed like that could be useful behaviour, so they documented it and left it that way. Then the POSIX committee came along and incorporated it into the standard so as to be compatible with existing practice. -- Greg From chambon.pascal at gmail.com Sun Sep 20 09:16:27 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sun, 20 Sep 2009 09:16:27 +0200 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <4AB58159.3040303@canterbury.ac.nz> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> Message-ID: <4AB5D6CB.6050604@gmail.com> Well, system compatibility argues strongl in favor of not letting filepointer > EOF. However, is that really necessary to move the pointer to EOF in ANY case ? I mean, if I extend the file, or if I reduce it without going lower than my current filepointer, I really don't expect at all the io system to move my pointer to the end of file, "just for fun". In these patterns, people would have to remember their current filepointer, to come back to where they were, and that's not pretty imo... If we agree on the simple mandatory expression 0 <= filepointer <= EOF (for cross-platform safety), then we just have to enforce it when the rule is broken : reducing the size lower than the filepointer, and seeking past the end of file. All other conditions should leav the filepointer where the user put it. Shouldnt it be so ? Concerning the naming of truncate(), would it be possible to deprecate it and alias it to "resize()" ? It's not very gratifying to have duplicated methods at the beginning of a major release, but I feel too that "truncate" is a misleading term, that had better be replaced asap. Regards, Pascal From chambon.pascal at gmail.com Sun Sep 20 09:24:16 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sun, 20 Sep 2009 09:24:16 +0200 Subject: [Python-Dev] POSIX [Fuzziness in io module specs] In-Reply-To: <4AB56C91.2080005@canterbury.ac.nz> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <87r5u3zsye.fsf@uwakimon.sk.tsukuba.ac.jp> <4AB489C9.7000908@gmail.com> <4AB48C72.8060603@gmail.com> <4AB56C91.2080005@canterbury.ac.nz> Message-ID: <4AB5D8A0.1060202@gmail.com> > > What we could do with is better platform-independent > ways of distinguishing particular error conditions, > such as file not found, out of space, etc., either > using subclasses of IOError or mapping error codes > to a set of platform-independent ones. > Well, mapping all errors (including C ones and windows-specific ones) to a common set would be extremely useful for developers indeed. I guess some advanced windows errors will never have equivalents elsewhere, but does anyone know an error code set which would be relevant to cover all memorty, filesystem, io and locking aspects ? Regards, Pascal From greg.ewing at canterbury.ac.nz Sun Sep 20 10:33:33 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 20 Sep 2009 20:33:33 +1200 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <4AB5D6CB.6050604@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> Message-ID: <4AB5E8DD.6050204@canterbury.ac.nz> Pascal Chambon wrote: > Concerning the naming of truncate(), would it be possible to deprecate > it and alias it to "resize()" ? It's not very gratifying to have > duplicated methods at the beginning of a major release, but I feel too > that "truncate" is a misleading term, that had better be replaced asap. There's something to be said for that, but there's also something to be said for following established conventions, and there's a long-established precedent from the C library for having a function called truncate() that behaves this way. -- Greg From chambon.pascal at gmail.com Sun Sep 20 11:48:49 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sun, 20 Sep 2009 11:48:49 +0200 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition In-Reply-To: <4AB5E8DD.6050204@canterbury.ac.nz> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> Message-ID: <4AB5FA81.5060408@gmail.com> Hello After weighing up here and that, here is what I have come with. Comments and issue notifications more than welcome, of course. The exception thingy is not yet addressed. Regards, Pascal *Truncate and file pointer semantics* Rationale : The current implementation of truncate() always move the file pointer to the new end of file. This behaviour is interesting for compatibility, if the file has been reduced and the file pointer is now past its end, since some platforms might require 0 <= filepointer <= filesize. However, there are several arguments against this semantic: * Most common standards (posix, win32...) allow the file pointer to be past the end of file, and define the behaviour of other stream methods in this case * In many cases, moving the filepointer when truncating has no reasons to happen (if we're extending the file, or reducing it without going beneath the file pointer) * Making 0 <= filepointer <= filesize a global rule of the python IO module doesn't seems possible, since it would require modifications of the semantic of other methods (eg. seek() should raise exceptions or silently disobey when asked to move the filepointer past the end of file), and lead to incoherent situations when concurrently accessing files without locking (what if another process truncates to 0 bytes the file you're writing ?) So here is the proposed semantic, which matches established conventions: *RawIOBase.truncate(n: int = None) -> int* *(same for BufferedIOBase.truncate(pos: int = None) -> int)* Resizes the file to the size specified by the positive integer n, or by the current filepointer position if n is None. The file must be opened with write permissions. If the file was previously larger than n, the extra data is discarded. If the file was previously shorter than n, its size is increased, and the extended area appears as if it were zero-filled. In any case, the file pointer is left unchanged, and may point beyond the end of file. Note: trying to read past the end of file returns an empty string, and trying to write past the end of file extends it by zero-ing the gap. On rare platforms which don't support file pointers to be beyond the end of file, all these behaviours shall be faked thanks to internal storage of the "wanted" file pointer position (silently extending the file, if necessary, when a write operation occurs). *Proposition of doc update* *RawIOBase*.read(n: int) -> bytes Read up to n bytes from the object and return them. Fewer than n bytes may be returned if the operating system call returns fewer than n bytes. If 0 bytes are returned, and n was not 0, this indicates end of file. If the object is in non-blocking mode and no bytes are available, the call returns None. *RawIOBase*.readinto(b: bytes) -> int Read up to len(b) bytes from the object and stores them in b, returning the number of bytes read. Like .read, fewer than len(b) bytes may be read, and 0 indicates end of file if b is not 0. None is returned if a non-blocking object has no bytes available. The length of b is never changed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at mrabarnett.plus.com Sun Sep 20 14:57:45 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 20 Sep 2009 13:57:45 +0100 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition In-Reply-To: <4AB5FA81.5060408@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> Message-ID: <4AB626C9.1010304@mrabarnett.plus.com> Pascal Chambon wrote: > Hello > > After weighing up here and that, here is what I have come with. Comments > and issue notifications more than welcome, of course. The exception > thingy is not yet addressed. > > Regards, > Pascal > > > *Truncate and file pointer semantics* > > Rationale : > > The current implementation of truncate() always move the file pointer to > the new end of file. > > This behaviour is interesting for compatibility, if the file has been > reduced and the file pointer is now past its end, since some platforms > might require 0 <= filepointer <= filesize. > > However, there are several arguments against this semantic: > > * Most common standards (posix, win32?) allow the file pointer to be > past the end of file, and define the behaviour of other stream > methods in this case > * In many cases, moving the filepointer when truncating has no > reasons to happen (if we?re extending the file, or reducing it > without going beneath the file pointer) > * Making 0 <= filepointer <= filesize a global rule of the python IO > module doesn?t seems possible, since it would require > modifications of the semantic of other methods (eg. seek() should > raise exceptions or silently disobey when asked to move the > filepointer past the end of file), and lead to incoherent > situations when concurrently accessing files without locking (what > if another process truncates to 0 bytes the file you?re writing ?) > > So here is the proposed semantic, which matches established conventions: > > *RawIOBase.truncate(n: int = None) -> int* > > *(same for BufferedIOBase.truncate(pos: int = None) -> int)* > > Resizes the file to the size specified by the positive integer n, or by > the current filepointer position if n is None. > The new size could be positive or zero. > The file must be opened with write permissions. > > If the file was previously larger than n, the extra data is discarded. > If the file was previously shorter than n, its size is increased, and > the extended area appears as if it were zero-filled. > > In any case, the file pointer is left unchanged, and may point beyond > the end of file. > > Note: trying to read past the end of file returns an empty string, and > trying to write past the end of file extends it by zero-ing the gap. On > rare platforms which don?t support file pointers to be beyond the end of > file, all these behaviours shall be faked thanks to internal storage of > the ?wanted? file pointer position (silently extending the file, if > necessary, when a write operation occurs). > > > > *Proposition of doc update* > > *RawIOBase*.read(n: int) -> bytes > > Read up to n bytes from the object and return them. Fewer than n bytes > may be returned if the operating system call returns fewer than n bytes. > If 0 bytes are returned, and n was not 0, this indicates end of file. If > the object is in non-blocking mode and no bytes are available, the call > returns None. > > *RawIOBase*.readinto(b: bytes) -> int > > Read up to len(b) bytes from the object and stores them in b, returning > the number of bytes read. Like .read, fewer than len(b) bytes may be > read, and 0 indicates end of file if b is not 0. None is returned if a > non-blocking object has no bytes available. The length of b is never > changed. > I thought 'bytes' was immutable! If you're going to read into a list or array, it would be nice to also be able to give the start index and either the end index or the count (start defaults to 0, end defaults to len). From python at mrabarnett.plus.com Sun Sep 20 14:59:20 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 20 Sep 2009 13:59:20 +0100 Subject: [Python-Dev] Fuzziness in io module specs In-Reply-To: <4AB5E8DD.6050204@canterbury.ac.nz> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> Message-ID: <4AB62728.5080408@mrabarnett.plus.com> Greg Ewing wrote: > Pascal Chambon wrote: > >> Concerning the naming of truncate(), would it be possible to >> deprecate it and alias it to "resize()" ? It's not very gratifying to >> have duplicated methods at the beginning of a major release, but I >> feel too that "truncate" is a misleading term, that had better be >> replaced asap. > > There's something to be said for that, but there's also > something to be said for following established conventions, > and there's a long-established precedent from the C library > for having a function called truncate() that behaves this > way. > But Python isn't C. :-) From solipsis at pitrou.net Sun Sep 20 15:08:07 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 20 Sep 2009 13:08:07 +0000 (UTC) Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> Message-ID: Hello, > *Truncate and file pointer semantics* [snip] The major problem here is that you are changing the current semantics. I don't think the argument you are making for it is strong enough to counter-balance the backwards compatibility issue. The situation would be different if 3.0 hadn't been released yet. Besides, we already broke compatibility with 3.0/3.1, let's not give users the impression that we don't care about compatibility anymore. Regards Antoine. From zookog at gmail.com Sun Sep 20 16:02:09 2009 From: zookog at gmail.com (Zooko O'Whielacronx) Date: Sun, 20 Sep 2009 08:02:09 -0600 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 Message-ID: Dear Pythonistas: This issue causes serious problems. Users occasionally get binaries built for a compatible Linux and Python version but with a different UCS2-vs-UCS4 setting, and those users get mysterious memory corruption errors which are hard to diagnose. It is possible that these situations also open up security vulnerabilities. A couple such instances are documented on http://bugs.python.org/setuptools/issue78, but you can find more by googling. I would like to get this problem fixed! In order to help address this issue I sampled what UCS size is used by python executables in the wild. I instrumented a few buildslaves that are contributed by various people to the Tahoe-LAFS project to print out their platform, python version, and sys.maxunicode. The full results are appended below. maxunicode: 1114111 means that python executable was configured with --enable-unicode=ucs4, and maxunicode: 65535 means that python executable was configured with --enable-unicode=ucs2 or just with --enable-unicode . The only incompatibilities that I found are because some packagers have deliberately set UCS4 configuration and other packagers have left the default setting. In the three cases where someone configured python with UCS2, one of the three is certainly an accident (a custom-built python executable on an Ubuntu server) and the other two just use the default instead of specifically configuring ucs2 in their configurations of Python and I suspect that they don't know the difference and that it was an accident that they built a Python incompatible with other distributions of their operating system. In sum, while it would be good to add the unicode setting to the platform's ABI (as discussed in setuptools ticket #78), it would also be good to make the default value be UCS4 instead of UCS2. This would fix all three of the potential incompatibilities that I found (listed below), and once we have proper inclusion of the unicode setting in the ABI in order to prevent the memory corruption, defaulting to UCS4 would increase the likelihood that a binary built on one distribution would be usable on another. I'm sure that someone can come up with a reason why UCS2 is better than UCS4, but I'm also sure that the benefits of compatibility outweigh any benefits of UCS2 encoding, and that the widespread use of UCS4 demonstrates that there is nothing fatally wrong with it, and that people who really value UCS2 encoding more than compatibility can choose that for themselves by explicitly setting UCS2. Let me restate that I am not suggesting taking away anyone's options, only making the setting for people who don't specify default to the compatible option. Hm, I guess that means that it should default to UCS2 on Windows and Mac and to UCS4 on Linux and Solaris. Regards, Zooko Ubuntu 6.10 "edgy" i386: python: 2.4.4c1 (#2, Mar 7 2008, 03:03:38) [GCC 4.1.2 20060928 (prerelease) (Ubuntu 4.1.1-13ubuntu5)], maxunicode: 1114111 Ubuntu 7.04 "feisty": python: 2.5.1 (r251:54863, Jul 31 2008, 22:53:39) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)], maxunicode: 1114111 Ubuntu 7.10 "gutsy" i386: python: 2.5.1 (r251:54863, Jul 31 2008, 23:17:40) [GCC 4.1.3 20070929 (prerelease) (Ubuntu 4.1.2-16ubuntu2)], maxunicode: 1114111 Ubuntu 8.04 "hardy" amd64: python: 2.5.2 (r252:60911, Jul 22 2009, 15:33:10) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)], maxunicode: 1114111 Ubuntu 8.04 "hardy" i386: *custom* python: 2.6 (r26:66714, Oct 2 2008, 13:40:28) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)], maxunicode: 65535 Ubuntu 8.04 "hardy" i386: python: 2.5.2 (r252:60911, Jul 22 2009, 15:35:03) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)], maxunicode: 1114111 Ubuntu 9.04 "jaunty" amd64: *custom* python: 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) [GCC 4.3.3], maxunicode: 1114111 Debian 4.0 "etch" i386: python: 2.4.4 (#2, Oct 22 2008, 19:52:44) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)], maxunicode: 1114111 Debian 5.0 "lenny" i386: python: 2.5.2 (r252:60911, Jan 4 2009, 17:40:26) [GCC 4.3.2], maxunicode: 1114111 Debian 5.0 "lenny" amd64: python: 2.5.2 (r252:60911, Jan 4 2009, 21:59:32) [GCC 4.3.2], maxunicode: 1114111 Debian 5.0 "lenny" armv5tel: python: 2.5.2 (r252:60911, Jan 5 2009, 02:00:00) [GCC 4.3.2], maxunicode: 1114111 Debian unstable "squeeze/sid" i386: python: 2.5.4 (r254:67916, Feb 17 2009, 20:16:45) [GCC 4.3.3], maxunicode: 1114111 Fedora 11 "leonidas" amd64: python: 2.6 (r26:66714, Jul 4 2009, 17:37:13) [GCC 4.4.0 20090506 (Red Hat 4.4.0-4)], maxunicode: 1114111 ArchLinux: python: 2.6.2 (r262:71600, Jul 20 2009, 02:23:30) [GCC 4.4.0 20090630 (prerelease)], maxunicode: 65535 NetBSD 4: python: 2.5.2 (r252:60911, Mar 20 2009, 14:00:07) [GCC 4.1.2 20060628 prerelease (NetBSD nb2 20060711)], maxunicode: 65535 OpenSolaris SunOS-5.11-i86pc-i386-32bit: python: 2.4.4 (#1, Mar 10 2009, 09:35:36) [C], maxunicode: 65535 Nexenta NCP1 SunOS-5.11-i86pc-i386-32bit: python: 2.4.3 (#2, May 3 2006, 19:12:42) [GCC 4.0.3 (GNU_OpenSolaris 4.0.3-1nexenta4)], maxunicode: 1114111 Mac OS 10.6 "snow leopard" i386: python: 2.6.1 (r261:67515, Jul 7 2009, 23:51:51) [GCC 4.2.1 (Apple Inc. build 5646)], maxunicode: 65535 Mac OS 10.5 "leopard" i386: python: 2.5.1 (r251:54863, Feb 6 2009, 19:02:12) [GCC 4.0.1 (Apple Inc. build 5465)], maxunicode: 65535 Mac OS 10.4 "tiger" *custom* python: 2.5.4 (release25-maint:72153M, Apr 30 2009, 12:28:20) [GCC 4.0.1 (Apple Computer, Inc. build 5367)], maxunicode: 65535 Cygwin CYGWIN_NT-5.1-1.5.25-0.156-4-2-i686-32bit-WindowsPE: python: 2.5.2 (r252:60911, Dec 2 2008, 09:26:14) [GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)], maxunicode: 65535 Windows: python: 2.6.2 (r262:71600, Apr 21 2009, 15:05:37) [MSC v.1500 32 bit (Intel)], maxunicode: 65535 Windows: python: 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit (Intel)], maxunicode: 65535 From benjamin at python.org Sun Sep 20 16:06:53 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 20 Sep 2009 09:06:53 -0500 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: References: Message-ID: <1afaf6160909200706p4b2145f6qb21f8a4d6074a2c3@mail.gmail.com> 2009/9/20 Zooko O'Whielacronx : > Dear Pythonistas: > > This issue causes serious problems. ?Users occasionally get binaries built for a > compatible Linux and Python version but with a different UCS2-vs-UCS4 setting, > and those users get mysterious memory corruption errors which are hard to > diagnose. ?It is possible that these situations also open up security > vulnerabilities. ?A couple such instances are documented on > http://bugs.python.org/setuptools/issue78, but you can find more by googling. > I would like to get this problem fixed! You may want to have a look at the archives of the last time this was extensively discussed: http://mail.python.org/pipermail/python-dev/2008-July/080886.html -- Regards, Benjamin From zookog at gmail.com Sun Sep 20 16:16:01 2009 From: zookog at gmail.com (Zooko O'Whielacronx) Date: Sun, 20 Sep 2009 08:16:01 -0600 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: <1afaf6160909200706p4b2145f6qb21f8a4d6074a2c3@mail.gmail.com> References: <1afaf6160909200706p4b2145f6qb21f8a4d6074a2c3@mail.gmail.com> Message-ID: I'm sorry, I should have mentioned that I did read those archives before I posted my letter. That discussion was all about whether UCS2 or UCS4 is better. I consider that question to be mostly irrelevant to this issue, which is about compatibility for people who don't choose to configure that setting themselves. Platforms or people who prefer UCS2 will continue to use it as appropriate. UCS4 is clearly good enough for the vast majority of Linux users, and having fewer mysterious segfaults and potential security vulnerabilities would be an important improvement to the user experience of Python on Linux. I should mention that the reason I'm spending time on this right now is that it is currently blocking me from being able to distribute binaries of Python packages which will work for all of my Linux users. Regards, Zooko From solipsis at pitrou.net Sun Sep 20 16:27:18 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 20 Sep 2009 14:27:18 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?please_consider_changing_--enable-unicode_?= =?utf-8?q?default_to=09ucs4?= References: Message-ID: Zooko O'Whielacronx gmail.com> writes: > > Users occasionally get binaries built for a > compatible Linux and Python version but with a different UCS2-vs-UCS4 setting, > and those users get mysterious memory corruption errors which are hard to > diagnose. What "binaries" are you talking about? AFAIK, C extensions should fail loading when they have the wrong UCS2/4 setting. That's the reason we have all those #define's in unicodeobject.h: the actual function names end up being different and, therefore, are not found when linking. > In order to help address this issue I sampled what UCS size is used by python > executables in the wild. For information, all Mandriva versions I've used until now have had their Python's built with UCS2 (maxunicode == 65535). Regards Antoine. From daniel at stutzbachenterprises.com Sun Sep 20 16:43:10 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 20 Sep 2009 09:43:10 -0500 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition In-Reply-To: <4AB5FA81.5060408@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> Message-ID: On Sun, Sep 20, 2009 at 4:48 AM, Pascal Chambon wrote: > *RawIOBase*.readinto(b: bytes) -> int > "bytes" are immutable. The signature is: *RawIOBase*.readinto(b: bytearray) -> int Your efforts in working on clarifying these important corner cases is appreciated. :-) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From zookog at gmail.com Sun Sep 20 18:17:45 2009 From: zookog at gmail.com (Zooko O'Whielacronx) Date: Sun, 20 Sep 2009 10:17:45 -0600 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: References: Message-ID: On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou wrote: > > What "binaries" are you talking about? I mean extension modules with native code, which means .so shared library files on unix. > AFAIK, C extensions should fail loading when they have the wrong UCS2/4 setting. That would be an improvement! Unfortunately we instead get mysterious misbehavior of the module, e.g.: http://bugs.python.org/setuptools/msg309 http://allmydata.org/trac/tahoe/ticket/704#comment:5 > For information, all Mandriva versions I've used until now have had their > Python's built with UCS2 (maxunicode == 65535). Thank you for the data point. This means that binary extension modules built on Mandriva can't be ported to Ubuntu or vice versa. However, is this an argument for or against changing the default setting to UCS4? Changing the default setting wouldn't interfere with Mandriva's decision, right? Regards, Zooko From chambon.pascal at gmail.com Sun Sep 20 18:32:14 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sun, 20 Sep 2009 18:32:14 +0200 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> Message-ID: <4AB6590E.2070800@gmail.com> Daniel Stutzbach a ?crit : > On Sun, Sep 20, 2009 at 4:48 AM, Pascal Chambon > > wrote: > > *RawIOBase*.readinto(b: bytes) -> int > > > "bytes" are immutable. The signature is: > > *RawIOBase*.readinto(b: bytearray) -> int > > Your efforts in working on clarifying these important corner cases is > appreciated. :-) > You're welcome B-) Indeed my copy/paste of the current pep was an epic fail - you'll all have recognized readinto actually dealt with bytearrays, contrarily to what the current pep tells -> http://www.python.org/dev/peps/pep-3116/ RawIOBase.read(int) takes a positive-or-zero integer indeed (I am used to understanding this, as opposed to "strictly positive") Does MRAb's suggestion of providing beginning and end offsets for the bytearray meets people's expectations ? Personnaly, I feel readinto is a very low-level method, mostly used by read() to get a result from low-level native functions (fread, readfile), and read() always provides a buffer with the proper size... are there cases in which these two additional arguments would provide some real gain ? Concerning the "backward compatibility" problem, I agree we should not break specifications, but breaking impelmentation details is another thing for me. It's a golden rule in programmers' world : thou shalt NEVER rely on implementation details. Programs that count on these (eg. thinking that listdir() will always returns "." and ".." as first item0... until it doesnt anymore) encounter huge problems when changing of platform or API version. When programming with the current truncate(), I would always have moved the file pointer after truncating the file, simply because I have no idea of what might happen to it (nothing was documented on this at the moment, and looking at the sources is really not a sustainable behaviour). So well, it's a pity if some early 3.1 users relied on it, but if we stick to the current semantic we still have a real coherency problem - seek() is not limited in range, and some experienced programmers might be trapped by this non-conventionnal truncate() if they rely on posix or previous python versions... I really dislike the idea that truncate() might move my file offset even when there are no reasons for it. Regards, Pascal -------------- next part -------------- An HTML attachment was scrubbed... URL: From zookog at gmail.com Sun Sep 20 18:33:23 2009 From: zookog at gmail.com (Zooko O'Whielacronx) Date: Sun, 20 Sep 2009 10:33:23 -0600 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: References: Message-ID: On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou wrote: > For information, all Mandriva versions I've used until now have had their > Python's built with UCS2 (maxunicode == 65535). By the way, I was investigating this, and discovered an issue on the Mandriva tracker which suggests that they intend to switch to UCS4 in the next release in order to avoid compatibility problems like these. (Not because they think that UCS4 is better than UCS2.) https://qa.mandriva.com/show_bug.cgi?id=48570 Regards, Zooko From solipsis at pitrou.net Sun Sep 20 18:36:03 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 20 Sep 2009 16:36:03 +0000 (UTC) Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 References: Message-ID: Le Sun, 20 Sep 2009 10:17:45 -0600, Zooko O'Whielacronx a ?crit?: > >> AFAIK, C extensions should fail loading when they have the wrong UCS2/4 >> setting. > > That would be an improvement! Unfortunately we instead get mysterious > misbehavior of the module, e.g.: > > http://bugs.python.org/setuptools/msg309 > http://allmydata.org/trac/tahoe/ticket/704#comment:5 The bug reports in themselves aren't very explicit, and they don't seem to be related to any native extension. So I'm not sure why you're talking about "mysterious memory corruption errors" in your original mail, because there doesn't seem to be such a thing happening at all. Please note that there's a bug related to a non-portable peephole optimization of some unicode constants, perhaps it may explain the aforementioned problems (perhaps not) : http://bugs.python.org/issue5057 I expect the solution to this bug to be rather easy (just disable the optimization, since it isn't really useful), but someone has to care enough to produce a patch. >> For information, all Mandriva versions I've used until now have had >> their Python's built with UCS2 (maxunicode == 65535). > > Thank you for the data point. This means that binary extension modules > built on Mandriva can't be ported to Ubuntu or vice versa. "Ported" they can certainly be, you just have to recompile. > However, is > this an argument for or against changing the default setting to UCS4? > Changing the default setting wouldn't interfere with Mandriva's > decision, right? Well, let's put it this way: - either you expect the default setting to be observed by everyone, and it *will* interfere with someone's current decision - or you don't expect the default setting to be observed by everyone, and then there's no point in changing it because it won't stop your problems Either way, my mentioning of Mandriva was just meant as an additional data point to those you already provided ;-) Regards Antoine. From solipsis at pitrou.net Sun Sep 20 18:42:31 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 20 Sep 2009 16:42:31 +0000 (UTC) Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 References: Message-ID: Le Sun, 20 Sep 2009 10:33:23 -0600, Zooko O'Whielacronx a ?crit?: > > By the way, I was investigating this, and discovered an issue on the > Mandriva tracker which suggests that they intend to switch to UCS4 in > the next release in order to avoid compatibility problems like these. Trying to use a Fedora or Suse RPM under Mandriva (or the other way round) isn't reasonable and is certainly not supported. I don't understand why this so-called "compatibility problem" should be taken seriously by anyone. Regards Antoine. From mal at egenix.com Sun Sep 20 20:28:00 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Sun, 20 Sep 2009 20:28:00 +0200 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: References: Message-ID: <4AB67430.2060306@egenix.com> Zooko O'Whielacronx wrote: > On Sun, Sep 20, 2009 at 8:27 AM, Antoine Pitrou wrote: >> >> What "binaries" are you talking about? > > I mean extension modules with native code, which means .so shared > library files on unix. Those will not load unless they are for the right UCS-version of Python. The extensions will give an ImportError if they are using any Unicode APIs - we go through great lengths in the Unicode API to make sure that you cannot mix UCS2 and UCS4 APIs. I'm not exactly sure what you are trying to achieve by making UCS4 the default... if you build extensions using the system Python version, distutils will automatically build the right UCS-version for you. >> AFAIK, C extensions should fail loading when they have the wrong UCS2/4 setting. > > That would be an improvement! Unfortunately we instead get mysterious > misbehavior of the module, e.g.: > > http://bugs.python.org/setuptools/msg309 > http://allmydata.org/trac/tahoe/ticket/704#comment:5 Those don't appear to be related to UCS2 vs. UCS4 but rather some problem with the UTF-8 data those users are trying to load. The fact that setuptools completely ignores the fact that Python UCS2 and UCS4 are two different Python builds, is not really a Python Unicode problem, but one of the setuptools design, so you should probably complain there. >> For information, all Mandriva versions I've used until now have had their >> Python's built with UCS2 (maxunicode == 65535). > > Thank you for the data point. This means that binary extension > modules built on Mandriva can't be ported to Ubuntu or vice versa. > However, is this an argument for or against changing the default > setting to UCS4? Changing the default setting wouldn't interfere with > Mandriva's decision, right? Depends on what you mean with "ported": of course you can port a source RPM between UCS2 and UCS4 builds. This just requires a recompile. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 20 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From python at mrabarnett.plus.com Sun Sep 20 20:46:44 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 20 Sep 2009 19:46:44 +0100 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition In-Reply-To: <4AB6590E.2070800@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> Message-ID: <4AB67894.5020903@mrabarnett.plus.com> Pascal Chambon wrote: > Daniel Stutzbach a ?crit : >> On Sun, Sep 20, 2009 at 4:48 AM, Pascal Chambon >> > wrote: >> >> *RawIOBase*.readinto(b: bytes) -> int >> >> >> "bytes" are immutable. The signature is: >> >> *RawIOBase*.readinto(b: bytearray) -> int >> >> Your efforts in working on clarifying these important corner cases is >> appreciated. :-) >> > You're welcome B-) > > Indeed my copy/paste of the current pep was an epic fail - you'll all > have recognized readinto actually dealt with bytearrays, contrarily to > what the current pep tells > -> http://www.python.org/dev/peps/pep-3116/ > > RawIOBase.read(int) takes a positive-or-zero integer indeed (I am used > to understanding this, as opposed to "strictly positive") > > Does MRAb's suggestion of providing beginning and end offsets for the > bytearray meets people's expectations ? Personnaly, I feel readinto is a > very low-level method, mostly used by read() to get a result from > low-level native functions (fread, readfile), and read() always provides > a buffer with the proper size... are there cases in which these two > additional arguments would provide some real gain ? > It's useful if you want to fill the buffer but 'read' might return fewer bytes than you asked for because it returns only what's available. That might not happen for files, but it might for other forms of I/O. Other languages, like Delphi and Java, which read into pre-existing arrays, have or allow the extra parameters. > > Concerning the "backward compatibility" problem, I agree we should not > break specifications, but breaking impelmentation details is another > thing for me. It's a golden rule in programmers' world : thou shalt > NEVER rely on implementation details. Programs that count on these (eg. > thinking that listdir() will always returns "." and ".." as first > item0... until it doesnt anymore) encounter huge problems when changing > of platform or API version. When programming with the current > truncate(), I would always have moved the file pointer after truncating > the file, simply because I have no idea of what might happen to it > (nothing was documented on this at the moment, and looking at the > sources is really not a sustainable behaviour). > So well, it's a pity if some early 3.1 users relied on it, but if we > stick to the current semantic we still have a real coherency problem - > seek() is not limited in range, and some experienced programmers might > be trapped by this non-conventionnal truncate() if they rely on posix or > previous python versions... I really dislike the idea that truncate() > might move my file offset even when there are no reasons for it. > Well, if it's consistent and documented (and not totally stupid), I can't really complain. :-) From martin at v.loewis.de Sun Sep 20 21:27:06 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 20 Sep 2009 21:27:06 +0200 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: References: <1afaf6160909200706p4b2145f6qb21f8a4d6074a2c3@mail.gmail.com> Message-ID: <4AB6820A.8060202@v.loewis.de> Zooko O'Whielacronx wrote: > I'm sorry, I should have mentioned that I did read those archives > before I posted my letter. That discussion was all about whether UCS2 > or UCS4 is better. I consider that question to be mostly irrelevant > to this issue, which is about compatibility for people who don't > choose to configure that setting themselves. You surely must have missed the sentence "For that reason I think it's also better that the configure script continues to default to UTF-16 -- this will give the UTF-16 support code the necessary exercise." This is effectively a BDFL pronouncement. Nothing has changed the validity of the premise of the statement, so the conclusion remains valid, as well. Regards, Martin From solipsis at pitrou.net Mon Sep 21 23:30:58 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 21 Sep 2009 21:30:58 +0000 (UTC) Subject: [Python-Dev] buildbots Message-ID: Hello, I realize I may have missed some kind of formal announcement, but why have all the buildbots been down for a number of days (weeks?) now? Thanks Antoine. From ziade.tarek at gmail.com Tue Sep 22 15:21:06 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 22 Sep 2009 15:21:06 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format Message-ID: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Hello Here's a wrapup of the Distutils-SIG discussion we had on the "static metadata" topic. I realize that it's a good thing to send in. python-dev such wrapup on distutils design decisions, to keep everyone informed and get more feedback when required. I will try to do it for every upcoming changes that are not going in a PEP process (when it's not a 'big' change). The rate of such mails should not be very high. (around 1 mail in python-dev for +150 mails in distutils-SIG) If you feel that what we are about to change in distutils is wrong, you can go ahead and help us by participating in Distutils-ML, so we keep one and only one media for these discussions. The four sentences summary for people in a hurry: Getting metadata of a distribution that is not. installed means running its setup.py. This means. downloading the whole archive, and running. third party code on your system. To avoid it, we are adding a [setup] section in. setup.cfg where you can express the package metadata statically. Conditional sections, specific to some system. can be added to add some specific fields in [setup]. At the end, you will be able to get metadata fields without running any third-party code, and possibly get only the distribution setup.cfg for this. Now the long version. Rational ======== Today, if you want to list all the metadata (PEP 314) of a. distribution that is not installed, you need to use it's. setup.py CLI. So, basically, you download it, and run:: $ python setup.py --name Foo $ python setup.py --version 1.2 Where `name` and `version` are metadata fields. That's OK but as. soon as the developers add more code in setup.py, this feature might break or (worse) might do unwanted things on the target. system. Why should we run third-party code just to get its metadata ? So we worked on a way to express those metadata statically, by pushing them in `setup.cfg`. That's not hard, since all. the metadata fields are static values. Adding a setup section in setup.cfg =================================== So the first thing we want to introduce is a [setup] section, that may contain any field from the metadata:: [setup] name: Foo version: 1.2 Distutils will look for them, and will use them. Of course setup.py is still required, and in case an option that is. a metadata field is passed to setup(), it will override one. located in setup.cfg. PEP 341 is coming up ==================== Remember the last Pycon Summit ? We said that we would introduce a new metadata field to describe requirements.. That's basically what PEP 341 is about, and we are still. working on it. Basically, we will be able to write things like:: requires: Twisted == 8.2.0 What takes us so long is that adding requirements like this in the metadata requires more things: - the ability to compare versions (this was described in. PEP 386 but not finished yet) - the ability to make these requirements vary depending on. the target system And the later makes our "setup.cfg" new [setup] section. obsolete as soon as this new metadata field is added in. Python. So we need more that what I've shown in the previous section Context-dependant sections ========================== The second change we are going to introduce is context-dependant sections in setup.cfg. A context dependant section is a section with a condition that is. used only if the execution environment meets its condition. Here's an example:: [setup] name: Foo version: 1.3 [setup:sys_platform == 'win32'] requires: pywin32 requires: bar > 1.0 [setup:os_machine == '64bits'] requires: some_package [setup:python_version == '2.4' or python_version == '2.5'] requires: some_package Every [setup:condition] section will be added in [setup] only if the condition is met. The micro-language behind this is the simplest possible: it compares only strings, with usual string operators, and with the ability to combine expressions. It makes it also easy to understand to non-pythoneers (os packagers). The pseudo-grammar is (I don't know how to write those but you'll get it hopefully):: comp: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in' comparison: expr (comp_op expr)* expr: STRING test: or_test or_test: and_test ('or' and_test)* and_test: not_test ('and' not_test)* not_test: 'not' not_test | comparison where STRING belongs to any of those: - python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1]) - os_name = os.name - sys_platform = sys.platform - os_sysname = os.uname()[0]. - os_nodename = os.uname()[1] - os_release = os.uname()[2]. - os_version = os.uname()[3].. - os_machine = os.uname()[4]. - a free string, like '2.4', or 'win32' Distutils will provide a function that is able to read the metadata of a distribution, given a setup.cfg file, for the target environment:: >>> from distutils.util import local_metadata >>> local_metadata('setup.cfg') Meaning that a Vanilla Python will be able to read the metadata of a package without running any code. That's what distutils will use when setup.py is run. That's what any package manager will be able to use to work. with distributions. Limitations =========== If a distribution is unable for obscure reasons (or does not wish) to set all metadata fields in setup.cfg, that's fine :. the fields will be set to UNKNOWN when `local_metadata` is called. When setup.py is run, options passed to setup() will complete those. When a package manager gets 'UNKOWN' values, it knows it might need to do further processing by running setup.py. Why this change is good ===================== Once the requires metadata is added in PEP 341, being able to query the metadata for a distribution can be done without doing anything else than reading a static file and interpreting its conditions in a restricted and secured manner. So there's no need to run third-party code anymore. Possible use cases: - setup.cfg is published on PyPI. Packages managers like easy_install or pip will be able to list all the distributions needed and their versions for installing a distribution without having to download anything ! - OS packagers will be able to list the dependencies of a distribution without having to browse Python code. And, again, this new [setup] section is backward-compatible, and will be a good complement to the work done in PEP 376. Thanks for reading so far. This code will be added in 2.7 and 3.2. Tarek -- Tarek Ziad? | http://ziade.org |????????????! From phd at phd.pp.ru Tue Sep 22 15:24:15 2009 From: phd at phd.pp.ru (Oleg Broytmann) Date: Tue, 22 Sep 2009 17:24:15 +0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: <20090922132415.GA15767@phd.pp.ru> On Tue, Sep 22, 2009 at 03:21:06PM +0200, Tarek Ziad? wrote: > I realize that it's a good thing to send in. > python-dev such wrapup on distutils design > decisions, to keep everyone informed and get > more feedback when required. Sure it is. > I will try to do it for every upcoming changes > that are not going in a PEP process (when it's not > a 'big' change). Thank you! Oleg. -- Oleg Broytmann http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From pje at telecommunity.com Tue Sep 22 15:50:45 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 22 Sep 2009 09:50:45 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.co m> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: <20090922135049.583BB3A4079@sparrow.telecommunity.com> At 03:21 PM 9/22/2009 +0200, Tarek Ziad? wrote: >Hello Here's a wrapup of the Distutils-SIG discussion we had on the >"static metadata" topic. I realize that it's a good thing to send >in. python-dev such wrapup on distutils design decisions, to keep >everyone informed and get more feedback when required The above implies that a consensus was reached on this topic; you should perhaps at least note that in this case it actually means that you just terminated the discussion and pronounced your choice, just as a simpler solution for dynamic dependencies (proposed by Sridhar) was beginning to gain some support. From solipsis at pitrou.net Tue Sep 22 15:53:21 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 22 Sep 2009 13:53:21 +0000 (UTC) Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: Tarek Ziad? gmail.com> writes: > > [setup] > name: Foo > version: 1.2 I am probably missing something, but why doesn't it use the more common ini-style: [setup] name = Foo version = 1.2 Regards Antoine. From fuzzyman at voidspace.org.uk Tue Sep 22 15:55:31 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 22 Sep 2009 14:55:31 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: <4AB8D753.3040800@voidspace.org.uk> Antoine Pitrou wrote: > Tarek Ziad? gmail.com> writes: > >> [setup] >> name: Foo >> version: 1.2 >> > > I am probably missing something, but why doesn't it use the more common > ini-style: > > [setup] > name = Foo > version = 1.2 > ConfigParser recognises both, but other tools (*coff* ConfigObj) only recognises the latter (=). Michael > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ From chris at simplistix.co.uk Tue Sep 22 16:02:56 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 22 Sep 2009 15:02:56 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: <4AB8D910.8050601@simplistix.co.uk> Tarek Ziad? wrote: > The four sentences summary for people in a hurry: If you're going to promise sentences, please deliver sentences... > Getting metadata of a distribution that is not. > installed means running its setup.py. This means. > downloading the whole archive, and running. > third party code on your system. This is not a sentence, and I have no idea what you're trying to convey... > A context dependant section is a section with a condition that is. > used only if the execution environment meets its condition. > > Here's an example:: > > [setup] > name: Foo > version: 1.3 > > [setup:sys_platform == 'win32'] > requires: pywin32 > requires: bar > 1.0 > > [setup:os_machine == '64bits'] > requires: some_package > > [setup:python_version == '2.4' or python_version == '2.5'] > requires: some_package As was brought up on the distutils list, this seems unnecessarily complicated. Other, simpler, solutions were proposed: http://mail.python.org/pipermail/distutils-sig/2009-September/013173.html http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html However, you pronounced yourself bdfl(!): http://mail.python.org/pipermail/distutils-sig/2009-September/013318.html ...and decided to ignore them. Please don't do this over such an important matter. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Tue Sep 22 16:06:42 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 22 Sep 2009 15:06:42 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations Message-ID: <4AB8D9F2.6050807@simplistix.co.uk> Hi All, I didn't see any docs on this: http://docs.python.org/reference/datamodel.html?highlight=__eq__#object.__eq__ Where are the specifications on what happens if two objects are compared and both have implementations of __eq__? Which __eq__ is called? What happens if the first one called returns False? Is the second one called? What is one implements __eq__ and the other __ne__? If I've missed something, please point me in the right direction. To all those about to tell me to go read the source: that's not good enough here. I'm hoping there *are* "official" rules for how these interact and they just need better linking in, otherwise, I worry that IronPython could do one thing, Jython another and CPython a third... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From jflatow at gmail.com Tue Sep 22 16:20:04 2009 From: jflatow at gmail.com (Jared Flatow) Date: Tue, 22 Sep 2009 07:20:04 -0700 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4AB8D9F2.6050807@simplistix.co.uk> References: <4AB8D9F2.6050807@simplistix.co.uk> Message-ID: This might help: http://mail.python.org/pipermail/python-dev/2008-June/080111.html Here is the most relevant part (quoting Guido): > Does it help if I tell you that for "x y" we always try > x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the > case where y is an instance of a subclass of the class of x? jared On Sep 22, 2009, at 7:06 AM, Chris Withers wrote: > Hi All, > > I didn't see any docs on this: > > http://docs.python.org/reference/datamodel.html?highlight=__eq__#object.__eq__ > > Where are the specifications on what happens if two objects are > compared and both have implementations of __eq__? Which __eq__ is > called? What happens if the first one called returns False? Is the > second one called? What is one implements __eq__ and the other __ne__? > > If I've missed something, please point me in the right direction. > > To all those about to tell me to go read the source: that's not good > enough here. I'm hoping there *are* "official" rules for how these > interact and they just need better linking in, otherwise, I worry > that IronPython could do one thing, Jython another and CPython a > third... > > cheers, > > Chris > > -- > Simplistix - Content Management, Batch Processing & Python Consulting > - http://www.simplistix.co.uk > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/jflatow%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at simplistix.co.uk Tue Sep 22 16:37:42 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 22 Sep 2009 15:37:42 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: References: <4AB8D9F2.6050807@simplistix.co.uk> Message-ID: <4AB8E136.8040706@simplistix.co.uk> Jared Flatow wrote: > This might help: > > http://mail.python.org/pipermail/python-dev/2008-June/080111.html > > Here is the most relevant part (quoting Guido): > > > Does it help if I tell you that for "x y" we always try > > x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the > > case where y is an instance of a subclass of the class of x? Okay, but does that count as a pronouncement that should go across all versions and platforms? I believe there are differences between when __eq__ and __ne__ are called between Python 2 and 3, and I don't see any docs on the expected behaviour for python 2, dunno about 3. I'm willing to write these docs, http://docs.python.org/reference/datamodel.html#object.__eq__ feels like the right place, is there anywhere else they should go or be linked to from? Mark Dickinson gave some useful insights into this, and Milko Krachounov provided these useful comparisons on #python: Python 2 http://pastebin.com/f8f19ab3 Python 3 http://pastebin.com/f55e44630 Do they cover it all or has anything been missed? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From dickinsm at gmail.com Tue Sep 22 17:12:32 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 22 Sep 2009 16:12:32 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4AB8E136.8040706@simplistix.co.uk> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> Message-ID: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> On Tue, Sep 22, 2009 at 3:37 PM, Chris Withers wrote: > Where are the specifications on what happens if two objects are compared and both have implementations of __eq__? Which __eq__ is called? > What happens if the first one called returns False? Is the second one called? What is one implements __eq__ and the other __ne__? I (still :-) think this is covered, for Python 2.x at least, by: http://docs.python.org/reference/datamodel.html#coercion-rules Specifically, the bits that say: - For objects x and y, first x.__op__(y) is tried. If this is not implemented or returns NotImplemented, y.__rop__(x) is tried. If this is also not implemented or returns NotImplemented, a TypeError exception is raised. But see the following exception: - Exception to the previous item: if the left operand is an instance of a built-in type or a new-style class, and the right operand is an instance of a proper subclass of that type or class and overrides the base?s __rop__() method, the right operand?s __rop__() method is tried before the left operand?s __op__() method. I agree that having these rules in a section called 'Coercion rules' is a bit confusing. > Python 2 > http://pastebin.com/f8f19ab3 > > Python 3 > http://pastebin.com/f55e44630 The duplicate __eq__ calls in these pastes are disturbing: in short, if A() and B() are new-style classes defining __eq__, it seems that A() == B() ends up calling A.__eq__ twice *and* B.__eq__ twice, in the order A.__eq__, B.__eq__, B.__eq__, A.__eq__. In 3.x, slot_tp_richcompare (in typeobject.c) makes two calls to half_richcompare; I think the second is redundant. The coercion rules are already taken care of in do_richcompare (in object.c). I tried removing the second call to half_richcompare, and the entire test-suite still runs without errors. In 2.x, it's possible that this call is necessary for some bizarre combinations of __cmp__ and __eq__; I haven't tried to get my head around this yet. I'll open an issue for the duplicate __eq__ calls. Mark From metawilm at gmail.com Tue Sep 22 17:23:03 2009 From: metawilm at gmail.com (Willem Broekema) Date: Tue, 22 Sep 2009 17:23:03 +0200 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> Message-ID: On Tue, Sep 22, 2009 at 5:12 PM, Mark Dickinson wrote: > - Exception to the previous item: if the left operand is an instance > of a built-in type or a new-style class, and the right operand is an > instance of a proper subclass of that type or class AND overrides the > base?s __rop__() method, the right operand?s __rop__() method is tried > before the left operand?s __op__() method. The AND above (which I uppercased) is subtle but important. In the "x op y" case with y being of a subclass of the class of x, if there is no class in between x and y (excluding x, including y) that overrides the __rop__ method, then y,__rop__(x) is *not* tried before x.__op__(y). It's easy for other implementations to get this wrong. :) - Willem From dirkjan at ochtman.nl Tue Sep 22 16:02:55 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 22 Sep 2009 16:02:55 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <4AB8D753.3040800@voidspace.org.uk> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D753.3040800@voidspace.org.uk> Message-ID: On Tue, Sep 22, 2009 at 15:55, Michael Foord wrote: > ConfigParser recognises both, but other tools (*coff* ConfigObj) only > recognises the latter (=). So does Mercurial, BTW, because this aspect of ConfigParser was pretty annoying for some of the stuff we try to do (and not at all obvious to most users, who have never seen anything but = used in config-style files). Cheers, Dirkjan From dickinsm at gmail.com Tue Sep 22 18:01:20 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 22 Sep 2009 17:01:20 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> Message-ID: <5c6f2a5d0909220901o36d709a9n8842cb58c00bc863@mail.gmail.com> On Tue, Sep 22, 2009 at 4:12 PM, Mark Dickinson wrote: > I'll open an issue for the duplicate __eq__ calls. http://bugs.python.org/issue6970 Mark From ziade.tarek at gmail.com Tue Sep 22 20:06:55 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 22 Sep 2009 20:06:55 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <4AB8D910.8050601@simplistix.co.uk> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> Message-ID: <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> On Tue, Sep 22, 2009 at 4:02 PM, Chris Withers wrote: >> ? ?[setup:python_version == '2.4' or python_version == '2.5'] >> ? ?requires: some_package > > As was brought up on the distutils list, this seems unnecessarily > complicated. > > Other, simpler, solutions were proposed: > > http://mail.python.org/pipermail/distutils-sig/2009-September/013173.html > > http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html > > However, you pronounced yourself bdfl(!): > > http://mail.python.org/pipermail/distutils-sig/2009-September/013318.html > > ...and decided to ignore them. Please don't do this over such an important > matter. Yes, for the syntax bikeshedding on how conditions should be expressed, I have decided I'll be some kind of distutils dictator to end up the endless discussions on how they should be expressed so we can move on. Tarek From fuzzyman at voidspace.org.uk Tue Sep 22 20:07:56 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 22 Sep 2009 19:07:56 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> Message-ID: <4AB9127C.5070102@voidspace.org.uk> Tarek Ziad? wrote: > On Tue, Sep 22, 2009 at 4:02 PM, Chris Withers wrote: > >>> [setup:python_version == '2.4' or python_version == '2.5'] >>> requires: some_package >>> >> As was brought up on the distutils list, this seems unnecessarily >> complicated. >> >> Other, simpler, solutions were proposed: >> >> http://mail.python.org/pipermail/distutils-sig/2009-September/013173.html >> >> http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html >> >> However, you pronounced yourself bdfl(!): >> >> http://mail.python.org/pipermail/distutils-sig/2009-September/013318.html >> >> ...and decided to ignore them. Please don't do this over such an important >> matter. >> > > Yes, for the syntax bikeshedding on how conditions should be expressed, > I have decided I'll be some kind of distutils dictator to end up the > endless discussions > on how they should be expressed so we can move on. > +1 Michael > Tarek > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From ziade.tarek at gmail.com Tue Sep 22 20:12:24 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 22 Sep 2009 20:12:24 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <20090922135049.583BB3A4079@sparrow.telecommunity.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <20090922135049.583BB3A4079@sparrow.telecommunity.com> Message-ID: <94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.com> 2009/9/22 P.J. Eby : > At 03:21 PM 9/22/2009 +0200, Tarek Ziad? wrote: >> >> Hello Here's a wrapup of the Distutils-SIG discussion we had on the >> "static metadata" topic. I realize that it's a good thing to send in. >> python-dev such wrapup on distutils design decisions, to keep everyone >> informed and get more feedback when required > > The above implies that a consensus was reached on this topic; you should > perhaps at least note that in this case it actually means that you just > terminated the discussion and pronounced your choice, just as a simpler > solution for dynamic dependencies (proposed by Sridhar) was beginning to > gain some support. I pronounced my choice on the syntax of the conditions in the setup.cfg file, and the setuptools-like syntax to express them Sridhar presented looked more complex. If you are talking about another solution than using setup.cfg with conditional expressions, please start a new thread that demonstrates how it will work, we all want the best solution of course. Tarek From ziade.tarek at gmail.com Tue Sep 22 20:15:30 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 22 Sep 2009 20:15:30 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D753.3040800@voidspace.org.uk> Message-ID: <94bdd2610909221115r658a0258yd0c84a2628d8e4b7@mail.gmail.com> On Tue, Sep 22, 2009 at 4:02 PM, Dirkjan Ochtman wrote: > On Tue, Sep 22, 2009 at 15:55, Michael Foord wrote: >> ConfigParser recognises both, but other tools (*coff* ConfigObj) only >> recognises the latter (=). > > So does Mercurial, BTW, because this aspect of ConfigParser was pretty > annoying for some of the stuff we try to do (and not at all obvious to > most users, who have never seen anything but = used in config-style > files). yes, I've just used ConfigParser syntax because that's what Distutils uses. If ':' is deprecated in favor of '=' only, Distutils will be fine with that change. I would be +1 for its deprecation Tarek -- Tarek Ziad? | http://ziade.org |????????????! From ziade.tarek at gmail.com Tue Sep 22 20:16:03 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 22 Sep 2009 20:16:03 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <20090922135049.583BB3A4079@sparrow.telecommunity.com> <94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.com> Message-ID: <94bdd2610909221116m26549490k2c45cfe188566207@mail.gmail.com> On Tue, Sep 22, 2009 at 8:12 PM, Tarek Ziad? wrote: > If you are talking about another solution than using setup.cfg with conditional > expressions, please start a new thread (in Distutils SIG) From fuzzyman at voidspace.org.uk Tue Sep 22 20:26:42 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 22 Sep 2009 19:26:42 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909221115r658a0258yd0c84a2628d8e4b7@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D753.3040800@voidspace.org.uk> <94bdd2610909221115r658a0258yd0c84a2628d8e4b7@mail.gmail.com> Message-ID: -- http://www.ironpythoninaction.com On 22 Sep 2009, at 19:15, Tarek Ziad? wrote: > On Tue, Sep 22, 2009 at 4:02 PM, Dirkjan Ochtman > wrote: >> On Tue, Sep 22, 2009 at 15:55, Michael Foord > > wrote: >>> ConfigParser recognises both, but other tools (*coff* ConfigObj) >>> only >>> recognises the latter (=). >> >> So does Mercurial, BTW, because this aspect of ConfigParser was >> pretty >> annoying for some of the stuff we try to do (and not at all obvious >> to >> most users, who have never seen anything but = used in config-style >> files). > > yes, I've just used ConfigParser syntax because that's what > Distutils uses. > > If ':' is deprecated in favor of '=' only, Distutils will be fine with > that change. > > I would be +1 for its deprecation > I don't think : is formally deprecated but = is more common and it would be great if distutils could default to (and document) this syntax. Michael > Tarek > > -- > Tarek Ziad? | http://ziade.org | ???????????! From pje at telecommunity.com Tue Sep 22 20:52:45 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 22 Sep 2009 14:52:45 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.co m> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <20090922135049.583BB3A4079@sparrow.telecommunity.com> <94bdd2610909221112u48484e93oc49f70445cf1fbe4@mail.gmail.com> Message-ID: <20090922185248.377CC3A403D@sparrow.telecommunity.com> At 08:12 PM 9/22/2009 +0200, Tarek Ziad? wrote: >If you are talking about another solution than using setup.cfg with >conditional >expressions, please start a new thread that demonstrates how it will work, >we all want the best solution of course. I already did that, on the distutils-sig. Or more precisely, someone else did. I only elaborated on the idea they proposed. (I believe it was Sridhar who first proposed it, but my memory may be incorrect.) From chris at simplistix.co.uk Tue Sep 22 21:51:58 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 22 Sep 2009 20:51:58 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> Message-ID: <4AB92ADE.9060700@simplistix.co.uk> Tarek Ziad? wrote: > Yes, for the syntax bikeshedding on how conditions should be expressed, > I have decided I'll be some kind of distutils dictator to end up the > endless discussions > on how they should be expressed so we can move on. ...except in this case, spelling is important and I'm far from the only person who would prefer you not to foist an unpalatable mess on the entire python community just because the ideas that other people prefer are not yours. cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From brett at python.org Tue Sep 22 22:02:54 2009 From: brett at python.org (Brett Cannon) Date: Tue, 22 Sep 2009 13:02:54 -0700 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <4AB92ADE.9060700@simplistix.co.uk> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> Message-ID: On Tue, Sep 22, 2009 at 12:51, Chris Withers wrote: > Tarek Ziad? wrote: >> >> Yes, for the syntax bikeshedding on how conditions should be expressed, >> I have decided I'll be some kind of distutils dictator to end up the >> endless discussions >> on how they should be expressed so we can move on. > > ...except in this case, spelling is important and I'm far from the only > person who would prefer you not to foist an unpalatable mess on the entire > python community just because the ideas that other people prefer are not > yours. OK, stop before it gets any nastier as your email already sounds rude, Chris. Since the language summit at PyCon 2009 various committers, including me, have been encouraging Tarek to act as distutils dictator to get things finished as we all know people are prone to bikeshedding that would kill any forward momentum we have towards improving the packaging situation. Since Tarek has been doing the legwork on all of this and continues to be the main contributor/pusher of distutils improvements I see no reason why he should not be able to short-circuit discussions when he sees them as having gone on long enough when he is the one that is going to ultimately end up doing all the coding. I honestly think Tarek listens better than most when it comes to input and tends to be extremely cordial about things. Every decision Tarek makes, just like any decision involving Python made by anyone, is going to upset/alienate/piss off some subset of the Python community. That's just life. If it came down to distutils just sitting there with no improvement or Tarek choosing some syntax for config files that I didn't love, I personally would choose the latter as it's better than nothing and syntax is definitely one place where you can't please everyone. -Brett From dinov at microsoft.com Tue Sep 22 23:55:27 2009 From: dinov at microsoft.com (Dino Viehland) Date: Tue, 22 Sep 2009 21:55:27 +0000 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> Willem wrote: > > It's easy for other implementations to get this wrong. :) For IronPython we wrote a set of tests which go through and define the various operator methods in all sorts of combinations on both new-style and old-style classes as well as subclasses of those classes and then do the comparisons w/ logging. We've got this for both comparisons and the normal binary operators. We've also done it in other areas such as throwing exceptions and other complicated areas. But having the the coercion rules documented still helped a lot (although my recollection is they were only about 95% right). From chris at simplistix.co.uk Wed Sep 23 00:48:08 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Tue, 22 Sep 2009 23:48:08 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> Message-ID: <4AB95428.1040704@simplistix.co.uk> Brett Cannon wrote: > OK, stop before it gets any nastier as your email already sounds rude, Chris. Yes, apologies. > Since the language summit at PyCon 2009 various committers, including > me, have been encouraging Tarek to act as distutils dictator to get > things finished as we all know people are prone to bikeshedding that > would kill any forward momentum we have towards improving the > packaging situation. ...except that this isn't bikeshedding. Use of that term just to stifle productive discussion is also rude. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From sridharr at activestate.com Wed Sep 23 00:53:46 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Tue, 22 Sep 2009 15:53:46 -0700 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <4AB95428.1040704@simplistix.co.uk> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <4AB95428.1040704@simplistix.co.uk> Message-ID: On Tue, 22 Sep 2009 15:48:08 -0700, Chris Withers wrote: > > Since the language summit at PyCon 2009 various committers, including > me, have been encouraging Tarek to act as distutils dictator to get > things finished as we all know people are prone to bikeshedding that > would kill any forward momentum we have towards improving the > packaging situation. > ...except that this isn't bikeshedding. Use of that term just to stifle > productive discussion is also rude. As I see it, the choice of syntax *as such* is an issue of bikeshedding except that the choice of one syntax over the another could make it supportive to add an useful feature in setuptools to distutils. For instance, this syntax which I proposed - http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html - will make us think about adding install_requires/extra_requires to distutils *in addition to* the conditional metadata .. rather than inventing a new syntax to support only conditional metadata. -srid From brett at python.org Wed Sep 23 02:06:49 2009 From: brett at python.org (Brett Cannon) Date: Tue, 22 Sep 2009 17:06:49 -0700 Subject: [Python-Dev] how to properly object to something (was: Distutils ML wrap-up: setup.cfg new format) Message-ID: 2009/9/22 Chris Withers : > Brett Cannon wrote: >> >> OK, stop before it gets any nastier as your email already sounds rude, >> Chris. > > Yes, apologies. Thanks. > >> Since the language summit at PyCon 2009 various committers, including >> me, have been encouraging Tarek to act as distutils dictator to get >> things finished as we all know people are prone to bikeshedding that >> would kill any forward momentum we have towards improving the >> packaging situation. > > ...except that this isn't bikeshedding. Use of that term just to stifle > productive discussion is also rude. This sounds like a differing of opinion; Tarek doesn't think it's critical, you do. Obviously that changes what the two of you view as bikeshedding on this topic. I also just checked the thread based on the link you and Sridhar sent and I didn't see anyone publicly say they preferred Sridhar's version. If people preferred Sridhar's version they should have spoken up. While Sridhar's email came in Friday and Tarek pronounced Saturday (as a reply to one of Sridhar's emails to his idea), that doesn't mean no one could have spoken up over the the past three days in support of Sridhar's idea if they preferred it regardless of what Tarek decided; no code has been committed so decisions can be changed (gods know plenty of people have said after Guido has pronounced "I disagree" and no one has ever taken offense). So I don't think it's fair to claim that Tarek is not listening considering his pronouncement was in reply to one of the proposals you said he was ignoring. In other words, Chris, you brought up objections to Tarek's decision claiming he was ignoring two full days after he made it, and used as support at least one email which no one publicly showed support for and for which Tarek indirectly replied to when he made his decision. So I see no evidence that Tarek is ignoring people's input from where I stand. If you didn't like Tarek's final decision or preferred Sridhar's idea then you should have replied to one of those emails with at least a vote of +1/+0/-0/-1 over on the distutils-SIG, not here. As of right now this feels like an end-run around Tarek using extremely weak evidence which, in my opinion, is bikeshedding. -Brett From dinov at microsoft.com Wed Sep 23 02:23:56 2009 From: dinov at microsoft.com (Dino Viehland) Date: Wed, 23 Sep 2009 00:23:56 +0000 Subject: [Python-Dev] unsubscriptable vs object does not support indexing Message-ID: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> Is there a reason or a rule by which CPython reports different error message for different failures to subscript? For example: >>> set()[2] Traceback (most recent call last): File "", line 1, in TypeError: 'set' object does not support indexing >>> class c(object): pass ... >>> c()[2] Traceback (most recent call last): File "", line 1, in TypeError: 'c' object does not support indexing But compare this to: >>> [].append[42] Traceback (most recent call last): File "", line 1, in TypeError: 'builtin_function_or_method' object is unsubscriptable >>> (lambda: 42)[42] Traceback (most recent call last): File "", line 1, in TypeError: 'function' object is unsubscriptable >>> property()[42] Traceback (most recent call last): File "", line 1, in TypeError: 'property' object is unsubscriptable IronPython had a bug report that we were getting this wrong for set objects and that "does not support indexing" was also a clearer error message. I'm wondering if there's some reason why the different error messages occur which I'm missing. Otherwise I could switch to using the more clear message or start marking types which should report the unsubscriptable error. Does anyone have any insights into this? -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.lyon at preisshare.net Wed Sep 23 02:07:33 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 22 Sep 2009 20:07:33 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: Tarek, Are you claiming this as your own work and ideas? Given: http://mail.python.org/pipermail/distutils-sig/2009-August/012998.html Regards David On Tue, 22 Sep 2009 15:21:06 +0200, Tarek Ziad? wrote: > Hello > > Here's a wrapup of the Distutils-SIG discussion > we had on the "static metadata" topic. > > I realize that it's a good thing to send in. > python-dev such wrapup on distutils design > decisions, to keep everyone informed and get > more feedback when required. > > I will try to do it for every upcoming changes > that are not going in a PEP process (when it's not > a 'big' change). The rate of such mails should > not be very high. (around 1 mail in python-dev > for +150 mails in distutils-SIG) > > If you feel that what we are about to change in distutils > is wrong, you can go ahead and help us by participating > in Distutils-ML, so we keep one and only one media > for these discussions. > > The four sentences summary for people in a hurry: > > Getting metadata of a distribution that is not. > installed means running its setup.py. This means. > downloading the whole archive, and running. > third party code on your system. > > To avoid it, we are adding a [setup] section in. > setup.cfg where you can express the package metadata > statically. > > Conditional sections, specific to some system. > can be added to add some specific fields in [setup]. > > At the end, you will be able to get metadata fields > without running any third-party code, and possibly > get only the distribution setup.cfg for this. > > Now the long version. > > > Rational > ======== > > Today, if you want to list all the metadata (PEP 314) of a. > distribution that is not installed, you need to use it's. > setup.py CLI. > > So, basically, you download it, and run:: > > $ python setup.py --name > Foo > > $ python setup.py --version > 1.2 > > Where `name` and `version` are metadata fields. That's OK but as. > soon as the developers add more code in setup.py, this feature > might break or (worse) might do unwanted things on the target. > system. > > Why should we run third-party code just to get its metadata ? > > So we worked on a way to express those metadata statically, > by pushing them in `setup.cfg`. That's not hard, since all. > the metadata fields are static values. > > Adding a setup section in setup.cfg > =================================== > > So the first thing we want to introduce is a [setup] section, > that may contain any field from the metadata:: > > [setup] > name: Foo > version: 1.2 > > Distutils will look for them, and will use them. Of course > setup.py is still required, and in case an option that is. > a metadata field is passed to setup(), it will override one. > located in setup.cfg. > > PEP 341 is coming up > ==================== > > Remember the last Pycon Summit ? We said that we would > introduce a new metadata field to describe requirements.. > That's basically what PEP 341 is about, and we are still. > working on it. > > Basically, we will be able to write things like:: > > requires: Twisted == 8.2.0 > > What takes us so long is that adding requirements like > this in the metadata requires more things: > > - the ability to compare versions (this was described in. > PEP 386 but not finished yet) > > - the ability to make these requirements vary depending on. > the target system > > And the later makes our "setup.cfg" new [setup] section. > obsolete as soon as this new metadata field is added in. > Python. > > So we need more that what I've shown in the previous section > > Context-dependant sections > ========================== > > The second change we are going to introduce is context-dependant > sections in setup.cfg. > > A context dependant section is a section with a condition that is. > used only if the execution environment meets its condition. > > Here's an example:: > > [setup] > name: Foo > version: 1.3 > > [setup:sys_platform == 'win32'] > requires: pywin32 > requires: bar > 1.0 > > [setup:os_machine == '64bits'] > requires: some_package > > [setup:python_version == '2.4' or python_version == '2.5'] > requires: some_package > > > Every [setup:condition] section will be added in [setup] > only if the condition is met. > > The micro-language behind this is the simplest possible: > it compares only strings, with usual string operators, > and with the ability to combine expressions. It makes it also > easy to understand to non-pythoneers (os packagers). > > The pseudo-grammar is (I don't know how to write those but you'll > get it hopefully):: > > comp: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in' > comparison: expr (comp_op expr)* > expr: STRING > test: or_test > or_test: and_test ('or' and_test)* > and_test: not_test ('and' not_test)* > not_test: 'not' not_test | comparison > > where STRING belongs to any of those: > > - python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1]) > - os_name = os.name > - sys_platform = sys.platform > - os_sysname = os.uname()[0]. > - os_nodename = os.uname()[1] > - os_release = os.uname()[2]. > - os_version = os.uname()[3].. > - os_machine = os.uname()[4]. > - a free string, like '2.4', or 'win32' > > Distutils will provide a function that is able to read the metadata > of a distribution, given a setup.cfg file, for the target environment:: > > >>> from distutils.util import local_metadata > >>> local_metadata('setup.cfg') > > > Meaning that a Vanilla Python will be able to read the metadata > of a package without running any code. > > That's what distutils will use when setup.py is run. > > That's what any package manager will be able to use to work. > with distributions. > > Limitations > =========== > > If a distribution is unable for obscure reasons (or does not > wish) to set all metadata fields in setup.cfg, that's fine :. > the fields will be set to UNKNOWN when `local_metadata` is called. > > When setup.py is run, options passed to setup() will complete > those. > > When a package manager gets 'UNKOWN' values, it knows it might > need to do further processing by running setup.py. > > Why this change is good > ===================== > > Once the requires metadata is added in PEP 341, > being able to query the metadata for a distribution can be done > without doing anything else than reading a static file and interpreting > its conditions in a restricted and secured manner. > > So there's no need to run third-party code anymore. > > Possible use cases: > > - setup.cfg is published on PyPI. Packages managers like easy_install > or pip will be able to list all the distributions needed and their > versions for installing a distribution without having to download > anything ! > > - OS packagers will be able to list the dependencies of a distribution > without having to browse Python code. > > And, again, this new [setup] section is backward-compatible, and will > be a good complement to the work done in PEP 376. > > Thanks for reading so far. This code will be added in 2.7 and 3.2. > > Tarek From fwierzbicki at gmail.com Wed Sep 23 02:43:22 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Tue, 22 Sep 2009 20:43:22 -0400 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> Message-ID: <4dab5f760909221743l4de4d983s2d5eabe78f052e62@mail.gmail.com> On Tue, Sep 22, 2009 at 5:55 PM, Dino Viehland wrote: > For IronPython we wrote a set of tests which go through and define > the various operator methods in all sorts of combinations on both > new-style and old-style classes as well as subclasses of those classes > and then do the comparisons w/ logging. We've talked about this before, but now I'm reminded again -- do you happen to have a pointer to these tests just in case someone might want to grab them :) ? -Frank From python at mrabarnett.plus.com Wed Sep 23 03:01:15 2009 From: python at mrabarnett.plus.com (MRAB) Date: Wed, 23 Sep 2009 02:01:15 +0100 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> Message-ID: <4AB9735B.4000102@mrabarnett.plus.com> Dino Viehland wrote: > Is there a reason or a rule by which CPython reports different error > message for different failures to subscript? > > > > For example: > > > > >>> set()[2] > > Traceback (most recent call last): > > File "", line 1, in > > TypeError: 'set' object does not support indexing > > >>> class c(object): pass > > ... > > >>> c()[2] > > Traceback (most recent call last): > > File "", line 1, in > > TypeError: 'c' object does not support indexing > > > > But compare this to: > > > > >>> [].append[42] > > Traceback (most recent call last): > > File "", line 1, in > > TypeError: 'builtin_function_or_method' object is unsubscriptable > > >>> (lambda: 42)[42] > > Traceback (most recent call last): > > File "", line 1, in > > TypeError: 'function' object is unsubscriptable > > >>> property()[42] > > Traceback (most recent call last): > > File "", line 1, in > > TypeError: 'property' object is unsubscriptable > > > > IronPython had a bug report that we were getting this wrong for set > objects and that ?does not support indexing? was also a clearer error > message. I?m wondering if there?s some reason why the different error > messages occur which I?m missing. Otherwise I could switch to using > the more clear message or start marking types which should report the > unsubscriptable error. Does anyone have any insights into this? > I thought that the difference might be between iterable and non-iterable objects, but then I'd expect the class 'c' to behave like the other non-iterables. In fact, the result is different again if it's an old-style class: >>> class c: pass >>> c()[2] Traceback (most recent call last): File "", line 1, in c()[2] AttributeError: c instance has no attribute '__getitem__' From dinov at microsoft.com Wed Sep 23 03:15:22 2009 From: dinov at microsoft.com (Dino Viehland) Date: Wed, 23 Sep 2009 01:15:22 +0000 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4dab5f760909221743l4de4d983s2d5eabe78f052e62@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <4dab5f760909221743l4de4d983s2d5eabe78f052e62@mail.gmail.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD049385AF@TK5EX14MBXC120.redmond.corp.microsoft.com> Frank wrote: > > For IronPython we wrote a set of tests which go through and define > > the various operator methods in all sorts of combinations on both > > new-style and old-style classes as well as subclasses of those > classes > > and then do the comparisons w/ logging. > We've talked about this before, but now I'm reminded again -- do you > happen to have a pointer to these tests just in case someone might > want to grab them :) ? > We include them in the source distribution which you can download the 2.6RC1 release which just came out today here: http://ironpython.codeplex.com/Release/ProjectReleases.aspx?ReleaseId=30315#DownloadId=84669 All of the side-by-side tests are in: IronPython-2.6\Src\Tests\compat Or you can view the source via the web at: http://ironpython.codeplex.com/SourceControl/ListDownloadableCommits.aspx (this page also has instructions for Subversion and other source control clients) And the latest version there is in: IronPython_Main\Src\Tests\compat Hopefully the infrastructure will just work on Jython because it also runs on CPython but there may be some Windows specific code in there (e.g. import nt, a general problem w/ our tests from the days before we always ran tests w/ the CPython std lib present). If you run into any problems let me know and I'd be happy to fix it to make them more compatible. From guido at python.org Wed Sep 23 04:28:41 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 22 Sep 2009 19:28:41 -0700 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: David, Isn't it clear from Tarek's email that he is summarizing distutils-sig? Isn't it odd that you ask this question pointing to your own message? Are you worried that you won't get credit for the idea (assuming it's yours)? Should you perhaps just ask for credit instead of asking a rhetorical question? --Guido On Tue, Sep 22, 2009 at 5:07 PM, David Lyon wrote: > > Tarek, > > Are you claiming this as your own work and ideas? > > Given: > > http://mail.python.org/pipermail/distutils-sig/2009-August/012998.html > > Regards > > David > > > On Tue, 22 Sep 2009 15:21:06 +0200, Tarek Ziad? > wrote: >> Hello >> >> Here's a wrapup of the Distutils-SIG discussion >> we had on the "static metadata" topic. >> >> I realize that it's a good thing to send in. >> python-dev such wrapup on distutils design >> decisions, to keep everyone informed and get >> more feedback when required. >> >> I will try to do it for every upcoming changes >> that are not going in a PEP process (when it's not >> a 'big' change). The rate of such mails should >> not be very high. (around 1 mail in python-dev >> for +150 mails in distutils-SIG) >> >> If you feel that what we are about to change in distutils >> is wrong, you can go ahead and help us by participating >> in Distutils-ML, so we keep one and only one media >> for these discussions. >> >> The four sentences summary for people in a hurry: >> >> ? ? Getting metadata of a distribution that is not. >> ? ? installed means running its setup.py. This means. >> ? ? downloading the whole archive, and running. >> ? ? third party code on your system. >> >> ? ? To avoid it, we are adding a [setup] section in. >> ? ? setup.cfg where you can express the package metadata >> ? ? statically. >> >> ? ? Conditional sections, specific to some system. >> ? ? can be added to add some specific fields in [setup]. >> >> ? ? At the end, you will be able to get metadata fields >> ? ? without running any third-party code, and possibly >> ? ? get only the distribution setup.cfg for this. >> >> Now the long version. >> >> >> Rational >> ======== >> >> Today, if you want to list all the metadata (PEP 314) of a. >> distribution that is not installed, you need to use it's. >> setup.py CLI. >> >> So, basically, you download it, and run:: >> >> ? ? $ python setup.py --name >> ? ? Foo >> >> ? ? $ python setup.py --version >> ? ? 1.2 >> >> Where `name` and `version` are metadata fields. That's OK but as. >> soon as the developers add more code in setup.py, this feature >> might break or (worse) might do unwanted things on the target. >> system. >> >> Why should we run third-party code just to get its metadata ? >> >> So we worked on a way to express those metadata statically, >> by pushing them in `setup.cfg`. That's not hard, since all. >> the metadata fields are static values. >> >> Adding a setup section in setup.cfg >> =================================== >> >> So the first thing we want to introduce is a [setup] section, >> that may contain any field from the metadata:: >> >> ? ? [setup] >> ? ? name: Foo >> ? ? version: 1.2 >> >> Distutils will look for them, and will use them. Of course >> setup.py is still required, and in case an option that is. >> a metadata field is passed to setup(), it will override one. >> located in setup.cfg. >> >> PEP 341 is coming up >> ==================== >> >> Remember the last Pycon Summit ? We said that we would >> introduce a new metadata field to describe requirements.. >> That's basically what PEP 341 is about, and we are still. >> working on it. >> >> Basically, we will be able to write things like:: >> >> ? ? requires: Twisted == 8.2.0 >> >> What takes us so long is that adding requirements like >> this in the metadata requires more things: >> >> - the ability to compare versions (this was described in. >> ? PEP 386 but not finished yet) >> >> - the ability to make these requirements vary depending on. >> ? the target system >> >> And the later makes our "setup.cfg" new [setup] section. >> obsolete as soon as this new metadata field is added in. >> Python. >> >> So we need more that what I've shown in the previous section >> >> Context-dependant sections >> ========================== >> >> The second change we are going to introduce is context-dependant >> sections in setup.cfg. >> >> A context dependant section is a section with a condition that is. >> used only if the execution environment meets its condition. >> >> Here's an example:: >> >> ? ? [setup] >> ? ? name: Foo >> ? ? version: 1.3 >> >> ? ? [setup:sys_platform == 'win32'] >> ? ? requires: pywin32 >> ? ? requires: bar > 1.0 >> >> ? ? [setup:os_machine == '64bits'] >> ? ? requires: some_package >> >> ? ? [setup:python_version == '2.4' or python_version == '2.5'] >> ? ? requires: some_package >> >> >> Every [setup:condition] section will be added in [setup] >> only if the condition is met. >> >> The micro-language behind this is the simplest possible: >> it compares only strings, with usual string operators, >> and with the ability to combine expressions. It makes it also >> easy to understand to non-pythoneers (os packagers). >> >> The pseudo-grammar is (I don't know how to write those but you'll >> get it hopefully):: >> >> ? ? comp: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in' >> ? ? comparison: expr (comp_op expr)* >> ? ? expr: STRING >> ? ? test: or_test >> ? ? or_test: and_test ('or' and_test)* >> ? ? and_test: not_test ('and' not_test)* >> ? ? not_test: 'not' not_test | comparison >> >> where STRING belongs to any of those: >> >> - python_version = '%s.%s' % (sys.version_info[0], sys.version_info[1]) >> - os_name = os.name >> - sys_platform = sys.platform >> - os_sysname = os.uname()[0]. >> - os_nodename = os.uname()[1] >> - os_release = os.uname()[2]. >> - os_version = os.uname()[3].. >> - os_machine = os.uname()[4]. >> - a free string, like '2.4', or 'win32' >> >> Distutils will provide a function that is able to read the metadata >> of a distribution, given a setup.cfg file, for the target environment:: >> >> ? ? >>> from distutils.util import local_metadata >> ? ? >>> local_metadata('setup.cfg') >> ? ? >> >> Meaning that a Vanilla Python will be able to read the metadata >> of a package without running any code. >> >> That's what distutils will use when setup.py is run. >> >> That's what any package manager will be able to use to work. >> with distributions. >> >> Limitations >> =========== >> >> If a distribution is unable for obscure reasons (or does not >> wish) to set all metadata fields in setup.cfg, that's fine :. >> the fields will be set to UNKNOWN when `local_metadata` is called. >> >> When setup.py is run, options passed to setup() will complete >> those. >> >> When a package manager gets 'UNKOWN' values, it knows it might >> need to do further processing by running setup.py. >> >> Why this change is good >> ===================== >> >> Once the requires metadata is added in PEP 341, >> being able to query the metadata for a distribution can be done >> without doing anything else than reading a static file and interpreting >> its conditions in a restricted and secured manner. >> >> So there's no need to run third-party code anymore. >> >> Possible use cases: >> >> - setup.cfg is published on PyPI. Packages managers like easy_install >> ? or pip will be able to list all the distributions needed and their >> ? versions for installing a distribution without having to download >> ? anything ! >> >> - OS packagers will be able to list the dependencies of a distribution >> ? without having to browse Python code. >> >> And, again, this new [setup] section is backward-compatible, and will >> be a good complement to the work done in PEP 376. >> >> Thanks for reading so far. This code will be added in 2.7 and 3.2. >> >> Tarek > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rdmurray at bitdance.com Wed Sep 23 05:08:07 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 22 Sep 2009 23:08:07 -0400 (EDT) Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <4AB9735B.4000102@mrabarnett.plus.com> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> Message-ID: On Wed, 23 Sep 2009 at 02:01, MRAB wrote: > Dino Viehland wrote: >> Is there a reason or a rule by which CPython reports different error >> message for different failures to subscript? >> >> For example: >> >> > > > set()[2] >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: 'set' object does not support indexing >> > > > class c(object): pass >> ... [....] >> > > > [].append[42] >> Traceback (most recent call last): >> File "", line 1, in >> TypeError: 'builtin_function_or_method' object is unsubscriptable [...] >> IronPython had a bug report that we were getting this wrong for set >> objects and that ?does not support indexing? was also a clearer error >> message. I?m wondering if there?s some reason why the different error >> messages occur which I?m missing. Otherwise I could switch to using the >> more clear message or start marking types which should report the >> unsubscriptable error. Does anyone have any insights into this? >> > I thought that the difference might be between iterable and non-iterable > objects, but then I'd expect the class 'c' to behave like the other > non-iterables. In fact, the result is different again if it's an > old-style class: > >> > > class c: pass >> > > c()[2] > Traceback (most recent call last): > File "", line 1, in > c()[2] > AttributeError: c instance has no attribute '__getitem__' Looking at the source, these messages are generated from abstract.c, and the difference appears to be whether or not the tp_as_sequence slot is filled in or not. If it is, but there is no sq_item method, then PySequence_GetItem gives the "does not support indexing" message. If it isn't filled in, you get the 'not subscriptable"(*) message from PyObject_GetItem. The logic appears to be, roughly, if an object does not have mapping methods, or has them but does not have mp_subscript, and does have sequence methods but does not have sq_item, then you get a 'does not support indexing'. Otherwise you get 'not subscriptable'. The question is, is this a useful distinction, or is it an implementation artifact? --David (*) The error message changed format slightly in 2.7 due to http://bugs.python.org/issue5760, and the discussion there is worth reading in this context. From david.lyon at preisshare.net Wed Sep 23 05:55:29 2009 From: david.lyon at preisshare.net (David Lyon) Date: Tue, 22 Sep 2009 23:55:29 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: <3e228a5145369e7d9932040c73a7d58e@preisshare.net> Hello Guido, > Isn't it clear from Tarek's email that he is summarizing > distutils-sig? Well, I'm new to this. Summarising is a word that could be certainly applied. > Isn't it odd that you ask this question pointing to > your own message? It's not odd at all. When I see my ideas, which Tarek originally argued against, published with his name on them, and my name left out, I just felt it fair to myself to ask him how that might have happened. > Are you worried that you won't get credit for the > idea (assuming it's yours)? As I said, I'm new. I've been very happy to help out. This is all a learning process for me. I can accept whatever decision Tarek makes on distutils and the direction of the code. If he chose to ignore my postings and go a different way from what I suggested then that is his perogative. If the failing was on my part to not fully write up PEPS and do full examples, and diagrams and so forth, then I accept that. > Should you perhaps just ask for credit instead of asking a > rhetorical question? I don't really believe that the work is complete enough for anybody to claim credit for just yet. That's why I was surprised to see the announcement. Hence me asking about it. In all fairness, I'm trying to help Tarek here. And the python packaging effort. Getting my name on what I believe are my contributions shouldn't be such a big deal and I'm very sure you'd agree on that. But it is an ongoing struggle for whatever reason that I've been having. I'm sure your guidance can resolve the issues easily.. David From dinov at microsoft.com Wed Sep 23 06:23:59 2009 From: dinov at microsoft.com (Dino Viehland) Date: Wed, 23 Sep 2009 04:23:59 +0000 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD04939586@TK5EX14MBXC120.redmond.corp.microsoft.com> R. David Murray wrote: > Looking at the source, these messages are generated from abstract.c, > and the difference appears to be whether or not the tp_as_sequence slot > is filled in or not. If it is, but there is no sq_item method, then > PySequence_GetItem gives the "does not support indexing" message. > If it isn't filled in, you get the 'not subscriptable"(*) message from > PyObject_GetItem. > > The logic appears to be, roughly, if an object does not have mapping > methods, or has them but does not have mp_subscript, and does have > sequence methods but does not have sq_item, then you get a 'does not > support indexing'. Otherwise you get 'not subscriptable'. > > The question is, is this a useful distinction, or is it an > implementation artifact? First on the useful distinction: for better or worse users do seem to have an affinity towards CPython's error messages and especially so when they're better than ours :). But I certainly hope no one is taking a dependency on them by parsing them or anything like that such that it would actually break someone's program. So basically I regard all error messages as implementation artifacts but I have no problem matching them if it'll make someone happier and it's not too crazy to do it. That being said there does seem to be a useful distinction here. It sounds like the idea here is to report a better error message when something is more collection like than a full blown sequence - in particular I'm thinking of .NET's ICollection and Java's Collection interfaces that are very sequence like but don't support getting elements by index. The only oddity in the error message to me is that user defined objects are effectively always treated as sequences. That's completely an implementation artifact but I'm going to copy it too because it's easy, may prevent another bug report, and checking for all the various sequence methods seems like unnecessary overkill. Thanks for the information! From greg.ewing at canterbury.ac.nz Wed Sep 23 06:27:21 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 23 Sep 2009 16:27:21 +1200 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> Message-ID: <4AB9A3A9.1090800@canterbury.ac.nz> Willem Broekema wrote: > The AND above (which I uppercased) is subtle but important. In the "x > op y" case with y being of a subclass of the class of x, if there is > no class in between x and y (excluding x, including y) that overrides > the __rop__ method, then y,__rop__(x) is *not* tried before > x.__op__(y). How does this work at the C typeslot level, where there are no __rop__ methods? -- Greg From brett at python.org Wed Sep 23 07:39:04 2009 From: brett at python.org (Brett Cannon) Date: Tue, 22 Sep 2009 22:39:04 -0700 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <3e228a5145369e7d9932040c73a7d58e@preisshare.net> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <3e228a5145369e7d9932040c73a7d58e@preisshare.net> Message-ID: On Tue, Sep 22, 2009 at 20:55, David Lyon wrote: > > Hello Guido, > >> Isn't it clear from Tarek's email that he is summarizing >> distutils-sig? > > Well, I'm new to this. Summarising is a word that could be > certainly applied. > >> Isn't it odd that you ask this question pointing to >> your own message? > > It's not odd at all. When I see my ideas, which Tarek originally > argued against, published with his name on them, and my name left > out, I just felt it fair to myself to ask him how that might have > happened. > >> Are you worried that you won't get credit for the >> idea (assuming it's yours)? > > As I said, I'm new. I've been very happy to help out. This is > all a learning process for me. > > I can accept whatever decision Tarek makes on distutils and the > direction of the code. If he chose to ignore my postings and > go a different way from what I suggested then that is his > perogative. > > If the failing was on my part to not fully write up PEPS and > do full examples, and diagrams and so forth, then I accept that. > >> Should you perhaps just ask for credit instead of asking a >> rhetorical question? > > I don't really believe that the work is complete enough for > anybody to claim credit for just yet. That's why I was surprised > to see the announcement. Hence me asking about it. > > In all fairness, I'm trying to help Tarek here. And the python > packaging effort. > > Getting my name on what I believe are my contributions shouldn't > be such a big deal and I'm very sure you'd agree on that. But it > is an ongoing struggle for whatever reason that I've been having. > > I'm sure your guidance can resolve the issues easily.. If you re-read Tarek's email is uses "we" throughout the entire discussion, only using "I" when referring to the email itself. I don't think anyone on python-dev thought that it was exclusively Tarek's idea or necessarily even mostly his. Personally I just attribute the idea to the distutils-SIG and don't think beyond that. If you want explicit credit, you can co-author a PEP or get thanked in a checkin as you mentioned. But honestly, from my observations of open source, ideas are not what get you noticed, it's producing something tangible like code. Plus ideas and such pull from so many people on mailing lists you can claim you came up with the initial idea, but I am sure a ton of people provided feedback which makes ownership of any idea practically moot. Trust me, if you are doing open source for anything other than altruistic reasons you are bound to be disappointed. -Brett From david.lyon at preisshare.net Wed Sep 23 08:20:47 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 23 Sep 2009 02:20:47 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <3e228a5145369e7d9932040c73a7d58e@preisshare.net> Message-ID: On Tue, 22 Sep 2009 22:39:04 -0700, Brett Cannon wrote: > I don't think anyone on python-dev thought that it was exclusively Tarek's > idea or necessarily even mostly his. Actually, he originally argued against it, but that is irrelevant. I'm happy that he's come around and embraced it. Perhaps a bit too much could be my issue. > If you want explicit credit, you can co-author a PEP or get thanked in > a checkin as you mentioned. Good idea. How can I get check-in privileges on distutils ? What is the process? > But honestly, from my observations of open > source, ideas are not what get you noticed, it's producing something > tangible like code. Sure.. you're 100% right. So I need to be able to work on code and be able to check it in. I would love that. How do I get that? > Plus ideas and such pull from so many people on > mailing lists you can claim you came up with the initial idea, but I > am sure a ton of people provided feedback which makes ownership of any > idea practically moot. Claiming ownership? No No. Lets not go there. A worklist and checkin rights would completely suffice. David From brett at python.org Wed Sep 23 08:42:36 2009 From: brett at python.org (Brett Cannon) Date: Tue, 22 Sep 2009 23:42:36 -0700 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <3e228a5145369e7d9932040c73a7d58e@preisshare.net> Message-ID: On Tue, Sep 22, 2009 at 23:20, David Lyon wrote: > On Tue, 22 Sep 2009 22:39:04 -0700, Brett Cannon wrote: >> I don't think anyone on python-dev thought that it was exclusively > Tarek's >> idea or necessarily even mostly his. > > Actually, he originally argued against it, but that is irrelevant. > > I'm happy that he's come around and embraced it. Perhaps a bit too > much could be my issue. > >> If you want explicit credit, you can co-author a PEP or get thanked in >> a checkin as you mentioned. > > Good idea. > > How can I get check-in privileges on distutils ? What is the process? > Contribute patches on the Python issue tracker for six months to a year (depends on how much you contribute) and be generally liked by other committers. Distutils is not separate from Python itself. But notice I said being thanked in a checkin, not making a checkin yourself. >> But honestly, from my observations of open >> source, ideas are not what get you noticed, it's producing something >> tangible like code. > > Sure.. you're 100% right. So I need to be able to work on code and > be able to check it in. I would love that. How do I get that? > You are conflating contributing code and checking it in. You can do the former without the latter. >> Plus ideas and such pull from so many people on >> mailing lists you can claim you came up with the initial idea, but I >> am sure a ton of people provided feedback which makes ownership of any >> idea practically moot. > > Claiming ownership? No No. Lets not go there. > > A worklist and checkin rights would completely suffice. Talk to Tarek or look at bugs.python.org for a worklist; checkin rights come MUCH later. -Brett From stephen at xemacs.org Wed Sep 23 09:03:45 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 23 Sep 2009 16:03:45 +0900 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> Message-ID: <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> Brett Cannon writes: > OK, stop before it gets any nastier as your email already sounds > rude Rude, yes, but not entirely lacking in truth. Tarek's post to Python- Dev *is* a mess from the point of view of English and organization -- it's *way* below Tarek's normal command of the language. Granted, something that complex is much harder to polish than a 3-sentence post, but it could have been run by a native speaker before posting. Better yet, it could have been run by Distutils-SIG first. At the very least, that would have kept this discussion on Distutils- SIG, and Chris couldn't be accused of trying to make an end run around that process. I suggest that posting progress reports to Python-Dev is a good idea (attracting attention and maybe participation to the process), but making one an opportunity to test the degree of internal consensus (or lack of it) on Distutils-SIG by posting there first is an even better one. > I see no reason why [Tarek] should not be able to short-circuit > discussions when he sees them as having gone on long enough when he > is the one that is going to ultimately end up doing all the coding. Of course there are benefits to having a strong impartial chairman, and installing a benevolent dictator is often a second-best strategy for project management. However, in this case, the question of what is being short-circuited is important. If it's bikeshedding, that's good. If it's a substantial difference, it's not. And it matters, because the benefits of a New! Improved! distutils! really start piling up when adoption becomes nearly universal. How do you tell the difference? Simple. If Philip wants the bikeshed painted pink, but he'll still move his bike to Tarek's blue bikeshed from the existing pink one, further discussion at this point is bikeshedding. If Chris wants the new bikeshed painted chartreuse, and he'll keep parking in the existing green one if Tarek paints the new one blue, that's a real difference. (This assumes Chris represents a sizable group of users/projects, of course.) From mal at egenix.com Wed Sep 23 09:49:16 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 23 Sep 2009 09:49:16 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> Message-ID: <4AB9D2FC.6040705@egenix.com> Tarek Ziad? wrote: > Hello > > Here's a wrapup of the Distutils-SIG discussion > we had on the "static metadata" topic. > > I realize that it's a good thing to send in. > python-dev such wrapup on distutils design > decisions, to keep everyone informed and get > more feedback when required. > > I will try to do it for every upcoming changes > that are not going in a PEP process (when it's not > a 'big' change). The rate of such mails should > not be very high. (around 1 mail in python-dev > for +150 mails in distutils-SIG) > > If you feel that what we are about to change in distutils > is wrong, you can go ahead and help us by participating > in Distutils-ML, so we keep one and only one media > for these discussions. > > The four sentences summary for people in a hurry: > > Getting metadata of a distribution that is not. > installed means running its setup.py. This means. > downloading the whole archive, and running. > third party code on your system. > > To avoid it, we are adding a [setup] section in. > setup.cfg where you can express the package metadata > statically. > > Conditional sections, specific to some system. > can be added to add some specific fields in [setup]. > > At the end, you will be able to get metadata fields > without running any third-party code, and possibly > get only the distribution setup.cfg for this. While it's a good idea to put up some form of meta-data into an index, I wonder why you are using setup.cfg for this. setup.cfg has traditionally been used to configure distutils, not to define meta-data. As such you wouldn't want to put such a configuration file up on PyPI. Wouldn't it be better use a new file for this (with the same syntax), e.g. metadata.cfg ?! This would then just contain the meta data that needs to be published. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 23 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From chris at simplistix.co.uk Wed Sep 23 10:12:21 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 23 Sep 2009 09:12:21 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> Message-ID: <4AB9D865.1070701@simplistix.co.uk> Mark Dickinson wrote: > I (still :-) think this is covered, for Python 2.x at least, by: > > http://docs.python.org/reference/datamodel.html#coercion-rules But this isn't coercion! :-) > - For objects x and y, first x.__op__(y) is tried. If this is not > implemented or returns NotImplemented, y.__rop__(x) is tried. Also, the above is not so: Python 2.5.1 >>> class X: ... def __eq__(self,other): ... print "X __eq__" >>> class Z: pass ... >>> Z()==X() X __eq__ No __req__ in sight... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From floris.bruynooghe at gmail.com Wed Sep 23 10:37:45 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Wed, 23 Sep 2009 09:37:45 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <4AB9D2FC.6040705@egenix.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB9D2FC.6040705@egenix.com> Message-ID: <20090923083745.GA14012@laurie.devork> On Wed, Sep 23, 2009 at 09:49:16AM +0200, M.-A. Lemburg wrote: > While it's a good idea to put up some form of meta-data > into an index, I wonder why you are using setup.cfg > for this. > > setup.cfg has traditionally been used to configure distutils, > not to define meta-data. As such you wouldn't want to > put such a configuration file up on PyPI. > > Wouldn't it be better use a new file for this (with the > same syntax), e.g. metadata.cfg ?! This would then just > contain the meta data that needs to be published. IIRC the goal was to make minimal changes and setup.cfg is known already. It's use is similar too: it configures the setup.py script with static data. PyPI could only extract the [setup:*] sections if you'r worried about it having random configuration from developers. I don't think this is enough of a argument in favour or not of a metadata file separate from setup.cfg. Regards Floris -- Debian GNU/Linux -- The Power of Freedom www.debian.org | www.gnu.org | www.kernel.org From skip at pobox.com Wed Sep 23 05:43:59 2009 From: skip at pobox.com (skip at pobox.com) Date: Tue, 22 Sep 2009 22:43:59 -0500 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> Message-ID: <19129.39295.970536.916767@montanaro.dyndns.org> Dino> For IronPython we wrote a set of tests which go through and define Dino> the various operator methods in all sorts of combinations on both Dino> new-style and old-style classes as well as subclasses of those Dino> classes and then do the comparisons w/ logging. It would be very nice if these complex corner cases had a set of test cases which could be run by all implementations (CPython, Jython, IronPython, PyPy, etc). I don't know. Maybe the CPython test suite serves that purpose, but it seems like it would be helpful if this sort of "validation suite" was maintained as a separate project all implementations could use and contribute to. Skip From dickinsm at gmail.com Wed Sep 23 13:05:31 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 23 Sep 2009 12:05:31 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4AB9D865.1070701@simplistix.co.uk> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <4AB9D865.1070701@simplistix.co.uk> Message-ID: <5c6f2a5d0909230405k9a0cd8ew8b36c9b147145632@mail.gmail.com> On Wed, Sep 23, 2009 at 9:12 AM, Chris Withers wrote: > Mark Dickinson wrote: >> >> I (still :-) think this is covered, for Python 2.x at least, by: >> >> http://docs.python.org/reference/datamodel.html#coercion-rules > > But this isn't coercion! :-) Agreed. FWIW this behaviour for arithmetic operations is also mentioned in http://docs.python.org/reference/datamodel.html#emulating-numeric-types but then again that section doesn't include the comparison operators. > >> - For objects x and y, first x.__op__(y) is tried. If this is not >> implemented or returns NotImplemented, y.__rop__(x) is tried. > > Also, the above is not so: > > Python 2.5.1 >>>> class X: > ... ? def __eq__(self,other): > ... ? ? print "X __eq__" >>>> class Z: pass > ... >>>> Z()==X() > X __eq__ > > No __req__ in sight... Okay, so combine this with the sentence under: http://docs.python.org/reference/datamodel.html#object.__eq__ that says: "There are no swapped-argument versions of these methods (to be used when the left argument does not support the operation but the right argument does); rather, __lt__() and __gt__() are each other?s reflection, __le__() and __ge__() are each other?s reflection, and __eq__() and __ne__() are their own reflection." So in the earlier doc snippets, if __op__ is __eq__, then __rop__ should also be interpreted as __eq__; similarly if __op__ is __lt__ then __rop__ is __gt__. I'm not saying that the documentation here couldn't be improved; just that IMO the docs do (with a little bit of extrapolation) describe what should happen, giving the 'official' specification that you were after. I don't know where/whether the behaviour for classes that define both __cmp__ and __eq__ are documented, though, and I'm far from sure what the rules are in that case. One minor oddity is that for arithmetic operations like __add__(self, other), if type(self) == type(other) then __radd__ isn't checked, on the basis that if __add__ fails then the operation is presumably not supported. This makes sense, but I wonder why this doesn't apply equally well to __eq__: that is, in doing A() == A() for a class A, why do we try the __eq__ method of both instances? Mark From dickinsm at gmail.com Wed Sep 23 13:06:09 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 23 Sep 2009 12:06:09 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> Message-ID: <5c6f2a5d0909230406q217eb528oe52902d86f17a0e2@mail.gmail.com> On Wed, Sep 23, 2009 at 4:43 AM, wrote: > > ? ?Dino> For IronPython we wrote a set of tests which go through and define > ? ?Dino> the various operator methods in all sorts of combinations on both > ? ?Dino> new-style and old-style classes as well as subclasses of those > ? ?Dino> classes and then do the comparisons w/ logging. > > It would be very nice if these complex corner cases had a set of test > cases which could be run by all implementations (CPython, Jython, > IronPython, PyPy, etc). ?I don't know. ?Maybe the CPython test suite serves > that purpose, but it seems like it would be helpful if this sort of > "validation suite" was maintained as a separate project all implementations > could use and contribute to. +1 Mark From solipsis at pitrou.net Wed Sep 23 13:36:49 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 23 Sep 2009 11:36:49 +0000 (UTC) Subject: [Python-Dev] altruism References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <3e228a5145369e7d9932040c73a7d58e@preisshare.net> Message-ID: Brett Cannon python.org> writes: > > Trust me, if you are doing open source for > anything other than altruistic reasons you are bound to be > disappointed. I'm surprised by this statement but perhaps it's a matter of vocabulary. Having fun and doing things you like to do does not strike me as "altruistic". Being involved in a FOSS project is not the same as participating in a charity. See this quick presentation about motivation of FOSS developers (second slide says: "Altruism? Not really"): http://www.infonomics.nl/FLOSS/papers/20030619/index.htm The full study is at: http://www.infonomics.nl/FLOSS/report/index.htm ("Part IV: Survey of Developers") (but of course there's the question of whether "altruism" exists at large, or is a political and moral fiction) Regards Antoine. From g.brandl at gmx.net Wed Sep 23 13:48:21 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 23 Sep 2009 13:48:21 +0200 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4AB8E136.8040706@simplistix.co.uk> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> Message-ID: Chris Withers schrieb: > Jared Flatow wrote: >> This might help: >> >> http://mail.python.org/pipermail/python-dev/2008-June/080111.html >> >> Here is the most relevant part (quoting Guido): >> >> > Does it help if I tell you that for "x y" we always try >> > x.__binop__(y) before trying y.__reverse_binop__(x), *except* in the >> > case where y is an instance of a subclass of the class of x? > > Okay, but does that count as a pronouncement that should go across all > versions and platforms? Yes. Unfortunately it's just documented for the other binops, but not for the comparisons; the note in the docs should be copied to the respective section. Georg From ziade.tarek at gmail.com Wed Sep 23 13:58:54 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 23 Sep 2009 13:58:54 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <4AB95428.1040704@simplistix.co.uk> Message-ID: <94bdd2610909230458g27fb8229q58e0325e1418eb88@mail.gmail.com> On Wed, Sep 23, 2009 at 12:53 AM, Sridhar Ratnakumar wrote: > On Tue, 22 Sep 2009 15:48:08 -0700, Chris Withers > wrote: > >> >> Since the language summit at PyCon 2009 various committers, including >> me, have been encouraging Tarek to act as distutils dictator to get >> things finished as we all know people are prone to bikeshedding that >> would kill any forward momentum we have towards improving the >> packaging situation. >> ?...except that this isn't bikeshedding. Use of that term just to stifle >> productive discussion is also rude. > > As I see it, the choice of syntax *as such* is an issue of bikeshedding > except that the choice of one syntax over the another could make it > supportive to add an useful feature in setuptools to distutils. For > instance, this syntax which I proposed - > http://mail.python.org/pipermail/distutils-sig/2009-September/013289.html - > will make us think about adding install_requires/extra_requires to distutils > *in addition to* the conditional metadata .. rather than inventing a new > syntax to support only conditional metadata. Please keep in mind that the original proposal is having the ability to get static metadata (metadata: PEP 314) without running any code. Your mail contains two things: - yet another way to express the conditions (that's the bikeshedding) - a setuptools mechanism located in setup.py that works with an option to build a requires.txt file when you install a package or create a binary distribution. The later is not related to what we want to do, or if it is, please refine your proposal and make it clear at Distutils-SIG. Tarek -- Tarek Ziad? | http://ziade.org |????????????! From ziade.tarek at gmail.com Wed Sep 23 14:05:04 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 23 Sep 2009 14:05:04 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <4AB95428.1040704@simplistix.co.uk> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <4AB95428.1040704@simplistix.co.uk> Message-ID: <94bdd2610909230505t7bfebca8lf6cba182878e53fa@mail.gmail.com> 2009/9/23 Chris Withers : >> Since the language summit at PyCon 2009 various committers, including >> me, have been encouraging Tarek to act as distutils dictator to get >> things finished as we all know people are prone to bikeshedding that >> would kill any forward momentum we have towards improving the >> packaging situation. > > ...except that this isn't bikeshedding. Use of that term just to stifle > productive discussion is also rude. - choosing if we are going to use "py<2.6" instead of "python_version<2.6" is bikeshedding. - explaining that os.uname() contains irrelevant infos and we shouldn't add it as-is in the execution context is not bikeshedding (see Ronald last mail in the thread) Please go back to the thread at distutils-SIG and list all points in the proposal that seems to you as non-bikeshedding things. Tarek -- Tarek Ziad? | http://ziade.org |????????????! From ncoghlan at gmail.com Wed Sep 23 14:05:29 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 23 Sep 2009 22:05:29 +1000 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> Message-ID: <4ABA0F09.9060703@gmail.com> Mark Dickinson wrote: > On Tue, Sep 22, 2009 at 3:37 PM, Chris Withers > wrote: >> Where are the specifications on what happens if two objects are >> compared and both have implementations of __eq__? Which __eq__ is >> called? What happens if the first one called returns False? Is the >> second one called? What is one implements __eq__ and the other >> __ne__? > > I (still :-) think this is covered, for Python 2.x at least, by: > > http://docs.python.org/reference/datamodel.html#coercion-rules Quite some time ago, I attempted to accurately document all of this - my efforts are buried in an ODF document in the SVN sandbox: http://svn.python.org/view/sandbox/trunk/userref/ODF/Chapter02_StatementsAndExpressions.odt Extracting all of that text with some kind of ODT2ReST converter has now been on my to-do list for over a year :P Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Wed Sep 23 14:09:13 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 23 Sep 2009 22:09:13 +1000 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4AB9A3A9.1090800@canterbury.ac.nz> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <4AB9A3A9.1090800@canterbury.ac.nz> Message-ID: <4ABA0FE9.7020401@gmail.com> Greg Ewing wrote: > Willem Broekema wrote: > >> The AND above (which I uppercased) is subtle but important. In the "x >> op y" case with y being of a subclass of the class of x, if there is >> no class in between x and y (excluding x, including y) that overrides >> the __rop__ method, then y,__rop__(x) is *not* tried before >> x.__op__(y). > > How does this work at the C typeslot level, where > there are no __rop__ methods? The interpreter doesn't promise to call those slots with "self" first - self will be the second argument in the "rop" case. See binary_op1() in abstract.c for the gory details (I had to go look it up myself in order to rediscover how it worked). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ziade.tarek at gmail.com Wed Sep 23 14:20:31 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 23 Sep 2009 14:20:31 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> On Wed, Sep 23, 2009 at 9:03 AM, Stephen J. Turnbull wrote: > At the very least, that would have kept this discussion on Distutils- > SIG, and Chris couldn't be accused of trying to make an end run around > that process. ?I suggest that posting progress reports to Python-Dev > is a good idea (attracting attention and maybe participation to the > process), but making one an opportunity to test the degree of internal > consensus (or lack of it) on Distutils-SIG by posting there first is > an even better one. Please define what "internal consensus on Distutils-SIG" means. If it means that everyone present in the Mailing-List needs to agree on a distutils proposal, this will never happen for many reasons. Here's one : That's not the distutils mailing list anymore. Here's a list of the projects "officialy" hosted in this mailing list: - distutils - setuptools - zc.buildout So, while it's legitimate to get some feedback from setuptools and zc.buildout users and developers, (on anyone else) on what we are planning for distutils, don't you agree that it's impossible in this context to get a 100% consensus ? and that I need to take some decisions to move distutils on ? Especially for this topic. If you take the time to read everything you'll see that there were no real alternative design proposed, and that I am just working out the details because I maintain and code distutils. Tarek -- Tarek Ziad? | http://ziade.org |????????????! From ncoghlan at gmail.com Wed Sep 23 14:22:46 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 23 Sep 2009 22:22:46 +1000 Subject: [Python-Dev] altruism In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <3e228a5145369e7d9932040c73a7d58e@preisshare.net> Message-ID: <4ABA1316.9000502@gmail.com> Antoine Pitrou wrote: > Brett Cannon python.org> writes: >> Trust me, if you are doing open source for >> anything other than altruistic reasons you are bound to be >> disappointed. > > I'm surprised by this statement but perhaps it's a matter of vocabulary. > Having fun and doing things you like to do does not strike me as "altruistic". > Being involved in a FOSS project is not the same as participating in a charity. I'd agree that 'altruism' isn't quite the right word - there's also the fact that plenty of folks these days contribute to open source because someone is paying them to :) However, Brett's basic point that good input may sometimes go uncredited through no fault of the poster's remains valid. Patches and particularly good bug reports/suggestions get credited in commit messages and sometimes NEWS items and the What's New, significant patches generally earn a mention in the ACKS file and each PEP usually has an acknowledgement section that lists major contributors to the associated discussion and reviews. Posts (even well-thought out ones) on the various discussion lists? Not so much - it's too easy to lose track of who posted what in an involved discussion. The highest respect you can really earn in those is to make a valid point clearly enough that you convince others to adopt your point of view. Although sometimes you can still persuade others even when you later turn out to be wrong* ;) Cheers, Nick. * See a couple of the footnotes to PEP 343 if you want to know what that smiley is about :) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From fwierzbicki at gmail.com Wed Sep 23 15:04:41 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Wed, 23 Sep 2009 09:04:41 -0400 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> Message-ID: <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com> On Tue, Sep 22, 2009 at 11:43 PM, wrote: > > ? ?Dino> For IronPython we wrote a set of tests which go through and define > ? ?Dino> the various operator methods in all sorts of combinations on both > ? ?Dino> new-style and old-style classes as well as subclasses of those > ? ?Dino> classes and then do the comparisons w/ logging. > > It would be very nice if these complex corner cases had a set of test > cases which could be run by all implementations (CPython, Jython, > IronPython, PyPy, etc). ?I don't know. ?Maybe the CPython test suite serves > that purpose, but it seems like it would be helpful if this sort of > "validation suite" was maintained as a separate project all implementations > could use and contribute to. Talk has started up again on the stdlib-sig list about finding a core stdlib + tests that can be shared by all implementations, potentially living apart from CPython. I have volunteered to put together a PEP on the subject, with Jessie Noller and Brett Canon are helping me out. When I have something worth showing, I'll start the real PEP process. -Frank From fwierzbicki at gmail.com Wed Sep 23 15:06:35 2009 From: fwierzbicki at gmail.com (Frank Wierzbicki) Date: Wed, 23 Sep 2009 09:06:35 -0400 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <1A472770E042064698CB5ADC83A12ACD049385AF@TK5EX14MBXC120.redmond.corp.microsoft.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <4dab5f760909221743l4de4d983s2d5eabe78f052e62@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD049385AF@TK5EX14MBXC120.redmond.corp.microsoft.com> Message-ID: <4dab5f760909230606j6fc88a34tac6bc95fb42456ce@mail.gmail.com> On Tue, Sep 22, 2009 at 9:15 PM, Dino Viehland wrote: > And the latest version there is in: > > IronPython_Main\Src\Tests\compat > > Hopefully the infrastructure will just work on Jython because it also runs on > CPython but there may be some Windows specific code in there (e.g. import nt, > a general problem w/ our tests from the days before we always ran tests w/ > the CPython std lib present). > > If you run into any problems let me know and I'd be happy to fix it to make > them more compatible. Great, thanks! I'll take a look (for real this time :) -Frank From eric at trueblade.com Wed Sep 23 15:01:52 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 23 Sep 2009 09:01:52 -0400 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> Message-ID: <4ABA1C40.7080001@trueblade.com> skip at pobox.com wrote: > Dino> For IronPython we wrote a set of tests which go through and define > Dino> the various operator methods in all sorts of combinations on both > Dino> new-style and old-style classes as well as subclasses of those > Dino> classes and then do the comparisons w/ logging. > > It would be very nice if these complex corner cases had a set of test > cases which could be run by all implementations (CPython, Jython, > IronPython, PyPy, etc). I don't know. Maybe the CPython test suite serves > that purpose, but it seems like it would be helpful if this sort of > "validation suite" was maintained as a separate project all implementations > could use and contribute to. IIRC, one of the reasons for "breaking out"[1] the standard library (and its test suite) was to allow for things like this. Eric. [1] For whatever definition "breaking out" ends up having. From stephen at xemacs.org Wed Sep 23 16:04:05 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 23 Sep 2009 23:04:05 +0900 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> Message-ID: <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> Tarek Ziad? writes: > On Wed, Sep 23, 2009 at 9:03 AM, Stephen J. Turnbull wrote: > > At the very least, that would have kept this discussion on Distutils- > > SIG, and Chris couldn't be accused of trying to make an end run around > > that process. ?I suggest that posting progress reports to Python-Dev > > is a good idea (attracting attention and maybe participation to the > > process), but making one an opportunity to test the degree of internal > > consensus (or lack of it) on Distutils-SIG by posting there first is > > an even better one. > > Please define what "internal consensus on Distutils-SIG" means. Definition is your job, as chairman/dictator. I did offer a concrete criterion for an individual's participation in a "internal consensus": that you expect that they will adopt the new features of distutils as a foundation for their own distribution systems, or at least not implement and promote an alternative. As for who needs to be included, if the author of setuptools isn't part of the internal consensus (on that, I'm just guessing from the fact that he went off to "start a new thread"), I think you should be concerned. He's already implemented and promoted an alternative in the past, so he doesn't even have to do any implementation. Just keep on using and promoting his preferred tools and formats. A third point is that you can have a consensus on "agreeing to disagree". But it's unlikely that you will have 100% *dis*agreement. If you can get a consensus that on certain points certain people are simply not going to change their minds, it's often possible to move past that to work on things that you can agree on. (And if some people are unwilling to even admit that, it's usually clear to absolutely everybody else.) > distutils, don't you agree that it's impossible in this context to > get a 100% consensus ? Depends on how you define "100% consensus". If you mean 100% of people agree on 100% of the proposal, of course not. But there may be 40% of the proposal you can get 90% of the people to agree on, and maybe 90% of the proposal is acceptable to 40% of the people. (That would be pretty good in this case, but this community regularly manages to come close to that, so there's some hope.) If you can get the setuptools and buildout people to agree to use some parts of "new distutils", that's a form of consensus, and definite progress. > and that I need to take some decisions to move distutils on ? I don't know; a better distribution system is not something I need as a user or a developer. What worries me is that a simple progress report caused dissension to spill over onto the Python-Dev list. That is relatively rare in this community. And people like Brett think it's important that some progress be made on distutils, so I'd like to see it done as quickly as possible -- but no quicker. > Especially for this topic. If you take the time to read everything > you'll see that there were no real alternative design proposed, and > that I am just working out the details because I maintain and code > distutils. Well, from the behavior of Philip and Chris, it seems that their position is that there was insufficient time to put forward an alternative design before the summary was posted to Python-Dev. *I don't care whether its true or not*, it's your job as chairman/ dictator to decide that, and we shouldn't discuss it here. But merely leaving the *impression* is damaging, and I suggested a simple procedure (posting the summary to your mailing list and requesting comments) that would very likely improve the summary, and also be likely to keep unnecessary controversy off Python-Dev. From benjamin at python.org Wed Sep 23 16:35:07 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 23 Sep 2009 09:35:07 -0500 Subject: [Python-Dev] thinking about 2.7 Message-ID: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Hi everyone, I've started plotting the release of 2.7. I'd like to try for a final release mid next summer. 3.2 should be released, if not at the same time as 2.7, within a few weeks to avoid 2.x having features which 3.x doesn't. If no one has problems with this, I will draft a schedule. Are we still planning to make 3.3 the main development focus and start the 5 years of 2.x maintenance after this release? Additionally, I'm very apprehensive about doing any kind of release without the buildbots running. Does anyone know when they might be up? -- Regards, Benjamin From skip at pobox.com Wed Sep 23 17:06:48 2009 From: skip at pobox.com (skip at pobox.com) Date: Wed, 23 Sep 2009 10:06:48 -0500 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4ABA1C40.7080001@trueblade.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <4ABA1C40.7080001@trueblade.com> Message-ID: <19130.14728.970627.817149@montanaro.dyndns.org> Eric> IIRC, one of the reasons for "breaking out"[1] the standard library (and Eric> its test suite) was to allow for things like this. In my opinion the standard library and the core test suite (the language validation stuff) are entirely independent beasts. I can understand pieces of the standard library not being available in one variant or another, but key semantic aspects of the language proper should be constant across implementations. That said, I agree that if the standard library is split off from CPython then the relevant portions of the test suite should go along with it. Skip From exarkun at twistedmatrix.com Wed Sep 23 17:13:55 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 23 Sep 2009 15:13:55 -0000 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Message-ID: <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> On 02:35 pm, benjamin at python.org wrote: >Hi everyone, >I've started plotting the release of 2.7. I'd like to try for a final >release mid next summer. 3.2 should be released, if not at the same >time as 2.7, within a few weeks to avoid 2.x having features which 3.x >doesn't. If no one has problems with this, I will draft a schedule. > >Are we still planning to make 3.3 the main development focus and start >the 5 years of 2.x maintenance after this release? I hope that this decision will be delayed until the release is closer, so that it can be based on how 3.x adoption is progressing. >Additionally, I'm very apprehensive about doing any kind of release >without the buildbots running. Does anyone know when they might be up? I was planning on replying to Antoine's earlier message about the buildbots after a sufficiently long silence. I'll reply here instead. Quite a few years of experience with a distributed team of build slave managers has shown me that by far the most reliable way to keep slaves online is to have them managed by a dedicated team. This team doesn't need to be small, but since finding dedicated people can sometimes be challenging, I think small teams are the most likely outcome (possibly resulting in a team of one). Adding more people who are only mildly interested doesn't help. If, as I believe is the case with Python's buildbot configuration, the mildly interested people have sole control over certain slaves, then it is actually detrimental. It's easy for someone to volunteer to set up a new slave. It's even easy to make sure it keeps running for 6 months. But it's not as easy to keep it running indefinitely. This isn't about the software involved (at least not entirely). It's about someone paying attention to whether the slave restarts on reboots, and about paying attention to whether the slave host has lots its network connection, or been decomissioned, or whether a system upgrade disabled the slave, or whatever other random administrative-like tasks are necessary to keep things running. Casual volunteers generally just won't keep up with these tasks. I suggest finding someone who's seriously interested in the quality of CPython and giving them the responsibility of keeping things operating properly. This includes paying attention to the status of slaves, cajoling hardware operators into bringing hosts back online and fixing network issues, and finding replacements of the appropriate type (hardware/software platform) when a slave host is permanently lost. I would also personally recommend that this person first (well, after tracking down all the slave operators and convincing them to bring their slaves back online) acquire shell access to all of the slave machines so that the owners of the slave hosts themselves no longer need to be the gating factor for most issues. Jean-Paul From fuzzyman at voidspace.org.uk Wed Sep 23 17:36:44 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 23 Sep 2009 16:36:44 +0100 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Message-ID: <4ABA408C.6080103@voidspace.org.uk> Benjamin Peterson wrote: > Hi everyone, > I've started plotting the release of 2.7. I'd like to try for a final > release mid next summer. 3.2 should be released, if not at the same > time as 2.7, within a few weeks to avoid 2.x having features which 3.x > doesn't. If no one has problems with this, I will draft a schedule. > +1 I'm keen to see a 2.7 release as there are good new features on trunk (particularly in unittest :-) Michael > Are we still planning to make 3.3 the main development focus and start > the 5 years of 2.x maintenance after this release? > > Additionally, I'm very apprehensive about doing any kind of release > without the buildbots running. Does anyone know when they might be up? > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From dinov at microsoft.com Wed Sep 23 17:54:35 2009 From: dinov at microsoft.com (Dino Viehland) Date: Wed, 23 Sep 2009 15:54:35 +0000 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> Message-ID: <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com> Skip wrote: > Dino> For IronPython we wrote a set of tests which go through and define > Dino> the various operator methods in all sorts of combinations on both > Dino> new-style and old-style classes as well as subclasses of those > Dino> classes and then do the comparisons w/ logging. > > It would be very nice if these complex corner cases had a set of test > cases which could be run by all implementations (CPython, Jython, > IronPython, PyPy, etc). I don't know. Maybe the CPython test suite serves > that purpose, but it seems like it would be helpful if this sort of > "validation suite" was maintained as a separate project all implementations > could use and contribute to. We are going to start contributing tests back real soon now. I'm not sure that these are the best tests to contribute as they require a version of Python to compare against rather than being nice and stand alone. But I'm sure we have other tests which cover this as well just not as exhaustively. We could also possibly check in the baseline file and then CPython could compare it's self to previous versions but it'd probably be a pretty big file - so it probably shouldn't be included in the standard install in the tests directory. From solipsis at pitrou.net Wed Sep 23 18:29:12 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 23 Sep 2009 16:29:12 +0000 (UTC) Subject: [Python-Dev] thinking about 2.7 References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Message-ID: Benjamin Peterson python.org> writes: > > Hi everyone, > I've started plotting the release of 2.7. I'd like to try for a final > release mid next summer. 3.2 should be released, if not at the same > time as 2.7, within a few weeks to avoid 2.x having features which 3.x > doesn't. If no one has problems with this, I will draft a schedule. If you want to avoid the low-activity low-responsiveness holiday period, you might shoot for June or September/October instead. Regards Antoine. From dickinsm at gmail.com Wed Sep 23 18:34:37 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 23 Sep 2009 17:34:37 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com> Message-ID: <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com> On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland wrote: > We are going to start contributing tests back real soon now. ?I'm not sure > that these are the best tests to contribute as they require a version of > Python to compare against rather than being nice and stand alone. ?But I'm > sure we have other tests which cover this as well just not as exhaustively. > We could also possibly check in the baseline file and then CPython could > compare it's self to previous versions but it'd probably be a pretty > big file - so it probably shouldn't be included in the standard install > in the tests directory. How big is big? For comparison, CPython's Lib/test/decimaltestdata directory alone is already over 4Mb, so maybe size isn't an issue? Mark From dinov at microsoft.com Wed Sep 23 18:42:09 2009 From: dinov at microsoft.com (Dino Viehland) Date: Wed, 23 Sep 2009 16:42:09 +0000 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com> <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD0493D9E1@TK5EX14MBXC120.redmond.corp.microsoft.com> Mark wrote: > On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland wrote: > > We are going to start contributing tests back real soon now. ?I'm not sure > > that these are the best tests to contribute as they require a version of > > Python to compare against rather than being nice and stand alone. ?But I'm > > sure we have other tests which cover this as well just not as exhaustively. > > We could also possibly check in the baseline file and then CPython could > > compare it's self to previous versions but it'd probably be a pretty > > big file - so it probably shouldn't be included in the standard install > > in the tests directory. > > How big is big? For comparison, CPython's Lib/test/decimaltestdata > directory alone is already over 4Mb, so maybe size isn't an issue? Running all of the side by side tests except our exceptions test it's about 73Mb. It might be highly compressible though. From fuzzyman at voidspace.org.uk Wed Sep 23 18:50:16 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 23 Sep 2009 17:50:16 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0493D9E1@TK5EX14MBXC120.redmond.corp.microsoft.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com> <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD0493D9E1@TK5EX14MBXC120.redmond.corp.microsoft.com> Message-ID: <4ABA51C8.3020407@voidspace.org.uk> Dino Viehland wrote: > Mark wrote: > >> On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland wrote: >> >>> We are going to start contributing tests back real soon now. I'm not sure >>> that these are the best tests to contribute as they require a version of >>> Python to compare against rather than being nice and stand alone. But I'm >>> sure we have other tests which cover this as well just not as exhaustively. >>> We could also possibly check in the baseline file and then CPython could >>> compare it's self to previous versions but it'd probably be a pretty >>> big file - so it probably shouldn't be included in the standard install >>> in the tests directory. >>> >> How big is big? For comparison, CPython's Lib/test/decimaltestdata >> directory alone is already over 4Mb, so maybe size isn't an issue? >> > > Running all of the side by side tests except our exceptions test it's > about 73Mb. It might be highly compressible though. > It sounds like many of these belong in an 'implementation-compliance' test suite separate from the main test suite. On the other hand many of the tests probably cover areas that are really 'sparsely' covered by current Python tests. For many of them getting a 'defined' set of results and changing the tests to compare against that rather than against CPython could be really useful - but probably a lot of work to create. Michael > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From ziade.tarek at gmail.com Wed Sep 23 19:00:20 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 23 Sep 2009 19:00:20 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> On Wed, Sep 23, 2009 at 4:04 PM, Stephen J. Turnbull wrote: > I did offer a concrete criterion for an individual's participation in > a "internal consensus": that you expect that they will adopt the new > features of distutils as a foundation for their own distribution > systems, or at least not implement and promote an alternative. > > As for who needs to be included, if the author of setuptools isn't > part of the internal consensus (on that, I'm just guessing from the > fact that he went off to "start a new thread"), I think you should be > concerned. ?He's already implemented and promoted an alternative in > the past, so he doesn't even have to do any implementation. ?Just keep > on using and promoting his preferred tools and formats. While it's great to have Philipp being part of our distutils design discussions, for his experience, I am not concerned of not having him in this "internal consensus" since Setuptools is not maintained anymore. He said some months ago, he would work on a brand new setuptools version with zero dependency to distutils. But it's still vaporware (from his own words), and the previous version is unmaintained for a year, so it was forked. The Distribute (setuptools fork), which is likely to be the first and only public packaging system on the top of distutils working under Python 3 (the trunk is py3k-ready and should be released in a few days), is pretty active, and none of his contributor raised against the proposal. But you are right about the need of making sure every package management project is involved. We should make sure that Enthought, which has its own package management system, is part of that consensus. Also, I am more concerned of not having enough end users involved in that process. End users would be: any python developer that needs to package his code, or any os packager that needs to package a python distribution for his system. But those are hard to get involved. > Well, from the behavior of Philip and Chris, it seems that their > position is that there was insufficient time to put forward an > alternative design before the summary was posted to Python-Dev. ?*I > don't care whether its true or not*, it's your job as chairman/ > dictator to decide that, and we shouldn't discuss it here. ?But merely > leaving the *impression* is damaging, and I suggested a simple > procedure (posting the summary to your mailing list and requesting > comments) that would very likely improve the summary, and also be > likely to keep unnecessary controversy off Python-Dev. Please note that the controversy that popped in python-dev didn't popped in distutils-SIG after I clearly stated that I was going to send a summary in python-dev, (which I did four days after). No one commented on it then. Next time I'll wait a week and I will also send the summary as you suggested, to make sure everyone sees the message, not hidden inside a big thread. But I doubt this will cut all further controversy once it's in Pyton-dev, because being controversial in Python-dev doesn't have the same impact for the writer and the readers. Tarek From fuzzyman at voidspace.org.uk Wed Sep 23 19:08:31 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 23 Sep 2009 18:08:31 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> Message-ID: <4ABA560F.4040406@voidspace.org.uk> Tarek Ziad? wrote: > On Wed, Sep 23, 2009 at 4:04 PM, Stephen J. Turnbull wrote: > >> I did offer a concrete criterion for an individual's participation in >> a "internal consensus": that you expect that they will adopt the new >> features of distutils as a foundation for their own distribution >> systems, or at least not implement and promote an alternative. >> >> As for who needs to be included, if the author of setuptools isn't >> part of the internal consensus (on that, I'm just guessing from the >> fact that he went off to "start a new thread"), I think you should be >> concerned. He's already implemented and promoted an alternative in >> the past, so he doesn't even have to do any implementation. Just keep >> on using and promoting his preferred tools and formats. >> > > While it's great to have Philipp being part of our distutils design > discussions, > for his experience, I am not concerned of not having him in this "internal > consensus" since Setuptools is not maintained anymore. > > He said some months ago, he would work on a brand new setuptools > version with zero > dependency to distutils. But it's still vaporware (from his own > words), and the previous version is unmaintained for a year, so it was > forked. > > The Distribute (setuptools fork), which is likely to be the first and > only public packaging system > on the top of distutils working under Python 3 (the trunk is > py3k-ready and should be released > in a few days), is pretty active, and none of his contributor raised > against the proposal. > > But you are right about the need of making sure every package management > project is involved. We should make sure that Enthought, > which has its own package management system, is part of that consensus. > > Note that Activestate also have a fledgling package management system for Python (unreleased yet I *believe*) so it is probably worth reaching out to them as well. > Also, I am more concerned of not having enough end users involved in > that process. > End users would be: any python developer that needs > to package his code, or any os packager that needs to package a python > distribution > for his system. But those are hard to get involved. > Perhaps post your summaries on your blog as well (which is aggregated on Planet Python right?) - including a description of the problem it aims to solve and how it will be used. You'll *mostly* be reaching the same set of people, but at least it spreads the net a bit wider. Michael > >> Well, from the behavior of Philip and Chris, it seems that their >> position is that there was insufficient time to put forward an >> alternative design before the summary was posted to Python-Dev. *I >> don't care whether its true or not*, it's your job as chairman/ >> dictator to decide that, and we shouldn't discuss it here. But merely >> leaving the *impression* is damaging, and I suggested a simple >> procedure (posting the summary to your mailing list and requesting >> comments) that would very likely improve the summary, and also be >> likely to keep unnecessary controversy off Python-Dev. >> > > Please note that the controversy that popped in python-dev didn't popped in > distutils-SIG after I clearly stated that I was going to send a > summary in python-dev, > (which I did four days after). No one commented on it then. > > Next time I'll wait a week and I will also send the summary as you suggested, > to make sure everyone sees the message, not hidden inside a big thread. > > But I doubt this will cut all further controversy once it's in > Pyton-dev, because > being controversial in Python-dev doesn't have the same impact for the writer > and the readers. > > Tarek > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From ziade.tarek at gmail.com Wed Sep 23 19:11:18 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 23 Sep 2009 19:11:18 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <4ABA560F.4040406@voidspace.org.uk> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <4ABA560F.4040406@voidspace.org.uk> Message-ID: <94bdd2610909231011n1b81a102scbb5660619ca8697@mail.gmail.com> On Wed, Sep 23, 2009 at 7:08 PM, Michael Foord wrote: > Note that Activestate also have a fledgling package management system for > Python (unreleased yet I *believe*) so it is probably worth reaching out to > them as well. Yes I didn't mention it because the project is private afaik. I'll ask them if it's planned to be released. > Perhaps post your summaries on your blog as well (which is aggregated on > Planet Python right?) - including a description of the problem it aims to > solve and how it will be used. > > You'll *mostly* be reaching the same set of people, but at least it spreads > the net a bit wider. That's right, I'll do it as well. From solipsis at pitrou.net Wed Sep 23 19:15:07 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 23 Sep 2009 17:15:07 +0000 (UTC) Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> Message-ID: Tarek Ziad? gmail.com> writes: > > Also, I am more concerned of not having enough end users involved in > that process. > End users would be: any python developer that needs > to package his code, or any os packager that needs to package a python > distribution > for his system. But those are hard to get involved. I'm sure there are some of them on this list. If you are concerned about opinion diversity, you can always try to probe python-list (c.l.py) about it ;) As far as I'm concerned, anything which looks intuitive enough (e.g. ini-style) and not overly complicated is fine. The details of the syntax aren't really important as long as they make sense, and don't get in the way. Regards Antoine. From a.badger at gmail.com Wed Sep 23 19:49:56 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Wed, 23 Sep 2009 10:49:56 -0700 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> Message-ID: <4ABA5FC4.4080006@gmail.com> On 09/23/2009 10:00 AM, Tarek Ziad? wrote: > But you are right about the need of making sure every package management > project is involved. We should make sure that Enthought, > which has its own package management system, is part of that consensus. > > Also, I am more concerned of not having enough end users involved in > that process. > End users would be: any python developer that needs > to package his code, or any os packager that needs to package a python > distribution > for his system. But those are hard to get involved. > As one of the people who deals with packaging python modules for distributions, I'm sorry for not having spent more time looking into this. I simply haven't had the time lately. One helpful resource for engaging linux distributions is: ##distros on irc.freenode.net and their mailing list: http://lists.freedesktop.org/mailman/listinfo/distributions Both are low traffic but people from the various linux distributions are watching and responding on it even if they don't make much noise on their own. Just bear in mind that sometimes they will need to connect you with the right person within their distro rather than being able to give feedback directly. -Toshio From brett at python.org Wed Sep 23 19:53:50 2009 From: brett at python.org (Brett Cannon) Date: Wed, 23 Sep 2009 10:53:50 -0700 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> Message-ID: On Tue, Sep 22, 2009 at 20:08, R. David Murray wrote: > On Wed, 23 Sep 2009 at 02:01, MRAB wrote: >> >> Dino Viehland wrote: >>> >>> ?Is there a reason or a rule by which CPython reports different error >>> ?message for different failures to subscript? >>> >>> ?For example: >>> >>> > > > ?set()[2] >>> ?Traceback (most recent call last): >>> ? File "", line 1, in >>> ?TypeError: 'set' object does not support indexing >>> > > > ?class c(object): pass >>> ?... > > [....] >>> >>> > > > ?[].append[42] >>> ?Traceback (most recent call last): >>> ? File "", line 1, in >>> ?TypeError: 'builtin_function_or_method' object is unsubscriptable > > [...] >>> >>> ?IronPython had a bug report that we were getting this wrong for set >>> ?objects and that ?does not support indexing? was also a clearer error >>> ?message. ?I?m wondering if there?s some reason why the different error >>> ?messages occur which I?m missing. ? Otherwise I could switch to using >>> the >>> ?more clear message or start marking types which should report the >>> ?unsubscriptable error. ?Does anyone have any insights into this? >>> >> I thought that the difference might be between iterable and non-iterable >> objects, but then I'd expect the class 'c' to behave like the other >> non-iterables. In fact, the result is different again if it's an >> old-style class: >> >>> > > ?class c: pass >>> > > ?c()[2] >> >> Traceback (most recent call last): >> ?File "", line 1, in >> ? c()[2] >> AttributeError: c instance has no attribute '__getitem__' > > Looking at the source, these messages are generated from abstract.c, and > the difference appears to be whether or not the tp_as_sequence slot is > filled in or not. ?If it is, but there is no sq_item method, then > PySequence_GetItem gives the "does not support indexing" message. > If it isn't filled in, you get the 'not subscriptable"(*) message from > PyObject_GetItem. > > The logic appears to be, roughly, if an object does not have mapping > methods, or has them but does not have mp_subscript, and does have > sequence methods but does not have sq_item, then you get a 'does > not support indexing'. ?Otherwise you get 'not subscriptable'. > > The question is, is this a useful distinction, or is it an > implementation artifact? Let's ignore history, which I bet is the reason for the distinction, and just look at the error messages; does the distinction make sense to a newbie? I would say no and that the "does not support indexing" error message is more useful. For expert programmers they could figure out the problem with either error message. The only help is if you are trying to debug a type, but I am willing to bet most of us didn't know the distinction at the C level until David looked it up. So I am +1 on unified the message and +1 on using the "does not support indexing" one. -Brett From brett at python.org Wed Sep 23 19:57:40 2009 From: brett at python.org (Brett Cannon) Date: Wed, 23 Sep 2009 10:57:40 -0700 Subject: [Python-Dev] altruism In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <3e228a5145369e7d9932040c73a7d58e@preisshare.net> Message-ID: On Wed, Sep 23, 2009 at 04:36, Antoine Pitrou wrote: > Brett Cannon python.org> writes: >> >> Trust me, if you are doing open source for >> anything other than altruistic reasons you are bound to be >> disappointed. > > I'm surprised by this statement but perhaps it's a matter of vocabulary. > Having fun and doing things you like to do does not strike me as "altruistic". > Being involved in a FOSS project is not the same as participating in a charity. > > See this quick presentation about motivation of FOSS developers (second slide > says: "Altruism? Not really"): > http://www.infonomics.nl/FLOSS/papers/20030619/index.htm > The full study is at: > http://www.infonomics.nl/FLOSS/report/index.htm > ("Part IV: Survey of Developers") > > (but of course there's the question of whether "altruism" exists at large, or is > a political and moral fiction) Ignoring the temptation to pull out my philosophy degree and devolve into a ridiculously involved conversation, I meant "altruism" in regard to not expecting to get anything from others for what you do. I do think you should only do open source work if you have fun and enjoy it, but I also think you should not do it for anyone but yourself unless you are drawing a paycheck. -Brett From brett at python.org Wed Sep 23 20:03:37 2009 From: brett at python.org (Brett Cannon) Date: Wed, 23 Sep 2009 11:03:37 -0700 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Message-ID: On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson wrote: > Hi everyone, > I've started plotting the release of 2.7. I'd like to try for a final > release mid next summer. 3.2 should be released, if not at the same > time as 2.7, within a few weeks to avoid 2.x having features which 3.x > doesn't. If no one has problems with this, I will draft a schedule. > Yes. Does this mean you are volunteering to be the 3.2 release manager? Or simply prodding for volunteers? If it's the former are you at all worried about burn-out from doing two release so close together and being on the hook for two point releases at roughly the same time for about two years after? > Are we still planning to make 3.3 the main development focus and start > the 5 years of 2.x maintenance after this release? I think so. As Jean-Paul somewhat pointed out the decision can change, but at this point I know I am willing to say that I don't want to have to port to four branches anymore if I don't need to. > > Additionally, I'm very apprehensive about doing any kind of release > without the buildbots running. Does anyone know when they might be up? I don't know the answer, but it might be "never". We used to do releases without them, so it's not impossible. Just means you have to really push the alphas, betas, and RCs. Titus Brown is actually working on a buildbot alternative that's more volunteer-friendly (pony-build). Hopefully that will be up and going by then so we can contemplate using that. -Brett From exarkun at twistedmatrix.com Wed Sep 23 20:24:40 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Wed, 23 Sep 2009 18:24:40 -0000 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Message-ID: <20090923182440.1533.961315222.divmod.xquotient.202@localhost.localdomain> On 06:03 pm, brett at python.org wrote: >On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson >wrote: >[snip] >> >>Additionally, I'm very apprehensive about doing any kind of release >>without the buildbots running. Does anyone know when they might be up? > >I don't know the answer, but it might be "never". We used to do >releases without them, so it's not impossible. Just means you have to >really push the alphas, betas, and RCs. > >Titus Brown is actually working on a buildbot alternative that's more >volunteer-friendly (pony-build). Hopefully that will be up and going >by then so we can contemplate using that. I certainly wish Titus luck in his project, but I'm skeptical about pony-build magically fixing the problems CPython development has with buildbot. Jean-Paul From pje at telecommunity.com Wed Sep 23 22:05:23 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 23 Sep 2009 16:05:23 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com > References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> Message-ID: <20090923200526.007313A407A@sparrow.telecommunity.com> At 02:20 PM 9/23/2009 +0200, Tarek Ziad? wrote: >If you take the time to read everything you'll see that there were >no real alternative design proposed, You did not respond to repeated requests (from more than one person), for clarification regarding the requirements that your proposal was trying to fill. More than one person expressed reservations about the complexity of your candidate proposal, and wanted to know why we needed a fully conditional syntax, if the only use case for conditionals was expressing dependencies. That there was not a "complete" alternative proposal is true... but that's only because you would not answer the question that several people asked: that is, why do we *need* conditionals, for anything besides dependencies? What are the use cases? More than one person asked that question, but I never saw you answer it. (In one case, you answered with the use cases for what the conditionals needed to be able to *check*, but not *what the conditionals were conditioning* -- which was the whole point of the question.) A lack of polished alternatives to your proposal does not constitute a positive rationale for your own proposal, especially if people are asking for the rationale in order to determine whether a simpler proposal would suffice! I dropped my previous competing proposal some time ago when you presented strong use cases for static metadata. And I'd have happily dropped my support for Sridhar's proposal, too, if you'd given similarly strong use cases for the proposal you went with. But even if you *didn't* give those use cases, I'd have been fine with you saying to Python-Dev, "I got tired of the discussion and chose to Pronounce." (After all, that is what you more or less said on the distutils-sig.) But what hacked me off is that *here*, you presented your pronouncement as if it were a summary of distutils-sig discussion, when the last flurry of traffic on the distutils-sig right beforehand was a bunch of questions and requests for use cases. Not nice. From sridharr at activestate.com Wed Sep 23 22:10:16 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Wed, 23 Sep 2009 13:10:16 -0700 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909231011n1b81a102scbb5660619ca8697@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <4ABA560F.4040406@voidspace.org.uk> <94bdd2610909231011n1b81a102scbb5660619ca8697@mail.gmail.com> Message-ID: On Wed, 23 Sep 2009 10:11:18 -0700, Tarek Ziad? wrote: > On Wed, Sep 23, 2009 at 7:08 PM, Michael Foord > wrote: >> Note that Activestate also have a fledgling package management system >> for >> Python (unreleased yet I *believe*) so it is probably worth reaching >> out to >> them as well. > Yes I didn't mention it because the project is private afaik. I'll ask > them if it's planned > to be released. PyPM is planned to be released to the public shortly after the planned 2.6.3 release. Most likely during the first week of October. -srid From benjamin at python.org Wed Sep 23 22:21:12 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 23 Sep 2009 15:21:12 -0500 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Message-ID: <1afaf6160909231321p5ca27b95wec7347bb22cad89a@mail.gmail.com> 2009/9/23 Antoine Pitrou : > Benjamin Peterson python.org> writes: >> >> Hi everyone, >> I've started plotting the release of 2.7. I'd like to try for a final >> release mid next summer. 3.2 should be released, if not at the same >> time as 2.7, within a few weeks to avoid 2.x having features which 3.x >> doesn't. If no one has problems with this, I will draft a schedule. > > If you want to avoid the low-activity low-responsiveness holiday period, you > might shoot for June or September/October instead. That is a good point, so I think we should shoot for June. -- Regards, Benjamin From guido at python.org Wed Sep 23 22:22:31 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Sep 2009 13:22:31 -0700 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> Message-ID: All, I don't have the bandwidth right now to help out much in this thread, so I'm glad that the collective effort has mostly figured out what the rules are (as well as what they should be :-). It truly is a mess, mostly thanks to the various concerns for backwards compatibility (classic vs. new classes *and* __cmp__ vs rich comparisons makes for a truly maddening compatibility matrix). The double __eq__ calls in the pastebin examples are a surprise even to me; it seems the logic for first giving the left object a chance and then the right object is duplicated in the generic rich comparison logic in object.c and again in instance_richcompare() in classobject.c. As Brett cautioned in the bug, I would be hesitant to try and fix this in 2.x; there's simply too much code that relies on whatever odd behavior CPython happens to exhibit, especially in test suites that often over-constrain the behavior (see e.g. the other thread about error messages, where I have nothing to add that Brett didn't already say). In 3.2 I'm fine with the (ever so slight) semantic change, especially since the behavior will rarely be exhibited in practice. (You'd have to have two different objects with __eq__ methods that return NotImplemented *and* have some kind of side effect that is expected.) --Guido On Tue, Sep 22, 2009 at 8:12 AM, Mark Dickinson wrote: > On Tue, Sep 22, 2009 at 3:37 PM, Chris Withers wrote: >> Where are the specifications on what happens if two objects are compared and both have implementations of __eq__? Which __eq__ is called? >> What happens if the first one called returns False? Is the second one called? What is one implements __eq__ and the other __ne__? > > I (still :-) think this is covered, for Python 2.x at least, by: > > http://docs.python.org/reference/datamodel.html#coercion-rules > > Specifically, the bits that say: > > - For objects x and y, first x.__op__(y) is tried. If this is not > implemented or returns NotImplemented, y.__rop__(x) is tried. If this > is also not implemented or returns NotImplemented, a TypeError > exception is raised. But see the following exception: > > - Exception to the previous item: if the left operand is an instance > of a built-in type or a new-style class, and the right operand is an > instance of a proper subclass of that type or class and overrides the > base?s __rop__() method, the right operand?s __rop__() method is tried > before the left operand?s __op__() method. > > I agree that having these rules in a section called 'Coercion rules' > is a bit confusing. > >> Python 2 >> http://pastebin.com/f8f19ab3 >> >> Python 3 >> http://pastebin.com/f55e44630 > > The duplicate __eq__ calls in these pastes are disturbing: ?in short, > if A() and B() are new-style classes defining __eq__, it seems that > A() == B() ends up calling A.__eq__ twice *and* B.__eq__ twice, in the > order A.__eq__, B.__eq__, B.__eq__, A.__eq__. > > In 3.x, slot_tp_richcompare (in typeobject.c) makes two calls to > half_richcompare; ?I think the second is redundant. ?The coercion > rules are already taken care of in do_richcompare (in object.c). ?I > tried removing the second call to half_richcompare, and the entire > test-suite still runs without errors. > > In 2.x, it's possible that this call is necessary for some bizarre > combinations of __cmp__ and __eq__; ?I haven't tried to get my head > around this yet. > > I'll open an issue for the duplicate __eq__ calls. > > Mark > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From benjamin at python.org Wed Sep 23 22:34:10 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 23 Sep 2009 15:34:10 -0500 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Message-ID: <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> 2009/9/23 Brett Cannon : > On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson wrote: >> Hi everyone, >> I've started plotting the release of 2.7. I'd like to try for a final >> release mid next summer. 3.2 should be released, if not at the same >> time as 2.7, within a few weeks to avoid 2.x having features which 3.x >> doesn't. If no one has problems with this, I will draft a schedule. >> > > Yes. Does this mean you are volunteering to be the 3.2 release > manager? Or simply prodding for volunteers? If it's the former are you > at all worried about burn-out from doing two release so close together > and being on the hook for two point releases at roughly the same time > for about two years after? While I certainly wouldn't mind doing only 2.7, I don't know how we could preserve everyone's sanity and release them at about the same time. Different RMs would have different times they can do releases, so I would worry about there being a release in a slightly different stage of a different branch every couple weeks. As for burn out, I expected 2.7.x, as the last 2.x release, to be different in that several people would do the maintenance releases (perhaps on a 6 month schedule or so) for the 5 year period, so that would leave me with just 3.2.x (and maybe another 3.1.x release). ... >> >> Additionally, I'm very apprehensive about doing any kind of release >> without the buildbots running. Does anyone know when they might be up? > > I don't know the answer, but it might be "never". We used to do > releases without them, so it's not impossible. Just means you have to > really push the alphas, betas, and RCs. What do you mean "push"? -- Regards, Benjamin From pje at telecommunity.com Wed Sep 23 22:47:34 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 23 Sep 2009 16:47:34 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.co m> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> Message-ID: <20090923204737.A97DC3A4079@sparrow.telecommunity.com> At 07:00 PM 9/23/2009 +0200, Tarek Ziad? wrote: >While it's great to have Philipp being part of our distutils design >discussions, >for his experience, I am not concerned of not having him in this "internal >consensus" since Setuptools is not maintained anymore. > >He said some months ago, he would work on a brand new setuptools >version with zero >dependency to distutils. But it's still vaporware (from his own >words), and the previous version is unmaintained for a year, so it was >forked. Here's what actually happened, if anyone cares. Tarek and friends announced a fork of setuptools. I reviewed the work and saw that -- for the most part -- I was happy with it, and opined as how I might be willing to bless the the "package inquisition" team as official maintainers of the 0.6 branch of setuptools, so that I could work on the fun bits I've long planned for 0.7, but never felt free to start on while there was so much still needing to be done on 0.6. However, just as I mentioned this, and suggested an option for what I could do that would be helpful to his Distribute 0.7 project as well as various other tools (e.g. implementing some of Jim Fulton's long-requested features for better modularization of setuptools), Tarek accused me of somehow trying to undermine his plans. In addition, it appears Tarek was also offended by my earlier statement that there were only a few people in the Python community who had *already* earned my implicit trust to not only hack on setuptools unsupervised, but also to take over its *future* direction and BDFL-ship. (For example, Jim Fulton and Ian Bicking.) Tarek, however, appears to have taken this to mean that I personally thought he was an incompetent programmer or something (when I actually had no opinion one way or the other), and ever since he has taken to levelling potshots like the above at me on a semi-regular basis. I've tried to ignore this and play nice, because he is actually working on this stuff and I am not. But it's hard for me to actually give any help in practice, if Tarek is too busy projecting hidden plots onto everything I say and do. If you read Tarek's distutils-sig posts, it appears my already-existing trust in Ian and Jim was not only a personal insult to Tarek, but also a plot to ensure that nobody with any time to do so would ever work on setuptools, just as my excitement about working on setuptools again was a plot to steal thunder from his fork. All I want is for good stuff to happen for setuptools users and Python users in general, so I don't think all the suspicion and backbiting is merited. I certainly don't appreciate it, and I would like it to stop. It also isn't even relevant to the thread, since my lack of work on setuptools says exactly zero about the merits or lack thereof of Tarek's proposals for the distutils! Hell, I *support* the bulk of Tarek's setup.cfg proposal, and don't even object to him Pronouncing it or cutting off the discussion! My only issue on Python-Dev was his inaccurate implication that it was a SIG consensus rather than a pronouncement on it. There is and was no need for any of this to get personal, and I have continually strived to keep my posts here and distutils-sig civil, even when I didn't feel like being civil in response to Tarek's jabs. I have in fact bent over backwards to be *nice* to Tarek, because he seemed so damn sensitive about everything. Apparently, however, this does not actually help things. :-( From dinov at microsoft.com Wed Sep 23 22:47:41 2009 From: dinov at microsoft.com (Dino Viehland) Date: Wed, 23 Sep 2009 20:47:41 +0000 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> Message-ID: <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> Brett wrote: > Let's ignore history, which I bet is the reason for the distinction, > and just look at the error messages; does the distinction make sense > to a newbie? I would say no and that the "does not support indexing" > error message is more useful. For expert programmers they could figure > out the problem with either error message. The only help is if you are > trying to debug a type, but I am willing to bet most of us didn't know > the distinction at the C level until David looked it up. > > So I am +1 on unified the message and +1 on using the "does not > support indexing" one. I'd be +1 on the unified message as well - but it seems what that message should be may be contentious (and quite a bike shed discussion at that). The bug David linked to (http://bugs.python.org/issue5760) has a preference for subscript because that's what's used elsewhere in the language. From mal at egenix.com Wed Sep 23 23:17:16 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Wed, 23 Sep 2009 23:17:16 +0200 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> Message-ID: <4ABA905C.2040807@egenix.com> Brett Cannon wrote: > On Tue, Sep 22, 2009 at 20:08, R. David Murray wrote: >> On Wed, 23 Sep 2009 at 02:01, MRAB wrote: >>> >>> Dino Viehland wrote: >>>> >>>> Is there a reason or a rule by which CPython reports different error >>>> message for different failures to subscript? >>>> >>>> For example: >>>> >>>>>>> set()[2] >>>> Traceback (most recent call last): >>>> File "", line 1, in >>>> TypeError: 'set' object does not support indexing >>>>>>> class c(object): pass >>>> ... >> >> [....] >>>> >>>>>>> [].append[42] >>>> Traceback (most recent call last): >>>> File "", line 1, in >>>> TypeError: 'builtin_function_or_method' object is unsubscriptable >> >> [...] >>>> >>>> IronPython had a bug report that we were getting this wrong for set >>>> objects and that ?does not support indexing? was also a clearer error >>>> message. I?m wondering if there?s some reason why the different error >>>> messages occur which I?m missing. Otherwise I could switch to using >>>> the >>>> more clear message or start marking types which should report the >>>> unsubscriptable error. Does anyone have any insights into this? >>>> >>> I thought that the difference might be between iterable and non-iterable >>> objects, but then I'd expect the class 'c' to behave like the other >>> non-iterables. In fact, the result is different again if it's an >>> old-style class: >>> >>>>>> class c: pass >>>>>> c()[2] >>> >>> Traceback (most recent call last): >>> File "", line 1, in >>> c()[2] >>> AttributeError: c instance has no attribute '__getitem__' >> >> Looking at the source, these messages are generated from abstract.c, and >> the difference appears to be whether or not the tp_as_sequence slot is >> filled in or not. If it is, but there is no sq_item method, then >> PySequence_GetItem gives the "does not support indexing" message. >> If it isn't filled in, you get the 'not subscriptable"(*) message from >> PyObject_GetItem. >> >> The logic appears to be, roughly, if an object does not have mapping >> methods, or has them but does not have mp_subscript, and does have >> sequence methods but does not have sq_item, then you get a 'does >> not support indexing'. Otherwise you get 'not subscriptable'. >> >> The question is, is this a useful distinction, or is it an >> implementation artifact? > > Let's ignore history, which I bet is the reason for the distinction, > and just look at the error messages; does the distinction make sense > to a newbie? I would say no and that the "does not support indexing" > error message is more useful. For expert programmers they could figure > out the problem with either error message. The only help is if you are > trying to debug a type, but I am willing to bet most of us didn't know > the distinction at the C level until David looked it up. > > So I am +1 on unified the message and +1 on using the "does not > support indexing" one. +1 as well. Note that the wording of error message has never been something we guaranteed backwards compatibility for, so such changes are a lot easier to implement than API changes. (I just hope that doctest doesn't rely on the message texts.) -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 23 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From fuzzyman at voidspace.org.uk Wed Sep 23 23:19:00 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 23 Sep 2009 22:19:00 +0100 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <4ABA905C.2040807@egenix.com> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> <4ABA905C.2040807@egenix.com> Message-ID: <4ABA90C4.2040401@voidspace.org.uk> M.-A. Lemburg wrote: > Brett Cannon wrote: > >> On Tue, Sep 22, 2009 at 20:08, R. David Murray wrote: >> >>> On Wed, 23 Sep 2009 at 02:01, MRAB wrote: >>> >>>> Dino Viehland wrote: >>>> >>>>> Is there a reason or a rule by which CPython reports different error >>>>> message for different failures to subscript? >>>>> >>>>> For example: >>>>> >>>>> >>>>>>>> set()[2] >>>>>>>> >>>>> Traceback (most recent call last): >>>>> File "", line 1, in >>>>> TypeError: 'set' object does not support indexing >>>>> >>>>>>>> class c(object): pass >>>>>>>> >>>>> ... >>>>> >>> [....] >>> >>>>>>>> [].append[42] >>>>>>>> >>>>> Traceback (most recent call last): >>>>> File "", line 1, in >>>>> TypeError: 'builtin_function_or_method' object is unsubscriptable >>>>> >>> [...] >>> >>>>> IronPython had a bug report that we were getting this wrong for set >>>>> objects and that ?does not support indexing? was also a clearer error >>>>> message. I?m wondering if there?s some reason why the different error >>>>> messages occur which I?m missing. Otherwise I could switch to using >>>>> the >>>>> more clear message or start marking types which should report the >>>>> unsubscriptable error. Does anyone have any insights into this? >>>>> >>>>> >>>> I thought that the difference might be between iterable and non-iterable >>>> objects, but then I'd expect the class 'c' to behave like the other >>>> non-iterables. In fact, the result is different again if it's an >>>> old-style class: >>>> >>>> >>>>>>> class c: pass >>>>>>> c()[2] >>>>>>> >>>> Traceback (most recent call last): >>>> File "", line 1, in >>>> c()[2] >>>> AttributeError: c instance has no attribute '__getitem__' >>>> >>> Looking at the source, these messages are generated from abstract.c, and >>> the difference appears to be whether or not the tp_as_sequence slot is >>> filled in or not. If it is, but there is no sq_item method, then >>> PySequence_GetItem gives the "does not support indexing" message. >>> If it isn't filled in, you get the 'not subscriptable"(*) message from >>> PyObject_GetItem. >>> >>> The logic appears to be, roughly, if an object does not have mapping >>> methods, or has them but does not have mp_subscript, and does have >>> sequence methods but does not have sq_item, then you get a 'does >>> not support indexing'. Otherwise you get 'not subscriptable'. >>> >>> The question is, is this a useful distinction, or is it an >>> implementation artifact? >>> >> Let's ignore history, which I bet is the reason for the distinction, >> and just look at the error messages; does the distinction make sense >> to a newbie? I would say no and that the "does not support indexing" >> error message is more useful. For expert programmers they could figure >> out the problem with either error message. The only help is if you are >> trying to debug a type, but I am willing to bet most of us didn't know >> the distinction at the C level until David looked it up. >> >> So I am +1 on unified the message and +1 on using the "does not >> support indexing" one. >> > > +1 as well. > Also +1. I had a friend (an experienced programmer) who started using Python recently. The cryptic nature of some of the error messages was a sore point with him. Michael > Note that the wording of error message has never been something > we guaranteed backwards compatibility for, so such changes are a > lot easier to implement than API changes. > > (I just hope that doctest doesn't rely on the message texts.) > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From fuzzyman at voidspace.org.uk Wed Sep 23 23:19:31 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 23 Sep 2009 22:19:31 +0100 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> Message-ID: <4ABA90E3.60200@voidspace.org.uk> Benjamin Peterson wrote: > 2009/9/23 Brett Cannon : > >> On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson wrote: >> >>> Hi everyone, >>> I've started plotting the release of 2.7. I'd like to try for a final >>> release mid next summer. 3.2 should be released, if not at the same >>> time as 2.7, within a few weeks to avoid 2.x having features which 3.x >>> doesn't. If no one has problems with this, I will draft a schedule. >>> >>> >> Yes. Does this mean you are volunteering to be the 3.2 release >> manager? Or simply prodding for volunteers? If it's the former are you >> at all worried about burn-out from doing two release so close together >> and being on the hook for two point releases at roughly the same time >> for about two years after? >> > > While I certainly wouldn't mind doing only 2.7, I don't know how we > could preserve everyone's sanity and release them at about the same > time. Different RMs would have different times they can do releases, > so I would worry about there being a release in a slightly different > stage of a different branch every couple weeks. > > As for burn out, I expected 2.7.x, as the last 2.x release, Are we definitely decided that 2.7 will be the last major release in the 2.x cycle? Michael > to be > different in that several people would do the maintenance releases > (perhaps on a 6 month schedule or so) for the 5 year period, so that > would leave me with just 3.2.x (and maybe another 3.1.x release). > > ... > >>> Additionally, I'm very apprehensive about doing any kind of release >>> without the buildbots running. Does anyone know when they might be up? >>> >> I don't know the answer, but it might be "never". We used to do >> releases without them, so it's not impossible. Just means you have to >> really push the alphas, betas, and RCs. >> > > What do you mean "push"? > > > > > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From solipsis at pitrou.net Wed Sep 23 23:26:28 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 23 Sep 2009 21:26:28 +0000 (UTC) Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.co m> <20090923204737.A97DC3A4079@sparrow.telecommunity.com> Message-ID: P.J. Eby telecommunity.com> writes: > > Hell, I *support* the bulk of Tarek's setup.cfg proposal, and don't > even object to him Pronouncing it or cutting off the discussion! My > only issue on Python-Dev was his inaccurate implication that it was a > SIG consensus rather than a pronouncement on it. There is and was no > need for any of this to get personal, and I have continually strived > to keep my posts here and distutils-sig civil, even when I didn't > feel like being civil in response to Tarek's jabs. I have in fact > bent over backwards to be *nice* to Tarek, because he seemed so damn > sensitive about everything. Apparently, however, this does not > actually help things. Ok, so Tarek and Philip, are you both ok that those little disagreements should belong to the past now? :) Philip did an important job in bringing setuptools to the Python community, and Tarek took a difficult decision when he finally decided to fork setuptools (he had probably been privately encouraged for months by people like me, and was still reluctant to do it AFAICT). Both of you have been tremendously helpful to Python, and I'm sure we can move forward graciously. Regards Antoine. From solipsis at pitrou.net Wed Sep 23 23:28:04 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 23 Sep 2009 21:28:04 +0000 (UTC) Subject: [Python-Dev] thinking about 2.7 References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> Message-ID: Benjamin Peterson python.org> writes: > > Different RMs would have different times they can do releases, > so I would worry about there being a release in a slightly different > stage of a different branch every couple weeks. Assuming you can do it, +1 for you (Benjamin) being RM for both 2.7 and 3.1. Indeed, both branches need to be synchronized anyway and the best way to achieve this is for a single person to overview both of them. Regards Antoine. From benjamin at python.org Wed Sep 23 23:34:17 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 23 Sep 2009 16:34:17 -0500 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <4ABA90E3.60200@voidspace.org.uk> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <4ABA90E3.60200@voidspace.org.uk> Message-ID: <1afaf6160909231434t6922d30eu5eb0b894617b0340@mail.gmail.com> 2009/9/23 Michael Foord : > Benjamin Peterson wrote: >> As for burn out, I expected 2.7.x, as the last 2.x release, > > Are we definitely decided that 2.7 will be the last major release in the 2.x > cycle? No, but that's what we're planning for atm. -- Regards, Benjamin From brett at python.org Wed Sep 23 23:26:30 2009 From: brett at python.org (Brett Cannon) Date: Wed, 23 Sep 2009 14:26:30 -0700 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <4ABA90C4.2040401@voidspace.org.uk> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> <4ABA905C.2040807@egenix.com> <4ABA90C4.2040401@voidspace.org.uk> Message-ID: On Wed, Sep 23, 2009 at 14:19, Michael Foord wrote: [SNIP] > Also +1. I had a friend (an experienced programmer) who started using Python > recently. The cryptic nature of some of the error messages was a sore point > with him. Do you know which error messages? We can change them. We have always said that we view exception messages as something that can change in minor releases and that you should be very careful if you ever use them in tests. -Brett From brett at python.org Wed Sep 23 23:12:12 2009 From: brett at python.org (Brett Cannon) Date: Wed, 23 Sep 2009 14:12:12 -0700 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> Message-ID: On Wed, Sep 23, 2009 at 13:34, Benjamin Peterson wrote: > 2009/9/23 Brett Cannon : >> On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson wrote: >>> Hi everyone, >>> I've started plotting the release of 2.7. I'd like to try for a final >>> release mid next summer. 3.2 should be released, if not at the same >>> time as 2.7, within a few weeks to avoid 2.x having features which 3.x >>> doesn't. If no one has problems with this, I will draft a schedule. >>> >> >> Yes. Does this mean you are volunteering to be the 3.2 release >> manager? Or simply prodding for volunteers? If it's the former are you >> at all worried about burn-out from doing two release so close together >> and being on the hook for two point releases at roughly the same time >> for about two years after? > > While I certainly wouldn't mind doing only 2.7, I don't know how we > could preserve everyone's sanity and release them at about the same > time. Different RMs would have different times they can do releases, > so I would worry about there being a release in a slightly different > stage of a different branch every couple weeks. > If you are okay with it then that's fine. > As for burn out, I expected 2.7.x, as the last 2.x release, to be > different in that several people would do the maintenance releases > (perhaps on a 6 month schedule or so) for the 5 year period, so that > would leave me with just 3.2.x (and maybe another 3.1.x release). Well, if you want that to happen you should make that clear now as I expected you to do the first few micro releases of 2.7. > > ... >>> >>> Additionally, I'm very apprehensive about doing any kind of release >>> without the buildbots running. Does anyone know when they might be up? >> >> I don't know the answer, but it might be "never". We used to do >> releases without them, so it's not impossible. Just means you have to >> really push the alphas, betas, and RCs. > > What do you mean "push"? Promote. -Brett From solipsis at pitrou.net Wed Sep 23 23:36:30 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 23 Sep 2009 21:36:30 +0000 (UTC) Subject: [Python-Dev] thinking about 2.7 References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <4ABA90E3.60200@voidspace.org.uk> Message-ID: Michael Foord voidspace.org.uk> writes: > > Are we definitely decided that 2.7 will be the last major release in the > 2.x cycle? I don't think any "definitive" decision was made, but judgeing by Benjamin's and Brett's answers (and by my own sentiment :-)), it certainly is the expectation of some core developers that we focus primarily on 3.x after 2.7 is released. Of course, if a large enough number of core developers want to backport stuff to 2.x, they probably can arrange to do so. Regards Antoine. From brett at python.org Wed Sep 23 23:24:53 2009 From: brett at python.org (Brett Cannon) Date: Wed, 23 Sep 2009 14:24:53 -0700 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> Message-ID: On Wed, Sep 23, 2009 at 13:47, Dino Viehland wrote: > Brett wrote: >> Let's ignore history, which I bet is the reason for the distinction, >> and just look at the error messages; does the distinction make sense >> to a newbie? I would say no and that the "does not support indexing" >> error message is more useful. For expert programmers they could figure >> out the problem with either error message. The only help is if you are >> trying to debug a type, but I am willing to bet most of us didn't know >> the distinction at the C level until David looked it up. >> >> So I am +1 on unified the message and +1 on using the "does not >> support indexing" one. > > I'd be +1 on the unified message as well - but it seems what that message > should be may be contentious (and quite a bike shed discussion at that). > The bug David linked to (http://bugs.python.org/issue5760) has a > preference for subscript because that's what's used elsewhere in the > language. That's what's used in the language spec which VERY few people read. That can use more technical jargon, but I still don't see experienced programmers having issuesvfiguring out that note being able to index an object is the same as not being subscriptable. And we can change the docs as well if needed, but I don't think that is necessary. -Brett From benjamin at python.org Wed Sep 23 23:42:05 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 23 Sep 2009 16:42:05 -0500 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> Message-ID: <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> 2009/9/23 Brett Cannon : > On Wed, Sep 23, 2009 at 13:34, Benjamin Peterson wrote: >> As for burn out, I expected 2.7.x, as the last 2.x release, to be >> different in that several people would do the maintenance releases >> (perhaps on a 6 month schedule or so) for the 5 year period, so that >> would leave me with just 3.2.x (and maybe another 3.1.x release). > > Well, if you want that to happen you should make that clear now as I > expected you to do the first few micro releases of 2.7. I will do the first few 2.7 bug fix releases. >>>> Additionally, I'm very apprehensive about doing any kind of release >>>> without the buildbots running. Does anyone know when they might be up? >>> >>> I don't know the answer, but it might be "never". We used to do >>> releases without them, so it's not impossible. Just means you have to >>> really push the alphas, betas, and RCs. >> >> What do you mean "push"? > > Promote. That basically means we will be relying on 3rd party libraries for our quality control? -- Regards, Benjamin From fuzzyman at voidspace.org.uk Wed Sep 23 23:49:38 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 23 Sep 2009 22:49:38 +0100 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> <4ABA905C.2040807@egenix.com> <4ABA90C4.2040401@voidspace.org.uk> Message-ID: <4ABA97F2.6040803@voidspace.org.uk> Brett Cannon wrote: > On Wed, Sep 23, 2009 at 14:19, Michael Foord wrote: > [SNIP] > >> Also +1. I had a friend (an experienced programmer) who started using Python >> recently. The cryptic nature of some of the error messages was a sore point >> with him. >> > > Do you know which error messages? We can change them. We have always > said that we view exception messages as something that can change in > minor releases and that you should be very careful if you ever use > them in tests. > There were a couple and I can only remember the last one he talked about. He ran some code with "python -3" that used someone else's library and resulted in the following warning, that he felt was cryptic: DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in 3.x Hmm... not entirely sure how we can make it clearer. There is a great blog entry on the subject by Eric Lippert though (Microsoft I'm afraid): http://blogs.msdn.com/ericlippert/archive/2006/07/07/659259.aspx Error messages: diagnostic is preferable to prescriptive * Polite: making the user feel like an idiot is very, very bad. * Readable: poor grammar and tortured sentence structure is bad. * Accurate: error messages must accurately describe the problem. * Precise: "Something is wrong" is an accurate error message but not a very precise one! * Diagnostic but not prescriptive: describe the problem, not the solution. How about (?): If you implement equality (__eq__) you should also implement hashing (__hash__), it won't be inherited in 3.x. Although that is prescriptive of course... Michael > -Brett > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From brett at python.org Wed Sep 23 23:27:40 2009 From: brett at python.org (Brett Cannon) Date: Wed, 23 Sep 2009 14:27:40 -0700 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <4ABA90E3.60200@voidspace.org.uk> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <4ABA90E3.60200@voidspace.org.uk> Message-ID: On Wed, Sep 23, 2009 at 14:19, Michael Foord wrote: > Benjamin Peterson wrote: >> >> 2009/9/23 Brett Cannon : >> >>> >>> On Wed, Sep 23, 2009 at 07:35, Benjamin Peterson >>> wrote: >>> >>>> >>>> Hi everyone, >>>> I've started plotting the release of 2.7. I'd like to try for a final >>>> release mid next summer. 3.2 should be released, if not at the same >>>> time as 2.7, within a few weeks to avoid 2.x having features which 3.x >>>> doesn't. If no one has problems with this, I will draft a schedule. >>>> >>>> >>> >>> Yes. Does this mean you are volunteering to be the 3.2 release >>> manager? Or simply prodding for volunteers? If it's the former are you >>> at all worried about burn-out from doing two release so close together >>> and being on the hook for two point releases at roughly the same time >>> for about two years after? >>> >> >> While I certainly wouldn't mind doing only 2.7, I don't know how we >> could preserve everyone's sanity and release them at about the same >> time. Different RMs would have different times they can do releases, >> so I would worry about there being a release in a slightly different >> stage of a different branch every couple weeks. >> >> As for burn out, I expected 2.7.x, as the last 2.x release, > > Are we definitely decided that 2.7 will be the last major release in the 2.x > cycle? I would say we won't be sure until we are closer to release, but I would be willing to put money down that it will be. -Brett From fuzzyman at voidspace.org.uk Wed Sep 23 23:52:38 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 23 Sep 2009 22:52:38 +0100 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> Message-ID: <4ABA98A6.7060803@voidspace.org.uk> Benjamin Peterson wrote: > 2009/9/23 Brett Cannon : > >> On Wed, Sep 23, 2009 at 13:34, Benjamin Peterson wrote: >> >>> As for burn out, I expected 2.7.x, as the last 2.x release, to be >>> different in that several people would do the maintenance releases >>> (perhaps on a 6 month schedule or so) for the 5 year period, so that >>> would leave me with just 3.2.x (and maybe another 3.1.x release). >>> >> Well, if you want that to happen you should make that clear now as I >> expected you to do the first few micro releases of 2.7. >> > > I will do the first few 2.7 bug fix releases. > > >>>>> Additionally, I'm very apprehensive about doing any kind of release >>>>> without the buildbots running. Does anyone know when they might be up? >>>>> >>>> I don't know the answer, but it might be "never". We used to do >>>> releases without them, so it's not impossible. Just means you have to >>>> really push the alphas, betas, and RCs. >>>> >>> What do you mean "push"? >>> >> Promote. >> > > That basically means we will be relying on 3rd party libraries for our > quality control? > > > Isn't that the real compatibility test *anyway* - how successful a new version of Python is at running all the existing Python code... Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From brett at python.org Thu Sep 24 00:00:07 2009 From: brett at python.org (Brett Cannon) Date: Wed, 23 Sep 2009 15:00:07 -0700 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <4ABA97F2.6040803@voidspace.org.uk> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> <4ABA905C.2040807@egenix.com> <4ABA90C4.2040401@voidspace.org.uk> <4ABA97F2.6040803@voidspace.org.uk> Message-ID: On Wed, Sep 23, 2009 at 14:49, Michael Foord wrote: > Brett Cannon wrote: >> >> On Wed, Sep 23, 2009 at 14:19, Michael Foord >> wrote: >> [SNIP] >> >>> >>> Also +1. I had a friend (an experienced programmer) who started using >>> Python >>> recently. The cryptic nature of some of the error messages was a sore >>> point >>> with him. >>> >> >> Do you know which error messages? We can change them. We have always >> said that we view exception messages as something that can change in >> minor releases and that you should be very careful if you ever use >> them in tests. >> > > There were a couple and I can only remember the last one he talked about. He > ran some code with "python -3" that used someone else's library and resulted > in the following warning, that he felt was cryptic: > > ? DeprecationWarning: Overriding __eq__ blocks inheritance of __hash__ in > 3.x > > Hmm... not entirely sure how we can make it clearer. There is a great blog > entry on the subject by Eric Lippert though (Microsoft I'm afraid): > > ? http://blogs.msdn.com/ericlippert/archive/2006/07/07/659259.aspx > > Error messages: diagnostic is preferable to prescriptive > > ? * Polite: making the user feel like an idiot is very, very bad. > ? * Readable: poor grammar and tortured sentence structure is bad. > ? * Accurate: error messages must accurately describe the problem. > ? * Precise: "Something is wrong" is an accurate error message but not a > very precise one! > ? * Diagnostic but not prescriptive: describe the problem, not the solution. > > How about (?): > > ? If you implement equality (__eq__) you should also implement hashing > (__hash__), it won't be inherited in 3.x. > > Although that is prescriptive of course... So the original makes sense if you are implementing an object's comparison operator. But if you are not it won't make much sense, although if you are not implementing it then it shouldn't be your concern. Then again, if you are trying to upgrade your code and some library you used has not, I can see how that message would throw you for a loop. So +0 on your specific improvment and +1 for trying to think about newbies when writing exception messages. -Brett From ziade.tarek at gmail.com Thu Sep 24 00:40:30 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 24 Sep 2009 00:40:30 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <20090923204737.A97DC3A4079@sparrow.telecommunity.com> <94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com> <94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com> Message-ID: <94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com> (Sorry if it's top posting I am on a mobile) Come on phillip, no one is "plotting" against you. You didn't maintain setuptools for a year while people where begging you to do bug fixes. You blessed Ian and Jim to take over but they are too busy to do it . I even sent them a mail on my side to try to convince them. So we asked you to bless someone else that was active (not in particular me as your mail seem to say) but you did not. So we forked. And people were pissed off at you. (Which I am sorry about) If a project is not maintained and if the maintainer does not open it to other maintainers, that s the way to go . And the fact that I took the lead of that fork doesn't mean I am offended because you did not bless me to maintain setuptools. It just means that I want to move forward and have a working tool for python 3. So let me make it clear that when you say "Tarek appears" it is something I have never said but rather something you are thinking like being the truth. Now for the Distribute work , your patches are very welcome. It s a community project. On Sep 23, 2009 10:47 PM, "P.J. Eby" wrote: At 07:00 PM 9/23/2009 +0200, Tarek Ziad? wrote: > > While it's great to have Philipp being part of o... Here's what actually happened, if anyone cares. Tarek and friends announced a fork of setuptools. I reviewed the work and saw that -- for the most part -- I was happy with it, and opined as how I might be willing to bless the the "package inquisition" team as official maintainers of the 0.6 branch of setuptools, so that I could work on the fun bits I've long planned for 0.7, but never felt free to start on while there was so much still needing to be done on 0.6. However, just as I mentioned this, and suggested an option for what I could do that would be helpful to his Distribute 0.7 project as well as various other tools (e.g. implementing some of Jim Fulton's long-requested features for better modularization of setuptools), Tarek accused me of somehow trying to undermine his plans. In addition, it appears Tarek was also offended by my earlier statement that there were only a few people in the Python community who had *already* earned my implicit trust to not only hack on setuptools unsupervised, but also to take over its *future* direction and BDFL-ship. (For example, Jim Fulton and Ian Bicking.) Tarek, however, appears to have taken this to mean that I personally thought he was an incompetent programmer or something (when I actually had no opinion one way or the other), and ever since he has taken to levelling potshots like the above at me on a semi-regular basis. I've tried to ignore this and play nice, because he is actually working on this stuff and I am not. But it's hard for me to actually give any help in practice, if Tarek is too busy projecting hidden plots onto everything I say and do. If you read Tarek's distutils-sig posts, it appears my already-existing trust in Ian and Jim was not only a personal insult to Tarek, but also a plot to ensure that nobody with any time to do so would ever work on setuptools, just as my excitement about working on setuptools again was a plot to steal thunder from his fork. All I want is for good stuff to happen for setuptools users and Python users in general, so I don't think all the suspicion and backbiting is merited. I certainly don't appreciate it, and I would like it to stop. It also isn't even relevant to the thread, since my lack of work on setuptools says exactly zero about the merits or lack thereof of Tarek's proposals for the distutils! Hell, I *support* the bulk of Tarek's setup.cfg proposal, and don't even object to him Pronouncing it or cutting off the discussion! My only issue on Python-Dev was his inaccurate implication that it was a SIG consensus rather than a pronouncement on it. There is and was no need for any of this to get personal, and I have continually strived to keep my posts here and distutils-sig civil, even when I didn't feel like being civil in response to Tarek's jabs. I have in fact bent over backwards to be *nice* to Tarek, because he seemed so damn sensitive about everything. Apparently, however, this does not actually help things. :-( -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Thu Sep 24 00:43:59 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Thu, 24 Sep 2009 00:43:59 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <20090923204737.A97DC3A4079@sparrow.telecommunity.com> Message-ID: <94bdd2610909231543u16827048k98a773af43c993bf@mail.gmail.com> Fine with me. Let's move forward. On Sep 23, 2009 11:27 PM, "Antoine Pitrou" wrote: P.J. Eby telecommunity.com> writes: > > Hell, I *support* the bulk of Tarek's setup.cfg p... Ok, so Tarek and Philip, are you both ok that those little disagreements should belong to the past now? :) Philip did an important job in bringing setuptools to the Python community, and Tarek took a difficult decision when he finally decided to fork setuptools (he had probably been privately encouraged for months by people like me, and was still reluctant to do it AFAICT). Both of you have been tremendously helpful to Python, and I'm sure we can move forward graciously. Regards Antoine. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http... -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Thu Sep 24 01:02:37 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 23 Sep 2009 16:02:37 -0700 Subject: [Python-Dev] thinking about 2.7 References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <4ABA90E3.60200@voidspace.org.uk> Message-ID: <85E6E4B220224880B4AFD8A105D39832@RaymondLaptop1> [Michael Foord] > Are we definitely decided that 2.7 will be the last major release in the > 2.x cycle? ISTM, that would depend on the uptake for 3.2. The users get a say in the matter. Raymond From fuzzyman at voidspace.org.uk Thu Sep 24 01:05:35 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 24 Sep 2009 00:05:35 +0100 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <85E6E4B220224880B4AFD8A105D39832@RaymondLaptop1> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <4ABA90E3.60200@voidspace.org.uk> <85E6E4B220224880B4AFD8A105D39832@RaymondLaptop1> Message-ID: <4ABAA9BF.7040609@voidspace.org.uk> Raymond Hettinger wrote: > > [Michael Foord] >> Are we definitely decided that 2.7 will be the last major release in >> the 2.x cycle? > > ISTM, that would depend on the uptake for 3.2. > The users get a say in the matter. > That would be my feeling... Michael > > Raymond -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From benjamin at python.org Thu Sep 24 01:17:17 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 23 Sep 2009 18:17:17 -0500 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <4ABA98A6.7060803@voidspace.org.uk> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> <4ABA98A6.7060803@voidspace.org.uk> Message-ID: <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com> 2009/9/23 Michael Foord : > Isn't that the real compatibility test *anyway* - how successful a new > version of Python is at running all the existing Python code... Yes, but we should have expect 3rd party code to be detecting bugs for us that our test suite could have shown on a platform. -- Regards, Benjamin From fuzzyman at voidspace.org.uk Thu Sep 24 01:19:21 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 24 Sep 2009 00:19:21 +0100 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> <4ABA98A6.7060803@voidspace.org.uk> <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com> Message-ID: <4ABAACF9.60607@voidspace.org.uk> Benjamin Peterson wrote: > 2009/9/23 Michael Foord : > >> Isn't that the real compatibility test *anyway* - how successful a new >> version of Python is at running all the existing Python code... >> > > Yes, but we should have expect 3rd party code to be detecting bugs for > us that our test suite could have shown on a platform. > > > Well, Trent Nelson is going to devote some real time to Snakebite - so there is a very good chance that it will be up and active before the release. Naturally I agree it would be *preferable* to run tests on all supported platforms prior to release. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From benjamin at python.org Thu Sep 24 01:22:20 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 23 Sep 2009 18:22:20 -0500 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <4ABAACF9.60607@voidspace.org.uk> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> <4ABA98A6.7060803@voidspace.org.uk> <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com> <4ABAACF9.60607@voidspace.org.uk> Message-ID: <1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com> 2009/9/23 Michael Foord : > Benjamin Peterson wrote: >> >> 2009/9/23 Michael Foord : >> >>> >>> Isn't that the real compatibility test *anyway* - how successful a new >>> version of Python is at running all the existing Python code... >>> >> >> Yes, but we should have expect 3rd party code to be detecting bugs for >> us that our test suite could have shown on a platform. >> >> >> > > Well, Trent Nelson is going to devote some real time to Snakebite - so there > is a very good chance that it will be up and active before the release. > Naturally I agree it would be *preferable* to run tests on all supported > platforms prior to release. Could I inquiry about your source of this information? -- Regards, Benjamin From fuzzyman at voidspace.org.uk Thu Sep 24 01:23:46 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Thu, 24 Sep 2009 00:23:46 +0100 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> <4ABA98A6.7060803@voidspace.org.uk> <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com> <4ABAACF9.60607@voidspace.org.uk> <1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com> Message-ID: <4ABAAE02.1060800@voidspace.org.uk> Benjamin Peterson wrote: > 2009/9/23 Michael Foord : > >> Benjamin Peterson wrote: >> >>> 2009/9/23 Michael Foord : >>> >>> >>>> Isn't that the real compatibility test *anyway* - how successful a new >>>> version of Python is at running all the existing Python code... >>>> >>>> >>> Yes, but we should have expect 3rd party code to be detecting bugs for >>> us that our test suite could have shown on a platform. >>> >>> >>> >>> >> Well, Trent Nelson is going to devote some real time to Snakebite - so there >> is a very good chance that it will be up and active before the release. >> Naturally I agree it would be *preferable* to run tests on all supported >> platforms prior to release. >> > > Could I inquiry about your source of this information? > > > > From Trent on the snakebite mailing list. Too late for me to look it up though; an exercise I leave to the reader. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From steve at pearwood.info Thu Sep 24 01:33:08 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 24 Sep 2009 09:33:08 +1000 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> Message-ID: <200909240933.09773.steve@pearwood.info> On Thu, 24 Sep 2009 06:47:41 am Dino Viehland wrote: > > So I am +1 on unified the message and +1 on using the "does not > > support indexing" one. > > I'd be +1 on the unified message as well - but it seems what that > message should be may be contentious (and quite a bike shed > discussion at that). The bug David linked to > (http://bugs.python.org/issue5760) has a preference for subscript > because that's what's used elsewhere in the language. For what it's worth, "unsubscriptable object" seems to me to be mysterious to many newbies, and even a few non-newbies. It took me something like seven years of coding to stop reading it as "unscriptable object", and I'm sure I won't be the only one. As far as I can see, in practice, people talk about obj[i] as the item at index i, not the item at subscript i -- the term "subscript" in this context seems to be rare to non-existent except for the error message. -- Steven D'Aprano From janzert at janzert.com Thu Sep 24 01:56:22 2009 From: janzert at janzert.com (Janzert) Date: Wed, 23 Sep 2009 19:56:22 -0400 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <200909240933.09773.steve@pearwood.info> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> <200909240933.09773.steve@pearwood.info> Message-ID: Steven D'Aprano wrote: > On Thu, 24 Sep 2009 06:47:41 am Dino Viehland wrote: >>> So I am +1 on unified the message and +1 on using the "does not >>> support indexing" one. >> I'd be +1 on the unified message as well - but it seems what that >> message should be may be contentious (and quite a bike shed >> discussion at that). The bug David linked to >> (http://bugs.python.org/issue5760) has a preference for subscript >> because that's what's used elsewhere in the language. > > For what it's worth, "unsubscriptable object" seems to me to be > mysterious to many newbies, and even a few non-newbies. It took me > something like seven years of coding to stop reading it > as "unscriptable object", and I'm sure I won't be the only one. > > As far as I can see, in practice, people talk about obj[i] as the item > at index i, not the item at subscript i -- the term "subscript" in this > context seems to be rare to non-existent except for the error message. > > > How about if it's obj["item"]? To me the following makes complete sense, but then it seems that I may just be the odd one out. >>> class A(object): ... pass ... >>> a = A() >>> a[1] Traceback (most recent call last): File "", line 1, in TypeError: 'A' object is unindexable >>> a["a"] Traceback (most recent call last): File "", line 1, in TypeError: 'A' object is unsubscriptable >>> Janzert From david.lyon at preisshare.net Thu Sep 24 01:59:29 2009 From: david.lyon at preisshare.net (David Lyon) Date: Wed, 23 Sep 2009 19:59:29 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <4AB9D2FC.6040705@egenix.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB9D2FC.6040705@egenix.com> Message-ID: On Wed, 23 Sep 2009 09:49:16 +0200, "M.-A. Lemburg" wrote: > While it's a good idea to put up some form of meta-data > into an index, I wonder why you are using setup.cfg > for this. > > setup.cfg has traditionally been used to configure distutils, > not to define meta-data. As such you wouldn't want to > put such a configuration file up on PyPI. > > Wouldn't it be better use a new file for this (with the > same syntax), e.g. metadata.cfg ?! This would then just > contain the meta data that needs to be published. Forgive me if I answer this from being on the distutils-ml but it is easy to answer and might save somebody else time. Currently, the metadata is stored within setup.py and the biggest issue with that is maintaining the version number. For instance, getting the same version number to go into the documentation files, etc. So consensus was that keeping that information in setup.py is somewhat cumbersome and putting it somewhere else would be better. Tarek suggested setup.cfg and it seems to make the most sense. By moving the metadata values into a configParser format file, it would mean that external scripts could more easily access/update the version number as required. Of course, setup.cfg wouldn't get uploaded. Nobody would want to do that. But distutils would create the .PKG_INFO file and metadata from configuration, and not from hardcoded string values within setup.py as it does now. David From ben+python at benfinney.id.au Thu Sep 24 02:10:43 2009 From: ben+python at benfinney.id.au (Ben Finney) Date: Thu, 24 Sep 2009 10:10:43 +1000 Subject: [Python-Dev] unsubscriptable vs object does not support indexing References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> <200909240933.09773.steve@pearwood.info> Message-ID: <87tyyt8anw.fsf@benfinney.id.au> Steven D'Aprano writes: > As far as I can see, in practice, people talk about obj[i] as the item > at index i, not the item at subscript i -- the term "subscript" in > this context seems to be rare to non-existent except for the error > message. Presumably, the same people would also call ?obj[i]? the item at *key* ?i?, if ?obj? is a dictionary. For an object that supports neither indexes nor keys, though, how is Python to know which the user meant? It's a single operation as far as the parser is concerned, so there needs to be a single term for it. That term is ?subscript?. Your point about the awkward word ?unsubscriptable? is well-taken, though. Perhaps a simple improvement to the message wording: >>> foo = 3 >>> foo[6] Traceback (most recent call last): File "", line 1, in TypeError: 'int' object does not support subscripts >>> foo['spam'] Traceback (most recent call last): File "", line 1, in TypeError: 'int' object does not support subscripts -- \ ?I lost a button-hole.? ?Steven Wright | `\ | _o__) | Ben Finney From eric at trueblade.com Thu Sep 24 02:12:13 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 23 Sep 2009 20:12:13 -0400 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <4ABAAE02.1060800@voidspace.org.uk> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> <4ABA98A6.7060803@voidspace.org.uk> <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com> <4ABAACF9.60607@voidspace.org.uk> <1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com> <4ABAAE02.1060800@voidspace.org.uk> Message-ID: <4ABAB95D.1020906@trueblade.com> Michael Foord wrote: > Benjamin Peterson wrote: >> 2009/9/23 Michael Foord : >> >>> Benjamin Peterson wrote: >>> >>>> 2009/9/23 Michael Foord : >>>> >>>> >>>>> Isn't that the real compatibility test *anyway* - how successful a new >>>>> version of Python is at running all the existing Python code... >>>>> >>>>> >>>> Yes, but we should have expect 3rd party code to be detecting bugs for >>>> us that our test suite could have shown on a platform. >>>> >>>> >>>> >>>> >>> Well, Trent Nelson is going to devote some real time to Snakebite - >>> so there >>> is a very good chance that it will be up and active before the release. >>> Naturally I agree it would be *preferable* to run tests on all supported >>> platforms prior to release. >>> >> >> Could I inquiry about your source of this information? >> >> >> >> > From Trent on the snakebite mailing list. Too late for me to look it up > though; an exercise I leave to the reader. http://groups.google.com/group/snakebite-list/browse_thread/thread/d08642261f2cc502 Eric. From python at mrabarnett.plus.com Thu Sep 24 02:15:19 2009 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 24 Sep 2009 01:15:19 +0100 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> <200909240933.09773.steve@pearwood.info> Message-ID: <4ABABA17.5010403@mrabarnett.plus.com> Janzert wrote: > Steven D'Aprano wrote: >> On Thu, 24 Sep 2009 06:47:41 am Dino Viehland wrote: >>>> So I am +1 on unified the message and +1 on using the "does not >>>> support indexing" one. >>> I'd be +1 on the unified message as well - but it seems what that >>> message should be may be contentious (and quite a bike shed >>> discussion at that). The bug David linked to >>> (http://bugs.python.org/issue5760) has a preference for subscript >>> because that's what's used elsewhere in the language. >> For what it's worth, "unsubscriptable object" seems to me to be >> mysterious to many newbies, and even a few non-newbies. It took me >> something like seven years of coding to stop reading it >> as "unscriptable object", and I'm sure I won't be the only one. >> >> As far as I can see, in practice, people talk about obj[i] as the item >> at index i, not the item at subscript i -- the term "subscript" in this >> context seems to be rare to non-existent except for the error message. >> >> >> > > How about if it's obj["item"]? To me the following makes complete sense, > but then it seems that I may just be the odd one out. > >>>> class A(object): > ... pass > ... >>>> a = A() >>>> a[1] > Traceback (most recent call last): > File "", line 1, in > TypeError: 'A' object is unindexable >>>> a["a"] > Traceback (most recent call last): > File "", line 1, in > TypeError: 'A' object is unsubscriptable > I prefer "index" to "subscript" and "does not support foo" or "cannot be fooed" to "is unfooable" (because, to me, "not" is more explicit than "un-"). I also prefer the same term to be used irrespective of the value used. From python at mrabarnett.plus.com Thu Sep 24 02:20:22 2009 From: python at mrabarnett.plus.com (MRAB) Date: Thu, 24 Sep 2009 01:20:22 +0100 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <87tyyt8anw.fsf@benfinney.id.au> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> <200909240933.09773.steve@pearwood.info> <87tyyt8anw.fsf@benfinney.id.au> Message-ID: <4ABABB46.2040105@mrabarnett.plus.com> Ben Finney wrote: > Steven D'Aprano writes: > >> As far as I can see, in practice, people talk about obj[i] as the item >> at index i, not the item at subscript i -- the term "subscript" in >> this context seems to be rare to non-existent except for the error >> message. > > Presumably, the same people would also call ?obj[i]? the item at *key* > ?i?, if ?obj? is a dictionary. For an object that supports neither > indexes nor keys, though, how is Python to know which the user meant? > It's a single operation as far as the parser is concerned, so there > needs to be a single term for it. That term is ?subscript?. > > Your point about the awkward word ?unsubscriptable? is well-taken, > though. Perhaps a simple improvement to the message wording: > > >>> foo = 3 > >>> foo[6] > Traceback (most recent call last): > File "", line 1, in > TypeError: 'int' object does not support subscripts > >>> foo['spam'] > Traceback (most recent call last): > File "", line 1, in > TypeError: 'int' object does not support subscripts > It's called a 'subscript' because conventional mathematical notation uses subscripting. Newbies might be acquainted with the term 'index' from books, where the 'value' is non-numeric. It's a bit unfortunate that dicts have keys+value instead of index+value! :-) From solipsis at pitrou.net Thu Sep 24 02:25:37 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 24 Sep 2009 00:25:37 +0000 (UTC) Subject: [Python-Dev] unsubscriptable vs object does not support indexing References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> <200909240933.09773.steve@pearwood.info> <87tyyt8anw.fsf@benfinney.id.au> <4ABABB46.2040105@mrabarnett.plus.com> Message-ID: MRAB mrabarnett.plus.com> writes: > > It's called a 'subscript' because conventional mathematical notation > uses subscripting. Newbies might be acquainted with the term 'index' > from books, where the 'value' is non-numeric. It's a bit unfortunate > that dicts have keys+value instead of index+value! Well, "index" for me points to the mathematical notion of an index, which AFAIK is always numeric. So it would be a mistake to use that term for dicts. Regards Antoine. From guido at python.org Thu Sep 24 02:31:34 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Sep 2009 17:31:34 -0700 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> <200909240933.09773.steve@pearwood.info> <87tyyt8anw.fsf@benfinney.id.au> <4ABABB46.2040105@mrabarnett.plus.com> Message-ID: On Wed, Sep 23, 2009 at 5:25 PM, Antoine Pitrou wrote: > MRAB mrabarnett.plus.com> writes: >> >> It's called a 'subscript' because conventional mathematical notation >> uses subscripting. Newbies might be acquainted with the term 'index' >> from books, where the 'value' is non-numeric. It's a bit unfortunate >> that dicts have keys+value instead of index+value! > > Well, "index" for me points to the mathematical notion of an index, which AFAIK > is always numeric. So it would be a mistake to use that term for dicts. 'Key' and 'index' refer to semantics. 'Subscript' refers to syntax. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Thu Sep 24 02:34:40 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 24 Sep 2009 12:34:40 +1200 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4AB9D865.1070701@simplistix.co.uk> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <4AB9D865.1070701@simplistix.co.uk> Message-ID: <4ABABEA0.8040802@canterbury.ac.nz> Chris Withers wrote: > But this isn't coercion! :-) The raisin is probably hysterical. Once upon a time, Python had a __coerce__ special method that took care of things like this in a different (and inferior) way. It gradually got replaced by the current scheme, but nobody changed the section heading in the docs. -- Greg From greg.ewing at canterbury.ac.nz Thu Sep 24 02:44:33 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 24 Sep 2009 12:44:33 +1200 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4ABA0FE9.7020401@gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <4AB9A3A9.1090800@canterbury.ac.nz> <4ABA0FE9.7020401@gmail.com> Message-ID: <4ABAC0F1.3080603@canterbury.ac.nz> Nick Coghlan wrote: > The interpreter doesn't promise to call those slots with "self" first - > self will be the second argument in the "rop" case. I know. My question is: How does it know whether a subclass "has overridden __rop__" when there is no concept of an __rop__ method distinct from the __op__ method? -- Greg From greg.ewing at canterbury.ac.nz Thu Sep 24 03:33:43 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 24 Sep 2009 13:33:43 +1200 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: <87tyyt8anw.fsf@benfinney.id.au> References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <1A472770E042064698CB5ADC83A12ACD0493F955@TK5EX14MBXC120.redmond.corp.microsoft.com> <200909240933.09773.steve@pearwood.info> <87tyyt8anw.fsf@benfinney.id.au> Message-ID: <4ABACC77.2060601@canterbury.ac.nz> Ben Finney wrote: > It's a single operation as far as the parser is concerned, so there > needs to be a single term for it. That term is ?subscript?. How about something like "does not support the [] operation". This refers directly to the syntax involved, rather than using a typographical term that is at best a metaphor when applied to a language written linearly. -- Greg From ncoghlan at gmail.com Thu Sep 24 03:48:06 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 24 Sep 2009 11:48:06 +1000 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <4ABAB95D.1020906@trueblade.com> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <1afaf6160909231334v4a3a294h3bcdd036e69a4eab@mail.gmail.com> <1afaf6160909231442u43efb06exbc216ec78b1fc496@mail.gmail.com> <4ABA98A6.7060803@voidspace.org.uk> <1afaf6160909231617n7d7bb71am7ec5de809ef423d3@mail.gmail.com> <4ABAACF9.60607@voidspace.org.uk> <1afaf6160909231622o5f677b48yde371b2d8a8147f4@mail.gmail.com> <4ABAAE02.1060800@voidspace.org.uk> <4ABAB95D.1020906@trueblade.com> Message-ID: <4ABACFD6.3080401@gmail.com> Eric Smith wrote: >> From Trent on the snakebite mailing list. Too late for me to look it >> up though; an exercise I leave to the reader. > > http://groups.google.com/group/snakebite-list/browse_thread/thread/d08642261f2cc502 Hmm, I thought I was subscribed to the snakebite list... guess I will have to check my settings. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From pje at telecommunity.com Thu Sep 24 03:57:14 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 23 Sep 2009 21:57:14 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.co m> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <20090923204737.A97DC3A4079@sparrow.telecommunity.com> <94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com> <94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com> <94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com> Message-ID: <20090924015718.B11343A4079@sparrow.telecommunity.com> At 12:40 AM 9/24/2009 +0200, Tarek Ziad? wrote: >Come on phillip, no one is "plotting" against you. Perhaps this is a language issue. When I said, "if Tarek is too busy projecting hidden plots onto everything I say and do," I meant that you were acting as if I were plotting against *you*, not the other way around. (For example, you described one of my proposals as "suspicious", in a context that made it appear you were concerned it would undermine your plans for Distribute.) However, I find it "suspicious" myself, that, rather than actually address *any* of the substantial issues I brought up, you chose to re-argue points I'm not even disputing. For example, is it really necessary to make *every* post of yours that mentions me include an essay on how long it's been since the last setuptools release? As I said below, I don't see how that's remotely relevant to the value of my contributions... but somehow you find a way to bring it up constantly. Do you feel so guilty about forking that you need to continually re-justify yourself? If you're doing it on my account, do please stop. AFAIR, I haven't said a negative thing about your fork since it got off the ground, and have in fact said many positive things about it. Indeed, the only negative thing I would currently say about it, is that your characterization of it as a "friendly" fork is not consistent with your public behavior and demeanor towards me. Once again, I'd like for the badgering to stop. Thanks. At 12:40 AM 9/24/2009 +0200, Tarek Ziad? wrote: >You didn't maintain setuptools for a year while people where begging >you to do bug fixes. You blessed Ian and Jim to take over but they >are too busy to do it . I even sent them a mail on my side to try to >convince them. > >So we asked you to bless someone else that was active (not in >particular me as your mail seem to say) but you did not. So we >forked. And people were pissed off at you. (Which I am sorry about) > >If a project is not maintained and if the maintainer does not open >it to other maintainers, that s the way to go . > >And the fact that I took the lead of that fork doesn't mean I am >offended because you did not bless me to maintain setuptools. It >just means that I want to move forward and have a working tool for python 3. > >So let me make it clear that when you say "Tarek appears" it is >something I have never said but rather something you are thinking >like being the truth. > >Now for the Distribute work , your patches are very welcome. It s a >community project. > >>On Sep 23, 2009 10:47 PM, "P.J. Eby" >><pje at telecommunity.com> wrote: >> >>At 07:00 PM 9/23/2009 +0200, Tarek Ziad? wrote: > > While it's >>great to have Philipp being part of o... >>Here's what actually happened, if anyone cares. Tarek and friends >>announced a fork of setuptools. I reviewed the work and saw that >>-- for the most part -- I was happy with it, and opined as how I >>might be willing to bless the the "package inquisition" team as >>official maintainers of the 0.6 branch of setuptools, so that I >>could work on the fun bits I've long planned for 0.7, but never >>felt free to start on while there was so much still needing to be done on 0.6. >> >>However, just as I mentioned this, and suggested an option for what >>I could do that would be helpful to his Distribute 0.7 project as >>well as various other tools (e.g. implementing some of Jim Fulton's >>long-requested features for better modularization of setuptools), >>Tarek accused me of somehow trying to undermine his plans. >> >>In addition, it appears Tarek was also offended by my earlier >>statement that there were only a few people in the Python community >>who had *already* earned my implicit trust to not only hack on >>setuptools unsupervised, but also to take over its *future* >>direction and BDFL-ship. (For example, Jim Fulton and Ian Bicking.) >> >>Tarek, however, appears to have taken this to mean that I >>personally thought he was an incompetent programmer or something >>(when I actually had no opinion one way or the other), and ever >>since he has taken to levelling potshots like the above at me on a >>semi-regular basis. >> >>I've tried to ignore this and play nice, because he is actually >>working on this stuff and I am not. But it's hard for me to >>actually give any help in practice, if Tarek is too busy projecting >>hidden plots onto everything I say and do. >> >>If you read Tarek's distutils-sig posts, it appears my >>already-existing trust in Ian and Jim was not only a personal >>insult to Tarek, but also a plot to ensure that nobody with any >>time to do so would ever work on setuptools, just as my excitement >>about working on setuptools again was a plot to steal thunder from his fork. >> >>All I want is for good stuff to happen for setuptools users and >>Python users in general, so I don't think all the suspicion and >>backbiting is merited. I certainly don't appreciate it, and I >>would like it to stop. It also isn't even relevant to the thread, >>since my lack of work on setuptools says exactly zero about the >>merits or lack thereof of Tarek's proposals for the distutils! >> >>Hell, I *support* the bulk of Tarek's setup.cfg proposal, and don't >>even object to him Pronouncing it or cutting off the >>discussion! My only issue on Python-Dev was his inaccurate >>implication that it was a SIG consensus rather than a pronouncement >>on it. There is and was no need for any of this to get personal, >>and I have continually strived to keep my posts here and >>distutils-sig civil, even when I didn't feel like being civil in >>response to Tarek's jabs. I have in fact bent over backwards to be >>*nice* to Tarek, because he seemed so damn sensitive about >>everything. Apparently, however, this does not actually help things. :-( From guido at python.org Thu Sep 24 04:02:43 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Sep 2009 19:02:43 -0700 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4ABAC0F1.3080603@canterbury.ac.nz> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <4AB9A3A9.1090800@canterbury.ac.nz> <4ABA0FE9.7020401@gmail.com> <4ABAC0F1.3080603@canterbury.ac.nz> Message-ID: On Wed, Sep 23, 2009 at 5:44 PM, Greg Ewing wrote: > Nick Coghlan wrote: > >> The interpreter doesn't promise to call those slots with "self" first - >> self will be the second argument in the "rop" case. > > I know. My question is: How does it know whether a subclass > "has overridden __rop__" when there is no concept of an > __rop__ method distinct from the __op__ method? This is a constraint on types implemented in C -- the same definition is required to apply to __op__ and __rop__. The datetime module actually has a slight problem here because it wants to override datetime - timedelta but not timedelta - datetime (since the latter would have to return a negative time, which is meaningless). It solves this easily by doing a type check on each argument and returning Py_NotImplemented for the unimplemented combinations -- see the various _subtract functions in Modules/datetimemodule.c. Returning NotImplemented is close enough to not actually implementing the method. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Sep 24 04:04:18 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 24 Sep 2009 12:04:18 +1000 Subject: [Python-Dev] unsubscriptable vs object does not support indexing In-Reply-To: References: <1A472770E042064698CB5ADC83A12ACD04937FC5@TK5EX14MBXC120.redmond.corp.microsoft.com> <4AB9735B.4000102@mrabarnett.plus.com> <4ABA905C.2040807@egenix.com> <4ABA90C4.2040401@voidspace.org.uk> <4ABA97F2.6040803@voidspace.org.uk> Message-ID: <4ABAD3A2.7050303@gmail.com> Brett Cannon wrote: > So +0 on your specific improvment and +1 for trying to think about > newbies when writing exception messages. The __eq__/__hash__ messages are somewhat arcane because the problem they're describing is somewhat arcane. Michael's suggested improvement also isn't quite right (and I believe indicates that we're susceptible to issuing false alarms in this case). You really only need to override __hash__ if you want your object to remain hashable, but we currently issue the warning for any object that defines __eq__ without defining __hash__. Perhaps the implementation of that warning should be improved so that it is only issued if the default __hash__ implementation is actually invoked on an affected object? (the downside is that depending on the implementation technique the extra checks may slow down hashing under "python -3", but I guess that kind of the undesirable impact is the reason we have all those warnings switched off by default) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Thu Sep 24 04:05:15 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 23 Sep 2009 19:05:15 -0700 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: <20090924015718.B11343A4079@sparrow.telecommunity.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <20090923204737.A97DC3A4079@sparrow.telecommunity.com> <94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com> <94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com> <94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com> <20090924015718.B11343A4079@sparrow.telecommunity.com> Message-ID: On Wed, Sep 23, 2009 at 6:57 PM, P.J. Eby wrote: > Once again, I'd like for the badgering to stop. ?Thanks. Tarek already agreed to that ("Fine with me. Let's move forward"). I hope you will stop badgering him too. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Sep 24 04:07:03 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 24 Sep 2009 12:07:03 +1000 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com> <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com> Message-ID: <4ABAD447.408@gmail.com> Mark Dickinson wrote: > On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland wrote: >> We are going to start contributing tests back real soon now. I'm not sure >> that these are the best tests to contribute as they require a version of >> Python to compare against rather than being nice and stand alone. But I'm >> sure we have other tests which cover this as well just not as exhaustively. >> We could also possibly check in the baseline file and then CPython could >> compare it's self to previous versions but it'd probably be a pretty >> big file - so it probably shouldn't be included in the standard install >> in the tests directory. > > How big is big? For comparison, CPython's Lib/test/decimaltestdata > directory alone is already over 4Mb, so maybe size isn't an issue? For big test files, there's also the option of including download logic in the test to go retrieve it from a known URL (I believe we already do that for some of the codec tests). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Sep 24 04:16:01 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 24 Sep 2009 12:16:01 +1000 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> Message-ID: <4ABAD661.3020103@gmail.com> Antoine Pitrou wrote: > As far as I'm concerned, anything which looks intuitive enough (e.g. ini-style) > and not overly complicated is fine. The details of the syntax aren't really > important as long as they make sense, and don't get in the way. One small comment before all this goes back to the distutils list: the abbreviation "py" shouldn't be used to refer to the Python interpreter as it is somewhat ambiguous due to the existence of the independent "py" package [1]. The longer "python_version" has a genuine advantage in clarity here. Cheers, Nick. [1] http://codespeak.net/py/dist/index.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Thu Sep 24 04:26:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 24 Sep 2009 12:26:50 +1000 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> Message-ID: <4ABAD8EA.9060202@gmail.com> Antoine Pitrou wrote: > As far as I'm concerned, anything which looks intuitive enough (e.g. ini-style) > and not overly complicated is fine. The details of the syntax aren't really > important as long as they make sense, and don't get in the way. One small comment before all this goes back to the distutils list: the abbreviation "py" shouldn't be used to refer to the Python interpreter as it is somewhat ambiguous due to the existence of the independent "py" package [1]. The longer "python_version" has a genuine advantage in clarity here. Cheers, Nick. [1] http://codespeak.net/py/dist/index.html -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From pje at telecommunity.com Thu Sep 24 05:36:13 2009 From: pje at telecommunity.com (P.J. Eby) Date: Wed, 23 Sep 2009 23:36:13 -0400 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <20090923204737.A97DC3A4079@sparrow.telecommunity.com> <94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com> <94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com> <94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com> <20090924015718.B11343A4079@sparrow.telecommunity.com> Message-ID: <20090924033618.6AFE33A4079@sparrow.telecommunity.com> At 07:05 PM 9/23/2009 -0700, Guido van Rossum wrote: >On Wed, Sep 23, 2009 at 6:57 PM, P.J. Eby wrote: > > Once again, I'd like for the badgering to stop. Thanks. > >Tarek already agreed to that ("Fine with me. Let's move forward"). I >hope you will stop badgering him too. The only reason I said "once again" above instead of, "One last time", is because I didn't want to be misconstrued as making some sort of threat. However, I can assure you, the above is the very last time I will make that request. In any case, I hadn't yet seen the message of Tarek's you allude to above, in large part because it wasn't actually *addressed* to me. It went out only to Python-Dev, whereas the message calling me paranoid went straight to my inbox. I doubt that it was intentional on Tarek's part (checking the thread shows that Antoine's the one who first cut the cc: list), but the fact remains that I hadn't seen it when I sent my message. Otherwise, I'd have replied in *that* thread, rather than this one. From david.lyon at preisshare.net Thu Sep 24 06:23:31 2009 From: david.lyon at preisshare.net (David Lyon) Date: Thu, 24 Sep 2009 00:23:31 -0400 Subject: [Python-Dev] thinking about 2.7 / buildbots / testing In-Reply-To: <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> Message-ID: On Wed, 23 Sep 2009 15:13:55 -0000, exarkun at twistedmatrix.com wrote: > Quite a few years of experience with a distributed team of build slave > managers has shown me that by far the most reliable way to keep slaves > online is to have them managed by a dedicated team. This team doesn't > need to be small, but since finding dedicated people can sometimes be > challenging, I think small teams are the most likely outcome (possibly > resulting in a team of one). > .. > Casual volunteers generally just won't keep up with these tasks. True. > I suggest finding someone who's seriously interested in the quality of > CPython and giving them the responsibility of keeping things operating > properly. This includes paying attention to the status of slaves, > cajoling hardware operators into bringing hosts back online and fixing > network issues, and finding replacements of the appropriate type > (hardware/software platform) when a slave host is permanently lost. Well, I'm a system administrator now, and a casual developer. Yes, you need somebody who watches machine control panels and can type emails on python lists at the same time. > I would also personally recommend that this person first (well, after > tracking down all the slave operators and convincing them to bring their > slaves back online) acquire shell access to all of the slave machines so > that the owners of the slave hosts themselves no longer need to be the > gating factor for most issues. Depends on where the machines are. There are good tools do check all automatically. Nagios is one. Anyway, this would suite my work schedule for the next 12 months. Do we already have the machines? or do they need to be acquired? David From ctb at msu.edu Thu Sep 24 06:40:27 2009 From: ctb at msu.edu (C. Titus Brown) Date: Wed, 23 Sep 2009 21:40:27 -0700 Subject: [Python-Dev] thinking about 2.7 / buildbots / testing In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> Message-ID: <20090924044027.GB22595@idyll.org> On Thu, Sep 24, 2009 at 12:23:31AM -0400, David Lyon wrote: > On Wed, 23 Sep 2009 15:13:55 -0000, exarkun at twistedmatrix.com wrote: > > > I would also personally recommend that this person first (well, after > > tracking down all the slave operators and convincing them to bring their > > slaves back online) acquire shell access to all of the slave machines so > > that the owners of the slave hosts themselves no longer need to be the > > gating factor for most issues. > > Depends on where the machines are. There are good tools do check all > automatically. Nagios is one. > > Anyway, this would suite my work schedule for the next 12 months. > > Do we already have the machines? or do they need to be acquired? Hi David, as Brett leaked, I am developing software towards making this buildbot dilemma easier to deal with. It's not really ready for primetime yet; I'll send you a link off-list. I am also (physically) hosting Snakebite, which Trent Nelson will be tackling over the next few months. And, finally, I now have Windows VMs (XP, Vista, 7) and a Mac OS X box that I can dedicate to Python purposes. I even have a student that will be monitoring those boxes. I am happy to give you access to these machines if you want to set up buildbot on them; buildbot is compatible with my own endeavors, so that would actually help me out ;) I will be buying more/better/bigger Linux hardware over the next few months, too, and I am happy to slice that off to provide more VMs -- basically whatever x86-based platforms people think are needed. cheers, --titus -- C. Titus Brown, ctb at msu.edu From mal at egenix.com Thu Sep 24 10:55:24 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Thu, 24 Sep 2009 10:55:24 +0200 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB9D2FC.6040705@egenix.com> Message-ID: <4ABB33FC.5080105@egenix.com> David Lyon wrote: > On Wed, 23 Sep 2009 09:49:16 +0200, "M.-A. Lemburg" wrote: > >> While it's a good idea to put up some form of meta-data >> into an index, I wonder why you are using setup.cfg >> for this. >> >> setup.cfg has traditionally been used to configure distutils, >> not to define meta-data. As such you wouldn't want to >> put such a configuration file up on PyPI. >> >> Wouldn't it be better use a new file for this (with the >> same syntax), e.g. metadata.cfg ?! This would then just >> contain the meta data that needs to be published. > > Forgive me if I answer this from being on the distutils-ml > but it is easy to answer and might save somebody else time. > > Currently, the metadata is stored within setup.py and the > biggest issue with that is maintaining the version number. > For instance, getting the same version number to go into the > documentation files, etc. So consensus was that keeping that > information in setup.py is somewhat cumbersome and putting > it somewhere else would be better. Tarek suggested setup.cfg > and it seems to make the most sense. So the idea is to make the setup() parameters settable via the setup.cfg as well ?! (+1 on that) Or are you suggesting that the setup.cfg file be the *only* place to set the version number ? (I'd be -1 on that, because you often need to determine the version number dynamically, e.g. when doing snapshot releases) In my experience, it's usually better to place all the setup() keyword parameters into a separate module which can then be loaded by other tools as well - see e.g. egenix-mx-base for what I mean: http://www.egenix.com/products/python/mxBase/#Download The setup.py module itself then just loads this configuration module and uses the values from it to fire off distutils. > By moving the metadata values into a configParser format file, > it would mean that external scripts could more easily > access/update the version number as required. > > Of course, setup.cfg wouldn't get uploaded. Nobody would > want to do that. > > But distutils would create the .PKG_INFO file and metadata > from configuration, and not from hardcoded string values > within setup.py as it does now. I lost you there... if you have the setup.py file available, where is the problem in running "python setup.py somecommand" to extract the metadata into some other file ? As far as I understood Tarek's summary, the idea was to have a separate file with just the metadata to upload to PyPI in order to have tools working on the PyPI index extract that metadata without having to download the whole installation package. That's a very good idea. The only problem I saw was the name of the file that Tarek chose and whether or not he was actually referring to the setup.cfg used for configuring distutils or not. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 24 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From solipsis at pitrou.net Thu Sep 24 13:13:58 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 24 Sep 2009 11:13:58 +0000 (UTC) Subject: [Python-Dev] [OT] cutting the cc list References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <20090923204737.A97DC3A4079@sparrow.telecommunity.com> <94bdd2610909231526l62c20d1cub6f2dff9d7d0480b@mail.gmail.com> <94bdd2610909231539n6ff5932fwf6c9106bea322195@mail.gmail.com> <94bdd2610909231540o410cf30cif4a23d743b0c5e7d@mail.gmail.com> <20090924015718.B11343A4079@sparrow.telecommunity.com> <20090924033618.6AFE33A4079@sparrow.telecommunity.com> Message-ID: P.J. Eby telecommunity.com> writes: > > I doubt that it was intentional on Tarek's part (checking the thread > shows that Antoine's the one who first cut the cc: list), but the > fact remains that I hadn't seen it when I sent my > message. Just FYI, there are two reasons that I cut the CC list: - I usually do it deliberately, when replying from e-mail, because I don't want people to get duplicates, nor to get an e-mail not coming from the list which might then not match with their own ML filters (based on whatever headers the ML machine adds to e-mails) - In the python-dev case, I reply through gmane, and gmane doesn't do any CC, it posts only to the list cheers Antoine. From rdmurray at bitdance.com Thu Sep 24 15:08:50 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 24 Sep 2009 09:08:50 -0400 (EDT) Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <4ABAD447.408@gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <1A472770E042064698CB5ADC83A12ACD0493D412@TK5EX14MBXC120.redmond.corp.microsoft.com> <5c6f2a5d0909230934j16dd31ces97d87cfb39736491@mail.gmail.com> <4ABAD447.408@gmail.com> Message-ID: On Thu, 24 Sep 2009 at 12:07, Nick Coghlan wrote: > Mark Dickinson wrote: >> On Wed, Sep 23, 2009 at 4:54 PM, Dino Viehland wrote: >>> We are going to start contributing tests back real soon now. I'm not sure >>> that these are the best tests to contribute as they require a version of >>> Python to compare against rather than being nice and stand alone. But I'm >>> sure we have other tests which cover this as well just not as exhaustively. >>> We could also possibly check in the baseline file and then CPython could >>> compare it's self to previous versions but it'd probably be a pretty >>> big file - so it probably shouldn't be included in the standard install >>> in the tests directory. >> >> How big is big? For comparison, CPython's Lib/test/decimaltestdata >> directory alone is already over 4Mb, so maybe size isn't an issue? > > For big test files, there's also the option of including download logic > in the test to go retrieve it from a known URL (I believe we already do > that for some of the codec tests). We do. There's even support code in test.support for handling such downloads relatively cleanly (and cleaning up afterward, though that last was only added recently). --David From solipsis at pitrou.net Thu Sep 24 15:49:35 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 24 Sep 2009 13:49:35 +0000 (UTC) Subject: [Python-Dev] thinking about 2.7 / buildbots / testing References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> Message-ID: Le Thu, 24 Sep 2009 00:23:31 -0400, David Lyon a ?crit?: > > Depends on where the machines are. There are good tools do check all > automatically. Nagios is one. > > Anyway, this would suite my work schedule for the next 12 months. > > Do we already have the machines? or do they need to be acquired? I have (root) access to a powerful x86-64 Debian machine that I might give you non-root access to, if it's enough. I'll have to ask the owner of the machine, however. Tell me if that can be useful. Regards Antoine. From rdmurray at bitdance.com Thu Sep 24 16:14:46 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Thu, 24 Sep 2009 10:14:46 -0400 (EDT) Subject: [Python-Dev] thinking about 2.7 / buildbots / testing In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> Message-ID: On Thu, 24 Sep 2009 at 13:49, Antoine Pitrou wrote: > Le Thu, 24 Sep 2009 00:23:31 -0400, David Lyon a ??crit??: >> >> Depends on where the machines are. There are good tools do check all >> automatically. Nagios is one. >> >> Anyway, this would suite my work schedule for the next 12 months. >> >> Do we already have the machines? or do they need to be acquired? > > I have (root) access to a powerful x86-64 Debian machine that I might > give you non-root access to, if it's enough. I'll have to ask the owner > of the machine, however. Tell me if that can be useful. I've been a sysadmin for years (among other things). If Jean-Paul's suggestion of forming a small team to manage this is followed, I volunteer to be part of that team. And if Snakebite isn't enough I have a well-connected platform I could run Linux buildbots on in vserver virthosts, and a less-well-connected platform that could run KVM based stuff. --David (R. David Murray) From solipsis at pitrou.net Thu Sep 24 19:16:36 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 24 Sep 2009 17:16:36 +0000 (UTC) Subject: [Python-Dev] thinking about 2.7 / buildbots / testing References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> Message-ID: R. David Murray bitdance.com> writes: > > And if Snakebite isn't enough I > have a well-connected platform I could run Linux buildbots on in > vserver virthosts, and a less-well-connected platform that could run > KVM based stuff. I suppose Snakebite will be enough, but only once it is ready :) We still need some buildbots (or any equivalent) in the meantime... Regards Antoine. From martin at v.loewis.de Fri Sep 25 01:25:12 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 25 Sep 2009 01:25:12 +0200 Subject: [Python-Dev] thinking about 2.7 / buildbots / testing In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> Message-ID: <4ABBFFD8.8070707@v.loewis.de> > Do we already have the machines? or do they need to be acquired? Yes, the machines are all there, see http://www.python.org/dev/buildbot/all/ Regards, Martin From martin at v.loewis.de Fri Sep 25 01:27:52 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 25 Sep 2009 01:27:52 +0200 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> Message-ID: <4ABC0078.8030509@v.loewis.de> >> Additionally, I'm very apprehensive about doing any kind of release >> without the buildbots running. Does anyone know when they might be up? When I (or somebody else) contacts all the slave operators and asks them to restart the buildbot slaves. Regards, Martin From exarkun at twistedmatrix.com Fri Sep 25 02:06:54 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Fri, 25 Sep 2009 00:06:54 -0000 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <4ABC0078.8030509@v.loewis.de> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <4ABC0078.8030509@v.loewis.de> Message-ID: <20090925000654.1533.1151834611.divmod.xquotient.332@localhost.localdomain> On 24 Sep, 11:27 pm, martin at v.loewis.de wrote: >>>Additionally, I'm very apprehensive about doing any kind of release >>>without the buildbots running. Does anyone know when they might be >>>up? > >When I (or somebody else) contacts all the slave operators and asks >them >to restart the buildbot slaves. Does this mean you're taking responsibility for this task? Or are you looking for a volunteer? Jean-Paul From martin at v.loewis.de Fri Sep 25 02:38:18 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 25 Sep 2009 02:38:18 +0200 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: <20090925000654.1533.1151834611.divmod.xquotient.332@localhost.localdomain> References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <4ABC0078.8030509@v.loewis.de> <20090925000654.1533.1151834611.divmod.xquotient.332@localhost.localdomain> Message-ID: <4ABC10FA.2040900@v.loewis.de> >> When I (or somebody else) contacts all the slave operators and asks them >> to restart the buildbot slaves. > > Does this mean you're taking responsibility for this task? Or are you > looking for a volunteer? I had been running this buildbot installation ever since I created it, and will happily continue to take responsibility for it. Regards, Martin From weebinn at gmail.com Wed Sep 23 05:47:29 2009 From: weebinn at gmail.com (Wee Binn) Date: Wed, 23 Sep 2009 11:47:29 +0800 Subject: [Python-Dev] surprised to "++" and "--" Message-ID: Hi, I know that there is no "++" or "--" operator in python, but if "var++" or something like that in my code(you know, most of C/C++ coders may like this),there is nothing wrong reported and program goes on just like expected!! This is obscure, maybe a bug. -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew at matthewwilkes.co.uk Fri Sep 25 12:58:46 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Fri, 25 Sep 2009 11:58:46 +0100 Subject: [Python-Dev] surprised to "++" and "--" In-Reply-To: References: Message-ID: <22F36940-FCC8-49AB-A148-220EA3655E48@matthewwilkes.co.uk> > I know that there is no "++" or "--" operator in python, but if > "var++" or something like that in my code(you know, most of C/C++ > coders may like this),there is nothing wrong reported and program > goes on just like expected!! > This is obscure, maybe a bug. Hi, Firstly, this list is for the development of Python, not with Python, questions about problems you're having should generally go to the users' list. Bug reports should go to the bug tracker at http://bugs.python.org/ However, in this particular case, there's no point submitting it; you have made a mistake somewhere. As you say, there is no ++ or -- unary postfix operator, but this DOES raise a SyntaxError: > >>> var = 1 > >>> var++ > File "", line 1 > var++ > ^ > SyntaxError: invalid syntax The prefix form is valid, as + and - are both valid prefix unary operators: > >>> ++var > 1 > >>> --var > 1 Which are equivalent to: > >>> +(+1) > 1 > >>> -(-1) > 1 > If you were to try this with something that didn't implement __neg__ and __pos__, such as strings, you'd get: > >>> ++var > Traceback (most recent call last): > File "", line 1, in ? > TypeError: bad operand type for unary + Hope this clarifies things for you, Matthew From facundobatista at gmail.com Fri Sep 25 15:13:07 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Fri, 25 Sep 2009 10:13:07 -0300 Subject: [Python-Dev] surprised to "++" and "--" In-Reply-To: References: Message-ID: 2009/9/23 Wee Binn : > ????I know that there is no "++" or "--" operator in python, but if "var++" > or something like that in my code(you know, most of C/C++ coders may like > this),there is nothing wrong reported and program goes on just like > expected!! > ????This is obscure, maybe a bug. You can not write "var++": >>> var = 5 >>> var++ File "", line 1 var++ ^ SyntaxError: invalid syntax >>> -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From status at bugs.python.org Fri Sep 25 18:07:26 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 25 Sep 2009 18:07:26 +0200 (CEST) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090925160726.09CAA78638@psf.upfronthosting.co.za> ACTIVITY SUMMARY (09/18/09 - 09/25/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2410 open (+34) / 16406 closed (+20) / 18816 total (+54) Open issues with patches: 955 Average duration of open issues: 667 days. Median duration of open issues: 423 days. Open Issues Breakdown open 2373 (+31) pending 36 ( +3) Issues Created Or Reopened (54) _______________________________ Socket error when launching IDLE 09/18/09 http://bugs.python.org/issue6941 created 08jpurcell email.generator.Generator memory consumption 09/18/09 http://bugs.python.org/issue6942 created rpatterson setup.py fails to find headers of system libffi 09/19/09 http://bugs.python.org/issue6943 created Arfrever patch, easy socket.getnameinfo raises SystemError on bogus input 09/19/09 CLOSED http://bugs.python.org/issue6944 created marienz patch pprint.pprint does not pprint unsortable dicts in Python 3 09/19/09 CLOSED http://bugs.python.org/issue6945 created aronacher Document: Missing link to datetime.datetime 09/19/09 CLOSED http://bugs.python.org/issue6946 created ocean-city Fix distutils test on windows (SO extension) 09/19/09 CLOSED http://bugs.python.org/issue6947 created ocean-city patch list 09/19/09 CLOSED http://bugs.python.org/issue6948 created nutregina Support Berkeley DB 4.8 09/19/09 http://bugs.python.org/issue6949 created Arfrever patch online documentation error: PyObject* PyByteArray_Resize(PyObjec 09/19/09 CLOSED http://bugs.python.org/issue6950 created kaizhu OS X IDLE has two Preferences menus on 2.6.x with Tk > 8.4.7 09/20/09 CLOSED http://bugs.python.org/issue6951 created ned.deily easy deprecated conversion from string constant to char * 09/20/09 http://bugs.python.org/issue6952 created barry-scott readline documenation needs work 09/20/09 http://bugs.python.org/issue6953 created ronaldoussoren DISTUTILS_DEBUG causes stack trace, really simple fix 09/20/09 CLOSED http://bugs.python.org/issue6954 created nedbat Max() Not Working Properly 09/21/09 CLOSED http://bugs.python.org/issue6955 created lovelygoo2 Test creation in unittest.TestProgram should be done in one plac 09/21/09 http://bugs.python.org/issue6956 created michael.foord Extension modules fail to build on OS X 10.6 using python.org 2. 09/21/09 http://bugs.python.org/issue6957 created ned.deily Add Python command line flags to configure logging 09/21/09 http://bugs.python.org/issue6958 created theller patch OS X 10.6 / Snow Leopard: building 2.6 maintenance release fails 09/21/09 CLOSED http://bugs.python.org/issue6959 created cscscs test_telnetlib gives spurious output 09/21/09 http://bugs.python.org/issue6960 created pitrou test_distutils failure 09/21/09 CLOSED http://bugs.python.org/issue6961 created pitrou traceback.format_exception_only does not return SyntaxError caro 09/21/09 http://bugs.python.org/issue6962 created thewtex Add worker process lifetime to multiprocessing.Pool - patch incl 09/22/09 http://bugs.python.org/issue6963 created charlesc patch import new fails 09/22/09 CLOSED http://bugs.python.org/issue6964 created illume tmpnam should not be used if tempfile or mkstemp are available 09/22/09 http://bugs.python.org/issue6965 created djc Ability to refer to arguments in TestCase.fail* methods 09/22/09 http://bugs.python.org/issue6966 created tuben codec windows1256 should be windows windows-1256 09/22/09 CLOSED http://bugs.python.org/issue6967 created DerSascha patch numpy extensions to distutils... are a source of improvements fo 09/22/09 http://bugs.python.org/issue6968 created illume configparser 09/22/09 CLOSED http://bugs.python.org/issue6969 created exe Redundant calls made to comparison methods. 09/22/09 http://bugs.python.org/issue6970 created mark.dickinson needs review Add the SIO_KEEPALIVE_VALS option to socket.ioctl 09/22/09 CLOSED http://bugs.python.org/issue6971 created krisvale patch, patch, easy zipfile.ZipFile overwrites files outside destination path 09/22/09 http://bugs.python.org/issue6972 created schmir subprocess.Popen.send_signal doesn't check whether the process h 09/22/09 http://bugs.python.org/issue6973 created milko.krachounov test_posix getcwd test leaves tmp dir 09/23/09 http://bugs.python.org/issue6974 created iandekit symlinks incorrectly resolved on Linux 09/23/09 http://bugs.python.org/issue6975 created swarecki getcwd hangs and leaks mem on Solaris <= 9 in very long file nam 09/23/09 http://bugs.python.org/issue6976 created iandekit Getopt documentation ambiguity 09/23/09 http://bugs.python.org/issue6977 created SilentGhost compiler.transformer dict key bug d[1,] = 1 09/23/09 http://bugs.python.org/issue6978 created kees patch Passing output arguments to an ActiveX application 09/23/09 CLOSED http://bugs.python.org/issue6979 created benji13 fix ctypes build failure on armel-linux-gnueabi with -mfloat-abi 09/23/09 http://bugs.python.org/issue6980 created doko locale.getdefaultlocale() envvars default code and documentation 09/23/09 http://bugs.python.org/issue6981 created scop make clean does not remove pickle files 09/23/09 CLOSED http://bugs.python.org/issue6982 created egreen patch Add specific get_platform() for freebsd 09/24/09 http://bugs.python.org/issue6983 created stefw patch typo in Unicode HOWTO 09/24/09 CLOSED http://bugs.python.org/issue6984 created Yinon range() fails with long integers 09/24/09 CLOSED http://bugs.python.org/issue6985 created kszawala _json crash on scanner/encoder initialization error 09/24/09 http://bugs.python.org/issue6986 created haypo patch Idle not start 09/24/09 CLOSED http://bugs.python.org/issue6987 created dorina_n2005 shlex.split() converts unicode input to UCS-4 output 09/24/09 http://bugs.python.org/issue6988 created fenner Compile error for Python-2.6.2 on Solaris 09/24/09 http://bugs.python.org/issue6989 created ashish threading.local subclasses don't cleanup their state and it gets 09/25/09 http://bugs.python.org/issue6990 created pjenvey patch, needs review logging encoding failes some situation 09/25/09 http://bugs.python.org/issue6991 created naoki patch multiple authors in setup.by 09/25/09 http://bugs.python.org/issue6992 created techtonik importing of "time" module is terribly slow 09/25/09 http://bugs.python.org/issue6993 created ndyankov enumerate dosctring has a typo 09/25/09 http://bugs.python.org/issue6994 created nessita Issues Now Closed (41) ______________________ Passing 'None' if argtype is set to POINTER(...) doesn't always 283 days http://bugs.python.org/issue4606 theller patch Structure sub-subclass does not initialize with base class posit 237 days http://bugs.python.org/issue5042 theller patch urllib ctypes error on Mac OS X Server 10.5 200 days http://bugs.python.org/issue5413 ronaldoussoren patch 2.6.2 Python Manuals CHM file seems broken 159 days http://bugs.python.org/issue5764 dx617 os.popen causes illegal seek on AIX in Python 3.1rc 106 days http://bugs.python.org/issue6236 nestor patch Upgrading xml.etree to ElementTree 1.3 104 days http://bugs.python.org/issue6272 senthil.kumaran encode and decode should accept 'errors' as a keyword argument 93 days http://bugs.python.org/issue6300 benjamin.peterson patch, easy solaris/aix: Py_Initialize: can't initialize sys standard stream 87 days http://bugs.python.org/issue6348 pitrou ctypes.PyThreadState_SetAsyncExc does not work on linux 64bit ma 49 days http://bugs.python.org/issue6613 theller time.strftime does unnecessary range check 19 days http://bugs.python.org/issue6823 brett.cannon patch, easy urllib.urlopen crashes in a thread on Snow Leopard 13 days http://bugs.python.org/issue6851 ronaldoussoren patch, 26backport Support system readline on OS X 10.6 15 days http://bugs.python.org/issue6872 mark.dickinson patch enable compilation of readline module on Mac OS X 10.5 and 10.6 12 days http://bugs.python.org/issue6877 zvezdan patch, 26backport, needs review incorrect signature in doc for PyByteArray_Resize 9 days http://bugs.python.org/issue6881 ezio.melotti pdb documentation shows running as script using "python" not "py 7 days http://bugs.python.org/issue6885 georg.brandl ctypes compilation error on SnowLeopard 5 days http://bugs.python.org/issue6918 ronaldoussoren Doc for locals and vars 3 days http://bugs.python.org/issue6925 tjreedy [PATCH] PyUnicode_DecodeUTF16 docs wrong (byteorder param) 1 days http://bugs.python.org/issue6930 georg.brandl patch [PATCH] postflight.framework script (from the Mac OS X .dmg inst 3 days http://bugs.python.org/issue6934 ronaldoussoren patch multiprocessing lock on OS X Snow Leopard dumps core 2 days http://bugs.python.org/issue6937 ronaldoussoren cannot compute sizeof wchar_t on OS X 10.5.7, Python 2.6.2 0 days http://bugs.python.org/issue6940 rajarshi socket.getnameinfo raises SystemError on bogus input 0 days http://bugs.python.org/issue6944 georg.brandl patch pprint.pprint does not pprint unsortable dicts in Python 3 1 days http://bugs.python.org/issue6945 aronacher Document: Missing link to datetime.datetime 0 days http://bugs.python.org/issue6946 georg.brandl Fix distutils test on windows (SO extension) 0 days http://bugs.python.org/issue6947 tarek patch list 0 days http://bugs.python.org/issue6948 mark.dickinson online documentation error: PyObject* PyByteArray_Resize(PyObjec 0 days http://bugs.python.org/issue6950 ezio.melotti OS X IDLE has two Preferences menus on 2.6.x with Tk > 8.4.7 1 days http://bugs.python.org/issue6951 ronaldoussoren easy DISTUTILS_DEBUG causes stack trace, really simple fix 1 days http://bugs.python.org/issue6954 tarek Max() Not Working Properly 0 days http://bugs.python.org/issue6955 benjamin.peterson OS X 10.6 / Snow Leopard: building 2.6 maintenance release fails 1 days http://bugs.python.org/issue6959 cscscs test_distutils failure 1 days http://bugs.python.org/issue6961 tarek import new fails 1 days http://bugs.python.org/issue6964 georg.brandl codec windows1256 should be windows windows-1256 0 days http://bugs.python.org/issue6967 georg.brandl patch configparser 0 days http://bugs.python.org/issue6969 georg.brandl Add the SIO_KEEPALIVE_VALS option to socket.ioctl 3 days http://bugs.python.org/issue6971 krisvale patch, patch, easy Passing output arguments to an ActiveX application 0 days http://bugs.python.org/issue6979 mark.dickinson make clean does not remove pickle files 1 days http://bugs.python.org/issue6982 mark.dickinson patch typo in Unicode HOWTO 0 days http://bugs.python.org/issue6984 georg.brandl range() fails with long integers 0 days http://bugs.python.org/issue6985 brett.cannon Idle not start 0 days http://bugs.python.org/issue6987 amaury.forgeotdarc Top Issues Most Discussed (10) ______________________________ 19 Add Python command line flags to configure logging 4 days open http://bugs.python.org/issue6958 12 enable compilation of readline module on Mac OS X 10.5 and 10.6 12 days closed http://bugs.python.org/issue6877 9 improve xrange.__contains__ 785 days open http://bugs.python.org/issue1766304 8 IDLE 2.6.1 locks up on Mac OS 10.6 17 days open http://bugs.python.org/issue6864 7 shlex.split() converts unicode input to UCS-4 output 1 days open http://bugs.python.org/issue6988 7 Add specific get_platform() for freebsd 2 days open http://bugs.python.org/issue6983 7 Add gamma function, error functions and other C99 math.h functi 437 days open http://bugs.python.org/issue3366 6 make clean does not remove pickle files 1 days closed http://bugs.python.org/issue6982 5 range() fails with long integers 0 days closed http://bugs.python.org/issue6985 5 deprecated conversion from string constant to char * 5 days open http://bugs.python.org/issue6952 From waqas805 at hotmail.com Fri Sep 25 20:28:55 2009 From: waqas805 at hotmail.com (waqas ahmad) Date: Fri, 25 Sep 2009 20:28:55 +0200 Subject: [Python-Dev] help required Message-ID: Hi, I dont know it is the right place to post this question. I need help to change one search code line . can you help me please. here is my search method code: search=re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext) if search: ret=search.group() here i am searching for "#acl InternationalGroup" in the pageText and when it true is then give me search group. I want to change this for following requirement: I want to search for "#acl InternationalGroup" and "CatInternational" in the pageText. when "#acl InternationalGroup" is not there but "CatInternational" is there. then return me search group. I shall be thankful to you for any help. Best Regards, Waqas _________________________________________________________________ Invite your mail contacts to join your friends list with Windows Live Spaces. It's easy! http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Fri Sep 25 21:30:01 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 25 Sep 2009 20:30:01 +0100 Subject: [Python-Dev] help required In-Reply-To: References: Message-ID: <4ABD1A39.3070906@voidspace.org.uk> waqas ahmad wrote: > Hi, > > I dont know it is the right place to post this question. I need help > to change one search code line . can you help me please. > Hello Waqas, This is a list for the development *of* Python, not for development with Python. More appropriate lists, where hopefully there will be lots of people willing to help you, are python-list and python-tutor: http://mail.python.org/mailman/listinfo/python-list http://mail.python.org/mailman/listinfo/tutor Or the google groups gateway for the Python-list (comp.lang.python): http://groups.google.com/group/comp.lang.python/topics All the best, Michael Foord > here is my search method code: > > search=re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext) > if search: > ret=search.group() > > > here i am searching for "#acl InternationalGroup" in the pageText and > when it true is then give me search group. > > > I want to change this for following requirement: > > I want to search for "#acl InternationalGroup" and > "CatInternational" in the pageText. > when "#acl InternationalGroup" is not there but "CatInternational" is > there. then return me search group. > > I shall be thankful to you for any help. > > Best Regards, > Waqas > > > > > > > > ------------------------------------------------------------------------ > Invite your mail contacts to join your friends list with Windows Live > Spaces. It's easy! Try it! > > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From python at mrabarnett.plus.com Fri Sep 25 21:36:44 2009 From: python at mrabarnett.plus.com (MRAB) Date: Fri, 25 Sep 2009 20:36:44 +0100 Subject: [Python-Dev] help required In-Reply-To: References: Message-ID: <4ABD1BCC.6080902@mrabarnett.plus.com> waqas ahmad wrote: > Hi, > > I dont know it is the right place to post this question. I need help to > change one search code line . can you help me please. > > here is my search method code: > > search=re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext) > if search: > ret=search.group() > > > here i am searching for "#acl InternationalGroup" in the pageText and > when it true is then give me search group. > > > I want to change this for following requirement: > > I want to search for "#acl InternationalGroup" and "CatInternational" > in the pageText. > when "#acl InternationalGroup" is not there but "CatInternational" is > there. then return me search group. > > I shall be thankful to you for any help. > I'm not clear whether you want the search to succeed if pageText contains either, or if pageText contains "CatInternational" but not "#acl InternationalGroup". Whichever you want, you could use 2 simple separate regular expressions or 1 more complicated regular expression. Search for either: search = re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext) if not search: search = re.compile("^CatInternational.*\n", re.M).search(pagetext) if search: ret = search.group() Search for one but not the other: search = re.compile("^CatInternational.*\n", re.M).search(pagetext) if search: search_2 = re.compile("^#acl InternationalGroup.*\n", re.M).search(pagetext) if not search_2: ret = search.group() From barry at python.org Fri Sep 25 22:14:56 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 25 Sep 2009 16:14:56 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: <25524412.post@talk.nabble.com> References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> <25524412.post@talk.nabble.com> Message-ID: On Sep 19, 2009, at 2:51 PM, qwavel wrote: > What about this bug: > http://bugs.python.org/issue3890 > It appears to me that the SSL module is broken in the 2.6.x line on > all > platforms in one of its most common uses (non-blocking). It also > seems that > the problem and solution are well understood, so the solution would > make a > good candidate for 2.6.3? I made this a release blocker, but reserve the right to bump it down again. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From jnoller at gmail.com Fri Sep 25 22:18:06 2009 From: jnoller at gmail.com (Jesse Noller) Date: Fri, 25 Sep 2009 15:18:06 -0500 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> <25524412.post@talk.nabble.com> Message-ID: <4222a8490909251318h604038b7o45444d300417f317@mail.gmail.com> On Fri, Sep 25, 2009 at 3:14 PM, Barry Warsaw wrote: > On Sep 19, 2009, at 2:51 PM, qwavel wrote: > >> What about this bug: >> http://bugs.python.org/issue3890 >> It appears to me that the SSL module is broken in the 2.6.x line on all >> platforms in one of its most common uses (non-blocking). ?It also seems >> that >> the problem and solution are well understood, so the solution would make a >> good candidate for 2.6.3? > > I made this a release blocker, but reserve the right to bump it down again. > > -Barry > Barry - this is your call, but I think http://bugs.python.org/issue6990 should be a rel blocker too. jesse From db3l.net at gmail.com Fri Sep 25 22:19:57 2009 From: db3l.net at gmail.com (David Bolen) Date: Fri, 25 Sep 2009 16:19:57 -0400 Subject: [Python-Dev] thinking about 2.7 References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> Message-ID: exarkun at twistedmatrix.com writes: > It's easy for someone to volunteer to set up a new slave. It's even > easy to make sure it keeps running for 6 months. But it's not as easy > to keep it running indefinitely. This isn't about the software > involved (at least not entirely). It's about someone paying attention > to whether the slave restarts on reboots, and about paying attention > to whether the slave host has lots its network connection, or been > decomissioned, or whether a system upgrade disabled the slave, or > whatever other random administrative-like tasks are necessary to keep > things running. Casual volunteers generally just won't keep up with > these tasks. One suggestion from my perspective as a buildbot maintainer who, IMHO, has maintained generally stable and fairly well monitored buildbots (XP-4 and FreeBSD) for a while now.... I don't know if it's possible but it would be great if the buildbot master could send an email to a buildbot maintainer if it thinks a buildbot is down. There have actually been times in the past when from the perspective of my buildbot things were just dandy, but the master showed it as down. Stopping and restarting the buildbot, forcing it to reconnect, cleaned things up, but I have to monitor the web status page which is a manual process. And since it's the master's perspective that really matters when it comes to a buildbot being available, getting status from the master would be very helpful. -- David From barry at python.org Fri Sep 25 22:23:22 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 25 Sep 2009 16:23:22 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: <4222a8490909251318h604038b7o45444d300417f317@mail.gmail.com> References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> <25524412.post@talk.nabble.com> <4222a8490909251318h604038b7o45444d300417f317@mail.gmail.com> Message-ID: <923DCCFF-3475-4A04-AC46-6362D40E3F7E@python.org> On Sep 25, 2009, at 4:18 PM, Jesse Noller wrote: > Barry - this is your call, but I think > http://bugs.python.org/issue6990 should be a rel blocker too. Done. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From martin at v.loewis.de Sat Sep 26 01:03:52 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 26 Sep 2009 01:03:52 +0200 Subject: [Python-Dev] thinking about 2.7 In-Reply-To: References: <1afaf6160909230735h49b5a7cdlf206f185f4d55f7e@mail.gmail.com> <20090923151355.1533.1481070653.divmod.xquotient.187@localhost.localdomain> Message-ID: <4ABD4C58.6030405@v.loewis.de> > I don't know if it's possible but it would be great if the buildbot > master could send an email to a buildbot maintainer if it thinks a > buildbot is down. I think it should be possible to detect that. I wouldn't recommend immediate reaction, but rather a delayed reaction (say, 12h). Buildbot is very well capable of tracking timeouts of many hours. I could put that on my list of things to do, but if you beat me to it, that would be even better. One problem is that the buildbot configuration currently doesn't have email address of all slave operators, but that would be possible to add. > And since it's the master's perspective that really matters when it > comes to a buildbot being available, getting status from the master > would be very helpful. This sounds very reasonable. I'll ask on the buildbot list; maybe somebody already has that implemented. Regards, Martin From rob.cliffe at btinternet.com Fri Sep 25 12:37:27 2009 From: rob.cliffe at btinternet.com (Rob Cliffe) Date: Fri, 25 Sep 2009 11:37:27 +0100 Subject: [Python-Dev] surprised to "++" and "--" References: Message-ID: <999354BF3D184B7CA51D6E0512729F9F@robslaptop> I don't think this is quite true. ++var and --var are legal albeit redundant expressions that equal var var++ and var-- cause a SyntaxError, as they should. Rob ----- Original Message ----- From: Wee Binn To: python-dev at python.org Sent: Wednesday, September 23, 2009 4:47 AM Subject: [Python-Dev] surprised to "++" and "--" Hi, I know that there is no "++" or "--" operator in python, but if "var++" or something like that in my code(you know, most of C/C++ coders may like this),there is nothing wrong reported and program goes on just like expected!! This is obscure, maybe a bug. ------------------------------------------------------------------------------ _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/rob.cliffe%40btinternet.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at barrys-emacs.org Sat Sep 26 15:23:14 2009 From: barry at barrys-emacs.org (Barry Scott) Date: Sat, 26 Sep 2009 14:23:14 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AAFDA0D.5040602@scottdial.com> <1253044711.5678.56.camel@localhost> <4AB02471.7000701@scottdial.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> Message-ID: On 16 Sep 2009, at 01:05, Peter Moody wrote: > On Tue, Sep 15, 2009 at 4:34 PM, Scott Dial > wrote: >> R. David Murray wrote: >>> On Tue, 15 Sep 2009 at 21:58, Antoine Pitrou wrote: >>>> Le mardi 15 septembre 2009 ? 15:48 -0400, R. David Murray a ?crit : >>>>> However, I do not think >>>>> that the proposed API should accept, eg, IPv4Network >>>>> ('192.168.1.1/24') >>>>> as valid. That's just too confusing and error prone. >>>> >>>> Indeed, it should throw some kind of ValueError instead. >>> >>> Peter, what do you think? > > I disagree. It seems overly pedantic to throw a valueerror on this. > IPy does (used to do?) this and it's one of the primary reasons I > wrote ipaddr. I've seen user interfaces accept 192.168.1.1/24 as a short cut to set the ipaddr and netmask on an interface. For that use being able to parse that string into an IP Address and a Net Mask is what they want. If this is a common idiom having a utility function that returned an IPv4Address('192.168.1.1') and IPv4Network('192.168.0.0') would be useful. If someone really thinks that '192.168.1.1/16' is a network any good UI should detect it is invalid report that to the user. If you think this is a decision that the only the caller can decide why not add a keyword param to control the parsing? IPv4Network( '192.168.1.1/16', syntax='loose' ) (I'm sure syntax='loose' is not right but you get the idea.) Barry From barry at barrys-emacs.org Sat Sep 26 14:46:00 2009 From: barry at barrys-emacs.org (Barry Scott) Date: Sat, 26 Sep 2009 13:46:00 +0100 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk Message-ID: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> I'm working with http://svn.python.org/projects/python/trunk on Mac OS X 10.6.1 using Apples xcode gcc 4.2.1. When I run the following commands: ./configure --enable-framework --with-universal-archs=32-bit | tee build.config.log make clean all | tee build.make.log I end up with a x86_64 Python image. No matter what I use for archs its always the same. I would expect to see -arch arg to GCC but it is not there. export CFLAG="-arch i386" did not work either. Am I doing something wrong or is this broken on trunk? Barry From martin at v.loewis.de Sat Sep 26 19:44:45 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 26 Sep 2009 19:44:45 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <4AB17B29.8050708@canterbury.ac.nz> <20090917003027.3AC245AC0D6@longblack.object-craft.com.au> <8517e9350909161758k148cb4far2a7a69197399d5fd@mail.gmail.com> <20090917013219.6A3895AC0D6@longblack.object-craft.com.au> <8517e9350909161919w73cb4fech95eb63013477d47a@mail.gmail.com> <20090917032126.66F896006DE@longblack.object-craft.com.au> <8517e9350909162036t7265e596uf50b329960199e52@mail.gmail.com> <4AB1B2A8.9050204@scottdial.com> Message-ID: <4ABE530D.8090609@v.loewis.de> > - Masks are also 32- (128-) bit integers, which happen to have the > property that their leftmost N bits are all zero and the rest are all > one. As a side note, I would be in favor of dropping the concept of a mask from the library, and only support a prefix length. IPv6 doesn't support masks at all, and even for IPv4, I think there are conventions (if not RFCs) against using them in a way that does not correspond to a prefix length. Regards, Martin From martin at v.loewis.de Sat Sep 26 19:59:25 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 26 Sep 2009 19:59:25 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> Message-ID: <4ABE567D.6080808@v.loewis.de> > I don't think the RFCs forbid the zero address being used RFC 1122 does: "IP addresses are not permitted to have the value 0 or -1 for any of the , , or fields (except in the special cases listed above)." RFC 3021 modifies this requirement, allowing the zero address to be used for an 31-bit prefix. Regards, Martin From martin at v.loewis.de Sat Sep 26 20:19:16 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 26 Sep 2009 20:19:16 +0200 Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library In-Reply-To: <20090916090849.GH12810@ruber.office.udmvt.ru> References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> <4A8C0267.5020408@wildenhain.de> <8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com> <8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com> <4A8C4264.2040708@trueblade.com> <20090916090849.GH12810@ruber.office.udmvt.ru> Message-ID: <4ABE5B24.1090003@v.loewis.de> >>> I think using .network and .broadcast are pretty well understood to be the >>> [0] and [-1] of the network address block. I don't think we want to start >>> creating new terms or access patterns here. >>> >>> +1 on leaving .network and .broadcast as-is (including returning a >>> IPvXAddress object). >>> >> -1. I think 'network.number' or 'network.zero' is a lot clearer than >> 'network.network'. Maybe '.broadcast' would be okay, as long as it *can* be >> adjusted for those unusual, or maybe even only hypothetical, networks where >> it is not the [-1]. > Real life example: network with a /31 mask. > There are only two hosts: 0 and 1 > first host configures the other's host as broadcast address and vice versa. > NOTE - broadcasts are different here! This is RFC 3021. IIUC, it does not support directed broadcast; only link-local broadcast can be used on that link. So ISTM that .broadcast should raise an exception on a /31 network. Any installation that configures the partner as the broadcast address is broken (somebody correct me if I'm wrong). > Another real life examples include /32 networks on PPP. Just a point-to-point. > No need for broadcasts and networks, a host just have one IP address and > send all traffic via point-to-point link, no addressing is required there. > This is a working dialup configuration, it works for me, it works for you, probably. > It is not weird, it is common, it is used for PPPoE, for ADSL, for dialup. So where is that defined? Regards, Martin From drkjam at gmail.com Sat Sep 26 21:07:21 2009 From: drkjam at gmail.com (DrKJam) Date: Sat, 26 Sep 2009 20:07:21 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4ABE567D.6080808@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> <4ABE567D.6080808@v.loewis.de> Message-ID: <538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com> 2009/9/26 "Martin v. L?wis" > > I don't think the RFCs forbid the zero address being used > > > RFC 1122 does: "IP addresses are not permitted to have the value 0 or -1 > for any of the , , or number> fields (except in the special cases listed above)." > > The current version of the PEP and reference implementation do not mention or deal with IPv4 classful addressing (A, B, C, D and E). It would be good to know if any of this (admittedly older yet no less important) functionality is going to be supported. If the library is to concentrate solely on classless addressing (i.e. CIDR) please can this be stated in future revisions of the PEP. > RFC 3021 modifies this requirement, allowing the zero address to > be used for an 31-bit prefix. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/drkjam%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel at stutzbachenterprises.com Sat Sep 26 22:14:24 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 26 Sep 2009 15:14:24 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB03909.3090608@trueblade.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> <4ABE567D.6080808@v.loewis.de> <538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com> Message-ID: On Sat, Sep 26, 2009 at 2:07 PM, DrKJam wrote: > The current version of the PEP and reference implementation do not mention > or deal with IPv4 classful addressing (A, B, C, D and E). It would be good > to know if any of this (admittedly older yet no less important) > functionality is going to be supported. If the library is to concentrate > solely on classless addressing (i.e. CIDR) please can this be stated in > future revisions of the PEP. > Classful addressing was deprecated more than 15 years ago! Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such scenarios are no longer operationally relevant." -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC "The plumage don't enter into it. It's stone dead." -------------- next part -------------- An HTML attachment was scrubbed... URL: From drkjam at gmail.com Sat Sep 26 23:57:00 2009 From: drkjam at gmail.com (DrKJam) Date: Sat, 26 Sep 2009 22:57:00 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151805h1fee4571r8f61402ce86a9ff1@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> <4ABE567D.6080808@v.loewis.de> <538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com> Message-ID: <538a660a0909261457n1a956bc0i1524e58c4e48be4e@mail.gmail.com> 2009/9/26 Daniel Stutzbach > On Sat, Sep 26, 2009 at 2:07 PM, DrKJam wrote: > >> The current version of the PEP and reference implementation do not mention >> or deal with IPv4 classful addressing (A, B, C, D and E). It would be good >> to know if any of this (admittedly older yet no less important) >> functionality is going to be supported. If the library is to concentrate >> solely on classless addressing (i.e. CIDR) please can this be stated in >> future revisions of the PEP. >> > > Classful addressing was deprecated more than 15 years ago! > > Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such > scenarios are no longer operationally relevant." Interesting as evidence of classful IPv4 behaviour seems fairly pervasive in current IP stacks and software that supports IPv4 addressing (e.g. PostgreSQL inet and cidr data types). Here's an excerpt from the 'ifconfig' man page (taken from an Ubuntu 9.04 install) :- ^^^^^^^^^^^^^^^^^^^^^^^ ... netmask addr Set the IP network mask for this interface. This value defaults to the usual class A, B or C network mask (as derived from the interface IP address). but it can be set to any value. ... ^^^^^^^^^^^^^^^^^^^^^^^ The point being that you can't always assume /32 implicitly for all use cases. Here is how IP addresses without an explicit prefix or netmask are currently handled :- >>> import ipaddr >>> ipaddr.IPv4Network('10.0.0.1') IPv4Network('10.0.0.1/32') It may not be something we want to support (you could always force the end user to specify a prefix or netmask explicitly). This is fair enough, but let's indicate that it's a considered choice. Somewhere in the PEP text is might be a good place for treatment of this topic. > -- > Daniel Stutzbach, Ph.D. > President, Stutzbach Enterprises, LLC > "The plumage don't enter into it. It's stone dead." > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From daniel at stutzbachenterprises.com Sun Sep 27 01:27:34 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sat, 26 Sep 2009 18:27:34 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <538a660a0909261457n1a956bc0i1524e58c4e48be4e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909162053.00354.steve@pearwood.info> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> <4ABE567D.6080808@v.loewis.de> <538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com> <538a660a0909261457n1a956bc0i1524e58c4e48be4e@mail.gmail.com> Message-ID: On Sat, Sep 26, 2009 at 4:57 PM, DrKJam wrote: > 2009/9/26 Daniel Stutzbach > >> On Sat, Sep 26, 2009 at 2:07 PM, DrKJam wrote: >> >>> The current version of the PEP and reference implementation do not >>> mention or deal with IPv4 classful addressing (A, B, C, D and E). It would >>> be good to know if any of this (admittedly older yet no less important) >>> functionality is going to be supported. If the library is to concentrate >>> solely on classless addressing (i.e. CIDR) please can this be stated in >>> future revisions of the PEP. >>> >> >> Classful addressing was deprecated more than 15 years ago! >> >> Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such >> scenarios are no longer operationally relevant." > > > Interesting as evidence of classful IPv4 behaviour seems fairly pervasive > in current IP stacks and software that supports IPv4 addressing (e.g. > PostgreSQL inet and cidr data types). > PostgreSQL's inet type stores a host address and optionally a network mask. It's cidr type stores a network address and mask. Neither of them are classful. > Here's an excerpt from the 'ifconfig' man page (taken from an Ubuntu 9.04 > install) :- > netmask addr > > Set the IP network mask for this interface. This value defaults to the > usual class A, B or C network mask (as derived from the interface IP > address). but it can be set to any value. > Since the network mask can be set to any legal value, that means it's stored as CIDR. It falls back on the classful network mask as a default. Undoubtedly that was useful when ifconfig was updated for CIDR. There were still many classful networks and it needed to retain backward compatibility. After all, ifconfig is around 25 years old, and CIDR is only 16. Today, all IP networks are classless (CIDR), and a system administrator would have to do extra work to figure out if the default value happened to be just right for the network they're setting up. > The point being that you can't always assume /32 implicitly for all use > cases. > Certainly, which is why the user can explicitly set the netmask. > Here is how IP addresses without an explicit prefix or netmask are > currently handled :- > > >>> import ipaddr > >>> ipaddr.IPv4Network('10.0.0.1') > IPv4Network('10.0.0.1/32') > > It may not be something we want to support (you could always force the end > user to specify a prefix or netmask explicitly). This is fair enough, but > let's indicate that it's a considered choice. Somewhere in the PEP text is > might be a good place for treatment of this topic. > +1 on forcing the user to specify a netmask explicitly and documenting it. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From peter at hda3.com Sun Sep 27 02:28:49 2009 From: peter at hda3.com (Peter Moody) Date: Sat, 26 Sep 2009 17:28:49 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AB0C7A9.3030001@gmail.com> <4AB15FD3.5040804@canterbury.ac.nz> <20090917004138.2177C5AC0D6@longblack.object-craft.com.au> <4ABE567D.6080808@v.loewis.de> <538a660a0909261207m2bed7a9axe7d556e28d82af8f@mail.gmail.com> <538a660a0909261457n1a956bc0i1524e58c4e48be4e@mail.gmail.com> Message-ID: <8517e9350909261728q2f15ccd6g37103277ebba4b31@mail.gmail.com> On Sat, Sep 26, 2009 at 4:27 PM, Daniel Stutzbach wrote: > On Sat, Sep 26, 2009 at 4:57 PM, DrKJam wrote: >> >> 2009/9/26 Daniel Stutzbach >>> >>> On Sat, Sep 26, 2009 at 2:07 PM, DrKJam wrote: >>>> >>>> The current version of the PEP and reference implementation do not >>>> mention or deal with IPv4 classful addressing (A, B, C, D and E). It would >>>> be good to know if any of this (admittedly older yet no less important) >>>> functionality is going to be supported. If the library is to concentrate >>>> solely on classless addressing (i.e. CIDR) please can this be stated in >>>> future revisions of the PEP. I'll call this out in the PEP if necessary. As mentioned elsewhere, classful addressing is so long deprecated that I would expect it to cause confusion if it was assumed. >>> Classful addressing was deprecated more than 15 years ago! >>> >>> Quoting RFC 4632: "With the full deployment of CIDR on the Internet, such >>> scenarios are no longer operationally relevant." >> >> Interesting as evidence of classful IPv4 behaviour seems fairly pervasive >> in current IP stacks and software that supports IPv4 addressing (e.g. >> PostgreSQL inet and cidr data types). > > PostgreSQL's inet type stores a host address and optionally a network mask. > It's cidr type stores a network address and mask.? Neither of them are > classful. > >> >> Here's an excerpt from the 'ifconfig' man page (taken from an Ubuntu 9.04 >> install) :- >> netmask addr >> >> Set the IP network mask for this interface. This value defaults to the >> usual class A, B or C network mask (as derived from the interface IP >> address). but it can be set to any value. > > Since the network mask can be set to any legal value, that means it's stored > as CIDR.? It falls back on the classful network mask as a default. > Undoubtedly that was useful when ifconfig was updated for CIDR.? There were > still many classful networks and it needed to retain backward > compatibility.? After all, ifconfig is around 25 years old, and CIDR is only > 16. > > Today, all IP networks are classless (CIDR), and a system administrator > would have to do extra work to figure out if the default value happened to > be just right for the network they're setting up. > >> >> ?The point being that you can't always assume /32 implicitly for all use >> cases. > > Certainly, which is why the user can explicitly set the netmask. > >> >> Here is how IP addresses without an explicit prefix or netmask are >> currently handled :- >> >> >>> import ipaddr >> >>> ipaddr.IPv4Network('10.0.0.1') >> IPv4Network('10.0.0.1/32') >> >> It may not be something we want to support (you could always force the end >> user to specify a prefix or netmask explicitly). This is fair enough, but >> let's indicate that it's a considered choice. Somewhere in the PEP text is >> might be a good place for treatment of this topic. > > +1 on forcing the user to specify a netmask explicitly and documenting it. -(1/0) forcing a user to include a prefixlen when creating a network object doesn't actually serve any useful purpose. when dealing with network objects, it's very useful (and common) to assume a /32 prefix length when none is supplied. Cheers, /peter > -- > Daniel Stutzbach, Ph.D. > President, Stutzbach Enterprises, LLC > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > > From steve at pearwood.info Sun Sep 27 03:08:32 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 27 Sep 2009 11:08:32 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4ABE530D.8090609@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABE530D.8090609@v.loewis.de> Message-ID: <200909271108.32456.steve@pearwood.info> On Sun, 27 Sep 2009 03:44:45 am Martin v. L?wis wrote: > > - Masks are also 32- (128-) bit integers, which happen to have the > > property that their leftmost N bits are all zero and the rest > > are all one. > > As a side note, I would be in favor of dropping the concept of a mask > from the library, and only support a prefix length. -1 > IPv6 doesn't support masks at all, and even for IPv4, I think there > are conventions (if not RFCs) against using them in a way that does > not correspond to a prefix length. Then the module should only support netmasks of the form (say) '255.255.255.224' (equivalent to "/27"), and reject those like "255.3.255.255". It currently accepts them. Many applications still display netmasks in dot-quad form, and I would be terribly annoyed if I had to count the bits myself before passing it to IPv4Address. -- Steven D'Aprano From steve at pearwood.info Sun Sep 27 04:05:18 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 27 Sep 2009 12:05:18 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> Message-ID: <200909271205.18488.steve@pearwood.info> On Sat, 26 Sep 2009 11:23:14 pm Barry Scott wrote: > I've seen user interfaces accept 192.168.1.1/24 as a short cut > to set the ipaddr and netmask on an interface. > > For that use being able to parse that string into an IP Address and > a Net Mask is what they want. I think you're at least the second person who has requested this functionality, or something similar. > If this is a common idiom having a utility function that returned > an IPv4Address('192.168.1.1') and IPv4Network('192.168.0.0') > would be useful. > > If someone really thinks that '192.168.1.1/16' is a network any > good UI should detect it is invalid report that to the user. Currently the ipaddr module accepts that without complaint: >>> ipaddr.IPv4Network('192.168.1.1/16') IPv4Network('192.168.1.1/16') The current behaviour is confusing to me. For example: >>> netw1 = ipaddr.IPv4Network('192.168.1.1/24') >>> netw2 = ipaddr.IPv4Network('192.168.1.0/24') >>> netw1 == netw2 False >>> list(netw1) == list(netw2) True Two networks, containing exactly the same range of addresses, but they don't compare equal. I'm not convinced that netw1 should even be allowed, but if it is, surely it should be turned into canonical form netw2? E.g. I would expect this: >>> ipaddr.IPv4Network('192.168.1.1/24') IPv4Network('192.168.1.0/24') but that's not what it does. -- Steven D'Aprano From nad at acm.org Sun Sep 27 04:12:44 2009 From: nad at acm.org (Ned Deily) Date: Sat, 26 Sep 2009 19:12:44 -0700 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> Message-ID: In article <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A at barrys-emacs.org>, Barry Scott wrote: > I'm working with http://svn.python.org/projects/python/trunk on Mac OS > X 10.6.1 > using Apples xcode gcc 4.2.1. > > When I run the following commands: > > ./configure --enable-framework --with-universal-archs=32-bit | tee > build.config.log > make clean all | tee build.make.log > > I end up with a x86_64 Python image. > > No matter what I use for archs its always the same. > > I would expect to see -arch arg to GCC but it is not there. > > export CFLAG="-arch i386" > > did not work either. > > Am I doing something wrong or is this broken on trunk? You need to add the enable-universalsdk parameter to configure: ... --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk Be aware, though, that universal build support on 10.6 is a bit of a work in progress as there are still some interesting unexplained universal build issues when building on Snow Leopard (see, for instance, the comments in http://bugs.python.org/issue6957). At the moment, the focus is on getting 2.6.3 out the door and the standard installer for that will be built on 10.5. -- Ned Deily, nad at acm.org From peter at hda3.com Sun Sep 27 04:22:08 2009 From: peter at hda3.com (Peter Moody) Date: Sat, 26 Sep 2009 19:22:08 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909271205.18488.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> Message-ID: <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> I again invite interested parties to continue this discussion on ipaddr-py-dev at googlegroups.com. we're pushing 250 messages on PEP 3144 at this point; well beyond what most folks would call a "long open-ended discussion". anyway: > The current behaviour is confusing to me. For example: > >>>> netw1 = ipaddr.IPv4Network('192.168.1.1/24') >>>> netw2 = ipaddr.IPv4Network('192.168.1.0/24') >>>> netw1 == netw2 > False >>>> list(netw1) == list(netw2) > True I think you're intentionally confusing yourself here. 192.168.1.1 clearly doesn't equal 192.168.1.0, so I don't know why you'd expect 192.168.1.1/24 to equal 192.168.1.0/24. > Two networks, containing exactly the same range of addresses, but they > don't compare equal. I'm not convinced that netw1 should even be > allowed, but if it is, surely it should be turned into canonical form > netw2? E.g. I would expect this: > >>>> ipaddr.IPv4Network('192.168.1.1/24') > IPv4Network('192.168.1.0/24') > > but that's not what it does. now you're confusing me as before you've asked for this to either: 1. Raise an exception 2. Return two objects. your unconfusing (to you) example does neither one of those. > -- > Steven D'Aprano > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From guido at python.org Sun Sep 27 04:25:32 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 26 Sep 2009 19:25:32 -0700 Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library In-Reply-To: <4ABE5B24.1090003@v.loewis.de> References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> <4A8C0267.5020408@wildenhain.de> <8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com> <8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com> <4A8C4264.2040708@trueblade.com> <20090916090849.GH12810@ruber.office.udmvt.ru> <4ABE5B24.1090003@v.loewis.de> Message-ID: I realize I'm late to this party, but this is just a naming issue, right? For any network, there are two special addresses, one with the last bits all zeros, one with the last bits all ones. We can call them A and B, or network and broadcast, or zeros and ones, or whatever we care. But their definitions are two well-defined functions for all networks (assuming a network is defined as an IP address and a number of bits), even if in an extreme case the functions return the same value. What is actually configured on a particular host to be the broadcast address is a separate issue, even if *by convention* in most cases it is given by one of the above functions -- the network object doesn't care, the configuration object is something else (and outside the scope of this PEP). IMO "real life examples" don't matter for the definitions of the functions, and I would personally be happy to name them network and broadcast, since I know their definitions and their typical uses and these match pretty closely. The expectation should be clearly set that these are pure functions though and do not imply knowledge of the configuration of any given host. --Guido On Sat, Sep 26, 2009 at 11:19 AM, "Martin v. L?wis" wrote: >>>> I think using .network and .broadcast are pretty well understood to be the >>>> [0] and [-1] of the network address block. I don't think we want to start >>>> creating new terms or access patterns here. >>>> >>>> +1 on leaving .network and .broadcast as-is (including returning a >>>> IPvXAddress object). >>>> >>> -1. ?I think 'network.number' or 'network.zero' is a lot clearer than >>> 'network.network'. ?Maybe '.broadcast' would be okay, as long as it *can* be >>> adjusted for those unusual, or maybe even only hypothetical, networks where >>> it is not the [-1]. >> Real life example: network with a /31 mask. >> There are only two hosts: 0 and 1 >> first host configures the other's host as broadcast address and vice versa. >> NOTE - broadcasts are different here! > > This is RFC 3021. IIUC, it does not support directed broadcast; only > link-local broadcast can be used on that link. > > So ISTM that .broadcast should raise an exception on a /31 network. Any > installation that configures the partner as the broadcast address is > broken (somebody correct me if I'm wrong). > >> Another real life examples include /32 networks on PPP. Just a point-to-point. >> No need for broadcasts and networks, a host just have one IP address and >> send all traffic via point-to-point link, no addressing is required there. >> This is a working dialup configuration, it works for me, it works for you, probably. >> It is not weird, it is common, it is used for PPPoE, for ADSL, for dialup. > > So where is that defined? > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Sun Sep 27 05:50:08 2009 From: aahz at pythoncraft.com (Aahz) Date: Sat, 26 Sep 2009 20:50:08 -0700 Subject: [Python-Dev] LAST CHANCE: PyCon 2010: Call for Proposals Message-ID: <20090927035008.GA28270@panix.com> Just four more days to propose a presentation! Call for proposals -- PyCon 2010 -- =============================================================== Due date: October 1st, 2009 Want to showcase your skills as a Python Hacker? Want to have hundreds of people see your talk on the subject of your choice? Have some hot button issue you think the community needs to address, or have some package, code or project you simply love talking about? Want to launch your master plan to take over the world with python? PyCon is your platform for getting the word out and teaching something new to hundreds of people, face to face. Previous PyCon conferences have had a broad range of presentations, from reports on academic and commercial projects, tutorials on a broad range of subjects and case studies. All conference speakers are volunteers and come from a myriad of backgrounds. Some are new speakers, some are old speakers. Everyone is welcome so bring your passion and your code! We're looking to you to help us top the previous years of success PyCon has had. PyCon 2010 is looking for proposals to fill the formal presentation tracks. The PyCon conference days will be February 19-22, 2010 in Atlanta, Georgia, preceded by the tutorial days (February 17-18), and followed by four days of development sprints (February 22-25). Online proposal submission is open now! Proposals will be accepted through October 1st, with acceptance notifications coming out on November 15th. For the detailed call for proposals, please see: For videos of talks from previous years - check out: We look forward to seeing you in Atlanta! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ "Look, it's your affair if you want to play with five people, but don't go calling it doubles." --John Cleese anticipates Usenet From ncoghlan at gmail.com Sun Sep 27 07:38:37 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 27 Sep 2009 15:38:37 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> Message-ID: <4ABEFA5D.40506@gmail.com> Peter Moody wrote: > I again invite interested parties to continue this discussion on > ipaddr-py-dev at googlegroups.com. we're pushing 250 messages on PEP > 3144 at this point; well beyond what most folks would call a "long > open-ended discussion". > > anyway: > >> The current behaviour is confusing to me. For example: >> >>>>> netw1 = ipaddr.IPv4Network('192.168.1.1/24') >>>>> netw2 = ipaddr.IPv4Network('192.168.1.0/24') >>>>> netw1 == netw2 >> False >>>>> list(netw1) == list(netw2) >> True > > I think you're intentionally confusing yourself here. 192.168.1.1 > clearly doesn't equal 192.168.1.0, so I don't know why you'd expect > 192.168.1.1/24 to equal 192.168.1.0/24. No, this is exactly the "dual purpose" nature of IPNetwork that people have been complaining about: 192.168.1.1 & 255.255.255.0 == 192.168.1.0 & 255.255.255.0, so the two objects describe the exact same network, but the network objects don't currently respect their own netmask, instead remembering which particular host happened to be used to derive the network ID. "Arbitrary network host with associated netmask" and "IP Network" are different concepts, but the library currently uses a single class interface (IPNetwork) for both of them. What we're asking for is for the "ip" attribute to be removed from IP Network, with the constructor either forgetting the host IP that was used to derive the network address, or else only accepting the correct network address in the constructor and providing a separate helper function or class method to create an IPNetwork object from an arbitrary hostname. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From peter at hda3.com Sun Sep 27 07:54:21 2009 From: peter at hda3.com (Peter Moody) Date: Sat, 26 Sep 2009 22:54:21 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4ABEFA5D.40506@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> Message-ID: <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> On Sat, Sep 26, 2009 at 10:38 PM, Nick Coghlan wrote: > Peter Moody wrote: >> I again invite interested parties to continue this discussion on >> ipaddr-py-dev at googlegroups.com. ?we're pushing 250 messages on PEP >> 3144 at this point; well beyond what most folks would call a "long >> open-ended discussion". >> >> anyway: >> >>> The current behaviour is confusing to me. For example: >>> >>>>>> netw1 = ipaddr.IPv4Network('192.168.1.1/24') >>>>>> netw2 = ipaddr.IPv4Network('192.168.1.0/24') >>>>>> netw1 == netw2 >>> False >>>>>> list(netw1) == list(netw2) >>> True >> >> I think you're intentionally confusing yourself here. ?192.168.1.1 >> clearly doesn't equal 192.168.1.0, so I don't know why you'd expect >> 192.168.1.1/24 to equal 192.168.1.0/24. > > No, this is exactly the "dual purpose" nature of IPNetwork that people > have been complaining about: 192.168.1.1 & 255.255.255.0 == 192.168.1.0 > & 255.255.255.0, so the two objects describe the exact same network, but > the network objects don't currently respect their own netmask, instead > remembering which particular host happened to be used to derive the > network ID. > > "Arbitrary network host with associated netmask" and "IP Network" are > different concepts, but the library currently uses a single class > interface (IPNetwork) for both of them. > > What we're asking for is for the "ip" attribute to be removed from IP > Network, with the constructor either forgetting the host IP that was > used to derive the network address, or else only accepting the correct > network address in the constructor and providing a separate helper > function or class method to create an IPNetwork object from an arbitrary > hostname. what you want is strict=True, just checked in. I've been meaning to send an update to the PEP explaining why this choice was made. I'm hoping this will ameliorate your confusion. Cheers, /peter > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > From ncoghlan at gmail.com Sun Sep 27 08:08:30 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 27 Sep 2009 16:08:30 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> Message-ID: <4ABF015E.5060002@gmail.com> Peter Moody wrote: > what you want is strict=True, just checked in. I've been meaning to > send an update to the PEP explaining why this choice was made. I'm > hoping this will ameliorate your confusion. If the ip property remains and equality is still broken as Steven describes, then no, it won't really alleviate my concerns. I've also previously registered my objections to using a boolean flag for this instead of a separate method. However, I'll wait and see what the PEP update says before commenting further. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Sun Sep 27 08:56:40 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 27 Sep 2009 08:56:40 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909271108.32456.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABE530D.8090609@v.loewis.de> <200909271108.32456.steve@pearwood.info> Message-ID: <4ABF0CA8.2030408@v.loewis.de> >> As a side note, I would be in favor of dropping the concept of a mask >> from the library, and only support a prefix length. > > -1 > >> IPv6 doesn't support masks at all, and even for IPv4, I think there >> are conventions (if not RFCs) against using them in a way that does >> not correspond to a prefix length. > > Then the module should only support netmasks of the form > (say) '255.255.255.224' (equivalent to "/27"), and reject those > like "255.3.255.255". It currently accepts them. > > Many applications still display netmasks in dot-quad form, and I would > be terribly annoyed if I had to count the bits myself before passing it > to IPv4Address. I wouldn't ask for that: it should certainly be possible to supply masks. However, I would want to reject masks that don't correspond to a prefix, and have only the prefix length in the internal representation. Regards, Martin From martin at v.loewis.de Sun Sep 27 09:08:39 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 27 Sep 2009 09:08:39 +0200 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: References: Message-ID: <4ABF0F77.8050209@v.loewis.de> > One thing I'd like to see in the list are real names of committers, > not tracker names. Of course, people looking for people to assign a bug > to should not have to search for the tracker name, so I'd like to make > another request (that Brett already made when we switched trackers): > > Could we *please* have tracker names that match the committer names? I'm opposed. I never type my committer name - I don't even know what it is (martin.v.loewis, martin.vonloewis, martin.von.loewis, something else?). In the tracker, I have to type my account name, so I need to remember. I have been using "loewis" as my account name in the net for nearly two decades now, so I have little interest in starting to have different account names, now. > (This doesn't even need to be done by the individual users, I would > volunteer to rename all committer accounts and notify them by email that > their name changed. This will also be coordinated with the new names > used for Mercurial commits, if a change will be made.) IIUC, there will be relatively little control over Mercurial committer names. Every user will have to configure ~/.hgrc, which should list a single name/email address pair, to be used for all Mercurial repositories. So I put "Martin v. L?wis " into .hgrc, which is my name and my email address. It's not the same as the tracker name or the subversion committer name; it can't possibly be (since it comes from an entirely different namespace). Regards, Martin From barry at barrys-emacs.org Sat Sep 26 15:34:27 2009 From: barry at barrys-emacs.org (Barry Scott) Date: Sat, 26 Sep 2009 14:34:27 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: On 14 Sep 2009, at 17:44, Peter Moody wrote: > Folks, Guido, > > I believe PEP 3144 is ready for your review. When you get a chance, > can you take a look/make a pronouncement? > >>> x=ipaddr.IPv4Network( '192.168.1.1/240.255.0.0' ) >>> x IPv4Network('192.168.1.1/16') You can decide on which bug this shows. 1) parse must raise exception on bad input 2) repr does not show input Barry From dickinsm at gmail.com Sun Sep 27 09:21:16 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 27 Sep 2009 08:21:16 +0100 Subject: [Python-Dev] Python build question (fixing pymath.c). Message-ID: <5c6f2a5d0909270021n20438083yaf3f854e8dd96c6d@mail.gmail.com> Hello all, I'm looking for advice on how to clean up an ugliness (one that I'm at least partly responsible for) in the Python build setup. Here's the problem: some of the exported functions (e.g. atanh, log1p) in the Python/pymath.c file aren't needed by the Python core at all; they're exported solely for use in the math and cmath modules. Even worse, these exported functions have no '_Py' or 'Py' prefix. Since I'm currently working on adding some oft-requested functions to the math module (gamma, lgamma, erf, ...) it seemed like a good time to clean this up. So I've now got a file Modules/math_support.c that contains some functions needed by both mathmodule.c and cmathmodule.c, as well as a couple of functions only currently needed by the math module. How should I incorporate this file into the build? One obvious solution seems to be to build an extra math_support.so file that's then used by both mathmodule.so and cmathmodule.so. Is this a sensible solution? Are there better ways? A complication to bear in mind is that some users may want to alter Modules/Setup.dist so that either the math module and/or the cmath module is included in the core Python executable. Cluelessly yours, Mark From peter at hda3.com Sun Sep 27 09:34:42 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 00:34:42 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: <8517e9350909270034k76f88da3pe1bdf52c576001bb@mail.gmail.com> On Sat, Sep 26, 2009 at 6:34 AM, Barry Scott wrote: > > On 14 Sep 2009, at 17:44, Peter Moody wrote: > >> Folks, Guido, >> >> I believe PEP 3144 is ready for your review. ?When you get a chance, >> can you take a look/make a pronouncement? >> > > >>>> x=ipaddr.IPv4Network( '192.168.1.1/240.255.0.0' ) >>>> x > IPv4Network('192.168.1.1/16') > > You can decide on which bug this shows. > > 1) parse must raise exception on bad input > 2) repr does not show input > > Barry thank you, Barry, for your continued interest. Since it may have been lost in the 200+ messages on this topic, issues with the current implementation can be filed at: http://code.google.com/p/ipaddr-py/issues/entry In case you missed it, I'm trying to move discussion of the PEP over to ipaddr-py-dev at googlegroups.com b/c pep-0001 is fairly explicit about trying to avoid long open-ended discussions. I'm pretty sure bug reports like this at the end of 200+ messages qualifies as long and open-ended. Cheers, /peter > From ncoghlan at gmail.com Sun Sep 27 09:39:38 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 27 Sep 2009 17:39:38 +1000 Subject: [Python-Dev] Misc/maintainers.rst In-Reply-To: <4ABF0F77.8050209@v.loewis.de> References: <4ABF0F77.8050209@v.loewis.de> Message-ID: <4ABF16BA.4020306@gmail.com> Martin v. L?wis wrote: >> One thing I'd like to see in the list are real names of committers, >> not tracker names. Of course, people looking for people to assign a bug >> to should not have to search for the tracker name, so I'd like to make >> another request (that Brett already made when we switched trackers): >> >> Could we *please* have tracker names that match the committer names? > > I'm opposed. I never type my committer name - I don't even know what it > is (martin.v.loewis, martin.vonloewis, martin.von.loewis, something > else?). In the tracker, I have to type my account name, so I need to > remember. I have been using "loewis" as my account name in the net for > nearly two decades now, so I have little interest in starting to have > different account names, now. Same - my tracker is my normal online account name (ncoghlan), while my committer name is my full name (nick.coghlan). Having roundup store an extra field with committer names (and letting people see both names in the user lists on the tracker) seems like a better option than asking us to change our account names. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Sun Sep 27 09:41:28 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 27 Sep 2009 09:41:28 +0200 Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library In-Reply-To: References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> <4A8C0267.5020408@wildenhain.de> <8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com> <8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com> <4A8C4264.2040708@trueblade.com> <20090916090849.GH12810@ruber.office.udmvt.ru> <4ABE5B24.1090003@v.loewis.de> Message-ID: <4ABF1728.9@v.loewis.de> Guido van Rossum wrote: > I realize I'm late to this party, but this is just a naming issue, right? Not really (AFAIU). > For any network, there are two special addresses, one with the last > bits all zeros, one with the last bits all ones. We can call them A > and B, or network and broadcast, or zeros and ones, or whatever we > care. But their definitions are two well-defined functions for all > networks (assuming a network is defined as an IP address and a number > of bits), even if in an extreme case the functions return the same > value. For the broadcast address, it's different, since you might also use it in programming (i.e. when sending broadcasts). Now, it seems that people will typically use undirected broadcasts (INADDR_BROADCAST). However, I assume there are also applications for directed broadcasts; in this case, it would matter to know the broadcast address of a network. > What is actually configured on a particular host to be the broadcast > address is a separate issue, even if *by convention* in most cases it > is given by one of the above functions -- the network object doesn't > care, the configuration object is something else (and outside the > scope of this PEP). It's a little bit stronger than convention; there are RFCs saying that the all-ones (-1) address has to be the broadcast address. This is like saying that it is out of scope of the socket protocol whether TCP is the IP protocol 6, and that individual hosts may do it differently - yes, they may, but then everything stops working. > IMO "real life examples" don't matter for the definitions of the > functions That I agree with. > and I would personally be happy to name them network and > broadcast, since I know their definitions and their typical uses and > these match pretty closely. The expectation should be clearly set that > these are pure functions though and do not imply knowledge of the > configuration of any given host. That I also agree with. However, there is this issue of /31 networks, where an 8-year-old RFC specifies how to do broadcasts on these. I think RFCs should be considered wherever applicable. Regards, Martin From brett at python.org Sun Sep 27 09:48:13 2009 From: brett at python.org (Brett Cannon) Date: Sun, 27 Sep 2009 00:48:13 -0700 Subject: [Python-Dev] Python build question (fixing pymath.c). In-Reply-To: <5c6f2a5d0909270021n20438083yaf3f854e8dd96c6d@mail.gmail.com> References: <5c6f2a5d0909270021n20438083yaf3f854e8dd96c6d@mail.gmail.com> Message-ID: On Sun, Sep 27, 2009 at 00:21, Mark Dickinson wrote: > Hello all, > > I'm looking for advice on how to clean up an ugliness (one > that I'm at least partly responsible for) in the Python build setup. > > Here's the problem: ?some of the exported functions (e.g. atanh, > log1p) in the Python/pymath.c file aren't needed by the Python core > at all; they're exported solely for use in the math and cmath modules. > Even worse, these exported functions have no '_Py' or 'Py' prefix. > > Since I'm currently working on adding some oft-requested > functions to the math module (gamma, lgamma, erf, ...) it seemed > like a good time to clean this up. > > So I've now got a file Modules/math_support.c that contains > some functions needed by both mathmodule.c and > cmathmodule.c, as well as a couple of functions only > currently needed by the math module. ?How should I incorporate > this file into the build? > > One obvious solution seems to be to build an extra math_support.so > file that's then used by both mathmodule.so and cmathmodule.so. > Is this a sensible solution? ?Are there better ways? > Are you planning on exposing any of this outside of those two modules? If not then I would change the name to _math.so > A complication to bear in mind is that some users may want to > alter Modules/Setup.dist so that either the math module and/or > the cmath module is included in the core Python executable. If you are mucking with Modules.Setup.dist you better know how to figure out that those two modules depend on another .so. -Brett From peter at hda3.com Sun Sep 27 09:49:41 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 00:49:41 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4ABF015E.5060002@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> Message-ID: <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> On Sat, Sep 26, 2009 at 11:08 PM, Nick Coghlan wrote: > Peter Moody wrote: >> what you want is strict=True, just checked in. I've been meaning to >> send an update to the PEP explaining why this choice was made. I'm >> hoping this will ameliorate your confusion. > > If the ip property remains and equality is still broken as Steven > describes, then no, it won't really alleviate my concerns. > > I've also previously registered my objections to using a boolean flag > for this instead of a separate method. > > However, I'll wait and see what the PEP update says before commenting > further. I've mailed the PEP patch in, hopefully it will be checked in and updated shortly. To be explicit though, unless I'm drastically misreading what you're asking for, I consider the design you're proposing to be broken from a usability perspective (striving for academic/pedantic correctness over accepted practice). I have no intention of adding any sort of AddressWithNetmask classes or making other fundamental design changes you've suggested. If what the community requires is the library you've described, then ipaddr is not that library. Cheers, /peter > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > From chambon.pascal at gmail.com Sun Sep 27 09:58:51 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sun, 27 Sep 2009 09:58:51 +0200 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition V2 In-Reply-To: <4AB67894.5020903@mrabarnett.plus.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> Message-ID: <4ABF1B3B.4080106@gmail.com> Hello Below is a corrected version of the PEP update, adding the start/end indexes proposition and fixing functions signatures. Does anyone disagree with these specifications ? Or can we consider it as a target for the next versions of the io module ? I would have no problem to implement this behaviour in my own pure python FileIO system, however if someone is willing to patch the _fileio implementation, it'd save a lot of time - I most probably won't have the means to setup a C compilation environment under windows and linux, and properly update/test this, before January (when I get freelance...). I launch another thread on other to-be-discussed IO points B-) Regards, Pascal ================ PEP UPDATE for new I/O system - v2 =========== **Truncate and file pointer semantics** Rationale : The current implementation of truncate() always move the file pointer to the new end of file. This behaviour is interesting for compatibility, if the file has been reduced and the file pointer is now past its end, since some platforms might require 0 <= filepointer <= filesize. However, there are several arguments against this semantic: * Most common standards (posix, win32?) allow the file pointer to be past the end of file, and define the behaviour of other stream methods in this case * In many cases, moving the filepointer when truncating has no reasons to happen (if we?re extending the file, or reducing it without going beneath the file pointer) * Making 0 <= filepointer <= filesize a global rule of the python IO module doesn?t seems possible, since it would require modifications of the semantic of other methods (eg. seek() should raise exceptions or silently disobey when asked to move the filepointer past the end of file), and lead to incoherent situations when concurrently accessing files without locking (what if another process truncates to 0 bytes the file you?re writing ?) So here is the proposed semantic, which matches established conventions: *IOBase.truncate(n: int = None) -> int* Resizes the file to the size specified by the positive integer n, or by the current filepointer position if n is None. The file must be opened with write permissions. If the file was previously larger than size, the extra data is discarded. If the file was previously shorter than size, its size is increased, and the extended area appears as if it were zero-filled. In any case, the file pointer is left unchanged, and may point beyond the end of file. Note: trying to read past the end of file returns an empty string, and trying to write past the end of file extends it by zero-ing the gap. On rare platforms which don't support file pointers to be beyond the end of file, all these behaviours shall be faked thanks to internal storage of the "wanted" file pointer position (silently extending the file, if necessary, when a write operation occurs). *Propositions of doc update* *RawIOBase*.read(n: int) -> bytes Read up to n bytes from the object and return them. Fewer than n bytes may be returned if the operating system call returns fewer than n bytes. If 0 bytes are returned, and n was not 0, this indicates end of file. If the object is in non-blocking mode and no bytes are available, the call returns None. *RawIOBase*.readinto(b: bytearray, [start: int = None], [end: int = None]) -> int start and end are used as slice indexes, so that the bytearray taken into account is actually range = b[start:end] (or b[start:], b[:end] or b[:], depending on the arguments which are not None). Read up to len(range) bytes from the object and store them in b, returning the number of bytes read. Like .read, fewer than len(range) bytes may be read, and 0 indicates end of file if len(range) is not 0. None is returned if a non-blocking object has no bytes available. The length of b is never changed. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Sun Sep 27 10:00:02 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 27 Sep 2009 09:00:02 +0100 Subject: [Python-Dev] Python build question (fixing pymath.c). In-Reply-To: References: <5c6f2a5d0909270021n20438083yaf3f854e8dd96c6d@mail.gmail.com> Message-ID: <5c6f2a5d0909270100m7199a1f4r84f3be9e62455509@mail.gmail.com> On Sun, Sep 27, 2009 at 8:48 AM, Brett Cannon wrote: > On Sun, Sep 27, 2009 at 00:21, Mark Dickinson wrote: [...] >> So I've now got a file Modules/math_support.c that contains >> some functions needed by both mathmodule.c and >> cmathmodule.c, as well as a couple of functions only >> currently needed by the math module. ?How should I incorporate >> this file into the build? >> >> One obvious solution seems to be to build an extra math_support.so >> file that's then used by both mathmodule.so and cmathmodule.so. >> Is this a sensible solution? ?Are there better ways? >> > > Are you planning on exposing any of this outside of those two modules? No. > If not then I would change the name to _math.so That makes sense. >> A complication to bear in mind is that some users may want to >> alter Modules/Setup.dist so that either the math module and/or >> the cmath module is included in the core Python executable. > > If you are mucking with Modules.Setup.dist you better know how to > figure out that those two modules depend on another .so. Sure. I'll at least add a comment pointing out the dependence, though. Thanks, Mark From barry at barrys-emacs.org Sun Sep 27 10:07:47 2009 From: barry at barrys-emacs.org (Barry Scott) Date: Sun, 27 Sep 2009 09:07:47 +0100 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk In-Reply-To: References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> Message-ID: <3400FFB0-8A1D-4C16-9D7A-8664924CCF14@barrys-emacs.org> On 27 Sep 2009, at 03:12, Ned Deily wrote: > In article <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A at barrys-emacs.org>, > Barry Scott wrote: >> I'm working with http://svn.python.org/projects/python/trunk on Mac >> OS >> X 10.6.1 >> using Apples xcode gcc 4.2.1. >> >> When I run the following commands: >> >> ./configure --enable-framework --with-universal-archs=32-bit | tee >> build.config.log >> make clean all | tee build.make.log >> >> I end up with a x86_64 Python image. >> >> No matter what I use for archs its always the same. >> >> I would expect to see -arch arg to GCC but it is not there. >> >> export CFLAG="-arch i386" I should have used CFLAGS... >> >> did not work either. >> >> Am I doing something wrong or is this broken on trunk? > > You need to add the enable-universalsdk parameter to configure: > > ... --enable-universalsdk=/Developer/SDKs/MacOSX10.6.sdk Thanks that worked with the 10.5 sdk. > > Be aware, though, that universal build support on 10.6 is a bit of a > work in progress as there are still some interesting unexplained > universal build issues when building on Snow Leopard (see, for > instance, > the comments in http://bugs.python.org/issue6957). At the moment, the > focus is on getting 2.6.3 out the door and the standard installer for > that will be built on 10.5. O.k. Barry From chambon.pascal at gmail.com Sun Sep 27 10:20:23 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Sun, 27 Sep 2009 10:20:23 +0200 Subject: [Python-Dev] IO module precisions and exception hierarchy Message-ID: <4ABF2047.7050501@gmail.com> Found in current io PEP : Q: Do we want to mandate in the specification that switching between reading and writing on a read-write object implies a .flush()? Or is that an implementation convenience that users should not rely on? -> it seems that the only important matter is : file pointer positions and bytes/characters read should always be the ones that the user expects, as if there were no buffering. So flushing or not may stay a non-mandatory behaviour, as long as the buffered streams ensures this data integrity. Eg. If a user opens a file in r/w mode, writes two bytes in it (which stay buffered), and then reads 2 bytes, the two bytes read should be those on range [2:4] of course, even though the file pointer would, due to python buffering, still be at index 0. Q from me : What happens in read/write text files, when overwriting a three-bytes character with a single-byte character ? Or at the contrary, when a single chinese character overrides 3 ASCII characters in an UTF8 file ? Is there any system designed to avoid this data corruption ? Or should TextIO classes forbid read+write streams ? IO Exceptions : Currently, the situation is kind of fuzzy around EnvironmentError subclasses. * OSError represents errors notified by the OS via errno.h error codes (as mirrored in the python "errno" module). errno.h errors (less than 125 error codes) seem to represent the whole of *nix system errors. However, Windows has many more system errors (15000+). So windows errors, when they can't be mapped to one of the errno errors are raises as "WindowsError" instances (a subclass of OSError), with the special attribute "winerror" indicating that win32 error code. * IOError are "errors raised because of I/O problems", but they use errno codes, like OSError. Thus, at the moment IOErrors rather have the semantic of "particular case of OSError", and it's kind of confusing to have them remain in their own separate tree... Furthermore, OSErrors are often used where IOErrors would perfectly fit, eg. in low level I/O functions of the OS module. Since OSErrors and IOErrors are slightly mixed up when we deal with IO operations, maybe the easiest way to make it clearer would be to push to their limits already existing designs. - the os module should only raise OSErrors, whatever the os operation involved (maybe it's already the case in CPython, isn't it ?) - the io module should only raise IOErrors and its subclasses, so that davs can easily take measures depending on the cause of the io failure (except 1 OSError exception, it's already the case in _fileio) - other modules refering to i/o might maybe keep their current (fuzzy) behaviour, since they're more platform specific, and should in the end be replaced by a crossplatform solution (at least I'd love it to happen) Until there, there would be no real benefits for the user, compared to catching EnvironmentErrors as most probably do. But the sweet thing would be to offer a concise but meaningfull IOError hierarchy, so that we can easily handle most specific errors gracefully (having a disk full is not the same level of gravity as simply having another process locking your target file). Here is a very rough beginning of IOError hierarchy. I'd liek to have people's opinion on the relevance of these, as well as on what other exceptions should be distinguished from basic IOErrors. IOError +-InvalidStreamError (eg. we try to write on a stream opened in readonly mode) +-LockingError +-PermissionError (mostly *nix chmod stuffs) +-FileNotFoundError +-DiskFullError +-MaxFileSizeError (maybe hard to implement, happens when we exceed 4Gb on fat32 and stuffs...) +-InvalidFileNameError (filepath max lengths, or "? / : " characters in a windows file name...) Regards, Pascal From drkjam at gmail.com Sun Sep 27 11:06:02 2009 From: drkjam at gmail.com (David Moss) Date: Sun, 27 Sep 2009 10:06:02 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4ABF0CA8.2030408@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABE530D.8090609@v.loewis.de> <200909271108.32456.steve@pearwood.info> <4ABF0CA8.2030408@v.loewis.de> Message-ID: <9118C602-C5D3-49A9-B07F-CE61723D4D6A@gmail.com> Dave M. On 27 Sep 2009, at 07:56, "Martin v. L?wis" wrote: >>> As a side note, I would be in favor of dropping the concept of a >>> mask >>> from the library, and only support a prefix length. >> >> -1 >> >>> IPv6 doesn't support masks at all, and even for IPv4, I think there >>> are conventions (if not RFCs) against using them in a way that does >>> not correspond to a prefix length. >> >> Then the module should only support netmasks of the form >> (say) '255.255.255.224' (equivalent to "/27"), and reject those >> like "255.3.255.255". It currently accepts them. >> >> Many applications still display netmasks in dot-quad form, and I >> would >> be terribly annoyed if I had to count the bits myself before >> passing it >> to IPv4Address. > > I wouldn't ask for that: it should certainly be possible to supply > masks. However, I would want to reject masks that don't correspond to > a prefix, and have only the prefix length in the internal > representation. > +1 on rejection of netmasks without direct CIDR prefix equivalents. AFAIK Cisco routers accept them but I don't see how they would be useful in practice (unless someone can demonstrate their need for this). > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/drkjam%40gmail.com From greg.ewing at canterbury.ac.nz Sun Sep 27 13:16:38 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 27 Sep 2009 23:16:38 +1200 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4ABF2047.7050501@gmail.com> References: <4ABF2047.7050501@gmail.com> Message-ID: <4ABF4996.6000900@canterbury.ac.nz> Pascal Chambon wrote: > -> it seems that the only important matter is : file pointer positions > and bytes/characters read should always be the ones that the user > expects, as if there were no buffering. That sounds right to me. > Q from me : What happens in read/write text files, when overwriting a > three-bytes character with a single-byte character ? I think you deserve whatever you get. If you want to be able to overwrite things that accurately, you should be dealing with the stream at the byte level. > Here is a very rough beginning of IOError hierarchy. > +-InvalidFileNameError (filepath max lengths, or "? / : " characters in > a windows file name...) This might be a bit too precise. Unix just has EINVAL, which covers any kind of invalid parameter, not just file names. -- Greg From solipsis at pitrou.net Sun Sep 27 13:23:43 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 11:23:43 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > To be explicit though, unless I'm drastically misreading what you're > asking for, I consider the design you're proposing to be broken from a > usability perspective (striving for academic/pedantic correctness over > accepted practice). It is certainly not academic correctness, it is concern over the understandibility of the library. The behaviour you are proposing is confusing, and even if you think it is more practical (which I doubt it is), it should still be avoided IMO. While "192.168.1.2/24" may be an accepted notation in some situations, it is not a notation for a *network* but for an (address, mask) pair, or an (address, network) pair. There was a proposal to have a separate parse_address_and_mask method which would return a (Address, Network) tuple, I still don't know why you don't seem to consider it seriously, rather than trying to make the Network class a kind of all-in-one type conflating different concepts. Regards Antoine. From fuzzyman at voidspace.org.uk Sun Sep 27 13:31:46 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 27 Sep 2009 12:31:46 +0100 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4ABF2047.7050501@gmail.com> References: <4ABF2047.7050501@gmail.com> Message-ID: <4ABF4D22.4090201@voidspace.org.uk> Pascal Chambon wrote: > Found in current io PEP : > Q: Do we want to mandate in the specification that switching between > reading and writing on a read-write object implies a .flush()? Or is > that an implementation convenience that users should not rely on? > -> it seems that the only important matter is : file pointer positions > and bytes/characters read should always be the ones that the user > expects, as if there > were no buffering. So flushing or not may stay a non-mandatory > behaviour, as long as the buffered streams ensures this data integrity. > Eg. If a user opens a file in r/w mode, writes two bytes in it (which > stay buffered), and then reads 2 bytes, the two bytes read should be > those on range [2:4] of course, even though the file pointer would, > due to python buffering, still be at index 0. > > > Q from me : What happens in read/write text files, when overwriting a > three-bytes character with a single-byte character ? Or at the > contrary, when a single chinese character overrides 3 ASCII characters > in an UTF8 file ? Is there any system designed to avoid this data > corruption ? Or should TextIO classes forbid read+write streams ? > > > IO Exceptions : > Currently, the situation is kind of fuzzy around EnvironmentError > subclasses. > * OSError represents errors notified by the OS via errno.h error codes > (as mirrored in the python "errno" module). > errno.h errors (less than 125 error codes) seem to represent the whole > of *nix system errors. However, Windows has many more system errors > (15000+). So windows errors, when they can't be mapped to one of the > errno errors are raises as "WindowsError" instances (a subclass of > OSError), with the special attribute "winerror" indicating that win32 > error code. > * IOError are "errors raised because of I/O problems", but they use > errno codes, like OSError. > > Thus, at the moment IOErrors rather have the semantic of "particular > case of OSError", and it's kind of confusing to have them remain in > their own separate tree... Furthermore, OSErrors are often used where > IOErrors would perfectly fit, eg. in low level I/O functions of the OS > module. > Since OSErrors and IOErrors are slightly mixed up when we deal with IO > operations, maybe the easiest way to make it clearer would be to push > to their limits already existing designs. > > - the os module should only raise OSErrors, whatever the os operation > involved (maybe it's already the case in CPython, isn't it ?) > - the io module should only raise IOErrors and its subclasses, so that > davs can easily take measures depending on the cause of the io failure > (except 1 OSError exception, it's already the case in _fileio) > - other modules refering to i/o might maybe keep their current (fuzzy) > behaviour, since they're more platform specific, and should in the end > be replaced by a crossplatform solution (at least I'd love it to happen) > > Until there, there would be no real benefits for the user, compared to > catching EnvironmentErrors as most probably do. But the sweet thing > would be to offer a concise but meaningfull IOError hierarchy, so that > we can easily handle most specific errors gracefully (having a disk > full is not the same level of gravity as simply having another process > locking your target file). > > Here is a very rough beginning of IOError hierarchy. I'd liek to have > people's opinion on the relevance of these, as well as on what other > exceptions should be distinguished from basic IOErrors. > > IOError > +-InvalidStreamError (eg. we try to write on a stream opened in > readonly mode) > +-LockingError > +-PermissionError (mostly *nix chmod stuffs) > +-FileNotFoundError > +-DiskFullError > +-MaxFileSizeError (maybe hard to implement, happens when we exceed > 4Gb on fat32 and stuffs...) > +-InvalidFileNameError (filepath max lengths, or "? / : " characters > in a windows file name...) Personally I'd love to see a richer set of exceptions for IO errors, so long as they can be implemented for all supported platforms and no information (err number from the os) is lost. I've been implementing a fake 'file' type [1] for Silverlight which does IO operations using local browser storage. The use case is for an online Python tutorial running in the browser [2]. Whilst implementing the exception behaviour (writing to a file open in read mode, etc) I considered improving the exception messages as they are very poor - but decided that being similar to CPython was more important. Michael [1] http://code.google.com/p/trypython/source/browse/trunk/trypython/app/storage.py and http://code.google.com/p/trypython/source/browse/trunk/trypython/app/tests/test_storage.py [2] http://www.trypython.org/ > > Regards, > Pascal > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From solipsis at pitrou.net Sun Sep 27 13:33:03 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 11:33:03 +0000 (UTC) Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition V2 References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> <4ABF1B3B.4080106@gmail.com> Message-ID: Hello, > So here is the proposed semantic, which matches established conventions: > > *IOBase.truncate(n: int = None) -> int* [...] I still don't think there is a sufficient benefit in breaking compatibility. If you want the file pointer to remain the same, you can save it first and restore it afterwards manually. > *Propositions of doc update* Please open tracker issues for these kinds of suggestions. Regards Antoine. From solipsis at pitrou.net Sun Sep 27 13:49:06 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 11:49:06 +0000 (UTC) Subject: [Python-Dev] IO module precisions and exception hierarchy References: <4ABF2047.7050501@gmail.com> Message-ID: Le Sun, 27 Sep 2009 10:20:23 +0200, Pascal Chambon a ?crit?: > Q: Do we want to mandate in the specification that switching between > reading and writing on a read-write object implies a .flush()? It doesn't and shouldn't be mandated in the specification, IMO. An implementation should be free to optimize out those "implicit flush() calls" for performance reasons. > Eg. If a user opens a file in r/w mode, writes two bytes in it (which > stay buffered), and then reads 2 bytes, the two bytes read should be > those on range [2:4] of course, even though the file pointer would, due > to python buffering, still be at index 0. Actually the raw file pointer would be at index N, where N is at least the buffer size (say 4096). However, it is not specified how the raw file pointer behaves when using a Buffered{Reader, Writer, Random} wrapper over it. The buffered object is free to do what it wants with the raw stream until flush() is called (or the file is closed, which calls flush() implicitly). Even after flush() is called, the raw file pointer can still be at whatever place, but the raw file contents are consistent with the view given by the buffered object. > Q from me : What happens in read/write text files, when overwriting a > three-bytes character with a single-byte character ? Or at the contrary, > when a single chinese character overrides 3 ASCII characters in an UTF8 > file ? Is there any system designed to avoid this data corruption ? Or > should TextIO classes forbid read+write streams ? What happens isn't specified, but in practice (with the current implementation) the overwriting will happen at the byte level, without any check for correctness at the character level. Actually, read+write text streams are implemented quite crudely, and little testing is done of them. The reason, as you discovered, is that the semantics are too weak, and it is not obvious how stronger semantics could look like. People wanting to do sophisticated random reads+writes over a text file should probably handle the encoding themselves and access the file at the binary level. > Here is a very rough beginning of IOError hierarchy. I'd liek to have > people's opinion on the relevance of these, as well as on what other > exceptions should be distinguished from basic IOErrors. This deserves its own PEP IMO, although I'm not sure it would be accepted (ISTR the idea of a detailed IO exception hierarchy was already refused in the past). Regards Antoine. From python at mrabarnett.plus.com Sun Sep 27 16:51:34 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 27 Sep 2009 15:51:34 +0100 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition V2 In-Reply-To: <4ABF1B3B.4080106@gmail.com> References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> <4ABF1B3B.4080106@gmail.com> Message-ID: <4ABF7BF6.5070903@mrabarnett.plus.com> Pascal Chambon wrote: > Hello > > Below is a corrected version of the PEP update, adding the start/end > indexes proposition and fixing functions signatures. Does anyone > disagree with these specifications ? Or can we consider it as a target > for the next versions of the io module ? > I would have no problem to implement this behaviour in my own pure > python FileIO system, however if someone is willing to patch the _fileio > implementation, it'd save a lot of time - I most probably won't have the > means to setup a C compilation environment under windows and linux, and > properly update/test this, before January (when I get freelance...). > > I launch another thread on other to-be-discussed IO points B-) > > Regards, > Pascal > > ================ PEP UPDATE for new I/O system - v2 =========== > > **Truncate and file pointer semantics** > > Rationale : > > The current implementation of truncate() always move the file pointer to > the new end of file. > > This behaviour is interesting for compatibility, if the file has been > reduced and the file pointer is now past its end, since some platforms > might require 0 <= filepointer <= filesize. > > However, there are several arguments against this semantic: > > * Most common standards (posix, win32?) allow the file pointer to be > past the end of file, and define the behaviour of other stream > methods in this case > * In many cases, moving the filepointer when truncating has no > reasons to happen (if we?re extending the file, or reducing it > without going beneath the file pointer) > * Making 0 <= filepointer <= filesize a global rule of the python IO > module doesn?t seems possible, since it would require > modifications of the semantic of other methods (eg. seek() should > raise exceptions or silently disobey when asked to move the > filepointer past the end of file), and lead to incoherent > situations when concurrently accessing files without locking (what > if another process truncates to 0 bytes the file you?re writing ?) > > So here is the proposed semantic, which matches established conventions: > > *IOBase.truncate(n: int = None) -> int* > > Resizes the file to the size specified by the positive integer n, or by > the current filepointer position if n is None. > > The file must be opened with write permissions. > > If the file was previously larger than size, the extra data is discarded. > If the file was previously shorter than size, its size is increased, and > the extended area appears as if it were zero-filled. > Instead of "than size", perhaps "than n". > In any case, the file pointer is left unchanged, and may point beyond > the end of file. > > Note: trying to read past the end of file returns an empty string, and > trying to write past the end of file extends it by zero-ing the gap. On > rare platforms which don't support file pointers to be beyond the end of > file, all these behaviours shall be faked thanks to internal storage of > the "wanted" file pointer position (silently extending the file, if > necessary, when a write operation occurs). > > > > *Propositions of doc update* > > *RawIOBase*.read(n: int) -> bytes > > Read up to n bytes from the object and return them. Fewer than n bytes > may be returned if the operating system call returns fewer than n bytes. > If 0 bytes are returned, and n was not 0, this indicates end of file. If > the object is in non-blocking mode and no bytes are available, the call > returns None. > > > *RawIOBase*.readinto(b: bytearray, [start: int = None], [end: int = > None]) -> int > > start and end are used as slice indexes, so that the bytearray taken > into account is actually range = b[start:end] (or b[start:], b[:end] or > b[:], depending on the arguments which are not None). > > Read up to len(range) bytes from the object and store them in b, returning > the number of bytes read. Like .read, fewer than len(range) bytes may be > read, and 0 indicates end of file if len(range) is not 0. > None is returned if a non-blocking object has no bytes available. The > length of b is never changed. > Should an exception be raised if start and/or end are out of range? From stephen at xemacs.org Sun Sep 27 17:09:44 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Mon, 28 Sep 2009 00:09:44 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> Message-ID: <878wg0mnk7.fsf@uwakimon.sk.tsukuba.ac.jp> Antoine Pitrou writes: > There was a proposal to have a separate parse_address_and_mask > method which would return a (Address, Network) tuple, I still don't > know why you don't seem to consider it seriously, rather than > trying to make the Network class a kind of all-in-one type > conflating different concepts. Because he thinks about the problem space differently from you. Specifically, AFAICS he does not (maybe even "can't") see a reason to distinguish an AddressWithMask from a Network. If so, it's a reasonable API choice for him to choose not to have separate classes to represent such similar concepts. That's all there is to it. I personally do not have a problem with that, except that you apparently can't grasp his way of thinking, and he apparently can't grasp yours. I'm -1 on PEP 3144 primarily because of this communications gap. Secondarily because I agree that it's unnatural that a Network instance can have an arbitrary distinguished address other than those defined in the RFCs (the network and broadcast addresses), especially since it matters to equality comparisons. (But I personally would use ipaddr if it were in the stdlib despite that.) From python at mrabarnett.plus.com Sun Sep 27 17:14:35 2009 From: python at mrabarnett.plus.com (MRAB) Date: Sun, 27 Sep 2009 16:14:35 +0100 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4ABF2047.7050501@gmail.com> References: <4ABF2047.7050501@gmail.com> Message-ID: <4ABF815B.2010501@mrabarnett.plus.com> Pascal Chambon wrote: > Found in current io PEP : > Q: Do we want to mandate in the specification that switching between > reading and writing on a read-write object implies a .flush()? Or is > that an implementation convenience that users should not rely on? > -> it seems that the only important matter is : file pointer > positions and bytes/characters read should always be the ones that > the user expects, as if there were no buffering. So flushing or not > may stay a non-mandatory behaviour, as long as the buffered streams > ensures this data integrity. Eg. If a user opens a file in r/w mode, > writes two bytes in it (which stay buffered), and then reads 2 bytes, > the two bytes read should be those on range [2:4] of course, even > though the file pointer would, due to python buffering, still be at > index 0. > +1 > > Q from me : What happens in read/write text files, when overwriting a > three-bytes character with a single-byte character ? Or at the > contrary, when a single chinese character overrides 3 ASCII characters > in an UTF8 file ? Is there any system designed to avoid this data > corruption ? Or should TextIO classes forbid read+write streams ? > If the characters are always the same number of bytes) then overwriting could be possible; that would, however, make the behaviour more complicated (sometimes you can, sometimes you can't), so it's probably just simpler to forbid read+write text streams. From solipsis at pitrou.net Sun Sep 27 17:32:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 15:32:14 +0000 (UTC) Subject: [Python-Dev] IO module precisions and exception hierarchy References: <4ABF2047.7050501@gmail.com> <4ABF815B.2010501@mrabarnett.plus.com> Message-ID: MRAB mrabarnett.plus.com> writes: > > If the characters are always the same number of bytes) then overwriting > could be possible; that would, however, make the behaviour more > complicated (sometimes you can, sometimes you can't), so it's probably > just simpler to forbid read+write text streams. Forbidding them would be gratuitous. There are perfectly safe uses, for example if all writes are appends, or if you use a fixed-size encoding. (and besides, it would break backwards compatibility) Regards Antoine. From foom at fuhm.net Sun Sep 27 17:43:00 2009 From: foom at fuhm.net (James Y Knight) Date: Sun, 27 Sep 2009 11:43:00 -0400 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4ABF2047.7050501@gmail.com> References: <4ABF2047.7050501@gmail.com> Message-ID: On Sep 27, 2009, at 4:20 AM, Pascal Chambon wrote: > Thus, at the moment IOErrors rather have the semantic of "particular > case of OSError", and it's kind of confusing to have them remain in > their own separate tree... Furthermore, OSErrors are often used > where IOErrors would perfectly fit, eg. in low level I/O functions > of the OS module. > Since OSErrors and IOErrors are slightly mixed up when we deal with > IO operations, maybe the easiest way to make it clearer would be to > push to their limits already existing designs. How about just making IOError = OSError, and introducing your proposed subclasses? Does the usage of IOError vs OSError have *any* useful semantics? James From solipsis at pitrou.net Sun Sep 27 17:52:38 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 17:52:38 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <878wg0mnk7.fsf@uwakimon.sk.tsukuba.ac.jp> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <878wg0mnk7.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <1254066758.5532.40.camel@localhost> > I personally do not have a problem with that, except that you > apparently can't grasp his way of thinking, and he apparently can't > grasp yours. If I was the only one disagreeing it wouldn't be that annoying (except for me :-)). > I'm -1 on PEP 3144 primarily because of this > communications gap. I must say I have started to lean in that direction, too. If it is not possible to produce a result (a PEP) that solves all major design issues, it will produce the kind of little-liked modules that people have recently complained about in an stdlib-sig thread. Regards Antoine. From guido at python.org Sun Sep 27 18:37:40 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Sep 2009 09:37:40 -0700 Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library In-Reply-To: <4ABF1728.9@v.loewis.de> References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> <8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com> <8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com> <4A8C4264.2040708@trueblade.com> <20090916090849.GH12810@ruber.office.udmvt.ru> <4ABE5B24.1090003@v.loewis.de> <4ABF1728.9@v.loewis.de> Message-ID: On Sun, Sep 27, 2009 at 12:41 AM, "Martin v. L?wis" wrote: > Guido van Rossum wrote: >> I realize I'm late to this party, but this is just a naming issue, right? > > Not really (AFAIU). > >> For any network, there are two special addresses, one with the last >> bits all zeros, one with the last bits all ones. We can call them A >> and B, or network and broadcast, or zeros and ones, or whatever we >> care. But their definitions are two well-defined functions for all >> networks (assuming a network is defined as an IP address and a number >> of bits), even if in an extreme case the functions return the same >> value. > > For the broadcast address, it's different, since you might also use > it in programming (i.e. when sending broadcasts). So there is no need to look up the broadcast address in the configuration? Don't you have to look up the rest of the network configuration too? (Or otherwise how do you know your network address and the value of /N?) > Now, it seems that people will typically use undirected broadcasts > (INADDR_BROADCAST). However, I assume there are also applications > for directed broadcasts; in this case, it would matter to know the > broadcast address of a network. I don't know, and it seems you are not familiar with an actual case either. I would assume that apps doing such low-level work would have direct access to the complete network configuration, of which the broadcast address to use would be a separate argument. The ipaddr library seems most useful for doing manipulations and computations on addresses and networks that may bear no relationship to the current host's configuration -- e.g. off-line verification of firewall configurations, or whatever. >> What is actually configured on a particular host to be the broadcast >> address is a separate issue, even if *by convention* in most cases it >> is given by one of the above functions -- the network object doesn't >> care, the configuration object is something else (and outside the >> scope of this PEP). > > It's a little bit stronger than convention; there are RFCs saying > that the all-ones (-1) address has to be the broadcast address. Sure, but what is the status of those RFCs? (Plenty of RFCs are unimplemented or superseded by others etc.) In any case we seem to agree that 'broadcast' is a fine name for the 'all ones' address so I don't want to argue this point any further. :) > This is like saying that it is out of scope of the socket protocol > whether TCP is the IP protocol 6, and that individual hosts may do > it differently - yes, they may, but then everything stops working. Well, except that if al hosts on a closed network were to define their alternate value in the appropriate header, then everything would work (within that closed network!), even Python (which gets the TCP value from the system header too). But I don't think that that is relevant to ipaddr, which has to be able to make decisions about arbitrary IP addresses and networks without having access to the hosts or their configurations. It can only look at the syntax. >> IMO "real life examples" don't matter for the definitions of the >> functions > > That I agree with. > >> and I would personally be happy to name them network and >> broadcast, since I know their definitions and their typical uses and >> these match pretty closely. The expectation should be clearly set that >> these are pure functions though and do not imply knowledge of the >> configuration of any given host. > > That I also agree with. > > However, there is this issue of /31 networks, where an 8-year-old > RFC specifies how to do broadcasts on these. I think RFCs should > be considered wherever applicable. Considered wherever applicable, yes. I'm assuming you're talking about RFC 3021 -- which says "standards track" but I can't find whether it is actually a standard or how widespread its implementation is. It might well be obsolete by now, or widely ignored, or irrelevant outside a few drivers. Another question is whether it *always* applies when a /31 network is used. If not, then we're back to the situation where we can't know the configuration, and declaring the RFC applicable in all cases doesn't make it so. RFC 3021 adds an odd wart to the "all ones" and "all zeros" functions (or perhaps only to "all ones"?). If after further examination of the facts we find that it should be honored, then what do we do for .network and .broadcast on a /32 "network"? Is that an error or should it just return the one IP address of which the "network" consists? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From peter at hda3.com Sun Sep 27 18:47:27 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 09:47:27 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> Message-ID: <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> On Sun, Sep 27, 2009 at 4:23 AM, Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> >> To be explicit though, unless I'm drastically misreading what you're >> asking for, I consider the design you're proposing to be broken from a >> usability perspective (striving for academic/pedantic correctness over >> accepted practice). > > It is certainly not academic correctness, it is concern over the > understandibility of the library. The behaviour you are proposing is confusing, > and even if you think it is more practical (which I doubt it is), it should > still be avoided IMO. While "192.168.1.2/24" may be an accepted notation in some > situations, it is not a notation for a *network* but for an (address, mask) > pair, or an (address, network) pair. > > There was a proposal to have a separate parse_address_and_mask method which > would return a (Address, Network) tuple, I still don't know why you don't seem > to consider it seriously, rather than trying to make the Network class a kind of > all-in-one type conflating different concepts. The reason (aside from the name) that I'm not going to include this in ipaddr is that it would require the user to deal with two objects when one would suffice. It's similar to getting two return values from float(). integer, fraction = float('1.25') crazy, right? Finally, to Stephen's point about seeing the other side of the argument, I wrote this offlist a week ago: I *understand* what you're saying, I *understand* that 192.168.1.1/24 isn't a network, 192.168.1.0/24 is a network, and 192.168.1.1 is an ip address on that network and it has an associated netmask that helps describe its network, etc. etc. what I'm saying is that, enforcing that restriction by introducing 2 new classes (ipv4 is not comparable with ipv6, I've actually got a good quote from Vint Cerf about this, so any AddressWithNetmask (?!?) class would have to be duplicated for both addr types) or something else as of yet described, is not useful. it is in fact, detrimental (optionally enforcing that restriction via an argument to the Network constructors is, however, useful) Cheers, /peter > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From solipsis at pitrou.net Sun Sep 27 19:22:41 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 17:22:41 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909151705y46539219m84dc4d5e14ca9a57@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > The reason (aside from the name) that I'm not going to include this in > ipaddr is that it would require the user to deal with two objects when > one would suffice. You make it sound like it's a burden, but "dealing" with two objects is not something extraordinary or bizarre, it happens all the time when you do network programming: e.g. ('127.0.0.1', 80). (or would you argue that Address objects should have an optional distinguishing port number, for "convenience" reasons?) If you need the two objects, whether you access them through separate variables (`network` and `host`) or through attribute access (`network` and `network.host`) doesn't really matter. You can even pretend the tuple is an atomic object and store it as-is if you want. I'm not sure what annoys you here. > It's similar to getting two return values from > float(). I don't see how it's similar, since a float is a clearly defined atomic entity. Most float operations become meaningless if you consider the integral and the fractional part separately. Regards Antoine. From steve at pearwood.info Sun Sep 27 19:30:29 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 28 Sep 2009 03:30:29 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> Message-ID: <200909280330.30643.steve@pearwood.info> On Mon, 28 Sep 2009 02:47:27 am Peter Moody wrote: > > There was a proposal to have a separate parse_address_and_mask > > method which would return a (Address, Network) tuple, I still don't > > know why you don't seem to consider it seriously, rather than > > trying to make the Network class a kind of all-in-one type > > conflating different concepts. > > The reason (aside from the name) that I'm not going to include this > in ipaddr is that it would require the user to deal with two objects > when one would suffice. It's similar to getting two return values > from float(). > > integer, fraction = float('1.25') > > crazy, right? Not if you want a separate integer and fraction object. There are plenty of use-cases for such things, and the math module includes a function, modf(), that does precisely that.. > Finally, to Stephen's point about seeing the other side of the > argument, I wrote this offlist a week ago: > > I *understand* what you're saying, I *understand* that > 192.168.1.1/24 isn't a network, But you still want to treat it as one. Could you explain what benefit there is for allowing the user to create network objects that don't represent networks? Is there a use-case where these networks-that-aren't-networks are something other than a typo? Under what circumstances would I want to specify a network as 192.168.1.1/24 instead of 192.168.1.0/24? -- Steven D'Aprano From peter at hda3.com Sun Sep 27 19:53:27 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 10:53:27 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909280330.30643.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> Message-ID: <8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com> On Sun, Sep 27, 2009 at 10:30 AM, Steven D'Aprano wrote: > On Mon, 28 Sep 2009 02:47:27 am Peter Moody wrote: > >> > There was a proposal to have a separate parse_address_and_mask >> > method which would return a (Address, Network) tuple, I still don't >> > know why you don't seem to consider it seriously, rather than >> > trying to make the Network class a kind of all-in-one type >> > conflating different concepts. >> >> The reason (aside from the name) that I'm not going to include this >> in ipaddr is that it would require the user to deal with two objects >> when one would suffice. It's similar to getting two return values >> from float(). >> >> ?integer, fraction = float('1.25') >> >> crazy, right? > > Not if you want a separate integer and fraction object. There are plenty > of use-cases for such things, and the math module includes a function, > modf(), that does precisely that.. > > >> Finally, to Stephen's point about seeing the other side of the >> argument, I wrote this offlist a week ago: >> >> ? I *understand* what you're saying, I *understand* that >> 192.168.1.1/24 isn't a network, > > But you still want to treat it as one. > > Could you explain what benefit there is for allowing the user to create > network objects that don't represent networks? Is there a use-case > where these networks-that-aren't-networks are something other than a > typo? Under what circumstances would I want to specify a network as > 192.168.1.1/24 instead of 192.168.1.0/24? this is pretty ridiculous. if in the last two months you haven't seen a single use case, then you haven't been paying attention. > -- > Steven D'Aprano > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From peter at hda3.com Sun Sep 27 20:04:04 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 11:04:04 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> Message-ID: <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> On Sun, Sep 27, 2009 at 10:22 AM, Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> >> The reason (aside from the name) that I'm not going to include this in >> ipaddr is that it would require the user to deal with two objects when >> one would suffice. > > You make it sound like it's a burden, but "dealing" with two objects is not > something extraordinary or bizarre, it happens all the time when you do network > programming: e.g. ('127.0.0.1', 80). > > (or would you argue that Address objects should have an optional distinguishing > port number, for "convenience" reasons?) I'm not sure what you're talking about, I've never argued to add layer 4 information to ipaddr. > If you need the two objects, whether you access them through separate variables > (`network` and `host`) or through attribute access (`network` and > `network.host`) doesn't really matter. You can even pretend the tuple is an > atomic object and store it as-is if you want. I'm not sure what annoys you here. this is tautological. if you need two objects, then two objects is what you need. I'm saying that you don't need two objects for this, that common accepted practice is to represent this as one. Note, there is *nothing* stopping you currently from using two objects for this, an IPAddress object and a strict=True IPNetwork object. I'm not sure what annoys you about this. >> It's similar to getting two return values from >> float(). > > I don't see how it's similar, since a float is a clearly defined atomic entity. > Most float operations become meaningless if you consider the integral and the > fractional part separately. > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From solipsis at pitrou.net Sun Sep 27 20:17:34 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 18:17:34 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909271205.18488.steve@pearwood.info> <8517e9350909261922t2103d536maa2d3875b859b194@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > > (or would you argue that Address objects should have an optional distinguishing > > port number, for "convenience" reasons?) > > I'm not sure what you're talking about, I've never argued to add layer > 4 information to ipaddr. It was an analogy, just like your "float" analogy. > I'm not sure what annoys you about this. I have already explained it: what annoys me is that it makes `Network` a hybrid object conflating two independent concepts, and makes the library less understandable as a result. Others have already pointed out that it makes operations like equality confusing. Regards Antoine. From peter at hda3.com Sun Sep 27 21:18:58 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 12:18:58 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> Message-ID: <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> On Sun, Sep 27, 2009 at 11:17 AM, Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> >> > (or would you argue that Address objects should have an optional > distinguishing >> > port number, for "convenience" reasons?) >> >> I'm not sure what you're talking about, I've never argued to add layer >> 4 information to ipaddr. > > It was an analogy, just like your "float" analogy. > >> I'm not sure what annoys you about this. > > I have already explained it: what annoys me is that it makes `Network` a hybrid > object conflating two independent concepts, and makes the library less > understandable as a result. I understand that this is your assertion. Here's the thing, your use case is already supported. IPv?Network objects can only be created of network addresses, exceptions are raised if all the host bits aren't 0, you can create and only deal with IPv4Address('192.168.1.1') and IPv4Network('192.168.1.0/24', strict=True) etc etc. That's not how I (nor many other network administrators) would use it, but it's doable. what you're claiming is that my use case is invalid. that's what I claim is broken. There have been folks on both sides claiming that this design is both fundamentally confusing and fundamentally sound. The confused have obviously been more vocal and less willing to compromise. So, the basic design of the library stands as it is, minor implementation and documentation bugs not withstanding. I'm not going to make ipaddr less useful (strictly removing functionality), more bulky and confusing (adding more confusingly named classes and methods) or otherwise break the library in a vain attempt to have it included in the stdlib. Cheers, /peter > Others have already pointed out that it makes operations like equality > confusing. > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From foom at fuhm.net Sun Sep 27 21:40:22 2009 From: foom at fuhm.net (James Y Knight) Date: Sun, 27 Sep 2009 15:40:22 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABEFA5D.40506@gmail.com> <8517e9350909262254x6736c11fk6e6a3d04e6deafa0@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> Message-ID: <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> On Sep 27, 2009, at 3:18 PM, Peter Moody wrote: > administrators) would use it, but it's doable. what you're claiming is > that my use case is invalid. > > that's what I claim is broken. He's claiming your solution to address your use case is confusing, not that the use case is invalid. > I'm not going to make ipaddr > less useful (strictly removing functionality), more bulky and > confusing (adding more confusingly named classes and methods) or > otherwise break the library in a vain attempt to have it included in > the stdlib. If I understand correctly, the proposal for addressing the issue is to make two rather simple changes: 1) if strict=False, mask off the bits described by the netmask when creating an IPNetwork, such that the host bits are always 0. 2) add a single new function: def parse_net_and_addr(s): return (IPNetwork(s), IPAddress(s.split('/')[0])) James From martin at v.loewis.de Sun Sep 27 21:50:09 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 27 Sep 2009 21:50:09 +0200 Subject: [Python-Dev] PEP 3144: IP Address Manipulation Library for the Python Standard Library In-Reply-To: References: <8517e9350908181300v6b61c942p9ae85a3e5bc9709e@mail.gmail.com> <8517e9350908190819j137375h63c97e81ad55d71f@mail.gmail.com> <8517e9350908190955y393435d9w89fa25980155a7fc@mail.gmail.com> <4A8C4264.2040708@trueblade.com> <20090916090849.GH12810@ruber.office.udmvt.ru> <4ABE5B24.1090003@v.loewis.de> <4ABF1728.9@v.loewis.de> Message-ID: <4ABFC1F1.5050507@v.loewis.de> >> For the broadcast address, it's different, since you might also use >> it in programming (i.e. when sending broadcasts). > > So there is no need to look up the broadcast address in the > configuration? Don't you have to look up the rest of the network > configuration too? (Or otherwise how do you know your network address > and the value of /N?) You would have to know either the broadcast address directly, or the network (i.e. address and prefix length). >>> What is actually configured on a particular host to be the broadcast >>> address is a separate issue, even if *by convention* in most cases it >>> is given by one of the above functions -- the network object doesn't >>> care, the configuration object is something else (and outside the >>> scope of this PEP). >> It's a little bit stronger than convention; there are RFCs saying >> that the all-ones (-1) address has to be the broadcast address. > > Sure, but what is the status of those RFCs? (Plenty of RFCs are > unimplemented or superseded by others etc.) The one that says that the broadcast address is -1 (and 0 should also be supported) is STD 3. The one that talks about 31-bit prefixes (RFC 3021) is a proposed standard. > RFC 3021 adds an odd wart to the "all ones" and "all zeros" functions > (or perhaps only to "all ones"?). To both: it allows them to be used as host addresses, even though STD 3 says that they are reserved and must not be assigned to hosts. > If after further examination of the > facts we find that it should be honored, then what do we do for > .network and .broadcast on a /32 "network"? For these, I would first like to find out what their specification is. Regards, Martin From peter at hda3.com Sun Sep 27 21:56:03 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 12:56:03 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> Message-ID: <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> On Sun, Sep 27, 2009 at 12:40 PM, James Y Knight wrote: > > On Sep 27, 2009, at 3:18 PM, Peter Moody wrote: > >> administrators) would use it, but it's doable. what you're claiming is >> that my use case is invalid. >> >> that's what I claim is broken. > > He's claiming your solution to address your use case is confusing, not that > the use case is invalid. this isn't actually true. Steven D'Aprano wrote: Could you explain what benefit there is for allowing the user to create network objects that don't represent networks? Is there a use-case where these networks-that-aren't-networks are something other than a typo? Under what circumstances would I want to specify a network as 192.168.1.1/24 instead of 192.168.1.0/24? that pretty flatly states that my use case is invalid. >> I'm not going to make ipaddr >> less useful (strictly removing functionality), more bulky and >> confusing (adding more confusingly named classes and methods) or >> otherwise break the library in a vain attempt to have it included in >> the stdlib. > > If I understand correctly, the proposal for addressing the issue is to make > two rather simple changes: i wish it were this easy. > 1) if strict=False, mask off the bits described by the netmask when creating > an IPNetwork, such that the host bits are always 0. I haven't heard anyone suggest auto-masking bits, but otherwise that would be strict=True. > 2) add a single new function: > > def parse_net_and_addr(s): > ?return (IPNetwork(s), IPAddress(s.split('/')[0])) I've only heard talk of new classes and new methods, not new constructor functions. In fact, when I asked for explicit clarification of what was required, a new constructor function "ala IPAddress and IPNetwork (so the current classes remain largely the same and the various constructors enforce certain restrictions)", I was told, no, a new object AddressWithMask was actually required. I've no problem with a constructor like this, but this is not what people have been asking for. Cheers, /peter > James > From solipsis at pitrou.net Sun Sep 27 22:15:26 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 20:15:26 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > On Sun, Sep 27, 2009 at 12:40 PM, James Y Knight fuhm.net> wrote: > > > > On Sep 27, 2009, at 3:18 PM, Peter Moody wrote: > > > >> administrators) would use it, but it's doable. what you're claiming is > >> that my use case is invalid. > >> > >> that's what I claim is broken. > > > > He's claiming your solution to address your use case is confusing, not that > > the use case is invalid. > > this isn't actually true. > > Steven D'Aprano wrote: [...] That's Steven, your original sentence was about me. > > 1) if strict=False, mask off the bits described by the netmask when creating > > an IPNetwork, such that the host bits are always 0. > > I haven't heard anyone suggest auto-masking bits, but otherwise that > would be strict=True. I would expect strict=True to raise an error if the lower bits are non-zero, not to silently erase them. strict=False would be the option that silently erases lower bits. (that's why it's named `strict`, after all :-)) > > 2) add a single new function: > > > > def parse_net_and_addr(s): > > ?return (IPNetwork(s), IPAddress(s.split('/')[0])) > > I've only heard talk of new classes and new methods, not new > constructor functions. Well, "method" in that context meant "class method" since the results aren't dependent on a particular instance. Of course, both a class method or a module-level function would be fine. Regards Antoine. From peter at hda3.com Sun Sep 27 22:36:37 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 13:36:37 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> Message-ID: <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> On Sun, Sep 27, 2009 at 1:15 PM, Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> >> On Sun, Sep 27, 2009 at 12:40 PM, James Y Knight fuhm.net> wrote: >> > >> > On Sep 27, 2009, at 3:18 PM, Peter Moody wrote: >> > >> >> administrators) would use it, but it's doable. what you're claiming is >> >> that my use case is invalid. >> >> >> >> that's what I claim is broken. >> > >> > He's claiming your solution to address your use case is confusing, not that >> > the use case is invalid. >> >> this isn't actually true. >> >> Steven D'Aprano wrote: > [...] > > That's Steven, your original sentence was about me. my original sentence was about everyone who was arguing that the current implementation was designed for an invalid use-case. it was in reply to your email, but it was to everyone (you, Nick, rdm, Steven, etc) >> > 1) if strict=False, mask off the bits described by the netmask when creating >> > an IPNetwork, such that the host bits are always 0. >> >> I haven't heard anyone suggest auto-masking bits, but otherwise that >> would be strict=True. > > I would expect strict=True to raise an error if the lower bits are non-zero, not > to silently erase them. strict=False would be the option that silently erases > lower bits. > (that's why it's named `strict`, after all :-)) > >> > 2) add a single new function: >> > >> > def parse_net_and_addr(s): >> > ?return (IPNetwork(s), IPAddress(s.split('/')[0])) >> >> I've only heard talk of new classes and new methods, not new >> constructor functions. > > Well, "method" in that context meant "class method" since the results aren't > dependent on a particular instance. Of course, both a class method or a > module-level function would be fine. so this is not the response I got when I asked what was required before. Would adding this constructor function satisfy your concerns (given sensible strict settings in the constructor, etc)? > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From solipsis at pitrou.net Sun Sep 27 22:49:15 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 20:49:15 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > >> > def parse_net_and_addr(s): > >> > ?return (IPNetwork(s), IPAddress(s.split('/')[0])) > >> > >> I've only heard talk of new classes and new methods, not new > >> constructor functions. > > > > Well, "method" in that context meant "class method" since the results aren't > > dependent on a particular instance. Of course, both a class method or a > > module-level function would be fine. > > so this is not the response I got when I asked what was required > before. Would adding this constructor function satisfy your concerns > (given sensible strict settings in the constructor, etc)? Assuming the Network type loses the notion of a specific host (or host address, or `ip`) attached to it, yes. Or to put it more clearly: >>> Network('192.168.0.1/24', strict=False) Network('192.168.0.0/24') >>> Network('192.168.0.1/24', strict=False) == Network('192.168.0.0/24') True Regards Antoine. From peter at hda3.com Sun Sep 27 22:59:07 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 13:59:07 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> Message-ID: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> >> >> > def parse_net_and_addr(s): >> >> > ?return (IPNetwork(s), IPAddress(s.split('/')[0])) >> >> >> >> I've only heard talk of new classes and new methods, not new >> >> constructor functions. >> > >> > Well, "method" in that context meant "class method" since the results aren't >> > dependent on a particular instance. Of course, both a class method or a >> > module-level function would be fine. >> >> so this is not the response I got when I asked what was required >> before. Would adding this constructor function satisfy your concerns >> (given sensible strict settings in the constructor, etc)? > > Assuming the Network type loses the notion of a specific host (or host address, > or `ip`) attached to it, yes. this is "less useful (strictly removing functionality)" and is an example of what I explicitly said I was not going to do with ipaddr. Cheers, /peter > Or to put it more clearly: > >>>> Network('192.168.0.1/24', strict=False) > Network('192.168.0.0/24') >>>> Network('192.168.0.1/24', strict=False) == Network('192.168.0.0/24') > True > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From steven.bethard at gmail.com Sun Sep 27 22:59:32 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 27 Sep 2009 13:59:32 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module Message-ID: Below is a new PEP based on discussions from the stdlib-sig, which proposes to add the argparse module to the standard library in Python 2.7 and 3.2. Looking forward to your feedback! Steve http://www.python.org/dev/peps/pep-0389/ ---------------------------------------------------------------------- PEP: 389 Title: argparse - new command line parsing module Version: $Revision: 75097 $ Last-Modified: $Date: 2009-09-27 12:42:40 -0700 (Sun, 27 Sep 2009) $ Author: Steven Bethard Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 25-Sep-2009 Python-Version: 2.7 and 3.2 Post-History: Abstract ======== This PEP proposes inclusion of the argparse [1]_ module in the Python standard library in Python 2.7 and 3.2. Motivation ========== The argparse module is a command line parsing library which provides more functionality than the existing command line parsing modules in the standard library, getopt [2]_ and optparse [3]_. It includes support for positional arguments (not just options), subcommands, required options, options syntaxes like "/f" and "+rgb", zero-or-more and one-or-more style arguments, and many other features the other two lack. The argparse module is also already a popular third-party replacement for these modules. It is used in projects like IPython (the Scipy Python shell) [4]_, is included in Debian testing and unstable [5]_, and since 2007 has had various requests for its inclusion in the standard library [6]_ [7]_ [8]_. This popularity suggests it may be a valuable addition to the Python libraries. Why aren't getopt and optparse enough? ====================================== One argument against adding argparse is that thare are "already two different option parsing modules in the standard library" [9]_. The following is a list of features provided by argparse but not present in getopt or optparse: * While it is true there are two *option* parsing libraries, there are no full command line parsing libraries -- both getopt and optparse support only options and have no support for positional arguments. The argparse module handles both, and as a result, is able to generate better help messages, avoiding redundancies like the ``usage=`` string usually required by optparse. * The argparse module values practicality over purity. Thus, argparse allows required options and customization of which characters are used to identify options, while optparse explicitly states "the phrase 'required option' is self-contradictory" and that the option syntaxes ``-pf``, ``-file``, ``+f``, ``+rgb``, ``/f`` and ``/file`` "are not supported by optparse, and they never will be". * The argparse module allows options to accept a variable number of arguments using ``nargs='?'``, ``nargs='*'`` or ``nargs='+'``. The optparse module provides an untested recipe for some part of this functionality [10]_ but admits that "things get hairy when you want an option to take a variable number of arguments." * The argparse module supports subcommands, where a main command line parser dispatches to other command line parsers depending on the command line arguments. This is a common pattern in command line interfaces, e.g. ``svn co`` and ``svn up``. Why isn't the functionality just being added to optparse? ========================================================= Clearly all the above features offer improvements over what is available through optparse. A reasonable question then is why these features are not simply provided as patches to optparse, instead of introducing an entirely new module. In fact, the original development of argparse intended to do just that, but because of various fairly constraining design decisions of optparse, this wasn't really possible. Some of the problems included: * The optparse module exposes the internals of its parsing algorithm. In particular, ``parser.largs`` and ``parser.rargs`` are guaranteed to be available to callbacks [11]_. This makes it extremely difficult to improve the parsing algorithm as was necessary in argparse for proper handling of positional arguments and variable length arguments. For example, ``nargs='+'`` in argparse is matched using regular expressions and thus has no notion of things like ``parser.largs``. * The optparse extension APIs are extremely complex. For example, just to use a simple custom string-to-object conversion function, you have to subclass ``Option``, hack class attributes, and then specify your custom option type to the parser, like this:: class MyOption(Option): TYPES = Option.TYPES + ("mytype",) TYPE_CHECKER = copy(Option.TYPE_CHECKER) TYPE_CHECKER["mytype"] = check_mytype parser = optparse.OptionParser(option_class=MyOption) parser.add_option("-m", type="mytype") For comparison, argparse simply allows conversion functions to be used as ``type=`` arguments directly, e.g.:: parser = argparse.ArgumentParser() parser.add_option("-m", type=check_mytype) But given the baroque customization APIs of optparse, it is unclear how such a feature should interact with those APIs, and it is quite possible that introducing the simple argparse API would break existing custom Option code. * Both optparse and argparse parse command line arguments and assign them as attributes to an object returned by ``parse_args``. However, the optparse module guarantees that the ``take_action`` method of custom actions will always be passed a ``values`` object which provides an ``ensure_value`` method [12]_, while the argparse module allows attributes to be assigned to any object, e.g.:: foo_object = ... parser.parse_args(namespace=foo_object) foo_object.some_attribute_parsed_from_command_line Modifying optparse to allow any object to be passed in would be difficult because simply passing the ``foo_object`` around instead of a ``Values`` instance will break existing custom actions that depend on the ``ensure_value`` method. Because of issues like these, which made it unreasonably difficult for argparse to stay compatible with the optparse APIs, argparse was developed as an independent module. Given these issues, merging all the argparse features into optparse with no backwards incompatibilities seems unlikely. Deprecation of getopt and optparse ================================== There is still some debate over the best way (if at all) to encourage users to move from getopt and optparse to their replacement, argparse. The current recommendation of this PEP is the following conservative deprecation strategy: * Python 3.2, Python 2.7 and any later Python 2.X releases -- PendingDeprecation warnings, which by default are not displayed, and documentation notes directing users of getopt and optparse to argparse. * Python 3.3 -- Same as above * Python 3.4 -- Deprecation warnings for getopt and optparse, which by default *are* displayed. Though this is slower than the usual deprecation process, it seems prudent to avoid producing any casually visible Deprecation warnings until Python 3.X has had some additional time to attract developers. Open Issues =========== The argparse module supports Python from 2.3 up through 3.2 and as a result relies on traditional ``%(foo)s`` style string formatting. It has been suggested that it might be better to use the new style ``{foo}`` string formatting [13]_. This seems like a good idea, but would break backwards compatibility for existing argparse-based scripts unless we can come up with a way to reasonably support both syntaxes. References ========== .. [1] argparse (http://code.google.com/p/argparse/) .. [2] getopt (http://docs.python.org/library/getopt.html) .. [3] optparse (http://docs.python.org/library/optparse.html) .. [4] argparse in IPython (http://mail.scipy.org/pipermail/ipython-dev/2009-April/005102.html) .. [5] argparse in Debian (http://packages.debian.org/search?keywords=python-argparse) .. [6] 2007-01-03 request for argparse in the standard library (http://mail.python.org/pipermail/python-list/2007-January/592646.html) .. [7] 2009-06-09 request for argparse in the standard library (http://bugs.python.org/issue6247) .. [8] 2009-09-10 request for argparse in the standard library (http://mail.python.org/pipermail/stdlib-sig/2009-September/000342.html) .. [9] Fredrik Lundh response to [6]_ (http://mail.python.org/pipermail/python-list/2007-January/592675.html) .. [10] optparse variable args (http://docs.python.org/library/optparse.html#callback-example-6-variable-arguments) .. [11] parser.largs and parser.rargs (http://docs.python.org/library/optparse.html#how-callbacks-are-called) .. [12] take_action values argument (http://docs.python.org/library/optparse.html#adding-new-actions) .. [13] use {}-formatting instead of %-formatting (http://bugs.python.org/msg89279) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End: From ncoghlan at gmail.com Sun Sep 27 22:41:17 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 06:41:17 +1000 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: References: <4ABF2047.7050501@gmail.com> Message-ID: <4ABFCDED.8080105@gmail.com> Antoine Pitrou wrote: > This deserves its own PEP IMO, although I'm not sure it would be accepted > (ISTR the idea of a detailed IO exception hierarchy was already refused > in the past). Not as such - a big exception hierarchy rewrite was rejected, but nothing specifically limited to the IO exceptions. Michael's response cut to the heart of the issue though - a richer IO exception hierarchy can make life interesting for compatibility purposes (especially when creating "file-like" interfaces to non-file objects). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Sun Sep 27 23:24:52 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 27 Sep 2009 14:24:52 -0700 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition V2 In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> <4ABF1B3B.4080106@gmail.com> Message-ID: On Sun, Sep 27, 2009 at 4:33 AM, Antoine Pitrou wrote: >> So here is the proposed semantic, which matches established conventions: >> >> *IOBase.truncate(n: int = None) -> int* > [...] > > I still don't think there is a sufficient benefit in breaking > compatibility. If you want the file pointer to remain the same, you can > save it first and restore it afterwards manually. What compatibility, though? f.truncate() behaves different in 2.x than in 3.x, and in 2.x it seems to match the POSIX semantics (i.e. the seek position is unchanged even though the file size is). Perhaps the changed semantics was an oversight or a mistake? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From fuzzyman at voidspace.org.uk Sun Sep 27 23:32:06 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 27 Sep 2009 22:32:06 +0100 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4ABFCDED.8080105@gmail.com> References: <4ABF2047.7050501@gmail.com> <4ABFCDED.8080105@gmail.com> Message-ID: <4ABFD9D6.9070602@voidspace.org.uk> Nick Coghlan wrote: > Antoine Pitrou wrote: > >> This deserves its own PEP IMO, although I'm not sure it would be accepted >> (ISTR the idea of a detailed IO exception hierarchy was already refused >> in the past). >> > > Not as such - a big exception hierarchy rewrite was rejected, but > nothing specifically limited to the IO exceptions. > > Michael's response cut to the heart of the issue though - a richer IO > exception hierarchy can make life interesting for compatibility purposes > (especially when creating "file-like" interfaces to non-file objects). > Some of the error messages are truly awful though as things stand, especially for someone new to Python. Try to read from a file handle opened in read mode for example: IOError: [Errno 9] Bad file descriptor Michael > Cheers, > Nick. > > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From brett at python.org Sun Sep 27 23:57:34 2009 From: brett at python.org (Brett Cannon) Date: Sun, 27 Sep 2009 14:57:34 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: I am going to state upfront that I am +1 for this and I encouraged Steven to submit this PEP on the stdlib-SIG. I still remember watching Steven's lightning talk at PyCon 2009 on argparse and being impressed by it (along with the rest of the audience which was obviously impressed as well). I think argprase is a good improvement over what we have now (getopt and optparse), it's stable, considered best-of-breed by the community for a while (as shown as how many times argparse has been suggestion for inclusion), and Steven is already a core committer so is already set to maintain the code. That covers the usual checklist we have for even looking at a PEP to add a module to the standard library. On Sun, Sep 27, 2009 at 13:59, Steven Bethard wrote: [SNIP] > Deprecation of getopt and optparse > ================================== > There is still some debate over the best way (if at all) to encourage > users to move from getopt and optparse to their replacement, > argparse. The current recommendation of this PEP is the following > conservative deprecation strategy: > > * Python 3.2, Python 2.7 and any later Python 2.X releases -- > ?PendingDeprecation warnings, which by default are not displayed, > ?and documentation notes directing users of getopt and optparse to > ?argparse. > > * Python 3.3 -- Same as above > > * Python 3.4 -- Deprecation warnings for getopt and optparse, which > ?by default *are* displayed. > > Though this is slower than the usual deprecation process, it seems > prudent to avoid producing any casually visible Deprecation warnings > until Python 3.X has had some additional time to attract developers. > Just to give people ideas of how long these deprecations would last, if Python 3.2 is released in June 2010 and we continue on our 18 month release schedule (and actually release that quickly which we typically don't), then getopt/optparse would have a pending deprecation for 3 years (until June 2013) and a deprecation warning for 1.5 years (until January 2015), so 4.5 years total once the deprecations began and six years since Python 3.0 came out. And obviously the deprecation can be extended if it turns out Python 3 pick up is now at a rate we expect (but only if people who plan to port are having issues; this does not concern those who plan to stay on Python 2 indefinitely and do not care about Python 3). And we can also document suggestions on how to transition off of getopt/optparse like Steven has in the argparse code (http://argparse.googlecode.com/svn/tags/r101/doc/argparse-vs-optparse.html#upgrading-optparse-code). > > Open Issues > =========== > The argparse module supports Python from 2.3 up through 3.2 and as a > result relies on traditional ``%(foo)s`` style string formatting. It > has been suggested that it might be better to use the new style > ``{foo}`` string formatting [13]_. This seems like a good idea, but > would break backwards compatibility for existing argparse-based > scripts unless we can come up with a way to reasonably support both > syntaxes. I see two solutions to this. One is to auto-detect either format and then do the right thing. Seems potentially messy, although getting the regex right shouldn't be a problem. The other option is to rename the module if it is added to the standard library ala simplejson/json. That would alleviate any issues with someone including their own copy of argparse with their application using %()s string interpolation over {}. I don't have a name to suggest at the moment (nor do I think we should start discussing that now unless this is a chosen solution), but it would deal with the problem. Either way, with {} being the future and talk of someday dropping %, I think we should make sure the newer syntax is at least supported, if not the only way to do things. -Brett From fuzzyman at voidspace.org.uk Mon Sep 28 00:00:21 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 27 Sep 2009 23:00:21 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <4ABFE075.2070309@voidspace.org.uk> Brett Cannon wrote: > I am going to state upfront that I am +1 for this and I encouraged > Steven to submit this PEP on the stdlib-SIG. I still remember watching > Steven's lightning talk at PyCon 2009 on argparse and being impressed > by it (along with the rest of the audience which was obviously > impressed as well). > > I think argprase is a good improvement over what we have now (getopt > and optparse), it's stable, considered best-of-breed by the community > for a while (as shown as how many times argparse has been suggestion > for inclusion), and Steven is already a core committer so is already > set to maintain the code. That covers the usual checklist we have for > even looking at a PEP to add a module to the standard library. > > I've used argparse and really like it. I could also have done with the subcommand support in recent changes to unittest. +1 for inclusion. Michael > On Sun, Sep 27, 2009 at 13:59, Steven Bethard wrote: > [SNIP] > >> Deprecation of getopt and optparse >> ================================== >> There is still some debate over the best way (if at all) to encourage >> users to move from getopt and optparse to their replacement, >> argparse. The current recommendation of this PEP is the following >> conservative deprecation strategy: >> >> * Python 3.2, Python 2.7 and any later Python 2.X releases -- >> PendingDeprecation warnings, which by default are not displayed, >> and documentation notes directing users of getopt and optparse to >> argparse. >> >> * Python 3.3 -- Same as above >> >> * Python 3.4 -- Deprecation warnings for getopt and optparse, which >> by default *are* displayed. >> >> Though this is slower than the usual deprecation process, it seems >> prudent to avoid producing any casually visible Deprecation warnings >> until Python 3.X has had some additional time to attract developers. >> >> > > Just to give people ideas of how long these deprecations would last, > if Python 3.2 is released in June 2010 and we continue on our 18 month > release schedule (and actually release that quickly which we typically > don't), then getopt/optparse would have a pending deprecation for 3 > years (until June 2013) and a deprecation warning for 1.5 years (until > January 2015), so 4.5 years total once the deprecations began and six > years since Python 3.0 came out. And obviously the deprecation can be > extended if it turns out Python 3 pick up is now at a rate we expect > (but only if people who plan to port are having issues; this does not > concern those who plan to stay on Python 2 indefinitely and do not > care about Python 3). > > And we can also document suggestions on how to transition off of > getopt/optparse like Steven has in the argparse code > (http://argparse.googlecode.com/svn/tags/r101/doc/argparse-vs-optparse.html#upgrading-optparse-code). > > >> Open Issues >> =========== >> The argparse module supports Python from 2.3 up through 3.2 and as a >> result relies on traditional ``%(foo)s`` style string formatting. It >> has been suggested that it might be better to use the new style >> ``{foo}`` string formatting [13]_. This seems like a good idea, but >> would break backwards compatibility for existing argparse-based >> scripts unless we can come up with a way to reasonably support both >> syntaxes. >> > > I see two solutions to this. One is to auto-detect either format and > then do the right thing. Seems potentially messy, although getting the > regex right shouldn't be a problem. > > The other option is to rename the module if it is added to the > standard library ala simplejson/json. That would alleviate any issues > with someone including their own copy of argparse with their > application using %()s string interpolation over {}. I don't have a > name to suggest at the moment (nor do I think we should start > discussing that now unless this is a chosen solution), but it would > deal with the problem. > > Either way, with {} being the future and talk of someday dropping %, I > think we should make sure the newer syntax is at least supported, if > not the only way to do things. > > -Brett > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From ncoghlan at gmail.com Sun Sep 27 22:34:51 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 06:34:51 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> Message-ID: <4ABFCC6B.3020409@gmail.com> Peter Moody wrote: > Steven D'Aprano wrote: > > Could you explain what benefit there is for allowing the user to create > network objects that don't represent networks? Is there a use-case > where these networks-that-aren't-networks are something other than a > typo? Under what circumstances would I want to specify a network as > 192.168.1.1/24 instead of 192.168.1.0/24? > > that pretty flatly states that my use case is invalid. Looks like Steven was asking you to explain your use case to him rather than stating that it didn't exist. (although I will grant that the network definition use case had been well covered, I will also grant that these threads have been somewhat interminable and I can easily understand someone missing those use case definitions - the example of being able to define a network given only the network mask and the IP address of the host running the current application should be mentioned explicitly in the PEP) >> 1) if strict=False, mask off the bits described by the netmask when creating >> an IPNetwork, such that the host bits are always 0. > > I haven't heard anyone suggest auto-masking bits, but otherwise that > would be strict=True. Actually, that's exactly what I suggested in this thread - that IPNetwork should lose the "ip" property and that the definition of "network equality" should be based on the network address not on the host address that happened to be used when constructing the network object. The part that is confusing with the current library implementation is the fact that we can have two IPNetwork objects, supposedly describing the same network, compare unequal because they happened to be defined differently. It would be analagous to having "Decimal('3.0') != Decimal('3.00')". >> 2) add a single new function: >> >> def parse_net_and_addr(s): >> return (IPNetwork(s), IPAddress(s.split('/')[0])) > > I've only heard talk of new classes and new methods, not new > constructor functions. In fact, when I asked for explicit > clarification of what was required, a new constructor function "ala > IPAddress and IPNetwork (so the current classes remain largely the > same and the various constructors enforce certain restrictions)", I > was told, no, a new object AddressWithMask was actually required. I've > no problem with a constructor like this, but this is not what people > have been asking for. No, AddressWithMask was merely defined as a concept separate from an IP Network to indicate how the current IPNetwork class was being overloaded with two distinct concepts and hence exhibiting unnecessary behavioural quirks. The use case for "given a netmask and an arbitrary host on that network, give me the appropriate IPNetwork object" has been well established by this discussion (although still isn't particularly well described even in the latest PEP update). This is what "strict=False" covers, and I'm now happy that practicality means this is appropriate default behaviour for the IPNetwork constructor. The use case for "create an IPNetwork object and have it *remember* which particular host address was used to define it" is still not established at all. It is this latter use case which is covered by the AddressWithMask concept (which may be perfectly validly represented as a simple IPNetwork/IPAddress tuple, which was also mentioned when the concern was first raised). The main concern here is to take the "AddressWithMask" behaviour *off* the IPNetwork objects. It interferes with their interpretation as descriptions of networks because they maintain a piece of irrelevant state and pay attention to it in comparison operations. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Mon Sep 28 00:32:35 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 22:32:35 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review, and the inclusion process References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > this is "less useful (strictly removing functionality)" and is an > example of what I explicitly said I was not going to do with ipaddr. (please note the conditional wording here) Assuming that a significant number of people agree that there is a design problem, if you don't want to make the necessary changes, then I don't see a reason why ipaddr would enter the stdlib. The functionality (IP address handling) hasn't really seen a huge demand. On stdlib-sig recently, a number of people complained that our criteria for including existing libraries in the stdlib should be higher (they complained about the quality of some existing modules, including optparse, which by the way prompted the current proposal to get argparse in the stdlib). I think this PEP is a good moment to judge and decide how demanding or tolerant we (and especially the complainers ;-)) want to be. Regards Antoine. From solipsis at pitrou.net Mon Sep 28 00:38:31 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 22:38:31 +0000 (UTC) Subject: [Python-Dev] IO module precisions and exception hierarchy References: <4ABF2047.7050501@gmail.com> <4ABFCDED.8080105@gmail.com> Message-ID: Le Mon, 28 Sep 2009 06:41:17 +1000, Nick Coghlan a ?crit?: > > Not as such - a big exception hierarchy rewrite was rejected, but > nothing specifically limited to the IO exceptions. > > Michael's response cut to the heart of the issue though - a richer IO > exception hierarchy can make life interesting for compatibility purposes > (especially when creating "file-like" interfaces to non-file objects). Well, not more interesting than currently where you need to replicate errno numbers if you want to make the errors precise enough, since an API consumer wanting to check specific error conditions will discriminate on errno. If you don't want to go to that level of perfection, you just have to raise a plain IOError (or OSError :-)) without bothering about errno or subclasses; like you would do today. Regards Antoine. From solipsis at pitrou.net Mon Sep 28 00:44:22 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 27 Sep 2009 22:44:22 +0000 (UTC) Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition V2 References: <4AB3DCC9.5050209@gmail.com> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> <4ABF1B3B.4080106@gmail.com> Message-ID: Le Sun, 27 Sep 2009 14:24:52 -0700, Guido van Rossum a ?crit?: [truncate()] > > What compatibility, though? Compatibility accross the 3.x line. > f.truncate() behaves different in 2.x than > in 3.x, and in 2.x it seems to match the POSIX semantics (i.e. the seek > position is unchanged even though the file size is). Perhaps the changed > semantics was an oversight or a mistake? Perhaps it was, indeed. I don't know who made that decision in the first place. Regards Antoine. From peter at hda3.com Mon Sep 28 01:17:48 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 16:17:48 -0700 Subject: [Python-Dev] PEP 3144 review, and the inclusion process In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> Message-ID: <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> On Sun, Sep 27, 2009 at 3:32 PM, Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> >> this is "less useful (strictly removing functionality)" and is an >> example of what I explicitly said I was not going to do with ipaddr. > > (please note the conditional wording here) Assuming that a significant number of > people agree that there is a design problem, if you don't want to make the > necessary changes, then I don't see a reason why ipaddr would enter the stdlib. I've never said otherwise. In fact, from an email last night, "If what the community requires is the library you've described, then ipaddr is not that library." The changes *you* require make ipaddr significantly less useful to me. I'm not prepared to make those changes in an attempt seek acceptance to the stdlib, especially if the stdlib is in such flux that I'll get to do this again in 18 months. > The functionality (IP address handling) hasn't really seen a huge demand. then no one would be the worse off if I don't make those suggested changes. > On stdlib-sig recently, a number of people complained that our criteria for > including existing libraries in the stdlib should be higher (they complained > about the quality of some existing modules, including optparse, which by the way > prompted the current proposal to get argparse in the stdlib). I think this PEP > is a good moment to judge and decide how demanding or tolerant we (and > especially the complainers ;-)) want to be. Sounds like design by committee to satisfy the squeakiest wheel. I'm trying, but I can't think of anything worse. Cheers, /peter > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From steve at pearwood.info Mon Sep 28 02:13:38 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 28 Sep 2009 10:13:38 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com> Message-ID: <200909281013.39069.steve@pearwood.info> On Mon, 28 Sep 2009 03:53:27 am Peter Moody wrote: > >> ? I *understand* what you're saying, I *understand* that > >> 192.168.1.1/24 isn't a network, > > > > But you still want to treat it as one. > > > > Could you explain what benefit there is for allowing the user to > > create network objects that don't represent networks? Is there a > > use-case where these networks-that-aren't-networks are something > > other than a typo? Under what circumstances would I want to specify > > a network as 192.168.1.1/24 instead of 192.168.1.0/24? > > this is pretty ridiculous. if in the last two months you haven't seen > a single use case, then you haven't been paying attention. No, I haven't read every single email in excruciating detail in this extremely long thread. I'd bet most other people haven't. I'm trying to understand why you want something which, in *my* ignorance, seems patently ridiculous to me: allowing Network objects which aren't Networks, particularly since in private email to me you denied that there was a use-case for the requested (Address object + netmask). But it seems to me that this is exactly what you have in the current implementation, except you call it a Network object, and have rejected the suggestion that the non-network bits of the address should be zeroed. That is, you've rejected the idea of having: >>> IPv4Network(192.168.1.1/24) IPv4Network(192.168.1.0/24) as reducing functionality, presumably because the address 192.168.1.1 is lost. Sounds just like an Address+netmask to me, with added network-like behaviour. Some people have requested a way of explicitly calculating a network from an Address and netmask, and you've been hostile to the idea. But it seems to me that your API does exactly that. I'm not the only person who thinks your API conflates two different concepts into a single class, and I'm trying to see your side of the argument. But your hostile attitude isn't making it easy. -- Steven D'Aprano From fperez.net at gmail.com Mon Sep 28 02:31:00 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 28 Sep 2009 00:31:00 +0000 (UTC) Subject: [Python-Dev] PEP 389: argparse - new command line parsing module References: Message-ID: On Sun, 27 Sep 2009 14:57:34 -0700, Brett Cannon wrote: > I am going to state upfront that I am +1 for this and I encouraged > Steven to submit this PEP on the stdlib-SIG. I still remember watching > Steven's lightning talk at PyCon 2009 on argparse and being impressed by > it (along with the rest of the audience which was obviously impressed as > well). > > I think argprase is a good improvement over what we have now (getopt and > optparse), it's stable, considered best-of-breed by the community for a > while (as shown as how many times argparse has been suggestion for > inclusion), and Steven is already a core committer so is already set to > maintain the code. That covers the usual checklist we have for even > looking at a PEP to add a module to the standard library. FWIW, +1 from IPython: we ship a private copy of argparse as well (we use it in several places but we want the basic ipython to be installable just with the stdlib). Steven was gracious enough to relicense it as BSD at our request: http://mail.scipy.org/pipermail/ipython-dev/2009-September/005537.html so we could ship this copy without any GPL incompatibilities, but we'd much rather rely on stdlib components. Best, f From peter at hda3.com Mon Sep 28 02:31:56 2009 From: peter at hda3.com (Peter Moody) Date: Sun, 27 Sep 2009 17:31:56 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909281013.39069.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com> <200909281013.39069.steve@pearwood.info> Message-ID: <8517e9350909271731ra859ba5yda1d5f30e13a8b26@mail.gmail.com> On Sun, Sep 27, 2009 at 5:13 PM, Steven D'Aprano wrote: > On Mon, 28 Sep 2009 03:53:27 am Peter Moody wrote: > >> >> ? I *understand* what you're saying, I *understand* that >> >> 192.168.1.1/24 isn't a network, >> > >> > But you still want to treat it as one. >> > >> > Could you explain what benefit there is for allowing the user to >> > create network objects that don't represent networks? Is there a >> > use-case where these networks-that-aren't-networks are something >> > other than a typo? Under what circumstances would I want to specify >> > a network as 192.168.1.1/24 instead of 192.168.1.0/24? >> >> this is pretty ridiculous. if in the last two months you haven't seen >> a single use case, then you haven't been paying attention. > > No, I haven't read every single email in excruciating detail in this > extremely long thread. I'd bet most other people haven't. > > I'm trying to understand why you want something which, in *my* > ignorance, seems patently ridiculous to me: allowing Network objects > which aren't Networks, particularly since in private email to me you > denied that there was a use-case for the requested (Address object + > netmask). no, it seems as though you're either misrepresenting my position or you misunderstood what I said. I said that there wasn't a use-case for explicitly pulling that functionality out into a a new class, and having 6 classes. > But it seems to me that this is exactly what you have in the > current implementation, except you call it a Network object, and have > rejected the suggestion that the non-network bits of the address should > be zeroed. I have not rejected this. I have rejected the suggestion that the current IPv?Network classes do this by default. > That is, you've rejected the idea of having: > >>>> IPv4Network(192.168.1.1/24) > IPv4Network(192.168.1.0/24) yes, I and everyone have rejected that idea. this should either raise an error, or do what it does now, that is, return IPv4Network('192.168.1.1/24') > as reducing functionality, presumably because the address 192.168.1.1 is > lost. good guess. > Sounds just like an Address+netmask to me, with added > network-like behaviour. > > Some people have requested a way of explicitly calculating a network > from an Address and netmask, and you've been hostile to the idea. But > it seems to me that your API does exactly that. You mean the suggestion to include an IPv?Network object attribute with the IPv?Address objects? I thought that was dropped long ago. > I'm not the only person who thinks your API conflates two different > concepts into a single class, and I'm trying to see your side of the > argument. But your hostile attitude isn't making it easy. apologies if you find me hostile. I'm not sure how you'd expect me to be if, after 2 months 200+ messages, both on list and off, you were unable to recall a single use-case. > -- > Steven D'Aprano > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From solipsis at pitrou.net Mon Sep 28 02:32:49 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 00:32:49 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review, and the inclusion process References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> Message-ID: Peter Moody hda3.com> writes: > > I've never said otherwise. In fact, from an email last night, "If what > the community requires is the library you've described, then ipaddr is > not that library." The changes *you* require make ipaddr significantly > less useful to me. I'm not prepared to make those changes in an > attempt seek acceptance to the stdlib, especially if the stdlib is in > such flux that I'll get to do this again in 18 months. Well, then I'm not sure why we have a PEP at all. If you don't want any significant changes and if you consider it to be *your* library, ipaddr can remain a third-party package that interested people can easily install (no pun ;-)) since AFAIK it's pure Python. It will also make maintenance easier for you, while freeing us (core developers) from having to bother about it in our daily development tasks. At least that's what I would advocate right now - not sure about what others think. Regards Antoine. From bioinformed at gmail.com Mon Sep 28 02:43:16 2009 From: bioinformed at gmail.com (Kevin Jacobs ) Date: Sun, 27 Sep 2009 20:43:16 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <2e1434c10909271743k68b51a36m8b1717a89d48bcf@mail.gmail.com> On Sun, Sep 27, 2009 at 8:31 PM, Fernando Perez wrote: > On Sun, 27 Sep 2009 14:57:34 -0700, Brett Cannon wrote: > > > I am going to state upfront that I am +1 for this and I encouraged > > Steven to submit this PEP on the stdlib-SIG. I still remember watching > > Steven's lightning talk at PyCon 2009 on argparse and being impressed by > > it (along with the rest of the audience which was obviously impressed as > > well). > > > > I think argprase is a good improvement over what we have now (getopt and > > optparse), it's stable, considered best-of-breed by the community for a > > while (as shown as how many times argparse has been suggestion for > > inclusion), and Steven is already a core committer so is already set to > > maintain the code. That covers the usual checklist we have for even > > looking at a PEP to add a module to the standard library. > > FWIW, +1 from IPython: we ship a private copy of argparse as well (we use > it in several places but we want the basic ipython to be installable just > with the stdlib). Steven was gracious enough to relicense it as BSD at our > request: > +1 Coincidentally I was converting several packages to argparse as I saw the python-dev mail come in. A great deal of hairy code needed to jury-rig sub-commands and other "advanced processing" with optparse is simply disappearing. I like the idea that stdlib can evolve to encompass best of breed solutions and that effort is being expended to improve existing functionality even if it means increasing the diversity of APIs. ~Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Mon Sep 28 02:51:33 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 00:51:33 +0000 (UTC) Subject: [Python-Dev] PEP 389: argparse - new command line parsing module References: Message-ID: Hello, I am neutral on the idea of adding argparse. However, I'm -1 on deprecating optparse. It is very widely used (tons of scripts use it), and ok for many uses; deprecating it is totally unhelpful and gratuitous. Regards Antoine. From greg.ewing at canterbury.ac.nz Mon Sep 28 03:01:21 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 28 Sep 2009 14:01:21 +1300 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4ABFCC6B.3020409@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <4ABFCC6B.3020409@gmail.com> Message-ID: <4AC00AE1.7080009@canterbury.ac.nz> Nick Coghlan wrote: > The use case for "given a netmask and an arbitrary host on that network, > give me the appropriate IPNetwork object" has been well established by > this discussion (although still isn't particularly well described even > in the latest PEP update). This is what "strict=False" covers, I think I'd rather have a separate constructor function or method for this, rather than a somewhat cryptically named boolean parameter. -- Greg From rdmurray at bitdance.com Mon Sep 28 03:01:35 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Sun, 27 Sep 2009 21:01:35 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> Message-ID: On Sun, 27 Sep 2009 at 13:59, Peter Moody wrote: > On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou wrote: >> Peter Moody hda3.com> writes: >>> >>>>>> def parse_net_and_addr(s): >>>>>> ?return (IPNetwork(s), IPAddress(s.split('/')[0])) >>>>> >>>>> I've only heard talk of new classes and new methods, not new >>>>> constructor functions. >>>> >>>> Well, "method" in that context meant "class method" since the results aren't >>>> dependent on a particular instance. Of course, both a class method or a >>>> module-level function would be fine. >>> >>> so this is not the response I got when I asked what was required >>> before. Would adding this constructor function satisfy your concerns >>> (given sensible strict settings in the constructor, etc)? >> >> Assuming the Network type loses the notion of a specific host (or host address, >> or `ip`) attached to it, yes. > > this is "less useful (strictly removing functionality)" and is an > example of what I explicitly said I was not going to do with ipaddr. In that case, I vote -1 on the inclusion of ipaddr in the stdlib. --David From greg.ewing at canterbury.ac.nz Mon Sep 28 03:11:06 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 28 Sep 2009 14:11:06 +1300 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4ABFD9D6.9070602@voidspace.org.uk> References: <4ABF2047.7050501@gmail.com> <4ABFCDED.8080105@gmail.com> <4ABFD9D6.9070602@voidspace.org.uk> Message-ID: <4AC00D2A.6020504@canterbury.ac.nz> Michael Foord wrote: > Some of the error messages are truly awful though as things stand, > especially for someone new to Python. Try to read from a file handle > opened in read mode for example: IOError: [Errno 9] Bad file descriptor Subdividing the IOError exception won't help with that, because all you have to go on when deciding which exception to raise is the error code returned by the OS. If the same error code results from a bunch of different things, there's not much Python can do to sort them out. The error messages produced for the various error codes could perhaps be improved, but that's an orthogonal issue. -- Greg From greg.ewing at canterbury.ac.nz Mon Sep 28 03:26:17 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 28 Sep 2009 14:26:17 +1300 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> Message-ID: <4AC010B9.4050003@canterbury.ac.nz> Peter Moody wrote: > On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou wrote: > >> Assuming the Network type loses the notion of a specific host (or host address, >> or `ip`) attached to it, yes. > > this is "less useful (strictly removing functionality)" and is an > example of what I explicitly said I was not going to do with ipaddr. Would you be kind enough to explain exactly what use case you have for retaining this information? Apologies if you've done so before -- I've been trying to follow this discussion, but that point doesn't seem to have come through clearly. -- Greg From doug.hellmann at gmail.com Mon Sep 28 03:22:20 2009 From: doug.hellmann at gmail.com (Doug Hellmann) Date: Sun, 27 Sep 2009 21:22:20 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4ABFE075.2070309@voidspace.org.uk> References: <4ABFE075.2070309@voidspace.org.uk> Message-ID: On Sep 27, 2009, at 6:00 PM, Michael Foord wrote: > Brett Cannon wrote: >> I am going to state upfront that I am +1 for this and I encouraged >> Steven to submit this PEP on the stdlib-SIG. I still remember >> watching >> Steven's lightning talk at PyCon 2009 on argparse and being impressed >> by it (along with the rest of the audience which was obviously >> impressed as well). >> >> I think argprase is a good improvement over what we have now (getopt >> and optparse), it's stable, considered best-of-breed by the community >> for a while (as shown as how many times argparse has been suggestion >> for inclusion), and Steven is already a core committer so is already >> set to maintain the code. That covers the usual checklist we have for >> even looking at a PEP to add a module to the standard library. >> >> > > I've used argparse and really like it. I could also have done with > the subcommand support in recent changes to unittest. > > +1 for inclusion. +1 as well. I wish I had been able to use this library for a recent project using sub-commands. Doug From drobinow at gmail.com Mon Sep 28 04:01:09 2009 From: drobinow at gmail.com (David Robinow) Date: Sun, 27 Sep 2009 22:01:09 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC010B9.4050003@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> <4AC010B9.4050003@canterbury.ac.nz> Message-ID: <4eb0089f0909271901u3d61319bi1372ab1b6f3e7d37@mail.gmail.com> On Sun, Sep 27, 2009 at 9:26 PM, Greg Ewing wrote: > Would you be kind enough to explain exactly what use > case you have for retaining this information? > > Apologies if you've done so before -- I've been > trying to follow this discussion, but that point > doesn't seem to have come through clearly. Please don't apologize. It shouldn't matter how many times it's been mentioned on the mailing list. The PEP process is intended to summarize the discussion. A PEP should pretty much stand alone, with specific external links when necessary. The use case should be added to the PEP. From digitalxero at gmail.com Mon Sep 28 02:12:52 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Sun, 27 Sep 2009 18:12:52 -0600 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4ABFCC6B.3020409@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <4ABFCC6B.3020409@gmail.com> Message-ID: Looking though the tests you have setup for ipaddr it is clear that you want the following to be True ip1 = ipaddr.IPv4Network('1.1.1.0/24') ip2 = ipaddr.IPv4Network('1.1.1.1/24') ip1 == ip2 based on this test self.assertEquals(ip1.compare_networks(ip2), 0) but your ><= operators all compare against _ip instead of network. I submitted a patch for review @ http://codereview.appspot.com/124057 From steven.bethard at gmail.com Mon Sep 28 04:55:46 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 27 Sep 2009 19:55:46 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: On Sun, Sep 27, 2009 at 5:51 PM, Antoine Pitrou wrote: > I am neutral on the idea of adding argparse. However, I'm -1 on deprecating > optparse. It is very widely used (tons of scripts use it), and ok for many uses; > deprecating it is totally unhelpful and gratuitous. Could you elaborate? If you are worried about Python 2.X scripts, note from the PEP that the only things that will ever show up in Python 2.X are: PendingDeprecation warnings, which by default are not displayed, and documentation notes directing users of getopt and optparse to argparse. I think these messages are useful for folks using 2.X who some day would like to migrate to 3.X, and at the same time should have zero effect on existing scripts. If you think getopt and optparse should stick around in 3.X, why is that? If you think there are things that getopt and optparse do better than argparse, could you please give some examples? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From steven.bethard at gmail.com Mon Sep 28 05:18:17 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 27 Sep 2009 20:18:17 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> Message-ID: On Sun, Sep 27, 2009 at 8:08 PM, Benjamin Peterson wrote: > 2009/9/27 Steven Bethard : >> If you think getopt and optparse should stick around in 3.X, why is >> that? If you think there are things that getopt and optparse do better >> than argparse, could you please give some examples? > > Transitioning to Python 3 is already a pain. bytes/str/unicode things > are going to be enough by themselves to drive people nuts. We don't > want to be forcing them to change APIs if optparse is already working > just fine for them. The job is the stdlib is not to force people to > use the "best" or "right" tools. Several years in the future I would > be more supportive of depreacting optparse, but more ways in which 2.x > and 3.x grow farther apart are not going to help jumping the great > version divide. The first release where any real deprecation message would show up is Python 3.4, more than 3 years away. If you think 3 years isn't long enough for people to be over the Python 3 transition, let's stick in another version in there and make it 4.5 years. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From steven.bethard at gmail.com Mon Sep 28 05:46:03 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 27 Sep 2009 20:46:03 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> Message-ID: On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson wrote: > 2009/9/27 Steven Bethard : >> The first release where any real deprecation message would show up is >> Python 3.4, more than 3 years away. If you think 3 years isn't long >> enough for people to be over the Python 3 transition, let's stick in >> another version in there and make it 4.5 years. > > So, why even bother deprecating it if nobody is going to see the warnings? I feel like I'm repeating the PEP, but here it is again anyway. There will be messages in the docs and pending deprecation warnings (which don't show up by default but can be requested) starting in Python 2.7 and 3.2. Regular deprecation warnings wouldn't show up until Python 3.4, 3 years away. This compromise was intended exactly to address the issue you brought up about people getting over the Python 3 transition. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? ? ? ? ?--- The Hiphopopotamus From martin at v.loewis.de Mon Sep 28 05:57:04 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Sep 2009 05:57:04 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <200909280330.30643.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> Message-ID: <4AC03410.4010104@v.loewis.de> >> Finally, to Stephen's point about seeing the other side of the >> argument, I wrote this offlist a week ago: >> >> I *understand* what you're saying, I *understand* that >> 192.168.1.1/24 isn't a network, > > But you still want to treat it as one. > > Could you explain what benefit there is for allowing the user to create > network objects that don't represent networks? Is there a use-case > where these networks-that-aren't-networks are something other than a > typo? Under what circumstances would I want to specify a network as > 192.168.1.1/24 instead of 192.168.1.0/24? > It's fairly obvious to me why the library should support 192.168.1.1/24 as an input, and return a network. End-users are likely going to enter such things (e.g. 82.94.164.162/29), as they will know one IP address in the network (namely, of one machine that they are looking at), and they will know the prefix length (more often, how large the network is - 8 or 16 or 32). So very clearly, end users should not be required to make the computation in their heads. So Python code has to make the computation, and it seems most natural that the IP library is the piece of code that is able to compute a network out of that input. Does that answer your question? Regards, Martin From ubershmekel at gmail.com Mon Sep 28 05:59:45 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Mon, 28 Sep 2009 06:59:45 +0300 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> Message-ID: <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com> -1 for deprecating getopt. getopt is super-simple and especially useful for c programmers learning python. +1 for argparse.+1 for eventual deprecation of optparse - optparse and argparse have a very similar syntax and having both is just confusing. tsboapooowtdi On Mon, Sep 28, 2009 at 6:46 AM, Steven Bethard wrote: > On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson > wrote: > > 2009/9/27 Steven Bethard : > >> The first release where any real deprecation message would show up is > >> Python 3.4, more than 3 years away. If you think 3 years isn't long > >> enough for people to be over the Python 3 transition, let's stick in > >> another version in there and make it 4.5 years. > > > > So, why even bother deprecating it if nobody is going to see the > warnings? > > I feel like I'm repeating the PEP, but here it is again anyway. There > will be messages in the docs and pending deprecation warnings (which > don't show up by default but can be requested) starting in Python 2.7 > and 3.2. Regular deprecation warnings wouldn't show up until Python > 3.4, 3 years away. This compromise was intended exactly to address the > issue you brought up about people getting over the Python 3 > transition. > > Steve > -- > Where did you get that preposterous hypothesis? > Did Steve tell you that? > --- The Hiphopopotamus > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Mon Sep 28 06:09:53 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Sep 2009 06:09:53 +0200 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <4AC03711.6000203@v.loewis.de> > If you think getopt and optparse should stick around in 3.X, why is > that? If you think there are things that getopt and optparse do better > than argparse, could you please give some examples? I personally consider getopt superior to both optparse and argparse. Those are terribly verbose in specifying arguments, whereas getopt's sequence-of-letters is really nice and compact. Regards, Martin From benjamin at python.org Mon Sep 28 05:08:37 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 27 Sep 2009 22:08:37 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> 2009/9/27 Steven Bethard : > If you think getopt and optparse should stick around in 3.X, why is > that? If you think there are things that getopt and optparse do better > than argparse, could you please give some examples? Transitioning to Python 3 is already a pain. bytes/str/unicode things are going to be enough by themselves to drive people nuts. We don't want to be forcing them to change APIs if optparse is already working just fine for them. The job is the stdlib is not to force people to use the "best" or "right" tools. Several years in the future I would be more supportive of depreacting optparse, but more ways in which 2.x and 3.x grow farther apart are not going to help jumping the great version divide. -- Regards, Benjamin From benjamin at python.org Mon Sep 28 05:41:16 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 27 Sep 2009 22:41:16 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> Message-ID: <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> 2009/9/27 Steven Bethard : > On Sun, Sep 27, 2009 at 8:08 PM, Benjamin Peterson wrote: >> 2009/9/27 Steven Bethard : >>> If you think getopt and optparse should stick around in 3.X, why is >>> that? If you think there are things that getopt and optparse do better >>> than argparse, could you please give some examples? >> >> Transitioning to Python 3 is already a pain. bytes/str/unicode things >> are going to be enough by themselves to drive people nuts. We don't >> want to be forcing them to change APIs if optparse is already working >> just fine for them. The job is the stdlib is not to force people to >> use the "best" or "right" tools. Several years in the future I would >> be more supportive of depreacting optparse, but more ways in which 2.x >> and 3.x grow farther apart are not going to help jumping the great >> version divide. > > The first release where any real deprecation message would show up is > Python 3.4, more than 3 years away. If you think 3 years isn't long > enough for people to be over the Python 3 transition, let's stick in > another version in there and make it 4.5 years. So, why even bother deprecating it if nobody is going to see the warnings? -- Regards, Benjamin From steven.bethard at gmail.com Mon Sep 28 06:24:54 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 27 Sep 2009 21:24:54 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC03711.6000203@v.loewis.de> References: <4AC03711.6000203@v.loewis.de> Message-ID: On Sun, Sep 27, 2009 at 9:09 PM, "Martin v. L?wis" wrote: >> If you think getopt and optparse should stick around in 3.X, why is >> that? If you think there are things that getopt and optparse do better >> than argparse, could you please give some examples? > > I personally consider getopt superior to both optparse and argparse. > Those are terribly verbose in specifying arguments, whereas getopt's > sequence-of-letters is really nice and compact. Thanks for the concrete example. Although I'm unconvinced that the characters you save in the sequence of letters in the getopt.getopt call aren't afterwards wasted on type conversions, if/else statements and variable assignments in the subsequent loop, it would be pretty simple to add to argparse something like:: ArgumentParser.add_getopt_arguments(options[, long_options]) Then you could replace your getopt code:: try: opts, args = getopt.getopt(sys.argv[1:], "ho:v", ["help", "output="]) except getopt.GetoptError, err: # print help information and exit: print str(err) # will print something like "option -a not recognized" usage() sys.exit(2) output = None verbose = False for o, a in opts: if o == "-v": verbose = True elif o in ("-h", "--help"): usage() sys.exit() elif o in ("-o", "--output"): output = a else: assert False, "unhandled option" with argparse code like:: parser = argparse.ArgumentParser() parser.add_getopt_arguments("ho:v", ["help", "output="]) args = parser.parse_args() verbose = args.v output = args.o or args.output If something like this were available, would you be alright with deprecating getopt? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From steven.bethard at gmail.com Mon Sep 28 05:56:03 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 27 Sep 2009 20:56:03 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <1afaf6160909272049m30328a39tf1bf63faa31d33c2@mail.gmail.com> References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> <1afaf6160909272049m30328a39tf1bf63faa31d33c2@mail.gmail.com> Message-ID: On Sun, Sep 27, 2009 at 8:49 PM, Benjamin Peterson wrote: > 2009/9/27 Steven Bethard : >> On Sun, Sep 27, 2009 at 8:41 PM, Benjamin Peterson wrote: >>> 2009/9/27 Steven Bethard : >>>> The first release where any real deprecation message would show up is >>>> Python 3.4, more than 3 years away. If you think 3 years isn't long >>>> enough for people to be over the Python 3 transition, let's stick in >>>> another version in there and make it 4.5 years. >>> >>> So, why even bother deprecating it if nobody is going to see the warnings? >> >> I feel like I'm repeating the PEP, but here it is again anyway. There >> will be messages in the docs and pending deprecation warnings (which >> don't show up by default but can be requested) starting in Python 2.7 >> and 3.2. Regular deprecation warnings wouldn't show up until Python >> 3.4, 3 years away. This compromise was intended exactly to address the >> issue you brought up about people getting over the Python 3 >> transition. > > But that doesn't tell me why we should deprecate optparse, when it may > work perfectly well for some people. Because it's basically unmaintained, and anything you can do in optparse you can do in argparse with almost identical syntax. So encouraging people to move from getopt and optparse to argparse is a net gain for us as Python maintainers -- that's two fewer modules in the standard library that someone has to take care of bug reports for. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From solipsis at pitrou.net Mon Sep 28 08:51:03 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 06:51:03 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > > Could you explain what benefit there is for allowing the user to create > > network objects that don't represent networks? Is there a use-case > > where these networks-that-aren't-networks are something other than a > > typo? Under what circumstances would I want to specify a network as > > 192.168.1.1/24 instead of 192.168.1.0/24? > > > [...] > > So Python code has to make the computation, and it seems most natural > that the IP library is the piece of code that is able to compute a > network out of that input. The thing is, it doesn't create a network, it creates a hybrid "network plus host" which retains knowledge about the original host (that is, 192.168.1.1 rather than simply 192.168.1.0, if you enter "192.168.1.1/24"). That's what the OP meant with "networks-that-aren't-networks", and that's what people are objecting to. Regards Antoine. From chambon.pascal at gmail.com Mon Sep 28 10:08:16 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Mon, 28 Sep 2009 10:08:16 +0200 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition V2 In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB3E594.5090907@mrabarnett.plus.com> <97917549-9B72-451C-9C41-AE6BBCFF6A6B@fuhm.net> <200909201007.11923.steve@pearwood.info> <4AB58159.3040303@canterbury.ac.nz> <4AB5D6CB.6050604@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> <4ABF1B3B.4080106@gmail.com> Message-ID: <4AC06EF0.50509@gmail.com> Antoine Pitrou a ?crit : > Hello, > > >> So here is the proposed semantic, which matches established conventions: >> >> *IOBase.truncate(n: int = None) -> int* >> > [...] > > I still don't think there is a sufficient benefit in breaking > compatibility. If you want the file pointer to remain the same, you can > save it first and restore it afterwards manually. > > Sure, but won't this truncate become some kind of a burden for py3k, if it's twice misleading (it's not a real truncation since it can extend the file, and it's not even a truncation or resizing in posix/win32 style, since the filepointer is moved) ? Since it was an undocumented behaviour, and py3k doesn't seem to be present yet in production environments (or is it ?), I'd promote this late-but-maybe-not-too-late change. But if the consensus prefers the current behaviour, well, it'll be OK to me too, as long as it's sufficiently documented and advertised. >> *Propositions of doc update* >> > > Please open tracker issues for these kinds of suggestions. > Is the tracker Ok for simple suggestions too ? I thought it was rather for obvious bugfixes, and to-be-discused propositions had better be in mailing-lists... OK then, I'll open bugtracker issues for these. B-) > Instead of "than size", perhaps "than n". > Whoups, indeed >_< Actually the signature would rather be: *IOBase.truncate(size: int = None) -> int* And I forgot to mention that truncate returns the new file size (according to the current PEP)... > Should an exception be raised if start and/or end are out of range? I'd advocate it yep, for the sake of "explicit errors". However, should it be a ValueError (the ones io functions normally use) or an IndexError (which is technically more accurate, but might confuse the user) ? Regards, Pascal -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Mon Sep 28 10:25:45 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 28 Sep 2009 10:25:45 +0200 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: References: <4AB67430.2060306@egenix.com> Message-ID: <4AC07309.1010608@egenix.com> Zooko O'Whielacronx wrote: > Folks: > > I'm sorry, I think I didn't make my concern clear. My users, and lots > of other users, are having a problem with incompatibility between > Python binary extension modules. One way to improve the situation > would be if the Python devs would use their "bully pulpit" -- their > unique position as a source respected by all Linux distributions -- > and say "We recommend that Linux distributions use UCS4 for > compatibility with one another". This would not abrogate anyone's > ability to choose their preferred setting nor, as far as I can tell, > would it interfere with the ongoing development of Python. -1 Please note that we did not choose to ship Python as UCS4 binary on Linux - the Linux distributions did. The Python default is UCS2 for a good reason: it's a good trade-off between memory consumption, functionality and performance. As already mentioned, I also don't understand how the changing the Python default on Linux would help your users in any way - if you let distutils compile your extensions, it's automatically going to use the right Unicode setting for you (as well as your users). Unfortunately, this automatic support doesn't help you when shipping e.g. setuptools eggs, but this is a tool problem, not one of Python: setuptools completely ignores the fact that there are two ways to build Python. I'd suggest you ask the tool maintainers to adjust their tools to support the Python Unicode option. > Here are the details: > > I'm the maintainer of several Python packages. I work hard to make it > easy for users, even users who don't know anything about Python, to > use my software. There have been many pain points in this process and > I've spent a lot of time on it for about three years now working on > packaging, including the tools such as setuptools and distutils and > the new "distribute" tool. Python packaging has been improving during > these years -- things are looking up. > > One of the remaining pain points is that I can distribute binaries of > my Python extension modules for Windows or Mac, but if I distribute a > binary Python extension module on Linux, then if the user has a > different UCS2/UCS4 setting then they won't be able to use the > extension module. The current de facto standard for Linux is UCS4 -- > it is used by Debian, Ubuntu, Fedora, RHEL, OpenSuSE, etc. etc.. The > vast majority of Linux users in practice have UCS4, and most binary > Python modules are compiled for UCS4. > > That means that a few folks will get left out. Those folks, from my > experience, are people who built their python executable themselves > without specifying an override for the default, and the smaller Linux > distributions who insist on doing whatever upstream Python devs > recommend instead of doing whatever the other Linux distros are doing. > One of the data points that I reported was a Python interpreter that > was built locally on an Ubuntu server. Since the person building it > didn't know to override the default setting of --enable-unicode, he > ended up with a Python interpreter built for UCS2, even though all the > Python extension modules shipped by Ubuntu were built with UCS4. People building their own Python version will usually also build their own extensions, so I don't really believe that the above scenario is very common. Also note that Python will complain loudly when you try to load a UCS2 extension in a UCS4 build and vice-versa. We've made sure that any extension using the Python Unicode C API has to be built for the same UCS version of Python. This is done by using different names for the C APIs at the C level. > These are not isolated incidents. The following google searches > suggest that a number of people spend time trying to figure out why > Python extension modules fail on their linux systems: > > http://www.google.com/search?q=PyUnicodeUCS4_FromUnicode+undefined+symbol > http://www.google.com/search?q=+PyUnicodeUCS2_FromUnicode+undefined+symbol > http://www.google.com/search?q=_PyUnicodeUCS2_AsDefaultEncodedString+undefined+symbol Perhaps we should add a FAQ entry for these linker errors (which are caused by the mentioned C API changes to prevent mixing UCS version) ?! Here's a quick way to determine you Python Unicode build type: python -c "import sys;print((sys.maxunicode<66000)and'UCS2'or'UCS4')" Perhaps we should include this info as well as an 32/64-bit indicator and the processor type in the Python startup line: # python Python 2.6 (r26:66714, Feb 3 2009, 20:49:49, UCS4, 64-bit, x86_64) [GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2 Type "help", "copyright", "credits" or "license" for more information. This would help users find the right binaries to install as extension. > Another data point is the Mandriva Linux distribution. It is probably > much smaller than Debian, Ubuntu, or RedHat, but it is still one of > the major, well-known distributions. I requested of the Python > maintainer for Mandriva, Michael Scherer, that they switch from UCS2 > to UCS4 in order to reduce compatibility problems like these. His > answer as I understood it was that it is best to follow the > recommendations of the upstream Python devs by using the default > setting instead of choosing a setting for himself. Which is IMHO what all Linux distributions should have done. Distributions should really not be put in charge of upstream coding design decisions. Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 28 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Mon Sep 28 10:36:07 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 28 Sep 2009 10:36:07 +0200 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: <4AC07309.1010608@egenix.com> References: <4AB67430.2060306@egenix.com> <4AC07309.1010608@egenix.com> Message-ID: <4AC07577.2040903@egenix.com> M.-A. Lemburg wrote: > Also note that Python will complain loudly when you try to load > a UCS2 extension in a UCS4 build and vice-versa. We've made sure > that any extension using the Python Unicode C API has to be built > for the same UCS version of Python. This is done by using different > names for the C APIs at the C level. > >> These are not isolated incidents. The following google searches >> suggest that a number of people spend time trying to figure out why >> Python extension modules fail on their linux systems: >> >> http://www.google.com/search?q=PyUnicodeUCS4_FromUnicode+undefined+symbol >> http://www.google.com/search?q=+PyUnicodeUCS2_FromUnicode+undefined+symbol >> http://www.google.com/search?q=_PyUnicodeUCS2_AsDefaultEncodedString+undefined+symbol > > Perhaps we should add a FAQ entry for these linker errors > (which are caused by the mentioned C API changes to prevent > mixing UCS version) ?! There already is one: http://www.python.org/doc/faq/extending/#when-importing-module-x-why-do-i-get-undefined-symbol-pyunicodeucs2 I wonder why it doesn't show up in the Google searches. > Here's a quick way to determine you Python Unicode build type: > > python -c "import sys;print((sys.maxunicode<66000)and'UCS2'or'UCS4')" > > Perhaps we should include this info as well as an 32/64-bit indicator > and the processor type in the Python startup line: > > # python > Python 2.6 (r26:66714, Feb 3 2009, 20:49:49, UCS4, 64-bit, x86_64) > [GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > > This would help users find the right binaries to install as > extension. Regards, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 28 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From floris.bruynooghe at gmail.com Mon Sep 28 10:38:20 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Mon, 28 Sep 2009 09:38:20 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com> References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com> Message-ID: <20090928083820.GA488@laurie.devork> On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote: > -1 for deprecating getopt. getopt is super-simple and especially useful for > c programmers learning python. > > +1 for argparse.+1 for eventual deprecation of optparse - optparse and > argparse have a very similar syntax and having both is just > confusing. tsboapooowtdi +1 on all of this :-) It would be a shame to see getopt go but optparse -> argparse seems logical. Floris From chambon.pascal at gmail.com Mon Sep 28 10:50:51 2009 From: chambon.pascal at gmail.com (Pascal Chambon) Date: Mon, 28 Sep 2009 10:50:51 +0200 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4ABF4996.6000900@canterbury.ac.nz> References: <4ABF2047.7050501@gmail.com> <4ABF4996.6000900@canterbury.ac.nz> Message-ID: <4AC078EB.1090500@gmail.com> > >> +-InvalidFileNameError (filepath max lengths, or "? / : " characters >> in a windows file name...) > > This might be a bit too precise. Unix just has EINVAL, which > covers any kind of invalid parameter, not just file names. > Allright thanks, an InvalidParameter (or similar) exception should do it better then. > Personally I'd love to see a richer set of exceptions for IO errors, > so long as they can be implemented for all supported platforms and no > information (err number from the os) is lost. > > I've been implementing a fake 'file' type [1] for Silverlight which > does IO operations using local browser storage. The use case is for an > online Python tutorial running in the browser [2]. Whilst implementing > the exception behaviour (writing to a file open in read mode, etc) I > considered improving the exception messages as they are very poor - > but decided that being similar to CPython was more important. > > Michael > > [1] > http://code.google.com/p/trypython/source/browse/trunk/trypython/app/storage.py > and > http://code.google.com/p/trypython/source/browse/trunk/trypython/app/tests/test_storage.py > > [2] http://www.trypython.org/ > Cool stuff :-) It's indeed quite unsure at the moment which exceptions it will really be possible (and relevant) to implement in a cross-platform way... I guess I should use my own fileio implementation as a playground and a proof of concept, before we specify anything for CPython. > What happens isn't specified, but in practice (with the current > implementation) the overwriting will happen at the byte level, without > any check for correctness at the character level. > > Actually, read+write text streams are implemented quite crudely, and > little testing is done of them. The reason, as you discovered, is that > the semantics are too weak, and it is not obvious how stronger semantics > could look like. People wanting to do sophisticated random reads+writes > over a text file should probably handle the encoding themselves and > access the file at the binary level. > It sounds ok to me, as long as we notify users about this danger (I've myself just realized about it). Most newcomers may happily open an UTF8 text file, and read/write in it carelessly, without realizing that the characters they write actually screw up the file... > How about just making IOError = OSError, and introducing your proposed > subclasses? Does the usage of IOError vs OSError have *any* useful > semantics? > I though that OSError dealt with a larger set of errors than IOError, but after checking the errno codes, it seems that they're all more or less related to IO problems (if we include interprocess communication in I/O). So theoretically, IOErrors and OSErrors might be merged. Note that in this case, windowsErrors would have to become children of EnvironmentError, because windows error code really seem to go farther than io errors (they deal with recursion limits, thousands of PC parameters...). The legacy is so heavy that OSError would have to remain as is, I think, but we might simply forget it in new io modules, and concentrate on an IOError hierarchy to provide all the info needed by the developer. > >> Some of the error messages are truly awful though as things stand, >> especially for someone new to Python. Try to read from a file handle >> opened in read mode for example: IOError: [Errno 9] Bad file descriptor > Subdividing the IOError exception won't help with > that, because all you have to go on when deciding > which exception to raise is the error code returned > by the OS. If the same error code results from a > bunch of different things, there's not much Python > can do to sort them out. > Well, you don't only have the error number, you also have the context of this exception. IOErrors subclasses would particularly be useful in a "high level" IO contect, when each single method can issue lots of system calls (to check the file, lock it, edit it...). If the error is raised during your locking operation, you can decide to sort it as "LockingError" even if the error code provided might appear in several different situations. Regards, Pascal From fuzzyman at voidspace.org.uk Mon Sep 28 12:03:35 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 28 Sep 2009 11:03:35 +0100 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4AC00D2A.6020504@canterbury.ac.nz> References: <4ABF2047.7050501@gmail.com> <4ABFCDED.8080105@gmail.com> <4ABFD9D6.9070602@voidspace.org.uk> <4AC00D2A.6020504@canterbury.ac.nz> Message-ID: <4AC089F7.8030004@voidspace.org.uk> Greg Ewing wrote: > Michael Foord wrote: > >> Some of the error messages are truly awful though as things stand, >> especially for someone new to Python. Try to read from a file handle >> opened in read mode for example: IOError: [Errno 9] Bad file descriptor > > Subdividing the IOError exception won't help with > that, because all you have to go on when deciding > which exception to raise is the error code returned > by the OS. If the same error code results from a > bunch of different things, there's not much Python > can do to sort them out. > Well, the file type knows what mode it is opened in so catching these errors is really not difficult at all. Michael > The error messages produced for the various error > codes could perhaps be improved, but that's an > orthogonal issue. > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From ndbecker2 at gmail.com Mon Sep 28 13:09:59 2009 From: ndbecker2 at gmail.com (Neal Becker) Date: Mon, 28 Sep 2009 07:09:59 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module References: Message-ID: If the plan is to migrate from optparse to argparse, this could be made a bit easier. If it weren't for the fact that some names are different in argparse than optparse, I believe many optparse usages could port with no change. From fuzzyman at voidspace.org.uk Mon Sep 28 13:13:32 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 28 Sep 2009 12:13:32 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <4AC09A5C.9060007@voidspace.org.uk> Neal Becker wrote: > If the plan is to migrate from optparse to argparse, this could be made a > bit easier. If it weren't for the fact that some names are different in > argparse than optparse, I believe many optparse usages could port with no > change. > The new names in argparse fit with the fact that argparse has a different 'rationale' than optparse. What about an optparse compatibility layer to make porting easier? Although it can't cover the whole optparse API (as explained in the PEP) it could perhaps cover most 'normal' usage? Michael > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From rdmurray at bitdance.com Mon Sep 28 13:34:46 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 28 Sep 2009 07:34:46 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC03410.4010104@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: On Mon, 28 Sep 2009 at 05:57, "Martin v. L?wis" wrote: >>> Finally, to Stephen's point about seeing the other side of the >>> argument, I wrote this offlist a week ago: >>> >>> I *understand* what you're saying, I *understand* that >>> 192.168.1.1/24 isn't a network, >> >> But you still want to treat it as one. >> >> Could you explain what benefit there is for allowing the user to create >> network objects that don't represent networks? Is there a use-case >> where these networks-that-aren't-networks are something other than a >> typo? Under what circumstances would I want to specify a network as >> 192.168.1.1/24 instead of 192.168.1.0/24? >> > > It's fairly obvious to me why the library should support 192.168.1.1/24 > as an input, and return a network. > > End-users are likely going to enter such things (e.g. 82.94.164.162/29), > as they will know one IP address in the network (namely, of one machine > that they are looking at), and they will know the prefix length (more > often, how large the network is - 8 or 16 or 32). So very clearly, > end users should not be required to make the computation in their heads. > > So Python code has to make the computation, and it seems most natural > that the IP library is the piece of code that is able to compute a > network out of that input. > > Does that answer your question? The fundamental divide here is between two behaviors. ipaddr: >>> x = IPv4Network('192.168.1.1/24') >>> y = IPv4Network('192.168.1.0/24') >>> x == y False >>> x.ip IPv4Address('192.168.1.1') desired: >>> x = IPv4Network('192.168.1.1/24') >>> y = IPv4Network('192.168.1.0/24') >>> x == y True >>> x.ip Traceback (most recent call last): File "", line 1, in AttributeError: 'IPv4Network' object has no attribute 'ip' Everything else is pretty much bikeshedding and can be dealt with. This is fundamental and Peter has indicated he will not change it. --David From eric at trueblade.com Mon Sep 28 13:48:47 2009 From: eric at trueblade.com (Eric Smith) Date: Mon, 28 Sep 2009 07:48:47 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: <4AC0A29F.1090907@trueblade.com> R. David Murray wrote: > The fundamental divide here is between two behaviors. > > ipaddr: > > >>> x = IPv4Network('192.168.1.1/24') > >>> y = IPv4Network('192.168.1.0/24') > >>> x == y > False > >>> x.ip > IPv4Address('192.168.1.1') > > desired: > > >>> x = IPv4Network('192.168.1.1/24') > >>> y = IPv4Network('192.168.1.0/24') > >>> x == y > True > >>> x.ip > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'IPv4Network' object has no attribute 'ip' > > Everything else is pretty much bikeshedding and can be dealt with. This > is fundamental and Peter has indicated he will not change it. I think that's an excellent summary, David. And I'm -1 unless it changes to the "desired" behavior above. Eric. From rdmurray at bitdance.com Mon Sep 28 13:50:25 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 28 Sep 2009 07:50:25 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: On Mon, 28 Sep 2009 at 07:34, R. David Murray wrote: > The fundamental divide here is between two behaviors. > > ipaddr: > > > > > x = IPv4Network('192.168.1.1/24') > > > > y = IPv4Network('192.168.1.0/24') > > > > x == y > False > > > > x.ip > IPv4Address('192.168.1.1') > > desired: > > > > > x = IPv4Network('192.168.1.1/24') > > > > y = IPv4Network('192.168.1.0/24') > > > > x == y > True > > > > x.ip > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'IPv4Network' object has no attribute 'ip' > > Everything else is pretty much bikeshedding and can be dealt with. This > is fundamental and Peter has indicated he will not change it. By the way, I think this is a reasonable position on Peter's part. He's got an installed base that expects the current behavior (ie: the "field tested module" has this behavior; making this change would produce a fundamentally new module that has _not_ been field tested). But I also think that the current behavior is not a good behavior for a module in the stdlib. --David From ncoghlan at gmail.com Mon Sep 28 14:11:55 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 22:11:55 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909271731ra859ba5yda1d5f30e13a8b26@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <8517e9350909271053n174ee3dy9173ecf73e3dd39b@mail.gmail.com> <200909281013.39069.steve@pearwood.info> <8517e9350909271731ra859ba5yda1d5f30e13a8b26@mail.gmail.com> Message-ID: <4AC0A80B.1000407@gmail.com> Peter Moody wrote: >> That is, you've rejected the idea of having: >> >>>>> IPv4Network(192.168.1.1/24) >> IPv4Network(192.168.1.0/24) > > yes, I and everyone have rejected that idea. this should either raise > an error, or do what it does now, that is, return > IPv4Network('192.168.1.1/24') Um, no, several people (me included) have said that defining an IPNetwork based on its *network address* and its netmask, instead of the arbitrary hostname. >> as reducing functionality, presumably because the address 192.168.1.1 is >> lost. > > good guess. > >> Sounds just like an Address+netmask to me, with added >> network-like behaviour. >> >> Some people have requested a way of explicitly calculating a network >> from an Address and netmask, and you've been hostile to the idea. But >> it seems to me that your API does exactly that. > > You mean the suggestion to include an IPv?Network object attribute > with the IPv?Address objects? I thought that was dropped long ago. No, we're talking about any way of calculating the canonical network object given a host within that network and the netmask. With the current incarnation of ipaddr, the way you do that is to create an IPNetwork object with net.ip != net.network. But then you get a bizarre artifact, in that the same network will not compare equal if it is derived using a different IP address. There are *3* major concepts in ipaddr: - single address - network (defined by network address + net mask) - arbitrary host associated with a specific network Since those 3 concepts are being mapped to only two classes, 3 different ways have been proposed of handling the 3rd concept: - a "network" attribute on IPAddress objects (rejected, largely by consensus as far as I can tell) - a 2-tuple containing an IPAddress object and an IPNetwork object (rejected implicitly by your refusal to remove the .ip attribute from IPNetwork objects) - the current implementation, which uses an IPNetwork object with net.ip != net.network. That 3rd approach would be largely OK, *except* that the current implementation of it breaks the definition of network equality. Currently, IPNetwork("192.168.1.0/24") != IPNetwork("192.168.1.1/24"), despite the fact that they refer to the exact same network. I believe that is the fundamental point of contention here. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Sep 28 14:34:36 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 22:34:36 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <4ABFE075.2070309@voidspace.org.uk> Message-ID: <4AC0AD5C.1000008@gmail.com> Doug Hellmann wrote: > > On Sep 27, 2009, at 6:00 PM, Michael Foord wrote: > >> Brett Cannon wrote: >>> I am going to state upfront that I am +1 for this and I encouraged >>> Steven to submit this PEP on the stdlib-SIG. I still remember watching >>> Steven's lightning talk at PyCon 2009 on argparse and being impressed >>> by it (along with the rest of the audience which was obviously >>> impressed as well). >>> >>> I think argprase is a good improvement over what we have now (getopt >>> and optparse), it's stable, considered best-of-breed by the community >>> for a while (as shown as how many times argparse has been suggestion >>> for inclusion), and Steven is already a core committer so is already >>> set to maintain the code. That covers the usual checklist we have for >>> even looking at a PEP to add a module to the standard library. >>> >>> >> >> I've used argparse and really like it. I could also have done with the >> subcommand support in recent changes to unittest. >> >> +1 for inclusion. > > +1 as well. I wish I had been able to use this library for a recent > project using sub-commands. +1 here as well (although we should definitely find a way to use str.format strings instead of %-format ones... come to think of it, does even the logging module support str.format style formatting in Py3k?) Reading through the argparse vs optparse comparison in the argparse docs when the topic of possibly adding it came up a few weeks back I kept going "yep, I've rolled my own version of that, oh, and that, yeah, that too...". argparse probably wouldn't have helped me even if I'd known about it (due to license review overhead), but it certainly addressed a whole heap of problems I had encountered in practice, and in ways that were a lot cleaner than the comparatively hacky approaches I had used for my own purposes (internal testing and diagnostic scripts are like that...). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Sep 28 14:37:16 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 22:37:16 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> Message-ID: <4AC0ADFC.1030604@gmail.com> Steven Bethard wrote: > I feel like I'm repeating the PEP, but here it is again anyway. There > will be messages in the docs and pending deprecation warnings (which > don't show up by default but can be requested) starting in Python 2.7 > and 3.2. Regular deprecation warnings wouldn't show up until Python > 3.4, 3 years away. This compromise was intended exactly to address the > issue you brought up about people getting over the Python 3 > transition. For 2.x, I'd go even further and only have the PendingDeprecation warnings show up under the "-3" command line flag. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Sep 28 14:24:55 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 22:24:55 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: <4AC0AB17.5070801@gmail.com> R. David Murray wrote: > On Mon, 28 Sep 2009 at 07:34, R. David Murray wrote: >> The fundamental divide here is between two behaviors. >> >> ipaddr: >> >> > > > x = IPv4Network('192.168.1.1/24') >> > > > y = IPv4Network('192.168.1.0/24') >> > > > x == y >> False >> > > > x.ip >> IPv4Address('192.168.1.1') >> >> desired: >> >> > > > x = IPv4Network('192.168.1.1/24') >> > > > y = IPv4Network('192.168.1.0/24') >> > > > x == y >> True >> > > > x.ip >> Traceback (most recent call last): >> File "", line 1, in >> AttributeError: 'IPv4Network' object has no attribute 'ip' >> >> Everything else is pretty much bikeshedding and can be dealt with. This >> is fundamental and Peter has indicated he will not change it. > > By the way, I think this is a reasonable position on Peter's part. > He's got an installed base that expects the current behavior (ie: the > "field tested module" has this behavior; making this change would produce > a fundamentally new module that has _not_ been field tested). > > But I also think that the current behavior is not a good behavior for > a module in the stdlib. I should note that I've softened my position slightly from what I posted yesterday. I could live with the following compromise: >>> x = IPv4Network('192.168.1.1/24') >>> y = IPv4Network('192.168.1.0/24') >>> x == y # Equality is the part I really want to see changed True >>> x.ip IPv4Address('192.168.1.1') >>> y.ip IPv4Address('192.168.1.0') So IPNetwork equality classes would be rendered conceptually coherent (i.e. list(net1) == list(net2) would imply net1 == net2), but the "Host with associated network" functionality would also be retained. To check that the hosts were the same in addition to the network definition one could easily check the .ip attribute in addition to checking the networks themselves. Something I also noticed is that "compare_networks" in the ipaddr reference implementation doesn't follow the PEP - it defines IPv4 networks as being less than IPv6 networks instead of raising a TypeError. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Sep 28 14:40:44 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 22:40:44 +1000 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: References: <4ABF2047.7050501@gmail.com> <4ABFCDED.8080105@gmail.com> Message-ID: <4AC0AECC.4050104@gmail.com> Antoine Pitrou wrote: > Le Mon, 28 Sep 2009 06:41:17 +1000, Nick Coghlan a ?crit : >> Michael's response cut to the heart of the issue though - a richer IO >> exception hierarchy can make life interesting for compatibility purposes >> (especially when creating "file-like" interfaces to non-file objects). > > Well, not more interesting than currently where you need to replicate > errno numbers if you want to make the errors precise enough, since an API > consumer wanting to check specific error conditions will discriminate on > errno. > > If you don't want to go to that level of perfection, you just have to > raise a plain IOError (or OSError :-)) without bothering about errno or > subclasses; like you would do today. Fair point. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From exarkun at twistedmatrix.com Mon Sep 28 14:58:18 2009 From: exarkun at twistedmatrix.com (exarkun at twistedmatrix.com) Date: Mon, 28 Sep 2009 12:58:18 -0000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC03410.4010104@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: <20090928125818.1533.2070636475.divmod.xquotient.382@localhost.localdomain> On 03:57 am, martin at v.loewis.de wrote: >>>Finally, to Stephen's point about seeing the other side of the >>>argument, I wrote this offlist a week ago: >>> >>> I *understand* what you're saying, I *understand* that >>>192.168.1.1/24 isn't a network, >> >>But you still want to treat it as one. >> >>Could you explain what benefit there is for allowing the user to >>create >>network objects that don't represent networks? Is there a use-case >>where these networks-that-aren't-networks are something other than a >>typo? Under what circumstances would I want to specify a network as >>192.168.1.1/24 instead of 192.168.1.0/24? > >It's fairly obvious to me why the library should support 192.168.1.1/24 >as an input, and return a network. > >End-users are likely going to enter such things (e.g. >82.94.164.162/29), >as they will know one IP address in the network (namely, of one machine >that they are looking at), and they will know the prefix length (more >often, how large the network is - 8 or 16 or 32). So very clearly, >end users should not be required to make the computation in their >heads. > >So Python code has to make the computation, and it seems most natural >that the IP library is the piece of code that is able to compute a >network out of that input. And this is a rather classic example of a misfeature. "(Non-developer) End users will type it in" is not argument for supporting a particular string format as the primary constructor for an object. Constructors are for *developers*. They should be made most useful for *developers*. The issue of dealing with user input (which may have many other quirks) is separate and should be dealt with separately. Jean-Paul From jon+python-dev at unequivocal.co.uk Mon Sep 28 15:03:17 2009 From: jon+python-dev at unequivocal.co.uk (Jon Ribbens) Date: Mon, 28 Sep 2009 14:03:17 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <20090928083820.GA488@laurie.devork> References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com> <20090928083820.GA488@laurie.devork> Message-ID: <20090928130317.GR23367@snowy.squish.net> On Mon, Sep 28, 2009 at 09:38:20AM +0100, Floris Bruynooghe wrote: > On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote: > > -1 for deprecating getopt. getopt is super-simple and especially useful for > > c programmers learning python. > > > > +1 for argparse.+1 for eventual deprecation of optparse - optparse and > > argparse have a very similar syntax and having both is just > > confusing. tsboapooowtdi > > +1 on all of this :-) > > It would be a shame to see getopt go but optparse -> argparse seems > logical. +1 from me too - keep getopt, deprecate optparse. From digitalxero at gmail.com Mon Sep 28 15:12:57 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Mon, 28 Sep 2009 07:12:57 -0600 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: On Mon, Sep 28, 2009 at 5:34 AM, R. David Murray wrote: > The fundamental divide here is between two behaviors. > > ipaddr: > > ? ?>>> x = IPv4Network('192.168.1.1/24') > ? ?>>> y = IPv4Network('192.168.1.0/24') > ? ?>>> x == y > ? ?False > ? ?>>> x.ip > ? ?IPv4Address('192.168.1.1') > > desired: > > ? ?>>> x = IPv4Network('192.168.1.1/24') > ? ?>>> y = IPv4Network('192.168.1.0/24') > ? ?>>> x == y > ? ?True > ? ?>>> x.ip > ? ?Traceback (most recent call last): > ? ? ?File "", line 1, in > ? ?AttributeError: 'IPv4Network' object has no attribute 'ip' I submitted a patch to ipadrdr http://codereview.appspot.com/124057 which would do >>> x = IPv4Network('192.168.1.1/24') >>> y = IPv4Network('192.168.1.0/24') >>> x == y True >>> x.ip IPv4Address('192.168.1.1') From ncoghlan at gmail.com Mon Sep 28 13:49:34 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 21:49:34 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC00AE1.7080009@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABF015E.5060002@gmail.com> <8517e9350909270049t68dc0161q9a32502c10b2dd47@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <4ABFCC6B.3020409@gmail.com> <4AC00AE1.7080009@canterbury.ac.nz> Message-ID: <4AC0A2CE.7010302@gmail.com> Greg Ewing wrote: > Nick Coghlan wrote: > >> The use case for "given a netmask and an arbitrary host on that network, >> give me the appropriate IPNetwork object" has been well established by >> this discussion (although still isn't particularly well described even >> in the latest PEP update). This is what "strict=False" covers, > > I think I'd rather have a separate constructor function > or method for this, rather than a somewhat cryptically > named boolean parameter. Yes, I would prefer a separate method as well. It doesn't bother as much as the broken definition of equality though. In fact, if the definition of IPNetwork equality was fixed then leaving strict mode out entirely would probably be OK. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From daniel at stutzbachenterprises.com Mon Sep 28 16:04:58 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 28 Sep 2009 09:04:58 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC0AB17.5070801@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC0AB17.5070801@gmail.com> Message-ID: On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan wrote: > I should note that I've softened my position slightly from what I posted > yesterday. I could live with the following compromise: > > >>> x = IPv4Network('192.168.1.1/24') > >>> y = IPv4Network('192.168.1.0/24') > >>> x == y # Equality is the part I really want to see changed > True > >>> x.ip > IPv4Address('192.168.1.1') > >>> y.ip > IPv4Address('192.168.1.0') > With those semantics, IPv4Network objects with distinct IP addresses (but the same network) could no longer be stored in a dictionary or set. IMO, it is a little counter-intuitive for objects to compare equal yet have different properties. I don't think this is a good compromise. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From dickinsm at gmail.com Mon Sep 28 16:06:58 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 28 Sep 2009 15:06:58 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC0AB17.5070801@gmail.com> Message-ID: <5c6f2a5d0909280706n2a83689bl9e04fe705b91c202@mail.gmail.com> On Mon, Sep 28, 2009 at 3:04 PM, Daniel Stutzbach wrote: > On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan wrote: >> >> I should note that I've softened my position slightly from what I posted >> yesterday. I could live with the following compromise: >> >> ? ?>>> x = IPv4Network('192.168.1.1/24') >> ? ?>>> y = IPv4Network('192.168.1.0/24') >> ? ?>>> x == y # Equality is the part I really want to see changed >> ? ?True >> ? ?>>> x.ip >> ? ?IPv4Address('192.168.1.1') >> ? ?>>> y.ip >> ? ?IPv4Address('192.168.1.0') > > With those semantics, IPv4Network objects with distinct IP addresses (but > the same network) could no longer be stored in a dictionary or set.? IMO, it > is a little counter-intuitive for objects to compare equal yet have > different properties.? I don't think this is a good compromise. This worries me too. It seems like a potentially dangerous half-measure. Mark From eric at trueblade.com Mon Sep 28 16:16:30 2009 From: eric at trueblade.com (Eric Smith) Date: Mon, 28 Sep 2009 10:16:30 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <5c6f2a5d0909280706n2a83689bl9e04fe705b91c202@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC0AB17.5070801@gmail.com> <5c6f2a5d0909280706n2a83689bl9e04fe705b91c202@mail.gmail.com> Message-ID: <4AC0C53E.3020805@trueblade.com> Mark Dickinson wrote: > On Mon, Sep 28, 2009 at 3:04 PM, Daniel Stutzbach > wrote: >> On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan wrote: >>> I should note that I've softened my position slightly from what I posted >>> yesterday. I could live with the following compromise: >>> >>> >>> x = IPv4Network('192.168.1.1/24') >>> >>> y = IPv4Network('192.168.1.0/24') >>> >>> x == y # Equality is the part I really want to see changed >>> True >>> >>> x.ip >>> IPv4Address('192.168.1.1') >>> >>> y.ip >>> IPv4Address('192.168.1.0') >> With those semantics, IPv4Network objects with distinct IP addresses (but >> the same network) could no longer be stored in a dictionary or set. IMO, it >> is a little counter-intuitive for objects to compare equal yet have >> different properties. I don't think this is a good compromise. > > This worries me too. It seems like a potentially dangerous half-measure. I agree. I think keeping the concepts distinct and adding 2 new classes is a better solution. Eric. From steven.bethard at gmail.com Mon Sep 28 16:28:39 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 28 Sep 2009 07:28:39 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <20090928130317.GR23367@snowy.squish.net> References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com> <20090928083820.GA488@laurie.devork> <20090928130317.GR23367@snowy.squish.net> Message-ID: On Mon, Sep 28, 2009 at 6:03 AM, Jon Ribbens wrote: > On Mon, Sep 28, 2009 at 09:38:20AM +0100, Floris Bruynooghe wrote: >> On Mon, Sep 28, 2009 at 06:59:45AM +0300, Yuvgoog Greenle wrote: >> > -1 for deprecating getopt. getopt is super-simple and especially useful for >> > c programmers learning python. >> > >> > +1 for argparse.+1 for eventual deprecation of optparse - optparse and >> > argparse have a very similar syntax and having both is just >> > confusing. tsboapooowtdi >> >> +1 on all of this :-) >> >> It would be a shame to see getopt go but optparse -> argparse seems >> logical. > > +1 from me too - keep getopt, deprecate optparse. Ok, sounds like there are a number of supporters for keeping getopt around and just deprecating optparse. For those who'd like to keep getopt around, I have a few questions: * Would you be opposed to a note in the getopt documentation suggesting argparse as an alternative? * Would you like argparse to grow an add_getopt_arguments method (as in my other post)? * If argparse grew an add_getopt_arguments, would you still want to keep getopt around? And if so, why? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From rdmurray at bitdance.com Mon Sep 28 16:35:57 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 28 Sep 2009 10:35:57 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <9118C602-C5D3-49A9-B07F-CE61723D4D6A@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABE530D.8090609@v.loewis.de> <200909271108.32456.steve@pearwood.info> <4ABF0CA8.2030408@v.loewis.de> <9118C602-C5D3-49A9-B07F-CE61723D4D6A@gmail.com> Message-ID: On Sun, 27 Sep 2009 at 10:06, David Moss wrote: > On 27 Sep 2009, at 07:56, "Martin v. L??wis" wrote: >> I wouldn't ask for that: it should certainly be possible to supply >> masks. However, I would want to reject masks that don't correspond to >> a prefix, and have only the prefix length in the internal representation. >> > +1 on rejection of netmasks without direct CIDR prefix equivalents. AFAIK > Cisco routers accept them but I don't see how they would be useful in > practice (unless someone can demonstrate their need for this). Several years ago I ran into a network with a non-CIDR netmask. This was in the early days of CIDR, and I hope that that network is no longer configured that way.... A little googling produced this hit: http://tools.ietf.org/wg/mip4/draft-ietf-mip4-nemo-v4-base/draft-ietf-mip4-nemo-v4-base-11-from-10.diff.txt which while not a standard itself clearly believes that non-contiguous netmasks are valid, and is dated 2008. Earlier RFCs (950 and 1219) give them as valid but discouraged. The linux stack (or at least the ifconfig command on my system) does not appear to support non-contiguous netmasks. From a brief search on Cisco's site it looks like Cisco only partially supports non-contiguous masks (you can use them in wildmasks in access lists, but not in subnet masks on interfaces). (I just tried setting a non-contiguous netmask on an interface on one of my Ciscos, and it gave me a 'bad mask' error message.) I think support for non-contiguous netmasks could be safely omitted, but it would mean the module did not fully support the standards, which should be noted in the docs (and why). --David From steve at pearwood.info Mon Sep 28 16:36:56 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 29 Sep 2009 00:36:56 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <200909290036.56742.steve@pearwood.info> On Mon, 28 Sep 2009 06:59:32 am Steven Bethard wrote: > Why aren't getopt and optparse enough? [snip] As a newbie, I found optparse intimidating because it provided so many features. I expect argparse will be the same, only more so. (Disclaimer: I haven't used argparse, I'm merely making a prediction based on the available functionality.) As it turned out, optparse wasn't anywhere near as difficult to use as I expected it to be, but I suggest that newbies will have the same response to argparse as I had to optparse -- "this looks complicated, I'll just roll my own". At least I had the option to use getopt. Not every module needs to be newbie-friendly, but something as fundamental as argument parsing should at least provide a gentle learning curve. Currently, getopt more-or-less plays the role of "bare-bones" command line parsing, which I think is important. Assuming argparse is added to the std lib, I'm -0.5 on deprecating getopt, 0 on deprecating optparse. -- Steven D'Aprano From digitalxero at gmail.com Mon Sep 28 16:42:15 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Mon, 28 Sep 2009 08:42:15 -0600 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC0AB17.5070801@gmail.com> Message-ID: On Mon, Sep 28, 2009 at 8:04 AM, Daniel Stutzbach wrote: > On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan wrote: >> >> I should note that I've softened my position slightly from what I posted >> yesterday. I could live with the following compromise: >> >> ? ?>>> x = IPv4Network('192.168.1.1/24') >> ? ?>>> y = IPv4Network('192.168.1.0/24') >> ? ?>>> x == y # Equality is the part I really want to see changed >> ? ?True >> ? ?>>> x.ip >> ? ?IPv4Address('192.168.1.1') >> ? ?>>> y.ip >> ? ?IPv4Address('192.168.1.0') > > With those semantics, IPv4Network objects with distinct IP addresses (but > the same network) could no longer be stored in a dictionary or set.? IMO, it > is a little counter-intuitive for objects to compare equal yet have > different properties.? I don't think this is a good compromise. Thats not true, the patch I submitted http://codereview.appspot.com/124057 still allows the networks to be included in a set or as a dict key >>> net1 = IPNetwork("10.1.2.3/24") >>> net2 = IPNetwork("10.1.2.0/24") >>> print hash(net1) == hash(net2) False >>> print net1 == net2 True >>> test = {net1: "something", net2: "something else"} >>> print test {IPv4Network('10.1.2.0/24'): 'something else', IPv4Network('10.1.2.3/24'): 'something'} >>> test2 = set([net1, net2]) >>> print test2 set([IPv4Network('10.1.2.0/24'), IPv4Network('10.1.2.3/24')]) From steven.bethard at gmail.com Mon Sep 28 16:43:04 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 28 Sep 2009 07:43:04 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: On Mon, Sep 28, 2009 at 4:09 AM, Neal Becker wrote: > If the plan is to migrate from optparse to argparse, this could be made a > bit easier. ?If it weren't for the fact that some names are different in > argparse than optparse, I believe many optparse usages could port with no > change. I could conceivably add an OptionParser class which:: * Has an add_option method which delegates to add_argument * Has a parse_args method which delegates to parse_known_args * Registers the string names for types that optparse uses That might allow some users to just switch their import as long as they're not using any callbacks, catching any exceptions, extending any APIs, etc. I'm not 100% convinced that it's good to add a subtly different API that people would have to unlearn, but if there's enough support for this, I'm open to the idea. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From skip at pobox.com Mon Sep 28 16:47:38 2009 From: skip at pobox.com (skip at pobox.com) Date: Mon, 28 Sep 2009 09:47:38 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC0AD5C.1000008@gmail.com> References: <4ABFE075.2070309@voidspace.org.uk> <4AC0AD5C.1000008@gmail.com> Message-ID: <19136.52362.562678.628471@montanaro.dyndns.org> Nick> +1 here as well (although we should definitely find a way to use Nick> str.format strings instead of %-format ones... come to think of Nick> it, does even the logging module support str.format style Nick> formatting in Py3k?) Assuming argparse currently works with versions of Python < 2.6 I see no reason to make such a change. This would just introduce needless differences between the version delivered with Python and the PyPI version and make it more difficult for the author to keep the two code bases in sync. Skip From guido at python.org Mon Sep 28 16:56:29 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Sep 2009 07:56:29 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC0AB17.5070801@gmail.com> Message-ID: On Mon, Sep 28, 2009 at 7:42 AM, Dj Gilcrease wrote: > On Mon, Sep 28, 2009 at 8:04 AM, Daniel Stutzbach > wrote: >> On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan wrote: >>> >>> I should note that I've softened my position slightly from what I posted >>> yesterday. I could live with the following compromise: >>> >>> ? ?>>> x = IPv4Network('192.168.1.1/24') >>> ? ?>>> y = IPv4Network('192.168.1.0/24') >>> ? ?>>> x == y # Equality is the part I really want to see changed >>> ? ?True >>> ? ?>>> x.ip >>> ? ?IPv4Address('192.168.1.1') >>> ? ?>>> y.ip >>> ? ?IPv4Address('192.168.1.0') >> >> With those semantics, IPv4Network objects with distinct IP addresses (but >> the same network) could no longer be stored in a dictionary or set.? IMO, it >> is a little counter-intuitive for objects to compare equal yet have >> different properties.? I don't think this is a good compromise. > > Thats not true, the patch I submitted > http://codereview.appspot.com/124057 still allows the networks to be > included in a set or as a dict key > >>>> net1 = IPNetwork("10.1.2.3/24") >>>> net2 = IPNetwork("10.1.2.0/24") >>>> print hash(net1) == hash(net2) > False >>>> print net1 == net2 > True Hold it right there! That's wrong. You can't have two objects that compare equal but whose hashes differ. It will break dict lookup. The other way around is fine: two objects may differ and still have the same hash. >>>> test = {net1: "something", net2: "something else"} >>>> print test > {IPv4Network('10.1.2.0/24'): 'something else', > IPv4Network('10.1.2.3/24'): 'something'} >>>> test2 = set([net1, net2]) >>>> print test2 > set([IPv4Network('10.1.2.0/24'), IPv4Network('10.1.2.3/24')]) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Mon Sep 28 16:57:28 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 28 Sep 2009 15:57:28 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC0AB17.5070801@gmail.com> Message-ID: <5c6f2a5d0909280757n4e79e1a9laf49f4991e43345e@mail.gmail.com> On Mon, Sep 28, 2009 at 3:42 PM, Dj Gilcrease wrote: > On Mon, Sep 28, 2009 at 8:04 AM, Daniel Stutzbach > wrote: >> On Mon, Sep 28, 2009 at 7:24 AM, Nick Coghlan wrote: >>> >>> I should note that I've softened my position slightly from what I posted >>> yesterday. I could live with the following compromise: >>> >>> ? ?>>> x = IPv4Network('192.168.1.1/24') >>> ? ?>>> y = IPv4Network('192.168.1.0/24') >>> ? ?>>> x == y # Equality is the part I really want to see changed >>> ? ?True >>> ? ?>>> x.ip >>> ? ?IPv4Address('192.168.1.1') >>> ? ?>>> y.ip >>> ? ?IPv4Address('192.168.1.0') >> >> With those semantics, IPv4Network objects with distinct IP addresses (but >> the same network) could no longer be stored in a dictionary or set.? IMO, it >> is a little counter-intuitive for objects to compare equal yet have >> different properties.? I don't think this is a good compromise. > > Thats not true, the patch I submitted > http://codereview.appspot.com/124057 still allows the networks to be > included in a set or as a dict key > >>>> net1 = IPNetwork("10.1.2.3/24") >>>> net2 = IPNetwork("10.1.2.0/24") >>>> print hash(net1) == hash(net2) > False >>>> print net1 == net2 > True In that case, your patch breaks the rather fundamental rule that Python objects that compare equal should have equal hash. :-) Relying on hashes to be distinct isn't safe. Mark From solipsis at pitrou.net Mon Sep 28 16:57:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 14:57:14 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC0AB17.5070801@gmail.com> Message-ID: Dj Gilcrease gmail.com> writes: > > >>> net1 = IPNetwork("10.1.2.3/24") > >>> net2 = IPNetwork("10.1.2.0/24") > >>> print hash(net1) == hash(net2) > False > >>> print net1 == net2 > True Ouch :-) It is very bad practice for equal objects to hash differently. Differing hashes are supposed to imply non-equal objects, so as to allow the implementation of associative containers with good performance characteristics. Regards Antoine. From daniel at stutzbachenterprises.com Mon Sep 28 17:01:38 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 28 Sep 2009 10:01:38 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4ABE530D.8090609@v.loewis.de> <200909271108.32456.steve@pearwood.info> <4ABF0CA8.2030408@v.loewis.de> <9118C602-C5D3-49A9-B07F-CE61723D4D6A@gmail.com> Message-ID: On Mon, Sep 28, 2009 at 9:35 AM, R. David Murray wrote: > A little googling produced this hit: > > > http://tools.ietf.org/wg/mip4/draft-ietf-mip4-nemo-v4-base/draft-ietf-mip4-nemo-v4-base-11-from-10.diff.txt > > which while not a standard itself clearly believes that non-contiguous > netmasks are valid, and is dated 2008. Earlier RFCs (950 and 1219) > give them as valid but discouraged. > That's a draft for RFC 5177. In the RFC, all references to non-contiguous subnets have been removed. http://tools.ietf.org/html/rfc5177 -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From mal at egenix.com Mon Sep 28 17:03:55 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 28 Sep 2009 17:03:55 +0200 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <4AC0D05B.2060506@egenix.com> Antoine Pitrou wrote: > > Hello, > > I am neutral on the idea of adding argparse. However, I'm -1 on deprecating > optparse. It is very widely used (tons of scripts use it), and ok for many uses; > deprecating it is totally unhelpful and gratuitous. You can add me to that camp as well: +0 on adding argparse -1 on deprecating wide-used modules such as getopt and optparse Note that the last -1 only applies to the module APIs, not the code in those modules. If argparse provides an API compatibility layer for getopt and optparse, the original code of getopt and optparse could go without problem, IMHO, leaving only the mostly empty modules together with compatibility layers. OTOH, as discussed on the stdlib SIG list, those two modules hardly ever require any maintenance, so keeping them as they are is certainly an option as well. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 28 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From fuzzyman at voidspace.org.uk Mon Sep 28 17:08:48 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 28 Sep 2009 16:08:48 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC0D05B.2060506@egenix.com> References: <4AC0D05B.2060506@egenix.com> Message-ID: <4AC0D180.9030607@voidspace.org.uk> M.-A. Lemburg wrote: > Antoine Pitrou wrote: > >> Hello, >> >> I am neutral on the idea of adding argparse. However, I'm -1 on deprecating >> optparse. It is very widely used (tons of scripts use it), and ok for many uses; >> deprecating it is totally unhelpful and gratuitous. >> > > You can add me to that camp as well: > > +0 on adding argparse > -1 on deprecating wide-used modules such as getopt and optparse > > Note that the last -1 only applies to the module APIs, not the > code in those modules. > > If argparse provides an API compatibility layer for getopt and > optparse, the original code of getopt and optparse could go > without problem, IMHO, leaving only the mostly empty modules > together with compatibility layers. > > OTOH, as discussed on the stdlib SIG list, those two modules > hardly ever require any maintenance, so keeping them as they > are is certainly an option as well. > > Although as also discussed on the stdlib-sig there is also a cost to leaving unmaintained and unneeded modules in the standard library. As we are 'end-of-lifing' Python 2.X there seems to be little point in going down the deprecation route (there won't be enough releases to ever get to module removal) but discussion about the standard library in situations like this is badly needed for 3.X. Certainly a slow deprecation procedure, as suggested by Steven, is warranted. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From steve at pearwood.info Mon Sep 28 17:13:41 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 29 Sep 2009 01:13:41 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC03410.4010104@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: <200909290113.41871.steve@pearwood.info> On Mon, 28 Sep 2009 01:57:04 pm Martin v. L?wis wrote: > >> Finally, to Stephen's point about seeing the other side of the > >> argument, I wrote this offlist a week ago: > >> > >> I *understand* what you're saying, I *understand* that > >> 192.168.1.1/24 isn't a network, > > > > But you still want to treat it as one. > > > > Could you explain what benefit there is for allowing the user to > > create network objects that don't represent networks? Is there a > > use-case where these networks-that-aren't-networks are something > > other than a typo? Under what circumstances would I want to specify > > a network as 192.168.1.1/24 instead of 192.168.1.0/24? > > It's fairly obvious to me why the library should support > 192.168.1.1/24 as an input, and return a network. > > End-users are likely going to enter such things (e.g. > 82.94.164.162/29), as they will know one IP address in the network > (namely, of one machine that they are looking at), and they will know > the prefix length (more often, how large the network is - 8 or 16 or > 32). So very clearly, end users should not be required to make the > computation in their heads. Thank you Martin. I had somehow -- justifiably or not, it doesn't matter -- got the impression from Peter off-list that this was *not* a use-case he gave any credence to, but of course the current API supports it, which confused me no end. I'm not sure that just because the user enters '82.94.164.162/29' into the UI, the API must accept it. As Jean-Paul suggests, the issue of dealing with user-input is different from API aimed at developers. There are at least three options: (1) The status quo: IPv4Network accepts the host address plus mask, and returns an object which behaves like a network but remembers the host address. (2) Like (1), but the address is normalised to the canonical network address (which if I've calculated it correctly should be '82.94.164.160'). In this case, whether the user enters '82.94.164.162/29' or '82.94.164.169/29', the network returned will be the same. If the developer wants to save the host address as well as the network for some reason, he should store the host address as a separate object. (3) Like (2), except the constructor is strict and only accepts the canonical network address, and will raise an exception if given '82.94.164.162/29'. This may require at least one helper function to calculate the network/mask from the address/mask. The PEP now states: "It should be understood that, above all, this module is designed with the network administrator in mind. In practice, this means that a number of assumptions are made with regards to common usage and the library prefers the usefulness of accepted practice over strict adherence to RFCs. For example, ipaddr accepts '192.168.1.1/24' as a network definition because this is a very common way of describing an address + netmask despite the fact that 192.168.1.1 is actually an IP address on the network 192.168.1.0/24. ..." which nicely explains why (3) is not used, but doesn't explain why (1) should be preferred over (2). -- Steven D'Aprano From foom at fuhm.net Mon Sep 28 17:13:47 2009 From: foom at fuhm.net (James Y Knight) Date: Mon, 28 Sep 2009 11:13:47 -0400 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: <4AC07309.1010608@egenix.com> References: <4AB67430.2060306@egenix.com> <4AC07309.1010608@egenix.com> Message-ID: <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> On Sep 28, 2009, at 4:25 AM, M.-A. Lemburg wrote: > Distributions should really not be put in charge of upstream > coding design decisions. I don't think you can blame distros for this one.... From PEP 0261: It is also proposed that one day --enable-unicode will just default to the width of your platforms wchar_t. On linux, wchar_t is 4 bytes. If there's a consensus amongst python upstream that all the distros should be shipping Python with UCS2 unicode strings, you should reach out to them and say this, in a rather more clear fashion. Currently, most signs point towards UCS4 builds as being the better option. Or, one might reasonably wonder why UCS-4 is an option at all, if nobody should enable it. > People building their own Python version will usually also build > their own extensions, so I don't really believe that the above > scenario is very common. I'd just like to note that I've run into this trap multiple times. I built a custom python, and expected it to work with all the existing, installed, extensions (same major version as the system install, just patched). And then had to build it again with UCS4, for it to actually work. Of course building twice isn't the end of the world, and I'm certainly used to having to twiddle build options on software to get it working, but, this *does* happen, and *is* a tiny bit irritating. James From ubershmekel at gmail.com Mon Sep 28 17:18:45 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Mon, 28 Sep 2009 18:18:45 +0300 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com> <20090928083820.GA488@laurie.devork> <20090928130317.GR23367@snowy.squish.net> Message-ID: <9d153b7c0909280818v30760a44p11ab24ba9251d4ef@mail.gmail.com> > > * Would you be opposed to a note in the getopt documentation suggesting > argparse as an alternative? from the top of http://docs.python.org/library/getopt.html - "A more convenient, flexible, and powerful alternative is the optparse module."I think this statement should be emphasized better but it's already there so no worries. * Would you like argparse to grow an add_getopt_arguments method (as in my > other post)? * If argparse grew an add_getopt_arguments, would you still want to keep > getopt around? And if so, why? Argparse/optparse use a parser instance and getopt doesn't. I think that's a saved line you have to keep saving so this would be the better syntax imho: optlist, args = argparse.getopt(args, 'abc:d:') I was thinking on the general issue of what's better for the std-lib: 1. Mini modules - 3+ small modules in each subject that can be useful in different contexts. For example the xml libs. or 2. Big modules - 1 uber-module for each subject that does everything. Maybe logging is an example of this. I think in general python std-lib went the path of mini modules. Its advantages 1. Each mini module can be coherent and while python isn't really OnlyOneWayToDoIt, each module in itself has OnlyOneWayToDoIt. 2. Documentation is less cluttered by a large amount of functions that all do the same thing but differently. This is very important for python's learning curve. disadvantages: 1. You have to craft your documentation carefully as to be very explicit on why you have all these modules for the same thing. Probably each module should explain it's purpose in relation to the other modules and this explanation should appear at the top of the documentation of each of the mini modules in the subject. Today it would be very hard for me to figure out how the xml modules interrelate, urllib has improved in python3 but still isn't perfect. 2. I could be using some silly way of doing things (getopt) while I really wanted to do it the good way (argparse). I think that while adding getopt functionality to argparse might be nice but would eventually cause clutter. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sesquile at gmail.com Mon Sep 28 17:23:22 2009 From: sesquile at gmail.com (m h) Date: Mon, 28 Sep 2009 09:23:22 -0600 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: Perhaps this is OT, but since command line parsing is part of configuration, I figure I'd throw it out there. My scripts often have configuration that the command line can override and I loosely follow the example hierarchy[0] listed in The Art of Unix Programming. Some configuration I want in a config file (but I want to override from the command line) and sometimes it's very nice to use environment variables for configuration. So I do something like this: Chaining Configuration ---------------------- Ugly code to cascade configuration >>> class Unset(object): pass >>> def cascade_value(opt=None, opt_name=None, env_name=None, cfg=None, cfg_section=None, cfg_name=None, default=None): ... """ ... opt - result of OptionParser.parse_args ... opt_name - string of opt name you want to access ... """ ... # get from cmd line ... value = Unset() ... if opt and opt_name: ... try: ... value = opt.__getattr__(opt_name) ... except AttributeError, e: ... pass ... if not isinstance(value, Unset): ... return value ... # get from ENV ... if env_name: ... try: ... value = os.environ[env_name] ... except KeyError, e: ... pass ... if not isinstance(value, Unset): ... return value ... # get from config file ... if cfg and cfg_section and cfg_name: ... try: ... value = cfg.get(cfg_section, cfg_name) ... except ConfigParser.NoOptionError, e: ... pass ... if not isinstance(value, Unset): ... return value ... return default >>> cascade_value(opt=opt, opt_name='author', cfg=cfg, cfg_section='Properties', cfg_name='author') 'Matt' Does anyone else have interest in such functionality? Is it outside the realm of this PEP? Ideally I'd like to have something like ``env_name``, ``config_key`` named parameters for options, then pass an optional list of configs files to the parser instance. Then rather than looking solely at command line options, argparse could also pull values from config files, then env and finally the command line. cheers, matt harrison 0 - http://www.faqs.org/docs/artu/ch10s02.html From p.f.moore at gmail.com Mon Sep 28 17:25:54 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 28 Sep 2009 16:25:54 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: <79990c6b0909280825q57fce79ej6b6c390699c3f32d@mail.gmail.com> 2009/9/28 Peter Moody : >> That is, you've rejected the idea of having: >> >>>>> IPv4Network(192.168.1.1/24) >> IPv4Network(192.168.1.0/24) > > yes, I and everyone have rejected that idea. this should either raise > an error, or do what it does now, that is, return > IPv4Network('192.168.1.1/24') Speaking as a member of the group "everyone", I have not rejected that idea. It seems perfectly sensible to me. Yes, it "loses" the information about the full IP address 192.168.1.1, but in my view, that's what I asked it to do by requesting a network object back. 2009/9/28 R. David Murray : > The fundamental divide here is between two behaviors. > > ipaddr: > > ? ?>>> x = IPv4Network('192.168.1.1/24') > ? ?>>> y = IPv4Network('192.168.1.0/24') > ? ?>>> x == y > ? ?False > ? ?>>> x.ip > ? ?IPv4Address('192.168.1.1') > > desired: > > ? ?>>> x = IPv4Network('192.168.1.1/24') > ? ?>>> y = IPv4Network('192.168.1.0/24') > ? ?>>> x == y > ? ?True > ? ?>>> x.ip > ? ?Traceback (most recent call last): > ? ? ?File "", line 1, in > ? ?AttributeError: 'IPv4Network' object has no attribute 'ip' > > Everything else is pretty much bikeshedding and can be dealt with. ?This > is fundamental and Peter has indicated he will not change it. Agreed. I would prefer the option marked "desired". As a naive "casual user" of the module, I claim that the current behaviour is non-intuitive and I anticipate it causing me issues (not many, and I won't use the module often, so I concede - before someone asks - that it'll be a minor inconvenience to me). Given that I am a minor user, as noted above, I remain -1 on inclusion, largely because it feels to me like the module will suffer the same fate as optparse - rejecting certain changes which have relatively widespread support on essentially philosophical grounds. Paul. From fuzzyman at voidspace.org.uk Mon Sep 28 17:26:52 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Mon, 28 Sep 2009 16:26:52 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <4AC0D5BC.40108@voidspace.org.uk> m h wrote: > Perhaps this is OT, but since command line parsing is part of > configuration, I figure I'd throw it out there. My scripts often have > configuration that the command line can override and I loosely follow > the example hierarchy[0] listed in The Art of Unix Programming. > > Some configuration I want in a config file (but I want to override > from the command line) and sometimes it's very nice to use environment > variables for configuration. So I do something like this: > > Integration with command line options is an occasionally-requested feature for ConfigObj. I've always said I would be open to patches... In other words, yes I think there is demand for it. Whether it belongs *immediately* in the standard library is another matter, but if you wrote a layer that unified ConfigParser and argparse I think you will find that people use it. It is well outside the realm of this PEP however. All the best, Michael Foord > Chaining Configuration > ---------------------- > > Ugly code to cascade configuration > > >>>> class Unset(object): pass >>>> def cascade_value(opt=None, opt_name=None, env_name=None, cfg=None, cfg_section=None, cfg_name=None, default=None): >>>> > ... """ > ... opt - result of OptionParser.parse_args > ... opt_name - string of opt name you want to access > ... """ > ... # get from cmd line > ... value = Unset() > ... if opt and opt_name: > ... try: > ... value = opt.__getattr__(opt_name) > ... except AttributeError, e: > ... pass > ... if not isinstance(value, Unset): > ... return value > ... # get from ENV > ... if env_name: > ... try: > ... value = os.environ[env_name] > ... except KeyError, e: > ... pass > ... if not isinstance(value, Unset): > ... return value > ... # get from config file > ... if cfg and cfg_section and cfg_name: > ... try: > ... value = cfg.get(cfg_section, cfg_name) > ... except ConfigParser.NoOptionError, e: > ... pass > ... if not isinstance(value, Unset): > ... return value > ... return default > > > >>>> cascade_value(opt=opt, opt_name='author', cfg=cfg, cfg_section='Properties', cfg_name='author') >>>> > 'Matt' > > Does anyone else have interest in such functionality? Is it outside > the realm of this PEP? Ideally I'd like to have something like > ``env_name``, ``config_key`` named parameters for options, then pass > an optional list of configs files to the parser instance. Then rather > than looking solely at command line options, argparse could also pull > values from config files, then env and finally the command line. > > cheers, > > matt harrison > > > 0 - http://www.faqs.org/docs/artu/ch10s02.html > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From steve at pearwood.info Mon Sep 28 17:27:02 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 29 Sep 2009 01:27:02 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> Message-ID: <200909290127.03215.steve@pearwood.info> On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote: > Ok, sounds like there are a number of supporters for keeping getopt > around and just deprecating optparse. For those who'd like to keep > getopt around, I have a few questions: > > * Would you be opposed to a note in the getopt documentation > suggesting argparse as an alternative? +1 > * Would you like argparse to grow an add_getopt_arguments method (as > in my other post)? 0 > * If argparse grew an add_getopt_arguments, would you still want to > keep getopt around? And if so, why? Simplicity of the learning curve. Using it is as simple as: getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) Does argparse allow anything as simple as that? -- Steven D'Aprano From phd at phd.pp.ru Mon Sep 28 17:33:35 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Mon, 28 Sep 2009 19:33:35 +0400 Subject: [Python-Dev] Application configuration (was: PEP 389: argparse) In-Reply-To: References: Message-ID: <20090928153335.GA4747@phd.pp.ru> On Mon, Sep 28, 2009 at 09:23:22AM -0600, m h wrote: > Perhaps this is OT, but since command line parsing is part of > configuration, I figure I'd throw it out there. My scripts often have > configuration that the command line can override and I loosely follow > the example hierarchy[0] listed in The Art of Unix Programming. > > Some configuration I want in a config file (but I want to override > from the command line) and sometimes it's very nice to use environment > variables for configuration. The idea was discussed in the list many years ago but it seems to be forgotten now. > Does anyone else have interest in such functionality? Is it outside > the realm of this PEP? It is outside the scope of this particular PEP, but it is certainly an interesting idea. Actually it is the Right Thing, but I doubt it is possible to implement such a generic configuration *framework* in the Python standard *library*. Applications use different configfile layouts, some programs use w32 registry instead of configfiles... Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From jnoller at gmail.com Mon Sep 28 17:35:26 2009 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 28 Sep 2009 10:35:26 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <4222a8490909280835va690e38u2f5d97a35bf9fce0@mail.gmail.com> On Sun, Sep 27, 2009 at 4:57 PM, Brett Cannon wrote: > I am going to state upfront that I am +1 for this and I encouraged > Steven to submit this PEP on the stdlib-SIG. I still remember watching > Steven's lightning talk at PyCon 2009 on argparse and being impressed > by it (along with the rest of the audience which was obviously > impressed as well). > > I think argprase is a good improvement over what we have now (getopt > and optparse), it's stable, considered best-of-breed by the community > for a while (as shown as how many times argparse has been suggestion > for inclusion), and Steven is already a core committer so is already > set to maintain the code. That covers the usual checklist we have for > even looking at a PEP to add a module to the standard library. +1 to what brett said. jesse From steven.bethard at gmail.com Mon Sep 28 17:37:16 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 28 Sep 2009 08:37:16 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC0D5BC.40108@voidspace.org.uk> References: <4AC0D5BC.40108@voidspace.org.uk> Message-ID: On Mon, Sep 28, 2009 at 8:26 AM, Michael Foord wrote: > m h wrote: >> >> Perhaps this is OT, but since command line parsing is part of >> configuration, I figure I'd throw it out there. ?My scripts often have >> configuration that the command line can override and I loosely follow >> the example hierarchy[0] listed in The Art of Unix Programming. >> >> Some configuration I want in a config file (but I want to override >> from the command line) and sometimes it's very nice to use environment >> variables for configuration. ?So I do something like this: > > Integration with command line options is an occasionally-requested feature > for ConfigObj. I've always said I would be open to patches... > > In other words, yes I think there is demand for it. Whether it belongs > *immediately* in the standard library is another matter, but if you wrote a > layer that unified ConfigParser and argparse I think you will find that > people use it. > > It is well outside the realm of this PEP however. What Michael said. ;-) If you'd like to provide a patch for argparse that provides such functionality, I'd be happy to review it. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From digitalxero at gmail.com Mon Sep 28 17:30:05 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Mon, 28 Sep 2009 09:30:05 -0600 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC0AB17.5070801@gmail.com> Message-ID: On Mon, Sep 28, 2009 at 8:56 AM, Guido van Rossum wrote: > Hold it right there! That's wrong. You can't have two objects that > compare equal but whose hashes differ. It will break dict lookup. The > other way around is fine: two objects may differ and still have the > same hash. mm I never knew that, guess I'll go fix my patch to change the __hash__ method as well, though when I tested it dict lookup seemed to work fine >>> net1 = IPNetwork("10.1.2.0/29") >>> i = 0 >>> test = {} >>> for ip in net1: test[IPNetwork("%s/29" %(ip))] = i i += 1 >>> print test[IPv4Network('10.1.2.4/29')] 4 >>> print test {IPv4Network('10.1.2.0/29'): 0, IPv4Network('10.1.2.1/29'): 1, IPv4Network('10.1.2.2/29'): 2, IPv4Network('10.1.2.3/29'): 3, IPv4Network('10.1.2.4/29'): 4, IPv4Network('10.1.2.5/29'): 5, IPv4Network('10.1.2.6/29'): 6, IPv4Network('10.1.2.7/29'): 7} >>> From steven.bethard at gmail.com Mon Sep 28 17:49:15 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 28 Sep 2009 08:49:15 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <200909290127.03215.steve@pearwood.info> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: On Mon, Sep 28, 2009 at 8:27 AM, Steven D'Aprano wrote: > On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote: >> * Would you like argparse to grow an add_getopt_arguments method (as >> in my other post)? > > 0 > >> * If argparse grew an add_getopt_arguments, would you still want to >> keep getopt around? And if so, why? > > Simplicity of the learning curve. Using it is as simple as: > > getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) You forgot the for-loop, nested if/else statements and type conversions. ;-) > Does argparse allow anything as simple as that? If you mean, does argparse allow configuration to be specified using the getopt style ("a:b", ["alpha=", "beta"]), no, it currently doesn't. But if this is useful functionality, and would reasonably replace the getopt use cases, then I'd be happy to add it. In the simplest version, we might add something like:: options, args = argparse.getopt("a:b", ["alpha=", "beta"]) where you could then use "options" without any looping:: alpha = options.a or options.alpha beta = options.b or options.beta But if people still like the traditional getopt loop with nested if/elses better, then we might as well just keep getopt around and not add anything to argparse. I'm fine with that too, I just want to make sure that there isn't an obvious deficiency in argparse that we could easily fix. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From floris.bruynooghe at gmail.com Mon Sep 28 17:54:52 2009 From: floris.bruynooghe at gmail.com (Floris Bruynooghe) Date: Mon, 28 Sep 2009 16:54:52 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com> <20090928083820.GA488@laurie.devork> <20090928130317.GR23367@snowy.squish.net> Message-ID: <20090928155452.GB13277@laurie.devork> On Mon, Sep 28, 2009 at 07:28:39AM -0700, Steven Bethard wrote: > Ok, sounds like there are a number of supporters for keeping getopt > around and just deprecating optparse. For those who'd like to keep > getopt around, I have a few questions: > > * Would you be opposed to a note in the getopt documentation > suggesting argparse as an alternative? No, I'd encourage such a note. > * Would you like argparse to grow an add_getopt_arguments method (as > in my other post)? That looks nice indeed. > * If argparse grew an add_getopt_arguments, would you still want to > keep getopt around? And if so, why? If clearly documented as "Hey, getopt users, check this out!" then I'd be indifferent to the proposed deprecation of getopt. Regards Floris From olemis at gmail.com Mon Sep 28 18:03:50 2009 From: olemis at gmail.com (Olemis Lang) Date: Mon, 28 Sep 2009 11:03:50 -0500 Subject: [Python-Dev] Application configuration (was: PEP 389: argparse) In-Reply-To: <20090928153335.GA4747@phd.pp.ru> References: <20090928153335.GA4747@phd.pp.ru> Message-ID: <24ea26600909280903v106403a7y23e1839aba40d83f@mail.gmail.com> On Mon, Sep 28, 2009 at 10:33 AM, Oleg Broytman wrote: > On Mon, Sep 28, 2009 at 09:23:22AM -0600, m h wrote: > >> Does anyone else have interest in such functionality? ?Is it outside >> the realm of this PEP? > > ? It is outside the scope of this particular PEP, but it is certainly an > interesting idea. :) > ? Actually it is the Right Thing, but I doubt it is possible to implement > such a generic configuration *framework* in the Python standard *library*. > Applications use different configfile layouts, some programs use w32 > registry instead of configfiles... > If this is the only reason : Q: - What if only INI files are considered ? In the end, that's the format included in stdlib and used by stdlib commands (e.g. distutils ;o), and also by others (e.g. Trac, Hg, ...). Reducing the scope in this direction could yield good results. OTOH, if other cfg need to be supported, then possibly an ?object layer? (e.g. compatible with ConfigParser) could be used in order to support further config layouts built outside stdlib. For instance, in the case of WinReg : {{{ #!python RegParser = RegConfigParser(HKEY_CURRENT_USER, r'Software\MyApp\xxx') RegParser.sections() # Children of this reg key RegParser.options('s1') # Values under # HKEY_CURRENT_USER\Software\MyApp\xxx\s1 ... }}} ... and similar solutions could be implemented outside stdlib for e.g. YAML, ... PS: BTW, is there something like this already for WinReg ? -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Sobrepasa las 300 descargas el m?dulo dutest - http://feedproxy.google.com/~r/simelo-es/~3/U5rff5iTQPI/sobrepasa-las-300-descargas-el-modulo.html From mal at egenix.com Mon Sep 28 18:12:56 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 28 Sep 2009 18:12:56 +0200 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> References: <4AB67430.2060306@egenix.com> <4AC07309.1010608@egenix.com> <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> Message-ID: <4AC0E088.8080000@egenix.com> James Y Knight wrote: > On Sep 28, 2009, at 4:25 AM, M.-A. Lemburg wrote: >> Distributions should really not be put in charge of upstream >> coding design decisions. > > I don't think you can blame distros for this one.... > > From PEP 0261: > It is also proposed that one day --enable-unicode will just > default to the width of your platforms wchar_t. > > On linux, wchar_t is 4 bytes. The PEP also has this to say: This has the effect of doubling the size of most Unicode strings. In order to avoid imposing this cost on every user, Python 2.2 will allow the 4-byte implementation as a build-time option. Users can choose whether they care about wide characters or prefer to preserve memory. And that's still true today. It was the main reason for not making it the default on those days. Today, Python 3.x uses Unicode for all strings, so while the RAM situation has changed somewhat since Python 2.2, the change has a much wider effect on the Python memory foot-print than in late 2001. > If there's a consensus amongst python upstream that all the distros > should be shipping Python with UCS2 unicode strings, you should reach > out to them and say this, in a rather more clear fashion. Currently, > most signs point towards UCS4 builds as being the better option. UCS4 is the better option if you use lots of non-BMP code points and if you have to regularly interface with C APIs using wchar_t on Unix. > Or, one might reasonably wonder why UCS-4 is an option at all, if nobody > should enable it. See above: there are use cases where this does make a lot of sense. E.g. non-BMP code points can only be represented using surrogates on UCS2 builds and these can be tricky to deal with (or at least many people feel like it's tricky to deal with them ;-). >> People building their own Python version will usually also build >> their own extensions, so I don't really believe that the above >> scenario is very common. > > I'd just like to note that I've run into this trap multiple times. I > built a custom python, and expected it to work with all the existing, > installed, extensions (same major version as the system install, just > patched). And then had to build it again with UCS4, for it to actually > work. Of course building twice isn't the end of the world, and I'm > certainly used to having to twiddle build options on software to get it > working, but, this *does* happen, and *is* a tiny bit irritating. Which is why I think that Python should include some more information on the type of built being used, e.g. by placing the information prominently on the startup line. I still don't believe the above use case is a common one, though. That said, Zooko's original motivation for the proposed change is making installation of extensions easier for users. That's a tools question much more than a Python Unicode one. Aside: --enable-unicode is gone in Python 3.x. You now only have the choice to use the default (which is UCS2) or switch on the optional support for UCS4 by using --with-wide-unicode. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 28 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From digitalxero at gmail.com Mon Sep 28 18:24:30 2009 From: digitalxero at gmail.com (Dj Gilcrease) Date: Mon, 28 Sep 2009 10:24:30 -0600 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <79990c6b0909280825q57fce79ej6b6c390699c3f32d@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <79990c6b0909280825q57fce79ej6b6c390699c3f32d@mail.gmail.com> Message-ID: Ok updated patch so it meets the following conditions >>> IPv4Network(192.168.1.1/24) IPv4Network(192.168.1.0/24) >>> x = IPv4Network('192.168.1.1/24') >>> y = IPv4Network('192.168.1.0/24') >>> x == y True >>> hash(x) == hash(y) True The ip you instantiated the Network with is still accessible though, so >>> x.ip IPv4Address('192.168.1.1') >>> y.ip IPv4Address('192.168.1.0') I left it this way since there seems to be conflicting opinions on if this should exist in the Network object or not http://codereview.appspot.com/124057 From guido at python.org Mon Sep 28 18:39:06 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Sep 2009 09:39:06 -0700 Subject: [Python-Dev] PEP 3144 review, and the inclusion process In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> Message-ID: On Sun, Sep 27, 2009 at 5:32 PM, Antoine Pitrou wrote: > Peter Moody hda3.com> writes: >> >> I've never said otherwise. In fact, from an email last night, "If what >> the community requires is the library you've described, then ipaddr is >> not that library." The changes *you* require make ipaddr significantly >> less useful to me. I'm not prepared to make those changes in an >> attempt seek acceptance to the stdlib, especially if the stdlib is in >> such flux that I'll get to do this again in 18 months. > > Well, then I'm not sure why we have a PEP at all. > If you don't want any significant changes and if you consider it to be *your* > library, ipaddr can remain a third-party package that interested people can > easily install (no pun ;-)) since AFAIK it's pure Python. It will also make > maintenance easier for you, while freeing us (core developers) from having to > bother about it in our daily development tasks. > > At least that's what I would advocate right now - not sure about what others > think. I think Peter is pretty frustrated by the many attacks on "his" library. There are probably a number of different things going on simultaneous: Peter has been driven into the defense by attacks both reasonable and unreasonable, there have been misunderstandings all around, teasing out use cases (by both parties) has been a problem. Things might have gone differently if the PEP had started out with multiple authors. Maybe it's not too late to add one or more other interested parties to the PEP with the purpose of making the PEP more clearly the result of a consensus-gathering process. Any volunteers? At the same time I don't think a complete reset of the proposed API is necessary. I am rather more thinking of judicious API tweaks in order to cover some new use cases, without requiring a complete rewrite or destroying the usability of the proposal for Peter's original use cases. (In general I am pretty happy with the ipaddr code and API; it looks like what I would have done, but then I am blissfully unaware of some of the issues brought up in this thread.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From robert.kern at gmail.com Mon Sep 28 18:39:43 2009 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 28 Sep 2009 11:39:43 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <4AC0D5BC.40108@voidspace.org.uk> Message-ID: On 2009-09-28 10:37 AM, Steven Bethard wrote: > On Mon, Sep 28, 2009 at 8:26 AM, Michael Foord > wrote: >> m h wrote: >>> >>> Perhaps this is OT, but since command line parsing is part of >>> configuration, I figure I'd throw it out there. My scripts often have >>> configuration that the command line can override and I loosely follow >>> the example hierarchy[0] listed in The Art of Unix Programming. >>> >>> Some configuration I want in a config file (but I want to override >>> from the command line) and sometimes it's very nice to use environment >>> variables for configuration. So I do something like this: >> >> Integration with command line options is an occasionally-requested feature >> for ConfigObj. I've always said I would be open to patches... >> >> In other words, yes I think there is demand for it. Whether it belongs >> *immediately* in the standard library is another matter, but if you wrote a >> layer that unified ConfigParser and argparse I think you will find that >> people use it. >> >> It is well outside the realm of this PEP however. > > What Michael said. ;-) If you'd like to provide a patch for argparse > that provides such functionality, I'd be happy to review it. cfgparse appears to fill this role for optparse. It might be straightforward to port it to argparse. http://cfgparse.sourceforge.net/ -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From peter at hda3.com Mon Sep 28 19:22:05 2009 From: peter at hda3.com (Peter Moody) Date: Mon, 28 Sep 2009 10:22:05 -0700 Subject: [Python-Dev] PEP 3144 review, and the inclusion process In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> Message-ID: <8517e9350909281022v39aeb83clb352709822e0b26b@mail.gmail.com> [cc += david moss] On Mon, Sep 28, 2009 at 9:39 AM, Guido van Rossum wrote: > On Sun, Sep 27, 2009 at 5:32 PM, Antoine Pitrou wrote: >> Peter Moody hda3.com> writes: >>> >>> I've never said otherwise. In fact, from an email last night, "If what >>> the community requires is the library you've described, then ipaddr is >>> not that library." The changes *you* require make ipaddr significantly >>> less useful to me. I'm not prepared to make those changes in an >>> attempt seek acceptance to the stdlib, especially if the stdlib is in >>> such flux that I'll get to do this again in 18 months. >> >> Well, then I'm not sure why we have a PEP at all. >> If you don't want any significant changes and if you consider it to be *your* >> library, ipaddr can remain a third-party package that interested people can >> easily install (no pun ;-)) since AFAIK it's pure Python. It will also make >> maintenance easier for you, while freeing us (core developers) from having to >> bother about it in our daily development tasks. >> >> At least that's what I would advocate right now - not sure about what others >> think. > > I think Peter is pretty frustrated by the many attacks on "his" > library. There are probably a number of different things going on > simultaneous: Peter has been driven into the defense by attacks both > reasonable and unreasonable, there have been misunderstandings all > around, teasing out use cases (by both parties) has been a problem. > > Things might have gone differently if the PEP had started out with > multiple authors. Maybe it's not too late to add one or more other > interested parties to the PEP with the purpose of making the PEP more > clearly the result of a consensus-gathering process. ?Any volunteers? David called me a little over a week ago and expressed an interest in doing exactly this cross continent/ocean coordination has been a little difficult thus far and I'm not certain what his feelings on this are now. > At the same time I don't think a complete reset of the proposed API is > necessary. ?I am rather more thinking of judicious API tweaks in order > to cover some new use cases, without requiring a complete rewrite or > destroying the usability of the proposal for Peter's original use > cases. (In general I am pretty happy with the ipaddr code and API; it > looks like what I would have done, but then I am blissfully unaware of > some of the issues brought up in this thread.) > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From guido at python.org Mon Sep 28 19:33:21 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Sep 2009 10:33:21 -0700 Subject: [Python-Dev] Fuzziness in io module specs - PEP update proposition V2 In-Reply-To: References: <4AB3DCC9.5050209@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> <4ABF1B3B.4080106@gmail.com> Message-ID: On Sun, Sep 27, 2009 at 3:44 PM, Antoine Pitrou wrote: > Le Sun, 27 Sep 2009 14:24:52 -0700, Guido van Rossum a ?crit?: > [truncate()] >> >> What compatibility, though? > > Compatibility accross the 3.x line. Well, in this case, maybe compatibility with 2.x is more important -- this isn't something we can easily warn about in 2.6+. In addition there's the POSIX rules. >> f.truncate() behaves different in 2.x than >> in 3.x, and in 2.x it seems to match the POSIX semantics (i.e. the seek >> position is unchanged even though the file size is). Perhaps the changed >> semantics was an oversight or a mistake? > > Perhaps it was, indeed. I don't know who made that decision in the first > place. It might well have been me (when implementing the earliest version of io.py), and I might well have *though* that I implemented the same rules as 2.x, and never bothered to check. :-( All in all I think we should change this before it's too late; it will affect a very small number of apps (perhaps none?), but I would rather have the right semantics in the future. Also, it's trivial to write code that doesn't care (in fact code running under 2.x and 3.x probably will have to be written so that it doesn't care) so it's not like changing this is going to make life harder for people wanting multiple-version support. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Mon Sep 28 20:17:01 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 18:17:01 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review, and the inclusion process References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> Message-ID: Guido van Rossum python.org> writes: > > At the same time I don't think a complete reset of the proposed API is > necessary. I am rather more thinking of judicious API tweaks in order > to cover some new use cases, without requiring a complete rewrite or > destroying the usability of the proposal for Peter's original use > cases. I think the current API is mostly fine as well, except for one roadblock for which several people has raised concerns, and that Peter stated he doesn't want to change. Regards Antoine. From steve at pearwood.info Mon Sep 28 20:21:20 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 29 Sep 2009 04:21:20 +1000 Subject: [Python-Dev] PEP 3144 review, and the inclusion process In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: <200909290421.21622.steve@pearwood.info> On Tue, 29 Sep 2009 02:39:06 am Guido van Rossum wrote: > I think Peter is pretty frustrated by the many attacks on "his" > library. There are probably a number of different things going on > simultaneous: Peter has been driven into the defense by attacks both > reasonable and unreasonable, there have been misunderstandings all > around, teasing out use cases (by both parties) has been a problem. As one of those people who has been critical of parts of Peter's module, I'd like to publicly say that I *mostly* like it. Sometimes we focus too much on the negatives and don't remember to give credit where credit is due. Hopefully we can reach consensus, or at least understanding, on the remaining issue(s). -- Steven D'Aprano From sesquile at gmail.com Mon Sep 28 20:26:46 2009 From: sesquile at gmail.com (m h) Date: Mon, 28 Sep 2009 12:26:46 -0600 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <4AC0D5BC.40108@voidspace.org.uk> Message-ID: On Mon, Sep 28, 2009 at 9:37 AM, Steven Bethard wrote: > On Mon, Sep 28, 2009 at 8:26 AM, Michael Foord > wrote: >> m h wrote: >>> >>> Perhaps this is OT, but since command line parsing is part of >>> configuration, I figure I'd throw it out there. ?My scripts often have >>> configuration that the command line can override and I loosely follow >>> the example hierarchy[0] listed in The Art of Unix Programming. >>> >>> Some configuration I want in a config file (but I want to override >>> from the command line) and sometimes it's very nice to use environment >>> variables for configuration. ?So I do something like this: >> >> Integration with command line options is an occasionally-requested feature >> for ConfigObj. I've always said I would be open to patches... >> >> In other words, yes I think there is demand for it. Whether it belongs >> *immediately* in the standard library is another matter, but if you wrote a >> layer that unified ConfigParser and argparse I think you will find that >> people use it. Having it integrated with the option parser is ideal. More people will use it if it is in the stdlib. Just like more people will use argparse. >> >> It is well outside the realm of this PEP however. > > What Michael said. ;-) If you'd like to provide a patch for argparse > that provides such functionality, I'd be happy to review it. > Patch submitted [0]. Accidentally marked as "Defect" instead of enhancement. Comments/critiques welcome. 0 - http://code.google.com/p/argparse/issues/detail?id=35&q=label%3AType-Defect Cheers, -matt > Steve > -- > Where did you get that preposterous hypothesis? > Did Steve tell you that? > ? ? ? ?--- The Hiphopopotamus > From drkjam at gmail.com Mon Sep 28 20:24:37 2009 From: drkjam at gmail.com (DrKJam) Date: Mon, 28 Sep 2009 19:24:37 +0100 Subject: [Python-Dev] PEP 3144 review, and the inclusion process In-Reply-To: <8517e9350909281022v39aeb83clb352709822e0b26b@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> <8517e9350909281022v39aeb83clb352709822e0b26b@mail.gmail.com> Message-ID: <538a660a0909281124i63358f8asd7e3c4d8245d9ebe@mail.gmail.com> 2009/9/28 Peter Moody > [cc += david moss] > > On Mon, Sep 28, 2009 at 9:39 AM, Guido van Rossum > wrote: > > On Sun, Sep 27, 2009 at 5:32 PM, Antoine Pitrou > wrote: > >> Peter Moody hda3.com> writes: > >>> > >>> I've never said otherwise. In fact, from an email last night, "If what > >>> the community requires is the library you've described, then ipaddr is > >>> not that library." The changes *you* require make ipaddr significantly > >>> less useful to me. I'm not prepared to make those changes in an > >>> attempt seek acceptance to the stdlib, especially if the stdlib is in > >>> such flux that I'll get to do this again in 18 months. > >> > >> Well, then I'm not sure why we have a PEP at all. > >> If you don't want any significant changes and if you consider it to be > *your* > >> library, ipaddr can remain a third-party package that interested people > can > >> easily install (no pun ;-)) since AFAIK it's pure Python. It will also > make > >> maintenance easier for you, while freeing us (core developers) from > having to > >> bother about it in our daily development tasks. > >> > >> At least that's what I would advocate right now - not sure about what > others > >> think. > > > > I think Peter is pretty frustrated by the many attacks on "his" > > library. There are probably a number of different things going on > > simultaneous: Peter has been driven into the defense by attacks both > > reasonable and unreasonable, there have been misunderstandings all > > around, teasing out use cases (by both parties) has been a problem. > > > > Things might have gone differently if the PEP had started out with > > multiple authors. Maybe it's not too late to add one or more other > > interested parties to the PEP with the purpose of making the PEP more > > clearly the result of a consensus-gathering process. Any volunteers? > > David called me a little over a week ago and expressed an interest in > doing exactly this cross continent/ocean coordination has been a > little difficult thus far and I'm not certain what his feelings on > this are now. > > Sure, I'm happy to volunteer and help out. Let's have a good hard look at all this and see what we can come up with. > > At the same time I don't think a complete reset of the proposed API is > > necessary. I am rather more thinking of judicious API tweaks in order > > to cover some new use cases, without requiring a complete rewrite or > > destroying the usability of the proposal for Peter's original use > > cases. (In general I am pretty happy with the ipaddr code and API; it > > looks like what I would have done, but then I am blissfully unaware of > > some of the issues brought up in this thread.) > > > > -- > > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > > _______________________________________________ > > Python-Dev mailing list > > Python-Dev at python.org > > http://mail.python.org/mailman/listinfo/python-dev > > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Sep 28 20:43:06 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Sep 2009 11:43:06 -0700 Subject: [Python-Dev] PEP 3144 review, and the inclusion process In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> Message-ID: On Mon, Sep 28, 2009 at 11:17 AM, Antoine Pitrou wrote: > Guido van Rossum python.org> writes: >> >> At the same time I don't think a complete reset of the proposed API is >> necessary. ?I am rather more thinking of judicious API tweaks in order >> to cover some new use cases, without requiring a complete rewrite or >> destroying the usability of the proposal for Peter's original use >> cases. > > I think the current API is mostly fine as well, except for one roadblock for > which several people has raised concerns, and that Peter stated he doesn't want > to change. Well, I'm sure somebody can come up with a compromise that works for everyone. Assuming this is the issue about e.g. IPNetwork('7.7.7.7/30') != IPNetwork('7.7.7.4/30') even though these describe the same network (and in fact their .network and .broadcast attributes have the same values), there are several compromises; I don't know the use cases well enough to pick one: - There could be an optional keyword parameter that decides whether the last bits are cleared or not (one would have to argue which behavior to use by default). - The __eq__ (and hence __hash__) could be made to return equality even though the .ip attribute returns a different value and the repr() is different. This is not unheard-of; Python's __eq__ is best thought of as some sort of equivalence. For example two dicts that only differ in the order of the items still compare equal (even though .keys() and repr() return different values). I note that it's easy to convert the unnormalized form to the normalized form: a = IPNetwork('7.7.7.7/30') b = IPNetwork(a.network, a.prefixlen) It would also be easy to have a property or method that returns the normalized version; as an optimization it could return self if self is already normalized. And there could be an explicit API to test whether a network is normalized (something more explicit than a.network == a.ip). I'll leave it to others to discuss the pros and cons of each alternative. I don't know why Peter wants to preserve the unnormalized data but I'm sure he has a good use case, otherwise he wouldn't be so adamant to keep it. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Mon Sep 28 21:52:53 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Sep 2009 21:52:53 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: <4AC11415.9060305@v.loewis.de> > The fundamental divide here is between two behaviors. > > ipaddr: > > >>> x = IPv4Network('192.168.1.1/24') > >>> y = IPv4Network('192.168.1.0/24') > >>> x == y > False > >>> x.ip > IPv4Address('192.168.1.1') > > desired: > > >>> x = IPv4Network('192.168.1.1/24') > >>> y = IPv4Network('192.168.1.0/24') > >>> x == y > True > >>> x.ip > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'IPv4Network' object has no attribute 'ip' I can understand the debate around the comparison. I cannot understand the complaint that networks where the base address is not null-terminated should be banned, or useless (which was the original concern that I responded to). > Everything else is pretty much bikeshedding and can be dealt with. This > is fundamental and Peter has indicated he will not change it. I can see why either behavior is desirable (i.e. comparison should either use the specified address, or the network address); following the Python Zen, we should refuse the temptation to guess. Unfortunately, that would mean that we should provide two comparison operations, and have neither available as == (unless one is more clearly right than the other, which I'm skeptical about). Regards, Martin From martin at v.loewis.de Mon Sep 28 21:55:03 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 28 Sep 2009 21:55:03 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <20090928125818.1533.2070636475.divmod.xquotient.382@localhost.localdomain> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <20090928125818.1533.2070636475.divmod.xquotient.382@localhost.localdomain> Message-ID: <4AC11497.4060905@v.loewis.de> >> It's fairly obvious to me why the library should support 192.168.1.1/24 >> as an input, and return a network. >> >> End-users are likely going to enter such things (e.g. 82.94.164.162/29), >> as they will know one IP address in the network (namely, of one machine >> that they are looking at), and they will know the prefix length (more >> often, how large the network is - 8 or 16 or 32). So very clearly, >> end users should not be required to make the computation in their heads. >> >> So Python code has to make the computation, and it seems most natural >> that the IP library is the piece of code that is able to compute a >> network out of that input. > > And this is a rather classic example of a misfeature. "(Non-developer) > End users will type it in" is not argument for supporting a particular > string format as the primary constructor for an object. Constructors > are for *developers*. They should be made most useful for *developers*. > The issue of dealing with user input (which may have many other quirks) > is separate and should be dealt with separately. Still, people proposing that 192.168.1.1/24 should be rejected (I still don't know whether anybody is actually proposing that) *have* to provide a proposal how this should be input instead. Otherwise, such proposals must be rejected, since there is a clear and evident need to support such input somehow. Regards, Martin From brett at python.org Mon Sep 28 21:22:55 2009 From: brett at python.org (Brett Cannon) Date: Mon, 28 Sep 2009 12:22:55 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: On Mon, Sep 28, 2009 at 08:49, Steven Bethard wrote: > On Mon, Sep 28, 2009 at 8:27 AM, Steven D'Aprano wrote: >> On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote: >>> * Would you like argparse to grow an add_getopt_arguments method (as >>> in my other post)? >> >> 0 >> >>> * If argparse grew an add_getopt_arguments, would you still want to >>> keep getopt around? And if so, why? >> >> Simplicity of the learning curve. Using it is as simple as: >> >> getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) > > You forgot the for-loop, nested if/else statements and type conversions. ;-) > =) I do wonder if people who are advocating for getopt sticking around realize how much extra code must be written to make sure what it gives back to you is in some sane manner. Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])`` as an example and simply assume that 'alpha' takes a string as an argument and that it's required and that 'beta' is a boolean flag. To pull everything out you could do:: options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) options_dict = dict(options) alpha = options_dict.get('-a', options_dict.get('--alpha', '')) beta = '-b' in options_dict or '--beta' in options_dict main(alpha, beta, args) Obviously if one of the getopt supporters has a better way of doing this then please speak up. Now, Steven, can you show how best to do this in argparse? I am willing to bet that the total number of lines to do this is not that much more and does not require you to know to use 'or' or the dict constructor along with dict.get() in order to keep it compact. I can only imagine what some newbie might try to do in order to be correct (if they even try). -Brett From steven.bethard at gmail.com Mon Sep 28 21:57:08 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 28 Sep 2009 12:57:08 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: On Mon, Sep 28, 2009 at 12:22 PM, Brett Cannon wrote: > On Mon, Sep 28, 2009 at 08:49, Steven Bethard wrote: >> On Mon, Sep 28, 2009 at 8:27 AM, Steven D'Aprano wrote: >>> On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote: >>>> * Would you like argparse to grow an add_getopt_arguments method (as >>>> in my other post)? >>> >>> 0 >>> >>>> * If argparse grew an add_getopt_arguments, would you still want to >>>> keep getopt around? And if so, why? >>> >>> Simplicity of the learning curve. Using it is as simple as: >>> >>> getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) >> >> You forgot the for-loop, nested if/else statements and type conversions. ;-) >> > > =) I do wonder if people who are advocating for getopt sticking around > realize how much extra code must be written to make sure what it gives > back to you is in some sane manner. > > Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])`` > as an example and simply assume that 'alpha' takes a string as an > argument and that it's required and that 'beta' is a boolean flag. To > pull everything out you could do:: > > ?options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) > ?options_dict = dict(options) > ?alpha = options_dict.get('-a', options_dict.get('--alpha', '')) > ?beta = '-b' in options_dict or '--beta' in options_dict > > ?main(alpha, beta, args) > > Obviously if one of the getopt supporters has a better way of doing > this then please speak up. > > Now, Steven, can you show how best to do this in argparse? Here's the same option parsing in argparse: parser = argparse.ArgumentParser() parser.add_argument('-a', '--alpha') parser.add_argument('-b', '--beta', action='store_true') args = parser.parse_args() main(args.alpha, args.beta) Or if those final positional arguments were actually meaningful, then you would add one more argument like this:: parser = argparse.ArgumentParser() parser.add_argument('-a', '--alpha') parser.add_argument('-b', '--beta', action='store_true') parser.add_argument('gammas', nargs='*') args = parser.parse_args() main(args.alpha, args.beta, args.gammas) Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From martin at v.loewis.de Mon Sep 28 22:11:01 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 28 Sep 2009 22:11:01 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> Message-ID: <4AC11855.7080406@v.loewis.de> > Martin v. L?wis v.loewis.de> writes: >>> Could you explain what benefit there is for allowing the user to create >>> network objects that don't represent networks? Is there a use-case >>> where these networks-that-aren't-networks are something other than a >>> typo? Under what circumstances would I want to specify a network as >>> 192.168.1.1/24 instead of 192.168.1.0/24? >>> > [...] >> So Python code has to make the computation, and it seems most natural >> that the IP library is the piece of code that is able to compute a >> network out of that input. > > The thing is, it doesn't create a network, it creates a hybrid "network plus > host" which retains knowledge about the original host (that is, 192.168.1.1 > rather than simply 192.168.1.0, if you enter "192.168.1.1/24"). > > That's what the OP meant with "networks-that-aren't-networks", and that's what > people are objecting to. That's not the question that was asked, though - the question asked was "Under what circumstances would I want to specify...". I hope most people agree that it is desirable to be able to specify a network not just by its network address. Regards, Martin From solipsis at pitrou.net Mon Sep 28 22:11:56 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 20:11:56 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11415.9060305@v.loewis.de> Message-ID: > Unfortunately, that would mean that we should provide two comparison > operations, and have neither available as == (unless one is more clearly > right than the other, which I'm skeptical about). That's probably why some people have argued to have two separate classes for the case of "networks" with significant lower bits. (by "lower bits" I mean bits which would be zeroed when ANDing with the netmask) From solipsis at pitrou.net Mon Sep 28 22:14:32 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 20:14:32 +0000 (UTC) Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <20090928125818.1533.2070636475.divmod.xquotient.382@localhost.localdomain> <4AC11497.4060905@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > > Still, people proposing that 192.168.1.1/24 should be rejected (I still > don't know whether anybody is actually proposing that) *have* to provide > a proposal how this should be input instead. It was proposed to have a dedicated function or class method which would parse such strings and return a (Network, Address) tuple corresponding to the two objects that are described by that single notation. >>> parse_network_and_host('192.168.1.1/24') (Network('192.168.1.0/24'), Address('192.168.1.1')) Regards Antoine. From scott+python-dev at scottdial.com Mon Sep 28 22:19:15 2009 From: scott+python-dev at scottdial.com (Scott Dial) Date: Mon, 28 Sep 2009 16:19:15 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC11855.7080406@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: <4AC11A43.2010401@scottdial.com> Martin v. L?wis wrote: >> Martin v. L?wis v.loewis.de> writes: >>>> Could you explain what benefit there is for allowing the user to create >>>> network objects that don't represent networks? Is there a use-case >>>> where these networks-that-aren't-networks are something other than a >>>> typo? Under what circumstances would I want to specify a network as >>>> 192.168.1.1/24 instead of 192.168.1.0/24? >>>> >> [...] >>> So Python code has to make the computation, and it seems most natural >>> that the IP library is the piece of code that is able to compute a >>> network out of that input. >> The thing is, it doesn't create a network, it creates a hybrid "network plus >> host" which retains knowledge about the original host (that is, 192.168.1.1 >> rather than simply 192.168.1.0, if you enter "192.168.1.1/24"). >> >> That's what the OP meant with "networks-that-aren't-networks", and that's what >> people are objecting to. > > That's not the question that was asked, though - the question asked > was "Under what circumstances would I want to specify...". I hope > most people agree that it is desirable to be able to specify a network > not just by its network address. I can't help but feel that you and many others jumping on Antoine et al. have skipped reading or only skimmed this topic of discussion. Antoine Pitrou wrote: > We just need a factory function which returns a tuple after parsing: > > >>> addr, net = parse_address_with_mask('192.168.1.1/24') > >>> addr == IPv4Address('192.168.1.1') > True > >>> net == IPv4Network('192.168.1.0/24') > True I still back this suggestion. I would also support a "loose=True" flag for the IPNetwork constructor, which would allow host bits to be non-zero. However, I will *never* support a proposal that includes retaining the host address. I believe I am not alone in that. However, Peter has made it quite clear that he will *never* change that about the IPNetwork classes. I would be quite glad to not have ipaddr included in the stdlib. As I can only imagine code that uses it being quite confusing to maintain (e.g., "wait is that actually a network or is that someone using IPNetwork to be an IPAddressWithMask?"). -1. -- Scott Dial scott at scottdial.com scodial at cs.indiana.edu From solipsis at pitrou.net Mon Sep 28 22:32:45 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 22:32:45 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC11855.7080406@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: <1254169965.4663.25.camel@localhost> Le lundi 28 septembre 2009 ? 22:11 +0200, "Martin v. L?wis" a ?crit : > > That's not the question that was asked, though - the question asked > was "Under what circumstances would I want to specify...". I hope > most people agree that it is desirable to be able to specify a network > not just by its network address. To me it makes no sense to "specify a network not just by its network address", because by definition a prefix + mask is all that characterizes a network: that's all you need to specify it entirely. Adding a "base address" would be like adding a "port number" attribute to the Address class: it's abusing an existing class by adding irrelevant information. I may be mistaken, but it seems to me that, in the IP world, nowhere does such a "network + host" hybrid appear. You find networks (prefix + mask) in routing protocols, and host addresses in applicative protocols. From rdmurray at bitdance.com Mon Sep 28 22:36:10 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 28 Sep 2009 16:36:10 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC11855.7080406@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: On Mon, 28 Sep 2009 at 22:11, "Martin v. L??wis" wrote: >> Martin v. L??wis v.loewis.de> writes: >>>> Could you explain what benefit there is for allowing the user to create >>>> network objects that don't represent networks? Is there a use-case >>>> where these networks-that-aren't-networks are something other than a >>>> typo? Under what circumstances would I want to specify a network as >>>> 192.168.1.1/24 instead of 192.168.1.0/24? >>>> >> [...] >>> So Python code has to make the computation, and it seems most natural >>> that the IP library is the piece of code that is able to compute a >>> network out of that input. >> >> The thing is, it doesn't create a network, it creates a hybrid "network plus >> host" which retains knowledge about the original host (that is, 192.168.1.1 >> rather than simply 192.168.1.0, if you enter "192.168.1.1/24"). >> >> That's what the OP meant with "networks-that-aren't-networks", and that's what >> people are objecting to. > > That's not the question that was asked, though - the question asked > was "Under what circumstances would I want to specify...". I hope > most people agree that it is desirable to be able to specify a network > not just by its network address. But then it's not a network, it is an ipaddress-plus-mask. It is exactly that conflation that we are objecting to. There is no question about the use case of _specifying_ a network ip plus a netmask and deriving a network object from that. That is unquestionably required by any useful API. The argument is about whether the object returned is a Network object, or a hybrid object representing _both_ an IP address and a network. It is the latter, which ipaddr does, which many of us find problematic and likely to lead to hard to read programs that will probably produce maintenance errors. I observe that this line in the current PEP rationale: IP addresses and IP networks are distinct. is not in fact achieved by the reference implementation. Peter, however, clearly thinks it is, since it is listed as a design goal of ipaddr. This is a language disconnect I don't understand, which I think has been the source of a lot of the difficulties in this thread. --David From solipsis at pitrou.net Mon Sep 28 22:39:43 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 28 Sep 2009 20:39:43 +0000 (UTC) Subject: [Python-Dev] PEP 389: argparse - optparse compatibility References: Message-ID: Steven Bethard gmail.com> writes: > > I could conceivably add an OptionParser class which:: > [...] Actually, I think that would be a waste of time. It would be less time-consuming to keep optparse.py than write a whole new compatibility bridge that you'll have to debug and maintain anyway, with the added problem of making people annoyed by whatever difference in behaviour happens to bite them. (*) So I think the two only reasonable options are: 1. either keep optparse durably 2. or deprecate it Given your precision about the proposed deprecation period (with the module vanishing in 3.5), I guess I'm only -0.5 on it, rather than -1. (*) (but of course I'm not against anyone wasting their time if they want to :-)) Regards Antoine. From guido at python.org Mon Sep 28 22:43:44 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Sep 2009 13:43:44 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: On Mon, Sep 28, 2009 at 1:36 PM, R. David Murray wrote: > On Mon, 28 Sep 2009 at 22:11, "Martin v. L?wis" wrote: >>> >>> Martin v. L?wis v.loewis.de> writes: >>>>> >>>>> Could you explain what benefit there is for allowing the user to create >>>>> network objects that don't represent networks? Is there a use-case >>>>> where these networks-that-aren't-networks are something other than a >>>>> typo? Under what circumstances would I want to specify a network as >>>>> 192.168.1.1/24 instead of 192.168.1.0/24? >>>>> >>> [...] >>>> >>>> So Python code has to make the computation, and it seems most natural >>>> that the IP library is the piece of code that is able to compute a >>>> network out of that input. >>> >>> The thing is, it doesn't create a network, it creates a hybrid "network >>> plus >>> host" which retains knowledge about the original host (that is, >>> 192.168.1.1 >>> rather than simply 192.168.1.0, if you enter "192.168.1.1/24"). >>> >>> That's what the OP meant with "networks-that-aren't-networks", and that's >>> what >>> people are objecting to. >> >> That's not the question that was asked, though - the question asked >> was "Under what circumstances would I want to specify...". I hope >> most people agree that it is desirable to be able to specify a network >> not just by its network address. > > But then it's not a network, it is an ipaddress-plus-mask. ?It is exactly > that conflation that we are objecting to. ?There is no question about > the use case of _specifying_ a network ip plus a netmask and deriving > a network object from that. ?That is unquestionably required by any > useful API. ?The argument is about whether the object returned is a > Network object, or a hybrid object representing _both_ an IP address > and a network. ?It is the latter, which ipaddr does, which many of us > find problematic and likely to lead to hard to read programs that will > probably produce maintenance errors. > > I observe that this line in the current PEP rationale: > > ? ?IP addresses and IP networks are distinct. > > is not in fact achieved by the reference implementation. ?Peter, however, > clearly thinks it is, since it is listed as a design goal of ipaddr. > This is a language disconnect I don't understand, which I think has > been the source of a lot of the difficulties in this thread. I say the case has been argued extensively, let's wait for Peter to respond. I would say that there certainly are precedents in other areas for keeping the information about the input form around. For example, occasionally it would be handy if parsing a hex integer returned an object that was compatible with other integers but somehow kept a hint that would cause printing it to use hex by default. I see keeping around the IP address used to create a network object as a similar edge case. I would probably define the __eq__ method to implement equivalency (ignoring the form of the input), just as I would in the case of the (hypothetical) hex integers. If you wanted to do a comparison that includes the input IP address, you could use (a, a.ip) == (b, b.ip). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rdmurray at bitdance.com Mon Sep 28 22:47:26 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 28 Sep 2009 16:47:26 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <1254169965.4663.25.camel@localhost> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <1254169965.4663.25.camel@localhost> Message-ID: On Mon, 28 Sep 2009 at 22:32, Antoine Pitrou wrote: > Le lundi 28 septembre 2009 ?? 22:11 +0200, "Martin v. L??wis" a ??crit : >> >> That's not the question that was asked, though - the question asked >> was "Under what circumstances would I want to specify...". I hope >> most people agree that it is desirable to be able to specify a network >> not just by its network address. > > To me it makes no sense to "specify a network not just by its network > address", because by definition a prefix + mask is all that > characterizes a network: that's all you need to specify it entirely. > Adding a "base address" would be like adding a "port number" attribute > to the Address class: it's abusing an existing class by adding > irrelevant information. > > I may be mistaken, but it seems to me that, in the IP world, nowhere > does such a "network + host" hybrid appear. You find networks (prefix + > mask) in routing protocols, and host addresses in applicative protocols. You do find "address plus mask" when specifying an IP address for an interface. This is a shorthand for specifying the IP address plus the network, since the network can be derived from the IP plus the mask. But it is a _shorthand_, it isn't an entity in its own right. There is no such thing as "a network that has an IP", there is only an _interface configuration_ that has an IP _and_ an associated network. --David From ubershmekel at gmail.com Mon Sep 28 23:14:59 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Tue, 29 Sep 2009 00:14:59 +0300 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> for a live demo of how getopt is useful and flexible, I like how Audacity uses it: http://www.google.com/codesearch/p?hl=en&sa=N&cd=6&ct=rc#_hWFOhGz9lE/mezzo/scons/sconsign.py&q=getopt%20%22import%20getopt%22%20file:%5C.py$&l=264 To answer your question, it goes like this: options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) for o, a in options: if o in ('-a', '--alpha'): alpha = a elif o in ('-b', '--beta'): beta = a main(alpha, beta, args) Notice a few things: 1. There is no chance of the script killing itself. In argparse and optparse exit() is called on every parsing error (btw because of this it sucks to debug parse_args in an interpreter). 2. There is no chance the parser will print things I don't want it to print. 3. You can do whatever you want using this flow - call a function, a binary operator (ie | is used in audacity scons example above) 4. in argparse this flow can be emulated and would be nicer, in ('-a', '--alpha') becomes == 'alpha' In a perfect world, getopt would be the low level parser that argparse and optparse rely on. This is not too far fetched btw, all that needs to be done is add another optional argument to getopt that would allow '-' and '--' to be replaced with arbitrary signs, OR moving the parsing code from argparse into getopt. On Mon, Sep 28, 2009 at 10:57 PM, Steven Bethard wrote: > On Mon, Sep 28, 2009 at 12:22 PM, Brett Cannon wrote: > > On Mon, Sep 28, 2009 at 08:49, Steven Bethard > wrote: > >> On Mon, Sep 28, 2009 at 8:27 AM, Steven D'Aprano > wrote: > >>> On Tue, 29 Sep 2009 12:28:39 am Steven Bethard wrote: > >>>> * Would you like argparse to grow an add_getopt_arguments method (as > >>>> in my other post)? > >>> > >>> 0 > >>> > >>>> * If argparse grew an add_getopt_arguments, would you still want to > >>>> keep getopt around? And if so, why? > >>> > >>> Simplicity of the learning curve. Using it is as simple as: > >>> > >>> getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) > >> > >> You forgot the for-loop, nested if/else statements and type conversions. > ;-) > >> > > > > =) I do wonder if people who are advocating for getopt sticking around > > realize how much extra code must be written to make sure what it gives > > back to you is in some sane manner. > > > > Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])`` > > as an example and simply assume that 'alpha' takes a string as an > > argument and that it's required and that 'beta' is a boolean flag. To > > pull everything out you could do:: > > > > options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) > > options_dict = dict(options) > > alpha = options_dict.get('-a', options_dict.get('--alpha', '')) > > beta = '-b' in options_dict or '--beta' in options_dict > > > > main(alpha, beta, args) > > > > Obviously if one of the getopt supporters has a better way of doing > > this then please speak up. > > > > Now, Steven, can you show how best to do this in argparse? > > Here's the same option parsing in argparse: > > parser = argparse.ArgumentParser() > parser.add_argument('-a', '--alpha') > parser.add_argument('-b', '--beta', action='store_true') > args = parser.parse_args() > > main(args.alpha, args.beta) > > Or if those final positional arguments were actually meaningful, then > you would add one more argument like this:: > > parser = argparse.ArgumentParser() > parser.add_argument('-a', '--alpha') > parser.add_argument('-b', '--beta', action='store_true') > parser.add_argument('gammas', nargs='*') > args = parser.parse_args() > > main(args.alpha, args.beta, args.gammas) > > Steve > -- > Where did you get that preposterous hypothesis? > Did Steve tell you that? > --- The Hiphopopotamus > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Mon Sep 28 23:16:57 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Sep 2009 23:16:57 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <1254169965.4663.25.camel@localhost> Message-ID: <4AC127C9.5080504@v.loewis.de> > You do find "address plus mask" when specifying an IP address for an > interface. This is a shorthand for specifying the IP address plus the > network, since the network can be derived from the IP plus the mask. > But it is a _shorthand_ Just as he says. Martin From martin at v.loewis.de Mon Sep 28 23:29:42 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 28 Sep 2009 23:29:42 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: <4AC12AC6.80304@v.loewis.de> > I would say that there certainly are precedents in other areas for > keeping the information about the input form around. For example, > occasionally it would be handy if parsing a hex integer returned an > object that was compatible with other integers but somehow kept a hint > that would cause printing it to use hex by default. At the risk of bringing in false analogies: it seems that Python typically represents values of some type in their canonical form, rather than remembering the form in which they arrived in the program: - integer values "forget" how many preceding zeroes they have - string literals forget which of the characters had been escaped, and whether the string was single- or double-quoted - floating point values forget a lot more about their literal representation (including even the literal decimal value) I guess a close case would be rational numbers: clearly, 3?2 == 6?4; would a Python library still remember (and repr) the original numerator and denominator? Regards, Martin From daniel at stutzbachenterprises.com Mon Sep 28 23:37:38 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Mon, 28 Sep 2009 16:37:38 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC12AC6.80304@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> Message-ID: On Mon, Sep 28, 2009 at 4:29 PM, "Martin v. L?wis" wrote: > I guess a close case would be rational numbers: clearly, 3?2 == 6?4; > would a Python library still remember (and repr) the original numerator > and denominator? > No need for a hypothetical, rational numbers were added in Python 2.6: Python 2.6.2 (r262:71600, Apr 15 2009, 07:20:39) [GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin Type "help", "copyright", "credits" or "license" for more information. >>> import fractions >>> fractions.Fraction(6,4) Fraction(3, 2) -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From cs at zip.com.au Mon Sep 28 23:44:50 2009 From: cs at zip.com.au (Cameron Simpson) Date: Tue, 29 Sep 2009 07:44:50 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: Message-ID: <20090928214450.GA6996@cskk.homeip.net> On 27Sep2009 21:24, Steven Bethard wrote: | On Sun, Sep 27, 2009 at 9:09 PM, "Martin v. L?wis" wrote: | >> If you think getopt and optparse should stick around in 3.X, why is | >> that? If you think there are things that getopt and optparse do better | >> than argparse, could you please give some examples? | > | > I personally consider getopt superior to both optparse and argparse. | > Those are terribly verbose in specifying arguments, whereas getopt's | > sequence-of-letters is really nice and compact. | | Thanks for the concrete example. Although I'm unconvinced that the | characters you save in the sequence of letters in the getopt.getopt | call aren't afterwards wasted on type conversions, if/else statements | and variable assignments in the subsequent loop, it would be pretty | simple to add to argparse something like:: | | ArgumentParser.add_getopt_arguments(options[, long_options]) Yes please! I'm also very fond of the succinct getopt sequence-of-letters style; it works really well for the simple cases. Disclaimer: I've not used the optparse et al modules because getopt has covered my needs and my C background made getopt the natural module to start with. I have written simple getopt-equivalent option parsers a number of times though. (Of course, that choice also drives me to adapt my option wishes to stuff that getopt can do:-) Clarification: this isn't a vote for preferring getopt, it's a vote for availability of the sequence-of-letters style. Cheers, -- Cameron Simpson DoD#743 http://www.cskk.ezoshosting.com/cs/ The most annoying thing about being without my files after our disc crash was discovering once again how widespread BLINK was on the web. From phd at phd.pp.ru Mon Sep 28 23:51:07 2009 From: phd at phd.pp.ru (Oleg Broytman) Date: Tue, 29 Sep 2009 01:51:07 +0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <20090928214450.GA6996@cskk.homeip.net> References: <20090928214450.GA6996@cskk.homeip.net> Message-ID: <20090928215107.GA15173@phd.pp.ru> On Tue, Sep 29, 2009 at 07:44:50AM +1000, Cameron Simpson wrote: > | ArgumentParser.add_getopt_arguments(options[, long_options]) > > Yes please! > > I'm also very fond of the succinct getopt sequence-of-letters style; > it works really well for the simple cases. The syntax "a:b" is also widely known; people who are often write small shell scripts are certainly accustomed to it (I am). > Clarification: this isn't a vote for preferring getopt, it's a vote for > availability of the sequence-of-letters style. +1 Oleg. -- Oleg Broytman http://phd.pp.ru/ phd at phd.pp.ru Programmers don't die, they just GOSUB without RETURN. From greg.ewing at canterbury.ac.nz Tue Sep 29 01:19:30 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Sep 2009 11:19:30 +1200 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4AC089F7.8030004@voidspace.org.uk> References: <4ABF2047.7050501@gmail.com> <4ABFCDED.8080105@gmail.com> <4ABFD9D6.9070602@voidspace.org.uk> <4AC00D2A.6020504@canterbury.ac.nz> <4AC089F7.8030004@voidspace.org.uk> Message-ID: <4AC14482.1040807@canterbury.ac.nz> Michael Foord wrote: > Well, the file type knows what mode it is opened in so catching these > errors is really not difficult at all. Yes, in this particular case it could do some checking of its own before making the system call. But this still has nothing to do with the exception hierarchy. -- Greg From fuzzyman at voidspace.org.uk Tue Sep 29 01:21:50 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 29 Sep 2009 00:21:50 +0100 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4AC14482.1040807@canterbury.ac.nz> References: <4ABF2047.7050501@gmail.com> <4ABFCDED.8080105@gmail.com> <4ABFD9D6.9070602@voidspace.org.uk> <4AC00D2A.6020504@canterbury.ac.nz> <4AC089F7.8030004@voidspace.org.uk> <4AC14482.1040807@canterbury.ac.nz> Message-ID: <4AC1450E.1040207@voidspace.org.uk> Greg Ewing wrote: > Michael Foord wrote: > >> Well, the file type knows what mode it is opened in so catching these >> errors is really not difficult at all. > > Yes, in this particular case it could do some checking of > its own before making the system call. But this still has > nothing to do with the exception hierarchy. > You said that having an exception hierarchy was impossible because the only information we have is the error number. Some errors could easily be caught. Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From ncoghlan at gmail.com Mon Sep 28 13:58:22 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 28 Sep 2009 21:58:22 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> Message-ID: <4AC0A4DE.4080402@gmail.com> Peter Moody wrote: >> Assuming the Network type loses the notion of a specific host (or host address, >> or `ip`) attached to it, yes. > > this is "less useful (strictly removing functionality)" and is an > example of what I explicitly said I was not going to do with ipaddr. A possible compromise then: Just change the definition of network equality to only consider the network address and ignore the ip attribute. Optionally provide a separate comparison method that pays attention to the ip attribute (keeping in mind that it isn't particularly hard for people to write out "net1 == net2 and net1.ip == net2.ip" if they care about the extra state). Then "IPNetwork" can be largely documented as a true IP network definition, with the "Host with Netmask" use case handled by having an IPNetwork object with "net.ip != net.network". Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From andrewm at object-craft.com.au Tue Sep 29 03:40:18 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Tue, 29 Sep 2009 11:40:18 +1000 Subject: [Python-Dev] PEP 3144 review, and the inclusion process In-Reply-To: <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> <8517e9350909271617w6699b0b2o2d992ebe043483d@mail.gmail.com> Message-ID: <20090929014018.CEED559C0A4@longblack.object-craft.com.au> >I've never said otherwise. In fact, from an email last night, "If what >the community requires is the library you've described, then ipaddr is >not that library." The changes *you* require make ipaddr significantly >less useful to me. I'm not prepared to make those changes in an >attempt seek acceptance to the stdlib, especially if the stdlib is in >such flux that I'll get to do this again in 18 months. The point is that, having brought it to us, we all now have an interest in the outcome. Whatever goes into the standard library is going to be something that we have to live with for a long time, and now is our best chance to shape the result. I understand your concern over introducing more classes, however I still feel my suggested functional decomposition is worth that cost because I consider the behaviour of my suggested classes to be more intuitive. I should mention that I am not a computer scientist, and none of this is motivated by a desire for theoretical purity - just practical experience. One of my concerns now is that if a code block receives an IPv4Network instance, it does not know whether this represents a host address with mask, or a network address. In some contexts, this distinction is critical, and confounding them can result in delayed error reporting or erroneous behaviour. Your addition of a strict flag does not completely address this concern as it assumes the instantiation and use occur in proximity, which is often not the case in large projects. I suspect you are also mistaken in thinking my proposed changes make the module less useful for you - maybe you can describe the problem as you see it? As a reminder to people who have come late to this thread, I proposed three classes per protocol: IPv?Address A single address IPv?AddressWithMask A single address with implied IPv?Network IPv?Network A container-like network address (with strict mask parsing) Further: * Comparisons between classes should be disallowed. * The IPv?AddressWithMask class would have a .address and .mask attributes containing IPv?Addresses, and a .network attribute for the containing network (as specified by the mask, and lazily constructed). * The IPv?Network class would have similar .address and .mask attributes. In cases where you want to allow lax specification of network addresses, this would be spelt: IPv?AddressWithMask(some_address).network At first glance, this seems somewhat round-about, however it makes explicit the potential loss of bits. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From rdmurray at bitdance.com Tue Sep 29 04:17:54 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Mon, 28 Sep 2009 22:17:54 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: On Mon, 28 Sep 2009 at 13:43, Guido van Rossum wrote: > On Mon, Sep 28, 2009 at 1:36 PM, R. David Murray wrote: > I would say that there certainly are precedents in other areas for > keeping the information about the input form around. For example, > occasionally it would be handy if parsing a hex integer returned an > object that was compatible with other integers but somehow kept a hint > that would cause printing it to use hex by default. > > I see keeping around the IP address used to create a network object as > a similar edge case. I would probably define the __eq__ method to > implement equivalency (ignoring the form of the input), just as I > would in the case of the (hypothetical) hex integers. If you wanted to > do a comparison that includes the input IP address, you could use (a, > a.ip) == (b, b.ip). Ignoring Antoine's excellent points for the moment, to my mind one difference between your integer-with-hex-hint and ipaddr's ip-with-netmask is that there is a one to one mapping between the canonical hex representations and the integer. If you want to argue that the _exact_ input string be preserved in the integer object, then there might be more of a direct analogy. Except that ipaddr isn't preserving the input _string_, it's preserving a canonical representation of that string, so even there the analogy wouldn't be all that close. The difference here is that ipaddr is preserving input information that is irrelevant to the object being returned (a network). The fractions case is much closer. Or consider another analogy: the ipaddr case is like the mod function returning a single object that is supposed to be the remainder, but also has an extra field that records the original input number. This might be a useful operator and object type in certain contexts, but it is something that would be layered on top of the real mod function that returns a simple integer remainder. You would never approve this remainder-with-saved-input as the data type returned by the mod operator. (I hope :) Similarly, there should be a basic ipaddr parsing function (and I don't much care how you spell it, even though I have opinions) that returns an ip address and a network object when given an ipaddress-plus-netmask string. An additional object built upon those fundamental data types that remembers the input IP would be fine, but the data type that remembers the IP should _not_ be the fundamental data type in the system. That strikes me as poor design, OO or otherwise. In fact, your "sometimes it would be useful if" phrasing indicates that your 'integer with hex hint' data type would also not be the fundamental data type, but instead a subclass of int. --David From martin at v.loewis.de Tue Sep 29 04:18:25 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 Sep 2009 04:18:25 +0200 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> References: <4AB67430.2060306@egenix.com> <4AC07309.1010608@egenix.com> <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> Message-ID: <4AC16E71.3040400@v.loewis.de> James Y Knight wrote: > On Sep 28, 2009, at 4:25 AM, M.-A. Lemburg wrote: >> Distributions should really not be put in charge of upstream >> coding design decisions. > > I don't think you can blame distros for this one.... > > From PEP 0261: > It is also proposed that one day --enable-unicode will just > default to the width of your platforms wchar_t. > > On linux, wchar_t is 4 bytes. > > If there's a consensus amongst python upstream that all the distros > should be shipping Python with UCS2 unicode strings, you should reach > out to them and say this, in a rather more clear fashion. Currently, > most signs point towards UCS4 builds as being the better option. There is no such consensus. Linux distributions really should build Python in UCS-4 mode, and I would be in favor of making it the default to match wchar_t. Regards, Martin From guido at python.org Tue Sep 29 05:18:43 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 28 Sep 2009 20:18:43 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: On Mon, Sep 28, 2009 at 7:17 PM, R. David Murray wrote: > On Mon, 28 Sep 2009 at 13:43, Guido van Rossum wrote: >> >> On Mon, Sep 28, 2009 at 1:36 PM, R. David Murray >> wrote: >> I would say that there certainly are precedents in other areas for >> keeping the information about the input form around. For example, >> occasionally it would be handy if parsing a hex integer returned an >> object that was compatible with other integers but somehow kept a hint >> that would cause printing it to use hex by default. >> >> I see keeping around the IP address used to create a network object as >> a similar edge case. I would probably define the __eq__ method to >> implement equivalency (ignoring the form of the input), just as I >> would in the case of the (hypothetical) hex integers. If you wanted to >> do a comparison that includes the input IP address, you could use (a, >> a.ip) == (b, b.ip). > > Ignoring Antoine's excellent points for the moment, to my mind > one difference between your integer-with-hex-hint and ipaddr's > ip-with-netmask is that there is a one to one mapping between the > canonical hex representations and the integer. > > If you want to argue that the _exact_ input string be preserved in the > integer object, then there might be more of a direct analogy. ?Except that > ipaddr isn't preserving the input _string_, it's preserving a canonical > representation of that string, so even there the analogy wouldn't be > all that close. ?The difference here is that ipaddr is preserving input > information that is irrelevant to the object being returned (a network). I disagree that recording the exact input string would be a better analogy; as you say the hypothetical hex integer doesn't save the exact input string -- it remembers that the base was 16 That happens to be 1 bit of information if you're only considering hex vs. decimal, but one could imagine a further version that supports any base between 1 and 36 (or however far you want to go). Whether I typed 01.01.01.01/024 or 1.1.1.1/24 is (presumably) irrelevant for this case, since the byte values are the same; whether I typed 1.1.1.1/24 or 1.1.1.0/24 *is* relevant (for Peter). There's probably another form, 16843009/24, which is also equivalent. > The fractions case is much closer. Hardly -- I've never heard of someone who had a use case for denormalized fractions, but I don't doubt that Peter has a use case for denormalized IPNetwork objects. (Do you doubt that Peter has such a use case? If so, we have a much more fundamental disagreement.) > Or consider another analogy: the ipaddr case is like the mod function > returning a single object that is supposed to be the remainder, > but also has an extra field that records the original input number. > This might be a useful operator and object type in certain contexts, but > it is something that would be layered on top of the real mod function > that returns a simple integer remainder. ?You would never approve this > remainder-with-saved-input as the data type returned by the mod operator. > (I hope :) Not a very strong argument, since that use case is purely hypothetical. I brought up the hex integer example to show that it is possible to conceive of use cases for objects that record some sort of information about their creation history. Now you're bending my line of reasoning by going from the assertion "Guido would never approve of mod-returning-remainder-with-saved-input" (which happens to be true until a real use case is found) to "Guido would never approve of *any* operator that keeps some traces of its creation history" (which is false -- dict being just one counter-example :-). > Similarly, there should be a basic ipaddr parsing function (and I don't > much care how you spell it, even though I have opinions) that returns > an ip address and a network object when given an ipaddress-plus-netmask > string. An additional object built upon those fundamental data types that > remembers the input IP would be fine, but the data type that remembers > the IP should _not_ be the fundamental data type in the system. ?That > strikes me as poor design, OO or otherwise. Poor design is highly subjective, and I simply disagree that one design is a priori better or worse than the other. This is Python, where practicality beats purity, so things like expediency of implementation, frequency of various uses, etc., matter. Right now, without knowing more about Peter's use case, I'd sat that __eq__ should ignore the .ip attribute, but that's more based on trying to get enough people to drop their opposition than on a full understanding of the use case. I do note that if Peter's use case is at all common, reducing the number of classes is a worthy goal, and Python has a bit of a history of preferring a small number of Swiss-army-knife classes over a multitude of simple classes. > In fact, your "sometimes it would be useful if" phrasing indicates that > your 'integer with hex hint' data type would also not be the fundamental > data type, but instead a subclass of int. Which proves nothing, see above. As a matter of fact, my "sometimes it would be useful" was meant as a gentle nudge in the direction of keeping the .ip attribute. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Sep 29 05:44:30 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 Sep 2009 05:44:30 +0200 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: <4AC1829E.50802@v.loewis.de> > Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])`` > as an example and simply assume that 'alpha' takes a string as an > argument and that it's required and that 'beta' is a boolean flag. To > pull everything out you could do:: > > options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) > options_dict = dict(options) > alpha = options_dict.get('-a', options_dict.get('--alpha', '')) > beta = '-b' in options_dict or '--beta' in options_dict > > main(alpha, beta, args) > > Obviously if one of the getopt supporters has a better way of doing > this then please speak up. As Yuvgoog Greenle says, the canonical getopt way is to write alpha = None beta = False options, args = getopt.getopt(sys.argv[1:],"a:b",['alpha=','beta']): for opt, val in options: if arg in ('-a','--alpha'): alpha = val elif arg in ('-b','--beta'): beta = True main(alpha, beta, args) Even though this is many more lines, I prefer it over optparse/argparse: this code has only a single function call, whereas the argparse version has three function calls to remember. The actual processing uses standard Python data structures which I don't need to look up in the documentation. > Now, Steven, can you show how best to do this in argparse? This demonstrates my point: you were able to use getopt right away (even though not in the traditional way), whereas you need to ask for help on using argparse properly. > I am > willing to bet that the total number of lines to do this is not that > much more and does not require you to know to use 'or' or the dict > constructor along with dict.get() in order to keep it compact. See above - getopt users don't care about compactness in the processing. > I can > only imagine what some newbie might try to do in order to be correct > (if they even try). Depends on the background of the newbie. If they come from C, they immediately recognize the way of doing things. Regards, Martin From greg.ewing at canterbury.ac.nz Tue Sep 29 07:20:13 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Sep 2009 17:20:13 +1200 Subject: [Python-Dev] IO module precisions and exception hierarchy In-Reply-To: <4AC1450E.1040207@voidspace.org.uk> References: <4ABF2047.7050501@gmail.com> <4ABFCDED.8080105@gmail.com> <4ABFD9D6.9070602@voidspace.org.uk> <4AC00D2A.6020504@canterbury.ac.nz> <4AC089F7.8030004@voidspace.org.uk> <4AC14482.1040807@canterbury.ac.nz> <4AC1450E.1040207@voidspace.org.uk> Message-ID: <4AC1990D.8060706@canterbury.ac.nz> Michael Foord wrote: > You said that having an exception hierarchy was impossible because the > only information we have is the error number. No, I said that having an exception hierarchy wouldn't help you to sort out the different meanings of a single error code. If you have some *other* way of sorting them out, then obviously you can represent them using different exceptions if you want. -- Greg From steven.bethard at gmail.com Tue Sep 29 07:22:26 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 28 Sep 2009 22:22:26 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC1829E.50802@v.loewis.de> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> Message-ID: On Mon, Sep 28, 2009 at 8:44 PM, "Martin v. L?wis" wrote: >> Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])`` > > As Yuvgoog Greenle says, the canonical getopt way is to write [snip getopt code] > Even though this is many more lines, I prefer it over > optparse/argparse: this code has only a single function call, > whereas the argparse version has three function calls to remember. > The actual processing uses standard Python data structures which > I don't need to look up in the documentation. > >> Now, Steven, can you show how best to do this in argparse? > > This demonstrates my point: you were able to use getopt right away > (even though not in the traditional way), whereas you need to ask > for help on using argparse properly. I don't think this is fair at all. I am totally unable to write getopt code without checking the documentation -- I don't remember the format string syntax, nor what the function returns. But that's just how library modules work -- if you don't know the library, you have to read the documentation. The only reason getopt is easier for you is that you're already familiar with the API from C. That said, I can certainly understand that folks who use getopt in C would have an easy transition to getopt in Python and a harder transition to argparse. You didn't directly answer my question of whether adding an "add_getopt_arguments" function would meet your needs, but I infer from your answer above that it wouldn't because you'd still need to make multiple function calls which is your primary complaint. So at this point, I think it's clear that there's nothing I can reasonably add to argparse to make getopt people more comfortable that isn't just duplicating what's already in getopt. So let's just deprecate optparse (as in the PEP), and leave getopt alone (other than adding a note to the documentation suggesting argparse as an alternative). -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From greg.ewing at canterbury.ac.nz Tue Sep 29 08:11:32 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 29 Sep 2009 18:11:32 +1200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: <4AC1A514.6000000@canterbury.ac.nz> Guido van Rossum wrote: > I've never heard of someone who had a use case for > denormalized fractions, but I don't doubt that Peter has a use case > for denormalized IPNetwork objects. (Do you doubt that Peter has such > a use case? If so, we have a much more fundamental disagreement.) It would be interesting to know what his use case is. Maybe it can be addressed in a different way that doesn't require denormalised network objects. > my "sometimes it > would be useful" was meant as a gentle nudge in the direction of > keeping the .ip attribute. I think the point is that it would have to be useful a *lot* of the time, not just "sometimes", to be worth having it on every IPNetwork object. -- Greg From stephen at xemacs.org Tue Sep 29 11:18:36 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 29 Sep 2009 18:18:36 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: <87bpkukt1v.fsf@uwakimon.sk.tsukuba.ac.jp> Guido van Rossum writes: > I don't doubt that Peter has a use case for denormalized IPNetwork > objects. If you know what it is, would you please describe it, or say "that's proprietary"? Peter hasn't done either, despite repeated requests. > I do note that if Peter's use case is at all common, reducing the > number of classes is a worthy goal, and Python has a bit of a > history of preferring a small number of Swiss-army-knife classes > over a multitude of simple classes. Even if Peter's use case turns out to be at all common, two things bother me a lot. First, IPv4Network(a) == IPv4Network(b) has an obvious preferred interpretation as pseudocode. Second, equality comparison for the "abuse" of a network class to represent host-with-network-info can be implemented cleanly[1] as x = IPv4Network(a) y = IPv4Network(b) (x.ip, x) == (y.ip, y) as you've pointed out. It is self-documenting in that makes it plain that some things that are not "true networks" are being compared. OTOH, x = IPv4Network(a) y = IPv4Network(b) (x.network, x.prefixlen) == (y.network, y.prefixlen) # alternatively # list(x) == list(y) looks like nothing so much as an incomplete implementation to me. It just makes me itch to provide the appropriate definitions of __hash__ and __equal__ for IPv4Network. Do you feel differently? Footnotes: [1] It's clean only if we assume you've accepted the "abuse", of course. From ronaldoussoren at mac.com Tue Sep 29 12:42:13 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 29 Sep 2009 12:42:13 +0200 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk In-Reply-To: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> Message-ID: <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> On 26 Sep, 2009, at 14:46, Barry Scott wrote: > I'm working with http://svn.python.org/projects/python/trunk on Mac > OS X 10.6.1 > using Apples xcode gcc 4.2.1. > > When I run the following commands: > > ./configure --enable-framework --with-universal-archs=32-bit | tee > build.config.log > make clean all | tee build.make.log > > I end up with a x86_64 Python image. > > No matter what I use for archs its always the same. > > I would expect to see -arch arg to GCC but it is not there. > > export CFLAG="-arch i386" > > did not work either. > > Am I doing something wrong or is this broken on trunk? Use: ./configure --enable-framework --enable-universalsdk=/ The --with-universal-archs flag selects whichs architectures should be included when you build a universal binary, defaulting to 32-bit. I'll write some documentation on the build options on OSX, but don't know what's the best location to do so. I guess I should also add logic to configure that bails out when you specifiy --with-universal-archs without asking for a universal build. Ronald > > Barry > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From steve at pearwood.info Tue Sep 29 13:20:03 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 29 Sep 2009 21:20:03 +1000 Subject: [Python-Dev] [OT] Denormalized fractions [was Re: PEP 3144 review.] In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> Message-ID: <200909292120.03677.steve@pearwood.info> On Tue, 29 Sep 2009 01:18:43 pm Guido van Rossum wrote: > I've never heard of someone who had a use case for > denormalized fractions Off-topic, but for what it's worth, I have one -- there's a mathematical operator called the mediant: mediant(a/b, c/d) = (a+c)/(b+d) It has a few uses, including Farey fractions. Clearly the result you get from normalized fractions will be different from denormalized (compare mediant(1/2, 3/4) with mediant(5/10, 3/4)). This leads to Simpson's Paradox, which is of importance in medical research: http://en.wikipedia.org/wiki/Simpson's_paradox Brief summary: consider two medical studies comparing two different treatments for an illness, treatment A and B. According to the first study, treatment A is better; according to the second study, treatment A is also better. But combining the results of the two studies into a single comparison paradoxically shows that treatment B is better! The mediant is fascinating (to maths geeks at least) and important, and you need denormalized fractions. -- Steven D'Aprano From skip at pobox.com Tue Sep 29 13:48:36 2009 From: skip at pobox.com (skip at pobox.com) Date: Tue, 29 Sep 2009 06:48:36 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC1829E.50802@v.loewis.de> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> Message-ID: <19137.62484.513693.107525@montanaro.dyndns.org> Martin> alpha = None Martin> beta = False Martin> options, args = getopt.getopt(sys.argv[1:],"a:b",['alpha=','beta']): Martin> for opt, val in options: ... Martin> Even though this is many more lines, I prefer it over Martin> optparse/argparse: this code has only a single function call, ... Agreed. I have never completely wrapped my brain around optparse. Getopt I just remember. Skip From skip at pobox.com Tue Sep 29 13:56:29 2009 From: skip at pobox.com (skip at pobox.com) Date: Tue, 29 Sep 2009 06:56:29 -0500 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk In-Reply-To: <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> Message-ID: <19137.62957.761923.573377@montanaro.dyndns.org> Ronald> I'll write some documentation on the build options on OSX, but Ronald> don't know what's the best location to do so. The top-level README file of the distribution has a "Platform specific notes" section. Seems like that would be the most logical place to stuff such info. Skip From solipsis at pitrou.net Tue Sep 29 15:23:32 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 29 Sep 2009 13:23:32 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Fuzziness_in_io_module_specs_-_PEP_update?= =?utf-8?q?=09proposition_V2?= References: <4AB3DCC9.5050209@gmail.com> <4AB5E8DD.6050204@canterbury.ac.nz> <4AB5FA81.5060408@gmail.com> <4AB6590E.2070800@gmail.com> <4AB67894.5020903@mrabarnett.plus.com> <4ABF1B3B.4080106@gmail.com> Message-ID: Guido van Rossum python.org> writes: > > All in all I think we should change this before it's too late; it will > affect a very small number of apps (perhaps none?), but I would rather > have the right semantics in the future. Also, it's trivial to write > code that doesn't care (in fact code running under 2.x and 3.x > probably will have to be written so that it doesn't care) so it's not > like changing this is going to make life harder for people wanting > multiple-version support. Ok, it sounds reasonable to me :) Now somebody just has to write a patch (it shouldn't be too difficult, since the position restoring code exists in the 2.x file object). Regards Antoine. From bjourne at gmail.com Tue Sep 29 17:17:25 2009 From: bjourne at gmail.com (=?ISO-8859-1?Q?Bj=F6rn_Lindqvist?=) Date: Tue, 29 Sep 2009 17:17:25 +0200 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> References: <4AB67430.2060306@egenix.com> <4AC07309.1010608@egenix.com> <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> Message-ID: <740c3aec0909290817r78d8c5e3t2823f8c01aa4642a@mail.gmail.com> 2009/9/28 James Y Knight : >> People building their own Python version will usually also build >> their own extensions, so I don't really believe that the above >> scenario is very common. > > I'd just like to note that I've run into this trap multiple times. I built a > custom python, and expected it to work with all the existing, installed, > extensions (same major version as the system install, just patched). And > then had to build it again with UCS4, for it to actually work. Of course > building twice isn't the end of the world, and I'm certainly used to having > to twiddle build options on software to get it working, but, this *does* > happen, and *is* a tiny bit irritating. I've also encountered this trap multiple times. Obviously, the problem is not rebuilding Python which is quick, but to figure out the correct configure option to use (--enable-unicode=ucs4). Others have also spent some time scratching their heads over the strange PyUnicodeUCS4_FromUnicode error the misconfiguration results in, as Zooko's links show. If Python can't infer the unicode setting from the width of the platforms wchar_t, then perhaps it should be mandatory to specify to configure whether you want UCS2 or UCS4? For someone clueless like me, it would be easier to deal with the problem upfront than (much) further down the line. Explicit being better than implicit and all that. -- mvh Bj?rn From mal at egenix.com Tue Sep 29 17:28:59 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 29 Sep 2009 17:28:59 +0200 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk In-Reply-To: <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> Message-ID: <4AC227BB.9000607@egenix.com> Ronald Oussoren wrote: > > On 26 Sep, 2009, at 14:46, Barry Scott wrote: > >> I'm working with http://svn.python.org/projects/python/trunk on Mac OS >> X 10.6.1 >> using Apples xcode gcc 4.2.1. >> >> When I run the following commands: >> >> ./configure --enable-framework --with-universal-archs=32-bit | tee >> build.config.log >> make clean all | tee build.make.log >> >> I end up with a x86_64 Python image. >> >> No matter what I use for archs its always the same. >> >> I would expect to see -arch arg to GCC but it is not there. >> >> export CFLAG="-arch i386" >> >> did not work either. >> >> Am I doing something wrong or is this broken on trunk? > > Use: > > ./configure --enable-framework --enable-universalsdk=/ > > The --with-universal-archs flag selects whichs architectures should be > included when you build a universal binary, defaulting to 32-bit. The Python default on 10.6 is 64-bit, so wouldn't it be better to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?! > I'll write some documentation on the build options on OSX, but don't > know what's the best location to do so. Please put that information into Mac/README which already has documentation for how to build universal binaries on Mac. > I guess I should also add logic to configure that bails out when you > specifiy --with-universal-archs without asking for a universal build. That would be useful. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 29 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From solipsis at pitrou.net Tue Sep 29 17:33:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 29 Sep 2009 15:33:19 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?please_consider_changing_--enable-unicode_?= =?utf-8?q?default=09to_ucs4?= References: <4AB67430.2060306@egenix.com> <4AC07309.1010608@egenix.com> <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> <740c3aec0909290817r78d8c5e3t2823f8c01aa4642a@mail.gmail.com> Message-ID: Hello, > I've also encountered this trap multiple times. Obviously, the problem > is not rebuilding Python which is quick, but to figure out the correct > configure option to use (--enable-unicode=ucs4). Others have also > spent some time scratching their heads over the strange > PyUnicodeUCS4_FromUnicode error the misconfiguration results in, as > Zooko's links show. Isn't this overrated? First, if you have a Python version that has the wrong version, just print out its sys.maxunicode and choose the right version according to that (if sys.maxunicode == 65535, you need to compile an UCS-4 version, otherwise an UCS-2 version). Second, once you have encountered this issue, you know what you need the subsequent times. There are only two possibilities after all. > If Python can't infer the unicode setting from the width of the > platforms wchar_t, then perhaps it should be mandatory to specify to > configure whether you want UCS2 or UCS4? For someone clueless like me, > it would be easier to deal with the problem upfront than (much) > further down the line. I'm not sure why someone "clueless" (your word :-)) wants to compile his own Python, though. Regards Antoine. From ronaldoussoren at mac.com Tue Sep 29 17:44:40 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 29 Sep 2009 17:44:40 +0200 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk In-Reply-To: <4AC227BB.9000607@egenix.com> References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> <4AC227BB.9000607@egenix.com> Message-ID: <9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com> On 29 Sep, 2009, at 17:28, M.-A. Lemburg wrote: > > Ronald Oussoren wrote: >> >> On 26 Sep, 2009, at 14:46, Barry Scott wrote: >> >>> I'm working with http://svn.python.org/projects/python/trunk on >>> Mac OS >>> X 10.6.1 >>> using Apples xcode gcc 4.2.1. >>> >>> When I run the following commands: >>> >>> ./configure --enable-framework --with-universal-archs=32-bit | >>> tee >>> build.config.log >>> make clean all | tee build.make.log >>> >>> I end up with a x86_64 Python image. >>> >>> No matter what I use for archs its always the same. >>> >>> I would expect to see -arch arg to GCC but it is not there. >>> >>> export CFLAG="-arch i386" >>> >>> did not work either. >>> >>> Am I doing something wrong or is this broken on trunk? >> >> Use: >> >> ./configure --enable-framework --enable-universalsdk=/ >> >> The --with-universal-archs flag selects whichs architectures should >> be >> included when you build a universal binary, defaulting to 32-bit. > > The Python default on 10.6 is 64-bit, so wouldn't it be better > to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?! Defaulting to a 32-bit build has several advantages. The first is that the defaults match the binary installer on the python.org website, and secondly there are still 3th-party libraries that don't work in 64-bit mode (mostly GUI libraries, until recently Tk and wxWidgets wrapped the Carbon libraries which are not available in 64-bit mode; AFAIK both have betas that wrap the Cocoa libraries instead). To mimick the system default you'd have to default to 32-bit on 10.4, 4-way universal on 10.5 and 3-way universal on 10.6, and that is without considering deployment targets. All of those are available as options, I'd prefer to keep it this way for now to keep things simple. > >> I'll write some documentation on the build options on OSX, but don't >> know what's the best location to do so. > > Please put that information into Mac/README which already has > documentation for how to build universal binaries on Mac. I know that, I wrote most of that file ;-). > >> I guess I should also add logic to configure that bails out when you >> specifiy --with-universal-archs without asking for a universal build. > > That would be useful. I've committed a patch to the trunk and 2.6 that does this. Ronald > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Services directly from the Source (#1, Sep 29 > 2009) >>>> Python/Zope Consulting and Support ... http:// >>>> www.egenix.com/ >>>> mxODBC.Zope.Database.Adapter ... http:// >>>> zope.egenix.com/ >>>> mxODBC, mxDateTime, mxTextTools ... http:// >>>> python.egenix.com/ > ________________________________________________________________________ > > ::: Try our new mxODBC.Connect Python Database Interface for > free ! :::: > > > eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 > D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg > Registered at Amtsgericht Duesseldorf: HRB 46611 > http://www.egenix.com/company/contact/ -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From solipsis at pitrou.net Tue Sep 29 18:03:16 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 29 Sep 2009 18:03:16 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> Message-ID: <1254240196.5662.5.camel@localhost> > I've mentioned at least once (and others have mentioned as well) that > it's very common, when describing the ip address/prefix of a NIC to > write, "192.168.1.100/24" You say it yourself : it describes "the ip address/prefix of a NIC". It isn't the job of a Network class. A Network shouldn't describe a host, or a particular NIC. Regards Antoine. From peter at hda3.com Tue Sep 29 17:36:02 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 29 Sep 2009 08:36:02 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> Message-ID: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> Responding to a few points here. David and I were discussing these points off-list, I didn't mean to ignore. re: assumed disconnect between the statement "Addresses and Networks are distinct" and the implementation. I don't actually see a disconnect. it seems analogous to stating lists and ints are distinct yet lists can still contain ints. networks and addresses are distinct and yet networks contain addresses. re: denormalized networks: I've mentioned at least once (and others have mentioned as well) that it's very common, when describing the ip address/prefix of a NIC to write, "192.168.1.100/24" re: new comparison methods this is interesting. I would be open to doing this and making __lt__, __gt__, __eq__ and friends do the non-ip comparing by default. would this be sufficient or does the fact that the network has the .ip attribute break something pythonic? Cheers, /peter On Mon, Sep 28, 2009 at 1:43 PM, Guido van Rossum wrote: > On Mon, Sep 28, 2009 at 1:36 PM, R. David Murray wrote: >> On Mon, 28 Sep 2009 at 22:11, "Martin v. L?wis" wrote: >>>> >>>> Martin v. L?wis v.loewis.de> writes: >>>>>> >>>>>> Could you explain what benefit there is for allowing the user to create >>>>>> network objects that don't represent networks? Is there a use-case >>>>>> where these networks-that-aren't-networks are something other than a >>>>>> typo? Under what circumstances would I want to specify a network as >>>>>> 192.168.1.1/24 instead of 192.168.1.0/24? >>>>>> >>>> [...] >>>>> >>>>> So Python code has to make the computation, and it seems most natural >>>>> that the IP library is the piece of code that is able to compute a >>>>> network out of that input. >>>> >>>> The thing is, it doesn't create a network, it creates a hybrid "network >>>> plus >>>> host" which retains knowledge about the original host (that is, >>>> 192.168.1.1 >>>> rather than simply 192.168.1.0, if you enter "192.168.1.1/24"). >>>> >>>> That's what the OP meant with "networks-that-aren't-networks", and that's >>>> what >>>> people are objecting to. >>> >>> That's not the question that was asked, though - the question asked >>> was "Under what circumstances would I want to specify...". I hope >>> most people agree that it is desirable to be able to specify a network >>> not just by its network address. >> >> But then it's not a network, it is an ipaddress-plus-mask. ?It is exactly >> that conflation that we are objecting to. ?There is no question about >> the use case of _specifying_ a network ip plus a netmask and deriving >> a network object from that. ?That is unquestionably required by any >> useful API. ?The argument is about whether the object returned is a >> Network object, or a hybrid object representing _both_ an IP address >> and a network. ?It is the latter, which ipaddr does, which many of us >> find problematic and likely to lead to hard to read programs that will >> probably produce maintenance errors. >> >> I observe that this line in the current PEP rationale: >> >> ? ?IP addresses and IP networks are distinct. >> >> is not in fact achieved by the reference implementation. ?Peter, however, >> clearly thinks it is, since it is listed as a design goal of ipaddr. >> This is a language disconnect I don't understand, which I think has >> been the source of a lot of the difficulties in this thread. > > I say the case has been argued extensively, let's wait for Peter to respond. > > I would say that there certainly are precedents in other areas for > keeping the information about the input form around. For example, > occasionally it would be handy if parsing a hex integer returned an > object that was compatible with other integers but somehow kept a hint > that would cause printing it to use hex by default. > > I see keeping around the IP address used to create a network object as > a similar edge case. I would probably define the __eq__ method to > implement equivalency (ignoring the form of the input), just as I > would in the case of the (hypothetical) hex integers. If you wanted to > do a comparison that includes the input IP address, you could use (a, > a.ip) == (b, b.ip). > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > From mal at egenix.com Tue Sep 29 18:17:15 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 29 Sep 2009 18:17:15 +0200 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk In-Reply-To: <9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com> References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> <4AC227BB.9000607@egenix.com> <9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com> Message-ID: <4AC2330B.1000405@egenix.com> Ronald Oussoren wrote: > >>> Use: >>> >>> ./configure --enable-framework --enable-universalsdk=/ >>> >>> The --with-universal-archs flag selects whichs architectures should be >>> included when you build a universal binary, defaulting to 32-bit. >> >> The Python default on 10.6 is 64-bit, so wouldn't it be better >> to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?! > > Defaulting to a 32-bit build has several advantages. The first is that > the defaults match the binary installer on the python.org website, What build options does that installer use ? (the web-page doesn't say) > and > secondly there are still 3th-party libraries that don't work in 64-bit > mode (mostly GUI libraries, until recently Tk and wxWidgets wrapped the > Carbon libraries which are not available in 64-bit mode; AFAIK both have > betas that wrap the Cocoa libraries instead). > > To mimick the system default you'd have to default to 32-bit on 10.4, > 4-way universal on 10.5 and 3-way universal on 10.6, and that is without > considering deployment targets. All of those are available as options, > I'd prefer to keep it this way for now to keep things simple. Hmm, so I guess the only way to support them all is by building extensions using 4-way universal on 10.5. No wonder they are called "fat" binaries ;-) >>> I'll write some documentation on the build options on OSX, but don't >>> know what's the best location to do so. >> >> Please put that information into Mac/README which already has >> documentation for how to build universal binaries on Mac. > > I know that, I wrote most of that file ;-). Which is why I was surprised you asked :-) >>> I guess I should also add logic to configure that bails out when you >>> specifiy --with-universal-archs without asking for a universal build. >> >> That would be useful. > > I've committed a patch to the trunk and 2.6 that does this. Thanks. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 29 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ronaldoussoren at mac.com Tue Sep 29 18:22:25 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 29 Sep 2009 18:22:25 +0200 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk In-Reply-To: <4AC2330B.1000405@egenix.com> References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> <4AC227BB.9000607@egenix.com> <9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com> <4AC2330B.1000405@egenix.com> Message-ID: <15E9DA97-B2CF-4F01-8375-F2169E598CB6@mac.com> On 29 Sep, 2009, at 18:17, M.-A. Lemburg wrote: > Ronald Oussoren wrote: >> >>>> Use: >>>> >>>> ./configure --enable-framework --enable-universalsdk=/ >>>> >>>> The --with-universal-archs flag selects whichs architectures >>>> should be >>>> included when you build a universal binary, defaulting to 32-bit. >>> >>> The Python default on 10.6 is 64-bit, so wouldn't it be better >>> to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?! >> >> Defaulting to a 32-bit build has several advantages. The first is >> that >> the defaults match the binary installer on the python.org website, > > What build options does that installer use ? (the web-page doesn't > say) The installer is build using the script in Mac/BuildScript, and uses -- enable-framework --with-universalsk. This creates a 32-bit fat build that runs on 10.3.9 or later. > >> and >> secondly there are still 3th-party libraries that don't work in 64- >> bit >> mode (mostly GUI libraries, until recently Tk and wxWidgets wrapped >> the >> Carbon libraries which are not available in 64-bit mode; AFAIK both >> have >> betas that wrap the Cocoa libraries instead). >> >> To mimick the system default you'd have to default to 32-bit on 10.4, >> 4-way universal on 10.5 and 3-way universal on 10.6, and that is >> without >> considering deployment targets. All of those are available as >> options, >> I'd prefer to keep it this way for now to keep things simple. > > Hmm, so I guess the only way to support them all is by building > extensions > using 4-way universal on 10.5. No wonder they are called "fat" > binaries ;-) I like the technology though, much more convenient than having parallel directory tries as on Linux. > >>>> I'll write some documentation on the build options on OSX, but >>>> don't >>>> know what's the best location to do so. >>> >>> Please put that information into Mac/README which already has >>> documentation for how to build universal binaries on Mac. >> >> I know that, I wrote most of that file ;-). > > Which is why I was surprised you asked :-) I hoped to find a document on docs.python.org that explains how to install Python, but sadly enough there isn't. Mac/README contains the right information, but isn't easily found if you're searching on the web or even if you're looking for documentation in the source tree. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From guido at python.org Tue Sep 29 19:00:39 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Sep 2009 10:00:39 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <1254240196.5662.5.camel@localhost> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <1254240196.5662.5.camel@localhost> Message-ID: On Tue, Sep 29, 2009 at 9:03 AM, Antoine Pitrou wrote: > >> I've mentioned at least once (and others have mentioned as well) that >> it's very common, when describing the ip address/prefix of a NIC to >> write, "192.168.1.100/24" > > You say it yourself : it describes "the ip address/prefix of a NIC". > It isn't the job of a Network class. A Network shouldn't describe a > host, or a particular NIC. Hey Antoine, Can we drop the pedantic discussion about what "should" or "shouldn't" be the job of a "Network" class, and just proceed to a pragmatic compromise? Peter has already posted that he is okay with __eq__ and friends ignoring the .ip attribute, which sounds good enough to me. His use case (which he mentioned to me off-list) is simply that if the denormalized .ip attribute weren't saved as part of the IPNetwork class, in many cases he'd have to keep track of it separately, which just feels clumsy. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From zookog at gmail.com Tue Sep 29 19:03:25 2009 From: zookog at gmail.com (Zooko O'Whielacronx) Date: Tue, 29 Sep 2009 11:03:25 -0600 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: <4AC07309.1010608@egenix.com> References: <4AB67430.2060306@egenix.com> <4AC07309.1010608@egenix.com> Message-ID: Dear MAL and python-dev: I failed to explain the problem that users are having. I will try again, and this time I will omit my ideas about how to improve things and just focus on describing the problem. Some users are having trouble using Python packages containing binary extensions on Linux. I want to provide such binary Python packages for Linux for the pycryptopp project (http://allmydata.org/trac/pycryptopp ) and the zfec project (http://allmydata.org/trac/zfec ). I also want to make it possible for users to install the Tahoe-LAFS project (http://allmydata.org ) without having a compiler or Python header files. (You'd be surprised at how often Tahoe-LAFS users try to do this on Linux. Linux is no longer only for people who have the knowledge and patience to compile software themselves.) Tahoe-LAFS also depends on many packages that are maintained by other people and are not packaged or distributed by me -- pyOpenSSL, simplejson, etc.. There have been several hurdles in the way that we've overcome, and no doubt there will be more, but the current hurdle is that there are two "formats" for Python extension modules that are used on Linux -- UCS2 and UCS4. If a user gets a Python package containing a compiled extension module which was built for the wrong UCS2/4 setting, he will get mysterious (to him) "undefined symbol" errors at import time. On Mon, Sep 28, 2009 at 2:25 AM, M.-A. Lemburg wrote: > > The Python default is UCS2 for a good reason: it's a good trade-off > between memory consumption, functionality and performance. I'm sure you are right about this. At some point I will try to measure the performance implications in the context of our application. I don't think it will be an issue for us, as so far no users have complained about any performance or functionality problems that were traceable to the choice of UCS2/4. > As already mentioned, I also don't understand how the changing > the Python default on Linux would help your users in any way - > if you let distutils compile your extensions, it's automatically > going to use the right Unicode setting for you (as well as your > users). My users are using some Python packages built by me and some built by others. The binary packages they get from others could have the incompatible UCS2/4 setting. Also some of my users might be using a python configured with the opposite setting of the python interpreter that I use to build packages. > Unfortunately, this automatic support doesn't help you when > shipping e.g. setuptools eggs, but this is a tool problem, > not one of Python: setuptools completely ignores the fact > that there are two ways to build Python. This is the setuptools/distribute issue that I mentioned: http://bugs.python.org/setuptools/issue78 . If that issue were solved then if a user tried to install a specific package, for example with a command-line like "easy_install http://allmydata.org/source/tahoe/deps/tahoe-dep-eggs/pyOpenSSL-0.8-py2.5-linux-i686.egg", then instead of getting an undefined symbol error at import time, they would get an error message to the effect of "This package is not compatible with your Python interpreter." at install time. That would be good because it would be less confusing to the users. However, if they were using the default setuptools/distribute dependency-satisfaction feature, e.g. because they are installing a package and that package is marked as "install_requires=['pyOpenSSL']", then setuptools/distribute would do its fallback behavior in which it attempts to compile the package from source when it can't find a compatible binary package. This would probably confuse the users at least as much as the undefined symbol error currently does. In any case, improving the tools to handle incompatible packages nicely would not make more packages compatible. Let's do both! Improve tools to handle incompatible packages nicely, and encourage everyone who compiles python on Linux to use the same UCS2/4 setting. Thank you for your attention. Regards, Zooko From glyph at twistedmatrix.com Tue Sep 29 20:19:10 2009 From: glyph at twistedmatrix.com (Glyph Lefkowitz) Date: Tue, 29 Sep 2009 14:19:10 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <1254240196.5662.5.camel@localhost> Message-ID: On Tue, Sep 29, 2009 at 1:00 PM, Guido van Rossum wrote: > On Tue, Sep 29, 2009 at 9:03 AM, Antoine Pitrou > wrote: > > You say it yourself : it describes "the ip address/prefix of a NIC". > > It isn't the job of a Network class. A Network shouldn't describe a > > host, or a particular NIC. > > Hey Antoine, > > Can we drop the pedantic discussion about what "should" or "shouldn't" > be the job of a "Network" class, and just proceed to a pragmatic > compromise? Peter has already posted that he is okay with __eq__ and > friends ignoring the .ip attribute, which sounds good enough to me. > His use case (which he mentioned to me off-list) is simply that if the > denormalized .ip attribute weren't saved as part of the IPNetwork > class, in many cases he'd have to keep track of it separately, which > just feels clumsy. > I apologize in advance for missing a message that answers my question; I've done my best to read all the related traffic in the various threads that discuss this, but I'm sure I missed something. I don't see what's particularly "pragmatic", in terms of functionality, about confusing the responsibility of the Network class. Networks don't have addresses, in the sense that is being discussed here. Allowing them to have an IP presents a misleading model, and will encourage applications to be passing around networks where they should be passing around hosts. And yes, the discussion is pedantic, in that some people are certain to learn about the model of IP networking by reading the documentation of this module if it gets into the stdlib. I personally learned all about async networking from reading the asyncore, select, and socket modules in python 1.5.2, lo these many years past. The discussion seems to be centered around the inconvenience of adding a separate IPNetworkWithHost class that can encapsulate both of these things. Peter seems to think that this is hugely inconvenient, but classes are cheap. If we were talking about IPNetwork.from_string() instead of IPNetwork(), it seems to me that it would even be acceptable for it to return a IPNetwork subclass if the address were not canonical (i.e. without the bits already masked off and zeroed). Perhaps there should be such a method, or even just a free function, parse_mask(), as that would allow for dealing with other user-input use-cases that have been brought up in this thread. I don't understand why we can't just add that class and make everybody happy. IPNetwork could even have a .canonicalize() method which would return itself, and the subclass implementation in IPNetworkWithHost return the corresponding IPNetwork. (I wish I could come up with a better name, but I certainly agree that there are cases where a IPNetworkWithHost is what I would want.) In addition to the somebody-must-have-mentioned-this-already feeling that I got, I hesitated to post this message because it doesn't actually seem that important to me. While I'm firmly convinced that Network.ip is a design mistake, it's not like the rest of Python, or for that matter any software, is completely perfect. In fact I think this mistake is significantly less bad than some of the others already present in Python. If Peter remains unconvinced, I do think that we should put it in the stdlib, move on, and get to fixing some of the other stuff we agree needs fixing rather than continuing to re-hash this. Primarily because, as far as I can tell, if hashing and equality are defined the way that everyone seems to be agreeing they be defined (ignoring the .ip attribute) then those of us who think .ip is a design error can use the library and safely ignore it completely. So, I promise not to contribute further to the problem; I won't post again in this thread against someone who is actually going to do some work here wants to solicit a clarification of my opinion or some more ideas :). -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Tue Sep 29 20:30:56 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 30 Sep 2009 03:30:56 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> Message-ID: <873a65li1r.fsf@uwakimon.sk.tsukuba.ac.jp> Let me first say that the module is overall quite nice; the implementation seems concise and very efficient, and the API is well-balanced. I'm sorry there's been such controversy (and for my part in it, as I managed to completely misunderstand what you absolutely require), when AFAICT the only "line in the sand" is being drawn around the semantics of "net1 == net2". Peter Moody writes: > I don't actually see a disconnect. it seems analogous to stating > lists and ints are distinct yet lists can still contain ints. networks > and addresses are distinct and yet networks contain addresses. AIUI, the disconnect is that lists do not contain a distinguished element such that if that element differs, the lists differ. Ie, the disconnect is the very presence of the .ip attribute in IPv?Network. ISTM that all of the vehement opposition is related to this single feature. > re: denormalized networks: > > I've mentioned at least once (and others have mentioned as well) that > it's very common, when describing the ip address/prefix of a NIC to > write, "192.168.1.100/24" Please be careful to distinguish between "denormalized descriptions of networks" (strings like your example above) and "denormalized network objects" (ie, objects where net.ip != net.network). I'm happy with the former, with a slight preference for a separate method to do the parsing, as in Antoine's proposal. The latter makes me "unhappy", but I'll deal with that simply by refusing to use that feature (see below). > re: new comparison methods > > this is interesting. I would be open to doing this and making __lt__, > __gt__, __eq__ and friends do the non-ip comparing by default. would > this be sufficient or does the fact that the network has the .ip > attribute break something pythonic? What do you mean, "by default"? Surely you don't mean an equality method with some sort of flag!? Personally I want "net1 == net2" to have the same value as "(net1.network, net1.prefixlen) == (net2.network, net2.prefixlen)" when net1 and net2 are IPv?Networks, period. For me, this is the only sticking point; if it's not true, I'll never use this module, and I'd oppose adding it to stdlib. I don't care if the .ip attribute *exists* (and I don't care if you provide separate .denormalized_compare() methods) but I find the presence of .ip in IPv?Network objects counterintuitive. Enough so that if I need such an object, I will create a new class (with a name that expresses its purpose to me) to manage it in my app, if such a class isn't provided by ipaddr.py. That would be no great burden to me since AFAICS apps that need such behavior are quite specialized. In any case, it seems that many developers (you, GvR, and MvL, at least) aren't bothered by overloading the IPv?Network class as you do in ipaddr.py. (In fact, IIUC you don't even see using it for denormalized networks as overloading.) I see no point in getting in your way. From peter at hda3.com Tue Sep 29 20:28:07 2009 From: peter at hda3.com (Peter Moody) Date: Tue, 29 Sep 2009 11:28:07 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <873a65li1r.fsf@uwakimon.sk.tsukuba.ac.jp> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <873a65li1r.fsf@uwakimon.sk.tsukuba.ac.jp> Message-ID: <8517e9350909291128s2a4ce554qecaa8f8fb1b7488e@mail.gmail.com> On Tue, Sep 29, 2009 at 11:30 AM, Stephen J. Turnbull wrote: > Let me first say that the module is overall quite nice; the > implementation seems concise and very efficient, and the API is > well-balanced. ?I'm sorry there's been such controversy (and for my > part in it, as I managed to completely misunderstand what you > absolutely require), when AFAICT the only "line in the sand" is being > drawn around the semantics of "net1 == net2". > > Peter Moody writes: > > ?> I don't actually see a disconnect. ?it seems analogous to stating > ?> lists and ints are distinct yet lists can still contain ints. networks > ?> and addresses are distinct and yet networks contain addresses. > > AIUI, the disconnect is that lists do not contain a distinguished > element such that if that element differs, the lists differ. ?Ie, the > disconnect is the very presence of the .ip attribute in IPv?Network. > ISTM that all of the vehement opposition is related to this single > feature. > > ?> ? re: denormalized networks: > ?> > ?> I've mentioned at least once (and others have mentioned as well) that > ?> it's very common, when describing the ip address/prefix of a NIC to > ?> write, "192.168.1.100/24" > > Please be careful to distinguish between "denormalized descriptions of > networks" (strings like your example above) and "denormalized network > objects" (ie, objects where net.ip != net.network). ?I'm happy with > the former, with a slight preference for a separate method to do the > parsing, as in Antoine's proposal. ?The latter makes me "unhappy", but > I'll deal with that simply by refusing to use that feature (see below). > > ?> ? re: new comparison methods > ?> > ?> this is interesting. I would be open to doing this and making __lt__, > ?> __gt__, __eq__ and friends do the non-ip comparing by default. would > ?> this be sufficient or does the fact that the network has the .ip > ?> attribute break something pythonic? > > What do you mean, "by default"? ?Surely you don't mean an equality > method with some sort of flag!? no, I mean setting it up so that if I want to compare two network objects and take into account their .ip attribute, that I use a .compare_with_extra_bits() method or something. __eq__, __ne__, __lt__, __le__, __gt__, __ge__ (any more?) would all only compare .network and .prefixlen (as you have in your example). Cheers, /peter > Personally I want "net1 == net2" to have the same value as > "(net1.network, net1.prefixlen) == (net2.network, net2.prefixlen)" > when net1 and net2 are IPv?Networks, period. ?For me, this is the only > sticking point; if it's not true, I'll never use this module, and I'd > oppose adding it to stdlib. > > I don't care if the .ip attribute *exists* (and I don't care if you > provide separate .denormalized_compare() methods) but I find the > presence of .ip in IPv?Network objects counterintuitive. ?Enough so > that if I need such an object, I will create a new class (with a name > that expresses its purpose to me) to manage it in my app, if such a > class isn't provided by ipaddr.py. > > That would be no great burden to me since AFAICS apps that need such > behavior are quite specialized. ?In any case, it seems that many > developers (you, GvR, and MvL, at least) aren't bothered by > overloading the IPv?Network class as you do in ipaddr.py. ?(In fact, > IIUC you don't even see using it for denormalized networks as > overloading.) ?I see no point in getting in your way. > > From martin at v.loewis.de Tue Sep 29 20:36:42 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 29 Sep 2009 20:36:42 +0200 Subject: [Python-Dev] please consider changing --enable-unicode default to ucs4 In-Reply-To: References: <4AB67430.2060306@egenix.com> <4AC07309.1010608@egenix.com> <4DC621A3-7815-4575-B788-8597F6429A4B@fuhm.net> <740c3aec0909290817r78d8c5e3t2823f8c01aa4642a@mail.gmail.com> Message-ID: <4AC253BA.3040204@v.loewis.de> >> I've also encountered this trap multiple times. Obviously, the problem >> is not rebuilding Python which is quick, but to figure out the correct >> configure option to use (--enable-unicode=ucs4). Others have also >> spent some time scratching their heads over the strange >> PyUnicodeUCS4_FromUnicode error the misconfiguration results in, as >> Zooko's links show. > > Isn't this overrated? > > First, if you have a Python version that has the wrong version, just print out > its sys.maxunicode and choose the right version according to that (if > sys.maxunicode == 65535, you need to compile an UCS-4 version, otherwise an > UCS-2 version). To do so, you have to know that there is such a configuration option in the first place, and that the error message you get (missing symbols) has anything to do with it. This is quite puzzling to users. Once people know what the problem is, fixing it is indeed easy. > I'm not sure why someone "clueless" (your word :-)) wants to compile his own > Python, though. People install a lot of software that they don't understand. In fact, most people who ever install software don't know how it is written, and cannot enumerate the meaning of all configuration options that the software possesses. In Unix, there is a long tradition that "installing software" means "building from source"; if you have a configure script, you expect that it either works out of the box, or gives an error message if it finds that something is wrong with the environment. So it is quite normal that people who don't understand how the Python interpreter works (or that it has a Unicode type) install Python. Regards, Martin From brett at python.org Tue Sep 29 08:34:24 2009 From: brett at python.org (Brett Cannon) Date: Mon, 28 Sep 2009 23:34:24 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC1829E.50802@v.loewis.de> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> Message-ID: On Mon, Sep 28, 2009 at 20:44, "Martin v. L?wis" wrote: >> Let's take ``getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"])`` >> as an example and simply assume that 'alpha' takes a string as an >> argument and that it's required and that 'beta' is a boolean flag. To >> pull everything out you could do:: >> >> ? options, args = getopt.getopt(sys.argv[1:], "a:b", ["alpha=", "beta"]) >> ? options_dict = dict(options) >> ? alpha = options_dict.get('-a', options_dict.get('--alpha', '')) >> ? beta = '-b' in options_dict or '--beta' in options_dict >> >> ? main(alpha, beta, args) >> >> Obviously if one of the getopt supporters has a better way of doing >> this then please speak up. > > As Yuvgoog Greenle says, the canonical getopt way is to write > > alpha = None > beta = False > options, args = getopt.getopt(sys.argv[1:],"a:b",['alpha=','beta']): > for opt, val in options: > ? ?if arg in ('-a','--alpha'): > ? ? ? ?alpha = val > ? ?elif arg in ('-b','--beta'): > ? ? ? ?beta = True > main(alpha, beta, args) > > Even though this is many more lines, I prefer it over > optparse/argparse: this code has only a single function call, > whereas the argparse version has three function calls to remember. > The actual processing uses standard Python data structures which > I don't need to look up in the documentation. > >> Now, Steven, can you show how best to do this in argparse? > > This demonstrates my point: you were able to use getopt right away > (even though not in the traditional way), whereas you need to ask > for help on using argparse properly. > Actually, I had to read the docs for getopt. And I chose to not even try argparse when the creator of the module is cc'ed on the email and can obviously do a better example using his own code then I could. -Brett From barry at python.org Tue Sep 29 21:10:56 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Sep 2009 15:10:56 -0400 Subject: [Python-Dev] bug 3890 Message-ID: <7FE4E7BF-9434-4B60-90F4-252A5B16D910@python.org> I'd like to include the patch for bug 3890 in Python 2.6.3. There is a patch by Amaury that has gotten the blessing of Bill Janssen, however the patch has no test. Is there anybody who can come up with a test in the next few hours so we can get this into 2.6.3rc1? If not, is the fix important enough to include without the test? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From rdmurray at bitdance.com Tue Sep 29 21:24:38 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 29 Sep 2009 15:24:38 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <1254240196.5662.5.camel@localhost> Message-ID: On Tue, 29 Sep 2009 at 14:19, Glyph Lefkowitz wrote: > In addition to the somebody-must-have-mentioned-this-already feeling that I > got, I hesitated to post this message because it doesn't actually seem that > important to me. While I'm firmly convinced that Network.ip is a design > mistake, it's not like the rest of Python, or for that matter any software, > is completely perfect. In fact I think this mistake is significantly less > bad than some of the others already present in Python. If Peter remains > unconvinced, I do think that we should put it in the stdlib, move on, and > get to fixing some of the other stuff we agree needs fixing rather than > continuing to re-hash this. Primarily because, as far as I can tell, if > hashing and equality are defined the way that everyone seems to be agreeing > they be defined (ignoring the .ip attribute) then those of us who think .ip > is a design error can use the library and safely ignore it completely. I think Glyph is petty much on target here. I think I dislike the .ip attribute more than he does, but I'm willing to let go of it now that the equality issue has been addressed so that the class acts like a Network object _except_ for the ip attribute. I do want to show why I think the ip attribute is a bad idea, using a concrete example, rather than the hand waving we've done up to this point. This is a stripped down example of something that could be a real application. I started writing this to show what the difference between using the ipaddr classes versus the completely separate address and network classes would look like, but ended up with something where there would be very little difference in the final program. Given this simple configuration file: demo.ini -------------------------------------------------------------- [gatewayrouter] inside = e1/0 192.168.1.1/24 outside = e1/1 172.16.200.1/23 dmz = e1/2 192.168.2.1/24 [dmzhosts] webserver = 172.16.200.2 192.168.2.30 22 80 442 mailserver = 172.16.200.3 182.168.2.31 22 25 587 -------------------------------------------------------------- Here is a (runable) program to generate the skeleton of a Cisco router configuration file from that configuration (I've left out a bunch of additinoal Cisco configuration I'd include if this were a real application): -------------------------------------------------------------- import ipaddr import ConfigParser config = ConfigParser.SafeConfigParser() config.read('demo.ini') output = open('gatewayrouter.config', 'w') class InterfaceData(object): def __init__(self, interfacespec): self.interfacename, ipwithnetmask = interfacespec.split() self.network = ipaddr.IPv4Network(ipwithnetmask) self.ip = self.network.ip self.netmask = self.network.netmask class DMZHost(object): def __init__(self, hostdata): data = hostdata.split() outsideip, insideip = data[0:2] self.ports = data[2:] self.outsideip = ipaddr.IPv4Address(outsideip) self.insideip = ipaddr.IPv4Address(insideip) interfaces = {} for name in ('inside', 'outside', 'dmz'): interfacedata = InterfaceData(config.get('gatewayrouter', name)) interfaces[name] = interfacedata globals()[name] = interfacedata.network dmzhosts = {} for host, hostdata in config.items('dmzhosts'): dmzhosts[host] = DMZHost(hostdata) for idata in interfaces.values(): print >>output, 'interface {}'.format(idata.interfacename) print >>output, 'ip address {} {}'.format(idata.ip, idata.netmask) print >>output, 'ip extended access-list outside_in' for host in dmzhosts.values(): for port in host.ports: print >>output, 'permit ip any host {} eq {}'.format( host.outsideip, port) for icmp in ('echo', 'echo-reply', 'host-unreachable', 'time-exceeded'): print >>output, 'permit icmp any host {} {}'.format( interfaces['inside'].ip, icmp) print >>output, 'permit icmp any {} {} {}'.format( dmz.network, dmz.hostmask, icmp) print >>output, 'ip extended access-list nat' print >>output, 'permit ip {} {} any'.format(inside.network, inside.hostmask) print >>output, 'permit ip {} {} any'.format(dmz.network, dmz.hostmask) print >>output, ('ip nat inside source access-list nat ' 'interface {} overload').format(interfaces['outside'].interfacename) for host in dmzhosts.values(): print >>output, 'ip nat inside source static {} {}'.format( host.insideip, host.outsideip) -------------------------------------------------------------- There are several things to note about this code. The first is that I wanted my own wrapper classes to deal with the data, IPv4Network/Address by themselves weren't enough, nor would an IPWithNetmask class have been. I also notice that I immediately pulled out the 'ip' address from the IPv4Network object to a top level attribute in my wrapper, so for me a ParseIPWithMask that returned separate IPAddress and an IPNetwork classes would have been more natural. On the other hand, I also pulled out the 'netmask', which I would still have needed to do. There's one place in this code where the inclusion of the 'ip' information in the IPNetwork class could have been used. In the line that allows ICMP traffic to the router's outside port, I could have written 'inside.ip' instead of interfaces['inside'].ip. I think that would have confused me when I came back to read the code later, though, since 'inside' is otherwise a network object. But the real 'ah ha' moment in writing this program came when I wrote the lines for the 'nat' access list. I first wrote the line like this: print >>output, 'permit ip {} {} any'.format(inside.ip, inside.hostmask) because I wanted the base IP of the inside network to go into the access-list statement. The code would have run, but it would have produced incorrect output. It also felt unnatural to write '.network' there, since the object, in my mind, _was_ a network, which is presumably why I wrote 'inside.ip' at first, before catching myself. But that I can certainly live with, just as I can live with '.ip' if I have to :) --David From rdmurray at bitdance.com Tue Sep 29 21:31:31 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 29 Sep 2009 15:31:31 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <1254240196.5662.5.camel@localhost> Message-ID: On Tue, 29 Sep 2009 at 15:24, R. David Murray wrote: > There's one place in this code where the inclusion of the 'ip' information > in the IPNetwork class could have been used. In the line that allows ICMP > traffic to the router's outside port, I could have written 'inside.ip' > instead of interfaces['inside'].ip. I think that would have confused me > when I came back to read the code later, though, since 'inside' is > otherwise a network object. Er, that should have been 'outside' in both the code and the paragraph above, of course. Which would also have run but produced incorrect output...make of that what you will :) --David From mal at egenix.com Tue Sep 29 21:55:35 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Tue, 29 Sep 2009 21:55:35 +0200 Subject: [Python-Dev] Python 2.7 Mac universal builds seem broken on trunk In-Reply-To: <15E9DA97-B2CF-4F01-8375-F2169E598CB6@mac.com> References: <90A90A3C-E037-4FCA-95D2-A46A5C6DD60A@barrys-emacs.org> <8F99FB53-5D00-4DCB-AA17-530DD97D3688@mac.com> <4AC227BB.9000607@egenix.com> <9015487A-AA0F-4448-8ADF-C2B6E7B65377@mac.com> <4AC2330B.1000405@egenix.com> <15E9DA97-B2CF-4F01-8375-F2169E598CB6@mac.com> Message-ID: <4AC26637.6090401@egenix.com> Ronald Oussoren wrote: > > On 29 Sep, 2009, at 18:17, M.-A. Lemburg wrote: > >> Ronald Oussoren wrote: >>> >>>>> Use: >>>>> >>>>> ./configure --enable-framework --enable-universalsdk=/ >>>>> >>>>> The --with-universal-archs flag selects whichs architectures should be >>>>> included when you build a universal binary, defaulting to 32-bit. >>>> >>>> The Python default on 10.6 is 64-bit, so wouldn't it be better >>>> to default to that on 10.6 and use 32-bit as default on 10.3/4/5 ?! >>> >>> Defaulting to a 32-bit build has several advantages. The first is that >>> the defaults match the binary installer on the python.org website, >> >> What build options does that installer use ? (the web-page doesn't say) > > The installer is build using the script in Mac/BuildScript, and uses > --enable-framework --with-universalsk. > > This creates a 32-bit fat build that runs on 10.3.9 or later. Ok, so the installed has 32-bit x86 and ppc code, just like for the older releases. >> >>> and >>> secondly there are still 3th-party libraries that don't work in 64-bit >>> mode (mostly GUI libraries, until recently Tk and wxWidgets wrapped the >>> Carbon libraries which are not available in 64-bit mode; AFAIK both have >>> betas that wrap the Cocoa libraries instead). >>> >>> To mimick the system default you'd have to default to 32-bit on 10.4, >>> 4-way universal on 10.5 and 3-way universal on 10.6, and that is without >>> considering deployment targets. All of those are available as options, >>> I'd prefer to keep it this way for now to keep things simple. >> >> Hmm, so I guess the only way to support them all is by building >> extensions >> using 4-way universal on 10.5. No wonder they are called "fat" >> binaries ;-) > > I like the technology though, much more convenient than having parallel > directory tries as on Linux. True, but it also makes it harder to get rid off parts you don't need and the often necessary separate combine step (using lipo) can get tedious at times, e.g. when trying to build OpenSSL or other software that uses different configuration data or optimizations depending on the underlying architecture. >>>>> I'll write some documentation on the build options on OSX, but don't >>>>> know what's the best location to do so. >>>> >>>> Please put that information into Mac/README which already has >>>> documentation for how to build universal binaries on Mac. >>> >>> I know that, I wrote most of that file ;-). >> >> Which is why I was surprised you asked :-) > > I hoped to find a document on docs.python.org that explains how to > install Python, but sadly enough there isn't. Mac/README contains the > right information, but isn't easily found if you're searching on the web > or even if you're looking for documentation in the source tree. True. The download page used to have a bit of information on how to build Python, but that got trimmed away it seems. The Python docs only have this page: Unix: http://docs.python.org/using/unix.html Window: http://docs.python.org/using/windows.html#installing-python Mac: http://docs.python.org/using/mac.html#getting-and-installing-macpython Neither of those pages has the in-depth information of the README files and, frankly, I wouldn't be looking for an installation guide in a "Using Python" section either. Perhaps something for a sprint to change... -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Sep 29 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From hnassrat at gmail.com Tue Sep 29 22:20:35 2009 From: hnassrat at gmail.com (Hatem Nassrat) Date: Tue, 29 Sep 2009 17:20:35 -0300 Subject: [Python-Dev] Announcing PEP 3136 Message-ID: <20090929202035.GA5492@gmail.com> Tue Jul 3 10:14:17 CEST 2007, Guido van Rossum wrote: > On 6/30/07, Matt Chisholm wrote: > > I've created and submitted a new PEP proposing support for labels in > > Python's break and continue statements. Georg Brandl has graciously > > added it to the PEP list as PEP 3136: > > > > http://www.python.org/dev/peps/pep-3136/ > > I think this is a good summary of various proposals that have been > floated in the past, plus some new ones. As a PEP, it falls short > because it doesn't pick a solution but merely offers a large menu of > possible options. Also, there is nothing about implementation yet. > > However, I'm rejecting it on the basis that code so complicated to > require this feature is very rare. In most cases there are existing > work-arounds that produce clean code, for example using 'return'. I agree that this feature will only serve as a quick hack and in many cases it would be misused and ugly code will be the result. However, it seems that there are some shortcuts that have sneaked into python (I am a fairly new Python programmer, only since late 2.4, so don't shoot me). The specific one of which I speak about is: while_stmt ::= "while" expression ":" suite ["else" ":" suite] for_stmt ::= "for" target_list "in" expression_list ":" suite ["else" ":" suite] try1_stmt ::= "try" ":" suite ("except" [expression ["as" target]] ":" suite)+ ["else" ":" suite] ["finally" ":" suite] All these else's seem like shortcuts to me. I did find a use for them, once I found out they existed, but I get butterflies whenever I do. -- Hatem Nassrat From p.f.moore at gmail.com Tue Sep 29 22:31:58 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 29 Sep 2009 21:31:58 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> Message-ID: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> 2009/9/28 Yuvgoog Greenle : > 1. There is no chance of the script killing itself. In argparse and optparse > exit() is called on every parsing error (btw because of this it sucks to > debug parse_args in an interpreter). That one does worry me. I'd rather argparse (or any library function) didn't call sys.exit on my behalf - it should raise an exception. Is it actually true that argparse exits? (I can imagine that it might if --help was specified, for example. An exception may not be right here, but I still don't like the idea of a straight exit - I've used too many C libraries that think they know when I want to exit). > 2. There is no chance the parser will print things I don't want it to print. That may also be bad - for example, Windows GUI-mode programs raise an error if they write to stdout/stderr. I could imagine using argparse for such a program, and wanting to do something with --help other than write to stdout and exit (a message box, for example). And yet, I'd want access to the text argparse would otherwise write to stdout. Paul. From sesquile at gmail.com Tue Sep 29 22:38:23 2009 From: sesquile at gmail.com (m h) Date: Tue, 29 Sep 2009 14:38:23 -0600 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> Message-ID: On Tue, Sep 29, 2009 at 2:31 PM, Paul Moore wrote: > 2009/9/28 Yuvgoog Greenle : >> 1. There is no chance of the script killing itself. In argparse and optparse >> exit() is called on every parsing error (btw because of this it sucks to >> debug parse_args in an interpreter). > > That one does worry me. I'd rather argparse (or any library function) > didn't call sys.exit on my behalf - it should raise an exception. Is > it actually true that argparse exits? (I can imagine that it might if > --help was specified, for example. An exception may not be right here, > but I still don't like the idea of a straight exit - I've used too > many C libraries that think they know when I want to exit). > +1 From guido at python.org Tue Sep 29 22:47:32 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Sep 2009 13:47:32 -0700 Subject: [Python-Dev] Announcing PEP 3136 In-Reply-To: <20090929202035.GA5492@gmail.com> References: <20090929202035.GA5492@gmail.com> Message-ID: On Tue, Sep 29, 2009 at 1:20 PM, Hatem Nassrat wrote: > Tue Jul 3 10:14:17 CEST 2007, Guido van Rossum wrote: >> On 6/30/07, Matt Chisholm wrote: >> > I've created and submitted a new PEP proposing support for labels in >> > Python's break and continue statements. ?Georg Brandl has graciously >> > added it to the PEP list as PEP 3136: >> > >> > http://www.python.org/dev/peps/pep-3136/ >> >> I think this is a good summary of various proposals that have been >> floated in the past, plus some new ones. As a PEP, it falls short >> because it doesn't pick a solution but merely offers a large menu of >> possible options. Also, there is nothing about implementation yet. >> >> However, I'm rejecting it on the basis that code so complicated to >> require this feature is very rare. In most cases there are existing >> work-arounds that produce clean code, for example using 'return'. > > I agree that this feature will only serve as a quick hack and in many > cases it would be misused and ugly code will be the result. However, > it seems that there are some shortcuts that have sneaked into python > (I am a fairly new Python programmer, only since late 2.4, so don't > shoot me). The specific one of which I speak about is: > > while_stmt ::= ?"while" expression ":" suite > ? ? ? ? ? ? ? ?["else" ":" suite] > > for_stmt ::= ?"for" target_list "in" expression_list ":" suite > ? ? ? ? ? ? ?["else" ":" suite] > > try1_stmt ::= ?"try" ":" suite > ? ? ? ? ? ? ? ("except" [expression ["as" target]] ":" suite)+ > ? ? ? ? ? ? ? ["else" ":" suite] > ? ? ? ? ? ? ? ["finally" ":" suite] > > All these else's seem like shortcuts to me. I did find a use for them, > once I found out they existed, but I get butterflies whenever I do. In English, butterflies are usually a good thing (they mean you'e in love). These else clauses (assuming you're talking about that) have been in the language pretty much forever. The combined except/finally clause is newer, it was added because Java allows it and it was actually a pretty common usage. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric at trueblade.com Tue Sep 29 22:56:38 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 29 Sep 2009 16:56:38 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> Message-ID: <4AC27486.3020106@trueblade.com> Paul Moore wrote: > 2009/9/28 Yuvgoog Greenle : >> 1. There is no chance of the script killing itself. In argparse and optparse >> exit() is called on every parsing error (btw because of this it sucks to >> debug parse_args in an interpreter). > > That one does worry me. I'd rather argparse (or any library function) > didn't call sys.exit on my behalf - it should raise an exception. Is > it actually true that argparse exits? (I can imagine that it might if > --help was specified, for example. An exception may not be right here, > but I still don't like the idea of a straight exit - I've used too > many C libraries that think they know when I want to exit). You can override ArgumentParser.error() to raise an exception. >> 2. There is no chance the parser will print things I don't want it to print. > > That may also be bad - for example, Windows GUI-mode programs raise an > error if they write to stdout/stderr. I could imagine using argparse > for such a program, and wanting to do something with --help other than > write to stdout and exit (a message box, for example). And yet, I'd > want access to the text argparse would otherwise write to stdout. It looks like this might not be so easy to do. I'd suggest adding a file-like object to the constructor, defaulting to sys.stderr; or maybe an ArgumentParser.print() method that can be overridden. Eric. From steven.bethard at gmail.com Tue Sep 29 22:57:22 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 29 Sep 2009 13:57:22 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> Message-ID: On Tue, Sep 29, 2009 at 1:31 PM, Paul Moore wrote: > 2009/9/28 Yuvgoog Greenle : >> 1. There is no chance of the script killing itself. In argparse and optparse >> exit() is called on every parsing error (btw because of this it sucks to >> debug parse_args in an interpreter). > > That one does worry me. I'd rather argparse (or any library function) > didn't call sys.exit on my behalf - it should raise an exception. Is > it actually true that argparse exits? (I can imagine that it might if > --help was specified, for example. An exception may not be right here, > but I still don't like the idea of a straight exit - I've used too > many C libraries that think they know when I want to exit). This is behavior that argparse inherits from optparse, but I believe it's still what 99.9% of users want. If you're writing a command line interface, you don't want a stack trace when there's an error message (which is what you'd get if argparse just raised exceptions) you want an exit with an error code. That's what command line applications are supposed to do. If you're not using argparse to write command line applications, then I don't feel bad if you have to do a tiny bit of extra work to take care of that use case. In this particular situation, all you have to do is subclass ArgumentParser and override exit() to do whatever you think it should do. >> 2. There is no chance the parser will print things I don't want it to print. > > That may also be bad - for example, Windows GUI-mode programs raise an > error if they write to stdout/stderr. I could imagine using argparse > for such a program, and wanting to do something with --help other than > write to stdout and exit (a message box, for example). And yet, I'd > want access to the text argparse would otherwise write to stdout. There is only a single method in argparse that prints things, _print_message(). So if you want it to do something else, you can simply override it in a subclass. I can make that method public if this is a common use case. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From tjreedy at udel.edu Tue Sep 29 23:05:27 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Tue, 29 Sep 2009 17:05:27 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> Message-ID: This is my first post in this thread. I do not currently expect to directly use ipaddr, but I am interested in Python having good modules and a friendly development environment. It strikes me that a module that meets the desiderata of broad community support is going to have the 'baggage' of lots of users, who will not want current code broken. So that criterion conflicts with the desire to do major redesign. Peter Moody wrote: > re: assumed disconnect between the statement "Addresses and Networks > are distinct" and the implementation. > > I don't actually see a disconnect. I can, at least somewhat. But I do not think philosophical agreement is either possible or needed for practical compromise. There will still remain the issue of how the doc is organized and worded (see below). > re: denormalized networks: > > I've mentioned at least once (and others have mentioned as well) that > it's very common, when describing the ip address/prefix of a NIC to > write, "192.168.1.100/24" > > re: new comparison methods > > this is interesting. I would be open to doing this and making __lt__, > __gt__, __eq__ and friends do the non-ip comparing by default. would > this be sufficient or does the fact that the network has the .ip > attribute break something pythonic? To my naive viewpoint, yes and mostly no, and if Guido say yes and no, that would also be good enough for me. I do not think the cognitive load of .ip on learning the module is any greater than having a third class, especially if it is somewhat isolated in the doc as suggested below. My suggestion for the doc, based on my understanding derived from reading this thread (and therefore subject to correction ;-) is something as follows: ----- NN. IPADDR NN.1 Adresses .... NN.2 Networks Conceptually, a network is a power of 2 range of addresses whose last k bits range from 0...0 to 1...1. A network object is similar to a Py3 range object. Networks are often (usually? always?) defined by an address plus an indication of the bits to either keep or ignore. As a convenience for certain uses, the defining address is included with the network instance as its .ip attribute (see NN.3 Address + network). However, standard comparisons ignore this attribute, as do the methods discussed in this section. ... NN.3 (or possibly NN.2.1) Address (or Host) + Network As mentioned above, Network instances include their defining address as a .ip attribute so that they can alternatively be used as Host+Network. The following functions and methods use this attribute. ... ------- In other words, I suggest the the doc be organized more or less as it would be if there were a third class. I believe this would help both those who do not need or want this usage, as well as those who do. Terry Jan Reedy From guido at python.org Tue Sep 29 23:23:39 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Sep 2009 14:23:39 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> Message-ID: On Tue, Sep 29, 2009 at 2:05 PM, Terry Reedy wrote: >> this is interesting. I would be open to doing this and making __lt__, >> __gt__, __eq__ and friends do the non-ip comparing by default. would >> this be sufficient or does the fact that the network has the .ip >> attribute break something pythonic? > > To my naive viewpoint, yes and mostly no, and if Guido say yes and no, that > would also be good enough for me. Indeed I say yes and no. > I do not think the cognitive load of .ip > on learning the module is any greater than having a third class, especially > if it is somewhat isolated in the doc as suggested below. Here I disagree. I want users to have few choices of data types, since they usually make the wrong choice (Ever tried to figure out whether to use an ArrayList or a LinkedList in Java?) So if there are two different but similar classes to represent a network, distinguished only by whether they retain the denormalized input address or not, users have a 50% chance of picking the wrong one, while if there is only one class, there is a 100% chance of picking the right one. (There is still the IPAddress class, but the differences between IPAddress and IPNetwork are much clearer.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From v+python at g.nevcal.com Wed Sep 30 00:04:30 2009 From: v+python at g.nevcal.com (Glenn Linderman) Date: Tue, 29 Sep 2009 15:04:30 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> Message-ID: <4AC2846E.8060304@g.nevcal.com> On approximately 9/29/2009 1:57 PM, came the following characters from the keyboard of Steven Bethard: > On Tue, Sep 29, 2009 at 1:31 PM, Paul Moore wrote: >> 2009/9/28 Yuvgoog Greenle : >>> 1. There is no chance of the script killing itself. In argparse and optparse >>> exit() is called on every parsing error (btw because of this it sucks to >>> debug parse_args in an interpreter). >> That one does worry me. I'd rather argparse (or any library function) >> didn't call sys.exit on my behalf - it should raise an exception. Is >> it actually true that argparse exits? (I can imagine that it might if >> --help was specified, for example. An exception may not be right here, >> but I still don't like the idea of a straight exit - I've used too >> many C libraries that think they know when I want to exit). > > This is behavior that argparse inherits from optparse, but I believe > it's still what 99.9% of users want. If you're writing a command line > interface, you don't want a stack trace when there's an error message > (which is what you'd get if argparse just raised exceptions) you want > an exit with an error code. That's what command line applications are > supposed to do. > > If you're not using argparse to write command line applications, then > I don't feel bad if you have to do a tiny bit of extra work to take > care of that use case. In this particular situation, all you have to > do is subclass ArgumentParser and override exit() to do whatever you > think it should do. > >>> 2. There is no chance the parser will print things I don't want it to print. >> That may also be bad - for example, Windows GUI-mode programs raise an >> error if they write to stdout/stderr. I could imagine using argparse >> for such a program, and wanting to do something with --help other than >> write to stdout and exit (a message box, for example). And yet, I'd >> want access to the text argparse would otherwise write to stdout. > > There is only a single method in argparse that prints things, > _print_message(). So if you want it to do something else, you can > simply override it in a subclass. I can make that method public if > this is a common use case. Documenting both of these options would forestall people from thinking it is only useful for console applications. An example of using argparse with Tk (I think that is the only GUI that ships with Python) would also be good. In making the method public you might want to rename it to something other than print_message. If all the messages are errors, including "error" in the name would be good. If not, classifying the errors vs non-errors as an extra parameter might be good (in other words... inform the user and continue, or inform the user and exit). Separating the message from the classification is not good, because that leads to dialog boxes having only an "OK" button, and depending on the message, it can be really inappropriate to display an "OK" button... buttons named "Sorry", "Alas!", and "Exit" or "Quit" are often more appropriate, even when there is no user choice possible. Clearly if someone is writing a GUI, they are willing to write lots of lines of code to do things... a couple more well-documented lines to integrate argparse into their chosen GUI doesn't seem onerous. Especially if they can cut, paste, and hack from the above-suggested example code, like they do for the rest of their GUI code. Well, at least, cut, paste, and hack is how I write GUIs when I bother. -- Glenn -- http://nevcal.com/ =========================== A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking From greg.ewing at canterbury.ac.nz Wed Sep 30 00:40:51 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 30 Sep 2009 10:40:51 +1200 Subject: [Python-Dev] [OT] Denormalized fractions [was Re: PEP 3144 review.] In-Reply-To: <200909292120.03677.steve@pearwood.info> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909292120.03677.steve@pearwood.info> Message-ID: <4AC28CF3.6040900@canterbury.ac.nz> Steven D'Aprano wrote: > there's a mathematical operator called the mediant: > > mediant(a/b, c/d) = (a+c)/(b+d) That's a function of four arguments, not two! -- Greg From greg.ewing at canterbury.ac.nz Wed Sep 30 00:51:11 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 30 Sep 2009 10:51:11 +1200 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <19137.62484.513693.107525@montanaro.dyndns.org> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> <19137.62484.513693.107525@montanaro.dyndns.org> Message-ID: <4AC28F5F.9020003@canterbury.ac.nz> skip at pobox.com wrote: > I have never completely wrapped my brain around optparse. Getopt I > just remember. Seems to me that optparse and argparse are fairly similar in their APIs, and that argparse isn't going to be significantly easier to fit in one's brain than optparse. There's an art to coming up with an API that makes simple things easy and other things possible. I'm not convinced that argparse represents a subsantial enough advancement in that art to justify replacing optparse with it in the stdlib, and thereby forcing everyone to learn a similar-but-different API. -- Greg From p.f.moore at gmail.com Wed Sep 30 00:51:23 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Tue, 29 Sep 2009 23:51:23 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> Message-ID: <79990c6b0909291551j4f526f32i3dc42264f67c5c8e@mail.gmail.com> 2009/9/29 Steven Bethard : > If you're not using argparse to write command line applications, then > I don't feel bad if you have to do a tiny bit of extra work to take > care of that use case. In this particular situation, all you have to > do is subclass ArgumentParser and override exit() to do whatever you > think it should do. [...] > There is only a single method in argparse that prints things, > _print_message(). So if you want it to do something else, you can > simply override it in a subclass. I can make that method public if > this is a common use case. Thanks, that's fine for me (as things stand, no need to publicly expose and document _print_message). BTW, the helpful and responsive way you reply to queries is much appreciated. I'm +1 on the PEP (although I see why people want getopt to stay, so I'm happy to leave that and only deprecate optparse). Paul. From janzert at janzert.com Wed Sep 30 01:05:31 2009 From: janzert at janzert.com (Janzert) Date: Tue, 29 Sep 2009 19:05:31 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909271104w75c610e2n41986dc254d24649@mail.gmail.com> <8517e9350909271218o138c5122w8e62e5735713ce85@mail.gmail.com> <9676ED37-B679-4948-B658-257314BCD46B@fuhm.net> <8517e9350909271256i1650d47se9ddd38d52504045@mail.gmail.com> <8517e9350909271336o28e7b474t4d1632542ecfb9af@mail.gmail.com> <8517e9350909271359u8d0da8dpb7eed2774ff4464e@mail.gmail.com> Message-ID: Peter Moody wrote: > On Sun, Sep 27, 2009 at 1:49 PM, Antoine Pitrou wrote: >> Peter Moody hda3.com> writes: >>>>>> def parse_net_and_addr(s): >>>>>> return (IPNetwork(s), IPAddress(s.split('/')[0])) >>>>> I've only heard talk of new classes and new methods, not new >>>>> constructor functions. >>>> Well, "method" in that context meant "class method" since the results aren't >>>> dependent on a particular instance. Of course, both a class method or a >>>> module-level function would be fine. >>> so this is not the response I got when I asked what was required >>> before. Would adding this constructor function satisfy your concerns >>> (given sensible strict settings in the constructor, etc)? >> Assuming the Network type loses the notion of a specific host (or host address, >> or `ip`) attached to it, yes. > > this is "less useful (strictly removing functionality)" and is an > example of what I explicitly said I was not going to do with ipaddr. > > Cheers, > /peter > Not that I should have any say, but I'm also -1 on any proposal that conflates an "address with mask" and "network" into the same object. Janzert From greg.ewing at canterbury.ac.nz Wed Sep 30 01:20:18 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 30 Sep 2009 11:20:18 +1200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> Message-ID: <4AC29632.9050903@canterbury.ac.nz> Peter Moody wrote: > I've mentioned at least once (and others have mentioned as well) that > it's very common, when describing the ip address/prefix of a NIC to > write, "192.168.1.100/24" Yes, but if you want to *retain* all of that information, the object you have isn't just a "network", it's more like "network plus the address of its interface". So some people think that using the name "IPNetwork" for this object is confusing. -- Greg From steven.bethard at gmail.com Wed Sep 30 01:38:53 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 29 Sep 2009 16:38:53 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC2846E.8060304@g.nevcal.com> References: <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> <4AC2846E.8060304@g.nevcal.com> Message-ID: On Tue, Sep 29, 2009 at 3:04 PM, Glenn Linderman wrote: > On approximately 9/29/2009 1:57 PM, came the following characters from the > keyboard of Steven Bethard: >> If you're not using argparse to write command line applications, then >> I don't feel bad if you have to do a tiny bit of extra work to take >> care of that use case. In this particular situation, all you have to >> do is subclass ArgumentParser and override exit() to do whatever you >> think it should do. [snip] >> There is only a single method in argparse that prints things, >> _print_message(). So if you want it to do something else, you can >> simply override it in a subclass. I can make that method public if >> this is a common use case. > > Documenting both of these options would forestall people from thinking it is > only useful for console applications. I'm totally fine with people thinking it is only useful for console applications. That's what it's intended for. That said, if there are people out there who want to use it for other applications, I'm happy to make things easier for them if I know concretely what they want. > An example of using argparse with Tk > (I think that is the only GUI that ships with Python) would also be good. I don't really use GUI libraries, so I wouldn't be able to come up with such an example. I'd also rather not make API changes based on speculative use cases, so before I spend time documenting these things, I'd really like to hear from someone who has already, say, used getopt or optparse in conjunction with a GUI library, and what feedback they have about that. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From barry at python.org Wed Sep 30 01:39:16 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Sep 2009 19:39:16 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC28F5F.9020003@canterbury.ac.nz> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> <19137.62484.513693.107525@montanaro.dyndns.org> <4AC28F5F.9020003@canterbury.ac.nz> Message-ID: On Sep 29, 2009, at 6:51 PM, Greg Ewing wrote: > skip at pobox.com wrote: >> I have never completely wrapped my brain around optparse. Getopt I >> just remember. > > Seems to me that optparse and argparse are fairly similar > in their APIs, and that argparse isn't going to be significantly > easier to fit in one's brain than optparse. There's no question it is if you're doing more complicated stuff, like extending it or using subcommands. After I converted my code from optparse to argparse, I ended up with less stuff that was more regular and easier to understand. It convinced me that argparse is a win. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From v+python at g.nevcal.com Wed Sep 30 01:59:22 2009 From: v+python at g.nevcal.com (Glenn Linderman) Date: Tue, 29 Sep 2009 16:59:22 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> <4AC2846E.8060304@g.nevcal.com> Message-ID: <4AC29F5A.1040809@g.nevcal.com> On approximately 9/29/2009 4:38 PM, came the following characters from the keyboard of Steven Bethard: > On Tue, Sep 29, 2009 at 3:04 PM, Glenn Linderman wrote: >> On approximately 9/29/2009 1:57 PM, came the following characters from the >> keyboard of Steven Bethard: >>> If you're not using argparse to write command line applications, then >>> I don't feel bad if you have to do a tiny bit of extra work to take >>> care of that use case. In this particular situation, all you have to >>> do is subclass ArgumentParser and override exit() to do whatever you >>> think it should do. > [snip] >>> There is only a single method in argparse that prints things, >>> _print_message(). So if you want it to do something else, you can >>> simply override it in a subclass. I can make that method public if >>> this is a common use case. >> Documenting both of these options would forestall people from thinking it is >> only useful for console applications. > > I'm totally fine with people thinking it is only useful for console > applications. That's what it's intended for. That said, if there are > people out there who want to use it for other applications, I'm happy > to make things easier for them if I know concretely what they want. Hmm. Maybe that is partly why (sadly) so many GUI programs don't offer much in the way of command line options. Of course, many of their users in Windowsland wouldn't know how to tweak the shortcut to invoke them with options anyway, which might be another part. Fortunately, there are some good Windows programs that do offer rich GUIs and also rich command line options (e.g. IrfanView and ImgBurn) >> An example of using argparse with Tk >> (I think that is the only GUI that ships with Python) would also be good. > > I don't really use GUI libraries, so I wouldn't be able to come up > with such an example. I'd also rather not make API changes based on > speculative use cases, so before I spend time documenting these > things, I'd really like to hear from someone who has already, say, > used getopt or optparse in conjunction with a GUI library, and what > feedback they have about that. I'm not a Tk user, just coming to Python from Perl, where I used Win32::GUI, but when I was looking for a functional and portable GUI development package, Perl didn't have one (still doesn't have one that supports Unicode and printing), and Python does... so here I come to Python and Qt. From my experience in Perl GUI Windowsland, the primary issue with command line parsing is displaying the message in a dialog instead of STDOUT. But the message and the user choices have to be known at the same time to design the dialog box. And, there is nothing so aggravating as to be shown an error message, and the only option is a button that says "OK", when it clearly isn't OK. So the tone/type of the messages also needs to be known, even when there are no user choices. The --help option could display the help message, and offer OK. Many errors (particularly unrecoverable ones) should display the error message, and offer an Exit button, or just the close box. A few (although probably only highly customized user options) might want to give the user multiple recovery options. So, I guess I'm in the unfortunate position of seeing the need, but not having used these specific technologies enough to offer an example either. So far, I've only used optparse (just now hearing about argparse in this thread) for command line programs in Python, and I am still just experimenting with Qt, and don't have enough familiarity with it yet to think that what I'm doing is best practices. I think it would be sad to a new replacement for optparse that didn't GUI usage, though, at least in a limited form. The concepts I describe seem sufficient from experience in other environments, and I would think they would be sufficient in Python too, but I'm too new here to state that definitely. -- Glenn -- http://nevcal.com/ =========================== A protocol is complete when there is nothing left to remove. -- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking From ncoghlan at gmail.com Wed Sep 30 02:05:17 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 10:05:17 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <19136.52362.562678.628471@montanaro.dyndns.org> References: <4ABFE075.2070309@voidspace.org.uk> <4AC0AD5C.1000008@gmail.com> <19136.52362.562678.628471@montanaro.dyndns.org> Message-ID: <4AC2A0BD.1070006@gmail.com> skip at pobox.com wrote: > Nick> +1 here as well (although we should definitely find a way to use > Nick> str.format strings instead of %-format ones... come to think of > Nick> it, does even the logging module support str.format style > Nick> formatting in Py3k?) > > Assuming argparse currently works with versions of Python < 2.6 I see no > reason to make such a change. This would just introduce needless > differences between the version delivered with Python and the PyPI version > and make it more difficult for the author to keep the two code bases in > sync. Sorry, my phrasing was poor - I should have said "as well as" rather than "instead of". For both existing argparse users and to ease conversion from optparse to argparse, %-formatting support obviously needs to remain. We already have a problem with existing APIs not supporting the new string formatting - let's not make it worse by adding *new* APIs that only support the *old* formatting technique. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From steven.bethard at gmail.com Wed Sep 30 02:27:31 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 29 Sep 2009 17:27:31 -0700 Subject: [Python-Dev] transitioning from % to {} formatting Message-ID: There's a lot of code already out there (in the standard library and other places) that uses %-style formatting, when in Python 3.0 we should be encouraging {}-style formatting. We should really provide some sort of transition plan. Consider an example from the logging docs: logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") We'd like to support both this style as well as the following style: logging.Formatter("{asctime} - {name} - {levelname} - {message}") Perhaps we'd eventually deprecate the %-style formatting, but at least in the intervening time, we'd have to support both. I see a few possibilities: * Add a new class, NewFormatter, which uses the {}-style formatting. This is ugly because it makes the formatting we're trying to encourage look like the alternative implementation instead of the standard one. It also means we have to come up with new names for every API that uses format strings. * Have Formatter try to guess whether it got %-style formatting or {}-style formatting, and then delegate to the appropriate one. I don't know how accurately we can guess. We also end up still relying on both formatting styles under the hood. * Have Formatter convert all %-style formatting strings to {}-style formatting strings (automatically). This still involves some guessing, and involves some serious hacking to translate from one to the other (maybe it wouldn't even always be possible?). But at least we'd only be using {}-style formatting under the hood. Thoughts? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From ncoghlan at gmail.com Wed Sep 30 02:25:48 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 10:25:48 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC28F5F.9020003@canterbury.ac.nz> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> <19137.62484.513693.107525@montanaro.dyndns.org> <4AC28F5F.9020003@canterbury.ac.nz> Message-ID: <4AC2A58C.9080609@gmail.com> Greg Ewing wrote: > skip at pobox.com wrote: >> I have never completely wrapped my brain around optparse. Getopt I >> just remember. > > Seems to me that optparse and argparse are fairly similar > in their APIs, and that argparse isn't going to be significantly > easier to fit in one's brain than optparse. > > There's an art to coming up with an API that makes simple > things easy and other things possible. I'm not convinced that > argparse represents a subsantial enough advancement in > that art to justify replacing optparse with it in the stdlib, > and thereby forcing everyone to learn a similar-but-different > API. As someone that has written multiple optparse based utility scripts, I would say that yes, argparse is a *huge* improvement. Several things that I implemented for my own use in a rather clumsy fashion (subcommands, aggregated parsers, non-interspersed arguments) have far more elegant support built directly into argparse. For the getopt-vs-opt/argparse discussion, I believe the major distinction is in the relative balance between procedural and declarative style of argument handling. getopt is very procedural - you define a minimal amount regarding the options you accept, but then do the bulk of the command line processing yourself optparse is declarative to some degree, but forces you to drop back to a procedural style to handle arguments and subcommands. argparse takes things even further in a declarative direction by adding explicit support for positional arguments and subcommands. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From greg.ewing at canterbury.ac.nz Wed Sep 30 02:38:26 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Wed, 30 Sep 2009 12:38:26 +1200 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> Message-ID: <4AC2A882.7070108@canterbury.ac.nz> Paul Moore wrote: > I'd rather argparse (or any library function) > didn't call sys.exit on my behalf - it should raise an exception. Actually, sys.exit() *does* raise an exception (i.e. SystemExit) that you can catch if you want. -- Greg From ncoghlan at gmail.com Wed Sep 30 02:44:07 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 10:44:07 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC12AC6.80304@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> Message-ID: <4AC2A9D7.4080603@gmail.com> Martin v. L?wis wrote: >> I would say that there certainly are precedents in other areas for >> keeping the information about the input form around. For example, >> occasionally it would be handy if parsing a hex integer returned an >> object that was compatible with other integers but somehow kept a hint >> that would cause printing it to use hex by default. > > At the risk of bringing in false analogies: it seems that Python > typically represents values of some type in their canonical form, > rather than remembering the form in which they arrived in the program: > - integer values "forget" how many preceding zeroes they have > - string literals forget which of the characters had been escaped, and > whether the string was single- or double-quoted > - floating point values forget a lot more about their literal > representation (including even the literal decimal value) > > I guess a close case would be rational numbers: clearly, 3?2 == 6?4; > would a Python library still remember (and repr) the original numerator > and denominator? For a concrete example of an object which remembers details about its creation that it ignores when determining equality, we have decimal.Decimal: .>> from decimal import Decimal as d .>> x = d("3.0") .>> y = d("3.00") .>> x d("3.0") .>> y d("3.00") .>> repr(x) == repr(y) False .>> x.as_tuple() == y.as_tuple() False .>> x == y True (It was actually thinking of this example which led to me suggesting that the equivalence classes of IPNetwork just needed adjusting rather than the .ip attribute being removed completely) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From barry at python.org Wed Sep 30 02:56:22 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Sep 2009 20:56:22 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC2A58C.9080609@gmail.com> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> <19137.62484.513693.107525@montanaro.dyndns.org> <4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com> Message-ID: On Sep 29, 2009, at 8:25 PM, Nick Coghlan wrote: > getopt is very procedural - you define a minimal amount regarding the > options you accept, but then do the bulk of the command line > processing > yourself Right, and we shouldn't underestimate the cognitive load this imposes. All those twisty if-conditions are de-facto part of getopt's API. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From ncoghlan at gmail.com Wed Sep 30 03:07:28 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 11:07:28 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> Message-ID: <4AC2AF50.6010806@gmail.com> Guido van Rossum wrote: > On Tue, Sep 29, 2009 at 2:05 PM, Terry Reedy wrote: >> I do not think the cognitive load of .ip >> on learning the module is any greater than having a third class, especially >> if it is somewhat isolated in the doc as suggested below. > > Here I disagree. I want users to have few choices of data types, since > they usually make the wrong choice (Ever tried to figure out whether > to use an ArrayList or a LinkedList in Java?) > > So if there are two different but similar classes to represent a > network, distinguished only by whether they retain the denormalized > input address or not, users have a 50% chance of picking the wrong > one, while if there is only one class, there is a 100% chance of > picking the right one. (There is still the IPAddress class, but the > differences between IPAddress and IPNetwork are much clearer.) Note that Terry was just talking about the documentation there. That is, he was suggesting that the documentation first describe the "normal" IPNetwork methods which are unaffected by the .ip attribute and then, in a separate section, describe the methods that are specific to "denormalised" networks where "net.ip != net.network". Such an arrangement makes sense to me. At the risk of bikeshedding a bit, I'm still somewhat uncomfortable with the "net.network" and "net.ip" attribute names. RDMs example application elicited the reason for that discomfort pretty well: the current naming seems like an invitation to write code using 'net.ip' that should have used 'net.network' instead. Such code will then work correctly most of the time (i.e. when only given normalised IPNetwork objects) but will fail when given a denormalised one. I believe that discomfort could be eliminated best by changing the name of the ".ip" attribute to ".host_ip" to make it clear that it is referring to the IP address of the host that was used to derive the network definition rather than referring to the network ID itself. Shortening ".network" to ".net_ip" would also help (this latter change would also eliminate the mental disconnect caused by an attribute called .network returning an IPAddress instance). Those two naming suggestions are far less fundamental than fixing the definition of the IPNetwork equivalent classes though - and it sounds like Peter is happy with the proposed change on that front. Regards, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From steven.bethard at gmail.com Wed Sep 30 03:16:33 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 29 Sep 2009 18:16:33 -0700 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: <1afaf6160909291754r51197dd3jdcd05782bef11ebd@mail.gmail.com> References: <1afaf6160909291754r51197dd3jdcd05782bef11ebd@mail.gmail.com> Message-ID: On Tue, Sep 29, 2009 at 5:54 PM, Benjamin Peterson wrote: > * Provide a flag to Formatter which controls whether new or old > formatting is used. Emit a warning when it's not true. So then the transition strategy is something like: version N: Add formatting flag which uses {}-style formatting on True and %-style formatting on False, which defaults to False version N + 1: Deprecate False value for formatting flag (all code should start specifying flag=True) version N + 2: Raise error on False value for formatting flag (all code must specify flag=True) version N + 3: Deprecate formatting flag version N + 4: Remove formatting flag ? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From ncoghlan at gmail.com Wed Sep 30 03:18:59 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 11:18:59 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> <19137.62484.513693.107525@montanaro.dyndns.org> <4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com> Message-ID: <4AC2B203.9010002@gmail.com> Barry Warsaw wrote: > On Sep 29, 2009, at 8:25 PM, Nick Coghlan wrote: > >> getopt is very procedural - you define a minimal amount regarding the >> options you accept, but then do the bulk of the command line processing >> yourself > > Right, and we shouldn't underestimate the cognitive load this imposes. > All those twisty if-conditions are de-facto part of getopt's API. People assess the mental cost differently though - for getopt, they tend to assign the cost to the script they're currently writing, while for optparse/argparse they assign the cost to those two libraries being difficult to learn :) Keeping getopt around *and* including a "add_getopt_arguments" method in argparse is probably the best of both worlds, in that it allows for relatively straightforward evolution of an application: 1. Start with getopt 2. As the getopt argument parsing gets twisty and arcane and maintaining the help string becomes a nightmare, move to argparse with the "add_getopt_arguments" method. 3. Over time, convert more arguments to proper argparse arguments with full documentation. (Note that getting a good help string "for free" is one of the biggest gains I currently find in using optparse over getopt. This is especially so once you start making use of option groups) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Wed Sep 30 03:21:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 11:21:49 +1000 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC2A882.7070108@canterbury.ac.nz> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> <4AC2A882.7070108@canterbury.ac.nz> Message-ID: <4AC2B2AD.4060600@gmail.com> Greg Ewing wrote: > Paul Moore wrote: >> I'd rather argparse (or any library function) >> didn't call sys.exit on my behalf - it should raise an exception. > > Actually, sys.exit() *does* raise an exception (i.e. > SystemExit) that you can catch if you want. I was going to mention that. Between catching SystemExit, the '-i' switch to the interpeter and the PYTHONINSPECT environment variable, investigating applications that call sys.exit isn't *that* difficult. It's a far cry from the instant exits of a C level abort call. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From steven.bethard at gmail.com Wed Sep 30 03:26:04 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 29 Sep 2009 18:26:04 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC2B203.9010002@gmail.com> References: <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> <19137.62484.513693.107525@montanaro.dyndns.org> <4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com> <4AC2B203.9010002@gmail.com> Message-ID: On Tue, Sep 29, 2009 at 6:18 PM, Nick Coghlan wrote: > Keeping getopt around *and* including a "add_getopt_arguments" method in > argparse is probably the best of both worlds, in that it allows for > relatively straightforward evolution of an application: > > 1. Start with getopt > 2. As the getopt argument parsing gets twisty and arcane and maintaining > the help string becomes a nightmare, move to argparse with the > "add_getopt_arguments" method. > 3. Over time, convert more arguments to proper argparse arguments with > full documentation. One thing that wouldn't be good in this transition is that "add_getopt_arguments" can only generate the part of the help string that says "-a" and "--author" exist as options -- it can't add the additional bit of text that says what they do because that's not provided to the getopt API. I suspect this makes the transition less convenient because with getopt you were probably already manually maintaining a usage message that had at least some of this information, and switching to "add_getopt_arguments" would mean throwing this information away. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From barry at python.org Wed Sep 30 03:30:04 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Sep 2009 21:30:04 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC2B203.9010002@gmail.com> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <4AC1829E.50802@v.loewis.de> <19137.62484.513693.107525@montanaro.dyndns.org> <4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com> <4AC2B203.9010002@gmail.com> Message-ID: On Sep 29, 2009, at 9:18 PM, Nick Coghlan wrote: > Keeping getopt around *and* including a "add_getopt_arguments" > method in > argparse is probably the best of both worlds, in that it allows for > relatively straightforward evolution of an application: > > 1. Start with getopt > 2. As the getopt argument parsing gets twisty and arcane and > maintaining > the help string becomes a nightmare, move to argparse with the > "add_getopt_arguments" method. > 3. Over time, convert more arguments to proper argparse arguments with > full documentation. Maybe. I haven't been following this entire thread, but I don't see much point in making it easy to go from getopt to argparse. The two are so different that I think you're either going to jump all in or not. Maybe that's just me though as I really don't see much use for getopt any more (even for throwaway scripts). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From skip at pobox.com Wed Sep 30 03:43:10 2009 From: skip at pobox.com (skip at pobox.com) Date: Tue, 29 Sep 2009 20:43:10 -0500 (CDT) Subject: [Python-Dev] test_thread tracebacks Message-ID: <20090930014310.78E571250FE5@montanaro.dyndns.org> It's been awhile since I rebuilt Python and ran the test suite. This evening I noticed this on my Mac (OS X 10.5): test_thread Unhandled exception in thread started by > Traceback (most recent call last): File "/Users/skip/src/python/trunk/Lib/test/test_thread.py", line 51, in task self.done_mutex.release() thread.error: release unlocked lock Unhandled exception in thread started by > Traceback (most recent call last): File "/Users/skip/src/python/trunk/Lib/test/test_thread.py", line 51, in task self.done_mutex.release() thread.error: release unlocked lock Unhandled exception in thread started by > Traceback (most recent call last): File "/Users/skip/src/python/trunk/Lib/test/test_thread.py", line 51, in task self.done_mutex.release() thread.error: release unlocked lock Unhandled exception in thread started by > Traceback (most recent call last): File "/Users/skip/src/python/trunk/Lib/test/test_thread.py", line 51, in task self.done_mutex.release() thread.error: release unlocked lock Oddly enough, this didn't cause the test to fail. Is this a known problem? Should I open a ticket? Skip From fdrake at gmail.com Wed Sep 30 03:49:09 2009 From: fdrake at gmail.com (Fred Drake) Date: Tue, 29 Sep 2009 21:49:09 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <4AC1829E.50802@v.loewis.de> <19137.62484.513693.107525@montanaro.dyndns.org> <4AC28F5F.9020003@canterbury.ac.nz> <4AC2A58C.9080609@gmail.com> <4AC2B203.9010002@gmail.com> Message-ID: <9cee7ab80909291849o45b73613g1fb303bfe6e33c25@mail.gmail.com> On Tue, Sep 29, 2009 at 9:30 PM, Barry Warsaw wrote: > Maybe. ?I haven't been following this entire thread, but I don't see much > point in making it easy to go from getopt to argparse. I'm with you on this; specific getopt uses are more likely to be switched to opt/argparse than to shift gradually via hybrid APIs. > The two are so > different that I think you're either going to jump all in or not. ?Maybe > that's just me though as I really don't see much use for getopt any more > (even for throwaway scripts). Heh. I never liked getopt anyway. :-) -Fred -- Fred L. Drake, Jr. "Chaos is the score upon which reality is written." --Henry Miller From barry at python.org Wed Sep 30 04:07:52 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 29 Sep 2009 22:07:52 -0400 Subject: [Python-Dev] test_thread tracebacks In-Reply-To: <20090930014310.78E571250FE5@montanaro.dyndns.org> References: <20090930014310.78E571250FE5@montanaro.dyndns.org> Message-ID: <760DC0C1-64AB-491E-8C7F-7257249049D8@python.org> On Sep 29, 2009, at 9:43 PM, skip at pobox.com wrote: > It's been awhile since I rebuilt Python and ran the test suite. This > evening I noticed this on my Mac (OS X 10.5): What version of Python? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From rdmurray at bitdance.com Wed Sep 30 04:12:54 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Tue, 29 Sep 2009 22:12:54 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC2AF50.6010806@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <4AC2AF50.6010806@gmail.com> Message-ID: On Wed, 30 Sep 2009 at 11:07, Nick Coghlan wrote: > At the risk of bikeshedding a bit, I'm still somewhat uncomfortable with > the "net.network" and "net.ip" attribute names. RDMs example application > elicited the reason for that discomfort pretty well: the current naming > seems like an invitation to write code using 'net.ip' that should have > used 'net.network' instead. Such code will then work correctly most of > the time (i.e. when only given normalised IPNetwork objects) but will > fail when given a denormalised one. > > I believe that discomfort could be eliminated best by changing the name > of the ".ip" attribute to ".host_ip" to make it clear that it is > referring to the IP address of the host that was used to derive the > network definition rather than referring to the network ID itself. > Shortening ".network" to ".net_ip" would also help (this latter change > would also eliminate the mental disconnect caused by an attribute called > .network returning an IPAddress instance). +1 --David (RDM) From nad at acm.org Wed Sep 30 04:32:42 2009 From: nad at acm.org (Ned Deily) Date: Tue, 29 Sep 2009 19:32:42 -0700 Subject: [Python-Dev] test_thread tracebacks References: <20090930014310.78E571250FE5@montanaro.dyndns.org> <760DC0C1-64AB-491E-8C7F-7257249049D8@python.org> Message-ID: In article <760DC0C1-64AB-491E-8C7F-7257249049D8 at python.org>, Barry Warsaw wrote: > On Sep 29, 2009, at 9:43 PM, skip at pobox.com wrote: > > It's been awhile since I rebuilt Python and ran the test suite. This > > evening I noticed this on my Mac (OS X 10.5): It's not a new problem: http://bugs.python.org/issue6186 -- Ned Deily, nad at acm.org From guido at python.org Wed Sep 30 04:48:04 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 29 Sep 2009 19:48:04 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <4AC2AF50.6010806@gmail.com> Message-ID: On Tue, Sep 29, 2009 at 7:12 PM, R. David Murray wrote: > On Wed, 30 Sep 2009 at 11:07, Nick Coghlan wrote: >> >> At the risk of bikeshedding a bit, I'm still somewhat uncomfortable with >> the "net.network" and "net.ip" attribute names. RDMs example application >> elicited the reason for that discomfort pretty well: the current naming >> seems like an invitation to write code using 'net.ip' that should have >> used 'net.network' instead. Such code will then work correctly most of >> the time (i.e. when only given normalised IPNetwork objects) but will >> fail when given a denormalised one. >> >> I believe that discomfort could be eliminated best by changing the name >> of the ".ip" attribute to ".host_ip" to make it clear that it is >> referring to the IP address of the host that was used to derive the >> network definition rather than referring to the network ID itself. >> Shortening ".network" to ".net_ip" would also help (this latter change >> would also eliminate the mental disconnect caused by an attribute called >> .network returning an IPAddress instance). > > +1 +1 on that it's a bikeshed. +0 on renaming .ip to something longer. -0 on renaming .network. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nad at acm.org Wed Sep 30 04:50:14 2009 From: nad at acm.org (Ned Deily) Date: Tue, 29 Sep 2009 19:50:14 -0700 Subject: [Python-Dev] test_thread tracebacks References: <20090930014310.78E571250FE5@montanaro.dyndns.org> <760DC0C1-64AB-491E-8C7F-7257249049D8@python.org> Message-ID: In article , Ned Deily wrote: > In article <760DC0C1-64AB-491E-8C7F-7257249049D8 at python.org>, > Barry Warsaw wrote: > > On Sep 29, 2009, at 9:43 PM, skip at pobox.com wrote: > > > It's been awhile since I rebuilt Python and ran the test suite. This > > > evening I noticed this on my Mac (OS X 10.5): > It's not a new problem: > > http://bugs.python.org/issue6186 ... and minutes after I posted this, I saw another occurrence of it, this time with 2.6.3rc1 on 10.5 ppc, so, whatever the problem or severity, it is still around in 2.6, at least. -- Ned Deily, nad at acm.org From stephen at xemacs.org Wed Sep 30 05:17:49 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 30 Sep 2009 12:17:49 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC29632.9050903@canterbury.ac.nz> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <4AC29632.9050903@canterbury.ac.nz> Message-ID: <87ws3hjf36.fsf@uwakimon.sk.tsukuba.ac.jp> Greg Ewing writes: > Peter Moody wrote: > > > I've mentioned at least once (and others have mentioned as well) that > > it's very common, when describing the ip address/prefix of a NIC to > > write, "192.168.1.100/24" > > Yes, but if you want to *retain* all of that information, > the object you have isn't just a "network", it's more > like "network plus the address of its interface". So > some people think that using the name "IPNetwork" for > this object is confusing. Especially since to my mind networks don't have interfaces. Hosts have interfaces. "IPInterface", anyone? ... no, let's not go there, stick a fork in it, it's done. From stephen at xemacs.org Wed Sep 30 05:24:29 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 30 Sep 2009 12:24:29 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8517e9350909291128s2a4ce554qecaa8f8fb1b7488e@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <8517e9350909290836h6fbe26bavea31287d64293170@mail.gmail.com> <873a65li1r.fsf@uwakimon.sk.tsukuba.ac.jp> <8517e9350909291128s2a4ce554qecaa8f8fb1b7488e@mail.gmail.com> Message-ID: <87vdj1jes2.fsf@uwakimon.sk.tsukuba.ac.jp> Peter Moody writes: > > ?> this is interesting. I would be open to doing this and making __lt__, > > ?> __gt__, __eq__ and friends do the non-ip comparing by default. would > > ?> this be sufficient or does the fact that the network has the .ip > > ?> attribute break something pythonic? > > > > What do you mean, "by default"? ?Surely you don't mean an equality > > method with some sort of flag!? > > no, I mean setting it up so that if I want to compare two network > objects and take into account their .ip attribute, that I use a > .compare_with_extra_bits() method or something. __eq__, __ne__, > __lt__, __le__, __gt__, __ge__ (any more?) would all only compare > .network and .prefixlen (as you have in your example). OK. My position is that I'd appreciate it if you would review some of the existing bikeshedding to see if there are any acceptable tweaks to make more people a little happier, but as of now I'm +1 on the design. From martin at v.loewis.de Wed Sep 30 05:15:49 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 30 Sep 2009 05:15:49 +0200 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: Message-ID: <4AC2CD65.30905@v.loewis.de> Steven Bethard wrote: > There's a lot of code already out there (in the standard library and > other places) that uses %-style formatting, when in Python 3.0 we > should be encouraging {}-style formatting. I don't agree that we should do that. I see nothing wrong with using % substitution. > We should really provide some sort of transition plan. -1. > Consider an example from the logging > docs: > > logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") > > We'd like to support both this style as well as the following style: > > logging.Formatter() Now, that's different. IIUC, you are not actually performing the substitution here, but only at a later place. So changing to the new formatting mechanism is an API change. I agree that the new placeholder syntax needs to be supported. > * Add a new class, NewFormatter, which uses the {}-style formatting. > This is ugly because it makes the formatting we're trying to encourage > look like the alternative implementation instead of the standard one. It's also ugly because the class has the word "new" in its name, which no class should ever have. In a few years, it would still be around, but not new anymore. > * Have Formatter convert all %-style formatting strings to {}-style > formatting strings (automatically). This still involves some guessing, > and involves some serious hacking to translate from one to the other > (maybe it wouldn't even always be possible?). But at least we'd only > be using {}-style formatting under the hood. I don't see the point of having a converter. The tricky part, as you say, is the guessing. Whether the implementation then converts the string or has two alternative formatting algorithms is an implementation detail. I would favor continued use of the actual % substitution code. I would propose that the format argument gets an argument name, according to the syntax it is written in. For PEP 3101 format, I would call the argument "format" (like the method name of the string type), i.e. logging.Formatter( format="{asctime} - {name} - {levelname} - {message}") For the % formatting, I suggest "dicttemplate" (assuming that you *have* to use dictionary %(key)s style currently). The positional parameter would also mean dicttemplate, and would be deprecated (eventually requiring a keyword-only parameter). Regards, Martin From nad at acm.org Wed Sep 30 06:29:32 2009 From: nad at acm.org (Ned Deily) Date: Tue, 29 Sep 2009 21:29:32 -0700 Subject: [Python-Dev] Python 2.6.3 References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> Message-ID: In article <9D506035-7C2D-4929-A134-E88EEB7B7D9E at python.org>, Barry Warsaw wrote: > On Sep 9, 2009, at 1:29 PM, Ned Deily wrote: > > The recent release of OS X 10.6 (Snow Leopard) has triggered a fair > > amount of 2.6 bug tracker activity, since 10.6 now includes 2.6 > > (2.6.1) > > and a 64-bit version at that. A number of patches have either just > > been checked-in over the past couple of weeks or are getting some > > exposure before check-in. Given the timing and the (appropriate) > > infrequency of 2.6.x releases, I think it would be unfortunate to push > > 2.6.3 out the door without ensuring that it works well on 10.6. > > Therefore, I propose that 2.6.3 should have 10.6 compatibility as a > > "release goal". > > Without trying to put Ronald on the spot (too much!), it would be a > > good > > idea to get his assessment where things stand wrt 2.6 on 10.6 before > > setting a final release date. > > I'm hoping that Python won't have any issues building and running on > 10.6, but I don't have it yet so I can't personally test it out. > > How would you quantify "works well"? Do you have any thoughts on > tests you'd run other than the standard test suite? If 2.6.3 is shown > to pass its test suite on 10.5.x, is that good enough? Are the > specific bug fixes necessary for 10.6? To follow-up, I've now run the usual "standard" OS X installer builds and regression tests using 2.6.3rc1 plus a few additional tests (IDLE and package installation) and have found no regressions from 2.6.2 and verified that some release-blocker problems have indeed been fixed. In particular, Issue6957 regarding extensions module building on OS X 10.6 is fixed as discussed in the issue. The tests involved the target python.org installer configuration: - deployment target = 10.3, 2-way i386/ppc, Tk 10.4, built on 10.5 -> regrtest on 10.4 ppc, 10.5 ppc, 10.6 i386 plus the following additional configurations: - deployment target = 10.3, 2-way i386/ppc, Tk 10.4, built on 10.4: -> regrtest on 10.4 ppc, 10.5 ppc, 10.6 i386 - deployment target = 10.5, 4-way i386/x86_64/ppc/ppc64, Tk 10.4, built on 10.5: -> regrtest on 10.5 ppc, 10.6 i386, 10.6 x86_64 plus, a quick check of the currently unsupported configuration, with no new regressions noted: - deployment target = 10.6, 2-way i386/x86_64, Tk 10.5, built on 10.6: -> regrtest on 10.6 x86_64 As noted in various tracker issues, there are still a few unresolved issues with universal builds built on 10.6 (e.g. 32-bit vs 64-bit selection, IDLE with Tk 8.5) which is why this last installer build configuration is currently unsupported. In my opinion, the standard python.org OS X installer for 2.6.3 now "works well" on 10.4, 10.5, and 10.6 and is ready for release. -- Ned Deily, nad at acm.org From steven.bethard at gmail.com Wed Sep 30 06:35:10 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Tue, 29 Sep 2009 21:35:10 -0700 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: <4AC2CD65.30905@v.loewis.de> References: <4AC2CD65.30905@v.loewis.de> Message-ID: On Tue, Sep 29, 2009 at 8:15 PM, "Martin v. L?wis" wrote: > Steven Bethard wrote: >> Consider an example from the logging >> docs: >> >> logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s") >> >> We'd like to support both this style as well as the following style: >> >> logging.Formatter() > > Now, that's different. IIUC, you are not actually performing the > substitution here, but only at a later place. > > So changing to the new formatting mechanism is an API change. > I agree that the new placeholder syntax needs to be supported. Just to be clear, I don't think logging is the only place these kind of things happen. Some others I found looking around: BaseHTTPServer.BaseHTTPRequestHandler.error_message_format http://docs.python.org/library/basehttpserver.html#BaseHTTPServer.BaseHTTPRequestHandler.error_message_format BaseHTTPServer.BaseHTTPRequestHandler.log_message http://docs.python.org/3.1/library/http.server.html#http.server.BaseHTTPRequestHandler.log_message email.generator.DecodedGenerator http://docs.python.org/library/email.generator.html#email.generator.DecodedGenerator There may be more. > I would propose that the format argument gets an argument name, > according to the syntax it is written in. For PEP 3101 format, > I would call the argument "format" (like the method name of the > string type), i.e. > > logging.Formatter( > ?format="{asctime} - {name} - {levelname} - {message}") > > For the % formatting, I suggest "dicttemplate" (assuming that > you *have* to use dictionary %(key)s style currently). > > The positional parameter would also mean dicttemplate, and > would be deprecated (eventually requiring a keyword-only > parameter). This is a nice solution for the cases where we can be confident that the parameter is currently only used positionally. However, at least in Python 3.1, "fmt" is already documented as a keyword parameter: http://docs.python.org/3.1/library/logging.html#logging.Formatter I guess we could follow the same approach though, and have fmt= be the %-style formatting, and use some other keyword argument for {}-style formatting. We've got a similar problem for the BaseHTTPRequestHandler.error_message_format attribute. I guess we'd want to introduce some other attribute which is the error message format for the {}-style formatting? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From mhagger at alum.mit.edu Wed Sep 30 06:36:42 2009 From: mhagger at alum.mit.edu (Michael Haggerty) Date: Wed, 30 Sep 2009 06:36:42 +0200 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> Message-ID: <4AC2E05A.70904@alum.mit.edu> Steven Bethard wrote: > On Tue, Sep 29, 2009 at 1:31 PM, Paul Moore wrote: >> 2009/9/28 Yuvgoog Greenle : >>> 1. There is no chance of the script killing itself. In argparse and optparse >>> exit() is called on every parsing error (btw because of this it sucks to >>> debug parse_args in an interpreter). > [...] > > This is behavior that argparse inherits from optparse, but I believe > it's still what 99.9% of users want. If you're writing a command line > interface, you don't want a stack trace when there's an error message > (which is what you'd get if argparse just raised exceptions) you want > an exit with an error code. That's what command line applications are > supposed to do. > > If you're not using argparse to write command line applications, then > I don't feel bad if you have to do a tiny bit of extra work to take > care of that use case. In this particular situation, all you have to > do is subclass ArgumentParser and override exit() to do whatever you > think it should do. > >>> 2. There is no chance the parser will print things I don't want it to print. > [...] > > There is only a single method in argparse that prints things, > _print_message(). So if you want it to do something else, you can > simply override it in a subclass. I can make that method public if > this is a common use case. Instead of forcing the user to override the ArgumentParser class to change how errors are handled, I suggest adding a separate method ArgumentParser.parse_args_with_exceptions() that raises exceptions instead of writing to stdout/stderr and never calls sys.exit(). Then implement ArgumentParser.parse_args() as a wrapper around parse_args_with_exceptions(): class ArgparseError(Exception): """argparse-specific exception type.""" pass class ArgumentError(ArgparseError): # ... class ArgumentParser(...): # ... def parse_args_with_exceptions(...): # like the old parse_args(), except raises exceptions instead of # writing to stdout/stderr or calling sys.exit()... def parse_args(self, *args, **kwargs): try: self.parse_args_with_exceptions(*args, **kwargs) except ArgparseError as e: self.print_usage(_sys.stderr) self.exit(status=2, message=(_('%s: error: %s\n') % (self.prog, e,))) # perhaps catch other exceptions that need special handling... def error(self, message): raise ArgparseError(message) The exception classes should hold enough information to be useful to non-command-line users, and obviously contain error messages that are output to stderr by default. This would allow non-command-line users to call parse_args_with_exceptions() and handle the exceptions however they like. Michael From foom at fuhm.net Wed Sep 30 07:04:05 2009 From: foom at fuhm.net (James Y Knight) Date: Wed, 30 Sep 2009 01:04:05 -0400 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: I'm resending a message I sent in June, since it seems the same thread has come up again, and I don't believe anybody actually responded (positively or negatively) to the suggestion back then. http://mail.python.org/pipermail/python-dev/2009-June/090176.html On Jun 21, 2009, at 5:40 PM, Eric Smith wrote: > I've basically come to accept that %-formatting can never go away, > unfortunately. There are too many places where %-formatting is used, > for example in logging Formatters. %-formatting either has to exist > or it has to be emulated. It'd possibly be helpful if there were builtin objects which forced the format style to be either newstyle or oldstyle, independent of whether % or format was called on it. E.g. x = newstyle_formatstr("{} {} {}") x % (1,2,3) == x.format(1,2,3) == "1 2 3" and perhaps, for symmetry: y = oldstyle_formatstr("%s %s %s") y.format(1,2,3) == x % (1,2,3) == "1 2 3" This allows the format string "style" decision is to be made external to the API actually calling the formatting function. Thus, it need not matter as much whether the logging API uses % or .format() internally -- that only affects the *default* behavior when a bare string is passed in. This could allow for a controlled switch towards the new format string format, with a long deprecation period for users to migrate: 1) introduce the above feature, and recommend in docs that people only ever use new-style format strings, wrapping the string in newstyle_formatstr() when necessary for passing to an API which uses % internally. 2) A long time later...deprecate str.__mod__; don't deprecate newstyle_formatstr.__mod__. 3) A while after that (maybe), remove str.__mod__ and replace all calls in Python to % (used as a formatting operator) with .format() so that the default is to use newstyle format strings for all APIs from then on. From ncoghlan at gmail.com Wed Sep 30 07:48:32 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 15:48:32 +1000 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: <4AC2F130.3060202@gmail.com> James Y Knight wrote: > I'm resending a message I sent in June, since it seems the same thread > has come up again, and I don't believe anybody actually responded > (positively or negatively) to the suggestion back then. > > http://mail.python.org/pipermail/python-dev/2009-June/090176.html > > On Jun 21, 2009, at 5:40 PM, Eric Smith wrote: >> I've basically come to accept that %-formatting can never go away, >> unfortunately. There are too many places where %-formatting is used, >> for example in logging Formatters. %-formatting either has to exist or >> it has to be emulated. > > It'd possibly be helpful if there were builtin objects which forced the > format style to be either newstyle or oldstyle, independent of whether % > or format was called on it. > > E.g. > x = newstyle_formatstr("{} {} {}") > x % (1,2,3) == x.format(1,2,3) == "1 2 3" > > and perhaps, for symmetry: > y = oldstyle_formatstr("%s %s %s") > y.format(1,2,3) == x % (1,2,3) == "1 2 3" > > This allows the format string "style" decision is to be made external to > the API actually calling the formatting function. Thus, it need not > matter as much whether the logging API uses % or .format() internally -- > that only affects the *default* behavior when a bare string is passed in. > > This could allow for a controlled switch towards the new format string > format, with a long deprecation period for users to migrate: > > 1) introduce the above feature, and recommend in docs that people only > ever use new-style format strings, wrapping the string in > newstyle_formatstr() when necessary for passing to an API which uses % > internally. > 2) A long time later...deprecate str.__mod__; don't deprecate > newstyle_formatstr.__mod__. > 3) A while after that (maybe), remove str.__mod__ and replace all calls > in Python to % (used as a formatting operator) with .format() so that > the default is to use newstyle format strings for all APIs from then on. I must have missed this suggestion when it went past the first time. I certainly like this approach - it has the virtue of only having to solve the problem once, and then application developers can use it to adapt any existing use of %-mod formatting to str.format formatting. Something like: class formatstr(str): def __mod__(self, other): if isinstance(other, dict): return self.format(**dict) if isinstance(other, tuple) return self.format(*other) return self.format(other) APIs that did their own parsing based on %-formatting codes would still break, as would any that explicitly called "str" on the object (or otherwise stripped the subclass away, such as via "'%s' % fmt"), but most things should pass a string subclass through transparently. I wouldn't bother with a deprecation plan for 'normal' %-formatting though. I don't think it is going to be practical to actually get rid of that short of creating Python 4.0. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From steven.bethard at gmail.com Wed Sep 30 09:35:18 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 30 Sep 2009 00:35:18 -0700 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: On Tue, Sep 29, 2009 at 10:04 PM, James Y Knight wrote: > It'd possibly be helpful if there were builtin objects which forced the > format style to be either newstyle or oldstyle, independent of whether % or > format was called on it. > > E.g. > x = newstyle_formatstr("{} {} {}") > x % (1,2,3) == x.format(1,2,3) == "1 2 3" > > and perhaps, for symmetry: > y = oldstyle_formatstr("%s %s %s") > y.format(1,2,3) == x % (1,2,3) == "1 2 3" > > This allows the format string "style" decision is to be made external to the > API actually calling the formatting function. Thus, it need not matter as > much whether the logging API uses % or .format() internally -- that only > affects the *default* behavior when a bare string is passed in. > > This could allow for a controlled switch towards the new format string > format, with a long deprecation period for users to migrate: > > 1) introduce the above feature, and recommend in docs that people only ever > use new-style format strings, wrapping the string in newstyle_formatstr() > when necessary for passing to an API which uses % internally. > 2) A long time later...deprecate str.__mod__; don't deprecate > newstyle_formatstr.__mod__. > 3) A while after that (maybe), remove str.__mod__ and replace all calls in > Python to % (used as a formatting operator) with .format() so that the > default is to use newstyle format strings for all APIs from then on. So I understand how this might help a user to move from %-style formatting to {}-style formatting, but it's still not clear to me how to use this to evolve an API. In particular, if the goal is for the API to move from accepting %-style format strings to {}-style format strings, how should that API use newstyle_formatstr or oldstyle_formatstr to make this transition? Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From dickinsm at gmail.com Wed Sep 30 10:31:25 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 30 Sep 2009 09:31:25 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC2A9D7.4080603@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> Message-ID: <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> On Wed, Sep 30, 2009 at 1:44 AM, Nick Coghlan wrote: > Martin v. L?wis wrote: >>> I would say that there certainly are precedents in other areas for >>> keeping the information about the input form around. For example, >>> occasionally it would be handy if parsing a hex integer returned an >>> object that was compatible with other integers but somehow kept a hint >>> that would cause printing it to use hex by default. >> >> At the risk of bringing in false analogies: it seems that Python >> typically represents values of some type in their canonical form, >> rather than remembering the form in which they arrived in the program: >> - integer values "forget" how many preceding zeroes they have >> - string literals forget which of the characters had been escaped, and >> ? whether the string was single- or double-quoted >> - floating point values forget a lot more about their literal >> ? representation (including even the literal decimal value) >> >> I guess a close case would be rational numbers: clearly, 3?2 == 6?4; >> would a Python library still remember (and repr) the original numerator >> and denominator? > > For a concrete example of an object which remembers details about its > creation that it ignores when determining equality, we have decimal.Decimal: > > .>> from decimal import Decimal as d > .>> x = d("3.0") > .>> y = d("3.00") > .>> x > d("3.0") > .>> y > d("3.00") > .>> repr(x) == repr(y) > False > .>> x.as_tuple() == y.as_tuple() > False > .>> x == y > True [snipped] [More on the Decimal analogy below.] Please could someone who understands the uses of IPNetwork better than I do explain why the following wouldn't be a significant problem, if __eq__ and __hash__ were modified to disregard the .ip attribute as suggested: >>> linus = IPv4Network('172.16.200.1/24') >>> snoopy = IPv4Network('172.16.200.3/24') >>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'} >>> fqdn[linus] # expecting 'linus.peanuts.net' 'snoopy.peanuts.net' Is this just a problem of education, teaching the users not to abuse IPv4Network this way? Or is this just an unlikely use of IPv4Network? Or have I misunderstood the proposal altogether? As for Decimal, I see that as another whole kettle of tuna: equality for Decimal couldn't reasonably have been done any other way---if it weren't mandated by the standard, there would still be a very strong expectation that == would mean numeric equality. That is, I see the == operator as having two distinct but mostly compatible uses in Python: it's used for numeric equality, *and* it's used as the equivalence relation for determining container membership. Mostly these two different meanings get along fine, though they lead to some fun when trying to ensure that x == y implies hash(x) == hash(y) for x and y two different numeric types. But since Decimals and floats aren't used as set elements or dict keys that often, the fact that you can't store Decimal('1.0') and Decimal('1.00') together in a set doesn't often get in the way. I'd expect putting IPv4Network objects in a set or dict to be more common. Mark From p.f.moore at gmail.com Wed Sep 30 11:03:15 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 30 Sep 2009 10:03:15 +0100 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <4AC2A882.7070108@canterbury.ac.nz> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> <4AC2A882.7070108@canterbury.ac.nz> Message-ID: <79990c6b0909300203u20e719e5w1d492d124b96d0d2@mail.gmail.com> 2009/9/30 Greg Ewing : > Paul Moore wrote: >> >> I'd rather argparse (or any library function) >> didn't call sys.exit on my behalf - it should raise an exception. > > Actually, sys.exit() *does* raise an exception (i.e. > SystemExit) that you can catch if you want. That's a good point... Paul From p.f.moore at gmail.com Wed Sep 30 11:18:38 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 30 Sep 2009 10:18:38 +0100 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: <79990c6b0909300218o50cf6car91216b512266b0a2@mail.gmail.com> 2009/9/30 Steven Bethard : > On Tue, Sep 29, 2009 at 10:04 PM, James Y Knight wrote: >> It'd possibly be helpful if there were builtin objects which forced the >> format style to be either newstyle or oldstyle, independent of whether % or >> format was called on it. > So I understand how this might help a user to move from %-style > formatting to {}-style formatting, but it's still not clear to me how > to use this to evolve an API. In particular, if the goal is for the > API to move from accepting %-style format strings to {}-style format > strings, how should that API use newstyle_formatstr or > oldstyle_formatstr to make this transition? IIUC, the API doesn't change. It's just that the internal code goes as follows: 1. (Current) Use %. str and oldformat objects work as now, newformat objects work (using .format). 2. Convert the code to use .format - oldformat and newformat objects work as before, str objects must be in new format. The problem is, there's a point at which bare str arguments change behaviour. So unless people wrap their arguments when calling the API, there's still a point when things break (albeit with a simple workaround available). So maybe we need transition steps - wrap bare str objects in oldformat classes, and warn, then wrap str objects in newformat objects and warn, then stop wrapping. That sounds to me like "normal" usage (bare strings) will be annoyingly painful for a substantial transition period. I'm assuming that the oldformat and newformat classes are intended to be transitional things, and there's no intention that users should be using the wrapper objects always. (And of course better names than "oldformat" and "newformat" are needed - as Martin pointed out, having "old" and "new" in the names is inappropriate). Otherwise, I'm a strong -1 on the whole idea. Paul. From p.f.moore at gmail.com Wed Sep 30 11:52:29 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 30 Sep 2009 10:52:29 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> Message-ID: <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> 2009/9/30 Mark Dickinson : > Please could someone who understands the uses of IPNetwork better than > I do explain why the following wouldn't be a significant problem, if __eq__ > and __hash__ were modified to disregard the .ip attribute as suggested: > >>>> linus = IPv4Network('172.16.200.1/24') >>>> snoopy = IPv4Network('172.16.200.3/24') >>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'} >>>> fqdn[linus] ?# expecting 'linus.peanuts.net' > 'snoopy.peanuts.net' I certainly don't understand IPv4Network better than you :-) But that just looks wrong to me - linus and snoopy are hosts not networks, so making them IPv4Network classes seems wrong. I'd instinctively make them IPv4Address objects (which, I believe, would work). Paul. From dickinsm at gmail.com Wed Sep 30 12:18:24 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 30 Sep 2009 11:18:24 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> Message-ID: <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> On Wed, Sep 30, 2009 at 10:52 AM, Paul Moore wrote: > 2009/9/30 Mark Dickinson : >> Please could someone who understands the uses of IPNetwork better than >> I do explain why the following wouldn't be a significant problem, if __eq__ >> and __hash__ were modified to disregard the .ip attribute as suggested: >> >>>>> linus = IPv4Network('172.16.200.1/24') >>>>> snoopy = IPv4Network('172.16.200.3/24') >>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'} >>>>> fqdn[linus] ?# expecting 'linus.peanuts.net' >> 'snoopy.peanuts.net' > > I certainly don't understand IPv4Network better than you :-) But that > just looks wrong to me - linus and snoopy are hosts not networks, so > making them IPv4Network classes seems wrong. I'd instinctively make > them IPv4Address objects (which, I believe, would work). Okay, so maybe this is an abuse of IPv4Network. But I'd (mis?)understood that the retention of the .ip attribute was precisely a convenience to allow this sort of use. If not, then what's it for? I've read the PEP and almost all of this thread, but I can't help feeling I'm still missing something. If someone could point out the obvious to me I'd be grateful. I don't have any opinion on whether the ip attribute should be retained or not; but retaining it *and* ignoring it in comparisons just seems a bit odd. Mark From eric at trueblade.com Wed Sep 30 13:15:45 2009 From: eric at trueblade.com (Eric Smith) Date: Wed, 30 Sep 2009 07:15:45 -0400 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: <4AC2CD65.30905@v.loewis.de> References: <4AC2CD65.30905@v.loewis.de> Message-ID: <4AC33DE1.3040304@trueblade.com> Martin v. L?wis wrote: > Steven Bethard wrote: >> There's a lot of code already out there (in the standard library and >> other places) that uses %-style formatting, when in Python 3.0 we >> should be encouraging {}-style formatting. > > I don't agree that we should do that. I see nothing wrong with using > % substitution. It's a maintenance burden. There are several outstanding bugs with it, admittedly not of any great significance. I've been putting time into fixing at least one of them. When Mark and I did short-float-repr, at least half of my time was consumed with %-formatting, mostly because of how it does memory management. On the plus side, %-formatting is (and always will be) faster than str.format(). Its very limitations make it possible for it to be fast. I'd note that PEP 3101 calls str.format() a replacement for %-formatting, not an alternate mechanism to achieve the same end. >> * Add a new class, NewFormatter, which uses the {}-style formatting. >> This is ugly because it makes the formatting we're trying to encourage >> look like the alternative implementation instead of the standard one. > > It's also ugly because the class has the word "new" in its name, which > no class should ever have. In a few years, it would still be around, > but not new anymore. > >> * Have Formatter convert all %-style formatting strings to {}-style >> formatting strings (automatically). This still involves some guessing, >> and involves some serious hacking to translate from one to the other >> (maybe it wouldn't even always be possible?). But at least we'd only >> be using {}-style formatting under the hood. > > I don't see the point of having a converter. The tricky part, as you > say, is the guessing. Whether the implementation then converts the > string or has two alternative formatting algorithms is an implementation > detail. I would favor continued use of the actual % substitution > code. Having a converter and guessing are 2 distinct issues. I believe a convert from %-formatting specification strings to str.format() strings is possible. I point this out not because I think a converter solves this problem, but because in the past there's been a debate on whether a converter is even possible. Eric. From dickinsm at gmail.com Wed Sep 30 13:33:03 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 30 Sep 2009 12:33:03 +0100 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC33BA1.8050203@gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> <4AC33BA1.8050203@gmail.com> Message-ID: <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> On Wed, Sep 30, 2009 at 12:06 PM, Nick Coghlan wrote: > Mark Dickinson wrote: >> Okay, so maybe this is an abuse of IPv4Network. ?But I'd (mis?)understood >> that the retention of the .ip attribute was precisely a convenience to allow >> this sort of use. ?If not, then what's it for? ?I've read the PEP and almost >> all of this thread, but I can't help feeling I'm still missing something. ?If >> someone could point out the obvious to me I'd be grateful. > > You're not missing anything that I'm aware of - unlike the use case for > accepting a denormalised network definition in the IPNetwork constructor > (which has been made quite clear in the list discussion, even if it is > still a bit vague in the PEP), the use case for *retaining* the host > information on the network object hasn't been well articulated at all. > > The closest anyone has come to describing a use case is an indirect > comment via Guido that leaving out the attribute would involve real code > having to find somewhere else to stash the original address details > (e.g. by passing around an IPAddres/IPNetwork tuple rather than just an > IPNetwork object). Ah, thanks---I'd missed that bit. So the .ip attribute is mainly for backwards compatibility with existing uses/users of ipaddr. I guess that makes sense, then. In particular, if it's suggested that new code shouldn't make use of the .ip attribute, then the list/dict membership problems described above can't arise. > However, while I'd still be a little happier if the .ip attribute went > away all together and another means was found to conveniently associate > an IPAddress and an IPNetwork, keeping it doesn't bother me anywhere > near as much as having network equivalence defined in terms of something > other than the network ID and the netmask. Makes sense. Thanks, Mark From stephen at xemacs.org Wed Sep 30 14:13:38 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Wed, 30 Sep 2009 21:13:38 +0900 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> Message-ID: <87pr98k4ul.fsf@uwakimon.sk.tsukuba.ac.jp> Mark Dickinson writes: > >> Please could someone who understands the uses of IPNetwork > >> better than I do explain why the following wouldn't be a > >> significant problem, if __eq__ and __hash__ were modified to > >> disregard the .ip attribute as suggested: > >>>>> linus = IPv4Network('172.16.200.1/24') > >>>>> snoopy = IPv4Network('172.16.200.3/24') > >>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'} > >>>>> fqdn[linus] ?# expecting 'linus.peanuts.net' > >> 'snoopy.peanuts.net' Well, for one thing it would be broken anyway if __eq__ and __hash__ paid attention to the .ip attribute, because they *also* pay attention to the .prefixlen (and I guess .network, redundantly) attribute. Presumably the use in real code would not be fqdn[linus], you already know about linus. Rather, it would be equivalent to say >>>>> fqdn[IPv4Network('172.16.200.1/16')] KeyError where an IP and network parsed out of some packet were used to construct the key. Of course it would work almost all the time, and only break when, say, old-style class was used to infer the prefix length, and was wrong. So, no, I don't think we want to do this. From skip at pobox.com Wed Sep 30 14:18:11 2009 From: skip at pobox.com (skip at pobox.com) Date: Wed, 30 Sep 2009 07:18:11 -0500 Subject: [Python-Dev] test_thread tracebacks In-Reply-To: <760DC0C1-64AB-491E-8C7F-7257249049D8@python.org> References: <20090930014310.78E571250FE5@montanaro.dyndns.org> <760DC0C1-64AB-491E-8C7F-7257249049D8@python.org> Message-ID: <19139.19587.615917.647283@montanaro.dyndns.org> >> It's been awhile since I rebuilt Python and ran the test suite. This >> evening I noticed this on my Mac (OS X 10.5): Sorry, trunk. Skip From barry at python.org Wed Sep 30 14:21:18 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Sep 2009 08:21:18 -0400 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: <4AC2CD65.30905@v.loewis.de> References: <4AC2CD65.30905@v.loewis.de> Message-ID: On Sep 29, 2009, at 11:15 PM, Martin v. L?wis wrote: > I would propose that the format argument gets an argument name, > according to the syntax it is written in. For PEP 3101 format, > I would call the argument "format" (like the method name of the > string type), i.e. > > logging.Formatter( > format="{asctime} - {name} - {levelname} - {message}") > > For the % formatting, I suggest "dicttemplate" (assuming that > you *have* to use dictionary %(key)s style currently). > > The positional parameter would also mean dicttemplate, and > would be deprecated (eventually requiring a keyword-only > parameter). Although I hate the name 'dicttemplate', this seems like the best solution to me. Maybe it's good that 'dicttemplate' is so ugly though so that people will naturally prefer 'format' :). But I like this because there's not really any magic, it's explicit, and the decision is made by the coder at the call site. The implementation does not need to guess at all. If this is adopted, it should become a common idiom across Python so that once you've learned how to transition between the format strings, you pretty much know how to do it for any supporting API. So we should adopt it across all of the standard library. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From p.f.moore at gmail.com Wed Sep 30 14:29:11 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 30 Sep 2009 13:29:11 +0100 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: <79990c6b0909300529u4383e02esa21b464b957fb015@mail.gmail.com> 2009/9/30 Barry Warsaw : > Although I hate the name 'dicttemplate', this seems like the best solution > to me. ?Maybe it's good that 'dicttemplate' is so ugly though so that people > will naturally prefer 'format' :). ?But I like this because there's not > really any magic, it's explicit, and the decision is made by the coder at > the call site. ?The implementation does not need to guess at all. Ignoring the "make it ugly to persuade people not to use it" aspect, why not just use 'template'? Paul From barry at python.org Wed Sep 30 14:34:02 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Sep 2009 08:34:02 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> Message-ID: <21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org> On Sep 30, 2009, at 12:29 AM, Ned Deily wrote: > In my opinion, the standard python.org OS X installer for 2.6.3 now > "works well" on 10.4, 10.5, and 10.6 and is ready for release. Fantastic, thanks. Martin's uploaded the Windows binaries so I'll make the announcement now. No commits to the 2.6 tree are allowed without checking with me first (via IRC, bug tracker or email). I'll make an exception for svnmerge bookkeeping. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From barry at python.org Wed Sep 30 14:37:58 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Sep 2009 08:37:58 -0400 Subject: [Python-Dev] Python 2.6.3rc1 available Message-ID: The first (and hopefully last) release candidate for Python 2.6.3 is now available via http://www.python.org/download/releases/2.6.3/ Source releases and Windows binaries are currently available, and Mac OS X binaries should be forthcoming. Nearly 100 bugs have been fixed since 2.6.2. Barring any unforeseen problems, we will make the final 2.6.3 release this Friday, October 2nd. Please give this release candidate a spin and let us know if you encounter any show stopping problems. Enjoy, -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From vinay_sajip at yahoo.co.uk Wed Sep 30 15:27:41 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 30 Sep 2009 13:27:41 +0000 (UTC) Subject: [Python-Dev] Python logging and 1.5.2 compatibility Message-ID: I'm planning to "officially" drop support for Python 1.5.2 in the logging package. When the logging package was introduced in Python 2.3, many Linux distros were shipping 1.5.2 as the system's Python, even though 2.2 had been out for a while. So it seemed important to support 1.5.2 for those sysadmins who wanted to use logging with their system Python. The Linux landscape has changed a bit since then. Most Linux distros ship with much more recent versions of Python, and so I no longer see 1.5.2 support as important. Dropping support for 1.5.2 means that future changes to logging will not be concerned with 1.5.2 compatibility. For example, boolean values which were 0/1 in the logging package will at some point be replaced by False/True, and newer language features will start to be used when changes are made. There are no plans for a specific "cleanup" exercise at the moment. In fact some changes made a while ago inadvertently broke 1.5.2 compatibility, but no-one's complained. So I'm assuming the whole thing is really a non-issue, and this post is just to keep everyone in the picture. A 1.5.2-compatible version of the package is still available via http://www.red-dove.com/python_logging.html if anyone needs it. This version is not actively maintained, but that shouldn't be an issue. Regards, Vinay Sajip From solipsis at pitrou.net Wed Sep 30 15:39:22 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 30 Sep 2009 13:39:22 +0000 (UTC) Subject: [Python-Dev] Python logging and 1.5.2 compatibility References: Message-ID: Le Wed, 30 Sep 2009 13:27:41 +0000, Vinay Sajip a ?crit?: > I'm planning to "officially" drop support for Python 1.5.2 in the > logging package. Thanks for the announcement. So, what is the minimum supported version now? Regards Antoine. From rdmurray at bitdance.com Wed Sep 30 15:41:26 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 30 Sep 2009 09:41:26 -0400 (EDT) Subject: [Python-Dev] Python logging and 1.5.2 compatibility In-Reply-To: References: Message-ID: On Wed, 30 Sep 2009 at 13:27, Vinay Sajip wrote: > I'm planning to "officially" drop support for Python 1.5.2 in the logging > package. What's the minimum version of Python that the logging module now officially supports? --David (RDM) From chris at simplistix.co.uk Wed Sep 30 16:24:32 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 30 Sep 2009 15:24:32 +0100 Subject: [Python-Dev] operator precedence of __eq__, __ne__, etc, if both object have implementations In-Reply-To: <19129.39295.970536.916767@montanaro.dyndns.org> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> Message-ID: <4AC36A20.4060102@simplistix.co.uk> skip at pobox.com wrote: > Dino> For IronPython we wrote a set of tests which go through and define > Dino> the various operator methods in all sorts of combinations on both > Dino> new-style and old-style classes as well as subclasses of those > Dino> classes and then do the comparisons w/ logging. > > It would be very nice if these complex corner cases had a set of test > cases which could be run by all implementations (CPython, Jython, > IronPython, PyPy, etc). I don't know. Maybe the CPython test suite serves > that purpose, but it seems like it would be helpful if this sort of > "validation suite" was maintained as a separate project all implementations > could use and contribute to. +1, and supported by documentation. Something like a manuel-based table test would seem ideal for this: http://packages.python.org/manuel/manuel/table-example.html ..but what are the chances of something like that getting into the python core docs? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Wed Sep 30 16:28:58 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 30 Sep 2009 15:28:58 +0100 Subject: [Python-Dev] sharing stdlib across python implementations In-Reply-To: <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com> Message-ID: <4AC36B2A.6020406@simplistix.co.uk> Frank Wierzbicki wrote: > Talk has started up again on the stdlib-sig list about finding a core > stdlib + tests that can be shared by all implementations, potentially > living apart from CPython. I have volunteered to put together a PEP > on the subject, with Jessie Noller and Brett Canon are helping me out. > When I have something worth showing, I'll start the real PEP process. I'm on on stdlib-sig and I'm afraid I don't have the bandwidth to start on it, but I'd just like to throw in (yet again) that it would be great if the stdlib was actually a set of separate python packages with their own version metadata so that packaging tools could manage them, and upgrade them independently of python packages when there are bug fixes. If that were the case, then pure python packages in the stdlib, of which there are many, *really* could be used across python implementations with no changes whatsoever... The big changes I can see from here would be moving the tests to the packages from the central tests directory, and adding a setup.py file or some other form of metadata providion for each package. Not that big now that I've written it ;-) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From vinay_sajip at yahoo.co.uk Wed Sep 30 16:32:44 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 30 Sep 2009 14:32:44 +0000 (UTC) Subject: [Python-Dev] Python logging and 1.5.2 compatibility References: Message-ID: Antoine Pitrou pitrou.net> writes: > So, what is the minimum supported version now? Well, each version is tested with the release of Python which it's part of, and in theory it would be possible for the code in logging for that release to use features only available in that release of Python. So, I'm not sure if it now makes sense to support a minimum version. When changing the package I intend to be conservative in terms of using the simplest, rather than using the latest language features (say, coroutines) just because they're available. The reason it was done for 1.5.2 was that there was already a community of users of the "standalone" package (which was 1.5.2-compatible) before it got accepted into Python 2.3, and it didn't make sense at the time to maintain two different codebases - one 1.5.2-compatible and one not. Keeping it compatible meant that 1.5.2-dependent users/sysadmins could just copy the whole package and have it work, and take advantage of bug-fixes and enhancements as they appeared. PEP 291 (Backward Compatibility for Standard Library) mentions logging as having 1.5.2 compatibility, and I'm thinking that it can be removed from that PEP altogether. I don't think it's still necessary to hold back from using new features just to support 1.5.2, given that a working implementation is available if any 1.5.2 user should need it, and that the 1.5.2-dependent community will be pretty minimal. A lot of water has passed under the bridge. There are some new features which have been added in recent versions (say, WatchedFileHandler in 2.6, NullHandler in 2.7) and these should still work if copied and pasted into a codebase which runs with an older version of Python. Certain projects which need to support multiple Python versions - Django, say - can take advantage of this by e.g. offering a copy of these handlers in their own codebase for use when constrained to work with an older Python version. Regards, Vinay Sajip From steve at pearwood.info Wed Sep 30 16:34:27 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 1 Oct 2009 00:34:27 +1000 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: Message-ID: <200910010034.27427.steve@pearwood.info> On Wed, 30 Sep 2009 03:04:05 pm James Y Knight wrote: > It'd possibly be helpful if there were builtin objects which forced > the format style to be either newstyle or oldstyle, independent of > whether % or format was called on it. > > E.g. > x = newstyle_formatstr("{} {} {}") > x % (1,2,3) == x.format(1,2,3) == "1 2 3" People will want this formatstr object to behave like strings, with concatenation, slicing, etc.: >>> x = newstyle_formatstr("x{} {} : ") >>> y = newstyle_formatstr("{}") >>> (x[1:] + y) % (1, 2, 3) '1 2 : 3' Instead of having to support one type with %-formatting and {}-formatting (str), now the std lib will have two classes with %-formatting and {}-formatting. How is this an improvement? Moving along, let's suppose the newstyle_formatstr is introduced. What's the intention then? Do we go through the std lib and replace every call to (say) somestring % args with newstyle_formatstr(somestring) % args instead? That seems terribly pointless to me -- it does nothing about getting rid of % but adds a layer of indirection which slows down the code. Things are no better if the replacement code is: newstyle_formatstr(somestring).format(*args) (or similar). If we can do that, then why not just go all the way and use this as the replacement instead? somestring.format(*args) > and perhaps, for symmetry: > y = oldstyle_formatstr("%s %s %s") > y.format(1,2,3) == x % (1,2,3) == "1 2 3" Now we have three classes that support both % and {} formatting. Great. [...] > This could allow for a controlled switch towards the new format > string format, with a long deprecation period for users to migrate: > > 1) introduce the above feature, and recommend in docs that people > only ever use new-style format strings, wrapping the string in > newstyle_formatstr() when necessary for passing to an API which uses > % internally. And how are people supposed to know what the API uses internally? Personally, I think your chances of getting people to write: logging.Formatter(newstyle_formatstr("%(asctime)s - %(name)s - %(level)s - %(msg)s")) instead of logging.Formatter("%(asctime)s - %(name)s - %(level)s - %(msg)s") is slim to none -- especially when the second call still works. You'd better off putting the call to newstyle_formatstr() inside logging.Formatter, and not even telling the users. Instead of wrapping strings in a class that makes .__mod__() and .format() behave the same, at some cost on every call presumably, my preferred approach would be a converter function (perhaps taken from 2to3?) which modified strings like "%(asctime)s" to "{asctime}". That cost gets paid *once*, rather than on every call. (Obviously the details will need to be ironed out, and it will depend on the external API. If the external API depends on the caller using % explicitly, then this approach may not work.) > 2) A long time later...deprecate str.__mod__; How long? I hope that's not until I'm dead and buried. -- Steven D'Aprano From solipsis at pitrou.net Wed Sep 30 16:43:07 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 30 Sep 2009 14:43:07 +0000 (UTC) Subject: [Python-Dev] sharing stdlib across python implementations References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com> <4AC36B2A.6020406@simplistix.co.uk> Message-ID: Chris Withers simplistix.co.uk> writes: > > I'm on on stdlib-sig and I'm afraid I don't have the bandwidth to start > on it, but I'd just like to throw in (yet again) that it would be great > if the stdlib was actually a set of separate python packages with their > own version metadata so that packaging tools could manage them, and > upgrade them independently of python packages when there are bug fixes. This sounds like a bad idea to me. Each Python release is tested and debugged as a whole. If you have a lot of possible combinations (module A version 1.1 with module B version 1.2, etc.), it becomes impossible for us to ensure proper QA for the whole and as a result the quality might become lower, rather than higher. (of course, if we rigorously enforce APIs and preserve compatibility, this might not be a real issue; but our compatibility story is a bit irregular, IMHO) Regards Antoine. From chris at simplistix.co.uk Wed Sep 30 16:45:56 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 30 Sep 2009 15:45:56 +0100 Subject: [Python-Dev] sharing stdlib across python implementations In-Reply-To: References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com> <4AC36B2A.6020406@simplistix.co.uk> Message-ID: <4AC36F24.5020107@simplistix.co.uk> Antoine Pitrou wrote: > This sounds like a bad idea to me. Each Python release is tested and debugged as > a whole. If you have a lot of possible combinations (module A version 1.1 with > module B version 1.2, etc.), it becomes impossible for us to ensure proper QA > for the whole and as a result the quality might become lower, rather than higher. I'd certainly still see there being "blessed" python releases. (The Zope community does this in the form of "known good sets" and these, for me, would be "the python releases") However, it's pretty frustrating to not be able to upgrade a pure-python package with a critical bug (as happened recently to me with httplib) because it's part of the stdlib. It's also frustrating to have to if-then-else about a package or project's dependencies when using dependency management tools like buildout when libraries such as elementtree and argparse become part of the stdlib when they weren't before... > (of course, if we rigorously enforce APIs and preserve compatibility, this might > not be a real issue; but our compatibility story is a bit irregular, IMHO) Breaking it apart like this would *make* the compatibility story better... Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Wed Sep 30 16:57:27 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 30 Sep 2009 15:57:27 +0100 Subject: [Python-Dev] a new setuptools release? In-Reply-To: <20090923204737.A97DC3A4079@sparrow.telecommunity.com> References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> <20090923204737.A97DC3A4079@sparrow.telecommunity.com> Message-ID: <4AC371D7.9000904@simplistix.co.uk> P.J. Eby wrote: > Here's what actually happened, if anyone cares. Tarek and friends > announced a fork of setuptools. I reviewed the work and saw that -- for > the most part -- I was happy with it, and opined as how I might be > willing to bless the the "package inquisition" team as official > maintainers of the 0.6 branch of setuptools, so that I could work on the > fun bits I've long planned for 0.7, but never felt free to start on > while there was so much still needing to be done on 0.6. If this offer is still available, I'd lake to take you up on it. I'd be more than willing to merge changes on the 0.6 distribute branch back into the setuptools codebase and do whatever is needed to get a new setuptools release out. Why? Because there are a *lot* of copies of ez_setup.py and buildout's bootstrap.py that will need replacing if it doesn't happen. I think it'd be better for the python community all round if setuptools just continued in maintenance mode until whatever-ends-up-in-the-core exists and people want to use... I'm happy to submit to whatever supervision is needed for you to trust me to do this, and I promise to be as careful as I can with this. I *know* how important this is and want to make it work... > All I want is for good stuff to happen for setuptools users and Python > users in general, so I don't think all the suspicion and backbiting is > merited. Fine, if that's true, I apologize (even spelled correctly!) for any previous involvement in this, but please help me help you achieve your aims... To put this into a way that makes sense to me: I'm volunteering to keep distribute 0.6 and setuptools 0.6 in sync, no more, no less, and try and keep that as uncontroversial as possible, and get setuptools 0.6 releases out to match distribute 0.6 releases as soon as I can. Again, I feel I need to stress that the *only* reason I want to do this is to bring the benefits of the distribute work to the existing setuptools codebase, with appropriate attribution if that makes a difference. Apologies if any of this is offensive to anyone. For once (really!) I really mean that :-) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From chris at simplistix.co.uk Wed Sep 30 17:04:26 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Wed, 30 Sep 2009 16:04:26 +0100 Subject: [Python-Dev] Distutils ML wrap-up: setup.cfg new format In-Reply-To: References: <94bdd2610909220621u54984c94ucf522dbe1fe68671@mail.gmail.com> <4AB8D910.8050601@simplistix.co.uk> <94bdd2610909221106h868a158yb29bdd77f96eac9@mail.gmail.com> <4AB92ADE.9060700@simplistix.co.uk> <87vdjauoq6.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909230520y5e4d59by7f192137980b7613@mail.gmail.com> <87tyyt4v1m.fsf@uwakimon.sk.tsukuba.ac.jp> <94bdd2610909231000t4bbeb5b4x3ed715dad883414a@mail.gmail.com> Message-ID: <4AC3737A.8020701@simplistix.co.uk> Antoine Pitrou wrote: > As far as I'm concerned, anything which looks intuitive enough (e.g. ini-style) > and not overly complicated is fine. The details of the syntax aren't really > important as long as they make sense, and don't get in the way. This is a variant of "as simple as possible and required but no simpler", which I'm sure is what we're all aiming for. The problem is that it's fiendishly difficult ;-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk From olemis at gmail.com Wed Sep 30 17:18:09 2009 From: olemis at gmail.com (Olemis Lang) Date: Wed, 30 Sep 2009 10:18:09 -0500 Subject: [Python-Dev] sharing stdlib across python implementations In-Reply-To: <4AC36B2A.6020406@simplistix.co.uk> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com> <4AC36B2A.6020406@simplistix.co.uk> Message-ID: <24ea26600909300818h4663eafbqba5971d4b4d3d752@mail.gmail.com> On Wed, Sep 30, 2009 at 9:28 AM, Chris Withers wrote: > Frank Wierzbicki wrote: >> >> Talk has started up again on the stdlib-sig list about finding a core >> stdlib + tests that can be shared by all implementations, potentially >> living apart from CPython. > [...] > > if the > stdlib was actually a set of separate python packages with their own version > metadata so that packaging tools could manage them, and upgrade them > independently of python packages when there are bug fixes. If that were the > case, then pure python packages in the stdlib, of which there are many, > *really* could be used across python implementations with no changes > whatsoever... > nice ! That's something I really liked about Python.NET :) BTW ... is there something like that for Java ? I mean to use J2[SE]E classes using CPython ? This could also be useful to have personalized distributions. I mean, if I want to implement a Py app that will run in devices with limited capabilities, and let's say that it only imports `sockets` module (or a few more ;o), then it will be easier to prepare a subset of stdlib in order to deploy just what is needed in such devices, and save some space ;o). Projects like py2exe or others, could use something like that in order to extract relevant stdlib (modules | packages) and make them available to Windows apps distributed as exe files (e.g. Hg ) CMIIW anyway, perhaps I'm just dreaming. however ... > The big changes I can see from here would be moving the tests to the > packages from the central tests directory, and adding a setup.py file or > some other form of metadata providion for each package. Not that big now > that I've written it ;-) > In this case I envision the following issues if one setup.py file is generated for every module or top-level package (... which is -considering the previous message- how u plan to do it, isn't it ? ) - the maintenance effort might increase - what about dependencies between stdlib modules ? - there are many attributes which will take the same values for each and every packages (e.g. version info, issue tracker, ...) and some that will be specific (e,g, maintainer, author, contact info, dependencies ...) - Overhead when a new package is included in stdlib (need to create and maintain `setup.py` script, and so on ...) So my $0.02 here are : - to have a single `setup.py` file (in the end it's a script, isn't it ? ) - provide an argument in order to select module(s) to be included (full stdlib if missing) in source distribution. In general any other parameterization of the `setup.py` may be just ok, the goal is to have only one - use a mechanism in order to specify config options for specific pkgs modules, and make it available to the global `setup.py`. For example : * Specify metadata using top-level fields in modules (e.g. __author__, __maintainer__, ...) * Specify metadata using separate INI files for each target What d'u think ? There may be some issues with sdist anyway :-/ PS: Will those packages be submitted to PyPI too ? I mean if not sdists, at least meta-data ? -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Sobrepasa las 300 descargas el m?dulo dutest - http://feedproxy.google.com/~r/simelo-es/~3/U5rff5iTQPI/sobrepasa-las-300-descargas-el-modulo.html From steven.bethard at gmail.com Wed Sep 30 17:22:33 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 30 Sep 2009 08:22:33 -0700 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: On Wed, Sep 30, 2009 at 5:21 AM, Barry Warsaw wrote: > On Sep 29, 2009, at 11:15 PM, Martin v. L?wis wrote: > >> I would propose that the format argument gets an argument name, >> according to the syntax it is written in. For PEP 3101 format, >> I would call the argument "format" (like the method name of the >> string type), i.e. >> >> logging.Formatter( >> ?format="{asctime} - {name} - {levelname} - {message}") >> >> For the % formatting, I suggest "dicttemplate" (assuming that >> you *have* to use dictionary %(key)s style currently). >> >> The positional parameter would also mean dicttemplate, and >> would be deprecated (eventually requiring a keyword-only >> parameter). > > Although I hate the name 'dicttemplate', this seems like the best solution > to me. ?Maybe it's good that 'dicttemplate' is so ugly though so that people > will naturally prefer 'format' :). ?But I like this because there's not > really any magic, it's explicit, and the decision is made by the coder at > the call site. ?The implementation does not need to guess at all. Could you comment on what you think we should do when the parameter is not positional? As I mentioned upthread, in the case of logging.Formatter, it's already documented as taking the keyword parameter "fmt", so we'd have to use the name "fmt" for % formatting. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From barry at python.org Wed Sep 30 17:31:23 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Sep 2009 11:31:23 -0400 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: On Sep 30, 2009, at 11:22 AM, Steven Bethard wrote: > On Wed, Sep 30, 2009 at 5:21 AM, Barry Warsaw > wrote: >> On Sep 29, 2009, at 11:15 PM, Martin v. L?wis wrote: >> >>> I would propose that the format argument gets an argument name, >>> according to the syntax it is written in. For PEP 3101 format, >>> I would call the argument "format" (like the method name of the >>> string type), i.e. >>> >>> logging.Formatter( >>> format="{asctime} - {name} - {levelname} - {message}") >>> >>> For the % formatting, I suggest "dicttemplate" (assuming that >>> you *have* to use dictionary %(key)s style currently). >>> >>> The positional parameter would also mean dicttemplate, and >>> would be deprecated (eventually requiring a keyword-only >>> parameter). >> >> Although I hate the name 'dicttemplate', this seems like the best >> solution >> to me. Maybe it's good that 'dicttemplate' is so ugly though so >> that people >> will naturally prefer 'format' :). But I like this because there's >> not >> really any magic, it's explicit, and the decision is made by the >> coder at >> the call site. The implementation does not need to guess at all. > > Could you comment on what you think we should do when the parameter is > not positional? As I mentioned upthread, in the case of > logging.Formatter, it's already documented as taking the keyword > parameter "fmt", so we'd have to use the name "fmt" for % formatting. I'm okay with fmt==%-formatting and format=={}-formatting, but I'd also be okay with transitioning 'fmt' to 'dicttemplate' or whatever. I think the important thing is to be explicit in the method signature which one you want (secondary would be trying to standardize this across the stdlib). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From olemis at gmail.com Wed Sep 30 17:37:02 2009 From: olemis at gmail.com (Olemis Lang) Date: Wed, 30 Sep 2009 10:37:02 -0500 Subject: [Python-Dev] sharing stdlib across python implementations In-Reply-To: References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com> <4AC36B2A.6020406@simplistix.co.uk> Message-ID: <24ea26600909300837m577947b4s7e537b048740adbd@mail.gmail.com> On Wed, Sep 30, 2009 at 9:43 AM, Antoine Pitrou wrote: > Chris Withers simplistix.co.uk> writes: >> >> I'm on on stdlib-sig and I'm afraid I don't have the bandwidth to start >> on it, but I'd just like to throw in (yet again) that it would be great >> if the stdlib was actually a set of separate python packages with their >> own version metadata so that packaging tools could manage them, and >> upgrade them independently of python packages when there are bug fixes. > > This sounds like a bad idea to me. Each Python release is tested and debugged as > a whole. If you have a lot of possible combinations (module A version 1.1 with > module B version 1.2, etc.), it becomes impossible for us to ensure proper QA > for the whole and as a result the quality might become lower, rather than higher. > You are right here ! -1 for splitting the test code and test suite but I still think it could be a good idea ... > (of course, if we rigorously enforce APIs and preserve compatibility, -1 > this might > not be a real issue; but our compatibility story is a bit irregular, IMHO) > mainly because of this ;o) But it's still possible to use a parameterized `setup.py` and leave things just like they are right now. For instance, I have started something like that has been dome by the FLiOOPS project [1]_ (and possibly others, it's just an example ;o). In the SVN repository there's a single trunk containing the whole PyOOP package (which is not complete BTW). Inside of it there are separate `setup.py` files for other distributions too : - `PyOOP` (which should contain them all) employs default `setup.py` - `dutest` is a single file (there are more in there) under `oop.utils` package and is distributed separately too, so you can find in there `setup_dutest.py` script. All other packages rely on it to build test suites ;o) - CASPy (Comprehensive Assertion Support for Python) should be distributed separately too, so it should have its own `setup_dbc.py` script once it's ready. When I create (SVN) tags for each package, I have to rename it (extra-overhead), and I once global metadata changes, then I have to change them all PS: Another alternative for stdlib could be to have common data in `setup.cfg` and separate `setup.py` scripts, but I don't really like this one ... -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Sobrepasa las 300 descargas el m?dulo dutest - http://feedproxy.google.com/~r/simelo-es/~3/U5rff5iTQPI/sobrepasa-las-300-descargas-el-modulo.html From steven.bethard at gmail.com Wed Sep 30 17:39:14 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 30 Sep 2009 08:39:14 -0700 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: On Wed, Sep 30, 2009 at 8:31 AM, Barry Warsaw wrote: > On Sep 30, 2009, at 11:22 AM, Steven Bethard wrote: >> On Wed, Sep 30, 2009 at 5:21 AM, Barry Warsaw wrote: >>> On Sep 29, 2009, at 11:15 PM, Martin v. L?wis wrote: >>>> I would propose that the format argument gets an argument name, >>>> according to the syntax it is written in. For PEP 3101 format, >>>> I would call the argument "format" (like the method name of the >>>> string type), i.e. >>>> >>>> logging.Formatter( >>>> ?format="{asctime} - {name} - {levelname} - {message}") >>>> >>>> For the % formatting, I suggest "dicttemplate" (assuming that >>>> you *have* to use dictionary %(key)s style currently). >> >> Could you comment on what you think we should do when the parameter is >> not positional? As I mentioned upthread, in the case of >> logging.Formatter, it's already documented as taking the keyword >> parameter "fmt", so we'd have to use the name "fmt" for % formatting. > > I'm okay with fmt==%-formatting and format=={}-formatting, but I'd also be > okay with transitioning 'fmt' to 'dicttemplate' or whatever. ?I think the > important thing is to be explicit in the method signature which one you want > (secondary would be trying to standardize this across the stdlib). Thanks for the clarification. I generally like this approach, though it's not so convenient for argparse which already takes format strings like this:: parser = ArgumentParser(usage='%(prog)s [--foo]') parser.add_argument( '--foo', type=int, default=42, help='A foo of type %(type)s, defaulting to %(42)s) That is, existing keyword arguments that already have good names (and are pretty much always used as keyword arguments) take format strings. I'm not sure that changing the name of usage= or help= here is really an option. I guess in this case I'm stuck with something like Benjamin's suggestion of adding an additional flag to control which type of formatting, and the corresponding 4 versions of cleanup. Ew. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From barry at python.org Wed Sep 30 17:50:28 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Sep 2009 11:50:28 -0400 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: On Sep 30, 2009, at 11:39 AM, Steven Bethard wrote: > Thanks for the clarification. I generally like this approach, though > it's not so convenient for argparse which already takes format strings > like this:: > > parser = ArgumentParser(usage='%(prog)s [--foo]') > parser.add_argument( > '--foo', type=int, default=42, > help='A foo of type %(type)s, defaulting to %(42)s) > > That is, existing keyword arguments that already have good names (and > are pretty much always used as keyword arguments) take format strings. > I'm not sure that changing the name of usage= or help= here is really > an option. Ah right. > I guess in this case I'm stuck with something like Benjamin's > suggestion of adding an additional flag to control which type of > formatting, and the corresponding 4 versions of cleanup. Ew. I missed Benjamin's suggestion, but in this case I would say add a flag to ArgumentParser. I'm either going to want {} formatting all or nothing. E.g. import argparse parser = ArgumentParser(usage='{prog} [--foo]', format=argparse.BRACES) parser.add_argument( '--foo', type=int, default=42, help='A foo of type {type}, defaulting to {42}') (although that last looks weird ;). -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From olemis at gmail.com Wed Sep 30 17:51:27 2009 From: olemis at gmail.com (Olemis Lang) Date: Wed, 30 Sep 2009 10:51:27 -0500 Subject: [Python-Dev] sharing stdlib across python implementations In-Reply-To: <24ea26600909300837m577947b4s7e537b048740adbd@mail.gmail.com> References: <4AB8D9F2.6050807@simplistix.co.uk> <4AB8E136.8040706@simplistix.co.uk> <5c6f2a5d0909220812u2319453axe72053bb5f736c81@mail.gmail.com> <1A472770E042064698CB5ADC83A12ACD04915F11@TK5EX14MBXC120.redmond.corp.microsoft.com> <19129.39295.970536.916767@montanaro.dyndns.org> <4dab5f760909230604j2d5cc2cam9bf4fdf492f57e31@mail.gmail.com> <4AC36B2A.6020406@simplistix.co.uk> <24ea26600909300837m577947b4s7e537b048740adbd@mail.gmail.com> Message-ID: <24ea26600909300851n1f56e1fcs9d648329a1b81349@mail.gmail.com> On Wed, Sep 30, 2009 at 10:37 AM, Olemis Lang wrote: > On Wed, Sep 30, 2009 at 9:43 AM, Antoine Pitrou wrote: >> Chris Withers simplistix.co.uk> writes: >>> [...] > > For instance, I have started something like that has been dome by the > FLiOOPS ?project [1]_ Sorry for the missing reference and typos, should be For instance, something like that has been done by the FLiOOPS ?project [1]_ .. [1] py trunk @ FLiOOPS @ sf.net (http://sourceforge.net/apps/trac/flioops/browser/py/trunk) -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: Sobrepasa las 300 descargas el m?dulo dutest - http://feedproxy.google.com/~r/simelo-es/~3/U5rff5iTQPI/sobrepasa-las-300-descargas-el-modulo.html From rdmurray at bitdance.com Wed Sep 30 18:14:37 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 30 Sep 2009 12:14:37 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> Message-ID: On Wed, 30 Sep 2009 at 10:52, Paul Moore wrote: > 2009/9/30 Mark Dickinson : >> Please could someone who understands the uses of IPNetwork better than >> I do explain why the following wouldn't be a significant problem, if __eq__ >> and __hash__ were modified to disregard the .ip attribute as suggested: >> >>>>> linus = IPv4Network('172.16.200.1/24') >>>>> snoopy = IPv4Network('172.16.200.3/24') >>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'} >>>>> fqdn[linus] ?# expecting 'linus.peanuts.net' >> 'snoopy.peanuts.net' > > I certainly don't understand IPv4Network better than you :-) But that > just looks wrong to me - linus and snoopy are hosts not networks, so > making them IPv4Network classes seems wrong. I'd instinctively make > them IPv4Address objects (which, I believe, would work). I didn't reply to Guido's post about not wanting a third class (IPAddressWithMask or somesuch) because it made it harder for people to choose the "right" class to use. He went on to say that if there was only one class, they'd have a 100% chance of getting it right. Well, the original ipaddr implementation had one class(*), but it wasn't deemed acceptable by the community, and at least one real user of ipaddr has expressed appreciation of the 2.0 addition of the explicit IPAddress data type, so it seems like that was the right direction to go. Now the new implementation has two classes, and the above example shows that people _will_ make the wrong choice. I'd say that was _because of_ the loose parsing and the ip attribute. If IPv4Network only accepted valid network addresses and masks, and there was a separate parsing function that took an arbitrary IP+netmask and returned an (IPAddress, IPNetwork) tuple, then the user would choose the right class, IMO, because otherwise they couldn't even get their code to parse the input. That seems like good design to me. But I think I'm descending to beating a dead horse here.... --David (RDM) (*) yes, I'm ignoring the IPv4/IPv6 split throughout this message. From vinay_sajip at yahoo.co.uk Wed Sep 30 18:16:51 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 30 Sep 2009 16:16:51 +0000 (UTC) Subject: [Python-Dev] PEP 389: argparse - new command line parsing module References: <1afaf6160909272008t6647d389me88a0a039c515f7a@mail.gmail.com> <1afaf6160909272041v3dc6c079o9c8ca50a2605c017@mail.gmail.com> <9d153b7c0909272059l16f6ffecofe8b18ada6c24aac@mail.gmail.com> <20090928083820.GA488@laurie.devork> <20090928130317.GR23367@snowy.squish.net> <9d153b7c0909280818v30760a44p11ab24ba9251d4ef@mail.gmail.com> Message-ID: Yuvgoog Greenle gmail.com> writes: > +1 for adding argparse and eventually deprecating optparse, -0 for deprecating getopt. > 2. Big modules - 1 uber-module for each subject that does everything. Maybe logging is an example of this. I'm not sure that it is, if you mean code size. In Python 2.5, logging is 1300 SLOC, less than say tarfile, pickletools, pydoc and decimal. Regards, Vinay Sajip From foom at fuhm.net Wed Sep 30 18:19:31 2009 From: foom at fuhm.net (James Y Knight) Date: Wed, 30 Sep 2009 12:19:31 -0400 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: <200910010034.27427.steve@pearwood.info> References: <200910010034.27427.steve@pearwood.info> Message-ID: On Sep 30, 2009, at 10:34 AM, Steven D'Aprano wrote: >> E.g. >> x = newstyle_formatstr("{} {} {}") >> x % (1,2,3) == x.format(1,2,3) == "1 2 3" > > Moving along, let's suppose the newstyle_formatstr is introduced. > What's > the intention then? Do we go through the std lib and replace every > call > to (say) > somestring % args > with > newstyle_formatstr(somestring) % args > instead? That seems terribly pointless to me Indeed, that *would* be terribly pointless! Actually, more than pointless, it would be broken, as you've changed the API from taking oldstyle format strings to newstyle format strings. That is not the suggestion. The intention is to change /nearly nothing/ in the std lib, and yet allow users to use newstyle string substitution with every API. Many Python APIs (e.g. logging) currently take a %-type formatting string. It cannot simply be changed to take a {}-type format string, because of backwards compatibility concerns. Either a new API can be added to every one of those functions/classes, or, a single API can be added to inform those places to use newstyle format strings. >> This could allow for a controlled switch towards the new format >> string format, with a long deprecation period for users to migrate: >> >> 1) introduce the above feature, and recommend in docs that people >> only ever use new-style format strings, wrapping the string in >> newstyle_formatstr() when necessary for passing to an API which uses >> % internally. > > And how are people supposed to know what the API uses internally? It's documented, (as it already must be, today!). > Personally, I think your chances of getting people to write: > logging.Formatter(newstyle_formatstr("%(asctime)s - %(name)s - % > (level)s - %(msg)s")) > instead of > logging.Formatter("%(asctime)s - %(name)s - %(level)s - %(msg)s") That's not my proposal. The user could write either: logging.Formatter("%(asctime)s - %(name)s - %(level)s - %(msg)s") (as always -- that can't be changed without a long deprecation period), or: logging.Formatter(newstyle_formatstr("{asctime} - {name} - {level} - {msg}") This despite the fact that logging has not been changed to use {}- style formatting internally. It should continue to call "self._fmt % record.__dict__" for backward compatibility. That's not to say that this proposal would allow no work to be done to check the stdlib for issues. The Logging module presents one: it checks if the format string contains "%{asctime}" to see if it should bother to calculate the time. That of course would need to be changed. Best would be to stick an instance which lazily generates its string representation into the dict. The other APIs mentioned on this thread (BaseHTTPServer, email.generator) will work immediately without changes, however. James From vinay_sajip at yahoo.co.uk Wed Sep 30 18:21:52 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 30 Sep 2009 16:21:52 +0000 (UTC) Subject: [Python-Dev] PEP 389: argparse - new command line parsing module References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: Brett Cannon python.org> writes: > Obviously if one of the getopt supporters has a better way of doing > this then please speak up. I'm not a getopt supporter per se - I'm +1 for argparse - but Opster provides some nice syntax sugar on top of getopt, and can be seen here: http://blogg.ingspree.net/blog/2009/09/14/opster/ I've contacted Steven about a similar approach for argparse, and we're waiting for Yuvgoog's work on argparse(func) to be done to see if there's a feasible similar approach. From python at rcn.com Wed Sep 30 18:36:53 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 30 Sep 2009 09:36:53 -0700 Subject: [Python-Dev] transitioning from % to {} formatting References: <4AC2CD65.30905@v.loewis.de> Message-ID: <8A34E62222BC4795B959E26E5F313AA0@RaymondLaptop1> >>> The positional parameter would also mean dicttemplate, and >>> would be deprecated (eventually requiring a keyword-only >>> parameter). >> >> Although I hate the name 'dicttemplate', this seems like the best solution >> to me. Maybe it's good that 'dicttemplate' is so ugly though so that people >> will naturally prefer 'format' :). But I like this because there's not >> really any magic, it's explicit, and the decision is made by the coder at >> the call site. The implementation does not need to guess at all. > > Could you comment on what you think we should do when the parameter is > not positional? As I mentioned upthread, in the case of > logging.Formatter, it's already documented as taking the keyword > parameter "fmt", so we'd have to use the name "fmt" for % formatting. Unless there is a firm decision to switch to kill %-formatting across the board, I don't think anything should be done at all. Creating Py3.x was all about removing cruft and clutter. I don't think it would be improved by adding two ways to do it for everything in the standard library. That is a lot of additional code, API expansion, and new testing, fatter docs, and extra maintenance, but giving us no new functionality. Anytime we start hearing about newstyle/oldstyle combinations, I think a flag should go up. Anytime there is a proposal to make sweeping additions that do not add new capabilities, a flag should go up. I understand the desire to have all formatting support both ways, but I don't think it is worth the costs. People *never* need both ways though they may have differing preferences about which *one* to use. my-two-cents, Raymond From tjreedy at udel.edu Wed Sep 30 18:38:44 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 30 Sep 2009 12:38:44 -0400 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <8517e9350909270947w65e8e74ap243f25790279feb4@mail.gmail.com> <200909280330.30643.steve@pearwood.info> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> Message-ID: Mark Dickinson wrote: > On Wed, Sep 30, 2009 at 1:44 AM, Nick Coghlan wrote: > Please could someone who understands the uses of IPNetwork better than > I do explain why the following wouldn't be a significant problem, if __eq__ > and __hash__ were modified to disregard the .ip attribute as suggested: > >>>> linus = IPv4Network('172.16.200.1/24') >>>> snoopy = IPv4Network('172.16.200.3/24') >>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'} >>>> fqdn[linus] # expecting 'linus.peanuts.net' > 'snoopy.peanuts.net' > > Is this just a problem of education, teaching the users not to abuse > IPv4Network this way? Or is this just an unlikely use of IPv4Network? > Or have I misunderstood the proposal altogether? This gets at why I suggested the docs be organized as I suggested, with 'Network is range of addresses defined by address within range ...' and 'Definition address is retained' *immediately* followed by 'Definition address is mostly ignored, including for comparisons'. The latter caveat should mention the implication for hash() and set/dict membership. The point is to emphasize from the beginning that a network is a network (with an address, that many will ignore), rather than an address (with a network). Terry Jan Reedy From guido at python.org Wed Sep 30 18:38:50 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Sep 2009 09:38:50 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: On a tangent -- a use case that I see happening frequently but which is pretty poorly supported by optparse is a command-line that has a bunch of general flags, then a 'subcommand name', and then more flags, specific to the subcommand. Most here are probably familiar with this pattern from SVN, Hg, and other revision control systems (P4 anyone?) with a rich command-line interface. There are some variants, e.g. whether global and subcommand-specific flags may overlap, and whether flags may follow positional args (Hg and SVN seem to differ here a bit). I've helped write at least two tools at Google that have this structure; both used different approaches, and neither was particularly easy to get right. Getting all the different --help output to make sense was mostly a manual process. (E.g. "foo --help" should print the general flags and the list of known subcommands, whereas "foo --help subcommand" should print flags and other usage info about the specific subcommand.) Also switching out to different calls based on the subcommand should/might be part of this. I would be willing to live with a third option parser in the stdlib if it solved this problem well. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From robert.kern at gmail.com Wed Sep 30 18:44:23 2009 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 30 Sep 2009 11:44:23 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> <4AC2846E.8060304@g.nevcal.com> Message-ID: On 2009-09-29 18:38 PM, Steven Bethard wrote: > I don't really use GUI libraries, so I wouldn't be able to come up > with such an example. I'd also rather not make API changes based on > speculative use cases, so before I spend time documenting these > things, I'd really like to hear from someone who has already, say, > used getopt or optparse in conjunction with a GUI library, and what > feedback they have about that. I use argparse (and previously optparse) frequently to handle the command line arguments of GUI apps. I tend to use them in the same way as CLI programs, though, since I usually only use command line arguments when starting the GUIs from the terminal. I am blissfully unaware of the problems Paul mentioned about Windows GUI-mode programs. I'm not sure what would make a program "GUI-mode" or not. Certainly, I have written Python programs that use wxPython and PyQt on Windows that print to stdout/stderr, and they appear to work fine. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From martin at v.loewis.de Wed Sep 30 18:48:16 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 30 Sep 2009 18:48:16 +0200 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: <4AC33DE1.3040304@trueblade.com> References: <4AC2CD65.30905@v.loewis.de> <4AC33DE1.3040304@trueblade.com> Message-ID: <4AC38BD0.9070608@v.loewis.de> >>> There's a lot of code already out there (in the standard library and >>> other places) that uses %-style formatting, when in Python 3.0 we >>> should be encouraging {}-style formatting. >> >> I don't agree that we should do that. I see nothing wrong with using >> % substitution. > > It's a maintenance burden. Well - that's the cost of keeping it in the language. It's not a problem with using it while it *is* in the language. So if a decision was made to eventually remove % formatting, it would be reasonable to start migrating code to PEP 3101. However, no such decision has been made (and hopefully won't be throughout 3.x), so as the mechanism *is* available, there is no need to start changing existing code (except the for actual issue Steven discusses, namely libraries that expect strings in % template form). > I'd note that PEP 3101 calls str.format() a replacement for > %-formatting, not an alternate mechanism to achieve the same end. I think this is a mis-wording; the intent of the PEP apparently is to propose this mechanism as an option, not as an actual replacement. This becomes clear when reading the "Backwards Compatibility" section: # Backwards compatibility can be maintained by leaving the existing # mechanisms in place. The new system does not collide with any of # the method names of the existing string formatting techniques, so # both systems can co-exist until it comes time to deprecate the # older system. Regards, Martin From tjreedy at udel.edu Wed Sep 30 18:49:34 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 30 Sep 2009 12:49:34 -0400 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <200910010034.27427.steve@pearwood.info> Message-ID: James Y Knight wrote: > allow users to use newstyle string substitution with every API. However it is done, I think someone (like new Python programmers) should be able to program in Python3, and use everything in the stdlib, without ever learning % formatting -- and that I should be able to forget about it ;-). +10 on the goal. Terry Jan Reedy From martin at v.loewis.de Wed Sep 30 18:50:25 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 30 Sep 2009 18:50:25 +0200 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: <4AC38C51.5050807@v.loewis.de> > Although I hate the name 'dicttemplate', this seems like the best > solution to me. Maybe it's good that 'dicttemplate' is so ugly though > so that people will naturally prefer 'format' :). But I like this > because there's not really any magic, it's explicit, and the decision is > made by the coder at the call site. The implementation does not need to > guess at all. Unfortunately, as Steven pointed out, the parameter is *already* documented with the name "fmt". So one option would be to call it "fmt" and "format"; the other option would be to not only deprecate the positional passing, but also the passing under the name fmt=. As for calling it "dicttemplate" - I'm sure people can and will propose alternative spellings :-) Regards, Martin From robert.kern at gmail.com Wed Sep 30 18:58:20 2009 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 30 Sep 2009 11:58:20 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: On 2009-09-30 11:38 AM, Guido van Rossum wrote: > On a tangent -- a use case that I see happening frequently but which > is pretty poorly supported by optparse is a command-line that has a > bunch of general flags, then a 'subcommand name', and then more flags, > specific to the subcommand. Most here are probably familiar with this > pattern from SVN, Hg, and other revision control systems (P4 anyone?) > with a rich command-line interface. There are some variants, e.g. > whether global and subcommand-specific flags may overlap, and whether > flags may follow positional args (Hg and SVN seem to differ here a > bit). > > I've helped write at least two tools at Google that have this > structure; both used different approaches, and neither was > particularly easy to get right. Getting all the different --help > output to make sense was mostly a manual process. (E.g. "foo --help" > should print the general flags and the list of known subcommands, > whereas "foo --help subcommand" should print flags and other usage > info about the specific subcommand.) Also switching out to different > calls based on the subcommand should/might be part of this. > > I would be willing to live with a third option parser in the stdlib if > it solved this problem well. I don't think argparse supports the "foo --help subcommand" OOB. I think it would be simple to modify argparse to make it do so. It does support general options followed by a subcommand with options, though. http://argparse.googlecode.com/svn/tags/r101/doc/other-methods.html#sub-commands -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From solipsis at pitrou.net Wed Sep 30 19:01:35 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 30 Sep 2009 17:01:35 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?transitioning_from_=25_to_=7B=7D_formattin?= =?utf-8?q?g?= References: <200910010034.27427.steve@pearwood.info> Message-ID: James Y Knight fuhm.net> writes: > > The user could write either: > logging.Formatter("%(asctime)s - %(name)s - %(level)s - %(msg)s") > (as always -- that can't be changed without a long deprecation > period), or: > logging.Formatter(newstyle_formatstr("{asctime} - {name} - {level} - > {msg}") Why not allow logging.Formatter to take a callable, which would in turn call the callable with keyword arguments? Therefore, you could write: logging.Formatter("{asctime} - {name} - {level} - {msg}".format) and then: logging.critical(name="Python", msg="Buildbots are down") All this without having to learn about a separate "compatibility wrapper object". Regards Antoine. From python at rcn.com Wed Sep 30 19:05:05 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 30 Sep 2009 10:05:05 -0700 Subject: [Python-Dev] transitioning from % to {} formatting References: <200910010034.27427.steve@pearwood.info> Message-ID: <0AC8D406FC894629AA95257777B7B70C@RaymondLaptop1> [Terry Reedy] > However it is done, I think someone (like new Python programmers) should > be able to program in Python3, and use everything in the stdlib, without > ever learning % formatting -- and that I should be able to forget about > it ;-). > > +10 on the goal. If that were possible, it would be nice. But as long as the language supports %-formatting, it is going to be around in one form or another. Any non-casual user will bump into %-formatting in books, in third-party modules, in ASPN recipes, on the newsgroup, and in our own source code. If they maintain any exising software, they will likely encounter too. It doesn't seem to be a subject that can be ignored. Also, I think it premature to say that {}-formatting has been proven in battle. AFAICT, there has been very little uptake. I've personally made an effort to use {}-formatting more often but find that I frequently have to lookup the syntax and need to experiment with the interactive interpreter to get it right. I haven't found it easy to teach or to get other people to convert. This is especially true if the person has encountered %-formatting in other languages (it is a popular approach). Raymond From steven.bethard at gmail.com Wed Sep 30 19:14:17 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 30 Sep 2009 10:14:17 -0700 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: On Wed, Sep 30, 2009 at 8:50 AM, Barry Warsaw wrote: > On Sep 30, 2009, at 11:39 AM, Steven Bethard wrote: > >> Thanks for the clarification. I generally like this approach, though >> it's not so convenient for argparse which already takes format strings >> like this:: >> >> ? parser = ArgumentParser(usage='%(prog)s [--foo]') >> ? parser.add_argument( >> ? ? ? '--foo', type=int, default=42, >> ? ? ? help='A foo of type %(type)s, defaulting to %(42)s) Yep, sorry, typo, that should have been %(default)s, not %(42)s. >> I guess in this case I'm stuck with something like Benjamin's >> suggestion of adding an additional flag to control which type of >> formatting, and the corresponding 4 versions of cleanup. Ew. > > I missed Benjamin's suggestion, but in this case I would say add a flag to > ArgumentParser. ?I'm either going to want {} formatting all or nothing. > ?E.g. > > import argparse > parser = ArgumentParser(usage='{prog} [--foo]', format=argparse.BRACES) Yeah, that's basically Benjamin's suggestion, with the transition path being: (1) Introduce format= keyword argument, defaulting to PERCENTS (2) Deprecate format=PERCENTS (3) Error on format=PERCENTS (Benjamin suggested just changing the default here, but that would give a release where errors would pass silently) (4) Deprecate format= keyword argument. (5) Remove format= keyword argument. It's a little sad that it takes 5 versions to do this, but I guess if a user is on top of things, at version (1) they add format=BRACES to all their code, and then remove those at version (4). So even though there are 5 versions, there are only two code changes required. At least in the case of argparse, this can be a constructor argument as you suggest, and we only have to introduce this flag in one place. Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From guido at python.org Wed Sep 30 19:37:20 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Sep 2009 10:37:20 -0700 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: <4AC38BD0.9070608@v.loewis.de> References: <4AC2CD65.30905@v.loewis.de> <4AC33DE1.3040304@trueblade.com> <4AC38BD0.9070608@v.loewis.de> Message-ID: On Wed, Sep 30, 2009 at 9:48 AM, "Martin v. L?wis" wrote: >> I'd note that PEP 3101 calls str.format() a replacement for >> %-formatting, not an alternate mechanism to achieve the same end. > > I think this is a mis-wording; the intent of the PEP apparently is > to propose this mechanism as an option, not as an actual replacement. > This becomes clear when reading the "Backwards Compatibility" section: The problem is, PEP 3101 and our interpretation of it evolved. The original proposal for {}-formatting was certainly put forward with the aim to completely *replace* %-formatting, and care was taken in the design to cover all use cases, avoid known problems, etc. Then we started looking seriously at conversion from Python 2 to Python 3 and we discovered that converting %-formatting to {}-formatting was a huge can of worms, and decided it wasn't worth to try and do *at the time* given the Python 3 schedule. We considered some kind of gentle deprecation warning, but decided that even that would be too noisy. So now we have two competing mechanisms. In the long run, say Python 4, I think we don't need both, and we should get rid of one. My preference is still getting rid of %-formatting, due to the problems with it that prompted the design of {}-formatting (no need to reiterate the list here). So how do we get there? My proposal would be to let this be a gradual take-over of a new, superior species in the same niche as an older species. (Say, modern man over Neanderthal man.) Thus, as new code is written (especially example code, which will be copied widely), we should start using {}-formatting, and when new APIs are designed that tie in to some kind of formatting, they should use {}-formatting. Adding support for {}-formatting, in addition to %-formatting, to existing APIs like the logging package also strikes me as a good idea, as long as backwards compatibility can be preserved. (I have no strong ideas on how to do this right now.) If we do this right, by the time Python 4 comes around, {}-formatting will have won the race, and there won't be a question about removing %-formatting at the time. I wouldn't be surprised if by then static analysis techniques will have improved so that we *can* consider automatic conversion by then. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed Sep 30 19:51:12 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 30 Sep 2009 19:51:12 +0200 Subject: [Python-Dev] Python logging and 1.5.2 compatibility In-Reply-To: References: Message-ID: <4AC39A90.2050007@v.loewis.de> > PEP 291 (Backward Compatibility for Standard Library) mentions logging as having > 1.5.2 compatibility, and I'm thinking that it can be removed from that PEP > altogether. Done! Martin From barry at python.org Wed Sep 30 20:22:53 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Sep 2009 14:22:53 -0400 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: <212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org> On Sep 30, 2009, at 12:58 PM, Robert Kern wrote: > I don't think argparse supports the "foo --help subcommand" OOB. I > think it would be simple to modify argparse to make it do so. It > does support general options followed by a subcommand with options, > though. Right. I've made it kind of work in Mailman 3, but it would be nice for argparse to support this out of the box. Note that I think you want two forms: foo help subcommand foo subcommand --help to basically print the same help text. This is the way bzr does it for example and it works great. Other than that, I think argparse supports Guido's use case very nicely. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From barry at python.org Wed Sep 30 20:27:59 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Sep 2009 14:27:59 -0400 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <200910010034.27427.steve@pearwood.info> Message-ID: <5353D5BD-1862-42C2-913F-C75AD73002DC@python.org> On Sep 30, 2009, at 1:01 PM, Antoine Pitrou wrote: > Why not allow logging.Formatter to take a callable, which would in > turn call the > callable with keyword arguments? > > Therefore, you could write: > logging.Formatter("{asctime} - {name} - {level} - {msg}".format) This is a very interesting idea. Note that one of the reasons to /at least/ support {}-strings also is that %-strings are simply too error prone in many situations. For example, if I decide to support internationalization of log format strings, and all I can use is %-strings, it's almost guaranteed that I will have bugs because a translator forgot the trailing 's'. This exactly the motivation that led to PEP 292 $-strings. In fact, while we're at it, it would be kind of cool if I could use $- strings in log templates. Antoine's idea of accepting a callable might fit that bill nicely. -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From brett at python.org Wed Sep 30 19:46:28 2009 From: brett at python.org (Brett Cannon) Date: Wed, 30 Sep 2009 10:46:28 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: On Wed, Sep 30, 2009 at 09:21, Vinay Sajip wrote: > Brett Cannon python.org> writes: > >> Obviously if one of the getopt supporters has a better way of doing >> this then please speak up. > > I'm not a getopt supporter per se - I'm +1 for argparse - but Opster provides > some nice syntax sugar on top of getopt, and can be seen here: > > http://blogg.ingspree.net/blog/2009/09/14/opster/ > > I've contacted Steven about a similar approach for argparse, and we're waiting > for Yuvgoog's work on argparse(func) to be done to see if there's a feasible > similar approach. I am reluctant to jump on these argument parsing decorators until they have existed outside the standard library for a while and have settled down as I suspect some people want to use annotations to do type conversion/checking, others don't, etc. -Brett From sridharr at activestate.com Wed Sep 30 21:36:01 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Wed, 30 Sep 2009 12:36:01 -0700 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: <21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org> References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> <21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org> Message-ID: 2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86, Windows 32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did. -srid On Wed, 30 Sep 2009 05:34:02 -0700, Barry Warsaw wrote: > On Sep 30, 2009, at 12:29 AM, Ned Deily wrote: > >> In my opinion, the standard python.org OS X installer for 2.6.3 now >> "works well" on 10.4, 10.5, and 10.6 and is ready for release. > > Fantastic, thanks. Martin's uploaded the Windows binaries so I'll > make the announcement now. No commits to the 2.6 tree are allowed > without checking with me first (via IRC, bug tracker or email). I'll > make an exception for svnmerge bookkeeping. > > -Barry > From barry at python.org Wed Sep 30 21:44:14 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 30 Sep 2009 15:44:14 -0400 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> <21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org> Message-ID: On Sep 30, 2009, at 3:36 PM, Sridhar Ratnakumar wrote: > 2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86, > Windows 32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did. Thanks for the feedback! Did you run the test suite on any of these? -Barry -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 832 bytes Desc: This is a digitally signed message part URL: From python at rcn.com Wed Sep 30 22:03:06 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 30 Sep 2009 13:03:06 -0700 Subject: [Python-Dev] Python 2.6.3 References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org><4AA7ADCB.6080101@gmail.com><11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org><9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org><21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org> Message-ID: <446A32AEF715427CB9D76CB23A368E96@RaymondLaptop1> [Sridhar Ratnakumar] > 2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86, Windows > 32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did. Did the test suite pass too? Raymond From sridharr at activestate.com Wed Sep 30 22:06:47 2009 From: sridharr at activestate.com (Sridhar Ratnakumar) Date: Wed, 30 Sep 2009 13:06:47 -0700 Subject: [Python-Dev] Python 2.6.3 In-Reply-To: References: <1BEC6749-D905-456F-BD34-9AC8B3F4CF2F@python.org> <4AA7ADCB.6080101@gmail.com> <11A6545D-7204-4F61-B55B-1CC77CB5645E@python.org> <9D506035-7C2D-4929-A134-E88EEB7B7D9E@python.org> <21F203FA-05CE-433B-AD16-1E7804C2C08B@python.org> Message-ID: On Wed, 30 Sep 2009 12:44:14 -0700, Barry Warsaw wrote: > 2.6.3rc1 builds fine on Linux x86/x86_64, MacOSX 10.4 ppc/x86, Windows > 32bit/64bit, HP-UX, AIX and Solaris just like 2.6.2 did. > Thanks for the feedback! Did you run the test suite on any of these? I will run the tests sometime tonight and let you know. -srid From guido at python.org Wed Sep 30 22:11:12 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 30 Sep 2009 13:11:12 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> <4AC33BA1.8050203@gmail.com> <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> Message-ID: On Wed, Sep 30, 2009 at 4:33 AM, Mark Dickinson wrote: > On Wed, Sep 30, 2009 at 12:06 PM, Nick Coghlan wrote: >> Mark Dickinson wrote: >>> Okay, so maybe this is an abuse of IPv4Network. ?But I'd (mis?)understood >>> that the retention of the .ip attribute was precisely a convenience to allow >>> this sort of use. ?If not, then what's it for? ?I've read the PEP and almost >>> all of this thread, but I can't help feeling I'm still missing something. ?If >>> someone could point out the obvious to me I'd be grateful. >> >> You're not missing anything that I'm aware of - unlike the use case for >> accepting a denormalised network definition in the IPNetwork constructor >> (which has been made quite clear in the list discussion, even if it is >> still a bit vague in the PEP), the use case for *retaining* the host >> information on the network object hasn't been well articulated at all. >> >> The closest anyone has come to describing a use case is an indirect >> comment via Guido that leaving out the attribute would involve real code >> having to find somewhere else to stash the original address details >> (e.g. by passing around an IPAddres/IPNetwork tuple rather than just an >> IPNetwork object). > > Ah, thanks---I'd missed that bit. ?So the .ip attribute is mainly for > backwards compatibility with existing uses/users of ipaddr. ?I guess > that makes sense, then. ?In particular, if it's suggested that new code > shouldn't make use of the .ip attribute, then the list/dict membership > problems described above can't arise. I don't know -- this is (from what Peter told me) a common use case so he (and presumably others) would from time to time have to write extra code to keep track of that IP address in a new app. Since this is just one extra attribute on an IPNetwork object I don't think that adding extra classes which only differ from the IPvXNetwork classes in having this one extra attribute is worthy. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rdmurray at bitdance.com Wed Sep 30 22:33:27 2009 From: rdmurray at bitdance.com (R. David Murray) Date: Wed, 30 Sep 2009 16:33:27 -0400 (EDT) Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> <4AC33BA1.8050203@gmail.com> <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> Message-ID: On Wed, 30 Sep 2009 at 13:11, Guido van Rossum wrote: > On Wed, Sep 30, 2009 at 4:33 AM, Mark Dickinson wrote: >> On Wed, Sep 30, 2009 at 12:06 PM, Nick Coghlan wrote: >>> Mark Dickinson wrote: >>>> Okay, so maybe this is an abuse of IPv4Network. ?But I'd (mis?)understood >>>> that the retention of the .ip attribute was precisely a convenience to allow >>>> this sort of use. ?If not, then what's it for? ?I've read the PEP and almost >>>> all of this thread, but I can't help feeling I'm still missing something. ?If >>>> someone could point out the obvious to me I'd be grateful. >>> >>> You're not missing anything that I'm aware of - unlike the use case for >>> accepting a denormalised network definition in the IPNetwork constructor >>> (which has been made quite clear in the list discussion, even if it is >>> still a bit vague in the PEP), the use case for *retaining* the host >>> information on the network object hasn't been well articulated at all. >>> >>> The closest anyone has come to describing a use case is an indirect >>> comment via Guido that leaving out the attribute would involve real code >>> having to find somewhere else to stash the original address details >>> (e.g. by passing around an IPAddres/IPNetwork tuple rather than just an >>> IPNetwork object). >> >> Ah, thanks---I'd missed that bit. ?So the .ip attribute is mainly for >> backwards compatibility with existing uses/users of ipaddr. ?I guess >> that makes sense, then. ?In particular, if it's suggested that new code >> shouldn't make use of the .ip attribute, then the list/dict membership >> problems described above can't arise. > > I don't know -- this is (from what Peter told me) a common use case so > he (and presumably others) would from time to time have to write > extra code to keep track of that IP address in a new app. Since this > is just one extra attribute on an IPNetwork object I don't think that > adding extra classes which only differ from the IPvXNetwork classes in > having this one extra attribute is worthy. I don't believe anyone ever suggested adding a class that differed from IPvXNetwork by the presence or absence of an 'ip' attribute. The two approaches(*) suggested were: 1) do not add another class, just pass around (IPvXAddress, IPVXNetwork) tuples when the address needs to be retained (or write your own tailored trivial class, like I did in my example). 2) add a class that differs from IPvXAddress by having a 'network' attribute that points (possibly lazily) to an IPvXNetwork object, and perhaps 'netmask' and 'hostmask' attributes. Especially after my experience with writing a real example program, I prefer (1). --David (*) For completeness there was also a third approach: add a 'network' attribute to IPvXAddress that would be None if there were no associated netmask. Conceptual problems with this were raised which I won't summarize; IMO it would only be slightly better than IPvXNetwork having an '.ip' attribute. From amcnabb at mcnabbs.org Wed Sep 30 22:17:35 2009 From: amcnabb at mcnabbs.org (Andrew McNabb) Date: Wed, 30 Sep 2009 14:17:35 -0600 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org> Message-ID: <20090930201735.GD27796@mcnabbs.org> On Wed, Sep 30, 2009 at 02:22:53PM -0400, Barry Warsaw wrote: > > Right. I've made it kind of work in Mailman 3, but it would be nice for > argparse to support this out of the box. Note that I think you want two > forms: > > foo help subcommand > foo subcommand --help > > to basically print the same help text. This is the way bzr does it for > example and it works great. In some commands, options as well as subcommands can change subsequent parsing. The iptables command is a good example of a command-line program that follows this practice. From the man page: after [a module name is specified], various extra command line options become available, depending on the specific module. You can specify multiple extended match modules in one line, and you can use the -h or --help options after the module has been specified to receive help specific to that module. In the case of iptables, module names are specified as options, not as subcommands. >From my cursory reading of the documentation, it looks like argparse can only add subparsers for subcommands. Is there any way to add subparsers based on options instead (as iptables does)? Also, is it possible to add these subparsers dynamically? For example, you would want to be able to load a module immediately after parsing the name instead of having to keep a predetermined list of all module names. I'm pretty sure that bzr dynamically loads modules this way. Can argparse help with this? Sorry for all of the questions. I ask them because I have some experience with adding the above features to optparse, and it was a lot of work to get it right. It also seems like there are a lot of programs that need to load modules dynamically. I would be really excited if argparse made this easier than optparse did. -- Andrew McNabb http://www.mcnabbs.org/andrew/ PGP Fingerprint: 8A17 B57C 6879 1863 DE55 8012 AB4D 6098 8826 6868 From robert.kern at gmail.com Wed Sep 30 23:01:32 2009 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 30 Sep 2009 16:01:32 -0500 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <20090930201735.GD27796@mcnabbs.org> References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> <212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org> <20090930201735.GD27796@mcnabbs.org> Message-ID: On 2009-09-30 15:17 PM, Andrew McNabb wrote: > On Wed, Sep 30, 2009 at 02:22:53PM -0400, Barry Warsaw wrote: >> >> Right. I've made it kind of work in Mailman 3, but it would be nice for >> argparse to support this out of the box. Note that I think you want two >> forms: >> >> foo help subcommand >> foo subcommand --help >> >> to basically print the same help text. This is the way bzr does it for >> example and it works great. > > In some commands, options as well as subcommands can change subsequent > parsing. The iptables command is a good example of a command-line > program that follows this practice. From the man page: > > after [a module name is specified], various extra command line options > become available, depending on the specific module. You can specify > multiple extended match modules in one line, and you can use the -h or > --help options after the module has been specified to receive help > specific to that module. > > In the case of iptables, module names are specified as options, not as > subcommands. > >> From my cursory reading of the documentation, it looks like argparse can > only add subparsers for subcommands. Is there any way to add subparsers > based on options instead (as iptables does)? I have not done so, but I suspect so. The implementation of .add_subparsers() adds to the positional argument list, but one could be written to append to the option list. > Also, is it possible to add these subparsers dynamically? For example, > you would want to be able to load a module immediately after parsing the > name instead of having to keep a predetermined list of all module names. > I'm pretty sure that bzr dynamically loads modules this way. Can > argparse help with this? Not out-of-box, but it looks fairly straightforward to plug in. The subparser logic is mostly encapsulated in the _SubparsersAction class. You can register a new class for it: parser.register('action', 'parsers', MySubParsersAction) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ubershmekel at gmail.com Wed Sep 30 23:15:14 2009 From: ubershmekel at gmail.com (Yuvgoog Greenle) Date: Thu, 1 Oct 2009 00:15:14 +0300 Subject: [Python-Dev] Announcing PEP 3136 In-Reply-To: References: <20090929202035.GA5492@gmail.com> Message-ID: <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com> I like how python has a minimalistic and powerful syntax (-1 for the break ___ PEP). Also, I really dislike the for/else ambiguity "butterflies". When is the else after a loop executed?1. When the loop isn't entered at all. 2. When the loop terminates through exhaustion of the list (does this include when the list was empty?) 3. When the loop didn't exit because of a break statement. HINTS: The way django does it is opposite the way python does it and there may be more than one correct answer. Any chances of cleaning this one up for python 4? I'm not sure how though I have a few ideas. On Tue, Sep 29, 2009 at 11:47 PM, Guido van Rossum wrote: > On Tue, Sep 29, 2009 at 1:20 PM, Hatem Nassrat wrote: > > Tue Jul 3 10:14:17 CEST 2007, Guido van Rossum wrote: > >> On 6/30/07, Matt Chisholm wrote: > >> > I've created and submitted a new PEP proposing support for labels in > >> > Python's break and continue statements. Georg Brandl has graciously > >> > added it to the PEP list as PEP 3136: > >> > > >> > http://www.python.org/dev/peps/pep-3136/ > >> > >> I think this is a good summary of various proposals that have been > >> floated in the past, plus some new ones. As a PEP, it falls short > >> because it doesn't pick a solution but merely offers a large menu of > >> possible options. Also, there is nothing about implementation yet. > >> > >> However, I'm rejecting it on the basis that code so complicated to > >> require this feature is very rare. In most cases there are existing > >> work-arounds that produce clean code, for example using 'return'. > > > > I agree that this feature will only serve as a quick hack and in many > > cases it would be misused and ugly code will be the result. However, > > it seems that there are some shortcuts that have sneaked into python > > (I am a fairly new Python programmer, only since late 2.4, so don't > > shoot me). The specific one of which I speak about is: > > > > while_stmt ::= "while" expression ":" suite > > ["else" ":" suite] > > > > for_stmt ::= "for" target_list "in" expression_list ":" suite > > ["else" ":" suite] > > > > try1_stmt ::= "try" ":" suite > > ("except" [expression ["as" target]] ":" suite)+ > > ["else" ":" suite] > > ["finally" ":" suite] > > > > All these else's seem like shortcuts to me. I did find a use for them, > > once I found out they existed, but I get butterflies whenever I do. > > In English, butterflies are usually a good thing (they mean you'e in love). > > These else clauses (assuming you're talking about that) have been in > the language pretty much forever. The combined except/finally clause > is newer, it was added because Java allows it and it was actually a > pretty common usage. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Wed Sep 30 23:21:47 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Wed, 30 Sep 2009 22:21:47 +0100 Subject: [Python-Dev] Announcing PEP 3136 In-Reply-To: <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com> References: <20090929202035.GA5492@gmail.com> <9d153b7c0909301415u39c8f198qea4777036e441289@mail.gmail.com> Message-ID: <4AC3CBEB.1000303@voidspace.org.uk> Yuvgoog Greenle wrote: > I like how python has a minimalistic and powerful syntax (-1 for the > break ___ PEP). > > Also, I really dislike the for/else ambiguity "butterflies". > > When is the else after a loop executed? > 1. When the loop isn't entered at all. If you mean because the list / iterator is empty then yes. If you mean something else then explain. > 2. When the loop terminates through exhaustion of the list (does this > include when the list was empty?) Yes. > 3. When the loop didn't exit because of a break statement. > No. This is pretty much the definition of the else statement semantics and are clearly defined. > HINTS: The way django does it is opposite the way python does it and > there may be more than one correct answer. > If you mean the Django template language does it differently from Python then you'll have to speak to the Django guys. The Django template language may be inspired by Python and written in Python but it isn't Python and has no obligation to use the same semantics. Python is very clear. > Any chances of cleaning this one up for python 4? I'm not sure how > though I have a few ideas. > See above. The else statement semantics for both loops and exception handling are extremely useful and don't need cleaning up. If you have further ideas then the Python-ideas list is the right place to discuss potential language changes or additions. All the best, Michael Foord > On Tue, Sep 29, 2009 at 11:47 PM, Guido van Rossum > wrote: > > On Tue, Sep 29, 2009 at 1:20 PM, Hatem Nassrat > wrote: > > Tue Jul 3 10:14:17 CEST 2007, Guido van Rossum wrote: > >> On 6/30/07, Matt Chisholm > wrote: > >> > I've created and submitted a new PEP proposing support for > labels in > >> > Python's break and continue statements. Georg Brandl has > graciously > >> > added it to the PEP list as PEP 3136: > >> > > >> > http://www.python.org/dev/peps/pep-3136/ > >> > >> I think this is a good summary of various proposals that have been > >> floated in the past, plus some new ones. As a PEP, it falls short > >> because it doesn't pick a solution but merely offers a large > menu of > >> possible options. Also, there is nothing about implementation yet. > >> > >> However, I'm rejecting it on the basis that code so complicated to > >> require this feature is very rare. In most cases there are existing > >> work-arounds that produce clean code, for example using 'return'. > > > > I agree that this feature will only serve as a quick hack and in > many > > cases it would be misused and ugly code will be the result. However, > > it seems that there are some shortcuts that have sneaked into python > > (I am a fairly new Python programmer, only since late 2.4, so don't > > shoot me). The specific one of which I speak about is: > > > > while_stmt ::= "while" expression ":" suite > > ["else" ":" suite] > > > > for_stmt ::= "for" target_list "in" expression_list ":" suite > > ["else" ":" suite] > > > > try1_stmt ::= "try" ":" suite > > ("except" [expression ["as" target]] ":" suite)+ > > ["else" ":" suite] > > ["finally" ":" suite] > > > > All these else's seem like shortcuts to me. I did find a use for > them, > > once I found out they existed, but I get butterflies whenever I do. > > In English, butterflies are usually a good thing (they mean you'e > in love). > > These else clauses (assuming you're talking about that) have been in > the language pretty much forever. The combined except/finally clause > is newer, it was added because Java allows it and it was actually a > pretty common usage. > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/ > ) > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ubershmekel%40gmail.com > > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From vinay_sajip at yahoo.co.uk Wed Sep 30 23:23:12 2009 From: vinay_sajip at yahoo.co.uk (Vinay Sajip) Date: Wed, 30 Sep 2009 21:23:12 +0000 (UTC) Subject: [Python-Dev] PEP 389: argparse - new command line parsing module References: <20090928130317.GR23367@snowy.squish.net> <200909290127.03215.steve@pearwood.info> Message-ID: Brett Cannon python.org> writes: > I am reluctant to jump on these argument parsing decorators until they > have existed outside the standard library for a while and have settled > down as I suspect some people want to use annotations to do type > conversion/checking, others don't, etc. > I agree - it's just an approach which shows some promise and is worth exploring. I'm not suggesting jumping in with both feet. But it does show about getopt that perhaps there's life in the old dog yet ;-) Regards, Vinay Sajip From martin at v.loewis.de Wed Sep 30 23:34:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 30 Sep 2009 23:34:11 +0200 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> <4AC33BA1.8050203@gmail.com> <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> Message-ID: <4AC3CED3.80104@v.loewis.de> > I don't know -- this is (from what Peter told me) a common use case so > he (and presumably others) would from time to time have to write > extra code to keep track of that IP address in a new app. I, and probably others, would really, REALLY like to see one such "common use case". I do find it puzzling that non of the readers here can think of any, and the continued assertion that use cases do exist without them being presented is somewhat disturbing. Regards, Martin From steven.bethard at gmail.com Wed Sep 30 23:40:20 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 30 Sep 2009 14:40:20 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: <20090930201735.GD27796@mcnabbs.org> References: <200909290127.03215.steve@pearwood.info> <212C8A6F-CEC3-44A0-91F4-C5814DF44B04@python.org> <20090930201735.GD27796@mcnabbs.org> Message-ID: On Wed, Sep 30, 2009 at 1:17 PM, Andrew McNabb wrote: > >From my cursory reading of the documentation, it looks like argparse can > only add subparsers for subcommands. ?Is there any way to add subparsers > based on options instead (as iptables does)? Currently this is not supported, but it would be a nice feature. > Also, is it possible to add these subparsers dynamically? ?For example, > you would want to be able to load a module immediately after parsing the > name instead of having to keep a predetermined list of all module names. > I'm pretty sure that bzr dynamically loads modules this way. ?Can > argparse help with this? You can probably already do this. I'm not 100% sure what you want to do, but it's certainly possible to define an argparse.Action that loads a module when it's invoked. It might look something like:: class MyAction(argparse.Action): def __call__(self, parser, namespace, value, option_string=None): mod = __import__(value) # or whatever Steve -- Where did you get that preposterous hypothesis? Did Steve tell you that? --- The Hiphopopotamus From ncoghlan at gmail.com Wed Sep 30 12:43:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 20:43:50 +1000 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: References: <4AC2CD65.30905@v.loewis.de> Message-ID: <4AC33666.9080709@gmail.com> Steven Bethard wrote: > So I understand how this might help a user to move from %-style > formatting to {}-style formatting, but it's still not clear to me how > to use this to evolve an API. In particular, if the goal is for the > API to move from accepting %-style format strings to {}-style format > strings, how should that API use newstyle_formatstr or > oldstyle_formatstr to make this transition? The problem is that many (most?) of the problematic APIs (such as logging) will have multiple users in a given application, so getting the granularity of any behavioural switches right is going to be difficult. Providing a formatstr() type that makes .__mod__() work based on a .format() style string (and a formatmod() type that does the opposite) would allow for extremely fine-grained decision making, since every format string will either be an ordinary str instance or else an instance of the formatting subclass. (Note that the primary use case for both proposed types is an application developer adapting between two otherwise incompatible third party libraries - the choice of class just changes based on whether the old code being adapted is the code invoking mod on a format string or the code providing a format string that expects to be used with the mod operator). I don't see any way for delayed formatting of "normal" strings in any existing API to move away from %-formatting except via a long and painful deprecation process (i.e. first warning when bare strings are used for a release or two, then switching entirely to the new formatting method) or by duplicating the API and maintaining the two interfaces in parallel for the foreseeable future. As Paul noted, the two proposed classes may also be useful to the library developer during such a transition process - they could accept strings in the "wrong" format just by wrapping them appropriately rather than having to maintain the parallel APIs all the way through the software stack. Probably worth letting these concepts bake for a while longer, but it definitely be nice to do *something* to help enable this transition in 2.7/3.2. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From peter at hda3.com Wed Sep 30 23:48:25 2009 From: peter at hda3.com (Peter Moody) Date: Wed, 30 Sep 2009 14:48:25 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <4AC3CED3.80104@v.loewis.de> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> <4AC33BA1.8050203@gmail.com> <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> <4AC3CED3.80104@v.loewis.de> Message-ID: <8517e9350909301448v4a7f891q3e12cec01ebde44a@mail.gmail.com> [- at google.com] this is what I wrote: it's not so much a need as it is useful. it's useful to take an address like 192.168.1.100/24 and derive a bunch of information from it (like the network address, broadcast address, containing supernets, etc), but still remember that the original address was 192.168.1.100. having a separate class or two for this is overly burdensome in my mind. FWIW, netaddr does the same thing. Cheers, /peter On Wed, Sep 30, 2009 at 2:34 PM, "Martin v. L?wis" wrote: >> I don't know -- this is (from what Peter told me) a common use case so >> he (and presumably ?others) would from time to time have to write >> extra code to keep track of that IP address in a new app. > > I, and probably others, would really, REALLY like to see one such > "common use case". I do find it puzzling that non of the readers here > can think of any, and the continued assertion that use cases do exist > without them being presented is somewhat disturbing. > > Regards, > Martin > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From python at rcn.com Wed Sep 30 23:51:19 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 30 Sep 2009 14:51:19 -0700 Subject: [Python-Dev] PEP 3144 review. References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com><4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com><5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com><79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com><5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com><4AC33BA1.8050203@gmail.com> <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> <4AC3CED3.80104@v.loewis.de> Message-ID: <8656B2EE4F6648CCA5A717C5E99D93A8@RaymondLaptop1> >> I don't know -- this is (from what Peter told me) a common use case so >> he (and presumably others) would from time to time have to write >> extra code to keep track of that IP address in a new app. [MvL] > I, and probably others, would really, REALLY like to see one such > "common use case". I do find it puzzling that non of the readers here > can think of any, and the continued assertion that use cases do exist > without them being presented is somewhat disturbing. A concrete use case would help put the discussion on a better technical footing. FWIW, it doesn't look like any of the competing packages ever found a need to add support for this yet-to-be-presented use case: http://code.google.com/p/netaddr/wiki/YetAnotherPythonIPModule Raymond From peter at hda3.com Wed Sep 30 23:56:07 2009 From: peter at hda3.com (Peter Moody) Date: Wed, 30 Sep 2009 14:56:07 -0700 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <8656B2EE4F6648CCA5A717C5E99D93A8@RaymondLaptop1> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> <4AC33BA1.8050203@gmail.com> <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> <4AC3CED3.80104@v.loewis.de> <8656B2EE4F6648CCA5A717C5E99D93A8@RaymondLaptop1> Message-ID: <8517e9350909301456t2051fd48yab3948c0b903bc43@mail.gmail.com> [- at google.com] On Wed, Sep 30, 2009 at 2:51 PM, Raymond Hettinger wrote: > >>> I don't know -- this is (from what Peter told me) a common use case so >>> he (and presumably ?others) would from time to time have to write >>> extra code to keep track of that IP address in a new app. > > [MvL] >> >> I, and probably others, would really, REALLY like to see one such >> "common use case". I do find it puzzling that non of the readers here >> can think of any, and the continued assertion that use cases do exist >> without them being presented is somewhat disturbing. > > A concrete use case would help put the discussion on a better technical > footing. > > FWIW, it doesn't look like any of the competing packages ever > found a need to add support for this yet-to-be-presented use case: > ?http://code.google.com/p/netaddr/wiki/YetAnotherPythonIPModule netaddr actually does have support for this. >>> import netaddr >>> netaddr.IPNetwork('1.1.1.1/24').ip IPAddress('1.1.1.1') >>> cheers, /peter > Raymond > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/python-dev%40hda3.com > From daniel at stutzbachenterprises.com Wed Sep 30 23:59:56 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Wed, 30 Sep 2009 16:59:56 -0500 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> <4AC33BA1.8050203@gmail.com> <5c6f2a5d0909300433i5b0d0833qe1a9d239bd303dfc@mail.gmail.com> Message-ID: .On Wed, Sep 30, 2009 at 3:33 PM, R. David Murray wrote: > 1) do not add another class, just pass around (IPvXAddress, > IPVXNetwork) tuples when the address needs to be retained (or write > your own tailored trivial class, like I did in my example). > I've been puzzled by objections to forcing the user to decide what metadata to store. Every time I've ever needed to store an IP address or a network, I've always need to store a bunch of other affiliated data with it. E.g.: class GnutellaPeer: # IP address, port, timestamp, and dozen of other fields class NetworkInterface: # IP address, subnet mask, name, MAC address, etc. class Route: # interface, IP address, subnet mask, gateway address, route priority Is it such a burden for the programmer to store one extra field in the class they will inevitably write? For that matter, I do not see the advantage to an IPNetwork class that saves the programmer from having to separately store the IP address and subnet mask. Maybe I am biased by my background: I learned network programming in C, and I think of an IP address as little more than an integer type with a special string representation. People have voiced support for the IPNetwork class and for the occasional utility of an .ip field. I assume they have good use cases. It would be nice if the use cases were collected into the PEP, in a clear and articulate way. Preferably by someone other than ipaddr author, for whom the use cases are understandably a bit too obvious to explain at length with exasperation. It aught to be easy to justify the functionality of the library, if the use cases are clearly articulated. The current PEP begins by noting that many other IP address libraries are available, but doesn't describe what makes ipaddr unique or superior other than a claim to being lightweight. After downloading several of the other IP address libraries (http://bit.ly/483Yw4), ipaddr appears to be the second largest, after the huge netaddr package. I worry that this discussion has focused too much on the details of ipaddr (and the false dichotomy of "ipaddr versus nothing"), without properly tackling the question of "What use-cases for IP addresses are sufficiently universal* that they belong in the standard library?" -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From a.badger at gmail.com Wed Sep 30 01:59:45 2009 From: a.badger at gmail.com (Toshio Kuratomi) Date: Tue, 29 Sep 2009 16:59:45 -0700 Subject: [Python-Dev] PEP 389: argparse - new command line parsing module In-Reply-To: References: <200909290127.03215.steve@pearwood.info> <9d153b7c0909281414w55515a9blec86d55a3fdeb362@mail.gmail.com> <79990c6b0909291331u36bf0693r52ffd1a07e9ef500@mail.gmail.com> <4AC2846E.8060304@g.nevcal.com> Message-ID: <4AC29F71.3000307@gmail.com> On 09/29/2009 04:38 PM, Steven Bethard wrote: > On Tue, Sep 29, 2009 at 3:04 PM, Glenn Linderman wrote: >> On approximately 9/29/2009 1:57 PM, came the following characters from the >> keyboard of Steven Bethard: >>> If you're not using argparse to write command line applications, then >>> I don't feel bad if you have to do a tiny bit of extra work to take >>> care of that use case. In this particular situation, all you have to >>> do is subclass ArgumentParser and override exit() to do whatever you >>> think it should do. > [snip] >>> There is only a single method in argparse that prints things, >>> _print_message(). So if you want it to do something else, you can >>> simply override it in a subclass. I can make that method public if >>> this is a common use case. >> >> Documenting both of these options would forestall people from thinking it is >> only useful for console applications. > > I'm totally fine with people thinking it is only useful for console > applications. That's what it's intended for. That said, if there are > people out there who want to use it for other applications, I'm happy > to make things easier for them if I know concretely what they want. > Note: on Unix systems, --help should still print to the terminal, not pop up a GUI text box with the help information. So being able to override the behaviour might be good but it is more than a simple, GUI vs console distinction. Are we talking about anything else than --help output (for the printing question)? About exit(), I agree with others about wanting to catch the exception myself and then choosing to exit from the code. I'm not sure that it's actually useful in practice, though...it might just feel cleaner but not actually be that helpful. -Toshio -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: OpenPGP digital signature URL: From brett at python.org Wed Sep 30 20:12:02 2009 From: brett at python.org (Brett Cannon) Date: Wed, 30 Sep 2009 11:12:02 -0700 Subject: [Python-Dev] transitioning from % to {} formatting In-Reply-To: <79990c6b0909300529u4383e02esa21b464b957fb015@mail.gmail.com> References: <4AC2CD65.30905@v.loewis.de> <79990c6b0909300529u4383e02esa21b464b957fb015@mail.gmail.com> Message-ID: On Wed, Sep 30, 2009 at 05:29, Paul Moore wrote: > 2009/9/30 Barry Warsaw : >> Although I hate the name 'dicttemplate', this seems like the best solution >> to me. ?Maybe it's good that 'dicttemplate' is so ugly though so that people >> will naturally prefer 'format' :). ?But I like this because there's not >> really any magic, it's explicit, and the decision is made by the coder at >> the call site. ?The implementation does not need to guess at all. > > Ignoring the "make it ugly to persuade people not to use it" aspect, > why not just use 'template'? Might confuse people with string.Template. -Brett From ncoghlan at gmail.com Wed Sep 30 13:06:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 30 Sep 2009 21:06:09 +1000 Subject: [Python-Dev] PEP 3144 review. In-Reply-To: <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> References: <8517e9350909140944x104bcc63g62f86b3af7c6ca9@mail.gmail.com> <4AC03410.4010104@v.loewis.de> <4AC11855.7080406@v.loewis.de> <4AC12AC6.80304@v.loewis.de> <4AC2A9D7.4080603@gmail.com> <5c6f2a5d0909300131g5fcb255eoe9403f54777fc677@mail.gmail.com> <79990c6b0909300252w1dec6e2dlc8efdd104e5116ed@mail.gmail.com> <5c6f2a5d0909300318p4b5404d4wb41e2add4ea8f88a@mail.gmail.com> Message-ID: <4AC33BA1.8050203@gmail.com> Mark Dickinson wrote: > On Wed, Sep 30, 2009 at 10:52 AM, Paul Moore wrote: >> 2009/9/30 Mark Dickinson : >>> Please could someone who understands the uses of IPNetwork better than >>> I do explain why the following wouldn't be a significant problem, if __eq__ >>> and __hash__ were modified to disregard the .ip attribute as suggested: >>> >>>>>> linus = IPv4Network('172.16.200.1/24') >>>>>> snoopy = IPv4Network('172.16.200.3/24') >>>>>> fqdn = {linus: 'linus.peanuts.net', snoopy: 'snoopy.peanuts.net'} >>>>>> fqdn[linus] # expecting 'linus.peanuts.net' >>> 'snoopy.peanuts.net' >> I certainly don't understand IPv4Network better than you :-) But that >> just looks wrong to me - linus and snoopy are hosts not networks, so >> making them IPv4Network classes seems wrong. I'd instinctively make >> them IPv4Address objects (which, I believe, would work). > > Okay, so maybe this is an abuse of IPv4Network. But I'd (mis?)understood > that the retention of the .ip attribute was precisely a convenience to allow > this sort of use. If not, then what's it for? I've read the PEP and almost > all of this thread, but I can't help feeling I'm still missing something. If > someone could point out the obvious to me I'd be grateful. You're not missing anything that I'm aware of - unlike the use case for accepting a denormalised network definition in the IPNetwork constructor (which has been made quite clear in the list discussion, even if it is still a bit vague in the PEP), the use case for *retaining* the host information on the network object hasn't been well articulated at all. The closest anyone has come to describing a use case is an indirect comment via Guido that leaving out the attribute would involve real code having to find somewhere else to stash the original address details (e.g. by passing around an IPAddres/IPNetwork tuple rather than just an IPNetwork object). However, while I'd still be a little happier if the .ip attribute went away all together and another means was found to conveniently associate an IPAddress and an IPNetwork, keeping it doesn't bother me anywhere near as much as having network equivalence defined in terms of something other than the network ID and the netmask. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia ---------------------------------------------------------------