From tim_one@email.msn.com Wed Mar 1 05:04:39 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 1 Mar 2000 00:04:39 -0500 Subject: [Patches] New checkappend.py tool In-Reply-To: <200002291309.IAA04656@eric.cnri.reston.va.us> Message-ID: <000301bf833b$a5af4720$412d153f@tim> [Guido] > I fixed a small error in the getopt error handling. Do you routinely > run with string exceptions? No, never. What happened is that I tested bad options using 1.5.2-- specifically because checkappend is meant to *run* under 1.5.2, in preparation for moving to 1.6 (I posted it to c.l.py too, so people could get a head start) --and the 1.5.2 getopt module raised string (not class) exceptions. It took me a while to figure out what the hell you were getting at with that question! BTW, I approve of your fix . making-a-career-of-tabnanny-derivatives-ly y'rs - tim From guido@python.org Wed Mar 1 05:44:10 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 01 Mar 2000 00:44:10 -0500 Subject: [Patches] Reference cycle collection for Python In-Reply-To: Your message of "Tue, 29 Feb 2000 15:34:21 MST." <20000229153421.A16502@acs.ucalgary.ca> References: <20000229153421.A16502@acs.ucalgary.ca> Message-ID: <200003010544.AAA13155@eric.cnri.reston.va.us> [I don't like to cross-post to patches and python-dev, but I think this belongs in patches because it's a followup to Neil's post there and also in -dev because of its longer-term importance.] Thanks for the new patches, Neil! We had a visitor here at CNRI today, Eric Tiedemann , who had a look at your patches before. Eric knows his way around the Scheme, Lisp and GC literature, and presented a variant on your approach which takes the bite out of the recursive passes. Eric had commented earlier on Neil's previous code, and I had used the morning to make myself familiar with Neil's code. This was relatively easy because Neil's code is very clear. Today, Eric proposed to do away with Neil's hash table altogether -- as long as we're wasting memory, we might as well add 3 fields to each container object rather than allocating the same amount in a separate hash table. Eric expects that this will run faster, although this obviously needs to be tried. Container types are: dict, list, tuple, class, instance; plus potentially user-defined container types such as kjbuckets. I have a feeling that function objects should also be considered container types, because of the cycle involving globals. Eric's algorithm, then, consists of the following parts. Each container object has three new fields: gc_next, gc_prev, and gc_refs. (Eric calls the gc_refs "refcount-zero".) We color objects white (initial), gray (root), black (scanned root). (The terms are explained later; we believe we don't actually need bits in the objects to store the color; see later.) All container objects are chained together in a doubly-linked list -- this is the same as Neil's code except Neil does it only for dicts. (Eric postulates that you need a list header.) When GC is activated, all objects are colored white; we make a pass over the entire list and set gc_refs equal to the refcount for each object. Next, we make another pass over the list to collect the internal references. Internal references are (just like in Neil's version) references from other container types. In Neil's version, this was recursive; in Eric's version, we don't need recursion, since the list already contains all containers. So we simple visit the containers in the list in turn, and for each one we go over all the objects it references and subtract one from *its* gc_refs field. (Eric left out the little detail that we ened to be able to distinguish between container and non-container objects amongst those references; this can be a flag bit in the type field.) Now, similar to Neil's version, all objects for which gc_refs == 0 have only internal references, and are potential garbage; all objects for which gc_refs > 0 are "roots". These have references to them from other places, e.g. from globals or stack frames in the Python virtual machine. We now start a second list, to which we will move all roots. The way to do this is to go over the first list again and to move each object that has gc_refs > 0 to the second list. Objects placed on the second list in this phase are considered colored gray (roots). Of course, some roots will reference some non-roots, which keeps those non-roots alive. We now make a pass over the second list, where for each object on the second list, we look at every object it references. If a referenced object is a container and is still in the first list (colored white) we *append* it to the second list (colored gray). Because we append, objects thus added to the second list will eventually be considered by this same pass; when we stop finding objects that sre still white, we stop appending to the second list, and we will eventually terminate this pass. Conceptually, objects on the second list that have been scanned in this pass are colored black (scanned root); but there is no need to to actually make the distinction. (How do we know whether an object pointed to is white (in the first list) or gray or black (in the second)? We could use an extra bitfield, but that's a waste of space. Better: we could set gc_refs to a magic value (e.g. 0xffffffff) when we move the object to the second list. During the meeting, I proposed to set the back pointer to NULL; that might work too but I think the gc_refs field is more elegant. We could even just test for a non-zero gc_refs field; the roots moved to the second list initially all have a non-zero gc_refs field already, and for the objects with a zero gc_refs field we could indeed set it to something arbitrary.) Once we reach the end of the second list, all objects still left in the first list are garbage. We can destroy them in a similar to the way Neil does this in his code. Neil calls PyDict_Clear on the dictionaries, and ignores the rest. Under Neils assumption that all cycles (that he detects) involve dictionaries, that is sufficient. In our case, we may need a type-specific "clear" function for containers in the type object. We discussed more things, but not as thoroughly. Eric & Eric stressed the importance of making excellent statistics available about the rate of garbage collection -- probably as data structures that Python code can read rather than debugging print statements. Eric T also sketched an incremental version of the algorithm, usable for real-time applications. This involved keeping the gc_refs field ("external" reference counts) up-to-date at all times, which would require two different versions of the INCREF/DECREF macros: one for adding/deleting a reference from a container, and another for adding/deleting a root reference. Also, a 4th color (red) was added, to distinguish between scanned roots and scanned non-roots. We decided not to work this out in more detail because the overhead cost appeared to be much higher than for the previous algorithm; instead, we recommed that for real-time requirements the whole GC is disabled (there should be run-time controls for this, not just compile-time). We also briefly discussed possibilities for generational schemes. The general opinion was that we should first implement and test the algorithm as sketched above, and then changes or extensions could be made. I was pleasantly surprised to find Neil's code in my inbox when we came out of the meeting; I think it would be worthwhile to compare and contrast the two approaches. (Hm, maybe there's a paper in it?) The rest of the afternoon was spent discussing continuations, coroutines and generators, and the fundamental reason why continuations are so hard (the C stack getting in the way everywhere). But that's a topic for another mail, maybe. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim_one@email.msn.com Wed Mar 1 11:26:21 2000 From: tim_one@email.msn.com (Tim Peters) Date: Wed, 1 Mar 2000 06:26:21 -0500 Subject: [Python-Dev] Re: [Patches] Reference cycle collection for Python In-Reply-To: <200003010544.AAA13155@eric.cnri.reston.va.us> Message-ID: <001101bf8370$f881dfa0$412d153f@tim> Very briefly: [Guido] > ... > Today, Eric proposed to do away with Neil's hash table altogether -- > as long as we're wasting memory, we might as well add 3 fields to each > container object rather than allocating the same amount in a separate > hash table. Eric expects that this will run faster, although this > obviously needs to be tried. No, it doesn't : it will run faster. > Container types are: dict, list, tuple, class, instance; plus > potentially user-defined container types such as kjbuckets. I > have a feeling that function objects should also be considered > container types, because of the cycle involving globals. Note that the list-migrating steps you sketch later are basically the same as (but hairier than) the ones JimF and I worked out for M&S-on-RC a few years ago, right down to using appending to effect a breadth-first traversal without requiring recursion -- except M&S doesn't have to bother accounting for sources of refcounts. Since *this* scheme does more work per item per scan, to be as fast in the end it has to touch less stuff than M&S. But the more kinds of types you track, the more stuff this scheme will have to chase. The tradeoffs are complicated & unclear, so I'll just raise an uncomfortable meta-point : you balked at M&S the last time around because of the apparent need for two link fields + a bit or two per object of a "chaseable type". If that's no longer perceived as being a showstopper, M&S should be reconsidered too. I happen to be a fan of both approaches . The worst part of M&S-on-RC (== the one I never had a good answer for) is that a non-cooperating extension type E can't be chased, hence objects reachable only from objects of type E never get marked, so are vulnerable to bogus collection. In the Neil/Toby scheme, objects of type E merely act as sources of "external" references, so the scheme fails safe (in the sense of never doing a bogus collection due to non-cooperating types). Hmm ... if both approaches converge on keeping a list of all chaseable objects, and being careful of uncoopoerating types, maybe the only real difference in the end is whether the root set is given explicitly (as in traditional M&S) or inferred indirectly (but where "root set" has a different meaning in the scheme you sketched). > ... > In our case, we may need a type-specific "clear" function for containers > in the type object. I think definitely, yes. full-speed-sideways-ly y'rs - tim From gerrit@nl.linux.org Wed Mar 1 13:56:11 2000 From: gerrit@nl.linux.org (gerrit@nl.linux.org) Date: Wed, 1 Mar 2000 14:56:11 +0100 Subject: [Patches] filecmp.py: new function to compare file _objects_ Message-ID: <20000301145611.A6227@nl.linux.org> --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Hello, I changed the filecmp.py module a bit to put the comparing of the file objects in a seperate function 'cmpfp', which can be used by the programmer. I needed this functionality for comparing two cStringIO file objects. -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! --u3/rZRmxL6MmkK24 Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="filecmp.py.diff" *** /tmp/filecmp.py Tue Feb 29 10:33:36 2000 --- filecmp.py Tue Feb 29 10:50:22 2000 *************** *** 51,57 **** def _do_cmp(f1, f2): bufsize = BUFSIZE fp1 , fp2 = open(f1, 'rb'), open(f2, 'rb') while 1: ! b1, b2 = fp1.read(bufsize), fp2.read(bufsize) if b1!=b2: return 0 if not b1: return 1 --- 51,60 ---- def _do_cmp(f1, f2): bufsize = BUFSIZE fp1 , fp2 = open(f1, 'rb'), open(f2, 'rb') + return cmpfp(fp1, fp2) + + def cmpfp(fp1, fp2): while 1: ! b1, b2 = fp1.read(BUFSIZE), fp2.read(BUFSIZE) if b1!=b2: return 0 if not b1: return 1 --u3/rZRmxL6MmkK24-- From gerrit@nl.linux.org Wed Mar 1 13:57:21 2000 From: gerrit@nl.linux.org (gerrit@nl.linux.org) Date: Wed, 1 Mar 2000 14:57:21 +0100 Subject: [Patches] Re: filecmp.py: new function to compare file _objects_ In-Reply-To: <20000301145611.A6227@nl.linux.org>; from gerrit@nl.linux.org on Wed, Mar 01, 2000 at 02:56:11PM +0100 References: <20000301145611.A6227@nl.linux.org> Message-ID: <20000301145721.A6260@nl.linux.org> > Hello, > > I changed the filecmp.py module a bit to put the comparing of the ... Ooops, please ignore. I pressed the wrong key, I wanted to cancel this email but I sent it... sorry! ~~~~~ regards, Gerrit. -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! From Vladimir.Marangozov@inrialpes.fr Wed Mar 1 17:07:07 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Wed, 1 Mar 2000 18:07:07 +0100 (CET) Subject: [Python-Dev] Re: [Patches] Reference cycle collection for Python In-Reply-To: <200003010544.AAA13155@eric.cnri.reston.va.us> from "Guido van Rossum" at Mar 01, 2000 12:44:10 AM Message-ID: <200003011707.SAA01310@python.inrialpes.fr> Guido van Rossum wrote: > > Thanks for the new patches, Neil! Thanks from me too! I notice, however, that hash_resize() still uses a malloc call instead of PyMem_NEW. Neil, please correct this in your version immediately ;-) > > We had a visitor here at CNRI today, Eric Tiedemann > , who had a look at your patches before. Eric > knows his way around the Scheme, Lisp and GC literature, and presented > a variant on your approach which takes the bite out of the recursive > passes. Avoiding the recursion is valuable, as long we're optimizing the implementation of one particular scheme. It doesn't bother me that Neil's scheme is recursive, because I still perceive his code as a proof of concept. You're presenting here another scheme based on refcounts arithmetic, generalized for all container types. The linked list implementation of this generalized scheme is not directly related to the logic. I have some suspitions on the logic, so you'll probably want to elaborate a bit more on it, and convince me that this scheme would actually work. > Today, Eric proposed to do away with Neil's hash table altogether -- > as long as we're wasting memory, we might as well add 3 fields to each > container object rather than allocating the same amount in a separate > hash table. I cannot agree so easily with this statement, but you should have expecting this from me :-) If we're about to opimize storage, I have good reasons to believe that we don't need 3 additional slots per container (but 1 for gc_refs, yes). We could certainly envision allocating the containers within memory pools of 4K (just as it is done in pymalloc, and close to what we have for ints & floats). These pools would be labaled as "container's memory", they would obviously be under our control, and we'd have additional slots per pool, not per object. As long as we isolate the containers from the rest, we can enumerate them easily by walking though the pools. But I'm willing to defer this question for now, as it involves the object allocators (the builtin allocators + PyObject_NEW for extension types E -- user objects of type E would be automatically taken into account for GC if there's a flag in the type struct which identifies them as containers). > Eric expects that this will run faster, although this obviously needs > to be tried. Definitely, although I trust Eric & Tim :-) > > Container types are: dict, list, tuple, class, instance; plus > potentially user-defined container types such as kjbuckets. I have a > feeling that function objects should also be considered container > types, because of the cycle involving globals. + other extension container types. And I insist. Don't forget that we're planning to merge types and classes... > > Eric's algorithm, then, consists of the following parts. > > Each container object has three new fields: gc_next, gc_prev, and > gc_refs. (Eric calls the gc_refs "refcount-zero".) > > We color objects white (initial), gray (root), black (scanned root). > (The terms are explained later; we believe we don't actually need bits > in the objects to store the color; see later.) > > All container objects are chained together in a doubly-linked list -- > this is the same as Neil's code except Neil does it only for dicts. > (Eric postulates that you need a list header.) > > When GC is activated, all objects are colored white; we make a pass > over the entire list and set gc_refs equal to the refcount for each > object. Step 1: for all containers, c->gc_refs = c->ob_refcnt > > Next, we make another pass over the list to collect the internal > references. Internal references are (just like in Neil's version) > references from other container types. In Neil's version, this was > recursive; in Eric's version, we don't need recursion, since the list > already contains all containers. So we simple visit the containers in > the list in turn, and for each one we go over all the objects it > references and subtract one from *its* gc_refs field. (Eric left out > the little detail that we ened to be able to distinguish between > container and non-container objects amongst those references; this can > be a flag bit in the type field.) Step 2: c->gc_refs = c->gc_refs - Nb_referenced_containers_from_c I guess that you realize that after this step, gc_refs can be zero or negative. I'm not sure that you collect "internal" references here (references from other container types). A list referencing 20 containers, being itself referenced by one container + one static variable + two times from the runtime stack, has an initial refcount == 4, so we'll end up with gc_refs == -16. A tuple referencing 1 list, referenced once by the stack, will end up with gc_refs == 0. Neil's scheme doesn't seem to have this "property". > > Now, similar to Neil's version, all objects for which gc_refs == 0 > have only internal references, and are potential garbage; all objects > for which gc_refs > 0 are "roots". These have references to them from > other places, e.g. from globals or stack frames in the Python virtual > machine. > Agreed, some roots have gc_refs > 0 I'm not sure that all of them have it, though... Do they? > We now start a second list, to which we will move all roots. The way > to do this is to go over the first list again and to move each object > that has gc_refs > 0 to the second list. Objects placed on the second > list in this phase are considered colored gray (roots). > Step 3: Roots with gc_refs > 0 go to the 2nd list. All c->gc_refs <= 0 stay in the 1st list. > Of course, some roots will reference some non-roots, which keeps those > non-roots alive. We now make a pass over the second list, where for > each object on the second list, we look at every object it references. > If a referenced object is a container and is still in the first list > (colored white) we *append* it to the second list (colored gray). > Because we append, objects thus added to the second list will > eventually be considered by this same pass; when we stop finding > objects that sre still white, we stop appending to the second list, > and we will eventually terminate this pass. Conceptually, objects on > the second list that have been scanned in this pass are colored black > (scanned root); but there is no need to to actually make the > distinction. > Step 4: Closure on reachable containers which are all moved to the 2nd list. (Assuming that the objects are checked only via their type, without involving gc_refs) > (How do we know whether an object pointed to is white (in the first > list) or gray or black (in the second)? Good question? :-) > We could use an extra bitfield, but that's a waste of space. > Better: we could set gc_refs to a magic value (e.g. 0xffffffff) when > we move the object to the second list. I doubt that this would work for the reasons mentioned above. > During the meeting, I proposed to set the back pointer to NULL; that > might work too but I think the gc_refs field is more elegant. We could > even just test for a non-zero gc_refs field; the roots moved to the > second list initially all have a non-zero gc_refs field already, and > for the objects with a zero gc_refs field we could indeed set it to > something arbitrary.) Not sure that "arbitrary" is a good choice if the differentiation is based solely on gc_refs. > > Once we reach the end of the second list, all objects still left in > the first list are garbage. We can destroy them in a similar to the > way Neil does this in his code. Neil calls PyDict_Clear on the > dictionaries, and ignores the rest. Under Neils assumption that all > cycles (that he detects) involve dictionaries, that is sufficient. In > our case, we may need a type-specific "clear" function for containers > in the type object. Couldn't this be done in the object's dealloc function? Note that both Neil's and this scheme assume that garbage _detection_ and garbage _collection_ is an atomic operation. I must say that I don't care of having some living garbage if it doesn't hurt my work. IOW, the used criterion for triggering the detection phase _may_ eventually differ from the one used for the collection phase. But this is where we reach the incremental approaches, implying different reasoning as a whole. My point is that the introduction of a "clear" function depends on the adopted scheme, whose logic depends on pertinent statistics on memory consumption of the cyclic garbage. To make it simple, we first need stats on memory consumption, then we can discuss objectively on how to implement some particular GC scheme. I second Eric on the need for excellent statistics. > > The general opinion was that we should first implement and test the > algorithm as sketched above, and then changes or extensions could be > made. I'd like to see it discussed first in conjunction with (1) the possibility of having a proprietary malloc, (2) the envisioned type/class unification. Perhaps I'm getting too deep, but once something gets in, it's difficult to take it out, even when a better solution is found subsequently. Although I'm enthousiastic about this work on GC, I'm not in a position to evaluate the true benefits of the proposed schemes, as I still don't have a basis for evaluating how much garbage my program generates and whether it hurts the interpreter compared to its overal memory consumption. > > I was pleasantly surprised to find Neil's code in my inbox when we > came out of the meeting; I think it would be worthwhile to compare and > contrast the two approaches. (Hm, maybe there's a paper in it?) I'm all for it! -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From jeremy@cnri.reston.va.us Wed Mar 1 17:53:13 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Wed, 1 Mar 2000 12:53:13 -0500 (EST) Subject: [Python-Dev] Re: [Patches] Reference cycle collection for Python In-Reply-To: <200003011707.SAA01310@python.inrialpes.fr> References: <200003010544.AAA13155@eric.cnri.reston.va.us> <200003011707.SAA01310@python.inrialpes.fr> Message-ID: <14525.22793.963077.707198@goon.cnri.reston.va.us> >>>>> "VM" == Vladimir Marangozov writes: [">>" == Guido explaining Eric Tiedemann's GC design] >> Next, we make another pass over the list to collect the internal >> references. Internal references are (just like in Neil's >> version) references from other container types. In Neil's >> version, this was recursive; in Eric's version, we don't need >> recursion, since the list already contains all containers. So we >> simple visit the containers in the list in turn, and for each one >> we go over all the objects it references and subtract one from >> *its* gc_refs field. (Eric left out the little detail that we >> ened to be able to distinguish between container and >> non-container objects amongst those references; this can be a >> flag bit in the type field.) VM> Step 2: c->gc_refs = c->gc_refs - VM> Nb_referenced_containers_from_c VM> I guess that you realize that after this step, gc_refs can be VM> zero or negative. I think Guido's explanation is slightly ambiguous. When he says, "subtract one from *its" gc_refs field" he means subtract one from the _contained_ object's gc_refs field. VM> I'm not sure that you collect "internal" references here VM> (references from other container types). A list referencing 20 VM> containers, being itself referenced by one container + one VM> static variable + two times from the runtime stack, has an VM> initial refcount == 4, so we'll end up with gc_refs == -16. The strategy is not that the container's gc_refs is decremented once for each object it contains. Rather, the container decrements each contained object's gc_refs by one. So you should never end of with gc_refs < 0. >> During the meeting, I proposed to set the back pointer to NULL; >> that might work too but I think the gc_refs field is more >> elegant. We could even just test for a non-zero gc_refs field; >> the roots moved to the second list initially all have a non-zero >> gc_refs field already, and for the objects with a zero gc_refs >> field we could indeed set it to something arbitrary.) I believe we discussed this further and concluded that setting the back pointer to NULL would not work. If we make the second list doubly-linked (like the first one), it is trivial to end GC by swapping the first and second lists. If we've zapped the NULL pointer, then we have to go back and re-set them all. Jeremy From gward@cnri.reston.va.us Wed Mar 1 18:51:50 2000 From: gward@cnri.reston.va.us (Greg Ward) Date: Wed, 1 Mar 2000 13:51:50 -0500 Subject: [Patches] Fix compiler warning in posixmodule.c Message-ID: <20000301135149.A21750@cnri.reston.va.us> --wac7ysb48OaltWcw Content-Type: text/plain; charset=us-ascii There're a couple of minor compiler warnings in the CVS version of Modules/posixmodule.c (at least under Solaris 2.6 with gcc 2.8.1, when configured *without* threads). Turns out that it's due to missing prototypes for 'ctermid_r()' and 'tmpnam_r()' -- those functions *are* in libc (hence HAVE_CTERMID_R and HAVE_TMPNAME_R are defined), but you don't get the prototypes unless you compile with -D_REENTRANT -- which is not the case for an unthreaded Python. Attached is a two-line patch that silences the compiler warnings by only using the "_r" versions in a threaded Python. Greg --wac7ysb48OaltWcw Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=posix-patch Index: posixmodule.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Modules/posixmodule.c,v retrieving revision 2.124 diff -c -r2.124 posixmodule.c *** posixmodule.c 2000/02/25 17:51:00 2.124 --- posixmodule.c 2000/03/01 18:46:40 *************** *** 649,655 **** if (!PyArg_ParseTuple(args, ":ctermid")) return NULL; ! #ifdef HAVE_CTERMID_R ret = ctermid_r(buffer); #else ret = ctermid(buffer); --- 649,655 ---- if (!PyArg_ParseTuple(args, ":ctermid")) return NULL; ! #if defined(HAVE_CTERMID_R) && defined(WITH_THREAD) ret = ctermid_r(buffer); #else ret = ctermid(buffer); *************** *** 3342,3348 **** if (!PyArg_ParseTuple(args, ":tmpnam")) return NULL; ! #ifdef HAVE_TMPNAM_R name = tmpnam_r(buffer); #else name = tmpnam(buffer); --- 3342,3348 ---- if (!PyArg_ParseTuple(args, ":tmpnam")) return NULL; ! #if defined(HAVE_TMPNAM_R) && defined(WITH_THREAD) name = tmpnam_r(buffer); #else name = tmpnam(buffer); --wac7ysb48OaltWcw-- From guido@python.org Wed Mar 1 18:55:48 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 01 Mar 2000 13:55:48 -0500 Subject: [Patches] Fix compiler warning in posixmodule.c In-Reply-To: Your message of "Wed, 01 Mar 2000 13:51:50 EST." <20000301135149.A21750@cnri.reston.va.us> References: <20000301135149.A21750@cnri.reston.va.us> Message-ID: <200003011855.NAA15862@eric.cnri.reston.va.us> Greg, PLease go ahead and check these in yourself! --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake@acm.org Wed Mar 1 19:01:15 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 1 Mar 2000 14:01:15 -0500 (EST) Subject: [Patches] Fix compiler warning in posixmodule.c In-Reply-To: <20000301135149.A21750@cnri.reston.va.us> References: <20000301135149.A21750@cnri.reston.va.us> Message-ID: <14525.26875.990397.926029@weyr.cnri.reston.va.us> Greg Ward writes: > Attached is a two-line patch that silences the compiler warnings by only > using the "_r" versions in a threaded Python. Actually, you missed the second #ifdef HAVE_TMPNAM_R; fix that, *then* check this in. ;) -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From gward@cnri.reston.va.us Wed Mar 1 19:17:39 2000 From: gward@cnri.reston.va.us (Greg Ward) Date: Wed, 1 Mar 2000 14:17:39 -0500 Subject: [Patches] Fix compiler warning in posixmodule.c In-Reply-To: <14525.26875.990397.926029@weyr.cnri.reston.va.us>; from fdrake@acm.org on Wed, Mar 01, 2000 at 02:01:15PM -0500 References: <20000301135149.A21750@cnri.reston.va.us> <14525.26875.990397.926029@weyr.cnri.reston.va.us> Message-ID: <20000301141738.C10657@cnri.reston.va.us> --2fHTh5uZTiUOsy+g Content-Type: text/plain; charset=us-ascii On 01 March 2000, Fred L. Drake, Jr. said: > Greg Ward writes: > > Attached is a two-line patch that silences the compiler warnings by only > > using the "_r" versions in a threaded Python. > > Actually, you missed the second #ifdef HAVE_TMPNAM_R; fix that, > *then* check this in. ;) Oops! Thanks Fred -- just shows that no patch is too trivial to escape peer review. Second patch attached -- this is relative to the original code, not to my first patch. This one's cleaner, as it defines USE_CTERMID_R and USE_TMPNAM_R up at the top of the file, so the preprocessor logic deep in the guts is simple: "#ifdef USE_CTERMID_R". Is the "USE_XXX" terminology OK? What about the placement of these two #define's (after all the other conditional #define's at the top of the file)? Greg --2fHTh5uZTiUOsy+g Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=posix-patch Index: posixmodule.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Modules/posixmodule.c,v retrieving revision 2.124 diff -c -r2.124 posixmodule.c *** posixmodule.c 2000/02/25 17:51:00 2.124 --- posixmodule.c 2000/03/01 19:12:21 *************** *** 274,279 **** --- 274,289 ---- #endif /* UNION_WAIT */ + /* Don't use the "_r" form if we don't need it (also, won't have a + prototype for it, at least on Solaris -- maybe others as well?). */ + #if defined(HAVE_CTERMID_R) && defined(WITH_THREAD) + #define USE_CTERMID_R + #endif + + #if defined(HAVE_TMPNAM_R) && defined(WITH_THREAD) + #define USE_TMPNAM_R + #endif + /* Return a dictionary corresponding to the POSIX environment table */ #if !defined(_MSC_VER) && ( !defined(__WATCOMC__) || defined(__QNX__) ) *************** *** 649,655 **** if (!PyArg_ParseTuple(args, ":ctermid")) return NULL; ! #ifdef HAVE_CTERMID_R ret = ctermid_r(buffer); #else ret = ctermid(buffer); --- 659,665 ---- if (!PyArg_ParseTuple(args, ":ctermid")) return NULL; ! #ifdef USE_CTERMID_R ret = ctermid_r(buffer); #else ret = ctermid(buffer); *************** *** 3342,3348 **** if (!PyArg_ParseTuple(args, ":tmpnam")) return NULL; ! #ifdef HAVE_TMPNAM_R name = tmpnam_r(buffer); #else name = tmpnam(buffer); --- 3352,3358 ---- if (!PyArg_ParseTuple(args, ":tmpnam")) return NULL; ! #ifdef USE_TMPNAM_R name = tmpnam_r(buffer); #else name = tmpnam(buffer); *************** *** 3350,3356 **** if (name == NULL) { PyErr_SetObject(PyExc_OSError, Py_BuildValue("is", 0, ! #ifdef HAVE_TMPNAM_R "unexpected NULL from tmpnam_r" #else "unexpected NULL from tmpnam" --- 3360,3366 ---- if (name == NULL) { PyErr_SetObject(PyExc_OSError, Py_BuildValue("is", 0, ! #ifdef USE_TMPNAM_R "unexpected NULL from tmpnam_r" #else "unexpected NULL from tmpnam" --2fHTh5uZTiUOsy+g-- From nascheme@enme.ucalgary.ca Wed Mar 1 19:29:02 2000 From: nascheme@enme.ucalgary.ca (nascheme@enme.ucalgary.ca) Date: Wed, 1 Mar 2000 12:29:02 -0700 Subject: [Python-Dev] Re: [Patches] Reference cycle collection for Python In-Reply-To: <200003011707.SAA01310@python.inrialpes.fr>; from marangoz@python.inrialpes.fr on Wed, Mar 01, 2000 at 06:07:07PM +0100 References: <200003010544.AAA13155@eric.cnri.reston.va.us> <200003011707.SAA01310@python.inrialpes.fr> Message-ID: <20000301122902.B7773@acs.ucalgary.ca> On Wed, Mar 01, 2000 at 06:07:07PM +0100, Vladimir Marangozov wrote: > Guido van Rossum wrote: > > Once we reach the end of the second list, all objects still left in > > the first list are garbage. We can destroy them in a similar to the > > way Neil does this in his code. Neil calls PyDict_Clear on the > > dictionaries, and ignores the rest. Under Neils assumption that all > > cycles (that he detects) involve dictionaries, that is sufficient. In > > our case, we may need a type-specific "clear" function for containers > > in the type object. > > Couldn't this be done in the object's dealloc function? No, I don't think so. The object still has references to it. You have to be careful about how you break cycles so that memory is not accessed after it is freed. Neil -- "If elected mayor, my first act will be to kill the whole lot of you, and burn your town to cinders!" -- Groundskeeper Willie From fdrake@acm.org Wed Mar 1 19:47:33 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Wed, 1 Mar 2000 14:47:33 -0500 (EST) Subject: [Patches] Fix compiler warning in posixmodule.c In-Reply-To: <20000301141738.C10657@cnri.reston.va.us> References: <20000301135149.A21750@cnri.reston.va.us> <14525.26875.990397.926029@weyr.cnri.reston.va.us> <20000301141738.C10657@cnri.reston.va.us> Message-ID: <14525.29653.373806.584337@weyr.cnri.reston.va.us> Greg Ward writes: > Second patch attached -- this is relative to the original code, not to > my first patch. This one's cleaner, as it defines USE_CTERMID_R and > USE_TMPNAM_R up at the top of the file, so the preprocessor logic deep > in the guts is simple: "#ifdef USE_CTERMID_R". Is the "USE_XXX" > terminology OK? What about the placement of these two #define's (after > all the other conditional #define's at the top of the file)? I suppose that's OK, but I don't like having to add new #defines that have to be chased down. Since there's only three sites that use the two different tests, "just do it" seems to make more sense. -Fred From mhammond@skippinet.com.au Thu Mar 2 06:41:35 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 2 Mar 2000 17:41:35 +1100 Subject: [Patches] Bug in my recent error patch Message-ID: Doh! I had this code in place while we decided exactly where to put the new error, and forgot to remove it! I nearly have a win32reg, complete with test code :-) I am passing it by Trent first to check for 64 bit and alpha correctness... Thanks, Mark. RCS file: /projects/cvsroot/python/dist/src/Python/errors.c,v retrieving revision 2.43 diff -r2.43 errors.c 387c387 < PyErr_SetObject(PyExc_EnvironmentError, v); --- > PyErr_SetObject(PyExc_WindowsError, v); From mhammond@skippinet.com.au Thu Mar 2 08:16:18 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 2 Mar 2000 19:16:18 +1100 Subject: [Patches] Bug in my recent error patch (re-posted) Message-ID: Disclaimer and check-in message this time :-) Checkin: New windows error raising function raised the wrong error! Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Thanks, Mark. RCS file: /projects/cvsroot/python/dist/src/Python/errors.c,v retrieving revision 2.43 diff -r2.43 errors.c 387c387 < PyErr_SetObject(PyExc_EnvironmentError, v); --- > PyErr_SetObject(PyExc_WindowsError, v); _______________________________________________ Patches mailing list Patches@python.org http://www.python.org/mailman/listinfo/patches From sjoerd@oratrix.nl Thu Mar 2 16:50:22 2000 From: sjoerd@oratrix.nl (Sjoerd Mullender) Date: Thu, 02 Mar 2000 17:50:22 +0100 Subject: [Patches] pdb patch for multi-line argument lists Message-ID: <20000302165023.81D27301CF9@bireme.oratrix.nl> When you set a breakpoint on a function with a multi-line argument list, the breakpoint is actually set on the second line of the arguments instead of the first line of the body. This patch fixes that. Index: pdb.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/pdb.py,v retrieving revision 1.42 diff -u -r1.42 pdb.py --- pdb.py 2000/02/04 15:39:30 1.42 +++ pdb.py 2000/03/02 16:44:29 @@ -329,29 +329,35 @@ # code parse time. We don't want that, so all breakpoints # set at 'def' statements are moved one line onward if line[:3] == 'def': - incomment = '' + instr = '' + brackets = 0 while 1: + skipone = 0 + for c in line: + if instr: + if skipone: + skipone = 0 + elif c == '\\': + skipone = 1 + elif c == instr: + instr = '' + elif c == '#': + break + elif c in ('"',"'"): + instr = c + elif c in ('(','{','['): + brackets = brackets + 1 + elif c in (')','}',']'): + brackets = brackets - 1 lineno = lineno+1 line = linecache.getline(filename, lineno) if not line: print 'end of file' return 0 line = string.strip(line) - if incomment: - if len(line) < 3: continue - if (line[-3:] == incomment): - incomment = '' - continue if not line: continue # Blank line - if len(line) >= 3: - if (line[:3] == '"""' - or line[:3] == "'''"): - if line[-3:] == line[:3]: - # one-line string - continue - incomment = line[:3] - continue - if line[0] != '#': break + if brackets <= 0 and line[0] not in ('#','"',"'"): + break return lineno def do_enable(self, arg): I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Sjoerd Mullender From tim_one@email.msn.com Fri Mar 3 04:37:33 2000 From: tim_one@email.msn.com (Tim Peters) Date: Thu, 2 Mar 2000 23:37:33 -0500 Subject: [Patches] Prevent re module blowup in IDLE's PyParse.py Message-ID: <001101bf84ca$320c7280$682d153f@tim> Changes the one regexp in PyParse capable of making the re module blow the C stack when passed unreasonable <0.9 wink> program text. Jeremy Hylton provoked this with a program of the form: x = (1, 2, ... # 9997 lines deleted here 10000, ) Programs "like this" will no longer (no matter how many lines they contain) trigger re death. OTOH, you can now make another class of unreasonable program that will take much longer to parse. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. *** idle-0.5/PyParse.py Mon Jun 07 22:26:18 1999 --- new/PyParse.py Thu Mar 02 22:56:16 2000 *************** *** 83,97 **** \b """, re.VERBOSE).match ! # Chew up non-special chars as quickly as possible, but retaining ! # enough info to determine the last non-ws char seen; if match is ! # successful, and m.group(1) isn't None, m.end(1) less 1 is the ! # index of the last non-ws char matched. _chew_ordinaryre = re.compile(r""" ! (?: \s+ ! | ( [^\s[\](){}#'"\\]+ ) ! )+ """, re.VERBOSE).match # Build translation table to map uninteresting chars to "x", open --- 83,95 ---- \b """, re.VERBOSE).match ! # Chew up non-special chars as quickly as possible. If match is ! # successful, m.end() less 1 is the index of the last boring char ! # matched. If match is unsuccessful, the string starts with an ! # interesting char. _chew_ordinaryre = re.compile(r""" ! [^[\](){}#'"\\]+ """, re.VERBOSE).match # Build translation table to map uninteresting chars to "x", open *************** *** 386,395 **** # suck up all except ()[]{}'"#\\ m = _chew_ordinaryre(str, p, q) if m: ! i = m.end(1) - 1 # last non-ws (if any) if i >= 0: lastch = str[i] - p = m.end() if p >= q: break --- 384,397 ---- # suck up all except ()[]{}'"#\\ m = _chew_ordinaryre(str, p, q) if m: ! # we skipped at least one boring char ! p = m.end() ! # back up over totally boring whitespace ! i = p-1 # index of last boring char ! while i >= 0 and str[i] in " \t\n": ! i = i-1 if i >= 0: lastch = str[i] if p >= q: break From gerrit@nl.linux.org Fri Mar 3 16:22:41 2000 From: gerrit@nl.linux.org (gerrit@nl.linux.org) Date: Fri, 3 Mar 2000 17:22:41 +0100 Subject: [Patches] tempfile: /usr/tmp not FHS-compliant Message-ID: <20000303172241.A2317@nl.linux.org> --ikeVEW9yuYc//A+q Content-Type: text/plain; charset=us-ascii Hello, the File Hierarchy Standard only counts for Unices, but being FHS-for-unix-compliant isn't bad on other OS'es. Quoting from the FHS: 3.11 /tmp : Temporary files The /tmp directory shall be made available for programs that require temporary files. Although data stored in /tmp may be deleted in a site-specific manner, it is recommended that files and directories located in /tmp be deleted whenever the system is booted. Programs shall not assume that any files or directories in /tmp are preserved between invocations of the program. BEGIN RATIONALE IEEE standard P1003.2 (POSIX, part 2) makes requirements that are similar to the above section. FHS added the recommendation that /tmp be cleaned at boot time on the basis of historical precedent and common practice, but did not make it a requirement because system administration is not within the scope of this standard. END RATIONALE .../usr/tmp is old... The following symbolic links to directories may be present. This possibility is based on the need to preserve compatibility with older systems until all implementations can be assumed to use the /var hierarchy. /usr/spool -> /var/spool /usr/tmp -> /var/tmp /usr/spool/locks -> /var/lock Once a system no longer requires any one of the above symbolic links, the link may be removed, if desired. .../var/tmp is for more persistent temporary files... 5.12 /var/tmp : Temporary files preserved between system reboots The /var/tmp directory is made available for programs that require temporary files or directories that are preserved between system reboots. Therefore, data stored in /var/tmp is more persistent than data in /tmp. Files and directories located in /var/tmp must not be deleted when the system is booted. Although data stored in /var/tmp is typically deleted in a site-specific manner, it is recommended that deletions occur at a less frequent interval than /tmp. This patch changes the assignment of the 'attempdirs' list on line 26 to try /tmp first, /var/tmp next and /usr/tmp as third, instead of /usr/tmp first and /tmp next. The FHS can be found at: http://www.pathname.com/fhs/ ------------------------------------------------------------------ | I confirm that, to the best of my knowledge and belief, this | | contribution is free of any claims of third parties under | | copyright, patent or other rights or interests ("claims"). To | | the extent that I have any such claims, I hereby grant to CNRI a | | nonexclusive, irrevocable, royalty-free, worldwide license to | | reproduce, distribute, perform and/or display publicly, prepare | | derivative versions, and otherwise use this contribution as part | | of the Python software and its related documentation, or any | | derivative versions thereof, at no cost to CNRI or its licensed | | users, and to authorize others to do so. | | | | I acknowledge that CNRI may, at its sole discretion, decide | | whether or not to incorporate this contribution in the Python | | software and its related documentation. I further grant CNRI | | permission to use my name and other identifying information | | provided to CNRI by me for use in connection with the Python | | software and its related documentation. | ------------------------------------------------------------------ regards, Gerrit. -- Plies korekt enie bet ingglisj joe encauntur in mai imil mesusj! -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! --ikeVEW9yuYc//A+q Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="tempfile.py.diff" *** /tmp/tempfile.py Fri Mar 3 17:04:30 2000 --- tempfile.py Fri Mar 3 17:05:37 2000 *************** *** 23,29 **** pwd = os.getcwd() except (AttributeError, os.error): pwd = os.curdir ! attempdirs = ['/usr/tmp', '/tmp', pwd] if os.name == 'nt': attempdirs.insert(0, 'C:\\TEMP') attempdirs.insert(0, '\\TEMP') --- 23,29 ---- pwd = os.getcwd() except (AttributeError, os.error): pwd = os.curdir ! attempdirs = ['/tmp', '/var/tmp', '/usr/tmp', pwd] if os.name == 'nt': attempdirs.insert(0, 'C:\\TEMP') attempdirs.insert(0, '\\TEMP') --ikeVEW9yuYc//A+q-- From guido@python.org Fri Mar 3 16:31:26 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 03 Mar 2000 11:31:26 -0500 Subject: [Patches] tempfile: /usr/tmp not FHS-compliant In-Reply-To: Your message of "Fri, 03 Mar 2000 17:22:41 +0100." <20000303172241.A2317@nl.linux.org> References: <20000303172241.A2317@nl.linux.org> Message-ID: <200003031631.LAA21612@eric.cnri.reston.va.us> > This patch changes the assignment of the 'attempdirs' list on line 26 > to try /tmp first, /var/tmp next and /usr/tmp as third, instead of > /usr/tmp first and /tmp next. I think adding /var/tmp is a good one, but I'm not sure about moving /tmp first. I think that there are (used to be?) systems where /tmp doesn't have sufficient space but /usr/tmp does. On the other hand, on Solaris /tmp is a lot faster than /usr/tmp -- /tmp is an in-memory filesystem (backed by swap space) while /usr/tmp is on a regular local disk. Other opinions please? --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Fri Mar 3 18:07:22 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 3 Mar 2000 20:07:22 +0200 (IST) Subject: [Patches] tempfile: /usr/tmp not FHS-compliant In-Reply-To: <200003031631.LAA21612@eric.cnri.reston.va.us> Message-ID: On Fri, 3 Mar 2000, Guido van Rossum wrote: > I think adding /var/tmp is a good one, but I'm not sure about moving > /tmp first. I think that there are (used to be?) systems where /tmp > doesn't have sufficient space but /usr/tmp does. On the other hand, > on Solaris /tmp is a lot faster than /usr/tmp -- /tmp is an in-memory > filesystem (backed by swap space) while /usr/tmp is on a regular local > disk. > > Other opinions please? Personally I like Gerrit's patch, because of the FHS (note that my Linux system follows the FHS closely enough for the rationale to hold, for example.) The "no space on /tmp" argument is bogus: for *large* temporary file, the location will have to be patched at installation anyway, so it wouldn't do any good putting that wisdom in a cross-platform module. OTOH, the "sloppy python programs should be cleaned after" argument wins: I suspect there are more sloppy programs creating small files and forgetting to clean up (and /var/tmp is cleaned up less frquently) then programs needing large temporaries. -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html From gstein@lyra.org Fri Mar 3 23:07:49 2000 From: gstein@lyra.org (Greg Stein) Date: Fri, 3 Mar 2000 15:07:49 -0800 (PST) Subject: [Patches] tempfile: /usr/tmp not FHS-compliant In-Reply-To: <20000303172241.A2317@nl.linux.org> Message-ID: On Fri, 3 Mar 2000 gerrit@nl.linux.org wrote: >... > This patch changes the assignment of the 'attempdirs' list on line 26 > to try /tmp first, /var/tmp next and /usr/tmp as third, instead of > /usr/tmp first and /tmp next. +1 Makes eminent sense to me! Cheers, -g -- Greg Stein, http://www.lyra.org/ From Moshe Zadka Sat Mar 4 13:15:04 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 4 Mar 2000 15:15:04 +0200 (IST) Subject: [Patches] selfnanny.py: checking for "self" in every method Message-ID: This simple nanny checks a Python source for methods that don't have arguments, or that the first argument isn't called "self". This is an important enough style guideline that I believe it is worth putting in a check, besides, it would stop bugs like class foo: def __init__(): self.i = 1 a = foo() NameError: self ---------------- selfnanny.py ---------------------- #!/usr/local/bin/python import parser, symbol, types def check_file(file): return check_string(file.read()) def check_string(s): return check_ast(parser.suite(s)) def check_ast(ast): return check_tuple(ast.totuple(1)) # retain line numbers def check_tuple(tup, classname=None): if type(tup) != types.TupleType: return [] problems = [] if tup[0] == symbol.funcdef: if classname is not None: args = get_function_args(tup) if not args or args[0] != 'self': funcname, funcline = tup[2][1:] problems.append((funcline, funcname, classname)) classname = None if tup[0] == symbol.classdef: classname = tup[2][1] for t in tup[1:]: problems.extend(check_tuple(t, classname)) return problems def get_function_args(tup): ret = [] for t in tup[1:]: if t[0] == symbol.parameters: parameters = t arglist = () for t in parameters[1:]: if t[0] == symbol.varargslist: arglist = t for arg in arglist[1:]: if arg[0] == symbol.fpdef: ret.append(arg[1][1]) return ret def format_problem(problem): s = 'line %d: method %s in class %s missing first argument "self"' return s % problem def print_problems(problems): for problem in problems: print format_problem(problem) def nanny(filename): print_problems(check_file(open(filename))) if __name__=='__main__': import sys for filename in sys.argv[1:]: nanny(filename) ------------------- selfnanny.py --------------------- Legal: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html From guido@python.org Sat Mar 4 17:24:22 2000 From: guido@python.org (Guido van Rossum) Date: Sat, 04 Mar 2000 12:24:22 -0500 Subject: [Patches] selfnanny.py: checking for "self" in every method In-Reply-To: Your message of "Sat, 04 Mar 2000 15:15:04 +0200." References: Message-ID: <200003041724.MAA05053@eric.cnri.reston.va.us> Before we all start writing nannies and checkers, how about a standard API design first? I will want to call various nannies from a "Check" command that I plan to add to IDLE. I already did this with tabnanny, and found that it's barely possible -- it's really written to run like a script. Since parsing is expensive, we probably want to share the parse tree. Ideas? --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Sat Mar 4 18:02:54 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 4 Mar 2000 20:02:54 +0200 (IST) Subject: [Patches] selfnanny.py: checking for "self" in every method In-Reply-To: <200003041724.MAA05053@eric.cnri.reston.va.us> Message-ID: On Sat, 4 Mar 2000, Guido van Rossum wrote: > Before we all start writing nannies and checkers, how about a standard > API design first? I thoroughly agree -- we should have a standard API. I tried to write selfnanny so it could be callable from any API possible (e.g., it can take either a file, a string, an ast or a tuple representation) > I will want to call various nannies from a "Check" > command that I plan to add to IDLE. Very cool: what I imagine is a sort of modular PyLint. > I already did this with tabnanny, > and found that it's barely possible -- it's really written to run like > a script. Mine definitely isn't: it's designed to run both like a script and like a module. One outstanding bug: no docos. To be supplied upon request <0.5 wink>. I just wanted to float it out and see if people think that this particular nanny is worth while. > Since parsing is expensive, we probably want to share the parse tree. Yes. Probably as an AST, and transform to tuples/lists inside the checkers. > Ideas? Here's a strawman API: There's a package called Nanny Every module in that package should have a function called check_ast. It's argument is an AST object, and it's output should be a list of three-tuples: (line-number, error-message, None) or (line-number, error-message, (column-begin, column-end)) (each tuple can be a different form). Problems? (I'm CCing to python-dev. Please follow up to that discussion to python-dev only, as I don't believe it belongs in patches) -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html From Moshe Zadka Sat Mar 4 18:12:19 2000 From: Moshe Zadka (Moshe Zadka) Date: Sat, 4 Mar 2000 20:12:19 +0200 (IST) Subject: [Patches] in-overloading: done right, with Guido's comments folded in (fwd) In-Reply-To: <200002281507.KAA23049@eric.cnri.reston.va.us> Message-ID: On Mon, 28 Feb 2000, Guido van Rossum wrote: > Thanks, Moshe (and Greg for reviewing). All checked in! > > Request: please submit some testcases for the test suite. How could I say no to you? ================= test_contains.py ===================== from test_support import TestFailed class base_set: def __init__(self, el): self.el = el class set(base_set): def __contains__(self, el): return self.el == el class seq(base_set): def __getitem__(self, n): return [self.el][n] def check(ok, *args): if not ok: raise TestFailed, join(map(str, args), " ") a = base_set(1) b = set(1) c = seq(1) check(1 in b, "1 not in set(1)") check(0 not in b, "0 in set(1)") check(1 in c, "1 not in seq(1)") check(0 not in c, "0 in seq(1)") try: 1 in a check(0, "in base_set did not raise error") except AttributeError: pass try: 1 not in a check(0, "not in base_set did not raise error") except AttributeError: pass ========================= test_contains.py =================== ================= output/test_contains ===================== test_contains ==================output/test_contains ===================== > And of course, documentation is still missing. I'll try and handle that too. -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html From Moshe Zadka Sat Mar 4 22:45:27 2000 From: Moshe Zadka (Moshe Zadka) Date: Sun, 5 Mar 2000 00:45:27 +0200 (IST) Subject: [Patches] In overloading: not special casing strings Message-ID: Especially with moving unicode into the Python core, it makes little sense for strings to be treated specially by PySequence_Contains. Here's a patch (which basically just moves code around) to take care of this issue. Enjoy. Checkin message: Moving the character in string checking code into the string object. Diff: diff -c -r python/dist/src/Objects/abstract.c build/Objects/abstract.c *** python/dist/src/Objects/abstract.c Sat Mar 4 12:38:54 2000 --- build/Objects/abstract.c Sat Mar 4 09:00:03 2000 *************** *** 1121,1144 **** PyObject *x; PySequenceMethods *sq; - /* Special case for char in string */ - if (PyString_Check(w)) { - register char *s, *end; - register char c; - if (!PyString_Check(v) || PyString_Size(v) != 1) { - PyErr_SetString(PyExc_TypeError, - "string member test needs char left operand"); - return -1; - } - c = PyString_AsString(v)[0]; - s = PyString_AsString(w); - end = s + PyString_Size(w); - while (s < end) { - if (c == *s++) - return 1; - } - return 0; - } if(PyType_HasFeature(w->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) { sq = w->ob_type->tp_as_sequence; if(sq != NULL && sq->sq_contains != NULL) --- 1121,1126 ---- diff -c -r python/dist/src/Objects/stringobject.c build/Objects/stringobject.c *** python/dist/src/Objects/stringobject.c Sat Mar 4 12:39:02 2000 --- build/Objects/stringobject.c Sat Mar 4 08:55:04 2000 *************** *** 381,386 **** --- 381,404 ---- return PyString_FromStringAndSize(a->ob_sval + i, (int) (j-i)); } + static int + string_contains(a, el) + PyObject *a, *el; + { + register char *s, *end; + register char c; + if (!PyString_Check(el) || PyString_Size(el) != 1) + return 0; + c = PyString_AsString(el)[0]; + s = PyString_AsString(a); + end = s + PyString_Size(a); + while (s < end) { + if (c == *s++) + return 1; + } + return 0; + } + static PyObject * string_item(a, i) PyStringObject *a; *************** *** 516,521 **** --- 534,540 ---- (intintargfunc)string_slice, /*sq_slice*/ 0, /*sq_ass_item*/ 0, /*sq_ass_slice*/ + (objobjproc)string_contains /*sq_contains*/ }; static PyBufferProcs string_as_buffer = { %%%%%%%%%%%%%%%%%%%%%%% end patch %%%%%%%%%%%%%%%%%%%%%%%%%% Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html From bwarsaw@python.org Sun Mar 5 02:12:36 2000 From: bwarsaw@python.org (Barry Warsaw) Date: Sat, 4 Mar 2000 21:12:36 -0500 (EST) Subject: [Patches] Re: Fix for reference leak in current Python CVS References: <14528.58721.558708.827151@buffalo.fnal.gov> Message-ID: <14529.49812.391212.496053@anthem.cnri.reston.va.us> >>>>> "CGW" == Charles G Waldman writes: CGW> Apparently, somebody added code to `string_join' to do string CGW> coercion on non-string sequence elements (a nice addition), CGW> but forgot to decref the resulting values returned from CGW> PyObject_Str. That someone was probably me :) Anyway, I think you're right about the leaks, but your patch doesn't quite catch them all. Try this one. -Barry P.S. patches@python.org is now the best official place to send patches. -------------------- snip snip -------------------- Index: stringobject.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Objects/stringobject.c,v retrieving revision 2.55 diff -c -r2.55 stringobject.c *** stringobject.c 2000/02/29 13:59:28 2.55 --- stringobject.c 2000/03/05 02:03:59 *************** *** 709,716 **** goto finally; slen = PyString_GET_SIZE(sitem); while (reslen + slen + seplen >= sz) { ! if (_PyString_Resize(&res, sz*2)) goto finally; sz *= 2; p = PyString_AsString(res) + reslen; } --- 709,718 ---- goto finally; slen = PyString_GET_SIZE(sitem); while (reslen + slen + seplen >= sz) { ! if (_PyString_Resize(&res, sz*2)) { ! Py_DECREF(sitem); goto finally; + } sz *= 2; p = PyString_AsString(res) + reslen; } *************** *** 720,725 **** --- 722,728 ---- reslen += seplen; } memcpy(p, PyString_AS_STRING(sitem), slen); + Py_DECREF(sitem); p += slen; reslen += slen; } *************** *** 728,741 **** for (i = 0; i < seqlen; i++) { PyObject *item = PySequence_GetItem(seq, i); PyObject *sitem; ! if (!item || !(sitem = PyObject_Str(item))) { ! Py_XDECREF(item); goto finally; ! } slen = PyString_GET_SIZE(sitem); while (reslen + slen + seplen >= sz) { ! if (_PyString_Resize(&res, sz*2)) goto finally; sz *= 2; p = PyString_AsString(res) + reslen; } --- 731,750 ---- for (i = 0; i < seqlen; i++) { PyObject *item = PySequence_GetItem(seq, i); PyObject *sitem; ! ! if (!item) goto finally; ! sitem = PyObject_Str(item); ! Py_DECREF(item); ! if (!sitem) ! goto finally; ! slen = PyString_GET_SIZE(sitem); while (reslen + slen + seplen >= sz) { ! if (_PyString_Resize(&res, sz*2)) { ! Py_DECREF(sitem); goto finally; + } sz *= 2; p = PyString_AsString(res) + reslen; } *************** *** 745,750 **** --- 754,760 ---- reslen += seplen; } memcpy(p, PyString_AS_STRING(sitem), slen); + Py_DECREF(sitem); p += slen; reslen += slen; } From guido@python.org Mon Mar 6 16:12:18 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 06 Mar 2000 11:12:18 -0500 Subject: [Patches] PyMem [1/8] - intro In-Reply-To: Your message of "Fri, 25 Feb 2000 13:23:09 +0100." <200002251223.NAA20710@python.inrialpes.fr> References: <200002251223.NAA20710@python.inrialpes.fr> Message-ID: <200003061612.LAA10179@eric.cnri.reston.va.us> OK, in my copious spare time (TM Larry Wall) I had a good look at Vladimir's malloc patches. I'm torn. I like a lot of the changes, but I don't like having two new files. Especially the new include file "for internal use only" seriously bothers me: people are going to ignore the warning. There are no precedents: all files in the Include directory are for public use (and most are included by Python.h); all private files live in one of the other source directories (Parser, Objects, Python, Modules). I also dislike the fact that extensions are made second-class citizens who don't have a macro that expands directly to malloc(). I think I'd prefer to have the following set of APIs: Py_Malloc c.s.: unchanged (calls PyMem_MALLOC c.s., may call PyErr_NoMemory) _PyObject_New c.s.: unchanged PyObject_NEW c.s.: unchanged (#define using _PyObject_New c.s.) PyMem_MALLOC c.s.: #defined as malloc c.s., normally PyMem_Malloc c.s.: functions calling PyMem_MALLOC c.s. PyMem_NEW c.s.: #defined using PyMem_MALLOC c.s. I *like* the new PyObject_DEL and _PyObject_Del. (I would have liked to have them for my GC experiment, but that's another story.) I can explain the MS_COREDLL business: This is defined on Windows because the core is in a DLL. Since the caller may be in another DLL, and each DLL (potentially) has a different default allocator, and (in pre-Vladimir times) the type-specific deallocator typically calls free(), we (Mark & I) decided that the allocation should be done in the type-specific allocator. We changed the PyObject_NEW() macro to call malloc() and pass that into _PyObject_New() as a second argument. Looking back, I wish we had made this change unconditionally -- there's nothing wrong with doing it that way on Unix, too. But now that Vladimir is fixing all type-specific deallocators to call the new _PyObject_Del(), we can put the allocation back into _PyObject_New()! The only problem is compatibility with old DLLs on Windows. One solution is to keep the second argument to _PyObject_New() when MS_COREDLL is defined, but to change the PyObject_NEW() macro to pass NULL instead. The code of _PyObject_New() checks whether the pointer is NULL and then allocates the memory itself; if a non-NULL pointer is passed, it uses that and skips its own allocation. _PyObject_Del() can unconditionally free the memory: old DLLs won't call it (they call free() directly) and new DLLs will have used NULL for the second _PyObject_New() argument. I hate to ask this of Vladimir, but could you make a new patch set along the lines I sketched, without a pycore.h file? --Guido van Rossum (home page: http://www.python.org/~guido/) > From: Vladimir Marangozov > > I'm about to send a couple of patches relative to the malloc cleanup > discussed on python-dev. Since it affects many files in the distribution, > I'll send a patch per directory, which will facilitate their review. > > Please examine them carefully. Although I'm pretty confident in their > correctness, some details may have escaped my eyes due to the volume > of the modified code. > > Summary: > ======== > > Exported (public) interfaces > ---------------------------- > > - PyMem_{Malloc, Realloc, Free} -- don't call anything on failure > - Py_{Malloc, Realloc, Free} -- call PyErr_NoMemory() on failure > - _PyObject_{New, NewVar, Del} -- object constructors/destructor > > Macros: > - PyMem_{NEW, NEW_VAR, DEL} -- handy, use PyMem_{Malloc...} above > - PyObject_{NEW, NEW_VAR, DEL} -- handy, use _PyObject_{New...} above > > Private core interfaces > ----------------------- > > In addition to the public ones: > > - PyMem_{MALLOC, REALLOC, FREE} -- Python core allocator > > Macros: > - PyMem_{NEW, NEW_VAR, DEL} -- handy, use PyMem_{MALLOC...} above > - PyObject_DEL is inlined (PyMem_FREE) From Vladimir.Marangozov@inrialpes.fr Mon Mar 6 18:53:50 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Mon, 6 Mar 2000 19:53:50 +0100 (CET) Subject: [Patches] PyMem [1/8] - intro In-Reply-To: <200003061612.LAA10179@eric.cnri.reston.va.us> from "Guido van Rossum" at Mar 06, 2000 11:12:18 AM Message-ID: <200003061853.TAA09996@python.inrialpes.fr> Guido van Rossum wrote: > > OK, in my copious spare time (TM Larry Wall) I had a good look at > Vladimir's malloc patches. I'm torn. I like a lot of the changes, > but I don't like having two new files. Especially the new include > file "for internal use only" seriously bothers me: people are going to > ignore the warning. There are no precedents: all files in the Include > directory are for public use (and most are included by Python.h); all > private files live in one of the other source directories (Parser, > Objects, Python, Modules). I agree that pycore.h should be invisible, but eventually decided to put it in Include/ for easy review of what it does (black magic renaming of the public interfaces). Yes, it would bother me too if it gets exposed somehow... > > I also dislike the fact that extensions are made second-class citizens > who don't have a macro that expands directly to malloc(). I don't perceive them as 2nd class citizens, but as 1st class customers of Python memory, bought from the Python memory manager (cf. the short description I wrote in the API document). Note that extensions would allocate objects through _PyObject_New/Del, i.e. a function call would still be involved for _Python_objects_. Mem buffers (not objects) allocated in extensions can be obtained either through malloc() directly (perfectly legal), either through PyMem_Malloc(), as long as the chunks are released with the corresponding free()/PyMem_Free(). So your discomfort relates only with the extensions' mem buffers. The lack of macros that expand directly to malloc() is in fact the invisible implementation of the private Python memory concept. You don't need such macros -- one can still use malloc/free directly. But if Python memory is needed, the PyMem_* APIs are provided precisely for this purpose. (Think unresolved symbols in an object extension file -- if the extension uses Python memory, it should be redirected to the Python memory manager, i.e. have unresolved PyMem_* symbols. The extension can still use malloc() and free() for private purposes, though, resolved with libc's malloc/free). > > I think I'd prefer to have the following set of APIs: > > Py_Malloc c.s.: unchanged (calls PyMem_MALLOC c.s., may call PyErr_NoMemory) > _PyObject_New c.s.: unchanged > PyObject_NEW c.s.: unchanged (#define using _PyObject_New c.s.) > > PyMem_MALLOC c.s.: #defined as malloc c.s., normally > PyMem_Malloc c.s.: functions calling PyMem_MALLOC c.s. > > PyMem_NEW c.s.: #defined using PyMem_MALLOC c.s. What's the point of exporting PyMem_MALLOC? This would be misleading. A compiled 3rd party extension can be distributed with PyMem_MALLOC defined as malloc(), while the core at the customer's site may be compiled with PyMem_MALLOC defined as my_own_malloc_other_than_libc_malloc(). In this case, things would fail badly... The extension writer would be confident in using the Python API, and would expect that by using this API, the extension would work with any version of the core >= 1.6 In the patch, I was targetting "binary compatibility" between the core and the extensions. You seem to suggest only source code compatibility, where all extensions would need to be recompiled with the setup of the core... But even in this case, the difference between PyMem_MALLOC and PyMem_Malloc would be only the func call overhead (modulo _PyMem_EXTRA) => nobody would use PyMem_Malloc. Subtle issue: Note that I may have used PyMem_Malloc() instead of PyMem_MALLOC() in the core, by redefining PyMem_Malloc in pycore.h in terms of the core allocator. Thus, I would have had 1 API less and would have used the same APIs for the core and for the extensions. (i.e. no MALLOC/REALLOC/FREE macros at all) But the point is that PyMem_Malloc() has different semantics from PyMem_MALLOC() due to the _PyMem_EXTRA business and the non-NULL pointer result insurance. That's why I've left the MALLOC family in the core as is -- to mark the distinction... > > I *like* the new PyObject_DEL and _PyObject_Del. (I would have liked > to have them for my GC experiment, but that's another story.) > > I can explain the MS_COREDLL business: > > This is defined on Windows because the core is in a DLL. Since the > caller may be in another DLL, and each DLL (potentially) has a > different default allocator, and (in pre-Vladimir times) the > type-specific deallocator typically calls free(), we (Mark & I) > decided that the allocation should be done in the type-specific > allocator. We changed the PyObject_NEW() macro to call malloc() and > pass that into _PyObject_New() as a second argument. Aha - got it. > > Looking back, I wish we had made this change unconditionally -- > there's nothing wrong with doing it that way on Unix, too. > > But now that Vladimir is fixing all type-specific deallocators to call > the new _PyObject_Del(), we can put the allocation back into > _PyObject_New()! Right. > > The only problem is compatibility with old DLLs on Windows. One > solution is to keep the second argument to _PyObject_New() when > MS_COREDLL is defined, but to change the PyObject_NEW() macro to pass > NULL instead. The code of _PyObject_New() checks whether the pointer > is NULL and then allocates the memory itself; if a non-NULL pointer is > passed, it uses that and skips its own allocation. _PyObject_Del() > can unconditionally free the memory: old DLLs won't call it (they call > free() directly) and new DLLs will have used NULL for the second > _PyObject_New() argument. Yes, this would work even when in the old lib the malloc fails and a NULL pointer is passed to _PyObject_New()... > > I hate to ask this of Vladimir, but could you make a new patch set > along the lines I sketched, without a pycore.h file? Yes, in my copious future spare time, and if we agree on what to do. I'm not positive that your suggestion solves the problem as I intented to. -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From guido@python.org Mon Mar 6 19:39:16 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 06 Mar 2000 14:39:16 -0500 Subject: [Patches] PyMem [1/8] - intro In-Reply-To: Your message of "Mon, 06 Mar 2000 19:53:50 +0100." <200003061853.TAA09996@python.inrialpes.fr> References: <200003061853.TAA09996@python.inrialpes.fr> Message-ID: <200003061939.OAA10623@eric.cnri.reston.va.us> > Guido van Rossum wrote: > > > > OK, in my copious spare time (TM Larry Wall) I had a good look at > > Vladimir's malloc patches. I'm torn. I like a lot of the changes, > > but I don't like having two new files. Especially the new include > > file "for internal use only" seriously bothers me: people are going to > > ignore the warning. There are no precedents: all files in the Include > > directory are for public use (and most are included by Python.h); all > > private files live in one of the other source directories (Parser, > > Objects, Python, Modules). Vladimir: > I agree that pycore.h should be invisible, but eventually decided > to put it in Include/ for easy review of what it does (black magic > renaming of the public interfaces). Yes, it would bother me too if > it gets exposed somehow... Since "make install" uses "for i in $(srcdir)/Include/*.h" to install the headers, it is guaranteed to be exposed. Plus, people find where the source tree is and add its Include dir there. I'd really rather not have a new file for this. > > I also dislike the fact that extensions are made second-class citizens > > who don't have a macro that expands directly to malloc(). > > I don't perceive them as 2nd class citizens, but as 1st class customers > of Python memory, bought from the Python memory manager (cf. the short > description I wrote in the API document). I see them as 2nd class citizens since Python core code has macro to bypass a level of function calls but non-core code doesn't. Core users of PyCore_MALLOC and users of PyMem_MALLOC end up calling malloc directly, under normal circumstances. 3rd party code always has to call Py_Malloc which calls malloc. > Note that extensions would allocate objects through _PyObject_New/Del, > i.e. a function call would still be involved for _Python_objects_. As it always has been. > Mem buffers (not objects) allocated in extensions can be obtained either > through malloc() directly (perfectly legal), either through PyMem_Malloc(), > as long as the chunks are released with the corresponding free()/PyMem_Free(). > > So your discomfort relates only with the extensions' mem buffers. > The lack of macros that expand directly to malloc() is in fact the > invisible implementation of the private Python memory concept. You don't > need such macros -- one can still use malloc/free directly. But if Python > memory is needed, the PyMem_* APIs are provided precisely for this purpose. I want to export both a function and a macro (for each operation) to use the core allocator, where the macro is normally defined as malloc and the function uses the macro. Then 3rd party code that decides it is interested in binary compatibility can call the function while 3rd party code that decides it is more interested in speed can use the macro -- and still play nice with possible custom allocators, as long as you're willing to recompile. Note that a typical 3rd party binary that uses the macros will still be correct, as long as it frees its own memory -- its memory just won't be seen by a custom allocator. No big deal, usually. > (Think unresolved symbols in an object extension file -- if the extension > uses Python memory, it should be redirected to the Python memory manager, > i.e. have unresolved PyMem_* symbols. The extension can still use malloc() > and free() for private purposes, though, resolved with libc's malloc/free). But don't we want to discourage extensions from using malloc and free directly? Otherwise we'll lose some of the advantages of the custom allocator, right? (Tell me again what you need the custom allocator for?) > > I think I'd prefer to have the following set of APIs: > > > > Py_Malloc c.s.: unchanged (calls PyMem_MALLOC c.s., may call PyErr_NoMemory) > > _PyObject_New c.s.: unchanged > > PyObject_NEW c.s.: unchanged (#define using _PyObject_New c.s.) > > > > PyMem_MALLOC c.s.: #defined as malloc c.s., normally > > PyMem_Malloc c.s.: functions calling PyMem_MALLOC c.s. > > > > PyMem_NEW c.s.: #defined using PyMem_MALLOC c.s. > > What's the point of exporting PyMem_MALLOC? This would be misleading. Why misleading? It exists. I don't like to have macros that behave differently in the core than outside, because the core is so often used as an example. Also, things move into the core and sometimes out of the core; this shouldn't affect how you are supposed to code something. > A compiled 3rd party extension can be distributed with PyMem_MALLOC > defined as malloc(), while the core at the customer's site may be > compiled with PyMem_MALLOC defined as my_own_malloc_other_than_libc_malloc(). > In this case, things would fail badly... The extension writer would > be confident in using the Python API, and would expect that by using this > API, the extension would work with any version of the core >= 1.6 I'm not clear how this would "fail badly." It would simply not use the custom allocator in this case, but as I mentioned above, memory allocation and freeing would still work, using malloc. Since there are no additional semantics defined for using the custom allocator, I'm not sure what point there is in enforcing its use to this level. > In the patch, I was targetting "binary compatibility" between the core > and the extensions. You seem to suggest only source code compatibility, > where all extensions would need to be recompiled with the setup of the > core... See above -- only if they care. A custom allocator to me seems a pretty rare thing to encounter, and very unlikely in an installed version. > But even in this case, the difference between PyMem_MALLOC and PyMem_Malloc > would be only the func call overhead (modulo _PyMem_EXTRA) => nobody would > use PyMem_Malloc. There's a clear semantic difference (using PyMem_Malloc allows binary compatibility). That should be enough to let the user choose. > Subtle issue: Note that I may have used PyMem_Malloc() instead of ^^^ could > PyMem_MALLOC() in the core, by redefining PyMem_Malloc in pycore.h > in terms of the core allocator. Thus, I would have had 1 API less > and would have used the same APIs for the core and for the extensions. > (i.e. no MALLOC/REALLOC/FREE macros at all) Not nice though: I'd like to distinguish between (safe) Functions and (slightly less safe) MACROS. Often, I like to code things using Functions and only use MACROS when I find I have a performance problem. The MACROS mean more work for a human reader because you have to remember the rules that decide when it's safe to use the macros. > But the point is that PyMem_Malloc() has different semantics from > PyMem_MALLOC() due to the _PyMem_EXTRA business and the non-NULL > pointer result insurance. That's why I've left the MALLOC family > in the core as is -- to mark the distinction... Great. Another semantic difference: if you think you may be allocating 0 bytes and you definitely want a non-NULL pointer back, you have to use the function. Makes sense to me. > > I *like* the new PyObject_DEL and _PyObject_Del. (I would have liked > > to have them for my GC experiment, but that's another story.) > > > > I can explain the MS_COREDLL business: > > > > This is defined on Windows because the core is in a DLL. Since the > > caller may be in another DLL, and each DLL (potentially) has a > > different default allocator, and (in pre-Vladimir times) the > > type-specific deallocator typically calls free(), we (Mark & I) > > decided that the allocation should be done in the type-specific > > allocator. We changed the PyObject_NEW() macro to call malloc() and > > pass that into _PyObject_New() as a second argument. > > Aha - got it. > > > > > Looking back, I wish we had made this change unconditionally -- > > there's nothing wrong with doing it that way on Unix, too. > > > > But now that Vladimir is fixing all type-specific deallocators to call > > the new _PyObject_Del(), we can put the allocation back into > > _PyObject_New()! > > Right. > > > > > The only problem is compatibility with old DLLs on Windows. One > > solution is to keep the second argument to _PyObject_New() when > > MS_COREDLL is defined, but to change the PyObject_NEW() macro to pass > > NULL instead. The code of _PyObject_New() checks whether the pointer > > is NULL and then allocates the memory itself; if a non-NULL pointer is > > passed, it uses that and skips its own allocation. _PyObject_Del() > > can unconditionally free the memory: old DLLs won't call it (they call > > free() directly) and new DLLs will have used NULL for the second > > _PyObject_New() argument. > > Yes, this would work even when in the old lib the malloc fails and a NULL > pointer is passed to _PyObject_New()... > > > > > I hate to ask this of Vladimir, but could you make a new patch set > > along the lines I sketched, without a pycore.h file? > > Yes, in my copious future spare time, and if we agree on what to do. > I'm not positive that your suggestion solves the problem as I intented to. I hope you will give it some thought. The in-core/not-in-core distinction is evil in my mind. --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer@tismer.com Mon Mar 6 20:39:18 2000 From: tismer@tismer.com (Christian Tismer) Date: Mon, 06 Mar 2000 21:39:18 +0100 Subject: [Patches] Safe destruction of recursive objects Message-ID: <38C41776.B6FAAB07@tismer.com> When recursive objects are destroyed, the recursion depth of nested dealloc calls can cause a core dump. The trashcan macro/functions are written to avoid this. To keep the necessary changes as small as possible, the patch add just two macro lines which enclose the critical operation. The affected objects are dict, tuple, list, frame and traceback. The method: While destructing, a global counter tracks the recursion depth. It is incremented in the beginning macro and decremented in the ending macro. If the count hits its limit, the enclosed deallocation code is not executed. Instead, destruction is deferred by appending the object to a list. The ending macro checks wether we are on toplevel and if there is a list to destroy. If so, the list is made local and cleared out. This process will continue until there is nothing left over. I'd call this "elevator destructor." As an addition, I added some small changes to listmodule.c, in order to make strict argument handling in append et al optional. If this is undesired, simply remove that part of the patch. Checkin-message: added wrapping macros to dictobject.c, listobject.c, tupleobject.c, frameobject.c, traceback.c that safely prevends core dumps on stack overflow. New files: trashcan.c, trashcan.h . Diff: diff -c -b -P -r //d/python/python-1.5.2/Include/trashcan.h trash/Include/trashcan.h *** //d/python/python-1.5.2/Include/trashcan.h Thu Jan 01 01:00:00 1970 --- trash/Include/trashcan.h Mon Mar 06 22:13:56 2000 *************** *** 0 **** --- 1,54 ---- + #ifndef Py_TRASHCAN_H + #define Py_TRASHCAN_H + #ifdef __cplusplus + extern "C" { + #endif + /* + trashcan + CT 2k0130 + non-recursively destroy nested objects + + CT 2k0223 + redefinition for better locality and less overhead. + + Objects that want to be recursion safe need to use + the macroes + Py_TRASHCAN_SAFE_BEGIN(name) + and + Py_TRASHCAN_SAFE_END(name) + surrounding their actual deallocation code. + + It would be nice to do this using the thread state. + Also, we could do an exact stack measure then. + Unfortunately, deallocations also take place when + the thread state is undefined. + */ + + #define PyTrash_UNWIND_LEVEL 50 + + #define Py_TRASHCAN_SAFE_BEGIN(op) \ + { \ + ++_PyTrash_delete_nesting; \ + if (_PyTrash_delete_nesting < PyTrash_UNWIND_LEVEL) { \ + + #define Py_TRASHCAN_SAFE_END(op) \ + ;} \ + else \ + _PyTrash_deposit_object((PyObject*)op);\ + --_PyTrash_delete_nesting; \ + if (_PyTrash_delete_later && _PyTrash_delete_nesting <= 0) \ + _PyTrash_destroy_list(); \ + } \ + + extern DL_IMPORT(void) _PyTrash_deposit_object Py_PROTO((PyObject*)); + extern DL_IMPORT(void) _PyTrash_destroy_list Py_PROTO(()); + + extern DL_IMPORT(int) _PyTrash_delete_nesting; + extern DL_IMPORT(PyObject *) _PyTrash_delete_later; + + /* swap the "xx" to check the speed loss */ + + #define xxPy_TRASHCAN_SAFE_BEGIN(op) + #define xxPy_TRASHCAN_SAFE_END(op) ; + + #endif /* !Py_TRASHCAN_H */ diff -c -b -P -r //d/python/python-1.5.2/Objects/dictobject.c trash/Objects/dictobject.c *** //d/python/python-1.5.2/Objects/dictobject.c Sat Feb 26 20:35:38 2000 --- trash/Objects/dictobject.c Sat Feb 26 22:10:44 2000 *************** *** 33,38 **** --- 33,40 ---- #include "Python.h" + #include "trashcan.h" + /* * MINSIZE is the minimum size of a dictionary. *************** *** 479,484 **** --- 481,487 ---- { register int i; register dictentry *ep; + Py_TRASHCAN_SAFE_BEGIN(op) for (i = 0, ep = mp->ma_table; i < mp->ma_size; i++, ep++) { if (ep->me_key != NULL) { Py_DECREF(ep->me_key); *************** *** 489,494 **** --- 492,498 ---- } PyMem_XDEL(mp->ma_table); PyMem_DEL(mp); + Py_TRASHCAN_SAFE_END(mp) } static int diff -c -b -P -r //d/python/python-1.5.2/Objects/frameobject.c trash/Objects/frameobject.c *** //d/python/python-1.5.2/Objects/frameobject.c Sat Feb 26 20:35:40 2000 --- trash/Objects/frameobject.c Mon Mar 06 21:13:32 2000 *************** *** 38,43 **** --- 38,45 ---- #include "opcode.h" #include "structmember.h" + #include "trashcan.h" + #define OFF(x) offsetof(PyFrameObject, x) static struct memberlist frame_memberlist[] = { *************** *** 103,108 **** --- 105,111 ---- int i; PyObject **fastlocals; + Py_TRASHCAN_SAFE_BEGIN(f) /* Kill all local variables */ fastlocals = f->f_localsplus; for (i = f->f_nlocals; --i >= 0; ++fastlocals) { *************** *** 120,125 **** --- 123,129 ---- Py_XDECREF(f->f_exc_traceback); f->f_back = free_list; free_list = f; + Py_TRASHCAN_SAFE_END(f) } PyTypeObject PyFrame_Type = { diff -c -b -P -r //d/python/python-1.5.2/Objects/listobject.c trash/Objects/listobject.c *** //d/python/python-1.5.2/Objects/listobject.c Sat Feb 26 20:35:40 2000 --- trash/Objects/listobject.c Sun Mar 05 22:29:52 2000 *************** *** 33,38 **** --- 33,40 ---- #include "Python.h" + #include "trashcan.h" + #ifdef STDC_HEADERS #include #else *************** *** 215,220 **** --- 217,223 ---- PyListObject *op; { int i; + Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_item != NULL) { /* Do it backwards, for Christian Tismer. There's a simple test case where somehow this reduces *************** *** 227,232 **** --- 230,236 ---- free((ANY *)op->ob_item); } free((ANY *)op); + Py_TRASHCAN_SAFE_END(op) } static int *************** *** 562,574 **** return ins(self, i, v); } static PyObject * listappend(self, args) PyListObject *self; PyObject *args; { PyObject *v; ! if (!PyArg_ParseTuple(args, "O:append", &v)) return NULL; return ins(self, (int) self->ob_size, v); } --- 566,590 ---- return ins(self, i, v); } + #ifdef STRICT_LIST_APPEND + #define PyArg_ParseTuple_Compat1 PyArg_ParseTuple + #else + #define PyArg_ParseTuple_Compat1(args, format, ret) \ + ( \ + PyTuple_GET_SIZE(args) > 1 ? (*ret = args, 1) : \ + PyTuple_GET_SIZE(args) == 1 ? (*ret = PyTuple_GET_ITEM(args, 0), 1) : \ + PyArg_ParseTuple(args, format, ret) \ + ) + #endif + + static PyObject * listappend(self, args) PyListObject *self; PyObject *args; { PyObject *v; ! if (!PyArg_ParseTuple_Compat1(args, "O:append", &v)) return NULL; return ins(self, (int) self->ob_size, v); } *************** *** 1326,1332 **** int i; PyObject *v; ! if (!PyArg_ParseTuple(args, "O:index", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) --- 1342,1348 ---- int i; PyObject *v; ! if (!PyArg_ParseTuple_Compat1(args, "O:index", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) *************** *** 1347,1353 **** int i; PyObject *v; ! if (!PyArg_ParseTuple(args, "O:count", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) --- 1363,1369 ---- int i; PyObject *v; ! if (!PyArg_ParseTuple_Compat1(args, "O:count", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) *************** *** 1366,1372 **** int i; PyObject *v; ! if (!PyArg_ParseTuple(args, "O:remove", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) { --- 1382,1388 ---- int i; PyObject *v; ! if (!PyArg_ParseTuple_Compat1(args, "O:remove", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) { diff -c -b -P -r //d/python/python-1.5.2/Objects/tupleobject.c trash/Objects/tupleobject.c *** //d/python/python-1.5.2/Objects/tupleobject.c Sat Feb 26 20:35:44 2000 --- trash/Objects/tupleobject.c Sat Feb 26 22:23:22 2000 *************** *** 33,38 **** --- 33,40 ---- #include "Python.h" + #include "trashcan.h" + #ifndef MAXSAVESIZE #define MAXSAVESIZE 20 #endif *************** *** 172,177 **** --- 174,180 ---- { register int i; + Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_size > 0) { i = op->ob_size; while (--i >= 0) *************** *** 180,190 **** if (op->ob_size < MAXSAVESIZE) { op->ob_item[0] = (PyObject *) free_tuples[op->ob_size]; free_tuples[op->ob_size] = op; ! return; } #endif } free((ANY *)op); } static int --- 183,195 ---- if (op->ob_size < MAXSAVESIZE) { op->ob_item[0] = (PyObject *) free_tuples[op->ob_size]; free_tuples[op->ob_size] = op; ! goto done; /* return */ } #endif } free((ANY *)op); + done: + Py_TRASHCAN_SAFE_END(op) } static int diff -c -b -P -r //d/python/python-1.5.2/Python/traceback.c trash/Python/traceback.c *** //d/python/python-1.5.2/Python/traceback.c Sat Sep 18 22:49:40 1999 --- trash/Python/traceback.c Wed Feb 23 21:19:46 2000 *************** *** 33,38 **** --- 33,40 ---- #include "Python.h" + #include "trashcan.h" + #include "compile.h" #include "frameobject.h" #include "structmember.h" *************** *** 68,76 **** --- 70,80 ---- tb_dealloc(tb) tracebackobject *tb; { + Py_TRASHCAN_SAFE_BEGIN(tb) Py_XDECREF(tb->tb_next); Py_XDECREF(tb->tb_frame); PyMem_DEL(tb); + Py_TRASHCAN_SAFE_END(tb) } #define Tracebacktype PyTraceBack_Type diff -c -b -P -r //d/python/python-1.5.2/Python/trashcan.c trash/Python/trashcan.c *** //d/python/python-1.5.2/Python/trashcan.c Thu Jan 01 01:00:00 1970 --- trash/Python/trashcan.c Mon Mar 06 22:14:08 2000 *************** *** 0 **** --- 1,38 ---- + /* + trashcan + CT 2k0130 + non-recursively destroy nested objects + + CT 2k0223 + everything is now done in a macro. + + CT 2k0305 + modified to use functions, after Tim Peter's suggestion. + */ + + #include "Python.h" + + int _PyTrash_delete_nesting = 0; + PyObject * _PyTrash_delete_later = NULL; + + void + _PyTrash_deposit_object(op) + PyObject *op; + { + if (!_PyTrash_delete_later) + _PyTrash_delete_later = PyList_New(0); + if (_PyTrash_delete_later) + PyList_Append(_PyTrash_delete_later, (PyObject *)op); + } + + void + _PyTrash_destroy_list() + { + while (_PyTrash_delete_later) { + PyObject *shredder = _PyTrash_delete_later; + _PyTrash_delete_later = NULL; + ++_PyTrash_delete_nesting; + Py_DECREF(shredder); + --_PyTrash_delete_nesting; + } + } %%%%%%%%%%%%%%%%%%%%%%% end patch %%%%%%%%%%%%%%%%%%%%%%%%%% Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From guido@python.org Mon Mar 6 21:05:21 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 06 Mar 2000 16:05:21 -0500 Subject: [Patches] In overloading: not special casing strings In-Reply-To: Your message of "Sun, 05 Mar 2000 00:45:27 +0200." References: Message-ID: <200003062105.QAA11349@eric.cnri.reston.va.us> > Especially with moving unicode into the Python core, it makes little sense > for strings to be treated specially by PySequence_Contains. Here's a patch > (which basically just moves code around) to take care of this issue. Thanks. There's one thing broken in this version: you made the test for length==1 return false -- it should raise an exception. See the patches: > Enjoy. > > Checkin message: > Moving the character in string checking code into the string object. > > Diff: > diff -c -r python/dist/src/Objects/abstract.c build/Objects/abstract.c > *** python/dist/src/Objects/abstract.c Sat Mar 4 12:38:54 2000 > --- build/Objects/abstract.c Sat Mar 4 09:00:03 2000 > *************** > *** 1121,1144 **** > PyObject *x; > PySequenceMethods *sq; > > - /* Special case for char in string */ > - if (PyString_Check(w)) { > - register char *s, *end; > - register char c; > - if (!PyString_Check(v) || PyString_Size(v) != 1) { > - PyErr_SetString(PyExc_TypeError, > - "string member test needs char left operand"); > - return -1; > - } See: this is the exception I'm talking about. > - c = PyString_AsString(v)[0]; > - s = PyString_AsString(w); > - end = s + PyString_Size(w); > - while (s < end) { > - if (c == *s++) > - return 1; > - } > - return 0; > - } > if(PyType_HasFeature(w->ob_type, Py_TPFLAGS_HAVE_SEQUENCE_IN)) { > sq = w->ob_type->tp_as_sequence; > if(sq != NULL && sq->sq_contains != NULL) > --- 1121,1126 ---- > diff -c -r python/dist/src/Objects/stringobject.c build/Objects/stringobject.c > *** python/dist/src/Objects/stringobject.c Sat Mar 4 12:39:02 2000 > --- build/Objects/stringobject.c Sat Mar 4 08:55:04 2000 > *************** > *** 381,386 **** > --- 381,404 ---- > return PyString_FromStringAndSize(a->ob_sval + i, (int) (j-i)); > } > > + static int > + string_contains(a, el) > + PyObject *a, *el; > + { > + register char *s, *end; > + register char c; > + if (!PyString_Check(el) || PyString_Size(el) != 1) > + return 0; Here you just return false for the same condition! > + c = PyString_AsString(el)[0]; > + s = PyString_AsString(a); > + end = s + PyString_Size(a); > + while (s < end) { > + if (c == *s++) > + return 1; > + } > + return 0; > + } > + > static PyObject * > string_item(a, i) > PyStringObject *a; > *************** > *** 516,521 **** > --- 534,540 ---- > (intintargfunc)string_slice, /*sq_slice*/ > 0, /*sq_ass_item*/ > 0, /*sq_ass_slice*/ > + (objobjproc)string_contains /*sq_contains*/ > }; > > static PyBufferProcs string_as_buffer = { I'll fix it. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Mar 6 21:14:52 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 06 Mar 2000 16:14:52 -0500 Subject: [Patches] Safe destruction of recursive objects In-Reply-To: Your message of "Mon, 06 Mar 2000 21:39:18 +0100." <38C41776.B6FAAB07@tismer.com> References: <38C41776.B6FAAB07@tismer.com> Message-ID: <200003062114.QAA11563@eric.cnri.reston.va.us> > When recursive objects are destroyed, the recursion depth > of nested dealloc calls can cause a core dump. > The trashcan macro/functions are written to avoid this. > To keep the necessary changes as small as possible, > the patch add just two macro lines which enclose > the critical operation. Thanks, Christian! I was about to check this in, when I read the following: > + It would be nice to do this using the thread state. > + Also, we could do an exact stack measure then. > + Unfortunately, deallocations also take place when > + the thread state is undefined. Really? Do you have a reproducible example? One that only involves standard Python modules? It makes a difference whether this can happen with the thread state undefined but the global interpreter lock still held, or whether you think that there can be deallocations without the lock held -- the latter would break many of my assumptions! --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer@tismer.com Mon Mar 6 21:52:39 2000 From: tismer@tismer.com (Christian Tismer) Date: Mon, 06 Mar 2000 22:52:39 +0100 Subject: [Patches] Safe destruction of recursive objects References: <38C41776.B6FAAB07@tismer.com> <200003062114.QAA11563@eric.cnri.reston.va.us> Message-ID: <38C428A7.57CF095D@tismer.com> Hi Guido, > I was about to check this in, when I read the following: > > > + It would be nice to do this using the thread state. > > + Also, we could do an exact stack measure then. > > + Unfortunately, deallocations also take place when > > + the thread state is undefined. > > Really? Do you have a reproducible example? One that only involves > standard Python modules? I had forgotton about this already, but yes it is true. I had implemented this stuff with the tstate already but had to learn that this is impossible. > It makes a difference whether this can happen with the thread state > undefined but the global interpreter lock still held, or whether you > think that there can be deallocations without the lock held -- the > latter would break many of my assumptions! It happens on shutdown, for instance. There are many more cases in PythonWin btw, otherwise I would have changed this. Here the context when it happens in standard python: tupledealloc() line 183 + 3 bytes class_dealloc() line 142 + 28 bytes finierrors() line 2830 + 18 bytes Py_Finalize() line 250 Py_Main() line 306 PYTHON! 0040100f() 0012fb28() tstate is null in this case. I checked it with this debug stuff in tupleobject.c: """ /* Methods */ //debug begin #include "compile.h" #include "frameobject.h" //debug end static void tupledealloc(op) register PyTupleObject *op; { register int i; //debug begin PyThreadState *tstate = PyThreadState_GET(); if (tstate->frame && !PyFrame_Check(tstate->frame)) { int a = 42; /* breakpoint here */ } //debug end Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_size > 0) { i = op->ob_size; while (--i >= 0) Py_XDECREF(op->ob_item[i]); #if MAXSAVESIZE > 0 """ I was guessing that the memory would have been recycled, but in fact the tstate was simply NULL. cheers - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From Vladimir.Marangozov@inrialpes.fr Mon Mar 6 23:04:40 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Tue, 7 Mar 2000 00:04:40 +0100 (CET) Subject: [Patches] PyMem [1/8] - intro In-Reply-To: <200003061939.OAA10623@eric.cnri.reston.va.us> from "Guido van Rossum" at Mar 06, 2000 02:39:16 PM Message-ID: <200003062304.AAA10505@python.inrialpes.fr> Guido van Rossum wrote: > > [me] > > So your discomfort relates only with the extensions' mem buffers. > > The lack of macros that expand directly to malloc() is in fact the > > invisible implementation of the private Python memory concept. You don't > > need such macros -- one can still use malloc/free directly. But if Python > > memory is needed, the PyMem_* APIs are provided precisely for this purpose. > > I want to export both a function and a macro (for each operation) to > use the core allocator, where the macro is normally defined as malloc > and the function uses the macro. Obviously, you want them both :-) And I agree that the in-core/out-core distinction may be confusing. It's probably not the best way to implement the concept, but still, let me defend its design once again: - Besides the malloc cleanup, the goal is to be able to change the core allocator easily. If the core malloc is not changed, or if it cannot be changed easily, the net result of the cleanup is of little added value, for now. - We have to think about lots of legacy extensions. Actually, these extensions use malloc/free or PyMem_NEW/DEL. Obviously, extensions that free() objects allocated with PyObject_NEW need to be fixed (in source) for using them with another core malloc. But the ones using PyMem_DEL do not require fixes in the source, just recompilation, because this patch redirects them silently to the wrappers. Recompilation is by far easiler than fixing the sources, especially if one deals with 3rd party modules. With this consideration in mind, I concluded that it would be better to impose the wrappers for all extensions and to make the core a "black box" for them w.r.t the core allocator. This simplifies the understanding of the exposed interfaces for the extension writer and presents a simple choice: use the Python APIs for Python memory or non-Python APIs for non-Python memory. You raise another fair point: the user is smart enough for not mixing the APIs and is allowed to use Python APIs for memory which may not be handled by the Python mem manager. This is what would happen if we do not impose the "2nd class citizenship", i.e., do not impose the wrapper redirection for *_NEW/DEL and expose PyMem_MALLOC. (BTW, PyMem_MALLOC is non-existent for the moment) Now I'm torn too: one of the goals is to distinguish the Python heap from the rest, the distinction being made by the APIs. I'd like to know which piece of memory is under Python's control and which is not; I'd prefer to see the difference explicitely mentioned in the source. After all, I'd like to handle more efficiently the memory used by Python, for Python -- a mem mgt scheme which has its specificities. I know that we'll never gain control over foreign memory, used in 3rd party code for specific purposes (_tkinter, arrays dealing with specific hardware, etc.). The distinction will obviously be made by the extension writer, and we cannot discourage systematically the use of malloc/free... This is application dependent. The lack of a clear differentiation between what would be Python mem and what wouldn't (with a custom malloc), is probably what disturbs me in the APIs you propose... Otherwise, I agree with the the safe-func/less-safe-macro arguments completely. > > (Think unresolved symbols in an object extension file -- if the extension > > uses Python memory, it should be redirected to the Python memory manager, > > i.e. have unresolved PyMem_* symbols. The extension can still use malloc() > > and free() for private purposes, though, resolved with libc's malloc/free). > > But don't we want to discourage extensions from using malloc and free > directly? Otherwise we'll lose some of the advantages of the custom > allocator, right? Right, but we'll discourage their use "if at all possible". > > (Tell me again what you need the custom allocator for?) :-) Primarily for an overall better management of the memory used for Python objects and internals, i.e. the one used by custom objects + the one needed by the core objects and structures. It's about a special-purpose malloc, not a general-purpose one. > > > > What's the point of exporting PyMem_MALLOC? This would be misleading. > > Why misleading? It exists. not yet. But once we define it, it would be hard to go back or redefine it to something else. > I don't like to have macros that behave differently in the core > than outside, because the core is so often used as an example. Understood. I don't want them to behave differently. I want them the same, but tend to prefer exposing them as wrappers for outside. BTW, the fact that the core is often used as an example is scary too, because the core inlines frequently _PyObject_New... We need to emphasize the use of _PyObject_NEW/DEL in the docs. > > In this case, things would fail badly... The extension writer would > > be confident in using the Python API, and would expect that by using this > > API, the extension would work with any version of the core >= 1.6 > > I'm not clear how this would "fail badly." It would simply not use > the custom allocator in this case, but as I mentioned above, memory > allocation and freeing would still work, using malloc. My bad - an unweighted thought of mine. > > I hope you will give it some thought. The in-core/not-in-core > distinction is evil in my mind. I understand. On my side, the hic is the possibility of using the APIs without gaining control over the memory they're associated with, plus the issue about fixing vs. recompiling legacy C code (of the 1.x line). Hm. Tell me it's okay once again, and before implementing it, I'll think about it once again . -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From Moshe Zadka Tue Mar 7 04:38:05 2000 From: Moshe Zadka (Moshe Zadka) Date: Tue, 7 Mar 2000 06:38:05 +0200 (IST) Subject: [Patches] In overloading: not special casing strings In-Reply-To: <200003062105.QAA11349@eric.cnri.reston.va.us> Message-ID: On Mon, 6 Mar 2000, Guido van Rossum wrote: > Thanks. There's one thing broken in this version: you made the test > for length==1 return false -- it should raise an exception. See the > patches: Yes. Undocumented feature (which will be documented in a seperate doc patch once these get in). I have moral problems with raising an exception as a result of a "test containment" operation. An ADT of container either contains something, or it does not. Anything else is an optimization. > See: this is the exception I'm talking about. The reference manual sorts of says this behaviour is nasty. > I'll fix it. bug-fixing-finger-happy?-ly y'rs, Z. -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html From a.eyre@optichrome.com Tue Mar 7 13:58:49 2000 From: a.eyre@optichrome.com (Adrian Eyre) Date: Tue, 7 Mar 2000 13:58:49 -0000 Subject: [Patches] RE: inet_ntoa In-Reply-To: <001601bf883b$3cb50ed0$3acbd9c2@optichrome.com> Message-ID: <001701bf883d$437ecfb0$3acbd9c2@optichrome.com> > Am I missing something here: I am. This function is only in the CVS version of Python. ;-) > Just noticed the docstring is wrong. File: modules/socketmodule.c Version: 1.96 Line: 1889 "inet_aton(packed_ip) -> ip_address_string\n\ ...should be... "inet_ntoa(packed_ip) -> ip_address_string\n\ ----------------------------------------------------------------- Adrian Eyre - http://www.optichrome.com From fdrake@acm.org Tue Mar 7 14:05:51 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 7 Mar 2000 09:05:51 -0500 (EST) Subject: [Patches] RE: inet_ntoa In-Reply-To: <001701bf883d$437ecfb0$3acbd9c2@optichrome.com> References: <001601bf883b$3cb50ed0$3acbd9c2@optichrome.com> <001701bf883d$437ecfb0$3acbd9c2@optichrome.com> Message-ID: <14533.3263.597344.740430@weyr.cnri.reston.va.us> > Just noticed the docstring is wrong. This is now fixed in the CVS repository. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From sjoerd@oratrix.nl Tue Mar 7 15:08:41 2000 From: sjoerd@oratrix.nl (Sjoerd Mullender) Date: Tue, 07 Mar 2000 16:08:41 +0100 Subject: [Patches] Fix for aifc.py Message-ID: <20000307150842.7D4DD301CF9@bireme.oratrix.nl> The (relatively) new chunk module uses seek, not setpos. One instance of the call still needed to be fixed. Index: aifc.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/aifc.py,v retrieving revision 1.35 diff -u -r1.35 aifc.py --- aifc.py 2000/02/02 15:10:15 1.35 +++ aifc.py 2000/03/07 15:04:25 @@ -404,7 +404,7 @@ dummy = self._ssnd_chunk.read(8) pos = self._soundpos * self._framesize if pos: - self._ssnd_chunk.setpos(pos + 8) + self._ssnd_chunk.seek(pos + 8) self._ssnd_seek_needed = 0 if nframes == 0: return '' I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Sjoerd Mullender From guido@python.org Tue Mar 7 15:47:49 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 07 Mar 2000 10:47:49 -0500 Subject: [Patches] In overloading: not special casing strings In-Reply-To: Your message of "Tue, 07 Mar 2000 06:38:05 +0200." References: Message-ID: <200003071547.KAA14042@eric.cnri.reston.va.us> > > Thanks. There's one thing broken in this version: you made the test > > for length==1 return false -- it should raise an exception. See the > > patches: > > Yes. Undocumented feature (which will be documented in a seperate doc > patch once these get in). I have moral problems with raising an exception > as a result of a "test containment" operation. An ADT of container either > contains something, or it does not. Anything else is an optimization. Moral problems or not, it's a feature that I don't want to remove. If my code uses (i in s) where s is a string and i is an int, this would always be false -- probably it is a bug in the code, and that's why I raise an exception. Same with non-length-1 character strings. A newbie could easily believe that "bc" in "abcd" should return 1 -- rather than having them puzzle why it doesn't work (they probably tried ("xyz" in "abc") first and found that it correctly returned 0 :-), I give them an error explaining that it doesn't work that way. > > > See: this is the exception I'm talking about. > > The reference manual sorts of says this behaviour is nasty. Where? Which behavior? I'd say that it's nasty behavior to send in a patch whose documented purpose is to move some code from one file to another but which silently also turns an exception into a "false" return. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From gerrit@nl.linux.org Tue Mar 7 19:53:36 2000 From: gerrit@nl.linux.org (gerrit@nl.linux.org) Date: Tue, 7 Mar 2000 20:53:36 +0100 Subject: [Patches] Constants to use urlparse results Message-ID: <20000307205336.A2018@nl.linux.org> --SLDf9lqlvOQaIe6s Content-Type: text/plain; charset=us-ascii Hello, this patch adds some constants to the urlparse module, to improve readability. First you would write: >>> urlparse.urlparse(url)[4] Now, you can write: >>> urlparse.urlparse(url)[url.QUERY] In my opinion, this improves readability, as the stat modules improves readability in the results of os.stat (and so does statvfs). I've also changed the...(bah, how do I say this in English)... usage of the string module to usage of string methods. ------------------------------------------------------------------ | I confirm that, to the best of my knowledge and belief, this | | contribution is free of any claims of third parties under | | copyright, patent or other rights or interests ("claims"). To | | the extent that I have any such claims, I hereby grant to CNRI a | | nonexclusive, irrevocable, royalty-free, worldwide license to | | reproduce, distribute, perform and/or display publicly, prepare | | derivative versions, and otherwise use this contribution as part | | of the Python software and its related documentation, or any | | derivative versions thereof, at no cost to CNRI or its licensed | | users, and to authorize others to do so. | | | | I acknowledge that CNRI may, at its sole discretion, decide | | whether or not to incorporate this contribution in the Python | | software and its related documentation. I further grant CNRI | | permission to use my name and other identifying information | | provided to CNRI by me for use in connection with the Python | | software and its related documentation. | ------------------------------------------------------------------ regards, Gerrit. -- Plies korekt enie bet ingglisj joe encauntur in mai imil mesusj! -- Comparison Python GUI's: http://www.nl.linux.org/~gerrit/gui.html Please comment! --SLDf9lqlvOQaIe6s Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="urlparse.py.diff" --- /tmp/urlparse.py Tue Mar 7 20:25:57 2000 +++ urlparse.py Tue Mar 7 20:35:02 2000 @@ -6,7 +6,6 @@ # Standard/builtin Python modules import string -from string import joinfields, splitfields, rfind # A classification of schemes ('' means apply by default) uses_relative = ['ftp', 'http', 'gopher', 'nntp', 'wais', 'file', @@ -41,6 +40,12 @@ global _parse_cache _parse_cache = {} +SCHEME = 0 +NETLOC = 1 +PATH = 2 +PARAMS = 3 +QUERY = 4 +FRAGMENT = 5 def urlparse(url, scheme = '', allow_fragments = 1): """Parse a URL into 6 components: @@ -54,29 +59,28 @@ return cached if len(_parse_cache) >= MAX_CACHE_SIZE: # avoid runaway growth clear_cache() - find = string.find netloc = path = params = query = fragment = '' - i = find(url, ':') + i = url.find(':') if i > 0: if url[:i] == 'http': # optimize the common case - scheme = string.lower(url[:i]) + scheme = url[:i].lower() url = url[i+1:] if url[:2] == '//': - i = find(url, '/', 2) + i = url.find('/', 2) if i < 0: i = len(url) netloc = url[2:i] url = url[i:] if allow_fragments: - i = string.rfind(url, '#') + i = url.rfind('#') if i >= 0: fragment = url[i+1:] url = url[:i] - i = find(url, '?') + i = url.find('?') if i >= 0: query = url[i+1:] url = url[:i] - i = find(url, ';') + i = url.find(';') if i >= 0: params = url[i+1:] url = url[:i] @@ -87,23 +91,23 @@ if c not in scheme_chars: break else: - scheme, url = string.lower(url[:i]), url[i+1:] + scheme, url = url[:i].lower(), url[i+1:] if scheme in uses_netloc: if url[:2] == '//': - i = find(url, '/', 2) + i = url.find('/', 2) if i < 0: i = len(url) netloc, url = url[2:i], url[i:] if allow_fragments and scheme in uses_fragment: - i = string.rfind(url, '#') + i = url.rfind('#') if i >= 0: url, fragment = url[:i], url[i+1:] if scheme in uses_query: - i = find(url, '?') + i = url.find('?') if i >= 0: url, query = url[:i], url[i+1:] if scheme in uses_params: - i = find(url, ';') + i = url.find(';') if i >= 0: url, params = url[:i], url[i+1:] tuple = scheme, netloc, url, params, query, fragment @@ -151,10 +155,10 @@ if not path: return urlunparse((scheme, netloc, bpath, params, query or bquery, fragment)) - i = rfind(bpath, '/') + i = bpath.rfind('/') if i >= 0: path = bpath[:i] + '/' + path - segments = splitfields(path, '/') + segments = path.split('/') if segments[-1] == '.': segments[-1] = '' while '.' in segments: @@ -173,7 +177,7 @@ segments[-1] = '' elif len(segments) >= 2 and segments[-1] == '..': segments[-2:] = [''] - return urlunparse((scheme, netloc, joinfields(segments, '/'), + return urlunparse((scheme, netloc, '/'.join(segments), params, query, fragment)) def urldefrag(url): @@ -238,7 +242,7 @@ while 1: line = fp.readline() if not line: break - words = string.split(line) + words = line.split() if not words: continue url = words[0] --SLDf9lqlvOQaIe6s-- From guido@python.org Tue Mar 7 19:58:48 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 07 Mar 2000 14:58:48 -0500 Subject: [Patches] Constants to use urlparse results In-Reply-To: Your message of "Tue, 07 Mar 2000 20:53:36 +0100." <20000307205336.A2018@nl.linux.org> References: <20000307205336.A2018@nl.linux.org> Message-ID: <200003071958.OAA15674@eric.cnri.reston.va.us> Gerrit, Thanks for your patch. You know what would be really cool? If you wrote a utility that changes string.function(strobj, ...) calls into strobj.method(...) calls! Bonus points if it also deals with things like find = string.find ... i = find(p, q) --Guido van Rossum (home page: http://www.python.org/~guido/) From Moshe Zadka Wed Mar 8 05:28:51 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 8 Mar 2000 07:28:51 +0200 (IST) Subject: [Patches] In overloading: not special casing strings In-Reply-To: <200003071547.KAA14042@eric.cnri.reston.va.us> Message-ID: On Tue, 7 Mar 2000, Guido van Rossum wrote: > Moral problems or not, it's a feature that I don't want to remove. OK. I'm sorry I didn't write that in the checkin message -- I wanted to do that, but it slipped my mind. I won't argue the merits, it isn't that important. > Where? Which behavior? The part about the "in" operator. > I'd say that it's nasty behavior to send in a patch whose documented > purpose is to move some code from one file to another but which > silently also turns an exception into a "false" return. :-) You're right. Sorry. Just slippped my mind. -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html From guido@python.org Wed Mar 8 14:31:22 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Mar 2000 09:31:22 -0500 Subject: [Patches] Safe destruction of recursive objects In-Reply-To: Your message of "Wed, 08 Mar 2000 15:00:27 +0100." <38C65CFB.D666F688@tismer.com> References: <38C41776.B6FAAB07@tismer.com> <200003062114.QAA11563@eric.cnri.reston.va.us> <38C428A7.57CF095D@tismer.com> <38C65CFB.D666F688@tismer.com> Message-ID: <200003081431.JAA20135@eric.cnri.reston.va.us> > > > > + Unfortunately, deallocations also take place when > > > > + the thread state is undefined. > > Ok, you said you were about to check this in when > you read my comment concerning thread state. > But I don't see these things to be related. The patch > works fine. Is there anything besides lack of time, keeping you from > checking this in? Do you expect more input from me? I am worried about the possibility of this code being run without the interpreter lock held -- it allocates a list and calls PyList_Append(), it's not safe to do this when another thread could be doing the same thing. PyList_New() is thread-safe, but PyList_Append() to a list reachable via a global is not! Should I worry? Other comments on the patch (which I didn't have the time to review earlier): - Can we avoid creating new little files for every little feature? This stuff should go into object.{c,h}. - Your _PyTrash_deposit_object(op) doesn't check for errors from PyList_New() and PyList_Append(). Typically, destructors shouldn't do anything that could cause an exception to be raised, because destructors may be called while an existing exception is being handled. You'd have to wrap your code in PyErr_Fetch() and _Restore(), like classobject.c does when it calls __del__ from a deallocator. - Please submit the append compatibility patch separately. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@mojam.com (Skip Montanaro) Wed Mar 8 15:30:32 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Wed, 8 Mar 2000 09:30:32 -0600 (CST) Subject: [Patches] In overloading: not special casing strings In-Reply-To: <200003071547.KAA14042@eric.cnri.reston.va.us> References: <200003071547.KAA14042@eric.cnri.reston.va.us> Message-ID: <14534.29208.209644.769956@beluga.mojam.com> Guido> Same with non-length-1 character strings. A newbie could easily Guido> believe that "bc" in "abcd" should return 1 -- rather than having Guido> them puzzle why it doesn't work (they probably tried ("xyz" in Guido> "abc") first and found that it correctly returned 0 :-), I give Guido> them an error explaining that it doesn't work that way. Yes, please keep this behavior. I encounter it on occasion, most frequently in a context where both sides of the "in" operator are variables and I've somehow screwed up when generating the left operand so that its length is > 1 or screwed up generating the right operand as a string when it should have been a list. Silently returning zero would allow a relatively common (for me) bug to go unannounced. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ From tismer@tismer.com Wed Mar 8 16:47:06 2000 From: tismer@tismer.com (Christian Tismer) Date: Wed, 08 Mar 2000 17:47:06 +0100 Subject: [Patches] Safe destruction of recursive objects References: <38C41776.B6FAAB07@tismer.com> <200003062114.QAA11563@eric.cnri.reston.va.us> <38C428A7.57CF095D@tismer.com> <38C65CFB.D666F688@tismer.com> <200003081431.JAA20135@eric.cnri.reston.va.us> Message-ID: <38C6840A.CCB7D6D5@tismer.com> Guido van Rossum wrote: > > > > > > + Unfortunately, deallocations also take place when > > > > > + the thread state is undefined. ... > I am worried about the possibility of this code being run without the > interpreter lock held -- it allocates a list and calls > PyList_Append(), it's not safe to do this when another thread could be > doing the same thing. PyList_New() is thread-safe, but > PyList_Append() to a list reachable via a global is not! > > Should I worry? Well, we can avoid this completely by using tuples. No list.append at all, but I simply create a new 2-tuple for every to-be-deleted that holds (to-be-deleted, old-tuple) > Other comments on the patch (which I didn't have the time to review > earlier): But they didn't reach me - was it private email? > - Can we avoid creating new little files for every little feature? > This stuff should go into object.{c,h}. Agreed, no problem. > - Your _PyTrash_deposit_object(op) doesn't check for errors from > PyList_New() and PyList_Append(). Typically, destructors shouldn't do > anything that could cause an exception to be raised, because > destructors may be called while an existing exception is being > handled. You'd have to wrap your code in PyErr_Fetch() and > _Restore(), like classobject.c does when it calls __del__ from a > deallocator. I think the tuple way sketched above avoids this and is more elegant. Although I'm loosing the elegance of appending a 0-refcount object to a list. :-) > - Please submit the append compatibility patch separately. Will do so. I have one further comment: While I appreciate the general use of PyArg_ParseTuple for consitency and clarity, there is a remarkable run-time overhead for using it in every simple case. There are quite often situations where you have very easy things like "(O)", and in very simple functions like len(), the tuple parser consumes over 90% of the computation time. What I do in these simple cases is: If there is nothing more to check than the length of the argument tuple, I use a direct implementation, like I did in the list.append compatibility. PyArg_ParseTuple is then simply used to do the *error report*. This gives you the advantage of not computing obvious cases, but you still have verbose standard error messages. I think we (I) could supply a handful of macros for this purpose (give me quick access but be as verbose on errors as before) and win a good amount of speedup. cheers - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From guido@python.org Wed Mar 8 17:04:58 2000 From: guido@python.org (Guido van Rossum) Date: Wed, 08 Mar 2000 12:04:58 -0500 Subject: [Patches] Safe destruction of recursive objects In-Reply-To: Your message of "Wed, 08 Mar 2000 17:47:06 +0100." <38C6840A.CCB7D6D5@tismer.com> References: <38C41776.B6FAAB07@tismer.com> <200003062114.QAA11563@eric.cnri.reston.va.us> <38C428A7.57CF095D@tismer.com> <38C65CFB.D666F688@tismer.com> <200003081431.JAA20135@eric.cnri.reston.va.us> <38C6840A.CCB7D6D5@tismer.com> Message-ID: <200003081704.MAA20521@eric.cnri.reston.va.us> > Well, we can avoid this completely by using tuples. > No list.append at all, but I simply create a new 2-tuple > for every to-be-deleted that holds > (to-be-deleted, old-tuple) Except that you still have to store it in a global so there's still a race condition. Also, it's ironic that this way you can create deeply nested tuples -- and deallocating those was the very reason you had for this whole patch! :-) > > Other comments on the patch (which I didn't have the time to review > > earlier): > > But they didn't reach me - was it private email? Sorry for the confusion; I sent them just now because I studied the patch just now. > > - Can we avoid creating new little files for every little feature? > > This stuff should go into object.{c,h}. > > Agreed, no problem. > > > - Your _PyTrash_deposit_object(op) doesn't check for errors from > > PyList_New() and PyList_Append(). Typically, destructors shouldn't do > > anything that could cause an exception to be raised, because > > destructors may be called while an existing exception is being > > handled. You'd have to wrap your code in PyErr_Fetch() and > > _Restore(), like classobject.c does when it calls __del__ from a > > deallocator. > > I think the tuple way sketched above avoids this and is more > elegant. Although I'm loosing the elegance of appending > a 0-refcount object to a list. :-) But having a tuple with a 0-refcount object is just as bad... > > - Please submit the append compatibility patch separately. > > Will do so. > > I have one further comment: > While I appreciate the general use of PyArg_ParseTuple for > consitency and clarity, there is a remarkable run-time overhead > for using it in every simple case. There are quite often situations > where you have very easy things like "(O)", and in very simple > functions like len(), the tuple parser consumes over 90% of > the computation time. > > What I do in these simple cases is: If there is nothing more > to check than the length of the argument tuple, I use a direct > implementation, like I did in the list.append compatibility. > PyArg_ParseTuple is then simply used to do the *error report*. > This gives you the advantage of not computing obvious cases, > but you still have verbose standard error messages. > > I think we (I) could supply a handful of macros for this > purpose (give me quick access but be as verbose on errors > as before) and win a good amount of speedup. Sure, but when is the execution time of len() critical? I much prefer not so "fix" something that isn't broken. --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer@tismer.com Wed Mar 8 18:09:20 2000 From: tismer@tismer.com (Christian Tismer) Date: Wed, 08 Mar 2000 19:09:20 +0100 Subject: [Patches] Safe destruction of recursive objects References: <38C41776.B6FAAB07@tismer.com> <200003062114.QAA11563@eric.cnri.reston.va.us> <38C428A7.57CF095D@tismer.com> <38C65CFB.D666F688@tismer.com> <200003081431.JAA20135@eric.cnri.reston.va.us> <38C6840A.CCB7D6D5@tismer.com> <200003081704.MAA20521@eric.cnri.reston.va.us> Message-ID: <38C69750.7420B06E@tismer.com> Guido van Rossum wrote: > > > Well, we can avoid this completely by using tuples. > > No list.append at all, but I simply create a new 2-tuple > > for every to-be-deleted that holds > > (to-be-deleted, old-tuple) > > Except that you still have to store it in a global so there's still a > race condition. Also, it's ironic that this way you can create deeply > nested tuples -- and deallocating those was the very reason you had > for this whole patch! :-) Yes, ironic. But it is consistent. By (obj, old-tup) I create a chain that is only one longer than the number of delayed objects. The total length is one plus the length of the list that I used before. When there is a deeply nested list, it will be unpacked until level 50, andthen it will go into a tuple that waits for the next elevator cycle. The worst case condition is in the opposite situation: At nesting level 49, you have a list containing a million of tuples. This list becomes destroyed, but all the tuples now pile up, using another million of tuples. One could enhance this behavior by using a different tuple size, if it's worth it. ... > But having a tuple with a 0-refcount object is just as bad... Oh that was a joke - of course I would incref the object. The elegant part was that list.append() does it for me :-) ... > Sure, but when is the execution time of len() critical? I much prefer > not so "fix" something that isn't broken. Len() is just an example where execution time of the real operation becomes so ridiculous when compared to the overhead. In my continuation stuff, I had to use this trick to become faster than a standard Python function call for a continuation call. It was astonishingly significant. Well, speed is my game I guess :-) Back on the race condition: If it in fact exists, then this is really hard to solve. I presume that there is no statement that forbids to deallocate something without having the global lock? Well, thinking a bit further: You need to state that it is not allowed to decref an object without the global lock, since it is possible that there is a class instance deallocated which has a __del__ method. That would cause a recursive interpreter incarnation while the global lock is not set, and that is illegal. For me this means: The patch is valid, with a list or with a tuple, but all other code that does not obey the above rule is definately invalid, and we have to check this immediately! Oh-oh :-) ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From tismer@tismer.com Wed Mar 8 19:29:58 2000 From: tismer@tismer.com (Christian Tismer) Date: Wed, 08 Mar 2000 20:29:58 +0100 Subject: [Patches] Safe destruction of recursive objects References: <38C41776.B6FAAB07@tismer.com> <200003062114.QAA11563@eric.cnri.reston.va.us> <38C428A7.57CF095D@tismer.com> <38C65CFB.D666F688@tismer.com> <200003081431.JAA20135@eric.cnri.reston.va.us> Message-ID: <38C6AA36.C8CC2139@tismer.com> Again on this one, after some thought: Guido van Rossum wrote: > > > > > > + Unfortunately, deallocations also take place when > > > > > + the thread state is undefined. > > > > Ok, you said you were about to check this in when > > you read my comment concerning thread state. > > But I don't see these things to be related. The patch > > works fine. Is there anything besides lack of time, keeping you from > > checking this in? Do you expect more input from me? > > I am worried about the possibility of this code being run without the > interpreter lock held -- it allocates a list and calls > PyList_Append(), it's not safe to do this when another thread could be > doing the same thing. PyList_New() is thread-safe, but > PyList_Append() to a list reachable via a global is not! > > Should I worry? I think you should, but not concerning this patch. If a dealloc occours without the interpreter lock, then it is an error, since this would allow to start an interpreter from the __del__ hook. Then, if we know that the interpreter lock is there all the time, then it is also safe to use a list. What I'd like to do is to insert a check for the interpreter lock into my patch, and to use that to find hidden implementation errors, most probably in extension modules. Setting the unwind_level to something small like 5 or so would still be not too expensive, but catch such errors with quite high probability. cheers - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From gstein@lyra.org Thu Mar 9 20:30:32 2000 From: gstein@lyra.org (Greg Stein) Date: Thu, 9 Mar 2000 12:30:32 -0800 (PST) Subject: [Patches] Safe destruction of recursive objects In-Reply-To: <38C6AA36.C8CC2139@tismer.com> Message-ID: On Wed, 8 Mar 2000, Christian Tismer wrote: >... > I think you should, but not concerning this patch. > If a dealloc occours without the interpreter lock, then > it is an error, since this would allow to start an interpreter > from the __del__ hook. > Then, if we know that the interpreter lock is there all the time, > then it is also safe to use a list. Yes, yes, and yes. > What I'd like to do is to insert a check for the interpreter > lock into my patch, and to use that to find hidden implementation > errors, most probably in extension modules. That seems a bit silly. We don't check for the lock anywhere else. The interpreter makes a basic assumption across the entire code body that the lock is held when calls to the API are made. There is no reason to start questioning that assumption. Cheers, -g -- Greg Stein, http://www.lyra.org/ From tismer@tismer.com Thu Mar 9 21:38:15 2000 From: tismer@tismer.com (Christian Tismer) Date: Thu, 09 Mar 2000 22:38:15 +0100 Subject: [Patches] Safe destruction of recursive objects References: Message-ID: <38C819C7.D6C9842B@tismer.com> Greg Stein wrote: > > On Wed, 8 Mar 2000, Christian Tismer wrote: > >... > > I think you should, but not concerning this patch. > > If a dealloc occours without the interpreter lock, then > > it is an error, since this would allow to start an interpreter > > from the __del__ hook. > > Then, if we know that the interpreter lock is there all the time, > > then it is also safe to use a list. > > Yes, yes, and yes. > > > What I'd like to do is to insert a check for the interpreter > > lock into my patch, and to use that to find hidden implementation > > errors, most probably in extension modules. > > That seems a bit silly. We don't check for the lock anywhere else. The > interpreter makes a basic assumption across the entire code body that the > lock is held when calls to the API are made. There is no reason to start > questioning that assumption. Ok, great! I'm just ready with an exhaustive search for violations in the core. There was exactly the one bug that Guido found before in posixmodule: posix_listdir uses PyList_Append in line 1022 All the rest is clean! ciao - chris (do you want to see that huge protocol? :-) -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From tismer@tismer.com Thu Mar 9 23:29:01 2000 From: tismer@tismer.com (Christian Tismer) Date: Fri, 10 Mar 2000 00:29:01 +0100 Subject: [Patches] Safe destruction of recursive objects V2 Message-ID: <38C833BD.71B04666@tismer.com> This is the reworked patch version 2. When recursive objects are destroyed, the recursion depth of nested dealloc calls can cause a core dump. The trashcan macro/functions are written to avoid this. To keep the necessary changes as small as possible, the patch add just two macro lines which enclose the critical operation. The affected objects are dict, tuple, list, frame and traceback. The method: While destructing, a global counter tracks the recursion depth. It is incremented in the beginning macro and decremented in the ending macro. If the count hits its limit, the enclosed deallocation code is not executed. Instead, destruction is deferred by appending the object to a list. The ending macro checks wether we are on toplevel and if there is a list to destroy. If so, the list is made local and cleared out. This process will continue until there is nothing left over. I'd call this "elevator destructor." As an addition, I added some small changes to listmodule.c, in order to make strict argument handling in append et al optional. If this is undesired, simply remove that part of the patch. Checkin-message: added wrapping macros to dictobject.c, listobject.c, tupleobject.c, frameobject.c, traceback.c that safely prevends core dumps on stack overflow. Macros and functions in object.c, object.h. Diff: diff -c -b -P -r //d/python/python-1.5.2/Include/object.h trash/Include/object.h *** //d/python/python-1.5.2/Include/object.h Fri Mar 03 20:44:40 2000 --- trash/Include/object.h Fri Mar 10 01:23:26 2000 *************** *** 514,519 **** --- 514,566 ---- times. */ + /* + trashcan + CT 2k0130 + non-recursively destroy nested objects + + CT 2k0223 + redefinition for better locality and less overhead. + + Objects that want to be recursion safe need to use + the macroes + Py_TRASHCAN_SAFE_BEGIN(name) + and + Py_TRASHCAN_SAFE_END(name) + surrounding their actual deallocation code. + + It would be nice to do this using the thread state. + Also, we could do an exact stack measure then. + Unfortunately, deallocations also take place when + the thread state is undefined. + */ + + #define PyTrash_UNWIND_LEVEL 50 + + #define Py_TRASHCAN_SAFE_BEGIN(op) \ + { \ + ++_PyTrash_delete_nesting; \ + if (_PyTrash_delete_nesting < PyTrash_UNWIND_LEVEL) { \ + + #define Py_TRASHCAN_SAFE_END(op) \ + ;} \ + else \ + _PyTrash_deposit_object((PyObject*)op);\ + --_PyTrash_delete_nesting; \ + if (_PyTrash_delete_later && _PyTrash_delete_nesting <= 0) \ + _PyTrash_destroy_list(); \ + } \ + + extern DL_IMPORT(void) _PyTrash_deposit_object Py_PROTO((PyObject*)); + extern DL_IMPORT(void) _PyTrash_destroy_list Py_PROTO(()); + + extern DL_IMPORT(int) _PyTrash_delete_nesting; + extern DL_IMPORT(PyObject *) _PyTrash_delete_later; + + /* swap the "xx" to check the speed loss */ + + #define xxPy_TRASHCAN_SAFE_BEGIN(op) + #define xxPy_TRASHCAN_SAFE_END(op) ; #ifdef __cplusplus } #endif diff -c -b -P -r //d/python/python-1.5.2/Objects/dictobject.c trash/Objects/dictobject.c *** //d/python/python-1.5.2/Objects/dictobject.c Sat Feb 26 20:35:38 2000 --- trash/Objects/dictobject.c Wed Mar 08 19:07:30 2000 *************** *** 479,484 **** --- 479,485 ---- { register int i; register dictentry *ep; + Py_TRASHCAN_SAFE_BEGIN(op) for (i = 0, ep = mp->ma_table; i < mp->ma_size; i++, ep++) { if (ep->me_key != NULL) { Py_DECREF(ep->me_key); *************** *** 489,494 **** --- 490,496 ---- } PyMem_XDEL(mp->ma_table); PyMem_DEL(mp); + Py_TRASHCAN_SAFE_END(mp) } static int diff -c -b -P -r //d/python/python-1.5.2/Objects/frameobject.c trash/Objects/frameobject.c *** //d/python/python-1.5.2/Objects/frameobject.c Sat Feb 26 20:35:40 2000 --- trash/Objects/frameobject.c Wed Mar 08 19:07:30 2000 *************** *** 103,108 **** --- 103,109 ---- int i; PyObject **fastlocals; + Py_TRASHCAN_SAFE_BEGIN(f) /* Kill all local variables */ fastlocals = f->f_localsplus; for (i = f->f_nlocals; --i >= 0; ++fastlocals) { *************** *** 120,125 **** --- 121,127 ---- Py_XDECREF(f->f_exc_traceback); f->f_back = free_list; free_list = f; + Py_TRASHCAN_SAFE_END(f) } PyTypeObject PyFrame_Type = { diff -c -b -P -r //d/python/python-1.5.2/Objects/listobject.c trash/Objects/listobject.c *** //d/python/python-1.5.2/Objects/listobject.c Sat Feb 26 20:35:40 2000 --- trash/Objects/listobject.c Wed Mar 08 19:07:30 2000 *************** *** 215,220 **** --- 215,221 ---- PyListObject *op; { int i; + Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_item != NULL) { /* Do it backwards, for Christian Tismer. There's a simple test case where somehow this reduces *************** *** 227,232 **** --- 228,234 ---- free((ANY *)op->ob_item); } free((ANY *)op); + Py_TRASHCAN_SAFE_END(op) } static int diff -c -b -P -r //d/python/python-1.5.2/Objects/object.c trash/Objects/object.c *** //d/python/python-1.5.2/Objects/object.c Sat Feb 26 20:35:42 2000 --- trash/Objects/object.c Fri Mar 10 00:40:24 2000 *************** *** 887,889 **** --- 887,934 ---- } } } + + /* + trashcan + CT 2k0130 + non-recursively destroy nested objects + + CT 2k0223 + everything is now done in a macro. + + CT 2k0305 + modified to use functions, after Tim Peter's suggestion. + + CT 2k0309 + modified to restore a possible error. + */ + + int _PyTrash_delete_nesting = 0; + PyObject * _PyTrash_delete_later = NULL; + + void + _PyTrash_deposit_object(op) + PyObject *op; + { + PyObject *error_type, *error_value, *error_traceback; + PyErr_Fetch(&error_type, &error_value, &error_traceback); + + if (!_PyTrash_delete_later) + _PyTrash_delete_later = PyList_New(0); + if (_PyTrash_delete_later) + PyList_Append(_PyTrash_delete_later, (PyObject *)op); + + PyErr_Restore(error_type, error_value, error_traceback); + } + + void + _PyTrash_destroy_list() + { + while (_PyTrash_delete_later) { + PyObject *shredder = _PyTrash_delete_later; + _PyTrash_delete_later = NULL; + ++_PyTrash_delete_nesting; + Py_DECREF(shredder); + --_PyTrash_delete_nesting; + } + } diff -c -b -P -r //d/python/python-1.5.2/Objects/tupleobject.c trash/Objects/tupleobject.c *** //d/python/python-1.5.2/Objects/tupleobject.c Sat Feb 26 20:35:44 2000 --- trash/Objects/tupleobject.c Wed Mar 08 19:07:30 2000 *************** *** 172,177 **** --- 172,178 ---- { register int i; + Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_size > 0) { i = op->ob_size; while (--i >= 0) *************** *** 180,190 **** if (op->ob_size < MAXSAVESIZE) { op->ob_item[0] = (PyObject *) free_tuples[op->ob_size]; free_tuples[op->ob_size] = op; ! return; } #endif } free((ANY *)op); } static int --- 181,193 ---- if (op->ob_size < MAXSAVESIZE) { op->ob_item[0] = (PyObject *) free_tuples[op->ob_size]; free_tuples[op->ob_size] = op; ! goto done; /* return */ } #endif } free((ANY *)op); + done: + Py_TRASHCAN_SAFE_END(op) } static int diff -c -b -P -r //d/python/python-1.5.2/Python/traceback.c trash/Python/traceback.c *** //d/python/python-1.5.2/Python/traceback.c Sat Sep 18 22:49:40 1999 --- trash/Python/traceback.c Wed Mar 08 19:07:30 2000 *************** *** 68,76 **** --- 68,78 ---- tb_dealloc(tb) tracebackobject *tb; { + Py_TRASHCAN_SAFE_BEGIN(tb) Py_XDECREF(tb->tb_next); Py_XDECREF(tb->tb_frame); PyMem_DEL(tb); + Py_TRASHCAN_SAFE_END(tb) } #define Tracebacktype PyTraceBack_Type %%%%%%%%%%%%%%%%%%%%%%% end patch %%%%%%%%%%%%%%%%%%%%%%%%%% Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From tismer@tismer.com Fri Mar 10 13:05:17 2000 From: tismer@tismer.com (Christian Tismer) Date: Fri, 10 Mar 2000 14:05:17 +0100 Subject: [Patches] Safe destruction of recursive objects V2a Message-ID: <38C8F30D.E4655520@tismer.com> Version 2a with one less typo. Sigh... :-) This is the reworked patch version 2. When recursive objects are destroyed, the recursion depth of nested dealloc calls can cause a core dump. The trashcan macro/functions are written to avoid this. To keep the necessary changes as small as possible, the patch add just two macro lines which enclose the critical operation. The affected objects are dict, tuple, list, frame and traceback. The method: While destructing, a global counter tracks the recursion depth. It is incremented in the beginning macro and decremented in the ending macro. If the count hits its limit, the enclosed deallocation code is not executed. Instead, destruction is deferred by appending the object to a list. The ending macro checks wether we are on toplevel and if there is a list to destroy. If so, the list is made local and cleared out. This process will continue until there is nothing left over. I'd call this "elevator destructor." As an addition, I added some small changes to listmodule.c, in order to make strict argument handling in append et al optional. If this is undesired, simply remove that part of the patch. Checkin-message: added wrapping macros to dictobject.c, listobject.c, tupleobject.c, frameobject.c, traceback.c that safely prevends core dumps on stack overflow. Macros and functions in object.c, object.h. Diff: diff -c -b -P -r //d/python/python-1.5.2/Include/object.h trash/Include/object.h *** //d/python/python-1.5.2/Include/object.h Fri Mar 03 20:44:40 2000 --- trash/Include/object.h Fri Mar 10 01:23:26 2000 *************** *** 514,519 **** --- 514,566 ---- times. */ + /* + trashcan + CT 2k0130 + non-recursively destroy nested objects + + CT 2k0223 + redefinition for better locality and less overhead. + + Objects that want to be recursion safe need to use + the macroes + Py_TRASHCAN_SAFE_BEGIN(name) + and + Py_TRASHCAN_SAFE_END(name) + surrounding their actual deallocation code. + + It would be nice to do this using the thread state. + Also, we could do an exact stack measure then. + Unfortunately, deallocations also take place when + the thread state is undefined. + */ + + #define PyTrash_UNWIND_LEVEL 50 + + #define Py_TRASHCAN_SAFE_BEGIN(op) \ + { \ + ++_PyTrash_delete_nesting; \ + if (_PyTrash_delete_nesting < PyTrash_UNWIND_LEVEL) { \ + + #define Py_TRASHCAN_SAFE_END(op) \ + ;} \ + else \ + _PyTrash_deposit_object((PyObject*)op);\ + --_PyTrash_delete_nesting; \ + if (_PyTrash_delete_later && _PyTrash_delete_nesting <= 0) \ + _PyTrash_destroy_list(); \ + } \ + + extern DL_IMPORT(void) _PyTrash_deposit_object Py_PROTO((PyObject*)); + extern DL_IMPORT(void) _PyTrash_destroy_list Py_PROTO(()); + + extern DL_IMPORT(int) _PyTrash_delete_nesting; + extern DL_IMPORT(PyObject *) _PyTrash_delete_later; + + /* swap the "xx" to check the speed loss */ + + #define xxPy_TRASHCAN_SAFE_BEGIN(op) + #define xxPy_TRASHCAN_SAFE_END(op) ; #ifdef __cplusplus } #endif diff -c -b -P -r //d/python/python-1.5.2/Objects/dictobject.c trash/Objects/dictobject.c *** //d/python/python-1.5.2/Objects/dictobject.c Sat Feb 26 20:35:38 2000 --- trash/Objects/dictobject.c Wed Mar 08 19:07:30 2000 *************** *** 479,484 **** --- 479,485 ---- { register int i; register dictentry *ep; + Py_TRASHCAN_SAFE_BEGIN(mp) for (i = 0, ep = mp->ma_table; i < mp->ma_size; i++, ep++) { if (ep->me_key != NULL) { Py_DECREF(ep->me_key); *************** *** 489,494 **** --- 490,496 ---- } PyMem_XDEL(mp->ma_table); PyMem_DEL(mp); + Py_TRASHCAN_SAFE_END(mp) } static int diff -c -b -P -r //d/python/python-1.5.2/Objects/frameobject.c trash/Objects/frameobject.c *** //d/python/python-1.5.2/Objects/frameobject.c Sat Feb 26 20:35:40 2000 --- trash/Objects/frameobject.c Wed Mar 08 19:07:30 2000 *************** *** 103,108 **** --- 103,109 ---- int i; PyObject **fastlocals; + Py_TRASHCAN_SAFE_BEGIN(f) /* Kill all local variables */ fastlocals = f->f_localsplus; for (i = f->f_nlocals; --i >= 0; ++fastlocals) { *************** *** 120,125 **** --- 121,127 ---- Py_XDECREF(f->f_exc_traceback); f->f_back = free_list; free_list = f; + Py_TRASHCAN_SAFE_END(f) } PyTypeObject PyFrame_Type = { diff -c -b -P -r //d/python/python-1.5.2/Objects/listobject.c trash/Objects/listobject.c *** //d/python/python-1.5.2/Objects/listobject.c Sat Feb 26 20:35:40 2000 --- trash/Objects/listobject.c Wed Mar 08 19:07:30 2000 *************** *** 215,220 **** --- 215,221 ---- PyListObject *op; { int i; + Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_item != NULL) { /* Do it backwards, for Christian Tismer. There's a simple test case where somehow this reduces *************** *** 227,232 **** --- 228,234 ---- free((ANY *)op->ob_item); } free((ANY *)op); + Py_TRASHCAN_SAFE_END(op) } static int diff -c -b -P -r //d/python/python-1.5.2/Objects/object.c trash/Objects/object.c *** //d/python/python-1.5.2/Objects/object.c Sat Feb 26 20:35:42 2000 --- trash/Objects/object.c Fri Mar 10 00:40:24 2000 *************** *** 887,889 **** --- 887,934 ---- } } } + + /* + trashcan + CT 2k0130 + non-recursively destroy nested objects + + CT 2k0223 + everything is now done in a macro. + + CT 2k0305 + modified to use functions, after Tim Peter's suggestion. + + CT 2k0309 + modified to restore a possible error. + */ + + int _PyTrash_delete_nesting = 0; + PyObject * _PyTrash_delete_later = NULL; + + void + _PyTrash_deposit_object(op) + PyObject *op; + { + PyObject *error_type, *error_value, *error_traceback; + PyErr_Fetch(&error_type, &error_value, &error_traceback); + + if (!_PyTrash_delete_later) + _PyTrash_delete_later = PyList_New(0); + if (_PyTrash_delete_later) + PyList_Append(_PyTrash_delete_later, (PyObject *)op); + + PyErr_Restore(error_type, error_value, error_traceback); + } + + void + _PyTrash_destroy_list() + { + while (_PyTrash_delete_later) { + PyObject *shredder = _PyTrash_delete_later; + _PyTrash_delete_later = NULL; + ++_PyTrash_delete_nesting; + Py_DECREF(shredder); + --_PyTrash_delete_nesting; + } + } diff -c -b -P -r //d/python/python-1.5.2/Objects/tupleobject.c trash/Objects/tupleobject.c *** //d/python/python-1.5.2/Objects/tupleobject.c Sat Feb 26 20:35:44 2000 --- trash/Objects/tupleobject.c Wed Mar 08 19:07:30 2000 *************** *** 172,177 **** --- 172,178 ---- { register int i; + Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_size > 0) { i = op->ob_size; while (--i >= 0) *************** *** 180,190 **** if (op->ob_size < MAXSAVESIZE) { op->ob_item[0] = (PyObject *) free_tuples[op->ob_size]; free_tuples[op->ob_size] = op; ! return; } #endif } free((ANY *)op); } static int --- 181,193 ---- if (op->ob_size < MAXSAVESIZE) { op->ob_item[0] = (PyObject *) free_tuples[op->ob_size]; free_tuples[op->ob_size] = op; ! goto done; /* return */ } #endif } free((ANY *)op); + done: + Py_TRASHCAN_SAFE_END(op) } static int diff -c -b -P -r //d/python/python-1.5.2/Python/traceback.c trash/Python/traceback.c *** //d/python/python-1.5.2/Python/traceback.c Sat Sep 18 22:49:40 1999 --- trash/Python/traceback.c Wed Mar 08 19:07:30 2000 *************** *** 68,76 **** --- 68,78 ---- tb_dealloc(tb) tracebackobject *tb; { + Py_TRASHCAN_SAFE_BEGIN(tb) Py_XDECREF(tb->tb_next); Py_XDECREF(tb->tb_frame); PyMem_DEL(tb); + Py_TRASHCAN_SAFE_END(tb) } #define Tracebacktype PyTraceBack_Type %%%%%%%%%%%%%%%%%%%%%%% end patch %%%%%%%%%%%%%%%%%%%%%%%%%% Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From tismer@tismer.com Fri Mar 10 13:50:05 2000 From: tismer@tismer.com (Christian Tismer) Date: Fri, 10 Mar 2000 14:50:05 +0100 Subject: [Patches] Safe destruction of recursive objects V2b Message-ID: <38C8FD8D.314EE5F0@tismer.com> Version 2b without mentioning the compatibility check which was already gone. I'm getting old... Version 2a with one less typo. Sigh... :-) This is the reworked patch version 2. When recursive objects are destroyed, the recursion depth of nested dealloc calls can cause a core dump. The trashcan macro/functions are written to avoid this. To keep the necessary changes as small as possible, the patch add just two macro lines which enclose the critical operation. The affected objects are dict, tuple, list, frame and traceback. The method: While destructing, a global counter tracks the recursion depth. It is incremented in the beginning macro and decremented in the ending macro. If the count hits its limit, the enclosed deallocation code is not executed. Instead, destruction is deferred by appending the object to a list. The ending macro checks wether we are on toplevel and if there is a list to destroy. If so, the list is made local and cleared out. This process will continue until there is nothing left over. I'd call this "elevator destructor." Checkin-message: added wrapping macros to dictobject.c, listobject.c, tupleobject.c, frameobject.c, traceback.c that safely prevends core dumps on stack overflow. Macros and functions in object.c, object.h. The method is an "elevator destructor" that turns cascading deletes into tail recursive behavior when some limit is hit. Diff: diff -c -b -P -r //d/python/python-1.5.2/Include/object.h trash/Include/object.h *** //d/python/python-1.5.2/Include/object.h Fri Mar 03 20:44:40 2000 --- trash/Include/object.h Fri Mar 10 01:23:26 2000 *************** *** 514,519 **** --- 514,566 ---- times. */ + /* + trashcan + CT 2k0130 + non-recursively destroy nested objects + + CT 2k0223 + redefinition for better locality and less overhead. + + Objects that want to be recursion safe need to use + the macroes + Py_TRASHCAN_SAFE_BEGIN(name) + and + Py_TRASHCAN_SAFE_END(name) + surrounding their actual deallocation code. + + It would be nice to do this using the thread state. + Also, we could do an exact stack measure then. + Unfortunately, deallocations also take place when + the thread state is undefined. + */ + + #define PyTrash_UNWIND_LEVEL 50 + + #define Py_TRASHCAN_SAFE_BEGIN(op) \ + { \ + ++_PyTrash_delete_nesting; \ + if (_PyTrash_delete_nesting < PyTrash_UNWIND_LEVEL) { \ + + #define Py_TRASHCAN_SAFE_END(op) \ + ;} \ + else \ + _PyTrash_deposit_object((PyObject*)op);\ + --_PyTrash_delete_nesting; \ + if (_PyTrash_delete_later && _PyTrash_delete_nesting <= 0) \ + _PyTrash_destroy_list(); \ + } \ + + extern DL_IMPORT(void) _PyTrash_deposit_object Py_PROTO((PyObject*)); + extern DL_IMPORT(void) _PyTrash_destroy_list Py_PROTO(()); + + extern DL_IMPORT(int) _PyTrash_delete_nesting; + extern DL_IMPORT(PyObject *) _PyTrash_delete_later; + + /* swap the "xx" to check the speed loss */ + + #define xxPy_TRASHCAN_SAFE_BEGIN(op) + #define xxPy_TRASHCAN_SAFE_END(op) ; #ifdef __cplusplus } #endif diff -c -b -P -r //d/python/python-1.5.2/Objects/dictobject.c trash/Objects/dictobject.c *** //d/python/python-1.5.2/Objects/dictobject.c Sat Feb 26 20:35:38 2000 --- trash/Objects/dictobject.c Wed Mar 08 19:07:30 2000 *************** *** 479,484 **** --- 479,485 ---- { register int i; register dictentry *ep; + Py_TRASHCAN_SAFE_BEGIN(mp) for (i = 0, ep = mp->ma_table; i < mp->ma_size; i++, ep++) { if (ep->me_key != NULL) { Py_DECREF(ep->me_key); *************** *** 489,494 **** --- 490,496 ---- } PyMem_XDEL(mp->ma_table); PyMem_DEL(mp); + Py_TRASHCAN_SAFE_END(mp) } static int diff -c -b -P -r //d/python/python-1.5.2/Objects/frameobject.c trash/Objects/frameobject.c *** //d/python/python-1.5.2/Objects/frameobject.c Sat Feb 26 20:35:40 2000 --- trash/Objects/frameobject.c Wed Mar 08 19:07:30 2000 *************** *** 103,108 **** --- 103,109 ---- int i; PyObject **fastlocals; + Py_TRASHCAN_SAFE_BEGIN(f) /* Kill all local variables */ fastlocals = f->f_localsplus; for (i = f->f_nlocals; --i >= 0; ++fastlocals) { *************** *** 120,125 **** --- 121,127 ---- Py_XDECREF(f->f_exc_traceback); f->f_back = free_list; free_list = f; + Py_TRASHCAN_SAFE_END(f) } PyTypeObject PyFrame_Type = { diff -c -b -P -r //d/python/python-1.5.2/Objects/listobject.c trash/Objects/listobject.c *** //d/python/python-1.5.2/Objects/listobject.c Sat Feb 26 20:35:40 2000 --- trash/Objects/listobject.c Wed Mar 08 19:07:30 2000 *************** *** 215,220 **** --- 215,221 ---- PyListObject *op; { int i; + Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_item != NULL) { /* Do it backwards, for Christian Tismer. There's a simple test case where somehow this reduces *************** *** 227,232 **** --- 228,234 ---- free((ANY *)op->ob_item); } free((ANY *)op); + Py_TRASHCAN_SAFE_END(op) } static int diff -c -b -P -r //d/python/python-1.5.2/Objects/object.c trash/Objects/object.c *** //d/python/python-1.5.2/Objects/object.c Sat Feb 26 20:35:42 2000 --- trash/Objects/object.c Fri Mar 10 00:40:24 2000 *************** *** 887,889 **** --- 887,934 ---- } } } + + /* + trashcan + CT 2k0130 + non-recursively destroy nested objects + + CT 2k0223 + everything is now done in a macro. + + CT 2k0305 + modified to use functions, after Tim Peter's suggestion. + + CT 2k0309 + modified to restore a possible error. + */ + + int _PyTrash_delete_nesting = 0; + PyObject * _PyTrash_delete_later = NULL; + + void + _PyTrash_deposit_object(op) + PyObject *op; + { + PyObject *error_type, *error_value, *error_traceback; + PyErr_Fetch(&error_type, &error_value, &error_traceback); + + if (!_PyTrash_delete_later) + _PyTrash_delete_later = PyList_New(0); + if (_PyTrash_delete_later) + PyList_Append(_PyTrash_delete_later, (PyObject *)op); + + PyErr_Restore(error_type, error_value, error_traceback); + } + + void + _PyTrash_destroy_list() + { + while (_PyTrash_delete_later) { + PyObject *shredder = _PyTrash_delete_later; + _PyTrash_delete_later = NULL; + ++_PyTrash_delete_nesting; + Py_DECREF(shredder); + --_PyTrash_delete_nesting; + } + } diff -c -b -P -r //d/python/python-1.5.2/Objects/tupleobject.c trash/Objects/tupleobject.c *** //d/python/python-1.5.2/Objects/tupleobject.c Sat Feb 26 20:35:44 2000 --- trash/Objects/tupleobject.c Wed Mar 08 19:07:30 2000 *************** *** 172,177 **** --- 172,178 ---- { register int i; + Py_TRASHCAN_SAFE_BEGIN(op) if (op->ob_size > 0) { i = op->ob_size; while (--i >= 0) *************** *** 180,190 **** if (op->ob_size < MAXSAVESIZE) { op->ob_item[0] = (PyObject *) free_tuples[op->ob_size]; free_tuples[op->ob_size] = op; ! return; } #endif } free((ANY *)op); } static int --- 181,193 ---- if (op->ob_size < MAXSAVESIZE) { op->ob_item[0] = (PyObject *) free_tuples[op->ob_size]; free_tuples[op->ob_size] = op; ! goto done; /* return */ } #endif } free((ANY *)op); + done: + Py_TRASHCAN_SAFE_END(op) } static int diff -c -b -P -r //d/python/python-1.5.2/Python/traceback.c trash/Python/traceback.c *** //d/python/python-1.5.2/Python/traceback.c Sat Sep 18 22:49:40 1999 --- trash/Python/traceback.c Wed Mar 08 19:07:30 2000 *************** *** 68,76 **** --- 68,78 ---- tb_dealloc(tb) tracebackobject *tb; { + Py_TRASHCAN_SAFE_BEGIN(tb) Py_XDECREF(tb->tb_next); Py_XDECREF(tb->tb_frame); PyMem_DEL(tb); + Py_TRASHCAN_SAFE_END(tb) } #define Tracebacktype PyTraceBack_Type %%%%%%%%%%%%%%%%%%%%%%% end patch %%%%%%%%%%%%%%%%%%%%%%%%%% Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home _______________________________________________ Patches mailing list Patches@python.org http://www.python.org/mailman/listinfo/patches From tismer@tismer.com Fri Mar 10 14:25:47 2000 From: tismer@tismer.com (Christian Tismer) Date: Fri, 10 Mar 2000 15:25:47 +0100 Subject: [Patches] Optional list.append behavior Message-ID: <38C905EB.11F0B0AD@tismer.com> Current status: list.append and 3 other list methods have been changed to do strict checking that they have a single argument. This is the desired behavior for Python 1.6 and up. Until then, all scripts and extensions have to be cleaned up and may no longer be sloppy with list.append. Why this patch: Until not everything else is in shape, having the strict append version in the CVS is a bit problematic. In order to be able to deploy 1.5.2+ builds without causing unforeseen trouble, I needed a way to switch the new behavior off without removing it. How it works: In the functions listappend, listindex, listcount and listremove, the function PyArg_ParseTuple is replaced by the macro PyArg_ParseTuple_Compat1. This macro expands to either PyArg_ParseTuple or an inlined expression that gives the old behavior. The behavior is controlled by a #define of STRICT_LIST_APPEND. This can be set in the makefile. If the python version goes to 1.6, it will be defined automatically. The patch: *** trash/objects/listobject.c Fri Mar 10 16:01:50 2000 --- //d/python/spc/sp/objects/listobject.c Fri Mar 10 16:09:32 2000 *************** *** 564,576 **** return ins(self, i, v); } static PyObject * listappend(self, args) PyListObject *self; PyObject *args; { PyObject *v; ! if (!PyArg_ParseTuple(args, "O:append", &v)) return NULL; return ins(self, (int) self->ob_size, v); } --- 564,595 ---- return ins(self, i, v); } + /* make the strict append version optional for 1.5.x + but enforce it for 1.6 and up + */ + #if PY_MAJOR_VERSION > 1 || PY_MAJOR_VERSION == 1 && PY_MINOR_VERSION > 5 + #define STRICT_LIST_APPEND + #endif + + #ifdef STRICT_LIST_APPEND + #define PyArg_ParseTuple_Compat1 PyArg_ParseTuple + #else + #define PyArg_ParseTuple_Compat1(args, format, ret) \ + ( \ + PyTuple_GET_SIZE(args) > 1 ? (*ret = args, 1) : \ + PyTuple_GET_SIZE(args) == 1 ? (*ret = PyTuple_GET_ITEM(args, 0), 1) : \ + PyArg_ParseTuple(args, format, ret) \ + ) + #endif + + static PyObject * listappend(self, args) PyListObject *self; PyObject *args; { PyObject *v; ! if (!PyArg_ParseTuple_Compat1(args, "O:append", &v)) return NULL; return ins(self, (int) self->ob_size, v); } *************** *** 1328,1334 **** int i; PyObject *v; ! if (!PyArg_ParseTuple(args, "O:index", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) --- 1347,1353 ---- int i; PyObject *v; ! if (!PyArg_ParseTuple_Compat1(args, "O:index", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) *************** *** 1349,1355 **** int i; PyObject *v; ! if (!PyArg_ParseTuple(args, "O:count", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) --- 1368,1374 ---- int i; PyObject *v; ! if (!PyArg_ParseTuple_Compat1(args, "O:count", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) *************** *** 1368,1374 **** int i; PyObject *v; ! if (!PyArg_ParseTuple(args, "O:remove", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) { --- 1387,1393 ---- int i; PyObject *v; ! if (!PyArg_ParseTuple_Compat1(args, "O:remove", &v)) return NULL; for (i = 0; i < self->ob_size; i++) { if (PyObject_Compare(self->ob_item[i], v) == 0) { %%%%%%%%%%%%%%%%%%%%%%% end patch %%%%%%%%%%%%%%%%%%%%%%%%%% Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From Vladimir.Marangozov@inrialpes.fr Fri Mar 10 14:51:17 2000 From: Vladimir.Marangozov@inrialpes.fr (Vladimir Marangozov) Date: Fri, 10 Mar 2000 15:51:17 +0100 (CET) Subject: [Patches] Optional list.append behavior In-Reply-To: <38C905EB.11F0B0AD@tismer.com> from "Christian Tismer" at Mar 10, 2000 03:25:47 PM Message-ID: <200003101451.PAA18410@python.inrialpes.fr> Christian Tismer wrote: > > Why this patch: > Until not everything else is in shape, having the strict > append version in the CVS is a bit problematic. In order to > be able to deploy 1.5.2+ builds without causing unforeseen > trouble, I needed a way to switch the new behavior off > without removing it. You delay the problem. It's the extensions that need to be fixed, not the core. IMO, better provoke the unforseen troubles now and fix them where they should be fixed :) -- Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 From tismer@tismer.com Fri Mar 10 16:24:43 2000 From: tismer@tismer.com (Christian Tismer) Date: Fri, 10 Mar 2000 17:24:43 +0100 Subject: [Patches] Optional list.append behavior References: <200003101451.PAA18410@python.inrialpes.fr> Message-ID: <38C921CB.2EC3341F@tismer.com> Vladimir Marangozov wrote: > > Christian Tismer wrote: > > > > Why this patch: > > Until not everything else is in shape, having the strict > > append version in the CVS is a bit problematic. In order to > > be able to deploy 1.5.2+ builds without causing unforeseen > > trouble, I needed a way to switch the new behavior off > > without removing it. > > You delay the problem. It's the extensions that need to be fixed, > not the core. IMO, better provoke the unforseen troubles now and > fix them where they should be fixed :) No. I loose a problem. I don't want to carry two source trees around, and I want to publish my stackless python build without bringing the users into trouble. SLP is used in production code meanwhile. If I let them crash, then *I* will have to solve problems which are not mine in the first place. The patch is no delay at all since python 1.6 will definately change append's behavior. Having to build my stuff against the old 1.5.2 library is even worse, since I believe it helps Python if 1.5.2+ is tested by some more people. Now I can compile two versions, and people might use that to compare the behavior of their applications. I think this is no delay but a win. ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From skip@mojam.com (Skip Montanaro) Fri Mar 10 17:57:26 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 10 Mar 2000 11:57:26 -0600 Subject: [Patches] httplib.HTTP.connect raises wrong exception Message-ID: <200003101757.LAA24297@beluga.mojam.com> In httplib.HTTP.connect a socket.error is raised when a non-numeric port is given. Seems to me this is more correctly a ValueError. I hereby release this patch into the public domain. Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ *** /tmp/httplib.py.~1.14~I Fri Mar 10 11:54:34 2000 --- /tmp/httplib.py Fri Mar 10 11:54:34 2000 *************** *** 105,111 **** host, port = host[:i], host[i+1:] try: port = string.atoi(port) except string.atoi_error: ! raise socket.error, "nonnumeric port" if not port: port = HTTP_PORT self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) if self.debuglevel > 0: print 'connect:', (host, port) --- 105,111 ---- host, port = host[:i], host[i+1:] try: port = string.atoi(port) except string.atoi_error: ! raise ValueError, "nonnumeric port" if not port: port = HTTP_PORT self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) if self.debuglevel > 0: print 'connect:', (host, port) From mwh21@cam.ac.uk Sat Mar 11 18:53:47 2000 From: mwh21@cam.ac.uk (Michael Hudson) Date: Sat, 11 Mar 2000 18:53:47 +0000 (GMT) Subject: [Patches] Lib/encodings not being installed Message-ID: With the recent unicode landing, the subject-mentioned problem occurs; this little patch sorts that out. Vladimir Marangozov suggested I send it to his address, so here it is. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Which seems even sillier than normal, as it's longer than the patch: Index: Makefile.in =================================================================== RCS file: /projects/cvsroot/python/dist/src/Makefile.in,v retrieving revision 1.83 diff -u -r1.83 Makefile.in --- Makefile.in 2000/02/28 19:52:53 1.83 +++ Makefile.in 2000/03/11 18:50:46 @@ -289,7 +289,7 @@ # Install the library PLATDIR= plat-$(MACHDEP) MACHDEPS= $(PLATDIR) -LIBSUBDIRS= lib-old lib-tk test test/output $(MACHDEPS) +LIBSUBDIRS= lib-old lib-tk test test/output encodings $(MACHDEPS) libinstall: python $(srcdir)/Lib/$(PLATDIR) @for i in $(SCRIPTDIR) $(LIBDEST); \ do \ Cheers, M. From tim_one@email.msn.com Sun Mar 12 08:41:17 2000 From: tim_one@email.msn.com (Tim Peters) Date: Sun, 12 Mar 2000 03:41:17 -0500 Subject: [Patches] Fix to IDLE PyParse.py fix Message-ID: <000001bf8bfe$bc9fb340$052d153f@tim> Fix bad auto-indent I recently introduced when replacing the regexp that could cause re to blow up: if or_any_other_block_opener: # one indenting comment line ^ cursor ended up at the caret (the bug) ^ but belongs here (the post-patch behavior) I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. *** idle/old/PyParse.py Mon Mar 06 20:48:32 2000 --- idle/PyParse.py Sun Mar 12 03:21:38 2000 *************** *** 385,397 **** m = _chew_ordinaryre(str, p, q) if m: # we skipped at least one boring char ! p = m.end() # back up over totally boring whitespace ! i = p-1 # index of last boring char ! while i >= 0 and str[i] in " \t\n": i = i-1 ! if i >= 0: lastch = str[i] if p >= q: break --- 385,398 ---- m = _chew_ordinaryre(str, p, q) if m: # we skipped at least one boring char ! newp = m.end() # back up over totally boring whitespace ! i = newp - 1 # index of last boring char ! while i >= p and str[i] in " \t\n": i = i-1 ! if i >= p: lastch = str[i] + p = newp if p >= q: break From brian@garage.co.jp Mon Mar 13 13:09:18 2000 From: brian@garage.co.jp (Brian Takashi Hooper) Date: Mon, 13 Mar 2000 22:09:18 +0900 Subject: [Patches] typos patch Message-ID: <38CCE87E72.16E4BRIAN@smtp.garage.co.jp> --TW9uLCAxMyBNYXIgMjAwMCAyMjowOToxOCArMDkwMA== Content-Transfer-Encoding: 7bit Content-Type: text/plain Fix some typos and spelling errors, mostly in the new Unicode-related sources, plus a few others that turned up from grep-ping recursively for those same words in the rest of the Python source. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Brian Hooper --TW9uLCAxMyBNYXIgMjAwMCAyMjowOToxOCArMDkwMA== Content-Type: application/octet-stream; name="python-typos.patch" Content-Disposition: attachment; filename="python-typos.patch" Content-Transfer-Encoding: base64 ZGlmZiAtYyAtciBweXRob24vZGlzdC9zcmMvSW5jbHVkZS91bmljb2Rlb2JqZWN0LmggcHl0aG9u Lm5ldy9kaXN0L3NyYy9JbmNsdWRlL3VuaWNvZGVvYmplY3QuaAoqKiogcHl0aG9uL2Rpc3Qvc3Jj L0luY2x1ZGUvdW5pY29kZW9iamVjdC5oCVNhdCBNYXIgMTEgMDc6MzM6MDUgMjAwMAotLS0gcHl0 aG9uLm5ldy9kaXN0L3NyYy9JbmNsdWRlL3VuaWNvZGVvYmplY3QuaAlNb24gTWFyIDEzIDE5OjIw OjQwIDIwMDAKKioqKioqKioqKioqKioqCioqKiA2NDcsNjUzICoqKioKICAgICAgaW50IGRpcmVj dGlvbgkJLyogRmluZCBkaXJlY3Rpb246ICsxIGZvcndhcmQsIC0xIGJhY2t3YXJkICovCiAgICAg ICk7CiAgCiEgLyogQ291bnQgdGhlIG51bWJlciBvZiBvY2N1cmFuY2VzIG9mIHN1YnN0ciBpbiBz dHJbc3RhcnQ6ZW5kXS4gKi8KICAKICBleHRlcm4gRExfSU1QT1JUKGludCkgUHlVbmljb2RlX0Nv dW50KAogICAgICBQeU9iamVjdCAqc3RyLAkJLyogU3RyaW5nICovIAotLS0gNjQ3LDY1MyAtLS0t CiAgICAgIGludCBkaXJlY3Rpb24JCS8qIEZpbmQgZGlyZWN0aW9uOiArMSBmb3J3YXJkLCAtMSBi YWNrd2FyZCAqLwogICAgICApOwogIAohIC8qIENvdW50IHRoZSBudW1iZXIgb2Ygb2NjdXJyZW5j ZXMgb2Ygc3Vic3RyIGluIHN0cltzdGFydDplbmRdLiAqLwogIAogIGV4dGVybiBETF9JTVBPUlQo aW50KSBQeVVuaWNvZGVfQ291bnQoCiAgICAgIFB5T2JqZWN0ICpzdHIsCQkvKiBTdHJpbmcgKi8g CioqKioqKioqKioqKioqKgoqKiogNjU2LDY2MiAqKioqCiAgICAgIGludCBlbmQJCQkvKiBTdG9w IGluZGV4ICovCiAgICAgICk7CiAgCiEgLyogUmVwbGFjZSBhdCBtb3N0IG1heGNvdW50IG9jY3Vy YW5jZXMgb2Ygc3Vic3RyIGluIHN0ciB3aXRoIHJlcGxzdHIKICAgICBhbmQgcmV0dXJuIHRoZSBy ZXN1bHRpbmcgVW5pY29kZSBvYmplY3QuICovCiAgCiAgZXh0ZXJuIERMX0lNUE9SVChQeU9iamVj dCAqKSBQeVVuaWNvZGVfUmVwbGFjZSgKLS0tIDY1Niw2NjIgLS0tLQogICAgICBpbnQgZW5kCQkJ LyogU3RvcCBpbmRleCAqLwogICAgICApOwogIAohIC8qIFJlcGxhY2UgYXQgbW9zdCBtYXhjb3Vu dCBvY2N1cnJlbmNlcyBvZiBzdWJzdHIgaW4gc3RyIHdpdGggcmVwbHN0cgogICAgIGFuZCByZXR1 cm4gdGhlIHJlc3VsdGluZyBVbmljb2RlIG9iamVjdC4gKi8KICAKICBleHRlcm4gRExfSU1QT1JU KFB5T2JqZWN0ICopIFB5VW5pY29kZV9SZXBsYWNlKApkaWZmIC1jIC1yIHB5dGhvbi9kaXN0L3Ny Yy9MaWIvY29kZWNzLnB5IHB5dGhvbi5uZXcvZGlzdC9zcmMvTGliL2NvZGVjcy5weQoqKiogcHl0 aG9uL2Rpc3Qvc3JjL0xpYi9jb2RlY3MucHkJU2F0IE1hciAxMSAwODoyMDo0MyAyMDAwCi0tLSBw eXRob24ubmV3L2Rpc3Qvc3JjL0xpYi9jb2RlY3MucHkJTW9uIE1hciAxMyAxOToxNDo0NyAyMDAw CioqKioqKioqKioqKioqKgoqKiogNTUsNjEgKioqKgogICAgICAiIiIKICAgICAgZGVmIGVuY29k ZShzZWxmLGlucHV0LGVycm9ycz0nc3RyaWN0Jyk6CiAgICAgICAgICAKISAgICAgICAgICIiIiBF bmNvZGVzIHRoZSBvYmplY3QgaW50cHV0IGFuZCByZXR1cm5zIGEgdHVwbGUgKG91dHB1dAogICAg ICAgICAgICAgIG9iamVjdCwgbGVuZ3RoIGNvbnN1bWVkKS4KICAKICAgICAgICAgICAgICBlcnJv cnMgZGVmaW5lcyB0aGUgZXJyb3IgaGFuZGxpbmcgdG8gYXBwbHkuIEl0IGRlZmF1bHRzIHRvCi0t LSA1NSw2MSAtLS0tCiAgICAgICIiIgogICAgICBkZWYgZW5jb2RlKHNlbGYsaW5wdXQsZXJyb3Jz PSdzdHJpY3QnKToKICAgICAgICAgIAohICAgICAgICAgIiIiIEVuY29kZXMgdGhlIG9iamVjdCBp bnB1dCBhbmQgcmV0dXJucyBhIHR1cGxlIChvdXRwdXQKICAgICAgICAgICAgICBvYmplY3QsIGxl bmd0aCBjb25zdW1lZCkuCiAgCiAgICAgICAgICAgICAgZXJyb3JzIGRlZmluZXMgdGhlIGVycm9y IGhhbmRsaW5nIHRvIGFwcGx5LiBJdCBkZWZhdWx0cyB0bwpkaWZmIC1jIC1yIHB5dGhvbi9kaXN0 L3NyYy9MaWIvZW5jb2RpbmdzL19faW5pdF9fLnB5IHB5dGhvbi5uZXcvZGlzdC9zcmMvTGliL2Vu Y29kaW5ncy9fX2luaXRfXy5weQoqKiogcHl0aG9uL2Rpc3Qvc3JjL0xpYi9lbmNvZGluZ3MvX19p bml0X18ucHkJU2F0IE1hciAxMSAwODoxNzoxOCAyMDAwCi0tLSBweXRob24ubmV3L2Rpc3Qvc3Jj L0xpYi9lbmNvZGluZ3MvX19pbml0X18ucHkJTW9uIE1hciAxMyAxOTowNjo1OSAyMDAwCioqKioq KioqKioqKioqKgoqKiogMzAsNDIgKioqKgogIGltcG9ydCBzdHJpbmcsY29kZWNzLGFsaWFzZXMK ICAKICBfY2FjaGUgPSB7fQohIF91bmtvd24gPSAnLS11bmtvd24tLScKICAKICBkZWYgc2VhcmNo X2Z1bmN0aW9uKGVuY29kaW5nKToKICAgICAgCiAgICAgICMgQ2FjaGUgbG9va3VwCiEgICAgIGVu dHJ5ID0gX2NhY2hlLmdldChlbmNvZGluZyxfdW5rb3duKQohICAgICBpZiBlbnRyeSBpcyBub3Qg X3Vua293bjoKICAgICAgICAgIHJldHVybiBlbnRyeQogIAogICAgICAjIEltcG9ydCB0aGUgbW9k dWxlCi0tLSAzMCw0MiAtLS0tCiAgaW1wb3J0IHN0cmluZyxjb2RlY3MsYWxpYXNlcwogIAogIF9j YWNoZSA9IHt9CiEgX3Vua25vd24gPSAnLS11bmtub3duLS0nCiAgCiAgZGVmIHNlYXJjaF9mdW5j dGlvbihlbmNvZGluZyk6CiAgICAgIAogICAgICAjIENhY2hlIGxvb2t1cAohICAgICBlbnRyeSA9 IF9jYWNoZS5nZXQoZW5jb2RpbmcsX3Vua25vd24pCiEgICAgIGlmIGVudHJ5IGlzIG5vdCBfdW5r bm93bjoKICAgICAgICAgIHJldHVybiBlbnRyeQogIAogICAgICAjIEltcG9ydCB0aGUgbW9kdWxl CmRpZmYgLWMgLXIgcHl0aG9uL2Rpc3Qvc3JjL0xpYi9zdHJpbmcucHkgcHl0aG9uLm5ldy9kaXN0 L3NyYy9MaWIvc3RyaW5nLnB5CioqKiBweXRob24vZGlzdC9zcmMvTGliL3N0cmluZy5weQlTdW4g TWFyIDEyIDA0OjEwOjIyIDIwMDAKLS0tIHB5dGhvbi5uZXcvZGlzdC9zcmMvTGliL3N0cmluZy5w eQlNb24gTWFyIDEzIDE5OjIzOjI5IDIwMDAKKioqKioqKioqKioqKioqCioqKiAxMTYsMTIyICoq KioKICAgICAgIiIiam9pbihsaXN0IFssc2VwXSkgLT4gc3RyaW5nCiAgCiAgICAgIFJldHVybiBh IHN0cmluZyBjb21wb3NlZCBvZiB0aGUgd29yZHMgaW4gbGlzdCwgd2l0aAohICAgICBpbnRlcnZl bmluZyBvY2N1cmVuY2VzIG9mIHNlcC4gIFRoZSBkZWZhdWx0IHNlcGFyYXRvciBpcyBhCiAgICAg IHNpbmdsZSBzcGFjZS4KICAKICAgICAgKGpvaW5maWVsZHMgYW5kIGpvaW4gYXJlIHN5bm9ueW1v dXMpCi0tLSAxMTYsMTIyIC0tLS0KICAgICAgIiIiam9pbihsaXN0IFssc2VwXSkgLT4gc3RyaW5n CiAgCiAgICAgIFJldHVybiBhIHN0cmluZyBjb21wb3NlZCBvZiB0aGUgd29yZHMgaW4gbGlzdCwg d2l0aAohICAgICBpbnRlcnZlbmluZyBvY2N1cnJlbmNlcyBvZiBzZXAuICBUaGUgZGVmYXVsdCBz ZXBhcmF0b3IgaXMgYQogICAgICBzaW5nbGUgc3BhY2UuCiAgCiAgICAgIChqb2luZmllbGRzIGFu ZCBqb2luIGFyZSBzeW5vbnltb3VzKQpkaWZmIC1jIC1yIHB5dGhvbi9kaXN0L3NyYy9MaWIvc3Ry aW5nb2xkLnB5IHB5dGhvbi5uZXcvZGlzdC9zcmMvTGliL3N0cmluZ29sZC5weQoqKiogcHl0aG9u L2Rpc3Qvc3JjL0xpYi9zdHJpbmdvbGQucHkJRnJpIEZlYiAxMSAwMjoxNzoxNCAyMDAwCi0tLSBw eXRob24ubmV3L2Rpc3Qvc3JjL0xpYi9zdHJpbmdvbGQucHkJTW9uIE1hciAxMyAxOToyMzo1NiAy MDAwCioqKioqKioqKioqKioqKgoqKiogMTE4LDEyNCAqKioqCiAgICAgICIiImpvaW4obGlzdCBb LHNlcF0pIC0+IHN0cmluZwogIAogICAgICBSZXR1cm4gYSBzdHJpbmcgY29tcG9zZWQgb2YgdGhl IHdvcmRzIGluIGxpc3QsIHdpdGgKISAgICAgaW50ZXJ2ZW5pbmcgb2NjdXJlbmNlcyBvZiBzZXAu ICBUaGUgZGVmYXVsdCBzZXBhcmF0b3IgaXMgYQogICAgICBzaW5nbGUgc3BhY2UuCiAgCiAgICAg IChqb2luZmllbGRzIGFuZCBqb2luIGFyZSBzeW5vbnltb3VzKQotLS0gMTE4LDEyNCAtLS0tCiAg ICAgICIiImpvaW4obGlzdCBbLHNlcF0pIC0+IHN0cmluZwogIAogICAgICBSZXR1cm4gYSBzdHJp bmcgY29tcG9zZWQgb2YgdGhlIHdvcmRzIGluIGxpc3QsIHdpdGgKISAgICAgaW50ZXJ2ZW5pbmcg b2NjdXJyZW5jZXMgb2Ygc2VwLiAgVGhlIGRlZmF1bHQgc2VwYXJhdG9yIGlzIGEKICAgICAgc2lu Z2xlIHNwYWNlLgogIAogICAgICAoam9pbmZpZWxkcyBhbmQgam9pbiBhcmUgc3lub255bW91cykK ZGlmZiAtYyAtciBweXRob24vZGlzdC9zcmMvTW9kdWxlcy9zdHJvcG1vZHVsZS5jIHB5dGhvbi5u ZXcvZGlzdC9zcmMvTW9kdWxlcy9zdHJvcG1vZHVsZS5jCioqKiBweXRob24vZGlzdC9zcmMvTW9k dWxlcy9zdHJvcG1vZHVsZS5jCVR1ZSBGZWIgMjkgMjI6NTk6MjQgMjAwMAotLS0gcHl0aG9uLm5l dy9kaXN0L3NyYy9Nb2R1bGVzL3N0cm9wbW9kdWxlLmMJTW9uIE1hciAxMyAxOToyMjo0NCAyMDAw CioqKioqKioqKioqKioqKgoqKiogMTkzLDE5OSAqKioqCiAgam9pbmZpZWxkcyhsaXN0IFssc2Vw XSkgLT4gc3RyaW5nXG5cCiAgXG5cCiAgUmV0dXJuIGEgc3RyaW5nIGNvbXBvc2VkIG9mIHRoZSB3 b3JkcyBpbiBsaXN0LCB3aXRoXG5cCiEgaW50ZXJ2ZW5pbmcgb2NjdXJlbmNlcyBvZiBzZXAuICBT ZXAgZGVmYXVsdHMgdG8gYSBzaW5nbGVcblwKICBzcGFjZS5cblwKICBcblwKICAoam9pbiBhbmQg am9pbmZpZWxkcyBhcmUgc3lub255bW91cykiOwotLS0gMTkzLDE5OSAtLS0tCiAgam9pbmZpZWxk cyhsaXN0IFssc2VwXSkgLT4gc3RyaW5nXG5cCiAgXG5cCiAgUmV0dXJuIGEgc3RyaW5nIGNvbXBv c2VkIG9mIHRoZSB3b3JkcyBpbiBsaXN0LCB3aXRoXG5cCiEgaW50ZXJ2ZW5pbmcgb2NjdXJyZW5j ZXMgb2Ygc2VwLiAgU2VwIGRlZmF1bHRzIHRvIGEgc2luZ2xlXG5cCiAgc3BhY2UuXG5cCiAgXG5c CiAgKGpvaW4gYW5kIGpvaW5maWVsZHMgYXJlIHN5bm9ueW1vdXMpIjsKKioqKioqKioqKioqKioq CioqKiAxMDU0LDEwNjAgKioqKgogIAogICAgc3Ryc3RyIHJlcGxhY2VtZW50IGZvciBhcmJpdHJh cnkgYmxvY2tzIG9mIG1lbW9yeS4KICAKISAgIExvY2F0ZXMgdGhlIGZpcnN0IG9jY3VyYW5jZSBp biB0aGUgbWVtb3J5IHBvaW50ZWQgdG8gYnkgTUVNIG9mIHRoZQogICAgY29udGVudHMgb2YgbWVt b3J5IHBvaW50ZWQgdG8gYnkgUEFULiAgUmV0dXJucyB0aGUgaW5kZXggaW50byBNRU0gaWYKICAg IGZvdW5kLCBvciAtMSBpZiBub3QgZm91bmQuICBJZiBsZW4gb2YgUEFUIGlzIGdyZWF0ZXIgdGhh biBsZW5ndGggb2YKICAgIE1FTSwgdGhlIGZ1bmN0aW9uIHJldHVybnMgLTEuCi0tLSAxMDU0LDEw NjAgLS0tLQogIAogICAgc3Ryc3RyIHJlcGxhY2VtZW50IGZvciBhcmJpdHJhcnkgYmxvY2tzIG9m IG1lbW9yeS4KICAKISAgIExvY2F0ZXMgdGhlIGZpcnN0IG9jY3VycmVuY2UgaW4gdGhlIG1lbW9y eSBwb2ludGVkIHRvIGJ5IE1FTSBvZiB0aGUKICAgIGNvbnRlbnRzIG9mIG1lbW9yeSBwb2ludGVk IHRvIGJ5IFBBVC4gIFJldHVybnMgdGhlIGluZGV4IGludG8gTUVNIGlmCiAgICBmb3VuZCwgb3Ig LTEgaWYgbm90IGZvdW5kLiAgSWYgbGVuIG9mIFBBVCBpcyBncmVhdGVyIHRoYW4gbGVuZ3RoIG9m CiAgICBNRU0sIHRoZSBmdW5jdGlvbiByZXR1cm5zIC0xLgoqKioqKioqKioqKioqKioKKioqIDEx MTAsMTExOSAqKioqCiAgLyoKICAgICBteW1lbXJlcGxhY2UKICAKISAgICBSZXR1cm4gYSBzdHJp bmcgaW4gd2hpY2ggYWxsIG9jY3VyZW5jZXMgb2YgUEFUIGluIG1lbW9yeSBTVFIgYXJlCiAgICAg cmVwbGFjZWQgd2l0aCBTVUIuCiAgCiEgICAgSWYgbGVuZ3RoIG9mIFBBVCBpcyBsZXNzIHRoYW4g bGVuZ3RoIG9mIFNUUiBvciB0aGVyZSBhcmUgbm8gb2NjdXJlbmNlcwogICAgIG9mIFBBVCBpbiBT VFIsIHRoZW4gdGhlIG9yaWdpbmFsIHN0cmluZyBpcyByZXR1cm5lZC4gT3RoZXJ3aXNlLCBhIG5l dwogICAgIHN0cmluZyBpcyBhbGxvY2F0ZWQgaGVyZSBhbmQgcmV0dXJuZWQuCiAgCi0tLSAxMTEw LDExMTkgLS0tLQogIC8qCiAgICAgbXltZW1yZXBsYWNlCiAgCiEgICAgUmV0dXJuIGEgc3RyaW5n IGluIHdoaWNoIGFsbCBvY2N1cnJlbmNlcyBvZiBQQVQgaW4gbWVtb3J5IFNUUiBhcmUKICAgICBy ZXBsYWNlZCB3aXRoIFNVQi4KICAKISAgICBJZiBsZW5ndGggb2YgUEFUIGlzIGxlc3MgdGhhbiBs ZW5ndGggb2YgU1RSIG9yIHRoZXJlIGFyZSBubyBvY2N1cnJlbmNlcwogICAgIG9mIFBBVCBpbiBT VFIsIHRoZW4gdGhlIG9yaWdpbmFsIHN0cmluZyBpcyByZXR1cm5lZC4gT3RoZXJ3aXNlLCBhIG5l dwogICAgIHN0cmluZyBpcyBhbGxvY2F0ZWQgaGVyZSBhbmQgcmV0dXJuZWQuCiAgCmRpZmYgLWMg LXIgcHl0aG9uL2Rpc3Qvc3JjL09iamVjdHMvc3RyaW5nb2JqZWN0LmMgcHl0aG9uLm5ldy9kaXN0 L3NyYy9PYmplY3RzL3N0cmluZ29iamVjdC5jCioqKiBweXRob24vZGlzdC9zcmMvT2JqZWN0cy9z dHJpbmdvYmplY3QuYwlTdW4gTWFyIDEyIDA0OjEwOjM5IDIwMDAKLS0tIHB5dGhvbi5uZXcvZGlz dC9zcmMvT2JqZWN0cy9zdHJpbmdvYmplY3QuYwlNb24gTWFyIDEzIDE5OjI0OjQ0IDIwMDAKKioq KioqKioqKioqKioqCioqKiAxMzkzLDEzOTkgKioqKgogIAogICAgc3Ryc3RyIHJlcGxhY2VtZW50 IGZvciBhcmJpdHJhcnkgYmxvY2tzIG9mIG1lbW9yeS4KICAKISAgIExvY2F0ZXMgdGhlIGZpcnN0 IG9jY3VyYW5jZSBpbiB0aGUgbWVtb3J5IHBvaW50ZWQgdG8gYnkgTUVNIG9mIHRoZQogICAgY29u dGVudHMgb2YgbWVtb3J5IHBvaW50ZWQgdG8gYnkgUEFULiAgUmV0dXJucyB0aGUgaW5kZXggaW50 byBNRU0gaWYKICAgIGZvdW5kLCBvciAtMSBpZiBub3QgZm91bmQuICBJZiBsZW4gb2YgUEFUIGlz IGdyZWF0ZXIgdGhhbiBsZW5ndGggb2YKICAgIE1FTSwgdGhlIGZ1bmN0aW9uIHJldHVybnMgLTEu Ci0tLSAxMzkzLDEzOTkgLS0tLQogIAogICAgc3Ryc3RyIHJlcGxhY2VtZW50IGZvciBhcmJpdHJh cnkgYmxvY2tzIG9mIG1lbW9yeS4KICAKISAgIExvY2F0ZXMgdGhlIGZpcnN0IG9jY3VycmVuY2Ug aW4gdGhlIG1lbW9yeSBwb2ludGVkIHRvIGJ5IE1FTSBvZiB0aGUKICAgIGNvbnRlbnRzIG9mIG1l bW9yeSBwb2ludGVkIHRvIGJ5IFBBVC4gIFJldHVybnMgdGhlIGluZGV4IGludG8gTUVNIGlmCiAg ICBmb3VuZCwgb3IgLTEgaWYgbm90IGZvdW5kLiAgSWYgbGVuIG9mIFBBVCBpcyBncmVhdGVyIHRo YW4gbGVuZ3RoIG9mCiAgICBNRU0sIHRoZSBmdW5jdGlvbiByZXR1cm5zIC0xLgoqKioqKioqKioq KioqKioKKioqIDE0NTEsMTQ2MCAqKioqCiAgLyoKICAgICBteW1lbXJlcGxhY2UKICAKISAgICBS ZXR1cm4gYSBzdHJpbmcgaW4gd2hpY2ggYWxsIG9jY3VyZW5jZXMgb2YgUEFUIGluIG1lbW9yeSBT VFIgYXJlCiAgICAgcmVwbGFjZWQgd2l0aCBTVUIuCiAgCiEgICAgSWYgbGVuZ3RoIG9mIFBBVCBp cyBsZXNzIHRoYW4gbGVuZ3RoIG9mIFNUUiBvciB0aGVyZSBhcmUgbm8gb2NjdXJlbmNlcwogICAg IG9mIFBBVCBpbiBTVFIsIHRoZW4gdGhlIG9yaWdpbmFsIHN0cmluZyBpcyByZXR1cm5lZC4gT3Ro ZXJ3aXNlLCBhIG5ldwogICAgIHN0cmluZyBpcyBhbGxvY2F0ZWQgaGVyZSBhbmQgcmV0dXJuZWQu CiAgCi0tLSAxNDUxLDE0NjAgLS0tLQogIC8qCiAgICAgbXltZW1yZXBsYWNlCiAgCiEgICAgUmV0 dXJuIGEgc3RyaW5nIGluIHdoaWNoIGFsbCBvY2N1cnJlbmNlcyBvZiBQQVQgaW4gbWVtb3J5IFNU UiBhcmUKICAgICByZXBsYWNlZCB3aXRoIFNVQi4KICAKISAgICBJZiBsZW5ndGggb2YgUEFUIGlz IGxlc3MgdGhhbiBsZW5ndGggb2YgU1RSIG9yIHRoZXJlIGFyZSBubyBvY2N1cnJlbmNlcwogICAg IG9mIFBBVCBpbiBTVFIsIHRoZW4gdGhlIG9yaWdpbmFsIHN0cmluZyBpcyByZXR1cm5lZC4gT3Ro ZXJ3aXNlLCBhIG5ldwogICAgIHN0cmluZyBpcyBhbGxvY2F0ZWQgaGVyZSBhbmQgcmV0dXJuZWQu CiAgCmRpZmYgLWMgLXIgcHl0aG9uL2Rpc3Qvc3JjL09iamVjdHMvdW5pY29kZW9iamVjdC5jIHB5 dGhvbi5uZXcvZGlzdC9zcmMvT2JqZWN0cy91bmljb2Rlb2JqZWN0LmMKKioqIHB5dGhvbi9kaXN0 L3NyYy9PYmplY3RzL3VuaWNvZGVvYmplY3QuYwlTYXQgTWFyIDExIDA3OjUzOjIzIDIwMDAKLS0t IHB5dGhvbi5uZXcvZGlzdC9zcmMvT2JqZWN0cy91bmljb2Rlb2JqZWN0LmMJTW9uIE1hciAxMyAx OToxMTowNSAyMDAwCioqKioqKioqKioqKioqKgoqKiogODMsODkgKioqKgogICAgIGFsbCBvYmpl Y3RzIG9uIHRoZSBmcmVlIGxpc3QgaGF2aW5nIGEgc2l6ZSBsZXNzIHRoYW4gdGhpcwogICAgIGxp bWl0LiBUaGlzIHJlZHVjZXMgbWFsbG9jKCkgb3ZlcmhlYWQgZm9yIHNtYWxsIFVuaWNvZGUgb2Jq ZWN0cy4gIAogIAohICAgIEF0IHdvcnNlIHRoaXMgd2lsbCByZXN1bHQgaW4gTUFYX1VOSUNPREVf RlJFRUxJU1RfU0laRSAqCiAgICAgKHNpemVvZihQeVVuaWNvZGVPYmplY3QpICsgU1RBWUFMSVZF X1NJWkVfTElNSVQgKwogICAgIG1hbGxvYygpLW92ZXJoZWFkKSBieXRlcyBvZiB1bnVzZWQgZ2Fy YmFnZS4KICAKLS0tIDgzLDg5IC0tLS0KICAgICBhbGwgb2JqZWN0cyBvbiB0aGUgZnJlZSBsaXN0 IGhhdmluZyBhIHNpemUgbGVzcyB0aGFuIHRoaXMKICAgICBsaW1pdC4gVGhpcyByZWR1Y2VzIG1h bGxvYygpIG92ZXJoZWFkIGZvciBzbWFsbCBVbmljb2RlIG9iamVjdHMuICAKICAKISAgICBBdCB3 b3JzdCB0aGlzIHdpbGwgcmVzdWx0IGluIE1BWF9VTklDT0RFX0ZSRUVMSVNUX1NJWkUgKgogICAg IChzaXplb2YoUHlVbmljb2RlT2JqZWN0KSArIFNUQVlBTElWRV9TSVpFX0xJTUlUICsKICAgICBt YWxsb2MoKS1vdmVyaGVhZCkgYnl0ZXMgb2YgdW51c2VkIGdhcmJhZ2UuCiAgCioqKioqKioqKioq KioqKgoqKiogNDg5LDQ5NSAqKioqCiAgICAgIH0KICAgICAgZWxzZSB7CiAgICAgICAgICBQeUVy cl9Gb3JtYXQoUHlFeGNfVmFsdWVFcnJvciwKISAgICAgICAgICAgICAgICAgICAgICAiVVRGLTgg ZGVjb2RpbmcgZXJyb3I7IHVua293biBlcnJvciBoYW5kbGluZyBjb2RlOiAlcyIsCiAgICAgICAg ICAgICAgICAgICAgICAgZXJyb3JzKTsKICAgICAgICAgIHJldHVybiAtMTsKICAgICAgfQotLS0g NDg5LDQ5NSAtLS0tCiAgICAgIH0KICAgICAgZWxzZSB7CiAgICAgICAgICBQeUVycl9Gb3JtYXQo UHlFeGNfVmFsdWVFcnJvciwKISAgICAgICAgICAgICAgICAgICAgICAiVVRGLTggZGVjb2Rpbmcg ZXJyb3I7IHVua25vd24gZXJyb3IgaGFuZGxpbmcgY29kZTogJXMiLAogICAgICAgICAgICAgICAg ICAgICAgIGVycm9ycyk7CiAgICAgICAgICByZXR1cm4gLTE7CiAgICAgIH0KKioqKioqKioqKioq KioqCioqKiA2MTEsNjE3ICoqKioKICAgICAgZWxzZSB7CiAgCVB5RXJyX0Zvcm1hdChQeUV4Y19W YWx1ZUVycm9yLAogIAkJICAgICAiVVRGLTggZW5jb2RpbmcgZXJyb3I7ICIKISAJCSAgICAgInVu a293biBlcnJvciBoYW5kbGluZyBjb2RlOiAlcyIsCiAgCQkgICAgIGVycm9ycyk7CiAgCXJldHVy biAtMTsKICAgICAgfQotLS0gNjExLDYxNyAtLS0tCiAgICAgIGVsc2UgewogIAlQeUVycl9Gb3Jt YXQoUHlFeGNfVmFsdWVFcnJvciwKICAJCSAgICAgIlVURi04IGVuY29kaW5nIGVycm9yOyAiCiEg CQkgICAgICJ1bmtub3duIGVycm9yIGhhbmRsaW5nIGNvZGU6ICVzIiwKICAJCSAgICAgZXJyb3Jz KTsKICAJcmV0dXJuIC0xOwogICAgICB9CioqKioqKioqKioqKioqKgoqKiogNzMzLDczOSAqKioq CiAgICAgIH0KICAgICAgZWxzZSB7CiAgICAgICAgICBQeUVycl9Gb3JtYXQoUHlFeGNfVmFsdWVF cnJvciwKISAgICAgICAgICAgICAgICAgICAgICAiVVRGLTE2IGRlY29kaW5nIGVycm9yOyB1bmtv d24gZXJyb3IgaGFuZGxpbmcgY29kZTogJXMiLAogICAgICAgICAgICAgICAgICAgICAgIGVycm9y cyk7CiAgICAgICAgICByZXR1cm4gLTE7CiAgICAgIH0KLS0tIDczMyw3MzkgLS0tLQogICAgICB9 CiAgICAgIGVsc2UgewogICAgICAgICAgUHlFcnJfRm9ybWF0KFB5RXhjX1ZhbHVlRXJyb3IsCiEg ICAgICAgICAgICAgICAgICAgICAgIlVURi0xNiBkZWNvZGluZyBlcnJvcjsgdW5rbm93biBlcnJv ciBoYW5kbGluZyBjb2RlOiAlcyIsCiAgICAgICAgICAgICAgICAgICAgICAgZXJyb3JzKTsKICAg ICAgICAgIHJldHVybiAtMTsKICAgICAgfQoqKioqKioqKioqKioqKioKKioqIDkyMSw5MjcgKioq KgogICAgICBlbHNlIHsKICAgICAgICAgIFB5RXJyX0Zvcm1hdChQeUV4Y19WYWx1ZUVycm9yLAog ICAgICAgICAgICAgICAgICAgICAgICJVbmljb2RlLUVzY2FwZSBkZWNvZGluZyBlcnJvcjsgIgoh ICAgICAgICAgICAgICAgICAgICAgICJ1bmtvd24gZXJyb3IgaGFuZGxpbmcgY29kZTogJXMiLAog ICAgICAgICAgICAgICAgICAgICAgIGVycm9ycyk7CiAgICAgICAgICByZXR1cm4gLTE7CiAgICAg IH0KLS0tIDkyMSw5MjcgLS0tLQogICAgICBlbHNlIHsKICAgICAgICAgIFB5RXJyX0Zvcm1hdChQ eUV4Y19WYWx1ZUVycm9yLAogICAgICAgICAgICAgICAgICAgICAgICJVbmljb2RlLUVzY2FwZSBk ZWNvZGluZyBlcnJvcjsgIgohICAgICAgICAgICAgICAgICAgICAgICJ1bmtub3duIGVycm9yIGhh bmRsaW5nIGNvZGU6ICVzIiwKICAgICAgICAgICAgICAgICAgICAgICBlcnJvcnMpOwogICAgICAg ICAgcmV0dXJuIC0xOwogICAgICB9CioqKioqKioqKioqKioqKgoqKiogMTI5OCwxMzA0ICoqKioK ICAgICAgZWxzZSB7CiAgCVB5RXJyX0Zvcm1hdChQeUV4Y19WYWx1ZUVycm9yLAogIAkJICAgICAi TGF0aW4tMSBlbmNvZGluZyBlcnJvcjsgIgohIAkJICAgICAidW5rb3duIGVycm9yIGhhbmRsaW5n IGNvZGU6ICVzIiwKICAJCSAgICAgZXJyb3JzKTsKICAJcmV0dXJuIC0xOwogICAgICB9Ci0tLSAx Mjk4LDEzMDQgLS0tLQogICAgICBlbHNlIHsKICAJUHlFcnJfRm9ybWF0KFB5RXhjX1ZhbHVlRXJy b3IsCiAgCQkgICAgICJMYXRpbi0xIGVuY29kaW5nIGVycm9yOyAiCiEgCQkgICAgICJ1bmtub3du IGVycm9yIGhhbmRsaW5nIGNvZGU6ICVzIiwKICAJCSAgICAgZXJyb3JzKTsKICAJcmV0dXJuIC0x OwogICAgICB9CioqKioqKioqKioqKioqKgoqKiogMTM2OSwxMzc1ICoqKioKICAgICAgZWxzZSB7 CiAgCVB5RXJyX0Zvcm1hdChQeUV4Y19WYWx1ZUVycm9yLAogIAkJICAgICAiQVNDSUkgZGVjb2Rp bmcgZXJyb3I7ICIKISAJCSAgICAgInVua293biBlcnJvciBoYW5kbGluZyBjb2RlOiAlcyIsCiAg CQkgICAgIGVycm9ycyk7CiAgCXJldHVybiAtMTsKICAgICAgfQotLS0gMTM2OSwxMzc1IC0tLS0K ICAgICAgZWxzZSB7CiAgCVB5RXJyX0Zvcm1hdChQeUV4Y19WYWx1ZUVycm9yLAogIAkJICAgICAi QVNDSUkgZGVjb2RpbmcgZXJyb3I7ICIKISAJCSAgICAgInVua25vd24gZXJyb3IgaGFuZGxpbmcg Y29kZTogJXMiLAogIAkJICAgICBlcnJvcnMpOwogIAlyZXR1cm4gLTE7CiAgICAgIH0KKioqKioq KioqKioqKioqCioqKiAxNDMxLDE0MzcgKioqKgogICAgICBlbHNlIHsKICAJUHlFcnJfRm9ybWF0 KFB5RXhjX1ZhbHVlRXJyb3IsCiAgCQkgICAgICJBU0NJSSBlbmNvZGluZyBlcnJvcjsgIgohIAkJ ICAgICAidW5rb3duIGVycm9yIGhhbmRsaW5nIGNvZGU6ICVzIiwKICAJCSAgICAgZXJyb3JzKTsK ICAJcmV0dXJuIC0xOwogICAgICB9Ci0tLSAxNDMxLDE0MzcgLS0tLQogICAgICBlbHNlIHsKICAJ UHlFcnJfRm9ybWF0KFB5RXhjX1ZhbHVlRXJyb3IsCiAgCQkgICAgICJBU0NJSSBlbmNvZGluZyBl cnJvcjsgIgohIAkJICAgICAidW5rbm93biBlcnJvciBoYW5kbGluZyBjb2RlOiAlcyIsCiAgCQkg ICAgIGVycm9ycyk7CiAgCXJldHVybiAtMTsKICAgICAgfQoqKioqKioqKioqKioqKioKKioqIDE1 MDIsMTUwOCAqKioqCiAgICAgIGVsc2UgewogIAlQeUVycl9Gb3JtYXQoUHlFeGNfVmFsdWVFcnJv ciwKICAJCSAgICAgImNoYXJtYXAgZGVjb2RpbmcgZXJyb3I7ICIKISAJCSAgICAgInVua293biBl cnJvciBoYW5kbGluZyBjb2RlOiAlcyIsCiAgCQkgICAgIGVycm9ycyk7CiAgCXJldHVybiAtMTsK ICAgICAgfQotLS0gMTUwMiwxNTA4IC0tLS0KICAgICAgZWxzZSB7CiAgCVB5RXJyX0Zvcm1hdChQ eUV4Y19WYWx1ZUVycm9yLAogIAkJICAgICAiY2hhcm1hcCBkZWNvZGluZyBlcnJvcjsgIgohIAkJ ICAgICAidW5rbm93biBlcnJvciBoYW5kbGluZyBjb2RlOiAlcyIsCiAgCQkgICAgIGVycm9ycyk7 CiAgCXJldHVybiAtMTsKICAgICAgfQoqKioqKioqKioqKioqKioKKioqIDE2MTgsMTYyNCAqKioq CiAgICAgIGVsc2UgewogIAlQeUVycl9Gb3JtYXQoUHlFeGNfVmFsdWVFcnJvciwKICAJCSAgICAg ImNoYXJtYXAgZW5jb2RpbmcgZXJyb3I7ICIKISAJCSAgICAgInVua293biBlcnJvciBoYW5kbGlu ZyBjb2RlOiAlcyIsCiAgCQkgICAgIGVycm9ycyk7CiAgCXJldHVybiAtMTsKICAgICAgfQotLS0g MTYxOCwxNjI0IC0tLS0KICAgICAgZWxzZSB7CiAgCVB5RXJyX0Zvcm1hdChQeUV4Y19WYWx1ZUVy cm9yLAogIAkJICAgICAiY2hhcm1hcCBlbmNvZGluZyBlcnJvcjsgIgohIAkJICAgICAidW5rbm93 biBlcnJvciBoYW5kbGluZyBjb2RlOiAlcyIsCiAgCQkgICAgIGVycm9ycyk7CiAgCXJldHVybiAt MTsKICAgICAgfQoqKioqKioqKioqKioqKioKKioqIDE3NTAsMTc1NiAqKioqCiAgICAgIGVsc2Ug ewogIAlQeUVycl9Gb3JtYXQoUHlFeGNfVmFsdWVFcnJvciwKICAJCSAgICAgInRyYW5zbGF0ZSBl cnJvcjsgIgohIAkJICAgICAidW5rb3duIGVycm9yIGhhbmRsaW5nIGNvZGU6ICVzIiwKICAJCSAg ICAgZXJyb3JzKTsKICAJcmV0dXJuIC0xOwogICAgICB9Ci0tLSAxNzUwLDE3NTYgLS0tLQogICAg ICBlbHNlIHsKICAJUHlFcnJfRm9ybWF0KFB5RXhjX1ZhbHVlRXJyb3IsCiAgCQkgICAgICJ0cmFu c2xhdGUgZXJyb3I7ICIKISAJCSAgICAgInVua25vd24gZXJyb3IgaGFuZGxpbmcgY29kZTogJXMi LAogIAkJICAgICBlcnJvcnMpOwogIAlyZXR1cm4gLTE7CiAgICAgIH0KZGlmZiAtYyAtciBweXRo b24vZGlzdC9zcmMvUHl0aG9uL2NvZGVjcy5jIHB5dGhvbi5uZXcvZGlzdC9zcmMvUHl0aG9uL2Nv ZGVjcy5jCioqKiBweXRob24vZGlzdC9zcmMvUHl0aG9uL2NvZGVjcy5jCVNhdCBNYXIgMTEgMDc6 NTc6MjcgMjAwMAotLS0gcHl0aG9uLm5ldy9kaXN0L3NyYy9QeXRob24vY29kZWNzLmMJTW9uIE1h ciAxMyAxOToxMDo0MCAyMDAwCioqKioqKioqKioqKioqKgoqKiogMTQ2LDE1MiAqKioqCiAgICAg IGlmIChpID09IGxlbikgewogIAkvKiBYWFggUGVyaGFwcyB3ZSBzaG91bGQgY2FjaGUgbWlzc2Vz IHRvbyA/ICovCiAgCVB5RXJyX1NldFN0cmluZyhQeUV4Y19Mb29rdXBFcnJvciwKISAJCQkidW5r b3duIGVuY29kaW5nIik7CiAgCWdvdG8gb25FcnJvcjsKICAgICAgfQogIAotLS0gMTQ2LDE1MiAt LS0tCiAgICAgIGlmIChpID09IGxlbikgewogIAkvKiBYWFggUGVyaGFwcyB3ZSBzaG91bGQgY2Fj aGUgbWlzc2VzIHRvbyA/ICovCiAgCVB5RXJyX1NldFN0cmluZyhQeUV4Y19Mb29rdXBFcnJvciwK ISAJCQkidW5rbm93biBlbmNvZGluZyIpOwogIAlnb3RvIG9uRXJyb3I7CiAgICAgIH0KICAKZGlm ZiAtYyAtciBweXRob24vZGlzdC9zcmMvY29uZmlndXJlIHB5dGhvbi5uZXcvZGlzdC9zcmMvY29u ZmlndXJlCioqKiBweXRob24vZGlzdC9zcmMvY29uZmlndXJlCVN1biBNYXIgMTIgMDQ6MTA6MjAg MjAwMAotLS0gcHl0aG9uLm5ldy9kaXN0L3NyYy9jb25maWd1cmUJTW9uIE1hciAxMyAxOToxNDo0 NCAyMDAwCioqKioqKioqKioqKioqKgoqKiogNTA1MSw1MDU3ICoqKioKICAKICAKICAjIGNoZWNr IGZvciB1c2FibGUgd2NoYXJfdAohIHVzYWJsZV93Y2hhcl90PSJ1bmtvd24iCiAgZWNobyAkYWNf biAiY2hlY2tpbmcgZm9yIHVzYWJsZSB3Y2hhcl90IiIuLi4gJGFjX2MiIDE+JjYKICBlY2hvICJj b25maWd1cmU6NTA1NzogY2hlY2tpbmcgZm9yIHVzYWJsZSB3Y2hhcl90IiA+JjUKICBpZiB0ZXN0 ICIkY3Jvc3NfY29tcGlsaW5nIiA9IHllczsgdGhlbgotLS0gNTA1MSw1MDU3IC0tLS0KICAKICAK ICAjIGNoZWNrIGZvciB1c2FibGUgd2NoYXJfdAohIHVzYWJsZV93Y2hhcl90PSJ1bmtub3duIgog IGVjaG8gJGFjX24gImNoZWNraW5nIGZvciB1c2FibGUgd2NoYXJfdCIiLi4uICRhY19jIiAxPiY2 CiAgZWNobyAiY29uZmlndXJlOjUwNTc6IGNoZWNraW5nIGZvciB1c2FibGUgd2NoYXJfdCIgPiY1 CiAgaWYgdGVzdCAiJGNyb3NzX2NvbXBpbGluZyIgPSB5ZXM7IHRoZW4KZGlmZiAtYyAtciBweXRo b24vZGlzdC9zcmMvY29uZmlndXJlLmluIHB5dGhvbi5uZXcvZGlzdC9zcmMvY29uZmlndXJlLmlu CioqKiBweXRob24vZGlzdC9zcmMvY29uZmlndXJlLmluCVN1biBNYXIgMTIgMDQ6MTA6MjAgMjAw MAotLS0gcHl0aG9uLm5ldy9kaXN0L3NyYy9jb25maWd1cmUuaW4JTW9uIE1hciAxMyAxOToxMzoy OCAyMDAwCioqKioqKioqKioqKioqKgoqKiogMTAzOSwxMDQ1ICoqKioKICApCiAgCiAgIyBjaGVj ayBmb3IgdXNhYmxlIHdjaGFyX3QKISB1c2FibGVfd2NoYXJfdD0idW5rb3duIgogIEFDX01TR19D SEVDS0lORyhmb3IgdXNhYmxlIHdjaGFyX3QpCiAgQUNfVFJZX1JVTihbCiAgI2luY2x1ZGUgIndj aGFyLmgiCi0tLSAxMDM5LDEwNDUgLS0tLQogICkKICAKICAjIGNoZWNrIGZvciB1c2FibGUgd2No YXJfdAohIHVzYWJsZV93Y2hhcl90PSJ1bmtub3duIgogIEFDX01TR19DSEVDS0lORyhmb3IgdXNh YmxlIHdjaGFyX3QpCiAgQUNfVFJZX1JVTihbCiAgI2luY2x1ZGUgIndjaGFyLmgiCg== --TW9uLCAxMyBNYXIgMjAwMCAyMjowOToxOCArMDkwMA==-- From mal@lemburg.com Mon Mar 13 13:36:34 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Mon, 13 Mar 2000 14:36:34 +0100 Subject: [Patches] typos patch References: <38CCE87E72.16E4BRIAN@smtp.garage.co.jp> Message-ID: <38CCEEE2.A73C563E@lemburg.com> Brian Takashi Hooper wrote: > > Fix some typos and spelling errors, mostly in the new Unicode-related > sources, plus a few others that turned up from grep-ping recursively for > those same words in the rest of the Python source. I've folded these into my patch set. I guess I'll submit the patch set for inclusion by the end of the week. Thanks, -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From guido@python.org Mon Mar 13 15:43:22 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Mar 2000 10:43:22 -0500 Subject: [Patches] Optional list.append behavior In-Reply-To: Your message of "Fri, 10 Mar 2000 15:25:47 +0100." <38C905EB.11F0B0AD@tismer.com> References: <38C905EB.11F0B0AD@tismer.com> Message-ID: <200003131543.KAA00465@eric.cnri.reston.va.us> I've added this patch, with a twist: it's disabled by default. You have to manually define NO_STRICT_LIST_APPEND to enable multi-arg append(). --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer@tismer.com Mon Mar 13 15:56:42 2000 From: tismer@tismer.com (Christian Tismer) Date: Mon, 13 Mar 2000 16:56:42 +0100 Subject: [Patches] Optional list.append behavior References: <38C905EB.11F0B0AD@tismer.com> <200003131543.KAA00465@eric.cnri.reston.va.us> Message-ID: <38CD0FBA.C4789A8F@tismer.com> Guido van Rossum wrote: > > I've added this patch, with a twist: it's disabled by default. You > have to manually define NO_STRICT_LIST_APPEND to enable multi-arg > append(). Fine with me. About the recursive delete: I'm about to make SLP 1.1 available and would like to wait until that other patch is in. But I'm a bit worrying about the error handling code that I added. The error handling enforces the existence of tstate. Didn't we find out that it doesn't exist on finalization? Do I need to check tstate for NULL, maybe? cheers - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From guido@python.org Mon Mar 13 16:02:15 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Mar 2000 11:02:15 -0500 Subject: [Patches] Safe destruction of recursive objects V2b In-Reply-To: Your message of "Fri, 10 Mar 2000 14:50:05 +0100." <38C8FD8D.314EE5F0@tismer.com> References: <38C8FD8D.314EE5F0@tismer.com> Message-ID: <200003131602.LAA02948@eric.cnri.reston.va.us> I've checked in the trashcan patch version 2b. Thanks, Christian! --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Mar 13 16:09:00 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 13 Mar 2000 11:09:00 -0500 Subject: [Patches] Optional list.append behavior In-Reply-To: Your message of "Mon, 13 Mar 2000 16:56:42 +0100." <38CD0FBA.C4789A8F@tismer.com> References: <38C905EB.11F0B0AD@tismer.com> <200003131543.KAA00465@eric.cnri.reston.va.us> <38CD0FBA.C4789A8F@tismer.com> Message-ID: <200003131609.LAA03778@eric.cnri.reston.va.us> > About the recursive delete: I'm about to make SLP 1.1 > available and would like to wait until that other patch > is in. But I'm a bit worrying about the error handling > code that I added. The error handling enforces the > existence of tstate. Didn't we find out that it doesn't > exist on finalization? Do I need to check tstate for > NULL, maybe? Oops. I can't reproduce this however. I created a 100000-nested tuple that contained an object at the bottom to print "bye". It got deleted correctly when I exited. I think only very few things are deleted after the tstate is deleted... The stuff that gets called by these: _PyBuiltin_Fini_2(); PyMethod_Fini(); PyFrame_Fini(); PyCFunction_Fini(); PyTuple_Fini(); PyString_Fini(); PyInt_Fini(); PyFloat_Fini(); PyGrammar_RemoveAccelerators(&_PyParser_Grammar); call_ll_exitfuncs(); _Py_ResetReferences(); I don't have the time right now to inspect all these for destruction of user objects; most seem to get rid of free lists only (e.g. PyTuple_Fini()). --Guido van Rossum (home page: http://www.python.org/~guido/) From tismer@tismer.com Mon Mar 13 16:24:50 2000 From: tismer@tismer.com (Christian Tismer) Date: Mon, 13 Mar 2000 17:24:50 +0100 Subject: [Patches] Optional list.append behavior References: <38C905EB.11F0B0AD@tismer.com> <200003131543.KAA00465@eric.cnri.reston.va.us> <38CD0FBA.C4789A8F@tismer.com> <200003131609.LAA03778@eric.cnri.reston.va.us> Message-ID: <38CD1652.F45D2646@tismer.com> Guido van Rossum wrote: > Oops. :-)) > I can't reproduce this however. I created a 100000-nested tuple that > contained an object at the bottom to print "bye". It got deleted > correctly when I exited. I think only very few things are deleted > after the tstate is deleted... The stuff that gets called by these: ... > I don't have the time right now to inspect all these for destruction > of user objects; most seem to get rid of free lists only > (e.g. PyTuple_Fini()). I will check these. However: It would not hurt at all to do the error check like now, but embrace it with an if (PyThreadState_GET()) { ... } and otherwise simply forget about the object, since we are finalizing anyway. But I'll check these conditions and see if any action is necessary at all. ciao - chris -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home From sjoerd@oratrix.nl Thu Mar 23 13:52:40 2000 From: sjoerd@oratrix.nl (Sjoerd Mullender) Date: Thu, 23 Mar 2000 14:52:40 +0100 Subject: [Patches] cmp not used in freeze Message-ID: <20000323135241.5945B301CF9@bireme.oratrix.nl> cmp is not used in freeze, but is imported anyway. What's worse, cmp is no longer in the library, so freeze won't work like this. Index: freeze.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Tools/freeze/freeze.py,v retrieving revision 1.33 diff -u -r1.33 freeze.py --- freeze.py 1999/03/12 22:07:05 1.33 +++ freeze.py 2000/03/23 13:49:37 @@ -78,7 +78,6 @@ # Import standard modules -import cmp import getopt import os import string I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Sjoerd Mullender From bthomas@hirevelocity.com Thu Mar 23 16:17:19 2000 From: bthomas@hirevelocity.com (Bill Thomas) Date: Thu, 23 Mar 2000 11:17:19 -0500 Subject: [Patches] Quick Question Message-ID: <006801bf94e3$436a7300$c71aa4d8@atw.pa.cable.rcn.com> Thursday, 3/23/00 Hi, I got your name from the SF Web Firmlist. Any Idea where the best place is, to find Consultant level Salespeople for a San Fran based Web Sales Company. We've been trying for a while and no luck... Can you help, thanks, Bill Thomas From mal@lemburg.com Fri Mar 24 08:38:43 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 24 Mar 2000 09:38:43 +0100 Subject: [Patches] Unicode codec lookup core dump Message-ID: <38DB2993.5F892A00@lemburg.com> Andy Robinson noted a core dump in the codecs.c file. This was introduced by my latest patch which fixed a memory leak in codecs.c. The bug causes all successful codec lookups to fail. Here's the patch that fixes the problem: --- CVS-Python/Python/codecs.c Fri Mar 24 00:02:04 2000 +++ Python+Unicode/Python/codecs.c Fri Mar 24 00:01:49 2000 @@ -91,11 +91,11 @@ PyObject *lowercasestring(const char *st If no codec is found, a KeyError is set and NULL returned. */ PyObject *_PyCodec_Lookup(const char *encoding) { - PyObject *result, *args = NULL, *v = NULL; + PyObject *result, *args = NULL, *v; int i, len; if (_PyCodec_SearchCache == NULL || _PyCodec_SearchPath == NULL) { PyErr_SetString(PyExc_SystemError, "codec module not properly initialized"); @@ -117,27 +117,26 @@ PyObject *_PyCodec_Lookup(const char *en Py_DECREF(v); return result; } /* Next, scan the search functions in order of registration */ - len = PyList_Size(_PyCodec_SearchPath); - if (len < 0) - goto onError; - args = PyTuple_New(1); if (args == NULL) goto onError; PyTuple_SET_ITEM(args,0,v); - v = NULL; + + len = PyList_Size(_PyCodec_SearchPath); + if (len < 0) + goto onError; for (i = 0; i < len; i++) { PyObject *func; func = PyList_GetItem(_PyCodec_SearchPath, i); if (func == NULL) goto onError; - result = PyEval_CallObject(func,args); + result = PyEval_CallObject(func, args); if (result == NULL) goto onError; if (result == Py_None) { Py_DECREF(result); continue; @@ -161,11 +160,10 @@ PyObject *_PyCodec_Lookup(const char *en PyDict_SetItem(_PyCodec_SearchCache, v, result); Py_DECREF(args); return result; onError: - Py_XDECREF(v); Py_XDECREF(args); return NULL; } static -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas@xs4all.net Fri Mar 24 20:41:24 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Fri, 24 Mar 2000 21:41:24 +0100 Subject: [Patches] mailbox.UnixMailbox and rfc822.Message.unixfrom Message-ID: <20000324214124.B12922@xs4all.nl> --LQksG6bCIzRHxTLp Content-Type: text/plain; charset=us-ascii The attached patch fixes a bug in mailbox.UnixMailbox that I hit upon just this afternoon, tracking a bug in Mailman. mailbox.UnixMailbox eats the 'From ' line in each message, before it passes off the 'subfile' to rfc822.Message, which means rfc822.Message.unixfrom will be emtpy for all messages. I'm not really sure if this is the proper fix, but it seemed more proper than overriding next() in mailbox.UnixMailbox. I'm also not really sure if this has been a pending issue, but a quick Deja search revealed nothing of the kind. Sadly, I've been so swamped in work for the last three weeks that I haven't been able to read any of the python and mailman lists. (Barry & other mailman-devel readers, I'll post about the bug in mailman later, after I get a chance to read all pending mailman-devel mail. The bug involves moderated lists and their archives.) I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! --LQksG6bCIzRHxTLp Content-Type: text/plain; charset=us-ascii Content-Description: mailbox.py.diff Content-Disposition: attachment; filename="mailbox.diff" Index: mailbox.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/mailbox.py,v retrieving revision 1.18 diff -c -r1.18 mailbox.py *** mailbox.py 2000/02/10 17:17:13 1.18 --- mailbox.py 2000/03/24 20:09:23 *************** *** 7,13 **** import os class _Mailbox: ! def __init__(self, fp): self.fp = fp self.seekp = 0 --- 7,13 ---- import os class _Mailbox: ! _skipfirst = 0 def __init__(self, fp): self.fp = fp self.seekp = 0 *************** *** 29,34 **** --- 29,37 ---- self.seekp = self.fp.tell() return None start = self.fp.tell() + if self._skipfirst: + # Skip the first line when necessary + self.fp.readline() self._search_end() self.seekp = stop = self.fp.tell() if start <> stop: *************** *** 94,106 **** del self.fp class UnixMailbox(_Mailbox): ! def _search_start(self): while 1: line = self.fp.readline() if not line: raise EOFError if line[:5] == 'From ' and self._isrealfromline(line): return def _search_end(self): --- 97,111 ---- del self.fp class UnixMailbox(_Mailbox): ! _skipfirst = 1 def _search_start(self): while 1: + pos = self.fp.tell() line = self.fp.readline() if not line: raise EOFError if line[:5] == 'From ' and self._isrealfromline(line): + self.fp.seek(pos) return def _search_end(self): --LQksG6bCIzRHxTLp-- From guido@python.org Fri Mar 24 21:21:21 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 24 Mar 2000 16:21:21 -0500 Subject: [Patches] Peter Bosch: Re: [Auto-response] Re: Python/Linux audio module. Message-ID: <200003242121.QAA05454@eric.cnri.reston.va.us> Can someone with Linux audio knowledge go over this and see if it works adequately? I suppose the audiodev module should also be adapted to use this module... And perhaps the needed constants could be added to LINUXAUDIODEV.py. --Guido van Rossum (home page: http://www.python.org/~guido/) ------- Forwarded Message Date: Fri, 24 Mar 2000 11:27:00 +0100 From: Peter Bosch To: Guido van Rossum Subject: Re: [Auto-response] Re: Python/Linux audio module. >> http://www.python.org/~guido/help.html Ok: `I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation.' /* Hey Emacs, this is -*-C-*- ****************************************************************************** * linuxaudiodev.c -- Linux audio device for python. * * Author : Peter Bosch * Created On : Thu Mar 2 21:10:33 2000 * Last Modified By: Peter Bosch * Last Modified On: Fri Mar 24 11:27:00 2000 * Status : Unknown, Use with caution! * * Unless other notices are present in any part of this file * explicitly claiming copyrights for other people and/or * organizations, the contents of this file is fully copyright * (C) 2000 Peter Bosch, all rights reserved. ****************************************************************************** */ #include "Python.h" #include "structmember.h" #ifdef HAVE_UNISTD_H #include #endif #ifdef HAVE_FCNTL_H #include #endif #include #include #include typedef unsigned long uint32_t; typedef struct { PyObject_HEAD; int x_fd; /* The open file */ int x_icount; /* Input count */ int x_ocount; /* Output count */ uint32_t x_afmts; /* Supported audio formats */ } lad_t; static struct { int a_bps; uint32_t a_fmt; } audio_types[] = { { 8, AFMT_MU_LAW }, { 8, AFMT_U8 }, { 8, AFMT_S8 }, { 16, AFMT_U16_BE }, { 16, AFMT_U16_LE }, { 16, AFMT_S16_BE }, { 16, AFMT_S16_LE }, }; staticforward PyTypeObject Ladtype; static PyObject *LinuxAudioError; static lad_t * newladobject(PyObject *arg) { lad_t *xp; int fd, afmts, imode; char *mode; char *basedev; char *ctldev; char *opendev; /* Check arg for r/w/rw */ if (!PyArg_ParseTuple(arg, "s", &mode)) return NULL; if (strcmp(mode, "r") == 0) imode = 0; else if (strcmp(mode, "w") == 0) imode = 1; else { PyErr_SetString(LinuxAudioError, "Mode should be one of 'r', or 'w'"); return NULL; } /* Open the correct device. The base device name comes from the * AUDIODEV environment variable first, then /dev/audio. The * control device tacks "ctl" onto the base device name. */ basedev = getenv("AUDIODEV"); if (!basedev) basedev = "/dev/dsp"; if ((fd = open(basedev, imode)) < 0) { PyErr_SetFromErrnoWithFilename(LinuxAudioError, basedev); return NULL; } if (imode) { if (ioctl(fd, SNDCTL_DSP_NONBLOCK, NULL) < 0) { PyErr_SetFromErrnoWithFilename(LinuxAudioError, basedev); return NULL; } } if (ioctl(fd, SNDCTL_DSP_GETFMTS, &afmts) < 0) { PyErr_SetFromErrnoWithFilename(LinuxAudioError, basedev); return NULL; } /* Create and initialize the object */ if ((xp = PyObject_NEW(lad_t, &Ladtype)) == NULL) { close(fd); return NULL; } xp->x_fd = fd; xp->x_icount = xp->x_ocount = 0; xp->x_afmts = afmts; return xp; } static void lad_dealloc(lad_t *xp) { close(xp->x_fd); PyMem_DEL(xp); } static PyObject * lad_read(lad_t *self, PyObject *args) { int size, count; char *cp; PyObject *rv; if (!PyArg_ParseTuple(args, "i", &size)) return NULL; rv = PyString_FromStringAndSize(NULL, size); if (rv == NULL) return NULL; if (!(cp = PyString_AsString(rv))) { Py_DECREF(rv); return NULL; } if ((count = read(self->x_fd, cp, size)) < 0) { PyErr_SetFromErrno(LinuxAudioError); Py_DECREF(rv); return NULL; } self->x_icount += count; return rv; } static PyObject * lad_write(lad_t *self, PyObject *args) { char *cp; int rv, size; if (!PyArg_ParseTuple(args, "s#", &cp, &size)) return NULL; while (size > 0) { if ((rv = write(self->x_fd, cp, size)) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } self->x_ocount += rv; size -= rv; cp += rv; } Py_INCREF(Py_None); return Py_None; } static PyObject * lad_close(lad_t *self, PyObject *args) { if (!PyArg_ParseTuple(args, "")) return NULL; if (self->x_fd >= 0) { close(self->x_fd); self->x_fd = -1; } Py_INCREF(Py_None); return Py_None; } static PyObject * lad_fileno(lad_t *self, PyObject *args) { if (!PyArg_ParseTuple(args, "")) return NULL; return PyInt_FromLong(self->x_fd); } static PyObject * lad_setparameters(lad_t *self, PyObject *args) { int rate, ssize, nchannels, stereo, n, fmt; if (!PyArg_ParseTuple(args, "iiii", &rate, &ssize, &nchannels, &fmt)) return NULL; if (rate < 0 || ssize < 0 || (nchannels != 1 && nchannels != 2)) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } if (ioctl(self->x_fd, SOUND_PCM_WRITE_RATE, &rate) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } if (ioctl(self->x_fd, SNDCTL_DSP_SAMPLESIZE, &ssize) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } stereo = (nchannels == 1)? 0: (nchannels == 2)? 1: -1; if (ioctl(self->x_fd, SNDCTL_DSP_STEREO, &stereo) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } for (n = 0; n != sizeof(audio_types) / sizeof(audio_types[0]); n++) if (fmt == audio_types[n].a_fmt) break; if (n == sizeof(audio_types) / sizeof(audio_types[0]) || audio_types[n].a_bps != ssize || (self->x_afmts & audio_types[n].a_fmt) == 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } if (ioctl(self->x_fd, SNDCTL_DSP_SETFMT, &audio_types[n].a_fmt) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } Py_INCREF(Py_None); return Py_None; } static int _ssize(lad_t *self, int *nchannels, int *ssize) { int fmt; fmt = 0; if (ioctl(self->x_fd, SNDCTL_DSP_SETFMT, &fmt) < 0) return -errno; switch (fmt) { case AFMT_MU_LAW: case AFMT_A_LAW: case AFMT_U8: case AFMT_S8: *ssize = sizeof(char); break; case AFMT_S16_LE: case AFMT_S16_BE: case AFMT_U16_LE: case AFMT_U16_BE: *ssize = sizeof(short); break; case AFMT_MPEG: case AFMT_IMA_ADPCM: default: return -EOPNOTSUPP; } *nchannels = 0; if (ioctl(self->x_fd, SNDCTL_DSP_CHANNELS, nchannels) < 0) return -errno; return 0; } /* bufsize returns the size of the hardware audio buffer in number of samples */ static PyObject * lad_bufsize(lad_t *self, PyObject *args) { audio_buf_info ai; int nchannels, ssize; if (!PyArg_ParseTuple(args, "")) return NULL; if (_ssize(self, &nchannels, &ssize) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } if (ioctl(self->x_fd, SNDCTL_DSP_GETOSPACE, &ai) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } return PyInt_FromLong((ai.fragstotal * ai.fragsize) / (nchannels * ssize)); } /* obufcount returns the number of samples that are available in the hardware for playing */ static PyObject * lad_obufcount(lad_t *self, PyObject *args) { audio_buf_info ai; int nchannels, ssize; if (!PyArg_ParseTuple(args, "")) return NULL; if (_ssize(self, &nchannels, &ssize) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } if (ioctl(self->x_fd, SNDCTL_DSP_GETOSPACE, &ai) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } return PyInt_FromLong((ai.fragstotal * ai.fragsize - ai.bytes) / (ssize * nchannels)); } /* obufcount returns the number of samples that can be played without blocking */ static PyObject * lad_obuffree(lad_t *self, PyObject *args) { audio_buf_info ai; int nchannels, ssize; if (!PyArg_ParseTuple(args, "")) return NULL; if (_ssize(self, &nchannels, &ssize) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } if (ioctl(self->x_fd, SNDCTL_DSP_GETOSPACE, &ai) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } return PyInt_FromLong(ai.bytes / (ssize * nchannels)); } /* Flush the device */ static PyObject * lad_flush(lad_t *self, PyObject *args) { if (!PyArg_ParseTuple(args, "")) return NULL; if (ioctl(self->x_fd, SNDCTL_DSP_SYNC, NULL) < 0) { PyErr_SetFromErrno(LinuxAudioError); return NULL; } Py_INCREF(Py_None); return Py_None; } static PyMethodDef lad_methods[] = { { "read", (PyCFunction)lad_read, 1 }, { "write", (PyCFunction)lad_write, 1 }, { "setparameters", (PyCFunction)lad_setparameters, 1 }, { "bufsize", (PyCFunction)lad_bufsize, 1 }, { "obufcount", (PyCFunction)lad_obufcount, 1 }, { "obuffree", (PyCFunction)lad_obuffree, 1 }, { "flush", (PyCFunction)lad_flush, 1 }, { "close", (PyCFunction)lad_close, 1 }, { "fileno", (PyCFunction)lad_fileno, 1 }, { NULL, NULL} /* sentinel */ }; static PyObject * lad_getattr(lad_t *xp, char *name) { return Py_FindMethod(lad_methods, (PyObject *)xp, name); } static PyTypeObject Ladtype = { PyObject_HEAD_INIT(&PyType_Type) 0, /*ob_size*/ "linux_audio_device", /*tp_name*/ sizeof(lad_t), /*tp_size*/ 0, /*tp_itemsize*/ /* methods */ (destructor)lad_dealloc, /*tp_dealloc*/ 0, /*tp_print*/ (getattrfunc)lad_getattr, /*tp_getattr*/ 0, /*tp_setattr*/ 0, /*tp_compare*/ 0, /*tp_repr*/ }; static PyObject * ladopen(PyObject *self, PyObject *args) { return (PyObject *)newladobject(args); } static PyMethodDef linuxaudiodev_methods[] = { { "open", ladopen, 1 }, { 0, 0 }, }; static int ins(PyObject *d, char *symbol, long value) { PyObject* v = PyInt_FromLong(value); if (!v || PyDict_SetItemString(d, symbol, v) < 0) return -1; /* triggers fatal error */ Py_DECREF(v); return 0; } void initlinuxaudiodev() { PyObject *m, *d, *x; m = Py_InitModule("linuxaudiodev", linuxaudiodev_methods); d = PyModule_GetDict(m); LinuxAudioError = PyErr_NewException("linuxaudiodev.error", NULL, NULL); if (LinuxAudioError) PyDict_SetItemString(d, "error", LinuxAudioError); x = PyInt_FromLong((long) AFMT_MU_LAW); if (x == NULL || PyDict_SetItemString(d, "AFMT_MU_LAW", x) < 0) goto error; Py_DECREF(x); x = PyInt_FromLong((long) AFMT_U8); if (x == NULL || PyDict_SetItemString(d, "AFMT_U8", x) < 0) goto error; Py_DECREF(x); x = PyInt_FromLong((long) AFMT_S8); if (x == NULL || PyDict_SetItemString(d, "AFMT_S8", x) < 0) goto error; Py_DECREF(x); x = PyInt_FromLong((long) AFMT_U16_BE); if (x == NULL || PyDict_SetItemString(d, "AFMT_U16_BE", x) < 0) goto error; Py_DECREF(x); x = PyInt_FromLong((long) AFMT_U16_LE); if (x == NULL || PyDict_SetItemString(d, "AFMT_U16_LE", x) < 0) goto error; Py_DECREF(x); x = PyInt_FromLong((long) AFMT_S16_BE); if (x == NULL || PyDict_SetItemString(d, "AFMT_S16_BE", x) < 0) goto error; Py_DECREF(x); x = PyInt_FromLong((long) AFMT_S16_LE); if (x == NULL || PyDict_SetItemString(d, "AFMT_S16_LE", x) < 0) goto error; Py_DECREF(x); /* Check for errors */ if (PyErr_Occurred()) { error: Py_FatalError("can't initialize module linuxaudiodev"); } } ------- End of Forwarded Message From mal@lemburg.com Fri Mar 24 21:40:48 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 24 Mar 2000 22:40:48 +0100 Subject: [Patches] Unicode Patch Set 2000-03-24 Message-ID: <38DBE0E0.76A298FE@lemburg.com> This is a multi-part message in MIME format. --------------16C56446D7F83349DECA84A2 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Attached you find the latest update of the Unicode implementation. The patch is against the current CVS version. It includes the fix I posted yesterday for the core dump problem in codecs.c (was introduced by my previous patch set -- sorry), adds more tests for the codecs and two new parser markers "es" and "es#". -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ --------------16C56446D7F83349DECA84A2 Content-Type: text/plain; charset=us-ascii; name="Unicode-Implementation-2000-03-24.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="Unicode-Implementation-2000-03-24.patch" Only in CVS-Python/Doc/tools: anno-api.py diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/codecs.py Python+Unicode/Lib/codecs.py --- CVS-Python/Lib/codecs.py Thu Mar 23 23:58:41 2000 +++ Python+Unicode/Lib/codecs.py Fri Mar 17 23:51:01 2000 @@ -46,7 +46,7 @@ handling schemes by providing the errors argument. These string values are defined: - 'strict' - raise an error (or a subclass) + 'strict' - raise a ValueError error (or a subclass) 'ignore' - ignore the character and continue with the next 'replace' - replace with a suitable replacement character; Python will use the official U+FFFD REPLACEMENT diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/test/output/test_unicode Python+Unicode/Lib/test/output/test_unicode --- CVS-Python/Lib/test/output/test_unicode Fri Mar 24 22:21:26 2000 +++ Python+Unicode/Lib/test/output/test_unicode Sat Mar 11 00:23:21 2000 @@ -1,5 +1,4 @@ test_unicode Testing Unicode comparisons... done. -Testing Unicode contains method... done. Testing Unicode formatting strings... done. Testing unicodedata module... done. diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/test/test_unicode.py Python+Unicode/Lib/test/test_unicode.py --- CVS-Python/Lib/test/test_unicode.py Thu Mar 23 23:58:47 2000 +++ Python+Unicode/Lib/test/test_unicode.py Fri Mar 24 00:29:43 2000 @@ -293,3 +293,33 @@ assert unicodedata.combining(u'\u20e1') == 230 print 'done.' + +# Test builtin codecs +print 'Testing builtin codecs...', + +assert unicode('hello','ascii') == u'hello' +assert unicode('hello','utf-8') == u'hello' +assert unicode('hello','utf8') == u'hello' +assert unicode('hello','latin-1') == u'hello' + +assert u'hello'.encode('ascii') == 'hello' +assert u'hello'.encode('utf-8') == 'hello' +assert u'hello'.encode('utf8') == 'hello' +assert u'hello'.encode('utf-16-le') == 'h\000e\000l\000l\000o\000' +assert u'hello'.encode('utf-16-be') == '\000h\000e\000l\000l\000o' +assert u'hello'.encode('latin-1') == 'hello' + +u = u''.join(map(unichr, range(1024))) +for encoding in ('utf-8', 'utf-16', 'utf-16-le', 'utf-16-be', + 'raw_unicode_escape', 'unicode_escape', 'unicode_internal'): + assert unicode(u.encode(encoding),encoding) == u + +u = u''.join(map(unichr, range(256))) +for encoding in ('latin-1',): + assert unicode(u.encode(encoding),encoding) == u + +u = u''.join(map(unichr, range(128))) +for encoding in ('ascii',): + assert unicode(u.encode(encoding),encoding) == u + +print 'done.' diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Misc/unicode.txt Python+Unicode/Misc/unicode.txt --- CVS-Python/Misc/unicode.txt Thu Mar 23 23:58:48 2000 +++ Python+Unicode/Misc/unicode.txt Fri Mar 24 22:29:35 2000 @@ -715,21 +715,126 @@ These markers are used by the PyArg_ParseTuple() APIs: - 'U': Check for Unicode object and return a pointer to it + "U": Check for Unicode object and return a pointer to it - 's': For Unicode objects: auto convert them to the + "s": For Unicode objects: auto convert them to the and return a pointer to the object's buffer. - 's#': Access to the Unicode object via the bf_getreadbuf buffer interface + "s#": Access to the Unicode object via the bf_getreadbuf buffer interface (see Buffer Interface); note that the length relates to the buffer length, not the Unicode string length (this may be different depending on the Internal Format). - 't#': Access to the Unicode object via the bf_getcharbuf buffer interface + "t#": Access to the Unicode object via the bf_getcharbuf buffer interface (see Buffer Interface); note that the length relates to the buffer length, not necessarily to the Unicode string length (this may be different depending on the ). + "es": + Takes two parameters: encoding (const char *) and + buffer (char **). + + The input object is first coerced to Unicode in the usual way + and then encoded into a string using the given encoding. + + On output, a buffer of the needed size is allocated and + returned through *buffer as NULL-terminated string. + The encoded may not contain embedded NULL characters. + The caller is responsible for free()ing the allocated *buffer + after usage. + + "es#": + Takes three parameters: encoding (const char *), + buffer (char **) and buffer_len (int *). + + The input object is first coerced to Unicode in the usual way + and then encoded into a string using the given encoding. + + If *buffer is non-NULL, *buffer_len must be set to sizeof(buffer) + on input. Output is then copied to *buffer. + + If *buffer is NULL, a buffer of the needed size is + allocated and output copied into it. *buffer is then + updated to point to the allocated memory area. The caller + is responsible for free()ing *buffer after usage. + + In both cases *buffer_len is updated to the number of + characters written (excluding the trailing NULL-byte). + The output buffer is assured to be NULL-terminated. + +Examples: + +Using "es#" with auto-allocation: + + static PyObject * + test_parser(PyObject *self, + PyObject *args) + { + PyObject *str; + const char *encoding = "latin-1"; + char *buffer = NULL; + int buffer_len = 0; + + if (!PyArg_ParseTuple(args, "es#:test_parser", + encoding, &buffer, &buffer_len)) + return NULL; + if (!buffer) { + PyErr_SetString(PyExc_SystemError, + "buffer is NULL"); + return NULL; + } + str = PyString_FromStringAndSize(buffer, buffer_len); + free(buffer); + return str; + } + +Using "es" with auto-allocation returning a NULL-terminated string: + + static PyObject * + test_parser(PyObject *self, + PyObject *args) + { + PyObject *str; + const char *encoding = "latin-1"; + char *buffer = NULL; + + if (!PyArg_ParseTuple(args, "es:test_parser", + encoding, &buffer)) + return NULL; + if (!buffer) { + PyErr_SetString(PyExc_SystemError, + "buffer is NULL"); + return NULL; + } + str = PyString_FromString(buffer); + free(buffer); + return str; + } + +Using "es#" with a pre-allocated buffer: + + static PyObject * + test_parser(PyObject *self, + PyObject *args) + { + PyObject *str; + const char *encoding = "latin-1"; + char _buffer[10]; + char *buffer = _buffer; + int buffer_len = sizeof(_buffer); + + if (!PyArg_ParseTuple(args, "es#:test_parser", + encoding, &buffer, &buffer_len)) + return NULL; + if (!buffer) { + PyErr_SetString(PyExc_SystemError, + "buffer is NULL"); + return NULL; + } + str = PyString_FromStringAndSize(buffer, buffer_len); + return str; + } + File/Stream Output: ------------------- @@ -837,6 +942,7 @@ History of this Proposal: ------------------------- +1.3: Added new "es" and "es#" parser markers 1.2: Removed POD about codecs.open() 1.1: Added note about comparisons and hash values. Added note about case mapping algorithms. Changed stream codecs .read() and Only in CVS-Python/Objects: .#stringobject.c.2.59 Only in CVS-Python/Objects: stringobject.c.orig diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Python/getargs.c Python+Unicode/Python/getargs.c --- CVS-Python/Python/getargs.c Sat Mar 11 10:55:21 2000 +++ Python+Unicode/Python/getargs.c Fri Mar 24 20:22:26 2000 @@ -178,6 +178,8 @@ } else if (level != 0) ; /* Pass */ + else if (c == 'e') + ; /* Pass */ else if (isalpha(c)) max++; else if (c == '|') @@ -654,6 +656,122 @@ break; } + case 'e': /* encoded string */ + { + char **buffer; + const char *encoding; + PyObject *u, *s; + int size; + + /* Get 'e' parameter: the encoding name */ + encoding = (const char *)va_arg(*p_va, const char *); + if (encoding == NULL) + return "(encoding is NULL)"; + + /* Get 's' parameter: the output buffer to use */ + if (*format != 's') + return "(unkown parser marker combination)"; + buffer = (char **)va_arg(*p_va, char **); + format++; + if (buffer == NULL) + return "(buffer is NULL)"; + + /* Convert object to Unicode */ + u = PyUnicode_FromObject(arg); + if (u == NULL) + return "string, unicode or text buffer"; + + /* Encode object; use default error handling */ + s = PyUnicode_AsEncodedString(u, + encoding, + NULL); + Py_DECREF(u); + if (s == NULL) + return "(encoding failed)"; + if (!PyString_Check(s)) { + Py_DECREF(s); + return "(encoder failed to return a string)"; + } + size = PyString_GET_SIZE(s); + + /* Write output; output is guaranteed to be + 0-terminated */ + if (*format == '#') { + /* Using buffer length parameter '#': + + - if *buffer is NULL, a new buffer + of the needed size is allocated and + the data copied into it; *buffer is + updated to point to the new buffer; + the caller is responsible for + free()ing it after usage + + - if *buffer is not NULL, the data + is copied to *buffer; *buffer_len + has to be set to the size of the + buffer on input; buffer overflow is + signalled with an error; buffer has + to provide enough room for the + encoded string plus the trailing + 0-byte + + - in both cases, *buffer_len is + updated to the size of the buffer + /excluding/ the trailing 0-byte + + */ + int *buffer_len = va_arg(*p_va, int *); + + format++; + if (buffer_len == NULL) + return "(buffer_len is NULL)"; + if (*buffer == NULL) { + *buffer = PyMem_NEW(char, size + 1); + if (*buffer == NULL) { + Py_DECREF(s); + return "(memory error)"; + } + } else { + if (size + 1 > *buffer_len) { + Py_DECREF(s); + return "(buffer overflow)"; + } + } + memcpy(*buffer, + PyString_AS_STRING(s), + size + 1); + *buffer_len = size; + } else { + /* Using a 0-terminated buffer: + + - the encoded string has to be + 0-terminated for this variant to + work; if it is not, an error raised + + - a new buffer of the needed size + is allocated and the data copied + into it; *buffer is updated to + point to the new buffer; the caller + is responsible for free()ing it + after usage + + */ + if (strlen(PyString_AS_STRING(s)) != size) + return "(encoded string without "\ + "NULL bytes)"; + *buffer = PyMem_NEW(char, size + 1); + if (*buffer == NULL) { + Py_DECREF(s); + return "(memory error)"; + } + memcpy(*buffer, + PyString_AS_STRING(s), + size + 1); + } + Py_DECREF(s); + break; + } + case 'S': /* string object */ { PyObject **p = va_arg(*p_va, PyObject **); --------------16C56446D7F83349DECA84A2-- From skip@mojam.com (Skip Montanaro) Fri Mar 24 22:12:59 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Fri, 24 Mar 2000 16:12:59 -0600 Subject: [Patches] updates to robotparser.py Message-ID: <200003242212.QAA04828@beluga.mojam.com> The robotparser.py module currently lives in Tools/webchecker. In preparation for its migration to Lib, I made the following changes: * renamed the test() function _test * corrected the URLs in _test() so they refer to actual documents * added an "if __name__ == '__main__'" catcher to invoke _test() when run as a main program * added doc strings for the two main methods, parse and can_fetch * replaced usage of regsub and regex with corresponding re code Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Skip Montanaro | http://www.mojam.com/ skip@mojam.com | http://www.musi-cal.com/ *** /tmp/robotparser.py.~1.2~ Fri Mar 24 16:06:54 2000 --- /tmp/robotparser.py Fri Mar 24 16:06:54 2000 *************** *** 23,37 **** def set_url(self, url): self.url = url - ## import urlmisc - ## self.url = urlmisc.canonical_url(url) def read(self): import urllib self.parse(urllib.urlopen(self.url).readlines()) def parse(self, lines): ! import regsub, string, regex active = [] for line in lines: if self.debug: print '>', line, --- 23,36 ---- def set_url(self, url): self.url = url def read(self): import urllib self.parse(urllib.urlopen(self.url).readlines()) def parse(self, lines): ! """parse the input lines from a robot.txt file""" ! import string, re active = [] for line in lines: if self.debug: print '>', line, *************** *** 43,49 **** line = string.strip(line[:string.find(line, '#')]) if not line: continue ! line = regsub.split(line, ' *: *') if len(line) == 2: line[0] = string.lower(line[0]) if line[0] == 'user-agent': --- 42,48 ---- line = string.strip(line[:string.find(line, '#')]) if not line: continue ! line = re.split(' *: *', line) if len(line) == 2: line[0] = string.lower(line[0]) if line[0] == 'user-agent': *************** *** 56,62 **** if line[1]: if self.debug: print '>> disallow:', line[1] for agent in active: ! self.rules[agent].append(regex.compile(line[1])) else: pass for agent in active: --- 55,61 ---- if line[1]: if self.debug: print '>> disallow:', line[1] for agent in active: ! self.rules[agent].append(re.compile(line[1])) else: pass for agent in active: *************** *** 68,97 **** self.modified() # returns true if agent is allowed to fetch url ! def can_fetch(self, agent, url): import urlparse ! ag = agent if not self.rules.has_key(ag): ag = '*' if not self.rules.has_key(ag): ! if self.debug: print '>> allowing', url, 'fetch by', agent return 1 path = urlparse.urlparse(url)[2] for rule in self.rules[ag]: ! if rule.match(path) != -1: ! if self.debug: print '>> disallowing', url, 'fetch by', agent return 0 ! if self.debug: print '>> allowing', url, 'fetch by', agent return 1 ! def test(): rp = RobotFileParser() rp.debug = 1 ! rp.set_url('http://www.automatrix.com/robots.txt') rp.read() print rp.rules ! print rp.can_fetch('*', 'http://www.calendar.com/concerts/') print rp.can_fetch('Musi-Cal-Robot', ! 'http://dolphin:80/cgi-bin/music-search?performer=Rolling+Stones') ! print rp.can_fetch('Lycos', 'http://www/~skip/volkswagen/') ! print rp.can_fetch('Lycos', 'http://www/~skip/volkswagen/vanagon-list-001') --- 67,97 ---- self.modified() # returns true if agent is allowed to fetch url ! def can_fetch(self, useragent, url): ! """using the parsed robots.txt decide if useragent can fetch url""" import urlparse ! ag = useragent if not self.rules.has_key(ag): ag = '*' if not self.rules.has_key(ag): ! if self.debug: print '>> allowing', url, 'fetch by', useragent return 1 path = urlparse.urlparse(url)[2] for rule in self.rules[ag]: ! if rule.match(path) is not None: ! if self.debug: print '>> disallowing', url, 'fetch by', useragent return 0 ! if self.debug: print '>> allowing', url, 'fetch by', useragent return 1 ! def _test(): rp = RobotFileParser() rp.debug = 1 ! rp.set_url('http://www.musi-cal.com/robots.txt') rp.read() print rp.rules ! print rp.can_fetch('*', 'http://www.musi-cal.com.com/') print rp.can_fetch('Musi-Cal-Robot', ! 'http://www.musi-cal.com/cgi-bin/event-search?city=San+Francisco') ! if __name__ == "__main__": ! _test() From bugs5@birdmail.com Sat Mar 25 03:28:33 2000 From: bugs5@birdmail.com (bugs5@birdmail.com) Date: Sat, 25 Mar 2000 03:28:33 Subject: [Patches] Email Advertising Special--Ends Friday Message-ID: PUT EMAIL MARKETING TO WORK FOR YOU... Call NOW and receive 50,000 FREE emails with your order! WE HAVE OPT-IN LISTS!!!! see below for removal. Special Ends Friday March 31, 2000 MLM'ers, We can build your downline. Imagine having a product or idea and selling it for only $10. Now imagine sending an ad for your product or idea to 25 million people! If you only get a 1/10 of 1% response you have just made $250,000!! You hear about people getting rich off the Internet everyday on TV, now is the perfect time for you to jump in on all the action. FACT. With the introduction of the Internet, one primary KEY to conducting your business successfully is creating massive exposure in a cost effective manner. FACT. The experts agree that email marketing is one of the most cost effective forms of promotion in existence today. Electronic mail has overtaken the telephone as the primary means of business communication.(American Management Association) Of online users 41 percent check their email daily. "A gold mine for those who can take advantage of bulk email programs"- The New York Times "Email is an incredible lead generation tool" -Crains Magazine "Blows away traditional Mailing"-Advertising Age "It's truly arrived. Email is the killer app so far in the online world"-Kate Delhagen, Forrester Research Analyst Why not let a professional company handle your direct email marketing efforts for you? *We will assist you in developing your entire campaign! *We can even create your ad or annoucement for you! *No responses? We resend at no cost! For More Information CALL NOW-702-248-1043 For removal see below. SPECIAL RATES SPECIAL ENDS Friday March 31, 2000 Targeted Rates Upon Request. BONUS!!! Call In and receive 50,000 Extra Emails at No Cost! Call NOW - 702-248-1043 ++++++++++++++++++++++++++++++++++++++++++++++++++ We are terribly sorry if you received this message in error. If you wish to be removed. Please, type "REMOVE" in the subject line: outnow@fiberia.com ++++++++++++++++++++++++++++++++++++++++++++++++++ From tismer@tismer.com Sat Mar 25 14:51:13 2000 From: tismer@tismer.com (Christian Tismer) Date: Sat, 25 Mar 2000 15:51:13 +0100 Subject: [Patches] Security Patch for Trashcan Message-ID: <38DCD261.DB9E3EAD@tismer.com> This is a patch to my recent patch concerning destruction of deeply nested objects. Reason: There are rare cases where the thread state is undefined while destructions can occour. This seems to happen in the finalization phase only, but there are calls to external finalizers like the LLNL extensions where we cannot be sure if the error fetch/restore *might* happen with undefined thread state. Checkin-message: Added "better safe than sorry" patch to the new trashcan code in object.c, to ensure that tstate is not touched when it might be undefined. Diff: *** //d/cvsroot/python/dist/src/objects/object.c Mon Mar 13 19:00:26 2000 --- sp/objects/object.c Sat Mar 25 16:37:50 2000 *************** *** 920,925 **** --- 920,928 ---- CT 2k0309 modified to restore a possible error. + + CT 2k0325 + added better safe than sorry check for threadstate */ int _PyTrash_delete_nesting = 0; *************** *** 930,943 **** PyObject *op; { PyObject *error_type, *error_value, *error_traceback; ! PyErr_Fetch(&error_type, &error_value, &error_traceback); if (!_PyTrash_delete_later) _PyTrash_delete_later = PyList_New(0); if (_PyTrash_delete_later) PyList_Append(_PyTrash_delete_later, (PyObject *)op); ! PyErr_Restore(error_type, error_value, error_traceback); } void --- 933,949 ---- PyObject *op; { PyObject *error_type, *error_value, *error_traceback; ! ! if (PyThreadState_GET() != NULL) ! PyErr_Fetch(&error_type, &error_value, &error_traceback); if (!_PyTrash_delete_later) _PyTrash_delete_later = PyList_New(0); if (_PyTrash_delete_later) PyList_Append(_PyTrash_delete_later, (PyObject *)op); ! if (PyThreadState_GET() != NULL) ! PyErr_Restore(error_type, error_value, error_traceback); } void %%%%%%%%%%%%%%%%%%%%%%% end patch %%%%%%%%%%%%%%%%%%%%%%%%%% Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. -- Christian Tismer :^) Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaunstr. 26 : *Starship* http://starship.python.net 14163 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home _______________________________________________ Patches mailing list Patches@python.org http://www.python.org/mailman/listinfo/patches _______________________________________________ Patches mailing list Patches@python.org http://www.python.org/mailman/listinfo/patches From dan@cgsoftware.com Sat Mar 25 16:33:59 2000 From: dan@cgsoftware.com (Daniel Berlin+list.python-patches) Date: 25 Mar 2000 11:33:59 -0500 Subject: [Patches] Email Advertising Special--Ends Friday In-Reply-To: 's message of "Sat, 25 Mar 2000 03:28:33" References: Message-ID: writes: Well, as good as this patch is (it would be great to have it in python), I can't get it to apply cleanly to the current python sources. Somebody should clean it up, and modularize it. --Dan From martin@loewis.home.cs.tu-berlin.de Sat Mar 25 18:27:36 2000 From: martin@loewis.home.cs.tu-berlin.de (Martin v. Loewis) Date: Sat, 25 Mar 2000 19:27:36 +0100 Subject: [Patches] Using Tcl_Obj in _tkinter Message-ID: <200003251827.TAA25283@loewis.home.cs.tu-berlin.de> With the patch included below, _tkinter uses the Tcl_Obj API for command invocations. This means a more direct exchange of data between Tcl and Python; I have not yet measured how that impacts performance of _tkinter. Currently, it uses the object interface only for TkApp_Call; other potential candidates would be global_call, and the setvar family. For compatibility with Tkinter applications, it never attempts to use the object interface when returning values to Python - in these cases, strings are produced as before. According to Tcl's changes document, this patch should work for Tcl versions starting with 8.0; it has been tested with Tcl 8.3 on Linux. The code is disabled for earlier versions. In addition, it also calls Tcl_FindExecutable, which is documented as being available since Tcl 8.1. This is necessary so Tcl finds its library, in particular its character set conversion tables. The patch is essentially the same that has been send to string-sig before, the only change is that it won't use Tcl_NewUnicodeString, anymore. Please let me know if you find any problems with that code. Regards, Martin Index: _tkinter.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Modules/_tkinter.c,v retrieving revision 1.90 diff -u -p -r1.90 _tkinter.c --- _tkinter.c 2000/02/29 13:59:22 1.90 +++ _tkinter.c 2000/03/25 18:00:25 @@ -130,6 +130,11 @@ PERFORMANCE OF THIS SOFTWARE. #define WAIT_FOR_STDIN #endif +/* If Tcl provides the Tcl_Obj type, we use that interface. */ +#if TKMAJORMINOR >= 8000 +#define USING_OBJECTS +#endif + #ifdef WITH_THREAD /* The threading situation is complicated. Tcl is not thread-safe, except for @@ -441,6 +446,44 @@ Split(list) return v; } +#ifdef USING_OBJECTS + +static Tcl_Obj* +AsObj(value) + PyObject *value; +{ + Tcl_Obj *result; + if (PyUnicode_Check(value)) { + PyObject* utf8 = PyUnicode_AsUTF8String (value); + if (!utf8) + return 0; + return Tcl_NewStringObj (PyString_AS_STRING (utf8), + PyString_GET_SIZE (utf8)); + } + else if (PyString_Check(value)) + return Tcl_NewStringObj(PyString_AS_STRING(value), + PyString_GET_SIZE(value)); + else if (PyTuple_Check(value)) { + Tcl_Obj **argv = (Tcl_Obj**)ckalloc(PyTuple_Size(value)*sizeof(Tcl_Obj*)); + int i; + if(!argv) + return 0; + for(i=0;i ARGSZ) { + objv = (Tcl_Obj **)ckalloc(objc * sizeof(char *)); + if (objv == NULL) { + PyErr_NoMemory(); + goto finally; + } + } + + for (i = 0; i < objc; i++) { + PyObject *v = PyTuple_GetItem(args, i); + objv[i] = AsObj(v); + if (!objv[i]) + goto finally; + Tcl_IncrRefCount(objv[i]); + } + } + + ENTER_TCL + + i = Tcl_EvalObjv(interp, objc, objv, flags); + + ENTER_OVERLAP + if (i == TCL_ERROR) + Tkinter_Error(self); + else + /* We could request the object result here, but doing + so would confuse applications that expect a string. */ + res = PyString_FromString(Tcl_GetStringResult(interp)); + + LEAVE_OVERLAP_TCL + + finally: + for (i = 0; i < objc; i++) + Tcl_DecrRefCount(objv[i]); + if (objv != objStore) + ckfree(FREECAST objv); + return res; +} + +#else /* !USING_OBJECTS */ + static PyObject * Tkapp_Call(self, args) PyObject *self; @@ -650,8 +763,8 @@ Tkapp_Call(self, args) Py_DECREF(tmp); return res; } +#endif /* USING_OBJECTS */ - static PyObject * Tkapp_GlobalCall(self, args) PyObject *self; @@ -2044,6 +2157,16 @@ init_tkinter() Tktt_Type.ob_type = &PyType_Type; PyDict_SetItemString(d, "TkttType", (PyObject *)&Tktt_Type); + +#if TKMAJORMINOR >= 8001 + /* Starting with Tcl 8.1, the library initialisation won't + work correctly if it is not told a guess of what the + executable name is. In particular, it won't find its + encodings. It will ignore the name, and then proceed with + the compiled-in defaults. So passing ProgramName should be + ok. */ + Tcl_FindExecutable(Py_GetProgramName()); +#endif if (PyErr_Occurred()) return; From guido@python.org Mon Mar 27 19:22:54 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 27 Mar 2000 14:22:54 -0500 Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: Your message of "Fri, 10 Mar 2000 11:57:26 CST." <200003101757.LAA24297@beluga.mojam.com> References: <200003101757.LAA24297@beluga.mojam.com> Message-ID: <200003271922.OAA27844@eric.cnri.reston.va.us> > In httplib.HTTP.connect a socket.error is raised when a non-numeric port is > given. Seems to me this is more correctly a ValueError. Hmm... It explicitly catches the ValueError and raises socket.error. Why could it be doing this? (Also, it does the same thing elsewhere in the module.) -0 --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@mojam.com (Skip Montanaro) Mon Mar 27 19:46:53 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 27 Mar 2000 13:46:53 -0600 (CST) Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: <200003271922.OAA27844@eric.cnri.reston.va.us> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> Message-ID: <14559.47789.120873.716707@beluga.mojam.com> Skip> In httplib.HTTP.connect a socket.error is raised when a Skip> non-numeric port is given. Seems to me this is more correctly a Skip> ValueError. Guido> Hmm... It explicitly catches the ValueError and raises Guido> socket.error. Why could it be doing this? (Also, it does the Guido> same thing elsewhere in the module.) Not sure. The more I see of one module raising other modules' errors, the more I don't like it since it forces the programmer to be aware of implementation details of the module being called. When a module catches an exception I see three valid options: 1. Recover and continue executing 2. Embellish the value of the exception (perhaps adding a concrete filename or other value), then reraise the same exception. 3. Raise a new exception defined in this module. Both #2 & #3 should invoke raise with the original stack info so as not to confuse programmers trying to debug their mistakes. Skip From guido@python.org Mon Mar 27 20:48:52 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 27 Mar 2000 15:48:52 -0500 Subject: [Patches] mailbox.UnixMailbox and rfc822.Message.unixfrom In-Reply-To: Your message of "Fri, 24 Mar 2000 21:41:24 +0100." <20000324214124.B12922@xs4all.nl> References: <20000324214124.B12922@xs4all.nl> Message-ID: <200003272048.PAA19149@eric.cnri.reston.va.us> Thomas, The unixfrom feature of the rfc822 module is at best questionable -- it it not part of the mail headers as defined by RFC-822! Why do you need access the Unix from header through the mailbox module? Your patch worries me: the dependency between _search_start() and next() through the _skipfirst variable begs for problems in subclasses. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido@python.org Mon Mar 27 21:42:23 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 27 Mar 2000 16:42:23 -0500 Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: Your message of "Mon, 27 Mar 2000 13:46:53 CST." <14559.47789.120873.716707@beluga.mojam.com> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> Message-ID: <200003272142.QAA25914@eric.cnri.reston.va.us> > Skip> In httplib.HTTP.connect a socket.error is raised when a > Skip> non-numeric port is given. Seems to me this is more correctly a > Skip> ValueError. > > Guido> Hmm... It explicitly catches the ValueError and raises > Guido> socket.error. Why could it be doing this? (Also, it does the > Guido> same thing elsewhere in the module.) [Skip] > Not sure. The more I see of one module raising other modules' errors, the > more I don't like it since it forces the programmer to be aware of > implementation details of the module being called. > > When a module catches an exception I see three valid options: > > 1. Recover and continue executing > > 2. Embellish the value of the exception (perhaps adding a concrete > filename or other value), then reraise the same exception. > > 3. Raise a new exception defined in this module. > > Both #2 & #3 should invoke raise with the original stack info so as not to > confuse programmers trying to debug their mistakes. I'm wondering, what mistake were you debugging? There must be an actual anecdote associated with your patch. Not necessarily. Looking at the code it is actually catching ValueError and raising socket.error. It appears that it wants to be helpful by *hiding* the details of the exception: all that matters is that the port is not numeric. That this was caught by string.atoi() is not relevant. The assumption is also that all errors caused by a bad "host:port" error will raise socket.error, whether the host lookup fails or the port is invalid or whatever. (An unsupported port number is just as much a "value" error, but it raises a socket error: Connection refused.) I would support adding the actual value of the port string to the error message. I don't support raising ValueError. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip@mojam.com (Skip Montanaro) Mon Mar 27 22:09:01 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Mon, 27 Mar 2000 16:09:01 -0600 (CST) Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: <200003272142.QAA25914@eric.cnri.reston.va.us> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> <200003272142.QAA25914@eric.cnri.reston.va.us> Message-ID: <14559.56317.522468.658667@beluga.mojam.com> Guido> I'm wondering, what mistake were you debugging? There must be an Guido> actual anecdote associated with your patch. Actually, I noticed it while doing a big paste into my own code. I wanted to use the timeout_socket module with httplib.HTTP. Unfortunately, there's no clean way to do that. If you would like me to modify httplib to make it easier to plug in alternatives to the socket module, I'll take a crack at it. Guido> I would support adding the actual value of the port string to the Guido> error message. I don't support raising ValueError. That's fine (it's an incorrect value, so in my mind is a ValueError, but I won't quibble), but I still maintain that if you're not going to raise ValueError, httplib should raise its own exception (httplib.error or some such), not socket.error. In particular, at the point in coonect() at which the current implementation raises socket.error, no socket module calls have even been made. (Same for the hasattr function later that I originally missed.) The os module went through a similary change a couple years ago. If I remember correctly, the os module originally did nothing to define its own exceptions, so programmers had to catch posix.error on Unix systems and something different on other platforms, sort of nixing much of the os module's platform independence. Then you modified it so that the os module set its own error object to refer to the underlying module's error object. That allowed people to catch os.error and not worry about the underlying implementation. Now I see you've gone one step further and defined an OSError standard exception. As a compromise, I propose the following change near the top of httplib somewhere after socket has been imported: error = socket.error Then anywhere httplib raises an error, that should be what it raises. No code that currently catches socket.error will break, and httplib.error can be the documented exception used for transmitting errors out of that module. Skip From guido@python.org Mon Mar 27 22:16:06 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 27 Mar 2000 17:16:06 -0500 Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: Your message of "Mon, 27 Mar 2000 16:09:01 CST." <14559.56317.522468.658667@beluga.mojam.com> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> <200003272142.QAA25914@eric.cnri.reston.va.us> <14559.56317.522468.658667@beluga.mojam.com> Message-ID: <200003272216.RAA28884@eric.cnri.reston.va.us> > Guido> I'm wondering, what mistake were you debugging? There must be an > Guido> actual anecdote associated with your patch. > > Actually, I noticed it while doing a big paste into my own code. I wanted > to use the timeout_socket module with httplib.HTTP. Unfortunately, there's > no clean way to do that. If you would like me to modify httplib to make it > easier to plug in alternatives to the socket module, I'll take a crack at > it. Hm... it seems that there should be an architecture for providing alternatives to the socket module. I've never even heard of the timeout_socket module... Hacking httplib seems the wrong place to approach this. What are the requirements? > Guido> I would support adding the actual value of the port string to the > Guido> error message. I don't support raising ValueError. > > That's fine (it's an incorrect value, so in my mind is a ValueError, but I > won't quibble), but I still maintain that if you're not going to raise > ValueError, httplib should raise its own exception (httplib.error or some > such), not socket.error. In particular, at the point in coonect() at which > the current implementation raises socket.error, no socket module calls have > even been made. (Same for the hasattr function later that I originally > missed.) > > The os module went through a similary change a couple years ago. If I > remember correctly, the os module originally did nothing to define its own > exceptions, so programmers had to catch posix.error on Unix systems and > something different on other platforms, sort of nixing much of the os > module's platform independence. Then you modified it so that the os module > set its own error object to refer to the underlying module's error object. > That allowed people to catch os.error and not worry about the underlying > implementation. Now I see you've gone one step further and defined an > OSError standard exception. > > As a compromise, I propose the following change near the top of httplib > somewhere after socket has been imported: > > error = socket.error > > Then anywhere httplib raises an error, that should be what it raises. No > code that currently catches socket.error will break, and httplib.error can > be the documented exception used for transmitting errors out of that module. And all this because you believe that raising some other module's error is a sin? I don't have a problem with such behavior -- what's the reason? --Guido van Rossum (home page: http://www.python.org/~guido/) From mhammond@skippinet.com.au Tue Mar 28 01:40:27 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Tue, 28 Mar 2000 11:40:27 +1000 Subject: [Patches] Unicode changes. Message-ID: This is a multi-part message in MIME format. ------=_NextPart_000_0039_01BF98AA.69D0B110 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Ive made a few changes to the Unicode stuff, with MA's blessing (assuming "great!" is a blessing :-) getargs.c - Fix typo in error message and silence signed/unsigned warning on Windows. diff -r2.27 getargs.c 673c673 < return "(unkown parser marker combination)"; --- > return "(unknown parser marker combination)"; 759c759 < if (strlen(PyString_AS_STRING(s)) != size) --- > if ((int)strlen(PyString_AS_STRING(s)) != size) unicodeobject.c - Added MBCS codecs. In attached unicodeobject.c.dif unicodeobject.h - Prototypes for MBCS codecs. In attached unicodeobject.h.dif mbcs.py - New file for "lib\encodings" - Description: MBCS codecs for Windows. Once these have been accepted I will submit the winreg registry module, which uses these encodings. I will also submit a patch so open() also takes advantage of these on Windows. Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Mark. ------=_NextPart_000_0039_01BF98AA.69D0B110 Content-Type: application/octet-stream; name="unicodeobject.c.dif" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="unicodeobject.c.dif" diff -c -r2.3 unicodeobject.c *** unicodeobject.c 2000/03/20 16:36:44 2.3 --- unicodeobject.c 2000/03/28 01:23:50 *************** *** 73,78 **** --- 73,81 ---- #define INT_MAX 2147483647 #endif =20 + #ifdef MS_WIN32 + #include + #endif /* Limit for the Unicode object free list */ =20 #define MAX_UNICODE_FREELIST_SIZE 1024 *************** *** 1478,1483 **** --- 1481,1542 ---- PyUnicode_GET_SIZE(unicode), NULL); } +=20 + #ifdef MS_WIN32 + /* --- MBCS codecs for Windows = -------------------------------------------- */ + PyObject *PyUnicode_DecodeMBCS(const char *s, + int size, + const char *errors) + { + PyUnicodeObject *v; + Py_UNICODE *p; +=20 + /* First get the size of the result */ + DWORD usize =3D MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0); + if (usize=3D=3D0) + return PyErr_SetFromWindowsErrWithFilename(0, NULL); +=20 + v =3D _PyUnicode_New(usize); + if (v =3D=3D NULL) + return NULL; + if (usize =3D=3D 0) + return (PyObject *)v; + p =3D PyUnicode_AS_UNICODE(v); + if (0 =3D=3D MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) { + Py_DECREF(v); + return PyErr_SetFromWindowsErrWithFilename(0, NULL); + } +=20 + return (PyObject *)v; + } +=20 + PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p, + int size, + const char *errors) + { + PyObject *repr; + char *s; +=20 + /* First get the size of the result */ + DWORD mbcssize =3D WideCharToMultiByte(CP_ACP, 0, p, size, NULL, = 0, NULL, NULL); + if (mbcssize=3D=3D0) + return PyErr_SetFromWindowsErrWithFilename(0, NULL); +=20 + repr =3D PyString_FromStringAndSize(NULL, mbcssize); + if (repr =3D=3D NULL) + return NULL; + if (mbcssize=3D=3D0) + return repr; +=20 + /* Do the conversion */ + s =3D PyString_AS_STRING(repr); + if (0 =3D=3D WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, = NULL, NULL)) { + Py_DECREF(repr); + return PyErr_SetFromWindowsErrWithFilename(0, NULL); + } + return repr; + } + #endif /* MS_WIN32 */ =20 /* --- Character Mapping Codec = -------------------------------------------- */ =20 ------=_NextPart_000_0039_01BF98AA.69D0B110 Content-Type: application/octet-stream; name="unicodeobject.h.dif" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="unicodeobject.h.dif" diff -c -r2.3 unicodeobject.h *** unicodeobject.h 2000/03/20 16:36:22 2.3 --- unicodeobject.h 2000/03/28 01:29:22 *************** *** 561,566 **** --- 561,586 ---- const char *errors /* error handling */ ); =20 + #ifdef MS_WIN32 + /* --- MBCS codecs for Windows = -------------------------------------------- */ + extern DL_IMPORT(PyObject*) PyUnicode_DecodeMBCS( + const char *string, /* MBCS encoded string */ + int length, /* size of string */ + const char *errors /* error handling */ + ); +=20 + extern DL_IMPORT(PyObject*) PyUnicode_AsMBCSString( + PyObject *unicode /* Unicode object */ + ); +=20 + extern DL_IMPORT(PyObject*) PyUnicode_EncodeMBCS( + const Py_UNICODE *data, /* Unicode char buffer */ + int length, /* Number of Py_UNICODE chars to = encode */ + const char *errors /* error handling */ + ); +=20 +=20 + #endif /* MS_WIN32 */ /* --- Methods & Slots = ---------------------------------------------------- =20 These are capable of handling Unicode objects and strings on input ------=_NextPart_000_0039_01BF98AA.69D0B110 Content-Type: application/octet-stream; name="mbcs.py" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="mbcs.py" """ Python 'mbcs' Codec for Windows Cloned by Mark Hammond (mhammond@skippinet.com.au) from ascii.py, which was written by Marc-Andre Lemburg (mal@lemburg.com). (c) Copyright CNRI, All Rights Reserved. NO WARRANTY. """ import codecs ### Codec APIs class Codec(codecs.Codec): # Note: Binding these as C functions will result in the class not # converting them to methods. This is intended. encode = codecs.mbcs_encode decode = codecs.mbcs_decode class StreamWriter(Codec,codecs.StreamWriter): pass class StreamReader(Codec,codecs.StreamReader): pass class StreamConverter(StreamWriter,StreamReader): encode = codecs.mbcs_decode decode = codecs.mbcs_encode ### encodings module API def getregentry(): return (Codec.encode,Codec.decode,StreamReader,StreamWriter) ------=_NextPart_000_0039_01BF98AA.69D0B110-- From nascheme@enme.ucalgary.ca Tue Mar 28 01:59:36 2000 From: nascheme@enme.ucalgary.ca (Neil Schemenauer) Date: Mon, 27 Mar 2000 18:59:36 -0700 Subject: [Patches] remove Tk < 8.0 support from _tkinter Message-ID: <20000327185936.A4034@acs.ucalgary.ca> --qMm9M+Fa2AknHoGS Content-Type: text/plain; charset=us-ascii I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. --qMm9M+Fa2AknHoGS Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="tk.diff" *** _tkinter.old Mon Mar 27 18:29:32 2000 --- _tkinter.c Mon Mar 27 18:50:23 2000 *************** *** 34,47 **** /* TCL/TK VERSION INFO: ! Unix: ! Tcl/Tk 8.0 (even alpha or beta) or 7.6/4.2 are recommended. ! Versions 7.5/4.1 are the earliest versions still supported. ! Versions 7.4/4.0 or Tk 3.x are no longer supported. ! ! Mac and Windows: ! Use Tcl 8.0 if available (even alpha or beta). ! The oldest usable version is 4.1p1/7.5p1. XXX Further speed-up ideas, involving Tcl 8.0 features: --- 34,43 ---- /* TCL/TK VERSION INFO: ! Tcl/Tk 8.0 (even alpha or beta) are supported. Older versions are ! not supported. Use Python 1.5.2 if you cannot upgrade your Tcl/Tk ! libraries. ! XXX Further speed-up ideas, involving Tcl 8.0 features: *************** *** 80,97 **** #define TKMAJORMINOR (TK_MAJOR_VERSION*1000 + TK_MINOR_VERSION) ! #if TKMAJORMINOR < 4001 ! #error "Tk 4.0 or 3.x are not supported -- use 4.1 or higher" #endif ! #if TKMAJORMINOR >= 8000 && defined(macintosh) /* Sigh, we have to include this to get at the tcl qd pointer */ #include /* And this one we need to clear the menu bar */ #include #endif ! #if TKMAJORMINOR < 8000 || !defined(MS_WINDOWS) #define HAVE_CREATEFILEHANDLER #endif --- 76,93 ---- #define TKMAJORMINOR (TK_MAJOR_VERSION*1000 + TK_MINOR_VERSION) ! #if TKMAJORMINOR < 8000 ! #error "Tk older than 8.0 not supported" #endif ! #if defined(macintosh) /* Sigh, we have to include this to get at the tcl qd pointer */ #include /* And this one we need to clear the menu bar */ #include #endif ! #if !defined(MS_WINDOWS) #define HAVE_CREATEFILEHANDLER #endif *************** *** 108,121 **** #define FHANDLETYPE TCL_UNIX_FD #endif - #if TKMAJORMINOR < 8000 - #define FHANDLE Tcl_File - #define MAKEFHANDLE(fd) Tcl_GetFile((ClientData)(fd), FHANDLETYPE) - #else - #define FHANDLE int - #define MAKEFHANDLE(fd) (fd) - #endif - /* If Tcl can wait for a Unix file descriptor, define the EventHook() routine which uses this to handle Tcl events while the user is typing commands. */ --- 104,109 ---- *************** *** 219,228 **** #include /* For EventRecord */ typedef int (*TclMacConvertEventPtr) Py_PROTO((EventRecord *eventPtr)); - /* They changed the name... */ - #if TKMAJORMINOR < 8000 - #define Tcl_MacSetEventProc TclMacSetEventProc - #endif void Tcl_MacSetEventProc Py_PROTO((TclMacConvertEventPtr procPtr)); int TkMacConvertEvent Py_PROTO((EventRecord *eventPtr)); --- 207,212 ---- *************** *** 495,502 **** TclpInitLibraryPath(baseName); #endif /* TKMAJORMINOR */ ! #if defined(macintosh) && TKMAJORMINOR >= 8000 ! /* This seems to be needed since Tk 8.0 */ ClearMenuBar(); TkMacInitMenus(v->interp); #endif --- 479,486 ---- TclpInitLibraryPath(baseName); #endif /* TKMAJORMINOR */ ! #if defined(macintosh) ! /* This seems to be needed */ ClearMenuBar(); TkMacInitMenus(v->interp); #endif *************** *** 1440,1463 **** { FileHandler_ClientData *data; PyObject *file, *func; ! int mask, id; ! FHANDLE tfile; if (!PyArg_ParseTuple(args, "OiO:createfilehandler", &file, &mask, &func)) return NULL; ! id = GetFileNo(file); ! if (id < 0) return NULL; if (!PyCallable_Check(func)) { PyErr_SetString(PyExc_TypeError, "bad argument list"); return NULL; } ! data = NewFHCD(func, file, id); if (data == NULL) return NULL; - tfile = MAKEFHANDLE(id); /* Ought to check for null Tcl_File object... */ ENTER_TCL Tcl_CreateFileHandler(tfile, mask, FileHandler, (ClientData) data); --- 1424,1445 ---- { FileHandler_ClientData *data; PyObject *file, *func; ! int mask, tfile; if (!PyArg_ParseTuple(args, "OiO:createfilehandler", &file, &mask, &func)) return NULL; ! tfile = GetFileNo(file); ! if (tfile < 0) return NULL; if (!PyCallable_Check(func)) { PyErr_SetString(PyExc_TypeError, "bad argument list"); return NULL; } ! data = NewFHCD(func, file, tfile); if (data == NULL) return NULL; /* Ought to check for null Tcl_File object... */ ENTER_TCL Tcl_CreateFileHandler(tfile, mask, FileHandler, (ClientData) data); *************** *** 1473,1490 **** { PyObject *file; FileHandler_ClientData *data; ! int id; ! FHANDLE tfile; if (!PyArg_ParseTuple(args, "O:deletefilehandler", &file)) return NULL; ! id = GetFileNo(file); ! if (id < 0) return NULL; ! DeleteFHCD(id); - tfile = MAKEFHANDLE(id); /* Ought to check for null Tcl_File object... */ ENTER_TCL Tcl_DeleteFileHandler(tfile); --- 1455,1470 ---- { PyObject *file; FileHandler_ClientData *data; ! int tfile; if (!PyArg_ParseTuple(args, "O:deletefilehandler", &file)) return NULL; ! tfile = GetFileNo(file); ! if (tfile < 0) return NULL; ! DeleteFHCD(tfile); /* Ought to check for null Tcl_File object... */ ENTER_TCL Tcl_DeleteFileHandler(tfile); *************** *** 1905,1911 **** EventHook() { #ifndef MS_WINDOWS ! FHANDLE tfile; #endif #ifdef WITH_THREAD PyEval_RestoreThread(event_tstate); --- 1885,1891 ---- EventHook() { #ifndef MS_WINDOWS ! int tfile; #endif #ifdef WITH_THREAD PyEval_RestoreThread(event_tstate); *************** *** 1913,1919 **** stdin_ready = 0; errorInCmd = 0; #ifndef MS_WINDOWS ! tfile = MAKEFHANDLE(fileno(stdin)); Tcl_CreateFileHandler(tfile, TCL_READABLE, MyFileProc, NULL); #endif while (!errorInCmd && !stdin_ready) { --- 1893,1899 ---- stdin_ready = 0; errorInCmd = 0; #ifndef MS_WINDOWS ! tfile = fileno(stdin); Tcl_CreateFileHandler(tfile, TCL_READABLE, MyFileProc, NULL); #endif while (!errorInCmd && !stdin_ready) { *************** *** 2045,2055 **** Tktt_Type.ob_type = &PyType_Type; PyDict_SetItemString(d, "TkttType", (PyObject *)&Tktt_Type); - #if TKMAJORMINOR >= 8000 /* This helps the dynamic loader; in Unicode aware Tcl versions it also helps Tcl find its encodings. */ Tcl_FindExecutable(Py_GetProgramName()); - #endif if (PyErr_Occurred()) return; --- 2025,2033 ---- *************** *** 2058,2067 **** /* This was not a good idea; through bindings, Tcl_Finalize() may invoke Python code but at that point the interpreter and thread state have already been destroyed! */ - #if TKMAJORMINOR >= 8000 Py_AtExit(Tcl_Finalize); #endif - #endif #ifdef macintosh /* --- 2036,2043 ---- *************** *** 2069,2077 **** ** Most of the initializations in that routine (toolbox init calls and ** such) have already been done for us, so we only need these. */ - #if TKMAJORMINOR >= 8000 tcl_macQdPtr = &qd; - #endif Tcl_MacSetEventProc(PyMacConvertEvent); #if GENERATINGCFM --- 2045,2051 ---- *************** *** 2127,2192 **** } return TkMacConvertEvent(eventPtr); } - - #if defined(USE_GUSI) && TKMAJORMINOR < 8000 - /* - * For Python we have to override this routine (from TclMacNotify), - * since we use GUSI for our sockets, not Tcl streams. Hence, we have - * to use GUSI select to see whether our socket is ready. Note that - * createfilehandler (above) sets the type to TCL_UNIX_FD for our - * files and sockets. - * - * NOTE: this code was lifted from Tcl 7.6, it may need to be modified - * for other versions. */ - - int - Tcl_FileReady(file, mask) - Tcl_File file; /* File handle for a stream. */ - int mask; /* OR'ed combination of TCL_READABLE, - * TCL_WRITABLE, and TCL_EXCEPTION: - * indicates conditions caller cares about. */ - { - int type; - int fd; - - fd = (int) Tcl_GetFileInfo(file, &type); - - if (type == TCL_MAC_SOCKET) { - return TclMacSocketReady(file, mask); - } else if (type == TCL_MAC_FILE) { - /* - * Under the Macintosh, files are always ready, so we just - * return the mask that was passed in. - */ - - return mask; - } else if (type == TCL_UNIX_FD) { - fd_set readset, writeset, excset; - struct timeval tv; - - FD_ZERO(&readset); - FD_ZERO(&writeset); - FD_ZERO(&excset); - - if ( mask & TCL_READABLE ) FD_SET(fd, &readset); - if ( mask & TCL_WRITABLE ) FD_SET(fd, &writeset); - if ( mask & TCL_EXCEPTION ) FD_SET(fd, &excset); - - tv.tv_sec = tv.tv_usec = 0; - if ( select(fd+1, &readset, &writeset, &excset, &tv) <= 0 ) - return 0; - - mask = 0; - if ( FD_ISSET(fd, &readset) ) mask |= TCL_READABLE; - if ( FD_ISSET(fd, &writeset) ) mask |= TCL_WRITABLE; - if ( FD_ISSET(fd, &excset) ) mask |= TCL_EXCEPTION; - - return mask; - } - - return 0; - } - #endif /* USE_GUSI */ #if GENERATINGCFM --- 2101,2106 ---- --qMm9M+Fa2AknHoGS-- From guido@python.org Tue Mar 28 02:03:28 2000 From: guido@python.org (Guido van Rossum) Date: Mon, 27 Mar 2000 21:03:28 -0500 Subject: [Patches] Unicode changes. In-Reply-To: Your message of "Tue, 28 Mar 2000 11:40:27 +1000." References: Message-ID: <200003280203.VAA29161@eric.cnri.reston.va.us> Great :-) All checked in. Question -- I imagine there might be a use for an MBCS codec outside the Windows platform. How hard would it be to add one? --Guido van Rossum (home page: http://www.python.org/~guido/) From mhammond@skippinet.com.au Tue Mar 28 03:41:15 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Tue, 28 Mar 2000 13:41:15 +1000 Subject: [Patches] Unicode changes. In-Reply-To: <200003280203.VAA29161@eric.cnri.reston.va.us> Message-ID: > Question -- I imagine there might be a use for an MBCS codec outside > the Windows platform. How hard would it be to add one? All it would require is a sys.platform style check in mbcs.py, and the relevent #defines in unicodeobject.c. Having 2 MBCS codecs on Windows would probably cause problems :-) Unless I misunderstand the question... Mark. From mhammond@skippinet.com.au Tue Mar 28 04:13:37 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Tue, 28 Mar 2000 14:13:37 +1000 Subject: [Patches] winreg module Message-ID: This is a multi-part message in MIME format. ------=_NextPart_000_0004_01BF98BF.CF4B3050 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Attached is a proposed winreg module, which is an interface to the Windows registry module. A number of people have had a look at this (pre my Unicode additions) so thanks for your help. It is built as an extension module - ie, it creates winreg.pyd. Guido previously expressed he had no opinion on the .pyd-vs-builtin question, so I went for my preference :-) PC\winreg.c exposes the raw Windows registry API, and uses a Python object to represent the Windows registry handles. This provides auto-close functionality similar to file objects. The module has copious docstrings - their formatting could be improved, but this is the best I can come up with :-) PC\dllbase_nt.txt has been brought up to date. We need a better base-address solution than this, but the file is in CVS, so it may as well be accurate :-) PCbuild\winreg.dsp, the MSVC project file, and PCbuild\pcbuild.dsw, an update to the MSVC Workspace that includes this new project, are also attached. lib\test\test_winreg.py and lib\test\output\test_winreg form the test suite for this module. This includes testing of Unicode strings, although does not include non-ascii characters through the Unicode interfaces. Ummm - I think that is about it :-) Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Mark. ------=_NextPart_000_0004_01BF98BF.CF4B3050 Content-Type: application/octet-stream; name="winreg.c" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="winreg.c" /* winreg.c Windows Registry access module for Python. * Simple registry access written by Mark Hammond in win32api module circa 1995. * Bill Tutt expanded the support significantly not long after. * Numerous other people have submitted patches since then. * Ripped from win32api module 03-Feb-2000 by Mark Hammond, and basic Unicode support added. */ #include "windows.h" #include "Python.h" #include "structmember.h" #include "malloc.h" /* for alloca */ static BOOL PyHKEY_AsHKEY(PyObject *ob, HKEY *pRes, BOOL bNoneOK); static PyObject *PyHKEY_FromHKEY(HKEY h); static BOOL PyHKEY_Close(PyObject *obHandle); static char errNotAHandle[] =3D "Object is not a handle"; /* The win32api module reports the function name that failed, but this concept is not in the Python core. Hopefully it will one day, and in the meantime I dont want to lose this info... */ #define PyErr_SetFromWindowsErrWithFunction(rc, fnname) \ PyErr_SetFromWindowsErr(rc) /* Forward declares */ /* Doc strings */ static char module_doc[] =3D "This module provides access to the Windows registry API.\n" "\n" "Functions:\n" "\n" "CloseKey() - Closes a registry key.\n" "ConnectRegistry() - Establishes a connection to a predefined registry = handle\n" " on another computer.\n" "CreateKey() - Creates the specified key, or opens it if it already = exists.\n" "DeleteKey() - Deletes the specified key.\n" "DeleteValue() - Removes a named value from the specified registry = key.\n" "EnumKey() - Enumerates subkeys of the specified open registry key.\n" "EnumValue() - Enumerates values of the specified open registry key.\n" "FlushKey() - Writes all the attributes of the specified key to the = registry.\n" "LoadKey() - Creates a subkey under HKEY_USER or HKEY_LOCAL_MACHINE and = stores\n" " registration information from a specified file into that = subkey.\n" "OpenKey() - Alias for \n" "OpenKeyEx() - Opens the specified key.\n" "QueryValue() - Retrieves the value associated with the unnamed value = for a\n" " specified key in the registry.\n" "QueryValueEx() - Retrieves the type and data for a specified value = name\n" " associated with an open registry key.\n" "QueryInfoKey() - Returns information about the specified key.\n" "SaveKey() - Saves the specified key, and all its subkeys a file.\n" "SetValue() - Associates a value with a specified key.\n" "SetValueEx() - Stores data in the value field of an open registry = key.\n" "\n" "Special objects:\n" "\n" "HKEYType -- type object for HKEY objects\n" "error -- exception raised for Win32 errors\n" "\n" "Integer constants:\n" "Many constants are defined - see the documentation for each function\n" "to see what constants are used, and where."; static char CloseKey_doc[] =3D "CloseKey(hkey) - Closes a previously opened registry key.\n" "\n" "The hkey argument specifies a previously opened key.\n" "\n" "Note that if the key is not closed using this method, it will be\n" "closed when the hkey object is destroyed by Python."; static char ConnectRegistry_doc[] =3D "key =3D ConnectRegistry(computer_name, key) - " "Establishes a connection to a predefined registry handle on another = computer.\n" "\n" "computer_name is the name of the remote computer, of the form = \\\\computername.\n" " If None, the local computer is used.\n" "key is the predefined handle to connect to.\n" "\n" "The return value is the handle of the opened key.\n" "If the function fails, an exception is raised."; static char CreateKey_doc[] =3D "key =3D CreateKey(key, sub_key) - Creates or opens the specified = key.\n" "\n" "key is an already open key, or one of the predefined HKEY_* = constants\n" "sub_key is a string that names the key this method opens or creates.\n" " If key is one of the predefined keys, sub_key may be None. In that = case,\n" " the handle returned is the same key handle passed in to the = function.\n" "\n" "If the key already exists, this function opens the existing key\n" "\n" "The return value is the handle of the opened key.\n" "If the function fails, an exception is raised."; static char DeleteKey_doc[] =3D "DeleteKey(key, sub_key) - Deletes the specified key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "sub_key is a string that must be a subkey of the key identified by the = key parameter.\n" " This value must not be None, and the key may not have subkeys.\n" "\n" "This method can not delete keys with subkeys.\n" "\n" "If the method succeeds, the entire key, including all of its values,\n" "is removed. If the method fails, and exception is raised."; static char DeleteValue_doc[] =3D "DeleteValue(key, value) - Removes a named value from a registry key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "value is a string that identifies the value to remove."; static char EnumKey_doc[] =3D "string =3D EnumKey(key, index) - Enumerates subkeys of an open registry = key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "index is an integer that identifies the index of the key to = retrieve.\n" "\n" "The function retrieves the name of one subkey each time it is = called.\n" "It is typically called repeatedly until an exception is raised, = indicating\n" "no more values are available.\n"; static char EnumValue_doc[] =3D "tuple =3D EnumValue(key, index) - Enumerates values of an open registry = key.\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "index is an integer that identifies the index of the value to = retrieve.\n" "\n" "The function retrieves the name of one subkey each time it is = called.\n" "It is typically called repeatedly, until an exception is raised,\n" "indicating no more values.\n" "\n" "The result is a tuple of 3 items:\n" "value_name is a string that identifies the value.\n" "value_data is an object that holds the value data, and whose type = depends\n" " on the underlying registry type.\n" "data_type is an integer that identifies the type of the value data."; static char FlushKey_doc[] =3D "FlushKey(key) - Writes all the attributes of a key to the registry.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "\n" "It is not necessary to call RegFlushKey to change a key.\n" "Registry changes are flushed to disk by the registry using its lazy = flusher.\n" "Registry changes are also flushed to disk at system shutdown.\n" "Unlike CloseKey(), the FlushKey() method returns only when all the data = has\n" "been written to the registry.\n" "An application should only call FlushKey() if it requires absolute = certainty that registry changes are on disk.\n" "If you don't know whether a FlushKey() call is required, it probably = isn't.\n"; static char LoadKey_doc[] =3D "RegLoadKey(key, sub_key, file_name) - Creates a subkey under the = specified key.\n" "and stores registration information from a specified file into that = subkey.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "sub_key is a string that identifies the sub_key to load\n" "file_name is the name of the file to load registry data from.\n" " This file must have been created with the SaveKey() function.\n" " Under the file allocation table (FAT) file system, the filename may = not\n" "have an extension.\n" "\n" "A call to LoadKey() fails if the calling process does not have the\n" "SE_RESTORE_PRIVILEGE privilege.\n" "\n" "If key is a handle returned by ConnectRegistry(), then the path = specified\n" "in fileName is relative to the remote computer.\n" "\n" "The docs imply key must be in the HKEY_USER or HKEY_LOCAL_MACHINE = tree"; static char OpenKey_doc[] =3D "key =3D OpenKey(key, sub_key, res =3D 0, sam =3D KEY_READ) - Opens the = specified key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "sub_key is a string that identifies the sub_key to open\n" "res is a reserved integer, and must be zero. Default is zero.\n" "sam is an integer that specifies an access mask that describes the = desired\n" " security access for the key. Default is KEY_READ\n" "\n" "The result is a new handle to the specified key\n" "If the function fails, an exception is raised.\n"; static char OpenKeyEx_doc[] =3D "See OpenKey()"; static char QueryInfoKey_doc[] =3D "tuple =3D QueryInfoKey(key) - Returns information about a key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "\n" "The result is a tuple of 3 items:" "An integer that identifies the number of sub keys this key has.\n" "An integer that identifies the number of values this key has.\n" "A long integer that identifies when the key was last modified (if = available)\n" " as 100's of nanoseconds since Jan 1, 1600.\n"; static char QueryValue_doc[] =3D "string =3D QueryValue(key, sub_key) - retrieves the unnamed value for a = key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "sub_key is a string that holds The name of the subkey with which the = value\n" " is associated. If this parameter is None or empty, the function = retrieves\n" " the value set by the SetValue() method for the key identified by key." "\n" "Values in the registry have name, type, and data components. This = method\n" "retrieves the data for a key's first value that has a NULL name.\n" "But the underlying API call doesn't return the type, Lame Lame Lame, = DONT USE THIS!!!"; static char QueryValueEx_doc[] =3D "value,type_id =3D QueryValueEx(key, value_name) - Retrieves the type = and data for a specified value name associated with an open registry = key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "value_name is a string indicating the value to query"; static char SaveKey_doc[] =3D "SaveKey(key, file_name) - Saves the specified key, and all its subkeys = to the specified file.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "file_name is the name of the file to save registry data to.\n" " This file cannot already exist. If this filename includes an = extension,\n" " it cannot be used on file allocation table (FAT) file systems by = the\n" " LoadKey(), ReplaceKey() or RestoreKey() methods.\n" "\n" "If key represents a key on a remote computer, the path described by\n" "file_name is relative to the remote computer.\n" "The caller of this method must possess the SeBackupPrivilege security = privilege.\n" "This function passes NULL for security_attributes to the API."; static char SetValue_doc[] =3D "SetValue(key, sub_key, type, value) - Associates a value with a = specified key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "sub_key is a string that names the subkey with which the value is = associated.\n" "type is an integer that specifies the type of the data. Currently = this\n" " must be REG_SZ, meaning only strings are supported.\n" "value is a string that specifies the new value.\n" "\n" "If the key specified by the sub_key parameter does not exist, the = SetValue\n" "function creates it.\n" "\n" "Value lengths are limited by available memory. Long values (more = than\n" "2048 bytes) should be stored as files with the filenames stored in \n" "the configuration registry. This helps the registry perform = efficiently.\n" "\n" "The key identified by the key parameter must have been opened with\n" "KEY_SET_VALUE access."; static char SetValueEx_doc[] =3D "SetValueEx(key, value_name, reserved, type, value) - Stores data in the = value field of an open registry key.\n" "\n" "key is an already open key, or any one of the predefined HKEY_* = constants.\n" "sub_key is a string that names the subkey with which the value is = associated.\n" "type is an integer that specifies the type of the data. This should be = one of:\n" " REG_BINARY -- Binary data in any form.\n" " REG_DWORD -- A 32-bit number.\n" " REG_DWORD_LITTLE_ENDIAN -- A 32-bit number in little-endian = format.\n" " REG_DWORD_BIG_ENDIAN -- A 32-bit number in big-endian format.\n" " REG_EXPAND_SZ -- A null-terminated string that contains unexpanded = references\n" " to environment variables (for example, %PATH%).\n" " REG_LINK -- A Unicode symbolic link.\n" " REG_MULTI_SZ -- An array of null-terminated strings, terminated by\n" " two null characters.\n" " REG_NONE -- No defined value type.\n" " REG_RESOURCE_LIST -- A device-driver resource list.\n" " REG_SZ -- A null-terminated string.\n" "reserved can be anything - zero is always passed to the API.\n" "value is a string that specifies the new value.\n" "\n" "This method can also set additional value and type information for = the\n" "specified key. The key identified by the key parameter must have = been\n" "opened with KEY_SET_VALUE access.\n" "\n" "To open the key, use the CreateKeyEx() or OpenKeyEx() methods.\n" "\n" "Value lengths are limited by available memory. Long values (more = than\n" "2048 bytes) should be stored as files with the filenames stored in \n" "the configuration registry. This helps the registry perform = efficiently.\n"; /* PyHKEY docstrings */ static char PyHKEY_doc[] =3D "PyHKEY Object - A Python object, representing a win32 registry key.\n" "\n" "This object wraps a win32 HANDLE object, automatically closing it = when\n" "the object is destroyed. To guarantee cleanup, you can call either\n" "the Close() method on the PyHKEY, or the CloseKey() method.\n" "\n" "All functions which accept a handle object also accept an integer - \n" "however, use of the handle object is encouraged.\n" "\n" "Functions:\n" "Close() - Closes the underlying handle.\n" "Detach() - Returns the integer Win32 handle, detaching it from the = object\n" "\n" "Properties:\n" "handle - The integer Win32 handle.\n" "\n" "Operations:\n" "__nonzero__ - Handles with an open object return true, otherwise = false.\n" "__int__ - Converting a handle to an integer returns the Win32 = handle.\n" "__cmp__ - Handle objects are compared using the handle value.\n"; static char PyHKEY_Close_doc[] =3D "key.Close() - Closes the underlying Win32 handle.\n" "\n" "If the handle is already closed, no error is raised."; static char PyHKEY_Detach_doc[] =3D "int =3D key.Detach() - Detaches the Win32 handle from the handle = object.\n" "\n" "The result is the value of the handle before it is detached. If the\n" "handle is already detached, this will return zero.\n" "\n" "After calling this function, the handle is effectively invalidated,\n" "but the handle is not closed. You would call this function when you\n" "need the underlying win32 handle to exist beyond the lifetime of the\n" "handle object."; /************************************************************************= The PyHKEY object definition ************************************************************************/= typedef struct { PyObject_VAR_HEAD HKEY hkey; } PyHKEYObject; #define PyHKEY_Check(op) ((op)->ob_type =3D=3D &PyHKEY_Type) static char *failMsg =3D "bad operand type"; static PyObject * PyHKEY_unaryFailureFunc(PyObject *ob) { PyErr_SetString(PyExc_TypeError, failMsg); return NULL; } static PyObject * PyHKEY_binaryFailureFunc(PyObject *ob1, PyObject *ob2) { PyErr_SetString(PyExc_TypeError, failMsg); return NULL; } static PyObject * PyHKEY_ternaryFailureFunc(PyObject *ob1, PyObject *ob2, PyObject *ob3) { PyErr_SetString(PyExc_TypeError, failMsg); return NULL; } static void PyHKEY_deallocFunc(PyObject *ob) { /* Can not call PyHKEY_Close, as the ob->tp_type has already been cleared, thus causing the type check to fail! */ PyHKEYObject *obkey =3D (PyHKEYObject *)ob; if (obkey->hkey) RegCloseKey((HKEY)obkey->hkey); PyMem_DEL(ob); } static int PyHKEY_nonzeroFunc(PyObject *ob) { return ((PyHKEYObject *)ob)->hkey !=3D 0; } static PyObject * PyHKEY_intFunc(PyObject *ob) { PyHKEYObject *pyhkey =3D (PyHKEYObject *)ob; return PyLong_FromVoidPtr(pyhkey->hkey); } static int PyHKEY_printFunc(PyObject *ob, FILE *fp, int flags) { PyHKEYObject *pyhkey =3D (PyHKEYObject *)ob; char resBuf[160]; wsprintf(resBuf, "", ob, pyhkey->hkey); fputs(resBuf, fp); return 0; } static PyObject * PyHKEY_strFunc(PyObject *ob) { PyHKEYObject *pyhkey =3D (PyHKEYObject *)ob; char resBuf[160]; wsprintf(resBuf, "", pyhkey->hkey); return PyString_FromString(resBuf); } static int PyHKEY_compareFunc(PyObject *ob1, PyObject *ob2) { PyHKEYObject *pyhkey1 =3D (PyHKEYObject *)ob1; PyHKEYObject *pyhkey2 =3D (PyHKEYObject *)ob2; return pyhkey1 =3D=3D pyhkey2 ? 0 : (pyhkey1 < pyhkey2 ? -1 : 1); } static long PyHKEY_hashFunc(PyObject *ob) { /* Just use the address. XXX - should we use the handle value? */ return (long)ob; } static PyNumberMethods PyHKEY_NumberMethods =3D { PyHKEY_binaryFailureFunc, /* nb_add */ PyHKEY_binaryFailureFunc, /* nb_subtract */ PyHKEY_binaryFailureFunc, /* nb_multiply */ PyHKEY_binaryFailureFunc, /* nb_divide */ PyHKEY_binaryFailureFunc, /* nb_remainder */ PyHKEY_binaryFailureFunc, /* nb_divmod */ PyHKEY_ternaryFailureFunc, /* nb_power */ PyHKEY_unaryFailureFunc, /* nb_negative */ PyHKEY_unaryFailureFunc, /* nb_positive */ PyHKEY_unaryFailureFunc, /* nb_absolute */ PyHKEY_nonzeroFunc, /* nb_nonzero */ PyHKEY_unaryFailureFunc, /* nb_invert */ PyHKEY_binaryFailureFunc, /* nb_lshift */ PyHKEY_binaryFailureFunc, /* nb_rshift */ PyHKEY_binaryFailureFunc, /* nb_and */ PyHKEY_binaryFailureFunc, /* nb_xor */ PyHKEY_binaryFailureFunc, /* nb_or */ 0, /* nb_coerce (allowed to be zero) */ PyHKEY_intFunc, /* nb_int */ PyHKEY_unaryFailureFunc, /* nb_long */ PyHKEY_unaryFailureFunc, /* nb_float */ PyHKEY_unaryFailureFunc, /* nb_oct */ PyHKEY_unaryFailureFunc, /* nb_hex */ }; /* fwd declare __getattr__ */ static PyObject *PyHKEY_getattr(PyObject *self, char *name); /* The type itself */ PyTypeObject PyHKEY_Type =3D { PyObject_HEAD_INIT(0) /* fill in type at module init */ 0, "PyHKEY", sizeof(PyHKEYObject), 0, PyHKEY_deallocFunc, /* tp_dealloc */ PyHKEY_printFunc, /* tp_print */ PyHKEY_getattr, /* tp_getattr */ 0, /* tp_setattr */ PyHKEY_compareFunc, /* tp_compare */ 0, /* tp_repr */ &PyHKEY_NumberMethods, /* tp_as_number */ 0, /* tp_as_sequence */ 0, /* tp_as_mapping */ PyHKEY_hashFunc, /* tp_hash */ 0, /* tp_call */ PyHKEY_strFunc, /* tp_str */ 0, /* tp_getattro */ 0, /* tp_setattro */ 0, /* tp_as_buffer */ 0, /* tp_flags */ PyHKEY_doc, /* tp_doc */ }; #define OFF(e) offsetof(PyHKEYObject, e) static struct memberlist PyHKEY_memberlist[] =3D { {"handle", T_INT, OFF(hkey)}, {NULL} /* Sentinel */ }; /************************************************************************= The PyHKEY object methods ************************************************************************/= static PyObject * PyHKEY_CloseMethod(PyObject *self, PyObject *args) { if (!PyArg_ParseTuple(args, ":Close")) return NULL; if (!PyHKEY_Close(self)) return NULL; Py_INCREF(Py_None); return Py_None; } static PyObject * PyHKEY_DetachMethod(PyObject *self, PyObject *args) { void* ret; PyHKEYObject *pThis =3D (PyHKEYObject *)self; if (!PyArg_ParseTuple(args, ":Detach")) return NULL; ret =3D (void*)pThis->hkey; pThis->hkey =3D 0; return PyLong_FromVoidPtr(ret); } static struct PyMethodDef PyHKEY_methods[] =3D { {"Close", PyHKEY_CloseMethod, 1, PyHKEY_Close_doc}, {"Detach", PyHKEY_DetachMethod, 1, PyHKEY_Detach_doc}, {NULL} }; /*static*/ PyObject * PyHKEY_getattr(PyObject *self, char *name) { PyObject *res; res =3D Py_FindMethod(PyHKEY_methods, self, name); if (res !=3D NULL) return res; PyErr_Clear(); if (strcmp(name, "handle") =3D=3D 0) return PyLong_FromVoidPtr(((PyHKEYObject *)self)->hkey); return PyMember_Get((char *)self, PyHKEY_memberlist, name); } /************************************************************************= The public PyHKEY API (well, not public yet :-) ************************************************************************/= PyObject * PyHKEY_New(HKEY hInit) { PyHKEYObject *key =3D PyObject_NEW(PyHKEYObject, &PyHKEY_Type); if (key) key->hkey =3D hInit; return (PyObject *)key; } BOOL PyHKEY_Close(PyObject *ob_handle) { LONG rc; PyHKEYObject *key; if (!PyHKEY_Check(ob_handle)) { PyErr_SetString(PyExc_TypeError, "bad operand type"); return FALSE; } key =3D (PyHKEYObject *)ob_handle; rc =3D key->hkey ? RegCloseKey((HKEY)key->hkey) : ERROR_SUCCESS; key->hkey =3D 0; if (rc !=3D ERROR_SUCCESS) PyErr_SetFromWindowsErrWithFunction(rc, "RegCloseKey"); return rc =3D=3D ERROR_SUCCESS; } BOOL PyHKEY_AsHKEY(PyObject *ob, HKEY *pHANDLE, BOOL bNoneOK) { if (ob =3D=3D Py_None) { if (!bNoneOK) { PyErr_SetString( PyExc_TypeError, "None is not a valid HKEY in this context"); return FALSE; } *pHANDLE =3D (HKEY)0; } else if (PyHKEY_Check(ob)) { PyHKEYObject *pH =3D (PyHKEYObject *)ob; *pHANDLE =3D pH->hkey; } else if (PyInt_Check(ob) || PyLong_Check(ob)) { /* We also support integers */ PyErr_Clear(); *pHANDLE =3D (HKEY)PyLong_AsVoidPtr(ob); if (PyErr_Occurred()) return FALSE; *pHANDLE =3D (HKEY)PyInt_AsLong(ob); } else { PyErr_SetString( PyExc_TypeError, "The object is not a PyHKEY object"); return FALSE; } return TRUE; } PyObject * PyHKEY_FromHKEY(HKEY h) { PyHKEYObject *op =3D (PyHKEYObject *) malloc(sizeof(PyHKEYObject)); if (op =3D=3D NULL) return PyErr_NoMemory(); op->ob_type =3D &PyHKEY_Type; op->hkey =3D h; _Py_NewReference((PyObject *)op); return (PyObject *)op; } /************************************************************************= The module methods ************************************************************************/= BOOL PyWinObject_CloseHKEY(PyObject *obHandle) { BOOL ok; if (PyHKEY_Check(obHandle)) { ok =3D PyHKEY_Close(obHandle); } else if (PyInt_Check(obHandle)) { long rc =3D RegCloseKey((HKEY)PyInt_AsLong(obHandle)); ok =3D (rc =3D=3D ERROR_SUCCESS); if (!ok) PyErr_SetFromWindowsErrWithFunction(rc, "RegCloseKey"); } else { PyErr_SetString( PyExc_TypeError, "A handle must be a HKEY object or an integer"); return FALSE; } return ok; } /* Private Helper functions for the registry interfaces ** Note that fixupMultiSZ and countString have both had changes ** made to support "incorrect strings". The registry specification ** calls for strings to be terminated with 2 null bytes. It seems ** some commercial packages install strings whcich dont conform, ** causing this code to fail - however, "regedit" etc still work ** with these strings (ie only we dont!). */ static void fixupMultiSZ(char **str, char *data, int len) { char *P; int i; char *Q; Q =3D data + len; for (P =3D data, i =3D 0; P < Q && *P !=3D '\0'; P++, i++) { str[i] =3D P; for(; *P !=3D '\0'; P++) ; } } static int countStrings(char *data, int len) { int strings; char *P; char *Q =3D data + len; for (P =3D data, strings =3D 0; P < Q && *P !=3D '\0'; P++, strings++) for (; P < Q && *P !=3D '\0'; P++) ; return strings; } /* Convert PyObject into Registry data. Allocates space as needed. */ static BOOL Py2Reg(PyObject *value, DWORD typ, BYTE **retDataBuf, DWORD = *retDataSize) { int i,j; switch (typ) { case REG_DWORD: if (value !=3D Py_None && !PyInt_Check(value)) return FALSE; *retDataBuf =3D (BYTE *)PyMem_NEW(DWORD, sizeof(DWORD)); if (*retDataBuf=3D=3DNULL){ PyErr_NoMemory(); return FALSE; } *retDataSize =3D sizeof(DWORD); if (value =3D=3D Py_None) { DWORD zero =3D 0; memcpy(*retDataBuf, &zero, sizeof(DWORD)); } else memcpy(*retDataBuf, &PyInt_AS_LONG((PyIntObject *)value), sizeof(DWORD)); break; case REG_SZ: case REG_EXPAND_SZ: { int need_decref =3D 0; if (value =3D=3D Py_None) *retDataSize =3D 1; else { if (PyUnicode_Check(value)) { value =3D PyUnicode_AsEncodedString( value, "mbcs", NULL); if (value=3D=3DNULL) return FALSE; need_decref =3D 1; } if (!PyString_Check(value)) return FALSE; *retDataSize =3D 1 + strlen( PyString_AS_STRING( (PyStringObject *)value)); } *retDataBuf =3D (BYTE *)PyMem_NEW(DWORD, *retDataSize); if (*retDataBuf=3D=3DNULL){ PyErr_NoMemory(); return FALSE; } if (value =3D=3D Py_None) strcpy((char *)*retDataBuf, ""); else strcpy((char *)*retDataBuf, PyString_AS_STRING( (PyStringObject *)value)); if (need_decref) Py_DECREF(value); break; } case REG_MULTI_SZ: { DWORD size =3D 0; char *P; PyObject **obs =3D NULL; if (value =3D=3D Py_None) i =3D 0; else { if (!PyList_Check(value)) return FALSE; i =3D PyList_Size(value); } obs =3D malloc(sizeof(PyObject *) * i); memset(obs, 0, sizeof(PyObject *) * i); for (j =3D 0; j < i; j++) { PyObject *t; t =3D PyList_GET_ITEM( (PyListObject *)value,j); if (PyString_Check(t)) { obs[j] =3D t; Py_INCREF(t); } else if (PyUnicode_Check(t)) { obs[j] =3D PyUnicode_AsEncodedString( t, "mbcs", NULL); if (obs[j]=3D=3DNULL) goto reg_multi_fail; } else goto reg_multi_fail; size +=3D 1 + strlen( PyString_AS_STRING( (PyStringObject *)obs[j])); } *retDataSize =3D size + 1; *retDataBuf =3D (BYTE *)PyMem_NEW(char, *retDataSize); if (*retDataBuf=3D=3DNULL){ PyErr_NoMemory(); goto reg_multi_fail; } P =3D (char *)*retDataBuf; for (j =3D 0; j < i; j++) { PyObject *t; t =3D obs[j]; strcpy(P, PyString_AS_STRING( (PyStringObject *)t)); P +=3D 1 + strlen( PyString_AS_STRING( (PyStringObject *)t)); Py_DECREF(obs[j]); } /* And doubly-terminate the list... */ *P =3D '\0'; free(obs); break; reg_multi_fail: if (obs) { for (j =3D 0; j < i; j++) Py_XDECREF(obs[j]); free(obs); } return FALSE; } case REG_BINARY: /* ALSO handle ALL unknown data types here. Even if we can't support it natively, we should handle the bits. */ default: if (value =3D=3D Py_None) *retDataSize =3D 0; else { if (!PyString_Check(value)) return 0; *retDataSize =3D PyString_Size(value); *retDataBuf =3D (BYTE *)PyMem_NEW(char, *retDataSize); if (*retDataBuf=3D=3DNULL){ PyErr_NoMemory(); return FALSE; } memcpy(*retDataBuf, PyString_AS_STRING( (PyStringObject *)value), *retDataSize); } break; } return TRUE; } /* Convert Registry data into PyObject*/ static PyObject * Reg2Py(char *retDataBuf, DWORD retDataSize, DWORD typ) { PyObject *obData; switch (typ) { case REG_DWORD: if (retDataSize =3D=3D 0) obData =3D Py_BuildValue("i", 0); else obData =3D Py_BuildValue("i", *(int *)retDataBuf); break; case REG_SZ: case REG_EXPAND_SZ: /* retDataBuf may or may not have a trailing NULL in the buffer. */ if (retDataSize && retDataBuf[retDataSize-1] =3D=3D '\0') --retDataSize; if (retDataSize =3D=3D0) retDataBuf =3D ""; obData =3D PyUnicode_DecodeMBCS(retDataBuf, retDataSize, NULL); break; case REG_MULTI_SZ: if (retDataSize =3D=3D 0) obData =3D PyList_New(0); else { int index =3D 0; int s =3D countStrings(retDataBuf, retDataSize); char **str =3D (char **)malloc(sizeof(char *)*s); if (str =3D=3D NULL) return PyErr_NoMemory(); fixupMultiSZ(str, retDataBuf, retDataSize); obData =3D PyList_New(s); for (index =3D 0; index < s; index++) { PyList_SetItem(obData, index, PyUnicode_DecodeMBCS( (const char *)str[index], _mbstrlen(str[index]), NULL) ); } free(str); break; } case REG_BINARY: /* ALSO handle ALL unknown data types here. Even if we can't support it natively, we should handle the bits. */ default: if (retDataSize =3D=3D 0) { Py_INCREF(Py_None); obData =3D Py_None; } else obData =3D Py_BuildValue("s#", (char *)retDataBuf, retDataSize); break; } if (obData =3D=3D NULL) return NULL; else return obData; } /* The Python methods */ static PyObject * PyCloseKey(PyObject *self, PyObject *args) { PyObject *obKey; if (!PyArg_ParseTuple(args, "O:CloseKey", &obKey)) return NULL; if (!PyHKEY_Close(obKey)) return NULL; Py_INCREF(Py_None); return Py_None; } static PyObject * PyConnectRegistry(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *szCompName =3D NULL; HKEY retKey; long rc; if (!PyArg_ParseTuple(args, "zO:ConnectRegistry", &szCompName, &obKey)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; rc =3D RegConnectRegistry(szCompName, hKey, &retKey); if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "ConnectRegistry"); return PyHKEY_FromHKEY(retKey); } static PyObject * PyCreateKey(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *subKey; HKEY retKey; long rc; if (!PyArg_ParseTuple(args, "Oz:CreateKey", &obKey, &subKey)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; rc =3D RegCreateKey(hKey, subKey, &retKey); if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "CreateKey"); return PyHKEY_FromHKEY(retKey); } static PyObject * PyDeleteKey(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *subKey; long rc; if (!PyArg_ParseTuple(args, "Os:DeleteKey", &obKey, &subKey)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; rc =3D RegDeleteKey(hKey, subKey ); if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegDeleteKey"); Py_INCREF(Py_None); return Py_None; } static PyObject * PyDeleteValue(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *subKey; long rc; if (!PyArg_ParseTuple(args, "Oz:DeleteValue", &obKey, &subKey)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; Py_BEGIN_ALLOW_THREADS rc =3D RegDeleteValue(hKey, subKey); Py_END_ALLOW_THREADS if (rc !=3DERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegDeleteValue"); Py_INCREF(Py_None); return Py_None; } static PyObject * PyEnumKey(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; int index; long rc; char *retBuf; DWORD len; if (!PyArg_ParseTuple(args, "Oi:EnumKey", &obKey, &index)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; if ((rc =3D RegQueryInfoKey(hKey, NULL, NULL, NULL, NULL, &len, NULL, NULL, NULL, NULL, NULL, NULL)) !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegQueryInfoKey"); ++len; /* include null terminator */ retBuf =3D (char *)alloca(len); if ((rc =3D RegEnumKey(hKey, index, retBuf, len)) !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegEnumKey"); return Py_BuildValue("s", retBuf); } static PyObject * PyEnumValue(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; int index; long rc; char *retValueBuf; char *retDataBuf; DWORD retValueSize; DWORD retDataSize; DWORD typ; PyObject *obData; PyObject *retVal; if (!PyArg_ParseTuple(args, "Oi:EnumValue", &obKey, &index)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; if ((rc =3D RegQueryInfoKey(hKey, NULL, NULL, NULL, NULL, NULL, NULL, NULL, &retValueSize, &retDataSize, NULL, NULL)) !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegQueryInfoKey"); ++retValueSize; /* include null terminators */ ++retDataSize; retValueBuf =3D (char *)alloca(retValueSize); retDataBuf =3D (char *)alloca(retDataSize); Py_BEGIN_ALLOW_THREADS rc =3D RegEnumValue(hKey, index, retValueBuf, &retValueSize, NULL, &typ, (BYTE *)retDataBuf, &retDataSize); Py_END_ALLOW_THREADS if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "PyRegEnumValue"); obData =3D Reg2Py(retDataBuf, retDataSize, typ); if (obData =3D=3D NULL) return NULL; retVal =3D Py_BuildValue("sOi", retValueBuf, obData, typ); Py_DECREF(obData); return retVal; } static PyObject * PyFlushKey(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; long rc; if (!PyArg_ParseTuple(args, "O:FlushKey", &obKey)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; Py_BEGIN_ALLOW_THREADS rc =3D RegFlushKey(hKey); Py_END_ALLOW_THREADS if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegFlushKey"); Py_INCREF(Py_None); return Py_None; } static PyObject * PyLoadKey(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *subKey; char *fileName; long rc; if (!PyArg_ParseTuple(args, "Oss:LoadKey", &obKey, &subKey, &fileName)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; Py_BEGIN_ALLOW_THREADS rc =3D RegLoadKey(hKey, subKey, fileName ); Py_END_ALLOW_THREADS if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegLoadKey"); Py_INCREF(Py_None); return Py_None; } static PyObject * PyOpenKey(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *subKey; int res =3D 0; HKEY retKey; long rc; REGSAM sam =3D KEY_READ; if (!PyArg_ParseTuple(args, "Oz|ii:OpenKey", &obKey, &subKey, &res, &sam)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; Py_BEGIN_ALLOW_THREADS rc =3D RegOpenKeyEx(hKey, subKey, res, sam, &retKey); Py_END_ALLOW_THREADS if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegOpenKeyEx"); return PyHKEY_FromHKEY(retKey); } static PyObject * PyQueryInfoKey(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; long rc; DWORD nSubKeys, nValues; FILETIME ft; LARGE_INTEGER li; PyObject *l; PyObject *ret; if (!PyArg_ParseTuple(args, "O:QueryInfoKey", &obKey)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; if ((rc =3D RegQueryInfoKey(hKey, NULL, NULL, 0, &nSubKeys, NULL, = NULL, &nValues, NULL, NULL, NULL, &ft)) !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegQueryInfoKey"); li.LowPart =3D ft.dwLowDateTime; li.HighPart =3D ft.dwHighDateTime; l =3D PyLong_FromLongLong(li.QuadPart); if (l =3D=3D NULL) return NULL; ret =3D Py_BuildValue("iiO", nSubKeys, nValues, l); Py_DECREF(l); return ret; } static PyObject * PyQueryValue(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *subKey; long rc; char *retBuf; long bufSize =3D 0; if (!PyArg_ParseTuple(args, "Oz:QueryValue", &obKey, &subKey)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; if ((rc =3D RegQueryValue(hKey, subKey, NULL, &bufSize)) !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegQueryValue"); retBuf =3D (char *)alloca(bufSize); if ((rc =3D RegQueryValue(hKey, subKey, retBuf, &bufSize)) !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegQueryValue"); return Py_BuildValue("s", retBuf); } static PyObject * PyQueryValueEx(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *valueName; long rc; char *retBuf; DWORD bufSize =3D 0; DWORD typ; PyObject *obData; PyObject *result; if (!PyArg_ParseTuple(args, "Oz:QueryValueEx", &obKey, &valueName)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; if ((rc =3D RegQueryValueEx(hKey, valueName, NULL, NULL, NULL, &bufSize)) !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegQueryValueEx"); retBuf =3D (char *)alloca(bufSize); if ((rc =3D RegQueryValueEx(hKey, valueName, NULL, &typ, (BYTE *)retBuf, &bufSize)) !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegQueryValueEx"); obData =3D Reg2Py(retBuf, bufSize, typ); if (obData =3D=3D NULL) return NULL; result =3D Py_BuildValue("Oi", obData, typ); Py_DECREF(obData); return result; } static PyObject * PySaveKey(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *fileName; LPSECURITY_ATTRIBUTES pSA =3D NULL; long rc; if (!PyArg_ParseTuple(args, "Os:SaveKey", &obKey, &fileName)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; /* One day we may get security into the core? if (!PyWinObject_AsSECURITY_ATTRIBUTES(obSA, &pSA, TRUE)) return NULL; */ Py_BEGIN_ALLOW_THREADS rc =3D RegSaveKey(hKey, fileName, pSA ); Py_END_ALLOW_THREADS if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegSaveKey"); Py_INCREF(Py_None); return Py_None; } static PyObject * PySetValue(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *subKey; char *str; DWORD typ; DWORD len; long rc; PyObject *obStrVal; PyObject *obSubKey; if (!PyArg_ParseTuple(args, "OOiO:SetValue", &obKey, &obSubKey, &typ, &obStrVal)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; if (typ !=3D REG_SZ) { PyErr_SetString(PyExc_TypeError, "Type must be win32con.REG_SZ"); return NULL; } /* XXX - need Unicode support */ str =3D PyString_AsString(obStrVal); if (str =3D=3D NULL) return NULL; len =3D PyString_Size(obStrVal); if (obSubKey =3D=3D Py_None) subKey =3D NULL; else { subKey =3D PyString_AsString(obSubKey); if (subKey =3D=3D NULL) return NULL; } Py_BEGIN_ALLOW_THREADS rc =3D RegSetValue(hKey, subKey, REG_SZ, str, len+1); Py_END_ALLOW_THREADS if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegSetValue"); Py_INCREF(Py_None); return Py_None; } static PyObject * PySetValueEx(PyObject *self, PyObject *args) { HKEY hKey; PyObject *obKey; char *valueName; PyObject *obRes; PyObject *value; BYTE *data; DWORD len; DWORD typ; LONG rc; if (!PyArg_ParseTuple(args, "OzOiO:SetValueEx", &obKey, &valueName, &obRes, &typ, &value)) return NULL; if (!PyHKEY_AsHKEY(obKey, &hKey, FALSE)) return NULL; if (!Py2Reg(value, typ, &data, &len)) { if (!PyErr_Occurred()) PyErr_SetString(PyExc_ValueError, "Could not convert the data to the specified type."); return NULL; } Py_BEGIN_ALLOW_THREADS rc =3D RegSetValueEx(hKey, valueName, 0, typ, data, len); Py_END_ALLOW_THREADS PyMem_Free(data); if (rc !=3D ERROR_SUCCESS) return PyErr_SetFromWindowsErrWithFunction(rc, "RegSetValueEx"); Py_INCREF(Py_None); return Py_None; } static struct PyMethodDef winreg_methods[] =3D { {"CloseKey", PyCloseKey, 1, CloseKey_doc}, {"ConnectRegistry", PyConnectRegistry, 1, ConnectRegistry_doc}, {"CreateKey", PyCreateKey, 1, CreateKey_doc}, {"DeleteKey", PyDeleteKey, 1, DeleteKey_doc}, {"DeleteValue", PyDeleteValue, 1, DeleteValue_doc}, {"EnumKey", PyEnumKey, 1, EnumKey_doc}, {"EnumValue", PyEnumValue, 1, EnumValue_doc}, {"FlushKey", PyFlushKey, 1, FlushKey_doc}, {"LoadKey", PyLoadKey, 1, LoadKey_doc}, {"OpenKey", PyOpenKey, 1, OpenKey_doc}, {"OpenKeyEx", PyOpenKey, 1, OpenKeyEx_doc}, {"QueryValue", PyQueryValue, 1, QueryValue_doc}, {"QueryValueEx", PyQueryValueEx, 1, QueryValueEx_doc}, {"QueryInfoKey", PyQueryInfoKey, 1, QueryInfoKey_doc}, {"SaveKey", PySaveKey, 1, SaveKey_doc}, {"SetValue", PySetValue, 1, SetValue_doc}, {"SetValueEx", PySetValueEx, 1, SetValueEx_doc}, NULL, }; static void insint(PyObject * d, char * name, long value) { PyObject *v =3D PyInt_FromLong(value); if (!v || PyDict_SetItemString(d, name, v)) PyErr_Clear(); Py_XDECREF(v); } #define ADD_INT(val) insint(d, #val, val) static void inskey(PyObject * d, char * name, HKEY key) { PyObject *v =3D PyLong_FromVoidPtr(key); if (!v || PyDict_SetItemString(d, name, v)) PyErr_Clear(); Py_XDECREF(v); } #define ADD_KEY(val) inskey(d, #val, val) __declspec(dllexport) void initwinreg(void) { PyObject *m, *d; m =3D Py_InitModule3("winreg", winreg_methods, module_doc); d =3D PyModule_GetDict(m); PyHKEY_Type.ob_type =3D &PyType_Type; PyHKEY_Type.tp_doc =3D PyHKEY_doc; Py_INCREF(&PyHKEY_Type); if (PyDict_SetItemString(d, "HKEYType", (PyObject *)&PyHKEY_Type) !=3D 0) return; Py_INCREF(PyExc_WindowsError); if (PyDict_SetItemString(d, "error", PyExc_WindowsError) !=3D 0) return; /* Add the relevant constants */ ADD_KEY(HKEY_CLASSES_ROOT); ADD_KEY(HKEY_CURRENT_USER); ADD_KEY(HKEY_LOCAL_MACHINE); ADD_KEY(HKEY_USERS); ADD_KEY(HKEY_PERFORMANCE_DATA); #ifdef HKEY_CURRENT_CONFIG ADD_KEY(HKEY_CURRENT_CONFIG); #endif #ifdef HKEY_DYN_DATA ADD_KEY(HKEY_DYN_DATA); #endif ADD_INT(KEY_QUERY_VALUE); ADD_INT(KEY_SET_VALUE); ADD_INT(KEY_CREATE_SUB_KEY); ADD_INT(KEY_ENUMERATE_SUB_KEYS); ADD_INT(KEY_NOTIFY); ADD_INT(KEY_CREATE_LINK); ADD_INT(KEY_READ); ADD_INT(KEY_WRITE); ADD_INT(KEY_EXECUTE); ADD_INT(KEY_ALL_ACCESS); ADD_INT(REG_OPTION_RESERVED); ADD_INT(REG_OPTION_NON_VOLATILE); ADD_INT(REG_OPTION_VOLATILE); ADD_INT(REG_OPTION_CREATE_LINK); ADD_INT(REG_OPTION_BACKUP_RESTORE); ADD_INT(REG_OPTION_OPEN_LINK); ADD_INT(REG_LEGAL_OPTION); ADD_INT(REG_CREATED_NEW_KEY); ADD_INT(REG_OPENED_EXISTING_KEY); ADD_INT(REG_WHOLE_HIVE_VOLATILE); ADD_INT(REG_REFRESH_HIVE); ADD_INT(REG_NO_LAZY_FLUSH); ADD_INT(REG_NOTIFY_CHANGE_NAME); ADD_INT(REG_NOTIFY_CHANGE_ATTRIBUTES); ADD_INT(REG_NOTIFY_CHANGE_LAST_SET); ADD_INT(REG_NOTIFY_CHANGE_SECURITY); ADD_INT(REG_LEGAL_CHANGE_FILTER); ADD_INT(REG_NONE); ADD_INT(REG_SZ); ADD_INT(REG_EXPAND_SZ); ADD_INT(REG_BINARY); ADD_INT(REG_DWORD); ADD_INT(REG_DWORD_LITTLE_ENDIAN); ADD_INT(REG_DWORD_BIG_ENDIAN); ADD_INT(REG_LINK); ADD_INT(REG_MULTI_SZ); ADD_INT(REG_RESOURCE_LIST); ADD_INT(REG_FULL_RESOURCE_DESCRIPTOR); ADD_INT(REG_RESOURCE_REQUIREMENTS_LIST); } ------=_NextPart_000_0004_01BF98BF.CF4B3050 Content-Type: application/octet-stream; name="pcbuild.dsw" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="pcbuild.dsw" Microsoft Developer Studio Workspace File, Format Version 6.00 # WARNING: DO NOT EDIT OR DELETE THIS WORKSPACE FILE! #########################################################################= ###### Project: "_tkinter"=3D".\_tkinter.dsp" - Package Owner=3D<4> Package=3D<5> {{{ }}} Package=3D<4> {{{ Begin Project Dependency Project_Dep_Name python15 End Project Dependency }}} #########################################################################= ###### Project: "bsddb"=3D".\bsddb.dsp" - Package Owner=3D<4> Package=3D<5> {{{ }}} Package=3D<4> {{{ Begin Project Dependency Project_Dep_Name python15 End Project Dependency }}} #########################################################################= ###### Project: "parser"=3D".\parser.dsp" - Package Owner=3D<4> Package=3D<5> {{{ }}} Package=3D<4> {{{ Begin Project Dependency Project_Dep_Name python15 End Project Dependency }}} #########################################################################= ###### Project: "python"=3D".\python.dsp" - Package Owner=3D<4> Package=3D<5> {{{ }}} Package=3D<4> {{{ Begin Project Dependency Project_Dep_Name python15 End Project Dependency }}} #########################################################################= ###### Project: "python15"=3D".\python15.dsp" - Package Owner=3D<4> Package=3D<5> {{{ }}} Package=3D<4> {{{ }}} #########################################################################= ###### Project: "pythonw"=3D".\pythonw.dsp" - Package Owner=3D<4> Package=3D<5> {{{ }}} Package=3D<4> {{{ Begin Project Dependency Project_Dep_Name python15 End Project Dependency }}} #########################################################################= ###### Project: "winreg"=3D".\winreg.dsp" - Package Owner=3D<4> Package=3D<5> {{{ }}} Package=3D<4> {{{ Begin Project Dependency Project_Dep_Name python15 End Project Dependency }}} #########################################################################= ###### Project: "zlib"=3D".\zlib.dsp" - Package Owner=3D<4> Package=3D<5> {{{ }}} Package=3D<4> {{{ Begin Project Dependency Project_Dep_Name python15 End Project Dependency }}} #########################################################################= ###### Global: Package=3D<5> {{{ }}} Package=3D<3> {{{ }}} #########################################################################= ###### ------=_NextPart_000_0004_01BF98BF.CF4B3050 Content-Type: application/octet-stream; name="winreg.dsp" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="winreg.dsp" # Microsoft Developer Studio Project File - Name=3D"winreg" - Package = Owner=3D<4> # Microsoft Developer Studio Generated Build File, Format Version 6.00 # ** DO NOT EDIT ** # TARGTYPE "Win32 (x86) Dynamic-Link Library" 0x0102 CFG=3Dwinreg - Win32 Debug !MESSAGE This is not a valid makefile. To build this project using = NMAKE, !MESSAGE use the Export Makefile command and run !MESSAGE=20 !MESSAGE NMAKE /f "winreg.mak". !MESSAGE=20 !MESSAGE You can specify a configuration when running NMAKE !MESSAGE by defining the macro CFG on the command line. For example: !MESSAGE=20 !MESSAGE NMAKE /f "winreg.mak" CFG=3D"winreg - Win32 Debug" !MESSAGE=20 !MESSAGE Possible choices for configuration are: !MESSAGE=20 !MESSAGE "winreg - Win32 Release" (based on "Win32 (x86) Dynamic-Link = Library") !MESSAGE "winreg - Win32 Debug" (based on "Win32 (x86) Dynamic-Link = Library") !MESSAGE=20 # Begin Project # PROP AllowPerConfigDependencies 0 # PROP Scc_ProjName "" # PROP Scc_LocalPath "" CPP=3Dcl.exe MTL=3Dmidl.exe RSC=3Drc.exe !IF "$(CFG)" =3D=3D "winreg - Win32 Release" # PROP BASE Use_MFC 0 # PROP BASE Use_Debug_Libraries 0 # PROP BASE Output_Dir "Release" # PROP BASE Intermediate_Dir "Release" # PROP BASE Target_Dir "" # PROP Use_MFC 0 # PROP Use_Debug_Libraries 0 # PROP Output_Dir "." # PROP Intermediate_Dir "x86-temp-release\winreg" # PROP Ignore_Export_Lib 0 # PROP Target_Dir "" F90=3Ddf.exe # ADD BASE CPP /nologo /MT /W3 /GX /O2 /D "WIN32" /D "NDEBUG" /D = "_WINDOWS" /D "_MBCS" /D "_USRDLL" /D "winreg_EXPORTS" /YX /FD /c # ADD CPP /nologo /MD /W3 /GX /O2 /I "..\Include" /I "..\PC" /D "NDEBUG" = /D "WIN32" /D "_WINDOWS" /D "_MBCS" /D "_USRDLL" /YX /FD /c # ADD BASE MTL /nologo /D "NDEBUG" /mktyplib203 /win32 # ADD MTL /nologo /D "NDEBUG" /mktyplib203 /win32 # ADD BASE RSC /l 0xc09 /d "NDEBUG" # ADD RSC /l 0xc09 /d "NDEBUG" BSC32=3Dbscmake.exe # ADD BASE BSC32 /nologo # ADD BSC32 /nologo LINK32=3Dlink.exe # ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib = comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib = odbc32.lib odbccp32.lib /nologo /dll /machine:I386 # ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib = advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib = odbccp32.lib /nologo /base:"0x1e1C0000" /dll /machine:I386 = /out:"winreg.pyd" !ELSEIF "$(CFG)" =3D=3D "winreg - Win32 Debug" # PROP BASE Use_MFC 0 # PROP BASE Use_Debug_Libraries 1 # PROP BASE Output_Dir "Debug" # PROP BASE Intermediate_Dir "Debug" # PROP BASE Target_Dir "" # PROP Use_MFC 0 # PROP Use_Debug_Libraries 1 # PROP Output_Dir "." # PROP Intermediate_Dir "x86-temp-debug\winreg" # PROP Ignore_Export_Lib 0 # PROP Target_Dir "" F90=3Ddf.exe # ADD BASE CPP /nologo /MTd /W3 /Gm /GX /ZI /Od /D "WIN32" /D "_DEBUG" = /D "_WINDOWS" /D "_MBCS" /D "_USRDLL" /D "winreg_EXPORTS" /YX /FD /GZ /c # ADD CPP /nologo /MDd /W3 /Gm /GX /ZI /Od /I "..\Include" /I "..\PC" /D = "_DEBUG" /D "WIN32" /D "_WINDOWS" /D "_MBCS" /D "_USRDLL" /YX /FD /GZ /c # ADD BASE MTL /nologo /D "_DEBUG" /mktyplib203 /win32 # ADD MTL /nologo /D "_DEBUG" /mktyplib203 /win32 # ADD BASE RSC /l 0xc09 /d "_DEBUG" # ADD RSC /l 0xc09 /d "_DEBUG" BSC32=3Dbscmake.exe # ADD BASE BSC32 /nologo # ADD BSC32 /nologo LINK32=3Dlink.exe # ADD BASE LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib = comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib = odbc32.lib odbccp32.lib /nologo /dll /debug /machine:I386 /pdbtype:sept # ADD LINK32 kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib = advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib = odbccp32.lib /nologo /base:"0x1e1C0000" /dll /debug /machine:I386 = /out:"winreg_d.pyd" /pdbtype:sept !ENDIF=20 # Begin Target # Name "winreg - Win32 Release" # Name "winreg - Win32 Debug" # Begin Group "Source Files" # PROP Default_Filter "cpp;c;cxx;rc;def;r;odl;idl;hpj;bat" # Begin Source File SOURCE=3D..\PC\winreg.c !IF "$(CFG)" =3D=3D "winreg - Win32 Release" # ADD CPP /MD !ELSEIF "$(CFG)" =3D=3D "winreg - Win32 Debug" # ADD CPP /MDd !ENDIF=20 # End Source File # End Group # Begin Group "Header Files" # PROP Default_Filter "h;hpp;hxx;hm;inl" # End Group # Begin Group "Resource Files" # PROP Default_Filter "ico;cur;bmp;dlg;rc2;rct;bin;rgs;gif;jpg;jpeg;jpe" # End Group # End Target # End Project ------=_NextPart_000_0004_01BF98BF.CF4B3050 Content-Type: text/plain; name="dllbase_nt.txt" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="dllbase_nt.txt" In Win32, DLL's are "pre-linked" using a specified base address. When the DLL is loaded, an attempt is made to place it at that address. If that address is already in use, a new base address is selected, and the DLL subject to fixups. Apparently, these fixups are very slow, and significant performance gains can be made by selecting a good base address. This document is to allocate base addresses to core Python and Python .PYD files, to give a better change of optimal performance. This base address is passed to the linker using the /BASE command line switch. Python.dll - 1e100000 - 1e180000 (-1) Standard Extension Modules - 1e180000 - 1e200000 "" - bsddb 1e180000 - 1e188000 "" - _tkinter 1e190000 - 1e1A0000 - parser 1e1A0000 - 1e1B0000 - zlib 1e1B0000 - 1e1C0000 - winreg 1e1C0000 - 1e1D0000 Other extension modules - win32api 1e200000 - 1e220000 - win32ras 1e220000 - 1e230000 - win32lz 1e230000 - 1e240000 - timer 1e240000 - 1e250000 - mmapfile 1e250000 - 1e260000 - win32pipe 1e260000 - 1e270000 - avl 1e270000 - 1e270000 - dbhash 1e280000 - 1e290000 - win32net 1e290000 - 1e2A0000 - win32security 1e2A0000 - 1e2B0000 - win32print 1e2B0000 - 1e2c0000 - 1e2d0000 - 1e2e0000 - win32gui 1e2e0000 - 1e2f0000 - _imaging 1e2f0000 - 1e300000 - multiarray 1e300000 - 1e310000 - win32help 1e310000 - 1e320000 - win32clipboard 1e320000 - 1e330000 PythonWin win32ui 1e400000 - 1e500000 ------=_NextPart_000_0004_01BF98BF.CF4B3050 Content-Type: application/octet-stream; name="test_winreg.py" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="test_winreg.py" # Test the windows specific win32reg module. # Only win32reg functions not hit here: FlushKey, LoadKey and SaveKey from winreg import * import os, sys test_key_name =3D "SOFTWARE\\Python Registry Test Key - Delete Me" test_data =3D [ ("Int Value", 45, = REG_DWORD), ("String Val", "A string value", REG_SZ,), (u"Unicode Val", u"A Unicode value", REG_SZ,), ("StringExpand", "The path is %path%", = REG_EXPAND_SZ), ("UnicodeExpand", u"The path is %path%", = REG_EXPAND_SZ), ("Multi-string", ["Lots", "of", "string", "values"], = REG_MULTI_SZ), ("Multi-unicode", [u"Lots", u"of", u"unicode", u"values"], = REG_MULTI_SZ), ("Multi-mixed", [u"Unicode", u"and", "string", = "values"],REG_MULTI_SZ), ("Raw Data", ("binary"+chr(0)+"data"), = REG_BINARY), ] def WriteTestData(root_key): # Set the default value for this key. SetValue(root_key, test_key_name, REG_SZ, "Default value") key =3D CreateKey(root_key, test_key_name) # Create a sub-key sub_key =3D CreateKey(key, "sub_key") # Give the sub-key some named values for value_name, value_data, value_type in test_data: SetValueEx(sub_key, value_name, 0, value_type, value_data) # Check we wrote as many items as we thought. nkeys, nvalues, since_mod =3D QueryInfoKey(key) assert nkeys=3D=3D1, "Not the correct number of sub keys" assert nvalues=3D=3D1, "Not the correct number of values" nkeys, nvalues, since_mod =3D QueryInfoKey(sub_key) assert nkeys=3D=3D0, "Not the correct number of sub keys" assert nvalues=3D=3Dlen(test_data), "Not the correct number of = values" # Close this key this way... # (but before we do, copy the key as an integer - this allows # us to test that the key really gets closed). int_sub_key =3D int(sub_key) CloseKey(sub_key) try: QueryInfoKey(int_sub_key) raise RuntimeError, "It appears the CloseKey() function does not = close the actual key!" except EnvironmentError: pass # ... and close that key that way :-) int_key =3D int(key) key.Close() try: QueryInfoKey(int_key) raise RuntimeError, "It appears the key.Close() function does not = close the actual key!" except EnvironmentError: pass def ReadTestData(root_key): # Check we can get default value for this key. val =3D QueryValue(root_key, test_key_name) assert val=3D=3D"Default value", "Registry didnt give back the = correct value" key =3D OpenKey(root_key, test_key_name) # Read the sub-keys sub_key =3D OpenKey(key, "sub_key") # Check I can enumerate over the values. index =3D 0 while 1: try: data =3D EnumValue(sub_key, index) except EnvironmentError: break assert data in test_data, "didnt read back the correct test = data." index =3D index + 1 assert index=3D=3Dlen(test_data), "Didnt read the correct number of = items" # Check I can directly access each item for value_name, value_data, value_type in test_data: read_val, read_typ =3D QueryValueEx(sub_key, value_name) assert read_val=3D=3Dvalue_data and read_typ =3D=3D value_type, = \ "Could not directly read the value" sub_key.Close() # Enumerate our main key. read_val =3D EnumKey(key, 0) assert read_val =3D=3D "sub_key", "Read subkey value wrong" try: EnumKey(key, 1) assert 0, "Was able to get a second key when I only have one!" except EnvironmentError: pass key.Close() def DeleteTestData(root_key): key =3D OpenKey(root_key, test_key_name, 0, KEY_ALL_ACCESS) sub_key =3D OpenKey(key, "sub_key", 0, KEY_ALL_ACCESS) # It is not necessary to delete the values before deleting # the key (although subkeys must not exist). We delete them # manually just to prove we can :-) for value_name, value_data, value_type in test_data: DeleteValue(sub_key, value_name) nkeys, nvalues, since_mod =3D QueryInfoKey(sub_key) assert nkeys=3D=3D0 and nvalues=3D=3D0, "subkey not empty before = delete" sub_key.Close() DeleteKey(key, "sub_key") try: # Shouldnt be able to delete it twice! DeleteKey(key, "sub_key") assert 0, "Deleting the key twice succeeded" except EnvironmentError: pass key.Close() DeleteKey(root_key, test_key_name) # Opening should now fail! try: key =3D OpenKey(root_key, test_key_name) assert 0, "Could open the non-existant key" except WindowsError: # Use this error name this time pass def TestAll(root_key): WriteTestData(root_key) ReadTestData(root_key) # DeleteTestData(root_key) # Test on my local machine. TestAll(HKEY_CURRENT_USER) print "Local registry tests worked" try: remote_name =3D sys.argv[sys.argv.index("--remote")+1] except (IndexError, ValueError): remote_name =3D None if remote_name is not None: try: remote_key =3D ConnectRegistry(remote_name, HKEY_CURRENT_USER) except EnvironmentError, exc: print "Could not connect to the remote machine -", exc.strerror remote_key =3D None if remote_key is not None: TestAll(remote_key) print "Remote registry tests worked" else: print "Remote registry calls can be tested using", print "'test_winreg.py --remote \\\\machine_name'" ------=_NextPart_000_0004_01BF98BF.CF4B3050 Content-Type: application/octet-stream; name="test_winreg" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="test_winreg" test_winreg Local registry tests worked Remote registry calls can be tested using 'test_winreg.py --remote = \\machine_name' ------=_NextPart_000_0004_01BF98BF.CF4B3050-- From mal@lemburg.com Tue Mar 28 07:16:59 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 Mar 2000 09:16:59 +0200 Subject: [Patches] Unicode changes. References: Message-ID: <38E05C6B.404AC325@lemburg.com> Mark Hammond wrote: > > > Question -- I imagine there might be a use for an MBCS codec outside > > the Windows platform. How hard would it be to add one? > > All it would require is a sys.platform style check in mbcs.py, and the > relevent #defines in unicodeobject.c. Having 2 MBCS codecs on Windows would > probably cause problems :-) You'd only need the relevant #ifdefs in unicodeobject.c. _codecmodule.c interfaces to the C APIs and if those have the same name you'd be set. Are there any sample MBCS encoders/decoders around which we could uses on non-Windows platforms (provided the copyright fits) ? BTW, do you have a pointer to the definition of MBCS -- the M in there sound very much like the M in MS ;-) I would suspect this to be a MS variant of IBM's DBCS. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mal@lemburg.com Tue Mar 28 07:04:25 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 Mar 2000 09:04:25 +0200 Subject: [Patches] Unicode changes. References: Message-ID: <38E05979.7A91B2C8@lemburg.com> Mark Hammond wrote: > > Ive made a few changes to the Unicode stuff, with MA's blessing (assuming > "great!" is a blessing :-) Sure is :-) > getargs.c - Fix typo in error message and silence signed/unsigned warning on > Windows. > > diff -r2.27 getargs.c > 673c673 > < return "(unkown parser marker combination)"; > --- > > return "(unknown parser marker combination)"; > 759c759 > < if (strlen(PyString_AS_STRING(s)) != size) > --- > > if ((int)strlen(PyString_AS_STRING(s)) != size) Ah, yes... looks like I have to tell my fingers once and for all not to forget that 'n' anymore. > unicodeobject.c - Added MBCS codecs. > In attached unicodeobject.c.dif > > unicodeobject.h - Prototypes for MBCS codecs. > In attached unicodeobject.h.dif > > mbcs.py - New file for "lib\encodings" - Description: MBCS codecs for > Windows. For the Python world you'll also need to add the C APIs to Lib/_codecsmodule.c. Never mind I'll add these with the next patch set. > Once these have been accepted I will submit the winreg registry module, > which uses these encodings. I will also submit a patch so open() also takes > advantage of these on Windows. Great :-) -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mhammond@skippinet.com.au Tue Mar 28 09:15:03 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Tue, 28 Mar 2000 19:15:03 +1000 Subject: [Patches] Unicode changes. In-Reply-To: <38E05C6B.404AC325@lemburg.com> Message-ID: > BTW, do you have a pointer to the definition of MBCS -- the > M in there sound very much like the M in MS ;-) Officially, the M is "Multi", as DBCS can sometimes mean more than "double"? The best doc I can find is: http://msdn.microsoft.com/library/backgrnd/html/msdn_mbcssg.htm It makes the vague statement that DBCS is sometimes referred to as MBCS - Ive no idea what the story is :-) Mark. From mal@lemburg.com Tue Mar 28 09:30:11 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 Mar 2000 11:30:11 +0200 Subject: [Patches] Unicode Patch Set 2000-03-28 Message-ID: <38E07BA3.890FBE75@lemburg.com> This is a multi-part message in MIME format. --------------5ABFD5148A2E5499A86B9C08 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit The attached patch set includes a workaround to get Python with Unicode compile on BSDI 4.x (courtesy Thomas Wouters; the cause is a bug in the BSDI wchar.h header file) and Python interfaces for the MBCS codec donated by Mark Hammond. Also included are some minor corrections w/r to the docs of the new "es" and "es#" parser markers (use PyMem_Free() instead of free(); thanks to Mark Hammond for finding these). The unicodedata tests are now in a separate file (test_unicodedata.py) to avoid problems if the module cannot be found. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ --------------5ABFD5148A2E5499A86B9C08 Content-Type: text/plain; charset=iso-8859-1; name="Unicode-Implementation-2000-03-28.patch" Content-Transfer-Encoding: 8bit Content-Disposition: inline; filename="Unicode-Implementation-2000-03-28.patch" diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Include/unicodeobject.h Python+Unicode/Include/unicodeobject.h --- CVS-Python/Include/unicodeobject.h Tue Mar 28 09:19:14 2000 +++ Python+Unicode/Include/unicodeobject.h Tue Mar 28 09:10:22 2000 @@ -82,6 +82,10 @@ #endif #ifdef HAVE_WCHAR_H +/* Work around a cosmetic bug in BSDI 4.x wchar.h; thanks to Thomas Wouters */ +# ifdef _HAVE_BSDI +# include +# endif # include "wchar.h" #endif @@ -562,7 +566,9 @@ ); #ifdef MS_WIN32 + /* --- MBCS codecs for Windows -------------------------------------------- */ + extern DL_IMPORT(PyObject*) PyUnicode_DecodeMBCS( const char *string, /* MBCS encoded string */ int length, /* size of string */ @@ -579,8 +585,8 @@ const char *errors /* error handling */ ); - #endif /* MS_WIN32 */ + /* --- Methods & Slots ---------------------------------------------------- These are capable of handling Unicode objects and strings on input diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/encodings/mbcs.py Python+Unicode/Lib/encodings/mbcs.py --- CVS-Python/Lib/encodings/mbcs.py Tue Mar 28 03:58:50 2000 +++ Python+Unicode/Lib/encodings/mbcs.py Tue Mar 28 08:57:31 2000 @@ -34,4 +34,3 @@ def getregentry(): return (Codec.encode,Codec.decode,StreamReader,StreamWriter) - diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/test/output/test_unicode Python+Unicode/Lib/test/output/test_unicode --- CVS-Python/Lib/test/output/test_unicode Sat Mar 25 11:56:30 2000 +++ Python+Unicode/Lib/test/output/test_unicode Tue Mar 28 11:18:47 2000 @@ -1,4 +1,5 @@ test_unicode Testing Unicode comparisons... done. +Testing Unicode contains method... done. Testing Unicode formatting strings... done. -Testing unicodedata module... done. +Testing builtin codecs... done. diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/test/output/test_unicodedata Python+Unicode/Lib/test/output/test_unicodedata --- CVS-Python/Lib/test/output/test_unicodedata Thu Jan 1 01:00:00 1970 +++ Python+Unicode/Lib/test/output/test_unicodedata Tue Mar 28 11:18:47 2000 @@ -0,0 +1,2 @@ +test_unicodedata +Testing unicodedata module... done. diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/test/test_unicode.py Python+Unicode/Lib/test/test_unicode.py --- CVS-Python/Lib/test/test_unicode.py Sat Mar 25 11:56:30 2000 +++ Python+Unicode/Lib/test/test_unicode.py Sat Mar 25 11:30:52 2000 @@ -1,6 +1,5 @@ """ Test script for the Unicode implementation. - Written by Marc-Andre Lemburg (mal@lemburg.com). (c) Copyright CNRI, All Rights Reserved. NO WARRANTY. @@ -249,50 +248,6 @@ assert u"%(x)s, %(y)s" % {'x':u"abc", 'y':"def"} == u'abc, def' assert u"%(x)s, %(ä)s" % {'x':u"abc", u'ä'.encode('utf-8'):"def"} == u'abc, def' print 'done.' - -# Test Unicode database APIs -try: - import unicodedata -except ImportError: - pass -else: - print 'Testing unicodedata module...', - - assert unicodedata.digit(u'A',None) is None - assert unicodedata.digit(u'9') == 9 - assert unicodedata.digit(u'\u215b',None) is None - assert unicodedata.digit(u'\u2468') == 9 - - assert unicodedata.numeric(u'A',None) is None - assert unicodedata.numeric(u'9') == 9 - assert unicodedata.numeric(u'\u215b') == 0.125 - assert unicodedata.numeric(u'\u2468') == 9.0 - - assert unicodedata.decimal(u'A',None) is None - assert unicodedata.decimal(u'9') == 9 - assert unicodedata.decimal(u'\u215b',None) is None - assert unicodedata.decimal(u'\u2468',None) is None - - assert unicodedata.category(u'\uFFFE') == 'Cn' - assert unicodedata.category(u'a') == 'Ll' - assert unicodedata.category(u'A') == 'Lu' - - assert unicodedata.bidirectional(u'\uFFFE') == '' - assert unicodedata.bidirectional(u' ') == 'WS' - assert unicodedata.bidirectional(u'A') == 'L' - - assert unicodedata.decomposition(u'\uFFFE') == '' - assert unicodedata.decomposition(u'\u00bc') == ' 0031 2044 0034' - - assert unicodedata.mirrored(u'\uFFFE') == 0 - assert unicodedata.mirrored(u'a') == 0 - assert unicodedata.mirrored(u'\u2201') == 1 - - assert unicodedata.combining(u'\uFFFE') == 0 - assert unicodedata.combining(u'a') == 0 - assert unicodedata.combining(u'\u20e1') == 230 - - print 'done.' # Test builtin codecs print 'Testing builtin codecs...', diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/test/test_unicodedata.py Python+Unicode/Lib/test/test_unicodedata.py --- CVS-Python/Lib/test/test_unicodedata.py Thu Jan 1 01:00:00 1970 +++ Python+Unicode/Lib/test/test_unicodedata.py Sat Mar 25 11:31:16 2000 @@ -0,0 +1,50 @@ +""" Test script for the unicodedata module. + +Written by Marc-Andre Lemburg (mal@lemburg.com). + +(c) Copyright CNRI, All Rights Reserved. NO WARRANTY. + +"""#" +from test_support import verbose +import sys + +# Test Unicode database APIs +import unicodedata + +print 'Testing unicodedata module...', + +assert unicodedata.digit(u'A',None) is None +assert unicodedata.digit(u'9') == 9 +assert unicodedata.digit(u'\u215b',None) is None +assert unicodedata.digit(u'\u2468') == 9 + +assert unicodedata.numeric(u'A',None) is None +assert unicodedata.numeric(u'9') == 9 +assert unicodedata.numeric(u'\u215b') == 0.125 +assert unicodedata.numeric(u'\u2468') == 9.0 + +assert unicodedata.decimal(u'A',None) is None +assert unicodedata.decimal(u'9') == 9 +assert unicodedata.decimal(u'\u215b',None) is None +assert unicodedata.decimal(u'\u2468',None) is None + +assert unicodedata.category(u'\uFFFE') == 'Cn' +assert unicodedata.category(u'a') == 'Ll' +assert unicodedata.category(u'A') == 'Lu' + +assert unicodedata.bidirectional(u'\uFFFE') == '' +assert unicodedata.bidirectional(u' ') == 'WS' +assert unicodedata.bidirectional(u'A') == 'L' + +assert unicodedata.decomposition(u'\uFFFE') == '' +assert unicodedata.decomposition(u'\u00bc') == ' 0031 2044 0034' + +assert unicodedata.mirrored(u'\uFFFE') == 0 +assert unicodedata.mirrored(u'a') == 0 +assert unicodedata.mirrored(u'\u2201') == 1 + +assert unicodedata.combining(u'\uFFFE') == 0 +assert unicodedata.combining(u'a') == 0 +assert unicodedata.combining(u'\u20e1') == 230 + +print 'done.' diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Misc/unicode.txt Python+Unicode/Misc/unicode.txt --- CVS-Python/Misc/unicode.txt Sat Mar 25 11:56:31 2000 +++ Python+Unicode/Misc/unicode.txt Tue Mar 28 11:18:46 2000 @@ -740,8 +740,8 @@ On output, a buffer of the needed size is allocated and returned through *buffer as NULL-terminated string. The encoded may not contain embedded NULL characters. - The caller is responsible for free()ing the allocated *buffer - after usage. + The caller is responsible for calling PyMem_Free() + to free the allocated *buffer after usage. "es#": Takes three parameters: encoding (const char *), @@ -755,8 +755,9 @@ If *buffer is NULL, a buffer of the needed size is allocated and output copied into it. *buffer is then - updated to point to the allocated memory area. The caller - is responsible for free()ing *buffer after usage. + updated to point to the allocated memory area. + The caller is responsible for calling PyMem_Free() + to free the allocated *buffer after usage. In both cases *buffer_len is updated to the number of characters written (excluding the trailing NULL-byte). @@ -784,7 +785,7 @@ return NULL; } str = PyString_FromStringAndSize(buffer, buffer_len); - free(buffer); + PyMem_Free(buffer); return str; } @@ -807,7 +808,7 @@ return NULL; } str = PyString_FromString(buffer); - free(buffer); + PyMem_Free(buffer); return str; } diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Modules/_codecsmodule.c Python+Unicode/Modules/_codecsmodule.c --- CVS-Python/Modules/_codecsmodule.c Sat Mar 11 00:09:23 2000 +++ Python+Unicode/Modules/_codecsmodule.c Tue Mar 28 09:12:19 2000 @@ -286,6 +286,26 @@ size); } +#ifdef MS_WIN32 + +static PyObject * +mbcs_decode(PyObject *self, + PyObject *args) +{ + const char *data; + int size; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "t#|z:mbcs_decode", + &data, &size, &errors)) + return NULL; + + return codec_tuple(PyUnicode_DecodeMBCS(data, size, errors), + size); +} + +#endif /* MS_WIN32 */ + /* --- Encoder ------------------------------------------------------------ */ static PyObject * @@ -491,6 +511,28 @@ PyUnicode_GET_SIZE(str)); } +#ifdef MS_WIN32 + +static PyObject * +mbcs_encode(PyObject *self, + PyObject *args) +{ + PyObject *str; + const char *errors = NULL; + + if (!PyArg_ParseTuple(args, "U|z:mbcs_encode", + &str, &errors)) + return NULL; + + return codec_tuple(PyUnicode_EncodeMBCS( + PyUnicode_AS_UNICODE(str), + PyUnicode_GET_SIZE(str), + errors), + PyUnicode_GET_SIZE(str)); +} + +#endif /* MS_WIN32 */ + /* --- Module API --------------------------------------------------------- */ static PyMethodDef _codecs_functions[] = { @@ -519,6 +561,10 @@ {"charmap_decode", charmap_decode, 1}, {"readbuffer_encode", readbuffer_encode, 1}, {"charbuffer_encode", charbuffer_encode, 1}, +#ifdef MS_WIN32 + {"mbcs_encode", mbcs_encode, 1}, + {"mbcs_decode", mbcs_decode, 1}, +#endif {NULL, NULL} /* sentinel */ }; Only in CVS-Python/Objects: .#stringobject.c.2.59 Only in CVS-Python/Objects: stringobject.c.orig diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Python/getargs.c Python+Unicode/Python/getargs.c --- CVS-Python/Python/getargs.c Tue Mar 28 09:19:18 2000 +++ Python+Unicode/Python/getargs.c Tue Mar 28 09:00:34 2000 @@ -704,7 +704,7 @@ the data copied into it; *buffer is updated to point to the new buffer; the caller is responsible for - free()ing it after usage + PyMem_Free()ing it after usage - if *buffer is not NULL, the data is copied to *buffer; *buffer_len @@ -752,7 +752,7 @@ is allocated and the data copied into it; *buffer is updated to point to the new buffer; the caller - is responsible for free()ing it + is responsible for PyMem_Free()ing it after usage */ --------------5ABFD5148A2E5499A86B9C08-- From mal@lemburg.com Tue Mar 28 09:11:30 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 Mar 2000 11:11:30 +0200 Subject: [Patches] Unicode changes. References: Message-ID: <38E07742.6F8E3829@lemburg.com> Mark Hammond wrote: > > > BTW, do you have a pointer to the definition of MBCS -- the > > M in there sound very much like the M in MS ;-) > > Officially, the M is "Multi", as DBCS can sometimes mean more than > "double"? > > The best doc I can find is: > http://msdn.microsoft.com/library/backgrnd/html/msdn_mbcssg.htm > > It makes the vague statement that DBCS is sometimes referred to as > MBCS - Ive no idea what the story is :-) Thanks for the pointer. It seems as if MBCS is really just another name for DBCS (the whole document speaks about DBCS and only mentions MBCS as an alias). If this is true, then I don't know how the MBCS will actually work, since DBCS is a class of encodings, not a single one. Andy Robinson and friends on the i18n sig list are already looking into writing codecs for the various encodings using DBCS as basis. Looks like you just made Python CJK-aware on Windows by virtue of letting the Win32 API decide which the current encoding is ;-). I think I'll post a note on the i18n list to see if this does indeed work. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From piers@cs.su.oz.au Tue Mar 28 10:14:37 2000 From: piers@cs.su.oz.au (Piers Lauder) Date: Tue, 28 Mar 2000 20:14:37 +1000 Subject: [Patches] imaplib.py patch Message-ID: <954239451.183.878479648@cs.usyd.edu.au> --129.78.10.43.126.21197.954239451.188.10387 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi! Reason for patch: This patch fixes the "search" command in imaplib. The problem was that a search can take multiple arguments, but as defined, would only accept one. I have also made changes to the test code at the end to be less verbose by default, but to accept a verbosity argument. I've modified the code for search, and tested it, and here is the resulting "diff -c" patch against the version in the CVS tree as of 1/2 hour ago. Standard disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Piers Lauder. --129.78.10.43.126.21197.954239451.188.10387 Content-Type: application/octet-stream; name=imaplib.diffs Content-Transfer-Encoding: 7bit *** dist/src/Lib/imaplib.py Tue Mar 28 18:52:20 2000 --- piers/python/imaplib.py Tue Mar 28 20:15:06 2000 *************** *** 15,21 **** # # Authentication code contributed by Donn Cave June 1998. ! __version__ = "2.33" import binascii, re, socket, string, time, random, sys --- 15,21 ---- # # Authentication code contributed by Donn Cave June 1998. ! __version__ = "2.36" import binascii, re, socket, string, time, random, sys *************** *** 446,455 **** return self._simple_command('RENAME', oldmailbox, newmailbox) ! def search(self, charset, criteria): """Search mailbox for matching messages. ! (typ, [data]) = .search(charset, criteria) 'data' is space separated list of matching message numbers. """ --- 446,455 ---- return self._simple_command('RENAME', oldmailbox, newmailbox) ! def search(self, charset, *criteria): """Search mailbox for matching messages. ! (typ, [data]) = .search(charset, criterium, ...) 'data' is space separated list of matching message numbers. """ *************** *** 456,462 **** name = 'SEARCH' if charset: charset = 'CHARSET ' + charset ! typ, dat = self._simple_command(name, charset, criteria) return self._untagged_response(typ, dat, name) --- 456,462 ---- name = 'SEARCH' if charset: charset = 'CHARSET ' + charset ! typ, dat = apply(self._simple_command, (name, charset) + criteria) return self._untagged_response(typ, dat, name) *************** *** 1022,1032 **** if __name__ == '__main__': ! import getpass, sys ! host = '' ! if sys.argv[1:]: host = sys.argv[1] USER = getpass.getuser() PASSWD = getpass.getpass("IMAP password for %s on %s" % (USER, host or "localhost")) --- 1022,1042 ---- if __name__ == '__main__': ! import getopt, getpass, sys ! try: ! optlist, args = getopt.getopt(sys.argv[1:], 'd:') ! except getopt.error, val: ! pass + for opt,val in optlist: + if opt == '-d': + Debug = int(val) + + if not args: args = ('',) + + host = args[0] + USER = getpass.getuser() PASSWD = getpass.getpass("IMAP password for %s on %s" % (USER, host or "localhost")) *************** *** 1039,1045 **** ('append', ('/tmp/yyz 2', None, None, test_mesg)), ('list', ('/tmp', 'yy*')), ('select', ('/tmp/yyz 2',)), ! ('search', (None, '(TO zork)')), ('partial', ('1', 'RFC822', 1, 1024)), ('store', ('1', 'FLAGS', '(\Deleted)')), ('expunge', ()), --- 1049,1055 ---- ('append', ('/tmp/yyz 2', None, None, test_mesg)), ('list', ('/tmp', 'yy*')), ('select', ('/tmp/yyz 2',)), ! ('search', (None, 'SUBJECT', 'test')), ('partial', ('1', 'RFC822', 1, 1024)), ('store', ('1', 'FLAGS', '(\Deleted)')), ('expunge', ()), *************** *** 1063,1088 **** _mesg('%s => %s %s' % (cmd, typ, dat)) return dat ! Debug = 5 ! M = IMAP4(host) ! _mesg('PROTOCOL_VERSION = %s' % M.PROTOCOL_VERSION) ! for cmd,args in test_seq1: ! run(cmd, args) ! for ml in run('list', ('/tmp/', 'yy%')): ! mo = re.match(r'.*"([^"]+)"$', ml) ! if mo: path = mo.group(1) ! else: path = string.split(ml)[-1] ! run('delete', (path,)) ! for cmd,args in test_seq2: ! dat = run(cmd, args) ! if (cmd,args) != ('uid', ('SEARCH', 'ALL')): ! continue ! uid = string.split(dat[-1]) ! if not uid: continue ! run('uid', ('FETCH', '%s' % uid[-1], ! '(FLAGS INTERNALDATE RFC822.SIZE RFC822.HEADER RFC822.TEXT)')) --- 1073,1111 ---- _mesg('%s => %s %s' % (cmd, typ, dat)) return dat ! try: ! M = IMAP4(host) ! _mesg('PROTOCOL_VERSION = %s' % M.PROTOCOL_VERSION) ! for cmd,args in test_seq1: ! run(cmd, args) ! for ml in run('list', ('/tmp/', 'yy%')): ! mo = re.match(r'.*"([^"]+)"$', ml) ! if mo: path = mo.group(1) ! else: path = string.split(ml)[-1] ! run('delete', (path,)) ! for cmd,args in test_seq2: ! dat = run(cmd, args) ! if (cmd,args) != ('uid', ('SEARCH', 'ALL')): ! continue ! uid = string.split(dat[-1]) ! if not uid: continue ! run('uid', ('FETCH', '%s' % uid[-1], ! '(FLAGS INTERNALDATE RFC822.SIZE RFC822.HEADER RFC822.TEXT)')) ! ! print '\nAll tests OK.' ! ! except: ! print '\nTests failed.' ! ! if not Debug: ! print ''' ! If you would like to see debugging output, ! try: %s -d5 ! ''' % sys.argv[0] ! ! raise --129.78.10.43.126.21197.954239451.188.10387-- From thomas.heller@ion-tof.com Tue Mar 28 10:41:46 2000 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Tue, 28 Mar 2000 12:41:46 +0200 Subject: [Patches] Unicode changes. References: <38E07742.6F8E3829@lemburg.com> Message-ID: <00f601bf98a2$372e0530$4500a8c0@thomasnotebook> > Thanks for the pointer. It seems as if MBCS is really just > another name for DBCS (the whole document speaks about DBCS > and only mentions MBCS as an alias). > > If this is true, then I don't know how the MBCS will actually > work, since DBCS is a class of encodings, not a single one. > Andy Robinson and friends on the i18n sig list are already > looking into writing codecs for the various encodings using > DBCS as basis. > > Looks like you just made Python CJK-aware on Windows by virtue > of letting the Win32 API decide which the current encoding is ;-). > Is this the answer (from the article mentioned above)? 14. Tips 14.1. Development Don't mix and match Win32® APIs with CRT APIs The difference is that Win32 APIs rely on System information, whereas the CRT APIs rely on the user to initialize for the appropriate settings. CRT defaults to ANSI "C" locale. For example: The following example will fail even if psz really points to a Japanese lead byte and the system is running in a different codepage other than 932. // **** undefined behaviour **** setlocale(LC_ALL, "Japanese"); // set run-time to Japanese locale if (IsDBCSLeadByte(*psz)) // query system locale *** wrong *** .... // **** correct behaviour **** if (isleadbyte((_TXCHAR)*psz)) // correct locale is used. Also note that // (_TXCHAR) casting was used to make sure // integral conversion is correct Thomas Heller From thomas@xs4all.net Tue Mar 28 12:07:22 2000 From: thomas@xs4all.net (Thomas Wouters) Date: Tue, 28 Mar 2000 14:07:22 +0200 Subject: [Patches] mailbox.UnixMailbox and rfc822.Message.unixfrom In-Reply-To: <200003272048.PAA19149@eric.cnri.reston.va.us>; from guido@python.org on Mon, Mar 27, 2000 at 03:48:52PM -0500 References: <20000324214124.B12922@xs4all.nl> <200003272048.PAA19149@eric.cnri.reston.va.us> Message-ID: <20000328140722.F21895@xs4all.nl> On Mon, Mar 27, 2000 at 03:48:52PM -0500, Guido van Rossum wrote: > The unixfrom feature of the rfc822 module is at best questionable -- > it it not part of the mail headers as defined by RFC-822! I concur, but the reality is that the rfc822 modules *does* support the unix mailbox added feature 'unixfrom', whereas the unix mailbox class in the mailbox module *doesn't*. That's what the patch remidies. The problem is that the unixfrom line holds information that can't be reproduced from the other headers, currently. The address in the unixfrom line is not necessarily the same as the one in the From: header, and the date in the From line is the date the message was received on the local machine, in the local timezone. I'd be prefectly happy with subclassing rfc822.Message in mailbox and moving the 'unixfrom' support there (marking the rfc822.Message.unixfrom attribute deprecated) since it does make more sense, but I'm not sure if it's worth the effort. I have a gut feeling there is a lot of code out there that uses rfc822.Message.unixfrom. > Why do you need access the Unix from header through the mailbox > module? I thought Mailman needed it. Mailman has a situation where a message gets dumped to disk, using 'str(msg)', and it lost the unixfrom line in the process. I tried to reproduce it by opening one of my own unix mailboxes, and noticed that unixfrom wasn't begin set. The problem in mailman was unrelated, but I still think the mailbox module should keep the fromline. I could write myself a script that needs it if you wish, though :-) The point is that the module does a fair job of representing a mailbox, but doesn't quite make it to the finish line in this case. The problem as I see it is this: the unixfrom line is an important part of the UnixMailbox. Currently, after you read in the unix mailbox via the mailbox module, you *can't* recreate it by appending the individual messages. The unixfrom lines dont get stored anywhere in the mailbox module, either. You have to kludge around, faking the unixfrom line by using (possibly) the wrong addresses and the wrong date. (The address problem could possibly be solved by transforming the unixfrom line into a Return-Path header, like sendmail does internally, but in all honesty I'd find that a waste of the unixfrom support in rfc822, misplaced or not.) > Your patch worries me: the dependency between _search_start() and > next() through the _skipfirst variable begs for problems in > subclasses. And there I was, thinking it was an elegant solution to a tricky problem ;) The only other solution I could see was overriding next entirely, in UnixMailbox, or perhaps rewriting the msg search/skip methods... What problems do you forsee, in subclasses ? I must admit I dont really see what kind of problems it would cause, though I admit a warning/description at the variable declaration might be in place. -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@python.org Tue Mar 28 13:56:00 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 28 Mar 2000 08:56:00 -0500 Subject: [Patches] mailbox.UnixMailbox and rfc822.Message.unixfrom In-Reply-To: Your message of "Tue, 28 Mar 2000 14:07:22 +0200." <20000328140722.F21895@xs4all.nl> References: <20000324214124.B12922@xs4all.nl> <200003272048.PAA19149@eric.cnri.reston.va.us> <20000328140722.F21895@xs4all.nl> Message-ID: <200003281356.IAA29608@eric.cnri.reston.va.us> > I have a gut feeling there is a lot of code out there that uses > rfc822.Message.unixfrom. My gut feels the reverse. You are the first to ask about this in the lifetime of the mailbox module, and you didn't even really need it, you only thought you did. I personally process approximately 200 messages a day using MH, which never looks at the Unix From line. > > Why do you need access the Unix from header through the mailbox > > module? > > I thought Mailman needed it. [But it doesn't] > The problem as I see it is this: the unixfrom line is an important part of > the UnixMailbox. Currently, after you read in the unix mailbox via the > mailbox module, you *can't* recreate it by appending the individual > messages. The unixfrom lines dont get stored anywhere in the mailbox module, > either. You have to kludge around, faking the unixfrom line by using > (possibly) the wrong addresses and the wrong date. The mailbox module supports several other mailbox formats which don't have a "Unix From line" either. It is intended to provide a uniform API for reading mailboxes. Features specific to mailbox formats don't belong in this API. > (The address problem could possibly be solved by transforming the unixfrom > line into a Return-Path header, like sendmail does internally, but in all > honesty I'd find that a waste of the unixfrom support in rfc822, misplaced > or not.) The unixfrom feature was added to rfc822 because people kept misusing it. :-) > > Your patch worries me: the dependency between _search_start() and > > next() through the _skipfirst variable begs for problems in > > subclasses. > > And there I was, thinking it was an elegant solution to a tricky problem ;) > The only other solution I could see was overriding next entirely, in > UnixMailbox, or perhaps rewriting the msg search/skip methods... What > problems do you forsee, in subclasses ? I must admit I dont really see what > kind of problems it would cause, though I admit a warning/description at the > variable declaration might be in place. Well, since _search_end() is always called after a successful _search_start(), you could have solved the problem locally in _search_end(). --Guido van Rossum (home page: http://www.python.org/~guido/) From mal@lemburg.com Tue Mar 28 12:48:49 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Tue, 28 Mar 2000 14:48:49 +0200 Subject: [Patches] Unicode changes. References: <38E07742.6F8E3829@lemburg.com> <00f601bf98a2$372e0530$4500a8c0@thomasnotebook> Message-ID: <38E0AA31.D7CEB112@lemburg.com> Thomas Heller wrote: > > > Thanks for the pointer. It seems as if MBCS is really just > > another name for DBCS (the whole document speaks about DBCS > > and only mentions MBCS as an alias). > > > > If this is true, then I don't know how the MBCS will actually > > work, since DBCS is a class of encodings, not a single one. > > Andy Robinson and friends on the i18n sig list are already > > looking into writing codecs for the various encodings using > > DBCS as basis. > > > > Looks like you just made Python CJK-aware on Windows by virtue > > of letting the Win32 API decide which the current encoding is ;-). > > > > Is this the answer (from the article mentioned above)? > 14. Tips > 14.1. Development > Don't mix and match Win32® APIs with CRT APIs > The difference is that Win32 APIs rely on System information, whereas the CRT APIs rely on the user to initialize for the > appropriate settings. CRT defaults to ANSI "C" locale. > > For example: The following example will fail even if psz really points to a Japanese lead byte and the system is running in a > different codepage other than 932. > > // **** undefined behaviour **** > setlocale(LC_ALL, "Japanese"); // set run-time to Japanese locale > if (IsDBCSLeadByte(*psz)) // query system locale *** wrong *** > .... > // **** correct behaviour **** > if (isleadbyte((_TXCHAR)*psz)) // correct locale is used. Also note that > // (_TXCHAR) casting was used to make sure > // integral conversion is correct Hmm, Mark's mbcs codec uses the Win32 API, so if the user set up the system to a CJK code page, then that code page will be used... not really portable, but we have Unicode for that anyway :-) On the positive side, the system will automagically use the right encoding for the user's machine, on the negative side a Python script cannot use mbcs codec to encode data using a predefined code page: the data is probably only usable on his/her machine. The Asian codec package will take care of the latter though, so its not as bad as it may look. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From skip@mojam.com (Skip Montanaro) Tue Mar 28 14:36:57 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 28 Mar 2000 08:36:57 -0600 (CST) Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: <200003272216.RAA28884@eric.cnri.reston.va.us> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> <200003272142.QAA25914@eric.cnri.reston.va.us> <14559.56317.522468.658667@beluga.mojam.com> <200003272216.RAA28884@eric.cnri.reston.va.us> Message-ID: <14560.50057.179322.106312@beluga.mojam.com> Guido> Hm... it seems that there should be an architecture for Guido> providing alternatives to the socket module. I've never even Guido> heard of the timeout_socket module... When the Vaults of Parnassus are back online, check there. The timeout_socket module, as its name implies, allows you to specify timeouts for sockets. I find it quite useful for my Gig Gopher (software that peeks at a few thousand registered sites on a regular basis looking for new concert dates). Guido> Hacking httplib seems the wrong place to approach this. What are Guido> the requirements? I don't really hack httplib, just subclass httplib.HTTP and override the connect method. Here's all I did (minus the docstring): class HTTP(httplib.HTTP): def connect(self, host, port = 0): if not port: i = string.find(host, ':') if i >= 0: host, port = host[:i], host[i+1:] try: port = string.atoi(port) except string.atoi_error: raise ValueError, "nonnumeric port" if not port: port = httplib.HTTP_PORT self.sock = timeout_socket.timeout_socket(timeout=60) if self.debuglevel > 0: print 'connect:', (host, port) self.sock.connect(host, port) I suppose I should have added a timeout parameter to connect(). Timeout_socket.py isn't very big. It's ideas could probably be incorporated directly into the socket module without too much effort. (Taking things a bit further afield, the one that's really nasty to subclass is urllib.URLopener - subclassing it to use the above version of HTTP. That's where some rearchitecting would be useful. You need to paste 50+ lines from the open_http method to make the one-line change to replace httplib.HTTP with the subclass because the open_http method explicitly calls for httplib.HTTP. That's a fair bit of coupling between the original and the subclass. Overriding the open_ftp method would be similarly painful.) [ ... snip stuff about: * ValueError * example of similar history with the os module * proposed one-line change to httplib ... ] Guido> And all this because you believe that raising some other module's Guido> error is a sin? I don't have a problem with such behavior -- Guido> what's the reason? Well, let's see: 1. that httplib can raise socket.error is not documented. (this can, of course, be remedied.) 2. what about the situation where we rearchitect things to allow easy use of alternatives to the socket module? 3. it's not even an exception raised by the socket module that's getting passed on to the user. My main problem is that it's a misrepresentation of the facts. I'd worry less if it was actually an exception raised by the socket module that was either passing through httplib or being reraised by it. Do what you like. If you decide to make a change, note that smtplib also suffers from the same problem. Skip From jeremy@cnri.reston.va.us Tue Mar 28 17:18:59 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Tue, 28 Mar 2000 12:18:59 -0500 (EST) Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: <200003272216.RAA28884@eric.cnri.reston.va.us> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> <200003272142.QAA25914@eric.cnri.reston.va.us> <14559.56317.522468.658667@beluga.mojam.com> <200003272216.RAA28884@eric.cnri.reston.va.us> Message-ID: <14560.59779.535533.973833@goon.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: >> As a compromise, I propose the following change near the top of >> httplib somewhere after socket has been imported: >> >> error = socket.error >> >> Then anywhere httplib raises an error, that should be what it >> raises. No code that currently catches socket.error will break, >> and httplib.error can be the documented exception used for >> transmitting errors out of that module. GvR> And all this because you believe that raising some other GvR> module's error is a sin? I don't have a problem with such GvR> behavior -- what's the reason? I'm not completely opposed to one module raising an exception defined in another, although I think I share some of Skip's worries. First, the documentation ought to describe the exceptions that can be raised. Documenting the intended exceptions is nearly as important as documenting the return value. It serves the dual purpose of describing the contract between caller and callee and helping the programmer figure out when the bug is in the library and when the bug is in her code. If you call a function that raises an undocument ValueError, then there's a good chance there is a bug in the code. Second, there are cases where module A raising B.error seems wrong. In the case of httplib, the HTTP object is an abstraction that hides the details of mucking with sockets to talk HTTP. The socket error exception exposes the representation of the HTTP object, and forces the programmer to import socket & httplib in order to write robust httplib code. Skip's suggestion to httplib.error == socket.error helps in that regard. +0 Jeremy From guido@python.org Tue Mar 28 20:15:10 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 28 Mar 2000 15:15:10 -0500 Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: Your message of "Tue, 28 Mar 2000 12:18:59 EST." <14560.59779.535533.973833@goon.cnri.reston.va.us> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> <200003272142.QAA25914@eric.cnri.reston.va.us> <14559.56317.522468.658667@beluga.mojam.com> <200003272216.RAA28884@eric.cnri.reston.va.us> <14560.59779.535533.973833@goon.cnri.reston.va.us> Message-ID: <200003282015.PAA12365@eric.cnri.reston.va.us> [Skip] > >> As a compromise, I propose the following change near the top of > >> httplib somewhere after socket has been imported: > >> > >> error = socket.error > >> > >> Then anywhere httplib raises an error, that should be what it > >> raises. No code that currently catches socket.error will break, > >> and httplib.error can be the documented exception used for > >> transmitting errors out of that module. > > GvR> And all this because you believe that raising some other > GvR> module's error is a sin? I don't have a problem with such > GvR> behavior -- what's the reason? [Jeremy] > I'm not completely opposed to one module raising an exception defined > in another, although I think I share some of Skip's worries. > > First, the documentation ought to describe the exceptions that can be > raised. Documenting the intended exceptions is nearly as important as > documenting the return value. It serves the dual purpose of > describing the contract between caller and callee and helping the > programmer figure out when the bug is in the library and when the bug > is in her code. If you call a function that raises an undocument > ValueError, then there's a good chance there is a bug in the code. > > Second, there are cases where module A raising B.error seems wrong. > In the case of httplib, the HTTP object is an abstraction that hides > the details of mucking with sockets to talk HTTP. The socket error > exception exposes the representation of the HTTP object, and forces > the programmer to import socket & httplib in order to write robust > httplib code. Skip's suggestion to httplib.error == socket.error > helps in that regard. > > +0 > > Jeremy If it's a documentation issue, so fix the documentation. Although it is apparently not obvious, the HTTP class raises socket.error for lots of reasons -- invalid arguments being only a subclass. This is in its nature, because all it does is make socket calls to implement the HTTP protocol. For this reason the one argument validity check that is done explicitly in the HTTP code is also made to raise socket.error. Adding httplib.error = socket.error seems like splitting hairs, and seems to suggest a future migration path that I don't like (it suggests that in the future httplib.error might *not* be socket.error, and that it's wrong to catch socket.error when using httplib). --Guido van Rossum (home page: http://www.python.org/~guido/) From jeremy@cnri.reston.va.us Tue Mar 28 20:17:11 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Tue, 28 Mar 2000 15:17:11 -0500 (EST) Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: <200003282015.PAA12365@eric.cnri.reston.va.us> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> <200003272142.QAA25914@eric.cnri.reston.va.us> <14559.56317.522468.658667@beluga.mojam.com> <200003272216.RAA28884@eric.cnri.reston.va.us> <14560.59779.535533.973833@goon.cnri.reston.va.us> <200003282015.PAA12365@eric.cnri.reston.va.us> Message-ID: <14561.4935.299987.449333@goon.cnri.reston.va.us> >>>>> "GvR" == Guido van Rossum writes: GvR> If it's a documentation issue, so fix the documentation. Will work on it (unless Skip beats me to it). GvR> Although it is apparently not obvious, the HTTP class raises GvR> socket.error for lots of reasons -- invalid arguments being GvR> only a subclass. This is in its nature, because all it does is GvR> make socket calls to implement the HTTP protocol. GvR> For this reason the one argument validity check that is done GvR> explicitly in the HTTP code is also made to raise socket.error. That makes sense. GvR> Adding httplib.error = socket.error seems like splitting hairs, GvR> and seems to suggest a future migration path that I don't like GvR> (it suggests that in the future httplib.error might *not* be GvR> socket.error, and that it's wrong to catch socket.error when GvR> using httplib). I don't agree. It's merely a convenience for programmers; you can catch httplib.error or import.error. I don't feel strongly about this issue, but I think the change is reasonable. Jeremy From skip@mojam.com (Skip Montanaro) Tue Mar 28 20:59:17 2000 From: skip@mojam.com (Skip Montanaro) (Skip Montanaro) Date: Tue, 28 Mar 2000 14:59:17 -0600 (CST) Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: <14561.4935.299987.449333@goon.cnri.reston.va.us> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> <200003272142.QAA25914@eric.cnri.reston.va.us> <14559.56317.522468.658667@beluga.mojam.com> <200003272216.RAA28884@eric.cnri.reston.va.us> <14560.59779.535533.973833@goon.cnri.reston.va.us> <200003282015.PAA12365@eric.cnri.reston.va.us> <14561.4935.299987.449333@goon.cnri.reston.va.us> Message-ID: <14561.7461.640871.506561@beluga.mojam.com> GvR> If it's a documentation issue, so fix the documentation. Jeremy> Will work on it (unless Skip beats me to it). I hate to drag Fred into this, but Fred, is there a correct or preferred way to document the exceptions a module raises (or passes through from lower level modules)? Is this simply supposed to be documented in the individual function/method description? Skip From fdrake@acm.org Tue Mar 28 21:11:09 2000 From: fdrake@acm.org (Fred L. Drake, Jr.) Date: Tue, 28 Mar 2000 16:11:09 -0500 (EST) Subject: [Patches] httplib.HTTP.connect raises wrong exception In-Reply-To: <14561.7461.640871.506561@beluga.mojam.com> References: <200003101757.LAA24297@beluga.mojam.com> <200003271922.OAA27844@eric.cnri.reston.va.us> <14559.47789.120873.716707@beluga.mojam.com> <200003272142.QAA25914@eric.cnri.reston.va.us> <14559.56317.522468.658667@beluga.mojam.com> <200003272216.RAA28884@eric.cnri.reston.va.us> <14560.59779.535533.973833@goon.cnri.reston.va.us> <200003282015.PAA12365@eric.cnri.reston.va.us> <14561.4935.299987.449333@goon.cnri.reston.va.us> <14561.7461.640871.506561@beluga.mojam.com> Message-ID: <14561.8173.595087.159431@seahag.cnri.reston.va.us> Skip Montanaro writes: > I hate to drag Fred into this, but Fred, is there a correct or preferred way > to document the exceptions a module raises (or passes through from lower > level modules)? Is this simply supposed to be documented in the individual This is strange; I just got a similar question over the cubicle wall. Hmm.. must be a conspiracy. I'm not tellin'! ;) At this time, no, just explain in the text and assume it *will* change when we migrate to SGML, but I haven't decided what it will look like yet. I don't know how wise it would be to make extensive changes without some form of explicit markup that we can at least convert & extract. (That could create a requirement for a lot of human intervention to complete the conversion.) Per my documentation plan sent to the Doc-SIG yesterday, I expect the 1.6 docs to be generated from the LaTeX due to the accelerated schedule. -Fred -- Fred L. Drake, Jr. Corporation for National Research Initiatives From Fredrik Lundh" Message-ID: <00e601bf98f7$2623fce0$34aab5d4@hagrid> This is a multi-part message in MIME format. ------=_NextPart_000_00B7_01BF9906.249D3800 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable the new filecmp module has an optional argument called use_statcache which is documented as a true/false value, but used as an tuple index. this patches replaces the tuple stuff with a good old if- statement, and also removes a few other tuple pack/unpack constructs (if not else, this saves a few bytes in the PYC file, and a few microseconds when using the module ;-) I confirm that, to the best of my knowledge and belief, this = contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related = documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. =20 I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ------=_NextPart_000_00B7_01BF9906.249D3800 Content-Type: application/octet-stream; name="filecmp-patch-1" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="filecmp-patch-1" --- filecmp.py.bak Tue Mar 28 22:24:29 2000 +++ filecmp.py Tue Mar 28 22:26:51 2000 @@ -30,14 +30,21 @@ and the cache will never grow stale. """ - stat_function = (os.stat, statcache.stat)[use_statcache] - s1, s2 = _sig(stat_function(f1)), _sig(stat_function(f2)) - if s1[0]!=stat.S_IFREG or s2[0]!=stat.S_IFREG: return 0 - if shallow and s1 == s2: return 1 - if s1[1]!=s2[1]: return 0 + if use_statcache: + stat_function = statcache.stat + else: + stat_function = os.stat + s1 = _sig(stat_function(f1)) + s2 = _sig(stat_function(f2)) + if s1[0] != stat.S_IFREG or s2[0] != stat.S_IFREG: + return 0 + if shallow and s1 == s2: + return 1 + if s1[1] != s2[1]: + return 0 result = _cache.get((f1, f2)) - if result and (s1, s2)==result[:2]: + if result and (s1, s2) == result[:2]: return result[2] outcome = _do_cmp(f1, f2) _cache[f1, f2] = s1, s2, outcome @@ -50,8 +57,12 @@ def _do_cmp(f1, f2): bufsize = BUFSIZE - fp1 , fp2 = open(f1, 'rb'), open(f2, 'rb') + fp1 = open(f1, 'rb') + fp2 = open(f2, 'rb') while 1: - b1, b2 = fp1.read(bufsize), fp2.read(bufsize) - if b1!=b2: return 0 - if not b1: return 1 + b1 = fp1.read(bufsize) + b2 = fp2.read(bufsize) + if b1 != b2: + return 0 + if not b1: + return 1 ------=_NextPart_000_00B7_01BF9906.249D3800-- From Fredrik Lundh" This is a multi-part message in MIME format. ------=_NextPart_000_00E0_01BF9907.91498AC0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable this fixes a bunch of socket.connect(host, post) calls. note that I haven't tested all modules -- I don't have enough servers here... I confirm that, to the best of my knowledge and belief, this = contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related = documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. =20 I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. ------=_NextPart_000_00E0_01BF9907.91498AC0 Content-Type: application/octet-stream; name="connect-patch-1" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="connect-patch-1" diff -u bak/ftplib.py ./ftplib.py --- bak/ftplib.py Wed Mar 01 08:03:23 2000 +++ ./ftplib.py Tue Mar 28 22:42:43 2000 @@ -115,7 +115,7 @@ if port: self.port = port self.passiveserver = 0 self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.sock.connect(self.host, self.port) + self.sock.connect((self.host, self.port)) self.file = self.sock.makefile('rb') self.welcome = self.getresp() return self.welcome @@ -265,7 +265,7 @@ if self.passiveserver: host, port = parse227(self.sendcmd('PASV')) conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - conn.connect(host, port) + conn.connect((host, port)) resp = self.sendcmd(cmd) if resp[0] <> '1': raise error_reply, resp diff -u bak/gopherlib.py ./gopherlib.py --- bak/gopherlib.py Wed Mar 01 08:03:23 2000 +++ ./gopherlib.py Tue Mar 28 22:42:47 2000 @@ -66,7 +66,7 @@ elif type(port) == type(''): port = string.atoi(port) s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - s.connect(host, port) + s.connect((host, port)) s.send(selector + CRLF) s.shutdown(1) return s.makefile('rb') diff -u bak/httplib.py ./httplib.py --- bak/httplib.py Wed Mar 01 08:03:23 2000 +++ ./httplib.py Tue Mar 28 22:42:57 2000 @@ -109,7 +109,7 @@ if not port: port = HTTP_PORT self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) if self.debuglevel > 0: print 'connect:', (host, port) - self.sock.connect(host, port) + self.sock.connect((host, port)) def send(self, str): """Send `str' to the server.""" @@ -209,7 +209,7 @@ if not port: port = HTTPS_PORT sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) if self.debuglevel > 0: print 'connect:', (host, port) - sock.connect(host, port) + sock.connect((host, port)) ssl = socket.ssl(sock, self.key_file, self.cert_file) self.sock = FakeSocket(sock, ssl) diff -u bak/imaplib.py ./imaplib.py --- bak/imaplib.py Tue Mar 28 22:09:29 2000 +++ ./imaplib.py Tue Mar 28 22:43:05 2000 @@ -191,7 +191,7 @@ def open(self, host, port): """Setup 'self.sock' and 'self.file'.""" self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.sock.connect(self.host, self.port) + self.sock.connect((self.host, self.port)) self.file = self.sock.makefile('r') diff -u bak/nntplib.py ./nntplib.py --- bak/nntplib.py Wed Mar 01 08:03:23 2000 +++ ./nntplib.py Tue Mar 28 22:43:09 2000 @@ -108,7 +108,7 @@ self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.sock.connect(self.host, self.port) + self.sock.connect((self.host, self.port)) self.file = self.sock.makefile('rb') self.debugging = 0 self.welcome = self.getresp() diff -u bak/poplib.py ./poplib.py --- bak/poplib.py Wed Mar 01 08:03:25 2000 +++ ./poplib.py Tue Mar 28 22:43:13 2000 @@ -77,7 +77,7 @@ self.host = host self.port = port self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - self.sock.connect(self.host, self.port) + self.sock.connect((self.host, self.port)) self.file = self.sock.makefile('rb') self._debugging = 0 self.welcome = self._getresp() --- bak/smtplib.py Wed Mar 01 08:03:25 2000 +++ ./smtplib.py Tue Mar 28 22:43:21 2000 @@ -213,7 +213,7 @@ if not port: port = SMTP_PORT self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) if self.debuglevel > 0: print 'connect:', (host, port) - self.sock.connect(host, port) + self.sock.connect((host, port)) (code,msg)=self.getreply() if self.debuglevel >0 : print "connect:", msg return (code,msg) ------=_NextPart_000_00E0_01BF9907.91498AC0-- From Moshe Zadka Tue Mar 28 23:15:46 2000 From: Moshe Zadka (Moshe Zadka) Date: Wed, 29 Mar 2000 01:15:46 +0200 (IST) Subject: [Patches] minor filecmp.py patch In-Reply-To: <00e601bf98f7$2623fce0$34aab5d4@hagrid> Message-ID: On Tue, 28 Mar 2000, Fredrik Lundh wrote: > the new filecmp module has an optional argument called > use_statcache which is documented as a true/false value, > but used as an tuple index. > > this patches replaces the tuple stuff with a good old if- > statement, and also removes a few other tuple pack/unpack > constructs (if not else, this saves a few bytes in the PYC file, > and a few microseconds when using the module ;-) +1 As the original offender, I wish to thank Frerik for cleaning up my code From mhammond@skippinet.com.au Tue Mar 28 23:24:13 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Wed, 29 Mar 2000 09:24:13 +1000 Subject: [Patches] Discuss: Unicode patch for open() builtin. Message-ID: Not a patch yet, but Im working towards one :) Im thinking about how to change open() WRT unicode. First thought was that the change would be to a PyArg_ParseTuple() call, but inspection shows a PyFile_FromString(char *name, ...). So it makes sense to create a new API - PyFile_FromWCHAR()? On Windows, Im thinking that rather than performing a conversion to MBCS, this could perform a runtime check for WCHAR CRTL support - ie, basically "am I running NT", and call fopenW(). This could be a global static, so the first check would be "expensive", but after that we have a single "if" test. The bltinmodule call to this function can simply check the type of the arg and make the appropriate call (fopen() or fopenW()). (Note that on Windows 95/98 the W calls exist, they just dont work! This is by design, and designed for exactly this case) Something like: #ifdef HAVE_WCHAR_H PyObject PyFile_FromWCHAR(WCHAR *fname, ...) { ... #ifdef MS_WIN32 static int have_wchar_crt = -1; if (have_wchar_crt==-1) have_wchar_crt = AmIRunningNT() ? 1 : 0; if (have_wchar_crt) fp = fopenW(fname, ...); else { /* encode as MBCS */ fp = fopen(mbcs_name, ...); } ... #endif } #endif Sound reasonable? Mark. From guido@python.org Tue Mar 28 23:59:32 2000 From: guido@python.org (Guido van Rossum) Date: Tue, 28 Mar 2000 18:59:32 -0500 Subject: [Patches] Re: Using Tcl_Obj in _tkinter In-Reply-To: Your message of "Mon, 27 Mar 2000 22:39:39 +0200." <200003272039.WAA00923@loewis.home.cs.tu-berlin.de> References: <200003112035.VAA07296@loewis.home.cs.tu-berlin.de> <200003241440.JAA14924@eric.cnri.reston.va.us> <200003250804.JAA01067@loewis.home.cs.tu-berlin.de> <200003272004.PAA12791@eric.cnri.reston.va.us> <200003272039.WAA00923@loewis.home.cs.tu-berlin.de> Message-ID: <200003282359.SAA16010@eric.cnri.reston.va.us> Martin, Check out the current CVS version. It builds and links and seems to work with Tcl/Tk 8.0, 8.1, 8.2 and 8.3 on Solaris; with 8.1, 8.2 and 8.3 it does Unicode. It doesn't have your TclObj patch, but it does call Tcl_FindExecutable(). I removed the TclpInitLibraryPath() call, as it doesn't seem to be needed and it's clearly internal. (I'm still working on getting it to run on Windows, mostly a matter of waiting for my poor 100 MHz pentium to download, compile, install etc.) I'm adding in the TclObj part of your patch. I also added an extension to your patch, which uses Tcl_NewLongObj and Tcl_NewDoubleObj when the Python object is an int or float. Always skeptical, I decided to time it. I have a test program "Plot test lines" by Gordon Williams, which draws a sine wave using about 2000 straight line segments. I found that apparently Tcl/Tk 8.3 is a lot slower than previous versions! Here's what I tried (on Solaris 2.7): - Python 1.5.2 + Tcl 8.0.4 0.48 sec - Python 1.6 + Tcl 8.0.4 0.52 sec - Python 1.6 + Tcl 8.1.1 0.51 sec - Python 1.6 + Tcl 8.2.3 0.52 sec - Python 1.6 + Tcl 8.2.3 + TclObj patch 0.63 sec - Python 1.6 + Tcl 8.2.3 + extended TclObj patch 0.47 sec - Python 1.6 + Tcl 8.3.0 0.78 sec - Python 1.6 + Tcl 8.3.0 + TclObj patch 0.73 sec - Python 1.6 + Tcl 8.3.0 + extended TclObj patch 0.58 sec Aargh! I can understand that Python 1.6 added a few milliseconds; but why is Tcl 8.3 50% slower than previous versions?!?! Also, your original version of the TclObj patch seems to be slower with Tcl 8.2 but faster with Tcl 8.3. I don't understand this... Still, the conclusion seems to be that the extended version of the TclObj patch (with NewLong and NewFloat special cases added) might be the best thing all around -- not too much slower with Tcl/Tk 8.3, and actually slightly faster with Tcl/Tk 8.2. I'll do this now; if there are problems, we'll sort them out before the alpha is released. --Guido van Rossum (home page: http://www.python.org/~guido/) From bwarsaw@cnri.reston.va.us Wed Mar 29 05:42:34 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Date: Wed, 29 Mar 2000 00:42:34 -0500 (EST) Subject: [Patches] Re: [Python-Dev] yeah! for Jeremy and Greg References: <14561.19438.157799.810802@goon.cnri.reston.va.us> <14561.21075.637108.322536@anthem.cnri.reston.va.us> Message-ID: <14561.38858.41246.28460@anthem.cnri.reston.va.us> >>>>> "BAW" == Barry A Warsaw writes: BAW> Uh oh. Fresh CVS update and make clean, make: >>> sum(*n) | Traceback (innermost last): | File "", line 1, in ? | SystemError: bad argument to internal function Here's a proposed patch that will cause a TypeError to be raised instead. -Barry -------------------- snip snip -------------------- Index: abstract.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Objects/abstract.c,v retrieving revision 2.33 diff -c -r2.33 abstract.c *** abstract.c 2000/03/10 22:55:18 2.33 --- abstract.c 2000/03/29 05:36:21 *************** *** 860,866 **** PyObject *s; { PySequenceMethods *m; ! if (s == NULL) { null_error(); return -1; --- 860,867 ---- PyObject *s; { PySequenceMethods *m; ! int size = -1; ! if (s == NULL) { null_error(); return -1; *************** *** 868,877 **** m = s->ob_type->tp_as_sequence; if (m && m->sq_length) ! return m->sq_length(s); ! type_error("len() of unsized object"); ! return -1; } PyObject * --- 869,879 ---- m = s->ob_type->tp_as_sequence; if (m && m->sq_length) ! size = m->sq_length(s); ! if (size < 0) ! type_error("len() of unsized object"); ! return size; } PyObject * Index: ceval.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Python/ceval.c,v retrieving revision 2.169 diff -c -r2.169 ceval.c *** ceval.c 2000/03/28 23:49:16 2.169 --- ceval.c 2000/03/29 05:39:00 *************** *** 1636,1641 **** --- 1636,1649 ---- break; } nstar = PySequence_Length(stararg); + if (nstar < 0) { + if (!PyErr_Occurred) + PyErr_SetString( + PyExc_TypeError, + "len() of unsized object"); + x = NULL; + break; + } } if (nk > 0) { if (kwdict == NULL) { From mhammond@skippinet.com.au Wed Mar 29 07:06:05 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Wed, 29 Mar 2000 17:06:05 +1000 Subject: [Patches] Registry behaviour change for 1.6 Message-ID: Attached is a patch for PC\getpathp.c that changes the registry strategy for Windows. There are extensive comments that document the rules, so I wont bother repeating them here :-) Note however that this patch is "Unicode aware", in that it can be compiled against the Win32 Unicode APIs. The normal Python build still uses the ANSI API's; the Unicode versions were required for Windows CE. Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Mark. *** getpathp.c 2000/03/29 01:49:47 1.12 --- getpathp.c 2000/03/29 07:00:31 *************** *** 32,43 **** /* Return the initial module search path. */ /* Used by DOS, OS/2, Windows 3.1, Windows 95/98, Windows NT. */ #include "Python.h" #include "osdefs.h" #ifdef MS_WIN32 #include ! extern BOOL PyWin_IsWin32s(); #endif #include --- 32,95 ---- /* Return the initial module search path. */ /* Used by DOS, OS/2, Windows 3.1, Windows 95/98, Windows NT. */ + /* ---------------------------------------------------------------- + PATH RULES FOR WINDOWS: + This describes how sys.path is formed on Windows. It describes the + functionality, not the implementation (ie, the order in which these + are actually fetched is different) + + * Python always adds an empty entry at the start, which corresponds + to the current directory. + + * If the PYTHONPATH env. var. exists, it's entries are added next. + + * We look in the registry for "application paths" - that is, sub-keys + under the main PythonPath registry key. These are added next (the + order of sub-key processing is undefined). + HKEY_CURRENT_USER is searched and added first. + HKEY_LOCAL_MACHINE is searched and added next. + (Note that all known installers only use HKLM, so HKCU is typically + empty) + + * We attempt to locate the "Python Home" - if the PYTHONHOME env var + is set, we believe it. Otherwise, we use the path of our host .EXE's + to try and locate our "landmark" (lib\\string.py) and deduce our home. + - If we DO have a Python Home: The relevant sub-directories (Lib, + plat-win, lib-tk, etc) are based on the Python Home + - If we DO NOT have a Python Home, the core Python Path is + loaded from the registry. This is the main PythonPath key, + and both HKLM and HKCU are combined to form the path) + + * Iff - we can not locate the Python Home, have not had a PYTHONPATH + specified, and can't locate any Registry entries (ie, we have _nothing_ + we can assume is a good path), a default path with relative entries is + used (eg. .\Lib;.\plat-win, etc) + + + The end result of all this is: + * When running python.exe, or any other .exe in the main Python directory + (either an installed version, or directly from the PCbuild directory), + the core path is deduced, and the core paths in the registry are + ignored. Other "application paths" in the registry are always read. + + * When Python is hosted in another exe (different directory, embedded via + COM, etc), the Python Home will not be deduced, so the core path from + the registry is used. Other "application paths "in the registry are + always read. + + * If Python can't find its home and there is no registry (eg, frozen + exe, some very strange installation setup) you get a path with + some default, but relative, paths. + + --------------------------------------------------------------- - */ + + #include "Python.h" #include "osdefs.h" #ifdef MS_WIN32 #include ! #include #endif #include *************** *** 168,274 **** #ifdef MS_WIN32 ! #ifndef BUILD_LANDMARK ! #define BUILD_LANDMARK "PC\\getpathp.c" ! #endif - #include "malloc.h" // for alloca - see comments below! - extern const char *PyWin_DLLVersionString; // a string loaded from the DLL at startup. - /* Load a PYTHONPATH value from the registry. Load from either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER. Returns NULL, or a pointer that should be freed. */ static char * ! getpythonregpath(HKEY keyBase) { HKEY newKey = 0; - DWORD nameSize = 0; DWORD dataSize = 0; ! DWORD numEntries = 0; LONG rc; char *retval = NULL; ! char *dataBuf; ! const char keyPrefix[] = "Software\\Python\\PythonCore\\"; ! const char keySuffix[] = "\\PythonPath"; int versionLen; ! char *keyBuf; ! ! // Tried to use sysget("winver") but here is too early :-( ! versionLen = strlen(PyWin_DLLVersionString); ! // alloca == no free required, but memory only local to fn. ! // also no heap fragmentation! Am I being silly? ! keyBuf = alloca(sizeof(keyPrefix)-1 + versionLen + sizeof(keySuffix)); // chars only, plus 1 NULL. ! // lots of constants here for the compiler to optimize away :-) ! memcpy(keyBuf, keyPrefix, sizeof(keyPrefix)-1); ! memcpy(keyBuf+sizeof(keyPrefix)-1, PyWin_DLLVersionString, versionLen); ! memcpy(keyBuf+sizeof(keyPrefix)-1+versionLen, keySuffix, sizeof(keySuffix)); // NULL comes with this one! ! ! rc=RegOpenKey(keyBase, ! keyBuf, ! &newKey); ! if (rc==ERROR_SUCCESS) { ! RegQueryInfoKey(newKey, NULL, NULL, NULL, NULL, NULL, NULL, ! &numEntries, &nameSize, &dataSize, NULL, NULL); ! } ! if (numEntries) { ! /* Loop over all subkeys. */ ! /* Win32s doesnt know how many subkeys, so we do ! it twice */ ! char keyBuf[MAX_PATH+1]; ! int index = 0; ! int off = 0; ! for(index=0;;index++) { ! long reqdSize = 0; ! DWORD rc = RegEnumKey(newKey, ! index, keyBuf, MAX_PATH+1); ! if (rc) break; ! rc = RegQueryValue(newKey, keyBuf, NULL, &reqdSize); ! if (rc) break; ! dataSize += reqdSize + 1; /* 1 for the ";" */ } ! dataBuf = malloc(dataSize+1); ! if (dataBuf==NULL) ! return NULL; /* pretty serious? Raise error? */ ! /* Now loop over, grabbing the paths. ! Subkeys before main library */ ! for(index=0;;index++) { ! int adjust; ! long reqdSize = dataSize; ! DWORD rc = RegEnumKey(newKey, ! index, keyBuf,MAX_PATH+1); ! if (rc) break; ! rc = RegQueryValue(newKey, ! keyBuf, dataBuf+off, &reqdSize); ! if (rc) break; ! if (reqdSize>1) { ! /* If Nothing, or only '\0' copied. */ ! adjust = strlen(dataBuf+off); ! dataSize -= adjust; ! off += adjust; ! dataBuf[off++] = ';'; ! dataBuf[off] = '\0'; dataSize--; } } ! /* Additionally, win32s doesnt work as expected, so ! the specific strlen() is required for 3.1. */ ! rc = RegQueryValue(newKey, "", dataBuf+off, &dataSize); ! if (rc==ERROR_SUCCESS) { ! if (strlen(dataBuf)==0) ! free(dataBuf); ! else ! retval = dataBuf; /* caller will free */ } ! else ! free(dataBuf); } ! if (newKey) RegCloseKey(newKey); return retval; } #endif /* MS_WIN32 */ --- 220,365 ---- #ifdef MS_WIN32 ! /* a string loaded from the DLL at startup.*/ ! extern const char *PyWin_DLLVersionString; /* Load a PYTHONPATH value from the registry. Load from either HKEY_LOCAL_MACHINE or HKEY_CURRENT_USER. + Works in both Unicode and 8bit environments. Only uses the + Ex family of functions so it also works with Windows CE. + Returns NULL, or a pointer that should be freed. */ static char * ! getpythonregpath(HKEY keyBase, int skipcore) { HKEY newKey = 0; DWORD dataSize = 0; ! DWORD numKeys = 0; LONG rc; char *retval = NULL; ! TCHAR *dataBuf = NULL; ! static const TCHAR keyPrefix[] = _T("Software\\Python\\PythonCore\\"); ! static const TCHAR keySuffix[] = _T("\\PythonPath"); int versionLen; ! DWORD index; ! TCHAR *keyBuf = NULL; ! TCHAR *keyBufPtr; ! TCHAR **ppPaths = NULL; ! ! /* Tried to use sysget("winver") but here is too early :-( */ ! versionLen = _tcslen(PyWin_DLLVersionString); ! /* Space for all the chars, plus one \0 */ ! keyBuf = keyBufPtr = malloc(sizeof(keyPrefix) + ! sizeof(TCHAR)*(versionLen-1) + ! sizeof(keySuffix)); ! if (keyBuf==NULL) goto done; ! ! memcpy(keyBufPtr, keyPrefix, sizeof(keyPrefix)-sizeof(TCHAR)); ! keyBufPtr += sizeof(keyPrefix)/sizeof(TCHAR) - 1; ! memcpy(keyBufPtr, PyWin_DLLVersionString, versionLen * sizeof(TCHAR)); ! keyBufPtr += versionLen; ! /* NULL comes with this one! */ ! memcpy(keyBufPtr, keySuffix, sizeof(keySuffix)); ! /* Open the root Python key */ ! rc=RegOpenKeyEx(keyBase, ! keyBuf, /* subkey */ ! 0, /* reserved */ ! KEY_READ, ! &newKey); ! if (rc!=ERROR_SUCCESS) goto done; ! /* Find out how big our core buffer is, and how many subkeys we have */ ! rc = RegQueryInfoKey(newKey, NULL, NULL, NULL, &numKeys, NULL, NULL, ! NULL, NULL, &dataSize, NULL, NULL); ! if (rc!=ERROR_SUCCESS) goto done; ! if (skipcore) dataSize = 0; /* Only count core ones if we want them! */ ! /* Allocate a temp array of char buffers, so we only need to loop ! reading the registry once ! */ ! ppPaths = malloc( sizeof(TCHAR *) * numKeys ); ! if (ppPaths==NULL) goto done; ! memset(ppPaths, 0, sizeof(TCHAR *) * numKeys); ! /* Loop over all subkeys, allocating a temp sub-buffer. */ ! for(index=0;index 0) { ! *(szCur++) = _T(';'); dataSize--; } + len = _tcslen(ppPaths[index]); + _tcsncpy(szCur, ppPaths[index], len); + szCur += len; + dataSize -= len; } ! if (skipcore) ! *szCur = '\0'; ! else { ! *(szCur++) = _T(';'); ! dataSize--; ! /* Now append the core path entries - this will include the NULL */ ! rc = RegQueryValueEx(newKey, NULL, 0, NULL, (LPBYTE)szCur, &dataSize); } ! /* And set the result - caller must free ! If MBCS, it is fine as is. If Unicode, allocate new ! buffer and convert. ! */ ! #ifdef UNICODE ! retval = (char *)malloc(reqdSize+1); ! if (retval) ! WideCharToMultiByte(CP_ACP, 0, ! dataBuf, -1, /* source */ ! retval, dataSize+1, /* dest */ ! NULL, NULL); ! free(dataBuf); ! #else ! retval = dataBuf; ! #endif } ! done: ! /* Loop freeing my temp buffers */ ! if (ppPaths) { ! for(index=0;index Message-ID: <38E1D18E.1E12E900@lemburg.com> Mark Hammond wrote: > > Not a patch yet, but Im working towards one :) > > Im thinking about how to change open() WRT unicode. > > First thought was that the change would be to a PyArg_ParseTuple() > call, but inspection shows a PyFile_FromString(char *name, ...). > > So it makes sense to create a new API - PyFile_FromWCHAR()? Why not a PyFile_FromUnicode(Py_UNICODE *fname,... ) API ? The open() builtin could use the new "es" parser markers to have the input converted to Unicode first and then pass it along to this new API. On Windows, PyFile_FromUnicode() would then have to convert the Py_UNICODE value to wchar_t using the PyUnicode_AsWideChar() API and then proceed with fopenW()... > On Windows, Im thinking that rather than performing a conversion to > MBCS, this could perform a runtime check for WCHAR CRTL support - > ie, basically "am I running NT", and call fopenW(). This could be a > global static, so the first check would be "expensive", but after > that we have a single "if" test. > > The bltinmodule call to this function can simply check the type of > the arg and make the appropriate call (fopen() or fopenW()). (Note > that on Windows 95/98 the W calls exist, they just dont work! This > is by design, and designed for exactly this case) > > Something like: > > #ifdef HAVE_WCHAR_H > > PyObject PyFile_FromWCHAR(WCHAR *fname, ...) > { > ... > #ifdef MS_WIN32 > static int have_wchar_crt = -1; > if (have_wchar_crt==-1) > have_wchar_crt = AmIRunningNT() ? 1 : 0; > if (have_wchar_crt) > fp = fopenW(fname, ...); > else { > /* encode as MBCS */ > fp = fopen(mbcs_name, ...); > } > ... > #endif > > } > #endif > > Sound reasonable? > > Mark. > > _______________________________________________ > Patches mailing list > Patches@python.org > http://www.python.org/mailman/listinfo/patches -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From mhammond@skippinet.com.au Wed Mar 29 12:36:01 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Wed, 29 Mar 2000 22:36:01 +1000 Subject: [Patches] Discuss: Unicode patch for open() builtin. In-Reply-To: <38E1D18E.1E12E900@lemburg.com> Message-ID: [Dunno why I started this discussion on patches!? Sorry 'bout that!] [MAL writes] > Why not a PyFile_FromUnicode(Py_UNICODE *fname,... ) API ? The > open() builtin could use the new "es" parser markers to have > the input converted to Unicode first and then pass it along > to this new API. I prefer that open() perform no translations - eg: if (PyString_Check()) return PyFile_FromString(); // Even on NT else if (PyUnicode_Check()) return PyFile_FromWideString(); // Native on NT, MBCS conversion on 95. I dont see a good reason to force open to jump through all these encoding hoops - simple rule is strings work exactly as now, Unicode gets passed directly if possible. Mark. From mal@lemburg.com Wed Mar 29 13:38:43 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 29 Mar 2000 15:38:43 +0200 Subject: [Patches] Discuss: Unicode patch for open() builtin. References: Message-ID: <38E20763.6562F026@lemburg.com> Mark Hammond wrote: > > [Dunno why I started this discussion on patches!? Sorry 'bout > that!] Perhaps we should restart the discussion on python-dev ? > [MAL writes] > > Why not a PyFile_FromUnicode(Py_UNICODE *fname,... ) API ? The > > open() builtin could use the new "es" parser markers to have > > the input converted to Unicode first and then pass it along > > to this new API. > > I prefer that open() perform no translations - eg: > if (PyString_Check()) > return PyFile_FromString(); // Even on NT > else if (PyUnicode_Check()) > return PyFile_FromWideString(); // Native on NT, MBCS conversion > on 95. > > I dont see a good reason to force open to jump through all these > encoding hoops - simple rule is strings work exactly as now, Unicode > gets passed directly if possible. Using "es" and Py_UNICODE would simplifiy the open() API since all conversion would be done in the PyFile_FromUnicode() API, but you prefer the above if-then-else, fine with me. Still, I think a PyFile_FromUnicode() would be useful for extension writers since this hides the OS dependent conversions from them. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From jeremy@cnri.reston.va.us Wed Mar 29 15:34:58 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Wed, 29 Mar 2000 10:34:58 -0500 (EST) Subject: [Patches] Re: [Python-Dev] yeah! for Jeremy and Greg In-Reply-To: <14561.38858.41246.28460@anthem.cnri.reston.va.us> References: <14561.19438.157799.810802@goon.cnri.reston.va.us> <14561.21075.637108.322536@anthem.cnri.reston.va.us> <14561.38858.41246.28460@anthem.cnri.reston.va.us> Message-ID: <14562.8866.74809.483061@goon.cnri.reston.va.us> Barry, I'm not sure about this patch because it has the potential to obscure a bug in user code. If I write a class that has an __len__ method and that method has a bug that raises an exception, it looks like your patch would clobber the exception and set a new one. Perhaps it would be better to explicitly catch the AttributeError on __len__ and turn that into a TypeError. Jeremy From bwarsaw@cnri.reston.va.us Wed Mar 29 18:28:27 2000 From: bwarsaw@cnri.reston.va.us (bwarsaw@cnri.reston.va.us) Date: Wed, 29 Mar 2000 13:28:27 -0500 (EST) Subject: [Patches] Re: [Python-Dev] yeah! for Jeremy and Greg References: <14561.19438.157799.810802@goon.cnri.reston.va.us> <14561.21075.637108.322536@anthem.cnri.reston.va.us> <14561.38858.41246.28460@anthem.cnri.reston.va.us> <14562.8866.74809.483061@goon.cnri.reston.va.us> Message-ID: <14562.19275.772454.397175@anthem.cnri.reston.va.us> >>>>> "JH" == Jeremy Hylton writes: JH> I'm not sure about this patch because it has the potential to JH> obscure a bug in user code. If I write a class that has an JH> __len__ method and that method has a bug that raises an JH> exception, it looks like your patch would clobber the JH> exception and set a new one. JH> Perhaps it would be better to explicitly catch the JH> AttributeError on __len__ and turn that into a TypeError. As we discussed at lunch today, I concur. It was late so I'm just happy I was able to bat 50% on those patches :) The one to ceval.c to check the return value of PySequence_Length() is the key patch. I'll go ahead and check that one in. -Barry From jeremy@cnri.reston.va.us Wed Mar 29 21:23:49 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Wed, 29 Mar 2000 16:23:49 -0500 (EST) Subject: [Patches] two more fix for extended call syntax Message-ID: <14562.29797.402307.361849@goon.cnri.reston.va.us> I was feeling paranoid today, after Barry's bug report last night, and wrote some more obscure tests cases for the new call syntax. I found two more problems that need fixing, and have patches below. The second problem & patch are a little questionable, so I'm running them through the patches list. If they look ok, I can check them in. Bug #1: A user-defined class implements the sequence protocol incorrectly, but raises an IndexError for index i where i is less than the value returned by __length__. The current code creates a tuple from the instance, then happily indexes off the end of the tuple. The fix is to never give the instance a chance to lie; convert to a tuple first, then see how long the tuple is. Bug #2: The use of old-style keyword arguments along with **dict modifies dict, adding entries for each of the explicit keywords. The solution I've taken is to copy the **dict if there are old-style keyword arguments. I'm not happy about copying a potentially large dictionary, but the case seems to obscure to optimize. I also discovered that there was no Python C API call to copy a dictionary, so I added a PyDict_Copy method by stealing the body of dict_copy. Jeremy === cd /home/jhylton/python/src/Include/ === /depot/gnu/plat/bin/cvs diff -c dictobject.h Index: dictobject.h =================================================================== RCS file: /projects/cvsroot/python/dist/src/Include/dictobject.h,v retrieving revision 2.15 diff -c -r2.15 dictobject.h *** dictobject.h 1998/12/04 18:47:57 2.15 --- dictobject.h 2000/03/29 19:57:15 *************** *** 52,57 **** --- 52,59 ---- extern DL_IMPORT(PyObject *) PyDict_Values Py_PROTO((PyObject *mp)); extern DL_IMPORT(PyObject *) PyDict_Items Py_PROTO((PyObject *mp)); extern DL_IMPORT(int) PyDict_Size Py_PROTO((PyObject *mp)); + extern DL_IMPORT(PyObject *) PyDict_Copy Py_PROTO((PyObject *mp)); + extern DL_IMPORT(PyObject *) PyDict_GetItemString Py_PROTO((PyObject *dp, char *key)); extern DL_IMPORT(int) PyDict_SetItemString Py_PROTO((PyObject *dp, char *key, PyObject *item)); === Exit status: 1 === cd /home/jhylton/python/src/Lib/test/ === /depot/gnu/plat/bin/cvs diff -c test_extcall.py Index: test_extcall.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/test/test_extcall.py,v retrieving revision 1.2 diff -c -r1.2 test_extcall.py *** test_extcall.py 2000/03/28 23:53:18 1.2 --- test_extcall.py 2000/03/29 20:25:56 *************** *** 46,51 **** --- 46,95 ---- g(1, 2) g(1, 2, 3) g(1, 2, 3, *(4, 5)) + class Nothing: pass + try: + g(*Nothing()) + except AttributeError, attr: + assert attr[0] == '__len__' + else: + print "should raise AttributeError: __len__" + + class Nothing: + def __len__(self): + return 5 + try: + g(*Nothing()) + except AttributeError, attr: + assert attr[0] == '__getitem__' + else: + print "should raise AttributeError: __getitem__" + + class Nothing: + def __len__(self): + return 5 + def __getitem__(self, i): + if i < 3: + return i + else: + raise IndexError, i + g(*Nothing()) + + # make sure the function call doesn't stomp on the dictionary? + d = {'a': 1, 'b': 2, 'c': 3} + d2 = d.copy() + assert d == d2 + g(1, d=4, **d) + print d + print d2 + assert d == d2, "function call modified dictionary" + + # what about willful misconduct? + def saboteur(**kw): + kw['x'] = locals() + d = {} + saboteur(a=1, **d) + assert d == {} + try: g(1, 2, 3, **{'x':4, 'y':5}) except TypeError, err: === Exit status: 1 === cd /home/jhylton/python/src/Objects/ === /depot/gnu/plat/bin/cvs diff -c dictobject.c Index: dictobject.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Objects/dictobject.c,v retrieving revision 2.50 diff -c -r2.50 dictobject.c *** dictobject.c 2000/03/13 16:01:29 2.50 --- dictobject.c 2000/03/29 20:17:24 *************** *** 738,748 **** register dictobject *mp; PyObject *args; { register int i; dictobject *copy; dictentry *entry; ! if (!PyArg_Parse(args, "")) return NULL; copy = (dictobject *)PyDict_New(); if (copy == NULL) return NULL; --- 738,760 ---- register dictobject *mp; PyObject *args; { + if (!PyArg_Parse(args, "")) + return NULL; + return PyDict_Copy((PyObject*)mp); + } + + PyObject * + PyDict_Copy(o) + PyObject *o; + { + register dictobject *mp; register int i; dictobject *copy; dictentry *entry; ! ! if (!PyDict_Check(o)) return NULL; + mp = (dictobject *)o; copy = (dictobject *)PyDict_New(); if (copy == NULL) return NULL; === Exit status: 1 === cd /home/jhylton/python/src/Python/ === /depot/gnu/plat/bin/cvs diff -c ceval.c Index: ceval.c =================================================================== RCS file: /projects/cvsroot/python/dist/src/Python/ceval.c,v retrieving revision 2.171 diff -c -r2.171 ceval.c *** ceval.c 2000/03/29 18:36:49 2.171 --- ceval.c 2000/03/29 20:24:22 *************** *** 1635,1641 **** x = NULL; break; } ! nstar = PySequence_Length(stararg); if (nstar < 0) { x = NULL; break; --- 1635,1652 ---- x = NULL; break; } ! /* Convert abstract sequence to concrete tuple */ ! if (!PyTuple_Check(stararg)) { ! PyObject *t = NULL; ! t = PySequence_Tuple(stararg); ! if (t == NULL) { ! x = NULL; ! break; ! } ! Py_DECREF(stararg); ! stararg = t; ! } ! nstar = PyTuple_GET_SIZE(stararg); if (nstar < 0) { x = NULL; break; *************** *** 1649,1654 **** --- 1660,1674 ---- break; } } + else { + PyObject *d = PyDict_Copy(kwdict); + if (d == NULL) { + x = NULL; + break; + } + Py_DECREF(kwdict); + kwdict = d; + } err = 0; while (--nk >= 0) { PyObject *value = POP(); *************** *** 1678,1695 **** break; } if (stararg) { - PyObject *t = NULL; int i; - if (!PyTuple_Check(stararg)) { - /* must be sequence to pass earlier test */ - t = PySequence_Tuple(stararg); - if (t == NULL) { - x = NULL; - break; - } - Py_DECREF(stararg); - stararg = t; - } for (i = 0; i < nstar; i++) { PyObject *a = PyTuple_GET_ITEM(stararg, i); Py_INCREF(a); --- 1698,1704 ---- === Exit status: 1 From mhammond@skippinet.com.au Wed Mar 29 22:50:01 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Thu, 30 Mar 2000 08:50:01 +1000 Subject: [Patches] Discuss: Unicode patch for open() builtin. In-Reply-To: <38E20763.6562F026@lemburg.com> Message-ID: W01BTCB3cml0ZXNdDQo+IE1hcmsgSGFtbW9uZCB3cm90ZToNCj4gPiANCj4gPiBbRHVubm8gd2h5 IEkgc3RhcnRlZCB0aGlzIGRpc2N1c3Npb24gb24gcGF0Y2hlcyE/ICBTb3JyeSAnYm91dA0KPiA+ IHRoYXQhXQ0KPiANCj4gUGVyaGFwcyB3ZSBzaG91bGQgcmVzdGFydCB0aGUgZGlzY3Vzc2lvbiBv biBweXRob24tZGV2ID8NCg0KTmFoIC0gSSB0aGluayB0aGlzIGlzIHRoZSBsYXN0IG9uZSA6LSkN Cg0KPiA+IFtNQUwgd3JpdGVzXQ0KPiA+ID4gV2h5IG5vdCBhIFB5RmlsZV9Gcm9tVW5pY29kZShQ eV9VTklDT0RFICpmbmFtZSwuLi4gKSBBUEkgPyBUaGUNCj4gPiA+IG9wZW4oKSBidWlsdGluIGNv dWxkIHVzZSB0aGUgbmV3ICJlcyIgcGFyc2VyIG1hcmtlcnMgdG8gaGF2ZQ0KPiA+ID4gdGhlIGlu cHV0IGNvbnZlcnRlZCB0byBVbmljb2RlIGZpcnN0IGFuZCB0aGVuIHBhc3MgaXQgYWxvbmcNCj4g PiA+IHRvIHRoaXMgbmV3IEFQSS4NCg0KRWVlZWsgLSBteSBtaXN0YWtlIC0gSSBhcG9sb2dpemUu ICBJIHNhdyAiUHlfVU5JQ09ERSIgYXMgIlB5VW5pY29kZU9iamVjdCIgLSBnb3RjaHlhLg0KDQpZ ZWFoIC0gbm8gcHJvYmxlbS4uLg0KDQpTbywgZ2l2ZW4gbm8gbmVnYXRpdmUgdmliZXMgZnJvbSB0 aGUgRHV0Y2ggb25lLCBJIHRoaW5rIEknbGwganVzdCBkbyB0aGUgcGF0Y2guDQoNCk1hcmsuDQo= From Moshe Zadka Thu Mar 30 06:07:17 2000 From: Moshe Zadka (Moshe Zadka) Date: Thu, 30 Mar 2000 08:07:17 +0200 (IST) Subject: [Patches] two more fix for extended call syntax In-Reply-To: <14562.29797.402307.361849@goon.cnri.reston.va.us> Message-ID: On Wed, 29 Mar 2000, Jeremy Hylton wrote: > Bug #2: The use of old-style keyword arguments along with **dict > modifies dict, adding entries for each of the explicit keywords. > The solution I've taken is to copy the **dict if there are > old-style keyword arguments. I'm not happy about copying a > potentially large dictionary, but the case seems to obscure to > optimize. I also discovered that there was no Python C API call > to copy a dictionary, so I added a PyDict_Copy method by stealing > the body of dict_copy. If the *args can be any type implementing the sequence protocol, why can't the second argument can't be any type implementing the mapping protocol? And in that case, you'd have to copy it to a dictiionary. It does make sense to optimize the common f(*args, **kw) case, but not any other. -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html http://www.linux.org.il -- we put the penguin in .com From mwh21@cam.ac.uk Thu Mar 30 09:40:19 2000 From: mwh21@cam.ac.uk (Michael Hudson) Date: Thu, 30 Mar 2000 10:40:19 +0100 (BST) Subject: [Patches] touch up dis formatting Message-ID: With the (cool!) new call syntax, the longest opcode name is much longer, which fouls up dis's formatting slightly; this is a "fix" for that. Index: dis.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/dis.py,v retrieving revision 1.20 diff -u -r1.20 dis.py --- dis.py 2000/03/28 23:49:07 1.20 +++ dis.py 2000/03/30 09:32:11 @@ -65,7 +65,7 @@ if i in labels: print '>>', else: print ' ', print string.rjust(`i`, 4), - print string.ljust(opname[op], 15), + print string.ljust(opname[op], 20), i = i+1 if op >= HAVE_ARGUMENT: oparg = ord(code[i]) + ord(code[i+1])*256 nitpicking-ly y'rs M. I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. From thomas@xs4all.nl Thu Mar 30 13:01:17 2000 From: thomas@xs4all.nl (Thomas Wouters) Date: Thu, 30 Mar 2000 15:01:17 +0200 Subject: [Patches] mailbox.UnixMailbox and rfc822.Message.unixfrom In-Reply-To: <200003281356.IAA29608@eric.cnri.reston.va.us>; from guido@python.org on Tue, Mar 28, 2000 at 08:56:00AM -0500 References: <20000324214124.B12922@xs4all.nl> <200003272048.PAA19149@eric.cnri.reston.va.us> <20000328140722.F21895@xs4all.nl> <200003281356.IAA29608@eric.cnri.reston.va.us> Message-ID: <20000330150116.G20358@xs4all.nl> [ So I mail one last attempt... Feel free to ignore me or shut me up, I'm spending this weekend in Rome anyway, so it wont affect my mood ;) ] On Tue, Mar 28, 2000 at 08:56:00AM -0500, Guido van Rossum wrote: > > I have a gut feeling there is a lot of code out there that uses > > rfc822.Message.unixfrom. > My gut feels the reverse. You are the first to ask about this in the > lifetime of the mailbox module, and you didn't even really need it, > you only thought you did. I personally process approximately 200 > messages a day using MH, which never looks at the Unix From line. I actually meant 'too many users of rfc822.Message.unixfrom to bluntly remove it'. My main problem with this (mis)feature of the mailbox module is that it *looks* like it can handle all mailboxes, and it does it best, but it fails in the Unixmailbox case. Were one to write a tool to read, modify and write mailboxes, one would have to use a rewritten mailbox module, or special-case the unixmailbox and write a seperate module for that. I'm not arguing the sensibility of the unixfrom line here. We use qmails' maildir format as much as possible, for a large number of reasons (though we dont actually use qmail) and I much prefer it over unixmailboxes. But the unixmailbox format is the most widespread mailbox format -- all Unix mailclients read it, most write it by default, as do most MDAs (not only sendmail) and Netscape and Eudora use it too, for instance. But the fact remains that you can't use the mailbox module the same way for different mailboxes -- the other single-file mailbox formats use fixed end/start message indicators, that dont contain any other information. The Unixmailbox format does, and it's necessary to keep that data if you want to re-write the message somewhere. And there is no way to retrieve the unixfrom line *at all*, except by not using this module. > > > Why do you need access the Unix from header through the mailbox > > > module? > > > > I thought Mailman needed it. > [But it doesn't] The only reason (as far as I can see) that it doesn't need it is because programmers have always worked around this problem. Mailman's archiver creates two sets of archives for each maillist, an HTML version and a unix mailbox (The 'downloadable' version). I dont know if you ever tried to use the unix mailbox format, but some people have, and come away with a nasty surprise: It's not a valid mailbox. Why ? Because the unixfrom line has to be kludged, and the kludge uses the wrong date format, in addition to the wrong From address and the wrong time. (i posted a fix to fix the kludge in the mean time, by the way) (pipermail/HyperArch *do* use the mailbox module's UnixMailbox, you see.) And yes, I know this can all be fixed in Mailman (and if it wont be fixed in the python library, I'll post the patch there myself ;) but I still see it as a problem in the python mailbox module. If the mailbox module had done it right from the start, the kludge wouldn't even *be* there. > The mailbox module supports several other mailbox formats which don't > have a "Unix From line" either. It is intended to provide a uniform > API for reading mailboxes. Features specific to mailbox formats don't > belong in this API. If it forgets about unixfrom, it's not an API for reading mailboxes, it's an API for reading individual *messages*. It loses any and all information about the mailbox, and that's what I disagree with... It's not useful for real mailbox-editing. [ about the codingstyle of the fix ] > Well, since _search_end() is always called after a successful > _search_start(), you could have solved the problem locally in > _search_end(). Ah, well, I was worried it might screw up if someone else seek()ed to the start of a message, and then called next(), but I now see it can't happen... Not because next() seeks back to seekp, but because the seek() method on the mailbox class is nonfunctional :) Okay, if I write the patch like that (and maybe remove and/or fix the seek() method on the mailbox class ?) does it stand a chance of being accepted ? Or should I write a mailboxhandling module instead ? :-) Highest regards, as always<0.0 wink>, -- Thomas Wouters Hi! I'm a .signature virus! copy me into your .signature file to help me spread! From guido@python.org Thu Mar 30 15:04:46 2000 From: guido@python.org (Guido van Rossum) Date: Thu, 30 Mar 2000 10:04:46 -0500 Subject: [Patches] touch up dis formatting In-Reply-To: Your message of "Thu, 30 Mar 2000 10:40:19 +0100." References: Message-ID: <200003301504.KAA21024@eric.cnri.reston.va.us> > With the (cool!) new call syntax, the longest opcode name is much longer, > which fouls up dis's formatting slightly; this is a "fix" for that. Thanks! Done. --Guido van Rossum (home page: http://www.python.org/~guido/) From thomas.heller@ion-tof.com Thu Mar 30 19:32:48 2000 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Thu, 30 Mar 2000 21:32:48 +0200 Subject: [Patches] Metaclasses, customizing attribute access for classes Message-ID: <023201bf9a7e$bafb4750$4500a8c0@thomasnotebook> (Forwared because I mistyped the address) > Dear Python-developers, > > Recently I played with metaclasses from within python, > also with Jim Fulton's ExtensionClass. > I even tried to write my own metaclass in a C-extension, using the > famous Don Beaudry hook. > It seems that ExtensionClass does not completely what I want. > Metaclasses implemented in python are somewhat slow, > also writing them is a lot of work. > Writing a metaclass in C is even more work... > > Well, what do I want? > > Often, I use the following pattern: > class X: > def __init__ (self): > self.delegate = anObjectImplementedInC(...) > > def __getattr__ (self, key): > return self.delegate.dosomething(key) > > def __setattr__ (self, key, value): > self.delegate.doanotherthing(key, value) > > def __delattr__ (self, key): > self.delegate.doevenmore(key) > > This is too slow (for me). > So what I would like do to is: > > class X: > def __init__ (self): > self.__dict__ = aMappingObject(...) > > and now aMappingObject will automatically receive > all the setattr, getattr, and delattr calls. > > The *only* thing which is required for this is to remove > the restriction that the __dict__ attribute must be a dictionary. > This is only a small change to classobject.c (which unfortunately I > have only implemented for 1.5.2, not for the CVS version). > The performance impact for this change is unnoticable in pystone. > > What do you think? > Should I prepare a patch? > Any chance that this can be included in a future python version? > > Thomas Heller > From jeremy@cnri.reston.va.us Thu Mar 30 20:02:22 2000 From: jeremy@cnri.reston.va.us (Jeremy Hylton) Date: Thu, 30 Mar 2000 15:02:22 -0500 (EST) Subject: [Patches] two more fix for extended call syntax In-Reply-To: References: <14562.29797.402307.361849@goon.cnri.reston.va.us> Message-ID: <14563.45774.234220.600976@goon.cnri.reston.va.us> >>>>> "MZ" == Moshe Zadka writes: MZ> On Wed, 29 Mar 2000, Jeremy Hylton wrote: >> Bug #2: The use of old-style keyword arguments along with **dict >> modifies dict, adding entries for each of the explicit keywords. >> The solution I've taken is to copy the **dict if there are >> old-style keyword arguments. I'm not happy about copying a >> potentially large dictionary, but the case seems to obscure to >> optimize. I also discovered that there was no Python C API call >> to copy a dictionary, so I added a PyDict_Copy method by stealing >> the body of dict_copy. MZ> If the *args can be any type implementing the sequence protocol, MZ> why can't the second argument can't be any type implementing the MZ> mapping protocol? Why do you always answer with a question? The reason that the second argument requires a dictionary is that map also requires a dictionary. Guido doesn't sound interested in changing this behavior any time soon. I guess non-dict mappings are just second-class citizen.s MZ> And in that case, you'd have to copy it to a MZ> dictionary. It does make sense to optimize the common f(*args, MZ> **kw) case, but not any other. I'm not sure that any of these cases is common. Why would f(*args, **kw) be more common than f(x, *args, **kw) or than f(a, b, **kw) ? It ends up that the only case that will be really expensive is f(a, b=2, **kw) because that will require a copy of kw be made so that b=2 can be inserted into it. All the techniques I can think of to avoid the copy are too complicated to implement. Jeremy From bwarsaw@cnri.reston.va.us Thu Mar 30 17:18:45 2000 From: bwarsaw@cnri.reston.va.us (Barry A. Warsaw) Date: Thu, 30 Mar 2000 12:18:45 -0500 (EST) Subject: [Patches] two more fix for extended call syntax References: <14562.29797.402307.361849@goon.cnri.reston.va.us> Message-ID: <14563.35957.162804.43303@anthem.cnri.reston.va.us> >>>>> "MZ" == Moshe Zadka writes: MZ> If the *args can be any type implementing the sequence MZ> protocol, why can't the second argument can't be any type MZ> implementing the mapping protocol? And in that case, you'd MZ> have to copy it to a dictiionary. It does make sense to MZ> optimize the common f(*args, **kw) case, but not any other. PyEval_CallObjectWithKeywords() makes an explicit check for dict-ness, so ultimately this would have to be fixed first. It would be nice if these calls were elaborated to take anything that passes PyMapping_Check(). -Barry From mhammond@skippinet.com.au Thu Mar 30 22:23:55 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Fri, 31 Mar 2000 08:23:55 +1000 Subject: [Patches] Patch to Wise installer to include winreg.pyd Message-ID: Attached is a patch for python16.wse, to ensure the new standard module winreg is installed. NOTE: Wise changed every line in the WSE file. The attached diff ignores whitespace, but should still be obvious. Alternatively, you can open the .wse file yourself and add the new file manually... Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Mark. diff -c -w -r1.2 python16.wse *** python16.wse 2000/03/30 21:27:11 1.2 --- python16.wse 2000/03/30 22:18:30 *************** *** 686,691 **** --- 686,696 ---- Flags=0000000000000010 end item: Install File + Source=%_SRC_%\pcbuild\winreg.pyd + Destination=%MAINDIR%\DLLs\winreg.pyd + Flags=0000000000000010 + end + item: Install File Source=%_SRC_%\pcbuild\python16.dll Destination=%DLLDEST%\python16.dll Flags=0000000000000010 From mhammond@skippinet.com.au Fri Mar 31 00:44:28 2000 From: mhammond@skippinet.com.au (Mark Hammond) Date: Fri, 31 Mar 2000 10:44:28 +1000 Subject: [Patches] mmap changes for Win32. Message-ID: I hacked on this for Win32. Specific changes of note: * the Win32 version now accepts the same args as the Unix version. The win32 specific "tag" param is now optional. The end result is that the exact same test suite runs on Windows (definately a worth goal!). * I changed the error object. All occurences of the error, except for 1, corresponds to an underlying OS error. This one was changed to a ValueError (a better error for that condition), and the module error object is now simply EnvironmentError. All win32 error routines now call the new Windows specific error handler. Ive simply attached all relevant files. The .dsp goes in the "PCBuild" directory. Also attached is "PCbuild" workspace that includes the new project. NOT yet included is an update to the WISE package to install this new .pyd. Note that the test output has not changed, so this is not attached. [Attachments should be uuencoded this time - let me know if there are problems] Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Mark. begin 666 mmapmodule.c M+RH@+2HM($UO9&4Z($,K*SL@=&%B+7=I9'1H.B T("TJ+0T*("\)075T:&]R M.B!386T@4G5S:&EN9R \"!B>2!!+DTN($MU8VAL:6YG(#QA;6LQ0&)I9V9O;W0N M8V]M/B -"B O"21)9#H@;6UA<&UO9'5L92YC+'8@,BXQ(#(P,# O,#,O,S @ M,C$Z,30Z,S @86MU8VAL:6X@17AP("0-"@T*("\@;6UA<&UO9'5L92YC<' @ M+2T@;6%P(&$@=FEE=R!O9B!A(&9I;&4@:6YT;R!M96UO65T(2$A#0H@+PT*("\-"B O($YO=&4Z(%1H:7,@;6]D=6QE(&-U2!O;FQY(&1E86QS('=I=&@@,S(M8FET(&9I;&4-"B O(" @'0N#0HJ+PT*#0HC:6YC;'5D92 \4'ET M:&]N+F@^#0H-"B-I9FYD968@35-?5TE.,S(-"B-D969I;F4@54Y)6 T*(V5N M9&EF#0H-"B-I9F1E9B!-4U]724XS,@T*(VEN8VQU9&4@/'=I;F1O=W,N:#X- M"B-E;F1I9@T*#0HC:69D968@54Y)6 T*(VEN8VQU9&4@/'5N:7-T9"YH/@T* M(VEN8VQU9&4@/'-Y7!E7!E9&5F('-TF5?= ES M:7IE.PT*"0ES:7IE7W0)<&]S.PT*#0HC:69D968@35-?5TE.,S(-"@D)2$%. M1$Q%"6UA<%]H86YD;&4[#0H)"4A&24Q%"0EF:6QE7VAA;F1L93L-"@D)8VAA M"US<&5C:69I8R!I;F9OF4L($U37U-93D,@?"!-4U])3E9!3$E$051%*3L-"@D)"0EM=6YM M87 H;5]O8FHM/F1A=&$L(&U?;V)J+3YS:7IE*3L-"@D)?0T*(V5N9&EF("\J M(%5.25@@*B\-"@T*"0E0>4UE;5]$14PH;5]O8FHI.PT*?0T*#0IS=&%T:6,@ M4'E/8FIE8W0@*@T*;6UA<%]C;&]S95]M971H;V0@*&UM87!?;V)J96-T("H@ MPT*(VEF9&5F($U37U=)3C,R#0H) M"55N;6%P5FEE=T]F1FEL92 H5]. M;VYE*3L-"@D)5].;VYE*3L-"GT-"@T*(VEF9&5F($U37U=) M3C,R#0HC9&5F:6YE($-(14-+7U9!3$E$*&5R&-?5F%L=65%PD)"0D)"0D)"0D@(" @7 T*(" @( E0>45R M4]B:F5C=" J(&%R9W,I#0I[#0H)"6-H87(@=F%L=64[ M#0H)"6-H87(@*B!W:&5R92 ]("AS96QF+3YD871A*W-E;&8M/G!OF4I*2D@>PT*"0D)"79A;'5E M(#T@*&-H87(I("HH=VAE45RPT*"0EC:&%R("H@F4[#0H)"6-H87(@*B!E;VP[ M#0H)"5!Y3V)J96-T("H@R O*B!D;R!N;W1H:6YG("HO('T-"@T* M"0ER97-U;'0@/2!0>5]"=6EL9%9A;'5E*")S(R(L('-T87)T+" H;&]N9RD@ M*"LK96]L("T@71E M71EF4[#0H)"7T- M"@D)71E M4]B:F5C M=" J#0IM;6%P7V9I;F1?;65T:&]D("AM;6%P7V]B:F5C=" J4]B:F5C=" J87)G4%R9U]087)S951U<&QE("AAPT*"0D)"0D)8VAAPT*"0D)"0D)"0ES*RLL(&XK M*SL-"@D)"0D)"7T-"@D)"0D)"6EF("@A*FXI('L-"@D)"0D)"0D)PT*"0EL;VYG(&QE;F=T:#L-"@D)8VAA45X8U]686QU945R5])3D-2148@*%!Y M7TYO;F4I.PT*"0ER971U71E7VUE=&AO9" H;6UA<%]O8FIE8W0@ M*B!S96QF+ T*"0D)"0D)"5!Y3V)J96-T("H@87)G4%R9U]0 M87)S951U<&QE("AA5])3D-2148@*%!Y7TYO;F4I.PT*"0ER M971U$9&1D9&1D9&*2![#0H)"0D)5]"=6EL9%9A;'5E M("@B;"(L($=E=$9I;&53:7IE("@H2$%.1$Q%*7-E;&8M/F9I;&5?:&%N9&QE M+"!.54Q,*2DI.PT*"0E](&5LPT*"0D)"7)E='5R;B H4'E?0G5I;&16 M86QU92 H(FPB+"!S96QF+3YS:7IE*2 I.PT*"0E]#0HC96YD:68@+RH@35-? M5TE.,S(@*B\-"@T*(VEF9&5F(%5.25@-"@D)5]"=6EL9%9A M;'5E("@B;"(L('-E;&8M/G-I>F4I("D[#0HC96YD:68@+RH@54Y)6" J+PT* M?0T*#0HO*B!4:&ES(&%S71H;VX@8GD@:G5S="!C;&]S M:6YG(&%N9"!R92UO<&5N:6YG('=I=&@@=&AE#0H@+R!N97<@F5?;65T:&]D("AM M;6%P7V]B:F5C=" J('-E;&8L#0H)"0D)"0E0>4]B:F5C=" J(&%R9W,I#0I[ M#0H)"75NF4[#0H)"4-(14-+7U9!3$E$*$Y5 M3$PI.PT*"0EI9B H(5!Y07)G7U!APT*"0D)"7)E='5R;B!.54Q,.PT*(VEF9&5F($U37U=)3C,R M#0H)"7T@96QS92![( T*"0D)"4173U)$(&1W17)R0V]D92 ](# [#0H)"0D) M+RH@1FERF4@;V8@=&AE(&9I;&4@*B\-"@D)"0E3 M971%;F1/9D9I;&4@*"A(04Y$3$4I5])3D-2148@ M*%!Y7TYO;F4I.PT*"0D)"0D)"0ER971UPT*"0E0>45R&-?4WES M=&5M17)R;W(L(")M;6%P.B!R97-I>FEN9R!N;W0@879A:6QA8FQE+2UN;R!M MPT* M"0EV;VED("IN97=M87 [#0H-"@D);F5W;6%P(#T@;7)E;6%P*'-E;&8M/F1A M=&$L('-E;&8M/G-I>F4L(&YE=U]S:7IE+"!-4D5-05!?34%934]612D[#0H) M"6EF("AN97=M87 @/3T@*'9O:60@*BDM,2D@#0H)"7L-"@D)"0E0>45RPT*"0E#2$5#2U]604Q)1"A.54Q,*3L-"@D)5]"=6EL9%9A;'5E("@B;"(L('-E;&8M/G!OF5?="!S:7IE"3T@PT*"0D)"7)E='5R;B!.54Q,.PT*"0E] M(&5LF4I('L-"@D) M"0E0>45RPT*"0D)"0D)4'E%4]B:F5C=" J(&%R9W,I#0I[#0H)"2\J M('!TPT*"0D)"7)E='5R M;BA.54Q,*3L-"@D)?2!E;'-E('L-"@D)"0EU;G-I9VYE9"!L;VYG('=H97)E M.PT*"0D)"7-W:71C:" H:&]W*2![#0H)"0D)8V%S92 P.@T*"0D)"0D)=VAE M45RF4I*2D@>PT*"0D)"0D)5])3D-2148@*%!Y7TYO;F4I.PT*"0D) M"0D)5].;VYE*3L-"@D)"0E](&5LPT*"0D)"0D)4'E% M45X8U]686QU945R4]B:F5C=" J(&%R9W,I#0I[#0H)"75N&-?5F%L=65% M5])3D-2148@*%!Y7TYO;F4I.PT*"0D) M"0D)PT* M"0E[(F-L;W-E(BP)"2A0>4-&=6YC=&EO;BD@;6UA<%]C;&]S95]M971H;V0L M"0DQ?2P-"@D)>R)F:6YD(BP)"2A0>4-&=6YC=&EO;BD@;6UA<%]F:6YD7VUE M=&AO9"P)"0DQ?2P-"@D)>R)F;'5S:"(L"0DH4'E#1G5N8W1I;VXI(&UM87!? M9FQU71E M(BP@*%!Y0T9U;F-T:6]N*2!M;6%P7W)E861?8GET95]M971H;V0L"3%]+ T* M"0E[(G)E861L:6YE(BP)*%!Y0T9U;F-T:6]N*2!M;6%P7W)E861?;&EN95]M M971H;V0L"3%]+ T*"0E[(G)E4-&=6YC=&EO;BD@;6UA<%]R M97-I>F5?;65T:&]D+ D),7TL#0H)"7LB4-&=6YC=&EO;BD@;6UA<%]W71E M7VUE=&AO9"P),7TL#0H)"7M.54Q,+ D@("!.54Q,?0D@(" O*B!S96YT:6YE M;" J+PT*?3L-"@T*+RH@1G5N8W1I;VYS(&9O"P@<'1R*0T*(" @(" @("!M M;6%P7V]B:F5C=" J#L-"F-O;G-T('9O:60@*BIP M='([#0I[#0H)"4-(14-+7U9!3$E$*"TQ*3L-"@D):68@*"!I;F1E>" A/2 P M("D@>PT*"0D)"5!Y17)R7U-E=%-T45X8U]3>7-T96U%R @#0H)"4-(14-+7U9!3$E$*"TQ*3L-"@D):68@*"!I;F1E>" A/2 P("D@ M>PT*"0D)"5!Y17)R7U-E=%-T45X8U]3>7-T96U%PT*"0EI9B H(&EN M9&5X("$](# @*2![#0H)"0D)4'E%&ES=&5N="!B=69F M97(@4]B M:F5C=" J*7-E;&8L(&YA;64I.PT*?0T*#0IS=&%T:6,@:6YT#0IM;6%P7VQE M;F=T:"AS96QF*0T*(" @(" @("!M;6%P7V]B:F5C=" JPT* M"0D)"5!Y17)R7U-E=%-T45X8U]);F1E>$5R"!O=70@;V8@F4[#0H)#0H) M"7)E='5R;B!0>5-T4]B:F5C=" J M#0IM;6%P7V-O;F-A="AS96QF+"!B8BD-"B @(" @(" @;6UA<%]O8FIE8W0@ M*G-E;&8[#0I0>4]B:F5C=" J8F([#0I[#0H)"4-(14-+7U9!3$E$*$Y53$PI M.PT*"0E0>45R&-?4WES=&5M17)R;W(L(")M;6%P M4]B:F5C=" J#0IM;6%P7W)E<&5A="AS96QF M+"!N*0T*(" @(" @("!M;6%P7V]B:F5C=" JPT* M"0E#2$5#2U]604Q)1"A.54Q,*3L-"@D)4'E%F4I#0H)"0D):6AI9V@@/2!S96QF+3YS:7IE M.PT*"0T*"0EI9B H(2 H4'E3=')I;F=?0VAE8VLH=BDI("D@>PT*"0D)"5!Y M17)R7U-E=%-T45X8U]);F1E>$5R45R&-? M26YD97A%5-T7!E(#T@>PT* M"0E0>4]B:F5C=%](14%$7TE.250H,"D@+RH@<&%T8VAE9"!I;B!M;V1U;&4@ M:6YI=" J+PT*"0DP+ D)"0D)"0D)"2\J(&]B7W-I>F4@*B\-"@D)(FUM87 B M+ D)"0D)"0DO*B!T<%]N86UE("HO#0H)"7-I>F5O9BAM;6%P7V]B:F5C="DL M"0D)"2\J('1P7W-I>F4@*B\-"@D),"P)"0D)"0D)"0DO*B!T<%]I=&5M4]B:F5C=" J#0IN97=?;6UA<%]O8FIE8W0@*%!Y3V)J96-T("H@4]B:F5C=" J:W=D:6-T*0T*>PT*"0EM M;6%P7V]B:F5C=" J(&U?;V)J.PT*"0EU;G-I9VYE9"!L;VYG(&UA<%]S:7IE M.PT*"0EI;G0@9F0L(&9L86=S(#T@34%07U-(05)%1"P@<')O=" ](%!23U1? M5U))5$4@?"!04D]47U)%040[#0H)"6-H87(@*B!F:6QE;F%M93L-"@D):6YT M(&YA;65L96X[#0H)"6-H87(@*FME>7=O4%R9U]0 M87)S951U<&QE06YD2V5Y=V]R9',H87)G7!E*3L-"@D):68@*&U?;V)J(#T]($Y53$PI('MR971U MF5?="D@;6%P7W-I>F4[ M#0H)"6U?;V)J+3YP;W,@/2 HF4L( T*"0D)"0D)(" @<')O="P@9FQA9W,L M#0H)"0D)"0D@("!F9"P@,"D[#0H)"6EF("AM7V]B:BT^9&%T82 ]/2 H=F]I M9" J*2TQ*0T*"0E[#0H)"0D)4'E?1$5#4D5&*&U?;V)J*3L-"@D)"0E0>45R M4]B:F5C=" J(&%R9W,I#0I[#0H)"6UM87!?;V)J96-T("H@;5]O8FH[ M#0H)"75NF4[#0H)"6-H87(@*B!F:6QE;F%M M93L-"@D):6YT(&YA;65L96X[#0H)"6-H87(@*B!T86=N86UE(#T@(B([#0H- M"@D)1%=/4D0@9'=%7!E.PT*#0H)"6EF("@A4'E!B(L#0H)"0D)"0D@("9F:6QE;F\L#0H)"0D) M"0D@("9M87!?PT*"0D)"0E0>45RPT*"0D)"0D);5]O8FHM/G-I>F4@/2!M87!?$9&1D9&1D9&.PT*"0D)"6U?;V)J+3YS:7IE(#T@;6%P7W-I>F4[#0H) M"7T-"@T*"0DO*B!S970@=&AE(&EN:71I86P@<&]S:71I;VX@*B\-"@D);5]O M8FHM/G!OF4L#0H)"0D)"0D)"0D)"2 @('1A9VYA;64I.PT*"0EI9B H M;5]O8FHM/FUA<%]H86YD;&4@(3T@3E5,3"D@>PT*"0D)"6U?;V)J+3YD871A M(#T@*&-H87(@*BD@36%P5FEE=T]F1FEL92 H;5]O8FHM/FUA<%]H86YD;&4L M#0H)"0D)"0D)"0D)"0D)("!&24Q%7TU!4%]74DE412P-"@D)"0D)"0D)"0D) M"0D@(# L#0H)"0D)"0D)"0D)"0D)(" P+ T*"0D)"0D)"0D)"0D)"2 @,"D[ M#0H)"0D):68@*&U?;V)J+3YD871A("$]($Y53$PI('L-"@D)"0D)"7)E='5R M;B H*%!Y3V)J96-T("HI(&U?;V)J*3L-"@D)"0E](&5LPT*"0D)"0ED M=T5R'!O2!T:&ES(&UO9'5L92 J+PT*4UE M=&AO9$1E9B!M;6%P7V9U;F-T:6]NPT*"0E[(FUM87 B+ D)*%!Y M0T9U;F-T:6]N*2!N97=?;6UA<%]O8FIE8W0L( T*"0D@34542%]605)!4D=3 M?$U%5$A?2T595T]21%-]+ T*"0E[3E5,3"P)"0E.54Q,?0D)("\J(%-E;G1I M;F5L("HO#0I].PT*#0HC:69D968@35-?5TE.,S(-"E]?9&5C;'-P96,H9&QL M97AP;W)T*2!V;VED#0HC96YD:68@+RH@35-?5TE.,S(@*B\-"B-I9F1E9B!5 M3DE8#0IE>'1E5]) M;FET36]D=6QE("@B;6UA<"(L(&UM87!?9G5N8W1I;VYS*3L-"@D)9&EC=" ] M(%!Y36]D=6QE7T=E=$1I8W0@*&UO9'5L92D[#0H)"6UM87!?;6]D=6QE7V5R M&-?16YV:7)O;FUE;G1%4EN=%]&4EN=%]&4EN=%]&4EN=%]&4EN=%]&# Q,#(- M"@T*0T9'/6UM87 @+2!7:6XS,B!$96)U9PT*(4U%4U-!1T4@5&AI'!O2!A(&-O;F9I9W5R871I;VX@=VAE;B!R=6YN:6YG($Y-04M%#0HA3453 M4T%'12!B>2!D969I;FEN9R!T:&4@;6%C&%M<&QE.@T*(4U%4U-!1T4@#0HA34534T%'12!.34%+ M12 O9B B;6UA<"YM86LB($-&1STB;6UA<" M(%=I;C,R($1E8G5G(@T*(4U% M4U-!1T4@#0HA34534T%'12!0;W-S:6)L92!C:&]I8V5S(&9O&4-"E)30SUR8RYE>&4-"@T*(4E&(" B)"A#1D'!O&4-"B,@041$($)!4T4@0U!0("]N;VQO M9V\@+TU49" O5S,@+T=M("]'6" O6DD@+T]D("]$(")724XS,B(@+T0@(E]$ M14)51R(@+T0@(E]724Y$3U=3(B O1" B7TU"0U,B("]$(")?55-21$Q,(B O M1" B34U!4%]%6%!/4E13(B O65@@+T9$("]'6B @+V,-"B,@041$($-04" O M;F]L;V=O("]-1&0@+U7!E.G-E<'0-"B,@041$($Q)3DLS,B!K97)N96PS,BYL:6(@=7-EPT*?7U]#0H-"E!A8VMA9V4]/#0^#0I[>WL-"B @("!"96=I M;B!00T*?7U]#0H-"B,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,-"@T*4')O:F5C=#H@ M(F)S9&1B(CTB+EQBPT*?7U]#0H-"E!A8VMA9V4]/#0^#0I[>WL-"B @ M("!"96=I;B!00T*?7U] M#0H-"B,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,-"@T*4')O M:F5C=#H@(FUM87 B/2(N7&UM87 N9'-P(B M(%!A8VMA9V4@3W=N97(]/#0^ M#0H-"E!A8VMA9V4]/#4^#0I[>WL-"GU]?0T*#0I086-K86=E/3PT/@T*>WM[ M#0I]?7T-"@T*(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(PT* M#0I0WL-"GU]?0T*#0I086-K86=E M/3PT/@T*>WM[#0H@(" @0F5G:6X@4')O:F5C="!$97!E;F1E;F-Y#0H@(" @ M4')O:F5C=%]$97!?3F%M92!P>71H;VXQ-@T*(" @($5N9"!071H;VXB/2(N7'!Y=&AO;BYDPT*?7U]#0H- M"E!A8VMA9V4]/#0^#0I[>WL-"B @("!"96=I;B!00T*?7U]#0H-"B,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,-"@T*4')O:F5C=#H@(G!Y=&AO;C$V(CTB+EQP>71H M;VXQ-BYDPT*?7U]#0H-"E!A8VMA9V4]/#0^#0I[>WL-"GU]?0T*#0HC(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C#0H-"E!R;VIE8W0Z(")P>71H M;VYW(CTB+EQP>71H;VYW+F1S<"(@+2!086-K86=E($]W;F5R/3PT/@T*#0I0 M86-K86=E/3PU/@T*>WM[#0I]?7T-"@T*4&%C:V%G93T\-#X-"GM[>PT*(" @ M($)E9VEN(%!R;VIE8W0@1&5P96YD96YC>0T*(" @(%!R;VIE8W1?1&5P7TYA M;64@<'ET:&]N,38-"B @("!%;F0@4')O:F5C="!$97!E;F1E;F-Y#0I]?7T- M"@T*(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(PT*#0I0WL-"GU]?0T*#0I086-K86=E/3PT/@T* M>WM[#0H@(" @0F5G:6X@4')O:F5C="!$97!E;F1E;F-Y#0H@(" @4')O:F5C M=%]$97!?3F%M92!P>71H;VXQ-@T*(" @($5N9"!0WM[#0I]?7T-"@T*4&%C:V%G93T\ M-#X-"GM[>PT*(" @($)E9VEN(%!R;VIE8W0@1&5P96YD96YC>0T*(" @(%!R M;VIE8W1?1&5P7TYA;64@<'ET:&]N,38-"B @("!%;F0@4')O:F5C="!$97!E M;F1E;F-Y#0I]?7T-"@T*(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(PT*#0I';&]B86PZ#0H-"E!A8VMA9V4]/#4^#0I[>WL-"GU]?0T*#0I0 M86-K86=E/3PS/@T*>WM[#0I]?7T-"@T*(R,C(R,C(R,C(R,C(R,C(R,C(R,C M(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C(R,C 1(R,C(R,C(R,C(R,C(PT*#0H` ` end begin 666 test_mmap.py M#0II;7!O7,-"@T* M4$%'15-)6D4@/2!M;6%P+E!!1T5325I%#0H-"F1E9B!T97-T7V)O=&@H*3H- M"B @(" B5&5S="!M;6%P(&UO9'5L92!O;B!5;FEX('-Y2!C:&5C:W,-"B @("!P M71E"!M871C:"!O;B!M;6%P(&9A:6QE9"$G#0H@(" @96QS93H-"B @(" @ M(" @"!M871C:"!O;B!M;6%P("AP86=E('-T87)T+"!L96YG=&@@ M;V8@;6%T8V@I.B7,-"@T* M4$%'15-)6D4@/2!M;6%P+E!!1T5325I%#0H-"F1E9B!T97-T7V)O=&@H*3H- M"B @(" B5&5S="!M;6%P(&UO9'5L92!O;B!5;FEX('-Y71E(# Z)RP@2!T:&4@ M9FEL92=S(&-O;G1E;G0-"B @("!P6EN9R!F:6QE M)W,@8V]N=&5N="XN+B(-"B @("!M6S!=(#T@)S,G#0H@(" @;5M004=%4TE: M12 K,SH@4$%'15-)6D4@*S,K,UT])V)A71E(# Z)RP@ posixmodule compile failed... Maybe the typedef should go somewhere else, but this works :-) Checkin: mode_t does not exist on Windows. Release info: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Mark. diff -c -r2.128 posixmodule.c *** posixmodule.c 2000/03/31 00:48:21 2.128 --- posixmodule.c 2000/03/31 01:26:55 *************** *** 692,697 **** --- 692,700 ---- PyObject *self; PyObject *args; { + #ifdef MS_WIN32 + typedef int mode_t; + #endif char *path; int i; int res; From Moshe Zadka Fri Mar 31 07:04:25 2000 From: Moshe Zadka (Moshe Zadka) Date: Fri, 31 Mar 2000 09:04:25 +0200 (IST) Subject: [Patches] two more fix for extended call syntax In-Reply-To: <14563.45774.234220.600976@goon.cnri.reston.va.us> Message-ID: On Thu, 30 Mar 2000, Jeremy Hylton wrote: > Why do you always answer with a question? Why not? (That's the original joke, BTW) > I'm not sure that any of these cases is common. Why would > f(*args, **kw) > be more common than > f(x, *args, **kw) Probably less common. > It ends up that the only case that will be really expensive is > f(a, b=2, **kw) But this case will probably be so uncommon, it doesn't justify the brain cells to optimize, not to mention a simple algorithm is always more Bug Free(tm) -- Moshe Zadka . http://www.oreilly.com/news/prescod_0300.html http://www.linux.org.il -- we put the penguin in .com From mal@lemburg.com Fri Mar 31 08:35:29 2000 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 31 Mar 2000 10:35:29 +0200 Subject: [Patches] Unicode Patch Set 2000-03-31 Message-ID: <38E46351.66AFDFE9@lemburg.com> This is a multi-part message in MIME format. --------------4953D72560CE2BCDA7B64DD4 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit The attached patch set for the Unicode implementation includes the following changes: 1. Error reporting in the codec registry and lookup mechanism is enhanced to be more informative. 2. The large unicode database table is broken in pages of 4k entries each. This should fix compiler problems on some platforms. -- Marc-Andre Lemburg ______________________________________________________________________ Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ --------------4953D72560CE2BCDA7B64DD4 Content-Type: text/plain; charset=us-ascii; name="Unicode-Implementation-2000-03-31.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="Unicode-Implementation-2000-03-31.patch" diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/codecs.py Python+Unicode/Lib/codecs.py --- CVS-Python/Lib/codecs.py Sat Mar 25 11:56:29 2000 +++ Python+Unicode/Lib/codecs.py Wed Mar 29 18:25:57 2000 @@ -11,7 +11,11 @@ ### Registry and builtin stateless codec functions -from _codecs import * +try: + from _codecs import * +except ImportError,why: + raise SystemError,\ + 'Failed to load the builtin codecs: %s' % why ### Constants Only in CVS-Python/Lib: distutils diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Lib/encodings/aliases.py Python+Unicode/Lib/encodings/aliases.py --- CVS-Python/Lib/encodings/aliases.py Sat Mar 11 00:17:18 2000 +++ Python+Unicode/Lib/encodings/aliases.py Thu Mar 30 01:42:03 2000 @@ -24,8 +24,8 @@ 'u16': 'utf_16', 'utf_16be': 'utf_16_be', 'utf_16le': 'utf_16_le', - 'UnicodeBigUnmarked': 'utf_16_be', - 'UnicodeLittleUnmarked': 'utf_16_le', + 'unicodebigunmarked': 'utf_16_be', + 'unicodelittleunmarked': 'utf_16_le', # ASCII 'us_ascii': 'ascii', @@ -47,11 +47,11 @@ 'iso_8859_9': 'iso8859_9', # Mac - 'MacCentralEurope': 'mac_latin2', - 'MacCyrillic': 'mac_cyrillic', - 'MacGreek': 'mac_greek', - 'MacIceland': 'mac_iceland', - 'MacRoman': 'mac_roman', - 'MacTurkish': 'mac_turkish', + 'maccentraleurope': 'mac_latin2', + 'maccyrillic': 'mac_cyrillic', + 'macgreek': 'mac_greek', + 'maciceland': 'mac_iceland', + 'macroman': 'mac_roman', + 'macturkish': 'mac_turkish', } Only in CVS-Python/Lib/test/output: test_extcall Only in CVS-Python/Lib/test/output: test_winreg Only in CVS-Python/Lib/test: test_extcall.py Only in CVS-Python/Lib/test: test_winreg.py diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Modules/unicodedata.c Python+Unicode/Modules/unicodedata.c --- CVS-Python/Modules/unicodedata.c Sat Mar 11 00:10:21 2000 +++ Python+Unicode/Modules/unicodedata.c Fri Mar 31 10:27:45 2000 @@ -13,6 +13,18 @@ #include "Python.h" #include "unicodedatabase.h" +/* --- Helpers ------------------------------------------------------------ */ + +static +const _PyUnicode_DatabaseRecord *unicode_db(register int i) +{ + register int page = i >> 12; + + if (page < sizeof(_PyUnicode_Database)) + return &_PyUnicode_Database[page][i & 0x0fff]; + return &_PyUnicode_Database[0][0]; +} + /* --- Module API --------------------------------------------------------- */ static PyObject * @@ -132,7 +144,7 @@ "need a single Unicode character as parameter"); goto onError; } - index = (int)_PyUnicode_Database[(int)*PyUnicode_AS_UNICODE(v)].category; + index = (int)unicode_db((int)*PyUnicode_AS_UNICODE(v))->category; if (index < 0 || index > sizeof(_PyUnicode_CategoryNames) / sizeof(_PyUnicode_CategoryNames[0])) { @@ -162,8 +174,7 @@ "need a single Unicode character as parameter"); goto onError; } - index = (int)_PyUnicode_Database[ - (int)*PyUnicode_AS_UNICODE(v)].bidirectional; + index = (int)unicode_db((int)*PyUnicode_AS_UNICODE(v))->bidirectional; if (index < 0 || index > sizeof(_PyUnicode_CategoryNames) / sizeof(_PyUnicode_CategoryNames[0])) { @@ -193,8 +204,7 @@ "need a single Unicode character as parameter"); goto onError; } - value = (int)_PyUnicode_Database[ - (int)*PyUnicode_AS_UNICODE(v)].combining; + value = (int)unicode_db((int)*PyUnicode_AS_UNICODE(v))->combining; return PyInt_FromLong(value); onError: @@ -216,7 +226,7 @@ "need a single Unicode character as parameter"); goto onError; } - value = (int)_PyUnicode_Database[(int)*PyUnicode_AS_UNICODE(v)].mirrored; + value = (int)unicode_db((int)*PyUnicode_AS_UNICODE(v))->mirrored; return PyInt_FromLong(value); onError: @@ -238,7 +248,7 @@ "need a single Unicode character as parameter"); goto onError; } - value = _PyUnicode_Database[(int)*PyUnicode_AS_UNICODE(v)].decomposition; + value = unicode_db((int)*PyUnicode_AS_UNICODE(v))->decomposition; if (value == NULL) return PyString_FromString(""); else diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Modules/unicodedatabase.c Python+Unicode/Modules/unicodedatabase.c --- CVS-Python/Modules/unicodedatabase.c Sat Mar 11 00:08:02 2000 +++ Python+Unicode/Modules/unicodedatabase.c Fri Mar 31 10:18:32 2000 @@ -87,9 +87,9 @@ 0 /* Sentinel */ }; -/* --- Unicode Database --------------------------------------------------- */ +/* --- Unicode Database Pages --------------------------------------------- */ -const _PyUnicode_DatabaseRecord _PyUnicode_Database[65536] = { +const _PyUnicode_DatabaseRecord _PyUnicode_Database_0[4096] = { /* U+0000 */ { 13, 0, 15, 0, 0 }, /* U+0001 */ { 13, 0, 15, 0, 0 }, @@ -4187,6 +4187,11 @@ /* U+0ffd */ { 0, 0, 0, 0, 0 }, /* U+0ffe */ { 0, 0, 0, 0, 0 }, /* U+0fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_1[4096] = { + /* U+1000 */ { 19, 0, 1, 0, 0 }, /* U+1001 */ { 19, 0, 1, 0, 0 }, /* U+1002 */ { 19, 0, 1, 0, 0 }, @@ -8283,6 +8288,11 @@ /* U+1ffd */ { 29, 0, 19, 0, "00B4" }, /* U+1ffe */ { 29, 0, 19, 0, " 0020 0314" }, /* U+1fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_2[4096] = { + /* U+2000 */ { 10, 0, 18, 0, "2002" }, /* U+2001 */ { 10, 0, 18, 0, "2003" }, /* U+2002 */ { 10, 0, 18, 0, " 0020" }, @@ -12379,6 +12389,11 @@ /* U+2ffd */ { 0, 0, 0, 0, 0 }, /* U+2ffe */ { 0, 0, 0, 0, 0 }, /* U+2fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_3[4096] = { + /* U+3000 */ { 10, 0, 18, 0, " 0020" }, /* U+3001 */ { 26, 0, 19, 0, 0 }, /* U+3002 */ { 26, 0, 19, 0, 0 }, @@ -16475,6 +16490,11 @@ /* U+3ffd */ { 0, 0, 0, 0, 0 }, /* U+3ffe */ { 0, 0, 0, 0, 0 }, /* U+3fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_4[4096] = { + /* U+4000 */ { 0, 0, 0, 0, 0 }, /* U+4001 */ { 0, 0, 0, 0, 0 }, /* U+4002 */ { 0, 0, 0, 0, 0 }, @@ -20571,6 +20591,11 @@ /* U+4ffd */ { 0, 0, 0, 0, 0 }, /* U+4ffe */ { 0, 0, 0, 0, 0 }, /* U+4fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_5[4096] = { + /* U+5000 */ { 0, 0, 0, 0, 0 }, /* U+5001 */ { 0, 0, 0, 0, 0 }, /* U+5002 */ { 0, 0, 0, 0, 0 }, @@ -24667,6 +24692,11 @@ /* U+5ffd */ { 0, 0, 0, 0, 0 }, /* U+5ffe */ { 0, 0, 0, 0, 0 }, /* U+5fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_6[4096] = { + /* U+6000 */ { 0, 0, 0, 0, 0 }, /* U+6001 */ { 0, 0, 0, 0, 0 }, /* U+6002 */ { 0, 0, 0, 0, 0 }, @@ -28763,6 +28793,11 @@ /* U+6ffd */ { 0, 0, 0, 0, 0 }, /* U+6ffe */ { 0, 0, 0, 0, 0 }, /* U+6fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_7[4096] = { + /* U+7000 */ { 0, 0, 0, 0, 0 }, /* U+7001 */ { 0, 0, 0, 0, 0 }, /* U+7002 */ { 0, 0, 0, 0, 0 }, @@ -32859,6 +32894,11 @@ /* U+7ffd */ { 0, 0, 0, 0, 0 }, /* U+7ffe */ { 0, 0, 0, 0, 0 }, /* U+7fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_8[4096] = { + /* U+8000 */ { 0, 0, 0, 0, 0 }, /* U+8001 */ { 0, 0, 0, 0, 0 }, /* U+8002 */ { 0, 0, 0, 0, 0 }, @@ -36955,6 +36995,11 @@ /* U+8ffd */ { 0, 0, 0, 0, 0 }, /* U+8ffe */ { 0, 0, 0, 0, 0 }, /* U+8fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_9[4096] = { + /* U+9000 */ { 0, 0, 0, 0, 0 }, /* U+9001 */ { 0, 0, 0, 0, 0 }, /* U+9002 */ { 0, 0, 0, 0, 0 }, @@ -41051,6 +41096,11 @@ /* U+9ffd */ { 0, 0, 0, 0, 0 }, /* U+9ffe */ { 0, 0, 0, 0, 0 }, /* U+9fff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_10[4096] = { + /* U+a000 */ { 19, 0, 1, 0, 0 }, /* U+a001 */ { 19, 0, 1, 0, 0 }, /* U+a002 */ { 19, 0, 1, 0, 0 }, @@ -45147,6 +45197,11 @@ /* U+affd */ { 0, 0, 0, 0, 0 }, /* U+affe */ { 0, 0, 0, 0, 0 }, /* U+afff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_11[4096] = { + /* U+b000 */ { 0, 0, 0, 0, 0 }, /* U+b001 */ { 0, 0, 0, 0, 0 }, /* U+b002 */ { 0, 0, 0, 0, 0 }, @@ -49243,6 +49298,11 @@ /* U+bffd */ { 0, 0, 0, 0, 0 }, /* U+bffe */ { 0, 0, 0, 0, 0 }, /* U+bfff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_12[4096] = { + /* U+c000 */ { 0, 0, 0, 0, 0 }, /* U+c001 */ { 0, 0, 0, 0, 0 }, /* U+c002 */ { 0, 0, 0, 0, 0 }, @@ -53339,6 +53399,11 @@ /* U+cffd */ { 0, 0, 0, 0, 0 }, /* U+cffe */ { 0, 0, 0, 0, 0 }, /* U+cfff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_13[4096] = { + /* U+d000 */ { 0, 0, 0, 0, 0 }, /* U+d001 */ { 0, 0, 0, 0, 0 }, /* U+d002 */ { 0, 0, 0, 0, 0 }, @@ -57435,6 +57500,11 @@ /* U+dffd */ { 0, 0, 0, 0, 0 }, /* U+dffe */ { 0, 0, 0, 0, 0 }, /* U+dfff */ { 15, 0, 1, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_14[4096] = { + /* U+e000 */ { 16, 0, 1, 0, 0 }, /* U+e001 */ { 0, 0, 0, 0, 0 }, /* U+e002 */ { 0, 0, 0, 0, 0 }, @@ -61531,6 +61601,11 @@ /* U+effd */ { 0, 0, 0, 0, 0 }, /* U+effe */ { 0, 0, 0, 0, 0 }, /* U+efff */ { 0, 0, 0, 0, 0 }, + +}; + +const _PyUnicode_DatabaseRecord _PyUnicode_Database_15[4096] = { + /* U+f000 */ { 0, 0, 0, 0, 0 }, /* U+f001 */ { 0, 0, 0, 0, 0 }, /* U+f002 */ { 0, 0, 0, 0, 0 }, @@ -65627,4 +65702,27 @@ /* U+fffd */ { 30, 0, 19, 0, 0 }, /* U+fffe */ { 0, 0, 0, 0, 0 }, /* U+ffff */ { 0, 0, 0, 0, 0 }, + +}; + +/* --- Unicode Database --------------------------------------------------- */ + +const _PyUnicode_DatabaseRecord *_PyUnicode_Database[16] = { + _PyUnicode_Database_0, + _PyUnicode_Database_1, + _PyUnicode_Database_2, + _PyUnicode_Database_3, + _PyUnicode_Database_4, + _PyUnicode_Database_5, + _PyUnicode_Database_6, + _PyUnicode_Database_7, + _PyUnicode_Database_8, + _PyUnicode_Database_9, + _PyUnicode_Database_10, + _PyUnicode_Database_11, + _PyUnicode_Database_12, + _PyUnicode_Database_13, + _PyUnicode_Database_14, + _PyUnicode_Database_15, }; + diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Modules/unicodedatabase.h Python+Unicode/Modules/unicodedatabase.h --- CVS-Python/Modules/unicodedatabase.h Sat Mar 11 00:08:04 2000 +++ Python+Unicode/Modules/unicodedatabase.h Fri Mar 31 10:16:03 2000 @@ -30,4 +30,4 @@ /* --- Unicode Database --------------------------------------------------- */ -extern const _PyUnicode_DatabaseRecord _PyUnicode_Database[65536]; +extern const _PyUnicode_DatabaseRecord *_PyUnicode_Database[16]; Only in CVS-Python/Objects: .#stringobject.c.2.59 Only in CVS-Python/Objects: stringobject.c.orig diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Objects/unicodeobject.c Python+Unicode/Objects/unicodeobject.c --- CVS-Python/Objects/unicodeobject.c Tue Mar 28 09:19:18 2000 +++ Python+Unicode/Objects/unicodeobject.c Tue Mar 28 15:47:04 2000 @@ -1483,7 +1483,9 @@ } #ifdef MS_WIN32 + /* --- MBCS codecs for Windows -------------------------------------------- */ + PyObject *PyUnicode_DecodeMBCS(const char *s, int size, const char *errors) @@ -1536,6 +1538,7 @@ } return repr; } + #endif /* MS_WIN32 */ /* --- Character Mapping Codec -------------------------------------------- */ Only in CVS-Python/PC: winreg.c Only in CVS-Python/PCbuild: python16.dsp Only in CVS-Python/PCbuild: python16.wse Only in CVS-Python/PCbuild: winreg.dsp diff -u -rP -x *.o -x *.pyc -x Makefile -x *~ -x *.so -x add2lib -x pgen -x buildno -x config.* -x libpython* -x python -x Setup -x Setup.local -x Setup.thread -x hassignal -x Makefile.pre -x *.bak -x *.s -x DEADJOE -x Demo -x CVS CVS-Python/Python/codecs.c Python+Unicode/Python/codecs.c --- CVS-Python/Python/codecs.c Fri Mar 24 22:21:29 2000 +++ Python+Unicode/Python/codecs.c Wed Mar 29 18:17:19 2000 @@ -27,42 +27,61 @@ This is done in a lazy way so that the Unicode implementation does not downgrade startup time of scripts not needing it. - Errors are silently ignored by this function. Only one try is made. + ImportErrors are silently ignored by this function. Only one try is + made. */ static -void import_encodings() +int import_encodings() { PyObject *mod; import_encodings_called = 1; mod = PyImport_ImportModule("encodings"); if (mod == NULL) { - PyErr_Clear(); - return; + if (PyErr_ExceptionMatches(PyExc_ImportError)) { + /* Ignore ImportErrors... this is done so that + distributions can disable the encodings package. Note + that other errors are not masked, e.g. SystemErrors + raised to inform the user of an error in the Python + configuration are still reported back to the user. */ + PyErr_Clear(); + return 0; + } + return -1; } Py_DECREF(mod); + return 0; } /* Register a new codec search function. + As side effect, this tries to load the encodings package, if not + yet done, to make sure that it is always first in the list of + search functions. + The search_function's refcount is incremented by this function. */ int PyCodec_Register(PyObject *search_function) { - if (!import_encodings_called) - import_encodings(); + if (!import_encodings_called) { + if (import_encodings()) + goto onError; + } if (search_function == NULL) { PyErr_BadArgument(); - return -1; + goto onError; } if (!PyCallable_Check(search_function)) { PyErr_SetString(PyExc_TypeError, "argument must be callable"); - return -1; + goto onError; } return PyList_Append(_PyCodec_SearchPath, search_function); + + onError: + return -1; } static @@ -89,20 +108,29 @@ characters. This makes encodings looked up through this mechanism effectively case-insensitive. - If no codec is found, a KeyError is set and NULL returned. */ + If no codec is found, a KeyError is set and NULL returned. + + As side effect, this tries to load the encodings package, if not + yet done. This is part of the lazy load strategy for the encodings + package. + +*/ PyObject *_PyCodec_Lookup(const char *encoding) { PyObject *result, *args = NULL, *v; int i, len; - if (_PyCodec_SearchCache == NULL || _PyCodec_SearchPath == NULL) { + if (_PyCodec_SearchCache == NULL || + _PyCodec_SearchPath == NULL) { PyErr_SetString(PyExc_SystemError, "codec module not properly initialized"); goto onError; } - if (!import_encodings_called) - import_encodings(); + if (!import_encodings_called) { + if (import_encodings()) + goto onError; + } /* Convert the encoding to a lower-cased Python string */ v = lowercasestring(encoding); @@ -127,6 +155,12 @@ len = PyList_Size(_PyCodec_SearchPath); if (len < 0) goto onError; + if (len == 0) { + PyErr_SetString(PyExc_LookupError, + "no codec search functions registered: " + "can't find encoding"); + goto onError; + } for (i = 0; i < len; i++) { PyObject *func; --------------4953D72560CE2BCDA7B64DD4-- From thomas.heller@ion-tof.com Fri Mar 31 13:02:34 2000 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 31 Mar 2000 15:02:34 +0200 Subject: [Patches] Small typo in ceval.c Message-ID: <024901bf9b11$62561060$4500a8c0@thomasnotebook> *** ceval.c Fri Mar 31 14:58:48 2000 --- ceval.patched.c Fri Mar 31 15:00:18 2000 *************** *** 2774,2780 **** } if (!PyString_Check(name)) { PyErr_SetString(PyExc_SystemError, ! "build_class witn non-string name"); return NULL; } n = PyTuple_Size(bases); --- 2774,2780 ---- } if (!PyString_Check(name)) { PyErr_SetString(PyExc_SystemError, ! "build_class with non-string name"); return NULL; } n = PyTuple_Size(bases); Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Thomas Heller From thomas.heller@ion-tof.com Fri Mar 31 16:05:44 2000 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Fri, 31 Mar 2000 18:05:44 +0200 Subject: [Patches] distutils should use new winreg module on NT Message-ID: <032a01bf9b2a$f873e5e0$4500a8c0@thomasnotebook> This patch changes msvccompiler.py so that the new winreg module is used if available. Otherwise, win32reg and win32con is used. *** msvccompiler.py Fri Mar 31 18:01:20 2000 --- msvccompiler.new.py Fri Mar 31 17:55:02 2000 *************** *** 16,21 **** --- 16,47 ---- from distutils.ccompiler import \ CCompiler, gen_preprocess_options, gen_lib_options + try: + import winreg + _HKEY_CLASSES_ROOT = winreg.HKEY_CLASSES_ROOT + _HKEY_LOCAL_MACHINE = winreg.HKEY_LOCAL_MACHINE + _HKEY_CURRENT_USER = winreg.HKEY_CURRENT_USER + _HKEY_USERS = winreg.HKEY_USERS + _RegOpenKeyEx = winreg.OpenKeyEx + _RegEnumKey = winreg.EnumKey + _RegEnumValue = winreg.EnumValue + _RegError = winreg.error + _can_read_reg = 1 + except ImportError: + try: + import win32api + import win32con + _HKEY_CLASSES_ROOT = win32con.HKEY_CLASSES_ROOT + _HKEY_LOCAL_MACHINE = win32con.HKEY_LOCAL_MACHINE + _HKEY_CURRENT_USER = win32con.HKEY_CURRENT_USER + _HKEY_USERS = win32con.HKEY_USERS + _RegOpenKeyEx = win32api.RegOpenKeyEx + _RegEnumKey = win32api.RegEnumKey + _RegEnumValue = win32api.RegEnumValue + _RegError = win32api.error + _can_read_reg = 1 + except ImportError: + _can_read_reg = 0 def get_devstudio_versions (): """Get list of devstudio versions from the Windows registry. Return a *************** *** 24,53 **** a registry-access module) or the appropriate registry keys weren't found.""" ! try: ! import win32api ! import win32con ! except ImportError: return [] K = 'Software\\Microsoft\\Devstudio' L = [] ! for base in (win32con.HKEY_CLASSES_ROOT, ! win32con.HKEY_LOCAL_MACHINE, ! win32con.HKEY_CURRENT_USER, ! win32con.HKEY_USERS): try: ! k = win32api.RegOpenKeyEx(base,K) i = 0 while 1: try: ! p = win32api.RegEnumKey(k,i) if p[0] in '123456789' and p not in L: L.append(p) ! except win32api.error: break i = i + 1 ! except win32api.error: pass L.sort() L.reverse() --- 50,76 ---- a registry-access module) or the appropriate registry keys weren't found.""" ! if not _can_read_reg: return [] K = 'Software\\Microsoft\\Devstudio' L = [] ! for base in (_HKEY_CLASSES_ROOT, ! _HKEY_LOCAL_MACHINE, ! _HKEY_CURRENT_USER, ! _HKEY_USERS): try: ! k = _RegOpenKeyEx(base,K) i = 0 while 1: try: ! p = _RegEnumKey(k,i) if p[0] in '123456789' and p not in L: L.append(p) ! except _RegError: break i = i + 1 ! except _RegError: pass L.sort() L.reverse() *************** *** 61,70 **** a list of strings; will be empty list if unable to access the registry or appropriate registry keys not found.""" ! try: ! import win32api ! import win32con ! except ImportError: return [] L = [] --- 84,90 ---- a list of strings; will be empty list if unable to access the registry or appropriate registry keys not found.""" ! if not _can_read_reg: return [] L = [] *************** *** 74,89 **** K = ('Software\\Microsoft\\Devstudio\\%s\\' + 'Build System\\Components\\Platforms\\Win32 (%s)\\Directories') % \ (version,platform) ! for base in (win32con.HKEY_CLASSES_ROOT, ! win32con.HKEY_LOCAL_MACHINE, ! win32con.HKEY_CURRENT_USER, ! win32con.HKEY_USERS): try: ! k = win32api.RegOpenKeyEx(base,K) i = 0 while 1: try: ! (p,v,t) = win32api.RegEnumValue(k,i) if string.upper(p) == path: V = string.split(v,';') for v in V: --- 94,109 ---- K = ('Software\\Microsoft\\Devstudio\\%s\\' + 'Build System\\Components\\Platforms\\Win32 (%s)\\Directories') % \ (version,platform) ! for base in (_HKEY_CLASSES_ROOT, ! _HKEY_LOCAL_MACHINE, ! _HKEY_CURRENT_USER, ! _HKEY_USERS): try: ! k = _RegOpenKeyEx(base,K) i = 0 while 1: try: ! (p,v,t) = _RegEnumValue(k,i) if string.upper(p) == path: V = string.split(v,';') for v in V: *************** *** 91,99 **** L.append(v) break i = i + 1 ! except win32api.error: break ! except win32api.error: pass return L --- 111,119 ---- L.append(v) break i = i + 1 ! except _RegError: break ! except _RegError: pass return L Disclaimer: I confirm that, to the best of my knowledge and belief, this contribution is free of any claims of third parties under copyright, patent or other rights or interests ("claims"). To the extent that I have any such claims, I hereby grant to CNRI a nonexclusive, irrevocable, royalty-free, worldwide license to reproduce, distribute, perform and/or display publicly, prepare derivative versions, and otherwise use this contribution as part of the Python software and its related documentation, or any derivative versions thereof, at no cost to CNRI or its licensed users, and to authorize others to do so. I acknowledge that CNRI may, at its sole discretion, decide whether or not to incorporate this contribution in the Python software and its related documentation. I further grant CNRI permission to use my name and other identifying information provided to CNRI by me for use in connection with the Python software and its related documentation. Thomas Heller From guido@python.org Fri Mar 31 21:01:38 2000 From: guido@python.org (Guido van Rossum) Date: Fri, 31 Mar 2000 16:01:38 -0500 Subject: [Patches] mailbox.UnixMailbox and rfc822.Message.unixfrom In-Reply-To: Your message of "Thu, 30 Mar 2000 15:01:17 +0200." <20000330150116.G20358@xs4all.nl> References: <20000324214124.B12922@xs4all.nl> <200003272048.PAA19149@eric.cnri.reston.va.us> <20000328140722.F21895@xs4all.nl> <200003281356.IAA29608@eric.cnri.reston.va.us> <20000330150116.G20358@xs4all.nl> Message-ID: <200003312101.QAA04015@eric.cnri.reston.va.us> > I actually meant 'too many users of rfc822.Message.unixfrom to bluntly > remove it'. My main problem with this (mis)feature of the mailbox module is > that it *looks* like it can handle all mailboxes, and it does it best, but > it fails in the Unixmailbox case. Were one to write a tool to read, modify > and write mailboxes, one would have to use a rewritten mailbox module, or > special-case the unixmailbox and write a seperate module for that. Frankly, I don't see any support for *writing* mailboxes anywhere in the mailbox module. All it does it allow you to *read* it. Recreating a mailbox given its messages would require a lot of code that is not part of the current module. > I'm not arguing the sensibility of the unixfrom line here. We use qmails' > maildir format as much as possible, for a large number of reasons (though we > dont actually use qmail) and I much prefer it over unixmailboxes. But the > unixmailbox format is the most widespread mailbox format -- all Unix > mailclients read it, most write it by default, as do most MDAs (not only > sendmail) and Netscape and Eudora use it too, for instance. > > But the fact remains that you can't use the mailbox module the same way for > different mailboxes -- the other single-file mailbox formats use fixed > end/start message indicators, that dont contain any other information. The > Unixmailbox format does, and it's necessary to keep that data if you want to > re-write the message somewhere. And there is no way to retrieve the unixfrom > line *at all*, except by not using this module. > > > > > Why do you need access the Unix from header through the mailbox > > > > module? > > > > > > I thought Mailman needed it. > > > [But it doesn't] > > The only reason (as far as I can see) that it doesn't need it is because > programmers have always worked around this problem. Mailman's archiver > creates two sets of archives for each maillist, an HTML version and a unix > mailbox (The 'downloadable' version). I dont know if you ever tried to use > the unix mailbox format, but some people have, and come away with a nasty > surprise: It's not a valid mailbox. Why ? Because the unixfrom line has to > be kludged, and the kludge uses the wrong date format, in addition to the > wrong From address and the wrong time. (i posted a fix to fix the kludge in > the mean time, by the way) > > (pipermail/HyperArch *do* use the mailbox module's UnixMailbox, you see.) > > And yes, I know this can all be fixed in Mailman (and if it wont be fixed in > the python library, I'll post the patch there myself ;) but I still see it > as a problem in the python mailbox module. If the mailbox module had done it > right from the start, the kludge wouldn't even *be* there. > > > The mailbox module supports several other mailbox formats which don't > > have a "Unix From line" either. It is intended to provide a uniform > > API for reading mailboxes. Features specific to mailbox formats don't > > belong in this API. > > If it forgets about unixfrom, it's not an API for reading mailboxes, it's an > API for reading individual *messages*. It loses any and all information > about the mailbox, and that's what I disagree with... It's not useful for > real mailbox-editing. "It's not useful" is a bit of an exaggeration, don't you think? You seem to be fixed on the idea that the Unix header contains useful information. I have been living without this information for years and have never missed it... > [ about the codingstyle of the fix ] > > > Well, since _search_end() is always called after a successful > > _search_start(), you could have solved the problem locally in > > _search_end(). > > Ah, well, I was worried it might screw up if someone else seek()ed to the > start of a message, and then called next(), but I now see it can't happen... > Not because next() seeks back to seekp, but because the seek() method on the > mailbox class is nonfunctional :) > > Okay, if I write the patch like that (and maybe remove and/or fix the seek() > method on the mailbox class ?) does it stand a chance of being accepted ? Or > should I write a mailboxhandling module instead ? :-) Here's what I suggest for a patch. It works for me. If it works for you, I'll check it in. Index: mailbox.py =================================================================== RCS file: /projects/cvsroot/python/dist/src/Lib/mailbox.py,v retrieving revision 1.18 diff -c -r1.18 mailbox.py *** mailbox.py 2000/02/10 17:17:13 1.18 --- mailbox.py 2000/03/31 20:53:03 *************** *** 97,109 **** --- 97,112 ---- def _search_start(self): while 1: + pos = self.fp.tell() line = self.fp.readline() if not line: raise EOFError if line[:5] == 'From ' and self._isrealfromline(line): + self.fp.seek(pos) return def _search_end(self): + self.fp.readline() # Throw away header line while 1: pos = self.fp.tell() line = self.fp.readline() --Guido van Rossum (home page: http://www.python.org/~guido/)