From noreply at sourceforge.net Mon May 1 05:44:00 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 30 Apr 2006 20:44:00 -0700 Subject: [Patches] [ python-Patches-1473132 ] Improve docs for tp_clear and tp_traverse Message-ID: Patches item #1473132, was opened at 2006-04-19 13:43 Message generated for change (Comment added) made by collinwinter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1473132&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Collin Winter (collinwinter) Assigned to: Nobody/Anonymous (nobody) Summary: Improve docs for tp_clear and tp_traverse Initial Comment: The attached patch greatly enhances the documentation for the tp_clear and tp_traverse functions. The patch is against Doc/api/newtypes.tex, r45562. ---------------------------------------------------------------------- >Comment By: Collin Winter (collinwinter) Date: 2006-04-30 23:43 Message: Logged In: YES user_id=1344176 I've enhanced the patch per Tim Peters' comment. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2006-04-22 23:30 Message: Logged In: YES user_id=31435 I agree the additional info is helpful (thanks!). Alas, there's more to it, and it's hard to know when to stop :-(. For example, an author of a type may _want_ to visit, e.g., contained strings in tp_traverse, because they want gc.get_referents() to return the contained strings (typically as a debugging aid). The issues wrt to tp_clear are subtler. The real requirement is that the aggregate of all tp_clears called break all possible cycles. For one thing, that means there's no real reason for a tp_clear to touch a member that's known to be a Python string or integer (since such an object can't be in a cycle, clearing it can't help to break a cycle). It's only tp_dealloc that _must_ drop references to all containees. Subtler is that a gc'ed container type may choose not to implement tp_clear at all. If you look, you'll see that Python's tuple type in fact leaves its tp_clear slot empty. This isn't a problem because it's impossible to have a cycle composed _solely_ of tuples (that may not be obvious, but it's true -- it derives from that tuples are immutable). Any cycle a tuple may be in will be broken if the non-tuple objects in the cycle clear their containees, so there's no actually need for tuples to have a tp_clear. The possibility should be mentioned, although it's fine to recommend playing it safe. Indeed, I don't think it buys anything worth having for tuples not to have an obvious tp_clear implementation. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1473132&group_id=5470 From noreply at sourceforge.net Mon May 1 08:58:24 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 30 Apr 2006 23:58:24 -0700 Subject: [Patches] [ python-Patches-1479611 ] speed up function calls Message-ID: Patches item #1479611, was opened at 2006-04-30 23:58 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: speed up function calls Initial Comment: Results: 2.86% for 1 arg (len), 11.8% for 2 args (min), and 1.6% for pybench. trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.74 msec per loop trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 8.03 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.88 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 9.09 msec per loop pybench goes from 5688.00 down to 5598.00 Details about the patch: There are 2 unrelated changes. They both seem to provide equal benefits for calling varargs C. One is very simple and just inlines calling a varargs C function rather than calling PyCFunction_Call() which does extra checks that are already known. This moves meth and self up one block. and breaks the C_TRACE into 2. (When looking at the patch, this will make sense I hope.) The other change is more dangerous. It modifies load_args() to hold on to tuples so they aren't allocated and deallocated. The initialization is done one time in the new func _PyEval_Init(). It allocates 64 tuples of size 8 that are never deallocated. The idea is that there won't be usually be more than 64 frames with 8 or less parameters active on the stack at any one time (stack depth). There are cases where this can degenerate, but for the most part, it should only be marginally slower, but generally this should be a fair amount faster by skipping the alloc and dealloc and some extra work. My decrementing the _last_index inside the needs_free blocks, that could improve behaviour. This really needs comments added to the code. But I'm not gonna get there tonight. I'd be interested in comments about the code. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 From noreply at sourceforge.net Mon May 1 09:08:03 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 00:08:03 -0700 Subject: [Patches] [ python-Patches-1479611 ] speed up function calls Message-ID: Patches item #1479611, was opened at 2006-04-30 23:58 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: speed up function calls Initial Comment: Results: 2.86% for 1 arg (len), 11.8% for 2 args (min), and 1.6% for pybench. trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.74 msec per loop trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 8.03 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.88 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 9.09 msec per loop pybench goes from 5688.00 down to 5598.00 Details about the patch: There are 2 unrelated changes. They both seem to provide equal benefits for calling varargs C. One is very simple and just inlines calling a varargs C function rather than calling PyCFunction_Call() which does extra checks that are already known. This moves meth and self up one block. and breaks the C_TRACE into 2. (When looking at the patch, this will make sense I hope.) The other change is more dangerous. It modifies load_args() to hold on to tuples so they aren't allocated and deallocated. The initialization is done one time in the new func _PyEval_Init(). It allocates 64 tuples of size 8 that are never deallocated. The idea is that there won't be usually be more than 64 frames with 8 or less parameters active on the stack at any one time (stack depth). There are cases where this can degenerate, but for the most part, it should only be marginally slower, but generally this should be a fair amount faster by skipping the alloc and dealloc and some extra work. My decrementing the _last_index inside the needs_free blocks, that could improve behaviour. This really needs comments added to the code. But I'm not gonna get there tonight. I'd be interested in comments about the code. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-05-01 00:08 Message: Logged In: YES user_id=33168 I should note the numbers 64 and 8 are total guesses. It might be good to try and determine values based on empirical data. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 From noreply at sourceforge.net Mon May 1 10:27:45 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 01:27:45 -0700 Subject: [Patches] [ python-Patches-1479611 ] speed up function calls Message-ID: Patches item #1479611, was opened at 2006-05-01 08:58 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: speed up function calls Initial Comment: Results: 2.86% for 1 arg (len), 11.8% for 2 args (min), and 1.6% for pybench. trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.74 msec per loop trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 8.03 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.88 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 9.09 msec per loop pybench goes from 5688.00 down to 5598.00 Details about the patch: There are 2 unrelated changes. They both seem to provide equal benefits for calling varargs C. One is very simple and just inlines calling a varargs C function rather than calling PyCFunction_Call() which does extra checks that are already known. This moves meth and self up one block. and breaks the C_TRACE into 2. (When looking at the patch, this will make sense I hope.) The other change is more dangerous. It modifies load_args() to hold on to tuples so they aren't allocated and deallocated. The initialization is done one time in the new func _PyEval_Init(). It allocates 64 tuples of size 8 that are never deallocated. The idea is that there won't be usually be more than 64 frames with 8 or less parameters active on the stack at any one time (stack depth). There are cases where this can degenerate, but for the most part, it should only be marginally slower, but generally this should be a fair amount faster by skipping the alloc and dealloc and some extra work. My decrementing the _last_index inside the needs_free blocks, that could improve behaviour. This really needs comments added to the code. But I'm not gonna get there tonight. I'd be interested in comments about the code. ---------------------------------------------------------------------- >Comment By: Martin v. L??wis (loewis) Date: 2006-05-01 10:27 Message: Logged In: YES user_id=21627 The tuples should get deallocated when Py_Finalize is called. It would be good if there was (conditional) statistical analysis, showing how often no tuple was found because the number of arguments was too large, and how often no tuple was found because the candidate was in use. I think it should be more stack-like, starting off with no tuples allocated, then returning them inside the needs_free blocks only if the refcount is 1 (or 2?). This would avoid degeneralized cases where some function holds onto its argument tuple indefinitely, thus consuming all 64 tuples. For the other part, I think it would make the code more readable if it inlined PyCFunction_Call even more: the test for NOARGS|O could be integrated into the switch statement (one case for each), VARARGS and VARARGS|KEYWORDS would both load the arguments, then call the function directly (possibly with NULL keywords). OLDARGS should goto either METH_NOARGS, METH_O, or METH_VARARGS depending on na (if you don't like goto, modifying flags would work as well). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-05-01 09:08 Message: Logged In: YES user_id=33168 I should note the numbers 64 and 8 are total guesses. It might be good to try and determine values based on empirical data. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 From noreply at sourceforge.net Mon May 1 11:24:43 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 02:24:43 -0700 Subject: [Patches] [ python-Patches-1473132 ] Improve docs for tp_clear and tp_traverse Message-ID: Patches item #1473132, was opened at 2006-04-19 19:43 Message generated for change (Comment added) made by twouters You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1473132&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Collin Winter (collinwinter) Assigned to: Nobody/Anonymous (nobody) Summary: Improve docs for tp_clear and tp_traverse Initial Comment: The attached patch greatly enhances the documentation for the tp_clear and tp_traverse functions. The patch is against Doc/api/newtypes.tex, r45562. ---------------------------------------------------------------------- >Comment By: Thomas Wouters (twouters) Date: 2006-05-01 11:24 Message: Logged In: YES user_id=34209 As Tim said, there is more to it :) I think this is a fine start, though. One minor point: the use of Py_CLEAR() can do with some extra explanation. It obviously isn't enough to just 'NULL out' members, since that would leak references, but the docs should also explain that it is in fact important to set the actual member to NULL *before* DECREFing the reference, and then point out that the Py_CLEAR macro is a convenient way of doing that. That kind of tweak can happen after it's checked in, though (preferably by someone who can build documentation and see that the result looks okay ;) ---------------------------------------------------------------------- Comment By: Collin Winter (collinwinter) Date: 2006-05-01 05:43 Message: Logged In: YES user_id=1344176 I've enhanced the patch per Tim Peters' comment. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2006-04-23 05:30 Message: Logged In: YES user_id=31435 I agree the additional info is helpful (thanks!). Alas, there's more to it, and it's hard to know when to stop :-(. For example, an author of a type may _want_ to visit, e.g., contained strings in tp_traverse, because they want gc.get_referents() to return the contained strings (typically as a debugging aid). The issues wrt to tp_clear are subtler. The real requirement is that the aggregate of all tp_clears called break all possible cycles. For one thing, that means there's no real reason for a tp_clear to touch a member that's known to be a Python string or integer (since such an object can't be in a cycle, clearing it can't help to break a cycle). It's only tp_dealloc that _must_ drop references to all containees. Subtler is that a gc'ed container type may choose not to implement tp_clear at all. If you look, you'll see that Python's tuple type in fact leaves its tp_clear slot empty. This isn't a problem because it's impossible to have a cycle composed _solely_ of tuples (that may not be obvious, but it's true -- it derives from that tuples are immutable). Any cycle a tuple may be in will be broken if the non-tuple objects in the cycle clear their containees, so there's no actually need for tuples to have a tp_clear. The possibility should be mentioned, although it's fine to recommend playing it safe. Indeed, I don't think it buys anything worth having for tuples not to have an obvious tp_clear implementation. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1473132&group_id=5470 From noreply at sourceforge.net Mon May 1 13:18:26 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 04:18:26 -0700 Subject: [Patches] [ python-Patches-1474907 ] detect %zd format for PY_FORMAT_SIZE_T Message-ID: Patches item #1474907, was opened at 2006-04-23 08:18 Message generated for change (Settings changed) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1474907&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: None Status: Open Resolution: None Priority: 5 Submitted By: Brett Cannon (bcannon) >Assigned to: Brett Cannon (bcannon) Summary: detect %zd format for PY_FORMAT_SIZE_T Initial Comment: The patch modifies configure.in to add PY_FORMAT_SIZE_T to configure.in (meaning you need to run autoheader on configure.in) so that if %zd is supported for size_t it sets PY_FORMAT_SIZE_T to "z", otherwise it goes undefined and the preprocessor trickery in Include/pyport.h kicks in. This fix removes compiler warnings on OS X 10.4.6 with gcc 4.0.1 thanks to PY_FORMAT_SIZE_T being set to "". Initially assigned to Martin v. Loewis since he said this would be good to do and the Py_ssize_t stuff is his invention. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-04-28 06:51 Message: Logged In: YES user_id=357491 Yeah, I tried to use a string constant as a stack value, but that didn't work. =) My brain just was not thinking in C when I first came up with the patch. I have a new version that uses a char array as the buffer. I am on vacation so I don't have the time to apply it and break buildbot, so I will hold off on applying if no one finds problems with this version. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-04-27 07:29 Message: Logged In: YES user_id=21627 Looks fine to me, although it has "unusual" style of C: - sizeof(char) is guaranteed to be 1 by the C standard. The C standard defines "char" and "byte" as synonyms, even if that means that "byte" has more than 8 bits. sizeof gives the number of bytes, so for char, it is always 1. - for a fixed-size array, people would normally make this an automatic (stack) variable, instead of bothering with explicit memory allocation, i.e. char str_buffer[4] Just out of fear of buffer overruns, many people would also add some horrendous overallocation, such as str_buffer[1024] :-) ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-04-27 07:16 Message: Logged In: YES user_id=357491 Realized there is a better way: just strncmp() for the expected result. Uploaded a new version. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-04-27 06:59 Message: Logged In: YES user_id=357491 OK, uploaded a new version that uses strchr to check for '%', 'z', and 'd'. If it looks reasonable I will apply it and hope I don't break the buildbot. =) ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-04-26 18:15 Message: Logged In: YES user_id=21627 The patch seems to rely on printf returning <0 for the unrecognized format. That seems unreliable: atleast on Linux, printf just outputs the format as-is for unrecognized formats. Instead, I think it should use sprintf, and then check whether the result is the string "0" (in addition to checking whether the printf call itself failed). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1474907&group_id=5470 From noreply at sourceforge.net Mon May 1 21:50:49 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 12:50:49 -0700 Subject: [Patches] [ python-Patches-1479977 ] Heavy revisions to urllib2 howto Message-ID: Patches item #1479977, was opened at 2006-05-01 20:50 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479977&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: Heavy revisions to urllib2 howto Initial Comment: Lots of people have been complaining about lack of urllib2 docs (though I'm never quite sure what people are looking for, being too familiar with all the details), so a tutorial may well be a useful addition. I'm sure you'll understand that my brutal criticism :-) is intended to make it even more useful. Michael: feel free to make further revisions, but unless you have major objections I suggest that this is checked in first, then we make any further changes after that by uploading patches on SF for review (I haven't stepped back and re-read it with a fresh mind, and no doubt would be useful for somebody to do that). Editing this took me quite a while, and if I can help it I don't want to go through too many revisions or argue about the details before anything gets fixed!-). I've taken the liberty of mentioning myself as a reviewer somewhere at the end of the document :-) Important: I reformatted paragraphs to max 70 character width (it's conventional, and plain-text diffs are especially painful to read otherwise, though admittedly diffs are never great for paragraphs anyway... I hope emacs didn't muck up any ReST syntax). I've uploaded just that formatting change as reformatted.rst (which also removes trailing whitespace from all lines). This should be done in a separate initial commit of course. For this reason, I've uploaded the whole document for both reformatted (reformatted.rst) and edited versions (edited.rst) rather than using patches. I've made all of the changes I discuss below, *with the exception of* the missing example of GET with urlencoded data that's really needed (search for XXX in the comments below) -- that should just need a few lines. BTW, it would be a really fantastic idea to turn the whole document into a valid doctest (I know I'm myself almost incapable of writing correct examples unless I do something like that). All that would require of course is adding a few >>>s and ...s and running it through doctest.testfile until it stops complaining ;-) Now a list explaining and justifying the changes I made: Spelling / paragraph structure etc. fixes. I won't list these. Most importantly, you seem a bit unsure who your audience is. For example, on headers -- you explain that "HTTP is based on requests and responses", but dive into User-Agent without actually mentioning what a header is. In my changes, I ended up adding brief explanations of the concepts for people new to or fuzzy about HTTP, but didn't go into details of implementation. For example, introducing the concept of "HTTP header", but not explaining how HTTP implements them "on the wire" (though in fact I think it would be a good thing to add one example that showed an HTTP request and pointed out the request line, the headers and the data, since that makes everything very concrete and easy to grasp for newbies). Removed link to external howto on cookie handling. Despite the description ("How to handle cookies, when fetching web pages with Python."), this actually spends most of its time discussing what conditional imports are needed if you want to be maximally compatible across libraries and older versions of Python. While that is certainly useful for people who need that, I think this is rather obscure and distracting detail that seems out of place being referenced from the Python 2.5 documentation, even in a howto. Perhaps some general statement that further tutorials are available on your site? Referencing your basic auth tutorial seems fine. You limit mention of urllib2.urlopen(url) to a footnote, and in the text of the tutorial itself, you say: """urllib2 mirrors this by having you form a ``Request``""" . That's not true: a string URL is fine, as you explain in the footnote. That seems an innaccuracy with no obvious didactic payoff. In the footnote, you say: """You *can* fetch URLs directly with urlopen, without using a request object. It's more explicit, and therefore more Pythonic, to use ``urllib2.Request`` though. It also makes it easier to add headers to your request. I find that bizarre! Why is urlopen(url) unpythonic?? On the contrary, using an extra object for no reason *does* seem unpythonic to me. I rewrote this a bit. You needlessly assign the_url = "http:...", then request = Request(the_url) -- why not a single line? Where it's useful to do that (i.e. in the more complicated examples), I've s/the_url/url/, since I object to chaff like "the_" in variable names ;-) Your discussion of Request implies that it only represents HTTP requests. Fixed that. Use of the word "handle" to talk about response objects is unfortunate for two reasons: First, many objects in Python are "handles" in some sense ("object reference" semantics), so it's too vague to be a helpful name. Second, it's particularly unfortunate to use the word "handle" when urllib2 makes heavy use of "handler" objects that "handle" requests. The fact that methods on these handlers often return your "handles" only makes things more confusing! s/handle/response/ """Sometimes you want to **POST** data to a CGI (Common Gateway Interface) [#]_ or other web application""" It's clear to us old hands what you mean here, but in a tutorial at the level you seem to have picked we probably shouldn't expect the reader to have all these concepts straight, so being sloppy here is bad. - By "a CGI" I'm guessing you mean "a CGI script/program". Also, the whole sentence is unclear whether you're talking about a web application in the abstract, or some concrete CGI script. I certainly remember being very confused about this kind of thing as a newbie. - "...or other web application" implies that all POSTs go to web applications. That's using "web application" in a broader sense than it's usually understood. - You introduce "POST" without explanation. Would be nice to say "send data" instead of "POST", then explain POST. I rewrote this bit to try to address those points. Re POST: """This is what your browser does when you fill in a FORM on the web""" Thats needed qualifying: form submission can also result in a GET. I added a bit on side-effects and GET/POST. """You may be mimicking a FORM submission, or transmitting data to your own application.""" This reads oddly to me. I know what you're getting at (forms are not part of HTTP), but surely if you are submitting form data you're not "mimicking" form submission, you *are* submitting a form. And in an English sentence the "or" reads as an "exclusive or"; with that in mind: In what sense does form submission *not* involve "transmitting data to your own application"? Reworded and s/FORM/HTML form/, since we're talking about the abstract thing rather than specifically about the HTML element. """In either case the data needs to be encoded for safe transmission over HTTP""" Arbitrary binary data does not need to be URL-encoded. Rephrased. """The encoding is done using a function from the ``urllib`` library *not* from ``urllib2``. ::""" This is not true in general even for HTML forms. For example, HTML form file upload data is not encoded in this way. There are more obscure cases, too. Noted this. The quoted User-Agent string was out-of-date. Fixed, noting that it changes with each minor Python version. Headers / data : I added a bit of explanatory context to tell people what we're about to explain, and break up paragraphs / add sections to clarify the structure. Also explained the concept of "HTTP header", as I noted above. XXX example needed on GET with urlencoded data (as it's written ATM, this would go immediately before the "Headers" section). """Coping With Errors""" "Handling exceptions" seems more accurate. Not all HTTP status codes for which urllib2 raises an exception involve HTTP error responses. The text is also confused on this point, so I rewrote it. Errors: I believe urlopen can still actually raise socket.error. This is a bug, but I haven't dared to submit a patch to fix it, fearing backwards-compatibility issues. I guess it should probably be documented :-( But I suppose we should discuss that in a separate tracker item, rather than adding it to your howto straight away. You mention IOError. Without a motivating use case I don't know why you mention this. Since I'm not really sure what the use case for this subclassing was ever intended to be :-) I removed this example: feel free to add it back if you know of a use or can get Jeremy Hylton to explain it to you ;-) Re URLError : you imply that the only reason for URLError to be raised is failure to connect to the server. This is often the cause, but certainly not always. For HTTP status codes, you refer to a document that states "This is a historic document and is not accurate anymore". RFC 2616 is authoritative, and IMHO fairly readable on error codes. Removed the reference to the other document. """As of Python 2.5 a dictionary like this one has become part of ``urllib2``.""" In fact, this was moved to httplib. The reference to "HTTPBaseServer" (sic) is interesting: I think the copy in httplib should be removed, since it's already there in BaseHTTPServer (albeit missing 306, but that is unused) -- would you mind filing a patch, Michael? Your listing differed from BaseHTTPServer and from RFC 2616, so I replaced it with the BaseHTTPServer copy. """shows all the defined response codes""" These are only those defined by RFC 2616 of course: other standards can and do define other response status codes (e.g. DAV). Clarified this. """When an error is raised the server responds by returning an http error code *and* an error page.""" This is sloppy: HTTP doesn't define "raising" an error, so it can't respond to one. Fixed. httplib.HTTPMessage Reworded to avoid impling it's *always* going to be this concrete class. """In versions of Python prior to 2.3.4 it wasn't safe to iterate over the object directly, so you should iterate over the list returned by ``msg.keys()`` instead.""" Is this appropriate advice in the 2.5 docs? I removed this (am I too harsh on this point?). """Openers and handlers are slightly esoteric parts of **urllib2**.""" I don't want to scare people off: they're easy to use (if not to write). Removed this. I added a tiny bit more on what handlers do. Changed the text to avoid implying that build_opener() is the only way to create openers. Don't refer to ``opener`` in those typewriter-font ReST backticks, since that seems a little misleading: it's not a Python class name (unfortunately the class is named OpenerDirector, which rather clashes with the use of the name "opener" of course, but personally I'm with you in preferring "opener"). Wrote a bit more about opener construction. Changed realm name to make it clear it may contain spaces. Changed references to URI to URL in discussion of authentication -- seems an irrelevant and distracting distinction here. I edited the basic auth description a little. Comments conventionally come *before* code it refers to, not after. Fixed that, removed an over-obvious comment or two (even in docs, "create the handler" seems redundant if that's *all* it says), and the fixed the curious line breaks. """The only reason to explicitly supply these to ``build_opener`` (which chains handlers provided as a list), would be to change the order they appear in the chain.""" I don't know of a use case for that in the case of the handlers you list. Also, that doesn't actually work: handler ordering is determined by sorting. Removed this. """One thing not to get bitten by is that the ``top_level_url`` in the code above *must not* contain the protocol - the ``http://`` part. So if the URL we are trying to access is""" This is not correct usage (though I can see why it worked); removed it. Admittedly, urllib2 auth was the subject of a quite a few bug fixes recently (I seem to have just found yet another one five minutes ago, in fact :-( ), so the situation pre-2.5 was certainly messy. However, I advise against trying to document the old bugs! Note that I haven't given examples of "sub-URLs" since the RFC (2617) isn't clear to me on this point, and I haven't yet tested whether urllib2 gets it right according to de-facto standards (as defined by browsers, Apache, etc.) for "sub-URLs" of the one passed to .add_password(). It's on the list... In your note explaining that HTTPS proxies are not supported, you use "caution" rather than "note", which conveys the strange implication to me that this lack of support is somehow a consequence of using your previous recipe for switching off proxy handling (or am I weird in reading it that way??). s/caution/note/ """.. [#] Possibly some of this tutorial will make it into the standard library docs for versions of Python after 2.4.1.""" Removed this. Whew! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479977&group_id=5470 From noreply at sourceforge.net Mon May 1 21:59:20 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 12:59:20 -0700 Subject: [Patches] [ python-Patches-1479977 ] Heavy revisions to urllib2 howto Message-ID: Patches item #1479977, was opened at 2006-05-01 20:50 Message generated for change (Comment added) made by jjlee You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479977&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: Heavy revisions to urllib2 howto Initial Comment: Lots of people have been complaining about lack of urllib2 docs (though I'm never quite sure what people are looking for, being too familiar with all the details), so a tutorial may well be a useful addition. I'm sure you'll understand that my brutal criticism :-) is intended to make it even more useful. Michael: feel free to make further revisions, but unless you have major objections I suggest that this is checked in first, then we make any further changes after that by uploading patches on SF for review (I haven't stepped back and re-read it with a fresh mind, and no doubt would be useful for somebody to do that). Editing this took me quite a while, and if I can help it I don't want to go through too many revisions or argue about the details before anything gets fixed!-). I've taken the liberty of mentioning myself as a reviewer somewhere at the end of the document :-) Important: I reformatted paragraphs to max 70 character width (it's conventional, and plain-text diffs are especially painful to read otherwise, though admittedly diffs are never great for paragraphs anyway... I hope emacs didn't muck up any ReST syntax). I've uploaded just that formatting change as reformatted.rst (which also removes trailing whitespace from all lines). This should be done in a separate initial commit of course. For this reason, I've uploaded the whole document for both reformatted (reformatted.rst) and edited versions (edited.rst) rather than using patches. I've made all of the changes I discuss below, *with the exception of* the missing example of GET with urlencoded data that's really needed (search for XXX in the comments below) -- that should just need a few lines. BTW, it would be a really fantastic idea to turn the whole document into a valid doctest (I know I'm myself almost incapable of writing correct examples unless I do something like that). All that would require of course is adding a few >>>s and ...s and running it through doctest.testfile until it stops complaining ;-) Now a list explaining and justifying the changes I made: Spelling / paragraph structure etc. fixes. I won't list these. Most importantly, you seem a bit unsure who your audience is. For example, on headers -- you explain that "HTTP is based on requests and responses", but dive into User-Agent without actually mentioning what a header is. In my changes, I ended up adding brief explanations of the concepts for people new to or fuzzy about HTTP, but didn't go into details of implementation. For example, introducing the concept of "HTTP header", but not explaining how HTTP implements them "on the wire" (though in fact I think it would be a good thing to add one example that showed an HTTP request and pointed out the request line, the headers and the data, since that makes everything very concrete and easy to grasp for newbies). Removed link to external howto on cookie handling. Despite the description ("How to handle cookies, when fetching web pages with Python."), this actually spends most of its time discussing what conditional imports are needed if you want to be maximally compatible across libraries and older versions of Python. While that is certainly useful for people who need that, I think this is rather obscure and distracting detail that seems out of place being referenced from the Python 2.5 documentation, even in a howto. Perhaps some general statement that further tutorials are available on your site? Referencing your basic auth tutorial seems fine. You limit mention of urllib2.urlopen(url) to a footnote, and in the text of the tutorial itself, you say: """urllib2 mirrors this by having you form a ``Request``""" . That's not true: a string URL is fine, as you explain in the footnote. That seems an innaccuracy with no obvious didactic payoff. In the footnote, you say: """You *can* fetch URLs directly with urlopen, without using a request object. It's more explicit, and therefore more Pythonic, to use ``urllib2.Request`` though. It also makes it easier to add headers to your request. I find that bizarre! Why is urlopen(url) unpythonic?? On the contrary, using an extra object for no reason *does* seem unpythonic to me. I rewrote this a bit. You needlessly assign the_url = "http:...", then request = Request(the_url) -- why not a single line? Where it's useful to do that (i.e. in the more complicated examples), I've s/the_url/url/, since I object to chaff like "the_" in variable names ;-) Your discussion of Request implies that it only represents HTTP requests. Fixed that. Use of the word "handle" to talk about response objects is unfortunate for two reasons: First, many objects in Python are "handles" in some sense ("object reference" semantics), so it's too vague to be a helpful name. Second, it's particularly unfortunate to use the word "handle" when urllib2 makes heavy use of "handler" objects that "handle" requests. The fact that methods on these handlers often return your "handles" only makes things more confusing! s/handle/response/ """Sometimes you want to **POST** data to a CGI (Common Gateway Interface) [#]_ or other web application""" It's clear to us old hands what you mean here, but in a tutorial at the level you seem to have picked we probably shouldn't expect the reader to have all these concepts straight, so being sloppy here is bad. - By "a CGI" I'm guessing you mean "a CGI script/program". Also, the whole sentence is unclear whether you're talking about a web application in the abstract, or some concrete CGI script. I certainly remember being very confused about this kind of thing as a newbie. - "...or other web application" implies that all POSTs go to web applications. That's using "web application" in a broader sense than it's usually understood. - You introduce "POST" without explanation. Would be nice to say "send data" instead of "POST", then explain POST. I rewrote this bit to try to address those points. Re POST: """This is what your browser does when you fill in a FORM on the web""" Thats needed qualifying: form submission can also result in a GET. I added a bit on side-effects and GET/POST. """You may be mimicking a FORM submission, or transmitting data to your own application.""" This reads oddly to me. I know what you're getting at (forms are not part of HTTP), but surely if you are submitting form data you're not "mimicking" form submission, you *are* submitting a form. And in an English sentence the "or" reads as an "exclusive or"; with that in mind: In what sense does form submission *not* involve "transmitting data to your own application"? Reworded and s/FORM/HTML form/, since we're talking about the abstract thing rather than specifically about the HTML element. """In either case the data needs to be encoded for safe transmission over HTTP""" Arbitrary binary data does not need to be URL-encoded. Rephrased. """The encoding is done using a function from the ``urllib`` library *not* from ``urllib2``. ::""" This is not true in general even for HTML forms. For example, HTML form file upload data is not encoded in this way. There are more obscure cases, too. Noted this. The quoted User-Agent string was out-of-date. Fixed, noting that it changes with each minor Python version. Headers / data : I added a bit of explanatory context to tell people what we're about to explain, and break up paragraphs / add sections to clarify the structure. Also explained the concept of "HTTP header", as I noted above. XXX example needed on GET with urlencoded data (as it's written ATM, this would go immediately before the "Headers" section). """Coping With Errors""" "Handling exceptions" seems more accurate. Not all HTTP status codes for which urllib2 raises an exception involve HTTP error responses. The text is also confused on this point, so I rewrote it. Errors: I believe urlopen can still actually raise socket.error. This is a bug, but I haven't dared to submit a patch to fix it, fearing backwards-compatibility issues. I guess it should probably be documented :-( But I suppose we should discuss that in a separate tracker item, rather than adding it to your howto straight away. You mention IOError. Without a motivating use case I don't know why you mention this. Since I'm not really sure what the use case for this subclassing was ever intended to be :-) I removed this example: feel free to add it back if you know of a use or can get Jeremy Hylton to explain it to you ;-) Re URLError : you imply that the only reason for URLError to be raised is failure to connect to the server. This is often the cause, but certainly not always. For HTTP status codes, you refer to a document that states "This is a historic document and is not accurate anymore". RFC 2616 is authoritative, and IMHO fairly readable on error codes. Removed the reference to the other document. """As of Python 2.5 a dictionary like this one has become part of ``urllib2``.""" In fact, this was moved to httplib. The reference to "HTTPBaseServer" (sic) is interesting: I think the copy in httplib should be removed, since it's already there in BaseHTTPServer (albeit missing 306, but that is unused) -- would you mind filing a patch, Michael? Your listing differed from BaseHTTPServer and from RFC 2616, so I replaced it with the BaseHTTPServer copy. """shows all the defined response codes""" These are only those defined by RFC 2616 of course: other standards can and do define other response status codes (e.g. DAV). Clarified this. """When an error is raised the server responds by returning an http error code *and* an error page.""" This is sloppy: HTTP doesn't define "raising" an error, so it can't respond to one. Fixed. httplib.HTTPMessage Reworded to avoid impling it's *always* going to be this concrete class. """In versions of Python prior to 2.3.4 it wasn't safe to iterate over the object directly, so you should iterate over the list returned by ``msg.keys()`` instead.""" Is this appropriate advice in the 2.5 docs? I removed this (am I too harsh on this point?). """Openers and handlers are slightly esoteric parts of **urllib2**.""" I don't want to scare people off: they're easy to use (if not to write). Removed this. I added a tiny bit more on what handlers do. Changed the text to avoid implying that build_opener() is the only way to create openers. Don't refer to ``opener`` in those typewriter-font ReST backticks, since that seems a little misleading: it's not a Python class name (unfortunately the class is named OpenerDirector, which rather clashes with the use of the name "opener" of course, but personally I'm with you in preferring "opener"). Wrote a bit more about opener construction. Changed realm name to make it clear it may contain spaces. Changed references to URI to URL in discussion of authentication -- seems an irrelevant and distracting distinction here. I edited the basic auth description a little. Comments conventionally come *before* code it refers to, not after. Fixed that, removed an over-obvious comment or two (even in docs, "create the handler" seems redundant if that's *all* it says), and the fixed the curious line breaks. """The only reason to explicitly supply these to ``build_opener`` (which chains handlers provided as a list), would be to change the order they appear in the chain.""" I don't know of a use case for that in the case of the handlers you list. Also, that doesn't actually work: handler ordering is determined by sorting. Removed this. """One thing not to get bitten by is that the ``top_level_url`` in the code above *must not* contain the protocol - the ``http://`` part. So if the URL we are trying to access is""" This is not correct usage (though I can see why it worked); removed it. Admittedly, urllib2 auth was the subject of a quite a few bug fixes recently (I seem to have just found yet another one five minutes ago, in fact :-( ), so the situation pre-2.5 was certainly messy. However, I advise against trying to document the old bugs! Note that I haven't given examples of "sub-URLs" since the RFC (2617) isn't clear to me on this point, and I haven't yet tested whether urllib2 gets it right according to de-facto standards (as defined by browsers, Apache, etc.) for "sub-URLs" of the one passed to .add_password(). It's on the list... In your note explaining that HTTPS proxies are not supported, you use "caution" rather than "note", which conveys the strange implication to me that this lack of support is somehow a consequence of using your previous recipe for switching off proxy handling (or am I weird in reading it that way??). s/caution/note/ """.. [#] Possibly some of this tutorial will make it into the standard library docs for versions of Python after 2.4.1.""" Removed this. Whew! ---------------------------------------------------------------------- >Comment By: John J Lee (jjlee) Date: 2006-05-01 20:59 Message: Logged In: YES user_id=261020 (I guess if I had any sense in me, I would have uploaded those comments as an attachment instead of pasting them into the summary -- sorry.) I'm uploading the revised document now. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479977&group_id=5470 From noreply at sourceforge.net Mon May 1 22:08:14 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 13:08:14 -0700 Subject: [Patches] [ python-Patches-1479988 ] weakref dict methods Message-ID: Patches item #1479988, was opened at 2006-05-01 16:08 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479988&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Fred L. Drake, Jr. (fdrake) Assigned to: Tim Peters (tim_one) Summary: weakref dict methods Initial Comment: The WeakKeyDictionary and WeakValueDictionary don't provide any API to get just the weakrefs out, instead of the usual mapping API. This can be desirable when you want to get a list of everything without creating new references to the underlying objects at that moment. This patch adds methods to make the references themselves accessible using the API, avoiding requiring client code to have to depend on the implementation. The WeakKeyDictionary gains the .iterkeyrefs() and .keyrefs() methods, and the WeakValueDictionary gains the .itervaluerefs() and .valuerefs() methods. The patch includes tests and docs. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479988&group_id=5470 From noreply at sourceforge.net Mon May 1 22:17:47 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 13:17:47 -0700 Subject: [Patches] [ python-Patches-1411097 ] httplib patch to make _read_chunked() more robust Message-ID: Patches item #1411097, was opened at 2006-01-20 20:26 Message generated for change (Settings changed) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1411097&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. >Category: None >Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: httplib patch to make _read_chunked() more robust Initial Comment: To reproduce: import urllib2 print urllib2.urlopen("http://66.117.37.13/").read() The attached patch "fixes" the hang, but that patch is not acceptable because it also removes the .readline() and .readlines() methods on the response object returned by urllib2.urlopen(). The patch seems to demonstrate that the problem is caused by the (ab)use of socket._fileobject in urllib2.AbstractHTTPHandler (I believe this hack was introduced when urllib2 switched to using httplib.HTTPConnection). Not sure yet what the actual problem is... ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 21:20 Message: Logged In: YES user_id=261020 Please ignore last comment (posted to wrong tracker item). ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 21:18 Message: Logged In: YES user_id=261020 Conservative or not, I see no utility in changing the default, and several major harmful effects: old code breaks, and people have to pore over the specs to figure out why "urlopen() doesn't work". ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 20:36 Message: Logged In: YES user_id=261020 I missed the fact that, if the connection will not close at the end of the transaction, the behaviour should not change from what's currently in SVN (we should not assume that the chunked response has ended unless we see the proper terminating CRLF). I intend to upload a slightly modified patch that tests for self._will_close, and behaves accordingly. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 01:24 Message: Logged In: YES user_id=261020 Oops, fixed chunk.patch to .strip() before comparing to "". ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 00:38 Message: Logged In: YES user_id=261020 First, expanding a bit on what I wrote on 2006-01-21: The problem does relate to chunked encoding, and is unrelated to urllib2's use of _fileobject. My hack to remove use of socket._fileobject from urllib2 merely breaks handling of chunked encoding by cutting httplib.HTTPResponse out of the picture. The problem is seen in urllib2 in recent Pythons thanks to urllib2 switching to use of httplib.HTTPConnection and HTTP/1.1, hence chunked encoding is allowed. urllib still uses httplib.HTTP, hence HTTP/1.0, so is unaffected. To reproduce with httplib: import httplib conn = httplib.HTTPConnection("66.117.37.13") conn.request("GET", "/", headers={"Connection": "close"}) r1 = conn.getresponse() print r1.read() The Connection: close is required -- if it's not there the server doesn't use chunked transfer-encoding. I verified with a packet sniffer that the problem is that this server does not send the final trailing CRLF required by section 3.6.1 of RFC 2616. However, that section also says that trailers (trailing HTTP headers) MUST NOT be sent by the server unless either a TE header was present and indicated that trailers are acceptable (httplib does not send the TE header), or the trailers are optional metadata and may be discarded by the client. So, I propose the attached patch to httplib (chunk.patch) as a work-around. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-01-21 22:10 Message: Logged In: YES user_id=261020 In fact the commit message for rev 36871 states the real reason _fileobject is used (handling chunked encoding), showing my workaround is even more harmful than I thought. Moreover, doing a urlopen on 66.117.37.13 shows the response *is* chunked. The problem seems to be caused by httplib failing to find a CRLF at the end of the chunked response, so the loop at the end of _read_chunked() never terminates. Haven't looked in detail yet, but I'm guessing a) it's the server's fault and b) httplib should work around it. Here's the commit message from 36871: Fix urllib2.urlopen() handling of chunked content encoding. The change to use the newer httplib interface admitted the possibility that we'd get an HTTP/1.1 chunked response, but the code didn't handle it correctly. The raw socket object can't be pass to addinfourl(), because it would read the undecoded response. Instead, addinfourl() must call HTTPResponse.read(), which will handle the decoding. One extra wrinkle is that the HTTPReponse object can't be passed to addinfourl() either, because it doesn't implement readline() or readlines(). As a quick hack, use socket._fileobject(), which implements those methods on top of a read buffer. (suggested by mwh) Finally, add some tests based on test_urllibnet. Thanks to Andrew Sawyers for originally reporting the chunked problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1411097&group_id=5470 From noreply at sourceforge.net Mon May 1 23:20:35 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 14:20:35 -0700 Subject: [Patches] [ python-Patches-1479988 ] weakref dict methods Message-ID: Patches item #1479988, was opened at 2006-05-01 16:08 Message generated for change (Comment added) made by fdrake You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479988&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Fred L. Drake, Jr. (fdrake) Assigned to: Tim Peters (tim_one) Summary: weakref dict methods Initial Comment: The WeakKeyDictionary and WeakValueDictionary don't provide any API to get just the weakrefs out, instead of the usual mapping API. This can be desirable when you want to get a list of everything without creating new references to the underlying objects at that moment. This patch adds methods to make the references themselves accessible using the API, avoiding requiring client code to have to depend on the implementation. The WeakKeyDictionary gains the .iterkeyrefs() and .keyrefs() methods, and the WeakValueDictionary gains the .itervaluerefs() and .valuerefs() methods. The patch includes tests and docs. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2006-05-01 17:20 Message: Logged In: YES user_id=3066 Tim noted in email: http://mail.python.org/pipermail/python-dev/2006-May/064751.html that the implementation could and probably should be simplified. This second version of the patch does that, and updates the documentation to note the liveness issues of the references, as well as avoid repetition. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479988&group_id=5470 From noreply at sourceforge.net Tue May 2 00:35:47 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 15:35:47 -0700 Subject: [Patches] [ python-Patches-1480067 ] urllib2 digest auth redirection bug causes 400 error Message-ID: Patches item #1480067, was opened at 2006-05-01 23:35 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1480067&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2 digest auth redirection bug causes 400 error Initial Comment: urllib2 redirects HTTP digest authorisation credentials, which is never useful (because the redirection will change the digest), and may cause a 400 error if for example the handler finds credentials for an initial request, but fails to finds credentials for a redirected request. In that case a stale Authorization or Proxy-authorization header will get returned to the server, causing a 400 error. I've verified this makes the 400 go away for example in the case where http://localhost/foo gets 301 redirected to http://127.0.0.1/foo/ (i.e. with a slash on the end), where I've only added username/password for "localhost" and not "127.0.0.1". The fix is trivial. 2.4 backport candidate. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1480067&group_id=5470 From noreply at sourceforge.net Tue May 2 03:22:17 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 18:22:17 -0700 Subject: [Patches] [ python-Patches-1216942 ] Suggested Additional Material for urllib2 docs Message-ID: Patches item #1216942, was opened at 2005-06-08 10:41 Message generated for change (Comment added) made by jjlee You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1216942&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: None Status: Open Resolution: None Priority: 5 Submitted By: Mike Foord (mjfoord) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: Suggested Additional Material for urllib2 docs Initial Comment: This is some suggested additional material for the urllib2 docs. Particularly the part about error codes and the reason/code attributes of error objects is *missing* from the manual and needed. Also the example showing basic Authentication using password manager/handler/opener may help avoid some confusion. Alternatively you can link to my online tutorials at http://www.voidspace.org.uk/python/articles.shtml#http :-) ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-05-02 02:22 Message: Logged In: YES user_id=261020 Andrew Kuchling checked in Michael's tutorial as a howto, so I guess this can be closed. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2005-12-22 22:07 Message: Logged In: YES user_id=261020 Just to shout it out again: no need for said patches to contain TeX markup!-) Plain text / reST pasted into the existing docs is ok (though making it clear by some means what is a heading and what isn't &c. is obviously desirable). I only want a patch because that would make it clear how the additions are intended to be integrated with the existing docs. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2005-12-22 22:01 Message: Logged In: YES user_id=261020 Fred, what will you TeXify? Are you waiting for Mike to reply, or were you saying that you'll TeXify what he already submitted? Personally, I'm not happy with the original as-is, foremost because it's not clear how it is intended to fit with the existing docs (there are certainly other problems with the suggested additions, but not much point going into detail before there's a patch). I would be happy to review / edit at least some of the content it if it were presented as patch(es). ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2005-12-22 14:53 Message: Logged In: YES user_id=3066 I'll TeXify. I agree with John about reproducing the response code listing; that's a good place to simply defer to the HTTP spec. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2005-12-04 20:01 Message: Logged In: YES user_id=261020 I'm sure doc improvements are welcome here, so thank you :) However, I think you need to 1) split this up into small patches that address very specific issues, and briefly justify each change in the patch submission note on the SF patch tracker 2) present the patches by editing the original .tex source files from src/Doc/lib and then running 'diff -u' or 'svn diff' (it doesn't matter if you can't compile the docs or get the TeX markup wrong, just as long as everybody can see exactly what the intended changes to the text are) Also, one thing that caught my eye on a very brief scan was that the actual response code->name mapping (rather than a note to document the existence of that mapping) shouldn't be reproduced in the docs, I think. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1216942&group_id=5470 From noreply at sourceforge.net Tue May 2 06:43:29 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 21:43:29 -0700 Subject: [Patches] [ python-Patches-1479181 ] Split open() and file() Message-ID: Patches item #1479181, was opened at 2006-04-29 21:12 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479181&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Aahz (aahz) Assigned to: Neal Norwitz (nnorwitz) Summary: Split open() and file() Initial Comment: Make open() a factory function instead of an alias to file(). Includes doc patches and a bugfix to Lib/test/test_subprocess.py (which was relying on open() being an alias to file()). There were no other changes to the test suite -- this appears to be a completely transparent fix. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-05-01 21:43 Message: Logged In: YES user_id=33168 Minor doc mods. Committed revision 45850. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479181&group_id=5470 From noreply at sourceforge.net Tue May 2 08:13:42 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 23:13:42 -0700 Subject: [Patches] [ python-Patches-1479988 ] weakref dict methods Message-ID: Patches item #1479988, was opened at 2006-05-01 16:08 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479988&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open >Resolution: Accepted Priority: 5 Submitted By: Fred L. Drake, Jr. (fdrake) >Assigned to: Fred L. Drake, Jr. (fdrake) Summary: weakref dict methods Initial Comment: The WeakKeyDictionary and WeakValueDictionary don't provide any API to get just the weakrefs out, instead of the usual mapping API. This can be desirable when you want to get a list of everything without creating new references to the underlying objects at that moment. This patch adds methods to make the references themselves accessible using the API, avoiding requiring client code to have to depend on the implementation. The WeakKeyDictionary gains the .iterkeyrefs() and .keyrefs() methods, and the WeakValueDictionary gains the .itervaluerefs() and .valuerefs() methods. The patch includes tests and docs. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2006-05-02 02:13 Message: Logged In: YES user_id=31435 Looks good to me, Fred, and thanks! Marked Accepted and back to you. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2006-05-01 17:20 Message: Logged In: YES user_id=3066 Tim noted in email: http://mail.python.org/pipermail/python-dev/2006-May/064751.html that the implementation could and probably should be simplified. This second version of the patch does that, and updates the documentation to note the liveness issues of the references, as well as avoid repetition. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479988&group_id=5470 From noreply at sourceforge.net Tue May 2 08:55:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 01 May 2006 23:55:25 -0700 Subject: [Patches] [ python-Patches-1479988 ] weakref dict methods Message-ID: Patches item #1479988, was opened at 2006-05-01 16:08 Message generated for change (Comment added) made by fdrake You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479988&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed Resolution: Accepted Priority: 5 Submitted By: Fred L. Drake, Jr. (fdrake) Assigned to: Fred L. Drake, Jr. (fdrake) Summary: weakref dict methods Initial Comment: The WeakKeyDictionary and WeakValueDictionary don't provide any API to get just the weakrefs out, instead of the usual mapping API. This can be desirable when you want to get a list of everything without creating new references to the underlying objects at that moment. This patch adds methods to make the references themselves accessible using the API, avoiding requiring client code to have to depend on the implementation. The WeakKeyDictionary gains the .iterkeyrefs() and .keyrefs() methods, and the WeakValueDictionary gains the .itervaluerefs() and .valuerefs() methods. The patch includes tests and docs. ---------------------------------------------------------------------- >Comment By: Fred L. Drake, Jr. (fdrake) Date: 2006-05-02 02:55 Message: Logged In: YES user_id=3066 Committed in revision 45853. ---------------------------------------------------------------------- Comment By: Tim Peters (tim_one) Date: 2006-05-02 02:13 Message: Logged In: YES user_id=31435 Looks good to me, Fred, and thanks! Marked Accepted and back to you. ---------------------------------------------------------------------- Comment By: Fred L. Drake, Jr. (fdrake) Date: 2006-05-01 17:20 Message: Logged In: YES user_id=3066 Tim noted in email: http://mail.python.org/pipermail/python-dev/2006-May/064751.html that the implementation could and probably should be simplified. This second version of the patch does that, and updates the documentation to note the liveness issues of the references, as well as avoid repetition. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479988&group_id=5470 From noreply at sourceforge.net Tue May 2 13:40:30 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 02 May 2006 04:40:30 -0700 Subject: [Patches] [ python-Patches-1455898 ] patch for mbcs codecs Message-ID: Patches item #1455898, was opened at 2006-03-22 16:31 Message generated for change (Comment added) made by ocean-city You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1455898&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Windows Group: Python 2.4 Status: Open Resolution: None Priority: 7 Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Walter D?rwald (doerwalter) Summary: patch for mbcs codecs Initial Comment: Hello. I have noticed mbcs codecs sometimes generates broken string. I'm using Windows(Japanese) so mbcs is mapped to cp932 (close to shift_jis) When I run the attached script "a.zip", the entry "Error 00007"'s message becomes broken like attached file "b.txt". I think this happens because the string passed to PyUnicode_DecodeMBCS() sometimes terminates with leading byte, and MultiByteToWideChar() counts it for size of result string.buffer size. I hope attached patch "mbcs.patch" may fix the problem. It would be nice if this bug will be fixed in 2.4.3... Thank you. ---------------------------------------------------------------------- >Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-05-02 20:40 Message: Logged In: YES user_id=1200846 I updated the patch. (I copy and pasted "int final = 0" from above code (ex: utf_16_ex_decode), maybe they also should be changed for consistency?) And one more thing, I noticed "errors" is ignored now. We can detect invalid character if we set MB_ERR_INVALID_CHARS flag when calling MultiByteToWideChar, but we cannot tell where is the position of invalid character, and MSDN saids this flag is available Win2000SP4 or later (I don't know why) http://msdn.microsoft.com/library/default.asp?url=/library/en-us/intl/unicode_17si.asp So I didn't make the patch for it. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-04-26 02:22 Message: Logged In: YES user_id=89016 I think the default value for final in mbcs_decode() should be true, so that the stateless decoder detects incomplete byte sequences too. ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-04-07 18:10 Message: Logged In: YES user_id=1200846 I have sent contributor form via postal mail. Probably you can get it after 10 days. ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-28 17:16 Message: Logged In: YES user_id=1200846 You are right. I've updated the patch. (mbcs5.patch) >>> import codecs [20198 refs] >>> d = codecs.getincrementaldecoder("mbcs")() [20198 refs] >>> d.decode('\x82\xa0\x82') u'\u3042' [20198 refs] >>> d.decode('') u'' [20198 refs] >>> d.decode('', final=True) u'\x00' [20198 refs] ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-03-28 01:06 Message: Logged In: YES user_id=89016 _buffer_decode() in the IncrementalDecoder ignores the final argument. IncrementalDecoder._buffer_decode() should pass on its final argument to _codecsmodules.c::mbcs_decode(), which should be extended to accept the final argument. Also PyUnicode_DecodeMBCSStateful() must handle consumed == NULL correctly (with your patch it drops trailing lead bytes even if consumed == NULL) ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-27 16:41 Message: Logged In: YES user_id=1200846 I replaced tests. Probably this is better instead of comparing the two string generated by same decoder. ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-27 14:44 Message: Logged In: YES user_id=1200846 My real name is Hirokazu Yamamoto. But sorry, I don't have FAX. (It's needed to send contributor form, isn't it?) I'll attach the patch updated for trunk. And I'll attach the tests. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-03-27 06:05 Message: Logged In: YES user_id=21627 I have reservations against this patch because of the quasi-anonymous nature of the submission. ocean-city, can you please state your real name? Would you also be willing to fill out a contributor form, as shown on http://www.python.org/psf/contrib-form.html ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-24 23:02 Message: Logged In: YES user_id=1200846 OK, I'll try. ---------------------------------------------------------------------- Comment By: Walter D?rwald (doerwalter) Date: 2006-03-24 06:44 Message: Logged In: YES user_id=89016 This isn't a bugfix in the strictest sense, so IMHO this patch shouldn't go into 2.4. If the patch goes into 2.5, it would need the appropriate changes to encodings/mbcs.py (i.e. the IncrementalDecoder would have to be changed (inheriting from BufferedIncrementalDecoder). I realize that this patch might be hard to test, as results are dependent on locale. Nevertheless at least some tests would be good (even if they are only run or do something useful on a certain locale and are skipped otherwise). ocean-city, can you update the patch for the trunk and add tests? ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-23 11:51 Message: Logged In: YES user_id=1200846 Hello. This is my final patch. (mbcs4.patch) - mbcs3a.patch: _mbsbtype depends on locale not system ANSI code page. so probably it's not good to use it with MultiByteToWideChar. - mbcs3b.patch: CharNext may cause buffer overflow. and this patch always calls CharPrev but it's not needed if string is not terminated with "potensial" lead byte. I hope this is stable enough to commit on repositry. Thank you. ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-22 23:36 Message: Logged In: YES user_id=1200846 Sorry, I was stupid. MSDN (http://msdn.microsoft.com/library/default.asp?url=/library/en-us/intl/unicode_0o2t.asp) saids, > IsDBCSLeadByte can only indicate a potential lead byte value. IsDBCSLeadByte was returning 1 for some trail byte (ex: "???"[1]) The patch "mbcs3a.patch" worked for me, but _mbsbtype is probably compiler specific. Is that OK? The patch "mbcs3b.patch" also worked for me and it only uses Win32API, but I don't have enough faith on this implementation... ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-22 19:31 Message: Logged In: YES user_id=1200846 Sorry, I found problem when tried more long text file... Please wait. I'll investigate more intensibly. ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-22 19:13 Message: Logged In: YES user_id=1200846 Thank you for reply. How about this? (I'm a newbie, I hope this is right tex format but... can you confirm this? I created this patch by copy & paste from PyUnicode_DecodeUTF16Stateful and some modification) ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2006-03-22 18:12 Message: Logged In: YES user_id=38388 One more nit: the doc patch is missing. Please add a patch for the API docs. ---------------------------------------------------------------------- Comment By: M.-A. Lemburg (lemburg) Date: 2006-03-22 18:11 Message: Logged In: YES user_id=38388 As I understand your comment, the mbcs codec will have a problem if the input string terminates with a lead byte. Could you add a comment regarding this to the patch ?! I can't test the patch, since I don't have a Japanese Windows to check on, but from looking at the patch, it seems OK. ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-03-22 16:42 Message: Logged In: YES user_id=1200846 I forgot to mention this. "mbcs.patch" is for release24-maint branch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1455898&group_id=5470 From noreply at sourceforge.net Wed May 3 07:05:19 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 02 May 2006 22:05:19 -0700 Subject: [Patches] [ python-Patches-1480067 ] urllib2 digest auth redirection bug causes 400 error Message-ID: Patches item #1480067, was opened at 2006-05-01 22:35 Message generated for change (Settings changed) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1480067&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2 digest auth redirection bug causes 400 error Initial Comment: urllib2 redirects HTTP digest authorisation credentials, which is never useful (because the redirection will change the digest), and may cause a 400 error if for example the handler finds credentials for an initial request, but fails to finds credentials for a redirected request. In that case a stale Authorization or Proxy-authorization header will get returned to the server, causing a 400 error. I've verified this makes the 400 go away for example in the case where http://localhost/foo gets 301 redirected to http://127.0.0.1/foo/ (i.e. with a slash on the end), where I've only added username/password for "localhost" and not "127.0.0.1". The fix is trivial. 2.4 backport candidate. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 05:05 Message: Logged In: YES user_id=849994 Committed as rev. 45879, 45880 (2.4). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1480067&group_id=5470 From noreply at sourceforge.net Wed May 3 07:17:48 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 02 May 2006 22:17:48 -0700 Subject: [Patches] [ python-Patches-1411097 ] httplib patch to make _read_chunked() more robust Message-ID: Patches item #1411097, was opened at 2006-01-20 20:26 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1411097&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: httplib patch to make _read_chunked() more robust Initial Comment: To reproduce: import urllib2 print urllib2.urlopen("http://66.117.37.13/").read() The attached patch "fixes" the hang, but that patch is not acceptable because it also removes the .readline() and .readlines() methods on the response object returned by urllib2.urlopen(). The patch seems to demonstrate that the problem is caused by the (ab)use of socket._fileobject in urllib2.AbstractHTTPHandler (I believe this hack was introduced when urllib2 switched to using httplib.HTTPConnection). Not sure yet what the actual problem is... ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 05:17 Message: Logged In: YES user_id=849994 Are you still working on your slightly modified patch? ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 21:20 Message: Logged In: YES user_id=261020 Please ignore last comment (posted to wrong tracker item). ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 21:18 Message: Logged In: YES user_id=261020 Conservative or not, I see no utility in changing the default, and several major harmful effects: old code breaks, and people have to pore over the specs to figure out why "urlopen() doesn't work". ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 20:36 Message: Logged In: YES user_id=261020 I missed the fact that, if the connection will not close at the end of the transaction, the behaviour should not change from what's currently in SVN (we should not assume that the chunked response has ended unless we see the proper terminating CRLF). I intend to upload a slightly modified patch that tests for self._will_close, and behaves accordingly. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 01:24 Message: Logged In: YES user_id=261020 Oops, fixed chunk.patch to .strip() before comparing to "". ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 00:38 Message: Logged In: YES user_id=261020 First, expanding a bit on what I wrote on 2006-01-21: The problem does relate to chunked encoding, and is unrelated to urllib2's use of _fileobject. My hack to remove use of socket._fileobject from urllib2 merely breaks handling of chunked encoding by cutting httplib.HTTPResponse out of the picture. The problem is seen in urllib2 in recent Pythons thanks to urllib2 switching to use of httplib.HTTPConnection and HTTP/1.1, hence chunked encoding is allowed. urllib still uses httplib.HTTP, hence HTTP/1.0, so is unaffected. To reproduce with httplib: import httplib conn = httplib.HTTPConnection("66.117.37.13") conn.request("GET", "/", headers={"Connection": "close"}) r1 = conn.getresponse() print r1.read() The Connection: close is required -- if it's not there the server doesn't use chunked transfer-encoding. I verified with a packet sniffer that the problem is that this server does not send the final trailing CRLF required by section 3.6.1 of RFC 2616. However, that section also says that trailers (trailing HTTP headers) MUST NOT be sent by the server unless either a TE header was present and indicated that trailers are acceptable (httplib does not send the TE header), or the trailers are optional metadata and may be discarded by the client. So, I propose the attached patch to httplib (chunk.patch) as a work-around. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-01-21 22:10 Message: Logged In: YES user_id=261020 In fact the commit message for rev 36871 states the real reason _fileobject is used (handling chunked encoding), showing my workaround is even more harmful than I thought. Moreover, doing a urlopen on 66.117.37.13 shows the response *is* chunked. The problem seems to be caused by httplib failing to find a CRLF at the end of the chunked response, so the loop at the end of _read_chunked() never terminates. Haven't looked in detail yet, but I'm guessing a) it's the server's fault and b) httplib should work around it. Here's the commit message from 36871: Fix urllib2.urlopen() handling of chunked content encoding. The change to use the newer httplib interface admitted the possibility that we'd get an HTTP/1.1 chunked response, but the code didn't handle it correctly. The raw socket object can't be pass to addinfourl(), because it would read the undecoded response. Instead, addinfourl() must call HTTPResponse.read(), which will handle the decoding. One extra wrinkle is that the HTTPReponse object can't be passed to addinfourl() either, because it doesn't implement readline() or readlines(). As a quick hack, use socket._fileobject(), which implements those methods on top of a read buffer. (suggested by mwh) Finally, add some tests based on test_urllibnet. Thanks to Andrew Sawyers for originally reporting the chunked problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1411097&group_id=5470 From noreply at sourceforge.net Wed May 3 13:59:13 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 04:59:13 -0700 Subject: [Patches] [ python-Patches-1481032 ] patch smtplib:when SMTPDataError, rset crashes with sslerror Message-ID: Patches item #1481032, was opened at 2006-05-03 13:59 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481032&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: kxroberto (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: patch smtplib:when SMTPDataError, rset crashes with sslerror Initial Comment: File "smtplib.pyc", line 690, in sendmail File "smtplib.pyc", line 445, in rset File "smtplib.pyc", line 370, in docmd File "smtplib.pyc", line 344, in getreply File "smtplib.pyc", line 159, in readline sslerror: (8, 'EOF occurred in violation of protocol') traced from a py2.3 - yet unchanged. => hides original error SMTPDataError. such SMTPDataError may have forced a disconnect of server. patch for py2.3,py2.4,.. ( I have this patch in my MUST-DO-PATCHes after any Python installation. ) -- the patch passes on any error in this rset() location. it also patches a error in PLAIN authentication could cleanly catch socket.sslerror / socket.error / EnvironmentError, yet ssl not always there ... rset() is tested otherwise, so the nacked except is ok? (there should maybe be special a common base Exception class "IOBaseError" for : EnvironmentError(IOError, OSError), EOFError, socket.error, socket.sslerror, ftplib.all_errors, etc. Nice as it and not all IO sublibs have to be imported to catch such errors.) --- the same problem is with smtp.quit() on many SSL'ed connections (without any other errors occuring): a final socket.sslerror is raised during quit(). There may be a problem in the termination code of ssl-FakeSockets or ssl.c . Or the same type of error catch (on IOBaseError) should be applied. I am not sure - in my apps I (must) catch on smtp.quit() generally. (Compare also bug #978833 / shutdown(2)-remedy in httplib's SSL FakeSocket - this shutdown(2) remedy patch of #978833 I still have it on my MUST list (py2.3/py2.4 installations), otherwise this FakeSocket doesn't close fully in a FTPS application (where termination on data channel is crucial for getting response on the control channel) - and most likely puts tremendous connection load on HTTPS servers because of stale unterminated HTTPS connections while the bug may not be obvious in casual usage. I'm not completely clear about the nature of this error. Thus, what I say is based on trial-and-error. -robert ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481032&group_id=5470 From noreply at sourceforge.net Wed May 3 15:01:15 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 06:01:15 -0700 Subject: [Patches] [ python-Patches-1481079 ] Support HTTP_REFERER in CGIHTTPServer.py Message-ID: Patches item #1481079, was opened at 2006-05-03 15:01 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481079&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Submitted By: S??bastien Martini (ookoi) Assigned to: Nobody/Anonymous (nobody) Summary: Support HTTP_REFERER in CGIHTTPServer.py Initial Comment: In CGIHTTPServer.py simply put the referer's value (obtained from headers) in os.env associated to the key 'HTTP_REFERER'. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481079&group_id=5470 From noreply at sourceforge.net Wed May 3 15:59:26 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 06:59:26 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) Assigned to: Nobody/Anonymous (nobody) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Wed May 3 17:34:12 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 08:34:12 -0700 Subject: [Patches] [ python-Patches-1481079 ] Support of HTTP_REFERER in CGIHTTPServer.py Message-ID: Patches item #1481079, was opened at 2006-05-03 15:01 Message generated for change (Settings changed) made by ookoi You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481079&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Submitted By: S??bastien Martini (ookoi) Assigned to: Nobody/Anonymous (nobody) >Summary: Support of HTTP_REFERER in CGIHTTPServer.py Initial Comment: In CGIHTTPServer.py simply put the referer's value (obtained from headers) in os.env associated to the key 'HTTP_REFERER'. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481079&group_id=5470 From noreply at sourceforge.net Wed May 3 19:47:22 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 10:47:22 -0700 Subject: [Patches] [ python-Patches-1143695 ] Fix to allow urllib2 digest auth to talk to livejournal.com Message-ID: Patches item #1143695, was opened at 2005-02-18 11:14 Message generated for change (Settings changed) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1143695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Benno Rice (benno) Assigned to: Nobody/Anonymous (nobody) Summary: Fix to allow urllib2 digest auth to talk to livejournal.com Initial Comment: When trying to use feedparser.py to deal with RSS feeds from livejournal.com using digest auth (needed to access locked posts), urllib2 would report a digest auth failure. The solution appears to be to always specify the algorithm, even when it's MD5. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-15 14:49 Message: Logged In: YES user_id=261020 This was fixed in revision 38092. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2005-02-20 02:01 Message: Logged In: YES user_id=261020 Patch appears correct, and RFC 2617 allows always sending the algorithm. Haven't tested the patch, or verified that real browsers do indeed always send algorithm even when it'd MD5. ---------------------------------------------------------------------- Comment By: Benno Rice (benno) Date: 2005-02-19 09:02 Message: Logged In: YES user_id=9925 SourceForge is teh suxx0r. ---------------------------------------------------------------------- Comment By: Anthony Baxter (anthonybaxter) Date: 2005-02-19 08:57 Message: Logged In: YES user_id=29957 There's no uploaded file! You have to check the checkbox labeled "Check to Upload & Attach File" when you upload a file. In addition, even if you *did* check this checkbox, a bug in SourceForge prevents attaching a file when *creating* an issue. Please try again. (This is a SourceForge annoyance that we can do nothing about. :-( ) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1143695&group_id=5470 From noreply at sourceforge.net Wed May 3 20:13:58 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 11:13:58 -0700 Subject: [Patches] [ python-Patches-1472184 ] pdb: fix for #1472191('clear' command bug) Message-ID: Patches item #1472184, was opened at 2006-04-18 09:38 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1472184&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: None Priority: 6 Submitted By: Kuba Ko??czyk (jakamkon) Assigned to: Nobody/Anonymous (nobody) Summary: pdb: fix for #1472191('clear' command bug) Initial Comment: Pdb 'clear x' command doesn't clear selected breakpoints that are already set: $ ./python -m pdb ../test.py > /home/xyz/python/test.py(3)() -> def t(x): (Pdb) break 5 Breakpoint 1 at /home/xyz/python/test.py:5 (Pdb) break Num Type Disp Enb Where 1 breakpoint keep yes at /home/xyz/python/test.py:5 (Pdb) clear 1 No breakpoint numbered 1 (Pdb) ... for i in numberlist: * if not (0 <= i < len(bdb.Breakpoint.bpbynumber)): print 'No breakpoint numbered', i ... Each i is a string and it's compared to 0 and len(...), so condition * is always True. The fix is trivial: * if not (0 <= int(i) < len(bdb.Breakpoint.bpbynumber)): ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 18:13 Message: Logged In: YES user_id=849994 Problem was fixed in rev. 45891, 45892(2.4), including better error handling. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1472184&group_id=5470 From noreply at sourceforge.net Wed May 3 20:37:42 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 11:37:42 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Comment added) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) Assigned to: Nobody/Anonymous (nobody) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Wed May 3 20:46:10 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 11:46:10 -0700 Subject: [Patches] [ python-Patches-1481304 ] Cleaned up 16x16px icons for windows. Message-ID: Patches item #1481304, was opened at 2006-05-03 20:46 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481304&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Windows Group: None Status: Open Resolution: None Priority: 5 Submitted By: goxe (goxe) Assigned to: Nobody/Anonymous (nobody) Summary: Cleaned up 16x16px icons for windows. Initial Comment: Since the currently distributed icon files only include 32x32px images, Windows resizes them where 16x16px is needed. With the predictable result that they look blurred and dark. The attached icons include 16x16px versions of the current icons. It's the same friendly-snake-icon as always, just prettier in small sizes. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481304&group_id=5470 From noreply at sourceforge.net Wed May 3 20:53:38 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 11:53:38 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 13:59 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) >Assigned to: Martin v. L??wis (loewis) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 18:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 18:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Wed May 3 20:57:35 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 11:57:35 -0700 Subject: [Patches] [ python-Patches-1411097 ] httplib patch to make _read_chunked() more robust Message-ID: Patches item #1411097, was opened at 2006-01-20 20:26 Message generated for change (Comment added) made by jjlee You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1411097&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: httplib patch to make _read_chunked() more robust Initial Comment: To reproduce: import urllib2 print urllib2.urlopen("http://66.117.37.13/").read() The attached patch "fixes" the hang, but that patch is not acceptable because it also removes the .readline() and .readlines() methods on the response object returned by urllib2.urlopen(). The patch seems to demonstrate that the problem is caused by the (ab)use of socket._fileobject in urllib2.AbstractHTTPHandler (I believe this hack was introduced when urllib2 switched to using httplib.HTTPConnection). Not sure yet what the actual problem is... ---------------------------------------------------------------------- >Comment By: John J Lee (jjlee) Date: 2006-05-03 19:57 Message: Logged In: YES user_id=261020 I *hope* to get back to it soon. But if anybody beats me to it, that's fine :-) One problem: I don't understand the need for HTTPConnection._safe_read(), rather than checking for an EINTR resulting from the recv() call (or WSAEINTR on Windows). Can anybody explain that? ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 06:17 Message: Logged In: YES user_id=849994 Are you still working on your slightly modified patch? ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 21:20 Message: Logged In: YES user_id=261020 Please ignore last comment (posted to wrong tracker item). ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 21:18 Message: Logged In: YES user_id=261020 Conservative or not, I see no utility in changing the default, and several major harmful effects: old code breaks, and people have to pore over the specs to figure out why "urlopen() doesn't work". ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 20:36 Message: Logged In: YES user_id=261020 I missed the fact that, if the connection will not close at the end of the transaction, the behaviour should not change from what's currently in SVN (we should not assume that the chunked response has ended unless we see the proper terminating CRLF). I intend to upload a slightly modified patch that tests for self._will_close, and behaves accordingly. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 01:24 Message: Logged In: YES user_id=261020 Oops, fixed chunk.patch to .strip() before comparing to "". ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-02-06 00:38 Message: Logged In: YES user_id=261020 First, expanding a bit on what I wrote on 2006-01-21: The problem does relate to chunked encoding, and is unrelated to urllib2's use of _fileobject. My hack to remove use of socket._fileobject from urllib2 merely breaks handling of chunked encoding by cutting httplib.HTTPResponse out of the picture. The problem is seen in urllib2 in recent Pythons thanks to urllib2 switching to use of httplib.HTTPConnection and HTTP/1.1, hence chunked encoding is allowed. urllib still uses httplib.HTTP, hence HTTP/1.0, so is unaffected. To reproduce with httplib: import httplib conn = httplib.HTTPConnection("66.117.37.13") conn.request("GET", "/", headers={"Connection": "close"}) r1 = conn.getresponse() print r1.read() The Connection: close is required -- if it's not there the server doesn't use chunked transfer-encoding. I verified with a packet sniffer that the problem is that this server does not send the final trailing CRLF required by section 3.6.1 of RFC 2616. However, that section also says that trailers (trailing HTTP headers) MUST NOT be sent by the server unless either a TE header was present and indicated that trailers are acceptable (httplib does not send the TE header), or the trailers are optional metadata and may be discarded by the client. So, I propose the attached patch to httplib (chunk.patch) as a work-around. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-01-21 22:10 Message: Logged In: YES user_id=261020 In fact the commit message for rev 36871 states the real reason _fileobject is used (handling chunked encoding), showing my workaround is even more harmful than I thought. Moreover, doing a urlopen on 66.117.37.13 shows the response *is* chunked. The problem seems to be caused by httplib failing to find a CRLF at the end of the chunked response, so the loop at the end of _read_chunked() never terminates. Haven't looked in detail yet, but I'm guessing a) it's the server's fault and b) httplib should work around it. Here's the commit message from 36871: Fix urllib2.urlopen() handling of chunked content encoding. The change to use the newer httplib interface admitted the possibility that we'd get an HTTP/1.1 chunked response, but the code didn't handle it correctly. The raw socket object can't be pass to addinfourl(), because it would read the undecoded response. Instead, addinfourl() must call HTTPResponse.read(), which will handle the decoding. One extra wrinkle is that the HTTPReponse object can't be passed to addinfourl() either, because it doesn't implement readline() or readlines(). As a quick hack, use socket._fileobject(), which implements those methods on top of a read buffer. (suggested by mwh) Finally, add some tests based on test_urllibnet. Thanks to Andrew Sawyers for originally reporting the chunked problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1411097&group_id=5470 From noreply at sourceforge.net Wed May 3 20:59:59 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 11:59:59 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Comment added) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) >Assigned to: Nobody/Anonymous (nobody) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:59 Message: Logged In: YES user_id=791932 I'll redo the patch with vim now, emacs doesn't like doing 8 spaces indents, at least as far as I can get it configured... Anyway, I assign the copyright to any code contained in this patch to the PSF. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 20:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Wed May 3 21:15:22 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 12:15:22 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Settings changed) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) >Assigned to: Martin v. L??wis (loewis) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:59 Message: Logged In: YES user_id=791932 I'll redo the patch with vim now, emacs doesn't like doing 8 spaces indents, at least as far as I can get it configured... Anyway, I assign the copyright to any code contained in this patch to the PSF. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 20:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Wed May 3 21:32:31 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 12:32:31 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Comment added) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) Assigned to: Martin v. L??wis (loewis) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 21:32 Message: Logged In: YES user_id=791932 Final patch which should conform to PEP-7 completely. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:59 Message: Logged In: YES user_id=791932 I'll redo the patch with vim now, emacs doesn't like doing 8 spaces indents, at least as far as I can get it configured... Anyway, I assign the copyright to any code contained in this patch to the PSF. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 20:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Wed May 3 23:02:27 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 14:02:27 -0700 Subject: [Patches] [ python-Patches-1477281 ] __init__.py'less package import warnings Message-ID: Patches item #1477281, was opened at 2006-04-26 22:36 Message generated for change (Settings changed) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1477281&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: None >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Thomas Wouters (twouters) Assigned to: Guido van Rossum (gvanrossum) Summary: __init__.py'less package import warnings Initial Comment: New! Industrial strength pitchfork-repellant. Just sprinkle onto 2.5 and watch the pitchforks melt away, to be replaced by hearthfelt grouphugs. This patch (which probably needs some reformatting) adds warnings when Python would have imported a module, if only there had been an __init__.py. The text is currently helpful, but it can easily be changed into a FutureWarning with a warning that it'll change in 2.6 (or 'might change in the future'.) ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 21:02 Message: Logged In: YES user_id=849994 Looks like this got checked in in rev. 45770. ---------------------------------------------------------------------- Comment By: Thomas Wouters (twouters) Date: 2006-04-26 22:54 Message: Logged In: YES user_id=34209 Let's just check it into Python 2.5 instead (it's still an hour and 7 minutes until trunk freeze, I think? :>) ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2006-04-26 22:47 Message: Logged In: YES user_id=6380 The patch doesn't check for errors coming out of PyErr_Warn() -- remember, a warning can always be turned into an exception. I'll see if we can patch Google's Python like this. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1477281&group_id=5470 From noreply at sourceforge.net Thu May 4 03:42:06 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 18:42:06 -0700 Subject: [Patches] [ python-Patches-1481530 ] imputil "from" os.path import bug Message-ID: Patches item #1481530, was opened at 2006-05-03 18:42 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481530&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Eric Huss (ehuss) Assigned to: Nobody/Anonymous (nobody) Summary: imputil "from" os.path import bug Initial Comment: The following idiom appears to not work when using imputil: from os.path import join This patch should fix that problem. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481530&group_id=5470 From noreply at sourceforge.net Thu May 4 06:56:30 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 21:56:30 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) Assigned to: Martin v. L??wis (loewis) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Martin v. L??wis (loewis) Date: 2006-05-04 06:56 Message: Logged In: YES user_id=21627 I actually wonder what the rationale for this patch is. The command line options of Python seem very clear to me; I don't see the need for long options. This should be discussed on python-dev. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 21:32 Message: Logged In: YES user_id=791932 Final patch which should conform to PEP-7 completely. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:59 Message: Logged In: YES user_id=791932 I'll redo the patch with vim now, emacs doesn't like doing 8 spaces indents, at least as far as I can get it configured... Anyway, I assign the copyright to any code contained in this patch to the PSF. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 20:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Thu May 4 07:08:37 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 22:08:37 -0700 Subject: [Patches] [ python-Patches-1481530 ] imputil "from" os.path import bug Message-ID: Patches item #1481530, was opened at 2006-05-04 01:42 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481530&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Eric Huss (ehuss) Assigned to: Nobody/Anonymous (nobody) Summary: imputil "from" os.path import bug Initial Comment: The following idiom appears to not work when using imputil: from os.path import join This patch should fix that problem. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-04 05:08 Message: Logged In: YES user_id=849994 Thanks for the patch, committed as rev. 45895, 45896(2.4). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481530&group_id=5470 From noreply at sourceforge.net Thu May 4 07:51:45 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 22:51:45 -0700 Subject: [Patches] [ python-Patches-1475845 ] IndentationError for unexpected indent Message-ID: Patches item #1475845, was opened at 2006-04-25 01:12 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1475845&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Parser/Compiler Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Roger Miller (rcmiller) Assigned to: Martin v. L??wis (loewis) Summary: IndentationError for unexpected indent Initial Comment: This patch raises an IndentationError rather than a generic "invalid syntax" error for unexpected indentation. Code to do this was already in pythonrun.c:err_input() but was not being reached due to a failure to pass the INDENT token in the perrdetail structure. The patch also adds tests for the 3 kinds of indentation errors (unexpected indent, no indent where required, invalid outdent level) to test_syntax.py . ---------------------------------------------------------------------- >Comment By: Martin v. L??wis (loewis) Date: 2006-05-04 07:51 Message: Logged In: YES user_id=21627 IndentationError is already raised for bad indentation, e.g. for "def f():\nreturn" or if 1:\nfoo()" (which is the test_no_indent) However, the patch is right in filling the token in this case, also; I accepted it as r45897. As it changes the exceptio behaviour, I don't think it should be backported. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-04-30 13:20 Message: Logged In: YES user_id=849994 Martin, do we want to change this? I myself have always wondered what IndentationError was for if it was not raised in these cases. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1475845&group_id=5470 From noreply at sourceforge.net Thu May 4 08:06:03 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 23:06:03 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Comment added) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) Assigned to: Martin v. L??wis (loewis) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 08:06 Message: Logged In: YES user_id=791932 The rationale behind this patch is to enable python to answer to --version and --help, which are pretty much standard command-line options with GNU utilities, and increasingly common amongst BSD utilities. I developed this patch, answering to a request on c.l.p where people were asking for Python to answer to --version, and thought I could generalize it a bit so that long options can also be used for other arguments. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-05-04 06:56 Message: Logged In: YES user_id=21627 I actually wonder what the rationale for this patch is. The command line options of Python seem very clear to me; I don't see the need for long options. This should be discussed on python-dev. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 21:32 Message: Logged In: YES user_id=791932 Final patch which should conform to PEP-7 completely. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:59 Message: Logged In: YES user_id=791932 I'll redo the patch with vim now, emacs doesn't like doing 8 spaces indents, at least as far as I can get it configured... Anyway, I assign the copyright to any code contained in this patch to the PSF. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 20:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Thu May 4 08:14:36 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 03 May 2006 23:14:36 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Comment added) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) Assigned to: Martin v. L??wis (loewis) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 08:14 Message: Logged In: YES user_id=791932 I just posted a mail to python-dev explaining my rationale behind this patch. Maybe you could answer there... ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 08:06 Message: Logged In: YES user_id=791932 The rationale behind this patch is to enable python to answer to --version and --help, which are pretty much standard command-line options with GNU utilities, and increasingly common amongst BSD utilities. I developed this patch, answering to a request on c.l.p where people were asking for Python to answer to --version, and thought I could generalize it a bit so that long options can also be used for other arguments. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-05-04 06:56 Message: Logged In: YES user_id=21627 I actually wonder what the rationale for this patch is. The command line options of Python seem very clear to me; I don't see the need for long options. This should be discussed on python-dev. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 21:32 Message: Logged In: YES user_id=791932 Final patch which should conform to PEP-7 completely. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:59 Message: Logged In: YES user_id=791932 I'll redo the patch with vim now, emacs doesn't like doing 8 spaces indents, at least as far as I can get it configured... Anyway, I assign the copyright to any code contained in this patch to the PSF. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 20:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Thu May 4 21:16:09 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 04 May 2006 12:16:09 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Comment added) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) Assigned to: Martin v. L??wis (loewis) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 21:16 Message: Logged In: YES user_id=791932 The latest patch takes into account all constructive ideas that have been proposed on python-dev for this enhancement. It implements / support on Windows for long options, and adds ? as a possible option to get help. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 08:14 Message: Logged In: YES user_id=791932 I just posted a mail to python-dev explaining my rationale behind this patch. Maybe you could answer there... ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 08:06 Message: Logged In: YES user_id=791932 The rationale behind this patch is to enable python to answer to --version and --help, which are pretty much standard command-line options with GNU utilities, and increasingly common amongst BSD utilities. I developed this patch, answering to a request on c.l.p where people were asking for Python to answer to --version, and thought I could generalize it a bit so that long options can also be used for other arguments. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-05-04 06:56 Message: Logged In: YES user_id=21627 I actually wonder what the rationale for this patch is. The command line options of Python seem very clear to me; I don't see the need for long options. This should be discussed on python-dev. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 21:32 Message: Logged In: YES user_id=791932 Final patch which should conform to PEP-7 completely. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:59 Message: Logged In: YES user_id=791932 I'll redo the patch with vim now, emacs doesn't like doing 8 spaces indents, at least as far as I can get it configured... Anyway, I assign the copyright to any code contained in this patch to the PSF. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 20:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Thu May 4 22:21:38 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 04 May 2006 13:21:38 -0700 Subject: [Patches] [ python-Patches-1481112 ] Python long option support Message-ID: Patches item #1481112, was opened at 2006-05-03 15:59 Message generated for change (Comment added) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: Heiko Wundram (hwundram) Assigned to: Martin v. L??wis (loewis) Summary: Python long option support Initial Comment: The attached patch implements long option support for Python. It changes the optstring found in Modules/main.c, specifying brackets for the long name of a corresponding option name. The patch is backward compatible in that it doesn't change the behaviour of _PyOS_GetOpt for any old format string, except on [:()], which are explicitly excluded for matching an option. This shouldn't break any code, though. The patch is against Python 2.4.3. ---------------------------------------------------------------------- >Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 22:21 Message: Logged In: YES user_id=791932 Small extension to completely conform to Microsoft long-opt semantics: --= is equivalent to: /: under Windows with the latest patch. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 21:16 Message: Logged In: YES user_id=791932 The latest patch takes into account all constructive ideas that have been proposed on python-dev for this enhancement. It implements / support on Windows for long options, and adds ? as a possible option to get help. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 08:14 Message: Logged In: YES user_id=791932 I just posted a mail to python-dev explaining my rationale behind this patch. Maybe you could answer there... ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-04 08:06 Message: Logged In: YES user_id=791932 The rationale behind this patch is to enable python to answer to --version and --help, which are pretty much standard command-line options with GNU utilities, and increasingly common amongst BSD utilities. I developed this patch, answering to a request on c.l.p where people were asking for Python to answer to --version, and thought I could generalize it a bit so that long options can also be used for other arguments. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-05-04 06:56 Message: Logged In: YES user_id=21627 I actually wonder what the rationale for this patch is. The command line options of Python seem very clear to me; I don't see the need for long options. This should be discussed on python-dev. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 21:32 Message: Logged In: YES user_id=791932 Final patch which should conform to PEP-7 completely. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:59 Message: Logged In: YES user_id=791932 I'll redo the patch with vim now, emacs doesn't like doing 8 spaces indents, at least as far as I can get it configured... Anyway, I assign the copyright to any code contained in this patch to the PSF. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-03 20:53 Message: Logged In: YES user_id=849994 Please update the patch to follow Python C style guidelines (PEP 7), especially use 8-space tabs to indent. Also, it might be good to send a copyright assignment to the PSF for a patch of this magnitude. Otherwise, I think that this is a desirable feature, at least for --help and --version. ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-03 20:37 Message: Logged In: YES user_id=791932 The attached patch is against the current subversion trunk, and implements long options with possible arguments after an =-sign. It has partial matching for options, erroring out on ambiguous matches of the command line arguments. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481112&group_id=5470 From noreply at sourceforge.net Fri May 5 08:55:50 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 04 May 2006 23:55:50 -0700 Subject: [Patches] [ python-Patches-1481032 ] patch smtplib:when SMTPDataError, rset crashes with sslerror Message-ID: Patches item #1481032, was opened at 2006-05-03 13:59 Message generated for change (Comment added) made by hwundram You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481032&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: kxroberto (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: patch smtplib:when SMTPDataError, rset crashes with sslerror Initial Comment: File "smtplib.pyc", line 690, in sendmail File "smtplib.pyc", line 445, in rset File "smtplib.pyc", line 370, in docmd File "smtplib.pyc", line 344, in getreply File "smtplib.pyc", line 159, in readline sslerror: (8, 'EOF occurred in violation of protocol') traced from a py2.3 - yet unchanged. => hides original error SMTPDataError. such SMTPDataError may have forced a disconnect of server. patch for py2.3,py2.4,.. ( I have this patch in my MUST-DO-PATCHes after any Python installation. ) -- the patch passes on any error in this rset() location. it also patches a error in PLAIN authentication could cleanly catch socket.sslerror / socket.error / EnvironmentError, yet ssl not always there ... rset() is tested otherwise, so the nacked except is ok? (there should maybe be special a common base Exception class "IOBaseError" for : EnvironmentError(IOError, OSError), EOFError, socket.error, socket.sslerror, ftplib.all_errors, etc. Nice as it and not all IO sublibs have to be imported to catch such errors.) --- the same problem is with smtp.quit() on many SSL'ed connections (without any other errors occuring): a final socket.sslerror is raised during quit(). There may be a problem in the termination code of ssl-FakeSockets or ssl.c . Or the same type of error catch (on IOBaseError) should be applied. I am not sure - in my apps I (must) catch on smtp.quit() generally. (Compare also bug #978833 / shutdown(2)-remedy in httplib's SSL FakeSocket - this shutdown(2) remedy patch of #978833 I still have it on my MUST list (py2.3/py2.4 installations), otherwise this FakeSocket doesn't close fully in a FTPS application (where termination on data channel is crucial for getting response on the control channel) - and most likely puts tremendous connection load on HTTPS servers because of stale unterminated HTTPS connections while the bug may not be obvious in casual usage. I'm not completely clear about the nature of this error. Thus, what I say is based on trial-and-error. -robert ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-05 08:55 Message: Logged In: YES user_id=791932 Where is the patch? :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481032&group_id=5470 From noreply at sourceforge.net Fri May 5 10:27:33 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 05 May 2006 01:27:33 -0700 Subject: [Patches] [ python-Patches-1479611 ] speed up function calls Message-ID: Patches item #1479611, was opened at 2006-04-30 23:58 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: speed up function calls Initial Comment: Results: 2.86% for 1 arg (len), 11.8% for 2 args (min), and 1.6% for pybench. trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.74 msec per loop trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 8.03 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.88 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 9.09 msec per loop pybench goes from 5688.00 down to 5598.00 Details about the patch: There are 2 unrelated changes. They both seem to provide equal benefits for calling varargs C. One is very simple and just inlines calling a varargs C function rather than calling PyCFunction_Call() which does extra checks that are already known. This moves meth and self up one block. and breaks the C_TRACE into 2. (When looking at the patch, this will make sense I hope.) The other change is more dangerous. It modifies load_args() to hold on to tuples so they aren't allocated and deallocated. The initialization is done one time in the new func _PyEval_Init(). It allocates 64 tuples of size 8 that are never deallocated. The idea is that there won't be usually be more than 64 frames with 8 or less parameters active on the stack at any one time (stack depth). There are cases where this can degenerate, but for the most part, it should only be marginally slower, but generally this should be a fair amount faster by skipping the alloc and dealloc and some extra work. My decrementing the _last_index inside the needs_free blocks, that could improve behaviour. This really needs comments added to the code. But I'm not gonna get there tonight. I'd be interested in comments about the code. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-05-05 01:27 Message: Logged In: YES user_id=33168 v2 attached. You might not want to review yet. I mostly did the first part of your suggest (stats, _Fini, and stack-like if I understood you correctly). I didn't do anything on the second part about inlinting Function_Call. perf seems to be about the same. I'm not entirely sure the patch is correct yet. I found one or two problems in the original. I added some more comments. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-05-01 01:27 Message: Logged In: YES user_id=21627 The tuples should get deallocated when Py_Finalize is called. It would be good if there was (conditional) statistical analysis, showing how often no tuple was found because the number of arguments was too large, and how often no tuple was found because the candidate was in use. I think it should be more stack-like, starting off with no tuples allocated, then returning them inside the needs_free blocks only if the refcount is 1 (or 2?). This would avoid degeneralized cases where some function holds onto its argument tuple indefinitely, thus consuming all 64 tuples. For the other part, I think it would make the code more readable if it inlined PyCFunction_Call even more: the test for NOARGS|O could be integrated into the switch statement (one case for each), VARARGS and VARARGS|KEYWORDS would both load the arguments, then call the function directly (possibly with NULL keywords). OLDARGS should goto either METH_NOARGS, METH_O, or METH_VARARGS depending on na (if you don't like goto, modifying flags would work as well). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-05-01 00:08 Message: Logged In: YES user_id=33168 I should note the numbers 64 and 8 are total guesses. It might be good to try and determine values based on empirical data. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 From noreply at sourceforge.net Fri May 5 12:22:53 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Fri, 05 May 2006 03:22:53 -0700 Subject: [Patches] [ python-Patches-1481032 ] patch smtplib:when SMTPDataError, rset crashes with sslerror Message-ID: Patches item #1481032, was opened at 2006-05-03 13:59 Message generated for change (Comment added) made by kxroberto You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481032&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: kxroberto (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: patch smtplib:when SMTPDataError, rset crashes with sslerror Initial Comment: File "smtplib.pyc", line 690, in sendmail File "smtplib.pyc", line 445, in rset File "smtplib.pyc", line 370, in docmd File "smtplib.pyc", line 344, in getreply File "smtplib.pyc", line 159, in readline sslerror: (8, 'EOF occurred in violation of protocol') traced from a py2.3 - yet unchanged. => hides original error SMTPDataError. such SMTPDataError may have forced a disconnect of server. patch for py2.3,py2.4,.. ( I have this patch in my MUST-DO-PATCHes after any Python installation. ) -- the patch passes on any error in this rset() location. it also patches a error in PLAIN authentication could cleanly catch socket.sslerror / socket.error / EnvironmentError, yet ssl not always there ... rset() is tested otherwise, so the nacked except is ok? (there should maybe be special a common base Exception class "IOBaseError" for : EnvironmentError(IOError, OSError), EOFError, socket.error, socket.sslerror, ftplib.all_errors, etc. Nice as it and not all IO sublibs have to be imported to catch such errors.) --- the same problem is with smtp.quit() on many SSL'ed connections (without any other errors occuring): a final socket.sslerror is raised during quit(). There may be a problem in the termination code of ssl-FakeSockets or ssl.c . Or the same type of error catch (on IOBaseError) should be applied. I am not sure - in my apps I (must) catch on smtp.quit() generally. (Compare also bug #978833 / shutdown(2)-remedy in httplib's SSL FakeSocket - this shutdown(2) remedy patch of #978833 I still have it on my MUST list (py2.3/py2.4 installations), otherwise this FakeSocket doesn't close fully in a FTPS application (where termination on data channel is crucial for getting response on the control channel) - and most likely puts tremendous connection load on HTTPS servers because of stale unterminated HTTPS connections while the bug may not be obvious in casual usage. I'm not completely clear about the nature of this error. Thus, what I say is based on trial-and-error. -robert ---------------------------------------------------------------------- >Comment By: kxroberto (kxroberto) Date: 2006-05-05 12:22 Message: Logged In: YES user_id=972995 here (i forgot the upload checkbox?) ---------------------------------------------------------------------- Comment By: Heiko Wundram (hwundram) Date: 2006-05-05 08:55 Message: Logged In: YES user_id=791932 Where is the patch? :-) ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1481032&group_id=5470 From noreply at sourceforge.net Sat May 6 20:16:18 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 06 May 2006 11:16:18 -0700 Subject: [Patches] [ python-Patches-1457736 ] patch for building trunk with VC6 Message-ID: Patches item #1457736, was opened at 2006-03-24 21:40 Message generated for change (Comment added) made by infidel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1457736&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Windows Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Raymond Hettinger (rhettinger) Summary: patch for building trunk with VC6 Initial Comment: Hello. I tried to build trunk with VC6, but failed. The reasons are - _W64 is not defined on VC6. (PC/pyconfig.h) - intptr_t and uintptr_t are not decleared on VC6. (should use Py_intptr_t and Py_uintptr_t respectively) I'll submit the patch for these two issues as "build_trunk_for_vc6.patch". And more two issues. - zlib was make built into pythoncore, but PC/VC6/pythoncore.dsp is not updated for it yet. I'll submit the file itself. - long long cannot be used on VC6, so 0xFFFFULL is failed to compile with "invalid suffix" error. I workarounded this replaced ULL with UI64 (_int64's suffix) but I don't know how to make the patch. maybe can this tequnique be used? #define Py_ULL(x) x##ULL /* non VC6 */ #define Py_ULL(x) x##UI64 /* VC6 */ Py_ULL(0xFFFFFFFFFFFFFFFF) instead of 0xFFF...FULL ---------------------------------------------------------------------- Comment By: Luke Dunstan (infidel) Date: 2006-05-07 02:16 Message: Logged In: YES user_id=30442 Is there anything preventing this patch from being applied? It would help me with building the trunk using both VC6 and Microsoft eMbedded Visual C++ 4.0 (for Windows CE). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-03-27 01:02 Message: Logged In: YES user_id=33168 Raymond, maybe this will help get VC6 building? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1457736&group_id=5470 From noreply at sourceforge.net Sun May 7 07:37:37 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 06 May 2006 22:37:37 -0700 Subject: [Patches] [ python-Patches-1457736 ] patch for building trunk with VC6 Message-ID: Patches item #1457736, was opened at 2006-03-24 22:40 Message generated for change (Comment added) made by ocean-city You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1457736&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Windows Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Raymond Hettinger (rhettinger) Summary: patch for building trunk with VC6 Initial Comment: Hello. I tried to build trunk with VC6, but failed. The reasons are - _W64 is not defined on VC6. (PC/pyconfig.h) - intptr_t and uintptr_t are not decleared on VC6. (should use Py_intptr_t and Py_uintptr_t respectively) I'll submit the patch for these two issues as "build_trunk_for_vc6.patch". And more two issues. - zlib was make built into pythoncore, but PC/VC6/pythoncore.dsp is not updated for it yet. I'll submit the file itself. - long long cannot be used on VC6, so 0xFFFFULL is failed to compile with "invalid suffix" error. I workarounded this replaced ULL with UI64 (_int64's suffix) but I don't know how to make the patch. maybe can this tequnique be used? #define Py_ULL(x) x##ULL /* non VC6 */ #define Py_ULL(x) x##UI64 /* VC6 */ Py_ULL(0xFFFFFFFFFFFFFFFF) instead of 0xFFF...FULL ---------------------------------------------------------------------- >Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-05-07 14:37 Message: Logged In: YES user_id=1200846 Hello. I updated the patch. (Probably this is better) - defined ULL() macro locally in Modules/sha512module.c maybe it's better to declare Py_ULL or something globally, but I don't know how to do it. - more patch for zlib builtin (ie: PC/VC6/Readme.txt) I cannot try this patch on VC7 or later, but I confirmed lib/test/testall.py passed on VC6. ---------------------------------------------------------------------- Comment By: Luke Dunstan (infidel) Date: 2006-05-07 03:16 Message: Logged In: YES user_id=30442 Is there anything preventing this patch from being applied? It would help me with building the trunk using both VC6 and Microsoft eMbedded Visual C++ 4.0 (for Windows CE). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-03-27 02:02 Message: Logged In: YES user_id=33168 Raymond, maybe this will help get VC6 building? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1457736&group_id=5470 From noreply at sourceforge.net Sun May 7 07:40:31 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sat, 06 May 2006 22:40:31 -0700 Subject: [Patches] [ python-Patches-1457736 ] patch for building trunk with VC6 Message-ID: Patches item #1457736, was opened at 2006-03-24 22:40 Message generated for change (Comment added) made by ocean-city You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1457736&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Windows Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Hirokazu Yamamoto (ocean-city) Assigned to: Raymond Hettinger (rhettinger) Summary: patch for building trunk with VC6 Initial Comment: Hello. I tried to build trunk with VC6, but failed. The reasons are - _W64 is not defined on VC6. (PC/pyconfig.h) - intptr_t and uintptr_t are not decleared on VC6. (should use Py_intptr_t and Py_uintptr_t respectively) I'll submit the patch for these two issues as "build_trunk_for_vc6.patch". And more two issues. - zlib was make built into pythoncore, but PC/VC6/pythoncore.dsp is not updated for it yet. I'll submit the file itself. - long long cannot be used on VC6, so 0xFFFFULL is failed to compile with "invalid suffix" error. I workarounded this replaced ULL with UI64 (_int64's suffix) but I don't know how to make the patch. maybe can this tequnique be used? #define Py_ULL(x) x##ULL /* non VC6 */ #define Py_ULL(x) x##UI64 /* VC6 */ Py_ULL(0xFFFFFFFFFFFFFFFF) instead of 0xFFF...FULL ---------------------------------------------------------------------- >Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-05-07 14:40 Message: Logged In: YES user_id=1200846 Oops, I forgot to upload the file. - Apply x.patch. - Replace pythoncore.dsp and pcbuild.dsw in PC/VC6 with attached files. - Remove PC/VC6/zlib.dsp ---------------------------------------------------------------------- Comment By: Hirokazu Yamamoto (ocean-city) Date: 2006-05-07 14:37 Message: Logged In: YES user_id=1200846 Hello. I updated the patch. (Probably this is better) - defined ULL() macro locally in Modules/sha512module.c maybe it's better to declare Py_ULL or something globally, but I don't know how to do it. - more patch for zlib builtin (ie: PC/VC6/Readme.txt) I cannot try this patch on VC7 or later, but I confirmed lib/test/testall.py passed on VC6. ---------------------------------------------------------------------- Comment By: Luke Dunstan (infidel) Date: 2006-05-07 03:16 Message: Logged In: YES user_id=30442 Is there anything preventing this patch from being applied? It would help me with building the trunk using both VC6 and Microsoft eMbedded Visual C++ 4.0 (for Windows CE). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-03-27 02:02 Message: Logged In: YES user_id=33168 Raymond, maybe this will help get VC6 building? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1457736&group_id=5470 From noreply at sourceforge.net Sun May 7 14:43:06 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 05:43:06 -0700 Subject: [Patches] [ python-Patches-982340 ] applesingle endianness issue Message-ID: Patches item #982340, was opened at 2004-06-30 00:20 Message generated for change (Settings changed) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=982340&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Macintosh Group: Python 2.3 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Bob Ippolito (etrepum) Assigned to: Jack Jansen (jackjansen) Summary: applesingle endianness issue Initial Comment: the struct formats in applesingle.py do not declare endianness and thus won't work on little endian architectures. This patch adds the endian declarations to the struct formats. (from http://www.opensource.apple.com/darwinsource/ WWDC2004/) ---------------------------------------------------------------------- Comment By: Ronald Oussoren (ronaldoussoren) Date: 2006-04-17 13:41 Message: Logged In: YES user_id=580910 Fixed in revision 45487 on the trunk. Please confirm and close this patch. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=982340&group_id=5470 From noreply at sourceforge.net Sun May 7 15:33:45 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 06:33:45 -0700 Subject: [Patches] [ python-Patches-1483325 ] Patch fixing #1481770 (wrong shared lib ext on hpux ia64) Message-ID: Patches item #1483325, was opened at 2006-05-07 07:33 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483325&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Everly (deckrider) Assigned to: Nobody/Anonymous (nobody) Summary: Patch fixing #1481770 (wrong shared lib ext on hpux ia64) Initial Comment: (configure and pyconfig.h.in must be regenerated after applying this patch) Not heavily tested, since I only have Linux (i686) at home. Will test on HPUX ia64 tomorrow and report back. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483325&group_id=5470 From noreply at sourceforge.net Sun May 7 18:03:28 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 09:03:28 -0700 Subject: [Patches] [ python-Patches-1483395 ] Add new top-level domains to cookielib Message-ID: Patches item #1483395, was opened at 2006-05-07 17:03 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: Add new top-level domains to cookielib Initial Comment: IANA introduced some new top-level domains in addition to the original seven. This adds them to cookielib, and adds a test for the relevant behaviour (blacklisting of some "country-code TLDs"). 2.4 backport candidate. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 From noreply at sourceforge.net Sun May 7 19:13:25 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 10:13:25 -0700 Subject: [Patches] [ python-Patches-1479977 ] Heavy revisions to urllib2 howto Message-ID: Patches item #1479977, was opened at 2006-05-01 15:50 Message generated for change (Comment added) made by akuchling You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479977&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Documentation Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: John J Lee (jjlee) >Assigned to: A.M. Kuchling (akuchling) Summary: Heavy revisions to urllib2 howto Initial Comment: Lots of people have been complaining about lack of urllib2 docs (though I'm never quite sure what people are looking for, being too familiar with all the details), so a tutorial may well be a useful addition. I'm sure you'll understand that my brutal criticism :-) is intended to make it even more useful. Michael: feel free to make further revisions, but unless you have major objections I suggest that this is checked in first, then we make any further changes after that by uploading patches on SF for review (I haven't stepped back and re-read it with a fresh mind, and no doubt would be useful for somebody to do that). Editing this took me quite a while, and if I can help it I don't want to go through too many revisions or argue about the details before anything gets fixed!-). I've taken the liberty of mentioning myself as a reviewer somewhere at the end of the document :-) Important: I reformatted paragraphs to max 70 character width (it's conventional, and plain-text diffs are especially painful to read otherwise, though admittedly diffs are never great for paragraphs anyway... I hope emacs didn't muck up any ReST syntax). I've uploaded just that formatting change as reformatted.rst (which also removes trailing whitespace from all lines). This should be done in a separate initial commit of course. For this reason, I've uploaded the whole document for both reformatted (reformatted.rst) and edited versions (edited.rst) rather than using patches. I've made all of the changes I discuss below, *with the exception of* the missing example of GET with urlencoded data that's really needed (search for XXX in the comments below) -- that should just need a few lines. BTW, it would be a really fantastic idea to turn the whole document into a valid doctest (I know I'm myself almost incapable of writing correct examples unless I do something like that). All that would require of course is adding a few >>>s and ...s and running it through doctest.testfile until it stops complaining ;-) Now a list explaining and justifying the changes I made: Spelling / paragraph structure etc. fixes. I won't list these. Most importantly, you seem a bit unsure who your audience is. For example, on headers -- you explain that "HTTP is based on requests and responses", but dive into User-Agent without actually mentioning what a header is. In my changes, I ended up adding brief explanations of the concepts for people new to or fuzzy about HTTP, but didn't go into details of implementation. For example, introducing the concept of "HTTP header", but not explaining how HTTP implements them "on the wire" (though in fact I think it would be a good thing to add one example that showed an HTTP request and pointed out the request line, the headers and the data, since that makes everything very concrete and easy to grasp for newbies). Removed link to external howto on cookie handling. Despite the description ("How to handle cookies, when fetching web pages with Python."), this actually spends most of its time discussing what conditional imports are needed if you want to be maximally compatible across libraries and older versions of Python. While that is certainly useful for people who need that, I think this is rather obscure and distracting detail that seems out of place being referenced from the Python 2.5 documentation, even in a howto. Perhaps some general statement that further tutorials are available on your site? Referencing your basic auth tutorial seems fine. You limit mention of urllib2.urlopen(url) to a footnote, and in the text of the tutorial itself, you say: """urllib2 mirrors this by having you form a ``Request``""" . That's not true: a string URL is fine, as you explain in the footnote. That seems an innaccuracy with no obvious didactic payoff. In the footnote, you say: """You *can* fetch URLs directly with urlopen, without using a request object. It's more explicit, and therefore more Pythonic, to use ``urllib2.Request`` though. It also makes it easier to add headers to your request. I find that bizarre! Why is urlopen(url) unpythonic?? On the contrary, using an extra object for no reason *does* seem unpythonic to me. I rewrote this a bit. You needlessly assign the_url = "http:...", then request = Request(the_url) -- why not a single line? Where it's useful to do that (i.e. in the more complicated examples), I've s/the_url/url/, since I object to chaff like "the_" in variable names ;-) Your discussion of Request implies that it only represents HTTP requests. Fixed that. Use of the word "handle" to talk about response objects is unfortunate for two reasons: First, many objects in Python are "handles" in some sense ("object reference" semantics), so it's too vague to be a helpful name. Second, it's particularly unfortunate to use the word "handle" when urllib2 makes heavy use of "handler" objects that "handle" requests. The fact that methods on these handlers often return your "handles" only makes things more confusing! s/handle/response/ """Sometimes you want to **POST** data to a CGI (Common Gateway Interface) [#]_ or other web application""" It's clear to us old hands what you mean here, but in a tutorial at the level you seem to have picked we probably shouldn't expect the reader to have all these concepts straight, so being sloppy here is bad. - By "a CGI" I'm guessing you mean "a CGI script/program". Also, the whole sentence is unclear whether you're talking about a web application in the abstract, or some concrete CGI script. I certainly remember being very confused about this kind of thing as a newbie. - "...or other web application" implies that all POSTs go to web applications. That's using "web application" in a broader sense than it's usually understood. - You introduce "POST" without explanation. Would be nice to say "send data" instead of "POST", then explain POST. I rewrote this bit to try to address those points. Re POST: """This is what your browser does when you fill in a FORM on the web""" Thats needed qualifying: form submission can also result in a GET. I added a bit on side-effects and GET/POST. """You may be mimicking a FORM submission, or transmitting data to your own application.""" This reads oddly to me. I know what you're getting at (forms are not part of HTTP), but surely if you are submitting form data you're not "mimicking" form submission, you *are* submitting a form. And in an English sentence the "or" reads as an "exclusive or"; with that in mind: In what sense does form submission *not* involve "transmitting data to your own application"? Reworded and s/FORM/HTML form/, since we're talking about the abstract thing rather than specifically about the HTML element. """In either case the data needs to be encoded for safe transmission over HTTP""" Arbitrary binary data does not need to be URL-encoded. Rephrased. """The encoding is done using a function from the ``urllib`` library *not* from ``urllib2``. ::""" This is not true in general even for HTML forms. For example, HTML form file upload data is not encoded in this way. There are more obscure cases, too. Noted this. The quoted User-Agent string was out-of-date. Fixed, noting that it changes with each minor Python version. Headers / data : I added a bit of explanatory context to tell people what we're about to explain, and break up paragraphs / add sections to clarify the structure. Also explained the concept of "HTTP header", as I noted above. XXX example needed on GET with urlencoded data (as it's written ATM, this would go immediately before the "Headers" section). """Coping With Errors""" "Handling exceptions" seems more accurate. Not all HTTP status codes for which urllib2 raises an exception involve HTTP error responses. The text is also confused on this point, so I rewrote it. Errors: I believe urlopen can still actually raise socket.error. This is a bug, but I haven't dared to submit a patch to fix it, fearing backwards-compatibility issues. I guess it should probably be documented :-( But I suppose we should discuss that in a separate tracker item, rather than adding it to your howto straight away. You mention IOError. Without a motivating use case I don't know why you mention this. Since I'm not really sure what the use case for this subclassing was ever intended to be :-) I removed this example: feel free to add it back if you know of a use or can get Jeremy Hylton to explain it to you ;-) Re URLError : you imply that the only reason for URLError to be raised is failure to connect to the server. This is often the cause, but certainly not always. For HTTP status codes, you refer to a document that states "This is a historic document and is not accurate anymore". RFC 2616 is authoritative, and IMHO fairly readable on error codes. Removed the reference to the other document. """As of Python 2.5 a dictionary like this one has become part of ``urllib2``.""" In fact, this was moved to httplib. The reference to "HTTPBaseServer" (sic) is interesting: I think the copy in httplib should be removed, since it's already there in BaseHTTPServer (albeit missing 306, but that is unused) -- would you mind filing a patch, Michael? Your listing differed from BaseHTTPServer and from RFC 2616, so I replaced it with the BaseHTTPServer copy. """shows all the defined response codes""" These are only those defined by RFC 2616 of course: other standards can and do define other response status codes (e.g. DAV). Clarified this. """When an error is raised the server responds by returning an http error code *and* an error page.""" This is sloppy: HTTP doesn't define "raising" an error, so it can't respond to one. Fixed. httplib.HTTPMessage Reworded to avoid impling it's *always* going to be this concrete class. """In versions of Python prior to 2.3.4 it wasn't safe to iterate over the object directly, so you should iterate over the list returned by ``msg.keys()`` instead.""" Is this appropriate advice in the 2.5 docs? I removed this (am I too harsh on this point?). """Openers and handlers are slightly esoteric parts of **urllib2**.""" I don't want to scare people off: they're easy to use (if not to write). Removed this. I added a tiny bit more on what handlers do. Changed the text to avoid implying that build_opener() is the only way to create openers. Don't refer to ``opener`` in those typewriter-font ReST backticks, since that seems a little misleading: it's not a Python class name (unfortunately the class is named OpenerDirector, which rather clashes with the use of the name "opener" of course, but personally I'm with you in preferring "opener"). Wrote a bit more about opener construction. Changed realm name to make it clear it may contain spaces. Changed references to URI to URL in discussion of authentication -- seems an irrelevant and distracting distinction here. I edited the basic auth description a little. Comments conventionally come *before* code it refers to, not after. Fixed that, removed an over-obvious comment or two (even in docs, "create the handler" seems redundant if that's *all* it says), and the fixed the curious line breaks. """The only reason to explicitly supply these to ``build_opener`` (which chains handlers provided as a list), would be to change the order they appear in the chain.""" I don't know of a use case for that in the case of the handlers you list. Also, that doesn't actually work: handler ordering is determined by sorting. Removed this. """One thing not to get bitten by is that the ``top_level_url`` in the code above *must not* contain the protocol - the ``http://`` part. So if the URL we are trying to access is""" This is not correct usage (though I can see why it worked); removed it. Admittedly, urllib2 auth was the subject of a quite a few bug fixes recently (I seem to have just found yet another one five minutes ago, in fact :-( ), so the situation pre-2.5 was certainly messy. However, I advise against trying to document the old bugs! Note that I haven't given examples of "sub-URLs" since the RFC (2617) isn't clear to me on this point, and I haven't yet tested whether urllib2 gets it right according to de-facto standards (as defined by browsers, Apache, etc.) for "sub-URLs" of the one passed to .add_password(). It's on the list... In your note explaining that HTTPS proxies are not supported, you use "caution" rather than "note", which conveys the strange implication to me that this lack of support is somehow a consequence of using your previous recipe for switching off proxy handling (or am I weird in reading it that way??). s/caution/note/ """.. [#] Possibly some of this tutorial will make it into the standard library docs for versions of Python after 2.4.1.""" Removed this. Whew! ---------------------------------------------------------------------- >Comment By: A.M. Kuchling (akuchling) Date: 2006-05-07 13:13 Message: Logged In: YES user_id=11375 Edited.rst has been committed; thanks! ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-05-01 15:59 Message: Logged In: YES user_id=261020 (I guess if I had any sense in me, I would have uploaded those comments as an attachment instead of pasting them into the summary -- sorry.) I'm uploading the revised document now. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479977&group_id=5470 From noreply at sourceforge.net Sun May 7 22:45:17 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 13:45:17 -0700 Subject: [Patches] [ python-Patches-1483395 ] Add new top-level domains to cookielib Message-ID: Patches item #1483395, was opened at 2006-05-07 16:03 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: Add new top-level domains to cookielib Initial Comment: IANA introduced some new top-level domains in addition to the original seven. This adds them to cookielib, and adds a test for the relevant behaviour (blacklisting of some "country-code TLDs"). 2.4 backport candidate. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-07 20:45 Message: Logged In: YES user_id=849994 Committed in rev. 45934. Note that the patch contained a new test which imported a module "mechanize", which doesn't belong to the stdlib. I removed that test. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 From noreply at sourceforge.net Sun May 7 23:23:07 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 14:23:07 -0700 Subject: [Patches] [ python-Patches-1483395 ] Add new top-level domains to cookielib Message-ID: Patches item #1483395, was opened at 2006-05-07 17:03 Message generated for change (Comment added) made by jjlee You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Open Resolution: Accepted Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: Add new top-level domains to cookielib Initial Comment: IANA introduced some new top-level domains in addition to the original seven. This adds them to cookielib, and adds a test for the relevant behaviour (blacklisting of some "country-code TLDs"). 2.4 backport candidate. ---------------------------------------------------------------------- >Comment By: John J Lee (jjlee) Date: 2006-05-07 22:23 Message: Logged In: YES user_id=261020 Oops, could you commit the test with that "mechanize" replaced by "cookielib"? I just ran the tests with that change and it passes. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-07 21:45 Message: Logged In: YES user_id=849994 Committed in rev. 45934. Note that the patch contained a new test which imported a module "mechanize", which doesn't belong to the stdlib. I removed that test. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 From noreply at sourceforge.net Sun May 7 23:33:08 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 14:33:08 -0700 Subject: [Patches] [ python-Patches-1483395 ] Add new top-level domains to cookielib Message-ID: Patches item #1483395, was opened at 2006-05-07 17:03 Message generated for change (Settings changed) made by jjlee You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: Accepted Priority: 5 Submitted By: John J Lee (jjlee) >Assigned to: Georg Brandl (gbrandl) Summary: Add new top-level domains to cookielib Initial Comment: IANA introduced some new top-level domains in addition to the original seven. This adds them to cookielib, and adds a test for the relevant behaviour (blacklisting of some "country-code TLDs"). 2.4 backport candidate. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-05-07 22:23 Message: Logged In: YES user_id=261020 Oops, could you commit the test with that "mechanize" replaced by "cookielib"? I just ran the tests with that change and it passes. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-07 21:45 Message: Logged In: YES user_id=849994 Committed in rev. 45934. Note that the patch contained a new test which imported a module "mechanize", which doesn't belong to the stdlib. I removed that test. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 From noreply at sourceforge.net Mon May 8 00:27:09 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 15:27:09 -0700 Subject: [Patches] [ python-Patches-972322 ] urllib2 handler naming convention collision Message-ID: Patches item #972322, was opened at 2004-06-14 00:16 Message generated for change (Comment added) made by jjlee You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=972322&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Jeremy Hylton (jhylton) Summary: urllib2 handler naming convention collision Initial Comment: The method naming conventions of *_open and *_request in urllib2 are accidentally met by the following methods: AbstractHTTPHandler.do_open() ProxyHandler.proxy_open() AbstractHTTPHandler.redirect_request() So URLs like do://example.com/ are regarded as having a handler, and urllib2.urlopen("do://python.org/") causes a TypeError. I think *something* should be done about this, but I'm willing to provide a different patch if this one is frowned upon. The alternative would be to rename do_open and proxy_open, and leave the redirect_request case unchanged (see below for why). The first two methods are undocumented, so could in theory be renamed. However, people will likely be overriding them anyway, so perhaps it's better to apply this ugly patch than rename them. redirect_request is documented, so can't be renamed, but it will never be accidentally called unless somebody actually adds a handler with a method named "redirect_open". ---------------------------------------------------------------------- >Comment By: John J Lee (jjlee) Date: 2006-05-07 23:27 Message: Logged In: YES user_id=261020 OK, I see a slightly less ugly fix, don't apply this. I intend to upload a better one later. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-03-31 22:16 Message: Logged In: YES user_id=261020 Here's an updated patch (collision.patch) that applies against SVN HEAD. I also made the test a little clearer. collision.patch supercedes both urllib2.py.patch and test_urllib2.py.patch ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2005-05-19 21:53 Message: Logged In: YES user_id=261020 Since nobody seems to mind the slightly uglified code required to fix these bugs in a backwards-compatible way, could somebody please apply this patch? ---------------------------------------------------------------------- Comment By: Michael Chermside (mcherm) Date: 2004-10-22 17:36 Message: Logged In: YES user_id=99874 I have reviewed this patch and I recomend applying it. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=972322&group_id=5470 From noreply at sourceforge.net Mon May 8 01:23:40 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Sun, 07 May 2006 16:23:40 -0700 Subject: [Patches] [ python-Patches-1483545 ] Wave.py support for ulaw and alaw audio Message-ID: Patches item #1483545, was opened at 2006-05-07 19:23 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483545&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Eric Woudenberg (ewoudenberg) Assigned to: Nobody/Anonymous (nobody) Summary: Wave.py support for ulaw and alaw audio Initial Comment: Dear Python Patch Center: This is my first Python patch submission. Apologies for any errors of protocol. I have been using these submitted changes for several years and have even given it out (I mentioned it a few years back on a python mailing list.) I am confident to the best of my ability that these changes are solid. Unfortunately I don't have the capability of rebuilding the documentation, but the changes to the documentation are outlined below. Please do not hesitate to contact me for further information or assistance. I would be honored to have these changes become part of some Python revision, be it 2.5 or something further in the future. Thank you, Eric Woudenberg eaw at connact.com >From my version of the wave.py file: These changes allow .wav files containing u-law and a-law data to be read and written. The user visible changes are: 1) After a .wav file containing mu-law or a-law data is opened for reading, a call to getcomptype() returns 'ULAW' (resp. 'ALAW') and a call to getcompname() returns 'CCITT G.711 u-law' (resp. 'CCITT G.711 a-law'). 2) After a wave object is created for writing, setcomptype() can be called with the arguments ('ULAW', 'CCITT G.711 u-law') (resp. 'ALAW', 'CCITT G.711 a-law'). The second argument (text description) is ignored. 3) The comptype 'PCM' is now a synonym for 'NONE'. PCM-containing wave files will return 'PCM' instead of 'NONE' for their comptype. Note that this module does not do any u-law or a-law format conversion to PCM, it simply allows users to read or write u-law/a-law data from/to .wav files that have conforming headers. For audio conversion of PCM data to or from u-law, use the audioop module. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483545&group_id=5470 From noreply at sourceforge.net Mon May 8 19:29:00 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 08 May 2006 10:29:00 -0700 Subject: [Patches] [ python-Patches-1483395 ] Add new top-level domains to cookielib Message-ID: Patches item #1483395, was opened at 2006-05-07 16:03 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed Resolution: Accepted Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Georg Brandl (gbrandl) Summary: Add new top-level domains to cookielib Initial Comment: IANA introduced some new top-level domains in addition to the original seven. This adds them to cookielib, and adds a test for the relevant behaviour (blacklisting of some "country-code TLDs"). 2.4 backport candidate. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-08 17:29 Message: Logged In: YES user_id=849994 Done in 45938. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-05-07 21:23 Message: Logged In: YES user_id=261020 Oops, could you commit the test with that "mechanize" replaced by "cookielib"? I just ran the tests with that change and it passes. ---------------------------------------------------------------------- Comment By: Georg Brandl (gbrandl) Date: 2006-05-07 20:45 Message: Logged In: YES user_id=849994 Committed in rev. 45934. Note that the patch contained a new test which imported a module "mechanize", which doesn't belong to the stdlib. I removed that test. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483395&group_id=5470 From noreply at sourceforge.net Mon May 8 19:36:30 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 08 May 2006 10:36:30 -0700 Subject: [Patches] [ python-Patches-1479302 ] Make urllib2 digest auth and basic auth play together Message-ID: Patches item #1479302, was opened at 2006-04-30 13:15 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479302&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: Make urllib2 digest auth and basic auth play together Initial Comment: urllib2.HTTPDigestAuthHandler breaks urllib2's handler scheme by raising an exception instead of returning None to indicate another handler might handle the response. This stops everything in its tracks (the exception is not caught by urllib2) and prevents urllib2.HTTPBasicAuthHandler from handling basic auth scheme 40* responses. The patch simply removes the raise statement, so that the .http_error_auth_reqed(), and therefore .http_error_40*(), returns None. There is also a unit test. (will upload patch in a sec when I have the tracker ID to insert in the test) 2.4 backport candidate. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-08 17:36 Message: Logged In: YES user_id=849994 Applied as rev. 45939. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 14:37 Message: Logged In: YES user_id=261020 Argh, posted to the wrong tracker item for that last comment, too many bugs on the go at once, sorry. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 14:36 Message: Logged In: YES user_id=261020 (...and the new patch makes a tiny fix to a slightly-inaccurate statement in the module docstring) ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 13:42 Message: Logged In: YES user_id=261020 Hmm, on second thoughts: use of module logging only solves the debugging problem. People may want to programatically handle failure of authentication (and, say, report to the user "authentication failed, you entered the wrong username or password", or "authentication failed: hash algorithm YYY not implemented"). That doesn't make applying this patch a bad idea, because the HTTPDigestAuthHandler ValueError is not useful for that purpose. People wanting to handle this at run time can (already) and should catch the HTTPError that will eventually be raised when no handler handles the 40* reponse. (although the bug addressed by this patch breaks that in one very specific case, of course: where both digest + basic handlers are present, and a basic auth challenge is received) In summary, this patch should be applied, but we should also , as an additional feature, think up some way of allowing auth failure information to be reported by these handlers (probably by stuffing the info into the HTTPError). ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 13:25 Message: Logged In: YES user_id=261020 Just a note that an XXX comment at the top of the code comments that: """ If an authentication error handler that tries to perform authentication for some reason but fails, how should the error be signalled? The client needs to know the HTTP error code. But if the handler knows that the problem was, e.g., that it didn't know that hash algo that requested in the challenge, it would be good to pass that information along to the client, too. """ I think this problem should be handled using module logging, similarly to how module cookielib logs its reasoning for accepting and returning cookies. Do people agree? If so, I'll file another patch to add that. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479302&group_id=5470 From noreply at sourceforge.net Mon May 8 19:48:22 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 08 May 2006 10:48:22 -0700 Subject: [Patches] [ python-Patches-1478993 ] Take advantage of BaseException/Exception split in cookielib Message-ID: Patches item #1478993, was opened at 2006-04-29 16:43 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1478993&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: John J Lee (jjlee) Assigned to: Nobody/Anonymous (nobody) Summary: Take advantage of BaseException/Exception split in cookielib Initial Comment: The patch takes advantage of the exception hierarchy reorganisation to remove some ugly code in cookielib. It clarifies a couple of exception messages, too. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-08 17:48 Message: Logged In: YES user_id=849994 Applied in rev. 45940. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 14:38 Message: Logged In: YES user_id=261020 (...and the new patch makes a tiny fix to a slightly-inaccurate statement in the module docstring) ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 14:35 Message: Logged In: YES user_id=261020 I still think there should be documented guidelines on this for both stdlib users and contributors (though they should not live in the cookielib docs, of course!). I've added a new patch to this tracker item, cookielib_baseexception_2.patch, that adds an __all__ and uses the name _warn_unhandled_exception (in addition to the original patch contents). I verified that the set of documented module globals is identical to those listed in the new __all__. Let me know if I should also rename all other non-public module globals defined in cookielib to have initial underscores. Just for the record, here is the list of all module globals NOT listed in the new __all__ (as determined by doing dir(cookielib) prior to applying the patch, and removing the items I didn't add to __all__, so includes some noise from things like 'import sys'): ['Absent', 'DAYS', 'DEFAULT_HTTP_PORT', 'EPOCH_YEAR', 'ESCAPED_CHAR_RE', 'HEADER_ESCAPE_RE', 'HEADER_JOIN_ESCAPE_RE', 'HEADER_QUOTED_VALUE_RE', 'HEADER_TOKEN_RE', 'HEADER_VALUE_RE', 'HTTP_PATH_SAFE', 'IPV4_RE', 'ISO_DATE_RE', 'LOOSE_HTTP_DATE_RE', 'MISSING_FILENAME_TEXT', 'MONTHS', 'MONTHS_LOWER', 'STRICT_DATE_RE', 'TIMEZONE_RE', 'UTC_ZONES', 'WEEKDAY_RE', '__builtins__', '__doc__', '__file__', '__name__', '_str2time', '_threading', '_timegm', 'copy', 'cut_port_re', 'debug', 'deepvalues', 'domain_match', 'eff_request_host', 'escape_path', 'http2time', 'httplib', 'is_HDN', 'is_third_party', 'iso2time', 'join_header_words', 'liberal_is_HDN', 'logging', 'lwp_cookie_str', 'month', 'offset_from_tz_string', 'parse_ns_headers', 're', 'reach', 'request_host', 'request_path', 'request_port', 'split_header_words', 'sys', 'time', 'time2isoz', 'time2netscape', 'timegm', 'unmatched', 'uppercase_escaped_char', 'urllib', 'urlparse', 'user_domain_match', 'vals_sorted_by_key', 'warn_unhandled_exception'] ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-04-30 02:23 Message: Logged In: YES user_id=357491 You should use an underscore if you don't have an __all__ defined. This is mostly for protection for ``from cookielib import *`` code. But if you define an __all__ I think you are fine. You do not need to document that undocumented globals should not be relied upon. Yes, people should know better (I personally got nailed by the Debian folk for an undocumented function in site.py that I changed the parameters of). But the suggestions Neal and I are making are to protect people who do their doc checking from the command-line and thus just do ``import cookielib; dir(cookielib)``. I know Python is for use by adults, but sometimes going a small step to protect the adolescents is also okay. =) ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 01:31 Message: Logged In: YES user_id=261020 Hmm, perhaps by "do no worse" you simply meant not to rename the function in this tracker item to a name not beginning with an initial underscore (since that would introduce a new non-public module global that does not begin with an underscore). In which case, sorry for the rant. :-) My questions after the rant still stand. ;-) ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 01:26 Message: Logged In: YES user_id=261020 Bleh. e). Stdlib users should assume all undocumented module globals are not part of the public API (I guess this should be go somewhere near the library reference introduction) ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 01:22 Message: Logged In: YES user_id=261020 So, you (Neil) agree with my three numbered action points below? You repeat my suggestion that we document this as if it were a new suggestion; did you read my comment? Sigh, sorry for being a little grumpy about this, but it's hard to "do no worse" for a project if that project doesn't seem itself to be very sure what it considers "worse": While I must say I *agree* with you that such practices are not good, if I as somebody apparently unusually inclined to heavy use of underscores (even in most of my module names, in library code) actually thought, however foolishly, that I was *following stdlib conventions* by using *fewer* underscores (for reasons I'll try to refrain from debating further here), it does indeed seem pretty clear we're in need of explicit documentation on this! So your advice to "do no worse" is a little annoying at this point... :-) OK, so, what should get documented, specifically? And where should documentation for module authors go? a). Stdlib module authors should always use underscores for non-public module globals. b). Don't know about this one: should non-legacy stdlib modules (viz, those that follow rule a)) define __all__? (perhaps a point against doing this is that it may encourage import * ?). c). Stdlib packages should use __init__ to export public names. d). Any discrepancy between __all__ and the API documentation is a bug. e). Stdlib users should assume all non- (I guess this should be go somewhere near the library reference introduction) Finally, how about my point 2? Should I add underscores to cookielib module globals I consider non-public (== all undocumented module globals), or not? Thanks for the feedback, both of you! ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-04-30 00:37 Message: Logged In: YES user_id=33168 John, at this point (2.x) we should at least do no worse. Don't export unnecessary vars in any new code. We should also start documenting the situation and work towards improving it. For 3k, we should do better and solidify the rules and do massive cleanup (module by module). This will probably involve some arm twisting of Guido. :-) ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2006-04-30 00:27 Message: Logged In: YES user_id=261020 I changed the exception detail strings to use %r to get quotes around the filename and quoted "bad" line read from the file. That makes it clearer what is part of the explanatory English text and what is part of the filename (or part of the quoted bad line, as the case may be). Filenames can and do contain spaces, commas, etc. Your other point stunned me a bit. I don't think it had ever even really *occurred* to me that stdlib users might consider stdlib module globals that are not documented as public. Ironically, I think that's because the code from which cookielib derives is much stricter about this, all modules starting with '_' and package __init__ exporting a short list of names -- I guess I thought I was following stdlib conventions by *not* adding initial underscores all over the place. Looking at some other stdlib code, I see that underscores would have been more conventional after all. Searching for reassurance, I discovered this from one of your old python-dev summaries that confirms that undocumented stdlib module globals are not considered part of the module public interface: http://www.python.org/dev/summary/2004-07-16_2004-07-31/#use-the-docs-to-know-what-the-public-api-is-people e.g. from Tim Peters: """ As you noted later, it wasn't part of keyword's documented interface, and you *always* act at your own risk when you go beyond the docs. """ However, I don't see that this is explicitly documented, which seems unfortunate to me (even though Tim's statement is true regardless of any convention Python might have). So, I guess I should: 1. Write something explicit about this (along the lines of "Use undocumented module globals at your own risk") for the stdlib library docs -- perhaps starting from Tim's post -- and submit that as a doc patch. 2. Leave all module global names in cookielib unchanged (so people using those functions don't suffer gratuitous breakage, even though any such people are asking for trouble in the long run). However, in the thread above, Michael Hudson disagrees with that, and suggests all such module globals be renamed. So suggestions are welcome here on the best course of action. 3. As you suggest, submit a patch to add an __all__ to cookielib. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-04-29 21:12 Message: Logged In: YES user_id=357491 Overall the patch looks fine (on vacation so not up for applying and handling any possible failures so not going to assign to myself). But a question and a suggestion. Why were the error strings changed to use the repr instead of the string representation? What does it buy you? And if you are going to be changing the function name, you might want to consider using a leading underscore to prevent people from using it or getting exported. Otherwise I would define __all__ for the module. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1478993&group_id=5470 From noreply at sourceforge.net Tue May 9 06:13:33 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Mon, 08 May 2006 21:13:33 -0700 Subject: [Patches] [ python-Patches-1483325 ] Patch fixing #1481770 (wrong shared lib ext on hpux ia64) Message-ID: Patches item #1483325, was opened at 2006-05-07 07:33 Message generated for change (Comment added) made by deckrider You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483325&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: David Everly (deckrider) Assigned to: Nobody/Anonymous (nobody) Summary: Patch fixing #1481770 (wrong shared lib ext on hpux ia64) Initial Comment: (configure and pyconfig.h.in must be regenerated after applying this patch) Not heavily tested, since I only have Linux (i686) at home. Will test on HPUX ia64 tomorrow and report back. ---------------------------------------------------------------------- >Comment By: David Everly (deckrider) Date: 2006-05-08 22:13 Message: Logged In: YES user_id=1113403 I tested against hpux ia64 today, and found that the original patch required correction. Here is the result. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483325&group_id=5470 From noreply at sourceforge.net Tue May 9 15:51:18 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 09 May 2006 06:51:18 -0700 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 15:51 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Tue May 9 15:52:23 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 09 May 2006 06:52:23 -0700 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 15:51 Message generated for change (Comment added) made by gustaebel You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- >Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-09 15:52 Message: Logged In: YES user_id=642936 Here is testtar.tar to replace Lib/test/testtar.tar. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Tue May 9 17:14:20 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 09 May 2006 08:14:20 -0700 Subject: [Patches] [ python-Patches-1484758 ] cookielib: reduce (fatal) dependency on "beta" logging? Message-ID: Patches item #1484758, was opened at 2006-05-09 17:14 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484758&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: kxroberto (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: cookielib: reduce (fatal) dependency on "beta" logging? Initial Comment: The logging package is tagged "beta". Yet cookielib (as the ONLY module in the std. lib !?) uses Logger.debug() very excessively. I got occasional nasty crash traces (from users) when using cookielib Processors through urllib2 (multi-threaded usage) - see below. The causes are not errors in cookielib, but upon simple calls to Logger.debug() : varying AttributeError's in logging, which on the first glance seem to be impossible, as those attributes are set in the related __init__()'s but there are strange complex things going on with roots/hierarchies/copy etc. so.... thread/lock problems I'd guess. the patch uncomments several debug() calls in cookielib in import. only one's in important high-frequency execution flow path (not ones upon errors and exceptional states). And 2 minor fixes on pychecker warnings. After applying that, the nasty crash reports disappeared. I do not understand completely why the cookielib production code has to use the logging package (expensive) at all. At least for the high-frq used add_cookie_header its unnecessary. There could be some simpler (detached) test code for testing purposes. Importing the logging and setup is time consuming etc. (see other patch for urllib2 import optimization. ) I'd recommend: At least as far as logging is "beta" and cookielib NOT, all these debug()'s should be uncommented, or at least called ONLY upon a dispatching global 'use_logging' variable in cookielib, in case the test code cannot be externalized nicely. 2 example error traces: ...File "cookielib.pyo", line 1303, in add_cookie_header\\n\', \' File "logging\\\\__init__.pyo", line 878, in debug\\n\', \' File "logging\\\\__init__.pyo", line 1056, in getEffectiveLevel\\n\', "AttributeError: Logger instance has no attribute \'level\'\\n ...in http_request\\n\', \' File "cookielib.pyo", line 1303, in add_cookie_header\\n\', \' File "logging\\\\__init__.pyo", line 876, in debug\\n\', "AttributeError: Manager instance has no attribute \'disable\'\\n -robert ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484758&group_id=5470 From noreply at sourceforge.net Tue May 9 17:59:48 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 09 May 2006 08:59:48 -0700 Subject: [Patches] [ python-Patches-1484793 ] urllib2: resolves extremly slow import (of "everything") Message-ID: Patches item #1484793, was opened at 2006-05-09 17:59 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484793&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.4 Status: Open Resolution: None Priority: 5 Submitted By: kxroberto (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2: resolves extremly slow import (of "everything") Initial Comment: This superseeds the old patch #1053150 (for an older Python; it was stopped: "Jeremy doesn't like the idea") in order to import the expensive modules behind urllib2 late. I'm recommending now again to do this, as things are almost unacceptable meanwhile. In Py24, simply importing original urllib2 costs upto to a second on my slower machines. the startup time of some of my bigger apps/scripts goes mainly to importing urllib2. More than half of the time goes into importing cookielib (regarding profiler runs). Its almost unusable so now in CGI scripts. New modules were added to urllib2 meanwhile, and worst of all the cookielib was inserted into urllib2 the same old style "import everything on top of the file in a kind of C-#include manner". Python offers best dynamic modularization of code. That should be exploited for such an expensive virtualization module like urllib2. There are usually only very locations, where the sub-modules are referenced. This patch also enables to strip off unnecessary modules (down to _MozillaCookieJar!) for cx_freeze/py2exe distribution. ( Since long I have this patch on my list, which I apply after each Python installation regularly. ) -- As a side effect of this import-all practice a lazy cookielib dependency came into normal Request constructor code: "origin_req_host = cookielib.request_host(self)" I'd recommend, to copy/move this simple tool function request_host into urllib2 in order to resolve the cookielib dependency completely. (not done so far in the patch) -robert ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484793&group_id=5470 From mersch.bernhard at osnanet.de Tue May 9 20:33:07 2006 From: mersch.bernhard at osnanet.de (mersch.bernhard) Date: Tue, 9 May 2006 20:33:07 +0200 Subject: [Patches] Nastiest of fresh girls. Message-ID: From noreply at sourceforge.net Wed May 10 04:48:51 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Tue, 09 May 2006 19:48:51 -0700 Subject: [Patches] [ python-Patches-1478292 ] Fix doctest nit. Message-ID: Patches item #1478292, was opened at 2006-04-28 05:54 Message generated for change (Comment added) made by tim_one You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1478292&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Thomas Heller (theller) >Assigned to: Nobody/Anonymous (nobody) Summary: Fix doctest nit. Initial Comment: I was puzzled by this behaviour: C:\>py25 Python 2.5a2 (r25a2:45740, Apr 27 2006, 06:31:19) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> from doctest import register_optionflag >>> print register_optionflag("SPAM") 1024 >>> print register_optionflag("SPAM") 2048 >>> print register_optionflag("SPAM") 2048 >>> I suggest that register_optionflags does not re-register already registered flags. ---------------------------------------------------------------------- >Comment By: Tim Peters (tim_one) Date: 2006-05-09 22:48 Message: Logged In: YES user_id=31435 I agree that this behavior wasn't intended. Fixed in a simpler way, and added a test to ensure it stays fixed, in rev 45944 on the trunk and rev 45945 on the 2.4 branch. Thanks! ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1478292&group_id=5470 From noreply at sourceforge.net Wed May 10 18:26:31 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 10 May 2006 09:26:31 -0700 Subject: [Patches] [ python-Patches-1484695 ] tarfile.py fix for #1471427 and updates Message-ID: Patches item #1484695, was opened at 2006-05-09 13:51 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Lars Gust?bel (gustaebel) Assigned to: Nobody/Anonymous (nobody) Summary: tarfile.py fix for #1471427 and updates Initial Comment: I have assembled a patch that adds some features from my own development path of tarfile.py (http://www.gustaebel.de/lars/tarfile/) and fixes #1471427 which made some restructuring necessary. The patch affects Lib/tarfile.py, Lib/test/test_tarfile.py and Doc/lib/libtarfile.tex. The changes the patch makes are as follows: Sets the version to 0.8.0. Support for base256 encoding of number fields (nti() and itn()). Up to now this was hardcoded for the filesize field to allow filesizes greater than 8 GB but it is applicable to all number fields. TarInfo.tobuf() has a boolean argument "posix" which controls how number fields are written (base256 is non-posix). Both unsigned and signed (Sun and NeXT) checksums are calculated. Header validation moves from TarFile.next() to TarInfo.frombuf(). A header is valid only if its checksum is okay, in the past the checksum was calculated but ignored. The TarFile.next() method was rearranged which makes header processing clearer and more abstract and fixes bug #1471427. However, this change breaks the interface for subclassing in order to implement custom member types but makes it much easier at the same time. The mapping TYPE_METH was removed. A new test ReadGNULongTest was added to test_tarfile.py and testtar.tar was updated to be able to test the GNU extensions LONGNAME and LONGLINK. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-10 16:26 Message: Logged In: YES user_id=849994 Thanks for the patch, applied as rev. 45954. ---------------------------------------------------------------------- Comment By: Lars Gust?bel (gustaebel) Date: 2006-05-09 13:52 Message: Logged In: YES user_id=642936 Here is testtar.tar to replace Lib/test/testtar.tar. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1484695&group_id=5470 From noreply at sourceforge.net Wed May 10 19:13:45 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 10 May 2006 10:13:45 -0700 Subject: [Patches] [ python-Patches-721464 ] Remote debugging with pdb.py Message-ID: Patches item #721464, was opened at 2003-04-14 23:02 Message generated for change (Comment added) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=721464&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.5 >Status: Closed >Resolution: Accepted Priority: 5 Submitted By: Laurent Pelecq (lpelecq) Assigned to: Raymond Hettinger (rhettinger) Summary: Remote debugging with pdb.py Initial Comment: With this patch, instances of pdb.Pdb can read and write from arbitrary file objects. It is based on similar changes that have been made to cmd.py. It basically consists of replacing print statement with calls to self.stdout.write. So it is possible for example to control the debugger from another terminal to debug curses-based applications or CGI scripts. I can provide a basic client/server debugger. This patch has been tested on Mandrake Linux 9.1 with the current CVS version. ---------------------------------------------------------------------- >Comment By: Georg Brandl (gbrandl) Date: 2006-05-10 17:13 Message: Logged In: YES user_id=849994 I committed a version of the patch using output redirection as rev. 45955. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-04-14 16:04 Message: Logged In: YES user_id=21627 lpelecq, would you be willing to redo that patch for 2.5? Using print redirection (instead of .write calls) might be the easiest way to do it. rhettinger, do you want to come back to this patch now? If not, please unassign. ---------------------------------------------------------------------- Comment By: Raymond Hettinger (rhettinger) Date: 2003-06-22 17:15 Message: Logged In: YES user_id=80475 I think this is a good idea. It is past the the time for being added to 2.3. Unassigning, but will come back to it for 2.4. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=721464&group_id=5470 From noreply at sourceforge.net Thu May 11 05:50:54 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 10 May 2006 20:50:54 -0700 Subject: [Patches] [ python-Patches-1429539 ] pdb: fix for 1326406 (import __main__ pdb failure) Message-ID: Patches item #1429539, was opened at 2006-02-10 19:34 Message generated for change (Comment added) made by isandler You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1429539&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: None Status: Open Resolution: None Priority: 5 Submitted By: Ilya Sandler (isandler) Assigned to: Nobody/Anonymous (nobody) Summary: pdb: fix for 1326406 (import __main__ pdb failure) Initial Comment: The patch allows pdb to debug program which import from __main__ ---------------------------------------------------------------------- >Comment By: Ilya Sandler (isandler) Date: 2006-05-10 20:50 Message: Logged In: YES user_id=971153 I'm attaching an alternative patch: the program stil runs in __main__ namespace, but pdb gets imported first: import pdb pdb.main() So the main program cann't accidentally stomp on pdb internals (e.g by doing help=None) (there is still a bit of namespace pollution in the main program) ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2006-04-23 11:10 Message: Logged In: YES user_id=971153 > 1. Could you give some code examples for that? Do you mean examples of intentional interference with debugger? Well, you could just traverse the stack and check whether the program runs under debugger and then do anything you want... But why do you think intentional interference would ever be an issue? After all python is not a language to write debugger-resistant applications ;-) Anyway, here are some examples of unintentional interference: 1. If you need a custom version of std module, you can modify sys.path and then import the module.. Which works by itself. But if pdb is loaded first and imports the module, then it does not work... 2. Similar problem with any application which changes sys.stdout/sys.stdin (there is actually a SF bug for that) 3. Also I don't see how pdb in its current form can control any program which needs a full-screen control of the terminal... 4. Any program which tries to do any magic with stack and assumes that top level stack frame is the main application will not work under pdb (where top level stack frame is pdb) --------------------------------------------------- And there is a whole separate bunch of intereference issues when pdb restarts the program. --------------------------------------------------- When a program does run in pdb's namespace (as would be the case if this patch is applied), pdb could save copies of all module global symbols which it needs and thus become immune to the accidental overwriting of those symbols in the main program... There could be a better way... ---------------------------------------------------------------------- Comment By: Kuba Ko??czyk (jakamkon) Date: 2006-04-21 08:28 Message: Logged In: YES user_id=1491175 Sorry I forget to login in;)The comment below is from me. ---------------------------------------------------------------------- Comment By: Nobody/Anonymous (nobody) Date: 2006-04-21 08:25 Message: Logged In: NO 1. Could you give some code examples for that? 2,3. Did you notice that google search for "from __main__ import" give hits similar to: t = Timer("test()", "from __main__ import test") in most situations? I think it's hard to value uses of "from..." based on google search or similar method.Maybe we shoud ask on python-list what are the others opinions? >As a middle ground it might be a good idea to expand the >patch to reduce pdb's dependency on module global symbols I'am interesting how would you do that? ---------------------------------------------------------------------- Comment By: Ilya Sandler (isandler) Date: 2006-04-20 19:39 Message: Logged In: YES user_id=971153 I do see your point (In fact it was me who submitted the patch #896011 which separated pdb namespace from the program's -- and thus broke imports from __main__ ;-)).. I do want to bring a couple of points: 1. I don't think it matters whether a program can intentionally interfere with pdb...Even when pdb's namespace is separated, it's easy for the program to interfere with debugger.. (Or delete your home directory for that matter) 2. Importing from __main_ may not be common in the std lib, but that's simply because stdlib doesn't contain that many executable hence there are very few places where there is __main__ to import from. google search for "from __main__ import" results in about 1M hits. 3. Just for the record, profile module does not separate its namespace from programs's either... So, basically, it boils down to this: what's worse breaking imports from __main__ or risking accidental interference between pdb and the program (e.g if your program redefines a help symbol)... As a middle ground it might be a good idea to expand the patch to reduce pdb's dependency on module global symbols and thus reducing the risk of interference. What do you think? ---------------------------------------------------------------------- Comment By: Kuba Ko??czyk (jakamkon) Date: 2006-04-20 04:17 Message: Logged In: YES user_id=1491175 I think that exposing pdb's namespaces for debugged code is dangerous.When debugged code have this kind of access he can dynamic change pdb's behaviour without your control: y.py: die = """\ def destroy(x,y): print 'Iam crashing your HOME and deleting your FILES' Pdb.__dict__['do_break'] = destroy # pdb's break = destroy """ x.py: # innocently looking code;) import y exec(y.puff) print "X" with your patch: $ python2.5 -m pdb x.py > /home/xyz/python/x.py(1)() -> import y (Pdb) Pdb.__dict__['do_break'] (Pdb) break (Pdb) n > /home/xyz/python/x.py(2)() -> exec(y.puff) (Pdb) n > /home/xyz/python/x.py(3)() -> print "X" (Pdb) Pdb.__dict__['do_break'] (Pdb) break Iam crashing your HOME and deleting your FILES I think that this patch can't be accepted due to above reason.According to my advanced reaserch;) ( find Lib/ -name '*.py' -exec grep 'from __main__ import' {} -ls \; ) 'from __main__' is rare case so maybe it will be reasonable to simply handle ImportError and print something like '** 'from __main__ import' not supported' message.What do you think? ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1429539&group_id=5470 From noreply at sourceforge.net Thu May 11 06:31:11 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 10 May 2006 21:31:11 -0700 Subject: [Patches] [ python-Patches-1053150 ] urllib2: better import ftplib and gopherlib etc late Message-ID: Patches item #1053150, was opened at 2004-10-24 13:50 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1053150&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: None Group: None >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: kxroberto (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: urllib2: better import ftplib and gopherlib etc late Initial Comment: importing those libs like (ftplib, gopherlib, ..) unconditionally on top of urllib2 slows down and hinders distributing small app packages (py2exe'd, mcm.installer, ...). simple patch in attachment ---------------------------------------------------------------------- >Comment By: Martin v. L??wis (loewis) Date: 2006-05-11 06:31 Message: Logged In: YES user_id=21627 Closing this as out-of-date; it is replaced by #1484793. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2005-05-19 22:38 Message: Logged In: YES user_id=261020 Since Jeremy doesn't like the idea (see tracker item ref. below), this should probably be closed, but: Robert originally submitted this as bug 1046077. ---------------------------------------------------------------------- Comment By: John J Lee (jjlee) Date: 2005-02-05 00:10 Message: Logged In: YES user_id=261020 Looks good to me. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1053150&group_id=5470 From noreply at sourceforge.net Thu May 11 06:33:28 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 10 May 2006 21:33:28 -0700 Subject: [Patches] [ python-Patches-1483325 ] Patch fixing #1481770 (wrong shared lib ext on hpux ia64) Message-ID: Patches item #1483325, was opened at 2006-05-07 15:33 Message generated for change (Comment added) made by loewis You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483325&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.4 >Status: Closed >Resolution: Out of Date Priority: 5 Submitted By: David Everly (deckrider) Assigned to: Nobody/Anonymous (nobody) Summary: Patch fixing #1481770 (wrong shared lib ext on hpux ia64) Initial Comment: (configure and pyconfig.h.in must be regenerated after applying this patch) Not heavily tested, since I only have Linux (i686) at home. Will test on HPUX ia64 tomorrow and report back. ---------------------------------------------------------------------- >Comment By: Martin v. L??wis (loewis) Date: 2006-05-11 06:33 Message: Logged In: YES user_id=21627 This is being tracked in #1481770; closing it here as out-of-date. ---------------------------------------------------------------------- Comment By: David Everly (deckrider) Date: 2006-05-09 06:13 Message: Logged In: YES user_id=1113403 I tested against hpux ia64 today, and found that the original patch required correction. Here is the result. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1483325&group_id=5470 From noreply at sourceforge.net Thu May 11 07:15:20 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Wed, 10 May 2006 22:15:20 -0700 Subject: [Patches] [ python-Patches-1474907 ] detect %zd format for PY_FORMAT_SIZE_T Message-ID: Patches item #1474907, was opened at 2006-04-22 23:18 Message generated for change (Settings changed) made by bcannon You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1474907&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Build Group: None >Status: Closed >Resolution: Fixed Priority: 5 Submitted By: Brett Cannon (bcannon) Assigned to: Brett Cannon (bcannon) Summary: detect %zd format for PY_FORMAT_SIZE_T Initial Comment: The patch modifies configure.in to add PY_FORMAT_SIZE_T to configure.in (meaning you need to run autoheader on configure.in) so that if %zd is supported for size_t it sets PY_FORMAT_SIZE_T to "z", otherwise it goes undefined and the preprocessor trickery in Include/pyport.h kicks in. This fix removes compiler warnings on OS X 10.4.6 with gcc 4.0.1 thanks to PY_FORMAT_SIZE_T being set to "". Initially assigned to Martin v. Loewis since he said this would be good to do and the Py_ssize_t stuff is his invention. ---------------------------------------------------------------------- >Comment By: Brett Cannon (bcannon) Date: 2006-05-10 22:15 Message: Logged In: YES user_id=357491 Applied in r45960 . ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-04-27 21:51 Message: Logged In: YES user_id=357491 Yeah, I tried to use a string constant as a stack value, but that didn't work. =) My brain just was not thinking in C when I first came up with the patch. I have a new version that uses a char array as the buffer. I am on vacation so I don't have the time to apply it and break buildbot, so I will hold off on applying if no one finds problems with this version. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-04-26 22:29 Message: Logged In: YES user_id=21627 Looks fine to me, although it has "unusual" style of C: - sizeof(char) is guaranteed to be 1 by the C standard. The C standard defines "char" and "byte" as synonyms, even if that means that "byte" has more than 8 bits. sizeof gives the number of bytes, so for char, it is always 1. - for a fixed-size array, people would normally make this an automatic (stack) variable, instead of bothering with explicit memory allocation, i.e. char str_buffer[4] Just out of fear of buffer overruns, many people would also add some horrendous overallocation, such as str_buffer[1024] :-) ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-04-26 22:16 Message: Logged In: YES user_id=357491 Realized there is a better way: just strncmp() for the expected result. Uploaded a new version. ---------------------------------------------------------------------- Comment By: Brett Cannon (bcannon) Date: 2006-04-26 21:59 Message: Logged In: YES user_id=357491 OK, uploaded a new version that uses strchr to check for '%', 'z', and 'd'. If it looks reasonable I will apply it and hope I don't break the buildbot. =) ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-04-26 09:15 Message: Logged In: YES user_id=21627 The patch seems to rely on printf returning <0 for the unrecognized format. That seems unreliable: atleast on Linux, printf just outputs the format as-is for unrecognized formats. Instead, I think it should use sprintf, and then check whether the result is the string "0" (in addition to checking whether the printf call itself failed). ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1474907&group_id=5470 From noreply at sourceforge.net Thu May 11 09:43:05 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 11 May 2006 00:43:05 -0700 Subject: [Patches] [ python-Patches-1479611 ] speed up function calls Message-ID: Patches item #1479611, was opened at 2006-04-30 23:58 Message generated for change (Comment added) made by nnorwitz You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Core (C code) Group: Python 2.5 Status: Open Resolution: None Priority: 5 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Nobody/Anonymous (nobody) Summary: speed up function calls Initial Comment: Results: 2.86% for 1 arg (len), 11.8% for 2 args (min), and 1.6% for pybench. trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.74 msec per loop trunk-speed$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 8.03 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): len([])' 100 loops, best of 3: 4.88 msec per loop trunk-clean$ ./python.exe -m timeit 'for x in xrange(10000): min(1,2)' 100 loops, best of 3: 9.09 msec per loop pybench goes from 5688.00 down to 5598.00 Details about the patch: There are 2 unrelated changes. They both seem to provide equal benefits for calling varargs C. One is very simple and just inlines calling a varargs C function rather than calling PyCFunction_Call() which does extra checks that are already known. This moves meth and self up one block. and breaks the C_TRACE into 2. (When looking at the patch, this will make sense I hope.) The other change is more dangerous. It modifies load_args() to hold on to tuples so they aren't allocated and deallocated. The initialization is done one time in the new func _PyEval_Init(). It allocates 64 tuples of size 8 that are never deallocated. The idea is that there won't be usually be more than 64 frames with 8 or less parameters active on the stack at any one time (stack depth). There are cases where this can degenerate, but for the most part, it should only be marginally slower, but generally this should be a fair amount faster by skipping the alloc and dealloc and some extra work. My decrementing the _last_index inside the needs_free blocks, that could improve behaviour. This really needs comments added to the code. But I'm not gonna get there tonight. I'd be interested in comments about the code. ---------------------------------------------------------------------- >Comment By: Neal Norwitz (nnorwitz) Date: 2006-05-11 00:43 Message: Logged In: YES user_id=33168 This version actually works (in both normal and debug builds). It adds some stats which are useful and updates Misc/SpecialBuilds.txt. I modified to not preallocate and only hold a ref when the function didn't keep a ref. I still need to inline more of PyCFunction_Call. Speed is still the same as before. I'm not sure if I'll finish this before the sprint next week. Anyone there feel free to check this in if you finish it. ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-05-05 01:27 Message: Logged In: YES user_id=33168 v2 attached. You might not want to review yet. I mostly did the first part of your suggest (stats, _Fini, and stack-like if I understood you correctly). I didn't do anything on the second part about inlinting Function_Call. perf seems to be about the same. I'm not entirely sure the patch is correct yet. I found one or two problems in the original. I added some more comments. ---------------------------------------------------------------------- Comment By: Martin v. L??wis (loewis) Date: 2006-05-01 01:27 Message: Logged In: YES user_id=21627 The tuples should get deallocated when Py_Finalize is called. It would be good if there was (conditional) statistical analysis, showing how often no tuple was found because the number of arguments was too large, and how often no tuple was found because the candidate was in use. I think it should be more stack-like, starting off with no tuples allocated, then returning them inside the needs_free blocks only if the refcount is 1 (or 2?). This would avoid degeneralized cases where some function holds onto its argument tuple indefinitely, thus consuming all 64 tuples. For the other part, I think it would make the code more readable if it inlined PyCFunction_Call even more: the test for NOARGS|O could be integrated into the switch statement (one case for each), VARARGS and VARARGS|KEYWORDS would both load the arguments, then call the function directly (possibly with NULL keywords). OLDARGS should goto either METH_NOARGS, METH_O, or METH_VARARGS depending on na (if you don't like goto, modifying flags would work as well). ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2006-05-01 00:08 Message: Logged In: YES user_id=33168 I should note the numbers 64 and 8 are total guesses. It might be good to try and determine values based on empirical data. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1479611&group_id=5470 From noreply at sourceforge.net Thu May 11 19:19:37 2006 From: noreply at sourceforge.net (SourceForge.net) Date: Thu, 11 May 2006 10:19:37 -0700 Subject: [Patches] [ python-Patches-1486713 ] HTMLParser : A auto-tolerant parsing mode Message-ID: Patches item #1486713, was opened at 2006-05-11 19:19 Message generated for change (Tracker Item Submitted) made by Item Submitter You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=305470&aid=1486713&group_id=5470 Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Library (Lib) Group: Python 2.3 Status: Open Resolution: None Priority: 5 Submitted By: kxroberto (kxroberto) Assigned to: Nobody/Anonymous (nobody) Summary: HTMLParser : A auto-tolerant parsing mode Initial Comment: Changes: * Now allows missing spaces between attributes as its often seen on the web like this :