From gvanrossum at users.sourceforge.net Mon May 2 05:30:09 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Sun, 01 May 2005 20:30:09 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.12,1.13 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28163 Modified Files: pep-0340.txt Log Message: Establish the "alternative" version. The exception API is called __exit__(), and its signature is the same as that of the raise-statement. Still some loose ends. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- pep-0340.txt 29 Apr 2005 18:51:03 -0000 1.12 +++ pep-0340.txt 2 May 2005 03:30:07 -0000 1.13 @@ -26,30 +26,6 @@ this on python-dev recently [1], and I figured it would be time to write up a precise spec in PEP form. -Proposal Evolution - - The discussion on python-dev has changed my mind slightly on how - exceptions should be handled, but I don't have the time to do a - full update of the PEP right now. Basically, I'm now in favor of - a variation on the exception handling proposed in the section - "Alternative __next__() and Generator Exception Handling" below. - - The added twist is that instead of adding a flag argument to - next() and __next__() to indicate whether the previous argument is - a value or an exception, we use a separate API (an __exit__() - method taking an exception and perhaps a traceback) for the - exception. If an iterator doesn't implement __exit__(), the - exception is just re-raised. It is expected that, apart from - generators, very few iterators will implement __exit__(); one use - case would be a fast implementation of synchronized() written in - C. - - The built-in next() function only interfaces to the next() and - __next__() methods; there is no user-friendly API to call - __exit__(). (Or perhaps calling next(itr, exc, traceback) would - call itr.__exit__(exc, traceback) if itr has an __exit__ method - and otherwise raise exc.__class__, exc, traceback?) - Motivation and Summary (Thanks to Shane Hathaway -- Hi Shane!) @@ -96,40 +72,25 @@ TBD. For now, see the Examples section near the end. -Specification: the Iteration Exception Hierarchy - - Two new built-in exceptions are defined, and StopIteration is - moved in the exception hierarchy: - - class Iteration(Exception): - pass - - class StopIteration(Iteration): - pass - - class ContinueIteration(Iteration): - def __init__(self, value=None): - self.value = None - Specification: the __next__() Method A new method for iterators is proposed, called __next__(). It - takes one optional argument, which defaults to None. If not None, - the argument must be an Iteration instance. Calling the + takes one optional argument, which defaults to None. Calling the __next__() method without argument or with None is equivalent to using the old iterator API, next(). For backwards compatibility, it is recommended that iterators also implement a next() method as an alias for calling the __next__() method without an argument. - Calling the __next__() method with a StopIteration instance - signals the iterator that the caller wants to abort the iteration - sequence; the iterator should respond by doing any necessary - cleanup and raising StopIteration. Calling it with a - ContinueIteration instance signals the iterator that the caller - wants to continue the iteration; the ContinueIteration exception - has a 'value' attribute which may be used by the iterator as a - hint on what to do next. Calling it with a (base class) Iteration - instance is the same as calling it with None. + The argument to the __next__() method may be used by the iterator + as a hint on what to do next. + +Specification: the __exit__() Method + + An optional new method for iterators is proposed, called + __exit__(). It takes up to three arguments which correspond to + the three "arguments" to the raise-statement: type, value, and + traceback. If all three arguments are None, sys.exc_info() may be + consulted to provide suitable default values. Specification: the next() Built-in Function @@ -143,34 +104,43 @@ return itr.next() raise TypeError("next() with arg for old-style iterator") -Specification: the 'for' Loop + This function is proposed because there is often a need to call + the next() method outside a for-loop; the new API, and the + backwards compatibility code, is too ugly to have to repeat in + user code. + + Note that I'm not proposing a built-in function to call the + __exit__() method of an iterator. I don't expect that this will + be called much outside the block-statement. + +Specification: a Change to the 'for' Loop A small change in the translation of the for-loop is proposed. The statement for VAR1 in EXPR1: BLOCK1 + else: + BLOCK2 will be translated as follows: itr = iter(EXPR1) arg = None + brk = False while True: try: VAR1 = next(itr, arg) except StopIteration: + brk = True break arg = None BLOCK1 + if brk: + BLOCK2 - (However, 'itr' and 'arg' are hidden from the user, their scope - ends when the while-loop is exited, and they are not shared with - nested or outer for-loops, and the user cannot override the - built-ins referenced.) - - I'm leaving the translation of an else-clause up to the reader; - note that you can't simply affix the else-clause to the while-loop - since it is always broken out. + (However, the variables 'itr' etc. are not user-visible and the + built-in names used cannot be overridden by the user.) Specification: the Extended 'continue' Statement @@ -180,7 +150,7 @@ is legal and is translated into - arg = ContinueIteration(EXPR2) + arg = EXPR2 continue (Where 'arg' references the corresponding hidden variable from the @@ -195,21 +165,28 @@ block EXPR1 as VAR1: BLOCK1 + else: + BLOCK2 Here, 'block' and 'as' are new keywords; EXPR1 is an arbitrary expression (but not an expression-list) and VAR1 is an arbitrary assignment target (which may be a comma-separated list). - The "as VAR1" part is optional; if omitted, the assignment to VAR1 - in the translation below is omitted (but the next() call is not!). + The "as VAR1" part is optional; if omitted, the assignments to + VAR1 in the translation below are omitted (but the expressions + assigned are still evaluated!). - The choice of the 'block' keyword is contentious; it has even been - proposed not to use a keyword at all. PEP 310 uses 'with' for - similar semantics, but I would like to reserve that for a - with-statement similar to the one found in Pascal and VB. To - sidestep this issue momentarily I'm using 'block' until we can - agree on a keyword. (I just found that the C# designers don't - like 'with' [2].) + The choice of the 'block' keyword is contentious; many + alternatives have been proposed, including not to use a keyword at + all (which I actually like). PEP 310 uses 'with' for similar + semantics, but I would like to reserve that for a with-statement + similar to the one found in Pascal and VB. (Though I just found + that the C# designers don't like 'with' [2], and I have to agree + with their reasoning.) To sidestep this issue momentarily I'm + using 'block' until we can agree on the right keyword, if any. + + Note that the 'as' keyword is not contentious (it will finally be + elevated to proper keyword status). Note that it is left in the middle whether a block-statement represents a loop or not; this is up to the iterator, but in the @@ -218,61 +195,66 @@ The translation is subtly different from the translation of a for-loop: iter() is not called, so EXPR1 should already be an iterator (not just an iterable); and the iterator is guaranteed to - be exhausted when the block-statement is left: + be notified when the block-statement is left, regardless if this + is due to a break, return or exception: - itr = EXPR1 - val = arg = None - ret = False + itr = EXPR1 # The iterator + ret = False # True if a return statement is active + val = None # Return value, if ret == True + arg = None # Argument to __next__() (value from continue) + exc = None # sys.exc_info() tuple if an exception is active while True: try: - VAR1 = next(itr, arg) + if exc: + ext = getattr(itr, "__exit__", None) + if ext is not None: + VAR1 = ext(*exc) # May re-raise *exc + else: + raise *exc # Well, the moral equivalent :-) + else: + VAR1 = next(itr, arg) # May raise StopIteration except StopIteration: if ret: return val - if val is not None: - raise val break try: - val = arg = None ret = False + val = arg = exc = None BLOCK1 - except Exception, val: - arg = StopIteration() - - (Again, 'itr' etc. are hidden, and the user cannot override the - built-ins.) + except: + exc = sys.exc_info() - The "raise val" translation is inexact; this is supposed to - re-raise the exact exception that was raised inside BLOCK1, with - the same traceback. We can't use a bare raise-statement because - we've just caught StopIteration. + (Again, the variables and built-ins are hidden from the user.) Inside BLOCK1, the following special translations apply: - "continue" and "continue EXPR2" are always legal; the latter is translated as shown earlier: - arg = ContinueIteration(EXPR2) + arg = EXPR2 continue - "break" is always legal; it is translated into: - arg = StopIteration() + exc = (StopIteration,) continue - "return EXPR3" is only legal when the block-statement is contained in a function definition; it is translated into: - val = EXPR3 + exc = (StopIteration,) ret = True - arg = StopIteration() + val = EXPR3 continue The net effect is that break, continue and return behave much the same as if the block-statement were a for-loop, except that the iterator gets a chance at resource cleanup before the - block-statement is left. The iterator also gets a chance if the - block-statement is left through raising an exception. + block-statement is left, through the optional __exit__() method. + The iterator also gets a chance if the block-statement is left + through raising an exception. If the iterator doesn't have an + __exit__() method, there is no difference with a for-loop (except + that a for-loop calls iter() on EXPR1). Note that a yield-statement (or a yield-expression, see below) in a block-statement is not treated differently. It suspends the @@ -287,15 +269,18 @@ be resumed eventually. I haven't decided yet whether the block-statement should also - allow an optional else-clause, like the for-loop. I think it - would be confusing, and emphasize the "loopiness" of the - block-statement, while I want to emphasize its *difference* from a - for-loop. + allow an optional else-clause, like the for-loop, but I'm leaning + against it. I think it would be confusing, and emphasize the + "loopiness" of the block-statement, while I want to emphasize its + *difference* from a for-loop. In addition, there are several + possible semantics for an else-clause. Specification: Generator Exception Handling Generators will implement the new __next__() method API, as well - as the old argument-less next() method. + as the old argument-less next() method which becomes an alias for + calling __next__() without an argument. They will also implement + the new __exit__() method API. Generators will be allowed to have a yield statement inside a try-finally statement. @@ -330,32 +315,41 @@ are all illegal. (Some of the edge cases are motivated by the current legality of "yield 12, 42".) - When __next__() is called with a StopIteration instance argument, - the yield statement that is resumed by the __next__() call will - raise this StopIteration exception. The generator should re-raise - this exception; it should not yield another value. When the - *initial* call to __next__() receives a StopIteration instance - argument, the generator's execution is aborted and the exception - is re-raised without passing control to the generator's body. + When __exit__() is called, the generator is resumed but at the + point of the yield-statement or -expression the exception + represented by the __exit__ argument(s) is raised. The generator + may re-raise this exception, raise another exception, or yield + another value, execpt that if the exception passed in to + __exit__() was StopIteration, it ought to raise StopIteration + (otherwise the effect would be that a break is turned into + continue, which is unexpected at least). When the *initial* call + resuming the generator is an __exit__() call instead of a + __next__() call, the generator's execution is aborted and the + exception is re-raised without passing control to the generator's + body. - When __next__() is called with a ContinueIteration instance - argument, the yield-expression that it resumes will return the - value attribute of the argument. If it resumes a yield-statement, - the value is ignored. When the *initial* call to __next__() - receives a ContinueIteration instance argument, the generator's - execution is started normally; the argument's value attribute is - ignored. + When __next__() is called with an argument that is not None, the + yield-expression that it resumes will return the value attribute + of the argument. If it resumes a yield-statement, the value is + ignored (or should this be considered an error?). When the + *initial* call to __next__() receives an argument that is not + None, the generator's execution is started normally; the + argument's value attribute is ignored (or should this be + considered an error?). When __next__() is called without an + argument or with None as argument, and a yield-expression is + resumed, the yield-expression returns None. When a generator that has not yet terminated is garbage-collected (either through reference counting or by the cyclical garbage - collector), its __next__() method is called once with a - StopIteration instance argument. Together with the requirement - that __next__() should always re-raise a StopIteration argument, - this guarantees the eventual activation of any finally-clauses - that were active when the generator was last suspended. Of - course, under certain circumstances the generator may never be - garbage-collected. This is no different than the guarantees that - are made about finalizers (__del__() methods) of other objects. + collector), its __exit__() method is called once with + StopIteration as its first argument. Together with the + requirement that a generator ought to raise StopIteration when + __exit__() is called with StopIteration, this guarantees the + eventual activation of any finally-clauses that were active when + the generator was last suspended. Of course, under certain + circumstances the generator may never be garbage-collected. This + is no different than the guarantees that are made about finalizers + (__del__() methods) of other objects. Note: the syntactic extensions to yield make its use very similar to that in Ruby. This is intentional. Do note that in Python the @@ -367,71 +361,6 @@ cases work differently; in Python, you cannot save the block for later use, and you cannot test whether there is a block or not. -Specification: Alternative __next__() and Generator Exception Handling - - The above specification doesn't let the generator handle general - exceptions. If we want that, we could modify the __next__() API - to take either a value or an exception argument, with an - additional flag argument to distinguish between the two. When the - second argument is True, the first must be an Exception instance, - which raised at the point of the resuming yield; otherwise the - first argument is the value that is returned from the - yield-expression (or ignored by a yield-statement). Wrapping a - regular value in a ContinueIteration is then no longer necessary. - - The next() built-in would be modified likewise: - - def next(itr, arg=None, exc=False): - nxt = getattr(itr, "__next__", None) - if nxt is not None: - return nxt(arg, exc) - if arg is None and not exc: - return itr.next() - raise TypeError("next() with args for old-style iterator") - - The translation of a block-statement would become: - - itr = EXPR1 - arg = val = None - ret = exc = False - while True: - try: - VAR1 = next(itr, arg, exc) - except StopIteration: - if ret: - return val - break - try: - arg = val = None - ret = exc = False - BLOCK1 - except Exception, arg: - exc = True - - The translation of "continue EXPR2" would become: - - arg = EXPR2 - continue - - The translation of "break" inside a block-statement would become: - - arg = StopIteration() - exc = True - continue - - The translation of "return EXPR3" inside a block-statement would - become: - - val = EXPR3 - arg = StopIteration() - ret = exc = True - continue - - The translation of a for-loop would be the same as indicated - earlier (inside a for-loop only the translation of "continue - EXPR2" is changed; break and return translate to themselves in - that case). - Loose Ends These are things that need to be resolved before accepting the @@ -447,17 +376,8 @@ - Decide on the keyword ('block', 'with', '@', nothing, or something else?). - - Phillip Eby wants a way to pass tracebacks along with - exceptions. - - - The translation for the for-loop's else-clause. - - Whether a block-statement should allow an else-clause. - - Which API to use to pass in an exception: itr.__next__(exc), - itr.__next__(exc, True) or itr.__exit__(exc[, traceback]). - Hmm..., perhaps itr.__next__(exc, traceback)? - Comparison to Thunks Alternative semantics proposed for the block-statement turn the @@ -626,7 +546,7 @@ block auto_retry(3, IOError): f = urllib.urlopen("http://python.org/peps/pep-0340.html") - print f.read() + print f.read() 5. It is possible to nest blocks and combine templates: From gvanrossum at users.sourceforge.net Mon May 2 17:33:50 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Mon, 02 May 2005 08:33:50 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.13,1.14 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24412 Modified Files: pep-0340.txt Log Message: Delete the last reference to ContinuationIteration. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- pep-0340.txt 2 May 2005 03:30:07 -0000 1.13 +++ pep-0340.txt 2 May 2005 15:33:47 -0000 1.14 @@ -291,8 +291,7 @@ The yield-statement will be allowed to be used on the right-hand side of an assignment; in that case it is referred to as yield-expression. The value of this yield-expression is None - unless __next__() was called with a ContinueIteration argument; - see below. + unless __next__() was called with an argument; see below. A yield-expression must always be parenthesized except when it occurs at the top-level expression on the right-hand side of an From gvanrossum at users.sourceforge.net Tue May 3 01:35:55 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Mon, 02 May 2005 16:35:55 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.14,1.15 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16732 Modified Files: pep-0340.txt Log Message: Fix typo in example 4 found by Chris Ryland. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- pep-0340.txt 2 May 2005 15:33:47 -0000 1.14 +++ pep-0340.txt 2 May 2005 23:35:52 -0000 1.15 @@ -536,7 +536,7 @@ try: yield return - except Exception, err: + except exc, err: # perhaps log exception here continue raise # re-raise the exception we caught earlier @@ -565,13 +565,13 @@ Acknowledgements In no useful order: Alex Martelli, Barry Warsaw, Bob Ippolito, - Brett Cannon, Brian Sabbey, Doug Landauer, Duncan Booth, Fredrik - Lundh, Greg Ewing, Holger Krekel, Jason Diamond, Jim Jewett, - Josiah Carlson, Ka-Ping Yee, Michael Chermside, Michael Hudson, - Neil Schemenauer, Nick Coghlan, Paul Moore, Phillip Eby, Raymond - Hettinger, Samuele Pedroni, Shannon Behrens, Skip Montanaro, - Steven Bethard, Terry Reedy, Tim Delaney, Aahz, and others. - Thanks all for the valuable contributions! + Brett Cannon, Brian Sabbey, Chris Ryland, Doug Landauer, Duncan + Booth, Fredrik Lundh, Greg Ewing, Holger Krekel, Jason Diamond, + Jim Jewett, Josiah Carlson, Ka-Ping Yee, Michael Chermside, + Michael Hudson, Neil Schemenauer, Nick Coghlan, Paul Moore, + Phillip Eby, Raymond Hettinger, Samuele Pedroni, Shannon Behrens, + Skip Montanaro, Steven Bethard, Terry Reedy, Tim Delaney, Aahz, + and others. Thanks all for the valuable contributions! References From gvanrossum at users.sourceforge.net Tue May 3 18:32:27 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 03 May 2005 09:32:27 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.15,1.16 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16785 Modified Files: pep-0340.txt Log Message: Solidify loose ends. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- pep-0340.txt 2 May 2005 23:35:52 -0000 1.15 +++ pep-0340.txt 3 May 2005 16:32:24 -0000 1.16 @@ -22,10 +22,6 @@ (Reliable Acquisition/Release Pairs), and PEP 325 (Resource-Release Support for Generators). - This proposal is just a strawman; we've had a heated debate about - this on python-dev recently [1], and I figured it would be time to - write up a precise spec in PEP form. - Motivation and Summary (Thanks to Shane Hathaway -- Hi Shane!) @@ -70,7 +66,7 @@ Use Cases - TBD. For now, see the Examples section near the end. + See the Examples section near the end. Specification: the __next__() Method @@ -159,14 +155,15 @@ This is also the case in the body of the block-statement proposed below. + EXPR2 may contain commas; "continue 1, 2, 3" is equivalent to + "continue (1, 2, 3)". + Specification: the Anonymous Block Statement A new statement is proposed with the syntax block EXPR1 as VAR1: BLOCK1 - else: - BLOCK2 Here, 'block' and 'as' are new keywords; EXPR1 is an arbitrary expression (but not an expression-list) and VAR1 is an arbitrary @@ -268,14 +265,14 @@ the limitations of all finalization semantics) that the block will be resumed eventually. - I haven't decided yet whether the block-statement should also - allow an optional else-clause, like the for-loop, but I'm leaning - against it. I think it would be confusing, and emphasize the + Unlike the for-loop, the block-statement does not have an + else-clause. I think it would be confusing, and emphasize the "loopiness" of the block-statement, while I want to emphasize its *difference* from a for-loop. In addition, there are several - possible semantics for an else-clause. + possible semantics for an else-clause, and only a very weak use + case. -Specification: Generator Exception Handling +Specification: Generator Exit Handling Generators will implement the new __next__() method API, as well as the old argument-less next() method which becomes an alias for @@ -330,11 +327,10 @@ When __next__() is called with an argument that is not None, the yield-expression that it resumes will return the value attribute of the argument. If it resumes a yield-statement, the value is - ignored (or should this be considered an error?). When the - *initial* call to __next__() receives an argument that is not - None, the generator's execution is started normally; the - argument's value attribute is ignored (or should this be - considered an error?). When __next__() is called without an + ignored (this is similar to ignoring the value returned by a + function call). When the *initial* call to __next__() receives an + argument that is not None, TypeError is raised; this is likely + caused by some logic error. When __next__() is called without an argument or with None as argument, and a yield-expression is resumed, the yield-expression returns None. @@ -360,22 +356,20 @@ cases work differently; in Python, you cannot save the block for later use, and you cannot test whether there is a block or not. -Loose Ends - - These are things that need to be resolved before accepting the - PEP. +Alternatives Considered - - Fill in the remaining TBD sections. + - Many alternatives have been proposed for 'block', including '@' + and no keyword at all. I haven't seen a proposal for another + keyword that I like better than 'block' yet, and not using a + keyword at all makes many folks (including me) uncomfortable. - - Address Phillip Eby's proposal to have the block-statement use + - Phillip Eby has proposed to have the block-statement use an entirely different API than the for-loop, to differentiate - between the two (a generator would have to be wrapped in a - decorator to make it support the block API). - - - Decide on the keyword ('block', 'with', '@', nothing, or - something else?). - - - Whether a block-statement should allow an else-clause. + between the two. A generator would have to be wrapped in a + decorator to make it support the block API. IMO this adds more + complexity with very little benefit; and we can't relly deny + that the block-statement is conceptually a loop -- it supports + break and continue, after all. Comparison to Thunks @@ -418,7 +412,7 @@ and I'd be bummed if I couldn't write this as: def findSomething(self, key, default=None): - block synchronized(self.lock): + block locking(self.lock): for item in self.elements: if item.matches(key): return item @@ -427,7 +421,7 @@ This particular example can be rewritten using a break: def findSomething(self, key, default=None): - block synchronized(self.lock): + block locking(self.lock): for item in self.elements: if item.matches(key): break @@ -478,16 +472,17 @@ However, the use cases for multiple blocks seem elusive. -Alternatives Considered - - TBD. + (Proposals have since been made to change the implementation of + thunks to remove most of these objections, but the resulting + semantics are fairly complex to explain and to implement, so IMO + that defeats the purpose of using thunks in the first place.) Examples 1. A template for ensuring that a lock, acquired at the start of a block, is released when the block is left: - def synchronized(lock): + def locking(lock): lock.acquire() try: yield @@ -496,7 +491,7 @@ Used as follows: - block synchronized(myLock): + block locking(myLock): # Code here executes with myLock held. The lock is # guaranteed to be released when the block is left (even # if by an uncaught exception). @@ -549,18 +544,40 @@ 5. It is possible to nest blocks and combine templates: - def synchronized_opening(lock, filename, mode="r"): - block synchronized(lock): + def locking_opening(lock, filename, mode="r"): + block locking(lock): block opening(filename) as f: yield f Used as follows: - block synchronized_opening("/etc/passwd", myLock) as f: + block locking_opening("/etc/passwd", myLock) as f: for line in f: print line.rstrip() - 6. Coroutine example TBD. + 6. It is possible to write a regular iterator with the + semantics of example 1: + + class locking: + def __init__(self, lock): + self.lock = lock + self.state = 0 + def __next__(self, arg=None): + # ignores arg + if self.state: + assert self.state == 1 + self.lock.release() + self.state += 1 + raise StopIteration + else: + self.lock.acquire() + self.state += 1 + return None + def __exit__(self, type, value=None, traceback=None): + assert self.state in (0, 1, 2) + if self.state == 1: + self.lock.release() + raise type, value, traceback Acknowledgements From gvanrossum at users.sourceforge.net Tue May 3 18:57:20 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 03 May 2005 09:57:20 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.16,1.17 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24156 Modified Files: pep-0340.txt Log Message: Add another example (saving sys.stout). Eradicate tabs. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- pep-0340.txt 3 May 2005 16:32:24 -0000 1.16 +++ pep-0340.txt 3 May 2005 16:57:17 -0000 1.17 @@ -558,26 +558,42 @@ 6. It is possible to write a regular iterator with the semantics of example 1: - class locking: - def __init__(self, lock): - self.lock = lock - self.state = 0 - def __next__(self, arg=None): - # ignores arg - if self.state: - assert self.state == 1 - self.lock.release() - self.state += 1 - raise StopIteration - else: - self.lock.acquire() - self.state += 1 - return None - def __exit__(self, type, value=None, traceback=None): - assert self.state in (0, 1, 2) - if self.state == 1: - self.lock.release() - raise type, value, traceback + class locking: + def __init__(self, lock): + self.lock = lock + self.state = 0 + def __next__(self, arg=None): + # ignores arg + if self.state: + assert self.state == 1 + self.lock.release() + self.state += 1 + raise StopIteration + else: + self.lock.acquire() + self.state += 1 + return None + def __exit__(self, type, value=None, traceback=None): + assert self.state in (0, 1, 2) + if self.state == 1: + self.lock.release() + raise type, value, traceback + + 7. Redirect stdout temporarily: + + def saving_stdout(new_stdout): + save_stdout = sys.stdout + try: + sys.stdout = new_stdout + yield + finally: + sys.stdout = save_stdout + + Used as follows: + + block opening(filename, "w") as f: + block saving_stdout(f): + print "Hello world" Acknowledgements From gvanrossum at users.sourceforge.net Tue May 3 19:21:21 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 03 May 2005 10:21:21 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.17,1.18 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29297 Modified Files: pep-0340.txt Log Message: saving_stdout -> redirecting_stdout (Phillip Eby). Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- pep-0340.txt 3 May 2005 16:57:17 -0000 1.17 +++ pep-0340.txt 3 May 2005 17:21:13 -0000 1.18 @@ -581,7 +581,7 @@ 7. Redirect stdout temporarily: - def saving_stdout(new_stdout): + def redirecting_stdout(new_stdout): save_stdout = sys.stdout try: sys.stdout = new_stdout @@ -592,7 +592,7 @@ Used as follows: block opening(filename, "w") as f: - block saving_stdout(f): + block redirecting_stdout(f): print "Hello world" Acknowledgements From gvanrossum at users.sourceforge.net Tue May 3 20:31:37 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 03 May 2005 11:31:37 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.18,1.19 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13580 Modified Files: pep-0340.txt Log Message: Clarify that the block-statement is a loop to the parser, and don't use literally translated Dutch. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.18 retrieving revision 1.19 diff -u -d -r1.18 -r1.19 --- pep-0340.txt 3 May 2005 17:21:13 -0000 1.18 +++ pep-0340.txt 3 May 2005 18:31:34 -0000 1.19 @@ -185,9 +185,11 @@ Note that the 'as' keyword is not contentious (it will finally be elevated to proper keyword status). - Note that it is left in the middle whether a block-statement - represents a loop or not; this is up to the iterator, but in the - most common case BLOCK1 is executed exactly once. + Note that it is up to the iterator to decide whether a + block-statement represents a loop with multiple iterations; in the + most common use case BLOCK1 is executed exactly once. To the + parser, however, it is always a loop; break and continue return + transfer to the block's iterator (see below for details). The translation is subtly different from the translation of a for-loop: iter() is not called, so EXPR1 should already be an From gvanrossum at users.sourceforge.net Wed May 4 00:23:34 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 03 May 2005 15:23:34 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.19,1.20 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8496 Modified Files: pep-0340.txt Log Message: Some clarifications after Raymond's email. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.19 retrieving revision 1.20 diff -u -d -r1.19 -r1.20 --- pep-0340.txt 3 May 2005 18:31:34 -0000 1.19 +++ pep-0340.txt 3 May 2005 22:23:32 -0000 1.20 @@ -122,7 +122,7 @@ will be translated as follows: itr = iter(EXPR1) - arg = None + arg = None # Set by "continue EXPR2", see below brk = False while True: try: @@ -496,7 +496,7 @@ block locking(myLock): # Code here executes with myLock held. The lock is # guaranteed to be released when the block is left (even - # if by an uncaught exception). + # if via return or by an uncaught exception). 2. A template for opening a file that ensures the file is closed when the block is left: @@ -557,6 +557,11 @@ for line in f: print line.rstrip() + (If this example confuses you, consider that it is equivalent + to using a for-loop with a yield in its body in a regular + generator which is invoking another iterator or generator + recursively; see for example the source code for os.walk().) + 6. It is possible to write a regular iterator with the semantics of example 1: @@ -581,6 +586,10 @@ self.lock.release() raise type, value, traceback + (This example is easily modified to implement the other + examples; it shows how much simpler generators are for the same + purpose.) + 7. Redirect stdout temporarily: def redirecting_stdout(new_stdout): @@ -597,6 +606,30 @@ block redirecting_stdout(f): print "Hello world" + 8. A variant on opening() that also returns an error condition: + + def opening_w_error(filename, mode="r"): + try: + f = open(filename, mode) + except IOError, err: + yield None, err + else: + try: + yield f, None + finally: + f.close() + + Used as follows: + + block opening_w_error("/etc/passwd", "a") as f, err: + if err: + print "IOError:", err + else: + f.write("guido::0:0::/:/bin/sh\n") + + 9. More examples are needed: showing "continue EXPR", and the use + of continue, break and return in a block-statement. + Acknowledgements In no useful order: Alex Martelli, Barry Warsaw, Bob Ippolito, From gvanrossum at users.sourceforge.net Wed May 4 01:51:56 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 03 May 2005 16:51:56 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.20,1.21 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31381 Modified Files: pep-0340.txt Log Message: Fix a remnant of the old proposal found by Tim Delaney. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.20 retrieving revision 1.21 diff -u -d -r1.20 -r1.21 --- pep-0340.txt 3 May 2005 22:23:32 -0000 1.20 +++ pep-0340.txt 3 May 2005 23:51:54 -0000 1.21 @@ -327,14 +327,14 @@ body. When __next__() is called with an argument that is not None, the - yield-expression that it resumes will return the value attribute - of the argument. If it resumes a yield-statement, the value is - ignored (this is similar to ignoring the value returned by a - function call). When the *initial* call to __next__() receives an - argument that is not None, TypeError is raised; this is likely - caused by some logic error. When __next__() is called without an - argument or with None as argument, and a yield-expression is - resumed, the yield-expression returns None. + yield-expression that it resumes will return the argument. If it + resumes a yield-statement, the value is ignored (this is similar + to ignoring the value returned by a function call). When the + *initial* call to __next__() receives an argument that is not + None, TypeError is raised; this is likely caused by some logic + error. When __next__() is called without an argument or with None + as argument, and a yield-expression is resumed, the + yield-expression returns None. When a generator that has not yet terminated is garbage-collected (either through reference counting or by the cyclical garbage From gvanrossum at users.sourceforge.net Wed May 4 03:03:20 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 03 May 2005 18:03:20 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.21,1.22 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15670 Modified Files: pep-0340.txt Log Message: Clarify that there are some separable items. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- pep-0340.txt 3 May 2005 23:51:54 -0000 1.21 +++ pep-0340.txt 4 May 2005 01:03:17 -0000 1.22 @@ -22,6 +22,21 @@ (Reliable Acquisition/Release Pairs), and PEP 325 (Resource-Release Support for Generators). + I should clarify that there are a few separable proposals in this + PEP. + + - Using "continue EXPR" which calls its.__next__(EXPR) which + becomes the return value of a yield-expression is entirely + orthogonal with the rest of the PEP. + + - Similarly, using a generator to "drive" a block statement is + also separable; with just the definition of the block statement + from the PEP you could implement all the examples using a class + (similar to example 6, which is easily turned into a template). + + But the key idea is using a generator to drive a block statement; + the rest is elaboration. + Motivation and Summary (Thanks to Shane Hathaway -- Hi Shane!) From mwh at users.sourceforge.net Wed May 4 13:59:40 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Wed, 04 May 2005 04:59:40 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_class.py, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2596/Lib/test Modified Files: test_class.py Log Message: Don't use 'is not' to compare strings. (spotted by reading pypy-svn :) Index: test_class.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_class.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- test_class.py 19 Jul 2004 16:29:15 -0000 1.12 +++ test_class.py 4 May 2005 11:59:38 -0000 1.13 @@ -353,7 +353,7 @@ try: A().a # Raised AttributeError: A instance has no attribute 'a' except AttributeError, x: - if str(x) is not "booh": + if str(x) != "booh": print "attribute error for A().a got masked:", str(x) class E: From gvanrossum at users.sourceforge.net Wed May 4 23:00:49 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 04 May 2005 14:00:49 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.22,1.23 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17237 Modified Files: pep-0340.txt Log Message: Fix bug in example 5 found by Reinhold Birkenfeld. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.22 retrieving revision 1.23 diff -u -d -r1.22 -r1.23 --- pep-0340.txt 4 May 2005 01:03:17 -0000 1.22 +++ pep-0340.txt 4 May 2005 21:00:46 -0000 1.23 @@ -568,7 +568,7 @@ Used as follows: - block locking_opening("/etc/passwd", myLock) as f: + block locking_opening(myLock, "/etc/passwd") as f: for line in f: print line.rstrip() @@ -652,9 +652,10 @@ Booth, Fredrik Lundh, Greg Ewing, Holger Krekel, Jason Diamond, Jim Jewett, Josiah Carlson, Ka-Ping Yee, Michael Chermside, Michael Hudson, Neil Schemenauer, Nick Coghlan, Paul Moore, - Phillip Eby, Raymond Hettinger, Samuele Pedroni, Shannon Behrens, - Skip Montanaro, Steven Bethard, Terry Reedy, Tim Delaney, Aahz, - and others. Thanks all for the valuable contributions! + Phillip Eby, Raymond Hettinger, Reinhold Birkenfeld, Samuele + Pedroni, Shannon Behrens, Skip Montanaro, Steven Bethard, Terry + Reedy, Tim Delaney, Aahz, and others. Thanks all for the valuable + contributions! References From gvanrossum at users.sourceforge.net Thu May 5 17:39:21 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Thu, 05 May 2005 08:39:21 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.23,1.24 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25281 Modified Files: pep-0340.txt Log Message: Mention some rejected ideas (and explain why). Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- pep-0340.txt 4 May 2005 21:00:46 -0000 1.23 +++ pep-0340.txt 5 May 2005 15:39:19 -0000 1.24 @@ -373,12 +373,28 @@ cases work differently; in Python, you cannot save the block for later use, and you cannot test whether there is a block or not. -Alternatives Considered +Alternatives Considered and Rejected - - Many alternatives have been proposed for 'block', including '@' - and no keyword at all. I haven't seen a proposal for another - keyword that I like better than 'block' yet, and not using a - keyword at all makes many folks (including me) uncomfortable. + - Many alternatives have been proposed for 'block'. I haven't + seen a proposal for another keyword that I like better than + 'block' yet. Alas, 'block' is also not a good choice; it is a + rather popular name for variables, arguments and methods. + Perhaps 'with' is the best choice after all? + + - Instead of trying to pick the ideal keyword, the block-statement + could simply have the form: + + EXPR1 as VAR1: + BLOCK1 + + This is at first attractive because, together with a good choice + of function names (like those in the Examples section below) + used in EXPR1, it reads well, and feels like a "user-defined + statement". And yet, it makes me (and many others) + uncomfortable; without a keyword the syntax is very "bland", + difficult to look up in a manual (remember that 'as' is + optional), and it makes the meaning of break and continue in the + block-statement even more confusing. - Phillip Eby has proposed to have the block-statement use an entirely different API than the for-loop, to differentiate @@ -388,6 +404,20 @@ that the block-statement is conceptually a loop -- it supports break and continue, after all. + - This keeps getting proposed: "block VAR1 = EXPR1" instead of + "block EXPR1 as VAR1". That would be very misleading, since + VAR1 does *not* get assigned the value of EXPR1; EXPR1 results + in a generator which is assigned to an internal variable, and + VAR1 is the value returned by successive calls to the __next__() + method of that iterator. + + - Why not change the translation to apply iter(EXPR1)? All the + examples would continue to work. But this makes the + block-statement *more* like a for-loop, while the emphasis ought + to be on the *difference* between the two. Not calling iter() + catches a bunch of misunderstandings, like using a sequence as + EXPR1. + Comparison to Thunks Alternative semantics proposed for the block-statement turn the From kbk at users.sourceforge.net Fri May 6 01:29:56 2005 From: kbk at users.sourceforge.net (kbk@users.sourceforge.net) Date: Thu, 05 May 2005 16:29:56 -0700 Subject: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.56, 1.57 run.py, 1.31, 1.32 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/idlelib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18826 Modified Files: NEWS.txt run.py Log Message: Use Queue's blocking feature instead of sleeping in the main loop. Patch # 1190163 Michiel de Hoon Index: NEWS.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/NEWS.txt,v retrieving revision 1.56 retrieving revision 1.57 diff -u -d -r1.56 -r1.57 --- NEWS.txt 3 Feb 2005 01:37:13 -0000 1.56 +++ NEWS.txt 5 May 2005 23:29:54 -0000 1.57 @@ -3,6 +3,9 @@ *Release date: XX-XXX-2005* +- run.py: use Queue's blocking feature instead of sleeping in the main + loop. Patch # 1190163 Michiel de Hoon + - Add config-main option to make the 'history' feature non-cyclic. Default remains cyclic. Python Patch 914546 Noam Raphael. Index: run.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/run.py,v retrieving revision 1.31 retrieving revision 1.32 diff -u -d -r1.31 -r1.32 --- run.py 21 Dec 2004 22:10:32 -0000 1.31 +++ run.py 5 May 2005 23:29:54 -0000 1.32 @@ -82,9 +82,8 @@ # exiting but got an extra KBI? Try again! continue try: - seq, request = rpc.request_queue.get(0) + seq, request = rpc.request_queue.get(block=True, timeout=0.05) except Queue.Empty: - time.sleep(0.05) continue method, args, kwargs = request ret = method(*args, **kwargs) From gvanrossum at users.sourceforge.net Fri May 6 16:38:11 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Fri, 06 May 2005 07:38:11 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt,1.24,1.25 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31239 Modified Files: pep-0340.txt Log Message: There's no need to be fuzzy about the moreal equivalent of "raise *exc". Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.24 retrieving revision 1.25 diff -u -d -r1.24 -r1.25 --- pep-0340.txt 5 May 2005 15:39:19 -0000 1.24 +++ pep-0340.txt 6 May 2005 14:38:09 -0000 1.25 @@ -224,7 +224,7 @@ if ext is not None: VAR1 = ext(*exc) # May re-raise *exc else: - raise *exc # Well, the moral equivalent :-) + raise exc[0], exc[1], exc[2] else: VAR1 = next(itr, arg) # May raise StopIteration except StopIteration: @@ -250,13 +250,13 @@ - "break" is always legal; it is translated into: - exc = (StopIteration,) + exc = (StopIteration, None, None) continue - "return EXPR3" is only legal when the block-statement is contained in a function definition; it is translated into: - exc = (StopIteration,) + exc = (StopIteration, None, None) ret = True val = EXPR3 continue From goodger at users.sourceforge.net Sat May 7 14:13:52 2005 From: goodger at users.sourceforge.net (goodger@users.sourceforge.net) Date: Sat, 07 May 2005 05:13:52 -0700 Subject: [Python-checkins] python/nondist/peps pep-0341.txt, NONE, 1.1 pep-0000.txt, 1.303, 1.304 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv660 Modified Files: pep-0000.txt Added Files: pep-0341.txt Log Message: added PEP 341, "Unifying try-except and try-finally", by Reinhold Birkenfeld --- NEW FILE: pep-0341.txt --- PEP: 341 Title: Unifying try-except and try-finally Version: $Revision: 1.1 $ Last-Modified: $Date: 2005/05/07 12:13:49 $ Author: Reinhold Birkenfeld Status: Draft Type: Standards Track Content-Type: text/plain Created: 04-May-2005 Post-History: Abstract This PEP proposes a change in the syntax and semantics of try statements to allow combined try-except-finally blocks. This means in short that it would be valid to write try: except Exception: finally: Rationale/Proposal There are many use cases for the try-except statement and for the try-finally statement per se; however, often one needs to catch exceptions and execute some cleanup code afterwards. It is slightly annoying and not very intelligible that one has to write f = None try: try: f = open(filename) text = f.read() except IOError: print 'An error occured' finally: if f: f.close() So it is proposed that a construction like this try: except Ex1: else: finally: be exactly the same as the legacy try: try: except Ex1: else: finally: This is backwards compatible, and every try statement that is legal today would continue to work. Changes to the grammar The grammar for the try statement, which is currently try_stmt: ('try' ':' suite (except_clause ':' suite)+ ['else' ':' suite] | 'try' ':' suite 'finally' ':' suite) would have to become try_stmt: ('try' ':' suite (except_clause ':' suite)+ ['else' ':' suite] ['finally' ':' suite] | 'try' ':' suite (except_clause ':' suite)* ['else' ':' suite] 'finally' ':' suite) Implementation As the PEP author currently does not have sufficient knowledge of the CPython implementation, he is unfortunately not able to deliver one. However, according to Guido, it should be a piece of cake to implement[1] -- at least for a core hacker. References [1] http://mail.python.org/pipermail/python-dev/2005-May/053319.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.303 retrieving revision 1.304 diff -u -d -r1.303 -r1.304 --- pep-0000.txt 27 Apr 2005 07:26:55 -0000 1.303 +++ pep-0000.txt 7 May 2005 12:13:49 -0000 1.304 @@ -118,6 +118,7 @@ S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan S 340 Anonymous Block Statements GvR + S 341 Unifying try-except and try-finally Birkenfeld S 754 IEEE 754 Floating Point Special Values Warnes Finished PEPs (done, implemented in CVS) @@ -376,6 +377,7 @@ S 338 Executing modules inside packages with '-m' Coghlan I 339 How to Change CPython's Bytecode Cannon S 340 Anonymous Block Statements GvR + S 341 Unifying try-except and try-finally Birkenfeld SR 666 Reject Foolish Indentation Creighton S 754 IEEE 754 Floating Point Special Values Warnes I 3000 Python 3.0 Plans Kuchling, Cannon @@ -405,6 +407,7 @@ Batista, Facundo facundo at taniquetil.com.ar Baxter, Anthony anthony at interlink.com.au Bellman, Thomas bellman+pep-divmod at lysator.liu.se + Birkenfeld, Reinhold reinhold-birkenfeld-nospam at wolke7.net Cannon, Brett brett at python.org Carlson, Josiah jcarlson at uci.edu Carroll, W Isaac icarroll at pobox.com From FZJTPRX at mail.natm.ru Sun May 8 13:37:21 2005 From: FZJTPRX at mail.natm.ru (Carmine Lutz) Date: Sun, 08 May 2005 14:37:21 +0300 Subject: [Python-checkins] don't suffer with the old fashioned way anymore - let us design software to make your life easier Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20050508/605f8d0a/attachment.htm From rhettinger at users.sourceforge.net Tue May 10 05:20:26 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Mon, 09 May 2005 20:20:26 -0700 Subject: [Python-checkins] python/dist/src/Lib calendar.py,1.35,1.36 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11745 Modified Files: calendar.py Log Message: SF bug #1193890: calendar.weekheader not found in __all__ Index: calendar.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/calendar.py,v retrieving revision 1.35 retrieving revision 1.36 diff -u -d -r1.35 -r1.36 --- calendar.py 13 Nov 2004 16:18:31 -0000 1.35 +++ calendar.py 10 May 2005 03:20:12 -0000 1.36 @@ -10,7 +10,8 @@ __all__ = ["error","setfirstweekday","firstweekday","isleap", "leapdays","weekday","monthrange","monthcalendar", "prmonth","month","prcal","calendar","timegm", - "month_name", "month_abbr", "day_name", "day_abbr"] + "month_name", "month_abbr", "day_name", "day_abbr", + "weekheader"] # Exception raised for bad input (with string parameter for details) error = ValueError From rhettinger at users.sourceforge.net Tue May 10 05:22:24 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Mon, 09 May 2005 20:22:24 -0700 Subject: [Python-checkins] python/dist/src/Lib calendar.py,1.35,1.35.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12302 Modified Files: Tag: release24-maint calendar.py Log Message: SF bug #1193890: calendar.weekheader not found in __all__ Index: calendar.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/calendar.py,v retrieving revision 1.35 retrieving revision 1.35.2.1 diff -u -d -r1.35 -r1.35.2.1 --- calendar.py 13 Nov 2004 16:18:31 -0000 1.35 +++ calendar.py 10 May 2005 03:22:21 -0000 1.35.2.1 @@ -10,7 +10,8 @@ __all__ = ["error","setfirstweekday","firstweekday","isleap", "leapdays","weekday","monthrange","monthcalendar", "prmonth","month","prcal","calendar","timegm", - "month_name", "month_abbr", "day_name", "day_abbr"] + "month_name", "month_abbr", "day_name", "day_abbr", + "weekheader"] # Exception raised for bad input (with string parameter for details) error = ValueError From kbk at users.sourceforge.net Tue May 10 05:44:26 2005 From: kbk at users.sourceforge.net (kbk@users.sourceforge.net) Date: Mon, 09 May 2005 20:44:26 -0700 Subject: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.57, 1.58 PyShell.py, 1.95, 1.96 rpc.py, 1.31, 1.32 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/idlelib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16006 Modified Files: NEWS.txt PyShell.py rpc.py Log Message: Improve subprocess link error notification M NEWS.txt M PyShell.py M rpc.py Index: NEWS.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/NEWS.txt,v retrieving revision 1.57 retrieving revision 1.58 diff -u -d -r1.57 -r1.58 --- NEWS.txt 5 May 2005 23:29:54 -0000 1.57 +++ NEWS.txt 10 May 2005 03:44:24 -0000 1.58 @@ -3,6 +3,8 @@ *Release date: XX-XXX-2005* +- Improve subprocess link error notification. + - run.py: use Queue's blocking feature instead of sleeping in the main loop. Patch # 1190163 Michiel de Hoon Index: PyShell.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/PyShell.py,v retrieving revision 1.95 retrieving revision 1.96 diff -u -d -r1.95 -r1.96 --- PyShell.py 19 Jan 2005 00:22:58 -0000 1.95 +++ PyShell.py 10 May 2005 03:44:24 -0000 1.96 @@ -596,6 +596,8 @@ self.write("Unsupported characters in input") return try: + # InteractiveInterpreter.runsource() calls its runcode() method, + # which is overridden (see below) return InteractiveInterpreter.runsource(self, source, filename) finally: if self.save_warnings_filters is not None: @@ -720,6 +722,7 @@ else: self.showtraceback() except: + print>>sys.stderr, "IDLE internal error in runcode()" self.showtraceback() finally: if not use_subprocess: Index: rpc.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/rpc.py,v retrieving revision 1.31 retrieving revision 1.32 diff -u -d -r1.31 -r1.32 --- rpc.py 23 Dec 2004 04:39:55 -0000 1.31 +++ rpc.py 10 May 2005 03:44:24 -0000 1.32 @@ -330,9 +330,10 @@ try: r, w, x = select.select([], [self.sock], []) n = self.sock.send(s[:BUFSIZE]) - except (AttributeError, socket.error): - # socket was closed - raise IOError + except (AttributeError, TypeError): + raise IOError, "socket no longer exists" + except socket.error: + raise else: s = s[n:] From service at paypal.com Tue May 10 22:01:25 2005 From: service at paypal.com (PayPal Account Review Department) Date: Tue, 10 May 2005 13:01:25 -0700 Subject: [Python-checkins] Notification of Limited Account Access Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20050510/d33f59d8/attachment.html From gvanrossum at users.sourceforge.net Wed May 11 23:29:19 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 11 May 2005 14:29:19 -0700 Subject: [Python-checkins] python/nondist/peps pep-0341.txt,1.1,1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3539 Modified Files: pep-0341.txt Log Message: Fix the grammar, with Reinhold's approval. Index: pep-0341.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0341.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0341.txt 7 May 2005 12:13:49 -0000 1.1 +++ pep-0341.txt 11 May 2005 21:29:16 -0000 1.2 @@ -81,11 +81,14 @@ would have to become - try_stmt: ('try' ':' suite (except_clause ':' suite)+ - ['else' ':' suite] ['finally' ':' suite] | - 'try' ':' suite (except_clause ':' suite)* - ['else' ':' suite] 'finally' ':' suite) - + try_stmt: 'try' ':' suite + ( + (except_clause ':' suite)+ + ['else' ':' suite] + ['finally' ':' suite] + | + 'finally' ':' suite + ) Implementation From gvanrossum at users.sourceforge.net Wed May 11 23:58:49 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 11 May 2005 14:58:49 -0700 Subject: [Python-checkins] python/nondist/peps pep-0342.txt, NONE, 1.1 pep-0000.txt, 1.304, 1.305 pep-0340.txt, 1.25, 1.26 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9259 Modified Files: pep-0000.txt pep-0340.txt Added Files: pep-0342.txt Log Message: Split PEP 342 (Enhanced Iterators) off of PEP 340. --- NEW FILE: pep-0342.txt --- PEP: 342 Title: Enhanced Iterators Version: $Revision: 1.1 $ Last-Modified: $Date: 2005/05/11 21:58:43 $ Author: Guido van Rossum Status: Draft Type: Standards Track Content-Type: text/plain Created: 10-May-2005 Post-History: Introduction This PEP proposes a new iterator API that allows values to be passed into an iterator using "continue EXPR". These values are received in the iterator as an argument to the new __next__ method, and can be accessed in a generator with a yield-expression. The content of this PEP is derived from the original content of PEP 340, broken off into its own PEP as the new iterator API is pretty much orthogonal from the anonymous block statement discussion. Thanks to Steven Bethard for doing the editing. Motivation and Summary TBD. Use Cases See the Examples section near the end. Specification: the __next__() Method A new method for iterators is proposed, called __next__(). It takes one optional argument, which defaults to None. Calling the __next__() method without argument or with None is equivalent to using the old iterator API, next(). For backwards compatibility, it is recommended that iterators also implement a next() method as an alias for calling the __next__() method without an argument. The argument to the __next__() method may be used by the iterator as a hint on what to do next. Specification: the next() Built-in Function This is a built-in function defined as follows: def next(itr, arg=None): nxt = getattr(itr, "__next__", None) if nxt is not None: return nxt(arg) if arg is None: return itr.next() raise TypeError("next() with arg for old-style iterator") This function is proposed because there is often a need to call the next() method outside a for-loop; the new API, and the backwards compatibility code, is too ugly to have to repeat in user code. Specification: a Change to the 'for' Loop A small change in the translation of the for-loop is proposed. The statement for VAR1 in EXPR1: BLOCK1 else: BLOCK2 will be translated as follows: itr = iter(EXPR1) arg = None # Set by "continue EXPR2", see below brk = False while True: try: VAR1 = next(itr, arg) except StopIteration: brk = True break arg = None BLOCK1 if brk: BLOCK2 (However, the variables 'itr' etc. are not user-visible and the built-in names used cannot be overridden by the user.) Specification: the Extended 'continue' Statement In the translation of the for-loop, inside BLOCK1, the new syntax continue EXPR2 is legal and is translated into arg = EXPR2 continue (Where 'arg' references the corresponding hidden variable from the previous section.) This is also the case in the body of the block-statement proposed below. EXPR2 may contain commas; "continue 1, 2, 3" is equivalent to "continue (1, 2, 3)". Specification: Generators and Yield-Expressions Generators will implement the new __next__() method API, as well as the old argument-less next() method which becomes an alias for calling __next__() without an argument. The yield-statement will be allowed to be used on the right-hand side of an assignment; in that case it is referred to as yield-expression. The value of this yield-expression is None unless __next__() was called with an argument; see below. A yield-expression must always be parenthesized except when it occurs at the top-level expression on the right-hand side of an assignment. So x = yield 42 x = yield x = 12 + (yield 42) x = 12 + (yield) foo(yield 42) foo(yield) are all legal, but x = 12 + yield 42 x = 12 + yield foo(yield 42, 12) foo(yield, 12) are all illegal. (Some of the edge cases are motivated by the current legality of "yield 12, 42".) When __next__() is called with an argument that is not None, the yield-expression that it resumes will return the argument. If it resumes a yield-statement, the value is ignored (this is similar to ignoring the value returned by a function call). When the *initial* call to __next__() receives an argument that is not None, TypeError is raised; this is likely caused by some logic error. When __next__() is called without an argument or with None as argument, and a yield-expression is resumed, the yield-expression returns None. Note: the syntactic extensions to yield make its use very similar to that in Ruby. This is intentional. Do note that in Python the block passes a value to the generator using "continue EXPR" rather than "return EXPR", and the underlying mechanism whereby control is passed between the generator and the block is completely different. Blocks in Python are not compiled into thunks; rather, yield suspends execution of the generator's frame. Some edge cases work differently; in Python, you cannot save the block for later use, and you cannot test whether there is a block or not. Alternative An alternative proposal is still under consideration, where instead of adding a __next__() method, the existing next() method is given an optional argument. The next() built-in function is then unnecessary. The only line that changes in the translation is the line VAR1 = next(itr, arg) which will be replaced by this if arg is None: VAR1 = itr.next() else: VAR1 = itr.next(arg) If "continue EXPR2" is used and EXPR2 does not evaluate to None, and the iterator's next() method does not support the optional argument, a TypeError exception will be raised, which is the same behavior as above. This proposal is more compatible (no new method name, no new built-in needed) but less future-proof; in some sense it was a mistake to call this method next() instead of __next__(), since *all* other operations corresponding to function pointers in the C type structure have names with leading and trailing underscores. Acknowledgements See Acknowledgements of PEP 340. References TBD. Copyright This document has been placed in the public domain. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.304 retrieving revision 1.305 diff -u -d -r1.304 -r1.305 --- pep-0000.txt 7 May 2005 12:13:49 -0000 1.304 +++ pep-0000.txt 11 May 2005 21:58:13 -0000 1.305 @@ -119,6 +119,7 @@ S 338 Executing modules inside packages with '-m' Coghlan S 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld + S 342 Enhanced Iterators GvR S 754 IEEE 754 Floating Point Special Values Warnes Finished PEPs (done, implemented in CVS) @@ -378,6 +379,7 @@ I 339 How to Change CPython's Bytecode Cannon S 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld + S 342 Enhanced Iterators GvR SR 666 Reject Foolish Indentation Creighton S 754 IEEE 754 Floating Point Special Values Warnes I 3000 Python 3.0 Plans Kuchling, Cannon Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- pep-0340.txt 6 May 2005 14:38:09 -0000 1.25 +++ pep-0340.txt 11 May 2005 21:58:31 -0000 1.26 @@ -12,30 +12,26 @@ Introduction This PEP proposes a new type of compound statement which can be - used for resource management purposes, and a new iterator API to - go with it. The new statement type is provisionally called the - block-statement because the keyword to be used has not yet been - chosen. + used for resource management purposes. The new statement type + is provisionally called the block-statement because the keyword + to be used has not yet been chosen. This PEP competes with several other PEPs: PEP 288 (Generators Attributes and Exceptions; only the second part), PEP 310 (Reliable Acquisition/Release Pairs), and PEP 325 (Resource-Release Support for Generators). - I should clarify that there are a few separable proposals in this - PEP. - - - Using "continue EXPR" which calls its.__next__(EXPR) which - becomes the return value of a yield-expression is entirely - orthogonal with the rest of the PEP. - - - Similarly, using a generator to "drive" a block statement is - also separable; with just the definition of the block statement - from the PEP you could implement all the examples using a class - (similar to example 6, which is easily turned into a template). + I should clarify that using a generator to "drive" a block + statement is really a separable proposal; with just the definition + of the block statement from the PEP you could implement all the + examples using a class (similar to example 6, which is easily + turned into a template). But the key idea is using a generator to + drive a block statement; the rest is elaboration, so I'd like to + keep these two parts together. - But the key idea is using a generator to drive a block statement; - the rest is elaboration. + (PEP 342, Enhanced Iterators, was originally a part of this PEP; + but the two proposals are really independent and with Steven + Bethard's help I have moved it to a separate PEP.) Motivation and Summary @@ -83,18 +79,6 @@ See the Examples section near the end. -Specification: the __next__() Method - - A new method for iterators is proposed, called __next__(). It - takes one optional argument, which defaults to None. Calling the - __next__() method without argument or with None is equivalent to - using the old iterator API, next(). For backwards compatibility, - it is recommended that iterators also implement a next() method as - an alias for calling the __next__() method without an argument. - - The argument to the __next__() method may be used by the iterator - as a hint on what to do next. - Specification: the __exit__() Method An optional new method for iterators is proposed, called @@ -103,76 +87,6 @@ traceback. If all three arguments are None, sys.exc_info() may be consulted to provide suitable default values. -Specification: the next() Built-in Function - - This is a built-in function defined as follows: - - def next(itr, arg=None): - nxt = getattr(itr, "__next__", None) - if nxt is not None: - return nxt(arg) - if arg is None: - return itr.next() - raise TypeError("next() with arg for old-style iterator") - - This function is proposed because there is often a need to call - the next() method outside a for-loop; the new API, and the - backwards compatibility code, is too ugly to have to repeat in - user code. - - Note that I'm not proposing a built-in function to call the - __exit__() method of an iterator. I don't expect that this will - be called much outside the block-statement. - -Specification: a Change to the 'for' Loop - - A small change in the translation of the for-loop is proposed. - The statement - - for VAR1 in EXPR1: - BLOCK1 - else: - BLOCK2 - - will be translated as follows: - - itr = iter(EXPR1) - arg = None # Set by "continue EXPR2", see below - brk = False - while True: - try: - VAR1 = next(itr, arg) - except StopIteration: - brk = True - break - arg = None - BLOCK1 - if brk: - BLOCK2 - - (However, the variables 'itr' etc. are not user-visible and the - built-in names used cannot be overridden by the user.) - -Specification: the Extended 'continue' Statement - - In the translation of the for-loop, inside BLOCK1, the new syntax - - continue EXPR2 - - is legal and is translated into - - arg = EXPR2 - continue - - (Where 'arg' references the corresponding hidden variable from the - previous section.) - - This is also the case in the body of the block-statement proposed - below. - - EXPR2 may contain commas; "continue 1, 2, 3" is equivalent to - "continue (1, 2, 3)". - Specification: the Anonymous Block Statement A new statement is proposed with the syntax @@ -206,16 +120,15 @@ parser, however, it is always a loop; break and continue return transfer to the block's iterator (see below for details). - The translation is subtly different from the translation of a - for-loop: iter() is not called, so EXPR1 should already be an - iterator (not just an iterable); and the iterator is guaranteed to - be notified when the block-statement is left, regardless if this - is due to a break, return or exception: + The translation is subtly different from a for-loop: iter() is + not called, so EXPR1 should already be an iterator (not just an + iterable); and the iterator is guaranteed to be notified when + the block-statement is left, regardless if this is due to a + break, return or exception: itr = EXPR1 # The iterator ret = False # True if a return statement is active val = None # Return value, if ret == True - arg = None # Argument to __next__() (value from continue) exc = None # sys.exc_info() tuple if an exception is active while True: try: @@ -226,28 +139,23 @@ else: raise exc[0], exc[1], exc[2] else: - VAR1 = next(itr, arg) # May raise StopIteration + VAR1 = itr.next() # May raise StopIteration except StopIteration: if ret: return val break try: ret = False - val = arg = exc = None + val = exc = None BLOCK1 except: exc = sys.exc_info() - (Again, the variables and built-ins are hidden from the user.) + (However, the variables 'itr' etc. are not user-visible and the + built-in names used cannot be overridden by the user.) Inside BLOCK1, the following special translations apply: - - "continue" and "continue EXPR2" are always legal; the latter is - translated as shown earlier: - - arg = EXPR2 - continue - - "break" is always legal; it is translated into: exc = (StopIteration, None, None) @@ -261,26 +169,25 @@ val = EXPR3 continue - The net effect is that break, continue and return behave much the - same as if the block-statement were a for-loop, except that the - iterator gets a chance at resource cleanup before the - block-statement is left, through the optional __exit__() method. - The iterator also gets a chance if the block-statement is left - through raising an exception. If the iterator doesn't have an - __exit__() method, there is no difference with a for-loop (except - that a for-loop calls iter() on EXPR1). + The net effect is that break and return behave much the same as + if the block-statement were a for-loop, except that the iterator + gets a chance at resource cleanup before the block-statement is + left, through the optional __exit__() method. The iterator also + gets a chance if the block-statement is left through raising an + exception. If the iterator doesn't have an __exit__() method, + there is no difference with a for-loop (except that a for-loop + calls iter() on EXPR1). - Note that a yield-statement (or a yield-expression, see below) in - a block-statement is not treated differently. It suspends the - function containing the block *without* notifying the block's - iterator. The block's iterator is entirely unaware of this - yield, since the local control flow doesn't actually leave the - block. In other words, it is *not* like a break, continue or - return statement. When the loop that was resumed by the yield - calls next(), the block is resumed right after the yield. The - generator finalization semantics described below guarantee (within - the limitations of all finalization semantics) that the block will - be resumed eventually. + Note that a yield-statement in a block-statement is not treated + differently. It suspends the function containing the block + *without* notifying the block's iterator. The block's iterator is + entirely unaware of this yield, since the local control flow + doesn't actually leave the block. In other words, it is *not* + like a break or return statement. When the loop that was resumed + by the yield calls next(), the block is resumed right after the + yield. (See example 7 below.) The generator finalization + semantics described below guarantee (within the limitations of all + finalization semantics) that the block will be resumed eventually. Unlike the for-loop, the block-statement does not have an else-clause. I think it would be confusing, and emphasize the @@ -291,10 +198,7 @@ Specification: Generator Exit Handling - Generators will implement the new __next__() method API, as well - as the old argument-less next() method which becomes an alias for - calling __next__() without an argument. They will also implement - the new __exit__() method API. + Generators will implement the new __exit__() method API. Generators will be allowed to have a yield statement inside a try-finally statement. @@ -302,54 +206,17 @@ The expression argument to the yield-statement will become optional (defaulting to None). - The yield-statement will be allowed to be used on the right-hand - side of an assignment; in that case it is referred to as - yield-expression. The value of this yield-expression is None - unless __next__() was called with an argument; see below. - - A yield-expression must always be parenthesized except when it - occurs at the top-level expression on the right-hand side of an - assignment. So - - x = yield 42 - x = yield - x = 12 + (yield 42) - x = 12 + (yield) - foo(yield 42) - foo(yield) - - are all legal, but - - x = 12 + yield 42 - x = 12 + yield - foo(yield 42, 12) - foo(yield, 12) - - are all illegal. (Some of the edge cases are motivated by the - current legality of "yield 12, 42".) - When __exit__() is called, the generator is resumed but at the - point of the yield-statement or -expression the exception - represented by the __exit__ argument(s) is raised. The generator - may re-raise this exception, raise another exception, or yield - another value, execpt that if the exception passed in to - __exit__() was StopIteration, it ought to raise StopIteration - (otherwise the effect would be that a break is turned into - continue, which is unexpected at least). When the *initial* call - resuming the generator is an __exit__() call instead of a - __next__() call, the generator's execution is aborted and the - exception is re-raised without passing control to the generator's - body. - - When __next__() is called with an argument that is not None, the - yield-expression that it resumes will return the argument. If it - resumes a yield-statement, the value is ignored (this is similar - to ignoring the value returned by a function call). When the - *initial* call to __next__() receives an argument that is not - None, TypeError is raised; this is likely caused by some logic - error. When __next__() is called without an argument or with None - as argument, and a yield-expression is resumed, the - yield-expression returns None. + point of the yield-statement the exception represented by the + __exit__ argument(s) is raised. The generator may re-raise this + exception, raise another exception, or yield another value, + except that if the exception passed in to __exit__() was + StopIteration, it ought to raise StopIteration (otherwise the + effect would be that a break is turned into continue, which is + unexpected at least). When the *initial* call resuming the + generator is an __exit__() call instead of a next() call, the + generator's execution is aborted and the exception is re-raised + without passing control to the generator's body. When a generator that has not yet terminated is garbage-collected (either through reference counting or by the cyclical garbage @@ -363,16 +230,6 @@ is no different than the guarantees that are made about finalizers (__del__() methods) of other objects. - Note: the syntactic extensions to yield make its use very similar - to that in Ruby. This is intentional. Do note that in Python the - block passes a value to the generator using "continue EXPR" rather - than "return EXPR", and the underlying mechanism whereby control - is passed between the generator and the block is completely - different. Blocks in Python are not compiled into thunks; rather, - yield suspends execution of the generator's frame. Some edge - cases work differently; in Python, you cannot save the block for - later use, and you cannot test whether there is a block or not. - Alternatives Considered and Rejected - Many alternatives have been proposed for 'block'. I haven't @@ -654,26 +511,23 @@ 8. A variant on opening() that also returns an error condition: def opening_w_error(filename, mode="r"): - try: - f = open(filename, mode) - except IOError, err: - yield None, err - else: + try: + f = open(filename, mode) + except IOError, err: + yield None, err + else: try: - yield f, None - finally: - f.close() + yield f, None + finally: + f.close() Used as follows: block opening_w_error("/etc/passwd", "a") as f, err: if err: - print "IOError:", err - else: - f.write("guido::0:0::/:/bin/sh\n") - - 9. More examples are needed: showing "continue EXPR", and the use - of continue, break and return in a block-statement. + print "IOError:", err + else: + f.write("guido::0:0::/:/bin/sh\n") Acknowledgements @@ -695,7 +549,6 @@ [3] http://effbot.org/zone/asyncore-generators.htm - Copyright This document has been placed in the public domain. From gvanrossum at users.sourceforge.net Thu May 12 00:10:56 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 11 May 2005 15:10:56 -0700 Subject: [Python-checkins] python/nondist/peps pep-0340.txt, 1.26, 1.27 pep-0342.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10442 Modified Files: pep-0340.txt pep-0342.txt Log Message: Note that 'yield' w/o an EXPR is currently illegal. Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- pep-0340.txt 11 May 2005 21:58:31 -0000 1.26 +++ pep-0340.txt 11 May 2005 22:09:37 -0000 1.27 @@ -383,13 +383,16 @@ Examples + (Several of these examples contain "yield None". If PEP 342 is + accepted, these can be changed to just "yield" of course.) + 1. A template for ensuring that a lock, acquired at the start of a block, is released when the block is left: def locking(lock): lock.acquire() try: - yield + yield None finally: lock.release() @@ -421,7 +424,7 @@ def transactional(db): try: - yield + yield None except: db.rollback() raise @@ -433,7 +436,7 @@ def auto_retry(n=3, exc=Exception): for i in range(n): try: - yield + yield None return except exc, err: # perhaps log exception here @@ -498,7 +501,7 @@ save_stdout = sys.stdout try: sys.stdout = new_stdout - yield + yield None finally: sys.stdout = save_stdout Index: pep-0342.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0342.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0342.txt 11 May 2005 21:58:43 -0000 1.1 +++ pep-0342.txt 11 May 2005 22:09:37 -0000 1.2 @@ -140,6 +140,12 @@ are all illegal. (Some of the edge cases are motivated by the current legality of "yield 12, 42".) + Note that a yield-statement or yield-expression without an + expression is now legal. This makes sense: when the information + flow in the next() call is reversed, it should be possible to + yield without passing an explicit value ("yield" is of course + equivalent to "yield None"). + When __next__() is called with an argument that is not None, the yield-expression that it resumes will return the argument. If it resumes a yield-statement, the value is ignored (this is similar From montanaro at users.sourceforge.net Thu May 12 15:42:45 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Thu, 12 May 2005 06:42:45 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libsocksvr.tex, 1.18, 1.19 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1244 Modified Files: libsocksvr.tex Log Message: Incorporate a lightly edited version of the SocketServer module's docstring into the docs and segregate the method descriptions for the various classes into separate sections. Base on suggestion by Paul Rubin in c.l.py. Index: libsocksvr.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libsocksvr.tex,v retrieving revision 1.18 retrieving revision 1.19 diff -u -d -r1.18 -r1.19 --- libsocksvr.tex 21 Jul 2004 02:47:10 -0000 1.18 +++ libsocksvr.tex 12 May 2005 13:42:42 -0000 1.19 @@ -52,10 +52,87 @@ \setindexsubitem{(SocketServer protocol)} +\subsection{Server Creation Notes} + +There are five classes in an inheritance diagram, four of which represent +synchronous servers of four types: + +\begin{verbatim} + +------------+ + | BaseServer | + +------------+ + | + v + +-----------+ +------------------+ + | TCPServer |------->| UnixStreamServer | + +-----------+ +------------------+ + | + v + +-----------+ +--------------------+ + | UDPServer |------->| UnixDatagramServer | + +-----------+ +--------------------+ +\end{verbatim} + +Note that \class{UnixDatagramServer} derives from \class{UDPServer}, not +from \class{UnixStreamServer} -- the only difference between an IP and a +Unix stream server is the address family, which is simply repeated in both +unix server classes. + +Forking and threading versions of each type of server can be created using +the \class{ForkingMixIn} and \class{ThreadingMixIn} mix-in classes. For +instance, a threading UDP server class is created as follows: + +\begin{verbatim} + class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass +\end{verbatim} + +The mix-in class must come first, since it overrides a method defined in +\class{UDPServer}. Setting the various member variables also changes the +behavior of the underlying server mechanism. + +To implement a service, you must derive a class from +\class{BaseRequestHandler} and redefine its \method{handle()} method. You +can then run various versions of the service by combining one of the server +classes with your request handler class. The request handler class must be +different for datagram or stream services. This can be hidden by using the +mix-in request handler classes \class{StreamRequestHandler} or +\class{DatagramRequestHandler}. + +Of course, you still have to use your head! For instance, it makes no sense +to use a forking server if the service contains state in memory that can be +modified by different requests, since the modifications in the child process +would never reach the initial state kept in the parent process and passed to +each child. In this case, you can use a threading server, but you will +probably have to use locks to protect the integrity of the shared data. + +On the other hand, if you are building an HTTP server where all data is +stored externally (for instance, in the file system), a synchronous class +will essentially render the service "deaf" while one request is being +handled -- which may be for a very long time if a client is slow to receive +all the data it has requested. Here a threading or forking server is +appropriate. + +In some cases, it may be appropriate to process part of a request +synchronously, but to finish processing in a forked child depending on the +request data. This can be implemented by using a synchronous server and +doing an explicit fork in the request handler class \method{handle()} +method. + +Another approach to handling multiple simultaneous requests in an +environment that supports neither threads nor \function{fork()} (or where +these are too expensive or inappropriate for the service) is to maintain an +explicit table of partially finished requests and to use \function{select()} +to decide which request to work on next (or whether to handle a new incoming +request). This is particularly important for stream services where each +client can potentially be connected for a long time (if threads or +subprocesses cannot be used). + %XXX should data and methods be intermingled, or separate? % how should the distinction between class and instance variables be % drawn? +\subsection{Server Objects} + \begin{funcdesc}{fileno}{} Return an integer file descriptor for the socket on which the server is listening. This function is most commonly passed to @@ -160,7 +237,8 @@ % instance variables, adding new network families? \begin{funcdesc}{server_activate}{} -Called by the server's constructor to activate the server. +Called by the server's constructor to activate the server. The default +behavior just \method{listen}s to the server's socket. May be overridden. \end{funcdesc} @@ -176,6 +254,8 @@ The default implementation always returns \constant{True}. \end{funcdesc} +\subsection{RequestHandler Objects} + The request handler class must define a new \method{handle()} method, and can override any of the following methods. A new instance is created for each request. @@ -189,6 +269,7 @@ \begin{funcdesc}{handle}{} This function must do all the work required to service a request. +The default implementation does nothing. Several instance attributes are available to it; the request is available as \member{self.request}; the client address as \member{self.client_address}; and the server instance as From montanaro at users.sourceforge.net Thu May 12 15:43:42 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Thu, 12 May 2005 06:43:42 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libsocksvr.tex, 1.18, 1.18.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1594 Modified Files: Tag: release24-maint libsocksvr.tex Log Message: backport from head Index: libsocksvr.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libsocksvr.tex,v retrieving revision 1.18 retrieving revision 1.18.4.1 diff -u -d -r1.18 -r1.18.4.1 --- libsocksvr.tex 21 Jul 2004 02:47:10 -0000 1.18 +++ libsocksvr.tex 12 May 2005 13:43:40 -0000 1.18.4.1 @@ -52,10 +52,87 @@ \setindexsubitem{(SocketServer protocol)} +\subsection{Server Creation Notes} + +There are five classes in an inheritance diagram, four of which represent +synchronous servers of four types: + +\begin{verbatim} + +------------+ + | BaseServer | + +------------+ + | + v + +-----------+ +------------------+ + | TCPServer |------->| UnixStreamServer | + +-----------+ +------------------+ + | + v + +-----------+ +--------------------+ + | UDPServer |------->| UnixDatagramServer | + +-----------+ +--------------------+ +\end{verbatim} + +Note that \class{UnixDatagramServer} derives from \class{UDPServer}, not +from \class{UnixStreamServer} -- the only difference between an IP and a +Unix stream server is the address family, which is simply repeated in both +unix server classes. + +Forking and threading versions of each type of server can be created using +the \class{ForkingMixIn} and \class{ThreadingMixIn} mix-in classes. For +instance, a threading UDP server class is created as follows: + +\begin{verbatim} + class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass +\end{verbatim} + +The mix-in class must come first, since it overrides a method defined in +\class{UDPServer}. Setting the various member variables also changes the +behavior of the underlying server mechanism. + +To implement a service, you must derive a class from +\class{BaseRequestHandler} and redefine its \method{handle()} method. You +can then run various versions of the service by combining one of the server +classes with your request handler class. The request handler class must be +different for datagram or stream services. This can be hidden by using the +mix-in request handler classes \class{StreamRequestHandler} or +\class{DatagramRequestHandler}. + +Of course, you still have to use your head! For instance, it makes no sense +to use a forking server if the service contains state in memory that can be +modified by different requests, since the modifications in the child process +would never reach the initial state kept in the parent process and passed to +each child. In this case, you can use a threading server, but you will +probably have to use locks to protect the integrity of the shared data. + +On the other hand, if you are building an HTTP server where all data is +stored externally (for instance, in the file system), a synchronous class +will essentially render the service "deaf" while one request is being +handled -- which may be for a very long time if a client is slow to receive +all the data it has requested. Here a threading or forking server is +appropriate. + +In some cases, it may be appropriate to process part of a request +synchronously, but to finish processing in a forked child depending on the +request data. This can be implemented by using a synchronous server and +doing an explicit fork in the request handler class \method{handle()} +method. + +Another approach to handling multiple simultaneous requests in an +environment that supports neither threads nor \function{fork()} (or where +these are too expensive or inappropriate for the service) is to maintain an +explicit table of partially finished requests and to use \function{select()} +to decide which request to work on next (or whether to handle a new incoming +request). This is particularly important for stream services where each +client can potentially be connected for a long time (if threads or +subprocesses cannot be used). + %XXX should data and methods be intermingled, or separate? % how should the distinction between class and instance variables be % drawn? +\subsection{Server Objects} + \begin{funcdesc}{fileno}{} Return an integer file descriptor for the socket on which the server is listening. This function is most commonly passed to @@ -160,7 +237,8 @@ % instance variables, adding new network families? \begin{funcdesc}{server_activate}{} -Called by the server's constructor to activate the server. +Called by the server's constructor to activate the server. The default +behavior just \method{listen}s to the server's socket. May be overridden. \end{funcdesc} @@ -176,6 +254,8 @@ The default implementation always returns \constant{True}. \end{funcdesc} +\subsection{RequestHandler Objects} + The request handler class must define a new \method{handle()} method, and can override any of the following methods. A new instance is created for each request. @@ -189,6 +269,7 @@ \begin{funcdesc}{handle}{} This function must do all the work required to service a request. +The default implementation does nothing. Several instance attributes are available to it; the request is available as \member{self.request}; the client address as \member{self.client_address}; and the server instance as From montanaro at users.sourceforge.net Thu May 12 15:44:31 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Thu, 12 May 2005 06:44:31 -0700 Subject: [Python-checkins] python/dist/src/Misc ACKS,1.294,1.295 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1779 Modified Files: ACKS Log Message: credit source of idea for recent doc changes Index: ACKS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/ACKS,v retrieving revision 1.294 retrieving revision 1.295 diff -u -d -r1.294 -r1.295 --- ACKS 10 Feb 2005 20:48:03 -0000 1.294 +++ ACKS 12 May 2005 13:44:23 -0000 1.295 @@ -506,6 +506,7 @@ Saskia van Rossum Donald Wallace Rouse II Liam Routt +Paul Rubin Audun S. Runde Jeff Rush Sam Rushing From montanaro at users.sourceforge.net Thu May 12 15:45:14 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Thu, 12 May 2005 06:45:14 -0700 Subject: [Python-checkins] python/dist/src/Misc ACKS,1.289.2.1,1.289.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2030 Modified Files: Tag: release24-maint ACKS Log Message: backport Index: ACKS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/ACKS,v retrieving revision 1.289.2.1 retrieving revision 1.289.2.2 diff -u -d -r1.289.2.1 -r1.289.2.2 --- ACKS 10 Feb 2005 22:47:13 -0000 1.289.2.1 +++ ACKS 12 May 2005 13:45:11 -0000 1.289.2.2 @@ -503,6 +503,7 @@ Saskia van Rossum Donald Wallace Rouse II Liam Routt +Paul Rubin Audun S. Runde Jeff Rush Sam Rushing From gvanrossum at users.sourceforge.net Sat May 14 02:08:22 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Fri, 13 May 2005 17:08:22 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt, NONE, 1.1 pep-0000.txt, 1.305, 1.306 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7951 Modified Files: pep-0000.txt Added Files: pep-0343.txt Log Message: Add PEP 343: Anonymous Block Redux --- NEW FILE: pep-0343.txt --- PEP: 343 Title: Anonymous Block Redux Version: $Revision: 1.1 $ Last-Modified: $Date: 2005/05/14 00:08:20 $ Author: Guido van Rossum Status: Draft Type: Standards Track Content-Type: text/plain Created: 13-May-2005 Post-History: Introduction After a lot of discussion about PEP 340 and alternatives, I've decided to withdraw PEP 340 and propose a slight variant on PEP 310. Motivation and Summary TBD. Use Cases See the Examples section near the end. Specification A new statement is proposed with the syntax do EXPR as VAR: BLOCK Here, 'do' and 'as' are new keywords; EXPR is an arbitrary expression (but not an expression-list) and VAR is an arbitrary assignment target (which may be a comma-separated list). The "as VAR" part is optional. The choice of the 'do' keyword is provisional; an alternative under consideration is 'with'. A yield-statement is illegal inside BLOCK. This is because the do-statement is translated into a try/finally statement, and yield is illegal in a try/finally statement. The translation of the above statement is: abc = EXPR try: VAR = abc.__enter__() BLOCK finally: abc.__exit__(*sys.exc_info()) # XXX See below If the "as VAR" part of the syntax is omitted, the "VAR =" part of the translation is omitted (but abc.__enter__() is still called). The call to abc.__exit__() is only approximated as written. The actual calling convention is: If the finally-suite was reached through normal completion of BLOCK or through a "non-local goto" (a break, continue or return statement in BLOCK), abc.__exit__() is called without arguments (or perhaps with three None arguments). If the finally-suite was reached through an exception raised in BLOCK, abc.__exit__() is called with three arguments representing the exception type, value, and traceback. Optional Generator Decorator It is possible to write a decorator that makes it possible to use a generator that yields exactly once to control a do-statement. Here's a sketch of such a decorator: class Wrapper(object): def __init__(self, gen): self.gen = gen self.state = "initial" def __enter__(self): assert self.state == "initial" self.state = "entered" try: return self.gen.next() except StopIteration: self.state = "error" raise RuntimeError("template generator didn't yield") def __exit__(self, *args): assert self.state == "entered" self.state = "exited" try: self.gen.next() except StopIteration: return else: self.state = "error" raise RuntimeError("template generator didn't stop") def do_template(func): def helper(*args, **kwds): return Wrapper(func(*args, **kwds)) return helper This decorator could be used as follows: @do_template def opening(filename): f = open(filename) # IOError here is untouched by Wrapper yield f f.close() # Ditto for errors here (however unlikely) A robust implementation of such a decorator should be made part of the standard library. Examples Several of these examples contain "yield None". If PEP 342 is accepted, these can be changed to just "yield". 1. A template for ensuring that a lock, acquired at the start of a block, is released when the block is left: @do_template def locking(lock): lock.acquire() yield None lock.release() Used as follows: do locking(myLock): # Code here executes with myLock held. The lock is # guaranteed to be released when the block is left (even # if via return or by an uncaught exception). 2. A template for opening a file that ensures the file is closed when the block is left: @do_template def opening(filename, mode="r"): f = open(filename, mode) yield f f.close() Used as follows: do opening("/etc/passwd") as f: for line in f: print line.rstrip() 3. A template for committing or rolling back a database transaction; this is written as a class rather than as a decorator since it requires access to the exception information: class transactional: def __init__(self, db): self.db = db def __enter__(self): pass def __exit__(self, *args): if args and args[0] is not None: self.db.rollback() else: self.db.commit() 4. Example 1 rewritten without a generator: class locking: def __init__(self, lock): self.lock = lock def __enter__(self): self.lock.acquire() def __exit__(self, *args): self.lock.release() (This example is easily modified to implement the other examples; it shows how much simpler generators are for the same purpose.) 5. Redirect stdout temporarily: @do_template def redirecting_stdout(new_stdout): save_stdout = sys.stdout try: sys.stdout = new_stdout yield None finally: sys.stdout = save_stdout Used as follows: do opening(filename, "w") as f: do redirecting_stdout(f): print "Hello world" 6. A variant on opening() that also returns an error condition: @do_template def opening_w_error(filename, mode="r"): try: f = open(filename, mode) except IOError, err: yield None, err else: yield f, None f.close() Used as follows: do opening_w_error("/etc/passwd", "a") as f, err: if err: print "IOError:", err else: f.write("guido::0:0::/:/bin/sh\n") 7. Another useful example would be an operation that blocks signals. The use could be like this: import signal do signal.blocking(): # code executed without worrying about signals An optional argument might be a list of signals to be blocked; by default all signals are blocked. The implementation is left as an exercise to the reader. Copyright This document has been placed in the public domain. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.305 retrieving revision 1.306 diff -u -d -r1.305 -r1.306 --- pep-0000.txt 11 May 2005 21:58:13 -0000 1.305 +++ pep-0000.txt 14 May 2005 00:08:19 -0000 1.306 @@ -120,6 +120,7 @@ S 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld S 342 Enhanced Iterators GvR + S 343 Anonymous Block Redux GvR S 754 IEEE 754 Floating Point Special Values Warnes Finished PEPs (done, implemented in CVS) @@ -380,6 +381,7 @@ S 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld S 342 Enhanced Iterators GvR + S 343 Anonymous Block Redux GvR SR 666 Reject Foolish Indentation Creighton S 754 IEEE 754 Floating Point Special Values Warnes I 3000 Python 3.0 Plans Kuchling, Cannon From gvanrossum at users.sourceforge.net Sat May 14 04:02:42 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Fri, 13 May 2005 19:02:42 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.1,1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25715 Modified Files: pep-0343.txt Log Message: Don't be wishy-washy about the call to __exit__(). Fix the redirecting_stdout() example (remove the try/finally). Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0343.txt 14 May 2005 00:08:20 -0000 1.1 +++ pep-0343.txt 14 May 2005 02:02:40 -0000 1.2 @@ -46,23 +46,28 @@ The translation of the above statement is: abc = EXPR + exc = () # Or (None, None, None) ? try: - VAR = abc.__enter__() - BLOCK + try: + VAR = abc.__enter__() + BLOCK + except: + exc = sys.exc_info() + raise finally: - abc.__exit__(*sys.exc_info()) # XXX See below + abc.__exit__(exc) If the "as VAR" part of the syntax is omitted, the "VAR =" part of the translation is omitted (but abc.__enter__() is still called). - The call to abc.__exit__() is only approximated as written. The - actual calling convention is: If the finally-suite was reached - through normal completion of BLOCK or through a "non-local goto" - (a break, continue or return statement in BLOCK), abc.__exit__() - is called without arguments (or perhaps with three None - arguments). If the finally-suite was reached through an exception - raised in BLOCK, abc.__exit__() is called with three arguments - representing the exception type, value, and traceback. + The calling convention for abc.__exit__() is: as follows. If the + finally-suite was reached through normal completion of BLOCK or + through a "non-local goto" (a break, continue or return statement + in BLOCK), abc.__exit__() is called without arguments (or perhaps + with three None arguments?). If the finally-suite was reached + through an exception raised in BLOCK, abc.__exit__() is called + with three arguments representing the exception type, value, and + traceback. Optional Generator Decorator @@ -70,41 +75,41 @@ a generator that yields exactly once to control a do-statement. Here's a sketch of such a decorator: - class Wrapper(object): - def __init__(self, gen): - self.gen = gen - self.state = "initial" - def __enter__(self): - assert self.state == "initial" - self.state = "entered" - try: - return self.gen.next() - except StopIteration: - self.state = "error" - raise RuntimeError("template generator didn't yield") - def __exit__(self, *args): - assert self.state == "entered" - self.state = "exited" - try: - self.gen.next() - except StopIteration: - return - else: - self.state = "error" - raise RuntimeError("template generator didn't stop") + class Wrapper(object): + def __init__(self, gen): + self.gen = gen + self.state = "initial" + def __enter__(self): + assert self.state == "initial" + self.state = "entered" + try: + return self.gen.next() + except StopIteration: + self.state = "error" + raise RuntimeError("template generator didn't yield") + def __exit__(self, *args): + assert self.state == "entered" + self.state = "exited" + try: + self.gen.next() + except StopIteration: + return + else: + self.state = "error" + raise RuntimeError("template generator didn't stop") - def do_template(func): - def helper(*args, **kwds): - return Wrapper(func(*args, **kwds)) - return helper + def do_template(func): + def helper(*args, **kwds): + return Wrapper(func(*args, **kwds)) + return helper This decorator could be used as follows: - @do_template - def opening(filename): - f = open(filename) # IOError here is untouched by Wrapper - yield f - f.close() # Ditto for errors here (however unlikely) + @do_template + def opening(filename): + f = open(filename) # IOError here is untouched by Wrapper + yield f + f.close() # Ditto for errors here (however unlikely) A robust implementation of such a decorator should be made part of the standard library. @@ -151,14 +156,14 @@ class transactional: def __init__(self, db): - self.db = db + self.db = db def __enter__(self): pass - def __exit__(self, *args): - if args and args[0] is not None: - self.db.rollback() - else: - self.db.commit() + def __exit__(self, *args): + if args and args[0] is not None: + self.db.rollback() + else: + self.db.commit() 4. Example 1 rewritten without a generator: @@ -166,9 +171,9 @@ def __init__(self, lock): self.lock = lock def __enter__(self): - self.lock.acquire() + self.lock.acquire() def __exit__(self, *args): - self.lock.release() + self.lock.release() (This example is easily modified to implement the other examples; it shows how much simpler generators are for the same @@ -179,11 +184,9 @@ @do_template def redirecting_stdout(new_stdout): save_stdout = sys.stdout - try: - sys.stdout = new_stdout - yield None - finally: - sys.stdout = save_stdout + sys.stdout = new_stdout + yield None + sys.stdout = save_stdout Used as follows: @@ -200,7 +203,7 @@ except IOError, err: yield None, err else: - yield f, None + yield f, None f.close() Used as follows: From gvanrossum at users.sourceforge.net Sat May 14 06:02:14 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Fri, 13 May 2005 21:02:14 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.2,1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9821 Modified Files: pep-0343.txt Log Message: Various clarifications. Mention Decimal context example. Clarify that I don't like having files etc. implement __enter__ and __exit__ directly. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0343.txt 14 May 2005 02:02:40 -0000 1.2 +++ pep-0343.txt 14 May 2005 04:02:10 -0000 1.3 @@ -57,6 +57,10 @@ finally: abc.__exit__(exc) + Here, the variables 'abc' and 'exc' are internal variables and not + accessible to the user; they will most likely be implemented as + special registers or stack positions. + If the "as VAR" part of the syntax is omitted, the "VAR =" part of the translation is omitted (but abc.__enter__() is still called). @@ -114,6 +118,32 @@ A robust implementation of such a decorator should be made part of the standard library. +Other Optional Extensions + + It would be possible to endow certain objects, like files, + sockets, and locks, with __enter__ and __exit__ methods so that + instead of writing + + do locking(myLock): + BLOCK + + one could write simply + + do myLock: + BLOCK + + I think we should be careful with this; it could lead to mistakes + like + + f = open(filename) + do f: + BLOCK1 + do f: + BLOCK2 + + which does not do what one might think (f is closed when BLOCK2 is + entered). + Examples Several of these examples contain "yield None". If PEP 342 is @@ -194,6 +224,10 @@ do redirecting_stdout(f): print "Hello world" + This isn't thread-safe, of course, but neither is doing this + same dance manually. In a single-threaded program (e.g., a + script) it is a totally fine way of doing things. + 6. A variant on opening() that also returns an error condition: @do_template @@ -226,6 +260,10 @@ by default all signals are blocked. The implementation is left as an exercise to the reader. + 8. Another use for this feature is the Decimal context. It's left + as an exercise for the reader. (Mail it to me if you'd like to + see it here.) + Copyright This document has been placed in the public domain. From gvanrossum at users.sourceforge.net Sat May 14 07:02:31 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Fri, 13 May 2005 22:02:31 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.3,1.4 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16548 Modified Files: pep-0343.txt Log Message: Add a motivational section, remove tabs, add colons, and some very minor edits. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- pep-0343.txt 14 May 2005 04:02:10 -0000 1.3 +++ pep-0343.txt 14 May 2005 05:02:28 -0000 1.4 @@ -17,7 +17,103 @@ Motivation and Summary - TBD. + PEP 340, Anonymous Block Statements, combined many powerful ideas: + using generators as block templates, adding exception handling and + finalization to generators, and more. Besides praise it received + a lot of opposition from people who didn't like the fact that it + was, under the covers, a (optential) looping construct. This + meant that break and continue in a block-statement would break or + continue the block-statement, even if it was used as a non-looping + resource management tool. + + But the final blow came when I read Raymond Chen's rant about + flow-control macros[1]. Raymond argues convincingly that hiding + flow control in macros makes your code inscrutable, and I find + that his argument applies to Python as well as to C. I realized + that PEP 340 templates can hide all sorts of control flow; for + example, its example 4 (auto_retry()) catches exceptions and + repeats the block up to three times. + + However, the with-statement of PEP 310 does *not* hide control + flow, in my view: while a finally-suite temporarily suspends the + control flow, in the end, the control flow resumes as if the + finally-suite wasn't there at all. Consider this: + + with VAR = EXPR: + BLOCK1 + BLOCK2 + + Here, just as if the first line was "if True" instead, we know + that if BLOCK1 completes without an exception, BLOCK2 will be + reached; and if BLOCK1 raises an exception or executes a + non-local goto (a break, continue or return), BLOCK2 is *not* + reached. The magic added by the with-statement at the end doesn't + affect this. + + (You may ask, what if a bug in the __exit__ method causes an + exception? Then all is lost -- but this is no worse than with + other exceptions; the nature of exceptions is that they can happen + *anywhere*, and you just have to live with that. Even if you + write bug-free code, a KeyboardInterrupt exception can still cause + it to exit between any two virtual machine opcodes.) + + This argument almost led me to endorse PEP 310, but I had one idea + left from the PEP 340 euphoria that I wasn't ready to drop: using + generators as "templates" for abstractions like acquiring and + releasing a lock or opening and closing a file is a powerful idea, + as can be seen by looking at the examples in that PEP. + + Inspired by a counter-proposal to PEP 340 by Phillip Eby I tried + to create a decorator that would turn a suitable generator into an + object with the necessary __entry__ and __exit__ methods. Here I + ran into a snag: while it wasn't too hard for the locking example, + it was impossible to do this for the opening example. The idea + was to define the template like this: + + @with_template + def opening(filename): + f = open(filename) + yield f + f.close() + + and used it like this: + + with f = opening(filename): + ...read data from f... + + The problem is that in PEP 310, the result of calling EXPR is + assigned directly to VAR, and then VAR's __exit__ method is called + upon exit from BLOCK1. But here, VAR clearly needs to receive the + opened file, and that would mean that __exit__ would have to be a + method on the file. + + While this can be solved using a proxy class, this is awkward and + made me realize that a slightly different translation would make + writing the desired decorator a piece of cake: let VAR receive the + result from calling the __enter__ method, and save the value of + EXPR to call its __exit__ method later. Then the decorator can + return an instance of a wrapper class whose __enter__ method calls + the generator's next() method and returns whatever next() returns; + the wrapper instance's __exit__ method calls next() again but + expects it to raise StopIteration. (Details below in the section + Optional Generator Decorator.) + + So now the final hurdle was that the PEP 310 syntax: + + with VAR = EXPR: + BLOCK1 + + would be deceptive, since VAR does *not* receive the value of + EXPR. Given PEP 340, it was an easy step to: + + with EXPR as VAR: + BLOCK1 + + or, using an alternate keyword that has been proposed a number of + times: + + do EXPR as VAR: + BLOCK1 Use Cases @@ -25,7 +121,7 @@ Specification - A new statement is proposed with the syntax + A new statement is proposed with the syntax: do EXPR as VAR: BLOCK @@ -53,7 +149,7 @@ BLOCK except: exc = sys.exc_info() - raise + raise finally: abc.__exit__(exc) @@ -66,7 +162,7 @@ The calling convention for abc.__exit__() is: as follows. If the finally-suite was reached through normal completion of BLOCK or - through a "non-local goto" (a break, continue or return statement + through a non-local goto (a break, continue or return statement in BLOCK), abc.__exit__() is called without arguments (or perhaps with three None arguments?). If the finally-suite was reached through an exception raised in BLOCK, abc.__exit__() is called @@ -122,18 +218,18 @@ It would be possible to endow certain objects, like files, sockets, and locks, with __enter__ and __exit__ methods so that - instead of writing + instead of writing: do locking(myLock): BLOCK - one could write simply + one could write simply: do myLock: BLOCK I think we should be careful with this; it could lead to mistakes - like + like: f = open(filename) do f: @@ -264,6 +360,10 @@ as an exercise for the reader. (Mail it to me if you'd like to see it here.) +References + + [1] http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx + Copyright This document has been placed in the public domain. From gvanrossum at users.sourceforge.net Sat May 14 07:08:44 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Fri, 13 May 2005 22:08:44 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.4,1.5 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17313 Modified Files: pep-0343.txt Log Message: Slight elaboration for readers who don't recall PEP 310. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- pep-0343.txt 14 May 2005 05:02:28 -0000 1.4 +++ pep-0343.txt 14 May 2005 05:08:23 -0000 1.5 @@ -37,18 +37,34 @@ However, the with-statement of PEP 310 does *not* hide control flow, in my view: while a finally-suite temporarily suspends the control flow, in the end, the control flow resumes as if the - finally-suite wasn't there at all. Consider this: + finally-suite wasn't there at all. + + Remember, PEP 310 proposes rougly this syntax (the "VAR =" part is + optional): with VAR = EXPR: + BLOCK + + which roughly translates into + + VAR = EXPR + VAR.__enter__() + try: + BLOCK + finally: + VAR.__exit__() + + Now consider this example: + + with f = opening("/etc/passwd"): BLOCK1 BLOCK2 Here, just as if the first line was "if True" instead, we know that if BLOCK1 completes without an exception, BLOCK2 will be - reached; and if BLOCK1 raises an exception or executes a - non-local goto (a break, continue or return), BLOCK2 is *not* - reached. The magic added by the with-statement at the end doesn't - affect this. + reached; and if BLOCK1 raises an exception or executes a non-local + goto (a break, continue or return), BLOCK2 is *not* reached. The + magic added by the with-statement at the end doesn't affect this. (You may ask, what if a bug in the __exit__ method causes an exception? Then all is lost -- but this is no worse than with From gvanrossum at users.sourceforge.net Sat May 14 07:18:38 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Fri, 13 May 2005 22:18:38 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.5,1.6 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18203 Modified Files: pep-0343.txt Log Message: Explain the __exit__() signature. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- pep-0343.txt 14 May 2005 05:08:23 -0000 1.5 +++ pep-0343.txt 14 May 2005 05:18:36 -0000 1.6 @@ -185,6 +185,23 @@ with three arguments representing the exception type, value, and traceback. + The motivation for this API to __exit__(), as opposed to the + argument-less __exit__() from PEP 310, was given by the + transactional() use case, example 3 below. The template in that + example must commit or roll back the transaction depending on + whether an exception occurred or not. Rather than just having a + boolean flag indicating whether an exception occurred, we pass the + complete exception information, for the benefor of an + exception-logging facility for example. Relying on sys.exc_info() + to get att the exception information was rejected; sys.exc_info() + has very complex semantics and it is perfectly possible that it + returns the exception information for an exception that was caught + ages ago. It was also proposed to add an additional boolean to + distinguish between reaching the end of BLOCK and a non-local + goto. This was rejected as too complex and unnecessary; a + non-local goto should be considered unexceptional for the purposes + of a database transaction roll-back decision. + Optional Generator Decorator It is possible to write a decorator that makes it possible to use @@ -300,7 +317,7 @@ def __init__(self, db): self.db = db def __enter__(self): - pass + self.db.begin() def __exit__(self, *args): if args and args[0] is not None: self.db.rollback() From gvanrossum at users.sourceforge.net Sat May 14 07:20:24 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Fri, 13 May 2005 22:20:24 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.6,1.7 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18510 Modified Files: pep-0343.txt Log Message: Break last line of intro differently so "PEP 310" can be a hyperlink. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- pep-0343.txt 14 May 2005 05:18:36 -0000 1.6 +++ pep-0343.txt 14 May 2005 05:20:21 -0000 1.7 @@ -12,8 +12,8 @@ Introduction After a lot of discussion about PEP 340 and alternatives, I've - decided to withdraw PEP 340 and propose a slight variant on PEP - 310. + decided to withdraw PEP 340 and propose a slight variant on + PEP 310. Motivation and Summary From rhettinger at users.sourceforge.net Sat May 14 19:18:34 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 14 May 2005 10:18:34 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut glossary.tex,1.11,1.12 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4214 Modified Files: glossary.tex Log Message: SF bug #1201807: Glossary listing bug Index: glossary.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/glossary.tex,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- glossary.tex 11 Jan 2005 16:11:13 -0000 1.11 +++ glossary.tex 14 May 2005 17:18:31 -0000 1.12 @@ -86,7 +86,7 @@ integers starting from zero. Called a hash in Perl. \index{duck-typing} -\item{duck-typing} +\item[duck-typing] Pythonic programming style that determines an object's type by inspection of its method or attribute signature rather than by explicit relationship to some type object ("If it looks like a duck and quacks like a duck, it From rhettinger at users.sourceforge.net Sat May 14 19:20:27 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 14 May 2005 10:20:27 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut glossary.tex, 1.9.4.2, 1.9.4.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4634 Modified Files: Tag: release24-maint glossary.tex Log Message: SF bug #1201807: Glossary listing bug Index: glossary.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/glossary.tex,v retrieving revision 1.9.4.2 retrieving revision 1.9.4.3 diff -u -d -r1.9.4.2 -r1.9.4.3 --- glossary.tex 11 Jan 2005 16:14:55 -0000 1.9.4.2 +++ glossary.tex 14 May 2005 17:20:25 -0000 1.9.4.3 @@ -86,7 +86,7 @@ integers starting from zero. Called a hash in Perl. \index{duck-typing} -\item{duck-typing} +\item[duck-typing] Pythonic programming style that determines an object's type by inspection of its method or attribute signature rather than by explicit relationship to some type object ("If it looks like a duck and quacks like a duck, it From gvanrossum at users.sourceforge.net Sat May 14 19:36:17 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Sat, 14 May 2005 10:36:17 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.7,1.8 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7347 Modified Files: pep-0343.txt Log Message: __exit__ is always called with 3 args. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- pep-0343.txt 14 May 2005 05:20:21 -0000 1.7 +++ pep-0343.txt 14 May 2005 17:36:15 -0000 1.8 @@ -158,7 +158,7 @@ The translation of the above statement is: abc = EXPR - exc = () # Or (None, None, None) ? + exc = (None, None, None) try: try: VAR = abc.__enter__() @@ -167,7 +167,7 @@ exc = sys.exc_info() raise finally: - abc.__exit__(exc) + abc.__exit__(*exc) Here, the variables 'abc' and 'exc' are internal variables and not accessible to the user; they will most likely be implemented as @@ -178,12 +178,11 @@ The calling convention for abc.__exit__() is: as follows. If the finally-suite was reached through normal completion of BLOCK or - through a non-local goto (a break, continue or return statement - in BLOCK), abc.__exit__() is called without arguments (or perhaps - with three None arguments?). If the finally-suite was reached - through an exception raised in BLOCK, abc.__exit__() is called - with three arguments representing the exception type, value, and - traceback. + through a non-local goto (a break, continue or return statement in + BLOCK), abc.__exit__() is called with three None arguments. If + the finally-suite was reached through an exception raised in + BLOCK, abc.__exit__() is called with three arguments representing + the exception type, value, and traceback. The motivation for this API to __exit__(), as opposed to the argument-less __exit__() from PEP 310, was given by the From rhettinger at users.sourceforge.net Sat May 14 20:08:27 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 14 May 2005 11:08:27 -0700 Subject: [Python-checkins] python/dist/src/Objects dictobject.c, 2.164, 2.165 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13045 Modified Files: dictobject.c Log Message: SF patch #1200051: Small optimization for PyDict_Merge() (Contributed by Barry Warsaw and Matt Messier.) Index: dictobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/dictobject.c,v retrieving revision 2.164 retrieving revision 2.165 diff -u -d -r2.164 -r2.165 --- dictobject.c 15 Apr 2005 15:58:42 -0000 2.164 +++ dictobject.c 14 May 2005 18:08:25 -0000 2.165 @@ -1203,6 +1203,12 @@ if (other == mp || other->ma_used == 0) /* a.update(a) or a.update({}); nothing to do */ return 0; + if (mp->ma_used == 0) + /* Since the target dict is empty, PyDict_GetItem() + * always returns NULL. Setting override to 1 + * skips the unnecessary test. + */ + override = 1; /* Do one big resize at the start, rather than * incrementally resizing as we insert new items. Expect * that there will be no (or few) overlapping keys. From montanaro at users.sourceforge.net Sat May 14 22:54:18 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Sat, 14 May 2005 13:54:18 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1290,1.1291 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9826/Misc Modified Files: NEWS Log Message: Add better datetime support to xmlrpclib module. Closes patch #1120353. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1290 retrieving revision 1.1291 diff -u -d -r1.1290 -r1.1291 --- NEWS 26 Apr 2005 03:45:26 -0000 1.1290 +++ NEWS 14 May 2005 20:54:13 -0000 1.1291 @@ -123,6 +123,11 @@ Library ------- +- Patch #1120353: The xmlrpclib module provides better, more transparent, + support for datetime.{datetime,date,time} objects. With use_datetime set + to True, applications shouldn't have to fiddle with the DateTime wrapper + class at all. + - distutils.commands.upload was added to support uploading distribution files to PyPI. From montanaro at users.sourceforge.net Sat May 14 22:54:18 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Sat, 14 May 2005 13:54:18 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libxmlrpclib.tex, 1.22, 1.23 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9826/Doc/lib Modified Files: libxmlrpclib.tex Log Message: Add better datetime support to xmlrpclib module. Closes patch #1120353. Index: libxmlrpclib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libxmlrpclib.tex,v retrieving revision 1.22 retrieving revision 1.23 diff -u -d -r1.22 -r1.23 --- libxmlrpclib.tex 21 Mar 2005 19:39:16 -0000 1.22 +++ libxmlrpclib.tex 14 May 2005 20:54:15 -0000 1.23 @@ -19,7 +19,7 @@ \begin{classdesc}{ServerProxy}{uri\optional{, transport\optional{, encoding\optional{, verbose\optional{, - allow_none}}}}} + allow_none\optional{, use_datetime}}}}}} A \class{ServerProxy} instance is an object that manages communication with a remote XML-RPC server. The required first argument is a URI (Uniform Resource Indicator), and will normally be the URL of the @@ -33,6 +33,13 @@ This is a commonly-used extension to the XML-RPC specification, but isn't supported by all clients and servers; see \url{http://ontosys.com/xml-rpc/extensions.php} for a description. +The \var{use_datetime} flag can be used to cause date/time values to be +presented as \class{\refmodule{datetime}.datetime} objects; this is false +by default. \class{\refmodule{datetime}.datetime}, +\class{\refmodule{datetime}.date} and \class{\refmodule{datetime}.time} +objects may be passed to calls. \class{\refmodule{datetime}.date} objects +are converted with a time of ``00:00:00''. +\class{\refmodule{datetime}.time} objects are converted using today's date. Both the HTTP and HTTPS transports support the URL syntax extension for HTTP Basic Authentication: \code{http://user:pass at host:port/path}. The @@ -62,8 +69,11 @@ elements. Arrays are returned as lists} \lineii{structures}{A Python dictionary. Keys must be strings, values may be any conformable type.} - \lineii{dates}{in seconds since the epoch; pass in an instance of the - \class{DateTime} wrapper class} + \lineii{dates}{in seconds since the epoch (pass in an instance of the + \class{DateTime} class) or a + \class{\refmodule{datetime}.datetime}, + \class{\refmodule{datetime}.date} or + \class{\refmodule{datetime}.time} instance} \lineii{binary data}{pass in an instance of the \class{Binary} wrapper class} \end{tableii} @@ -87,6 +97,7 @@ \class{Server} is retained as an alias for \class{ServerProxy} for backwards compatibility. New code should use \class{ServerProxy}. +\versionchanged[The \var{use_datetime} flag was added]{2.5} \end{classdesc} @@ -96,7 +107,7 @@ client software in several languages. Contains pretty much everything an XML-RPC client developer needs to know.} \seetitle[http://xmlrpc-c.sourceforge.net/hacks.php] - {XML-RPC-Hacks page}{Extensions for various open-source + {XML-RPC Hacks page}{Extensions for various open-source libraries to support introspection and multicall.} \end{seealso} @@ -149,7 +160,8 @@ Introspection methods are currently supported by servers written in PHP, C and Microsoft .NET. Partial introspection support is included in recent updates to UserLand Frontier. Introspection support for -Perl, Python and Java is available at the XML-RPC Hacks page. +Perl, Python and Java is available at the \ulink{XML-RPC +Hacks}{http://xmlrpc-c.sourceforge.net/hacks.php} page. \subsection{Boolean Objects \label{boolean-objects}} @@ -170,21 +182,23 @@ \subsection{DateTime Objects \label{datetime-objects}} -This class may be initialized with seconds since the epoch, a -time tuple, or an ISO 8601 time/date string. It has the following -methods, supported mainly for internal use by the -marshalling/unmarshalling code: +This class may be initialized with seconds since the epoch, a time tuple, an +ISO 8601 time/date string, or a {}\class{\refmodule{datetime}.datetime}, +{}\class{\refmodule{datetime}.date} or {}\class{\refmodule{datetime}.time} +instance. It has the following methods, supported mainly for internal use +by the marshalling/unmarshalling code: \begin{methoddesc}{decode}{string} Accept a string as the instance's new time value. \end{methoddesc} \begin{methoddesc}{encode}{out} -Write the XML-RPC encoding of this DateTime item to the out stream object. +Write the XML-RPC encoding of this \class{DateTime} item to the +\var{out} stream object. \end{methoddesc} It also supports certain of Python's built-in operators through -\method{__cmp__} and \method{__repr__} methods. +\method{__cmp__()} and \method{__repr__()} methods. \subsection{Binary Objects \label{binary-objects}} @@ -296,7 +310,6 @@ \begin{funcdesc}{dumps}{params\optional{, methodname\optional{, methodresponse\optional{, encoding\optional{, allow_none}}}}} - Convert \var{params} into an XML-RPC request. or into a response if \var{methodresponse} is true. \var{params} can be either a tuple of arguments or an instance of the @@ -308,12 +321,21 @@ provide a true value for \var{allow_none}. \end{funcdesc} -\begin{funcdesc}{loads}{data} +\begin{funcdesc}{loads}{data\optional{, use_datetime}} Convert an XML-RPC request or response into Python objects, a \code{(\var{params}, \var{methodname})}. \var{params} is a tuple of argument; \var{methodname} is a string, or \code{None} if no method name is present in the packet. If the XML-RPC packet represents a fault condition, this function will raise a \exception{Fault} exception. +The \var{use_datetime} flag can be used to cause date/time values to be +presented as \class{\refmodule{datetime}.datetime} objects; this is false +by default. +Note that even if you call an XML-RPC method with +\class{\refmodule{datetime}.date} or \class{\refmodule{datetime}.time} +objects, they are converted to \class{DateTime} objects internally, so only +{}\class{\refmodule{datetime}.datetime} objects will be returned. + +\versionchanged[The \var{use_datetime} flag was added]{2.5} \end{funcdesc} From montanaro at users.sourceforge.net Sat May 14 22:54:19 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Sat, 14 May 2005 13:54:19 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_xmlrpc.py,1.8,1.9 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9826/Lib/test Modified Files: test_xmlrpc.py Log Message: Add better datetime support to xmlrpclib module. Closes patch #1120353. Index: test_xmlrpc.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_xmlrpc.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- test_xmlrpc.py 8 Apr 2005 18:00:59 -0000 1.8 +++ test_xmlrpc.py 14 May 2005 20:54:16 -0000 1.9 @@ -34,16 +34,48 @@ xmlrpclib.loads(xmlrpclib.dumps((alist,)))[0][0]) def test_dump_bare_datetime(self): - # This checks that an unwrapped datetime object can be handled - # by the marshalling code. This can't be done via - # test_dump_load() since the unmarshaller doesn't produce base - # datetime instances. + # This checks that an unwrapped datetime.date object can be handled + # by the marshalling code. This can't be done via test_dump_load() + # since with use_datetime set to 1 the unmarshaller would create + # datetime objects for the 'datetime[123]' keys as well dt = datetime.datetime(2005, 02, 10, 11, 41, 23) s = xmlrpclib.dumps((dt,)) - r, m = xmlrpclib.loads(s) - self.assertEquals(r, (xmlrpclib.DateTime('20050210T11:41:23'),)) + (newdt,), m = xmlrpclib.loads(s, use_datetime=1) + self.assertEquals(newdt, dt) + self.assertEquals(m, None) + + (newdt,), m = xmlrpclib.loads(s, use_datetime=0) + self.assertEquals(newdt, xmlrpclib.DateTime('20050210T11:41:23')) + + def test_dump_bare_date(self): + # This checks that an unwrapped datetime.date object can be handled + # by the marshalling code. This can't be done via test_dump_load() + # since the unmarshaller produces a datetime object + d = datetime.datetime(2005, 02, 10, 11, 41, 23).date() + s = xmlrpclib.dumps((d,)) + (newd,), m = xmlrpclib.loads(s, use_datetime=1) + self.assertEquals(newd.date(), d) + self.assertEquals(newd.time(), datetime.time(0, 0, 0)) + self.assertEquals(m, None) + + (newdt,), m = xmlrpclib.loads(s, use_datetime=0) + self.assertEquals(newdt, xmlrpclib.DateTime('20050210T00:00:00')) + + def test_dump_bare_time(self): + # This checks that an unwrapped datetime.time object can be handled + # by the marshalling code. This can't be done via test_dump_load() + # since the unmarshaller produces a datetime object + t = datetime.datetime(2005, 02, 10, 11, 41, 23).time() + s = xmlrpclib.dumps((t,)) + (newt,), m = xmlrpclib.loads(s, use_datetime=1) + today = datetime.datetime.now().date().strftime("%Y%m%d") + self.assertEquals(newt.time(), t) + self.assertEquals(newt.date(), datetime.datetime.now().date()) self.assertEquals(m, None) + (newdt,), m = xmlrpclib.loads(s, use_datetime=0) + self.assertEquals(newdt, xmlrpclib.DateTime('%sT11:41:23'%today)) + def test_dump_big_long(self): self.assertRaises(OverflowError, xmlrpclib.dumps, (2L**99,)) From montanaro at users.sourceforge.net Sat May 14 22:54:18 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Sat, 14 May 2005 13:54:18 -0700 Subject: [Python-checkins] python/dist/src/Lib xmlrpclib.py,1.40,1.41 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9826/Lib Modified Files: xmlrpclib.py Log Message: Add better datetime support to xmlrpclib module. Closes patch #1120353. Index: xmlrpclib.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/xmlrpclib.py,v retrieving revision 1.40 retrieving revision 1.41 diff -u -d -r1.40 -r1.41 --- xmlrpclib.py 11 Feb 2005 17:59:03 -0000 1.40 +++ xmlrpclib.py 14 May 2005 20:54:15 -0000 1.41 @@ -357,7 +357,14 @@ if datetime and isinstance(value, datetime.datetime): self.value = value.strftime("%Y%m%dT%H:%M:%S") return - elif not isinstance(value, (TupleType, time.struct_time)): + if datetime and isinstance(value, datetime.date): + self.value = value.strftime("%Y%m%dT%H:%M:%S") + return + if datetime and isinstance(value, datetime.time): + today = datetime.datetime.now().strftime("%Y%m%d") + self.value = value.strftime(today+"T%H:%M:%S") + return + if not isinstance(value, (TupleType, time.struct_time)): if value == 0: value = time.time() value = time.localtime(value) @@ -394,6 +401,10 @@ value.decode(data) return value +def _datetime_type(data): + t = time.strptime(data, "%Y%m%dT%H:%M:%S") + return datetime.datetime(*tuple(t)[:6]) + ## # Wrapper for binary data. This can be used to transport any kind # of binary data over XML-RPC, using BASE64 encoding. @@ -714,6 +725,19 @@ write("\n") dispatch[datetime.datetime] = dump_datetime + def dump_date(self, value, write): + write("") + write(value.strftime("%Y%m%dT00:00:00")) + write("\n") + dispatch[datetime.date] = dump_date + + def dump_time(self, value, write): + write("") + write(datetime.datetime.now().date().strftime("%Y%m%dT")) + write(value.strftime("%H:%M:%S")) + write("\n") + dispatch[datetime.time] = dump_time + def dump_instance(self, value, write): # check for special wrappers if value.__class__ in WRAPPERS: @@ -742,7 +766,7 @@ # and again, if you don't understand what's going on in here, # that's perfectly ok. - def __init__(self): + def __init__(self, use_datetime=0): self._type = None self._stack = [] self._marks = [] @@ -750,6 +774,9 @@ self._methodname = None self._encoding = "utf-8" self.append = self._stack.append + self._use_datetime = use_datetime + if use_datetime and not datetime: + raise ValueError, "the datetime module is not available" def close(self): # return response tuple and target method @@ -867,6 +894,8 @@ def end_dateTime(self, data): value = DateTime() value.decode(data) + if self._use_datetime: + value = _datetime_type(data) self.append(value) dispatch["dateTime.iso8601"] = end_dateTime @@ -968,17 +997,23 @@ # # return A (parser, unmarshaller) tuple. -def getparser(): +def getparser(use_datetime=0): """getparser() -> parser, unmarshaller Create an instance of the fastest available parser, and attach it to an unmarshalling object. Return both objects. """ + if use_datetime and not datetime: + raise ValueError, "the datetime module is not available" if FastParser and FastUnmarshaller: - target = FastUnmarshaller(True, False, _binary, _datetime, Fault) + if use_datetime: + mkdatetime = _datetime_type + else: + mkdatetime = _datetime + target = FastUnmarshaller(True, False, _binary, mkdatetime, Fault) parser = FastParser(target) else: - target = Unmarshaller() + target = Unmarshaller(use_datetime=use_datetime) if FastParser: parser = FastParser(target) elif SgmlopParser: @@ -1081,7 +1116,7 @@ # (None if not present). # @see Fault -def loads(data): +def loads(data, use_datetime=0): """data -> unmarshalled data, method name Convert an XML-RPC packet to unmarshalled data plus a method @@ -1090,7 +1125,7 @@ If the XML-RPC packet represents a fault condition, this function raises a Fault exception. """ - p, u = getparser() + p, u = getparser(use_datetime=use_datetime) p.feed(data) p.close() return u.close(), u.getmethodname() @@ -1122,6 +1157,9 @@ # client identifier (may be overridden) user_agent = "xmlrpclib.py/%s (by www.pythonware.com)" % __version__ + def __init__(self, use_datetime=0): + self._use_datetime = use_datetime + ## # Send a complete request, and parse the response. # @@ -1168,7 +1206,7 @@ def getparser(self): # get parser and unmarshaller - return getparser() + return getparser(use_datetime=self._use_datetime) ## # Get authorization info from host parameter @@ -1362,7 +1400,7 @@ """ def __init__(self, uri, transport=None, encoding=None, verbose=0, - allow_none=0): + allow_none=0, use_datetime=0): # establish a "logical" server connection # get the url @@ -1376,9 +1414,9 @@ if transport is None: if type == "https": - transport = SafeTransport() + transport = SafeTransport(use_datetime=use_datetime) else: - transport = Transport() + transport = Transport(use_datetime=use_datetime) self.__transport = transport self.__encoding = encoding From goodger at users.sourceforge.net Sun May 15 02:45:44 2005 From: goodger at users.sourceforge.net (goodger@users.sourceforge.net) Date: Sat, 14 May 2005 17:45:44 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.8,1.9 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16848 Modified Files: pep-0343.txt Log Message: fixed typos Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- pep-0343.txt 14 May 2005 17:36:15 -0000 1.8 +++ pep-0343.txt 15 May 2005 00:45:42 -0000 1.9 @@ -21,7 +21,7 @@ using generators as block templates, adding exception handling and finalization to generators, and more. Besides praise it received a lot of opposition from people who didn't like the fact that it - was, under the covers, a (optential) looping construct. This + was, under the covers, a (potential) looping construct. This meant that break and continue in a block-statement would break or continue the block-statement, even if it was used as a non-looping resource management tool. @@ -176,7 +176,7 @@ If the "as VAR" part of the syntax is omitted, the "VAR =" part of the translation is omitted (but abc.__enter__() is still called). - The calling convention for abc.__exit__() is: as follows. If the + The calling convention for abc.__exit__() is as follows. If the finally-suite was reached through normal completion of BLOCK or through a non-local goto (a break, continue or return statement in BLOCK), abc.__exit__() is called with three None arguments. If @@ -190,9 +190,9 @@ example must commit or roll back the transaction depending on whether an exception occurred or not. Rather than just having a boolean flag indicating whether an exception occurred, we pass the - complete exception information, for the benefor of an + complete exception information, for the benefit of an exception-logging facility for example. Relying on sys.exc_info() - to get att the exception information was rejected; sys.exc_info() + to get at the exception information was rejected; sys.exc_info() has very complex semantics and it is perfectly possible that it returns the exception information for an exception that was caught ages ago. It was also proposed to add an additional boolean to From arigo at users.sourceforge.net Sun May 15 15:29:28 2005 From: arigo at users.sourceforge.net (arigo@users.sourceforge.net) Date: Sun, 15 May 2005 06:29:28 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_mutants.py, 1.7, 1.8 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16135/Lib/test Modified Files: test_mutants.py Log Message: This test relied on short-circuiting details of dictobject.py to avoid crashing, and indirectly on the fact that hash codes in random.randrange(1000000000) were very unlikely to exhibit collisions. To see the problem, replace this number with 500 and observe the crash on either del target[key] or del keys[i]. The fix prevents recursive mutation, just as in the key insertion case. Index: test_mutants.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_mutants.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- test_mutants.py 23 Jul 2002 19:03:57 -0000 1.7 +++ test_mutants.py 15 May 2005 13:29:26 -0000 1.8 @@ -69,14 +69,12 @@ elif keys: # Delete a key at random. + mutate = 0 # disable mutation until key deleted i = random.randrange(len(keys)) key = keys[i] del target[key] - # CAUTION: don't use keys.remove(key) here. Or do . The - # point is that .remove() would trigger more comparisons, and so - # also more calls to this routine. We're mutating often enough - # without that. del keys[i] + mutate = 1 # A horrid class that triggers random mutations of dict1 and dict2 when # instances are compared. From arigo at users.sourceforge.net Sun May 15 17:32:11 2005 From: arigo at users.sourceforge.net (arigo@users.sourceforge.net) Date: Sun, 15 May 2005 08:32:11 -0700 Subject: [Python-checkins] python/dist/src/Objects typeobject.c, 2.265, 2.266 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4545 Modified Files: typeobject.c Log Message: Fixed a quite misleading comment: a "not" should not have been there. Index: typeobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/typeobject.c,v retrieving revision 2.265 retrieving revision 2.266 diff -u -d -r2.265 -r2.266 --- typeobject.c 3 Mar 2005 16:45:19 -0000 2.265 +++ typeobject.c 15 May 2005 15:32:08 -0000 2.266 @@ -5648,7 +5648,7 @@ return self; } if (su->ob_type != &PySuper_Type) - /* If su is not an instance of a subclass of super, + /* If su is an instance of a (strict) subclass of super, call its type */ return PyObject_CallFunction((PyObject *)su->ob_type, "OO", su->type, obj); From ping at users.sourceforge.net Sun May 15 21:30:41 2005 From: ping at users.sourceforge.net (ping@users.sourceforge.net) Date: Sun, 15 May 2005 12:30:41 -0700 Subject: [Python-checkins] python/nondist/peps pep-0344.txt,NONE,1.1 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18411 Added Files: pep-0344.txt Log Message: Add PEP 344: Exception Chaining and the Traceback Attribute. --- NEW FILE: pep-0344.txt --- PEP: 344 Title: Exception Chaining and the Traceback Attribute Version: $Revision: 1.1 $ Last-Modified: $Date: 2005/05/15 19:30:38 $ Author: Ka-Ping Yee Status: Active Type: Standards Track Content-Type: text/plain Created: 12-May-2005 Python-Version: 2.5 Abstract This PEP proposes two standard attributes on exception instances: the 'context' attribute for chained exceptions, and the 'traceback' attribute for the traceback. Motivation Sometimes, during the handling of one exception (exception A), another exception (exception B) can occur. In today's Python (version 2.4), if this happens, exception B is propagated outward and exception A is lost. But in order to debug the problem, it is useful to know about both exceptions. The 'context' attribute retains this information. In today's Python implementation, exceptions are composed of three parts: the type, the value, and the traceback. The 'sys' module, exposes the current exception in three parallel variables, exc_type, exc_value, and exc_traceback, the sys.exc_info() function returns a tuple of these three parts, and the 'raise' statement has a three-argument form accepting these three parts. Manipulating exceptions often requires passing these three things in parallel, which can be tedious and error-prone. Additionally, the 'except' statement can only provide access to the value, not the traceback. Adding the 'traceback' attribute to exception values makes all the exception information accessible from a single place. The reason both of these attributes are presented together in one proposal is that the 'traceback' attribute provides convenient access to the traceback on chained exceptions. History Raymond Hettinger [1] raised the issue of masked exceptions on Python-Dev in January 2003 and proposed a PyErr_FormatAppend() function that C modules could use to augment the currently active exception with more information. Brett Cannon [2] brought up chained exceptions again in June 2003 and a long discussion followed. Other suggested attribute names include 'cause', 'antecedent', 'reason', 'original', 'chain', 'chainedexc', 'exc_chain', 'excprev', 'previous', and 'precursor'. This PEP suggests 'context' because the intended meaning is more specific than temporal precedence and less specific than causation: an exception occurs in the *context* of handling another exception. Greg Ewing [3] identified the case of an exception occuring in a 'finally' block during unwinding triggered by an original exception, as distinct from the case of an exception occuring in an 'except' block that is handling the original exception. This PEP handles both situations in the same way; it is assumed to be unnecessary to add mechanisms for distinguishing them since the programmer can tell them apart by reading the traceback. Greg Ewing [4] and Guido van Rossum [5], and probably others, have previously mentioned adding the traceback attribute to Exception instances. This is noted in PEP 3000. This PEP was motivated by yet another recent Python-Dev reposting of the same ideas [6] [7]. *** Add rationale for: choice of name, handling both finally/except, handling finally/except the same, displaying outermost last. *** Compare to Java: exceptions are lost in catch or finally clauses. *** Compare to Ruby: exceptions are lost just like Java. *** Compare to C#: *** Compare to E: *** COmpare to Perl: RFC 88 a mess. *** Note http://pclt.cis.yale.edu/pclt/exceptions.htm Exception Chaining Here is an example to illustrate the 'context' attribute. def compute(a, b): try: a/b except Exception, exc: log(exc) def log(exc): file = open('logfile.txt') # oops, forgot the 'w' print >>file, exc file.close() Calling compute(0, 0) causes a ZeroDivisionError. The compute() function catches this exception and calls log(exc), but the log() function also raises an exception when it tries to write to a file that wasn't opened for writing. In today's Python, the caller of compute() gets thrown an IOError. The ZeroDivisionError is lost. With the proposed change, the instance of IOError has an additional 'context' attribute that retains the ZeroDivisionError. The following more elaborate example demonstrates the handling of a mix of 'finally' and 'except' clauses: def main(filename): file = open(filename) # oops, forgot the 'w' try: try: compute() except Exception, exc: log(file, exc) finally: file.clos() # oops, misspelled 'close' def compute(): 1/0 def log(file, exc): try: print >>file, exc # oops, file is not writable except: display(exc) def display(exc): print ex # oops, misspelled 'exc' Calling main() with the name of an existing file will trigger four exceptions. The ultimate result will be an AttributeError due to the misspelling of 'clos', which has a context attribute pointing to a NameError due to the misspelling of 'ex', which has a context attribute pointing to an IOError due to the file being read-only, which has a context attribute pointing to a ZeroDivisionError, which has a context attribute of None. The proposed semantics are as follows: 1. Each thread has an exception context initially set to None. 2. Whenever an exception is raised, if the exception instance does not already have a 'context' attribute, the interpreter sets it equal to the thread's exception context. 3. Immediately after an exception is raised, the thread's exception context is set to the exception. 4. Whenever the interpreter exits an 'except' block by reaching the end or executing a 'return', 'yield', 'continue', or 'break' statement, the thread's exception context is set to None. Traceback Attribute The following example illustrates the 'traceback' attribute. def do_logged(file, work): try: work() except Exception, exc: write_exception(file, exc) raise exc from traceback import format_tb def write_exception(file, exc): ... type = exc.__class__ message = str(exc) lines = format_tb(exc.traceback) file.write(... type ... message ... lines ...) ... In today's Python, the do_logged() function would have to extract the traceback from sys.exc_traceback or sys.exc_info()[2] and pass both the value and the traceback to write_exception(). With the proposed change, write_exception() simply gets one argument and obtains the exception using the 'traceback' attribute. The proposed semantics are as follows: 1. Whenever an exception is raised, if the exception instance does not already have a 'traceback' attribute, the interpreter sets it to the newly raised traceback. Enhanced Reporting The default exception handler will be modified to report chained exceptions. In keeping with the chronological order of tracebacks, the most recently raised exception is displayed last. The display begins with the description of the innermost exception and backs up the chain to the outermost exception. The tracebacks are formatted as usual, with the following line between tracebacks: During handling of the above exception, another exception occurred: In the 'traceback' module, the format_exception, print_exception, print_exc, and print_last functions will be updated to accept an optional 'context' argument, True by default. When this argument is True, these functions will format or display the entire chain of exceptions as just described. When it is False, these functions will format or display only the outermost exception. The 'cgitb' module will be updated to display the entire chain of exceptions. C API To keep things simpler, the PyErr_Set* calls for setting exceptions will not set the 'context' attribute on exceptions. Guido van Rossum has expressed qualms with making such changes to PyErr_Set* [8]. PyErr_NormalizeException will always set the 'traceback' attribute to its 'tb' argument and the 'context' attribute to None. A new API function, PyErr_SetContext(context), will help C programmers provide chained exception information. This function will first normalize the current exception so it is an instance, then set its 'context' attribute. Compatibility Chained exceptions expose their outermost type so that they will continue to match the same 'except' clauses as they do now. The proposed changes should not break any code except for code that currently sets and relies on the values of attributes named 'context' or 'traceback' on exceptions. As of 2005-05-12, the Python standard library contains no mention of such attributes. Open Issues Walter Dörwald [9] expressed a desire to attach extra information to an exception during its upward propagation, without changing its type. This could be a useful feature, but it is not addressed by this PEP. It could conceivably be addressed by a separate PEP establishing conventions for other informational attributes on exceptions. It is not clear whether the 'context' feature proposed here would be sufficient to cover all the use cases that Raymond Hettinger [1] originally had in mind. The exception context is lost when a 'yield' statement is executed; resuming the frame after the 'yield' does not restore the context. This is not a new problem, as demonstrated by the following example: >>> def gen(): ... try: ... 1/0 ... except: ... yield 3 ... raise ... >>> g = gen() >>> g.next() 3 >>> g.next() TypeError: exceptions must be classes, instances, or strings (deprecated), not NoneType For now, addressing this problem is out of the scope of this PEP. Possible Future Compatible Changes These changes are consistent with the appearance of exceptions as a single object rather than a triple at the interpreter level. - Deprecating sys.exc_type, sys.exc_value, sys.exc_traceback, and sys.exc_info() in favour of a single member, sys.exception. - Deprecating sys.last_type, sys.last_value, and sys.last_traceback in favour of a single member, sys.last_exception. - Deprecating the three-argument form of the 'raise' statement in favour of the one-argument form. - Upgrading cgitb.html() to accept a single value as its first argument as an alternative to a (type, value, traceback) tuple. Possible Future Incompatible Changes These changes might be worth considering for Python 3000. - Removing sys.exc_type, sys.exc_value, sys.exc_traceback, and sys.exc_info(). - Removing sys.last_type, sys.last_value, and sys.last_traceback. - Replacing the three-argument sys.excepthook with a one-argument API, and changing the 'cgitb' module to match. - Removing the three-argument form of the 'raise' statement. - Upgrading traceback.print_exception to accept an 'exception' argument instead of the type, value, and traceback arguments. Acknowledgements Brett Cannon, Greg Ewing, Guido van Rossum, Jeremy Hylton, Phillip J. Eby, Raymond Hettinger, Walter Dörwald, and others. References [1] Raymond Hettinger, "Idea for avoiding exception masking" http://mail.python.org/pipermail/python-dev/2003-January/032492.html [2] Brett Cannon explains chained exceptions http://mail.python.org/pipermail/python-dev/2003-June/036063.html [3] Greg Ewing points out masking caused by exceptions during finally http://mail.python.org/pipermail/python-dev/2003-June/036290.html [4] Greg Ewing suggests storing the traceback in the exception object http://mail.python.org/pipermail/python-dev/2003-June/036092.html [5] Guido van Rossum mentions exceptions having a traceback attribute http://mail.python.org/pipermail/python-dev/2005-April/053060.html [6] Ka-Ping Yee, "Tidier Exceptions" http://mail.python.org/pipermail/python-dev/2005-May/053671.html [7] Ka-Ping Yee, "Chained Exceptions" http://mail.python.org/pipermail/python-dev/2005-May/053672.html [8] Guido van Rossum discusses automatic chaining in PyErr_Set* http://mail.python.org/pipermail/python-dev/2003-June/036180.html [9] Walter Dörwald suggests wrapping exceptions to add details http://mail.python.org/pipermail/python-dev/2003-June/036148.html Copyright This document has been placed in the public domain. From ping at users.sourceforge.net Sun May 15 21:49:20 2005 From: ping at users.sourceforge.net (ping@users.sourceforge.net) Date: Sun, 15 May 2005 12:49:20 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.306, 1.307 pep-0344.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21258 Modified Files: pep-0000.txt pep-0344.txt Log Message: Add PEP 344. Shorten title to make it fit in PEP 0. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.306 retrieving revision 1.307 diff -u -d -r1.306 -r1.307 --- pep-0000.txt 14 May 2005 00:08:19 -0000 1.306 +++ pep-0000.txt 15 May 2005 19:49:01 -0000 1.307 @@ -121,6 +121,7 @@ S 341 Unifying try-except and try-finally Birkenfeld S 342 Enhanced Iterators GvR S 343 Anonymous Block Redux GvR + S 344 Exception Chaining and Embedded Tracebacks Yee S 754 IEEE 754 Floating Point Special Values Warnes Finished PEPs (done, implemented in CVS) @@ -382,6 +383,7 @@ S 341 Unifying try-except and try-finally Birkenfeld S 342 Enhanced Iterators GvR S 343 Anonymous Block Redux GvR + S 344 Exception Chaining and Embedded Tracebacks Yee SR 666 Reject Foolish Indentation Creighton S 754 IEEE 754 Floating Point Special Values Warnes I 3000 Python 3.0 Plans Kuchling, Cannon Index: pep-0344.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0344.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0344.txt 15 May 2005 19:30:38 -0000 1.1 +++ pep-0344.txt 15 May 2005 19:49:01 -0000 1.2 @@ -1,5 +1,5 @@ PEP: 344 -Title: Exception Chaining and the Traceback Attribute +Title: Exception Chaining and Embedded Tracebacks Version: $Revision$ Last-Modified: $Date$ Author: Ka-Ping Yee From ping at users.sourceforge.net Mon May 16 01:29:58 2005 From: ping at users.sourceforge.net (ping@users.sourceforge.net) Date: Sun, 15 May 2005 16:29:58 -0700 Subject: [Python-checkins] python/nondist/peps pep-0344.txt,1.2,1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30968 Modified Files: pep-0344.txt Log Message: Add __cause__ and language comparisons. Index: pep-0344.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0344.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0344.txt 15 May 2005 19:49:01 -0000 1.2 +++ pep-0344.txt 15 May 2005 23:29:56 -0000 1.3 @@ -12,19 +12,25 @@ Abstract - This PEP proposes two standard attributes on exception instances: - the 'context' attribute for chained exceptions, and the 'traceback' - attribute for the traceback. + This PEP proposes three standard attributes on exception instances: + the '__context__' attribute for implicitly chained exceptions, the + '__cause__' attribute for explicitly chained exceptions, and the + '__traceback__' attribute for the traceback. Motivation - Sometimes, during the handling of one exception (exception A), - another exception (exception B) can occur. In today's Python + During the handling of one exception (exception A), it is possible + that another exception (exception B) may occur. In today's Python (version 2.4), if this happens, exception B is propagated outward - and exception A is lost. But in order to debug the problem, it is - useful to know about both exceptions. The 'context' attribute - retains this information. + and exception A is lost. In order to debug the problem, it is + useful to know about both exceptions. The '__context__' attribute + retains this information automatically. + + Sometimes it can be useful for an exception handler to intentionally + re-raise an exception, either to provide extra information or to + translate an exception to another type. The '__cause__' attribute + provides an explicit way to record the direct cause of an exception. In today's Python implementation, exceptions are composed of three parts: the type, the value, and the traceback. The 'sys' module, @@ -35,12 +41,8 @@ exceptions often requires passing these three things in parallel, which can be tedious and error-prone. Additionally, the 'except' statement can only provide access to the value, not the traceback. - Adding the 'traceback' attribute to exception values makes all the - exception information accessible from a single place. - - The reason both of these attributes are presented together in one - proposal is that the 'traceback' attribute provides convenient - access to the traceback on chained exceptions. + Adding the '__traceback__' attribute to exception values makes all + the exception information accessible from a single place. History @@ -48,51 +50,83 @@ Raymond Hettinger [1] raised the issue of masked exceptions on Python-Dev in January 2003 and proposed a PyErr_FormatAppend() function that C modules could use to augment the currently active - exception with more information. - - Brett Cannon [2] brought up chained exceptions again in June 2003 - and a long discussion followed. Other suggested attribute names - include 'cause', 'antecedent', 'reason', 'original', 'chain', - 'chainedexc', 'exc_chain', 'excprev', 'previous', and 'precursor'. - This PEP suggests 'context' because the intended meaning is more - specific than temporal precedence and less specific than causation: - an exception occurs in the *context* of handling another exception. + exception with more information. Brett Cannon [2] brought up + chained exceptions again in June 2003, prompting a long discussion. Greg Ewing [3] identified the case of an exception occuring in a 'finally' block during unwinding triggered by an original exception, as distinct from the case of an exception occuring in an 'except' - block that is handling the original exception. This PEP handles - both situations in the same way; it is assumed to be unnecessary to - add mechanisms for distinguishing them since the programmer can tell - them apart by reading the traceback. + block that is handling the original exception. Greg Ewing [4] and Guido van Rossum [5], and probably others, have - previously mentioned adding the traceback attribute to Exception + previously mentioned adding a traceback attribute to Exception instances. This is noted in PEP 3000. This PEP was motivated by yet another recent Python-Dev reposting of the same ideas [6] [7]. -*** Add rationale for: choice of name, handling both finally/except, - handling finally/except the same, displaying outermost last. +Rationale -*** Compare to Java: exceptions are lost in catch or finally clauses. + This PEP distinguishes implicit chaining from explicit chaining of + exceptions because the unexpected raising of a secondary exception + and the intentional translation of an exception are two different + situations deserving quite different interpretations. -*** Compare to Ruby: exceptions are lost just like Java. + Several attribute names for chained exceptions have been suggested + on Python-Dev [2], including 'cause', 'antecedent', 'reason', + 'original', 'chain', 'chainedexc', 'exc_chain', 'excprev', + 'previous', and 'precursor'. For an explicitly chained exception, + this PEP suggests '__cause__' because of its specific meaning. For + an implicitly chained exception, this PEP proposes the name + '__context__' because the intended meaning is more specific than + temporal precedence but less specific than causation: an exception + occurs in the context of handling another exception. + + This PEP suggests names with leading and trailing double-underscores + for '__context__' and '__traceback__' because the attributes are set + by the Python VM. The name '__cause__' is not set automatically by + the VM, but it seems confusing and collision-prone to use 'cause'. -*** Compare to C#: + This PEP handles exceptions that occur during 'except' blocks and + 'finally' blocks in the same way. Reading the traceback makes it + clear where the exceptions occurred, so additional mechanisms for + distinguishing the two cases would only add unnecessary complexity. -*** Compare to E: + This PEP proposes that the outermost exception object (the one + exposed for matching by 'except' clauses) be the most recently + raised exception for compatibility with current behaviour. -*** COmpare to Perl: RFC 88 a mess. + This PEP proposes that tracebacks display the outermost exception + last, because it would be consistent with the chronological order + of tracebacks (from oldest to most recent frame) and because the + actual thrown exception is easier to find on the last line. -*** Note http://pclt.cis.yale.edu/pclt/exceptions.htm + Java and Ruby both discard the current exception when an exception + occurs in a 'catch'/'rescue' or 'finally'/'ensure' clause. Perl 5 + lacks built-in structured exception handling. For Perl 6, RFC 88 + proposes an exception mechanism that retains all the chained + exceptions in an array named @@. In that RFC, the most recently + raised exception is exposed for matching, as in this PEP; also, + arbitrary expressions (possibly involving @@) can be evaluated for + exception matching. + Exceptions in C# contain a read-only 'InnerException' property that + may point to another exception. This property is not set by the VM + automatically. Instead, the constructors of exception objects all + accept an optional 'innerException' argument to explicitly set this + property. The C# documentation says "an exception that is thrown as + a direct result of a previous exception should include a reference + to the previous exception in the InnerException property." -Exception Chaining + The reason all three of these attributes are presented together in + one proposal is that the '__traceback__' attribute provides + convenient access to the traceback on chained exceptions. - Here is an example to illustrate the 'context' attribute. + +Implicit Exception Chaining + + Here is an example to illustrate the '__context__' attribute. def compute(a, b): try: @@ -112,11 +146,11 @@ In today's Python, the caller of compute() gets thrown an IOError. The ZeroDivisionError is lost. With the proposed change, the - instance of IOError has an additional 'context' attribute that + instance of IOError has an additional '__context__' attribute that retains the ZeroDivisionError. The following more elaborate example demonstrates the handling of a - mix of 'finally' and 'except' clauses: + mixture of 'finally' and 'except' clauses: def main(filename): file = open(filename) # oops, forgot the 'w' @@ -142,19 +176,18 @@ Calling main() with the name of an existing file will trigger four exceptions. The ultimate result will be an AttributeError due to - the misspelling of 'clos', which has a context attribute pointing to - a NameError due to the misspelling of 'ex', which has a context - attribute pointing to an IOError due to the file being read-only, - which has a context attribute pointing to a ZeroDivisionError, which - has a context attribute of None. + the misspelling of 'clos', whose __context__ points to a NameError + due to the misspelling of 'ex', whose __context__ points to an + IOError due to the file being read-only, whose __context__ points to + a ZeroDivisionError, whose __context__ attribute is None. The proposed semantics are as follows: 1. Each thread has an exception context initially set to None. 2. Whenever an exception is raised, if the exception instance does - not already have a 'context' attribute, the interpreter sets it - equal to the thread's exception context. + not already have a '__context__' attribute, the interpreter sets + it equal to the thread's exception context. 3. Immediately after an exception is raised, the thread's exception context is set to the exception. @@ -164,9 +197,33 @@ statement, the thread's exception context is set to None. +Explicit Exception Chaining + + The '__cause__' attribute on exception objects is always initialized + to None. It is set by calling the 'setcause' method, a new method + defined on the base Exception class. For convenience, this method + returns the exception itself. + + In the following example, a database provides implementations for a + few different kinds of storage, with file storage as one kind. The + database designer wants errors to propagate as DatabaseError objects + so that the client doesn't have to be aware of the storage-specific + details, but doesn't want to lose the underlying error information. + + class DatabaseError(StandardError): + pass + + class FileDatabase(Database): + def __init__(self, filename): + try: + self.file = open(filename) + except IOError, exc: + raise DatabaseError('failed to open').setcause(exc) + + Traceback Attribute - The following example illustrates the 'traceback' attribute. + The following example illustrates the '__traceback__' attribute. def do_logged(file, work): try: @@ -181,7 +238,7 @@ ... type = exc.__class__ message = str(exc) - lines = format_tb(exc.traceback) + lines = format_tb(exc.__traceback__) file.write(... type ... message ... lines ...) ... @@ -189,63 +246,84 @@ the traceback from sys.exc_traceback or sys.exc_info()[2] and pass both the value and the traceback to write_exception(). With the proposed change, write_exception() simply gets one argument and - obtains the exception using the 'traceback' attribute. + obtains the exception using the '__traceback__' attribute. The proposed semantics are as follows: - 1. Whenever an exception is raised, if the exception instance does - not already have a 'traceback' attribute, the interpreter sets - it to the newly raised traceback. + 1. Whenever an exception is caught, if the exception instance does + not already have a '__traceback__' attribute, the interpreter + sets it to the newly caught traceback. Enhanced Reporting The default exception handler will be modified to report chained - exceptions. In keeping with the chronological order of tracebacks, - the most recently raised exception is displayed last. The display - begins with the description of the innermost exception and backs - up the chain to the outermost exception. The tracebacks are - formatted as usual, with the following line between tracebacks: + exceptions. The chain of exceptions is traversed by following the + '__cause__' and '__context__' attributes, with '__cause__' taking + priority. In keeping with the chronological order of tracebacks, + the most recently raised exception is displayed last; that is, the + display begins with the description of the innermost exception and + backs up the chain to the outermost exception. The tracebacks are + formatted as usual, with one of the lines: + + The above exception was the direct cause of the following exception: + + or During handling of the above exception, another exception occurred: + between tracebacks, depending whether they are linked by __cause__ + or __context__ respectively. Here is a sketch of the procedure: + + def print_chain(exc): + chain = [] + link = None + while exc: + chain.append((exc, link)) + if exc.__cause__: + exc, link = exc.__cause__, 'The above exception...' + else: + exc, link = exc.__context__, 'During handling...' + for exc, link in reversed(chain): + print_exc(exc) + print '\n' + link + '\n' + In the 'traceback' module, the format_exception, print_exception, print_exc, and print_last functions will be updated to accept an - optional 'context' argument, True by default. When this argument is + optional 'chain' argument, True by default. When this argument is True, these functions will format or display the entire chain of exceptions as just described. When it is False, these functions will format or display only the outermost exception. - The 'cgitb' module will be updated to display the entire chain of - exceptions. + The 'cgitb' module should also be updated to display the entire + chain of exceptions. C API To keep things simpler, the PyErr_Set* calls for setting exceptions - will not set the 'context' attribute on exceptions. Guido van Rossum + will not set the '__context__' attribute on exceptions. The BDFL has has expressed qualms with making such changes to PyErr_Set* [8]. PyErr_NormalizeException will always set the 'traceback' attribute - to its 'tb' argument and the 'context' attribute to None. + to its 'tb' argument and the '__context__' attribute to None. A new API function, PyErr_SetContext(context), will help C programmers provide chained exception information. This function will first normalize the current exception so it is an instance, - then set its 'context' attribute. + then set its '__context__' attribute. Compatibility - Chained exceptions expose their outermost type so that they will - continue to match the same 'except' clauses as they do now. - - The proposed changes should not break any code except for code - that currently sets and relies on the values of attributes named - 'context' or 'traceback' on exceptions. + Chained exceptions expose the type of the most recent exception, so + they will still match the same 'except' clauses as they do now. - As of 2005-05-12, the Python standard library contains no mention - of such attributes. + The proposed changes should not break any code except for code that + currently sets or depends on the values of attributes named + '__context__', '__cause__', or '__traceback__' on exception + instances. As of 2005-05-12, the Python standard library contains + no mention of such attributes. Open Issues @@ -257,13 +335,14 @@ establishing conventions for other informational attributes on exceptions. - It is not clear whether the 'context' feature proposed here would + It is not clear whether the '__context__' feature proposed here would be sufficient to cover all the use cases that Raymond Hettinger [1] originally had in mind. The exception context is lost when a 'yield' statement is executed; resuming the frame after the 'yield' does not restore the context. - This is not a new problem, as demonstrated by the following example: + Addressing this problem is out of the scope of this PEP; it is not a + new problem, as demonstrated by the following example: >>> def gen(): ... try: @@ -279,24 +358,25 @@ TypeError: exceptions must be classes, instances, or strings (deprecated), not NoneType - For now, addressing this problem is out of the scope of this PEP. - Possible Future Compatible Changes These changes are consistent with the appearance of exceptions as a single object rather than a triple at the interpreter level. - - Deprecating sys.exc_type, sys.exc_value, sys.exc_traceback, and + - If PEP 340 or PEP 343 is accepted, replace the three (type, value, + traceback) arguments to __exit__ with a single exception argument. + + - Deprecate sys.exc_type, sys.exc_value, sys.exc_traceback, and sys.exc_info() in favour of a single member, sys.exception. - - Deprecating sys.last_type, sys.last_value, and sys.last_traceback + - Deprecate sys.last_type, sys.last_value, and sys.last_traceback in favour of a single member, sys.last_exception. - - Deprecating the three-argument form of the 'raise' statement in + - Deprecate the three-argument form of the 'raise' statement in favour of the one-argument form. - - Upgrading cgitb.html() to accept a single value as its first + - Upgrade cgitb.html() to accept a single value as its first argument as an alternative to a (type, value, traceback) tuple. @@ -304,17 +384,17 @@ These changes might be worth considering for Python 3000. - - Removing sys.exc_type, sys.exc_value, sys.exc_traceback, and + - Remove sys.exc_type, sys.exc_value, sys.exc_traceback, and sys.exc_info(). - - Removing sys.last_type, sys.last_value, and sys.last_traceback. + - Remove sys.last_type, sys.last_value, and sys.last_traceback. - - Replacing the three-argument sys.excepthook with a one-argument + - Replace the three-argument sys.excepthook with a one-argument API, and changing the 'cgitb' module to match. - - Removing the three-argument form of the 'raise' statement. + - Remove the three-argument form of the 'raise' statement. - - Upgrading traceback.print_exception to accept an 'exception' + - Upgrade traceback.print_exception to accept an 'exception' argument instead of the type, value, and traceback arguments. From ping at users.sourceforge.net Mon May 16 01:53:58 2005 From: ping at users.sourceforge.net (ping@users.sourceforge.net) Date: Sun, 15 May 2005 16:53:58 -0700 Subject: [Python-checkins] python/nondist/peps pep-0344.txt,1.3,1.4 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2622 Modified Files: pep-0344.txt Log Message: Update Rationale. Index: pep-0344.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0344.txt,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- pep-0344.txt 15 May 2005 23:29:56 -0000 1.3 +++ pep-0344.txt 15 May 2005 23:53:56 -0000 1.4 @@ -102,22 +102,29 @@ of tracebacks (from oldest to most recent frame) and because the actual thrown exception is easier to find on the last line. - Java and Ruby both discard the current exception when an exception - occurs in a 'catch'/'rescue' or 'finally'/'ensure' clause. Perl 5 - lacks built-in structured exception handling. For Perl 6, RFC 88 - proposes an exception mechanism that retains all the chained - exceptions in an array named @@. In that RFC, the most recently - raised exception is exposed for matching, as in this PEP; also, - arbitrary expressions (possibly involving @@) can be evaluated for - exception matching. + To keep things simpler, the C API calls for setting an exception + will not automatically set the exception's '__context__'. Guido + van Rossum has has expressed concerns with making such changes [8]. + + As for other languages, Java and Ruby both discard the original + exception when another exception occurs in a 'catch'/'rescue' or + 'finally'/'ensure' clause. Perl 5 lacks built-in structured + exception handling. For Perl 6, RFC 88 proposes an exception + mechanism that retains chained exceptions in an array named @@. + In that RFC, the most recently raised exception is exposed for + matching, as in this PEP; also, arbitrary expressions (possibly + involving @@) can be evaluated for exception matching. Exceptions in C# contain a read-only 'InnerException' property that - may point to another exception. This property is not set by the VM - automatically. Instead, the constructors of exception objects all - accept an optional 'innerException' argument to explicitly set this - property. The C# documentation says "an exception that is thrown as - a direct result of a previous exception should include a reference - to the previous exception in the InnerException property." + may point to another exception [9]. According to its documentation, + "When an exception X is thrown as a direct result of a previous + exception Y, the InnerException property of X should contain a + reference to Y." This property is not set by the VM automatically; + rather, all exception constructors take an optional 'innerException' + argument to set it explicitly. The '__cause__' attribute fulfills + the same purpose as InnerException, but this PEP proposes adding a + single method to the base Exception class rather than extending the + constructors of all exceptions. The reason all three of these attributes are presented together in one proposal is that the '__traceback__' attribute provides @@ -202,7 +209,11 @@ The '__cause__' attribute on exception objects is always initialized to None. It is set by calling the 'setcause' method, a new method defined on the base Exception class. For convenience, this method - returns the exception itself. + returns the exception itself: + + def setcause(self, cause): + self.__cause__ = cause + return self In the following example, a database provides implementations for a few different kinds of storage, with file storage as one kind. The @@ -301,12 +312,10 @@ C API - To keep things simpler, the PyErr_Set* calls for setting exceptions - will not set the '__context__' attribute on exceptions. The BDFL has - has expressed qualms with making such changes to PyErr_Set* [8]. - - PyErr_NormalizeException will always set the 'traceback' attribute - to its 'tb' argument and the '__context__' attribute to None. + The PyErr_Set* calls for setting exceptions will not set the + '__context__' attribute on exceptions. PyErr_NormalizeException + will always set the 'traceback' attribute to its 'tb' argument and + the '__context__' and '__cause__' attributes to None. A new API function, PyErr_SetContext(context), will help C programmers provide chained exception information. This function @@ -319,25 +328,28 @@ Chained exceptions expose the type of the most recent exception, so they will still match the same 'except' clauses as they do now. - The proposed changes should not break any code except for code that - currently sets or depends on the values of attributes named - '__context__', '__cause__', or '__traceback__' on exception - instances. As of 2005-05-12, the Python standard library contains - no mention of such attributes. + The proposed changes should not break any code unless the code sets + or uses attributes named '__context__', '__cause__', 'setcause', or + '__traceback__' on exception instances. As of 2005-05-12, the + Python standard library contains no mention of such attributes. Open Issues - Walter Dörwald [9] expressed a desire to attach extra information - to an exception during its upward propagation, without changing its + Walter Dörwald [10] expressed a desire to attach extra information + to an exception during its upward propagation without changing its type. This could be a useful feature, but it is not addressed by this PEP. It could conceivably be addressed by a separate PEP establishing conventions for other informational attributes on exceptions. - It is not clear whether the '__context__' feature proposed here would - be sufficient to cover all the use cases that Raymond Hettinger [1] - originally had in mind. + It is not clear whether the '__context__' and '__cause__' features + proposed here would be sufficient to cover all the use cases that + Raymond Hettinger [1] originally had in mind. + + As written, this PEP makes it impossible to suppress '__context__', + since setting exc.__context__ to None will only result in it being + set again the moment that exc is raised. The exception context is lost when a 'yield' statement is executed; resuming the frame after the 'yield' does not restore the context. @@ -430,7 +442,10 @@ [8] Guido van Rossum discusses automatic chaining in PyErr_Set* http://mail.python.org/pipermail/python-dev/2003-June/036180.html - [9] Walter Dörwald suggests wrapping exceptions to add details + [9] MSDN .NET Framework Library, "Exception.InnerException Property" + http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemexceptionclassinnerexceptiontopic.asp + + [10] Walter Dörwald suggests wrapping exceptions to add details http://mail.python.org/pipermail/python-dev/2003-June/036148.html From gvanrossum at users.sourceforge.net Mon May 16 02:34:35 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Sun, 15 May 2005 17:34:35 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.307, 1.308 pep-0340.txt, 1.27, 1.28 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9626 Modified Files: pep-0000.txt pep-0340.txt Log Message: Rejecting PEP 340 in favor of PEP 343. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.307 retrieving revision 1.308 diff -u -d -r1.307 -r1.308 --- pep-0000.txt 15 May 2005 19:49:01 -0000 1.307 +++ pep-0000.txt 16 May 2005 00:34:33 -0000 1.308 @@ -117,7 +117,6 @@ S 336 Make None Callable McClelland S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan - S 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld S 342 Enhanced Iterators GvR S 343 Anonymous Block Redux GvR @@ -212,6 +211,7 @@ SR 317 Eliminate Implicit Exception Instantiation Taschuk SR 326 A Case for Top and Bottom Values Carlson, Reedy SR 329 Treating Builtins as Constants in the Standard Library Hettinger + SR 340 Anonymous Block Statements GvR SR 666 Reject Foolish Indentation Creighton @@ -379,7 +379,7 @@ S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan I 339 How to Change CPython's Bytecode Cannon - S 340 Anonymous Block Statements GvR + SR 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld S 342 Enhanced Iterators GvR S 343 Anonymous Block Redux GvR Index: pep-0340.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0340.txt,v retrieving revision 1.27 retrieving revision 1.28 diff -u -d -r1.27 -r1.28 --- pep-0340.txt 11 May 2005 22:09:37 -0000 1.27 +++ pep-0340.txt 16 May 2005 00:34:33 -0000 1.28 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Guido van Rossum -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/plain Created: 27-Apr-2005 @@ -33,6 +33,11 @@ but the two proposals are really independent and with Steven Bethard's help I have moved it to a separate PEP.) +Rejection Notice + + I am rejecting this PEP in favor of PEP 343. See the motivational + section in that PEP for the reasoning behind this rejection. + Motivation and Summary (Thanks to Shane Hathaway -- Hi Shane!) From gvanrossum at users.sourceforge.net Mon May 16 02:36:03 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Sun, 15 May 2005 17:36:03 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.9,1.10 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9870 Modified Files: pep-0343.txt Log Message: Move the __enter__ call out of the try suite. I was nuts putting it inside. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- pep-0343.txt 15 May 2005 00:45:42 -0000 1.9 +++ pep-0343.txt 16 May 2005 00:36:01 -0000 1.10 @@ -160,8 +160,8 @@ abc = EXPR exc = (None, None, None) try: + VAR = abc.__enter__() try: - VAR = abc.__enter__() BLOCK except: exc = sys.exc_info() From montanaro at users.sourceforge.net Mon May 16 04:42:23 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Sun, 15 May 2005 19:42:23 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.153,1.154 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27235/Doc/lib Modified Files: libos.tex Log Message: Add O_SHLOCK & O_EXLOCK. Closes patch #1103951. Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.153 retrieving revision 1.154 diff -u -d -r1.153 -r1.154 --- libos.tex 13 Feb 2005 22:50:03 -0000 1.153 +++ libos.tex 16 May 2005 02:42:21 -0000 1.154 @@ -594,7 +594,9 @@ The following data items are available for use in constructing the -\var{flags} parameter to the \function{open()} function. +\var{flags} parameter to the \function{open()} function. Some items will +not be available on all platforms. For descriptions of their availability +and use, consult \manpage{open}{2}. \begin{datadesc}{O_RDONLY} \dataline{O_WRONLY} @@ -614,6 +616,8 @@ \dataline{O_NDELAY} \dataline{O_NONBLOCK} \dataline{O_NOCTTY} +\dataline{O_SHLOCK} +\dataline{O_EXLOCK} More options for the \var{flag} argument to the \function{open()} function. Availability: Macintosh, \UNIX. \end{datadesc} From montanaro at users.sourceforge.net Mon May 16 04:42:24 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Sun, 15 May 2005 19:42:24 -0700 Subject: [Python-checkins] python/dist/src/Modules posixmodule.c, 2.334, 2.335 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27235/Modules Modified Files: posixmodule.c Log Message: Add O_SHLOCK & O_EXLOCK. Closes patch #1103951. Index: posixmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/posixmodule.c,v retrieving revision 2.334 retrieving revision 2.335 diff -u -d -r2.334 -r2.335 --- posixmodule.c 8 Mar 2005 09:10:29 -0000 2.334 +++ posixmodule.c 16 May 2005 02:42:21 -0000 2.335 @@ -7718,6 +7718,12 @@ #ifdef O_LARGEFILE if (ins(d, "O_LARGEFILE", (long)O_LARGEFILE)) return -1; #endif +#ifdef O_SHLOCK + if (ins(d, "O_SHLOCK", (long)O_SHLOCK)) return -1; +#endif +#ifdef O_EXLOCK + if (ins(d, "O_EXLOCK", (long)O_EXLOCK)) return -1; +#endif /* MS Windows */ #ifdef O_NOINHERIT From montanaro at users.sourceforge.net Mon May 16 04:42:25 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Sun, 15 May 2005 19:42:25 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1291,1.1292 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27235/Misc Modified Files: NEWS Log Message: Add O_SHLOCK & O_EXLOCK. Closes patch #1103951. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1291 retrieving revision 1.1292 diff -u -d -r1.1291 -r1.1292 --- NEWS 14 May 2005 20:54:13 -0000 1.1291 +++ NEWS 16 May 2005 02:42:22 -0000 1.1292 @@ -81,6 +81,9 @@ Extension Modules ----------------- +- Patch #1103951: Expose O_SHLOCK and O_EXLOCK in the posix module if + available on the platform. + - Bug #1166660: The readline module could segfault if hook functions were set in a different thread than that which called readline. From ping at users.sourceforge.net Mon May 16 08:58:15 2005 From: ping at users.sourceforge.net (ping@users.sourceforge.net) Date: Sun, 15 May 2005 23:58:15 -0700 Subject: [Python-checkins] python/nondist/peps pep-0344.txt,1.4,1.5 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv689 Modified Files: pep-0344.txt Log Message: Add "raise ... from" statement. Add reference to Perl 6 exception RFC. Index: pep-0344.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0344.txt,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- pep-0344.txt 15 May 2005 23:53:56 -0000 1.4 +++ pep-0344.txt 16 May 2005 06:58:12 -0000 1.5 @@ -15,7 +15,8 @@ This PEP proposes three standard attributes on exception instances: the '__context__' attribute for implicitly chained exceptions, the '__cause__' attribute for explicitly chained exceptions, and the - '__traceback__' attribute for the traceback. + '__traceback__' attribute for the traceback. A new "raise ... from" + statement sets the '__cause__' attribute. Motivation @@ -68,10 +69,11 @@ Rationale - This PEP distinguishes implicit chaining from explicit chaining of - exceptions because the unexpected raising of a secondary exception - and the intentional translation of an exception are two different - situations deserving quite different interpretations. + The Python-Dev discussions revealed interest in exception chaining + for two quite different purposes. To handle the unexpected raising + of a secondary exception, the exception must be retained implicitly. + To support intentional translation of an exception, there must be a + way to chain exceptions explicitly. This PEP addresses both. Several attribute names for chained exceptions have been suggested on Python-Dev [2], including 'cause', 'antecedent', 'reason', @@ -84,9 +86,8 @@ occurs in the context of handling another exception. This PEP suggests names with leading and trailing double-underscores - for '__context__' and '__traceback__' because the attributes are set - by the Python VM. The name '__cause__' is not set automatically by - the VM, but it seems confusing and collision-prone to use 'cause'. + for these three attributes because they are set by the Python VM. + Only in very special cases should they be set by normal assignment. This PEP handles exceptions that occur during 'except' blocks and 'finally' blocks in the same way. Reading the traceback makes it @@ -98,7 +99,7 @@ raised exception for compatibility with current behaviour. This PEP proposes that tracebacks display the outermost exception - last, because it would be consistent with the chronological order + last, because this would be consistent with the chronological order of tracebacks (from oldest to most recent frame) and because the actual thrown exception is easier to find on the last line. @@ -109,22 +110,21 @@ As for other languages, Java and Ruby both discard the original exception when another exception occurs in a 'catch'/'rescue' or 'finally'/'ensure' clause. Perl 5 lacks built-in structured - exception handling. For Perl 6, RFC 88 proposes an exception - mechanism that retains chained exceptions in an array named @@. - In that RFC, the most recently raised exception is exposed for - matching, as in this PEP; also, arbitrary expressions (possibly - involving @@) can be evaluated for exception matching. + exception handling. For Perl 6, RFC 88 [9] proposes an exception + mechanism that implicitly retains chained exceptions in an array + named @@. In that RFC, the most recently raised exception is + exposed for matching, as in this PEP; also, arbitrary expressions + (possibly involving @@) can be evaluated for exception matching. Exceptions in C# contain a read-only 'InnerException' property that - may point to another exception [9]. According to its documentation, + may point to another exception. Its documentation [10] says that "When an exception X is thrown as a direct result of a previous exception Y, the InnerException property of X should contain a reference to Y." This property is not set by the VM automatically; rather, all exception constructors take an optional 'innerException' argument to set it explicitly. The '__cause__' attribute fulfills - the same purpose as InnerException, but this PEP proposes adding a - single method to the base Exception class rather than extending the - constructors of all exceptions. + the same purpose as InnerException, but this PEP proposes a new form + of 'raise' rather than extending the constructors of all exceptions. The reason all three of these attributes are presented together in one proposal is that the '__traceback__' attribute provides @@ -207,14 +207,16 @@ Explicit Exception Chaining The '__cause__' attribute on exception objects is always initialized - to None. It is set by calling the 'setcause' method, a new method - defined on the base Exception class. For convenience, this method - returns the exception itself: + to None. It is set by a new form of the 'raise' statement: - def setcause(self, cause): - self.__cause__ = cause - return self + raise EXCEPTION from CAUSE + which is equivalent to: + + exc = EXCEPTION + exc.__cause__ = CAUSE + raise exc + In the following example, a database provides implementations for a few different kinds of storage, with file storage as one kind. The database designer wants errors to propagate as DatabaseError objects @@ -229,7 +231,11 @@ try: self.file = open(filename) except IOError, exc: - raise DatabaseError('failed to open').setcause(exc) + raise DatabaseError('failed to open') from exc + + If the call to open() raises an exception, the problem will be + reported as a DatabaseError, with a __cause__ attribute that reveals + the IOError as the original cause. Traceback Attribute @@ -297,7 +303,10 @@ exc, link = exc.__context__, 'During handling...' for exc, link in reversed(chain): print_exc(exc) - print '\n' + link + '\n' + if link: + print + print link + print In the 'traceback' module, the format_exception, print_exception, print_exc, and print_last functions will be updated to accept an @@ -320,7 +329,8 @@ A new API function, PyErr_SetContext(context), will help C programmers provide chained exception information. This function will first normalize the current exception so it is an instance, - then set its '__context__' attribute. + then set its '__context__' attribute. A similar API function, + PyErr_SetCause(cause), will set the '__cause__' attribute. Compatibility @@ -328,28 +338,50 @@ Chained exceptions expose the type of the most recent exception, so they will still match the same 'except' clauses as they do now. - The proposed changes should not break any code unless the code sets - or uses attributes named '__context__', '__cause__', 'setcause', or - '__traceback__' on exception instances. As of 2005-05-12, the - Python standard library contains no mention of such attributes. + The proposed changes should not break any code unless it sets or + uses attributes named '__context__', '__cause__', or '__traceback__' + on exception instances. As of 2005-05-12, the Python standard + library contains no mention of such attributes. Open Issues - Walter Dörwald [10] expressed a desire to attach extra information + Walter Dörwald [11] expressed a desire to attach extra information to an exception during its upward propagation without changing its type. This could be a useful feature, but it is not addressed by this PEP. It could conceivably be addressed by a separate PEP establishing conventions for other informational attributes on exceptions. - It is not clear whether the '__context__' and '__cause__' features - proposed here would be sufficient to cover all the use cases that - Raymond Hettinger [1] originally had in mind. - As written, this PEP makes it impossible to suppress '__context__', - since setting exc.__context__ to None will only result in it being - set again the moment that exc is raised. + since setting exc.__context__ to None in an 'except' or 'finally' + clause will only result in it being set again when exc is raised. + + To improve encapsulation, library implementors may want to wrap all + implementation-level exceptions with an application-level exception. + One could try to wrap exceptions by writing this: + + try: + ... implementation may raise an exception ... + except: + import sys + raise ApplicationError from sys.exc_value + + or this: + + try: + ... implementation may raise an exception ... + except Exception, exc: + raise ApplicationError from exc + + but both are somewhat flawed. It would be nice to be able to name + the current exception in a catch-all 'except' clause, but that isn't + addressed here. Such a feature would allow something like this: + + try: + ... implementation may raise an exception ... + except *, exc: + raise ApplicationError from exc The exception context is lost when a 'yield' statement is executed; resuming the frame after the 'yield' does not restore the context. @@ -441,11 +473,14 @@ [8] Guido van Rossum discusses automatic chaining in PyErr_Set* http://mail.python.org/pipermail/python-dev/2003-June/036180.html + + [9] Tony Olensky, "Omnibus Structured Exception/Error Handling Mechanism" + http://dev.perl.org/perl6/rfc/88.html - [9] MSDN .NET Framework Library, "Exception.InnerException Property" + [10] MSDN .NET Framework Library, "Exception.InnerException Property" http://msdn.microsoft.com/library/en-us/cpref/html/frlrfsystemexceptionclassinnerexceptiontopic.asp - [10] Walter Dörwald suggests wrapping exceptions to add details + [11] Walter Dörwald suggests wrapping exceptions to add details http://mail.python.org/pipermail/python-dev/2003-June/036148.html From ping at users.sourceforge.net Mon May 16 09:10:24 2005 From: ping at users.sourceforge.net (ping@users.sourceforge.net) Date: Mon, 16 May 2005 00:10:24 -0700 Subject: [Python-checkins] python/nondist/peps pep-0344.txt,1.5,1.6 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2757 Modified Files: pep-0344.txt Log Message: Mention GetBaseException in C#. Index: pep-0344.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0344.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- pep-0344.txt 16 May 2005 06:58:12 -0000 1.5 +++ pep-0344.txt 16 May 2005 07:10:22 -0000 1.6 @@ -125,6 +125,8 @@ argument to set it explicitly. The '__cause__' attribute fulfills the same purpose as InnerException, but this PEP proposes a new form of 'raise' rather than extending the constructors of all exceptions. + C# also provides a GetBaseException method that jumps directly to + the end of the InnerException chain; this PEP proposes no analog. The reason all three of these attributes are presented together in one proposal is that the '__traceback__' attribute provides From gvanrossum at users.sourceforge.net Mon May 16 15:42:55 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Mon, 16 May 2005 06:42:55 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.10,1.11 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23476 Modified Files: pep-0343.txt Log Message: *Really* move the __enter__ call where it belongs. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- pep-0343.txt 16 May 2005 00:36:01 -0000 1.10 +++ pep-0343.txt 16 May 2005 13:42:52 -0000 1.11 @@ -159,8 +159,8 @@ abc = EXPR exc = (None, None, None) + VAR = abc.__enter__() try: - VAR = abc.__enter__() try: BLOCK except: From ping at users.sourceforge.net Mon May 16 21:12:55 2005 From: ping at users.sourceforge.net (ping@users.sourceforge.net) Date: Mon, 16 May 2005 12:12:55 -0700 Subject: [Python-checkins] python/nondist/peps pep-0344.txt,1.6,1.7 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17533 Modified Files: pep-0344.txt Log Message: Simplify implementation of print_chain. Index: pep-0344.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0344.txt,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- pep-0344.txt 16 May 2005 07:10:22 -0000 1.6 +++ pep-0344.txt 16 May 2005 19:12:51 -0000 1.7 @@ -295,20 +295,13 @@ or __context__ respectively. Here is a sketch of the procedure: def print_chain(exc): - chain = [] - link = None - while exc: - chain.append((exc, link)) - if exc.__cause__: - exc, link = exc.__cause__, 'The above exception...' - else: - exc, link = exc.__context__, 'During handling...' - for exc, link in reversed(chain): - print_exc(exc) - if link: - print - print link - print + if exc.__cause__: + print_chain(exc.__cause__) + print '\nThe above exception was the direct cause...' + elif exc.__context__: + print_chain(exc.__context__) + print '\nDuring handling of the above exception, ...' + print_exc(exc) In the 'traceback' module, the format_exception, print_exception, print_exc, and print_last functions will be updated to accept an From gvanrossum at users.sourceforge.net Wed May 18 00:15:52 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 17 May 2005 15:15:52 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.11,1.12 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5548 Modified Files: pep-0343.txt Log Message: Elaborate Decimal context example, thanks to Michael Chermside. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- pep-0343.txt 16 May 2005 13:42:52 -0000 1.11 +++ pep-0343.txt 17 May 2005 22:15:50 -0000 1.12 @@ -388,9 +388,33 @@ by default all signals are blocked. The implementation is left as an exercise to the reader. - 8. Another use for this feature is the Decimal context. It's left - as an exercise for the reader. (Mail it to me if you'd like to - see it here.) + 8. Another use for this feature is the Decimal context. Here's a + simple example, after one posted by Michael Chermside: + + import decimal + + @do_template + def with_extra_precision(places=2): + c = decimal.getcontext() + saved_prec = c.prec + c.prec += places + yield None + c.prec = saved_prec + + Sample usage (adapted from the Python Library Reference): + + def sin(x): + "Return the sine of x as measured in radians." + do with_extra_precision(): + i, lasts, s, fact, num, sign = 1, 0, x, 1, x, 1 + while s != lasts: + lasts = s + i += 2 + fact *= i * (i-1) + num *= x * x + sign *= -1 + s += num / fact * sign + return +s References From tim_one at users.sourceforge.net Wed May 18 04:18:11 2005 From: tim_one at users.sourceforge.net (tim_one@users.sourceforge.net) Date: Tue, 17 May 2005 19:18:11 -0700 Subject: [Python-checkins] python/dist/src/Lib/distutils sysconfig.py, 1.65, 1.66 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/distutils In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18635 Modified Files: sysconfig.py Log Message: Whitespace normalization. Index: sysconfig.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/distutils/sysconfig.py,v retrieving revision 1.65 retrieving revision 1.66 diff -u -d -r1.65 -r1.66 --- sysconfig.py 25 Apr 2005 07:14:03 -0000 1.65 +++ sysconfig.py 18 May 2005 02:18:09 -0000 1.66 @@ -147,7 +147,7 @@ """ if compiler.compiler_type == "unix": (cc, cxx, opt, cflags, ccshared, ldshared, so_ext) = \ - get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS', + get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS', 'CCSHARED', 'LDSHARED', 'SO') if os.environ.has_key('CC'): From gvanrossum at users.sourceforge.net Wed May 18 04:53:28 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 17 May 2005 19:53:28 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.12,1.13 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23233 Modified Files: pep-0343.txt Log Message: Add a variant of the decimal example. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- pep-0343.txt 17 May 2005 22:15:50 -0000 1.12 +++ pep-0343.txt 18 May 2005 02:53:26 -0000 1.13 @@ -416,6 +416,20 @@ s += num / fact * sign return +s + An alternative version of the generator saves the entire + context instead of just the precision attribute. This is more + robust, but also more expensive, making it perhaps the better + choice when several attributes are modified together: + + @do_template + def with_extra_precision(places=2): + oldcontext = decimal.getcontext() + newcontext = oldcontext.copy() + newcontext.prec += places + decimal.setcontext(newcontext) + yield None + decimal.setcontext(oldcontext) + References [1] http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx From python at rcn.com Wed May 18 05:47:39 2005 From: python at rcn.com (Raymond Hettinger) Date: Tue, 17 May 2005 23:47:39 -0400 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.11,1.12 In-Reply-To: Message-ID: <000101c55b5c$56f14b20$ab29a044@oemcomputer> > + def sin(x): > + "Return the sine of x as measured in radians." > + do with_extra_precision(): > + i, lasts, s, fact, num, sign = 1, 0, x, 1, x, 1 > + while s != lasts: > + lasts = s > + i += 2 > + fact *= i * (i-1) > + num *= x * x > + sign *= -1 > + s += num / fact * sign > + return +s One more change: The final "return +s" should be unindented. It should be at the same level as the "do with_extra_precision()". The purpose of the "+s" is to force the result to be rounded back to the *original* precision. This nuance is likely to be the bane of folks who shift back and forth between different levels of precision. The following example shows the kind of oddity that can arise when working with quantities that have not been rounded to the current precision: >>> from decimal import getcontext, Decimal as D >>> getcontext().prec = 3 >>> D('3.104') + D('2.104') Decimal("5.21") >>> D('3.104') + D('0.000') + D('2.104') Decimal("5.20") Raymond From gvanrossum at users.sourceforge.net Wed May 18 05:58:41 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 17 May 2005 20:58:41 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.13,1.14 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1575 Modified Files: pep-0343.txt Log Message: Outdent "return +s" and explain why. Thanks Raymond! Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- pep-0343.txt 18 May 2005 02:53:26 -0000 1.13 +++ pep-0343.txt 18 May 2005 03:58:29 -0000 1.14 @@ -414,7 +414,9 @@ num *= x * x sign *= -1 s += num / fact * sign - return +s + # The "+s" rounds back to the original precision, + # so this must be outside the do-statement: + return +s An alternative version of the generator saves the entire context instead of just the precision attribute. This is more From rhettinger at users.sourceforge.net Wed May 18 08:07:10 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 17 May 2005 23:07:10 -0700 Subject: [Python-checkins] python/nondist/peps pep-0327.txt,1.7,1.8 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24900 Modified Files: pep-0327.txt Log Message: Fill-in missing word. Index: pep-0327.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0327.txt,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- pep-0327.txt 8 Jul 2004 22:09:48 -0000 1.7 +++ pep-0327.txt 18 May 2005 06:07:07 -0000 1.8 @@ -571,7 +571,7 @@ - The first release of the module should focus on that which is safe, minimal, and essential. -- While theoretically nice, real world use cases for interactions between +- While theoretically nice, real world use cases for interactions between floats and decimals are lacking. Java included float/decimal conversions to handle an obscure case where calculations are best performed in decimal eventhough a legacy data structure requires the inputs and outputs to be stored in From gvanrossum at users.sourceforge.net Wed May 18 16:24:29 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 18 May 2005 07:24:29 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.14,1.15 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv407 Modified Files: pep-0343.txt Log Message: More generic decimal example. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- pep-0343.txt 18 May 2005 03:58:29 -0000 1.14 +++ pep-0343.txt 18 May 2005 14:24:27 -0000 1.15 @@ -418,19 +418,28 @@ # so this must be outside the do-statement: return +s - An alternative version of the generator saves the entire - context instead of just the precision attribute. This is more - robust, but also more expensive, making it perhaps the better - choice when several attributes are modified together: + 9. Here's a more general Decimal-context-switching template: @do_template - def with_extra_precision(places=2): - oldcontext = decimal.getcontext() - newcontext = oldcontext.copy() - newcontext.prec += places - decimal.setcontext(newcontext) - yield None - decimal.setcontext(oldcontext) + def with_decimal_context(newctx=None): + oldctx = decimal.getcontext() + if newctx is None: + newctx = oldctx.copy() + decimal.setcontext(newctx) + yield newctx + decimal.setcontext(oldctx) + + Sample usage (adapted from the previous one): + + def sin(x): + do with_decimal_context() as ctx: + ctx.prec += 2 + # Rest of algorithm the same + return +s + + (Nick Coghlan has proposed to add __enter__ and __exit__ + methods to the decimal.Context class so that this example can + be simplified to "do decimal.getcontext() as ctx: ...".) References From jlt63 at users.sourceforge.net Fri May 20 02:56:57 2005 From: jlt63 at users.sourceforge.net (jlt63@users.sourceforge.net) Date: Thu, 19 May 2005 17:56:57 -0700 Subject: [Python-checkins] python/dist/src/Python import.c,2.240,2.241 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2513 Modified Files: import.c Log Message: Patch #1197318: Cygwin case-sensitive import patch A problem regarding importing symlinked modules was recently reported on the Cygwin mailing list: http://cygwin.com/ml/cygwin/2005-04/msg00257.html The following test case demonstrates the problem: $ ls -l total 1 lrwxrwxrwx 1 jt None 6 Apr 23 13:32 bar.py -> foo.py -rw-r--r-- 1 jt None 24 Apr 18 20:13 foo.py $ python -c 'import bar' Traceback (most recent call last): File "", line 1, in ? ImportError: No module named bar Since Cygwin's case_ok() uses a modified version of the Windows's version, the symlinked bar module actually resolves to file foo.py instead of bar.py. This obviously causes the matching code to fail (regardless of case). The patch fixes this problem by making Cygwin use the Mac OS X case_ok() instead of a modified Window's version. Index: import.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/import.c,v retrieving revision 2.240 retrieving revision 2.241 diff -u -d -r2.240 -r2.241 --- import.c 7 Oct 2004 06:46:25 -0000 2.240 +++ import.c 20 May 2005 00:56:54 -0000 2.241 @@ -1377,16 +1377,13 @@ /* First we may need a pile of platform-specific header files; the sequence * of #if's here should match the sequence in the body of case_ok(). */ -#if defined(MS_WINDOWS) || defined(__CYGWIN__) +#if defined(MS_WINDOWS) #include -#ifdef __CYGWIN__ -#include -#endif #elif defined(DJGPP) #include -#elif defined(__MACH__) && defined(__APPLE__) && defined(HAVE_DIRENT_H) +#elif (defined(__MACH__) && defined(__APPLE__) || defined(__CYGWIN__)) && defined(HAVE_DIRENT_H) #include #include @@ -1407,23 +1404,15 @@ * match the sequence just above. */ -/* MS_WINDOWS || __CYGWIN__ */ -#if defined(MS_WINDOWS) || defined(__CYGWIN__) +/* MS_WINDOWS */ +#if defined(MS_WINDOWS) WIN32_FIND_DATA data; HANDLE h; -#ifdef __CYGWIN__ - char tempbuf[MAX_PATH]; -#endif if (Py_GETENV("PYTHONCASEOK") != NULL) return 1; -#ifdef __CYGWIN__ - cygwin32_conv_to_win32_path(buf, tempbuf); - h = FindFirstFile(tempbuf, &data); -#else h = FindFirstFile(buf, &data); -#endif if (h == INVALID_HANDLE_VALUE) { PyErr_Format(PyExc_NameError, "Can't find file for module %.100s\n(filename %.300s)", @@ -1450,8 +1439,8 @@ } return strncmp(ffblk.ff_name, name, namelen) == 0; -/* new-fangled macintosh (macosx) */ -#elif defined(__MACH__) && defined(__APPLE__) && defined(HAVE_DIRENT_H) +/* new-fangled macintosh (macosx) or Cygwin */ +#elif (defined(__MACH__) && defined(__APPLE__) || defined(__CYGWIN__)) && defined(HAVE_DIRENT_H) DIR *dirp; struct dirent *dp; char dirname[MAXPATHLEN + 1]; From montanaro at users.sourceforge.net Fri May 20 05:07:23 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Thu, 19 May 2005 20:07:23 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_file.py,1.16,1.17 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22227/Lib/test Modified Files: test_file.py Log Message: Disallow opening files with modes 'aU' or 'wU' as specified by PEP 278. Closes bug 967182. Index: test_file.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_file.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- test_file.py 20 Apr 2005 19:41:36 -0000 1.16 +++ test_file.py 20 May 2005 03:07:04 -0000 1.17 @@ -40,6 +40,16 @@ raise TestFailed('expected exception setting file attr %r' % attr) f.close() +# check invalid mode strings +for mode in ("", "aU", "wU+"): + try: + f = file(TESTFN, mode) + except ValueError: + pass + else: + f.close() + raise TestFailed('%r is an invalid file mode' % mode) + # verify writelines with instance sequence l = UserList(['1', '2']) f = open(TESTFN, 'wb') From montanaro at users.sourceforge.net Fri May 20 05:07:38 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Thu, 19 May 2005 20:07:38 -0700 Subject: [Python-checkins] python/dist/src/Objects fileobject.c, 2.193, 2.194 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22227/Objects Modified Files: fileobject.c Log Message: Disallow opening files with modes 'aU' or 'wU' as specified by PEP 278. Closes bug 967182. Index: fileobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/fileobject.c,v retrieving revision 2.193 retrieving revision 2.194 diff -u -d -r2.193 -r2.194 --- fileobject.c 7 Nov 2004 14:15:28 -0000 2.193 +++ fileobject.c 20 May 2005 03:07:05 -0000 2.194 @@ -128,6 +128,54 @@ return (PyObject *) f; } +/* check for known incorrect mode strings - problem is, platforms are + free to accept any mode characters they like and are supposed to + ignore stuff they don't understand... write or append mode with + universal newline support is expressly forbidden by PEP 278. */ +/* zero return is kewl - one is un-kewl */ +static int +check_the_mode(char *mode) +{ + unsigned int len = strlen(mode); + + switch (len) { + case 0: + PyErr_SetString(PyExc_ValueError, "empty mode string"); + return 1; + + /* reject wU, aU */ + case 2: + switch (mode[0]) { + case 'w': + case 'a': + if (mode[1] == 'U') { + PyErr_SetString(PyExc_ValueError, + "invalid mode string"); + return 1; + } + break; + } + break; + + /* reject w+U, a+U, wU+, aU+ */ + case 3: + switch (mode[0]) { + case 'w': + case 'a': + if ((mode[1] == '+' && mode[2] == 'U') || + (mode[1] == 'U' && mode[2] == '+')) { + PyErr_SetString(PyExc_ValueError, + "invalid mode string"); + return 1; + } + break; + } + break; + } + + return 0; +} + static PyObject * open_the_file(PyFileObject *f, char *name, char *mode) { @@ -142,6 +190,9 @@ assert(mode != NULL); assert(f->f_fp == NULL); + if (check_the_mode(mode)) + return NULL; + /* rexec.py can't stop a user from getting the file() constructor -- all they have to do is get *any* file object f, and then do type(f). Here we prevent them from doing damage with it. */ From montanaro at users.sourceforge.net Fri May 20 05:07:39 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Thu, 19 May 2005 20:07:39 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1292,1.1293 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22227/Misc Modified Files: NEWS Log Message: Disallow opening files with modes 'aU' or 'wU' as specified by PEP 278. Closes bug 967182. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1292 retrieving revision 1.1293 diff -u -d -r1.1292 -r1.1293 --- NEWS 16 May 2005 02:42:22 -0000 1.1292 +++ NEWS 20 May 2005 03:07:06 -0000 1.1293 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- bug #967182: disallow opening files with 'wU' or 'aU' as specified by PEP + 278. + - patch #1109424: int, long, float, complex, and unicode now check for the proper magic slot for type conversions when subclassed. Previously the magic slot was ignored during conversion. Semantics now match the way From pje at users.sourceforge.net Sat May 21 23:43:00 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 21 May 2005 14:43:00 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.9, 1.10 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32219 Modified Files: pkg_resources.py Log Message: Add basic "Requirement" class that can tell whether a distribution or version meets its version requirements. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- pkg_resources.py 3 Apr 2005 18:52:21 -0000 1.9 +++ pkg_resources.py 21 May 2005 21:42:57 -0000 1.10 @@ -19,7 +19,7 @@ 'resource_stream', 'resource_filename', 'set_extraction_path', 'cleanup_resources', 'parse_requirements', 'parse_version', 'compatible_platforms', 'get_platform', - 'Distribution', # 'glob_resources' + 'Distribution', 'Requirement', # 'glob_resources' ] import sys, os, zipimport, time, re @@ -740,7 +740,7 @@ """Yield ``Requirement`` objects for each specification in `strs` `strs` must be an instance of ``basestring``, or a (possibly-nested) - sequence thereof. + iterable thereof. """ # create a steppable iterator, so we can handle \-continuations lines = iter(yield_lines(strs)) @@ -772,10 +772,61 @@ elif not LINE_END(line,p): raise ValueError("Expected ',' or EOL in",line,"at",line[p:]) - yield distname.replace('_','-'), specs + yield Requirement(distname.replace('_','-'), specs) + + + + +class Requirement: + + def __init__(self, distname, specs=()): + self.distname = distname + self.key = distname.lower() + index = [(parse_version(v),state_machine[op],op,v) for op,v in specs] + index.sort() + self.specs = [(op,ver) for parsed,trans,op,ver in index] + self.index = index + def __str__(self): + return self.distname + ','.join([''.join(s) for s in self.specs]) + def __repr__(self): + return "Requirement(%r, %r)" % (self.distname, self.specs) + def __eq__(self,other): + return isinstance(other,Requirement) \ + and self.key==other.key and self.specs==other.specs + + def __contains__(self,item): + if isinstance(item,Distribution): + if item.key <> self.key: + return False + item = item.parsed_version + elif isinstance(item,basestring): + item = parse_version(item) + last = True + for parsed,trans,op,ver in self.index: + action = trans[cmp(item,parsed)] + if action=='F': + return False + elif action=='T': + return True + elif action=='+': + last = True + elif action=='-': + last = False + return last + + +state_machine = { + # =>< + '<' : '--T', + '<=': 'T-T', + '>' : 'F+F', + '>=': 'T+F', + '==': 'T..', + '!=': 'F..', +} def _get_mro(cls): """Get an mro for a type or classic class""" @@ -808,13 +859,3 @@ _initialize(globals()) - - - - - - - - - - From pje at users.sourceforge.net Sat May 21 23:43:00 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 21 May 2005 14:43:00 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/tests test_resources.py, 1.5, 1.6 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32219/setuptools/tests Modified Files: test_resources.py Log Message: Add basic "Requirement" class that can tell whether a distribution or version meets its version requirements. Index: test_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests/test_resources.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- test_resources.py 3 Apr 2005 18:52:21 -0000 1.5 +++ test_resources.py 21 May 2005 21:42:57 -0000 1.6 @@ -80,6 +80,47 @@ +class RequirementsTests(TestCase): + + def testBasics(self): + r = Requirement("Twisted", [('>=','1.2')]) + self.assertEqual(str(r),"Twisted>=1.2") + self.assertEqual(repr(r),"Requirement('Twisted', [('>=', '1.2')])") + self.assertEqual(r, Requirement("Twisted", [('>=','1.2')])) + self.assertEqual(r, Requirement("twisTed", [('>=','1.2')])) + self.assertNotEqual(r, Requirement("Twisted", [('>=','2.0')])) + self.assertNotEqual(r, Requirement("Zope", [('>=','1.2')])) + self.assertNotEqual(r, Requirement("Zope", [('>=','3.0')])) + + def testOrdering(self): + r1 = Requirement("Twisted", [('==','1.2c1'),('>=','1.2')]) + r2 = Requirement("Twisted", [('>=','1.2'),('==','1.2c1')]) + self.assertEqual(r1,r2) + self.assertEqual(str(r1),str(r2)) + self.assertEqual(str(r2),"Twisted==1.2c1,>=1.2") + + def testBasicContains(self): + r = Requirement("Twisted", [('>=','1.2')]) + foo_dist = Distribution.from_filename("FooPkg-1.3_1.egg") + twist11 = Distribution.from_filename("Twisted-1.1.egg") + twist12 = Distribution.from_filename("Twisted-1.2.egg") + self.failUnless(parse_version('1.2') in r) + self.failUnless(parse_version('1.1') not in r) + self.failUnless('1.2' in r) + self.failUnless('1.1' not in r) + self.failUnless(foo_dist not in r) + self.failUnless(twist11 not in r) + self.failUnless(twist12 in r) + + def testAdvancedContains(self): + r, = parse_requirements("Foo>=1.2,<=1.3,==1.9,>2.0,!=2.5,<3.0,==4.5") + for v in ('1.2','1.2.2','1.3','1.9','2.0.1','2.3','2.6','3.0c1','4.5'): + self.failUnless(v in r, (v,r)) + for v in ('1.2c1','1.3.1','1.5','1.9.1','2.0','2.5','3.0','4.0'): + self.failUnless(v not in r, (v,r)) + + + class ParseTests(TestCase): def testEmptyParse(self): @@ -92,14 +133,14 @@ ]: self.assertEqual(list(pkg_resources.yield_lines(inp)),out) - def testSimple(self): + def testSimpleRequirements(self): self.assertEqual( list(parse_requirements('Twis-Ted>=1.2-1')), - [('Twis-Ted',[('>=','1.2-1')])] + [Requirement('Twis-Ted',[('>=','1.2-1')])] ) self.assertEqual( list(parse_requirements('Twisted >=1.2, \ # more\n<2.0')), - [('Twisted',[('>=','1.2'),('<','2.0')])] + [Requirement('Twisted',[('>=','1.2'),('<','2.0')])] ) self.assertRaises(ValueError,lambda:list(parse_requirements(">=2.3"))) self.assertRaises(ValueError,lambda:list(parse_requirements("x\\"))) From pje at users.sourceforge.net Sun May 22 20:17:02 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 11:17:02 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.10, 1.11 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31069 Modified Files: pkg_resources.py Log Message: Refine dependency resolution algorithm so it won't take exponential time, or bomb on cyclic dependencies. (But it's still an untested sketch.) Added list of things that need to be implemented before dependency resolution can actually work. Added tests for lower-level parts of the dependency resolution system, and a hook to support subclasses doing automatic download of needed dependencies. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- pkg_resources.py 21 May 2005 21:42:57 -0000 1.10 +++ pkg_resources.py 22 May 2005 18:16:59 -0000 1.11 @@ -19,6 +19,7 @@ 'resource_stream', 'resource_filename', 'set_extraction_path', 'cleanup_resources', 'parse_requirements', 'parse_version', 'compatible_platforms', 'get_platform', + 'ResolutionError', 'VersionConflict', 'DistributionNotFound', 'Distribution', 'Requirement', # 'glob_resources' ] @@ -27,17 +28,16 @@ def _sort_dists(dists): tmp = [(dist.version,dist) for dist in dists] tmp.sort() - tmp.reverse() - dists[:] = [d for v,d in tmp] - - - - - - + dists[::-1] = [d for v,d in tmp] +class ResolutionError(ImportError): + """Abstract base for dependency resolution errors""" +class VersionConflict(ResolutionError): + """An already-installed version conflicts with the requested version""" +class DistributionNotFound(ResolutionError): + """A requested distribution was not found""" _provider_factories = {} @@ -76,7 +76,7 @@ return True # easy case # XXX all the tricky cases go here - + return False @@ -137,7 +137,7 @@ was first imported.) You may explicitly set `platform` to ``None`` if you wish to map *all* - distributions, not just those compatible with the running platform. + distributions, not just those compatible with a single platform. """ self._distmap = {} @@ -179,10 +179,10 @@ search_path = sys.path add = self.add for item in search_path: - source = get_dist_source(item) + source = get_distro_source(item) for dist in source.iter_distributions(requirement): if compatible_platforms(dist.platform, platform): - add(dist) + add(dist) # XXX should also check python version! def __getitem__(self,key): """Return a newest-to-oldest list of distributions for the given key @@ -213,6 +213,77 @@ """Remove `dist` from the distribution map""" self._distmap[dist.key].remove(dist) + def best_match(self,requirement,path=None): + """Find distribution best matching `requirement` and usable on `path` + + If a distribution that's already installed on `path` is unsuitable, + a VersionConflict is raised. If one or more suitable distributions are + already installed, the leftmost distribution (i.e., the one first in + the search path) is returned. Otherwise, the available distribution + with the highest version number is returned, or a deferred distribution + object is returned if a suitable ``obtain()`` method exists. If there + is no way to meet the requirement, None is returned. + """ + if path is None: + path = sys.path + + distros = self.get(requirement.key, ()) + find = dict([(dist.path,dist) for dist in distros]).get + + for item in path: + dist = find(item) + if dist is not None: + if dist in requirement: + return dist + else: + raise VersionConflict(dist,requirement) # XXX add more info + + for dist in distros: + if dist in requirement: + return dist + + return self.obtain(requirement) # as a last resort, try and download + + def resolve(self, requirements, path=None): + """List all distributions needed to (recursively) meet requirements""" + + if path is None: + path = sys.path + + requirements = list(requirements)[::1] # set up the stack + processed = {} # set of processed requirements + best = {} # key -> dist + + while requirements: + + req = requirements.pop() + if req in processed: + # Ignore cyclic or redundant dependencies + continue + + dist = best.get(req.key) + + if dist is None: + # Find the best distribution and add it to the map + dist = best[req.key] = self.best_match(req,path) + if dist is None: + raise DistributionNotFound(req) # XXX put more info here + + elif dist not in requirement: + # Oops, the "best" so far conflicts with a dependency + raise VersionConflict(req,dist) # XXX put more info here + + requirements.extend(dist.depends(req.options)[::-1]) + processed[req] = True + + return best.values() # return list of distros to install + + + def obtain(self, requirement): + """Obtain a distro that matches requirement (e.g. via download)""" + return None # override this in subclasses + + class ResourceManager: """Manage resource extraction and packages""" @@ -244,6 +315,17 @@ self, resource_name ) + + + + + + + + + + + def get_cache_path(self, archive_name, names=()): """Return absolute location in cache for `archive_name` and `names` @@ -331,35 +413,35 @@ `requirements` must be a string or a (possibly-nested) sequence thereof, specifying the distributions and versions required. - XXX THIS IS DRAFT CODE FOR DESIGN PURPOSES ONLY RIGHT NOW + + XXX This doesn't work yet, because: + + * get_distro_source() isn't implemented + * Distribution.depends() isn't implemented + * Distribution.install_on() isn't implemented + * Requirement.options isn't implemented + * AvailableDistributions.resolve() is untested + * AvailableDistributions.scan() is untested + + There may be other things missing as well, but this definitely won't work + as long as any of the above items remain unimplemented. """ - all_distros = AvailableDistributions() - installed = {} - all_requirements = {} - def _require(requirements,source=None): - for req in parse_requirements(requirements): - name,vers = req # XXX - key = name.lower() - all_requirements.setdefault(key,[]).append((req,source)) - if key in installed and not req.matches(installed[key]): - raise ImportError( - "The installed %s distribution does not match" # XXX - ) # XXX should this be a subclass of ImportError? - all_distros[key] = distros = [ - dist for dist in all_distros.get(key,[]) - if req.matches(dist) - ] - if not distros: - raise ImportError( - "No %s distribution matches all criteria for " % name - ) # XXX should this be a subclass of ImportError? - for key in all_requirements.keys(): # XXX sort them - pass - # find "best" distro for key and install it - # after _require()-ing its requirements + requirements = parse_requirements(requirements) + + for dist in AvailableDistributions().resolve(requirements): + dist.install_on(sys.path) + + + + + + + + + + - _require(requirements) @@ -669,31 +751,12 @@ self.py_version = py_version self.platform = platform self.path = path_str - self.normalized_path = os.path.normpath(os.path.normcase(path_str)) def installed_on(self,path=None): """Is this distro installed on `path`? (defaults to ``sys.path``)""" if path is None: path = sys.path - if self.path in path or self.normalized_path in path: - return True - for item in path: - normalized = os.path.normpath(os.path.normcase(item)) - if normalized == self.normalized_path: - return True - return False - - - - - - - - - - - - + return self.path in path #@classmethod def from_filename(cls,filename,metadata=None): @@ -711,6 +774,9 @@ ) from_filename = classmethod(from_filename) + + + # These properties have to be lazy so that we don't have to load any # metadata until/unless it's actually needed. (i.e., some distributions # may not know their name or version without loading PKG-INFO) @@ -736,6 +802,22 @@ + + + + + + + + + + + + + + + + def parse_requirements(strs): """Yield ``Requirement`` objects for each specification in `strs` @@ -818,6 +900,17 @@ return last + #@staticmethod + def parse(s): + reqs = list(parse_requirements(s)) + if reqs: + if len(reqs)==1: + return reqs[0] + raise ValueError("Expected only one requirement", s) + raise ValueError("No requirements found", s) + + parse = staticmethod(parse) + state_machine = { # =>< '<' : '--T', From pje at users.sourceforge.net Sun May 22 20:17:02 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 11:17:02 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/tests test_resources.py, 1.6, 1.7 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31069/setuptools/tests Modified Files: test_resources.py Log Message: Refine dependency resolution algorithm so it won't take exponential time, or bomb on cyclic dependencies. (But it's still an untested sketch.) Added list of things that need to be implemented before dependency resolution can actually work. Added tests for lower-level parts of the dependency resolution system, and a hook to support subclasses doing automatic download of needed dependencies. Index: test_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests/test_resources.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- test_resources.py 21 May 2005 21:42:57 -0000 1.6 +++ test_resources.py 22 May 2005 18:17:00 -0000 1.7 @@ -39,6 +39,47 @@ [dist.version for dist in ad['FooPkg']], ['1.9','1.4','1.2'] ) + path = [] + req, = parse_requirements("FooPkg>=1.3") + + # Nominal case: no distros on path, should yield all applicable + self.assertEqual(ad.best_match(req,path).version, '1.9') + + # If a matching distro is already installed, should return only that + path.append("FooPkg-1.4-py2.4-win32.egg") + self.assertEqual(ad.best_match(req,path).version, '1.4') + + # If the first matching distro is unsuitable, it's a version conflict + path.insert(0,"FooPkg-1.2-py2.4.egg") + self.assertRaises(VersionConflict, ad.best_match, req, path) + + # If more than one match on the path, the first one takes precedence + path.insert(0,"FooPkg-1.4-py2.4-win32.egg") + self.assertEqual(ad.best_match(req,path).version, '1.4') + + + + + + + + + + + + + + + + + + + + + + + + def checkFooPkg(self,d): self.assertEqual(d.name, "FooPkg") self.assertEqual(d.key, "foopkg") @@ -83,7 +124,7 @@ class RequirementsTests(TestCase): def testBasics(self): - r = Requirement("Twisted", [('>=','1.2')]) + r = Requirement.parse("Twisted>=1.2") self.assertEqual(str(r),"Twisted>=1.2") self.assertEqual(repr(r),"Requirement('Twisted', [('>=', '1.2')])") self.assertEqual(r, Requirement("Twisted", [('>=','1.2')])) @@ -142,15 +183,15 @@ list(parse_requirements('Twisted >=1.2, \ # more\n<2.0')), [Requirement('Twisted',[('>=','1.2'),('<','2.0')])] ) - self.assertRaises(ValueError,lambda:list(parse_requirements(">=2.3"))) - self.assertRaises(ValueError,lambda:list(parse_requirements("x\\"))) - self.assertRaises(ValueError,lambda:list(parse_requirements("x==2 q"))) - - - - - - + self.assertEqual( + Requirement.parse("FooBar==1.99a3"), + Requirement("FooBar", [('==','1.99a3')]) + ) + self.assertRaises(ValueError,Requirement.parse,">=2.3") + self.assertRaises(ValueError,Requirement.parse,"x\\") + self.assertRaises(ValueError,Requirement.parse,"x==2 q") + self.assertRaises(ValueError,Requirement.parse,"X==1\nY==2") + self.assertRaises(ValueError,Requirement.parse,"#") From pje at users.sourceforge.net Sun May 22 21:40:24 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 12:40:24 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.11, 1.12 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13209 Modified Files: pkg_resources.py Log Message: Distribution metadata parsing: distribution objects can now extract their version from PKG-INFO and their dependencies from depends.txt, including optional dependencies. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- pkg_resources.py 22 May 2005 18:16:59 -0000 1.11 +++ pkg_resources.py 22 May 2005 19:40:22 -0000 1.12 @@ -14,23 +14,19 @@ method. """ __all__ = [ - 'register_loader_type', 'get_provider', 'IResourceProvider', + 'register_loader_type', 'get_provider', 'IResourceProvider', 'ResourceManager', 'AvailableDistributions', 'require', 'resource_string', 'resource_stream', 'resource_filename', 'set_extraction_path', 'cleanup_resources', 'parse_requirements', 'parse_version', - 'compatible_platforms', 'get_platform', + 'compatible_platforms', 'get_platform', 'IMetadataProvider', 'ResolutionError', 'VersionConflict', 'DistributionNotFound', - 'Distribution', 'Requirement', # 'glob_resources' + 'InvalidOption', 'Distribution', 'Requirement', 'yield_lines', + 'split_sections', # 'glob_resources' ] import sys, os, zipimport, time, re -def _sort_dists(dists): - tmp = [(dist.version,dist) for dist in dists] - tmp.sort() - dists[::-1] = [d for v,d in tmp] - -class ResolutionError(ImportError): +class ResolutionError(Exception): """Abstract base for dependency resolution errors""" class VersionConflict(ResolutionError): @@ -39,6 +35,10 @@ class DistributionNotFound(ResolutionError): """A requested distribution was not found""" +class InvalidOption(ResolutionError): + """Invalid or unrecognized option name for a distribution""" + + _provider_factories = {} def register_loader_type(loader_type, provider_factory): @@ -80,7 +80,22 @@ return False -class IResourceProvider: +class IMetadataProvider: + + def has_metadata(name): + """Does the package's distribution contain the named metadata?""" + + def get_metadata(name): + """The named metadata resource as a string""" + + def get_metadata_lines(name): + """Yield named metadata resource as list of non-blank non-comment lines + + Leading and trailing whitespace is stripped from each line, and lines + with ``#`` as the first non-blank character are omitted. + """ + +class IResourceProvider(IMetadataProvider): """An object that provides access to package resources""" @@ -102,25 +117,10 @@ def has_resource(resource_name): """Does the package contain the named resource?""" - def has_metadata(name): - """Does the package's distribution contain the named metadata?""" - - def get_metadata(name): - """The named metadata resource as a string""" - - def get_metadata_lines(name): - """Yield named metadata resource as list of non-blank non-comment lines - - Leading and trailing whitespace is stripped from each line, and lines - with ``#`` as the first non-blank character are omitted. - """ - # XXX list_resources? glob_resources? - - class AvailableDistributions(object): """Searchable snapshot of distributions on a search path""" @@ -417,7 +417,6 @@ XXX This doesn't work yet, because: * get_distro_source() isn't implemented - * Distribution.depends() isn't implemented * Distribution.install_on() isn't implemented * Requirement.options isn't implemented * AvailableDistributions.resolve() is untested @@ -449,6 +448,7 @@ + class DefaultProvider: """Provides access to package resources in the filesystem""" @@ -746,11 +746,12 @@ if name: self.name = name.replace('_','-') if version: - self.version = version.replace('_','-') + self._version = version.replace('_','-') self.py_version = py_version self.platform = platform self.path = path_str + self.metadata = metadata def installed_on(self,path=None): """Is this distro installed on `path`? (defaults to ``sys.path``)""" @@ -776,7 +777,6 @@ - # These properties have to be lazy so that we don't have to load any # metadata until/unless it's actually needed. (i.e., some distributions # may not know their name or version without loading PKG-INFO) @@ -800,17 +800,58 @@ parsed_version = property(parsed_version) + #@property + def version(self): + try: + return self._version + except AttributeError: + for line in self.metadata.get_metadata_lines('PKG-INFO'): + if line.lower().startswith('version:'): + self._version = line.split(':',1)[1].strip() + return self._version + else: + raise AttributeError( + "Missing Version: header in PKG-INFO", self + ) + version = property(version) + #@property + def _dep_map(self): + try: + return self.__dep_map + except AttributeError: + dm = self.__dep_map = {None: []} + if self.metadata.has_metadata('depends.txt'): + for section,contents in split_sections( + self.metadata.get_metadata_lines('depends.txt') + ): + dm[section] = list(parse_requirements(contents)) + return dm + _dep_map = property(_dep_map) + def depends(self,options=()): + """List of Requirements needed for this distro if `options` are used""" + dm = self._dep_map + deps = [] + deps.extend(dm.get(None,())) + for opt in options: + try: + deps.extend(dm[opt.lower()]) + except KeyError: + raise InvalidOption("No such option", self, opt) + return deps - +def _sort_dists(dists): + tmp = [(dist.version,dist) for dist in dists] + tmp.sort() + dists[::-1] = [d for v,d in tmp] @@ -941,6 +982,33 @@ os.makedirs(dirname) +def split_sections(s): + """Split a string or iterable thereof into (section,content) pairs + + Each ``section`` is a lowercase version of the section header ("[section]") + and each ``content`` is a list of stripped lines excluding blank lines and + comment-only lines. If there are any such lines before the first section + header, they're returned in a first ``section`` of ``None``. + """ + section = None + content = [] + for line in yield_lines(s): + if line.startswith("["): + if line.endswith("]"): + if content: + yield section, content + section = line[1:-1].strip().lower() + content = [] + else: + raise ValueError("Invalid section heading", line) + else: + content.append(line) + + # wrap up last segment + if content: + yield section, content + + # Set up global resource manager _manager = ResourceManager() @@ -952,3 +1020,6 @@ _initialize(globals()) + + + From pje at users.sourceforge.net Sun May 22 21:40:24 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 12:40:24 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/tests test_resources.py, 1.7, 1.8 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13209/setuptools/tests Modified Files: test_resources.py Log Message: Distribution metadata parsing: distribution objects can now extract their version from PKG-INFO and their dependencies from depends.txt, including optional dependencies. Index: test_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests/test_resources.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- test_resources.py 22 May 2005 18:17:00 -0000 1.7 +++ test_resources.py 22 May 2005 19:40:22 -0000 1.8 @@ -2,6 +2,23 @@ from pkg_resources import * import pkg_resources, sys +class Metadata: + """Mock object to return metadata as if from an on-disk distribution""" + + def __init__(self,*pairs): + self.metadata = dict(pairs) + + def has_metadata(self,name): + return name in self.metadata + + def get_metadata(self,name): + return self.metadata[name] + + def get_metadata_lines(self,name): + return yield_lines(self.get_metadata(name)) + + + class DistroTests(TestCase): def testCollection(self): @@ -26,13 +43,11 @@ self.assertEqual( [dist.version for dist in ad['FooPkg']], ['1.4','1.3-1','1.2'] ) - # Removing a distribution leaves sequence alone ad.remove(ad['FooPkg'][1]) self.assertEqual( [dist.version for dist in ad.get('FooPkg')], ['1.4','1.2'] ) - # And inserting adds them in order ad.add(Distribution.from_filename("FooPkg-1.9.egg")) self.assertEqual( @@ -51,34 +66,11 @@ # If the first matching distro is unsuitable, it's a version conflict path.insert(0,"FooPkg-1.2-py2.4.egg") - self.assertRaises(VersionConflict, ad.best_match, req, path) + self.assertRaises(VersionConflict, ad.best_match, req, path) # If more than one match on the path, the first one takes precedence path.insert(0,"FooPkg-1.4-py2.4-win32.egg") self.assertEqual(ad.best_match(req,path).version, '1.4') - - - - - - - - - - - - - - - - - - - - - - - def checkFooPkg(self,d): self.assertEqual(d.name, "FooPkg") @@ -105,6 +97,55 @@ d = Distribution.from_filename("FooPkg-1.3_1-py2.4-win32.egg") self.checkFooPkg(d) + def testDistroMetadata(self): + d = Distribution( + "/some/path", name="FooPkg", py_version="2.4", platform="win32", + metadata = Metadata( + ('PKG-INFO',"Metadata-Version: 1.0\nVersion: 1.3-1\n") + ) + ) + self.checkFooPkg(d) + + + def distDepends(self, txt): + return Distribution("/foo", metadata=Metadata(('depends.txt', txt))) + + def checkDepends(self, dist, txt, opts=()): + self.assertEqual( + list(dist.depends(opts)), + list(parse_requirements(txt)) + ) + + def testDistroDependsSimple(self): + for v in "Twisted>=1.5", "Twisted>=1.5\nZConfig>=2.0": + self.checkDepends(self.distDepends(v), v) + + + def testDistroDependsOptions(self): + d = self.distDepends(""" + Twisted>=1.5 + [docgen] + ZConfig>=2.0 + docutils>=0.3 + [fastcgi] + fcgiapp>=0.1""") + self.checkDepends(d,"Twisted>=1.5") + self.checkDepends( + d,"Twisted>=1.5 ZConfig>=2.0 docutils>=0.3".split(), ["docgen"] + ) + self.checkDepends( + d,"Twisted>=1.5 fcgiapp>=0.1".split(), ["fastcgi"] + ) + self.checkDepends( + d,"Twisted>=1.5 ZConfig>=2.0 docutils>=0.3 fcgiapp>=0.1".split(), + ["docgen","fastcgi"] + ) + self.checkDepends( + d,"Twisted>=1.5 fcgiapp>=0.1 ZConfig>=2.0 docutils>=0.3".split(), + ["fastcgi", "docgen"] + ) + self.assertRaises(InvalidOption, d.depends, ["foo"]) + @@ -174,6 +215,35 @@ ]: self.assertEqual(list(pkg_resources.yield_lines(inp)),out) + def testSplitting(self): + self.assertEqual( + list( + pkg_resources.split_sections(""" + x + [Y] + z + + a + [b ] + # foo + c + [ d] + [q] + v + """ + ) + ), + [(None,["x"]), ("y",["z","a"]), ("b",["c"]), ("q",["v"])] + ) + self.assertRaises(ValueError,list,pkg_resources.split_sections("[foo")) + + + + + + + + def testSimpleRequirements(self): self.assertEqual( list(parse_requirements('Twis-Ted>=1.2-1')), @@ -194,6 +264,18 @@ self.assertRaises(ValueError,Requirement.parse,"#") + def testVersionEquality(self): + def c(s1,s2): + p1, p2 = parse_version(s1),parse_version(s2) + self.assertEqual(p1,p2, (s1,s2,p1,p2)) + + c('0.4', '0.4.0') + c('0.4.0.0', '0.4.0') + c('0.4.0-0', '0.4-0') + c('0pl1', '0.0pl1') + c('0pre1', '0.0c1') + c('0.0.0preview1', '0c1') + c('0.0c1', '0rc1') @@ -244,44 +326,3 @@ - def testVersionEquality(self): - def c(s1,s2): - p1, p2 = parse_version(s1),parse_version(s2) - self.assertEqual(p1,p2, (s1,s2,p1,p2)) - - c('0.4', '0.4.0') - c('0.4.0.0', '0.4.0') - c('0.4.0-0', '0.4-0') - c('0pl1', '0.0pl1') - c('0pre1', '0.0c1') - c('0.0.0preview1', '0c1') - c('0.0c1', '0rc1') - - - - - - - - - - - - - - - - - - - - - - - - - - - - - From pje at users.sourceforge.net Sun May 22 22:28:49 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 13:28:49 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/tests test_resources.py, 1.8, 1.9 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21794/setuptools/tests Modified Files: test_resources.py Log Message: Added support for specifying options on requirements, so that a package's optional dependencies can be included when processing nested dependencies. Next up: tests for the resolve() algorithm. Index: test_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests/test_resources.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- test_resources.py 22 May 2005 19:40:22 -0000 1.8 +++ test_resources.py 22 May 2005 20:28:47 -0000 1.9 @@ -1,6 +1,7 @@ from unittest import TestCase, makeSuite from pkg_resources import * import pkg_resources, sys +from sets import ImmutableSet class Metadata: """Mock object to return metadata as if from an on-disk distribution""" @@ -18,7 +19,6 @@ return yield_lines(self.get_metadata(name)) - class DistroTests(TestCase): def testCollection(self): @@ -167,12 +167,13 @@ def testBasics(self): r = Requirement.parse("Twisted>=1.2") self.assertEqual(str(r),"Twisted>=1.2") - self.assertEqual(repr(r),"Requirement('Twisted', [('>=', '1.2')])") + self.assertEqual(repr(r),"Requirement('Twisted', [('>=', '1.2')], ())") self.assertEqual(r, Requirement("Twisted", [('>=','1.2')])) self.assertEqual(r, Requirement("twisTed", [('>=','1.2')])) self.assertNotEqual(r, Requirement("Twisted", [('>=','2.0')])) self.assertNotEqual(r, Requirement("Zope", [('>=','1.2')])) self.assertNotEqual(r, Requirement("Zope", [('>=','3.0')])) + self.assertNotEqual(r, Requirement.parse("Twisted[extras]>=1.2")) def testOrdering(self): r1 = Requirement("Twisted", [('==','1.2c1'),('>=','1.2')]) @@ -202,6 +203,46 @@ self.failUnless(v not in r, (v,r)) + def testOptionsAndHashing(self): + r1 = Requirement.parse("Twisted[foo,bar]>=1.2") + r2 = Requirement.parse("Twisted[bar,FOO]>=1.2") + r3 = Requirement.parse("Twisted[BAR,FOO]>=1.2.0") + self.assertEqual(r1,r2) + self.assertEqual(r1,r3) + self.assertEqual(r1.options, ("foo","bar")) + self.assertEqual(r2.options, ("bar","FOO")) + self.assertEqual(hash(r1), hash(r2)) + self.assertEqual( + hash(r1), hash(("twisted", ((">=",parse_version("1.2")),), + ImmutableSet(["foo","bar"]))) + ) + + + + + + + + + + + + + + + + + + + + + + + + + + + class ParseTests(TestCase): @@ -233,7 +274,7 @@ """ ) ), - [(None,["x"]), ("y",["z","a"]), ("b",["c"]), ("q",["v"])] + [(None,["x"]), ("y",["z","a"]), ("b",["c"]), ("d",[]), ("q",["v"])] ) self.assertRaises(ValueError,list,pkg_resources.split_sections("[foo")) From pje at users.sourceforge.net Sun May 22 22:28:49 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 13:28:49 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21794 Modified Files: pkg_resources.py Log Message: Added support for specifying options on requirements, so that a package's optional dependencies can be included when processing nested dependencies. Next up: tests for the resolve() algorithm. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- pkg_resources.py 22 May 2005 19:40:22 -0000 1.12 +++ pkg_resources.py 22 May 2005 20:28:46 -0000 1.13 @@ -14,7 +14,7 @@ method. """ __all__ = [ - 'register_loader_type', 'get_provider', 'IResourceProvider', + 'register_loader_type', 'get_provider', 'IResourceProvider', 'ResourceManager', 'AvailableDistributions', 'require', 'resource_string', 'resource_stream', 'resource_filename', 'set_extraction_path', 'cleanup_resources', 'parse_requirements', 'parse_version', @@ -25,6 +25,7 @@ ] import sys, os, zipimport, time, re +from sets import ImmutableSet class ResolutionError(Exception): """Abstract base for dependency resolution errors""" @@ -38,7 +39,6 @@ class InvalidOption(ResolutionError): """Invalid or unrecognized option name for a distribution""" - _provider_factories = {} def register_loader_type(loader_type, provider_factory): @@ -418,7 +418,6 @@ * get_distro_source() isn't implemented * Distribution.install_on() isn't implemented - * Requirement.options isn't implemented * AvailableDistributions.resolve() is untested * AvailableDistributions.scan() is untested @@ -449,6 +448,7 @@ + class DefaultProvider: """Provides access to package resources in the filesystem""" @@ -668,9 +668,11 @@ LINE_END = re.compile(r"\s*(#.*)?$").match # whitespace and comment CONTINUE = re.compile(r"\s*\\\s*(#.*)?$").match # line continuation -DISTRO = re.compile(r"\s*(\w+)").match # Distribution name +DISTRO = re.compile(r"\s*(\w+)").match # Distribution or option VERSION = re.compile(r"\s*(<=?|>=?|==|!=)\s*((\w|\.)+)").match # version info COMMA = re.compile(r"\s*,").match # comma between items +OBRACKET = re.compile(r"\s*\[").match +CBRACKET = re.compile(r"\s*\]").match EGG_NAME = re.compile( r"(?P[^-]+)" @@ -693,8 +695,6 @@ yield '*final' # ensure that alpha/beta/candidate are before final - - def parse_version(s): """Convert a version string to a sortable key @@ -867,15 +867,12 @@ """ # create a steppable iterator, so we can handle \-continuations lines = iter(yield_lines(strs)) - for line in lines: - line = line.replace('-','_') - match = DISTRO(line) - if not match: - raise ValueError("Missing distribution spec", line) - distname = match.group(1) - p = match.end() - specs = [] - while not LINE_END(line,p): + + def scan_list(ITEM,TERMINATOR,line,p,groups,item_name): + + items = [] + + while not TERMINATOR(line,p): if CONTINUE(line,p): try: line = lines.next().replace('-','_'); p = 0 @@ -883,64 +880,111 @@ raise ValueError( "\\ must not appear on the last nonblank line" ) - match = VERSION(line,p) + + match = ITEM(line,p) if not match: - raise ValueError("Expected version spec in",line,"at",line[p:]) - op,val = match.group(1,2) - specs.append((op,val.replace('_','-'))) + raise ValueError("Expected "+item_name+" in",line,"at",line[p:]) + + items.append(match.group(*groups)) p = match.end() + match = COMMA(line,p) if match: p = match.end() # skip the comma - elif not LINE_END(line,p): - raise ValueError("Expected ',' or EOL in",line,"at",line[p:]) + elif not TERMINATOR(line,p): + raise ValueError( + "Expected ',' or end-of-list in",line,"at",line[p:] + ) + + match = TERMINATOR(line,p) + if match: p = match.end() # skip the terminator, if any + return line, p, items + + for line in lines: + line = line.replace('-','_') + match = DISTRO(line) + if not match: + raise ValueError("Missing distribution spec", line) + distname = match.group(1) + p = match.end() + options = [] + + match = OBRACKET(line,p) + if match: + p = match.end() + line, p, options = scan_list(DISTRO,CBRACKET,line,p,(1,),"option") + + line, p, specs = scan_list(VERSION,LINE_END,line,p,(1,2),"version spec") + specs = [(op,val.replace('_','-')) for op,val in specs] + yield Requirement(distname.replace('_','-'), specs, options) + + + + + + + + + + + + + + + + + + + - yield Requirement(distname.replace('_','-'), specs) class Requirement: - def __init__(self, distname, specs=()): + def __init__(self, distname, specs=(), options=()): self.distname = distname self.key = distname.lower() index = [(parse_version(v),state_machine[op],op,v) for op,v in specs] index.sort() self.specs = [(op,ver) for parsed,trans,op,ver in index] - self.index = index + self.index, self.options = index, tuple(options) + self.hashCmp = ( + self.key, tuple([(op,parsed) for parsed,trans,op,ver in index]), + ImmutableSet(map(str.lower,options)) + ) + self.__hash = hash(self.hashCmp) def __str__(self): return self.distname + ','.join([''.join(s) for s in self.specs]) def __repr__(self): - return "Requirement(%r, %r)" % (self.distname, self.specs) + return "Requirement(%r, %r, %r)" % \ + (self.distname,self.specs,self.options) def __eq__(self,other): - return isinstance(other,Requirement) \ - and self.key==other.key and self.specs==other.specs + return isinstance(other,Requirement) and self.hashCmp==other.hashCmp def __contains__(self,item): if isinstance(item,Distribution): - if item.key <> self.key: - return False + if item.key <> self.key: return False item = item.parsed_version elif isinstance(item,basestring): item = parse_version(item) last = True for parsed,trans,op,ver in self.index: action = trans[cmp(item,parsed)] - if action=='F': - return False - elif action=='T': - return True - elif action=='+': - last = True - elif action=='-': - last = False + if action=='F': return False + elif action=='T': return True + elif action=='+': last = True + elif action=='-': last = False return last + def __hash__(self): + return self.__hash + #@staticmethod def parse(s): reqs = list(parse_requirements(s)) @@ -948,10 +992,11 @@ if len(reqs)==1: return reqs[0] raise ValueError("Expected only one requirement", s) - raise ValueError("No requirements found", s) + raise ValueError("No requirements found", s) parse = staticmethod(parse) + state_machine = { # =>< '<' : '--T', @@ -962,6 +1007,7 @@ '!=': 'F..', } + def _get_mro(cls): """Get an mro for a type or classic class""" if not isinstance(cls,type): @@ -969,6 +1015,7 @@ return cls.__mro__[1:] return cls.__mro__ + def _find_adapter(registry, ob): """Return an adapter factory for `ob` from `registry`""" for t in _get_mro(getattr(ob, '__class__', type(ob))): @@ -995,7 +1042,7 @@ for line in yield_lines(s): if line.startswith("["): if line.endswith("]"): - if content: + if section or content: yield section, content section = line[1:-1].strip().lower() content = [] @@ -1005,8 +1052,16 @@ content.append(line) # wrap up last segment - if content: - yield section, content + yield section, content + + + + + + + + + # Set up global resource manager @@ -1023,3 +1078,30 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + From goodger at users.sourceforge.net Mon May 23 02:59:56 2005 From: goodger at users.sourceforge.net (goodger@users.sourceforge.net) Date: Sun, 22 May 2005 17:59:56 -0700 Subject: [Python-checkins] python/nondist/peps pep-0345.txt, NONE, 1.1 pep-0000.txt, 1.308, 1.309 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10012 Modified Files: pep-0000.txt Added Files: pep-0345.txt Log Message: added PEP 345, Medatadata for Python Software Packages 1.2, by Richard Jones --- NEW FILE: pep-0345.txt --- PEP: 345 Title: Medatadata for Python Software Packages 1.2 Version: $Revision: 1.1 $ Last-Modified: $Date: 2005/05/23 00:59:54 $ Author: Richard Jones Discussions-To: Distutils SIG Status: Draft Type: Standards Track Content-type: text/x-rst Created: 28-Apr-2005 Python-Version: 2.5 Post-History: Replaces: 314 Abstract ======== This PEP describes a mechanism for adding metadata to Python packages. It includes specifics of the field names, and their semantics and usage. This document specifies version 1.2 of the metadata format. Version 1.0 is specified in PEP 241. Version 1.1 is specified in PEP 314. Version 1.2 of the metadata format adds a number of optional fields designed to make third-party packaging of Python Software easier. These fields are "Requires-Python" and "Requires-External". Also, the "Metadata-Version" field is updated. Fields ====== This section specifies the names and semantics of each of the supported metadata fields. Fields marked with "(Multiple use)" may be specified multiple times in a single PKG-INFO file. Other fields may only occur once in a PKG-INFO file. Fields marked with "(optional)" are not required to appear in a valid PKG-INFO file; all other fields must be present. Metadata-Version Version of the file format; "1.0", "1.1" and "1.2" are the only legal values here. Example:: Metadata-Version: 1.2 Name The name of the package. Example:: Name: BeagleVote Version A string containing the package's version number. This field should be parseable by one of the Version classes (StrictVersion or LooseVersion) in the distutils.version module. Example:: Version: 1.0a2 Platform (multiple use) A comma-separated list of platform specifications, summarizing the operating systems supported by the package which are not listed in the "Operating System" Trove classifiers. See "Classifier" below. Example:: Platform: ObscureUnix, RareDOS Supported-Platform (multiple use) Binary distributions containing a PKG-INFO file will use the Supported-Platform field in their metadata to specify the OS and CPU for which the binary package was compiled. The semantics of the Supported-Platform field are not specified in this PEP. Example:: Supported-Platform: RedHat 7.2 Supported-Platform: i386-win32-2791 Summary A one-line summary of what the package does. Example:: Summary: A module for collecting votes from beagles. Description (optional) A longer description of the package that can run to several paragraphs. Software that deals with metadata should not assume any maximum size for this field, though people shouldn't include their instruction manual as the description. The contents of this field can be written using reStructuredText markup [1]_. For programs that work with the metadata, supporting markup is optional; programs can also display the contents of the field as-is. This means that authors should be conservative in the markup they use. Example:: Description: This module collects votes from beagles in order to determine their electoral wishes. Do *not* try to use this module with basset hounds; it makes them grumpy. Keywords (optional) A list of additional keywords to be used to assist searching for the package in a larger catalog. Example:: Keywords: dog puppy voting election Home-page (optional) A string containing the URL for the package's home page. Example:: Home-page: http://www.example.com/~cschultz/bvote/ Download-URL A string containing the URL from which this version of the package can be downloaded. (This means that the URL can't be something like ".../package-latest.tgz", but instead must be "../package-0.45.tgz".) Author (optional) A string containing the author's name at a minimum; additional contact information may be provided. Example:: Author: C. Schultz, Universal Features Syndicate, Los Angeles, CA Author-email A string containing the author's e-mail address. It can contain a name and e-mail address in the legal forms for a RFC-822 'From:' header. It's not optional because cataloging systems can use the e-mail portion of this field as a unique key representing the author. A catalog might provide authors the ability to store their GPG key, personal home page, and other additional metadata *about the author*, and optionally the ability to associate several e-mail addresses with the same person. Author-related metadata fields are not covered by this PEP. Example:: Author-email: "C. Schultz" License Text indicating the license covering the package where the license is not a selection from the "License" Trove classifiers. See "Classifier" below. Example:: License: This software may only be obtained by sending the author a postcard, and then the user promises not to redistribute it. Classifier (multiple use) Each entry is a string giving a single classification value for the package. Classifiers are described in PEP 301 [2]. Examples:: Classifier: Development Status :: 4 - Beta Classifier: Environment :: Console (Text Based) Requires (multiple use) Each entry contains a string describing some other module or package required by this package. The format of a requirement string is identical to that of a module or package name usable with the 'import' statement, optionally followed by a version declaration within parentheses. A version declaration is a series of conditional operators and version numbers, separated by commas. Conditional operators must be one of "<", ">", "<=", ">=", "==", and "!=". Version numbers must be in the format accepted by the distutils.version.StrictVersion class: two or three dot-separated numeric components, with an optional "pre-release" tag on the end consisting of the letter 'a' or 'b' followed by a number. Example version numbers are "1.0", "2.3a2", "1.3.99", Any number of conditional operators can be specified, e.g. the string ">1.0, !=1.3.4, <2.0" is a legal version declaration. All of the following are possible requirement strings: "rfc822", "zlib (>=1.1.4)", "zope". There's no canonical list of what strings should be used; the Python community is left to choose its own standards. Example:: Requires: re Requires: sys Requires: zlib Requires: xml.parsers.expat (>1.0) Requires: psycopg Provides (multiple use) Each entry contains a string describing a package or module that will be provided by this package once it is installed. These strings should match the ones used in Requirements fields. A version declaration may be supplied (without a comparison operator); the package's version number will be implied if none is specified. Example:: Provides: xml Provides: xml.utils Provides: xml.utils.iso8601 Provides: xml.dom Provides: xmltools (1.3) Obsoletes (multiple use) Each entry contains a string describing a package or module that this package renders obsolete, meaning that the two packages should not be installed at the same time. Version declarations can be supplied. The most common use of this field will be in case a package name changes, e.g. Gorgon 2.3 gets subsumed into Torqued Python 1.0. When you install Torqued Python, the Gorgon package should be removed. Example:: Obsoletes: Gorgon Requires-Python This field specifies the Python version(s) that the package is guaranteed to be compatible with. The format of the field is a series of conditional operators and version numbers, separated by commas. Conditional operators must be one of "<", ">", "<=", ">=", "==", and "!=". Version numbers must be in the format accepted by the distutils.version.StrictVersion class: two or three dot-separated numeric components, with an optional "pre-release" tag on the end consisting of the letter 'a' or 'b' followed by a number. Example version numbers are "1.0", "2.3a2" and "1.3.99". Any number of conditional operators can be specified, e.g. the string ">1.0, !=1.3.4, <2.0" is a legal version declaration. Example:: Requires-Python: >2.1 Requires-Python: >=2.3.4 XXX This field doesn't take into account possible future incompatibilities through deprecation. We could specify that the ">" operator only work for up to two releases? This is based on the typical deprecation plan that Python usually follows (warning for two releases and then error). That is, >=2.3.4 works for 2.3.4, 2.4 and 2.5. Requires-External (multiple use) Each entry contains a string describing some dependency in the system that the package is to be used. The format of a requirement string is a name of an external dependency, optionally followed by a version declaration within parentheses. A version declaration is a series of conditional operators and version numbers, separated by commas. Conditional operators must be one of "<", ">", "<=", ">=", "==", and "!=". Version numbers must be in the format accepted by the distutils.version.StrictVersion class: two or three dot-separated numeric components, with an optional "pre-release" tag on the end consisting of the letter 'a' or 'b' followed by a number. Example version numbers are "1.0", "2.3a2", "1.3.99", Any number of conditional operators can be specified, e.g. the string ">1.0, !=1.3.4, <2.0" is a legal version declaration. The canonical list of what strings are allowed is available in the `Cheese Shop`_ database. New names may be added to the database either through the web or using the command-line; Python community is left to choose its own standards. Some dependencies are anticipated to be quite broad, eg. "C", indicating a C compiler is required. Example:: Requires: C Requires: libpng Copyright Indicates the party or parties, and the year of copyright covering the package. Example:: Copyright: Guido van Rossum, 1991 Copyright: Python Software Foundation, 2005 Copyright: Public Domain External References Registry ============================ Stores in the `Cheese Shop`_ database a list of (name, description, URI) identifying an external reference that may be used as a value in a Requires-External field. The name and description are required, but URI is not (as there is no single useful URI for "C"). Submissions to the registry are open to the community, and may be performed through the web or using the command-line. The names in the registry will be created under a first-comes first-wins basis. Other packagers of Python software (eg. to deb, rpm, etc) should be able to translate the requires-external field to names in their own packaging system. XXX command-line interface needs work, obviously References ========== This document specifies version 1.2 of the metadata format. Version 1.0 is specified in PEP 241. Version 1.1 is specified in PEP 314. .. [1] reStructuredText markup: http://docutils.sourceforge.net/ .. _`Cheese Shop`: http://cheeseshop.python.org/ Copyright ========= This document has been placed in the public domain. Acknowledgements ================ Fred Drake, Anthony Baxter and Matthias Klose have all contributed to the ideas presented in this PEP. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.308 retrieving revision 1.309 diff -u -d -r1.308 -r1.309 --- pep-0000.txt 16 May 2005 00:34:33 -0000 1.308 +++ pep-0000.txt 23 May 2005 00:59:53 -0000 1.309 @@ -121,6 +121,7 @@ S 342 Enhanced Iterators GvR S 343 Anonymous Block Redux GvR S 344 Exception Chaining and Embedded Tracebacks Yee + S 345 Medatadata for Python Software Packages 1.2 Jones S 754 IEEE 754 Floating Point Special Values Warnes Finished PEPs (done, implemented in CVS) @@ -384,6 +385,7 @@ S 342 Enhanced Iterators GvR S 343 Anonymous Block Redux GvR S 344 Exception Chaining and Embedded Tracebacks Yee + S 345 Medatadata for Python Software Packages 1.2 Jones SR 666 Reject Foolish Indentation Creighton S 754 IEEE 754 Floating Point Special Values Warnes I 3000 Python 3.0 Plans Kuchling, Cannon @@ -445,7 +447,7 @@ Hylton, Jeremy jeremy at zope.com Jansen, Jack jack at cwi.nl Jewett, Jim jimjjewett at users.sourceforge.net - Jones, Richard rjones at ekit-inc.com + Jones, Richard richard at mechanicalcat.net Koltsov, Stepan yozh at mx1.ru Kuchling, A.M. amk at amk.ca Lemburg, Marc-Andre mal at lemburg.com From pje at users.sourceforge.net Mon May 23 03:56:29 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 18:56:29 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.13, 1.14 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17806 Modified Files: pkg_resources.py Log Message: Add tests for AvailableDistributions().resolve(). This effectively completes the core dependency resolution engine; all we need now is a way to turn sys.path entries into "distribution sources" that can list Distribution objects for inclusion in an instance of AvailableDistributions, and the 'require("SomePkg>=2.7")' API will be usable. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- pkg_resources.py 22 May 2005 20:28:46 -0000 1.13 +++ pkg_resources.py 23 May 2005 01:56:26 -0000 1.14 @@ -250,12 +250,12 @@ if path is None: path = sys.path - requirements = list(requirements)[::1] # set up the stack + requirements = list(requirements)[::-1] # set up the stack processed = {} # set of processed requirements best = {} # key -> dist + to_install = [] while requirements: - req = requirements.pop() if req in processed: # Ignore cyclic or redundant dependencies @@ -268,15 +268,16 @@ dist = best[req.key] = self.best_match(req,path) if dist is None: raise DistributionNotFound(req) # XXX put more info here + to_install.append(dist) - elif dist not in requirement: + elif dist not in req: # Oops, the "best" so far conflicts with a dependency raise VersionConflict(req,dist) # XXX put more info here requirements.extend(dist.depends(req.options)[::-1]) processed[req] = True - return best.values() # return list of distros to install + return to_install # return list of distros to install def obtain(self, requirement): @@ -284,7 +285,6 @@ return None # override this in subclasses - class ResourceManager: """Manage resource extraction and packages""" @@ -418,7 +418,6 @@ * get_distro_source() isn't implemented * Distribution.install_on() isn't implemented - * AvailableDistributions.resolve() is untested * AvailableDistributions.scan() is untested There may be other things missing as well, but this definitely won't work @@ -449,6 +448,7 @@ + class DefaultProvider: """Provides access to package resources in the filesystem""" From pje at users.sourceforge.net Mon May 23 03:56:29 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 18:56:29 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/tests test_resources.py, 1.9, 1.10 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17806/setuptools/tests Modified Files: test_resources.py Log Message: Add tests for AvailableDistributions().resolve(). This effectively completes the core dependency resolution engine; all we need now is a way to turn sys.path entries into "distribution sources" that can list Distribution objects for inclusion in an instance of AvailableDistributions, and the 'require("SomePkg>=2.7")' API will be usable. Index: test_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests/test_resources.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- test_resources.py 22 May 2005 20:28:47 -0000 1.9 +++ test_resources.py 23 May 2005 01:56:27 -0000 1.10 @@ -121,6 +121,47 @@ self.checkDepends(self.distDepends(v), v) + def testResolve(self): + ad = AvailableDistributions([]) + + # Resolving no requirements -> nothing to install + self.assertEqual( list(ad.resolve([],[])), [] ) + + # Request something not in the collection -> DistributionNotFound + self.assertRaises( + DistributionNotFound, ad.resolve, parse_requirements("Foo"), [] + ) + + Foo = Distribution.from_filename( + "/foo_dir/Foo-1.2.egg", + metadata=Metadata(('depends.txt', "[bar]\nBaz>=2.0")) + ) + ad.add(Foo) + + # Request thing(s) that are available -> list to install + self.assertEqual( + list(ad.resolve(parse_requirements("Foo"),[])), [Foo] + ) + + # Request an option that causes an unresolved dependency for "Baz" + self.assertRaises( + DistributionNotFound, ad.resolve,parse_requirements("Foo[bar]"),[] + ) + Baz = Distribution.from_filename( + "/foo_dir/Baz-2.1.egg", metadata=Metadata(('depends.txt', "Foo")) + ) + ad.add(Baz) + + # Install list now includes resolved dependency + self.assertEqual( + list(ad.resolve(parse_requirements("Foo[bar]"),[])), [Foo,Baz] + ) + # Requests for conflicting versions produce VersionConflict + self.assertRaises( + VersionConflict, + ad.resolve, parse_requirements("Foo==1.2\nFoo!=1.2"), [] + ) + def testDistroDependsOptions(self): d = self.distDepends(""" Twisted>=1.5 From pje at users.sourceforge.net Mon May 23 04:00:44 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 19:00:44 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.14, 1.15 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18564 Modified Files: pkg_resources.py Log Message: Add a rough draft of Distribution.install_on(), to let others experiment with 'require()' before the "official" version is complete. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- pkg_resources.py 23 May 2005 01:56:26 -0000 1.14 +++ pkg_resources.py 23 May 2005 02:00:42 -0000 1.15 @@ -417,7 +417,6 @@ XXX This doesn't work yet, because: * get_distro_source() isn't implemented - * Distribution.install_on() isn't implemented * AvailableDistributions.scan() is untested There may be other things missing as well, but this definitely won't work @@ -449,6 +448,7 @@ + class DefaultProvider: """Provides access to package resources in the filesystem""" @@ -846,6 +846,11 @@ raise InvalidOption("No such option", self, opt) return deps + def install_on(self,path=None): + # XXX this needs to interface with namespace packages and such + if path is None: path = sys.path + if self.path not in path: + path.append(self.path) def _sort_dists(dists): @@ -854,11 +859,6 @@ dists[::-1] = [d for v,d in tmp] - - - - - def parse_requirements(strs): """Yield ``Requirement`` objects for each specification in `strs` From pje at users.sourceforge.net Mon May 23 04:17:55 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 22 May 2005 19:17:55 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.15, 1.16 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21643 Modified Files: pkg_resources.py Log Message: Make AvailableDistributions check distributions for Python version compatibility as well as platform compatibility. Rename get_distro_source to 'find_distributions', and get rid of intermediate distro-source objects. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- pkg_resources.py 23 May 2005 02:00:42 -0000 1.15 +++ pkg_resources.py 23 May 2005 02:17:52 -0000 1.16 @@ -40,6 +40,7 @@ """Invalid or unrecognized option name for a distribution""" _provider_factories = {} +PY_MAJOR = sys.version[:3] def register_loader_type(loader_type, provider_factory): """Register `provider_factory` to make providers for `loader_type` @@ -76,7 +77,6 @@ return True # easy case # XXX all the tricky cases go here - return False @@ -124,25 +124,16 @@ class AvailableDistributions(object): """Searchable snapshot of distributions on a search path""" - def __init__(self, search_path=None, platform=get_platform()): + def __init__(self,search_path=None,platform=get_platform(),python=PY_MAJOR): """Snapshot distributions available on a search path - `search_path` should be a sequence of ``sys.path`` items. If not - supplied, ``sys.path`` is used. - - The `platform` is an optional string specifying the name of the - platform that platform-specific distributions must be compatible - with. If not specified, it defaults to the current platform - (as defined by the result of ``get_platform()`` when ``pkg_resources`` - was first imported.) - - You may explicitly set `platform` to ``None`` if you wish to map *all* - distributions, not just those compatible with a single platform. + The constructor arguments are the same as those for the ``scan()`` + method; see that method's documentation for details. """ self._distmap = {} self._cache = {} - self.scan(search_path,platform) + self.scan(search_path,platform,python) def __iter__(self): """Iterate over distribution keys""" @@ -162,7 +153,7 @@ else: return default - def scan(self, search_path=None, platform=get_platform()): + def scan(self, search_path=None, platform=get_platform(), python=PY_MAJOR): """Scan `search_path` for distributions usable on `platform` Any distributions found are added to the distribution map. @@ -170,19 +161,27 @@ supplied, ``sys.path`` is used. `platform` is an optional string specifying the name of the platform that platform-specific distributions must be compatible with. If unspecified, it defaults to - the current platform. + the current platform. `python` is an optional string naming the + desired version of Python (e.g. ``'2.4'``); it defaults to the current + version. - You may explicitly set `platform` to ``None`` if you wish to map *all* - distributions, not just those compatible with the running platform. + You may explicitly set `platform` (and/or `python`) to ``None`` if you + wish to map *all* distributions, not just those compatible with the + running platform or Python version. """ + if search_path is None: search_path = sys.path + add = self.add + for item in search_path: - source = get_distro_source(item) - for dist in source.iter_distributions(requirement): + for dist in find_distributions(item): + if python is not None and dist.py_version!=python: + continue if compatible_platforms(dist.platform, platform): - add(dist) # XXX should also check python version! + add(dist) + def __getitem__(self,key): """Return a newest-to-oldest list of distributions for the given key @@ -190,6 +189,7 @@ The returned list may be modified in-place, e.g. for narrowing down usable distributions. """ + try: return self._cache[key] except KeyError: @@ -416,7 +416,7 @@ XXX This doesn't work yet, because: - * get_distro_source() isn't implemented + * find_distributions() isn't implemented * AvailableDistributions.scan() is untested There may be other things missing as well, but this definitely won't work @@ -641,8 +641,8 @@ return StringIO(*args,**kw) -def get_distro_source(path_item): - pass # XXX +def find_distributions(path_item): + return () # XXX @@ -741,7 +741,7 @@ def __init__(self, path_str, metadata=None, name=None, version=None, - py_version=sys.version[:3], platform=None + py_version=PY_MAJOR, platform=None ): if name: self.name = name.replace('_','-') From akuchling at users.sourceforge.net Mon May 23 15:08:11 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Mon, 23 May 2005 06:08:11 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.309, 1.310 pep-0345.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10118 Modified Files: pep-0000.txt pep-0345.txt Log Message: Fix title typo Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.309 retrieving revision 1.310 diff -u -d -r1.309 -r1.310 --- pep-0000.txt 23 May 2005 00:59:53 -0000 1.309 +++ pep-0000.txt 23 May 2005 13:08:09 -0000 1.310 @@ -121,7 +121,7 @@ S 342 Enhanced Iterators GvR S 343 Anonymous Block Redux GvR S 344 Exception Chaining and Embedded Tracebacks Yee - S 345 Medatadata for Python Software Packages 1.2 Jones + S 345 Metadata for Python Software Packages 1.2 Jones S 754 IEEE 754 Floating Point Special Values Warnes Finished PEPs (done, implemented in CVS) @@ -385,7 +385,7 @@ S 342 Enhanced Iterators GvR S 343 Anonymous Block Redux GvR S 344 Exception Chaining and Embedded Tracebacks Yee - S 345 Medatadata for Python Software Packages 1.2 Jones + S 345 Metadata for Python Software Packages 1.2 Jones SR 666 Reject Foolish Indentation Creighton S 754 IEEE 754 Floating Point Special Values Warnes I 3000 Python 3.0 Plans Kuchling, Cannon Index: pep-0345.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0345.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0345.txt 23 May 2005 00:59:54 -0000 1.1 +++ pep-0345.txt 23 May 2005 13:08:09 -0000 1.2 @@ -1,5 +1,5 @@ PEP: 345 -Title: Medatadata for Python Software Packages 1.2 +Title: Metadata for Python Software Packages 1.2 Version: $Revision$ Last-Modified: $Date$ Author: Richard Jones From pje at users.sourceforge.net Tue May 24 03:15:27 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 23 May 2005 18:15:27 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.16, 1.17 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11404 Modified Files: pkg_resources.py Log Message: Implement a draft version of 'find_distributions()' with hardcoded support for packed and unpacked .egg files and .egg-info dirs. This will need to be refactored later to support registering adapters for arbitrary PEP 302 importers, so that it can deal with non-filesystem sys.path entries. In the meantime, however, this means that 'require()' now works and that intrepid souls may now witness the power of this fully operational battle station... Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- pkg_resources.py 23 May 2005 02:17:52 -0000 1.16 +++ pkg_resources.py 24 May 2005 01:15:24 -0000 1.17 @@ -414,13 +414,8 @@ `requirements` must be a string or a (possibly-nested) sequence thereof, specifying the distributions and versions required. - XXX This doesn't work yet, because: - - * find_distributions() isn't implemented - * AvailableDistributions.scan() is untested - - There may be other things missing as well, but this definitely won't work - as long as any of the above items remain unimplemented. + XXX This doesn't support arbitrary PEP 302 sys.path items yet, because + ``find_distributions()`` is hardcoded at the moment. """ requirements = parse_requirements(requirements) @@ -449,15 +444,21 @@ -class DefaultProvider: - """Provides access to package resources in the filesystem""" + + + + +class NullProvider: + """Try to implement resources and metadata for arbitrary PEP 302 loaders""" + + egg_name = None egg_info = None + loader = None def __init__(self, module): - self.module = module self.loader = getattr(module, '__loader__', None) - self.module_path = os.path.dirname(module.__file__) + self.module_path = os.path.dirname(getattr(module, '__file__', '')) def get_resource_filename(self, manager, resource_name): return self._fn(resource_name) @@ -484,33 +485,6 @@ def get_metadata_lines(self, name): return yield_lines(self.get_metadata(name)) - - - - - - - def _has(self, path): - return os.path.exists(path) - - def _get(self, path): - stream = open(path, 'rb') - try: - return stream.read() - finally: - stream.close() - - def _fn(self, resource_name): - return os.path.join(self.module_path, *resource_name.split('/')) - - -register_loader_type(type(None), DefaultProvider) - - - -class NullProvider(DefaultProvider): - """Try to implement resource support for arbitrary PEP 302 loaders""" - def _has(self, path): raise NotImplementedError( "Can't perform this operation for unregistered loader type" @@ -523,27 +497,18 @@ "Can't perform this operation for loaders without 'get_data()'" ) + def _fn(self, resource_name): + return os.path.join(self.module_path, *resource_name.split('/')) register_loader_type(object, NullProvider) +class DefaultProvider(NullProvider): + """Provides access to package resources in the filesystem""" - - - -class ZipProvider(DefaultProvider): - """Resource support for zips and eggs""" - - egg_name = None - eagers = None - - def __init__(self, module): - self.module = module - self.loader = module.__loader__ - self.zipinfo = zipimport._zip_directory_cache[self.loader.archive] - self.zip_pre = self.loader.archive+os.sep - - path = self.module_path = os.path.dirname(module.__file__) + def __init__(self,module): + NullProvider.__init__(self,module) + path = self.module_path old = None self.prefix = [] while path!=old: @@ -555,6 +520,27 @@ path, base = os.path.split(path) self.prefix.append(base) + def _has(self, path): return os.path.exists(path) + + def _get(self, path): + stream = open(path, 'rb') + try: + return stream.read() + finally: + stream.close() + +register_loader_type(type(None), DefaultProvider) + +class ZipProvider(DefaultProvider): + """Resource support for zips and eggs""" + + eagers = None + + def __init__(self, module): + DefaultProvider.__init__(self,module) + self.zipinfo = zipimport._zip_directory_cache[self.loader.archive] + self.zip_pre = self.loader.archive+os.sep + def _short_name(self, path): if path.startswith(self.zip_pre): return path[len(self.zip_pre):] @@ -569,6 +555,20 @@ def get_resource_stream(self, manager, resource_name): return StringIO(self.get_resource_string(manager, resource_name)) + def get_resource_filename(self, manager, resource_name): + if not self.egg_name: + raise NotImplementedError( + "resource_filename() only supported for .egg, not .zip" + ) + + # should lock for extraction here + eagers = self._get_eager_resources() + if resource_name in eagers: + for name in eagers: + self._extract_resource(manager, name) + + return self._extract_resource(manager, resource_name) + @@ -608,27 +608,50 @@ return self.eagers +register_loader_type(zipimport.zipimporter, ZipProvider) +class PathMetadata(DefaultProvider): + """Metadata provider for egg directories - def get_resource_filename(self, manager, resource_name): - if not self.egg_name: - raise NotImplementedError( - "resource_filename() only supported for .egg, not .zip" - ) + Usage:: - # should lock for extraction here - eagers = self._get_eager_resources() - if resource_name in eagers: - for name in eagers: - self._extract_resource(manager, name) + # Development eggs: - return self._extract_resource(manager, resource_name) + egg_info = "/path/to/PackageName.egg-info" + base_dir = os.path.dirname(egg_info) + metadata = PathMetadata(base_dir, egg_info) + dist_name = os.path.splitext(os.path.basename(egg_info))[0] + dist = Distribution(basedir,name=dist_name,metadata=metadata) + # Unpacked egg directories: -register_loader_type(zipimport.zipimporter, ZipProvider) + egg_path = "/path/to/PackageName-ver-pyver-etc.egg" + metadata = PathMetadata(egg_path, os.path.join(egg_path,'EGG-INFO')) + dist = Distribution.from_filename(egg_path, metadata=metadata) + """ + + def __init__(self, path, egg_info): + self.module_path = path + self.egg_info = egg_info + + +class EggMetadata(ZipProvider): + """Metadata provider for .egg files""" + + def __init__(self, importer): + """Create a metadata provider from a zipimporter""" + + self.zipinfo = zipimport._zip_directory_cache[importer.archive] + self.zip_pre = importer.archive+os.sep + self.loader = importer + self.module_path = os.path.join(importer.archive, importer.prefix) + + # we assume here that our metadata may be nested inside a "basket" + # of multiple eggs; that's why we use module_path instead of .archive + self.egg_info = os.path.join(self.module_path, 'EGG-INFO') def StringIO(*args, **kw): @@ -642,16 +665,34 @@ def find_distributions(path_item): - return () # XXX - - - - - - - - - + """Yield distributions accessible via `path_item`""" + if not os.path.exists(path_item): + return + elif os.path.isdir(path_item): + if path_item.lower().endswith('.egg'): + # unpacked egg + yield Distribution.from_filename( + egg_path, metadata=PathMetadata( + path_item,os.path.join(path_item,'EGG-INFO') + ) + ) + else: + # scan for .egg and .egg-info in directory + for entry in os.listdir(path_item): + fullpath = os.path.join(path_item, entry) + if entry.lower().endswith('.egg'): + for dist in find_distributions(fullpath): + yield dist + elif entry.lower().endswith('.egg-info'): + if os.path.isdir(fullpath): + # development egg + metadata = PathMetadata(path_item, fullpath) + dist_name = os.path.splitext(entry)[0] + yield Distribution(path_item,metadata,name=dist_name) + elif path_item.lower().endswith('.egg'): + # packed egg + metadata = EggMetadata(zipimport.zipimporter(path_item)) + yield Distribution.from_filename(path_item, metadata=metadata) def yield_lines(strs): @@ -761,7 +802,7 @@ #@classmethod def from_filename(cls,filename,metadata=None): - name,version,py_version,platform = [None]*4 + name,version,py_version,platform = [None,None,PY_MAJOR,None] basename,ext = os.path.splitext(os.path.basename(filename)) if ext.lower()==".egg": match = EGG_NAME(basename) From pje at users.sourceforge.net Wed May 25 02:50:42 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 24 May 2005 17:50:42 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.17, 1.18 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32239 Modified Files: pkg_resources.py Log Message: Support registering distribution finders for arbitrary PEP 302 importer types, so that the directory scanner isn't a hardcoded case. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- pkg_resources.py 24 May 2005 01:15:24 -0000 1.17 +++ pkg_resources.py 25 May 2005 00:50:39 -0000 1.18 @@ -1,5 +1,4 @@ -"""\ -Package resource API +"""Package resource API -------------------- A resource is a logical file contained within a package, or a logical @@ -21,6 +20,7 @@ 'compatible_platforms', 'get_platform', 'IMetadataProvider', 'ResolutionError', 'VersionConflict', 'DistributionNotFound', 'InvalidOption', 'Distribution', 'Requirement', 'yield_lines', + 'get_importer', 'find_distributions', 'find_on_path', 'register_finder', 'split_sections', # 'glob_resources' ] @@ -654,6 +654,107 @@ self.egg_info = os.path.join(self.module_path, 'EGG-INFO') +class ImpWrapper: + """PEP 302 Importer that wraps Python's "normal" import algorithm""" + + def __init__(self, path=None): + if path is not None and not os.path.isdir(path): + raise ImportError + self.path = path + + def find_module(self, fullname, path=None): + subname = fullname.split(".")[-1] + if subname != fullname and self.path is None: + return None + if self.path is None: + path = None + else: + path = [self.path] + try: + file, filename, etc = imp.find_module(subname, path) + except ImportError: + return None + return ImpLoader(file, filename, etc) + + +class ImpLoader: + """PEP 302 Loader that wraps Python's "normal" import algorithm""" + + def __init__(self, file, filename, etc): + self.file = file + self.filename = filename + self.etc = etc + + def load_module(self, fullname): + try: + mod = imp.load_module(fullname, self.file, self.filename, self.etc) + finally: + if self.file: self.file.close() + # Note: we don't set __loader__ because we want the module to look + # normal; i.e. this is just a wrapper for standard import machinery + return mod + + +def get_importer(path_item): + """Retrieve a PEP 302 "importer" for the given path item + + If there is no importer, this returns a wrapper around the builtin import + machinery. The returned importer is only cached if it was created by a + path hook. + """ + try: + importer = sys.path_importer_cache[path_item] + except KeyError: + for hook in sys.path_hooks: + try: + importer = hook(path_item) + except ImportError: + pass + else: + break + else: + importer = None + + sys.path_importer_cache.setdefault(path_item,importer) + if importer is None: + try: + importer = ImpWrapper(path_item) + except ImportError: + pass + return importer + + + + + + + + + + + + + + +_distribution_finders = {} + +def register_finder(importer_type, distribution_finder): + """Register `distribution_finder` to find distributions in sys.path items + + `importer_type` is the type or class of a PEP 302 "Importer" (sys.path item + handler), and `distribution_finder_type` is a callable that, passed a path + item and the importer instance, yields ``Distribution`` instances found on + that path item. See ``pkg_resources.find_on_path`` for an example.""" + _distribution_finders[importer_type] = distribution_finder + + +def find_distributions(path_item): + """Yield distributions accessible via `path_item`""" + importer = get_importer(path_item) + finder = _find_adapter(_distribution_finders, importer) + return finder(importer,path_item) + + def StringIO(*args, **kw): """Thunk to load the real StringIO on demand""" global StringIO @@ -664,8 +765,26 @@ return StringIO(*args,**kw) -def find_distributions(path_item): - """Yield distributions accessible via `path_item`""" + + + + + + + + + + + + +def find_nothing(importer,path_item): + return () + +register_finder(object,find_nothing) + + +def find_on_path(importer,path_item): + """Yield distributions accessible on a sys.path directory""" if not os.path.exists(path_item): return elif os.path.isdir(path_item): @@ -681,7 +800,7 @@ for entry in os.listdir(path_item): fullpath = os.path.join(path_item, entry) if entry.lower().endswith('.egg'): - for dist in find_distributions(fullpath): + for dist in find_on_path(importer,fullpath): yield dist elif entry.lower().endswith('.egg-info'): if os.path.isdir(fullpath): @@ -694,6 +813,10 @@ metadata = EggMetadata(zipimport.zipimporter(path_item)) yield Distribution.from_filename(path_item, metadata=metadata) +register_finder(ImpWrapper,find_on_path) + + + def yield_lines(strs): """Yield non-empty/non-comment lines of a ``basestring`` or sequence""" From pje at users.sourceforge.net Wed May 25 05:06:56 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 24 May 2005 20:06:56 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.18, 1.19 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22070 Modified Files: pkg_resources.py Log Message: Implement draft support for namespace packages, both declaring them initially and fixing them up when new eggs are added to sys.path. At the moment, all namespace packages are fixed up whenever any new egg is added to sys.path, but this might not scale well if there are lots of eggs and lots of namespace packages. Perhaps we should limit namespace fixups to namespace packages explicitly declared in the egg? Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.18 retrieving revision 1.19 diff -u -d -r1.18 -r1.19 --- pkg_resources.py 25 May 2005 00:50:39 -0000 1.18 +++ pkg_resources.py 25 May 2005 03:06:53 -0000 1.19 @@ -742,7 +742,7 @@ """Register `distribution_finder` to find distributions in sys.path items `importer_type` is the type or class of a PEP 302 "Importer" (sys.path item - handler), and `distribution_finder_type` is a callable that, passed a path + handler), and `distribution_finder` is a callable that, passed a path item and the importer instance, yields ``Distribution`` instances found on that path item. See ``pkg_resources.find_on_path`` for an example.""" _distribution_finders[importer_type] = distribution_finder @@ -818,6 +818,129 @@ +_namespace_handlers = {} +_namespace_packages = {} + +def register_namespace_handler(importer_type, namespace_handler): + """Register `namespace_handler` to declare namespace packages + + `importer_type` is the type or class of a PEP 302 "Importer" (sys.path item + handler), and `namespace_handler` is a callable like this:: + + def namespace_handler(importer,path_entry,moduleName,module): + # return a path_entry to use for child packages + + Namespace handlers are only called if the importer object has already + agreed that it can handle the relevant path item, and they should only + return a subpath if the module __path__ does not already contain an + equivalent subpath. For an example namespace handler, see + ``pkg_resources.file_ns_handler``. + """ + _namespace_handlers[importer_type] = namespace_handler + + +def _handle_ns(packageName, path_item): + """Ensure that named package includes a subpath of path_item (if needed)""" + importer = get_importer(path_item) + loader = importer.find_module(packageName) + if loader is None: + return None + + module = sys.modules.get(packageName) or loader.load_module(packageName) + if not hasattr(module,'__path__'): + raise TypeError("Not a package:", packageName) + + handler = _find_adapter(_distribution_finders, importer) + subpath = handler(importer,path_item,packageName,module) + + if subpath is not None: + module.__path__.append(subpath) + + return subpath + + +def declare_namespace(packageName): + """Declare that package 'packageName' is a namespace package""" + + # XXX nslock.acquire() + try: + if packageName in _namespace_packages: + return + + path, parent = sys.path, None + if '.' in packageName: + parent = '.'.join(package.split('.')[:-1]) + declare_namespace(parent) + __import__(parent) + try: + path = sys.modules[parent].__path__ + except AttributeError: + raise TypeError("Not a package:", parent) + + for path_item in path: + # Ensure all the parent's path items are reflected in the child, + # if they apply + _handle_ns(packageName, path_item) + + # Track what packages are namespaces, so when new path items are added, + # they can be updated + _namespace_packages.setdefault(parent,[]).append(packageName) + _namespace_packages.setdefault(packageName,[]) + + finally: + pass # XXX nslock.release() + +def fixup_namespace_packages(path_item, parent=None): + """Ensure that previously-declared namespace packages include path_item""" + # XXX nslock.acquire() + try: + for package in _namespace_packages.get(parent,()): + subpath = _handle_ns(package, path_item) + if subpath: fixup_namespace_packages(subpath,package) + finally: + pass # XXX nslock.release() + +def file_ns_handler(importer, path_item, packageName, module): + """Compute an ns-package subpath for a filesystem or zipfile importer""" + + subpath = os.path.join(path_item, packageName.split('.')[-1]) + normalized = os.path.normpath(os.path.normcase(subpath)) + for item in module.__path__: + if os.path.normpath(os.path.normcase(item))==normalized: + break + else: + # Only return the path if it's not already there + return subpath + +register_namespace_handler(ImpWrapper,file_ns_handler) +register_namespace_handler(zipimport.zipimporter,file_ns_handler) + + + + + + + + + + + + + + + + + + + + + + + + + + + def yield_lines(strs): """Yield non-empty/non-comment lines of a ``basestring`` or sequence""" if isinstance(strs,basestring): @@ -988,11 +1111,10 @@ return self.__dep_map except AttributeError: dm = self.__dep_map = {None: []} - if self.metadata.has_metadata('depends.txt'): - for section,contents in split_sections( - self.metadata.get_metadata_lines('depends.txt') - ): - dm[section] = list(parse_requirements(contents)) + for section,contents in split_sections( + self._get_metadata('depends.txt') + ): + dm[section] = list(parse_requirements(contents)) return dm _dep_map = property(_dep_map) @@ -1010,18 +1132,19 @@ raise InvalidOption("No such option", self, opt) return deps + def _get_metadata(self,name): + if self.metadata.has_metadata(name): + for line in self.metadata.get_metadata_lines(name): + yield line + def install_on(self,path=None): - # XXX this needs to interface with namespace packages and such + """Ensure distribution is importable on `path` (default=sys.path)""" if path is None: path = sys.path if self.path not in path: path.append(self.path) - - -def _sort_dists(dists): - tmp = [(dist.version,dist) for dist in dists] - tmp.sort() - dists[::-1] = [d for v,d in tmp] - + if path is sys.path: + fixup_namespace_packages(self.path) + map(declare_namespace, self._get_metadata('namespace_packages.txt')) def parse_requirements(strs): """Yield ``Requirement`` objects for each specification in `strs` @@ -1083,10 +1206,10 @@ yield Requirement(distname.replace('_','-'), specs, options) - - - - +def _sort_dists(dists): + tmp = [(dist.version,dist) for dist in dists] + tmp.sort() + dists[::-1] = [d for v,d in tmp] From pje at users.sourceforge.net Wed May 25 05:22:34 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 24 May 2005 20:22:34 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.19, 1.20 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25376 Modified Files: pkg_resources.py Log Message: Fix some typos and a missing import Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.19 retrieving revision 1.20 diff -u -d -r1.19 -r1.20 --- pkg_resources.py 25 May 2005 03:06:53 -0000 1.19 +++ pkg_resources.py 25 May 2005 03:22:32 -0000 1.20 @@ -24,7 +24,7 @@ 'split_sections', # 'glob_resources' ] -import sys, os, zipimport, time, re +import sys, os, zipimport, time, re, imp from sets import ImmutableSet class ResolutionError(Exception): @@ -842,6 +842,8 @@ def _handle_ns(packageName, path_item): """Ensure that named package includes a subpath of path_item (if needed)""" importer = get_importer(path_item) + if importer is None: + return None loader = importer.find_module(packageName) if loader is None: return None @@ -850,12 +852,10 @@ if not hasattr(module,'__path__'): raise TypeError("Not a package:", packageName) - handler = _find_adapter(_distribution_finders, importer) + handler = _find_adapter(_namespace_handlers, importer) subpath = handler(importer,path_item,packageName,module) - if subpath is not None: module.__path__.append(subpath) - return subpath @@ -876,12 +876,12 @@ path = sys.modules[parent].__path__ except AttributeError: raise TypeError("Not a package:", parent) - + for path_item in path: # Ensure all the parent's path items are reflected in the child, # if they apply _handle_ns(packageName, path_item) - + # Track what packages are namespaces, so when new path items are added, # they can be updated _namespace_packages.setdefault(parent,[]).append(packageName) @@ -916,10 +916,10 @@ register_namespace_handler(zipimport.zipimporter,file_ns_handler) +def null_ns_handler(importer, path_item, packageName, module): + return None - - - +register_namespace_handler(object,null_ns_handler) From fdrake at users.sourceforge.net Wed May 25 07:29:19 2005 From: fdrake at users.sourceforge.net (fdrake@users.sourceforge.net) Date: Tue, 24 May 2005 22:29:19 -0700 Subject: [Python-checkins] python/dist/src/Doc/ref ref2.tex,1.56,1.57 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ref In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12418/ref Modified Files: ref2.tex Log Message: update documentation on what constitutes a line in a source file (closes SF bug #1167922) Index: ref2.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ref/ref2.tex,v retrieving revision 1.56 retrieving revision 1.57 diff -u -d -r1.56 -r1.57 --- ref2.tex 10 Nov 2004 16:51:17 -0000 1.56 +++ ref2.tex 25 May 2005 05:29:17 -0000 1.57 @@ -54,11 +54,18 @@ \subsection{Physical lines\label{physical}} -A physical line ends in whatever the current platform's convention is -for terminating lines. On \UNIX, this is the \ASCII{} LF (linefeed) -character. On Windows, it is the \ASCII{} sequence CR LF (return -followed by linefeed). On Macintosh, it is the \ASCII{} CR (return) -character. +A physical line is a sequence of characters terminated by an end-of-line +sequence. In source files, any of the standard platform line +termination sequences can be used - the \UNIX form using \ASCII{} LF +(linefeed), the Windows form using the \ASCII{} sequence CR LF (return +followed by linefeed), or the Macintosh form using the \ASCII{} CR +(return) character. All of these forms can be used equally, regardless +of platform. + +When embedding Python, source code strings should be passed to Python +APIs using the standard C conventions for newline characters (the +\code{\e n} character, representing \ASCII{} LF, is the line +terminator). \subsection{Comments\label{comments}} From fdrake at users.sourceforge.net Wed May 25 07:30:38 2005 From: fdrake at users.sourceforge.net (fdrake@users.sourceforge.net) Date: Tue, 24 May 2005 22:30:38 -0700 Subject: [Python-checkins] python/dist/src/Doc/ref ref2.tex,1.56,1.56.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ref In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12708/ref Modified Files: Tag: release24-maint ref2.tex Log Message: update documentation on what constitutes a line in a source file (closes SF bug #1167922) Index: ref2.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ref/ref2.tex,v retrieving revision 1.56 retrieving revision 1.56.2.1 diff -u -d -r1.56 -r1.56.2.1 --- ref2.tex 10 Nov 2004 16:51:17 -0000 1.56 +++ ref2.tex 25 May 2005 05:30:36 -0000 1.56.2.1 @@ -54,11 +54,18 @@ \subsection{Physical lines\label{physical}} -A physical line ends in whatever the current platform's convention is -for terminating lines. On \UNIX, this is the \ASCII{} LF (linefeed) -character. On Windows, it is the \ASCII{} sequence CR LF (return -followed by linefeed). On Macintosh, it is the \ASCII{} CR (return) -character. +A physical line is a sequence of characters terminated by an end-of-line +sequence. In source files, any of the standard platform line +termination sequences can be used - the \UNIX form using \ASCII{} LF +(linefeed), the Windows form using the \ASCII{} sequence CR LF (return +followed by linefeed), or the Macintosh form using the \ASCII{} CR +(return) character. All of these forms can be used equally, regardless +of platform. + +When embedding Python, source code strings should be passed to Python +APIs using the standard C conventions for newline characters (the +\code{\e n} character, representing \ASCII{} LF, is the line +terminator). \subsection{Comments\label{comments}} From fdrake at users.sourceforge.net Wed May 25 07:39:38 2005 From: fdrake at users.sourceforge.net (fdrake@users.sourceforge.net) Date: Tue, 24 May 2005 22:39:38 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libfuncs.tex, 1.181, 1.182 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14039/lib Modified Files: libfuncs.tex Log Message: fix description of the input range of unichr() (closes SF bug #1120777) Index: libfuncs.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libfuncs.tex,v retrieving revision 1.181 retrieving revision 1.182 diff -u -d -r1.181 -r1.182 --- libfuncs.tex 19 Mar 2005 16:27:33 -0000 1.181 +++ libfuncs.tex 25 May 2005 05:39:36 -0000 1.182 @@ -1077,7 +1077,8 @@ Return the Unicode string of one character whose Unicode code is the integer \var{i}. For example, \code{unichr(97)} returns the string \code{u'a'}. This is the inverse of \function{ord()} for Unicode - strings. The argument must be in the range [0..65535], inclusive. + strings. The valid range for the argument depends how Python was + configured -- it may be either UCS2 [0..0xFFFF] or UCS4 [0..0x10FFFF]. \exception{ValueError} is raised otherwise. \versionadded{2.0} \end{funcdesc} From fdrake at users.sourceforge.net Wed May 25 07:42:58 2005 From: fdrake at users.sourceforge.net (fdrake@users.sourceforge.net) Date: Tue, 24 May 2005 22:42:58 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libfuncs.tex, 1.175.2.2, 1.175.2.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14537/lib Modified Files: Tag: release24-maint libfuncs.tex Log Message: fix description of the input range of unichr() (closes SF bug #1120777) Index: libfuncs.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libfuncs.tex,v retrieving revision 1.175.2.2 retrieving revision 1.175.2.3 diff -u -d -r1.175.2.2 -r1.175.2.3 --- libfuncs.tex 7 Jan 2005 04:35:08 -0000 1.175.2.2 +++ libfuncs.tex 25 May 2005 05:42:51 -0000 1.175.2.3 @@ -1039,7 +1039,8 @@ Return the Unicode string of one character whose Unicode code is the integer \var{i}. For example, \code{unichr(97)} returns the string \code{u'a'}. This is the inverse of \function{ord()} for Unicode - strings. The argument must be in the range [0..65535], inclusive. + strings. The valid range for the argument depends how Python was + configured -- it may be either UCS2 [0..0xFFFF] or UCS4 [0..0x10FFFF]. \exception{ValueError} is raised otherwise. \versionadded{2.0} \end{funcdesc} From mwh at users.sourceforge.net Thu May 26 09:58:25 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Thu, 26 May 2005 00:58:25 -0700 Subject: [Python-checkins] python/dist/src/Doc/ref ref7.tex,1.44,1.45 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ref In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16924 Modified Files: ref7.tex Log Message: Fix: [ 1207501 ] Issue in grammar We didn't define dotted_name in the pseudo-grammar in the reference docs. Backport candidate. Index: ref7.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ref/ref7.tex,v retrieving revision 1.44 retrieving revision 1.45 diff -u -d -r1.44 -r1.45 --- ref7.tex 9 Apr 2005 03:03:00 -0000 1.44 +++ ref7.tex 26 May 2005 07:58:22 -0000 1.45 @@ -323,6 +323,8 @@ {\token{decorator}+} \production{decorator} {"@" \token{dotted_name} ["(" [\token{argument_list} [","]] ")"] NEWLINE} + \production{dotted_name} + {\token{identifier} ("." \token{identifier})*} \production{parameter_list} {(\token{defparameter} ",")*} \productioncont{(~~"*" \token{identifier} [, "**" \token{identifier}]} From rhettinger at users.sourceforge.net Fri May 27 12:43:57 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 27 May 2005 03:43:57 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libstdtypes.tex, 1.177, 1.178 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19557 Modified Files: libstdtypes.tex Log Message: SF bug #1209671: dict.popitem documentation should mention empty dict case Index: libstdtypes.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libstdtypes.tex,v retrieving revision 1.177 retrieving revision 1.178 diff -u -d -r1.177 -r1.178 --- libstdtypes.tex 19 Apr 2005 04:29:44 -0000 1.177 +++ libstdtypes.tex 27 May 2005 10:43:55 -0000 1.178 @@ -1398,7 +1398,8 @@ the dictionary as the value of \var{k}. \var{x} defaults to \var{None}. \item[(6)] \function{popitem()} is useful to destructively iterate -over a dictionary, as often used in set algorithms. +over a dictionary, as often used in set algorithms. If the dictionary +is empty, calling \function{popitem()} raises a \exception{KeyError}. \item[(7)] \function{fromkeys()} is a class method that returns a new dictionary. \var{value} defaults to \code{None}. \versionadded{2.3} From rhettinger at users.sourceforge.net Fri May 27 12:45:17 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 27 May 2005 03:45:17 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libstdtypes.tex, 1.170.2.7, 1.170.2.8 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19754 Modified Files: Tag: release24-maint libstdtypes.tex Log Message: SF bug #1209671: dict.popitem documentation should mention empty dict case Index: libstdtypes.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libstdtypes.tex,v retrieving revision 1.170.2.7 retrieving revision 1.170.2.8 diff -u -d -r1.170.2.7 -r1.170.2.8 --- libstdtypes.tex 19 Apr 2005 04:31:38 -0000 1.170.2.7 +++ libstdtypes.tex 27 May 2005 10:45:14 -0000 1.170.2.8 @@ -1398,7 +1398,8 @@ the dictionary as the value of \var{k}. \var{x} defaults to \var{None}. \item[(6)] \function{popitem()} is useful to destructively iterate -over a dictionary, as often used in set algorithms. +over a dictionary, as often used in set algorithms. If the dictionary +is empty, calling \function{popitem()} raises a \exception{KeyError}. \item[(7)] \function{fromkeys()} is a class method that returns a new dictionary. \var{value} defaults to \code{None}. \versionadded{2.3} From rhettinger at users.sourceforge.net Fri May 27 13:23:47 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 27 May 2005 04:23:47 -0700 Subject: [Python-checkins] python/dist/src/Doc/ref ref7.tex,1.42,1.42.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ref In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25972 Modified Files: Tag: release24-maint ref7.tex Log Message: Backport fix to decorator grammar update. Index: ref7.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ref/ref7.tex,v retrieving revision 1.42 retrieving revision 1.42.2.1 diff -u -d -r1.42 -r1.42.2.1 --- ref7.tex 2 Nov 2004 19:18:20 -0000 1.42 +++ ref7.tex 27 May 2005 11:23:44 -0000 1.42.2.1 @@ -323,6 +323,8 @@ {\token{decorator}+} \production{decorator} {"@" \token{dotted_name} ["(" [\token{argument_list} [","]] ")"] NEWLINE} + \production{dotted_name} + {\token{identifier} ("." \token{identifier})*} \production{parameter_list} {(\token{defparameter} ",")*} \productioncont{(~~"*" \token{identifier} [, "**" \token{identifier}]} From mwh at users.sourceforge.net Fri May 27 16:58:09 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 27 May 2005 07:58:09 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_site.py,1.6,1.7 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9279 Modified Files: test_site.py Log Message: Fix test_site to not call open('...', 'wU'), as that now raises an error. Is anyone running the test suite regularly at the moment? Index: test_site.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_site.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- test_site.py 18 Jul 2004 06:14:50 -0000 1.6 +++ test_site.py 27 May 2005 14:58:06 -0000 1.7 @@ -117,7 +117,7 @@ Make sure to call self.cleanup() to undo anything done by this method. """ - FILE = open(self.file_path, 'wU') + FILE = open(self.file_path, 'w') try: print>>FILE, "#import @bad module name" print>>FILE, "\n" From mwh at users.sourceforge.net Fri May 27 17:21:33 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 27 May 2005 08:21:33 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_float.py,NONE,1.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13607/Lib/test Added Files: test_float.py Log Message: This is my patch: [ 1181301 ] make float packing copy bytes when they can which hasn't been reviewed, despite numerous threats to check it in anyway if noone reviews it. Please read the diff on the checkin list, at least! The basic idea is to examine the bytes of some 'probe values' to see if the current platform is a IEEE 754-ish platform, and if so _PyFloat_{Pack,Unpack}{4,8} just copy bytes around. The rest is hair for testing, and tests. --- NEW FILE: test_float.py --- import unittest, struct from test import test_support class FormatFunctionsTestCase(unittest.TestCase): def setUp(self): self.save_formats = {'double':float.__getformat__('double'), 'float':float.__getformat__('float')} def tearDown(self): float.__setformat__('double', self.save_formats['double']) float.__setformat__('float', self.save_formats['float']) def test_getformat(self): self.assert_(float.__getformat__('double') in ['unknown', 'IEEE, big-endian', 'IEEE, little-endian']) self.assert_(float.__getformat__('float') in ['unknown', 'IEEE, big-endian', 'IEEE, little-endian']) self.assertRaises(ValueError, float.__getformat__, 'chicken') self.assertRaises(TypeError, float.__getformat__, 1) def test_setformat(self): for t in 'double', 'float': float.__setformat__(t, 'unknown') if self.save_formats[t] == 'IEEE, big-endian': self.assertRaises(ValueError, float.__setformat__, t, 'IEEE, little-endian') elif self.save_formats[t] == 'IEEE, little-endian': self.assertRaises(ValueError, float.__setformat__, t, 'IEEE, big-endian') else: self.assertRaises(ValueError, float.__setformat__, t, 'IEEE, big-endian') self.assertRaises(ValueError, float.__setformat__, t, 'IEEE, little-endian') self.assertRaises(ValueError, float.__setformat__, t, 'chicken') self.assertRaises(ValueError, float.__setformat__, 'chicken', 'unknown') BE_DOUBLE_INF = '\x7f\xf0\x00\x00\x00\x00\x00\x00' LE_DOUBLE_INF = ''.join(reversed(BE_DOUBLE_INF)) BE_DOUBLE_NAN = '\x7f\xf8\x00\x00\x00\x00\x00\x00' LE_DOUBLE_NAN = ''.join(reversed(BE_DOUBLE_NAN)) BE_FLOAT_INF = '\x7f\x80\x00\x00' LE_FLOAT_INF = ''.join(reversed(BE_FLOAT_INF)) BE_FLOAT_NAN = '\x7f\xc0\x00\x00' LE_FLOAT_NAN = ''.join(reversed(BE_FLOAT_NAN)) # on non-IEEE platforms, attempting to unpack a bit pattern # representing an infinity or a NaN should raise an exception. class UnknownFormatTestCase(unittest.TestCase): def setUp(self): self.save_formats = {'double':float.__getformat__('double'), 'float':float.__getformat__('float')} float.__setformat__('double', 'unknown') float.__setformat__('float', 'unknown') def tearDown(self): float.__setformat__('double', self.save_formats['double']) float.__setformat__('float', self.save_formats['float']) def test_double_specials_dont_unpack(self): for fmt, data in [('>d', BE_DOUBLE_INF), ('>d', BE_DOUBLE_NAN), ('f', BE_FLOAT_INF), ('>f', BE_FLOAT_NAN), ('d', BE_DOUBLE_INF), ('>d', BE_DOUBLE_NAN), ('f', BE_FLOAT_INF), ('>f', BE_FLOAT_NAN), (' Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14998/Python Modified Files: pythonrun.c Log Message: This is my patch: [ 1181301 ] make float packing copy bytes when they can which hasn't been reviewed, despite numerous threats to check it in anyway if noone reviews it. Please read the diff on the checkin list, at least! The basic idea is to examine the bytes of some 'probe values' to see if the current platform is a IEEE 754-ish platform, and if so _PyFloat_{Pack,Unpack}{4,8} just copy bytes around. The rest is hair for testing, and tests. Index: pythonrun.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/pythonrun.c,v retrieving revision 2.213 retrieving revision 2.214 diff -u -d -r2.213 -r2.214 --- pythonrun.c 29 Mar 2005 13:36:16 -0000 2.213 +++ pythonrun.c 27 May 2005 15:23:20 -0000 2.214 @@ -172,6 +172,8 @@ if (!_PyInt_Init()) Py_FatalError("Py_Initialize: can't init ints"); + _PyFloat_Init(); + interp->modules = PyDict_New(); if (interp->modules == NULL) Py_FatalError("Py_Initialize: can't make modules dictionary"); From mwh at users.sourceforge.net Fri May 27 17:23:22 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 27 May 2005 08:23:22 -0700 Subject: [Python-checkins] python/dist/src/Objects floatobject.c, 2.135, 2.136 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14998/Objects Modified Files: floatobject.c Log Message: This is my patch: [ 1181301 ] make float packing copy bytes when they can which hasn't been reviewed, despite numerous threats to check it in anyway if noone reviews it. Please read the diff on the checkin list, at least! The basic idea is to examine the bytes of some 'probe values' to see if the current platform is a IEEE 754-ish platform, and if so _PyFloat_{Pack,Unpack}{4,8} just copy bytes around. The rest is hair for testing, and tests. Index: floatobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/floatobject.c,v retrieving revision 2.135 retrieving revision 2.136 diff -u -d -r2.135 -r2.136 --- floatobject.c 26 Apr 2005 03:45:25 -0000 2.135 +++ floatobject.c 27 May 2005 15:23:14 -0000 2.136 @@ -983,8 +983,139 @@ return Py_BuildValue("(d)", v->ob_fval); } +/* this is for the benefit of the pack/unpack routines below */ + +typedef enum { + unknown_format, ieee_big_endian_format, ieee_little_endian_format +} float_format_type; + +static float_format_type double_format, float_format; +static float_format_type detected_double_format, detected_float_format; + +static PyObject * +float_getformat(PyTypeObject *v, PyObject* arg) +{ + char* s; + float_format_type r; + + if (!PyString_Check(arg)) { + PyErr_Format(PyExc_TypeError, + "__getformat__() argument must be string, not %.500s", + arg->ob_type->tp_name); + return NULL; + } + s = PyString_AS_STRING(arg); + if (strcmp(s, "double") == 0) { + r = double_format; + } + else if (strcmp(s, "float") == 0) { + r = float_format; + } + else { + PyErr_SetString(PyExc_ValueError, + "__getformat__() argument 1 must be " + "'double' or 'float'"); + return NULL; + } + + switch (r) { + case unknown_format: + return PyString_FromString("unknown"); + case ieee_little_endian_format: + return PyString_FromString("IEEE, little-endian"); + case ieee_big_endian_format: + return PyString_FromString("IEEE, big-endian"); + default: + Py_FatalError("insane float_format or double_format"); + return NULL; + } +} + +PyDoc_STRVAR(float_getformat_doc, +"float.__getformat__(typestr) -> string\n" +"\n" +"You probably don't want to use this function. It exists mainly to be\n" +"used in Python's test suite.\n" +"\n" +"typestr must be 'double' or 'float'. This function returns whichever of\n" +"'unknown', 'IEEE, big-endian' or 'IEEE, little-endian' best describes the\n" +"format of floating point numbers used by the C type named by typestr."); + +static PyObject * +float_setformat(PyTypeObject *v, PyObject* args) +{ + char* typestr; + char* format; + float_format_type f; + float_format_type detected; + float_format_type *p; + + if (!PyArg_ParseTuple(args, "ss:__setformat__", &typestr, &format)) + return NULL; + + if (strcmp(typestr, "double") == 0) { + p = &double_format; + detected = detected_double_format; + } + else if (strcmp(typestr, "float") == 0) { + p = &float_format; + detected = detected_float_format; + } + else { + PyErr_SetString(PyExc_ValueError, + "__setformat__() argument 1 must " + "be 'double' or 'float'"); + return NULL; + } + + if (strcmp(format, "unknown") == 0) { + f = unknown_format; + } + else if (strcmp(format, "IEEE, little-endian") == 0) { + f = ieee_little_endian_format; + } + else if (strcmp(format, "IEEE, big-endian") == 0) { + f = ieee_big_endian_format; + } + else { + PyErr_SetString(PyExc_ValueError, + "__setformat__() argument 2 must be " + "'unknown', 'IEEE, little-endian' or " + "'IEEE, big-endian'"); + return NULL; + + } + + if (f != unknown_format && f != detected) { + PyErr_Format(PyExc_ValueError, + "can only set %s format to 'unknown' or the " + "detected platform value", typestr); + return NULL; + } + + *p = f; + Py_RETURN_NONE; +} + +PyDoc_STRVAR(float_setformat_doc, +"float.__setformat__(typestr, fmt) -> None\n" +"\n" +"You probably don't want to use this function. It exists mainly to be\n" +"used in Python's test suite.\n" +"\n" +"typestr must be 'double' or 'float'. fmt must be one of 'unknown',\n" +"'IEEE, big-endian' or 'IEEE, little-endian', and in addition can only be\n" +"one of the latter two if it appears to match the underlying C reality.\n" +"\n" +"Overrides the automatic determination of C-level floating point type.\n" +"This affects how floats are converted to and from binary strings."); + static PyMethodDef float_methods[] = { {"__getnewargs__", (PyCFunction)float_getnewargs, METH_NOARGS}, + {"__getformat__", (PyCFunction)float_getformat, + METH_O|METH_CLASS, float_getformat_doc}, + {"__setformat__", (PyCFunction)float_setformat, + METH_VARARGS|METH_CLASS, float_setformat_doc}, {NULL, NULL} /* sentinel */ }; @@ -1079,6 +1210,56 @@ }; void +_PyFloat_Init(void) +{ + /* We attempt to determine if this machine is using IEEE + floating point formats by peering at the bits of some + carefully chosen values. If it looks like we are on an + IEEE platform, the float packing/unpacking routines can + just copy bits, if not they resort to arithmetic & shifts + and masks. The shifts & masks approach works on all finite + values, but what happens to infinities, NaNs and signed + zeroes on packing is an accident, and attempting to unpack + a NaN or an infinity will raise an exception. + + Note that if we're on some whacked-out platform which uses + IEEE formats but isn't strictly little-endian or big- + endian, we will fall back to the portable shifts & masks + method. */ + +#if SIZEOF_DOUBLE == 8 + { + double x = 9006104071832581.0; + if (memcmp(&x, "\x43\x3f\xff\x01\x02\x03\x04\x05", 8) == 0) + detected_double_format = ieee_big_endian_format; + else if (memcmp(&x, "\x05\x04\x03\x02\x01\xff\x3f\x43", 8) == 0) + detected_double_format = ieee_little_endian_format; + else + detected_double_format = unknown_format; + } +#else + detected_double_format = unknown_format; +#endif + +#if SIZEOF_FLOAT == 4 + { + float y = 16711938.0; + if (memcmp(&y, "\x4b\x7f\x01\x02", 4) == 0) + detected_float_format = ieee_big_endian_format; + else if (memcmp(&y, "\x02\x01\x7f\x4b", 4) == 0) + detected_float_format = ieee_little_endian_format; + else + detected_float_format = unknown_format; + } +#else + detected_float_format = unknown_format; +#endif + + double_format = detected_double_format; + float_format = detected_float_format; +} + +void PyFloat_Fini(void) { PyFloatObject *p; @@ -1165,306 +1346,395 @@ int _PyFloat_Pack4(double x, unsigned char *p, int le) { - unsigned char sign; - int e; - double f; - unsigned int fbits; - int incr = 1; - - if (le) { - p += 3; - incr = -1; - } + if (float_format == unknown_format) { + unsigned char sign; + int e; + double f; + unsigned int fbits; + int incr = 1; - if (x < 0) { - sign = 1; - x = -x; - } - else - sign = 0; + if (le) { + p += 3; + incr = -1; + } - f = frexp(x, &e); + if (x < 0) { + sign = 1; + x = -x; + } + else + sign = 0; - /* Normalize f to be in the range [1.0, 2.0) */ - if (0.5 <= f && f < 1.0) { - f *= 2.0; - e--; - } - else if (f == 0.0) - e = 0; - else { - PyErr_SetString(PyExc_SystemError, - "frexp() result out of range"); - return -1; - } + f = frexp(x, &e); - if (e >= 128) - goto Overflow; - else if (e < -126) { - /* Gradual underflow */ - f = ldexp(f, 126 + e); - e = 0; - } - else if (!(e == 0 && f == 0.0)) { - e += 127; - f -= 1.0; /* Get rid of leading 1 */ - } + /* Normalize f to be in the range [1.0, 2.0) */ + if (0.5 <= f && f < 1.0) { + f *= 2.0; + e--; + } + else if (f == 0.0) + e = 0; + else { + PyErr_SetString(PyExc_SystemError, + "frexp() result out of range"); + return -1; + } - f *= 8388608.0; /* 2**23 */ - fbits = (unsigned int)(f + 0.5); /* Round */ - assert(fbits <= 8388608); - if (fbits >> 23) { - /* The carry propagated out of a string of 23 1 bits. */ - fbits = 0; - ++e; - if (e >= 255) + if (e >= 128) goto Overflow; - } + else if (e < -126) { + /* Gradual underflow */ + f = ldexp(f, 126 + e); + e = 0; + } + else if (!(e == 0 && f == 0.0)) { + e += 127; + f -= 1.0; /* Get rid of leading 1 */ + } - /* First byte */ - *p = (sign << 7) | (e >> 1); - p += incr; + f *= 8388608.0; /* 2**23 */ + fbits = (unsigned int)(f + 0.5); /* Round */ + assert(fbits <= 8388608); + if (fbits >> 23) { + /* The carry propagated out of a string of 23 1 bits. */ + fbits = 0; + ++e; + if (e >= 255) + goto Overflow; + } - /* Second byte */ - *p = (char) (((e & 1) << 7) | (fbits >> 16)); - p += incr; + /* First byte */ + *p = (sign << 7) | (e >> 1); + p += incr; - /* Third byte */ - *p = (fbits >> 8) & 0xFF; - p += incr; + /* Second byte */ + *p = (char) (((e & 1) << 7) | (fbits >> 16)); + p += incr; - /* Fourth byte */ - *p = fbits & 0xFF; + /* Third byte */ + *p = (fbits >> 8) & 0xFF; + p += incr; - /* Done */ - return 0; + /* Fourth byte */ + *p = fbits & 0xFF; - Overflow: - PyErr_SetString(PyExc_OverflowError, - "float too large to pack with f format"); - return -1; + /* Done */ + return 0; + + Overflow: + PyErr_SetString(PyExc_OverflowError, + "float too large to pack with f format"); + return -1; + } + else { + float y = x; + const char *s = (char*)&y; + int i, incr = 1; + + if ((float_format == ieee_little_endian_format && !le) + || (float_format == ieee_big_endian_format && le)) { + p += 3; + incr = -1; + } + + for (i = 0; i < 4; i++) { + *p = *s++; + p += incr; + } + return 0; + } } int _PyFloat_Pack8(double x, unsigned char *p, int le) { - unsigned char sign; - int e; - double f; - unsigned int fhi, flo; - int incr = 1; + if (double_format == unknown_format) { + unsigned char sign; + int e; + double f; + unsigned int fhi, flo; + int incr = 1; - if (le) { - p += 7; - incr = -1; - } + if (le) { + p += 7; + incr = -1; + } - if (x < 0) { - sign = 1; - x = -x; - } - else - sign = 0; + if (x < 0) { + sign = 1; + x = -x; + } + else + sign = 0; - f = frexp(x, &e); + f = frexp(x, &e); - /* Normalize f to be in the range [1.0, 2.0) */ - if (0.5 <= f && f < 1.0) { - f *= 2.0; - e--; - } - else if (f == 0.0) - e = 0; - else { - PyErr_SetString(PyExc_SystemError, - "frexp() result out of range"); - return -1; - } + /* Normalize f to be in the range [1.0, 2.0) */ + if (0.5 <= f && f < 1.0) { + f *= 2.0; + e--; + } + else if (f == 0.0) + e = 0; + else { + PyErr_SetString(PyExc_SystemError, + "frexp() result out of range"); + return -1; + } - if (e >= 1024) - goto Overflow; - else if (e < -1022) { - /* Gradual underflow */ - f = ldexp(f, 1022 + e); - e = 0; - } - else if (!(e == 0 && f == 0.0)) { - e += 1023; - f -= 1.0; /* Get rid of leading 1 */ - } + if (e >= 1024) + goto Overflow; + else if (e < -1022) { + /* Gradual underflow */ + f = ldexp(f, 1022 + e); + e = 0; + } + else if (!(e == 0 && f == 0.0)) { + e += 1023; + f -= 1.0; /* Get rid of leading 1 */ + } - /* fhi receives the high 28 bits; flo the low 24 bits (== 52 bits) */ - f *= 268435456.0; /* 2**28 */ - fhi = (unsigned int)f; /* Truncate */ - assert(fhi < 268435456); + /* fhi receives the high 28 bits; flo the low 24 bits (== 52 bits) */ + f *= 268435456.0; /* 2**28 */ + fhi = (unsigned int)f; /* Truncate */ + assert(fhi < 268435456); - f -= (double)fhi; - f *= 16777216.0; /* 2**24 */ - flo = (unsigned int)(f + 0.5); /* Round */ - assert(flo <= 16777216); - if (flo >> 24) { - /* The carry propagated out of a string of 24 1 bits. */ - flo = 0; - ++fhi; - if (fhi >> 28) { - /* And it also progagated out of the next 28 bits. */ - fhi = 0; - ++e; - if (e >= 2047) - goto Overflow; + f -= (double)fhi; + f *= 16777216.0; /* 2**24 */ + flo = (unsigned int)(f + 0.5); /* Round */ + assert(flo <= 16777216); + if (flo >> 24) { + /* The carry propagated out of a string of 24 1 bits. */ + flo = 0; + ++fhi; + if (fhi >> 28) { + /* And it also progagated out of the next 28 bits. */ + fhi = 0; + ++e; + if (e >= 2047) + goto Overflow; + } } - } - /* First byte */ - *p = (sign << 7) | (e >> 4); - p += incr; + /* First byte */ + *p = (sign << 7) | (e >> 4); + p += incr; - /* Second byte */ - *p = (unsigned char) (((e & 0xF) << 4) | (fhi >> 24)); - p += incr; + /* Second byte */ + *p = (unsigned char) (((e & 0xF) << 4) | (fhi >> 24)); + p += incr; - /* Third byte */ - *p = (fhi >> 16) & 0xFF; - p += incr; + /* Third byte */ + *p = (fhi >> 16) & 0xFF; + p += incr; - /* Fourth byte */ - *p = (fhi >> 8) & 0xFF; - p += incr; + /* Fourth byte */ + *p = (fhi >> 8) & 0xFF; + p += incr; - /* Fifth byte */ - *p = fhi & 0xFF; - p += incr; + /* Fifth byte */ + *p = fhi & 0xFF; + p += incr; - /* Sixth byte */ - *p = (flo >> 16) & 0xFF; - p += incr; + /* Sixth byte */ + *p = (flo >> 16) & 0xFF; + p += incr; - /* Seventh byte */ - *p = (flo >> 8) & 0xFF; - p += incr; + /* Seventh byte */ + *p = (flo >> 8) & 0xFF; + p += incr; - /* Eighth byte */ - *p = flo & 0xFF; - p += incr; + /* Eighth byte */ + *p = flo & 0xFF; + p += incr; - /* Done */ - return 0; + /* Done */ + return 0; - Overflow: - PyErr_SetString(PyExc_OverflowError, - "float too large to pack with d format"); - return -1; + Overflow: + PyErr_SetString(PyExc_OverflowError, + "float too large to pack with d format"); + return -1; + } + else { + const char *s = (char*)&x; + int i, incr = 1; + + if ((double_format == ieee_little_endian_format && !le) + || (double_format == ieee_big_endian_format && le)) { + p += 7; + incr = -1; + } + + for (i = 0; i < 8; i++) { + *p = *s++; + p += incr; + } + return 0; + } } double _PyFloat_Unpack4(const unsigned char *p, int le) { - unsigned char sign; - int e; - unsigned int f; - double x; - int incr = 1; + if (float_format == unknown_format) { + unsigned char sign; + int e; + unsigned int f; + double x; + int incr = 1; - if (le) { - p += 3; - incr = -1; - } + if (le) { + p += 3; + incr = -1; + } - /* First byte */ - sign = (*p >> 7) & 1; - e = (*p & 0x7F) << 1; - p += incr; + /* First byte */ + sign = (*p >> 7) & 1; + e = (*p & 0x7F) << 1; + p += incr; - /* Second byte */ - e |= (*p >> 7) & 1; - f = (*p & 0x7F) << 16; - p += incr; + /* Second byte */ + e |= (*p >> 7) & 1; + f = (*p & 0x7F) << 16; + p += incr; - /* Third byte */ - f |= *p << 8; - p += incr; + if (e == 255) { + PyErr_SetString( + PyExc_ValueError, + "can't unpack IEEE 754 special value " + "on non-IEEE platform"); + return -1; + } - /* Fourth byte */ - f |= *p; + /* Third byte */ + f |= *p << 8; + p += incr; - x = (double)f / 8388608.0; + /* Fourth byte */ + f |= *p; - /* XXX This sadly ignores Inf/NaN issues */ - if (e == 0) - e = -126; - else { - x += 1.0; - e -= 127; - } - x = ldexp(x, e); + x = (double)f / 8388608.0; - if (sign) - x = -x; + /* XXX This sadly ignores Inf/NaN issues */ + if (e == 0) + e = -126; + else { + x += 1.0; + e -= 127; + } + x = ldexp(x, e); - return x; + if (sign) + x = -x; + + return x; + } + else { + if ((float_format == ieee_little_endian_format && !le) + || (float_format == ieee_big_endian_format && le)) { + char buf[8]; + char *d = &buf[3]; + int i; + + for (i = 0; i < 4; i++) { + *d-- = *p++; + } + return *(float*)&buf[0]; + } + else { + return *(float*)p; + } + } } double _PyFloat_Unpack8(const unsigned char *p, int le) { - unsigned char sign; - int e; - unsigned int fhi, flo; - double x; - int incr = 1; + if (double_format == unknown_format) { + unsigned char sign; + int e; + unsigned int fhi, flo; + double x; + int incr = 1; - if (le) { - p += 7; - incr = -1; - } + if (le) { + p += 7; + incr = -1; + } - /* First byte */ - sign = (*p >> 7) & 1; - e = (*p & 0x7F) << 4; - p += incr; + /* First byte */ + sign = (*p >> 7) & 1; + e = (*p & 0x7F) << 4; + + p += incr; - /* Second byte */ - e |= (*p >> 4) & 0xF; - fhi = (*p & 0xF) << 24; - p += incr; + /* Second byte */ + e |= (*p >> 4) & 0xF; + fhi = (*p & 0xF) << 24; + p += incr; - /* Third byte */ - fhi |= *p << 16; - p += incr; + if (e == 2047) { + PyErr_SetString( + PyExc_ValueError, + "can't unpack IEEE 754 special value " + "on non-IEEE platform"); + return -1.0; + } - /* Fourth byte */ - fhi |= *p << 8; - p += incr; + /* Third byte */ + fhi |= *p << 16; + p += incr; - /* Fifth byte */ - fhi |= *p; - p += incr; + /* Fourth byte */ + fhi |= *p << 8; + p += incr; - /* Sixth byte */ - flo = *p << 16; - p += incr; + /* Fifth byte */ + fhi |= *p; + p += incr; - /* Seventh byte */ - flo |= *p << 8; - p += incr; + /* Sixth byte */ + flo = *p << 16; + p += incr; - /* Eighth byte */ - flo |= *p; + /* Seventh byte */ + flo |= *p << 8; + p += incr; - x = (double)fhi + (double)flo / 16777216.0; /* 2**24 */ - x /= 268435456.0; /* 2**28 */ + /* Eighth byte */ + flo |= *p; - /* XXX This sadly ignores Inf/NaN */ - if (e == 0) - e = -1022; - else { - x += 1.0; - e -= 1023; - } - x = ldexp(x, e); + x = (double)fhi + (double)flo / 16777216.0; /* 2**24 */ + x /= 268435456.0; /* 2**28 */ - if (sign) - x = -x; + if (e == 0) + e = -1022; + else { + x += 1.0; + e -= 1023; + } + x = ldexp(x, e); - return x; + if (sign) + x = -x; + + return x; + } + else { + if ((double_format == ieee_little_endian_format && !le) + || (double_format == ieee_big_endian_format && le)) { + char buf[8]; + char *d = &buf[7]; + int i; + + for (i = 0; i < 8; i++) { + *d-- = *p++; + } + return *(double*)&buf[0]; + } + else { + return *(double*)p; + } + } } From mwh at users.sourceforge.net Fri May 27 17:23:46 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 27 May 2005 08:23:46 -0700 Subject: [Python-checkins] python/dist/src/Include floatobject.h, 2.22, 2.23 pythonrun.h, 2.65, 2.66 Message-ID: Update of /cvsroot/python/python/dist/src/Include In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14998/Include Modified Files: floatobject.h pythonrun.h Log Message: This is my patch: [ 1181301 ] make float packing copy bytes when they can which hasn't been reviewed, despite numerous threats to check it in anyway if noone reviews it. Please read the diff on the checkin list, at least! The basic idea is to examine the bytes of some 'probe values' to see if the current platform is a IEEE 754-ish platform, and if so _PyFloat_{Pack,Unpack}{4,8} just copy bytes around. The rest is hair for testing, and tests. Index: floatobject.h =================================================================== RCS file: /cvsroot/python/python/dist/src/Include/floatobject.h,v retrieving revision 2.22 retrieving revision 2.23 diff -u -d -r2.22 -r2.23 --- floatobject.h 20 Mar 2003 20:53:29 -0000 2.22 +++ floatobject.h 27 May 2005 15:23:13 -0000 2.23 @@ -55,13 +55,18 @@ * routines produce a C double from such a string. The suffix (4 or 8) * specifies the number of bytes in the string. * - * Excepting NaNs and infinities (which aren't handled correctly), the 4- - * byte format is identical to the IEEE-754 single precision format, and - * the 8-byte format to the IEEE-754 double precision format. On non- - * IEEE platforms with more precision, or larger dynamic range, than - * 754 supports, not all values can be packed; on non-IEEE platforms with - * less precision, or smaller dynamic range, not all values can be - * unpacked. What happens in such cases is partly accidental (alas). + * On platforms that appear to use (see _PyFloat_Init()) IEEE-754 formats + * these functions work by copying bits. On other platforms, the formats the + * 4- byte format is identical to the IEEE-754 single precision format, and + * the 8-byte format to the IEEE-754 double precision format, although the + * packing of INFs and NaNs (if such things exist on the platform) isn't + * handled correctly, and attempting to unpack a string containing an IEEE + * INF or NaN will raise an exception. + * + * On non-IEEE platforms with more precision, or larger dynamic range, than + * 754 supports, not all values can be packed; on non-IEEE platforms with less + * precision, or smaller dynamic range, not all values can be unpacked. What + * happens in such cases is partly accidental (alas). */ /* The pack routines write 4 or 8 bytes, starting at p. le is a bool @@ -70,8 +75,9 @@ * first, at p). * Return value: 0 if all is OK, -1 if error (and an exception is * set, most likely OverflowError). - * Bug: What this does is undefined if x is a NaN or infinity. - * Bug: -0.0 and +0.0 produce the same string. + * There are two problems on non-IEEE platforms: + * 1): What this does is undefined if x is a NaN or infinity. + * 2): -0.0 and +0.0 produce the same string. */ PyAPI_FUNC(int) _PyFloat_Pack4(double x, unsigned char *p, int le); PyAPI_FUNC(int) _PyFloat_Pack8(double x, unsigned char *p, int le); @@ -81,9 +87,8 @@ * last, at p+3 or p+7), false if big-endian (exponent first, at p). * Return value: The unpacked double. On error, this is -1.0 and * PyErr_Occurred() is true (and an exception is set, most likely - * OverflowError). - * Bug: What this does is undefined if the string represents a NaN or - * infinity. + * OverflowError). Note that on a non-IEEE platform this will refuse + * to unpack a string that represents a NaN or infinity. */ PyAPI_FUNC(double) _PyFloat_Unpack4(const unsigned char *p, int le); PyAPI_FUNC(double) _PyFloat_Unpack8(const unsigned char *p, int le); Index: pythonrun.h =================================================================== RCS file: /cvsroot/python/python/dist/src/Include/pythonrun.h,v retrieving revision 2.65 retrieving revision 2.66 diff -u -d -r2.65 -r2.66 --- pythonrun.h 7 Oct 2004 03:58:06 -0000 2.65 +++ pythonrun.h 27 May 2005 15:23:13 -0000 2.66 @@ -105,6 +105,7 @@ PyAPI_FUNC(void) _PyImportHooks_Init(void); PyAPI_FUNC(int) _PyFrame_Init(void); PyAPI_FUNC(int) _PyInt_Init(void); +PyAPI_FUNC(void) _PyFloat_Init(void); /* Various internal finalizers */ PyAPI_FUNC(void) _PyExc_Fini(void); From goodger at users.sourceforge.net Sun May 29 00:44:27 2005 From: goodger at users.sourceforge.net (goodger@users.sourceforge.net) Date: Sat, 28 May 2005 15:44:27 -0700 Subject: [Python-checkins] python/nondist/peps pep-0346.txt, NONE, 1.1 pep-0000.txt, 1.310, 1.311 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23178 Modified Files: pep-0000.txt Added Files: pep-0346.txt Log Message: added PEP 346, User Defined ("with") Statements, by Nick Coghlan; withdrawn by the author --- NEW FILE: pep-0346.txt --- PEP: 346 Title: User Defined ("``with``") Statements Version: $Revision: 1.1 $ Last-Modified: $Date: 2005/05/28 22:44:24 $ Author: Nick Coghlan Status: Withdrawn Type: Standards Track Content-Type: text/x-rst Created: 6-May-2005 Python-Version: 2.5 Post-History: Abstract ======== This PEP is a combination of PEP 310's "Reliable Acquisition/Release Pairs" with the "Anonymous Block Statements" of Guido's PEP 340. This PEP aims to take the good parts of PEP 340, blend them with parts of [...1264 lines suppressed...] .. [7] A rant against flow control macros (http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx) .. [8] Why doesn't C# have a 'with' statement? (http://msdn.microsoft.com/vcsharp/programming/language/ask/withstatement/) Copyright ========= This document has been placed in the public domain. .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.310 retrieving revision 1.311 diff -u -d -r1.310 -r1.311 --- pep-0000.txt 23 May 2005 13:08:09 -0000 1.310 +++ pep-0000.txt 28 May 2005 22:44:24 -0000 1.311 @@ -213,6 +213,7 @@ SR 326 A Case for Top and Bottom Values Carlson, Reedy SR 329 Treating Builtins as Constants in the Standard Library Hettinger SR 340 Anonymous Block Statements GvR + SR 346 User Defined ("with") Statements Coghlan SR 666 Reject Foolish Indentation Creighton @@ -386,6 +387,7 @@ S 343 Anonymous Block Redux GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones + SR 346 User Defined ("with") Statements Coghlan SR 666 Reject Foolish Indentation Creighton S 754 IEEE 754 Floating Point Special Values Warnes I 3000 Python 3.0 Plans Kuchling, Cannon @@ -419,7 +421,7 @@ Cannon, Brett brett at python.org Carlson, Josiah jcarlson at uci.edu Carroll, W Isaac icarroll at pobox.com - Coghlan, Nick ncoghlan at email.com + Coghlan, Nick ncoghlan at gmail.com Cole, Dave djc at object-craft.com.au Craig, Christopher python-pep at ccraig.org Creighton, Laura lac at strakt.com From pje at users.sourceforge.net Sun May 29 01:06:02 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 28 May 2005 16:06:02 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.7, 1.8 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31303/setuptools/command Modified Files: bdist_egg.py Log Message: Add experimental 'install_data' support to 'bdist_egg'. The most common distutils custom command hack in the field is to make 'install_data' put data in with the target packages by changing the --install-data to match --install-lib, so this should let bdist_egg work with more packages "out of the box". Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- bdist_egg.py 3 Apr 2005 18:52:21 -0000 1.7 +++ bdist_egg.py 28 May 2005 23:05:59 -0000 1.8 @@ -96,35 +96,18 @@ f.close() - - - - - - - - - - - - - - - - - - - - - - - - - def run(self): if not self.skip_build: self.run_command('build') + if self.distribution.data_files: + install = self.reinitialize_command('install_data') + install.install_dir = self.bdist_dir + install.force = 0 + install.root = None + log.info("installing package data to %s" % self.bdist_dir) + self.run_command('install_data') + install = self.reinitialize_command('install_lib', reinit_subcommands=1) install.install_dir = self.bdist_dir install.skip_build = self.skip_build @@ -134,9 +117,10 @@ install._mutate_outputs(self.distribution.has_ext_modules(), 'build_ext', 'build_lib', '') - log.info("installing to %s" % self.bdist_dir) + log.info("installing library code to %s" % self.bdist_dir) self.run_command('install_lib') + to_compile = [] for ext_name in ext_outputs: filename,ext = os.path.splitext(ext_name) @@ -174,6 +158,10 @@ if not self.dry_run: self.distribution.metadata.write_pkg_info(self.egg_info) + + + + native_libs = os.path.join(self.egg_info,"native_libs.txt") if ext_outputs: log.info("writing %s" % native_libs) @@ -193,6 +181,8 @@ if os.path.isfile(path): self.copy_file(path,os.path.join(egg_info,filename)) + + # Make the archive make_zipfile(pseudoinstall_root+'.egg', archive_root, verbose=self.verbose, @@ -203,6 +193,16 @@ + + + + + + + + + + def make_zipfile (zip_filename, base_dir, verbose=0, dry_run=0): """Create a zip file from all the files under 'base_dir'. The output zip file will be named 'base_dir' + ".zip". Uses either the "zipfile" From rhettinger at users.sourceforge.net Sun May 29 01:20:11 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 28 May 2005 16:20:11 -0700 Subject: [Python-checkins] python/dist/src/Misc developers.txt,1.11,1.12 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7539 Modified Files: developers.txt Log Message: Add permissions for Reinhold Birkenfeld. Index: developers.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/developers.txt,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- developers.txt 30 Apr 2005 21:25:19 -0000 1.11 +++ developers.txt 28 May 2005 23:20:09 -0000 1.12 @@ -17,6 +17,11 @@ Permissions History ------------------- +- Reinhold Birkenfeld was given SF tracker permissions on 28 May 2005 + by RDH. Since the beginning of 2005, he has been active in discussions + on python-dev and has submitted a dozen patch reviews. The permissions + add the ability to change tracker status and to attach patches. + - Terry Reedy was given SF tracker permissions on 7 Apr 2005 by RDH. - Nick Coghlan was given SF tracker permissions on 5 Apr 2005 by RDH. From pje at users.sourceforge.net Sun May 29 02:17:25 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 28 May 2005 17:17:25 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, NONE, 1.1 pkg_resources.py, 1.20, 1.21 setup.py, 1.4, 1.5 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3039 Modified Files: pkg_resources.py setup.py Added Files: easy_install.py Log Message: Add "easy_install" script that downloads distutils source (or .egg files) and builds and installs them as eggs, with support for managing .pth files. Built distributions are installed in individual subdirectories, so you can either add the directory to a .pth (automatically done by default), or you can use pkg_resources.require() to manage your dependencies explicitly. Because each distribution is in its own directory (or .egg file), uninstallation and clean upgrades are trivial, without the aid of any sort of package manager. --- NEW FILE: easy_install.py --- #!python.exe import site #!python """\ Easy Install ============ Easy Install is a python module (easy_install) that lets you automatically download, build, install, and manage Python packages. .. contents:: **Table of Contents** Downloading and Installing a Package ------------------------------------ For basic use of ``easy_install``, you need only supply the filename or URL of a source distribution or .egg file (Python Egg). **Example 1**. Download a source distribution, automatically building and installing it:: easy_install http://example.com/path/to/MyPackage-1.2.3.tgz **Example 2**. Install an already-downloaded .egg file:: easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg Easy Install recognizes distutils *source* (not binary) distribution files with extensions of .tgz, .tar, .tar.gz, .tar.bz2, or .zip. And of course it handles already-built .egg distributions. By default, packages are installed to the running Python installation's ``site-packages`` directory, unless you provide the ``-d`` or ``--install-dir`` option to specify an alternative directory. Packages installed to ``site-packages`` are added to an ``easy-install.pth`` file, so that Python will be able to import the package by default. If you do not want this to happen, you should use the ``-m`` or ``--multi`` option, which allows multiple versions of the same package to be selected at runtime. Note that installing to a directory other than ``site-packages`` already implies the ``-m`` option, so if you cannot install to ``site-packages``, please see the `Options`_ section below (under ``--multi``) to find out how to select packages at runtime. Upgrading a Package ------------------- You don't need to do anything special to upgrade a package: just install the new version. If you're using ``-m`` or ``--multi`` (or installing outside of ``site-packages``), the runtime system automatically selects the newest available version of a package. If you're installing to ``site-packages`` and not using ``-m``, installing a package automatically replaces its older version in the ``easy-install.pth`` file, so that Python will import the latest version by default. ``easy_install`` never actually deletes packages (unless you're installing a package with the same name and version number as an existing package), so if you want to get rid of older versions of a package, please see `Uninstalling Packages`_, below. Changing the Active Version (``site-packages`` installs only) ------------------------------------------------------------- If you've upgraded a package, but need to revert to a previously-installed version, you can do so like this:: easy_install PackageName==1.2.3 Where ``1.2.3`` is replaced by the exact version number you wish to switch to. Note that the named package and version must already have been installed to ``site-packages``. If you'd like to switch to the latest version of ``PackageName``, you can do so like this:: easy_install PackageName This will activate the latest installed version. Uninstalling Packages --------------------- If you have replaced a package with another version, then you can just delete the package(s) you don't need by deleting the PackageName-versioninfo.egg file or directory (found in the installation directory). If you want to delete the currently installed version of a package (or all versions of a package), you should first run:: easy_install -m PackageName This will ensure that Python doesn't continue to search for a package you're planning to remove. After you've done this, you can safely delete the .egg files or directories. Options ------- ``--zip, -z`` Enable installing the package as a zip file. This can significantly increase Python's overall import performance if you're installing to ``site-packages`` and not using the ``--multi`` option, because Python process zipfile entries on ``sys.path`` much faster than it does directories. So, if you don't use this option, and you install a lot of packages, some of them may be slower to import. But, this option is disabled by default, unless you're installing from an already-built binary zipfile (``.egg`` file). This is to avoid problems when using packages that dosn't support running from a zip file. Such packages usually access data files in their package directories using the Python ``__file__`` or ``__path__`` attribute, instead of the ``pkg_resources`` API. So, if you find that a package doesn't work properly when used with this option, you may want to suggest to the author that they switch to using the ``pkg_resources`` resource API, which will allow their package to work whether it's installed as a zipfile or not. (Note: this option only affects the installation of newly-built packages that are not already installed in the target directory; if you want to convert an existing installed version from zipped to unzipped or vice versa, you'll need to delete the existing version first.) ``--multi-version, -m`` "Multi-version" mode. Specifying this option prevents ``easy_install`` from adding an ``easy-install.pth`` entry for the package being installed, and if an entry for any version the package already exists, it will be removed upon successful installation. In multi-version mode, no specific version of the package is available for importing, unless you use ``pkg_resources.require()`` to put it on ``sys.path``. This can be as simple as:: from pkg_resources import require require("SomePackage", "OtherPackage", "MyPackage") which will put the latest installed version of the specified packages on ``sys.path`` for you. (For more advanced uses, like selecting specific versions and enabling optional dependencies, see the ``pkg_resources`` API doc.) Note that if you install to a directory other than ``site-packages``, this option is automatically in effect, because ``.pth`` files can only be used in ``site-packages`` (at least in Python 2.3 and 2.4). So, if you use the ``--install-dir`` or ``-i`` options, you must also use ``require()`` to enable packages at runtime ``--install-dir=DIR, -d DIR`` Set the installation directory. It is up to you to ensure that this directory is on ``sys.path`` at runtime, and to use ``pkg_resources.require()`` to enable the installed package(s) that you need. """ import sys, os.path, pkg_resources, re, zipimport from pkg_resources import * class Installer: """Manage a download/build/install process""" pth_file = None def __init__(self, instdir=None, zip_ok=False, multi=None, tmpdir=None): from distutils.sysconfig import get_python_lib site_packages = get_python_lib() if tmpdir is None: from tempfile import mkdtemp tmpdir = mkdtemp(prefix="easy_install-") self.tmpdir = tmpdir if instdir is None or self.samefile(site_packages,instdir): instdir = site_packages self.pth_file = PthDistributions( os.path.join(instdir,'easy-install.pth') ) elif multi is None: multi = True elif not multi: # explicit false, raise an error raise RuntimeError( "Can't do single-version installs outside site-packages" ) self.instdir = instdir self.zip_ok = zip_ok self.multi = multi def close(self): if os.path.isdir(self.tmpdir): from shutil import rmtree rmtree(self.tmpdir,True) def __del__(self): self.close() def samefile(self,p1,p2): try: os.path.samefile except AttributeError: return ( os.path.normpath(os.path.normcase(p1)) == os.path.normpath(os.path.normcase(p2)) ) else: return os.path.samefile(p1,p2) def download(self, spec): """Locate and/or download or `spec`, returning a local filename""" if isinstance(spec,Requirement): pass else: scheme = URL_SCHEME(spec) if scheme: # It's a url, download it to self.tmpdir return self._download_url(scheme, spec) elif os.path.exists(spec): # Existing file or directory, just return it return spec else: try: spec = Requirement.parse(spec) except ValueError: raise RuntimeError( "Not a URL, existing file, or requirement spec: %r" % (spec,) ) # process a Requirement dist = AvailableDistributions().best_match(spec,[]) if dist is not None and dist.path.endswith('.egg'): return dist.path # TODO: search here for a distro to download raise DistributionNotFound(spec) def install_eggs(self, dist_filename): # .egg dirs or files are already built, so just return them if dist_filename.lower().endswith('.egg'): return self.install_egg(dist_filename,True) # Anything else, try to extract and build import zipfile, tarfile if zipfile.is_zipfile(dist_filename): self._extract_zip(dist_filename, self.tmpdir) else: import tarfile try: tar = tarfile.open(dist_filename) except tarfile.TarError: raise RuntimeError( "Not a valid tar or zip archive: %s" % dist_filename ) else: self._extract_tar(tar) # Find the setup.py file from glob import glob setup_script = os.path.join(self.tmpdir, 'setup.py') if not os.path.exists(setup_script): setups = glob(os.path.join(self.tmpdir, '*', 'setup.py')) if not setups: raise RuntimeError( "Couldn't find a setup script in %s" % dist_filename ) if len(setups)>1: raise RuntimeError( "Multiple setup scripts in %s" % dist_filename ) setup_script = setups[0] self._run_setup(setups[0]) for egg in glob( os.path.join(os.path.dirname(setup_script),'dist','*.egg') ): self.install_egg(egg, self.zip_ok) def _extract_zip(self,zipname,extract_dir): import zipfile z = zipfile.ZipFile(zipname) try: for info in z.infolist(): name = info.filename # don't extract absolute paths or ones with .. in them if name.startswith('/') or '..' in name: continue target = os.path.join(extract_dir,name) if name.endswith('/'): # directory ensure_directory(target) else: # file ensure_directory(target) data = z.read(info.filename) f = open(target,'wb') try: f.write(data) finally: f.close() del data finally: z.close() def _extract_tar(self,tarobj): try: tarobj.chown = lambda *args: None # don't do any chowning! for member in tarobj: if member.isfile() or member.isdir(): name = member.name # don't extract absolute paths or ones with .. in them if not name.startswith('/') and '..' not in name: tarobj.extract(member,self.tmpdir) finally: tarobj.close() def _run_setup(self, setup_script): from setuptools.command import bdist_egg sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) old_dir = os.getcwd() save_argv = sys.argv[:] save_path = sys.path[:] try: os.chdir(os.path.dirname(setup_script)) try: sys.argv[:] = [setup_script, '-q', 'bdist_egg'] sys.path.insert(0,os.getcwd()) execfile(setup_script, {'__file__':setup_script}) except SystemExit, v: if v.args and v.args[0]: raise RuntimeError( "Setup script exited with %s" % (v.args[0],) ) finally: os.chdir(old_dir) sys.path[:] = save_path sys.argv[:] = save_argv def install_egg(self, egg_path, zip_ok): import shutil destination = os.path.join(self.instdir, os.path.basename(egg_path)) ensure_directory(destination) if not self.samefile(egg_path, destination): if os.path.isdir(destination): shutil.rmtree(destination) elif os.path.isfile(destination): os.unlink(destination) if zip_ok: if egg_path.startswith(self.tmpdir): shutil.move(egg_path, destination) else: shutil.copy2(egg_path, destination) elif os.path.isdir(egg_path): shutil.move(egg_path, destination) else: os.mkdir(destination) self._extract_zip(egg_path, destination) if self.pth_file is not None: if os.path.isdir(destination): dist = Distribution.from_filename( destination, metadata=PathMetadata( destination, os.path.join(destination,'EGG-INFO') ) ) else: metadata = EggMetadata(zipimport.zipimporter(destination)) dist = Distribution.from_filename(destination,metadata=metadata) # remove old map(self.pth_file.remove, self.pth_file.get(dist.key,())) if not self.multi: self.pth_file.add(dist) # add new self.pth_file.save() def _download_url(self, scheme, url): # Determine download filename from urlparse import urlparse name = filter(None,urlparse(url)[2].split('/'))[-1] while '..' in name: name = name.replace('..','.').replace('\\','_') # Download the file from urllib import FancyURLopener, URLopener class opener(FancyURLopener): http_error_default = URLopener.http_error_default try: filename,headers = opener().retrieve( url,os.path.join(self.tmpdir,name) ) except IOError,v: if v.args and v.args[0]=='http error': raise RuntimeError( "Download error: %s %s" % v.args[1:3] ) else: raise # and return its filename return filename class PthDistributions(AvailableDistributions): """A .pth file with Distribution paths in it""" dirty = False def __init__(self, filename): self.filename = filename; self._load() AvailableDistributions.__init__( self, list(yield_lines(self.paths)), None, None ) def _load(self): self.paths = [] if os.path.isfile(self.filename): self.paths = [line.rstrip() for line in open(self.filename,'rt')] while self.paths and not self.paths[-1].strip(): self.paths.pop() def save(self): """Write changed .pth file back to disk""" if self.dirty: data = '\n'.join(self.paths+['']) f = open(self.filename,'wt') f.write(data) f.close() self.dirty = False def add(self,dist): """Add `dist` to the distribution map""" if dist.path not in self.paths: self.paths.append(dist.path); self.dirty = True AvailableDistributions.add(self,dist) def remove(self,dist): """Remove `dist` from the distribution map""" while dist.path in self.paths: self.paths.remove(dist.path); self.dirty = True AvailableDistributions.remove(self,dist) URL_SCHEME = re.compile('([-+.a-z0-9]{2,}):',re.I).match def main(argv): from optparse import OptionParser parser = OptionParser(usage = "usage: %prog [options] url [url...]") parser.add_option("-d", "--install-dir", dest="instdir", default=None, help="install package to DIR", metavar="DIR") parser.add_option("-z", "--zip", action="store_true", dest="zip_ok", default=False, help="install package as a zipfile") parser.add_option("-m", "--multi-version", action="store_true", dest="multi", default=None, help="make apps have to require() a version") (options, args) = parser.parse_args() try: if not args: raise RuntimeError("No urls, filenames, or requirements specified") for spec in args: inst = Installer(options.instdir, options.zip_ok, options.multi) try: print "Downloading", spec downloaded = inst.download(spec) print "Installing", os.path.basename(downloaded) inst.install_eggs(downloaded) finally: inst.close() except RuntimeError, v: parser.error(str(v)) if __name__ == '__main__': main(sys.argv[1:]) Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.20 retrieving revision 1.21 diff -u -d -r1.20 -r1.21 --- pkg_resources.py 25 May 2005 03:22:32 -0000 1.20 +++ pkg_resources.py 29 May 2005 00:17:22 -0000 1.21 @@ -13,15 +13,15 @@ method. """ __all__ = [ - 'register_loader_type', 'get_provider', 'IResourceProvider', + 'register_loader_type', 'get_provider', 'IResourceProvider','PathMetadata', 'ResourceManager', 'AvailableDistributions', 'require', 'resource_string', 'resource_stream', 'resource_filename', 'set_extraction_path', - 'cleanup_resources', 'parse_requirements', 'parse_version', - 'compatible_platforms', 'get_platform', 'IMetadataProvider', - 'ResolutionError', 'VersionConflict', 'DistributionNotFound', + 'cleanup_resources', 'parse_requirements', 'ensure_directory', + 'compatible_platforms', 'get_platform', 'IMetadataProvider','parse_version', + 'ResolutionError', 'VersionConflict', 'DistributionNotFound','EggMetadata', 'InvalidOption', 'Distribution', 'Requirement', 'yield_lines', 'get_importer', 'find_distributions', 'find_on_path', 'register_finder', - 'split_sections', # 'glob_resources' + 'split_sections', 'declare_namespace', 'register_namespace_handler', ] import sys, os, zipimport, time, re, imp @@ -342,7 +342,7 @@ extract_path = self.extraction_path extract_path = extract_path or os.path.expanduser('~/.python-eggs') target_path = os.path.join(extract_path, archive_name, *names) - _ensure_directory(target_path) + ensure_directory(target_path) self.cached_files.append(target_path) return target_path @@ -791,7 +791,7 @@ if path_item.lower().endswith('.egg'): # unpacked egg yield Distribution.from_filename( - egg_path, metadata=PathMetadata( + path_item, metadata=PathMetadata( path_item,os.path.join(path_item,'EGG-INFO') ) ) @@ -1310,7 +1310,7 @@ return registry[t] -def _ensure_directory(path): +def ensure_directory(path): dirname = os.path.dirname(path) if not os.path.isdir(dirname): os.makedirs(dirname) Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- setup.py 3 Apr 2005 17:23:22 -0000 1.4 +++ setup.py 29 May 2005 00:17:22 -0000 1.5 @@ -7,7 +7,7 @@ setup( name="setuptools", - version="0.0.1", + version="0.3a1", description="Distutils enhancements", author="Phillip J. Eby", @@ -22,6 +22,7 @@ Require('PyUnit', None, 'unittest', "http://pyunit.sf.net/"), ], packages = find_packages(), - py_modules = ['pkg_resources'], + py_modules = ['pkg_resources', 'easy_install'], + scripts = ['easy_install.py'] ) From pje at users.sourceforge.net Sun May 29 02:18:27 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 28 May 2005 17:18:27 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4736 Modified Files: easy_install.py Log Message: Remove exemaker spew from easy_install.py Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- easy_install.py 29 May 2005 00:17:22 -0000 1.1 +++ easy_install.py 29 May 2005 00:18:25 -0000 1.2 @@ -1,6 +1,3 @@ -#!python.exe -import site - #!python """\ Easy Install From pje at users.sourceforge.net Sun May 29 03:36:04 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 28 May 2005 18:36:04 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10742 Modified Files: easy_install.py Log Message: Add link to Python Eggs page in doc; fix a problem with non-standard source distros (where setup.py is in the archive root). Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- easy_install.py 29 May 2005 00:18:25 -0000 1.2 +++ easy_install.py 29 May 2005 01:36:01 -0000 1.3 @@ -13,7 +13,9 @@ ------------------------------------ For basic use of ``easy_install``, you need only supply the filename or URL of -a source distribution or .egg file (Python Egg). +a source distribution or .egg file (`Python Egg`__). + +__ http://peak.telecommunity.com/DevCenter/PythonEggs **Example 1**. Download a source distribution, automatically building and installing it:: @@ -160,8 +162,6 @@ - - class Installer: """Manage a download/build/install process""" @@ -279,7 +279,7 @@ ) setup_script = setups[0] - self._run_setup(setups[0]) + self._run_setup(setup_script) for egg in glob( os.path.join(os.path.dirname(setup_script),'dist','*.egg') ): From pje at users.sourceforge.net Sun May 29 07:20:01 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 28 May 2005 22:20:01 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.8, 1.9 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23525/setuptools/command Modified Files: bdist_egg.py Log Message: Handle distributions with ' ' in their names Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- bdist_egg.py 28 May 2005 23:05:59 -0000 1.8 +++ bdist_egg.py 29 May 2005 05:19:59 -0000 1.9 @@ -53,7 +53,7 @@ def finalize_options (self): - self.egg_name = self.distribution.get_name().replace('-','_') + self.egg_name = self.distribution.get_name().replace('-','_').replace(' ','_') self.egg_version = self.distribution.get_version().replace('-','_') try: list( From pje at users.sourceforge.net Sun May 29 07:21:02 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 28 May 2005 22:21:02 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.21, 1.22 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23732 Modified Files: pkg_resources.py Log Message: Correctly recognize .egg files that are already on sys.path (or whatever path AvailableDistributions is scanning) Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- pkg_resources.py 29 May 2005 00:17:22 -0000 1.21 +++ pkg_resources.py 29 May 2005 05:20:59 -0000 1.22 @@ -755,6 +755,15 @@ return finder(importer,path_item) +def find_in_zip(importer,path_item): + # for now, we only yield the .egg file itself, if applicable; + # i.e., we don't support "baskets" yet, just eggs + for item in find_on_path(importer,path_item): + yield item + +register_finder(zipimport.zipimporter,find_in_zip) + + def StringIO(*args, **kw): """Thunk to load the real StringIO on demand""" global StringIO @@ -770,13 +779,6 @@ - - - - - - - def find_nothing(importer,path_item): return () From pje at users.sourceforge.net Sun May 29 08:41:17 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 28 May 2005 23:41:17 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.3, 1.4 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29069 Modified Files: easy_install.py Log Message: Oddly enough, some setup.py files actually check __name__=='__main__' Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- easy_install.py 29 May 2005 01:36:01 -0000 1.3 +++ easy_install.py 29 May 2005 06:41:14 -0000 1.4 @@ -184,7 +184,7 @@ ) elif multi is None: multi = True - + elif not multi: # explicit false, raise an error raise RuntimeError( @@ -337,7 +337,9 @@ try: sys.argv[:] = [setup_script, '-q', 'bdist_egg'] sys.path.insert(0,os.getcwd()) - execfile(setup_script, {'__file__':setup_script}) + execfile(setup_script, + {'__file__':setup_script, '__name__':'__main__'} + ) except SystemExit, v: if v.args and v.args[0]: raise RuntimeError( @@ -365,8 +367,6 @@ - - def install_egg(self, egg_path, zip_ok): import shutil destination = os.path.join(self.instdir, os.path.basename(egg_path)) @@ -383,10 +383,10 @@ shutil.move(egg_path, destination) else: shutil.copy2(egg_path, destination) - + elif os.path.isdir(egg_path): shutil.move(egg_path, destination) - + else: os.mkdir(destination) self._extract_zip(egg_path, destination) @@ -502,7 +502,7 @@ parser.add_option("-z", "--zip", action="store_true", dest="zip_ok", default=False, help="install package as a zipfile") - + parser.add_option("-m", "--multi-version", action="store_true", dest="multi", default=None, help="make apps have to require() a version") @@ -512,7 +512,7 @@ try: if not args: raise RuntimeError("No urls, filenames, or requirements specified") - + for spec in args: inst = Installer(options.instdir, options.zip_ok, options.multi) try: From pje at users.sourceforge.net Sun May 29 09:13:45 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 29 May 2005 00:13:45 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.4, 1.5 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13044 Modified Files: easy_install.py Log Message: Add support for using Installer subclasses, especially ones with support for searching a package database. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- easy_install.py 29 May 2005 06:41:14 -0000 1.4 +++ easy_install.py 29 May 2005 07:13:43 -0000 1.5 @@ -235,14 +235,14 @@ "Not a URL, existing file, or requirement spec: %r" % (spec,) ) + # process a Requirement dist = AvailableDistributions().best_match(spec,[]) if dist is not None and dist.path.endswith('.egg'): return dist.path - # TODO: search here for a distro to download + return self.download(self._find_package(spec)) - raise DistributionNotFound(spec) def install_eggs(self, dist_filename): # .egg dirs or files are already built, so just return them @@ -351,10 +351,10 @@ sys.argv[:] = save_argv - - - - + def _find_package(self, req): + # TODO: search here for a distro to download, matching Requirement + # 'req' and return the package URL or filename + raise DistributionNotFound(spec) @@ -492,7 +492,7 @@ URL_SCHEME = re.compile('([-+.a-z0-9]{2,}):',re.I).match -def main(argv): +def main(argv, factory=Installer): from optparse import OptionParser parser = OptionParser(usage = "usage: %prog [options] url [url...]") @@ -514,7 +514,7 @@ raise RuntimeError("No urls, filenames, or requirements specified") for spec in args: - inst = Installer(options.instdir, options.zip_ok, options.multi) + inst = factory(options.instdir, options.zip_ok, options.multi) try: print "Downloading", spec downloaded = inst.download(spec) From pje at users.sourceforge.net Mon May 30 00:05:42 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 29 May 2005 15:05:42 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.9, 1.10 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4227/setuptools/command Modified Files: bdist_egg.py Log Message: Added options to alter eggs' version number by tagging with the subversion revision number, date, and/or a custom tag. This should make it easier for people to produce e.g. automated nightly builds of eggs. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- bdist_egg.py 29 May 2005 05:19:59 -0000 1.9 +++ bdist_egg.py 29 May 2005 22:05:39 -0000 1.10 @@ -13,31 +13,31 @@ from pkg_resources import parse_requirements, get_platform class bdist_egg(Command): - description = "create an \"egg\" distribution" - - user_options = [('egg-base=', 'e', - "directory containing .egg-info directories" - "(default: top of the source tree)"), - ('bdist-dir=', 'd', - "temporary directory for creating the distribution"), - ('plat-name=', 'p', + user_options = [ + ('egg-base=', 'e', "directory containing .egg-info directories" + " (default: top of the source tree)"), + ('bdist-dir=', 'd', + "temporary directory for creating the distribution"), + ('tag-svn-revision', None, + "Add subversion revision ID to version number"), + ('tag-date', None, "Add date stamp (e.g. 20050528) to version number"), + ('tag-build=', None, "Specify explicit tag to add to version number"), + ('plat-name=', 'p', "platform name to embed in generated filenames " "(default: %s)" % get_platform()), - ('keep-temp', 'k', + ('keep-temp', 'k', "keep the pseudo-installation tree around after " + "creating the distribution archive"), - ('dist-dir=', 'd', + ('dist-dir=', 'd', "directory to put final built distributions in"), - ('skip-build', None, + ('skip-build', None, "skip rebuilding everything (for testing/debugging)"), - ('relative', None, - "build the archive using relative paths" - "(default: false)"), - ] - - boolean_options = ['keep-temp', 'skip-build', 'relative'] + ] + boolean_options = [ + 'keep-temp', 'skip-build', 'relative','tag-date','tag-svn-revision' + ] def initialize_options (self): self.egg_name = None @@ -49,12 +49,13 @@ self.keep_temp = 0 self.dist_dir = None self.skip_build = 0 - self.relative = 0 - + self.tag_build = None + self.tag_svn_revision = 0 + self.tag_date = 0 def finalize_options (self): - self.egg_name = self.distribution.get_name().replace('-','_').replace(' ','_') - self.egg_version = self.distribution.get_version().replace('-','_') + self.egg_name = self.distribution.get_name().replace(' ','-') + self.egg_version = self.tagged_version() try: list( parse_requirements('%s==%s' % (self.egg_name,self.egg_version)) @@ -79,7 +80,6 @@ self.plat_name = get_platform() self.set_undefined_options('bdist',('dist_dir', 'dist_dir')) - def write_stub(self, resource, pyfile): f = open(pyfile,'w') f.write('\n'.join([ @@ -135,9 +135,8 @@ # And make an archive relative to the root of the # pseudo-installation tree. - archive_basename = "%s-%s-py%s" % (self.egg_name, self.egg_version, - get_python_version()) - + archive_basename = "%s-%s-py%s" % ( self.egg_name.replace('-','_'), + self.egg_version.replace('-','_'), get_python_version()) if ext_outputs: archive_basename += "-" + self.plat_name @@ -156,11 +155,12 @@ log.info("writing %s" % os.path.join(self.egg_info,'PKG-INFO')) if not self.dry_run: - self.distribution.metadata.write_pkg_info(self.egg_info) - - - - + metadata = self.distribution.metadata + metadata.version, oldver = self.egg_version, metadata.version + try: + metadata.write_pkg_info(self.egg_info) + finally: + metadata.version = oldver native_libs = os.path.join(self.egg_info,"native_libs.txt") if ext_outputs: @@ -181,8 +181,6 @@ if os.path.isfile(path): self.copy_file(path,os.path.join(egg_info,filename)) - - # Make the archive make_zipfile(pseudoinstall_root+'.egg', archive_root, verbose=self.verbose, @@ -191,16 +189,28 @@ if not self.keep_temp: remove_tree(self.bdist_dir, dry_run=self.dry_run) + def tagged_version(self): + version = self.distribution.get_version() + if self.tag_build: + version+='-'+self.tag_build + if self.tag_svn_revision and os.path.exists('.svn'): + version += '-%s' % self.get_svn_revision() + if self.tag_date: + import time + version += time.strftime("-%Y%m%d") + return version - - - - - - + def get_svn_revision(self): + stdin, stdout = os.popen4("svn info"); stdin.close() + result = stdout.read(); stdout.close() + import re + match = re.search(r'Last Changed Rev: (\d+)', result) + if not match: + raise RuntimeError("svn info error: %s" % result.strip()) + return match.group(1) def make_zipfile (zip_filename, base_dir, verbose=0, dry_run=0): @@ -211,7 +221,6 @@ raises DistutilsExecError. Returns the name of the output zip file. """ import zipfile - mkpath(os.path.dirname(zip_filename), dry_run=dry_run) # If zipfile module is not available, try spawning an external @@ -230,17 +239,8 @@ if not dry_run: z = zipfile.ZipFile(zip_filename, "w", compression=zipfile.ZIP_DEFLATED) - os.path.walk(base_dir, visit, z) z.close() return zip_filename -# make_zipfile () - - - - - - - From pje at users.sourceforge.net Mon May 30 00:08:49 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 29 May 2005 15:08:49 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.5, 1.6 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5768 Modified Files: easy_install.py Log Message: Add subversion support, loosely based on an implementation by Ian Bicking. EasyInstall now recognizes both 'svn' URLs (including 'svn+ssh' et al), and notices when it connects to an HTTP server that's actually a subversion repository. In either case it does a subversion checkout of the package. Also, fixed a bug on platforms with os.samefile(). Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- easy_install.py 29 May 2005 07:13:43 -0000 1.5 +++ easy_install.py 29 May 2005 22:08:46 -0000 1.6 @@ -153,23 +153,21 @@ need. """ -import sys, os.path, pkg_resources, re, zipimport +import sys, os.path, pkg_resources, re, zipimport, zipfile, tarfile, shutil +from distutils.sysconfig import get_python_lib +from shutil import rmtree # must have, because it can be called from __del__ from pkg_resources import * - - class Installer: """Manage a download/build/install process""" pth_file = None def __init__(self, instdir=None, zip_ok=False, multi=None, tmpdir=None): - from distutils.sysconfig import get_python_lib - site_packages = get_python_lib() if tmpdir is None: from tempfile import mkdtemp @@ -177,6 +175,7 @@ self.tmpdir = tmpdir + site_packages = get_python_lib() if instdir is None or self.samefile(site_packages,instdir): instdir = site_packages self.pth_file = PthDistributions( @@ -197,22 +196,23 @@ def close(self): if os.path.isdir(self.tmpdir): - from shutil import rmtree rmtree(self.tmpdir,True) def __del__(self): self.close() + + def samefile(self,p1,p2): - try: - os.path.samefile - except AttributeError: - return ( - os.path.normpath(os.path.normcase(p1)) == - os.path.normpath(os.path.normcase(p2)) - ) - else: + if hasattr(os.path,'samefile') and ( + os.path.exists(p1) and os.path.exists(p2) + ): return os.path.samefile(p1,p2) + return ( + os.path.normpath(os.path.normcase(p1)) == + os.path.normpath(os.path.normcase(p2)) + ) + def download(self, spec): """Locate and/or download or `spec`, returning a local filename""" @@ -222,7 +222,7 @@ scheme = URL_SCHEME(spec) if scheme: # It's a url, download it to self.tmpdir - return self._download_url(scheme, spec) + return self._download_url(scheme.group(1), spec) elif os.path.exists(spec): # Existing file or directory, just return it @@ -250,19 +250,8 @@ return self.install_egg(dist_filename,True) # Anything else, try to extract and build - import zipfile, tarfile - if zipfile.is_zipfile(dist_filename): - self._extract_zip(dist_filename, self.tmpdir) - else: - import tarfile - try: - tar = tarfile.open(dist_filename) - except tarfile.TarError: - raise RuntimeError( - "Not a valid tar or zip archive: %s" % dist_filename - ) - else: - self._extract_tar(tar) + if os.path.isfile(dist_filename): + self._extract_file(dist_filename) # Find the setup.py file from glob import glob @@ -285,6 +274,17 @@ ): self.install_egg(egg, self.zip_ok) + + + + + + + + + + + def _extract_zip(self,zipname,extract_dir): import zipfile z = zipfile.ZipFile(zipname) @@ -351,7 +351,7 @@ sys.argv[:] = save_argv - def _find_package(self, req): + def _find_package(self, req): # TODO: search here for a distro to download, matching Requirement # 'req' and return the package URL or filename raise DistributionNotFound(spec) @@ -412,16 +412,24 @@ # Determine download filename from urlparse import urlparse name = filter(None,urlparse(url)[2].split('/'))[-1] + while '..' in name: name = name.replace('..','.').replace('\\','_') + filename = os.path.join(self.tmpdir,name) + + if scheme=='svn' or scheme.startswith('svn+'): + return self._download_svn(url, filename) + # Download the file from urllib import FancyURLopener, URLopener - class opener(FancyURLopener): + + class _opener(FancyURLopener): http_error_default = URLopener.http_error_default + try: - filename,headers = opener().retrieve( - url,os.path.join(self.tmpdir,name) + filename,headers = _opener().retrieve( + url,filename ) except IOError,v: if v.args and v.args[0]=='http error': @@ -430,6 +438,10 @@ ) else: raise + + if headers['content-type'].lower().startswith('text/html'): + return self._download_html(url, headers, filename) + # and return its filename return filename @@ -437,9 +449,38 @@ + def _extract_file(self, dist_filename): + if zipfile.is_zipfile(dist_filename): + self._extract_zip(dist_filename, self.tmpdir) + else: + try: + tar = tarfile.open(dist_filename) + except tarfile.TarError: + raise RuntimeError( + "Not a valid tar or zip archive: %s" % dist_filename + ) + else: + self._extract_tar(tar) + def _download_html(self, url, headers, filename): + # Check if it is a subversion index page: + file = open(filename) + for line in file: + if line.strip(): + if re.search(r'Revision \d+:', line): + file.close() + os.unlink(filename) + return self._download_svn(url, filename) + else: + break # not an index page + file.close() + raise RuntimeError("Unexpected HTML page found at "+url) + + def _download_svn(self, url, filename): + os.system("svn checkout -q %s %s" % (url, filename)) + return filename @@ -511,7 +552,7 @@ try: if not args: - raise RuntimeError("No urls, filenames, or requirements specified") + parser.error("No urls, filenames, or requirements specified") for spec in args: inst = factory(options.instdir, options.zip_ok, options.multi) @@ -523,11 +564,11 @@ finally: inst.close() except RuntimeError, v: - parser.error(str(v)) + print >>sys.stderr,"error:",v + sys.exit(1) if __name__ == '__main__': main(sys.argv[1:]) - From pje at users.sourceforge.net Mon May 30 00:14:33 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 29 May 2005 15:14:33 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools setup.py, 1.5, 1.6 Message-ID: <E1DcW3E-0002e6-W7@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10063 Modified Files: setup.py Log Message: Bump version to 0.3a2 for release Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- setup.py 29 May 2005 00:17:22 -0000 1.5 +++ setup.py 29 May 2005 22:14:30 -0000 1.6 @@ -7,7 +7,7 @@ setup( name="setuptools", - version="0.3a1", + version="0.3a2", description="Distutils enhancements", author="Phillip J. Eby", From pje at users.sourceforge.net Mon May 30 05:37:52 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 29 May 2005 20:37:52 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.6, 1.7 Message-ID: <E1Dcb68-0005KK-LX@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20147 Modified Files: easy_install.py Log Message: Add SourceForge download support, graciously contributed by Ian Bicking. Also, move some more imports to the top. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- easy_install.py 29 May 2005 22:08:46 -0000 1.6 +++ easy_install.py 30 May 2005 03:37:50 -0000 1.7 @@ -154,6 +154,7 @@ """ import sys, os.path, pkg_resources, re, zipimport, zipfile, tarfile, shutil +import urlparse, urllib, tempfile from distutils.sysconfig import get_python_lib from shutil import rmtree # must have, because it can be called from __del__ from pkg_resources import * @@ -161,7 +162,6 @@ - class Installer: """Manage a download/build/install process""" @@ -170,8 +170,7 @@ def __init__(self, instdir=None, zip_ok=False, multi=None, tmpdir=None): if tmpdir is None: - from tempfile import mkdtemp - tmpdir = mkdtemp(prefix="easy_install-") + tmpdir = tempfile.mkdtemp(prefix="easy_install-") self.tmpdir = tmpdir @@ -203,6 +202,7 @@ + def samefile(self,p1,p2): if hasattr(os.path,'samefile') and ( os.path.exists(p1) and os.path.exists(p2) @@ -286,9 +286,7 @@ def _extract_zip(self,zipname,extract_dir): - import zipfile z = zipfile.ZipFile(zipname) - try: for info in z.infolist(): name = info.filename @@ -326,6 +324,8 @@ finally: tarobj.close() + + def _run_setup(self, setup_script): from setuptools.command import bdist_egg sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) @@ -368,7 +368,7 @@ def install_egg(self, egg_path, zip_ok): - import shutil + destination = os.path.join(self.instdir, os.path.basename(egg_path)) ensure_directory(destination) @@ -409,9 +409,9 @@ self.pth_file.save() def _download_url(self, scheme, url): + # Determine download filename - from urlparse import urlparse - name = filter(None,urlparse(url)[2].split('/'))[-1] + name = filter(None,urlparse.urlparse(url)[2].split('/'))[-1] while '..' in name: name = name.replace('..','.').replace('\\','_') @@ -422,10 +422,8 @@ return self._download_svn(url, filename) # Download the file - from urllib import FancyURLopener, URLopener - - class _opener(FancyURLopener): - http_error_default = URLopener.http_error_default + class _opener(urllib.FancyURLopener): + http_error_default = urllib.URLopener.http_error_default try: filename,headers = _opener().retrieve( @@ -449,6 +447,8 @@ + + def _extract_file(self, dist_filename): if zipfile.is_zipfile(dist_filename): self._extract_zip(dist_filename, self.tmpdir) @@ -464,16 +464,28 @@ def _download_html(self, url, headers, filename): - # Check if it is a subversion index page: + # Check for a sourceforge URL + sf_url = url.startswith('http://prdownloads.') file = open(filename) for line in file: if line.strip(): + # Check for a subversion index page if re.search(r'<title>Revision \d+:', line): + # it's a subversion index page: file.close() os.unlink(filename) return self._download_svn(url, filename) - else: - break # not an index page + # Check for a SourceForge header + elif sf_url: + if re.search(r'^<HTML><HEAD>', line, re.I): + continue # skip first line + elif re.search(r'<TITLE>Select a Mirror for File:',line): + # Sourceforge mirror page + page = file.read() + file.close() + os.unlink(filename) + return self._download_sourceforge(url, page) + break # not an index page file.close() raise RuntimeError("Unexpected HTML page found at "+url) @@ -482,6 +494,35 @@ os.system("svn checkout -q %s %s" % (url, filename)) return filename + def _download_sourceforge(self, source_url, sf_page): + """Download package from randomly-selected SourceForge mirror""" + + mirror_regex = re.compile(r'HREF=(/.*?\?use_mirror=[^>]*)') + urls = [m.group(1) for m in mirror_regex.finditer(sf_page)] + if not urls: + raise RuntimeError( + "URL looks like a Sourceforge mirror page, but no URLs found" + ) + + import random + url = urlparse.urljoin(source_url, random.choice(urls)) + f = urllib.urlopen(url) + match = re.search( + r'<META HTTP-EQUIV="refresh" content=".*?URL=(.*?)"', + f.read() + ) + f.close() + + if match: + download_url = match.group(1) + scheme = URL_SCHEME(download_url) + return self._download_url(scheme.group(1), download_url) + else: + raise RuntimeError( + 'No META HTTP-EQUIV="refresh" found in Sourceforge page at %s' + % url + ) + From pje at users.sourceforge.net Mon May 30 08:46:04 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 29 May 2005 23:46:04 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.10, 1.11 Message-ID: <E1Dce2G-0003AG-07@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9321/setuptools/command Modified Files: bdist_egg.py Log Message: Reorganize bdist_egg's handling of 'install_data' to better deal with the various kludges legacy packages are using to install data in their package directories. Some use absolute paths in 'distribution.data_files', while others create various subclasses of 'install_data', each with their own way of finding out what directory to use! So 'bdist_egg' now does all its 'install_lib' activity before 'install_data', and pokes the desired build directory into a wide variety of places, so that all of the known kludges so far will work correctly. It also checks for absolute paths in 'data_files' (carefully working around other packages' 'data_files' kludges!) and converts them back to relative ones, if they are subpaths of site-packages. Clearly, we need to get the word out about 'package_files' in Python 2.4 and above, and suggest using 'setuptools' for Python 2.3. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- bdist_egg.py 29 May 2005 22:05:39 -0000 1.10 +++ bdist_egg.py 30 May 2005 06:46:01 -0000 1.11 @@ -7,7 +7,7 @@ from distutils.core import Command from distutils.util import get_platform from distutils.dir_util import create_tree, remove_tree, ensure_relative,mkpath -from distutils.sysconfig import get_python_version +from distutils.sysconfig import get_python_version, get_python_lib from distutils.errors import * from distutils import log from pkg_resources import parse_requirements, get_platform @@ -95,19 +95,39 @@ ])) f.close() + def do_install_data(self): + self.get_finalized_command('install').install_lib = self.bdist_dir + site_packages = os.path.normcase(os.path.realpath(get_python_lib())) + old, self.distribution.data_files = self.distribution.data_files,[] + for item in old: + if isinstance(item,tuple) and len(item)==2: + if os.path.isabs(item[0]): + realpath = os.path.realpath(item[0]) + normalized = os.path.normcase(realpath) + if normalized==site_packages or normalized.startswith( + site_packages+os.sep + ): + item = realpath[len(site_packages)+1:], item[1] + # XXX else: raise ??? + self.distribution.data_files.append(item) + try: + install = self.reinitialize_command('install_data') + # kludge for setups that use a 3-tuple inst_data + install.install_dir = install.install_base = \ + install.install_data = install.install_lib = self.bdist_dir + install.force = 0; install.root = None + log.info("installing package data to %s" % self.bdist_dir) + self.run_command('install_data') + finally: + self.distribution.data_files = old def run(self): + if not self.skip_build: self.run_command('build') - if self.distribution.data_files: - install = self.reinitialize_command('install_data') - install.install_dir = self.bdist_dir - install.force = 0 - install.root = None - log.info("installing package data to %s" % self.bdist_dir) - self.run_command('install_data') - + # We run install_lib before install_data, because some data hacks + # pull their data path from the install_lib command. install = self.reinitialize_command('install_lib', reinit_subcommands=1) install.install_dir = self.bdist_dir install.skip_build = self.skip_build @@ -120,7 +140,6 @@ log.info("installing library code to %s" % self.bdist_dir) self.run_command('install_lib') - to_compile = [] for ext_name in ext_outputs: filename,ext = os.path.splitext(ext_name) @@ -133,6 +152,9 @@ if to_compile: install.byte_compile(to_compile) + if self.distribution.data_files: + self.do_install_data() + # And make an archive relative to the root of the # pseudo-installation tree. archive_basename = "%s-%s-py%s" % ( self.egg_name.replace('-','_'), @@ -213,6 +235,15 @@ return match.group(1) + + + + + + + + + def make_zipfile (zip_filename, base_dir, verbose=0, dry_run=0): """Create a zip file from all the files under 'base_dir'. The output zip file will be named 'base_dir' + ".zip". Uses either the "zipfile" @@ -244,3 +275,13 @@ return zip_filename + + + + + + + + + + From pje at users.sourceforge.net Tue May 31 01:20:37 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 30 May 2005 16:20:37 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.7, 1.8 Message-ID: <E1DctYj-0002NU-EO@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8435 Modified Files: easy_install.py Log Message: Add setup script "sandboxing" -- abort a setup script if it tries to write to the filesystem outside of the installer's temporary directory. This is accomplished by temporarily replacing 'os.*' functions and the 'open' builtin with path-validation wrappers. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- easy_install.py 30 May 2005 03:37:50 -0000 1.7 +++ easy_install.py 30 May 2005 23:20:34 -0000 1.8 @@ -154,12 +154,12 @@ """ import sys, os.path, pkg_resources, re, zipimport, zipfile, tarfile, shutil -import urlparse, urllib, tempfile +import urlparse, urllib, tempfile, __builtin__ from distutils.sysconfig import get_python_lib from shutil import rmtree # must have, because it can be called from __del__ from pkg_resources import * - - +_os = sys.modules[os.name] +_open = open class Installer: @@ -337,8 +337,11 @@ try: sys.argv[:] = [setup_script, '-q', 'bdist_egg'] sys.path.insert(0,os.getcwd()) - execfile(setup_script, - {'__file__':setup_script, '__name__':'__main__'} + DirectorySandbox(self.tmpdir).run( + lambda: execfile( + setup_script, + {'__file__':setup_script, '__name__':'__main__'} + ) ) except SystemExit, v: if v.args and v.args[0]: @@ -364,9 +367,6 @@ - - - def install_egg(self, egg_path, zip_ok): destination = os.path.join(self.instdir, os.path.basename(egg_path)) @@ -531,6 +531,170 @@ +class AbstractSandbox: + """Wrap 'os' module and 'open()' builtin for virtualizing setup scripts""" + + _active = False + + def __init__(self): + self._attrs = [ + name for name in dir(_os) + if not name.startswith('_') and hasattr(self,name) + ] + + def _copy(self, source): + for name in self._attrs: + setattr(os, name, getattr(source,name)) + + def run(self, func): + """Run 'func' under os sandboxing""" + try: + self._copy(self) + __builtin__.open = __builtin__.file = self._open + self._active = True + return func() + finally: + self._active = False + __builtin__.open = __builtin__.file = _open + self._copy(_os) + + + def _mk_dual_path_wrapper(name): + original = getattr(_os,name) + def wrap(self,src,dst,*args,**kw): + if self._active: + src,dst = self._remap_pair(name,src,dst,*args,**kw) + return original(src,dst,*args,**kw) + return wrap + + + for name in ["rename", "link", "symlink"]: + if hasattr(_os,name): locals()[name] = _mk_dual_path_wrapper(name) + + + def _mk_single_path_wrapper(name, original=None): + original = original or getattr(_os,name) + def wrap(self,path,*args,**kw): + if self._active: + path = self._remap_input(name,path,*args,**kw) + return original(path,*args,**kw) + return wrap + + _open = _mk_single_path_wrapper('file', _open) + for name in [ + "stat", "listdir", "chdir", "open", "chmod", "chown", "mkdir", + "remove", "unlink", "rmdir", "utime", "lchown", "chroot", "lstat", + "startfile", "mkfifo", "mknod", "pathconf", "access" + ]: + if hasattr(_os,name): locals()[name] = _mk_single_path_wrapper(name) + + + def _mk_single_with_return(name): + original = getattr(_os,name) + def wrap(self,path,*args,**kw): + if self._active: + path = self._remap_input(name,path,*args,**kw) + return self._remap_output(name, original(path,*args,**kw)) + return original(path,*args,**kw) + return wrap + + for name in ['readlink', 'tempnam']: + if hasattr(_os,name): locals()[name] = _mk_single_with_return(name) + + def _mk_query(name): + original = getattr(_os,name) + def wrap(self,*args,**kw): + retval = original(*args,**kw) + if self._active: + return self._remap_output(name, retval) + return retval + return wrap + + for name in ['getcwd', 'tmpnam']: + if hasattr(_os,name): locals()[name] = _mk_query(name) + + def _validate_path(self,path): + """Called to remap or validate any path, whether input or output""" + return path + + def _remap_input(self,operation,path,*args,**kw): + """Called for path inputs""" + return self._validate_path(path) + + def _remap_output(self,operation,path): + """Called for path outputs""" + return self._validate_path(path) + + def _remap_pair(self,operation,src,dst,*args,**kw): + """Called for path pairs like rename, link, and symlink operations""" + return ( + self._remap_input(operation+'-from',src,*args,**kw), + self._remap_input(operation+'-to',dst,*args,**kw) + ) + + +class DirectorySandbox(AbstractSandbox): + """Restrict operations to a single subdirectory - pseudo-chroot""" + + write_ops = dict.fromkeys([ + "open", "chmod", "chown", "mkdir", "remove", "unlink", "rmdir", + "utime", "lchown", "chroot", "mkfifo", "mknod", "tempnam", + ]) + + def __init__(self,sandbox): + self._sandbox = os.path.realpath(sandbox) + self._prefix = os.path.join(self._sandbox,'') + AbstractSandbox.__init__(self) + + def _violation(self, operation, *args, **kw): + raise SandboxViolation(operation, args, kw) + + def _open(self, path, mode='r', *args, **kw): + if mode not in ('r', 'rt', 'rb', 'rU') and not self._ok(path): + self._violation("open", path, mode, *args, **kw) + return _open(path,mode,*args,**kw) + + def tmpnam(self): + self._violation("tmpnam") + + def _ok(self,path): + active = self._active + try: + self._active = False + realpath = os.path.realpath(path) + if realpath==self._sandbox or realpath.startswith(self._prefix): + return True + finally: + self._active = active + + def _remap_input(self,operation,path,*args,**kw): + """Called for path inputs""" + if operation in self.write_ops and not self._ok(path): + self._violation(operation, path, *args, **kw) + return path + + def _remap_pair(self,operation,src,dst,*args,**kw): + """Called for path pairs like rename, link, and symlink operations""" + if not self._ok(src) or not self._ok(dst): + self._violation(operation, src, dst, *args, **kw) + return (src,dst) + + +class SandboxViolation(RuntimeError): + """A setup script attempted to modify the filesystem outside the sandbox""" + + def __str__(self): + return """SandboxViolation: %s%r %s + +The package setup script has attempted to modify files on your system +that are not within the EasyInstall build area, and has been aborted. + +This package cannot be safely installed by EasyInstall, and may not +support alternate installation locations even if you run its setup +script by hand. Please inform the package's author and the EasyInstall +maintainers to find out if a fix or workaround is available.""" % self.args + + class PthDistributions(AvailableDistributions): """A .pth file with Distribution paths in it""" From pje at users.sourceforge.net Tue May 31 01:22:12 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 30 May 2005 16:22:12 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.11, 1.12 Message-ID: <E1DctaG-0002cK-Nm@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9371/setuptools/command Modified Files: bdist_egg.py Log Message: Ensure that the distribution name written to PKG-INFO is the same as the name you'll use in 'require()' operations for that distribution. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- bdist_egg.py 30 May 2005 06:46:01 -0000 1.11 +++ bdist_egg.py 30 May 2005 23:22:10 -0000 1.12 @@ -174,15 +174,15 @@ egg_info = os.path.join(archive_root,'EGG-INFO') self.mkpath(egg_info) self.mkpath(self.egg_info) - log.info("writing %s" % os.path.join(self.egg_info,'PKG-INFO')) if not self.dry_run: metadata = self.distribution.metadata metadata.version, oldver = self.egg_version, metadata.version + metadata.name, oldname = self.egg_name, metadata.name try: metadata.write_pkg_info(self.egg_info) finally: - metadata.version = oldver + metadata.name, metadata.version = oldname, oldver native_libs = os.path.join(self.egg_info,"native_libs.txt") if ext_outputs: From pje at users.sourceforge.net Tue May 31 01:52:36 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 30 May 2005 16:52:36 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.8, 1.9 Message-ID: <E1Dcu3g-000707-3a@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv26408 Modified Files: easy_install.py Log Message: Add option to allow specifying a download/extract/build directory, which will be kept after installation completes. Added an "installation report" that tells you how to use 'require()' to activate a particular package version. Installer.install_eggs() now returns a list of Distribution objects for the eggs it found and installed, so that the command-line tool can print an installation report for each one. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- easy_install.py 30 May 2005 23:20:34 -0000 1.8 +++ easy_install.py 30 May 2005 23:52:33 -0000 1.9 @@ -166,13 +166,15 @@ """Manage a download/build/install process""" pth_file = None + cleanup = False def __init__(self, instdir=None, zip_ok=False, multi=None, tmpdir=None): - if tmpdir is None: tmpdir = tempfile.mkdtemp(prefix="easy_install-") - - self.tmpdir = tmpdir + self.cleanup = True + elif not os.path.isdir(tmpdir): + os.makedirs(tmpdir) + self.tmpdir = os.path.realpath(tmpdir) site_packages = get_python_lib() if instdir is None or self.samefile(site_packages,instdir): @@ -194,15 +196,13 @@ self.multi = multi def close(self): - if os.path.isdir(self.tmpdir): + if self.cleanup and os.path.isdir(self.tmpdir): rmtree(self.tmpdir,True) def __del__(self): self.close() - - def samefile(self,p1,p2): if hasattr(os.path,'samefile') and ( os.path.exists(p1) and os.path.exists(p2) @@ -269,14 +269,14 @@ setup_script = setups[0] self._run_setup(setup_script) + + eggs = [] for egg in glob( os.path.join(os.path.dirname(setup_script),'dist','*.egg') ): - self.install_egg(egg, self.zip_ok) - - - + eggs.append(self.install_egg(egg, self.zip_ok)) + return eggs @@ -339,7 +339,7 @@ sys.path.insert(0,os.getcwd()) DirectorySandbox(self.tmpdir).run( lambda: execfile( - setup_script, + "setup.py", {'__file__':setup_script, '__name__':'__main__'} ) ) @@ -391,22 +391,22 @@ os.mkdir(destination) self._extract_zip(egg_path, destination) - if self.pth_file is not None: - if os.path.isdir(destination): - dist = Distribution.from_filename( - destination, metadata=PathMetadata( - destination, os.path.join(destination,'EGG-INFO') - ) + if os.path.isdir(destination): + dist = Distribution.from_filename( + destination, metadata=PathMetadata( + destination, os.path.join(destination,'EGG-INFO') ) - else: - metadata = EggMetadata(zipimport.zipimporter(destination)) - dist = Distribution.from_filename(destination,metadata=metadata) + ) + else: + metadata = EggMetadata(zipimport.zipimporter(destination)) + dist = Distribution.from_filename(destination,metadata=metadata) - # remove old - map(self.pth_file.remove, self.pth_file.get(dist.key,())) + if self.pth_file is not None: + map(self.pth_file.remove, self.pth_file.get(dist.key,())) # drop old if not self.multi: self.pth_file.add(dist) # add new self.pth_file.save() + return dist def _download_url(self, scheme, url): @@ -531,6 +531,47 @@ + def installation_report(self, dist): + """Helpful installation message for display to package users""" + + msg = "Installed %(eggloc)s to %(instdir)s" + if self.multi: + msg += """ + +Because this distribution was installed --multi-version or --install-dir, +before you can import modules from this package in an application, you +will need to 'import pkg_resources' and then use a 'require()' call +similar to one of these examples, in order to select the desired version: + + pkg_resources.require("%(name)s") # latest installed version + pkg_resources.require("%(name)s==%(version)s") # this exact version + pkg_resources.require("%(name)s>=%(version)s") # this version or higher +""" + if not self.samefile(get_python_lib(),self.instdir): + msg += """ + +Note also that the installation directory must be on sys.path at runtime for +this to work. (e.g. by being the application's script directory, by being on +PYTHONPATH, or by being added to sys.path by your code.) +""" + eggloc = os.path.basename(dist.path) + instdir = os.path.realpath(self.instdir) + name = dist.name + version = dist.version + return msg % locals() + + + + + + + + + + + + + class AbstractSandbox: """Wrap 'os' module and 'open()' builtin for virtualizing setup scripts""" @@ -670,7 +711,7 @@ def _remap_input(self,operation,path,*args,**kw): """Called for path inputs""" if operation in self.write_ops and not self._ok(path): - self._violation(operation, path, *args, **kw) + self._violation(operation, os.path.realpath(path), *args, **kw) return path def _remap_pair(self,operation,src,dst,*args,**kw): @@ -740,7 +781,6 @@ def main(argv, factory=Installer): from optparse import OptionParser - parser = OptionParser(usage = "usage: %prog [options] url [url...]") parser.add_option("-d", "--install-dir", dest="instdir", default=None, help="install package to DIR", metavar="DIR") @@ -753,27 +793,69 @@ action="store_true", dest="multi", default=None, help="make apps have to require() a version") + parser.add_option("-b", "--build-directory", dest="tmpdir", metavar="DIR", + default=None, + help="download/extract/build in DIR; keep the results") (options, args) = parser.parse_args() try: if not args: parser.error("No urls, filenames, or requirements specified") - for spec in args: - inst = factory(options.instdir, options.zip_ok, options.multi) + inst = factory( + options.instdir, options.zip_ok, options.multi, options.tmpdir + ) try: print "Downloading", spec downloaded = inst.download(spec) print "Installing", os.path.basename(downloaded) - inst.install_eggs(downloaded) + for dist in inst.install_eggs(downloaded): + print inst.installation_report(dist) finally: inst.close() except RuntimeError, v: print >>sys.stderr,"error:",v sys.exit(1) + if __name__ == '__main__': main(sys.argv[1:]) + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + From pje at users.sourceforge.net Tue May 31 02:24:47 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 30 May 2005 17:24:47 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, NONE, 1.1 easy_install.py, 1.9, 1.10 TODO.txt, 1.3, NONE Message-ID: <E1DcuYp-000355-Q8@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11166 Modified Files: easy_install.py Added Files: EasyInstall.txt Removed Files: TODO.txt Log Message: Ditch outdated TODO file, move docs to a new 'EasyInstall.txt' file. Fix installation report for .egg files/directories. --- NEW FILE: EasyInstall.txt --- ============ Easy Install ============ Easy Install is a python module (``easy_install``) that lets you automatically download, build, install, and manage Python packages. (Please share your experiences with us! Whether you encountered success or difficulty installing a particular package, please add your notes to the `Experience Reports <http://peak.telecommunity.com/DevCenter/PackageNotes>`_ page. You'll need to register for a Wiki ID if you don't already have one; you can do that from the `User Preferences <http://peak.telecommunity.com/DevCenter/UserPreferences>`_ page. Thanks!) .. contents:: **Table of Contents** Using "Easy Install" ==================== Installing "Easy Install" ------------------------- Unix-like Systems (including Mac OS X and Cygwin) Download either the `Python 2.3 easy_install shell script <http://peak.telecommunity.com/dist/ez_install-0.3a3-py2.3-unix.sh>`_ or the `Python 2.4 easy_install shell script <http://peak.telecommunity.com/dist/ez_install-0.3a3-py2.4-unix.sh>`_. Place the file somewhere on your PATH, after renaming it to ``easy_install``. Note that these scripts assume you have ``python2.3`` or ``python2.4`` accessible via the ``PATH`` environment variable. Then, you can use ``easy_install`` to finish its own installation, by running one of the following, depending on your Python version:: # Python 2.3 easy_install http://peak.telecommunity.com/dist/setuptools-0.3a3-py2.3.egg # Python 2.4 easy_install http://peak.telecommunity.com/dist/setuptools-0.3a3-py2.4.egg All Other Systems Download the `easy_install (aka setuptools) source distribution <http://peak.telecommunity.com/dist/setuptools-0.3a3.zip>`_, and follow the normal procedure for installing a source package with distutils. An ``easy_install.py`` script will be installed in the normal location for Python scripts on your platform. In the examples below, you'll need to replace references to ``easy_install`` with the correct invocation to run ``easy_install.py`` on your system. If you have Python 2.4 or better, you can also use ``python -m easy_install``, which will have the same effect, but which may be easier for you to type. Downloading and Installing a Package ------------------------------------ For basic use of ``easy_install``, you need only supply the filename or URL of a source distribution or .egg file (`Python Egg`__). __ http://peak.telecommunity.com/DevCenter/PythonEggs **Example 1**. Download a source distribution, automatically building and installing it:: easy_install http://example.com/path/to/MyPackage-1.2.3.tgz **Example 2**. Install an already-downloaded .egg file:: easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg Easy Install recognizes distutils *source* (not binary) distribution files with extensions of .tgz, .tar, .tar.gz, .tar.bz2, or .zip. And of course it handles already-built .egg distributions. By default, packages are installed to the running Python installation's ``site-packages`` directory, unless you provide the ``-d`` or ``--install-dir`` option to specify an alternative directory. Packages installed to ``site-packages`` are added to an ``easy-install.pth`` file, so that Python will be able to import the package by default. If you do not want this to happen, you should use the ``-m`` or ``--multi`` option, which allows multiple versions of the same package to be selected at runtime. Note that installing to a directory other than ``site-packages`` already implies the ``-m`` option, so if you cannot install to ``site-packages``, please see the `Command-Line Options`_ section below (under ``--multi``) to find out how to select packages at runtime. Upgrading a Package ------------------- You don't need to do anything special to upgrade a package: just install the new version. If you're using ``-m`` or ``--multi`` (or installing outside of ``site-packages``), the runtime system automatically selects the newest available version of a package. If you're installing to ``site-packages`` and not using ``-m``, installing a package automatically replaces its older version in the ``easy-install.pth`` file, so that Python will import the latest version by default. ``easy_install`` never actually deletes packages (unless you're installing a package with the same name and version number as an existing package), so if you want to get rid of older versions of a package, please see `Uninstalling Packages`_, below. Changing the Active Version (``site-packages`` installs only) ------------------------------------------------------------- If you've upgraded a package, but need to revert to a previously-installed version, you can do so like this:: easy_install PackageName==1.2.3 Where ``1.2.3`` is replaced by the exact version number you wish to switch to. Note that the named package and version must already have been installed to ``site-packages``. If you'd like to switch to the latest version of ``PackageName``, you can do so like this:: easy_install PackageName This will activate the latest installed version. Uninstalling Packages --------------------- If you have replaced a package with another version, then you can just delete the package(s) you don't need by deleting the PackageName-versioninfo.egg file or directory (found in the installation directory). If you want to delete the currently installed version of a package (or all versions of a package), you should first run:: easy_install -m PackageName This will ensure that Python doesn't continue to search for a package you're planning to remove. After you've done this, you can safely delete the .egg files or directories. Reference Manual ================ Command-Line Options -------------------- ``--zip, -z`` Enable installing the package as a zip file. This can significantly increase Python's overall import performance if you're installing to ``site-packages`` and not using the ``--multi`` option, because Python process zipfile entries on ``sys.path`` much faster than it does directories. So, if you don't use this option, and you install a lot of packages, some of them may be slower to import. But, this option is disabled by default, unless you're installing from an already-built binary zipfile (``.egg`` file). This is to avoid problems when using packages that dosn't support running from a zip file. Such packages usually access data files in their package directories using the Python ``__file__`` or ``__path__`` attribute, instead of the ``pkg_resources`` API. So, if you find that a package doesn't work properly when used with this option, you may want to suggest to the author that they switch to using the ``pkg_resources`` resource API, which will allow their package to work whether it's installed as a zipfile or not. (Note: this option only affects the installation of newly-built packages that are not already installed in the target directory; if you want to convert an existing installed version from zipped to unzipped or vice versa, you'll need to delete the existing version first.) ``--multi-version, -m`` "Multi-version" mode. Specifying this option prevents ``easy_install`` from adding an ``easy-install.pth`` entry for the package being installed, and if an entry for any version the package already exists, it will be removed upon successful installation. In multi-version mode, no specific version of the package is available for importing, unless you use ``pkg_resources.require()`` to put it on ``sys.path``. This can be as simple as:: from pkg_resources import require require("SomePackage", "OtherPackage", "MyPackage") which will put the latest installed version of the specified packages on ``sys.path`` for you. (For more advanced uses, like selecting specific versions and enabling optional dependencies, see the ``pkg_resources`` API doc.) Note that if you install to a directory other than ``site-packages``, this option is automatically in effect, because ``.pth`` files can only be used in ``site-packages`` (at least in Python 2.3 and 2.4). So, if you use the ``--install-dir`` or ``-i`` options, you must also use ``require()`` to enable packages at runtime ``--install-dir=DIR, -d DIR`` Set the installation directory. It is up to you to ensure that this directory is on ``sys.path`` at runtime, and to use ``pkg_resources.require()`` to enable the installed package(s) that you need. ``--build-directory=DIR, -b DIR`` (New in 0.3a3) Set the directory used to download, extract, and install the package. The directory is not cleared before or after installation, so the downloaded packages and extracted contents will remain there afterwards, allowing you to read any documentation, examples, scripts, etc. that may have been included with the source distribution (if any). Release Notes/Change History ============================ 0.3a3 * Added ``--build-directory=DIR/-b DIR`` option. * Added "installation report" that explains how to use 'require()' when doing a multiversion install or alternate installation directory. * Added SourceForge mirror auto-select (Contributed by Ian Bicking) * Added "sandboxing" that stops a setup script from running if it attempts to write to the filesystem outside of the build area * Added more workarounds for packages with quirky ``install_data`` hacks 0.3a2 * Added subversion download support for ``svn:`` and ``svn+`` URLs, as well as automatic recognition of HTTP subversion URLs (Contributed by Ian Bicking) * Added new options to ``bdist_egg`` to allow tagging the egg's version number with a subversion revision number, the current date, or an explicit tag value. Run ``setup.py bdist_egg --help`` to get more information. * Misc. bug fixes 0.3a1 Initial release. Future Plans ============ * Support packages that include scripts * Automatic package download URL discovery via PyPI/CheeseShop Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- easy_install.py 30 May 2005 23:52:33 -0000 1.9 +++ easy_install.py 31 May 2005 00:24:45 -0000 1.10 @@ -1,165 +1,42 @@ #!python """\ -Easy Install -============ - -Easy Install is a python module (easy_install) that lets you automatically -download, build, install, and manage Python packages. - -.. contents:: **Table of Contents** - - -Downloading and Installing a Package ------------------------------------- - -For basic use of ``easy_install``, you need only supply the filename or URL of -a source distribution or .egg file (`Python Egg`__). - -__ http://peak.telecommunity.com/DevCenter/PythonEggs - -**Example 1**. Download a source distribution, automatically building and -installing it:: - - easy_install http://example.com/path/to/MyPackage-1.2.3.tgz - -**Example 2**. Install an already-downloaded .egg file:: - - easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg - -Easy Install recognizes distutils *source* (not binary) distribution files with -extensions of .tgz, .tar, .tar.gz, .tar.bz2, or .zip. And of course it handles -already-built .egg distributions. - -By default, packages are installed to the running Python installation's -``site-packages`` directory, unless you provide the ``-d`` or ``--install-dir`` -option to specify an alternative directory. - -Packages installed to ``site-packages`` are added to an ``easy-install.pth`` -file, so that Python will be able to import the package by default. If you do -not want this to happen, you should use the ``-m`` or ``--multi`` option, which -allows multiple versions of the same package to be selected at runtime. - -Note that installing to a directory other than ``site-packages`` already -implies the ``-m`` option, so if you cannot install to ``site-packages``, -please see the `Options`_ section below (under ``--multi``) to find out how to -select packages at runtime. - -Upgrading a Package -------------------- - -You don't need to do anything special to upgrade a package: just install the -new version. If you're using ``-m`` or ``--multi`` (or installing outside of -``site-packages``), the runtime system automatically selects the newest -available version of a package. If you're installing to ``site-packages`` and -not using ``-m``, installing a package automatically replaces its older version -in the ``easy-install.pth`` file, so that Python will import the latest version -by default. - -``easy_install`` never actually deletes packages (unless you're installing a -package with the same name and version number as an existing package), so if -you want to get rid of older versions of a package, please see `Uninstalling -Packages`_, below. - - -Changing the Active Version (``site-packages`` installs only) -------------------------------------------------------------- - -If you've upgraded a package, but need to revert to a previously-installed -version, you can do so like this:: - - easy_install PackageName==1.2.3 - -Where ``1.2.3`` is replaced by the exact version number you wish to switch to. -Note that the named package and version must already have been installed to -``site-packages``. - -If you'd like to switch to the latest version of ``PackageName``, you can do so -like this:: - - easy_install PackageName - -This will activate the latest installed version. - - -Uninstalling Packages ---------------------- - -If you have replaced a package with another version, then you can just delete -the package(s) you don't need by deleting the PackageName-versioninfo.egg file -or directory (found in the installation directory). - -If you want to delete the currently installed version of a package (or all -versions of a package), you should first run:: +Easy Install +------------ - easy_install -m PackageName +A tool for doing automatic download/extract/build of distutils-based Python +packages. For detailed documentation, see the accompanying EasyInstall.txt +file, or visit the `EasyInstall home page`__. -This will ensure that Python doesn't continue to search for a package you're -planning to remove. After you've done this, you can safely delete the .egg -files or directories. +__ http://peak.telecommunity.com/DevCenter/EasyInstall +""" -Options -------- +import sys +import os.path +import pkg_resources +import re +import zipimport +import zipfile +import tarfile +import shutil +import urlparse +import urllib +import tempfile +import __builtin__ -``--zip, -z`` - Enable installing the package as a zip file. This can significantly - increase Python's overall import performance if you're installing to - ``site-packages`` and not using the ``--multi`` option, because Python - process zipfile entries on ``sys.path`` much faster than it does - directories. So, if you don't use this option, and you install a lot of - packages, some of them may be slower to import. +from distutils.sysconfig import get_python_lib +from shutil import rmtree # must have, because it can be called from __del__ +from pkg_resources import * - But, this option is disabled by default, unless you're installing from an - already-built binary zipfile (``.egg`` file). This is to avoid problems - when using packages that dosn't support running from a zip file. Such - packages usually access data files in their package directories using the - Python ``__file__`` or ``__path__`` attribute, instead of the - ``pkg_resources`` API. So, if you find that a package doesn't work properly - when used with this option, you may want to suggest to the author that they - switch to using the ``pkg_resources`` resource API, which will allow their - package to work whether it's installed as a zipfile or not. +_os = sys.modules[os.name] +_open = open - (Note: this option only affects the installation of newly-built packages - that are not already installed in the target directory; if you want to - convert an existing installed version from zipped to unzipped or vice - versa, you'll need to delete the existing version first.) -``--multi-version, -m`` - "Multi-version" mode. Specifying this option prevents ``easy_install`` from - adding an ``easy-install.pth`` entry for the package being installed, and - if an entry for any version the package already exists, it will be removed - upon successful installation. In multi-version mode, no specific version of - the package is available for importing, unless you use - ``pkg_resources.require()`` to put it on ``sys.path``. This can be as - simple as:: - from pkg_resources import require - require("SomePackage", "OtherPackage", "MyPackage") - which will put the latest installed version of the specified packages on - ``sys.path`` for you. (For more advanced uses, like selecting specific - versions and enabling optional dependencies, see the ``pkg_resources`` API - doc.) Note that if you install to a directory other than ``site-packages``, - this option is automatically in effect, because ``.pth`` files can only be - used in ``site-packages`` (at least in Python 2.3 and 2.4). So, if you use - the ``--install-dir`` or ``-i`` options, you must also use ``require()`` to - enable packages at runtime -``--install-dir=DIR, -d DIR`` - Set the installation directory. It is up to you to ensure that this - directory is on ``sys.path`` at runtime, and to use - ``pkg_resources.require()`` to enable the installed package(s) that you - need. -""" -import sys, os.path, pkg_resources, re, zipimport, zipfile, tarfile, shutil -import urlparse, urllib, tempfile, __builtin__ -from distutils.sysconfig import get_python_lib -from shutil import rmtree # must have, because it can be called from __del__ -from pkg_resources import * -_os = sys.modules[os.name] -_open = open class Installer: @@ -247,7 +124,7 @@ def install_eggs(self, dist_filename): # .egg dirs or files are already built, so just return them if dist_filename.lower().endswith('.egg'): - return self.install_egg(dist_filename,True) + return [self.install_egg(dist_filename,True)] # Anything else, try to extract and build if os.path.isfile(dist_filename): --- TODO.txt DELETED --- From pje at users.sourceforge.net Tue May 31 02:35:07 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 30 May 2005 17:35:07 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools setup.py, 1.6, 1.7 Message-ID: <E1Dcuip-0004ZM-Ov@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17479 Modified Files: setup.py Log Message: Release 0.3a3 Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- setup.py 29 May 2005 22:14:30 -0000 1.6 +++ setup.py 31 May 2005 00:34:54 -0000 1.7 @@ -7,9 +7,9 @@ setup( name="setuptools", - version="0.3a2", + version="0.3a3", - description="Distutils enhancements", + description="Distutils packaging and installation enhancements", author="Phillip J. Eby", author_email="peak at eby-sarna.com", license="PSF or ZPL", From rhettinger at users.sourceforge.net Tue May 31 12:26:30 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 31 May 2005 03:26:30 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libstdtypes.tex, 1.178, 1.179 Message-ID: <E1Dd3x8-0005Ih-Kr@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19835 Modified Files: libstdtypes.tex Log Message: SF bug #1202395: Description of string.lstrip() needs improvement Clarify the role of the chars argument in the strip() methods. Index: libstdtypes.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libstdtypes.tex,v retrieving revision 1.178 retrieving revision 1.179 diff -u -d -r1.178 -r1.179 --- libstdtypes.tex 27 May 2005 10:43:55 -0000 1.178 +++ libstdtypes.tex 31 May 2005 10:26:28 -0000 1.179 @@ -699,11 +699,17 @@ \end{methoddesc} \begin{methoddesc}[string]{lstrip}{\optional{chars}} -Return a copy of the string with leading characters removed. If -\var{chars} is omitted or \code{None}, whitespace characters are -removed. If given and not \code{None}, \var{chars} must be a string; -the characters in the string will be stripped from the beginning of -the string this method is called on. +Return a copy of the string with leading characters removed. The +\var{chars} argument is a string specifying the set of characters +to be removed. If omitted or \code{None}, the \var{chars} argument +defaults to removing whitespace. The \var{chars} argument is not +a prefix; rather, all combinations of its values are stripped: +\begin{verbatim} + >>> ' spacious '.lstrip() + 'spacious ' + >>> 'www.example.com'.lstrip('cmowz.') + 'example.com' +\end{verbatim} \versionchanged[Support for the \var{chars} argument]{2.2.2} \end{methoddesc} @@ -745,11 +751,17 @@ \end{methoddesc} \begin{methoddesc}[string]{rstrip}{\optional{chars}} -Return a copy of the string with trailing characters removed. If -\var{chars} is omitted or \code{None}, whitespace characters are -removed. If given and not \code{None}, \var{chars} must be a string; -the characters in the string will be stripped from the end of the -string this method is called on. +Return a copy of the string with trailing characters removed. The +\var{chars} argument is a string specifying the set of characters +to be removed. If omitted or \code{None}, the \var{chars} argument +defaults to removing whitespace. The \var{chars} argument is not +a suffix; rather, all combinations of its values are stripped: +\begin{verbatim} + >>> ' spacious '.rstrip() + ' spacious' + >>> 'mississippi'.rstrip('ipz') + 'mississ' +\end{verbatim} \versionchanged[Support for the \var{chars} argument]{2.2.2} \end{methoddesc} @@ -791,11 +803,17 @@ \end{methoddesc} \begin{methoddesc}[string]{strip}{\optional{chars}} -Return a copy of the string with leading and trailing characters -removed. If \var{chars} is omitted or \code{None}, whitespace -characters are removed. If given and not \code{None}, \var{chars} -must be a string; the characters in the string will be stripped from -the both ends of the string this method is called on. +Return a copy of the string with the leading and trailing characters +removed. The \var{chars} argument is a string specifying the set of +characters to be removed. If omitted or \code{None}, the \var{chars} +argument defaults to removing whitespace. The \var{chars} argument is not +a prefix or suffix; rather, all combinations of its values are stripped: +\begin{verbatim} + >>> ' spacious '.strip() + 'spacious' + >>> 'www.example.com'.strip('cmowz.') + 'example' +\end{verbatim} \versionchanged[Support for the \var{chars} argument]{2.2.2} \end{methoddesc} From rhettinger at users.sourceforge.net Tue May 31 12:28:09 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 31 May 2005 03:28:09 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libstdtypes.tex, 1.170.2.8, 1.170.2.9 Message-ID: <E1Dd3yj-0005Wg-Mf@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21101 Modified Files: Tag: release24-maint libstdtypes.tex Log Message: SF bug #1202395: Description of string.lstrip() needs improvement Clarify the role of the chars argument in the strip() methods. Index: libstdtypes.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libstdtypes.tex,v retrieving revision 1.170.2.8 retrieving revision 1.170.2.9 diff -u -d -r1.170.2.8 -r1.170.2.9 --- libstdtypes.tex 27 May 2005 10:45:14 -0000 1.170.2.8 +++ libstdtypes.tex 31 May 2005 10:28:07 -0000 1.170.2.9 @@ -699,11 +699,17 @@ \end{methoddesc} \begin{methoddesc}[string]{lstrip}{\optional{chars}} -Return a copy of the string with leading characters removed. If -\var{chars} is omitted or \code{None}, whitespace characters are -removed. If given and not \code{None}, \var{chars} must be a string; -the characters in the string will be stripped from the beginning of -the string this method is called on. +Return a copy of the string with leading characters removed. The +\var{chars} argument is a string specifying the set of characters +to be removed. If omitted or \code{None}, the \var{chars} argument +defaults to removing whitespace. The \var{chars} argument is not +a prefix; rather, all combinations of its values are stripped: +\begin{verbatim} + >>> ' spacious '.lstrip() + 'spacious ' + >>> 'www.example.com'.lstrip('cmowz.') + 'example.com' +\end{verbatim} \versionchanged[Support for the \var{chars} argument]{2.2.2} \end{methoddesc} @@ -745,11 +751,17 @@ \end{methoddesc} \begin{methoddesc}[string]{rstrip}{\optional{chars}} -Return a copy of the string with trailing characters removed. If -\var{chars} is omitted or \code{None}, whitespace characters are -removed. If given and not \code{None}, \var{chars} must be a string; -the characters in the string will be stripped from the end of the -string this method is called on. +Return a copy of the string with trailing characters removed. The +\var{chars} argument is a string specifying the set of characters +to be removed. If omitted or \code{None}, the \var{chars} argument +defaults to removing whitespace. The \var{chars} argument is not +a suffix; rather, all combinations of its values are stripped: +\begin{verbatim} + >>> ' spacious '.rstrip() + ' spacious' + >>> 'mississippi'.rstrip('ipz') + 'mississ' +\end{verbatim} \versionchanged[Support for the \var{chars} argument]{2.2.2} \end{methoddesc} @@ -791,11 +803,17 @@ \end{methoddesc} \begin{methoddesc}[string]{strip}{\optional{chars}} -Return a copy of the string with leading and trailing characters -removed. If \var{chars} is omitted or \code{None}, whitespace -characters are removed. If given and not \code{None}, \var{chars} -must be a string; the characters in the string will be stripped from -the both ends of the string this method is called on. +Return a copy of the string with the leading and trailing characters +removed. The \var{chars} argument is a string specifying the set of +characters to be removed. If omitted or \code{None}, the \var{chars} +argument defaults to removing whitespace. The \var{chars} argument is not +a prefix or suffix; rather, all combinations of its values are stripped: +\begin{verbatim} + >>> ' spacious '.strip() + 'spacious' + >>> 'www.example.com'.strip('cmowz.') + 'example' +\end{verbatim} \versionchanged[Support for the \var{chars} argument]{2.2.2} \end{methoddesc} From rhettinger at users.sourceforge.net Tue May 31 13:04:03 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 31 May 2005 04:04:03 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libstdtypes.tex, 1.179, 1.180 libfuncs.tex, 1.182, 1.183 Message-ID: <E1Dd4XT-0001zE-9R@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7415 Modified Files: libstdtypes.tex libfuncs.tex Log Message: SF bug #1209411: divmod documentation shd reference // not / Index: libstdtypes.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libstdtypes.tex,v retrieving revision 1.179 retrieving revision 1.180 diff -u -d -r1.179 -r1.180 --- libstdtypes.tex 31 May 2005 10:26:28 -0000 1.179 +++ libstdtypes.tex 31 May 2005 11:03:59 -0000 1.180 @@ -256,7 +256,7 @@ \lineiii{float(\var{x})}{\var{x} converted to floating point}{} \lineiii{complex(\var{re},\var{im})}{a complex number with real part \var{re}, imaginary part \var{im}. \var{im} defaults to zero.}{} \lineiii{\var{c}.conjugate()}{conjugate of the complex number \var{c}}{} - \lineiii{divmod(\var{x}, \var{y})}{the pair \code{(\var{x} / \var{y}, \var{x} \%{} \var{y})}}{(3)(4)} + \lineiii{divmod(\var{x}, \var{y})}{the pair \code{(\var{x} // \var{y}, \var{x} \%{} \var{y})}}{(3)(4)} \lineiii{pow(\var{x}, \var{y})}{\var{x} to the power \var{y}}{} \lineiii{\var{x} ** \var{y}}{\var{x} to the power \var{y}}{} \end{tableiii} Index: libfuncs.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libfuncs.tex,v retrieving revision 1.182 retrieving revision 1.183 diff -u -d -r1.182 -r1.183 --- libfuncs.tex 25 May 2005 05:39:36 -0000 1.182 +++ libfuncs.tex 31 May 2005 11:04:00 -0000 1.183 @@ -297,7 +297,7 @@ consisting of their quotient and remainder when using long division. With mixed operand types, the rules for binary arithmetic operators apply. For plain and long integers, the result is the same as - \code{(\var{a} / \var{b}, \var{a} \%{} \var{b})}. + \code{(\var{a} // \var{b}, \var{a} \%{} \var{b})}. For floating point numbers the result is \code{(\var{q}, \var{a} \%{} \var{b})}, where \var{q} is usually \code{math.floor(\var{a} / \var{b})} but may be 1 less than that. In any case \code{\var{q} * From gvanrossum at users.sourceforge.net Tue May 31 22:27:17 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 31 May 2005 13:27:17 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.15,1.16 Message-ID: <E1DdDKX-0004pf-TC@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18549 Modified Files: pep-0343.txt Log Message: Warn readers of coming attractions. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- pep-0343.txt 18 May 2005 14:24:27 -0000 1.15 +++ pep-0343.txt 31 May 2005 20:27:15 -0000 1.16 @@ -15,6 +15,14 @@ decided to withdraw PEP 340 and propose a slight variant on PEP 310. +Evolutionary Note + + After ample discussion on python-dev, I'll add back a mechanism + for raising an exception in a suspended generator using a throw() + method, and a close() method which throws a new GeneratorExit + exception. Until I get a chance to update the PEP, see reference + [2]. I'm also leaning towards 'with' as the keyword. + Motivation and Summary PEP 340, Anonymous Block Statements, combined many powerful ideas: @@ -445,6 +453,8 @@ [1] http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx + [2] http://mail.python.org/pipermail/python-dev/2005-May/053885.html + Copyright This document has been placed in the public domain.