From mwh at users.sourceforge.net Wed Jun 1 13:34:25 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Wed, 01 Jun 2005 04:34:25 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1293,1.1294 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1001 Modified Files: NEWS Log Message: NEWS entry for the patch I checked in last week. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1293 retrieving revision 1.1294 diff -u -d -r1.1293 -r1.1294 --- NEWS 20 May 2005 03:07:06 -0000 1.1293 +++ NEWS 1 Jun 2005 11:34:22 -0000 1.1294 @@ -12,6 +12,10 @@ Core and builtins ----------------- +- SF patch #1181301: on platforms that appear to use IEEE 754 floats, + the routines that promise to produce IEEE 754 binary representations + of floats now simply copy bytes around. + - bug #967182: disallow opening files with 'wU' or 'aU' as specified by PEP 278. From gvanrossum at users.sourceforge.net Wed Jun 1 17:13:46 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 01 Jun 2005 08:13:46 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.16,1.17 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22078 Modified Files: pep-0343.txt Log Message: Specify generator enhancements. Change keyword to 'with'. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- pep-0343.txt 31 May 2005 20:27:15 -0000 1.16 +++ pep-0343.txt 1 Jun 2005 15:13:37 -0000 1.17 @@ -1,5 +1,5 @@ PEP: 343 -Title: Anonymous Block Redux +Title: Anonymous Block Redux and Generator Enhancements Version: $Revision$ Last-Modified: $Date$ Author: Guido van Rossum @@ -11,17 +11,13 @@ Introduction - After a lot of discussion about PEP 340 and alternatives, I've - decided to withdraw PEP 340 and propose a slight variant on - PEP 310. - -Evolutionary Note - - After ample discussion on python-dev, I'll add back a mechanism + After a lot of discussion about PEP 340 and alternatives, I + decided to withdraw PEP 340 and proposed a slight variant on PEP + 310. After more discussion, I have added back a mechanism for raising an exception in a suspended generator using a throw() method, and a close() method which throws a new GeneratorExit - exception. Until I get a chance to update the PEP, see reference - [2]. I'm also leaning towards 'with' as the keyword. + exception; these additions were first proposed in [2] and + universally approved of. I'm also changing the keyword to 'with'. Motivation and Summary @@ -53,7 +49,7 @@ with VAR = EXPR: BLOCK - which roughly translates into + which roughly translates into this: VAR = EXPR VAR.__enter__() @@ -74,7 +70,7 @@ goto (a break, continue or return), BLOCK2 is *not* reached. The magic added by the with-statement at the end doesn't affect this. - (You may ask, what if a bug in the __exit__ method causes an + (You may ask, what if a bug in the __exit__() method causes an exception? Then all is lost -- but this is no worse than with other exceptions; the nature of exceptions is that they can happen *anywhere*, and you just have to live with that. Even if you @@ -89,16 +85,18 @@ Inspired by a counter-proposal to PEP 340 by Phillip Eby I tried to create a decorator that would turn a suitable generator into an - object with the necessary __entry__ and __exit__ methods. Here I - ran into a snag: while it wasn't too hard for the locking example, - it was impossible to do this for the opening example. The idea - was to define the template like this: + object with the necessary __enter__() and __exit__() methods. + Here I ran into a snag: while it wasn't too hard for the locking + example, it was impossible to do this for the opening example. + The idea was to define the template like this: @with_template def opening(filename): f = open(filename) - yield f - f.close() + try: + yield f + finally: + f.close() and used it like this: @@ -106,21 +104,21 @@ ...read data from f... The problem is that in PEP 310, the result of calling EXPR is - assigned directly to VAR, and then VAR's __exit__ method is called - upon exit from BLOCK1. But here, VAR clearly needs to receive the - opened file, and that would mean that __exit__ would have to be a - method on the file. + assigned directly to VAR, and then VAR's __exit__() method is + called upon exit from BLOCK1. But here, VAR clearly needs to + receive the opened file, and that would mean that __exit__() would + have to be a method on the file. While this can be solved using a proxy class, this is awkward and made me realize that a slightly different translation would make writing the desired decorator a piece of cake: let VAR receive the - result from calling the __enter__ method, and save the value of - EXPR to call its __exit__ method later. Then the decorator can - return an instance of a wrapper class whose __enter__ method calls - the generator's next() method and returns whatever next() returns; - the wrapper instance's __exit__ method calls next() again but - expects it to raise StopIteration. (Details below in the section - Optional Generator Decorator.) + result from calling the __enter__() method, and save the value of + EXPR to call its __exit__() method later. Then the decorator can + return an instance of a wrapper class whose __enter__() method + calls the generator's next() method and returns whatever next() + returns; the wrapper instance's __exit__() method calls next() + again but expects it to raise StopIteration. (Details below in + the section Optional Generator Decorator.) So now the final hurdle was that the PEP 310 syntax: @@ -128,41 +126,61 @@ BLOCK1 would be deceptive, since VAR does *not* receive the value of - EXPR. Given PEP 340, it was an easy step to: + EXPR. Borrowing from PEP 340, it was an easy step to: with EXPR as VAR: BLOCK1 - or, using an alternate keyword that has been proposed a number of - times: + Additional discussion showed that people really liked being able + to "see" the exception in the generator, even if it was only to + log it; the generator is not allowed to yield another value, since + the with-statement should not be usable as a loop (raising a + different exception is marginally acceptable). To enable this, a + new throw() method for generators is proposed, which takes three + arguments representing an exception in the usual fashion (type, + value, traceback) and raises it at the point where the generator + is suspended. - do EXPR as VAR: - BLOCK1 + Once we have this, it is a small step to proposing another + generator method, close(), which calls throw() with a special + exception, GeneratorExit. This tells the generator to exit, and + from there it's another small step to proposing that close() be + called automatically when the generator is garbage-collected. + + Then, finally, we can allow a yield-statement inside a try-finally + statement, since we can now guarantee that the finally-clause will + (eventually) be executed. The usual cautions about finalization + apply -- the process may be terminated abruptly without finalizing + any objects, and objects may be kept alive forever by cycles or + memory leaks in the application (as opposed to cycles or leaks in + the Python implementation, which are taken care of by GC). + + Note that we're not guaranteeing that the finally-clause is + executed immediately after the generator object becomes unused, + even though this is how it will work in CPython. This is similar + to auto-closing files: while a reference-counting implementation + like CPython deallocates an object as soon as the last reference + to it goes away, implementations that use other GC algorithms do + not make the same guarantee. This applies to Jython, IronPython, + and probably to Python running on Parrot. Use Cases See the Examples section near the end. -Specification +Specification: The 'with' Statement A new statement is proposed with the syntax: - do EXPR as VAR: + with EXPR as VAR: BLOCK - Here, 'do' and 'as' are new keywords; EXPR is an arbitrary + Here, 'with' and 'as' are new keywords; EXPR is an arbitrary expression (but not an expression-list) and VAR is an arbitrary assignment target (which may be a comma-separated list). The "as VAR" part is optional. - The choice of the 'do' keyword is provisional; an alternative - under consideration is 'with'. - - A yield-statement is illegal inside BLOCK. This is because the - do-statement is translated into a try/finally statement, and yield - is illegal in a try/finally statement. - The translation of the above statement is: abc = EXPR @@ -209,94 +227,184 @@ non-local goto should be considered unexceptional for the purposes of a database transaction roll-back decision. -Optional Generator Decorator +Specification: Generator Enhancements + + Let a generator object be the iterator produced by calling a + generator function. Below, 'g' always refers to a generator + object. + + New syntax: yield allowed inside try-finally + + The syntax for generator functions is extended to allow a + yield-statement inside a try-finally statement. + + New generator method: throw(type, value, traceback) + + g.throw(type, value, traceback) causes the specified exception to + be thrown at the point where the generator g is currently + suspended (i.e. at a yield-statement, or at the start of its + function body if next() has not been called yet). If the + generator catches the exception and yields another value, that is + the return value of g.throw(). If it doesn't catch the exception, + the throw() appears to raise the same exception passed it (it + "falls through"). If the generator raises another exception (this + includes the StopIteration produced when it returns) that + exception is raised by the throw() call. In summary, throw() + behaves like next() except it raises an exception at the + suspension point. If the generator is already in the closed + state, throw() just raises the exception it was passed without + executing any of the generator's code. + + The effect of raising the exception is exactly as if the + statement: + + raise type, value, traceback + + was executed at the suspension point. The type argument should + not be None. + + New standard exception: GeneratorExit + + A new standard exception is defined, GeneratorExit, inheriting + from Exception. A generator should handle this by re-raising it + or by raising StopIteration. + + New generator method: close() + + g.close() is defined by the following pseudo-code: + + def close(self): + try: + self.throw(GeneratorExit, GeneratorExit(), None) + except (GeneratorExit, StopIteration): + pass + else: + raise TypeError("generator ignored GeneratorExit") + # Other exceptions are not caught + + New generator method: __del__() + + g.__del__() is an alias for g.close(). This will be called when + the generator object is garbage-collected (in CPython, this is + when its reference count goes to zero). If close() raises an + exception, a traceback for the exception is printed to sys.stderr + and further ignored; it is not propagated back to the place that + triggered the garbage collection. This is consistent with the + handling of exceptions in __del__() methods on class instances. + + If the generator object participates in a cycle, g.__del__() may + not be called. This is the behavior of CPython's current garbage + collector. The reason for the restriction is that the GC code + needs to "break" a cycle at an arbitrary point in order to collect + it, and from then on no Python code should be allowed to see the + objects that formed the cycle, as they may be in an invalid state. + Objects "hanging off" a cycle are not subject to this restriction. + Note that it is unlikely to see a generator object participate in + a cycle in practice. However, storing a generator object in a + global variable creates a cycle via the generator frame's + f_globals pointer. Another way to create a cycle would be to + store a reference to the generator object in a data structure that + is passed to the generator as an argument. Neither of these cases + are very likely given the typical pattern of generator use. + +Generator Decorator It is possible to write a decorator that makes it possible to use - a generator that yields exactly once to control a do-statement. + a generator that yields exactly once to control a with-statement. Here's a sketch of such a decorator: class Wrapper(object): + def __init__(self, gen): self.gen = gen - self.state = "initial" + def __enter__(self): - assert self.state == "initial" - self.state = "entered" try: return self.gen.next() except StopIteration: - self.state = "error" - raise RuntimeError("template generator didn't yield") - def __exit__(self, *args): - assert self.state == "entered" - self.state = "exited" - try: - self.gen.next() - except StopIteration: - return + raise RuntimeError("generator didn't yield") + + def __exit__(self, type, value, traceback): + if type is None: + try: + self.gen.next() + except StopIteration: + return + else: + raise RuntimeError("generator didn't stop") else: - self.state = "error" - raise RuntimeError("template generator didn't stop") + try: + self.gen.throw(type, value, traceback) + except (type, StopIteration): + return + else: + raise RuntimeError("generator caught exception") - def do_template(func): + def with_template(func): def helper(*args, **kwds): return Wrapper(func(*args, **kwds)) return helper This decorator could be used as follows: - @do_template + @with_template def opening(filename): f = open(filename) # IOError here is untouched by Wrapper yield f f.close() # Ditto for errors here (however unlikely) - A robust implementation of such a decorator should be made part of - the standard library. + A robust implementation of this decorator should be made part of + the standard library, but not necessarily as a built-in function. + (I'm not sure which exception it should raise for errors; + RuntimeError is used above as an example only.) -Other Optional Extensions +Optional Extensions It would be possible to endow certain objects, like files, - sockets, and locks, with __enter__ and __exit__ methods so that - instead of writing: + sockets, and locks, with __enter__() and __exit__() methods so + that instead of writing: - do locking(myLock): + with locking(myLock): BLOCK one could write simply: - do myLock: + with myLock: BLOCK I think we should be careful with this; it could lead to mistakes like: f = open(filename) - do f: + with f: BLOCK1 - do f: + with f: BLOCK2 - which does not do what one might think (f is closed when BLOCK2 is - entered). + which does not do what one might think (f is closed before BLOCK2 + is entered). + + OTOH such mistakes are easily diagnosed. Examples - Several of these examples contain "yield None". If PEP 342 is - accepted, these can be changed to just "yield". + (Note: several of these examples contain "yield None". If PEP 342 + is accepted, these can be changed to just "yield".) 1. A template for ensuring that a lock, acquired at the start of a block, is released when the block is left: - @do_template + @with_template def locking(lock): lock.acquire() - yield None - lock.release() + try: + yield None + finally: + lock.release() Used as follows: - do locking(myLock): + with locking(myLock): # Code here executes with myLock held. The lock is # guaranteed to be released when the block is left (even # if via return or by an uncaught exception). @@ -304,29 +412,32 @@ 2. A template for opening a file that ensures the file is closed when the block is left: - @do_template + @with_template def opening(filename, mode="r"): f = open(filename, mode) - yield f - f.close() + try: + yield f + finally: + f.close() Used as follows: - do opening("/etc/passwd") as f: + with opening("/etc/passwd") as f: for line in f: print line.rstrip() 3. A template for committing or rolling back a database transaction; this is written as a class rather than as a - decorator since it requires access to the exception information: + decorator since it requires access to the exception + information: class transactional: def __init__(self, db): self.db = db def __enter__(self): self.db.begin() - def __exit__(self, *args): - if args and args[0] is not None: + def __exit__(self, type, value, tb): + if type is not None: self.db.rollback() else: self.db.commit() @@ -338,47 +449,51 @@ self.lock = lock def __enter__(self): self.lock.acquire() - def __exit__(self, *args): + def __exit__(self, type, value, tb): self.lock.release() (This example is easily modified to implement the other - examples; it shows how much simpler generators are for the same - purpose.) + examples; it shows the relative advantage of using a generator + template.) 5. Redirect stdout temporarily: - @do_template + @with_template def redirecting_stdout(new_stdout): save_stdout = sys.stdout sys.stdout = new_stdout - yield None - sys.stdout = save_stdout + try: + yield None + finally: + sys.stdout = save_stdout Used as follows: - do opening(filename, "w") as f: - do redirecting_stdout(f): + with opening(filename, "w") as f: + with redirecting_stdout(f): print "Hello world" This isn't thread-safe, of course, but neither is doing this - same dance manually. In a single-threaded program (e.g., a - script) it is a totally fine way of doing things. + same dance manually. In single-threaded programs (for example, + in scripts) it is a popular way of doing things. 6. A variant on opening() that also returns an error condition: - @do_template + @with_template def opening_w_error(filename, mode="r"): try: f = open(filename, mode) except IOError, err: yield None, err else: - yield f, None - f.close() + try: + yield f, None + finally: + f.close() Used as follows: - do opening_w_error("/etc/passwd", "a") as f, err: + with opening_w_error("/etc/passwd", "a") as f, err: if err: print "IOError:", err else: @@ -389,7 +504,7 @@ import signal - do signal.blocking(): + with signal.blocking(): # code executed without worrying about signals An optional argument might be a list of signals to be blocked; @@ -401,19 +516,21 @@ import decimal - @do_template - def with_extra_precision(places=2): + @with_template + def extra_precision(places=2): c = decimal.getcontext() saved_prec = c.prec c.prec += places - yield None - c.prec = saved_prec + try: + yield None + finally: + c.prec = saved_prec Sample usage (adapted from the Python Library Reference): def sin(x): "Return the sine of x as measured in radians." - do with_extra_precision(): + with extra_precision(): i, lasts, s, fact, num, sign = 1, 0, x, 1, x, 1 while s != lasts: lasts = s @@ -423,31 +540,33 @@ sign *= -1 s += num / fact * sign # The "+s" rounds back to the original precision, - # so this must be outside the do-statement: + # so this must be outside the with-statement: return +s 9. Here's a more general Decimal-context-switching template: - @do_template - def with_decimal_context(newctx=None): + @with_template + def decimal_context(newctx=None): oldctx = decimal.getcontext() if newctx is None: newctx = oldctx.copy() decimal.setcontext(newctx) - yield newctx - decimal.setcontext(oldctx) + try: + yield newctx + finally: + decimal.setcontext(oldctx) - Sample usage (adapted from the previous one): + Sample usage: def sin(x): - do with_decimal_context() as ctx: + with decimal_context() as ctx: ctx.prec += 2 - # Rest of algorithm the same + # Rest of algorithm the same as above return +s - (Nick Coghlan has proposed to add __enter__ and __exit__ + (Nick Coghlan has proposed to add __enter__() and __exit__() methods to the decimal.Context class so that this example can - be simplified to "do decimal.getcontext() as ctx: ...".) + be simplified to "with decimal.getcontext() as ctx: ...".) References From gvanrossum at users.sourceforge.net Wed Jun 1 17:16:51 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 01 Jun 2005 08:16:51 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt,1.311,1.312 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23454 Modified Files: pep-0000.txt Log Message: Updated PEP 343 title. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.311 retrieving revision 1.312 diff -u -d -r1.311 -r1.312 --- pep-0000.txt 28 May 2005 22:44:24 -0000 1.311 +++ pep-0000.txt 1 Jun 2005 15:16:33 -0000 1.312 @@ -119,7 +119,7 @@ S 338 Executing modules inside packages with '-m' Coghlan S 341 Unifying try-except and try-finally Birkenfeld S 342 Enhanced Iterators GvR - S 343 Anonymous Block Redux GvR + S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones S 754 IEEE 754 Floating Point Special Values Warnes @@ -384,7 +384,7 @@ SR 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld S 342 Enhanced Iterators GvR - S 343 Anonymous Block Redux GvR + S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones SR 346 User Defined ("with") Statements Coghlan From akuchling at users.sourceforge.net Wed Jun 1 17:26:27 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 08:26:27 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib liburllib.tex,1.56,1.57 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27277 Modified Files: liburllib.tex Log Message: [Bug #1194249] Fix duplicate assignment in example code Index: liburllib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/liburllib.tex,v retrieving revision 1.56 retrieving revision 1.57 diff -u -d -r1.56 -r1.57 --- liburllib.tex 9 Mar 2005 03:01:31 -0000 1.56 +++ liburllib.tex 1 Jun 2005 15:26:24 -0000 1.57 @@ -97,7 +97,7 @@ \begin{verbatim} # Use http://www.someproxy.com:3128 for http proxying -proxies = proxies={'http': 'http://www.someproxy.com:3128'} +proxies = {'http': 'http://www.someproxy.com:3128'} filehandle = urllib.urlopen(some_url, proxies=proxies) # Don't use any proxies filehandle = urllib.urlopen(some_url, proxies={}) From akuchling at users.sourceforge.net Wed Jun 1 17:27:29 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 08:27:29 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib liburllib.tex, 1.54.4.2, 1.54.4.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27673 Modified Files: Tag: release24-maint liburllib.tex Log Message: [Bug #1194249] Fix duplicate assignment in example code Index: liburllib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/liburllib.tex,v retrieving revision 1.54.4.2 retrieving revision 1.54.4.3 diff -u -d -r1.54.4.2 -r1.54.4.3 --- liburllib.tex 9 Mar 2005 03:02:17 -0000 1.54.4.2 +++ liburllib.tex 1 Jun 2005 15:27:26 -0000 1.54.4.3 @@ -97,7 +97,7 @@ \begin{verbatim} # Use http://www.someproxy.com:3128 for http proxying -proxies = proxies={'http': 'http://www.someproxy.com:3128'} +proxies = {'http': 'http://www.someproxy.com:3128'} filehandle = urllib.urlopen(some_url, proxies=proxies) # Don't use any proxies filehandle = urllib.urlopen(some_url, proxies={}) From akuchling at users.sourceforge.net Wed Jun 1 17:28:03 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 08:28:03 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib liburllib.tex, 1.50.8.2, 1.50.8.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27881 Modified Files: Tag: release23-maint liburllib.tex Log Message: [Bug #1194249] Fix duplicate assignment in example code Index: liburllib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/liburllib.tex,v retrieving revision 1.50.8.2 retrieving revision 1.50.8.3 diff -u -d -r1.50.8.2 -r1.50.8.3 --- liburllib.tex 1 Apr 2004 04:11:05 -0000 1.50.8.2 +++ liburllib.tex 1 Jun 2005 15:28:01 -0000 1.50.8.3 @@ -97,7 +97,7 @@ \begin{verbatim} # Use http://www.someproxy.com:3128 for http proxying -proxies = proxies={'http': 'http://www.someproxy.com:3128'} +proxies = {'http': 'http://www.someproxy.com:3128'} filehandle = urllib.urlopen(some_url, proxies=proxies) # Don't use any proxies filehandle = urllib.urlopen(some_url, proxies={}) From akuchling at users.sourceforge.net Wed Jun 1 17:39:59 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 08:39:59 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libasyncore.tex, 1.17, 1.18 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1844 Modified Files: libasyncore.tex Log Message: [Bug #1181939] Remove incorrect text about __init__; move map discussion into a separate paragraph Index: libasyncore.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libasyncore.tex,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- libasyncore.tex 30 Jun 2004 09:02:33 -0000 1.17 +++ libasyncore.tex 1 Jun 2005 15:39:57 -0000 1.18 @@ -53,12 +53,11 @@ \function{poll()} call, measured in seconds; the default is 30 seconds. The \var{use_poll} parameter, if true, indicates that \function{poll()} should be used in preference to \function{select()} (the default is - \code{False}). The \var{map} parameter is a dictionary whose items are - the channels to watch. As channels are closed they are deleted from their - map. If \var{map} is omitted, a global map is used (this map is updated - by the default class \method{__init__()} -- make sure you extend, rather - than override, \method{__init__()} if you want to retain this behavior). + \code{False}). + The \var{map} parameter is a dictionary whose items are + the channels to watch. As channels are closed they are deleted from their + map. If \var{map} is omitted, a global map is used. Channels (instances of \class{asyncore.dispatcher}, \class{asynchat.async_chat} and subclasses thereof) can freely be mixed in the map. \end{funcdesc} From gvanrossum at users.sourceforge.net Wed Jun 1 18:45:27 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 01 Jun 2005 09:45:27 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.17,1.18 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5610 Modified Files: pep-0343.txt Log Message: Address most of Phillip Eby's comments. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- pep-0343.txt 1 Jun 2005 15:13:37 -0000 1.17 +++ pep-0343.txt 1 Jun 2005 16:45:25 -0000 1.18 @@ -282,6 +282,8 @@ raise TypeError("generator ignored GeneratorExit") # Other exceptions are not caught + (XXX is TypeError an acceptable exception here?) + New generator method: __del__() g.__del__() is an alias for g.close(). This will be called when @@ -350,8 +352,10 @@ @with_template def opening(filename): f = open(filename) # IOError here is untouched by Wrapper - yield f - f.close() # Ditto for errors here (however unlikely) + try: + yield f + finally: + f.close() # Ditto for errors here (however unlikely) A robust implementation of this decorator should be made part of the standard library, but not necessarily as a built-in function. @@ -427,20 +431,17 @@ print line.rstrip() 3. A template for committing or rolling back a database - transaction; this is written as a class rather than as a - decorator since it requires access to the exception - information: + transaction: - class transactional: - def __init__(self, db): - self.db = db - def __enter__(self): - self.db.begin() - def __exit__(self, type, value, tb): - if type is not None: - self.db.rollback() - else: - self.db.commit() + @with_template + def transactional(db): + db.begin() + try: + yield None + except: + db.rollback() + else: + db.commit() 4. Example 1 rewritten without a generator: From gvanrossum at users.sourceforge.net Wed Jun 1 23:01:17 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Wed, 01 Jun 2005 14:01:17 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.18,1.19 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19274 Modified Files: pep-0343.txt Log Message: List PEPs that can be rejected once this is accepted. This turned up a surprise: PEP 319. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.18 retrieving revision 1.19 diff -u -d -r1.18 -r1.19 --- pep-0343.txt 1 Jun 2005 16:45:25 -0000 1.18 +++ pep-0343.txt 1 Jun 2005 21:01:11 -0000 1.19 @@ -19,6 +19,33 @@ exception; these additions were first proposed in [2] and universally approved of. I'm also changing the keyword to 'with'. + If this PEP is approved, the following PEPs will be rejected due + to overlap: + + - PEP 288, Generators Attributes and Exceptions. The current PEP + covers its second half, generator exceptions (in fact the + throw() method name was taken from this PEP). I'm not in favor + of generator attributes, since they can easily be handled by + making the generator a method or by passing a mutable argument. + + - PEP 310, Reliable Acquisition/Release Pairs. This is the + original with-statement proposal. + + - PEP 325, Resource-Release Support for Generators. The current + PEP covers this (in fact the close() method name was taken from + this PEP). + + - PEP 319, Python Synchronize/Asynchronize Block. Its use cases + can be covered by the current PEP by providing suitable + with-statement controllers: for 'synchronize' we can use the + "locking" template from example 1; for 'asynchronize' we can use + a similar "unlocking" template. I don't think having an + "anonymous" lock associated with a code block is all that + important; in fact it may be better to always be explicit about + the mutex being used. + + (PEP 340 and PEP 346 have already been withdrawn.) + Motivation and Summary PEP 340, Anonymous Block Statements, combined many powerful ideas: @@ -413,6 +440,10 @@ # guaranteed to be released when the block is left (even # if via return or by an uncaught exception). + PEP 319 gives a use case for also having an unlocking() + template; this can be written very similarly (just swap the + acquire() and release() calls). + 2. A template for opening a file that ensures the file is closed when the block is left: From akuchling at users.sourceforge.net Thu Jun 2 01:22:14 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 16:22:14 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libcurses.tex,1.49,1.50 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24968 Modified Files: libcurses.tex Log Message: [Bug #1123268] Fix typo Index: libcurses.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libcurses.tex,v retrieving revision 1.49 retrieving revision 1.50 diff -u -d -r1.49 -r1.50 --- libcurses.tex 1 Jan 2005 00:28:35 -0000 1.49 +++ libcurses.tex 1 Jun 2005 23:22:11 -0000 1.50 @@ -161,7 +161,7 @@ calls, LINES is set to 1; the capabilities clear, cup, cud, cud1, cuu1, cuu, vpa are disabled; and the home string is set to the value of cr. The effect is that the cursor is confined to the current line, and so -are screen updates. This may be used for enabling cgaracter-at-a-time +are screen updates. This may be used for enabling character-at-a-time line editing without touching the rest of the screen. \end{funcdesc} From akuchling at users.sourceforge.net Thu Jun 2 01:22:42 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 16:22:42 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libcurses.tex, 1.48.2.1, 1.48.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25170 Modified Files: Tag: release24-maint libcurses.tex Log Message: [Bug #1123268] Fix typo Index: libcurses.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libcurses.tex,v retrieving revision 1.48.2.1 retrieving revision 1.48.2.2 diff -u -d -r1.48.2.1 -r1.48.2.2 --- libcurses.tex 1 Jan 2005 00:34:53 -0000 1.48.2.1 +++ libcurses.tex 1 Jun 2005 23:22:39 -0000 1.48.2.2 @@ -161,7 +161,7 @@ calls, LINES is set to 1; the capabilities clear, cup, cud, cud1, cuu1, cuu, vpa are disabled; and the home string is set to the value of cr. The effect is that the cursor is confined to the current line, and so -are screen updates. This may be used for enabling cgaracter-at-a-time +are screen updates. This may be used for enabling character-at-a-time line editing without touching the rest of the screen. \end{funcdesc} From akuchling at users.sourceforge.net Thu Jun 2 01:31:21 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 16:31:21 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libcurses.tex,1.50,1.51 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30996 Modified Files: libcurses.tex Log Message: [Bug #1105706] Use correct name for constant Index: libcurses.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libcurses.tex,v retrieving revision 1.50 retrieving revision 1.51 diff -u -d -r1.50 -r1.51 --- libcurses.tex 1 Jun 2005 23:22:11 -0000 1.50 +++ libcurses.tex 1 Jun 2005 23:31:18 -0000 1.51 @@ -635,8 +635,8 @@ \lineiii{bs}{Bottom}{\constant{ACS_HLINE}} \lineiii{tl}{Upper-left corner}{\constant{ACS_ULCORNER}} \lineiii{tr}{Upper-right corner}{\constant{ACS_URCORNER}} - \lineiii{bl}{Bottom-left corner}{\constant{ACS_BLCORNER}} - \lineiii{br}{Bottom-right corner}{\constant{ACS_BRCORNER}} + \lineiii{bl}{Bottom-left corner}{\constant{ACS_LLCORNER}} + \lineiii{br}{Bottom-right corner}{\constant{ACS_LRCORNER}} \end{tableiii} \end{methoddesc} From akuchling at users.sourceforge.net Thu Jun 2 01:32:34 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 16:32:34 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libcurses.tex, 1.48.2.2, 1.48.2.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31745 Modified Files: Tag: release24-maint libcurses.tex Log Message: [Bug #1105706] Use correct name for constant Index: libcurses.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libcurses.tex,v retrieving revision 1.48.2.2 retrieving revision 1.48.2.3 diff -u -d -r1.48.2.2 -r1.48.2.3 --- libcurses.tex 1 Jun 2005 23:22:39 -0000 1.48.2.2 +++ libcurses.tex 1 Jun 2005 23:32:31 -0000 1.48.2.3 @@ -635,8 +635,8 @@ \lineiii{bs}{Bottom}{\constant{ACS_HLINE}} \lineiii{tl}{Upper-left corner}{\constant{ACS_ULCORNER}} \lineiii{tr}{Upper-right corner}{\constant{ACS_URCORNER}} - \lineiii{bl}{Bottom-left corner}{\constant{ACS_BLCORNER}} - \lineiii{br}{Bottom-right corner}{\constant{ACS_BRCORNER}} + \lineiii{bl}{Bottom-left corner}{\constant{ACS_LLCORNER}} + \lineiii{br}{Bottom-right corner}{\constant{ACS_LRCORNER}} \end{tableiii} \end{methoddesc} From pierslauder at users.sourceforge.net Thu Jun 2 01:50:54 2005 From: pierslauder at users.sourceforge.net (pierslauder@users.sourceforge.net) Date: Wed, 01 Jun 2005 16:50:54 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libimaplib.tex, 1.32, 1.33 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9053/dist/src/Doc/lib Modified Files: libimaplib.tex Log Message: added GET/SETANNOTATION methods Index: libimaplib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libimaplib.tex,v retrieving revision 1.32 retrieving revision 1.33 diff -u -d -r1.32 -r1.33 --- libimaplib.tex 19 Jan 2005 04:44:07 -0000 1.32 +++ libimaplib.tex 1 Jun 2005 23:50:52 -0000 1.33 @@ -222,6 +222,11 @@ The method is non-standard, but is supported by the \samp{Cyrus} server. \end{methoddesc} +\begin{methoddesc}{getannotation}{mailbox, entry, attribute} + Retrieve the specified \samp{ANNOTATION}s for \var{mailbox}. + The method is non-standard, but is supported by the \samp{Cyrus} server. +\end{methoddesc} + \begin{methoddesc}{getquota}{root} Get the \samp{quota} \var{root}'s resource usage and limits. This method is part of the IMAP4 QUOTA extension defined in rfc2087. @@ -357,6 +362,11 @@ The method is non-standard, but is supported by the \samp{Cyrus} server. \end{methoddesc} +\begin{methoddesc}{setannotation}{mailbox, entry, attribute\optional{, ...}} + Set \samp{ANNOTATION}s for \var{mailbox}. + The method is non-standard, but is supported by the \samp{Cyrus} server. +\end{methoddesc} + \begin{methoddesc}{setquota}{root, limits} Set the \samp{quota} \var{root}'s resource \var{limits}. This method is part of the IMAP4 QUOTA extension defined in rfc2087. From pierslauder at users.sourceforge.net Thu Jun 2 01:50:54 2005 From: pierslauder at users.sourceforge.net (pierslauder@users.sourceforge.net) Date: Wed, 01 Jun 2005 16:50:54 -0700 Subject: [Python-checkins] python/dist/src/Lib imaplib.py,1.75,1.76 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9053/dist/src/Lib Modified Files: imaplib.py Log Message: added GET/SETANNOTATION methods Index: imaplib.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/imaplib.py,v retrieving revision 1.75 retrieving revision 1.76 diff -u -d -r1.75 -r1.76 --- imaplib.py 2 Mar 2005 09:13:45 -0000 1.75 +++ imaplib.py 1 Jun 2005 23:50:51 -0000 1.76 @@ -18,8 +18,9 @@ # IMAP4_SSL contributed by Tino Lange March 2002. # GET/SETQUOTA contributed by Andreas Zeidler June 2002. # PROXYAUTH contributed by Rick Holbert November 2002. +# GET/SETANNOTATION contributed by Tomas Lindroos June 2005. -__version__ = "2.55" +__version__ = "2.56" import binascii, os, random, re, socket, sys, time @@ -51,6 +52,7 @@ 'EXPUNGE': ('SELECTED',), 'FETCH': ('SELECTED',), 'GETACL': ('AUTH', 'SELECTED'), + 'GETANNOTATION':('AUTH', 'SELECTED'), 'GETQUOTA': ('AUTH', 'SELECTED'), 'GETQUOTAROOT': ('AUTH', 'SELECTED'), 'MYRIGHTS': ('AUTH', 'SELECTED'), @@ -66,6 +68,7 @@ 'SEARCH': ('SELECTED',), 'SELECT': ('AUTH', 'SELECTED'), 'SETACL': ('AUTH', 'SELECTED'), + 'SETANNOTATION':('AUTH', 'SELECTED'), 'SETQUOTA': ('AUTH', 'SELECTED'), 'SORT': ('SELECTED',), 'STATUS': ('AUTH', 'SELECTED'), @@ -133,10 +136,10 @@ the command re-tried. "readonly" exceptions imply the command should be re-tried. - Note: to use this module, you must read the RFCs pertaining - to the IMAP4 protocol, as the semantics of the arguments to - each IMAP4 command are left to the invoker, not to mention - the results. + Note: to use this module, you must read the RFCs pertaining to the + IMAP4 protocol, as the semantics of the arguments to each IMAP4 + command are left to the invoker, not to mention the results. Also, + most IMAP servers implement a sub-set of the commands available here. """ class error(Exception): pass # Logical errors - debug required @@ -186,11 +189,10 @@ else: raise self.error(self.welcome) - cap = 'CAPABILITY' - self._simple_command(cap) - if not cap in self.untagged_responses: + typ, dat = self.capability() + if dat == [None]: raise self.error('no CAPABILITY response from server') - self.capabilities = tuple(self.untagged_responses[cap][-1].upper().split()) + self.capabilities = tuple(dat[-1].upper().split()) if __debug__: if self.debug >= 3: @@ -345,6 +347,15 @@ return typ, dat + def capability(self): + """(typ, [data]) = .capability() + Fetch capabilities list from server.""" + + name = 'CAPABILITY' + typ, dat = self._simple_command(name) + return self._untagged_response(typ, dat, name) + + def check(self): """Checkpoint mailbox on server. @@ -436,6 +447,14 @@ return self._untagged_response(typ, dat, 'ACL') + def getannotation(self, mailbox, entry, attribute): + """(typ, [data]) = .getannotation(mailbox, entry, attribute) + Retrieve ANNOTATIONs.""" + + typ, dat = self._simple_command('GETANNOTATION', mailbox, entry, attribute) + return self._untagged_response(typ, dat, 'ANNOTATION') + + def getquota(self, root): """Get the quota root's resource usage and limits. @@ -643,6 +662,14 @@ return self._simple_command('SETACL', mailbox, who, what) + def setannotation(self, *args): + """(typ, [data]) = .setannotation(mailbox[, entry, attribute]+) + Set ANNOTATIONs.""" + + typ, dat = self._simple_command('SETANNOTATION', *args) + return self._untagged_response(typ, dat, 'ANNOTATION') + + def setquota(self, root, limits): """Set the quota root's resource limits. From akuchling at users.sourceforge.net Thu Jun 2 02:10:07 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 17:10:07 -0700 Subject: [Python-checkins] python/dist/src/Lib/curses textpad.py,1.11,1.12 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/curses In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18874/Lib/curses Modified Files: textpad.py Log Message: [Bug #1152762] Ensure _end_of_line() returns an x-coordinate that's within the text box Index: textpad.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/curses/textpad.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- textpad.py 19 Oct 2004 19:36:09 -0000 1.11 +++ textpad.py 2 Jun 2005 00:10:04 -0000 1.12 @@ -53,7 +53,7 @@ last = self.maxx while 1: if ascii.ascii(self.win.inch(y, last)) != ascii.SP: - last = last + 1 + last = min(self.maxx, last+1) break elif last == 0: break From akuchling at users.sourceforge.net Thu Jun 2 02:11:57 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 01 Jun 2005 17:11:57 -0700 Subject: [Python-checkins] python/dist/src/Lib/curses textpad.py, 1.11, 1.11.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/curses In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20088/Lib/curses Modified Files: Tag: release24-maint textpad.py Log Message: [Bug #1152762] Ensure _end_of_line() returns an x-coordinate that's within the text box Index: textpad.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/curses/textpad.py,v retrieving revision 1.11 retrieving revision 1.11.2.1 diff -u -d -r1.11 -r1.11.2.1 --- textpad.py 19 Oct 2004 19:36:09 -0000 1.11 +++ textpad.py 2 Jun 2005 00:11:55 -0000 1.11.2.1 @@ -53,7 +53,7 @@ last = self.maxx while 1: if ascii.ascii(self.win.inch(y, last)) != ascii.SP: - last = last + 1 + last = min(self.maxx, last+1) break elif last == 0: break From nascheme at users.sourceforge.net Thu Jun 2 07:14:37 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Wed, 01 Jun 2005 22:14:37 -0700 Subject: [Python-checkins] python/dist/src/Lib/test/output test_grammar, 1.19.2.1, 1.19.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test/output In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13883/Lib/test/output Modified Files: Tag: ast-branch test_grammar Log Message: Use block type attribute from symbol table to catch 'return' and 'yield' syntax errors. Index: test_grammar =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/output/test_grammar,v retrieving revision 1.19.2.1 retrieving revision 1.19.2.2 diff -u -d -r1.19.2.1 -r1.19.2.2 --- test_grammar 7 Jan 2005 06:59:20 -0000 1.19.2.1 +++ test_grammar 2 Jun 2005 05:14:34 -0000 1.19.2.2 @@ -34,6 +34,7 @@ continue + try/finally ok testing continue and break in try/except in loop return_stmt +yield_stmt raise_stmt import_name import_from From nascheme at users.sourceforge.net Thu Jun 2 07:14:36 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Wed, 01 Jun 2005 22:14:36 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_grammar.py, 1.40.8.3, 1.40.8.4 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13883/Lib/test Modified Files: Tag: ast-branch test_grammar.py Log Message: Use block type attribute from symbol table to catch 'return' and 'yield' syntax errors. Index: test_grammar.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_grammar.py,v retrieving revision 1.40.8.3 retrieving revision 1.40.8.4 diff -u -d -r1.40.8.3 -r1.40.8.4 --- test_grammar.py 7 Jan 2005 06:59:09 -0000 1.40.8.3 +++ test_grammar.py 2 Jun 2005 05:14:34 -0000 1.40.8.4 @@ -413,6 +413,10 @@ def g2(): return 1 g1() x = g2() +check_syntax("class foo:return 1") + +print 'yield_stmt' +check_syntax("class foo:yield 1") print 'raise_stmt' # 'raise' test [',' test] try: raise RuntimeError, 'just testing' From nascheme at users.sourceforge.net Thu Jun 2 07:14:37 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Wed, 01 Jun 2005 22:14:37 -0700 Subject: [Python-checkins] python/dist/src/Python newcompile.c, 1.1.2.107, 1.1.2.108 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13883/Python Modified Files: Tag: ast-branch newcompile.c Log Message: Use block type attribute from symbol table to catch 'return' and 'yield' syntax errors. Index: newcompile.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/Attic/newcompile.c,v retrieving revision 1.1.2.107 retrieving revision 1.1.2.108 diff -u -d -r1.1.2.107 -r1.1.2.108 --- newcompile.c 15 Apr 2005 01:49:23 -0000 1.1.2.107 +++ newcompile.c 2 Jun 2005 05:14:34 -0000 1.1.2.108 @@ -1992,8 +1992,8 @@ case ClassDef_kind: return compiler_class(c, s); case Return_kind: - if (c->c_nestlevel <= 1) - return compiler_error(c, "'return' outside function"); + if (c->u->u_ste->ste_type != FunctionBlock) + return compiler_error(c, "'return' outside function"); if (s->v.Return.value) { if (c->u->u_ste->ste_generator) { return compiler_error(c, @@ -2006,7 +2006,7 @@ ADDOP(c, RETURN_VALUE); break; case Yield_kind: - if (c->c_nestlevel <= 1) + if (c->u->u_ste->ste_type != FunctionBlock) return compiler_error(c, "'yield' outside function"); for (i = 0; i < c->u->u_nfblocks; i++) { if (c->u->u_fblock[i].fb_type == FINALLY_TRY) From nascheme at users.sourceforge.net Thu Jun 2 07:34:38 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Wed, 01 Jun 2005 22:34:38 -0700 Subject: [Python-checkins] python/dist/src/Python ast.c,1.1.2.62,1.1.2.63 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23165/Python Modified Files: Tag: ast-branch ast.c Log Message: Add invalid trailing comma syntax check for import statements. Closes SF patch #1194895. Index: ast.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/Attic/ast.c,v retrieving revision 1.1.2.62 retrieving revision 1.1.2.63 diff -u -d -r1.1.2.62 -r1.1.2.63 --- ast.c 28 Apr 2005 03:32:39 -0000 1.1.2.62 +++ ast.c 2 Jun 2005 05:34:35 -0000 1.1.2.63 @@ -2079,8 +2079,18 @@ /* XXX this needs to be cleaned up */ from_modules = STR(CHILD(n, 3)); - if (!from_modules || from_modules[0] == '*') + if (!from_modules) { n = CHILD(n, 3); /* from ... import x, y, z */ + if (NCH(n) % 2 == 0) { + /* it ends with a comma, not valid but the parser allows it */ + ast_error(n, "trailing comma not allowed without" + " surrounding parentheses"); + return NULL; + } + } + else if (from_modules[0] == '*') { + n = CHILD(n,3); /* from ... import * */ + } else if (from_modules[0] == '(') n = CHILD(n, 4); /* from ... import (x, y, z) */ else From nascheme at users.sourceforge.net Thu Jun 2 07:55:21 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Wed, 01 Jun 2005 22:55:21 -0700 Subject: [Python-checkins] python/dist/src/Lib/compiler ast.py,1.27,1.28 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/compiler In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv367/Lib/compiler Modified Files: ast.py Log Message: Fix compiler.ast.flatten function so that it works on lists. Index: ast.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/compiler/ast.py,v retrieving revision 1.27 retrieving revision 1.28 diff -u -d -r1.27 -r1.28 --- ast.py 12 Sep 2004 03:49:29 -0000 1.27 +++ ast.py 2 Jun 2005 05:55:18 -0000 1.28 @@ -4,9 +4,9 @@ """ from consts import CO_VARARGS, CO_VARKEYWORDS -def flatten(list): +def flatten(seq): l = [] - for elt in list: + for elt in seq: t = type(elt) if t is tuple or t is list: for elt2 in flatten(elt): @@ -15,8 +15,8 @@ l.append(elt) return l -def flatten_nodes(list): - return [n for n in flatten(list) if isinstance(n, Node)] +def flatten_nodes(seq): + return [n for n in flatten(seq) if isinstance(n, Node)] nodes = {} From nascheme at users.sourceforge.net Thu Jun 2 07:55:22 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Wed, 01 Jun 2005 22:55:22 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_compiler.py, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv367/Lib/test Modified Files: test_compiler.py Log Message: Fix compiler.ast.flatten function so that it works on lists. Index: test_compiler.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_compiler.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- test_compiler.py 20 Apr 2005 17:45:12 -0000 1.12 +++ test_compiler.py 2 Jun 2005 05:55:19 -0000 1.13 @@ -1,4 +1,5 @@ import compiler +from compiler.ast import flatten import os import test.test_support import unittest @@ -60,6 +61,10 @@ for child in node.getChildNodes(): self.check_lineno(child) + def testFlatten(self): + self.assertEquals(flatten([1, [2]]), [1, 2]) + self.assertEquals(flatten((1, (2,))), [1, 2]) + NOLINENO = (compiler.ast.Module, compiler.ast.Stmt, compiler.ast.Discard) ############################################################################### From nascheme at users.sourceforge.net Thu Jun 2 07:55:22 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Wed, 01 Jun 2005 22:55:22 -0700 Subject: [Python-checkins] python/dist/src/Tools/compiler astgen.py, 1.10, 1.11 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/compiler In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv367/Tools/compiler Modified Files: astgen.py Log Message: Fix compiler.ast.flatten function so that it works on lists. Index: astgen.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/compiler/astgen.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- astgen.py 12 Sep 2004 03:49:31 -0000 1.10 +++ astgen.py 2 Jun 2005 05:55:20 -0000 1.11 @@ -234,9 +234,9 @@ """ from consts import CO_VARARGS, CO_VARKEYWORDS -def flatten(list): +def flatten(seq): l = [] - for elt in list: + for elt in seq: t = type(elt) if t is tuple or t is list: for elt2 in flatten(elt): @@ -245,8 +245,8 @@ l.append(elt) return l -def flatten_nodes(list): - return [n for n in flatten(list) if isinstance(n, Node)] +def flatten_nodes(seq): + return [n for n in flatten(seq) if isinstance(n, Node)] nodes = {} From antivirusadmin at mediag8way.ph Thu Jun 2 11:16:13 2005 From: antivirusadmin at mediag8way.ph (antivirusadmin@mediag8way.ph) Date: Thu, 02 Jun 2005 17:16:13 +0800 Subject: [Python-checkins] Virus Alert Message-ID: <20050602091613.5E3426307E@mail.mediag8way.ph> The mail message (file: Data.zip) you sent to pcclinic at pcworld.com.ph contains a virus. (on mail.mediag8way.ph) From perky at users.sourceforge.net Thu Jun 2 15:09:32 2005 From: perky at users.sourceforge.net (perky@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:09:32 -0700 Subject: [Python-checkins] python/dist/src/Modules posixmodule.c, 2.335, 2.336 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19602/Modules Modified Files: posixmodule.c Log Message: Patch #1212117: Add optional attribute st_flags to os.stat_result when the member is available on the platform. (Contributed by Diego Petteno) Index: posixmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/posixmodule.c,v retrieving revision 2.335 retrieving revision 2.336 diff -u -d -r2.335 -r2.336 --- posixmodule.c 16 May 2005 02:42:21 -0000 2.335 +++ posixmodule.c 2 Jun 2005 13:09:28 -0000 2.336 @@ -674,8 +674,8 @@ (mode, ino, dev, nlink, uid, gid, size, atime, mtime, ctime)\n\ or via the attributes st_mode, st_ino, st_dev, st_nlink, st_uid, and so on.\n\ \n\ -Posix/windows: If your platform supports st_blksize, st_blocks, or st_rdev,\n\ -they are available as attributes only.\n\ +Posix/windows: If your platform supports st_blksize, st_blocks, st_rdev,\n\ +or st_flags, they are available as attributes only.\n\ \n\ See os.stat for more information."); @@ -703,6 +703,9 @@ #ifdef HAVE_STRUCT_STAT_ST_RDEV {"st_rdev", "device type (if inode device)"}, #endif +#ifdef HAVE_STRUCT_STAT_ST_FLAGS + {"st_flags", "user defined flags for file"}, +#endif {0} }; @@ -724,6 +727,12 @@ #define ST_RDEV_IDX ST_BLOCKS_IDX #endif +#ifdef HAVE_STRUCT_STAT_ST_FLAGS +#define ST_FLAGS_IDX (ST_RDEV_IDX+1) +#else +#define ST_FLAGS_IDX ST_RDEV_IDX +#endif + static PyStructSequence_Desc stat_result_desc = { "stat_result", /* name */ stat_result__doc__, /* doc */ @@ -887,6 +896,10 @@ PyStructSequence_SET_ITEM(v, ST_RDEV_IDX, PyInt_FromLong((long)st.st_rdev)); #endif +#ifdef HAVE_STRUCT_STAT_ST_FLAGS + PyStructSequence_SET_ITEM(v, ST_FLAGS_IDX, + PyInt_FromLong((long)st.st_flags)); +#endif if (PyErr_Occurred()) { Py_DECREF(v); From perky at users.sourceforge.net Thu Jun 2 15:09:33 2005 From: perky at users.sourceforge.net (perky@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:09:33 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.154,1.155 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19602/Doc/lib Modified Files: libos.tex Log Message: Patch #1212117: Add optional attribute st_flags to os.stat_result when the member is available on the platform. (Contributed by Diego Petteno) Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.154 retrieving revision 1.155 diff -u -d -r1.154 -r1.155 --- libos.tex 16 May 2005 02:42:21 -0000 1.154 +++ libos.tex 2 Jun 2005 13:09:30 -0000 1.155 @@ -966,6 +966,7 @@ \member{st_blocks} (number of blocks allocated for file), \member{st_blksize} (filesystem blocksize), \member{st_rdev} (type of device if an inode device). +\member{st_flags} (user defined flags for file). On Mac OS systems, the following attributes may also be available: \member{st_rsize}, From perky at users.sourceforge.net Thu Jun 2 15:10:05 2005 From: perky at users.sourceforge.net (perky@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:10:05 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1294,1.1295 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19602/Misc Modified Files: NEWS Log Message: Patch #1212117: Add optional attribute st_flags to os.stat_result when the member is available on the platform. (Contributed by Diego Petteno) Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1294 retrieving revision 1.1295 diff -u -d -r1.1294 -r1.1295 --- NEWS 1 Jun 2005 11:34:22 -0000 1.1294 +++ NEWS 2 Jun 2005 13:09:26 -0000 1.1295 @@ -88,6 +88,9 @@ Extension Modules ----------------- +- Patch #1212117: os.stat().st_flags is now accessible as a attribute + if available on the platform. + - Patch #1103951: Expose O_SHLOCK and O_EXLOCK in the posix module if available on the platform. From perky at users.sourceforge.net Thu Jun 2 15:10:05 2005 From: perky at users.sourceforge.net (perky@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:10:05 -0700 Subject: [Python-checkins] python/dist/src configure, 1.471, 1.472 configure.in, 1.484, 1.485 pyconfig.h.in, 1.106, 1.107 Message-ID: Update of /cvsroot/python/python/dist/src In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19602 Modified Files: configure configure.in pyconfig.h.in Log Message: Patch #1212117: Add optional attribute st_flags to os.stat_result when the member is available on the platform. (Contributed by Diego Petteno) Index: configure =================================================================== RCS file: /cvsroot/python/python/dist/src/configure,v retrieving revision 1.471 retrieving revision 1.472 diff -u -d -r1.471 -r1.472 --- configure 28 Mar 2005 23:23:46 -0000 1.471 +++ configure 2 Jun 2005 13:08:55 -0000 1.472 @@ -1,5 +1,5 @@ #! /bin/sh -# From configure.in Revision: 1.483 . +# From configure.in Revision: 1.484 . # Guess values for system-dependent variables and create Makefiles. # Generated by GNU Autoconf 2.59 for python 2.5. # @@ -16414,6 +16414,116 @@ fi +echo "$as_me:$LINENO: checking for struct stat.st_flags" >&5 +echo $ECHO_N "checking for struct stat.st_flags... $ECHO_C" >&6 +if test "${ac_cv_member_struct_stat_st_flags+set}" = set; then + echo $ECHO_N "(cached) $ECHO_C" >&6 +else + cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +$ac_includes_default +int +main () +{ +static struct stat ac_aggr; +if (ac_aggr.st_flags) +return 0; + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_cv_member_struct_stat_st_flags=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +cat >conftest.$ac_ext <<_ACEOF +/* confdefs.h. */ +_ACEOF +cat confdefs.h >>conftest.$ac_ext +cat >>conftest.$ac_ext <<_ACEOF +/* end confdefs.h. */ +$ac_includes_default +int +main () +{ +static struct stat ac_aggr; +if (sizeof ac_aggr.st_flags) +return 0; + ; + return 0; +} +_ACEOF +rm -f conftest.$ac_objext +if { (eval echo "$as_me:$LINENO: \"$ac_compile\"") >&5 + (eval $ac_compile) 2>conftest.er1 + ac_status=$? + grep -v '^ *+' conftest.er1 >conftest.err + rm -f conftest.er1 + cat conftest.err >&5 + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); } && + { ac_try='test -z "$ac_c_werror_flag" + || test ! -s conftest.err' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; } && + { ac_try='test -s conftest.$ac_objext' + { (eval echo "$as_me:$LINENO: \"$ac_try\"") >&5 + (eval $ac_try) 2>&5 + ac_status=$? + echo "$as_me:$LINENO: \$? = $ac_status" >&5 + (exit $ac_status); }; }; then + ac_cv_member_struct_stat_st_flags=yes +else + echo "$as_me: failed program was:" >&5 +sed 's/^/| /' conftest.$ac_ext >&5 + +ac_cv_member_struct_stat_st_flags=no +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +fi +rm -f conftest.err conftest.$ac_objext conftest.$ac_ext +fi +echo "$as_me:$LINENO: result: $ac_cv_member_struct_stat_st_flags" >&5 +echo "${ECHO_T}$ac_cv_member_struct_stat_st_flags" >&6 +if test $ac_cv_member_struct_stat_st_flags = yes; then + +cat >>confdefs.h <<_ACEOF +#define HAVE_STRUCT_STAT_ST_FLAGS 1 +_ACEOF + + +fi + echo "$as_me:$LINENO: checking for struct stat.st_blocks" >&5 echo $ECHO_N "checking for struct stat.st_blocks... $ECHO_C" >&6 if test "${ac_cv_member_struct_stat_st_blocks+set}" = set; then Index: configure.in =================================================================== RCS file: /cvsroot/python/python/dist/src/configure.in,v retrieving revision 1.484 retrieving revision 1.485 diff -u -d -r1.484 -r1.485 --- configure.in 28 Mar 2005 23:23:47 -0000 1.484 +++ configure.in 2 Jun 2005 13:09:24 -0000 1.485 @@ -2421,6 +2421,7 @@ AC_STRUCT_TIMEZONE AC_CHECK_MEMBERS([struct stat.st_rdev]) AC_CHECK_MEMBERS([struct stat.st_blksize]) +AC_CHECK_MEMBERS([struct stat.st_flags]) AC_STRUCT_ST_BLOCKS AC_MSG_CHECKING(for time.h that defines altzone) Index: pyconfig.h.in =================================================================== RCS file: /cvsroot/python/python/dist/src/pyconfig.h.in,v retrieving revision 1.106 retrieving revision 1.107 diff -u -d -r1.106 -r1.107 --- pyconfig.h.in 23 Jan 2005 09:27:22 -0000 1.106 +++ pyconfig.h.in 2 Jun 2005 13:09:25 -0000 1.107 @@ -489,6 +489,9 @@ /* Define to 1 if `st_blocks' is member of `struct stat'. */ #undef HAVE_STRUCT_STAT_ST_BLOCKS +/* Define to 1 if `st_flags' is member of `struct stat'. */ +#undef HAVE_STRUCT_STAT_ST_FLAGS + /* Define to 1 if `st_rdev' is member of `struct stat'. */ #undef HAVE_STRUCT_STAT_ST_RDEV From akuchling at users.sourceforge.net Thu Jun 2 15:35:55 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:35:55 -0700 Subject: [Python-checkins] python/dist/src/Lib sre_compile.py,1.57,1.58 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2498/Lib Modified Files: sre_compile.py Log Message: [Bug #1177831] Fix generation of code for GROUPREF_EXISTS. Thanks to Andre Malo for the fix. Index: sre_compile.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/sre_compile.py,v retrieving revision 1.57 retrieving revision 1.58 diff -u -d -r1.57 -r1.58 --- sre_compile.py 28 Feb 2005 19:27:52 -0000 1.57 +++ sre_compile.py 2 Jun 2005 13:35:52 -0000 1.58 @@ -167,7 +167,7 @@ emit(av-1) elif op is GROUPREF_EXISTS: emit(OPCODES[op]) - emit((av[0]-1)*2) + emit(av[0]-1) skipyes = _len(code); emit(0) _compile(code, av[1], flags) if av[2]: From akuchling at users.sourceforge.net Thu Jun 2 15:38:58 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:38:58 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_re.py,1.53,1.54 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3992/Lib/test Modified Files: test_re.py Log Message: [Bug #1177831] Exercise (?(id)yes|no) for a group other than the first one Index: test_re.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_re.py,v retrieving revision 1.53 retrieving revision 1.54 diff -u -d -r1.53 -r1.54 --- test_re.py 12 Sep 2004 03:49:31 -0000 1.53 +++ test_re.py 2 Jun 2005 13:38:45 -0000 1.54 @@ -235,6 +235,16 @@ self.assertEqual(re.match('^(?:(a)|c)((?(1)|d))$', 'a').groups(), ('a', '')) + # Tests for bug #1177831: exercise groups other than the first group + p = re.compile('(?Pa)(?Pb)?((?(g2)c|d))') + self.assertEqual(p.match('abc').groups(), + ('a', 'b', 'c')) + self.assertEqual(p.match('ad').groups(), + ('a', None, 'd')) + self.assertEqual(p.match('abd'), None) + self.assertEqual(p.match('ac'), None) + + def test_re_groupref(self): self.assertEqual(re.match(r'^(\|)?([^()]+)\1$', '|a|').groups(), ('|', 'a')) From akuchling at users.sourceforge.net Thu Jun 2 15:40:14 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:40:14 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_re.py, 1.53, 1.53.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4768/Lib/test Modified Files: Tag: release24-maint test_re.py Log Message: [Bug #1177831] Fix (?(id)yes|no) for a group other than the first one, and add a test case Index: test_re.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_re.py,v retrieving revision 1.53 retrieving revision 1.53.2.1 diff -u -d -r1.53 -r1.53.2.1 --- test_re.py 12 Sep 2004 03:49:31 -0000 1.53 +++ test_re.py 2 Jun 2005 13:40:12 -0000 1.53.2.1 @@ -235,6 +235,16 @@ self.assertEqual(re.match('^(?:(a)|c)((?(1)|d))$', 'a').groups(), ('a', '')) + # Tests for bug #1177831: exercise groups other than the first group + p = re.compile('(?Pa)(?Pb)?((?(g2)c|d))') + self.assertEqual(p.match('abc').groups(), + ('a', 'b', 'c')) + self.assertEqual(p.match('ad').groups(), + ('a', None, 'd')) + self.assertEqual(p.match('abd'), None) + self.assertEqual(p.match('ac'), None) + + def test_re_groupref(self): self.assertEqual(re.match(r'^(\|)?([^()]+)\1$', '|a|').groups(), ('|', 'a')) From akuchling at users.sourceforge.net Thu Jun 2 15:40:14 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:40:14 -0700 Subject: [Python-checkins] python/dist/src/Lib sre_compile.py, 1.56, 1.56.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4768/Lib Modified Files: Tag: release24-maint sre_compile.py Log Message: [Bug #1177831] Fix (?(id)yes|no) for a group other than the first one, and add a test case Index: sre_compile.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/sre_compile.py,v retrieving revision 1.56 retrieving revision 1.56.2.1 diff -u -d -r1.56 -r1.56.2.1 --- sre_compile.py 15 Oct 2004 06:15:08 -0000 1.56 +++ sre_compile.py 2 Jun 2005 13:40:12 -0000 1.56.2.1 @@ -156,7 +156,7 @@ emit(av-1) elif op is GROUPREF_EXISTS: emit(OPCODES[op]) - emit((av[0]-1)*2) + emit(av[0]-1) skipyes = _len(code); emit(0) _compile(code, av[1], flags) if av[2]: From akuchling at users.sourceforge.net Thu Jun 2 15:50:22 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:50:22 -0700 Subject: [Python-checkins] python/dist/src/Doc/ref ref1.tex,1.14,1.15 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ref In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10376 Modified Files: ref1.tex Log Message: [Bug #1193001] Make the notation section use the same productionlist env. as other grammar rules Index: ref1.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ref/ref1.tex,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- ref1.tex 28 Nov 2001 07:26:15 -0000 1.14 +++ ref1.tex 2 Jun 2005 13:50:19 -0000 1.15 @@ -43,10 +43,10 @@ \index{syntax} \index{notation} -\begin{verbatim} -name: lc_letter (lc_letter | "_")* -lc_letter: "a"..."z" -\end{verbatim} +\begin{productionlist} + \production{name}{\token{lc_letter} (\token{lc_letter} | "_")*} + \production{lc_letter}{"a"..."z"} +\end{productionlist} The first line says that a \code{name} is an \code{lc_letter} followed by a sequence of zero or more \code{lc_letter}s and underscores. An @@ -55,7 +55,7 @@ names defined in lexical and grammar rules in this document.) Each rule begins with a name (which is the name defined by the rule) -and a colon. A vertical bar (\code{|}) is used to separate +and \code{::=}. A vertical bar (\code{|}) is used to separate alternatives; it is the least binding operator in this notation. A star (\code{*}) means zero or more repetitions of the preceding item; likewise, a plus (\code{+}) means one or more repetitions, and a From akuchling at users.sourceforge.net Thu Jun 2 15:52:10 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 06:52:10 -0700 Subject: [Python-checkins] python/dist/src/Doc/ref ref1.tex,1.14,1.14.28.1 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ref In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11495 Modified Files: Tag: release24-maint ref1.tex Log Message: [Bug #1193001] Make the notation section use the same productionlist env. as other grammar rules Index: ref1.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ref/ref1.tex,v retrieving revision 1.14 retrieving revision 1.14.28.1 diff -u -d -r1.14 -r1.14.28.1 --- ref1.tex 28 Nov 2001 07:26:15 -0000 1.14 +++ ref1.tex 2 Jun 2005 13:52:07 -0000 1.14.28.1 @@ -43,10 +43,10 @@ \index{syntax} \index{notation} -\begin{verbatim} -name: lc_letter (lc_letter | "_")* -lc_letter: "a"..."z" -\end{verbatim} +\begin{productionlist} + \production{name}{\token{lc_letter} (\token{lc_letter} | "_")*} + \production{lc_letter}{"a"..."z"} +\end{productionlist} The first line says that a \code{name} is an \code{lc_letter} followed by a sequence of zero or more \code{lc_letter}s and underscores. An @@ -55,7 +55,7 @@ names defined in lexical and grammar rules in this document.) Each rule begins with a name (which is the name defined by the rule) -and a colon. A vertical bar (\code{|}) is used to separate +and \code{::=}. A vertical bar (\code{|}) is used to separate alternatives; it is the least binding operator in this notation. A star (\code{*}) means zero or more repetitions of the preceding item; likewise, a plus (\code{+}) means one or more repetitions, and a From gvanrossum at users.sourceforge.net Thu Jun 2 16:56:39 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Thu, 02 Jun 2005 07:56:39 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.19,1.20 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12669 Modified Files: pep-0343.txt Log Message: Raise RuntimeError in close(). Add list of open issues. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.19 retrieving revision 1.20 diff -u -d -r1.19 -r1.20 --- pep-0343.txt 1 Jun 2005 21:01:11 -0000 1.19 +++ pep-0343.txt 2 Jun 2005 14:56:36 -0000 1.20 @@ -306,11 +306,9 @@ except (GeneratorExit, StopIteration): pass else: - raise TypeError("generator ignored GeneratorExit") + raise RuntimeError("generator ignored GeneratorExit") # Other exceptions are not caught - (XXX is TypeError an acceptable exception here?) - New generator method: __del__() g.__del__() is an alias for g.close(). This will be called when @@ -417,6 +415,50 @@ OTOH such mistakes are easily diagnosed. +Open Issues + + Discussion on python-dev has revealed some open issues. I list + them here, with my preferred resolution and its motivation. The + PEP as currently written reflects this preferred resolution. + + 1. What exception should be raised by close() when the generator + yields another value as a response to the GeneratorExit + exception? + + I originally chose TypeError because it represents gross + misbehavior of the generator function, which should be fixed by + changing the code. But the with_template decorator class uses + RuntimeError for similar offenses. Arguably they should all + use the same exception. I'd rather not introduce a new + exception class just for this purpose, since it's not an + exception that I want people to catch: I want it to turn into a + traceback which is seen by the programmer who then fixes the + code. So now I believe they should both raise RuntimeError. + There are some precedents for that: it's raised by the core + Python code in situations where endless recursion is detected, + and for uninitialized objects (and for a variety of + miscellaneous conditions). + + 2. Both the generator close() method and the __exit__() method of + the with_template decorator class catch StopIteration and + consider it equivalent to re-raising the exception passed to + throw(). Is allowing StopIteration right here? + + This is so that a generator doing cleanup depending on the + exception thrown (like the transactional() example below) can + *catch* the exception thrown if it wants to and doesn't have to + worry about re-raising it. I find this more convenient for the + generator writer. Against this was brought in that the + generator *appears* to suppress an exception that it cannot + suppress: the transactional() example would be more clear + according to this view if it re-raised the original exception + after the call to db.rollback(). I personally would find the + requirement to re-raise the exception an annoyance in a + generator used as a with-template, since all the code after + yield is used for is cleanup, and it is invoked from a + finally-clause (the one implicit in the with-statement) which + re-raises the original exception anyway. + Examples (Note: several of these examples contain "yield None". If PEP 342 From gvanrossum at users.sourceforge.net Thu Jun 2 17:13:57 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Thu, 02 Jun 2005 08:13:57 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.20,1.21 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22249 Modified Files: pep-0343.txt Log Message: Include suggestions from Nick Coghlan. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.20 retrieving revision 1.21 diff -u -d -r1.20 -r1.21 --- pep-0343.txt 2 Jun 2005 14:56:36 -0000 1.20 +++ pep-0343.txt 2 Jun 2005 15:13:55 -0000 1.21 @@ -413,7 +413,9 @@ which does not do what one might think (f is closed before BLOCK2 is entered). - OTOH such mistakes are easily diagnosed. + OTOH such mistakes are easily diagnosed; for example, the + with_template decorator above raises RuntimeError when the second + with-statement calls f.__enter__() again. Open Issues @@ -642,6 +644,29 @@ methods to the decimal.Context class so that this example can be simplified to "with decimal.getcontext() as ctx: ...".) + 10. A generic "object-closing" template: + + @with_template + def closing(obj): + try: + yield obj + finally: + obj.close() + + This can be used to deterministically close anything with a + close method, be it file, generator, or something else: + + # emulate opening(): + with closing(open("argument.txt")) as contradiction: + for line in contradiction: + print line + + # deterministically finalize a generator: + with closing(some_gen()) as data: + for datum in data: + process(datum) + + References [1] http://blogs.msdn.com/oldnewthing/archive/2005/01/06/347666.aspx From akuchling at users.sourceforge.net Thu Jun 2 18:59:21 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 09:59:21 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libthread.tex, 1.30, 1.31 libthreading.tex, 1.20, 1.21 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16432 Modified Files: libthread.tex libthreading.tex Log Message: [Bug #1209880] Describe only the True/False return values from lock.acquire() Index: libthread.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libthread.tex,v retrieving revision 1.30 retrieving revision 1.31 diff -u -d -r1.30 -r1.31 --- libthread.tex 9 Jul 2003 12:41:55 -0000 1.30 +++ libthread.tex 2 Jun 2005 16:59:18 -0000 1.31 @@ -81,11 +81,11 @@ Without the optional argument, this method acquires the lock unconditionally, if necessary waiting until it is released by another thread (only one thread at a time can acquire a lock --- that's their -reason for existence), and returns \code{None}. If the integer +reason for existence). If the integer \var{waitflag} argument is present, the action depends on its value: if it is zero, the lock is only acquired if it can be acquired immediately without waiting, while if it is nonzero, the lock is -acquired unconditionally as before. If an argument is present, the +acquired unconditionally as before. The return value is \code{True} if the lock is acquired successfully, \code{False} if not. \end{methoddesc} Index: libthreading.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libthreading.tex,v retrieving revision 1.20 retrieving revision 1.21 diff -u -d -r1.20 -r1.21 --- libthreading.tex 17 Jul 2004 13:35:43 -0000 1.20 +++ libthreading.tex 2 Jun 2005 16:59:18 -0000 1.21 @@ -167,8 +167,7 @@ Acquire a lock, blocking or non-blocking. When invoked without arguments, block until the lock is -unlocked, then set it to locked, and return. There is no -return value in this case. +unlocked, then set it to locked, and return true. When invoked with the \var{blocking} argument set to true, do the same thing as when called without arguments, and return true. From akuchling at users.sourceforge.net Thu Jun 2 19:01:12 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 10:01:12 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libthread.tex, 1.30, 1.30.12.1 libthreading.tex, 1.20, 1.20.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17391 Modified Files: Tag: release24-maint libthread.tex libthreading.tex Log Message: [Bug #1209880] Describe only the True/False return values from lock.acquire() Index: libthread.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libthread.tex,v retrieving revision 1.30 retrieving revision 1.30.12.1 diff -u -d -r1.30 -r1.30.12.1 --- libthread.tex 9 Jul 2003 12:41:55 -0000 1.30 +++ libthread.tex 2 Jun 2005 17:01:08 -0000 1.30.12.1 @@ -81,11 +81,11 @@ Without the optional argument, this method acquires the lock unconditionally, if necessary waiting until it is released by another thread (only one thread at a time can acquire a lock --- that's their -reason for existence), and returns \code{None}. If the integer +reason for existence). If the integer \var{waitflag} argument is present, the action depends on its value: if it is zero, the lock is only acquired if it can be acquired immediately without waiting, while if it is nonzero, the lock is -acquired unconditionally as before. If an argument is present, the +acquired unconditionally as before. The return value is \code{True} if the lock is acquired successfully, \code{False} if not. \end{methoddesc} Index: libthreading.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libthreading.tex,v retrieving revision 1.20 retrieving revision 1.20.4.1 diff -u -d -r1.20 -r1.20.4.1 --- libthreading.tex 17 Jul 2004 13:35:43 -0000 1.20 +++ libthreading.tex 2 Jun 2005 17:01:08 -0000 1.20.4.1 @@ -167,8 +167,7 @@ Acquire a lock, blocking or non-blocking. When invoked without arguments, block until the lock is -unlocked, then set it to locked, and return. There is no -return value in this case. +unlocked, then set it to locked, and return true. When invoked with the \var{blocking} argument set to true, do the same thing as when called without arguments, and return true. From akuchling at users.sourceforge.net Thu Jun 2 19:07:13 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 02 Jun 2005 10:07:13 -0700 Subject: [Python-checkins] python/dist/src/Modules threadmodule.c, 2.60, 2.61 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19999 Modified Files: threadmodule.c Log Message: Delete some vestigial code; execution will never reach the 'if' statement if args is NULL Index: threadmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/threadmodule.c,v retrieving revision 2.60 retrieving revision 2.61 diff -u -d -r2.60 -r2.61 --- threadmodule.c 20 Feb 2005 03:02:16 -0000 2.60 +++ threadmodule.c 2 Jun 2005 17:07:11 -0000 2.61 @@ -63,12 +63,7 @@ i = PyThread_acquire_lock(self->lock_lock, i); Py_END_ALLOW_THREADS - if (args == NULL) { - Py_INCREF(Py_None); - return Py_None; - } - else - return PyBool_FromLong((long)i); + return PyBool_FromLong((long)i); } PyDoc_STRVAR(acquire_doc, From python at rcn.com Thu Jun 2 21:34:17 2005 From: python at rcn.com (Raymond Hettinger) Date: Thu, 2 Jun 2005 15:34:17 -0400 Subject: [Python-checkins] python/dist/src/Lib sre_compile.py,1.57,1.58 In-Reply-To: Message-ID: <002501c567aa$11428300$3bb02c81@oemcomputer> > Index: sre_compile.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/sre_compile.py,v > retrieving revision 1.57 > retrieving revision 1.58 > diff -u -d -r1.57 -r1.58 > --- sre_compile.py 28 Feb 2005 19:27:52 -0000 1.57 > +++ sre_compile.py 2 Jun 2005 13:35:52 -0000 1.58 > @@ -167,7 +167,7 @@ > emit(av-1) > elif op is GROUPREF_EXISTS: > emit(OPCODES[op]) > - emit((av[0]-1)*2) > + emit(av[0]-1) > skipyes = _len(code); emit(0) > _compile(code, av[1], flags) > if av[2]: The times two operation also occurs twice in nearby code. Are those also incorrect? Raymond Hettinger From greg at users.sourceforge.net Fri Jun 3 09:03:09 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Fri, 03 Jun 2005 00:03:09 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb dbobj.py,1.9,1.10 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32169/bsddb Modified Files: dbobj.py Log Message: pybsddb 4.3.1, adds support for DB.set_bt_compare database btree comparison functions written in python. contributed by Index: dbobj.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/dbobj.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- dbobj.py 28 Jun 2004 04:06:48 -0000 1.9 +++ dbobj.py 3 Jun 2005 07:03:06 -0000 1.10 @@ -164,6 +164,8 @@ return apply(self._cobj.rename, args, kwargs) def set_bt_minkey(self, *args, **kwargs): return apply(self._cobj.set_bt_minkey, args, kwargs) + def set_bt_compare(self, *args, **kwargs): + return apply(self._cobj.set_bt_compare, args, kwargs) def set_cachesize(self, *args, **kwargs): return apply(self._cobj.set_cachesize, args, kwargs) def set_flags(self, *args, **kwargs): From greg at users.sourceforge.net Fri Jun 3 09:03:09 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Fri, 03 Jun 2005 00:03:09 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_compare.py, NONE, 1.1 test_all.py, 1.4, 1.5 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32169/bsddb/test Modified Files: test_all.py Added Files: test_compare.py Log Message: pybsddb 4.3.1, adds support for DB.set_bt_compare database btree comparison functions written in python. contributed by --- NEW FILE: test_compare.py --- """ TestCases for python DB Btree key comparison function. """ import sys, os import test_all import unittest from bsddb3 import db def lexical_cmp (db, left, right): return cmp (left, right) def lowercase_cmp(db, left, right): return cmp (left.lower(), right.lower()) def make_reverse_comparator (cmp): def reverse (db, left, right, delegate=cmp): return - delegate (db, left, right) return reverse _expected_lexical_test_data = ['', 'CCCP', 'a', 'aaa', 'b', 'c', 'cccce', 'ccccf'] _expected_lowercase_test_data = ['', 'a', 'aaa', 'b', 'c', 'CC', 'cccce', 'ccccf', 'CCCP'] class ComparatorTests (unittest.TestCase): def comparator_test_helper (self, comparator, expected_data): data = expected_data[:] data.sort (lambda l, r, cmp=comparator: cmp (None, l, r)) self.failUnless (data == expected_data, "comparator `%s' is not right: %s vs. %s" % (comparator, expected_data, data)) def test_lexical_comparator (self): self.comparator_test_helper (lexical_cmp, _expected_lexical_test_data) def test_reverse_lexical_comparator (self): rev = _expected_lexical_test_data[:] rev.reverse () self.comparator_test_helper (make_reverse_comparator (lexical_cmp), rev) def test_lowercase_comparator (self): self.comparator_test_helper (lowercase_cmp, _expected_lowercase_test_data) class AbstractBtreeKeyCompareTestCase (unittest.TestCase): env = None db = None def setUp (self): self.filename = self.__class__.__name__ + '.db' homeDir = os.path.join (os.path.dirname (sys.argv[0]), 'db_home') self.homeDir = homeDir try: os.mkdir (homeDir) except os.error: pass env = db.DBEnv () env.open (homeDir, db.DB_CREATE | db.DB_INIT_MPOOL | db.DB_INIT_LOCK | db.DB_THREAD) self.env = env def tearDown (self): self.closeDB () if self.env is not None: self.env.close () self.env = None import glob map (os.remove, glob.glob (os.path.join (self.homeDir, '*'))) def addDataToDB (self, data): i = 0 for item in data: self.db.put (item, str (i)) i = i + 1 def createDB (self, key_comparator): self.db = db.DB (self.env) self.setupDB (key_comparator) self.db.open (self.filename, "test", db.DB_BTREE, db.DB_CREATE) def setupDB (self, key_comparator): self.db.set_bt_compare (key_comparator) def closeDB (self): if self.db is not None: self.db.close () self.db = None def startTest (self): pass def finishTest (self, expected = None): if expected is not None: self.check_results (expected) self.closeDB () def check_results (self, expected): curs = self.db.cursor () try: index = 0 rec = curs.first () while rec: key, ignore = rec self.failUnless (index < len (expected), "to many values returned from cursor") self.failUnless (expected[index] == key, "expected value `%s' at %d but got `%s'" % (expected[index], index, key)) index = index + 1 rec = curs.next () self.failUnless (index == len (expected), "not enough values returned from cursor") finally: curs.close () class BtreeKeyCompareTestCase (AbstractBtreeKeyCompareTestCase): def runCompareTest (self, comparator, data): self.startTest () self.createDB (comparator) self.addDataToDB (data) self.finishTest (data) def test_lexical_ordering (self): self.runCompareTest (lexical_cmp, _expected_lexical_test_data) def test_reverse_lexical_ordering (self): expected_rev_data = _expected_lexical_test_data[:] expected_rev_data.reverse () self.runCompareTest (make_reverse_comparator (lexical_cmp), expected_rev_data) def test_compare_function_useless (self): self.startTest () def socialist_comparator (db, l, r): return 0 self.createDB (socialist_comparator) self.addDataToDB (['b', 'a', 'd']) # all things being equal the first key will be the only key # in the database... (with the last key's value fwiw) self.finishTest (['b']) class BtreeExceptionsTestCase (AbstractBtreeKeyCompareTestCase): def test_raises_non_callable (self): self.startTest () self.assertRaises (TypeError, self.createDB, 'abc') self.assertRaises (TypeError, self.createDB, None) self.finishTest () def test_set_bt_compare_with_function (self): self.startTest () self.createDB (lexical_cmp) self.finishTest () def check_results (self, results): pass def test_compare_function_incorrect (self): self.startTest () def bad_comparator (db, l, r): return 1 # verify that set_bt_compare checks that comparator(db, '', '') == 0 self.assertRaises (TypeError, self.createDB, bad_comparator) self.finishTest () def test_compare_function_exception (self): self.startTest () def bad_comparator (db, l, r): if l == r: # pass the set_bt_compare test return 0 raise RuntimeError, "i'm a naughty comparison function" self.createDB (bad_comparator) print "\n*** this test should print 2 uncatchable tracebacks ***" self.addDataToDB (['a', 'b', 'c']) # this should raise, but... self.finishTest () def test_compare_function_bad_return (self): self.startTest () def bad_comparator (db, l, r): if l == r: # pass the set_bt_compare test return 0 return l self.createDB (bad_comparator) print "\n*** this test should print 2 errors about returning an int ***" self.addDataToDB (['a', 'b', 'c']) # this should raise, but... self.finishTest () def test_cannot_assign_twice (self): def my_compare (db, a, b): return 0 self.startTest () self.createDB (my_compare) try: self.db.set_bt_compare (my_compare) assert False, "this set should fail" except RuntimeError, msg: pass def test_suite (): res = unittest.TestSuite () res.addTest (unittest.makeSuite (ComparatorTests)) if db.version () >= (3, 3, 11): res.addTest (unittest.makeSuite (BtreeExceptionsTestCase)) res.addTest (unittest.makeSuite (BtreeKeyCompareTestCase)) return res if __name__ == '__main__': unittest.main (defaultTest = 'suite') Index: test_all.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_all.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- test_all.py 16 Mar 2004 07:07:06 -0000 1.4 +++ test_all.py 3 Jun 2005 07:03:07 -0000 1.5 @@ -50,6 +50,7 @@ 'test_associate', 'test_basics', 'test_compat', + 'test_compare', 'test_dbobj', 'test_dbshelve', 'test_dbtables', From greg at users.sourceforge.net Fri Jun 3 09:03:11 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Fri, 03 Jun 2005 00:03:11 -0700 Subject: [Python-checkins] python/dist/src/Modules _bsddb.c,1.40,1.41 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32169/extsrc Modified Files: _bsddb.c Log Message: pybsddb 4.3.1, adds support for DB.set_bt_compare database btree comparison functions written in python. contributed by Index: _bsddb.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_bsddb.c,v retrieving revision 1.40 retrieving revision 1.41 diff -u -d -r1.40 -r1.41 --- _bsddb.c 16 Dec 2004 09:47:28 -0000 1.40 +++ _bsddb.c 3 Jun 2005 07:03:07 -0000 1.41 @@ -97,7 +97,7 @@ #error "eek! DBVER can't handle minor versions > 9" #endif -#define PY_BSDDB_VERSION "4.3.0" +#define PY_BSDDB_VERSION "4.3.1" static char *rcs_id = "$Id$"; @@ -244,6 +244,7 @@ struct behaviourFlags moduleFlags; #if (DBVER >= 33) PyObject* associateCallback; + PyObject* btCompareCallback; int primaryDBType; #endif #ifdef HAVE_WEAKREF @@ -741,6 +742,7 @@ self->myenvobj = NULL; #if (DBVER >= 33) self->associateCallback = NULL; + self->btCompareCallback = NULL; self->primaryDBType = 0; #endif #ifdef HAVE_WEAKREF @@ -815,6 +817,10 @@ Py_DECREF(self->associateCallback); self->associateCallback = NULL; } + if (self->btCompareCallback != NULL) { + Py_DECREF(self->btCompareCallback); + self->btCompareCallback = NULL; + } #endif PyObject_Del(self); } @@ -1959,6 +1965,161 @@ RETURN_NONE(); } +static int +_default_cmp (const DBT *leftKey, + const DBT *rightKey) +{ + int res; + int lsize = leftKey->size, rsize = rightKey->size; + + res = memcmp (leftKey->data, rightKey->data, + lsize < rsize ? lsize : rsize); + + if (res == 0) { + if (lsize < rsize) { + res = -1; + } + else if (lsize > rsize) { + res = 1; + } + } + return res; +} + +static int +_db_compareCallback (DB* db, + const DBT *leftKey, + const DBT *rightKey) +{ + int res = 0; + PyObject *args; + PyObject *result; + PyObject *leftObject; + PyObject *rightObject; + DBObject *self = (DBObject *) db->app_private; + + if (self == NULL || self->btCompareCallback == NULL) { + MYDB_BEGIN_BLOCK_THREADS; + PyErr_SetString (PyExc_TypeError, + (self == 0 + ? "DB_bt_compare db is NULL." + : "DB_bt_compare callback is NULL.")); + /* we're in a callback within the DB code, we can't raise */ + PyErr_Print (); + res = _default_cmp (leftKey, rightKey); + MYDB_END_BLOCK_THREADS; + } + else { + MYDB_BEGIN_BLOCK_THREADS; + + leftObject = PyString_FromStringAndSize (leftKey->data, leftKey->size); + rightObject = PyString_FromStringAndSize (rightKey->data, rightKey->size); + + args = PyTuple_New (3); + Py_INCREF (self); + PyTuple_SET_ITEM (args, 0, (PyObject *) self); + PyTuple_SET_ITEM (args, 1, leftObject); /* steals reference */ + PyTuple_SET_ITEM (args, 2, rightObject); /* steals reference */ + + result = PyEval_CallObject (self->btCompareCallback, args); + if (result == 0) { + /* we're in a callback within the DB code, we can't raise */ + PyErr_Print (); // XXX-gps or can we? either way the DB is screwed + res = _default_cmp (leftKey, rightKey); + } + else if (PyInt_Check (result)) { + res = PyInt_AsLong (result); + } + else { + PyErr_SetString (PyExc_TypeError, + "DB_bt_compare callback MUST return an int."); + /* we're in a callback within the DB code, we can't raise */ + PyErr_Print (); + res = _default_cmp (leftKey, rightKey); + } + + Py_DECREF (args); + Py_XDECREF (result); + + MYDB_END_BLOCK_THREADS; + } + return res; +} + +static PyObject* +DB_set_bt_compare (DBObject* self, PyObject* args) +{ + int err; + PyObject *comparator; + PyObject *tuple, *emptyStr, *result; + + if (!PyArg_ParseTuple(args,"O:set_bt_compare", &comparator )) + return NULL; + + CHECK_DB_NOT_CLOSED (self); + + if (! PyCallable_Check (comparator)) { + makeTypeError ("Callable", comparator); + return NULL; + } + + /* + * Perform a test call of the comparator function with two empty + * string objects here. verify that it returns an int (0). + * err if not. + */ + tuple = PyTuple_New (3); + Py_INCREF (self); + PyTuple_SET_ITEM (tuple, 0, (PyObject *) self); + + emptyStr = PyString_FromStringAndSize (NULL, 0); + Py_INCREF(emptyStr); + PyTuple_SET_ITEM (tuple, 1, emptyStr); + PyTuple_SET_ITEM (tuple, 2, emptyStr); /* steals reference */ + result = PyEval_CallObject (comparator, tuple); + Py_DECREF (tuple); + if (result == 0 || !PyInt_Check(result)) { + PyErr_SetString (PyExc_TypeError, + "callback MUST return an int"); + return NULL; + } + else if (PyInt_AsLong(result) != 0) { + PyErr_SetString (PyExc_TypeError, + "callback failed to return 0 on two empty strings"); + return NULL; + } + + /* We don't accept multiple set_bt_compare operations, in order to + * simplify the code. This would have no real use, as one cannot + * change the function once the db is opened anyway */ + if (self->btCompareCallback != NULL) { + PyErr_SetString (PyExc_RuntimeError, "set_bt_compare () cannot be called more than once"); + return NULL; + } + + Py_INCREF (comparator); + self->btCompareCallback = comparator; + + /* This is to workaround a problem with un-initialized threads (see + comment in DB_associate) */ +#ifdef WITH_THREAD + PyEval_InitThreads(); +#endif + + err = self->db->set_bt_compare (self->db, + (comparator != NULL ? + _db_compareCallback : NULL)); + + if (err) { + /* restore the old state in case of error */ + Py_DECREF (comparator); + self->btCompareCallback = NULL; + } + + RETURN_IF_ERR (); + RETURN_NONE (); +} + static PyObject* DB_set_cachesize(DBObject* self, PyObject* args) @@ -4400,6 +4561,7 @@ {"remove", (PyCFunction)DB_remove, METH_VARARGS|METH_KEYWORDS}, {"rename", (PyCFunction)DB_rename, METH_VARARGS}, {"set_bt_minkey", (PyCFunction)DB_set_bt_minkey, METH_VARARGS}, + {"set_bt_compare", (PyCFunction)DB_set_bt_compare, METH_VARARGS}, {"set_cachesize", (PyCFunction)DB_set_cachesize, METH_VARARGS}, #if (DBVER >= 41) {"set_encrypt", (PyCFunction)DB_set_encrypt, METH_VARARGS|METH_KEYWORDS}, From mwh at users.sourceforge.net Fri Jun 3 11:32:59 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 03 Jun 2005 02:32:59 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.21,1.22 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16166 Modified Files: pep-0343.txt Log Message: Incredibly minor clarification. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- pep-0343.txt 2 Jun 2005 15:13:55 -0000 1.21 +++ pep-0343.txt 3 Jun 2005 09:32:57 -0000 1.22 @@ -13,11 +13,12 @@ After a lot of discussion about PEP 340 and alternatives, I decided to withdraw PEP 340 and proposed a slight variant on PEP - 310. After more discussion, I have added back a mechanism - for raising an exception in a suspended generator using a throw() + 310. After more discussion, I have added back a mechanism for + raising an exception in a suspended generator using a throw() method, and a close() method which throws a new GeneratorExit - exception; these additions were first proposed in [2] and - universally approved of. I'm also changing the keyword to 'with'. + exception; these additions were first proposed on python-dev in + [2] and universally approved of. I'm also changing the keyword to + 'with'. If this PEP is approved, the following PEPs will be rejected due to overlap: From rhettinger at users.sourceforge.net Fri Jun 3 13:15:08 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 03 Jun 2005 04:15:08 -0700 Subject: [Python-checkins] python/dist/src/Misc developers.txt,1.12,1.13 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1854 Modified Files: developers.txt Log Message: Update permissions for Birkenfeld. Index: developers.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/developers.txt,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- developers.txt 28 May 2005 23:20:09 -0000 1.12 +++ developers.txt 3 Jun 2005 11:15:03 -0000 1.13 @@ -20,7 +20,8 @@ - Reinhold Birkenfeld was given SF tracker permissions on 28 May 2005 by RDH. Since the beginning of 2005, he has been active in discussions on python-dev and has submitted a dozen patch reviews. The permissions - add the ability to change tracker status and to attach patches. + add the ability to change tracker status and to attach patches. On + 3 June 2005, this was expanded by RDH to include checkin permissions. - Terry Reedy was given SF tracker permissions on 7 Apr 2005 by RDH. From mwh at users.sourceforge.net Fri Jun 3 15:56:01 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 03 Jun 2005 06:56:01 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_re.py,1.54,1.55 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25069 Modified Files: test_re.py Log Message: M-x untabify Index: test_re.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_re.py,v retrieving revision 1.54 retrieving revision 1.55 diff -u -d -r1.54 -r1.55 --- test_re.py 2 Jun 2005 13:38:45 -0000 1.54 +++ test_re.py 3 Jun 2005 13:55:58 -0000 1.55 @@ -235,15 +235,15 @@ self.assertEqual(re.match('^(?:(a)|c)((?(1)|d))$', 'a').groups(), ('a', '')) - # Tests for bug #1177831: exercise groups other than the first group - p = re.compile('(?Pa)(?Pb)?((?(g2)c|d))') - self.assertEqual(p.match('abc').groups(), - ('a', 'b', 'c')) - self.assertEqual(p.match('ad').groups(), - ('a', None, 'd')) - self.assertEqual(p.match('abd'), None) - self.assertEqual(p.match('ac'), None) - + # Tests for bug #1177831: exercise groups other than the first group + p = re.compile('(?Pa)(?Pb)?((?(g2)c|d))') + self.assertEqual(p.match('abc').groups(), + ('a', 'b', 'c')) + self.assertEqual(p.match('ad').groups(), + ('a', None, 'd')) + self.assertEqual(p.match('abd'), None) + self.assertEqual(p.match('ac'), None) + def test_re_groupref(self): self.assertEqual(re.match(r'^(\|)?([^()]+)\1$', '|a|').groups(), From anthonybaxter at users.sourceforge.net Fri Jun 3 16:02:51 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:02:51 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_re.py, 1.53.2.1, 1.53.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28144 Modified Files: Tag: release24-maint test_re.py Log Message: fix exceptions.TabError. _Please_ run test suite before checking in. Index: test_re.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_re.py,v retrieving revision 1.53.2.1 retrieving revision 1.53.2.2 diff -u -d -r1.53.2.1 -r1.53.2.2 --- test_re.py 2 Jun 2005 13:40:12 -0000 1.53.2.1 +++ test_re.py 3 Jun 2005 14:02:48 -0000 1.53.2.2 @@ -235,15 +235,15 @@ self.assertEqual(re.match('^(?:(a)|c)((?(1)|d))$', 'a').groups(), ('a', '')) - # Tests for bug #1177831: exercise groups other than the first group - p = re.compile('(?Pa)(?Pb)?((?(g2)c|d))') - self.assertEqual(p.match('abc').groups(), - ('a', 'b', 'c')) - self.assertEqual(p.match('ad').groups(), - ('a', None, 'd')) - self.assertEqual(p.match('abd'), None) - self.assertEqual(p.match('ac'), None) - + # Tests for bug #1177831: exercise groups other than the first group + p = re.compile('(?Pa)(?Pb)?((?(g2)c|d))') + self.assertEqual(p.match('abc').groups(), + ('a', 'b', 'c')) + self.assertEqual(p.match('ad').groups(), + ('a', None, 'd')) + self.assertEqual(p.match('abd'), None) + self.assertEqual(p.match('ac'), None) + def test_re_groupref(self): self.assertEqual(re.match(r'^(\|)?([^()]+)\1$', '|a|').groups(), From anthonybaxter at users.sourceforge.net Fri Jun 3 16:10:51 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:10:51 -0700 Subject: [Python-checkins] python/dist/src/Lib/idlelib configHandler.py, 1.35.4.1, 1.35.4.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/idlelib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32335/idlelib Modified Files: Tag: release24-maint configHandler.py Log Message: whitespace normalisation Index: configHandler.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/configHandler.py,v retrieving revision 1.35.4.1 retrieving revision 1.35.4.2 diff -u -d -r1.35.4.1 -r1.35.4.2 --- configHandler.py 19 Jan 2005 17:12:49 -0000 1.35.4.1 +++ configHandler.py 3 Jun 2005 14:10:48 -0000 1.35.4.2 @@ -193,7 +193,7 @@ """ Creates (if required) and returns a filesystem directory for storing user config files. - + """ cfgDir = '.idlerc' userDir = os.path.expanduser('~') From anthonybaxter at users.sourceforge.net Fri Jun 3 16:10:52 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:10:52 -0700 Subject: [Python-checkins] python/dist/src/Lib/plat-mac bundlebuilder.py, 1.36.10.1, 1.36.10.2 pimp.py, 1.35.4.3, 1.35.4.4 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/plat-mac In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32335/plat-mac Modified Files: Tag: release24-maint bundlebuilder.py pimp.py Log Message: whitespace normalisation Index: bundlebuilder.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/plat-mac/bundlebuilder.py,v retrieving revision 1.36.10.1 retrieving revision 1.36.10.2 diff -u -d -r1.36.10.1 -r1.36.10.2 --- bundlebuilder.py 31 Dec 2004 11:23:20 -0000 1.36.10.1 +++ bundlebuilder.py 3 Jun 2005 14:10:49 -0000 1.36.10.2 @@ -110,7 +110,7 @@ # Verbosity level. verbosity = 1 - + # Destination root directory destroot = "" Index: pimp.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/plat-mac/pimp.py,v retrieving revision 1.35.4.3 retrieving revision 1.35.4.4 diff -u -d -r1.35.4.3 -r1.35.4.4 --- pimp.py 7 Jan 2005 13:48:49 -0000 1.35.4.3 +++ pimp.py 3 Jun 2005 14:10:49 -0000 1.35.4.4 @@ -907,10 +907,10 @@ def installPackageOnly(self, output=None): """Install a single source package. - + If output is given it should be a file-like object and it will receive a log of what happened.""" - + if self._dict.has_key('Post-install-command'): return "%s: Installer package cannot have Post-install-command" % self.fullname() @@ -918,7 +918,7 @@ if _cmd(output, '/tmp', self._dict['Pre-install-command']): return "pre-install %s: running \"%s\" failed" % \ (self.fullname(), self._dict['Pre-install-command']) - + self.beforeInstall() installcmd = self._dict.get('Install-command') @@ -926,7 +926,7 @@ if '%' in installcmd: installcmd = installcmd % self.archiveFilename else: - installcmd = 'open \"%s\"' % self.archiveFilename + installcmd = 'open \"%s\"' % self.archiveFilename if _cmd(output, "/tmp", installcmd): return '%s: install command failed (use verbose for details)' % self.fullname() return '%s: downloaded and opened. Install manually and restart Package Manager' % self.archiveFilename From anthonybaxter at users.sourceforge.net Fri Jun 3 16:10:54 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:10:54 -0700 Subject: [Python-checkins] python/dist/src/Lib/test regrtest.py, 1.165.2.1, 1.165.2.2 test_subprocess.py, 1.15.2.2, 1.15.2.3 test_zlib.py, 1.26.4.1, 1.26.4.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32335/test Modified Files: Tag: release24-maint regrtest.py test_subprocess.py test_zlib.py Log Message: whitespace normalisation Index: regrtest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/regrtest.py,v retrieving revision 1.165.2.1 retrieving revision 1.165.2.2 diff -u -d -r1.165.2.1 -r1.165.2.2 --- regrtest.py 3 Mar 2005 20:51:32 -0000 1.165.2.1 +++ regrtest.py 3 Jun 2005 14:10:49 -0000 1.165.2.2 @@ -93,7 +93,7 @@ in the standard library and test suite. This takes a long time. - subprocess Run all tests for the subprocess module. + subprocess Run all tests for the subprocess module. To enable all resources except one, use '-uall,-'. For example, to run all the tests except for the bsddb tests, give the Index: test_subprocess.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_subprocess.py,v retrieving revision 1.15.2.2 retrieving revision 1.15.2.3 diff -u -d -r1.15.2.2 -r1.15.2.3 --- test_subprocess.py 3 Mar 2005 20:51:32 -0000 1.15.2.2 +++ test_subprocess.py 3 Jun 2005 14:10:50 -0000 1.15.2.3 @@ -346,7 +346,7 @@ if test_support.is_resource_enabled("subprocess") and not mswindows: max_handles = 1026 # too much for most UNIX systems else: - max_handles = 65 + max_handles = 65 for i in range(max_handles): p = subprocess.Popen([sys.executable, "-c", "import sys;sys.stdout.write(sys.stdin.read())"], Index: test_zlib.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_zlib.py,v retrieving revision 1.26.4.1 retrieving revision 1.26.4.2 diff -u -d -r1.26.4.1 -r1.26.4.2 --- test_zlib.py 28 Dec 2004 20:12:29 -0000 1.26.4.1 +++ test_zlib.py 3 Jun 2005 14:10:50 -0000 1.26.4.2 @@ -299,7 +299,7 @@ self.failUnless(co.flush()) # Returns a zlib header dco = zlib.decompressobj() self.assertEqual(dco.flush(), "") # Returns nothing - + def genblock(seed, length, step=1024, generator=random): """length-byte stream of random data from a seed (in step-byte blocks).""" From anthonybaxter at users.sourceforge.net Fri Jun 3 16:12:38 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:12:38 -0700 Subject: [Python-checkins] python/dist/src/Objects typeobject.c, 2.266, 2.267 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1043/Objects Modified Files: typeobject.c Log Message: fix object.__divmod__.__doc__ backport candidate Index: typeobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/typeobject.c,v retrieving revision 2.266 retrieving revision 2.267 diff -u -d -r2.266 -r2.267 --- typeobject.c 15 May 2005 15:32:08 -0000 2.266 +++ typeobject.c 3 Jun 2005 14:12:21 -0000 2.267 @@ -4908,6 +4908,12 @@ #define RBINSLOT(NAME, SLOT, FUNCTION, DOC) \ ETSLOT(NAME, as_number.SLOT, FUNCTION, wrap_binaryfunc_r, \ "x." NAME "(y) <==> y" DOC "x") +#define BINSLOTNOTINFIX(NAME, SLOT, FUNCTION, DOC) \ + ETSLOT(NAME, as_number.SLOT, FUNCTION, wrap_binaryfunc_l, \ + "x." NAME "(y) <==> " DOC) +#define RBINSLOTNOTINFIX(NAME, SLOT, FUNCTION, DOC) \ + ETSLOT(NAME, as_number.SLOT, FUNCTION, wrap_binaryfunc_r, \ + "x." NAME "(y) <==> " DOC) static slotdef slotdefs[] = { SQSLOT("__len__", sq_length, slot_sq_length, wrap_inquiry, @@ -4976,9 +4982,9 @@ "%"), RBINSLOT("__rmod__", nb_remainder, slot_nb_remainder, "%"), - BINSLOT("__divmod__", nb_divmod, slot_nb_divmod, + BINSLOTNOTINFIX("__divmod__", nb_divmod, slot_nb_divmod, "divmod(x, y)"), - RBINSLOT("__rdivmod__", nb_divmod, slot_nb_divmod, + RBINSLOTNOTINFIX("__rdivmod__", nb_divmod, slot_nb_divmod, "divmod(y, x)"), NBSLOT("__pow__", nb_power, slot_nb_power, wrap_ternaryfunc, "x.__pow__(y[, z]) <==> pow(x, y[, z])"), From anthonybaxter at users.sourceforge.net Fri Jun 3 16:13:50 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:13:50 -0700 Subject: [Python-checkins] python/dist/src/Objects typeobject.c, 2.264.2.2, 2.264.2.3 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1610/Objects Modified Files: Tag: release24-maint typeobject.c Log Message: fix int.__divmod__.__doc__ Index: typeobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/typeobject.c,v retrieving revision 2.264.2.2 retrieving revision 2.264.2.3 diff -u -d -r2.264.2.2 -r2.264.2.3 --- typeobject.c 4 Mar 2005 04:47:03 -0000 2.264.2.2 +++ typeobject.c 3 Jun 2005 14:13:47 -0000 2.264.2.3 @@ -4909,6 +4909,12 @@ #define RBINSLOT(NAME, SLOT, FUNCTION, DOC) \ ETSLOT(NAME, as_number.SLOT, FUNCTION, wrap_binaryfunc_r, \ "x." NAME "(y) <==> y" DOC "x") +#define BINSLOTNOTINFIX(NAME, SLOT, FUNCTION, DOC) \ + ETSLOT(NAME, as_number.SLOT, FUNCTION, wrap_binaryfunc_l, \ + "x." NAME "(y) <==> " DOC) +#define RBINSLOTNOTINFIX(NAME, SLOT, FUNCTION, DOC) \ + ETSLOT(NAME, as_number.SLOT, FUNCTION, wrap_binaryfunc_r, \ + "x." NAME "(y) <==> " DOC) static slotdef slotdefs[] = { SQSLOT("__len__", sq_length, slot_sq_length, wrap_inquiry, @@ -4977,9 +4983,9 @@ "%"), RBINSLOT("__rmod__", nb_remainder, slot_nb_remainder, "%"), - BINSLOT("__divmod__", nb_divmod, slot_nb_divmod, + BINSLOTNOTINFIX("__divmod__", nb_divmod, slot_nb_divmod, "divmod(x, y)"), - RBINSLOT("__rdivmod__", nb_divmod, slot_nb_divmod, + RBINSLOTNOTINFIX("__rdivmod__", nb_divmod, slot_nb_divmod, "divmod(y, x)"), NBSLOT("__pow__", nb_power, slot_nb_power, wrap_ternaryfunc, "x.__pow__(y[, z]) <==> pow(x, y[, z])"), From birkenfeld at users.sourceforge.net Fri Jun 3 16:24:56 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:24:56 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_posixpath.py, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5595/Lib/test Modified Files: test_posixpath.py Log Message: Bug #1213894: os.path.realpath didn't resolve symlinks that were the first component of the path. Index: test_posixpath.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_posixpath.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- test_posixpath.py 30 Aug 2004 10:19:56 -0000 1.12 +++ test_posixpath.py 3 Jun 2005 14:24:44 -0000 1.13 @@ -476,6 +476,26 @@ self.safe_rmdir(ABSTFN + "/k/y") self.safe_rmdir(ABSTFN + "/k") self.safe_rmdir(ABSTFN) + + def test_realpath_resolve_first(self): + # Bug #1213894: The first component of the path, if not absolute, + # must be resolved too. + + try: + old_path = abspath('.') + os.mkdir(ABSTFN) + os.mkdir(ABSTFN + "/k") + os.symlink(ABSTFN, ABSTFN + "link") + os.chdir(dirname(ABSTFN)) + + base = basename(ABSTFN) + self.assertEqual(realpath(base + "link"), ABSTFN) + self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") + finally: + os.chdir(old_path) + self.safe_remove(ABSTFN + "link") + self.safe_rmdir(ABSTFN + "/k") + self.safe_rmdir(ABSTFN) # Convenience functions for removing temporary files. def pass_os_error(self, func, filename): From birkenfeld at users.sourceforge.net Fri Jun 3 16:25:16 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:25:16 -0700 Subject: [Python-checkins] python/dist/src/Lib posixpath.py,1.73,1.74 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5595/Lib Modified Files: posixpath.py Log Message: Bug #1213894: os.path.realpath didn't resolve symlinks that were the first component of the path. Index: posixpath.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/posixpath.py,v retrieving revision 1.73 retrieving revision 1.74 diff -u -d -r1.73 -r1.74 --- posixpath.py 30 Aug 2004 10:19:55 -0000 1.73 +++ posixpath.py 3 Jun 2005 14:24:43 -0000 1.74 @@ -414,7 +414,7 @@ if isabs(filename): bits = ['/'] + filename.split('/')[1:] else: - bits = filename.split('/') + bits = [''] + filename.split('/') for i in range(2, len(bits)+1): component = join(*bits[0:i]) From birkenfeld at users.sourceforge.net Fri Jun 3 16:28:57 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:28:57 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1295,1.1296 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8326/Misc Modified Files: NEWS Log Message: Bug #1213894: os.path.realpath didn't resolve symlinks that were the first component of the path. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1295 retrieving revision 1.1296 diff -u -d -r1.1295 -r1.1296 --- NEWS 2 Jun 2005 13:09:26 -0000 1.1295 +++ NEWS 3 Jun 2005 14:28:50 -0000 1.1296 @@ -136,6 +136,9 @@ Library ------- +- Bug #1213894: os.path.realpath didn't resolve symlinks that were the first + component of the path. + - Patch #1120353: The xmlrpclib module provides better, more transparent, support for datetime.{datetime,date,time} objects. With use_datetime set to True, applications shouldn't have to fiddle with the DateTime wrapper From birkenfeld at users.sourceforge.net Fri Jun 3 16:31:58 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:31:58 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_posixpath.py, 1.12, 1.12.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9621/Lib/test Modified Files: Tag: release24-maint test_posixpath.py Log Message: Backport bug #1213894: os.path.realpath didn't resolve symlinks that were the first component of the path. Index: test_posixpath.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_posixpath.py,v retrieving revision 1.12 retrieving revision 1.12.2.1 diff -u -d -r1.12 -r1.12.2.1 --- test_posixpath.py 30 Aug 2004 10:19:56 -0000 1.12 +++ test_posixpath.py 3 Jun 2005 14:31:49 -0000 1.12.2.1 @@ -476,6 +476,26 @@ self.safe_rmdir(ABSTFN + "/k/y") self.safe_rmdir(ABSTFN + "/k") self.safe_rmdir(ABSTFN) + + def test_realpath_resolve_first(self): + # Bug #1213894: The first component of the path, if not absolute, + # must be resolved too. + + try: + old_path = abspath('.') + os.mkdir(ABSTFN) + os.mkdir(ABSTFN + "/k") + os.symlink(ABSTFN, ABSTFN + "link") + os.chdir(dirname(ABSTFN)) + + base = basename(ABSTFN) + self.assertEqual(realpath(base + "link"), ABSTFN) + self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") + finally: + os.chdir(old_path) + self.safe_remove(ABSTFN + "link") + self.safe_rmdir(ABSTFN + "/k") + self.safe_rmdir(ABSTFN) # Convenience functions for removing temporary files. def pass_os_error(self, func, filename): From birkenfeld at users.sourceforge.net Fri Jun 3 16:31:58 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:31:58 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.50, 1.1193.2.51 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9621/Misc Modified Files: Tag: release24-maint NEWS Log Message: Backport bug #1213894: os.path.realpath didn't resolve symlinks that were the first component of the path. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1193.2.50 retrieving revision 1.1193.2.51 diff -u -d -r1.1193.2.50 -r1.1193.2.51 --- NEWS 24 Apr 2005 19:32:27 -0000 1.1193.2.50 +++ NEWS 3 Jun 2005 14:31:55 -0000 1.1193.2.51 @@ -35,6 +35,9 @@ Library ------- +- Bug #1213894: os.path.realpath didn't resolve symlinks that were the first + component of the path. + - distutils.commands.register now encodes the data as UTF-8 before posting them to PyPI. From birkenfeld at users.sourceforge.net Fri Jun 3 16:32:21 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:32:21 -0700 Subject: [Python-checkins] python/dist/src/Lib posixpath.py,1.73,1.73.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9621/Lib Modified Files: Tag: release24-maint posixpath.py Log Message: Backport bug #1213894: os.path.realpath didn't resolve symlinks that were the first component of the path. Index: posixpath.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/posixpath.py,v retrieving revision 1.73 retrieving revision 1.73.2.1 diff -u -d -r1.73 -r1.73.2.1 --- posixpath.py 30 Aug 2004 10:19:55 -0000 1.73 +++ posixpath.py 3 Jun 2005 14:31:48 -0000 1.73.2.1 @@ -414,7 +414,7 @@ if isabs(filename): bits = ['/'] + filename.split('/')[1:] else: - bits = filename.split('/') + bits = [''] + filename.split('/') for i in range(2, len(bits)+1): component = join(*bits[0:i]) From mwh at users.sourceforge.net Fri Jun 3 16:42:01 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:42:01 -0700 Subject: [Python-checkins] python/dist/src/Include marshal.h,2.14,2.15 Message-ID: Update of /cvsroot/python/python/dist/src/Include In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14030/Include Modified Files: marshal.h Log Message: This is my patch: [ 1180995 ] binary formats for marshalling floats Adds 2 new type codes for marshal (binary floats and binary complexes), a new marshal version (2), updates MAGIC and fiddles the de-serializing of code objects to be less likely to clobber the real reason for failing if it fails. Index: marshal.h =================================================================== RCS file: /cvsroot/python/python/dist/src/Include/marshal.h,v retrieving revision 2.14 retrieving revision 2.15 diff -u -d -r2.14 -r2.15 --- marshal.h 27 Jun 2004 16:51:45 -0000 2.14 +++ marshal.h 3 Jun 2005 14:41:44 -0000 2.15 @@ -7,7 +7,7 @@ extern "C" { #endif -#define Py_MARSHAL_VERSION 1 +#define Py_MARSHAL_VERSION 2 PyAPI_FUNC(void) PyMarshal_WriteLongToFile(long, FILE *, int); PyAPI_FUNC(void) PyMarshal_WriteObjectToFile(PyObject *, FILE *, int); From mwh at users.sourceforge.net Fri Jun 3 16:42:01 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:42:01 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_marshal.py, 1.8, 1.9 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14030/Lib/test Modified Files: test_marshal.py Log Message: This is my patch: [ 1180995 ] binary formats for marshalling floats Adds 2 new type codes for marshal (binary floats and binary complexes), a new marshal version (2), updates MAGIC and fiddles the de-serializing of code objects to be less likely to clobber the real reason for failing if it fails. Index: test_marshal.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_marshal.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- test_marshal.py 11 Jan 2005 03:03:06 -0000 1.8 +++ test_marshal.py 3 Jun 2005 14:41:55 -0000 1.9 @@ -73,20 +73,34 @@ n /= 123.4567 f = 0.0 - s = marshal.dumps(f) + s = marshal.dumps(f, 2) got = marshal.loads(s) self.assertEqual(f, got) + # and with version <= 1 (floats marshalled differently then) + s = marshal.dumps(f, 1) + got = marshal.loads(s) + self.assertEqual(f, got) n = sys.maxint * 3.7e-250 while n < small: for expected in (-n, n): f = float(expected) + s = marshal.dumps(f) got = marshal.loads(s) self.assertEqual(f, got) + + s = marshal.dumps(f, 1) + got = marshal.loads(s) + self.assertEqual(f, got) + marshal.dump(f, file(test_support.TESTFN, "wb")) got = marshal.load(file(test_support.TESTFN, "rb")) self.assertEqual(f, got) + + marshal.dump(f, file(test_support.TESTFN, "wb"), 1) + got = marshal.load(file(test_support.TESTFN, "rb")) + self.assertEqual(f, got) n *= 123.4567 os.unlink(test_support.TESTFN) From mwh at users.sourceforge.net Fri Jun 3 16:42:01 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:42:01 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1296,1.1297 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14030/Misc Modified Files: NEWS Log Message: This is my patch: [ 1180995 ] binary formats for marshalling floats Adds 2 new type codes for marshal (binary floats and binary complexes), a new marshal version (2), updates MAGIC and fiddles the de-serializing of code objects to be less likely to clobber the real reason for failing if it fails. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1296 retrieving revision 1.1297 diff -u -d -r1.1296 -r1.1297 --- NEWS 3 Jun 2005 14:28:50 -0000 1.1296 +++ NEWS 3 Jun 2005 14:41:55 -0000 1.1297 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- SF patch #1180995: marshal now uses a binary format by default when + serializing floats. + - SF patch #1181301: on platforms that appear to use IEEE 754 floats, the routines that promise to produce IEEE 754 binary representations of floats now simply copy bytes around. From mwh at users.sourceforge.net Fri Jun 3 16:42:01 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 03 Jun 2005 07:42:01 -0700 Subject: [Python-checkins] python/dist/src/Python marshal.c, 1.82, 1.83 import.c, 2.241, 2.242 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14030/Python Modified Files: marshal.c import.c Log Message: This is my patch: [ 1180995 ] binary formats for marshalling floats Adds 2 new type codes for marshal (binary floats and binary complexes), a new marshal version (2), updates MAGIC and fiddles the de-serializing of code objects to be less likely to clobber the real reason for failing if it fails. Index: marshal.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/marshal.c,v retrieving revision 1.82 retrieving revision 1.83 diff -u -d -r1.82 -r1.83 --- marshal.c 12 Jan 2005 16:00:55 -0000 1.82 +++ marshal.c 3 Jun 2005 14:41:54 -0000 1.83 @@ -15,28 +15,30 @@ */ #define MAX_MARSHAL_STACK_DEPTH 5000 -#define TYPE_NULL '0' -#define TYPE_NONE 'N' -#define TYPE_FALSE 'F' -#define TYPE_TRUE 'T' -#define TYPE_STOPITER 'S' -#define TYPE_ELLIPSIS '.' -#define TYPE_INT 'i' -#define TYPE_INT64 'I' -#define TYPE_FLOAT 'f' -#define TYPE_COMPLEX 'x' -#define TYPE_LONG 'l' -#define TYPE_STRING 's' -#define TYPE_INTERNED 't' -#define TYPE_STRINGREF 'R' -#define TYPE_TUPLE '(' -#define TYPE_LIST '[' -#define TYPE_DICT '{' -#define TYPE_CODE 'c' -#define TYPE_UNICODE 'u' -#define TYPE_UNKNOWN '?' -#define TYPE_SET '<' -#define TYPE_FROZENSET '>' +#define TYPE_NULL '0' +#define TYPE_NONE 'N' +#define TYPE_FALSE 'F' +#define TYPE_TRUE 'T' +#define TYPE_STOPITER 'S' +#define TYPE_ELLIPSIS '.' +#define TYPE_INT 'i' +#define TYPE_INT64 'I' +#define TYPE_FLOAT 'f' +#define TYPE_BINARY_FLOAT 'g' +#define TYPE_COMPLEX 'x' +#define TYPE_BINARY_COMPLEX 'y' +#define TYPE_LONG 'l' +#define TYPE_STRING 's' +#define TYPE_INTERNED 't' +#define TYPE_STRINGREF 'R' +#define TYPE_TUPLE '(' +#define TYPE_LIST '[' +#define TYPE_DICT '{' +#define TYPE_CODE 'c' +#define TYPE_UNICODE 'u' +#define TYPE_UNKNOWN '?' +#define TYPE_SET '<' +#define TYPE_FROZENSET '>' typedef struct { FILE *fp; @@ -47,6 +49,7 @@ char *ptr; char *end; PyObject *strings; /* dict on marshal, list on unmarshal */ + int version; } WFILE; #define w_byte(c, p) if (((p)->fp)) putc((c), (p)->fp); \ @@ -165,32 +168,62 @@ w_short(ob->ob_digit[i], p); } else if (PyFloat_Check(v)) { - char buf[256]; /* Plenty to format any double */ - PyFloat_AsReprString(buf, (PyFloatObject *)v); - n = strlen(buf); - w_byte(TYPE_FLOAT, p); - w_byte(n, p); - w_string(buf, n, p); + if (p->version > 1) { + char buf[8]; + if (_PyFloat_Pack8(PyFloat_AsDouble(v), + buf, 1) < 0) { + p->error = 1; + return; + } + w_byte(TYPE_BINARY_FLOAT, p); + w_string(buf, 8, p); + } + else { + char buf[256]; /* Plenty to format any double */ + PyFloat_AsReprString(buf, (PyFloatObject *)v); + n = strlen(buf); + w_byte(TYPE_FLOAT, p); + w_byte(n, p); + w_string(buf, n, p); + } } #ifndef WITHOUT_COMPLEX else if (PyComplex_Check(v)) { - char buf[256]; /* Plenty to format any double */ - PyFloatObject *temp; - w_byte(TYPE_COMPLEX, p); - temp = (PyFloatObject*)PyFloat_FromDouble( - PyComplex_RealAsDouble(v)); - PyFloat_AsReprString(buf, temp); - Py_DECREF(temp); - n = strlen(buf); - w_byte(n, p); - w_string(buf, n, p); - temp = (PyFloatObject*)PyFloat_FromDouble( - PyComplex_ImagAsDouble(v)); - PyFloat_AsReprString(buf, temp); - Py_DECREF(temp); - n = strlen(buf); - w_byte(n, p); - w_string(buf, n, p); + if (p->version > 1) { + char buf[8]; + if (_PyFloat_Pack8(PyComplex_RealAsDouble(v), + buf, 1) < 0) { + p->error = 1; + return; + } + w_byte(TYPE_BINARY_COMPLEX, p); + w_string(buf, 8, p); + if (_PyFloat_Pack8(PyComplex_ImagAsDouble(v), + buf, 1) < 0) { + p->error = 1; + return; + } + w_string(buf, 8, p); + } + else { + char buf[256]; /* Plenty to format any double */ + PyFloatObject *temp; + w_byte(TYPE_COMPLEX, p); + temp = (PyFloatObject*)PyFloat_FromDouble( + PyComplex_RealAsDouble(v)); + PyFloat_AsReprString(buf, temp); + Py_DECREF(temp); + n = strlen(buf); + w_byte(n, p); + w_string(buf, n, p); + temp = (PyFloatObject*)PyFloat_FromDouble( + PyComplex_ImagAsDouble(v)); + PyFloat_AsReprString(buf, temp); + Py_DECREF(temp); + n = strlen(buf); + w_byte(n, p); + w_string(buf, n, p); + } } #endif else if (PyString_Check(v)) { @@ -335,6 +368,7 @@ wf.error = 0; wf.depth = 0; wf.strings = NULL; + wf.version = version; w_long(x, &wf); } @@ -346,6 +380,7 @@ wf.error = 0; wf.depth = 0; wf.strings = (version > 0) ? PyDict_New() : NULL; + wf.version = version; w_object(x, &wf); Py_XDECREF(wf.strings); } @@ -519,6 +554,22 @@ return PyFloat_FromDouble(dx); } + case TYPE_BINARY_FLOAT: + { + char buf[8]; + double x; + if (r_string(buf, 8, p) != 8) { + PyErr_SetString(PyExc_EOFError, + "EOF read where object expected"); + return NULL; + } + x = _PyFloat_Unpack8(buf, 1); + if (x == -1.0 && PyErr_Occurred()) { + return NULL; + } + return PyFloat_FromDouble(x); + } + #ifndef WITHOUT_COMPLEX case TYPE_COMPLEX: { @@ -546,6 +597,31 @@ PyFPE_END_PROTECT(c) return PyComplex_FromCComplex(c); } + + case TYPE_BINARY_COMPLEX: + { + char buf[8]; + Py_complex c; + if (r_string(buf, 8, p) != 8) { + PyErr_SetString(PyExc_EOFError, + "EOF read where object expected"); + return NULL; + } + c.real = _PyFloat_Unpack8(buf, 1); + if (c.real == -1.0 && PyErr_Occurred()) { + return NULL; + } + if (r_string(buf, 8, p) != 8) { + PyErr_SetString(PyExc_EOFError, + "EOF read where object expected"); + return NULL; + } + c.imag = _PyFloat_Unpack8(buf, 1); + if (c.imag == -1.0 && PyErr_Occurred()) { + return NULL; + } + return PyComplex_FromCComplex(c); + } #endif case TYPE_INTERNED: @@ -707,30 +783,63 @@ return NULL; } else { - int argcount = r_long(p); - int nlocals = r_long(p); - int stacksize = r_long(p); - int flags = r_long(p); - PyObject *code = r_object(p); - PyObject *consts = r_object(p); - PyObject *names = r_object(p); - PyObject *varnames = r_object(p); - PyObject *freevars = r_object(p); - PyObject *cellvars = r_object(p); - PyObject *filename = r_object(p); - PyObject *name = r_object(p); - int firstlineno = r_long(p); - PyObject *lnotab = r_object(p); + int argcount; + int nlocals; + int stacksize; + int flags; + PyObject *code = NULL; + PyObject *consts = NULL; + PyObject *names = NULL; + PyObject *varnames = NULL; + PyObject *freevars = NULL; + PyObject *cellvars = NULL; + PyObject *filename = NULL; + PyObject *name = NULL; + int firstlineno; + PyObject *lnotab = NULL; + + v = NULL; - if (!PyErr_Occurred()) { - v = (PyObject *) PyCode_New( + argcount = r_long(p); + nlocals = r_long(p); + stacksize = r_long(p); + flags = r_long(p); + code = r_object(p); + if (code == NULL) + goto code_error; + consts = r_object(p); + if (consts == NULL) + goto code_error; + names = r_object(p); + if (names == NULL) + goto code_error; + varnames = r_object(p); + if (varnames == NULL) + goto code_error; + freevars = r_object(p); + if (freevars == NULL) + goto code_error; + cellvars = r_object(p); + if (cellvars == NULL) + goto code_error; + filename = r_object(p); + if (filename == NULL) + goto code_error; + name = r_object(p); + if (name == NULL) + goto code_error; + firstlineno = r_long(p); + lnotab = r_object(p); + if (lnotab == NULL) + goto code_error; + + v = (PyObject *) PyCode_New( argcount, nlocals, stacksize, flags, code, consts, names, varnames, freevars, cellvars, filename, name, firstlineno, lnotab); - } - else - v = NULL; + + code_error: Py_XDECREF(code); Py_XDECREF(consts); Py_XDECREF(names); @@ -882,6 +991,7 @@ wf.end = wf.ptr + PyString_Size(wf.str); wf.error = 0; wf.depth = 0; + wf.version = version; wf.strings = (version > 0) ? PyDict_New() : NULL; w_object(x, &wf); Py_XDECREF(wf.strings); Index: import.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/import.c,v retrieving revision 2.241 retrieving revision 2.242 diff -u -d -r2.241 -r2.242 --- import.c 20 May 2005 00:56:54 -0000 2.241 +++ import.c 3 Jun 2005 14:41:54 -0000 2.242 @@ -50,8 +50,9 @@ Python 2.4a0: 62041 Python 2.4a3: 62051 Python 2.4b1: 62061 + Python 2.5a0: 62071 */ -#define MAGIC (62061 | ((long)'\r'<<16) | ((long)'\n'<<24)) +#define MAGIC (62071 | ((long)'\r'<<16) | ((long)'\n'<<24)) /* Magic word as global; note that _PyImport_Init() can change the value of this global to accommodate for alterations of how the From anthonybaxter at users.sourceforge.net Fri Jun 3 17:04:17 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 03 Jun 2005 08:04:17 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_locale.py, 1.9, 1.10 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24406/Lib/test Modified Files: test_locale.py Log Message: [ 1197218 ] test_locale fix on modern linux On more modern linuxes (and probably others) straight 'en_US' isn't a valid locale. Make the code try a couple of alternates. backport candidate Index: test_locale.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_locale.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- test_locale.py 4 Aug 2004 06:33:51 -0000 1.9 +++ test_locale.py 3 Jun 2005 15:04:15 -0000 1.10 @@ -7,16 +7,18 @@ oldlocale = locale.setlocale(locale.LC_NUMERIC) if sys.platform.startswith("win"): - tloc = "en" -elif sys.platform.startswith("freebsd"): - tloc = "en_US.US-ASCII" + tlocs = ("en",) else: - tloc = "en_US" + tlocs = ("en_US.UTF-8", "en_US.US-ASCII", "en_US") -try: - locale.setlocale(locale.LC_NUMERIC, tloc) -except locale.Error: - raise ImportError, "test locale %s not supported" % tloc +for tloc in tlocs: + try: + locale.setlocale(locale.LC_NUMERIC, tloc) + break + except locale.Error: + continue +else: + raise ImportError, "test locale not supported (tried %s)"%(', '.join(tlocs)) def testformat(formatstr, value, grouping = 0, output=None): if verbose: From mwh at users.sourceforge.net Fri Jun 3 17:17:19 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Fri, 03 Jun 2005 08:17:19 -0700 Subject: [Python-checkins] python/dist/src/Python marshal.c,1.83,1.84 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29871 Modified Files: marshal.c Log Message: Fix a couple of crashing-on-malformed data marshal bugs. Index: marshal.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/marshal.c,v retrieving revision 1.83 retrieving revision 1.84 diff -u -d -r1.83 -r1.84 --- marshal.c 3 Jun 2005 14:41:54 -0000 1.83 +++ marshal.c 3 Jun 2005 15:17:16 -0000 1.84 @@ -632,13 +632,13 @@ return NULL; } v = PyString_FromStringAndSize((char *)NULL, n); - if (v != NULL) { - if (r_string(PyString_AS_STRING(v), (int)n, p) != n) { - Py_DECREF(v); - v = NULL; - PyErr_SetString(PyExc_EOFError, + if (v == NULL) + return v; + if (r_string(PyString_AS_STRING(v), (int)n, p) != n) { + Py_DECREF(v); + PyErr_SetString(PyExc_EOFError, "EOF read where object expected"); - } + return NULL; } if (type == TYPE_INTERNED) { PyString_InternInPlace(&v); @@ -766,6 +766,8 @@ } PyTuple_SET_ITEM(v, (int)i, v2); } + if (v == NULL) + return v; if (type == TYPE_SET) v3 = PyObject_CallFunctionObjArgs( (PyObject *)&PySet_Type, v, NULL); From anthonybaxter at users.sourceforge.net Fri Jun 3 18:01:33 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 03 Jun 2005 09:01:33 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_locale.py, 1.9, 1.9.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22310/Lib/test Modified Files: Tag: release24-maint test_locale.py Log Message: backport of [ 1197218 ] test_locale fix on modern linux On more modern linuxes (and probably others) straight 'en_US' isn't a valid locale. Make the code try a couple of alternates. Index: test_locale.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_locale.py,v retrieving revision 1.9 retrieving revision 1.9.2.1 diff -u -d -r1.9 -r1.9.2.1 --- test_locale.py 4 Aug 2004 06:33:51 -0000 1.9 +++ test_locale.py 3 Jun 2005 16:01:29 -0000 1.9.2.1 @@ -7,16 +7,18 @@ oldlocale = locale.setlocale(locale.LC_NUMERIC) if sys.platform.startswith("win"): - tloc = "en" -elif sys.platform.startswith("freebsd"): - tloc = "en_US.US-ASCII" + tlocs = ("en",) else: - tloc = "en_US" + tlocs = ("en_US.UTF-8", "en_US.US-ASCII", "en_US") -try: - locale.setlocale(locale.LC_NUMERIC, tloc) -except locale.Error: - raise ImportError, "test locale %s not supported" % tloc +for tloc in tlocs: + try: + locale.setlocale(locale.LC_NUMERIC, tloc) + break + except locale.Error: + continue +else: + raise ImportError, "test locale not supported (tried %s)"%(', '.join(tlocs)) def testformat(formatstr, value, grouping = 0, output=None): if verbose: From birkenfeld at users.sourceforge.net Fri Jun 3 21:45:55 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 12:45:55 -0700 Subject: [Python-checkins] python/dist/src/Modules bz2module.c, 1.23, 1.23.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7372/Modules Modified Files: Tag: release24-maint bz2module.c Log Message: Backport bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. Index: bz2module.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/bz2module.c,v retrieving revision 1.23 retrieving revision 1.23.2.1 diff -u -d -r1.23 -r1.23.2.1 --- bz2module.c 1 Nov 2004 17:10:19 -0000 1.23 +++ bz2module.c 3 Jun 2005 19:45:52 -0000 1.23.2.1 @@ -1308,6 +1308,10 @@ break; } + if (mode_char == 0) { + mode_char = 'r'; + } + mode = (mode_char == 'r') ? "rb" : "wb"; self->file = PyObject_CallFunction((PyObject*)&PyFile_Type, "(Osi)", From birkenfeld at users.sourceforge.net Fri Jun 3 21:45:55 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 12:45:55 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_bz2.py, 1.16, 1.16.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7372/Lib/test Modified Files: Tag: release24-maint test_bz2.py Log Message: Backport bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. Index: test_bz2.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_bz2.py,v retrieving revision 1.16 retrieving revision 1.16.2.1 diff -u -d -r1.16 -r1.16.2.1 --- test_bz2.py 1 Nov 2004 17:10:19 -0000 1.16 +++ test_bz2.py 3 Jun 2005 19:45:53 -0000 1.16.2.1 @@ -235,6 +235,16 @@ # "Test opening a nonexistent file" self.assertRaises(IOError, BZ2File, "/non/existent") + def testModeU(self): + # Bug #1194181: bz2.BZ2File opened for write with mode "U" + self.createTempFile() + bz2f = BZ2File(self.filename, "U") + bz2f.close() + f = file(self.filename) + f.seek(0, 2) + self.assertEqual(f.tell(), len(self.DATA)) + f.close() + class BZ2CompressorTest(BaseTest): def testCompress(self): # "Test BZ2Compressor.compress()/flush()" From birkenfeld at users.sourceforge.net Fri Jun 3 21:45:56 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 12:45:56 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.51, 1.1193.2.52 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7372/Misc Modified Files: Tag: release24-maint NEWS Log Message: Backport bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1193.2.51 retrieving revision 1.1193.2.52 diff -u -d -r1.1193.2.51 -r1.1193.2.52 --- NEWS 3 Jun 2005 14:31:55 -0000 1.1193.2.51 +++ NEWS 3 Jun 2005 19:45:53 -0000 1.1193.2.52 @@ -27,6 +27,8 @@ Extension Modules ----------------- +- Bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. + - Bug #1166660: The readline module could segfault if hook functions were set in a different thread than that which called readline. From birkenfeld at users.sourceforge.net Fri Jun 3 21:47:02 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 12:47:02 -0700 Subject: [Python-checkins] python/dist/src/Modules bz2module.c,1.23,1.24 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8373/Modules Modified Files: bz2module.c Log Message: Bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. Index: bz2module.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/bz2module.c,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- bz2module.c 1 Nov 2004 17:10:19 -0000 1.23 +++ bz2module.c 3 Jun 2005 19:47:00 -0000 1.24 @@ -1308,6 +1308,10 @@ break; } + if (mode_char == 0) { + mode_char = 'r'; + } + mode = (mode_char == 'r') ? "rb" : "wb"; self->file = PyObject_CallFunction((PyObject*)&PyFile_Type, "(Osi)", From birkenfeld at users.sourceforge.net Fri Jun 3 21:47:02 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 12:47:02 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_bz2.py,1.16,1.17 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8373/Lib/test Modified Files: test_bz2.py Log Message: Bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. Index: test_bz2.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_bz2.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- test_bz2.py 1 Nov 2004 17:10:19 -0000 1.16 +++ test_bz2.py 3 Jun 2005 19:47:00 -0000 1.17 @@ -235,6 +235,16 @@ # "Test opening a nonexistent file" self.assertRaises(IOError, BZ2File, "/non/existent") + def testModeU(self): + # Bug #1194181: bz2.BZ2File opened for write with mode "U" + self.createTempFile() + bz2f = BZ2File(self.filename, "U") + bz2f.close() + f = file(self.filename) + f.seek(0, 2) + self.assertEqual(f.tell(), len(self.DATA)) + f.close() + class BZ2CompressorTest(BaseTest): def testCompress(self): # "Test BZ2Compressor.compress()/flush()" From birkenfeld at users.sourceforge.net Fri Jun 3 21:47:02 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 03 Jun 2005 12:47:02 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1297,1.1298 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8373/Misc Modified Files: NEWS Log Message: Bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1297 retrieving revision 1.1298 diff -u -d -r1.1297 -r1.1298 --- NEWS 3 Jun 2005 14:41:55 -0000 1.1297 +++ NEWS 3 Jun 2005 19:47:00 -0000 1.1298 @@ -91,6 +91,8 @@ Extension Modules ----------------- +- Bug #1194181: bz2.BZ2File didn't handle mode 'U' correctly. + - Patch #1212117: os.stat().st_flags is now accessible as a attribute if available on the platform. From tim_one at users.sourceforge.net Sat Jun 4 00:40:30 2005 From: tim_one at users.sourceforge.net (tim_one@users.sourceforge.net) Date: Fri, 03 Jun 2005 15:40:30 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_compare.py, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv423/Lib/bsddb/test Modified Files: test_compare.py Log Message: Whitespace normalization. Index: test_compare.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_compare.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- test_compare.py 3 Jun 2005 07:03:07 -0000 1.1 +++ test_compare.py 3 Jun 2005 22:40:27 -0000 1.2 @@ -9,207 +9,207 @@ from bsddb3 import db def lexical_cmp (db, left, right): - return cmp (left, right) + return cmp (left, right) def lowercase_cmp(db, left, right): - return cmp (left.lower(), right.lower()) + return cmp (left.lower(), right.lower()) def make_reverse_comparator (cmp): - def reverse (db, left, right, delegate=cmp): - return - delegate (db, left, right) - return reverse + def reverse (db, left, right, delegate=cmp): + return - delegate (db, left, right) + return reverse _expected_lexical_test_data = ['', 'CCCP', 'a', 'aaa', 'b', 'c', 'cccce', 'ccccf'] _expected_lowercase_test_data = ['', 'a', 'aaa', 'b', 'c', 'CC', 'cccce', 'ccccf', 'CCCP'] class ComparatorTests (unittest.TestCase): - def comparator_test_helper (self, comparator, expected_data): - data = expected_data[:] - data.sort (lambda l, r, cmp=comparator: cmp (None, l, r)) - self.failUnless (data == expected_data, - "comparator `%s' is not right: %s vs. %s" - % (comparator, expected_data, data)) - def test_lexical_comparator (self): - self.comparator_test_helper (lexical_cmp, _expected_lexical_test_data) - def test_reverse_lexical_comparator (self): - rev = _expected_lexical_test_data[:] - rev.reverse () - self.comparator_test_helper (make_reverse_comparator (lexical_cmp), - rev) - def test_lowercase_comparator (self): - self.comparator_test_helper (lowercase_cmp, - _expected_lowercase_test_data) + def comparator_test_helper (self, comparator, expected_data): + data = expected_data[:] + data.sort (lambda l, r, cmp=comparator: cmp (None, l, r)) + self.failUnless (data == expected_data, + "comparator `%s' is not right: %s vs. %s" + % (comparator, expected_data, data)) + def test_lexical_comparator (self): + self.comparator_test_helper (lexical_cmp, _expected_lexical_test_data) + def test_reverse_lexical_comparator (self): + rev = _expected_lexical_test_data[:] + rev.reverse () + self.comparator_test_helper (make_reverse_comparator (lexical_cmp), + rev) + def test_lowercase_comparator (self): + self.comparator_test_helper (lowercase_cmp, + _expected_lowercase_test_data) class AbstractBtreeKeyCompareTestCase (unittest.TestCase): - env = None - db = None - - def setUp (self): - self.filename = self.__class__.__name__ + '.db' - homeDir = os.path.join (os.path.dirname (sys.argv[0]), 'db_home') - self.homeDir = homeDir - try: - os.mkdir (homeDir) - except os.error: - pass + env = None + db = None - env = db.DBEnv () - env.open (homeDir, - db.DB_CREATE | db.DB_INIT_MPOOL - | db.DB_INIT_LOCK | db.DB_THREAD) - self.env = env + def setUp (self): + self.filename = self.__class__.__name__ + '.db' + homeDir = os.path.join (os.path.dirname (sys.argv[0]), 'db_home') + self.homeDir = homeDir + try: + os.mkdir (homeDir) + except os.error: + pass - def tearDown (self): - self.closeDB () - if self.env is not None: - self.env.close () - self.env = None - import glob - map (os.remove, glob.glob (os.path.join (self.homeDir, '*'))) + env = db.DBEnv () + env.open (homeDir, + db.DB_CREATE | db.DB_INIT_MPOOL + | db.DB_INIT_LOCK | db.DB_THREAD) + self.env = env - def addDataToDB (self, data): - i = 0 - for item in data: - self.db.put (item, str (i)) - i = i + 1 + def tearDown (self): + self.closeDB () + if self.env is not None: + self.env.close () + self.env = None + import glob + map (os.remove, glob.glob (os.path.join (self.homeDir, '*'))) - def createDB (self, key_comparator): - self.db = db.DB (self.env) - self.setupDB (key_comparator) - self.db.open (self.filename, "test", db.DB_BTREE, db.DB_CREATE) + def addDataToDB (self, data): + i = 0 + for item in data: + self.db.put (item, str (i)) + i = i + 1 - def setupDB (self, key_comparator): - self.db.set_bt_compare (key_comparator) + def createDB (self, key_comparator): + self.db = db.DB (self.env) + self.setupDB (key_comparator) + self.db.open (self.filename, "test", db.DB_BTREE, db.DB_CREATE) - def closeDB (self): - if self.db is not None: - self.db.close () - self.db = None + def setupDB (self, key_comparator): + self.db.set_bt_compare (key_comparator) - def startTest (self): - pass - - def finishTest (self, expected = None): - if expected is not None: - self.check_results (expected) - self.closeDB () + def closeDB (self): + if self.db is not None: + self.db.close () + self.db = None - def check_results (self, expected): - curs = self.db.cursor () - try: - index = 0 - rec = curs.first () - while rec: - key, ignore = rec - self.failUnless (index < len (expected), - "to many values returned from cursor") - self.failUnless (expected[index] == key, - "expected value `%s' at %d but got `%s'" - % (expected[index], index, key)) - index = index + 1 - rec = curs.next () - self.failUnless (index == len (expected), - "not enough values returned from cursor") - finally: - curs.close () + def startTest (self): + pass + + def finishTest (self, expected = None): + if expected is not None: + self.check_results (expected) + self.closeDB () + + def check_results (self, expected): + curs = self.db.cursor () + try: + index = 0 + rec = curs.first () + while rec: + key, ignore = rec + self.failUnless (index < len (expected), + "to many values returned from cursor") + self.failUnless (expected[index] == key, + "expected value `%s' at %d but got `%s'" + % (expected[index], index, key)) + index = index + 1 + rec = curs.next () + self.failUnless (index == len (expected), + "not enough values returned from cursor") + finally: + curs.close () class BtreeKeyCompareTestCase (AbstractBtreeKeyCompareTestCase): - def runCompareTest (self, comparator, data): - self.startTest () - self.createDB (comparator) - self.addDataToDB (data) - self.finishTest (data) + def runCompareTest (self, comparator, data): + self.startTest () + self.createDB (comparator) + self.addDataToDB (data) + self.finishTest (data) - def test_lexical_ordering (self): - self.runCompareTest (lexical_cmp, _expected_lexical_test_data) + def test_lexical_ordering (self): + self.runCompareTest (lexical_cmp, _expected_lexical_test_data) - def test_reverse_lexical_ordering (self): - expected_rev_data = _expected_lexical_test_data[:] - expected_rev_data.reverse () - self.runCompareTest (make_reverse_comparator (lexical_cmp), - expected_rev_data) + def test_reverse_lexical_ordering (self): + expected_rev_data = _expected_lexical_test_data[:] + expected_rev_data.reverse () + self.runCompareTest (make_reverse_comparator (lexical_cmp), + expected_rev_data) + + def test_compare_function_useless (self): + self.startTest () + def socialist_comparator (db, l, r): + return 0 + self.createDB (socialist_comparator) + self.addDataToDB (['b', 'a', 'd']) + # all things being equal the first key will be the only key + # in the database... (with the last key's value fwiw) + self.finishTest (['b']) - def test_compare_function_useless (self): - self.startTest () - def socialist_comparator (db, l, r): - return 0 - self.createDB (socialist_comparator) - self.addDataToDB (['b', 'a', 'd']) - # all things being equal the first key will be the only key - # in the database... (with the last key's value fwiw) - self.finishTest (['b']) - class BtreeExceptionsTestCase (AbstractBtreeKeyCompareTestCase): - def test_raises_non_callable (self): - self.startTest () - self.assertRaises (TypeError, self.createDB, 'abc') - self.assertRaises (TypeError, self.createDB, None) - self.finishTest () + def test_raises_non_callable (self): + self.startTest () + self.assertRaises (TypeError, self.createDB, 'abc') + self.assertRaises (TypeError, self.createDB, None) + self.finishTest () - def test_set_bt_compare_with_function (self): - self.startTest () - self.createDB (lexical_cmp) - self.finishTest () + def test_set_bt_compare_with_function (self): + self.startTest () + self.createDB (lexical_cmp) + self.finishTest () - def check_results (self, results): - pass + def check_results (self, results): + pass - def test_compare_function_incorrect (self): - self.startTest () - def bad_comparator (db, l, r): - return 1 - # verify that set_bt_compare checks that comparator(db, '', '') == 0 - self.assertRaises (TypeError, self.createDB, bad_comparator) - self.finishTest () + def test_compare_function_incorrect (self): + self.startTest () + def bad_comparator (db, l, r): + return 1 + # verify that set_bt_compare checks that comparator(db, '', '') == 0 + self.assertRaises (TypeError, self.createDB, bad_comparator) + self.finishTest () - def test_compare_function_exception (self): - self.startTest () - def bad_comparator (db, l, r): - if l == r: - # pass the set_bt_compare test - return 0 - raise RuntimeError, "i'm a naughty comparison function" - self.createDB (bad_comparator) - print "\n*** this test should print 2 uncatchable tracebacks ***" - self.addDataToDB (['a', 'b', 'c']) # this should raise, but... - self.finishTest () + def test_compare_function_exception (self): + self.startTest () + def bad_comparator (db, l, r): + if l == r: + # pass the set_bt_compare test + return 0 + raise RuntimeError, "i'm a naughty comparison function" + self.createDB (bad_comparator) + print "\n*** this test should print 2 uncatchable tracebacks ***" + self.addDataToDB (['a', 'b', 'c']) # this should raise, but... + self.finishTest () - def test_compare_function_bad_return (self): - self.startTest () - def bad_comparator (db, l, r): - if l == r: - # pass the set_bt_compare test - return 0 - return l - self.createDB (bad_comparator) - print "\n*** this test should print 2 errors about returning an int ***" - self.addDataToDB (['a', 'b', 'c']) # this should raise, but... - self.finishTest () + def test_compare_function_bad_return (self): + self.startTest () + def bad_comparator (db, l, r): + if l == r: + # pass the set_bt_compare test + return 0 + return l + self.createDB (bad_comparator) + print "\n*** this test should print 2 errors about returning an int ***" + self.addDataToDB (['a', 'b', 'c']) # this should raise, but... + self.finishTest () - def test_cannot_assign_twice (self): + def test_cannot_assign_twice (self): - def my_compare (db, a, b): - return 0 - - self.startTest () - self.createDB (my_compare) - try: - self.db.set_bt_compare (my_compare) - assert False, "this set should fail" + def my_compare (db, a, b): + return 0 + + self.startTest () + self.createDB (my_compare) + try: + self.db.set_bt_compare (my_compare) + assert False, "this set should fail" + + except RuntimeError, msg: + pass - except RuntimeError, msg: - pass - def test_suite (): - res = unittest.TestSuite () + res = unittest.TestSuite () - res.addTest (unittest.makeSuite (ComparatorTests)) - if db.version () >= (3, 3, 11): - res.addTest (unittest.makeSuite (BtreeExceptionsTestCase)) - res.addTest (unittest.makeSuite (BtreeKeyCompareTestCase)) - return res + res.addTest (unittest.makeSuite (ComparatorTests)) + if db.version () >= (3, 3, 11): + res.addTest (unittest.makeSuite (BtreeExceptionsTestCase)) + res.addTest (unittest.makeSuite (BtreeKeyCompareTestCase)) + return res if __name__ == '__main__': - unittest.main (defaultTest = 'suite') + unittest.main (defaultTest = 'suite') From tim_one at users.sourceforge.net Sat Jun 4 00:40:30 2005 From: tim_one at users.sourceforge.net (tim_one@users.sourceforge.net) Date: Fri, 03 Jun 2005 15:40:30 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_float.py, 1.1, 1.2 test_marshal.py, 1.9, 1.10 test_posixpath.py, 1.13, 1.14 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv423/Lib/test Modified Files: test_float.py test_marshal.py test_posixpath.py Log Message: Whitespace normalization. Index: test_float.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_float.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- test_float.py 27 May 2005 15:21:31 -0000 1.1 +++ test_float.py 3 Jun 2005 22:40:28 -0000 1.2 @@ -58,7 +58,7 @@ 'float':float.__getformat__('float')} float.__setformat__('double', 'unknown') float.__setformat__('float', 'unknown') - + def tearDown(self): float.__setformat__('double', self.save_formats['double']) float.__setformat__('float', self.save_formats['float']) Index: test_marshal.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_marshal.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- test_marshal.py 3 Jun 2005 14:41:55 -0000 1.9 +++ test_marshal.py 3 Jun 2005 22:40:28 -0000 1.10 @@ -78,26 +78,26 @@ self.assertEqual(f, got) # and with version <= 1 (floats marshalled differently then) s = marshal.dumps(f, 1) - got = marshal.loads(s) - self.assertEqual(f, got) + got = marshal.loads(s) + self.assertEqual(f, got) n = sys.maxint * 3.7e-250 while n < small: for expected in (-n, n): f = float(expected) - + s = marshal.dumps(f) got = marshal.loads(s) self.assertEqual(f, got) - + s = marshal.dumps(f, 1) got = marshal.loads(s) self.assertEqual(f, got) - + marshal.dump(f, file(test_support.TESTFN, "wb")) got = marshal.load(file(test_support.TESTFN, "rb")) self.assertEqual(f, got) - + marshal.dump(f, file(test_support.TESTFN, "wb"), 1) got = marshal.load(file(test_support.TESTFN, "rb")) self.assertEqual(f, got) Index: test_posixpath.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_posixpath.py,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- test_posixpath.py 3 Jun 2005 14:24:44 -0000 1.13 +++ test_posixpath.py 3 Jun 2005 22:40:28 -0000 1.14 @@ -476,7 +476,7 @@ self.safe_rmdir(ABSTFN + "/k/y") self.safe_rmdir(ABSTFN + "/k") self.safe_rmdir(ABSTFN) - + def test_realpath_resolve_first(self): # Bug #1213894: The first component of the path, if not absolute, # must be resolved too. @@ -487,7 +487,7 @@ os.mkdir(ABSTFN + "/k") os.symlink(ABSTFN, ABSTFN + "link") os.chdir(dirname(ABSTFN)) - + base = basename(ABSTFN) self.assertEqual(realpath(base + "link"), ABSTFN) self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") From pje at users.sourceforge.net Sat Jun 4 23:18:24 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 04 Jun 2005 14:18:24 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.22, 1.23 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28685 Modified Files: pkg_resources.py Log Message: Add support for resource isdir/listdir operations. Support directory extraction and "egg baskets" for .zips. Add import locking for namespace package manipulation routines. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.22 retrieving revision 1.23 diff -u -d -r1.22 -r1.23 --- pkg_resources.py 29 May 2005 05:20:59 -0000 1.22 +++ pkg_resources.py 4 Jun 2005 21:18:19 -0000 1.23 @@ -92,11 +92,9 @@ """Yield named metadata resource as list of non-blank non-comment lines Leading and trailing whitespace is stripped from each line, and lines - with ``#`` as the first non-blank character are omitted. - """ + with ``#`` as the first non-blank character are omitted.""" class IResourceProvider(IMetadataProvider): - """An object that provides access to package resources""" def get_resource_filename(manager, resource_name): @@ -117,9 +115,11 @@ def has_resource(resource_name): """Does the package contain the named resource?""" - # XXX list_resources? glob_resources? - + def resource_isdir(resource_name): + """Is the named resource a directory? (like ``os.path.isdir()``)""" + def resource_listdir(resource_name): + """List of resource names in the directory (like ``os.listdir()``)""" class AvailableDistributions(object): """Searchable snapshot of distributions on a search path""" @@ -297,6 +297,10 @@ """Does the named resource exist in the named package?""" return get_provider(package_name).has_resource(self, resource_name) + def resource_isdir(self, package_name, resource_name): + """Does the named resource exist in the named package?""" + return get_provider(package_name).resource_isdir(self, resource_name) + def resource_filename(self, package_name, resource_name): """Return a true filesystem path for specified resource""" return get_provider(package_name).get_resource_filename( @@ -315,12 +319,8 @@ self, resource_name ) - - - - - - + def list_resources(self, package_name, resource_name): + return get_provider(package_name).resource_listdir(self, resource_name) @@ -485,6 +485,11 @@ def get_metadata_lines(self, name): return yield_lines(self.get_metadata(name)) + def resource_isdir(self,name): return False + + def resource_listdir(self,name): + return [] + def _has(self, path): raise NotImplementedError( "Can't perform this operation for unregistered loader type" @@ -503,11 +508,39 @@ register_loader_type(object, NullProvider) + + + + + + + + + + + + + + + + + + + + + + + class DefaultProvider(NullProvider): """Provides access to package resources in the filesystem""" def __init__(self,module): NullProvider.__init__(self,module) + self._setup_prefix() + + def _setup_prefix(self): + # we assume here that our metadata may be nested inside a "basket" + # of multiple eggs; that's why we use module_path instead of .archive path = self.module_path old = None self.prefix = [] @@ -520,7 +553,14 @@ path, base = os.path.split(path) self.prefix.append(base) - def _has(self, path): return os.path.exists(path) + def _has(self, path): + return os.path.exists(path) + + def resource_isdir(self,name): + return os.path.isdir(self._fn(name)) + + def resource_listdir(self,name): + return os.listdir(self._fn(name)) def _get(self, path): stream = open(path, 'rb') @@ -531,6 +571,7 @@ register_loader_type(type(None), DefaultProvider) + class ZipProvider(DefaultProvider): """Resource support for zips and eggs""" @@ -546,11 +587,9 @@ return path[len(self.zip_pre):] return path - def _has(self, path): - return self._short_name(path) in self.zipinfo + def _has(self, path): return self._short_name(path) in self.zipinfo or self.resource_isdir(path) - def _get(self, path): - return self.loader.get_data(path) + def _get(self, path): return self.loader.get_data(path) def get_resource_stream(self, manager, resource_name): return StringIO(self.get_resource_string(manager, resource_name)) @@ -569,10 +608,27 @@ return self._extract_resource(manager, resource_name) + def resource_isdir(self, resource_name): + if resource_name.endswith('/'): + resource_name = resource_name[:-1] + return resource_name in self._index() + def resource_listdir(self, resource_name): + if resource_name.endswith('/'): + resource_name = resource_name[:-1] + return list(self._index().get(resource_name, ())) - + def _extract_directory(self, manager, resource_name): + if resource_name.endswith('/'): + resource_name = resource_name[:-1] + for resource in self.resource_listdir(resource_name): + last = self._extract_resource(manager, resource_name+'/'+resource) + return os.path.dirname(last) # return the directory path + def _extract_resource(self, manager, resource_name): + if self.resource_isdir(resource_name): + return self._extract_dir(resource_name) + parts = resource_name.split('/') zip_path = os.path.join(self.module_path, *parts) zip_stat = self.zipinfo[os.path.join(*self.prefix+parts)] @@ -607,12 +663,38 @@ self.eagers = eagers return self.eagers + def _index(self): + try: + return self._dirindex + except AttributeError: + ind = {}; skip = len(self.prefix) + for path in self.zipinfo: + parts = path.split(os.sep) + if parts[:skip] != self.prefix: + continue # only include items under our prefix + parts = parts[skip:] # but don't include prefix in paths + while parts: + parent = '/'.join(parts[:-1]) + if parent in ind: + ind[parent].append(parts[-1]) + break + else: + ind[parent] = [parts.pop()] + self._dirindex = ind + return ind + register_loader_type(zipimport.zipimporter, ZipProvider) + + + + + + class PathMetadata(DefaultProvider): """Metadata provider for egg directories @@ -647,11 +729,11 @@ self.zipinfo = zipimport._zip_directory_cache[importer.archive] self.zip_pre = importer.archive+os.sep self.loader = importer - self.module_path = os.path.join(importer.archive, importer.prefix) - - # we assume here that our metadata may be nested inside a "basket" - # of multiple eggs; that's why we use module_path instead of .archive - self.egg_info = os.path.join(self.module_path, 'EGG-INFO') + if importer.prefix: + self.module_path = os.path.join(importer.archive, importer.prefix) + else: + self.module_path = importer.archive + self._setup_prefix() class ImpWrapper: @@ -754,13 +836,16 @@ finder = _find_adapter(_distribution_finders, importer) return finder(importer,path_item) - def find_in_zip(importer,path_item): - # for now, we only yield the .egg file itself, if applicable; - # i.e., we don't support "baskets" yet, just eggs - for item in find_on_path(importer,path_item): - yield item - + metadata = EggMetadata(importer) + if metadata.has_metadata('PKG-INFO'): + yield Distribution.from_filename(path_item, metadata=metadata) + for subitem in metadata.resource_listdir('/'): + if subitem.endswith('.egg'): + subpath = os.path.join(path_item, subitem) + for dist in find_in_zip(zipimport.zipimporter(subpath), subpath): + yield dist + register_finder(zipimport.zipimporter,find_in_zip) @@ -774,11 +859,6 @@ return StringIO(*args,**kw) - - - - - def find_nothing(importer,path_item): return () @@ -864,7 +944,7 @@ def declare_namespace(packageName): """Declare that package 'packageName' is a namespace package""" - # XXX nslock.acquire() + imp.acquire_lock() try: if packageName in _namespace_packages: return @@ -890,17 +970,17 @@ _namespace_packages.setdefault(packageName,[]) finally: - pass # XXX nslock.release() + imp.release_lock() def fixup_namespace_packages(path_item, parent=None): """Ensure that previously-declared namespace packages include path_item""" - # XXX nslock.acquire() + imp.acquire_lock() try: for package in _namespace_packages.get(parent,()): subpath = _handle_ns(package, path_item) if subpath: fixup_namespace_packages(subpath,package) finally: - pass # XXX nslock.release() + imp.release_lock() def file_ns_handler(importer, path_item, packageName, module): """Compute an ns-package subpath for a filesystem or zipfile importer""" From pje at users.sourceforge.net Sat Jun 4 23:35:31 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 04 Jun 2005 14:35:31 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.1, 1.2 easy_install.py, 1.10, 1.11 setup.py, 1.7, 1.8 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6103 Modified Files: EasyInstall.txt easy_install.py setup.py Log Message: Fix a minor problem with -b option, and prep for 0.3a4 release. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- EasyInstall.txt 31 May 2005 00:24:45 -0000 1.1 +++ EasyInstall.txt 4 Jun 2005 21:35:13 -0000 1.2 @@ -206,10 +206,31 @@ to read any documentation, examples, scripts, etc. that may have been included with the source distribution (if any). + This option can only be used when you are specifying a single installation + URL or filename, so that the installer will not be confused by the presence + of multiple ``setup.py`` files in the build directory. + Release Notes/Change History ============================ +0.3a4 + * ``pkg_resources`` now supports resource directories, not just the resources + in them. In particular, there are ``resource_listdir()`` and + ``resource_isdir()`` APIs. + + * ``pkg_resources`` now supports "egg baskets" -- .egg zipfiles which contain + multiple distributions in subdirectories whose names end with ``.egg``. + Having such a "basket" in a directory on ``sys.path`` is equivalent to + having the individual eggs in that directory, but the contained eggs can + be individually added (or not) to ``sys.path``. Currently, however, there + is no automated way to create baskets. + + * Namespace package manipulation is now protected by the Python import lock. + + * Restrict ``--build-directory=DIR/-b DIR`` option to only be used with single + URL installs, to avoid running the wrong setup.py. + 0.3a3 * Added ``--build-directory=DIR/-b DIR`` option. @@ -226,7 +247,7 @@ 0.3a2 * Added subversion download support for ``svn:`` and ``svn+`` URLs, as well as - automatic recognition of HTTP subversion URLs (Contributed by Ian Bicking) + automatic recognition of HTTP subversion URLs (Contributed by Ian Bicking) * Added new options to ``bdist_egg`` to allow tagging the egg's version number with a subversion revision number, the current date, or an explicit tag @@ -234,7 +255,6 @@ * Misc. bug fixes - 0.3a1 Initial release. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- easy_install.py 31 May 2005 00:24:45 -0000 1.10 +++ easy_install.py 4 Jun 2005 21:35:13 -0000 1.11 @@ -674,10 +674,11 @@ default=None, help="download/extract/build in DIR; keep the results") (options, args) = parser.parse_args() - try: if not args: parser.error("No urls, filenames, or requirements specified") + elif len(args)>1 and options.tmpdir is not None: + parser.error("Build directory can only be set when using one URL") for spec in args: inst = factory( options.instdir, options.zip_ok, options.multi, options.tmpdir @@ -694,7 +695,6 @@ print >>sys.stderr,"error:",v sys.exit(1) - if __name__ == '__main__': main(sys.argv[1:]) Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- setup.py 31 May 2005 00:34:54 -0000 1.7 +++ setup.py 4 Jun 2005 21:35:13 -0000 1.8 @@ -7,7 +7,7 @@ setup( name="setuptools", - version="0.3a3", + version="0.3a4", description="Distutils packaging and installation enhancements", author="Phillip J. Eby", From pje at users.sourceforge.net Sun Jun 5 02:47:38 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 04 Jun 2005 17:47:38 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.23, 1.24 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16912 Modified Files: pkg_resources.py Log Message: Add support for prioritized sorting of distributions by distribution type, to assist easy_install in indexing packages from PyPI. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- pkg_resources.py 4 Jun 2005 21:18:19 -0000 1.23 +++ pkg_resources.py 5 Jun 2005 00:47:36 -0000 1.24 @@ -15,8 +15,8 @@ __all__ = [ 'register_loader_type', 'get_provider', 'IResourceProvider','PathMetadata', 'ResourceManager', 'AvailableDistributions', 'require', 'resource_string', - 'resource_stream', 'resource_filename', 'set_extraction_path', - 'cleanup_resources', 'parse_requirements', 'ensure_directory', + 'resource_stream', 'resource_filename', 'set_extraction_path', 'EGG_DIST', + 'cleanup_resources', 'parse_requirements', 'ensure_directory','SOURCE_DIST', 'compatible_platforms', 'get_platform', 'IMetadataProvider','parse_version', 'ResolutionError', 'VersionConflict', 'DistributionNotFound','EggMetadata', 'InvalidOption', 'Distribution', 'Requirement', 'yield_lines', @@ -1105,9 +1105,12 @@ +EGG_DIST = 2 +SOURCE_DIST = 1 + class Distribution(object): """Wrap an actual or potential sys.path entry w/metadata""" - + typecode = EGG_DIST def __init__(self, path_str, metadata=None, name=None, version=None, py_version=PY_MAJOR, platform=None @@ -1116,7 +1119,6 @@ self.name = name.replace('_','-') if version: self._version = version.replace('_','-') - self.py_version = py_version self.platform = platform self.path = path_str @@ -1144,8 +1146,6 @@ ) from_filename = classmethod(from_filename) - - # These properties have to be lazy so that we don't have to load any # metadata until/unless it's actually needed. (i.e., some distributions # may not know their name or version without loading PKG-INFO) @@ -1174,13 +1174,13 @@ try: return self._version except AttributeError: - for line in self.metadata.get_metadata_lines('PKG-INFO'): + for line in self._get_metadata('PKG-INFO'): if line.lower().startswith('version:'): self._version = line.split(':',1)[1].strip() return self._version else: raise AttributeError( - "Missing Version: header in PKG-INFO", self + "Missing 'Version:' header in PKG-INFO", self ) version = property(version) @@ -1215,7 +1215,7 @@ return deps def _get_metadata(self,name): - if self.metadata.has_metadata(name): + if self.metadata is not None and self.metadata.has_metadata(name): for line in self.metadata.get_metadata_lines(name): yield line @@ -1289,9 +1289,9 @@ def _sort_dists(dists): - tmp = [(dist.version,dist) for dist in dists] + tmp = [(dist.version,dist.typecode,dist) for dist in dists] tmp.sort() - dists[::-1] = [d for v,d in tmp] + dists[::-1] = [d for v,t,d in tmp] From pje at users.sourceforge.net Sun Jun 5 02:48:13 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 04 Jun 2005 17:48:13 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17486 Modified Files: EasyInstall.txt Log Message: Updated for v0.3a4 Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- EasyInstall.txt 4 Jun 2005 21:35:13 -0000 1.2 +++ EasyInstall.txt 5 Jun 2005 00:48:11 -0000 1.3 @@ -24,9 +24,9 @@ Unix-like Systems (including Mac OS X and Cygwin) Download either the `Python 2.3 easy_install shell script - `_ or the + `_ or the `Python 2.4 easy_install shell script - `_. Place + `_. Place the file somewhere on your PATH, after renaming it to ``easy_install``. Note that these scripts assume you have ``python2.3`` or ``python2.4`` accessible via the ``PATH`` environment variable. Then, you can use ``easy_install`` to @@ -34,15 +34,15 @@ your Python version:: # Python 2.3 - easy_install http://peak.telecommunity.com/dist/setuptools-0.3a3-py2.3.egg + easy_install http://peak.telecommunity.com/dist/setuptools-0.3a4-py2.3.egg # Python 2.4 - easy_install http://peak.telecommunity.com/dist/setuptools-0.3a3-py2.4.egg + easy_install http://peak.telecommunity.com/dist/setuptools-0.3a4-py2.4.egg All Other Systems Download the `easy_install (aka setuptools) source distribution - `_, and follow the + `_, and follow the normal procedure for installing a source package with distutils. An ``easy_install.py`` script will be installed in the normal location for Python scripts on your platform. In the examples below, you'll need to From pje at users.sourceforge.net Sun Jun 5 03:20:11 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 04 Jun 2005 18:20:11 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.24, 1.25 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2589 Modified Files: pkg_resources.py Log Message: Make ``AvailableDistributions`` keep track of the desired platform/python. Add a ``can_add()`` method that indicates whether a distribution matches the collection's desired platform and Python version. Fix a bug in ``Distribution.from_filename()`` when the filename has no Python version. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.24 retrieving revision 1.25 diff -u -d -r1.24 -r1.25 --- pkg_resources.py 5 Jun 2005 00:47:36 -0000 1.24 +++ pkg_resources.py 5 Jun 2005 01:20:09 -0000 1.25 @@ -127,13 +127,30 @@ def __init__(self,search_path=None,platform=get_platform(),python=PY_MAJOR): """Snapshot distributions available on a search path - The constructor arguments are the same as those for the ``scan()`` - method; see that method's documentation for details. - """ + Any distributions found on `search_path` are added to the distribution + map. `search_path` should be a sequence of ``sys.path`` items. If not + supplied, ``sys.path`` is used. + `platform` is an optional string specifying the name of the platform + that platform-specific distributions must be compatible with. If + unspecified, it defaults to the current platform. `python` is an + optional string naming the desired version of Python (e.g. ``'2.4'``); + it defaults to the current version. + + You may explicitly set `platform` (and/or `python`) to ``None`` if you + wish to map *all* distributions, not just those compatible with the + running platform or Python version. + """ self._distmap = {} self._cache = {} - self.scan(search_path,platform,python) + self.platform = platform + self.python = python + self.scan(search_path) + + def can_add(self, dist): + """Is distribution `dist` acceptable for this collection?""" + return (self.python is None or dist.py_version==self.python) \ + and compatible_platforms(dist.platform,self.platform) def __iter__(self): """Iterate over distribution keys""" @@ -143,8 +160,7 @@ """Has a distribution named `name` ever been added to this map?""" return name.lower() in self._distmap - def __len__(self): - return len(self._distmap) + def __len__(self): return len(self._distmap) def get(self,key,default=None): """Return ``self[key]`` if `key` in self, otherwise return `default`""" @@ -153,35 +169,20 @@ else: return default - def scan(self, search_path=None, platform=get_platform(), python=PY_MAJOR): + def scan(self, search_path=None): """Scan `search_path` for distributions usable on `platform` Any distributions found are added to the distribution map. `search_path` should be a sequence of ``sys.path`` items. If not - supplied, ``sys.path`` is used. `platform` is an optional string - specifying the name of the platform that platform-specific - distributions must be compatible with. If unspecified, it defaults to - the current platform. `python` is an optional string naming the - desired version of Python (e.g. ``'2.4'``); it defaults to the current - version. - - You may explicitly set `platform` (and/or `python`) to ``None`` if you - wish to map *all* distributions, not just those compatible with the - running platform or Python version. + supplied, ``sys.path`` is used. Only distributions conforming to + the platform/python version defined at initialization are added. """ - if search_path is None: search_path = sys.path - add = self.add - for item in search_path: for dist in find_distributions(item): - if python is not None and dist.py_version!=python: - continue - if compatible_platforms(dist.platform, platform): - add(dist) - + self.add(dist) def __getitem__(self,key): """Return a newest-to-oldest list of distributions for the given key @@ -189,7 +190,6 @@ The returned list may be modified in-place, e.g. for narrowing down usable distributions. """ - try: return self._cache[key] except KeyError: @@ -204,10 +204,11 @@ return self._cache[key] def add(self,dist): - """Add `dist` to the distribution map""" - self._distmap.setdefault(dist.key,[]).append(dist) - if dist.key in self._cache: - _sort_dists(self._cache[dist.key]) + """Add `dist` to the distribution map, only if it's suitable""" + if self.can_add(dist): + self._distmap.setdefault(dist.key,[]).append(dist) + if dist.key in self._cache: + _sort_dists(self._cache[dist.key]) def remove(self,dist): """Remove `dist` from the distribution map""" @@ -241,8 +242,7 @@ for dist in distros: if dist in requirement: return dist - - return self.obtain(requirement) # as a last resort, try and download + return self.obtain(requirement) # try and download def resolve(self, requirements, path=None): """List all distributions needed to (recursively) meet requirements""" @@ -1132,7 +1132,7 @@ #@classmethod def from_filename(cls,filename,metadata=None): - name,version,py_version,platform = [None,None,PY_MAJOR,None] + name,version,py_version,platform = [None]*4 basename,ext = os.path.splitext(os.path.basename(filename)) if ext.lower()==".egg": match = EGG_NAME(basename) @@ -1142,7 +1142,7 @@ ) return cls( filename, metadata, name=name, version=version, - py_version=py_version, platform=platform + py_version=py_version or PY_MAJOR, platform=platform ) from_filename = classmethod(from_filename) From pje at users.sourceforge.net Sun Jun 5 03:20:12 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 04 Jun 2005 18:20:12 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/tests test_resources.py, 1.10, 1.11 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2589/setuptools/tests Modified Files: test_resources.py Log Message: Make ``AvailableDistributions`` keep track of the desired platform/python. Add a ``can_add()`` method that indicates whether a distribution matches the collection's desired platform and Python version. Fix a bug in ``Distribution.from_filename()`` when the filename has no Python version. Index: test_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests/test_resources.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- test_resources.py 23 May 2005 01:56:27 -0000 1.10 +++ test_resources.py 5 Jun 2005 01:20:09 -0000 1.11 @@ -23,7 +23,7 @@ def testCollection(self): # empty path should produce no distributions - ad = AvailableDistributions([]) + ad = AvailableDistributions([], python=None) self.assertEqual(list(ad), []) self.assertEqual(len(ad),0) self.assertEqual(ad.get('FooPkg'),None) From pje at users.sourceforge.net Sun Jun 5 03:24:32 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 04 Jun 2005 18:24:32 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools easy_install.py, 1.11, 1.12 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4123 Modified Files: easy_install.py Log Message: Partial support for autolocation of packages via link harvesting. Still needs a little work to get PyPI support working, and automatic dependency resolution. But for now you can do: easy_install -s http://example.com/download.html SomePackage And easy_install will find the best-looking download link on the page for instaling SomePackage. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- easy_install.py 4 Jun 2005 21:35:13 -0000 1.11 +++ easy_install.py 5 Jun 2005 01:24:30 -0000 1.12 @@ -39,13 +39,140 @@ +EXTENSIONS = ( + (EGG_DIST, ".egg"), + (SOURCE_DIST, ".tar.gz"), + (SOURCE_DIST, ".tar.bz2"), + (SOURCE_DIST, ".tar"), + (SOURCE_DIST, ".zip"), + (SOURCE_DIST, ".tgz"), +) + +class URLDistribution(Distribution): + """A distribution that has not been installed""" + + def __init__(self, url, metadata=None): + path = urlparse.urlparse(url)[2] + base = path.split('/')[-1] + + for typecode, ext in EXTENSIONS: + if base.endswith(ext): + base = base[:-len(ext)] + break + else: + raise DistributionNotFound(url) + + self.typecode = typecode + name, version, py_version, platform = [None]*4 + match = pkg_resources.EGG_NAME(base) + if match: + name,version,py_version,platform = match.group( + 'name','ver','pyver','plat' + ) + else: + name = base + Distribution.__init__(self, + url, metadata=metadata, name=name, version=version or "0", + py_version=py_version or pkg_resources.PY_MAJOR, platform=platform + ) + + + + + +class PackageIndex(AvailableDistributions): + """A distribution index that scans web pages for download URLs""" + + def __init__(self,index_url="http://www.python.org/pypi",*args,**kw): + AvailableDistributions.__init__(self,*args,**kw) + self.index_url = index_url + self.scanned_urls = {} + + def scan_url(self, url): + self.process_url(url, True) + + def process_url(self, url, retrieve=False): + if url in self.scanned_urls: + return + try: + dist = URLDistribution(url) + except DistributionNotFound: # not a distro, so scan the page + if not retrieve: + return # unless we're skipping retrieval + else: + # It's a distro, just process it + self.scanned_urls[url] = True + self.add(dist) # XXX should check py_ver/platform! + return + + f = urllib.urlopen(url) + self.scanned_urls[url] = True + + if 'html' not in f.headers['content-type'].lower(): + f.close() # not html, we can't process it + return + url = f.url # handle redirects + href = re.compile(r"""href\s*=\s*['"]?([^'"> ]+)""", re.I) + page = f.read() + f.close() + for match in href.finditer(page): + link = urlparse.urljoin(url, match.group(1)) + self.process_url(link) + + + + def obtain(self,requirement): + self.find_packages(requirement) + for dist in self.get(requirement.key, ()): + if dist in requirement: + return dist + + def find_packages(self,requirement): + pass # XXX process PyPI entries for package + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + class Installer: """Manage a download/build/install process""" pth_file = None cleanup = False - def __init__(self, instdir=None, zip_ok=False, multi=None, tmpdir=None): + def __init__(self, + instdir=None, zip_ok=False, multi=None, tmpdir=None, index=None + ): + if index is None: + index = AvailableDistributions() if tmpdir is None: tmpdir = tempfile.mkdtemp(prefix="easy_install-") self.cleanup = True @@ -67,7 +194,7 @@ raise RuntimeError( "Can't do single-version installs outside site-packages" ) - + self.index = index self.instdir = instdir self.zip_ok = zip_ok self.multi = multi @@ -79,7 +206,6 @@ def __del__(self): self.close() - def samefile(self,p1,p2): if hasattr(os.path,'samefile') and ( os.path.exists(p1) and os.path.exists(p2) @@ -90,7 +216,6 @@ os.path.normpath(os.path.normcase(p2)) ) - def download(self, spec): """Locate and/or download or `spec`, returning a local filename""" if isinstance(spec,Requirement): @@ -114,12 +239,10 @@ ) # process a Requirement - dist = AvailableDistributions().best_match(spec,[]) - if dist is not None and dist.path.endswith('.egg'): - return dist.path - - return self.download(self._find_package(spec)) - + dist = self.index.best_match(spec,[]) + if dist is not None: + return self.download(dist.path) + return None def install_eggs(self, dist_filename): # .egg dirs or files are already built, so just return them @@ -231,10 +354,10 @@ sys.argv[:] = save_argv - def _find_package(self, req): - # TODO: search here for a distro to download, matching Requirement - # 'req' and return the package URL or filename - raise DistributionNotFound(spec) + + + + @@ -277,7 +400,7 @@ else: metadata = EggMetadata(zipimport.zipimporter(destination)) dist = Distribution.from_filename(destination,metadata=metadata) - + self.index.add(dist) if self.pth_file is not None: map(self.pth_file.remove, self.pth_file.get(dist.key,())) # drop old if not self.multi: @@ -314,7 +437,7 @@ else: raise - if headers['content-type'].lower().startswith('text/html'): + if 'html' in headers['content-type'].lower(): return self._download_html(url, headers, filename) # and return its filename @@ -657,7 +780,9 @@ URL_SCHEME = re.compile('([-+.a-z0-9]{2,}):',re.I).match def main(argv, factory=Installer): + from optparse import OptionParser + parser = OptionParser(usage = "usage: %prog [options] url [url...]") parser.add_option("-d", "--install-dir", dest="instdir", default=None, help="install package to DIR", metavar="DIR") @@ -673,28 +798,55 @@ parser.add_option("-b", "--build-directory", dest="tmpdir", metavar="DIR", default=None, help="download/extract/build in DIR; keep the results") + + parser.add_option("-u", "--index-url", dest="index_url", metavar="URL", + default="http://www.python.org/pypi", + help="Base URL of Python Package Index") + + parser.add_option("-s", "--scan-url", dest="scan_urls", metavar="URL", + action="append", + help="Additional URL(s) to search for packages") + (options, args) = parser.parse_args() + + if not args: + parser.error("No urls, filenames, or requirements specified") + elif len(args)>1 and options.tmpdir is not None: + parser.error("Build directory can only be set when using one URL") + + + + + try: - if not args: - parser.error("No urls, filenames, or requirements specified") - elif len(args)>1 and options.tmpdir is not None: - parser.error("Build directory can only be set when using one URL") + index = PackageIndex(options.index_url) + if options.scan_urls: + for url in options.scan_urls: + index.scan_url(url) + for spec in args: inst = factory( - options.instdir, options.zip_ok, options.multi, options.tmpdir + options.instdir, options.zip_ok, options.multi, options.tmpdir, + index ) try: print "Downloading", spec downloaded = inst.download(spec) + if downloaded is None: + raise RuntimeError( + "Could not find distribution for %r" % spec + ) print "Installing", os.path.basename(downloaded) for dist in inst.install_eggs(downloaded): print inst.installation_report(dist) finally: inst.close() + except RuntimeError, v: print >>sys.stderr,"error:",v sys.exit(1) + if __name__ == '__main__': main(sys.argv[1:]) @@ -707,32 +859,3 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - From birkenfeld at users.sourceforge.net Sun Jun 5 12:57:02 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 05 Jun 2005 03:57:02 -0700 Subject: [Python-checkins] python/dist/src/Doc/ext extending.tex,1.32,1.33 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ext In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5140/Doc/ext Modified Files: extending.tex Log Message: Bug #1184380: extending&embedding example broken Index: extending.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ext/extending.tex,v retrieving revision 1.32 retrieving revision 1.33 diff -u -d -r1.32 -r1.33 --- extending.tex 13 Feb 2005 22:50:02 -0000 1.32 +++ extending.tex 5 Jun 2005 10:56:59 -0000 1.33 @@ -1306,7 +1306,7 @@ /* C API functions */ #define PySpam_System_NUM 0 #define PySpam_System_RETURN int -#define PySpam_System_PROTO (char *command) +#define PySpam_System_PROTO (const char *command) /* Total number of C API pointers */ #define PySpam_API_pointers 1 From birkenfeld at users.sourceforge.net Sun Jun 5 12:57:02 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 05 Jun 2005 03:57:02 -0700 Subject: [Python-checkins] python/dist/src/Doc/ext extending.tex, 1.30.4.2, 1.30.4.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ext In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5089/Doc/ext Modified Files: Tag: release24-maint extending.tex Log Message: Bug #1184380: extending&embedding example broken Index: extending.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ext/extending.tex,v retrieving revision 1.30.4.2 retrieving revision 1.30.4.3 diff -u -d -r1.30.4.2 -r1.30.4.3 --- extending.tex 13 Feb 2005 22:56:35 -0000 1.30.4.2 +++ extending.tex 5 Jun 2005 10:57:00 -0000 1.30.4.3 @@ -1306,7 +1306,7 @@ /* C API functions */ #define PySpam_System_NUM 0 #define PySpam_System_RETURN int -#define PySpam_System_PROTO (char *command) +#define PySpam_System_PROTO (const char *command) /* Total number of C API pointers */ #define PySpam_API_pointers 1 From pje at users.sourceforge.net Sun Jun 5 20:59:39 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 05 Jun 2005 11:59:39 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/tests test_resources.py, 1.11, 1.12 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21981/setuptools/tests Modified Files: test_resources.py Log Message: Add "safe_name" and "safe_version" functions to allow sanitizing of distribution names and versions in arbitrary packages that might be built using EasyInstall. Index: test_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests/test_resources.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- test_resources.py 5 Jun 2005 01:20:09 -0000 1.11 +++ test_resources.py 5 Jun 2005 18:59:37 -0000 1.12 @@ -319,13 +319,20 @@ ) self.assertRaises(ValueError,list,pkg_resources.split_sections("[foo")) + def testSafeName(self): + self.assertEqual(safe_name("adns-python"), "adns-python") + self.assertEqual(safe_name("WSGI Utils"), "WSGI-Utils") + self.assertEqual(safe_name("WSGI Utils"), "WSGI-Utils") + self.assertEqual(safe_name("Money$$$Maker"), "Money-Maker") + self.assertEqual(safe_name("peak.web"), "peak-web") - - - - - - + def testSafeVersion(self): + self.assertEqual(safe_version("1.2-1"), "1.2-1") + self.assertEqual(safe_version("1.2 alpha"), "1.2.alpha") + self.assertEqual(safe_version("2.3.4 20050521"), "2.3.4.20050521") + self.assertEqual(safe_version("Money$$$Maker"), "Money-Maker") + self.assertEqual(safe_version("peak.web"), "peak.web") + def testSimpleRequirements(self): self.assertEqual( list(parse_requirements('Twis-Ted>=1.2-1')), @@ -358,14 +365,7 @@ c('0pre1', '0.0c1') c('0.0.0preview1', '0c1') c('0.0c1', '0rc1') - - - - - - - - + c('1.2a1', '1.2.a.1'); c('1.2...a', '1.2a') def testVersionOrdering(self): def c(s1,s2): From pje at users.sourceforge.net Sun Jun 5 20:59:39 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 05 Jun 2005 11:59:39 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.25, 1.26 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21981 Modified Files: pkg_resources.py Log Message: Add "safe_name" and "safe_version" functions to allow sanitizing of distribution names and versions in arbitrary packages that might be built using EasyInstall. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- pkg_resources.py 5 Jun 2005 01:20:09 -0000 1.25 +++ pkg_resources.py 5 Jun 2005 18:59:36 -0000 1.26 @@ -22,11 +22,23 @@ 'InvalidOption', 'Distribution', 'Requirement', 'yield_lines', 'get_importer', 'find_distributions', 'find_on_path', 'register_finder', 'split_sections', 'declare_namespace', 'register_namespace_handler', + 'safe_name', 'safe_version' ] import sys, os, zipimport, time, re, imp from sets import ImmutableSet + + + + + + + + + + + class ResolutionError(Exception): """Abstract base for dependency resolution errors""" @@ -57,6 +69,17 @@ loader = getattr(module, '__loader__', None) return _find_adapter(_provider_factories, loader)(module) + + + + + + + + + + + def get_platform(): """Return this platform's string for platform-specific distributions @@ -80,6 +103,24 @@ return False + + + + + + + + + + + + + + + + + + class IMetadataProvider: def has_metadata(name): @@ -424,22 +465,22 @@ dist.install_on(sys.path) +def safe_name(name): + """Convert an arbitrary string to a standard distribution name + Any runs of non-alphanumeric characters are replaced with a single '-'. + """ + return re.sub('[^A-Za-z0-9]+', '-', name) +def safe_version(version): + """Convert an arbitrary string to a standard version string - - - - - - - - - - - - + Spaces become dots, and all other non-alphanumeric characters become + dashes, with runs of multiple dashes condensed to a single dash. + """ + version = version.replace(' ','.') + return re.sub('[^A-Za-z0-9.]+', '-', version) From pje at users.sourceforge.net Sun Jun 5 20:59:39 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 05 Jun 2005 11:59:39 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21981/setuptools/command Modified Files: bdist_egg.py Log Message: Add "safe_name" and "safe_version" functions to allow sanitizing of distribution names and versions in arbitrary packages that might be built using EasyInstall. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- bdist_egg.py 30 May 2005 23:22:10 -0000 1.12 +++ bdist_egg.py 5 Jun 2005 18:59:36 -0000 1.13 @@ -10,7 +10,7 @@ from distutils.sysconfig import get_python_version, get_python_lib from distutils.errors import * from distutils import log -from pkg_resources import parse_requirements, get_platform +from pkg_resources import parse_requirements, get_platform, safe_name, safe_version class bdist_egg(Command): description = "create an \"egg\" distribution" @@ -54,7 +54,7 @@ self.tag_date = 0 def finalize_options (self): - self.egg_name = self.distribution.get_name().replace(' ','-') + self.egg_name = safe_name(self.distribution.get_name()) self.egg_version = self.tagged_version() try: list( @@ -122,7 +122,6 @@ self.distribution.data_files = old def run(self): - if not self.skip_build: self.run_command('build') @@ -161,6 +160,7 @@ self.egg_version.replace('-','_'), get_python_version()) if ext_outputs: archive_basename += "-" + self.plat_name + ext_outputs = [out.replace(os.sep,'/') for out in ext_outputs] # OS/2 objects to any ":" characters in a filename (such as when # a timestamp is used in a version) so change them to underscores. @@ -223,7 +223,7 @@ import time version += time.strftime("-%Y%m%d") - return version + return safe_version(version) def get_svn_revision(self): stdin, stdout = os.popen4("svn info"); stdin.close() From vsajip at users.sourceforge.net Sun Jun 5 22:39:05 2005 From: vsajip at users.sourceforge.net (vsajip@users.sourceforge.net) Date: Sun, 05 Jun 2005 13:39:05 -0700 Subject: [Python-checkins] python/dist/src/Lib/logging config.py,1.12,1.13 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/logging In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11960 Modified Files: config.py Log Message: Documentation clarified re. config socket listener protocol Index: config.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/logging/config.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- config.py 13 Mar 2005 09:57:46 -0000 1.12 +++ config.py 5 Jun 2005 20:39:02 -0000 1.13 @@ -225,9 +225,9 @@ """ Handle a request. - Each request is expected to be a 4-byte length, - followed by the config file. Uses fileConfig() to do the - grunt work. + Each request is expected to be a 4-byte length, packed using + struct.pack(">L", n), followed by the config file. + Uses fileConfig() to do the grunt work. """ import tempfile try: From vsajip at users.sourceforge.net Sun Jun 5 22:39:39 2005 From: vsajip at users.sourceforge.net (vsajip@users.sourceforge.net) Date: Sun, 05 Jun 2005 13:39:39 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib liblogging.tex, 1.36, 1.37 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12431/lib Modified Files: liblogging.tex Log Message: Documentation clarified re. config socket listener protocol Index: liblogging.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/liblogging.tex,v retrieving revision 1.36 retrieving revision 1.37 diff -u -d -r1.36 -r1.37 --- liblogging.tex 31 Mar 2005 20:18:06 -0000 1.36 +++ liblogging.tex 5 Jun 2005 20:39:36 -0000 1.37 @@ -1396,7 +1396,10 @@ will be sent as a file suitable for processing by \function{fileConfig()}. Returns a \class{Thread} instance on which you can call \method{start()} to start the server, and which you can \method{join()} when appropriate. -To stop the server, call \function{stopListening()}. +To stop the server, call \function{stopListening()}. To send a configuration +to the socket, read in the configuration file and send it to the socket +as a string of bytes preceded by a four-byte length packed in binary using +struct.\code{pack(">L", n)}. \end{funcdesc} \begin{funcdesc}{stopListening}{} From pje at users.sourceforge.net Sun Jun 5 23:33:53 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 05 Jun 2005 14:33:53 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/tests test_resources.py, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11175/setuptools/tests Modified Files: test_resources.py Log Message: Implement PyPI screenscraping for EasyInstall. Fix a bug in requirement version checking. Document new options for screen scraping. Index: test_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/tests/test_resources.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- test_resources.py 5 Jun 2005 18:59:37 -0000 1.12 +++ test_resources.py 5 Jun 2005 21:33:51 -0000 1.13 @@ -258,12 +258,19 @@ ImmutableSet(["foo","bar"]))) ) + def testVersionEquality(self): + r1 = Requirement.parse("setuptools==0.3a2") + r2 = Requirement.parse("setuptools!=0.3a4") + d = Distribution.from_filename + self.failIf(d("setuptools-0.3a4.egg") in r1) + self.failIf(d("setuptools-0.3a1.egg") in r1) + self.failIf(d("setuptools-0.3a4.egg") in r2) - - - - + self.failUnless(d("setuptools-0.3a2.egg") in r1) + self.failUnless(d("setuptools-0.3a2.egg") in r2) + self.failUnless(d("setuptools-0.3a3.egg") in r2) + self.failUnless(d("setuptools-0.3a5.egg") in r2) From pje at users.sourceforge.net Sun Jun 5 23:33:54 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 05 Jun 2005 14:33:54 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.3, 1.4 easy_install.py, 1.12, 1.13 pkg_resources.py, 1.26, 1.27 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11175 Modified Files: EasyInstall.txt easy_install.py pkg_resources.py Log Message: Implement PyPI screenscraping for EasyInstall. Fix a bug in requirement version checking. Document new options for screen scraping. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- EasyInstall.txt 5 Jun 2005 00:48:11 -0000 1.3 +++ EasyInstall.txt 5 Jun 2005 21:33:50 -0000 1.4 @@ -210,10 +210,42 @@ URL or filename, so that the installer will not be confused by the presence of multiple ``setup.py`` files in the build directory. +``--scan-url=URL, -s URL`` (New in 0.4a1) + Scan the specified "download page" for direct links to downloadable eggs or + source distributions. Any usable packages will be downloaded if they are + required by a command line argument. For example, this:: + + easy_install -s http://peak.telecommunity.com/dist PyProtocols + + will download and install the latest version of PyProtocols linked from + the PEAK downloads page, but ignore the other download links on that page. + + You may use this option more than once, to list multiple download pages. + If all requested packages can be found using the specified download pages, + the Python Package Index will *not* be consulted. + +``--index-url=URL, -u URL`` (New in 0.4a1) + Specifies the base URL of the Python Package Index. The default is + http://www.python.org/pypi if not specified. When a package is requested + that is not locally available or linked from a ``--scan-url`` download + page, the package index will be searched for download pages for the needed + package, and those download pages will be searched for links to download + an egg or source distribution. + Release Notes/Change History ============================ +0.4a1 + * Added ``--scan-url`` and ``--index-url`` options, to scan download pages + and search PyPI for needed packages. + + * Fixed a bug in requirements processing for exact versions (i.e. ``==`` and + ``!=``) when only one condition was included. + + * Added ``safe_name()`` and ``safe_version()`` APIs to clean up handling of + arbitrary distribution names and versions found on PyPI. + 0.3a4 * ``pkg_resources`` now supports resource directories, not just the resources in them. In particular, there are ``resource_listdir()`` and Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- easy_install.py 5 Jun 2005 01:24:30 -0000 1.12 +++ easy_install.py 5 Jun 2005 21:33:51 -0000 1.13 @@ -28,98 +28,133 @@ from distutils.sysconfig import get_python_lib from shutil import rmtree # must have, because it can be called from __del__ from pkg_resources import * - _os = sys.modules[os.name] _open = open +class Opener(urllib.FancyURLopener): + def http_error_default(self, url, fp, errcode, errmsg, headers): + """Default error handling -- don't raise an exception.""" + info = urllib.addinfourl(fp, headers, "http:" + url) + info.status, info.reason = errcode, errmsg + return info +opener = Opener() +HREF = re.compile(r"""href\s*=\s*['"]?([^'"> ]+)""", re.I) +EXTENSIONS = ".tar.gz .tar.bz2 .tar .zip .tgz".split() +def distros_for_url(url, metadata=None): + """Yield egg or source distribution objects that might be found at a URL""" + path = urlparse.urlparse(url)[2] + base = urllib.unquote(path.split('/')[-1]) + if base.endswith('.egg'): + dist = Distribution.from_filename(base, metadata) + dist.path = url + yield dist + return # only one, unambiguous interpretation + for ext in EXTENSIONS: + if base.endswith(ext): + base = base[:-len(ext)] + break + else: + return # no extension matched + # Generate alternative interpretations of a source distro name + # Because some packages are ambiguous as to name/versions split + # e.g. "adns-python-1.1.0", "egenix-mx-commercial", etc. + # So, we generate each possible interepretation (e.g. "adns, python-1.1.0" + # "adns-python, 1.1.0", and "adns-python-1.1.0, no version"). In practice, + # the spurious interpretations should be ignored, because in the event + # there's also an "adns" package, the spurious "python-1.1.0" version will + # compare lower than any numeric version number, and is therefore unlikely + # to match a request for it. It's still a potential problem, though, and + # in the long run PyPI and the distutils should go for "safe" names and + # versions in source distribution names. -EXTENSIONS = ( - (EGG_DIST, ".egg"), - (SOURCE_DIST, ".tar.gz"), - (SOURCE_DIST, ".tar.bz2"), - (SOURCE_DIST, ".tar"), - (SOURCE_DIST, ".zip"), - (SOURCE_DIST, ".tgz"), -) - -class URLDistribution(Distribution): - """A distribution that has not been installed""" - - def __init__(self, url, metadata=None): - path = urlparse.urlparse(url)[2] - base = path.split('/')[-1] - - for typecode, ext in EXTENSIONS: - if base.endswith(ext): - base = base[:-len(ext)] - break - else: - raise DistributionNotFound(url) - - self.typecode = typecode - name, version, py_version, platform = [None]*4 - match = pkg_resources.EGG_NAME(base) - if match: - name,version,py_version,platform = match.group( - 'name','ver','pyver','plat' - ) - else: - name = base - Distribution.__init__(self, - url, metadata=metadata, name=name, version=version or "0", - py_version=py_version or pkg_resources.PY_MAJOR, platform=platform + parts = base.split('-') + for p in range(1,len(parts)+1): + yield Distribution( + url, metadata, '-'.join(parts[:p]), '-'.join(parts[p:]), + distro_type = SOURCE_DIST ) - - - - class PackageIndex(AvailableDistributions): """A distribution index that scans web pages for download URLs""" def __init__(self,index_url="http://www.python.org/pypi",*args,**kw): AvailableDistributions.__init__(self,*args,**kw) - self.index_url = index_url + self.index_url = index_url + "/"[:not index_url.endswith('/')] self.scanned_urls = {} + self.fetched_urls = {} + self.package_pages = {} def scan_url(self, url): self.process_url(url, True) def process_url(self, url, retrieve=False): - if url in self.scanned_urls: - return - try: - dist = URLDistribution(url) - except DistributionNotFound: # not a distro, so scan the page - if not retrieve: - return # unless we're skipping retrieval - else: - # It's a distro, just process it - self.scanned_urls[url] = True - self.add(dist) # XXX should check py_ver/platform! + if url in self.scanned_urls and not retrieve: return - f = urllib.urlopen(url) self.scanned_urls[url] = True + dists = list(distros_for_url(url)) + map(self.add, dists) + + if dists or not retrieve or url in self.fetched_urls: + # don't need the actual page + return + f = opener.open(url) + self.fetched_urls[url] = self.fetched_urls[f.url] = True if 'html' not in f.headers['content-type'].lower(): f.close() # not html, we can't process it return - url = f.url # handle redirects - href = re.compile(r"""href\s*=\s*['"]?([^'"> ]+)""", re.I) + + base = f.url # handle redirects page = f.read() f.close() - for match in href.finditer(page): - link = urlparse.urljoin(url, match.group(1)) - self.process_url(link) + if url.startswith(self.index_url): + self.process_index(url, page) + else: + for match in HREF.finditer(page): + link = urlparse.urljoin(base, match.group(1)) + self.process_url(link) + def find_packages(self,requirement): + self.scan_url(self.index_url + requirement.distname) + if not self.package_pages.get(requirement.key): + # We couldn't find the target package, so search the index page too + self.scan_url(self.index_url) + for url in self.package_pages.get(requirement.key,()): + # scan each page that might be related to the desired package + self.scan_url(url) + def process_index(self,url,page): + def scan(link): + if link.startswith(self.index_url): + parts = map( + urllib.unquote, link[len(self.index_url):].split('/') + ) + if len(parts)==2: + # it's a package page, sanitize and index it + pkg = safe_name(parts[0]) + ver = safe_version(parts[1]) + self.package_pages.setdefault(pkg.lower(),{})[link] = True + if url==self.index_url or 'Index of Packages' in page: + # process an index page into the package-page index + for match in HREF.finditer(page): + scan( urlparse.urljoin(url, match.group(1)) ) + else: + scan(url) # ensure this page is in the page index + # process individual package page + for tag in ("Home Page", "Download URL"): + pos = page.find(tag) + if pos!=-1: + match = HREF.search(page,pos) + if match: + # Process the found URL + self.scan_url(urlparse.urljoin(url, match.group(1))) def obtain(self,requirement): self.find_packages(requirement) @@ -127,41 +162,6 @@ if dist in requirement: return dist - def find_packages(self,requirement): - pass # XXX process PyPI entries for package - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - class Installer: """Manage a download/build/install process""" @@ -801,11 +801,11 @@ parser.add_option("-u", "--index-url", dest="index_url", metavar="URL", default="http://www.python.org/pypi", - help="Base URL of Python Package Index") + help="base URL of Python Package Index") parser.add_option("-s", "--scan-url", dest="scan_urls", metavar="URL", action="append", - help="Additional URL(s) to search for packages") + help="additional URL(s) to search for packages") (options, args) = parser.parse_args() Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- pkg_resources.py 5 Jun 2005 18:59:36 -0000 1.26 +++ pkg_resources.py 5 Jun 2005 21:33:51 -0000 1.27 @@ -54,6 +54,9 @@ _provider_factories = {} PY_MAJOR = sys.version[:3] +EGG_DIST = 2 +SOURCE_DIST = 1 + def register_loader_type(loader_type, provider_factory): """Register `provider_factory` to make providers for `loader_type` @@ -77,9 +80,6 @@ - - - def get_platform(): """Return this platform's string for platform-specific distributions @@ -1146,23 +1146,21 @@ -EGG_DIST = 2 -SOURCE_DIST = 1 - class Distribution(object): """Wrap an actual or potential sys.path entry w/metadata""" - typecode = EGG_DIST + def __init__(self, path_str, metadata=None, name=None, version=None, - py_version=PY_MAJOR, platform=None + py_version=PY_MAJOR, platform=None, distro_type = EGG_DIST ): if name: - self.name = name.replace('_','-') + self.name = safe_name(name) if version: - self._version = version.replace('_','-') + self._version = safe_version(version) self.py_version = py_version self.platform = platform self.path = path_str + self.distro_type = distro_type self.metadata = metadata def installed_on(self,path=None): @@ -1187,6 +1185,8 @@ ) from_filename = classmethod(from_filename) + + # These properties have to be lazy so that we don't have to load any # metadata until/unless it's actually needed. (i.e., some distributions # may not know their name or version without loading PKG-INFO) @@ -1330,7 +1330,7 @@ def _sort_dists(dists): - tmp = [(dist.version,dist.typecode,dist) for dist in dists] + tmp = [(dist.version,dist.distro_type,dist) for dist in dists] tmp.sort() dists[::-1] = [d for v,t,d in tmp] @@ -1382,16 +1382,16 @@ item = item.parsed_version elif isinstance(item,basestring): item = parse_version(item) - last = True + last = None for parsed,trans,op,ver in self.index: action = trans[cmp(item,parsed)] if action=='F': return False elif action=='T': return True elif action=='+': last = True - elif action=='-': last = False + elif action=='-' or last is None: last = False + if last is None: last = True # no rules encountered return last - def __hash__(self): return self.__hash @@ -1414,7 +1414,7 @@ '>' : 'F+F', '>=': 'T+F', '==': 'T..', - '!=': 'F..', + '!=': 'F++', } From pje at users.sourceforge.net Sun Jun 5 23:55:45 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 05 Jun 2005 14:55:45 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.4, 1.5 setup.py, 1.8, 1.9 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24847 Modified Files: EasyInstall.txt setup.py Log Message: Update docs for PyPI support, prep for 0.4a1 release Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- EasyInstall.txt 5 Jun 2005 21:33:50 -0000 1.4 +++ EasyInstall.txt 5 Jun 2005 21:55:41 -0000 1.5 @@ -24,9 +24,9 @@ Unix-like Systems (including Mac OS X and Cygwin) Download either the `Python 2.3 easy_install shell script - `_ or the + `_ or the `Python 2.4 easy_install shell script - `_. Place + `_. Place the file somewhere on your PATH, after renaming it to ``easy_install``. Note that these scripts assume you have ``python2.3`` or ``python2.4`` accessible via the ``PATH`` environment variable. Then, you can use ``easy_install`` to @@ -34,15 +34,15 @@ your Python version:: # Python 2.3 - easy_install http://peak.telecommunity.com/dist/setuptools-0.3a4-py2.3.egg + easy_install http://peak.telecommunity.com/dist/setuptools-0.4a1-py2.3.egg # Python 2.4 - easy_install http://peak.telecommunity.com/dist/setuptools-0.3a4-py2.4.egg + easy_install http://peak.telecommunity.com/dist/setuptools-0.4a1-py2.4.egg All Other Systems Download the `easy_install (aka setuptools) source distribution - `_, and follow the + `_, and follow the normal procedure for installing a source package with distutils. An ``easy_install.py`` script will be installed in the normal location for Python scripts on your platform. In the examples below, you'll need to @@ -60,18 +60,33 @@ __ http://peak.telecommunity.com/DevCenter/PythonEggs -**Example 1**. Download a source distribution, automatically building and -installing it:: +**Example 1**. Install a package by name, searching PyPI for the latest +version, and automatically downloading, building, and installing it:: + + easy_install SQLObject + +**Example 2**. Install a package by name and version from a given +"download page":: + + easy_install -s http://peak.telecommunity.com/dist "setuptools>=0.4a1" + +**Example 3**. Download a source distribution from a specified URL, +automatically building and installing it:: easy_install http://example.com/path/to/MyPackage-1.2.3.tgz -**Example 2**. Install an already-downloaded .egg file:: +**Example 4**. Install an already-downloaded .egg file:: easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg -Easy Install recognizes distutils *source* (not binary) distribution files with -extensions of .tgz, .tar, .tar.gz, .tar.bz2, or .zip. And of course it handles -already-built .egg distributions. +Easy Install accepts URLs, filenames, PyPI package names (i.e., ``distutils`` +"distribution" names), and package+version specifiers. In each case, it will +attempt to locate the latest available version that meets your criteria. + +When downloading or processing downloaded files, Easy Install recognizes +distutils *source* (not binary) distribution files with extensions of .tgz, +.tar, .tar.gz, .tar.bz2, or .zip. And of course it handles already-built .egg +distributions as well. By default, packages are installed to the running Python installation's ``site-packages`` directory, unless you provide the ``-d`` or ``--install-dir`` @@ -92,12 +107,31 @@ ------------------- You don't need to do anything special to upgrade a package: just install the -new version. If you're using ``-m`` or ``--multi`` (or installing outside of -``site-packages``), the runtime system automatically selects the newest -available version of a package. If you're installing to ``site-packages`` and -not using ``-m``, installing a package automatically replaces its older version -in the ``easy-install.pth`` file, so that Python will import the latest version -by default. +new version, either by requesting a specific version, e.g.:: + + easy_install "SomePackage==2.0" + +a version greater than the one you have now:: + + easy_install "SomePackage>2.0" + +or by using a download page, direct download URL, or package filename:: + + easy_install -s http://example.com/downloads ExamplePackage + + easy_install http://example.com/downloads/ExamplePackage-2.0-py2.4.egg + + easy_install my_downloads/ExamplePackage-2.0.tgz + +If you're using ``-m`` or ``--multi`` (or installing outside of +``site-packages``), the ``require()`` function automatically selects the newest +available version of a package that meets your version criteria at runtime, so +installation is the only step needed. + +If you're installing to ``site-packages`` and not using ``-m``, installing a +package automatically replaces any previous version in the ``easy-install.pth`` +file, so that Python will import the most-recently installed version by +default. ``easy_install`` never actually deletes packages (unless you're installing a package with the same name and version number as an existing package), so if @@ -296,6 +330,8 @@ ============ * Support packages that include scripts - -* Automatic package download URL discovery via PyPI/CheeseShop +* Log progress to a logger, with -v and -q options to control verbosity +* Process the installed package's dependencies as well as the base package +* Additional utilities to list/remove/verify packages +* Signature checking? SSL? Ability to suppress PyPI search? Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- setup.py 4 Jun 2005 21:35:13 -0000 1.8 +++ setup.py 5 Jun 2005 21:55:41 -0000 1.9 @@ -7,7 +7,7 @@ setup( name="setuptools", - version="0.3a4", + version="0.4a1", description="Distutils packaging and installation enhancements", author="Phillip J. Eby", From pje at users.sourceforge.net Mon Jun 6 05:30:50 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 05 Jun 2005 20:30:50 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools setup.py, 1.9, 1.10 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11400 Modified Files: setup.py Log Message: Update distribution metadata so 'setup.py register' works; add 'extra_path' so that setuptools can install itself in egg form. Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- setup.py 5 Jun 2005 21:55:41 -0000 1.9 +++ setup.py 6 Jun 2005 03:30:48 -0000 1.10 @@ -1,18 +1,34 @@ #!/usr/bin/env python - """Distutils setup file, used to install or test 'setuptools'""" +VERSION = "0.4a1" from setuptools import setup, find_packages, Require from distutils.version import LooseVersion setup( name="setuptools", - version="0.4a1", + version=VERSION, + + description="Download, build, install, upgrade, and uninstall Python " + "packages -- easily!", - description="Distutils packaging and installation enhancements", author="Phillip J. Eby", author_email="peak at eby-sarna.com", license="PSF or ZPL", + long_description = + "Setuptools enhances the distutils with support for Python Eggs " + "(http://peak.telecommunity.com/DevCenter/PythonEggs) and more. Its " + "'EasyInstall' tool " + "(http://peak.telecommunity.com/DevCenter/EasyInstall) lets you " + "download and install (or cleanly upgrade) Python packages on your " + "system, from source distributions, subversion checkouts, SourceForge " + "download mirrors, or from Python Eggs. Been looking for a CPAN " + "clone for Python? When combined with PyPI, this gets pretty darn " + "close. See the home page and download page for details and docs.", + + keywords = "CPAN PyPI distutils eggs package management", + url = "http://peak.telecommunity.com/PythonEggs", + download_url = "http://peak.telecommunity.com/DevCenter/EasyInstall", test_suite = 'setuptools.tests.test_suite', requires = [ @@ -21,8 +37,46 @@ ), Require('PyUnit', None, 'unittest', "http://pyunit.sf.net/"), ], + + packages = find_packages(), py_modules = ['pkg_resources', 'easy_install'], - scripts = ['easy_install.py'] + scripts = ['easy_install.py'], + extra_path = ('setuptools', 'setuptools-%s.egg' % VERSION), + + classifiers = [f.strip() for f in """ + Development Status :: 3 - Alpha + Intended Audience :: Developers + License :: OSI Approved :: Python Software Foundation License + License :: OSI Approved :: Zope Public License + Operating System :: OS Independent + Programming Language :: Python + Topic :: Software Development :: Libraries :: Python Modules + Topic :: System :: Archiving :: Packaging + Topic :: System :: Systems Administration + Topic :: Utilities + """.splitlines() if f.strip()] ) + + + + + + + + + + + + + + + + + + + + + + From greg at users.sourceforge.net Mon Jun 6 11:52:13 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Mon, 06 Jun 2005 02:52:13 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_associate.py, 1.8, 1.9 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14394 Modified Files: test_associate.py Log Message: test DB.associate using transactions. somewhat related to SF pybsddb bug #1215432 Index: test_associate.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_associate.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- test_associate.py 28 Jun 2004 04:06:49 -0000 1.8 +++ test_associate.py 6 Jun 2005 09:52:10 -0000 1.9 @@ -91,6 +91,8 @@ class AssociateTestCase(unittest.TestCase): keytype = '' + envFlags = 0 + dbFlags = 0 def setUp(self): self.filename = self.__class__.__name__ + '.db' @@ -100,7 +102,7 @@ except os.error: pass self.env = db.DBEnv() self.env.open(homeDir, db.DB_CREATE | db.DB_INIT_MPOOL | - db.DB_INIT_LOCK | db.DB_THREAD) + db.DB_INIT_LOCK | db.DB_THREAD | self.envFlags) def tearDown(self): self.closeDB() @@ -110,17 +112,17 @@ for file in files: os.remove(file) - def addDataToDB(self, d): + def addDataToDB(self, d, txn=None): for key, value in musicdata.items(): if type(self.keytype) == type(''): key = "%02d" % key - d.put(key, string.join(value, '|')) + d.put(key, string.join(value, '|'), txn=txn) - def createDB(self): + def createDB(self, txn=None): self.primary = db.DB(self.env) self.primary.set_get_returns_none(2) self.primary.open(self.filename, "primary", self.dbtype, - db.DB_CREATE | db.DB_THREAD) + db.DB_CREATE | db.DB_THREAD | self.dbFlags, txn=txn) def closeDB(self): self.primary.close() @@ -128,6 +130,7 @@ def getDB(self): return self.primary + def test01_associateWithDB(self): if verbose: print '\n', '-=' * 30 @@ -140,7 +143,7 @@ secDB.set_flags(db.DB_DUP) secDB.set_get_returns_none(2) secDB.open(self.filename, "secondary", db.DB_BTREE, - db.DB_CREATE | db.DB_THREAD) + db.DB_CREATE | db.DB_THREAD | self.dbFlags) self.getDB().associate(secDB, self.getGenre) self.addDataToDB(self.getDB()) @@ -160,7 +163,7 @@ secDB = db.DB(self.env) secDB.set_flags(db.DB_DUP) secDB.open(self.filename, "secondary", db.DB_BTREE, - db.DB_CREATE | db.DB_THREAD) + db.DB_CREATE | db.DB_THREAD | self.dbFlags) # adding the DB_CREATE flag will cause it to index existing records self.getDB().associate(secDB, self.getGenre, db.DB_CREATE) @@ -168,12 +171,12 @@ self.finish_test(secDB) - def finish_test(self, secDB): + def finish_test(self, secDB, txn=None): # 'Blues' should not be in the secondary database - vals = secDB.pget('Blues') + vals = secDB.pget('Blues', txn=txn) assert vals == None, vals - vals = secDB.pget('Unknown') + vals = secDB.pget('Unknown', txn=txn) assert vals[0] == 99 or vals[0] == '99', vals vals[1].index('Unknown') vals[1].index('Unnamed') @@ -181,7 +184,7 @@ if verbose: print "Primary key traversal:" - c = self.getDB().cursor() + c = self.getDB().cursor(txn) count = 0 rec = c.first() while rec is not None: @@ -198,7 +201,7 @@ if verbose: print "Secondary key traversal:" - c = secDB.cursor() + c = secDB.cursor(txn) count = 0 # test cursor pget @@ -246,6 +249,35 @@ dbtype = db.DB_RECNO keytype = 0 +#---------------------------------------------------------------------- + +class AssociateBTreeTxnTestCase(AssociateBTreeTestCase): + envFlags = db.DB_INIT_TXN + dbFlags = 0 + + def test13_associateAutoCommit(self): + if verbose: + print '\n', '-=' * 30 + print "Running %s.test13_associateAutoCommit..." % \ + self.__class__.__name__ + + txn = self.env.txn_begin() + try: + self.createDB(txn=txn) + + secDB = db.DB(self.env) + secDB.set_flags(db.DB_DUP) + secDB.set_get_returns_none(2) + secDB.open(self.filename, "secondary", db.DB_BTREE, + db.DB_CREATE | db.DB_THREAD, txn=txn) + self.getDB().associate(secDB, self.getGenre, txn=txn) + + self.addDataToDB(self.getDB(), txn=txn) + + self.finish_test(secDB, txn=txn) + finally: + txn.commit() + #---------------------------------------------------------------------- @@ -335,6 +367,8 @@ suite.addTest(unittest.makeSuite(AssociateBTreeTestCase)) suite.addTest(unittest.makeSuite(AssociateRecnoTestCase)) + suite.addTest(unittest.makeSuite(AssociateBTreeTxnTestCase)) + suite.addTest(unittest.makeSuite(ShelveAssociateHashTestCase)) suite.addTest(unittest.makeSuite(ShelveAssociateBTreeTestCase)) suite.addTest(unittest.makeSuite(ShelveAssociateRecnoTestCase)) From greg at users.sourceforge.net Mon Jun 6 11:55:09 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Mon, 06 Jun 2005 02:55:09 -0700 Subject: [Python-checkins] python/dist/src/Modules _bsddb.c,1.42,1.43 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16106/extsrc Modified Files: _bsddb.c Log Message: fixes pybsddb SF bug id 1215432. DB.associate() would crash when a DBError was supposed to be raised. needs backport to 2.4.x and possibly 2.3.x. Index: _bsddb.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_bsddb.c,v retrieving revision 1.42 retrieving revision 1.43 diff -u -d -r1.42 -r1.43 --- _bsddb.c 4 Jun 2005 06:46:59 -0000 1.42 +++ _bsddb.c 6 Jun 2005 09:55:06 -0000 1.43 @@ -1181,9 +1181,7 @@ } /* Save a reference to the callback in the secondary DB. */ - if (self->associateCallback != NULL) { - Py_DECREF(self->associateCallback); - } + Py_XDECREF(secondaryDB->associateCallback); Py_INCREF(callback); secondaryDB->associateCallback = callback; secondaryDB->primaryDBType = _DB_get_type(self); @@ -1217,8 +1215,8 @@ MYDB_END_ALLOW_THREADS; if (err) { - Py_DECREF(self->associateCallback); - self->associateCallback = NULL; + Py_XDECREF(secondaryDB->associateCallback); + secondaryDB->associateCallback = NULL; secondaryDB->primaryDBType = 0; } From greg at users.sourceforge.net Mon Jun 6 12:26:27 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Mon, 06 Jun 2005 03:26:27 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_associate.py, 1.9, 1.10 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2717/test Modified Files: test_associate.py Log Message: test case for pybsddb SF bug id 1215432 Index: test_associate.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_associate.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- test_associate.py 6 Jun 2005 09:52:10 -0000 1.9 +++ test_associate.py 6 Jun 2005 10:26:25 -0000 1.10 @@ -88,6 +88,54 @@ #---------------------------------------------------------------------- +class AssociateErrorTestCase(unittest.TestCase): + def setUp(self): + self.filename = self.__class__.__name__ + '.db' + homeDir = os.path.join(os.path.dirname(sys.argv[0]), 'db_home') + self.homeDir = homeDir + try: os.mkdir(homeDir) + except os.error: pass + self.env = db.DBEnv() + self.env.open(homeDir, db.DB_CREATE | db.DB_INIT_MPOOL) + + def tearDown(self): + self.env.close() + import glob + files = glob.glob(os.path.join(self.homeDir, '*')) + for file in files: + os.remove(file) + + + def test00_associateDBError(self): + if verbose: + print '\n', '-=' * 30 + print "Running %s.test00_associateDBError..." % \ + self.__class__.__name__ + + dupDB = db.DB(self.env) + dupDB.set_flags(db.DB_DUP) + dupDB.open(self.filename, "primary", db.DB_BTREE, db.DB_CREATE) + + secDB = db.DB(self.env) + secDB.open(self.filename, "secondary", db.DB_BTREE, db.DB_CREATE) + + # dupDB has been configured to allow duplicates, it can't + # associate with a secondary. BerkeleyDB will return an error. + try: + dupDB.associate(secDB, lambda a, b: a+b) + except db.DBError: + # good + secDB.close() + dupDB.close() + else: + secDB.close() + dupDB.close() + self.fail("DBError exception was expected") + + + +#---------------------------------------------------------------------- + class AssociateTestCase(unittest.TestCase): keytype = '' @@ -363,6 +411,8 @@ suite = unittest.TestSuite() if db.version() >= (3, 3, 11): + suite.addTest(unittest.makeSuite(AssociateErrorTestCase)) + suite.addTest(unittest.makeSuite(AssociateHashTestCase)) suite.addTest(unittest.makeSuite(AssociateBTreeTestCase)) suite.addTest(unittest.makeSuite(AssociateRecnoTestCase)) From greg at users.sourceforge.net Mon Jun 6 12:28:09 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Mon, 06 Jun 2005 03:28:09 -0700 Subject: [Python-checkins] python/dist/src/Modules _bsddb.c,1.43,1.44 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3872 Modified Files: _bsddb.c Log Message: additional sanity check. secondaryDB cannot be closed. Index: _bsddb.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_bsddb.c,v retrieving revision 1.43 retrieving revision 1.44 diff -u -d -r1.43 -r1.44 --- _bsddb.c 6 Jun 2005 09:55:06 -0000 1.43 +++ _bsddb.c 6 Jun 2005 10:28:06 -0000 1.44 @@ -1172,6 +1172,7 @@ makeTypeError("DB", (PyObject*)secondaryDB); return NULL; } + CHECK_DB_NOT_CLOSED(secondaryDB); if (callback == Py_None) { callback = NULL; } From gvanrossum at users.sourceforge.net Mon Jun 6 19:15:20 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Mon, 06 Jun 2005 10:15:20 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.22,1.23 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv26212 Modified Files: pep-0343.txt Log Message: Add Wiki link. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.22 retrieving revision 1.23 diff -u -d -r1.22 -r1.23 --- pep-0343.txt 3 Jun 2005 09:32:57 -0000 1.22 +++ pep-0343.txt 6 Jun 2005 17:15:17 -0000 1.23 @@ -7,7 +7,7 @@ Type: Standards Track Content-Type: text/plain Created: 13-May-2005 -Post-History: +Post-History: 2-Jun-2005 Introduction @@ -20,6 +20,9 @@ [2] and universally approved of. I'm also changing the keyword to 'with'. + On-line discussion of this PEP should take place in the Python + Wiki [3]. + If this PEP is approved, the following PEPs will be rejected due to overlap: @@ -674,6 +677,8 @@ [2] http://mail.python.org/pipermail/python-dev/2005-May/053885.html + [3] http://wiki.python.org/moin/WithStatement + Copyright This document has been placed in the public domain. From greg at users.sourceforge.net Mon Jun 6 19:30:24 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Mon, 06 Jun 2005 10:30:24 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_compare.py, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1164/test Modified Files: test_compare.py Log Message: make the tests that expect uncatchable exceptions from a callback test for them in a roundabout way (catching and parsing stderr) keeps test output clean. Index: test_compare.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_compare.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- test_compare.py 3 Jun 2005 22:40:27 -0000 1.2 +++ test_compare.py 6 Jun 2005 17:30:22 -0000 1.3 @@ -2,21 +2,21 @@ TestCases for python DB Btree key comparison function. """ -import sys, os +import sys, os, re import test_all +from cStringIO import StringIO import unittest from bsddb3 import db -def lexical_cmp (db, left, right): - return cmp (left, right) +lexical_cmp = cmp -def lowercase_cmp(db, left, right): +def lowercase_cmp(left, right): return cmp (left.lower(), right.lower()) def make_reverse_comparator (cmp): - def reverse (db, left, right, delegate=cmp): - return - delegate (db, left, right) + def reverse (left, right, delegate=cmp): + return - delegate (left, right) return reverse _expected_lexical_test_data = ['', 'CCCP', 'a', 'aaa', 'b', 'c', 'cccce', 'ccccf'] @@ -25,7 +25,7 @@ class ComparatorTests (unittest.TestCase): def comparator_test_helper (self, comparator, expected_data): data = expected_data[:] - data.sort (lambda l, r, cmp=comparator: cmp (None, l, r)) + data.sort (comparator) self.failUnless (data == expected_data, "comparator `%s' is not right: %s vs. %s" % (comparator, expected_data, data)) @@ -131,7 +131,7 @@ def test_compare_function_useless (self): self.startTest () - def socialist_comparator (db, l, r): + def socialist_comparator (l, r): return 0 self.createDB (socialist_comparator) self.addDataToDB (['b', 'a', 'd']) @@ -157,40 +157,69 @@ def test_compare_function_incorrect (self): self.startTest () - def bad_comparator (db, l, r): + def bad_comparator (l, r): return 1 - # verify that set_bt_compare checks that comparator(db, '', '') == 0 + # verify that set_bt_compare checks that comparator('', '') == 0 self.assertRaises (TypeError, self.createDB, bad_comparator) self.finishTest () - def test_compare_function_exception (self): + def verifyStderr(self, method, successRe): + """ + Call method() while capturing sys.stderr output internally and + call self.fail() if successRe.search() does not match the stderr + output. This is used to test for uncatchable exceptions. + """ + stdErr = sys.stderr + sys.stderr = StringIO() + try: + method() + finally: + temp = sys.stderr + sys.stderr = stdErr + errorOut = temp.getvalue() + if not successRe.search(errorOut): + self.fail("unexpected stderr output:\n"+errorOut) + + def _test_compare_function_exception (self): self.startTest () - def bad_comparator (db, l, r): + def bad_comparator (l, r): if l == r: # pass the set_bt_compare test return 0 raise RuntimeError, "i'm a naughty comparison function" self.createDB (bad_comparator) - print "\n*** this test should print 2 uncatchable tracebacks ***" - self.addDataToDB (['a', 'b', 'c']) # this should raise, but... - self.finishTest () + #print "\n*** test should print 2 uncatchable tracebacks ***" + self.addDataToDB (['a', 'b', 'c']) # this should raise, but... + self.finishTest () - def test_compare_function_bad_return (self): + def test_compare_function_exception(self): + self.verifyStderr( + self._test_compare_function_exception, + re.compile('(^RuntimeError:.* naughty.*){2}', re.M|re.S) + ) + + def _test_compare_function_bad_return (self): self.startTest () - def bad_comparator (db, l, r): + def bad_comparator (l, r): if l == r: # pass the set_bt_compare test return 0 return l self.createDB (bad_comparator) - print "\n*** this test should print 2 errors about returning an int ***" + #print "\n*** test should print 2 errors about returning an int ***" self.addDataToDB (['a', 'b', 'c']) # this should raise, but... self.finishTest () + def test_compare_function_bad_return(self): + self.verifyStderr( + self._test_compare_function_bad_return, + re.compile('(^TypeError:.* return an int.*){2}', re.M|re.S) + ) + def test_cannot_assign_twice (self): - def my_compare (db, a, b): + def my_compare (a, b): return 0 self.startTest () From greg at users.sourceforge.net Mon Jun 6 19:31:34 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Mon, 06 Jun 2005 10:31:34 -0700 Subject: [Python-checkins] python/dist/src/Modules _bsddb.c,1.44,1.45 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1775 Modified Files: _bsddb.c Log Message: change set_bt_compare() callback comparison function to only take two arguments (left, right) like any sane comparison function. no need to pass in the db object as an argument. Index: _bsddb.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_bsddb.c,v retrieving revision 1.44 retrieving revision 1.45 diff -u -d -r1.44 -r1.45 --- _bsddb.c 6 Jun 2005 10:28:06 -0000 1.44 +++ _bsddb.c 6 Jun 2005 17:31:32 -0000 1.45 @@ -97,7 +97,7 @@ #error "eek! DBVER can't handle minor versions > 9" #endif -#define PY_BSDDB_VERSION "4.3.2" +#define PY_BSDDB_VERSION "4.3.3" static char *rcs_id = "$Id$"; @@ -2019,11 +2019,10 @@ leftObject = PyString_FromStringAndSize (leftKey->data, leftKey->size); rightObject = PyString_FromStringAndSize (rightKey->data, rightKey->size); - args = PyTuple_New (3); + args = PyTuple_New (2); Py_INCREF (self); - PyTuple_SET_ITEM (args, 0, (PyObject *) self); - PyTuple_SET_ITEM (args, 1, leftObject); /* steals reference */ - PyTuple_SET_ITEM (args, 2, rightObject); /* steals reference */ + PyTuple_SET_ITEM (args, 0, leftObject); /* steals reference */ + PyTuple_SET_ITEM (args, 1, rightObject); /* steals reference */ result = PyEval_CallObject (self->btCompareCallback, args); if (result == 0) { @@ -2072,14 +2071,12 @@ * string objects here. verify that it returns an int (0). * err if not. */ - tuple = PyTuple_New (3); - Py_INCREF (self); - PyTuple_SET_ITEM (tuple, 0, (PyObject *) self); + tuple = PyTuple_New (2); emptyStr = PyString_FromStringAndSize (NULL, 0); Py_INCREF(emptyStr); - PyTuple_SET_ITEM (tuple, 1, emptyStr); - PyTuple_SET_ITEM (tuple, 2, emptyStr); /* steals reference */ + PyTuple_SET_ITEM (tuple, 0, emptyStr); + PyTuple_SET_ITEM (tuple, 1, emptyStr); /* steals reference */ result = PyEval_CallObject (comparator, tuple); Py_DECREF (tuple); if (result == 0 || !PyInt_Check(result)) { From greg at users.sourceforge.net Mon Jun 6 19:59:03 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Mon, 06 Jun 2005 10:59:03 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_associate.py, 1.10, 1.11 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14822 Modified Files: test_associate.py Log Message: fix Errors (not Failures) in test cases when running with BerkeleyDB<4.2 Index: test_associate.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_associate.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- test_associate.py 6 Jun 2005 10:26:25 -0000 1.10 +++ test_associate.py 6 Jun 2005 17:59:00 -0000 1.11 @@ -93,8 +93,13 @@ self.filename = self.__class__.__name__ + '.db' homeDir = os.path.join(os.path.dirname(sys.argv[0]), 'db_home') self.homeDir = homeDir - try: os.mkdir(homeDir) - except os.error: pass + try: + os.mkdir(homeDir) + except os.error: + import glob + files = glob.glob(os.path.join(self.homeDir, '*')) + for file in files: + os.remove(file) self.env = db.DBEnv() self.env.open(homeDir, db.DB_CREATE | db.DB_INIT_MPOOL) @@ -146,8 +151,13 @@ self.filename = self.__class__.__name__ + '.db' homeDir = os.path.join(os.path.dirname(sys.argv[0]), 'db_home') self.homeDir = homeDir - try: os.mkdir(homeDir) - except os.error: pass + try: + os.mkdir(homeDir) + except os.error: + import glob + files = glob.glob(os.path.join(self.homeDir, '*')) + for file in files: + os.remove(file) self.env = db.DBEnv() self.env.open(homeDir, db.DB_CREATE | db.DB_INIT_MPOOL | db.DB_INIT_LOCK | db.DB_THREAD | self.envFlags) @@ -167,12 +177,18 @@ d.put(key, string.join(value, '|'), txn=txn) def createDB(self, txn=None): + self.cur = None + self.secDB = None self.primary = db.DB(self.env) self.primary.set_get_returns_none(2) self.primary.open(self.filename, "primary", self.dbtype, db.DB_CREATE | db.DB_THREAD | self.dbFlags, txn=txn) def closeDB(self): + if self.cur: + self.cur.close() + if self.secDB: + self.secDB.close() self.primary.close() def getDB(self): @@ -187,16 +203,16 @@ self.createDB() - secDB = db.DB(self.env) - secDB.set_flags(db.DB_DUP) - secDB.set_get_returns_none(2) - secDB.open(self.filename, "secondary", db.DB_BTREE, + self.secDB = db.DB(self.env) + self.secDB.set_flags(db.DB_DUP) + self.secDB.set_get_returns_none(2) + self.secDB.open(self.filename, "secondary", db.DB_BTREE, db.DB_CREATE | db.DB_THREAD | self.dbFlags) - self.getDB().associate(secDB, self.getGenre) + self.getDB().associate(self.secDB, self.getGenre) self.addDataToDB(self.getDB()) - self.finish_test(secDB) + self.finish_test(self.secDB) def test02_associateAfterDB(self): @@ -208,15 +224,15 @@ self.createDB() self.addDataToDB(self.getDB()) - secDB = db.DB(self.env) - secDB.set_flags(db.DB_DUP) - secDB.open(self.filename, "secondary", db.DB_BTREE, + self.secDB = db.DB(self.env) + self.secDB.set_flags(db.DB_DUP) + self.secDB.open(self.filename, "secondary", db.DB_BTREE, db.DB_CREATE | db.DB_THREAD | self.dbFlags) # adding the DB_CREATE flag will cause it to index existing records - self.getDB().associate(secDB, self.getGenre, db.DB_CREATE) + self.getDB().associate(self.secDB, self.getGenre, db.DB_CREATE) - self.finish_test(secDB) + self.finish_test(self.secDB) def finish_test(self, secDB, txn=None): @@ -232,9 +248,9 @@ if verbose: print "Primary key traversal:" - c = self.getDB().cursor(txn) + self.cur = self.getDB().cursor(txn) count = 0 - rec = c.first() + rec = self.cur.first() while rec is not None: if type(self.keytype) == type(''): assert string.atoi(rec[0]) # for primary db, key is a number @@ -243,36 +259,38 @@ count = count + 1 if verbose: print rec - rec = c.next() + rec = self.cur.next() assert count == len(musicdata) # all items accounted for if verbose: print "Secondary key traversal:" - c = secDB.cursor(txn) + self.cur = secDB.cursor(txn) count = 0 # test cursor pget - vals = c.pget('Unknown', flags=db.DB_LAST) + vals = self.cur.pget('Unknown', flags=db.DB_LAST) assert vals[1] == 99 or vals[1] == '99', vals assert vals[0] == 'Unknown' vals[2].index('Unknown') vals[2].index('Unnamed') vals[2].index('unknown') - vals = c.pget('Unknown', data='wrong value', flags=db.DB_GET_BOTH) + vals = self.cur.pget('Unknown', data='wrong value', flags=db.DB_GET_BOTH) assert vals == None, vals - rec = c.first() + rec = self.cur.first() assert rec[0] == "Jazz" while rec is not None: count = count + 1 if verbose: print rec - rec = c.next() + rec = self.cur.next() # all items accounted for EXCEPT for 1 with "Blues" genre assert count == len(musicdata)-1 + self.cur = None + def getGenre(self, priKey, priData): assert type(priData) == type("") if verbose: @@ -303,7 +321,17 @@ envFlags = db.DB_INIT_TXN dbFlags = 0 - def test13_associateAutoCommit(self): + def txn_finish_test(self, sDB, txn): + try: + self.finish_test(sDB, txn=txn) + finally: + if self.cur: + self.cur.close() + self.cur = None + if txn: + txn.commit() + + def test13_associate_in_transaction(self): if verbose: print '\n', '-=' * 30 print "Running %s.test13_associateAutoCommit..." % \ @@ -313,18 +341,19 @@ try: self.createDB(txn=txn) - secDB = db.DB(self.env) - secDB.set_flags(db.DB_DUP) - secDB.set_get_returns_none(2) - secDB.open(self.filename, "secondary", db.DB_BTREE, + self.secDB = db.DB(self.env) + self.secDB.set_flags(db.DB_DUP) + self.secDB.set_get_returns_none(2) + self.secDB.open(self.filename, "secondary", db.DB_BTREE, db.DB_CREATE | db.DB_THREAD, txn=txn) - self.getDB().associate(secDB, self.getGenre, txn=txn) + self.getDB().associate(self.secDB, self.getGenre, txn=txn) self.addDataToDB(self.getDB(), txn=txn) + except: + txn.abort() + raise - self.finish_test(secDB, txn=txn) - finally: - txn.commit() + self.txn_finish_test(self.secDB, txn=txn) #---------------------------------------------------------------------- From greg at users.sourceforge.net Mon Jun 6 20:12:27 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Mon, 06 Jun 2005 11:12:27 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_associate.py, 1.11, 1.12 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21023 Modified Files: test_associate.py Log Message: fix more Errors (not Failures) when run using BerkeleyDB <= 4.0 Index: test_associate.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_associate.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- test_associate.py 6 Jun 2005 17:59:00 -0000 1.11 +++ test_associate.py 6 Jun 2005 18:12:24 -0000 1.12 @@ -127,7 +127,8 @@ # dupDB has been configured to allow duplicates, it can't # associate with a secondary. BerkeleyDB will return an error. try: - dupDB.associate(secDB, lambda a, b: a+b) + def f(a,b): return a+b + dupDB.associate(secDB, f) except db.DBError: # good secDB.close() @@ -181,8 +182,12 @@ self.secDB = None self.primary = db.DB(self.env) self.primary.set_get_returns_none(2) - self.primary.open(self.filename, "primary", self.dbtype, + if db.version() >= (4, 1): + self.primary.open(self.filename, "primary", self.dbtype, db.DB_CREATE | db.DB_THREAD | self.dbFlags, txn=txn) + else: + self.primary.open(self.filename, "primary", self.dbtype, + db.DB_CREATE | db.DB_THREAD | self.dbFlags) def closeDB(self): if self.cur: @@ -346,7 +351,10 @@ self.secDB.set_get_returns_none(2) self.secDB.open(self.filename, "secondary", db.DB_BTREE, db.DB_CREATE | db.DB_THREAD, txn=txn) - self.getDB().associate(self.secDB, self.getGenre, txn=txn) + if db.version() >= (4,1): + self.getDB().associate(self.secDB, self.getGenre, txn=txn) + else: + self.getDB().associate(self.secDB, self.getGenre) self.addDataToDB(self.getDB(), txn=txn) except: @@ -446,7 +454,8 @@ suite.addTest(unittest.makeSuite(AssociateBTreeTestCase)) suite.addTest(unittest.makeSuite(AssociateRecnoTestCase)) - suite.addTest(unittest.makeSuite(AssociateBTreeTxnTestCase)) + if db.version() >= (4, 1): + suite.addTest(unittest.makeSuite(AssociateBTreeTxnTestCase)) suite.addTest(unittest.makeSuite(ShelveAssociateHashTestCase)) suite.addTest(unittest.makeSuite(ShelveAssociateBTreeTestCase)) From pje at users.sourceforge.net Tue Jun 7 06:41:53 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 06 Jun 2005 21:41:53 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.13, 1.14 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27547/setuptools/command Modified Files: bdist_egg.py Log Message: Package scripts under EGG-INFO/scripts. Refactor subcommand invocations for less duplication and greater clarity. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- bdist_egg.py 5 Jun 2005 18:59:36 -0000 1.13 +++ bdist_egg.py 7 Jun 2005 04:41:51 -0000 1.14 @@ -96,9 +96,12 @@ f.close() def do_install_data(self): + # Hack for packages that install data to install's --install-lib self.get_finalized_command('install').install_lib = self.bdist_dir + site_packages = os.path.normcase(os.path.realpath(get_python_lib())) old, self.distribution.data_files = self.distribution.data_files,[] + for item in old: if isinstance(item,tuple) and len(item)==2: if os.path.isabs(item[0]): @@ -110,36 +113,27 @@ item = realpath[len(site_packages)+1:], item[1] # XXX else: raise ??? self.distribution.data_files.append(item) + try: - install = self.reinitialize_command('install_data') - # kludge for setups that use a 3-tuple inst_data - install.install_dir = install.install_base = \ - install.install_data = install.install_lib = self.bdist_dir - install.force = 0; install.root = None log.info("installing package data to %s" % self.bdist_dir) - self.run_command('install_data') + self.call_command('install_data', force=0, root=None) finally: self.distribution.data_files = old - def run(self): - if not self.skip_build: - self.run_command('build') + def run(self): # We run install_lib before install_data, because some data hacks # pull their data path from the install_lib command. - install = self.reinitialize_command('install_lib', reinit_subcommands=1) - install.install_dir = self.bdist_dir - install.skip_build = self.skip_build - install.warn_dir = 0 - ext_outputs = \ - install._mutate_outputs(self.distribution.has_ext_modules(), - 'build_ext', 'build_lib', - '') log.info("installing library code to %s" % self.bdist_dir) - self.run_command('install_lib') + cmd = self.call_command('install_lib', warn_dir=0) + + ext_outputs = cmd._mutate_outputs( + self.distribution.has_ext_modules(), 'build_ext', 'build_lib', '' + ) to_compile = [] + for ext_name in ext_outputs: filename,ext = os.path.splitext(ext_name) pyfile = os.path.join(self.bdist_dir, filename + '.py') @@ -149,15 +143,21 @@ to_compile.append(pyfile) if to_compile: - install.byte_compile(to_compile) + cmd.byte_compile(to_compile) if self.distribution.data_files: self.do_install_data() + if self.distribution.scripts: + script_dir = os.path.join(self.bdist_dir,'EGG-INFO','scripts') + log.info("installing scripts to %s" % script_dir) + self.call_command('install_scripts', install_dir=script_dir) + # And make an archive relative to the root of the # pseudo-installation tree. archive_basename = "%s-%s-py%s" % ( self.egg_name.replace('-','_'), self.egg_version.replace('-','_'), get_python_version()) + if ext_outputs: archive_basename += "-" + self.plat_name ext_outputs = [out.replace(os.sep,'/') for out in ext_outputs] @@ -207,7 +207,6 @@ make_zipfile(pseudoinstall_root+'.egg', archive_root, verbose=self.verbose, dry_run=self.dry_run) - if not self.keep_temp: remove_tree(self.bdist_dir, dry_run=self.dry_run) @@ -234,14 +233,23 @@ raise RuntimeError("svn info error: %s" % result.strip()) return match.group(1) + def call_command(self,cmdname,**kw): + cmd = self.reinitialize_command(cmdname) + for dirname in INSTALL_DIRECTORY_ATTRS: + if dirname in cmd.__dict__: # don't overwrite methods! + setattr(cmd,dirname,self.bdist_dir) + cmd.skip_build = self.skip_build + for k,v in kw.items(): + setattr(cmd,k,v) + self.run_command(cmdname) + return cmd +# Attribute names of options for commands that might need to be convinced to +# install to the egg build directory - - - - - - +INSTALL_DIRECTORY_ATTRS = [ + 'install_lib', 'install_dir', 'install_data', 'install_base' +] def make_zipfile (zip_filename, base_dir, verbose=0, dry_run=0): @@ -277,11 +285,3 @@ - - - - - - - - From goodger at users.sourceforge.net Tue Jun 7 15:17:41 2005 From: goodger at users.sourceforge.net (goodger@users.sourceforge.net) Date: Tue, 07 Jun 2005 06:17:41 -0700 Subject: [Python-checkins] python/nondist/peps pep-0012.txt,1.4,1.5 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31188 Modified Files: pep-0012.txt Log Message: updated links; thanks to Felix Wiemann Index: pep-0012.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0012.txt,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- pep-0012.txt 22 Sep 2003 04:51:50 -0000 1.4 +++ pep-0012.txt 7 Jun 2005 13:17:37 -0000 1.5 @@ -602,15 +602,15 @@ The processing of reStructuredText PEPs is done using Docutils_. If you have a question or require assistance with reStructuredText or -Docutils, please `post a message`_ to the `Docutils-Users mailing +Docutils, please `post a message`_ to the `Docutils-users mailing list`_. The `Docutils project web site`_ has more information. -.. _Docutils: http://docutils.sourceforge.net/ +.. _Docutils: +.. _Docutils project web site: http://docutils.sourceforge.net/ .. _post a message: mailto:docutils-users at lists.sourceforge.net?subject=PEPs -.. _Docutils-Users mailing list: - http://lists.sourceforge.net/lists/listinfo/docutils-users -.. _Docutils project web site: http://docutils.sourceforge.net/ +.. _Docutils-users mailing list: + http://docutils.sf.net/docs/user/mailing-lists.html#docutils-users References From rhettinger at users.sourceforge.net Tue Jun 7 20:50:59 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 07 Jun 2005 11:50:59 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libdecimal.tex, 1.25, 1.26 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9065 Modified Files: libdecimal.tex Log Message: Add a decimal FAQ Index: libdecimal.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libdecimal.tex,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- libdecimal.tex 21 Feb 2005 15:46:52 -0000 1.25 +++ libdecimal.tex 7 Jun 2005 18:50:56 -0000 1.26 @@ -525,11 +525,11 @@ large number of methods for doing arithmetic directly in a given context. \begin{methoddesc}{clear_flags}{} - Sets all of the flags to \constant{0}. + Resets all of the flags to \constant{0}. \end{methoddesc} \begin{methoddesc}{copy}{} - Returns a duplicate of the context. + Return a duplicate of the context. \end{methoddesc} \begin{methoddesc}{create_decimal}{num} @@ -1118,3 +1118,156 @@ return +s \end{verbatim} + + + +\subsection{Decimal FAQ \label{decimal-faq}} + + +Q. It is cumbersome to type \code{decimal.Decimal('1234.5')}. Is there a way +to minimize typing when using the interactive interpreter? + +A. Some users abbreviate the constructor to just a single letter: + +\begin{verbatim} +>>> D = decimal.Decimal +>>> D('1.23') + D('3.45') +Decimal("4.68") +\end{verbatim} + + +Q. In a fixed-point application to two decimal places, some inputs +have many places and need to be rounded. Others are not supposed to have +excess digits and need to be validated. What methods should be used? + +A. The \method{quantize()} method rounds to a fixed number of decimal places. +If the \constant{Inexact} trap is set, it is also useful for validation: + +\begin{verbatim} +>>> TWOPLACES = Decimal(10) ** -2 # same as Decimal('0.01') + +>>> # Round to two places +>>> Decimal("3.214").quantize(TWOPLACES) +Decimal("3.21") + +>>> # Validate that a number does not exceed two places +>>> Decimal("3.21").quantize(TWOPLACES, context=Context(traps=[Inexact])) +Decimal("3.21") + +>>> Decimal("3.214").quantize(TWOPLACES, context=Context(traps=[Inexact])) +Traceback (most recent call last): + ... +Inexact: Changed in rounding +\end{verbatim} + + +Q. Once I have valid two place inputs, how do I maintain that invariant +throughout an application? + +A. Some operations like addition and subtraction automatically preserve fixed +point. Others, like multiplication and division, change the number of decimal +places and need to be followed-up with a \method{quantize()} step. + + +Q. There are many ways to write express the same value. The numbers +\constant{200}, \constant{200.000}, \constant{2E2}, and \constant{.02E+4} all +have the same value at various precisions. Is there a way to transform them to +a single recognizable canonical value? + +A. The \method{normalize()} method maps all equivalent values to a single +representive: + +\begin{verbatim} +>>> values = map(Decimal, '200 200.000 2E2 .02E+4'.split()) +>>> [v.normalize() for v in values] +[Decimal("2E+2"), Decimal("2E+2"), Decimal("2E+2"), Decimal("2E+2")] +\end{verbatim} + + +Q. Is there a way to convert a regular float to a \class{Decimal}? + +A. Yes, all binary floating point numbers can be exactly expressed as a +Decimal. An exact conversion may take more precision than intuition would +suggest, so trapping \constant{Inexact} will signal a need for more precision: + +\begin{verbatim} +def floatToDecimal(f): + "Convert a floating point number to a Decimal with no loss of information" + # Transform (exactly) a float to a mantissa (0.5 <= abs(m) < 1.0) and an + # exponent. Double the mantissa until it is an integer. Use the integer + # mantissa and exponent to compute an equivalent Decimal. If this cannot + # be done exactly, then retry with more precision. + + mantissa, exponent = math.frexp(f) + while mantissa != int(mantissa): + mantissa *= 2.0 + exponent -= 1 + mantissa = int(mantissa) + oldcontext = getcontext() + setcontext(Context(traps=[Inexact])) + try: + while True: + try: + return mantissa * Decimal(2) ** exponent + except Inexact: + getcontext().prec += 1 + finally: + setcontext(oldcontext) +\end{verbatim} + + +Q. Why isn't the \function{floatToDecimal()} routine included in the module? + +A. There is some question about whether it is advisable to mix binary and +decimal floating point. Also, its use requires some care to avoid the +representation issues associated with binary floating point: + +\begin{verbatim} +>>> floatToDecimal(1.1) +Decimal("1.100000000000000088817841970012523233890533447265625") +\end{verbatim} + + +Q. Within a complex calculation, how can I make sure that I haven't gotten a +spurious result because of insufficient precision or rounding anomalies. + +A. The decimal module makes it easy to test results. A best practice is to +re-run calculations using greater precision and with various rounding modes. +Widely differing results indicate insufficient precision, rounding mode +issues, ill-conditioned inputs, or a numerically unstable algorithm. + + +Q. I noticed that context precision is applied to the results of operations +but not to the inputs. Is there anything to watch out for when mixing +values of different precisions? + +A. Yes. The principle is that all values are considered to be exact and so +is the arithmetic on those values. Only the results are rounded. The +advantage for inputs is that ``what you type is what you get''. A +disadvantage is that the results can look odd if you forget that the inputs +haven't been rounded: + +\begin{verbatim} +>>> getcontext().prec = 3 +>>> Decimal('3.104') + D('2.104') +Decimal("5.21") +>>> Decimal('3.104') + D('0.000') + D('2.104') +Decimal("5.20") +\end{verbatim} + +The solution is either to increase precision or to force rounding of inputs +using the unary plus operation: + +\begin{verbatim} +>>> getcontext().prec = 3 +>>> +Decimal('1.23456789') # unary plus triggers rounding +Decimal("1.23") +\end{verbatim} + +Alternatively, inputs can be rounded upon creation using the +\method{Context.create_decimal()} method: + +\begin{verbatim} +>>> Context(prec=5, rounding=ROUND_DOWN).create_decimal('1.2345678') +Decimal("1.2345") +\end{verbatim} From akuchling at users.sourceforge.net Tue Jun 7 20:51:44 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Tue, 07 Jun 2005 11:51:44 -0700 Subject: [Python-checkins] python/dist/src/Doc/dist dist.tex,1.93,1.94 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/dist In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9427 Modified Files: dist.tex Log Message: [Bug #1170422] Use correct option name Index: dist.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/dist/dist.tex,v retrieving revision 1.93 retrieving revision 1.94 diff -u -d -r1.93 -r1.94 --- dist.tex 23 Mar 2005 10:38:59 -0000 1.93 +++ dist.tex 7 Jun 2005 18:51:42 -0000 1.94 @@ -403,7 +403,7 @@ with, etc.). All of this is done through another keyword argument to -\function{setup()}, the \option{extensions} option. \option{extensions} +\function{setup()}, the \option{ext_modules} option. \option{ext_modules} is just a list of \class{Extension} instances, each of which describes a single extension module. Suppose your distribution includes a single extension, called \module{foo} and implemented by \file{foo.c}. If no From rhettinger at users.sourceforge.net Tue Jun 7 20:52:51 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 07 Jun 2005 11:52:51 -0700 Subject: [Python-checkins] python/dist/src/Lib decimal.py,1.36,1.37 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9998 Modified Files: decimal.py Log Message: Minor namespace clean-up. Index: decimal.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/decimal.py,v retrieving revision 1.36 retrieving revision 1.37 diff -u -d -r1.36 -r1.37 --- decimal.py 27 Mar 2005 10:47:38 -0000 1.36 +++ decimal.py 7 Jun 2005 18:52:34 -0000 1.37 @@ -134,7 +134,7 @@ 'setcontext', 'getcontext' ] -import copy +import copy as _copy #Rounding ROUND_DOWN = 'ROUND_DOWN' @@ -2210,7 +2210,7 @@ del s for name, val in locals().items(): if val is None: - setattr(self, name, copy.copy(getattr(DefaultContext, name))) + setattr(self, name, _copy.copy(getattr(DefaultContext, name))) else: setattr(self, name, val) del self.self From akuchling at users.sourceforge.net Tue Jun 7 21:05:10 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Tue, 07 Jun 2005 12:05:10 -0700 Subject: [Python-checkins] python/dist/src/Doc/dist dist.tex, 1.86.2.3, 1.86.2.4 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/dist In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17052 Modified Files: Tag: release24-maint dist.tex Log Message: [Bug #1170422] Use correct option name Index: dist.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/dist/dist.tex,v retrieving revision 1.86.2.3 retrieving revision 1.86.2.4 diff -u -d -r1.86.2.3 -r1.86.2.4 --- dist.tex 10 Mar 2005 03:47:24 -0000 1.86.2.3 +++ dist.tex 7 Jun 2005 19:05:06 -0000 1.86.2.4 @@ -403,7 +403,7 @@ with, etc.). All of this is done through another keyword argument to -\function{setup()}, the \option{extensions} option. \option{extensions} +\function{setup()}, the \option{ext_modules} option. \option{ext_modules} is just a list of \class{Extension} instances, each of which describes a single extension module. Suppose your distribution includes a single extension, called \module{foo} and implemented by \file{foo.c}. If no From akuchling at users.sourceforge.net Tue Jun 7 21:36:13 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Tue, 07 Jun 2005 12:36:13 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_dumbdbm.py, 1.10, 1.11 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2036/Lib/test Modified Files: test_dumbdbm.py Log Message: [Bug #1172763] dumbdbm uses eval() on lines, so it chokes if there's an extra \r on the end of a line; fixed by stripping off trailing whitespace. Index: test_dumbdbm.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_dumbdbm.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- test_dumbdbm.py 13 Jul 2003 17:21:10 -0000 1.10 +++ test_dumbdbm.py 7 Jun 2005 19:36:10 -0000 1.11 @@ -74,6 +74,24 @@ self.assertEqual(f['1'], 'hello2') f.close() + def test_line_endings(self): + # test for bug #1172763: dumbdbm would die if the line endings + # weren't what was expected. + f = dumbdbm.open(_fname) + f['1'] = 'hello' + f['2'] = 'hello2' + f.close() + + # Mangle the file by adding \r before each newline + data = open(_fname + '.dir').read() + data = data.replace('\n', '\r\n') + open(_fname + '.dir', 'wb').write(data) + + f = dumbdbm.open(_fname) + self.assertEqual(f['1'], 'hello') + self.assertEqual(f['2'], 'hello2') + + def read_helper(self, f): keys = self.keys_helper(f) for key in self._dict: From akuchling at users.sourceforge.net Tue Jun 7 21:36:12 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Tue, 07 Jun 2005 12:36:12 -0700 Subject: [Python-checkins] python/dist/src/Lib dumbdbm.py,1.29,1.30 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2036/Lib Modified Files: dumbdbm.py Log Message: [Bug #1172763] dumbdbm uses eval() on lines, so it chokes if there's an extra \r on the end of a line; fixed by stripping off trailing whitespace. Index: dumbdbm.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/dumbdbm.py,v retrieving revision 1.29 retrieving revision 1.30 diff -u -d -r1.29 -r1.30 --- dumbdbm.py 14 Jul 2003 12:15:15 -0000 1.29 +++ dumbdbm.py 7 Jun 2005 19:36:09 -0000 1.30 @@ -81,6 +81,7 @@ pass else: for line in f: + line = line.rstrip() key, pos_and_siz_pair = eval(line) self._index[key] = pos_and_siz_pair f.close() From akuchling at users.sourceforge.net Tue Jun 7 21:57:03 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Tue, 07 Jun 2005 12:57:03 -0700 Subject: [Python-checkins] python/dist/src/Lib dumbdbm.py,1.29,1.29.10.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14062/Lib Modified Files: Tag: release24-maint dumbdbm.py Log Message: [Bug #1172763] dumbdbm uses eval() on lines, so it chokes if there's an extra \r on the end of a line; fixed by stripping off trailing whitespace. Index: dumbdbm.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/dumbdbm.py,v retrieving revision 1.29 retrieving revision 1.29.10.1 diff -u -d -r1.29 -r1.29.10.1 --- dumbdbm.py 14 Jul 2003 12:15:15 -0000 1.29 +++ dumbdbm.py 7 Jun 2005 19:57:00 -0000 1.29.10.1 @@ -81,6 +81,7 @@ pass else: for line in f: + line = line.rstrip() key, pos_and_siz_pair = eval(line) self._index[key] = pos_and_siz_pair f.close() From akuchling at users.sourceforge.net Tue Jun 7 21:57:03 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Tue, 07 Jun 2005 12:57:03 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_dumbdbm.py, 1.10, 1.10.10.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14062/Lib/test Modified Files: Tag: release24-maint test_dumbdbm.py Log Message: [Bug #1172763] dumbdbm uses eval() on lines, so it chokes if there's an extra \r on the end of a line; fixed by stripping off trailing whitespace. Index: test_dumbdbm.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_dumbdbm.py,v retrieving revision 1.10 retrieving revision 1.10.10.1 diff -u -d -r1.10 -r1.10.10.1 --- test_dumbdbm.py 13 Jul 2003 17:21:10 -0000 1.10 +++ test_dumbdbm.py 7 Jun 2005 19:57:01 -0000 1.10.10.1 @@ -74,6 +74,24 @@ self.assertEqual(f['1'], 'hello2') f.close() + def test_line_endings(self): + # test for bug #1172763: dumbdbm would die if the line endings + # weren't what was expected. + f = dumbdbm.open(_fname) + f['1'] = 'hello' + f['2'] = 'hello2' + f.close() + + # Mangle the file by adding \r before each newline + data = open(_fname + '.dir').read() + data = data.replace('\n', '\r\n') + open(_fname + '.dir', 'wb').write(data) + + f = dumbdbm.open(_fname) + self.assertEqual(f['1'], 'hello') + self.assertEqual(f['2'], 'hello2') + + def read_helper(self, f): keys = self.keys_helper(f) for key in self._dict: From montanaro at users.sourceforge.net Wed Jun 8 04:28:30 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Tue, 07 Jun 2005 19:28:30 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libfcntl.tex,1.38,1.39 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15713 Modified Files: libfcntl.tex Log Message: Tweak note about using os.open to lock files if O_SHLOCK and O_EXLOCK are present. Index: libfcntl.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libfcntl.tex,v retrieving revision 1.38 retrieving revision 1.39 diff -u -d -r1.38 -r1.39 --- libfcntl.tex 30 Mar 2005 16:25:34 -0000 1.38 +++ libfcntl.tex 8 Jun 2005 02:28:11 -0000 1.39 @@ -166,9 +166,9 @@ better. \begin{seealso} - \seemodule{os}{The \function{os.open()} function supports locking flags - and is available on a wider variety of platforms than - the \function{lockf()} and \function{flock()} - functions, providing a more platform-independent file - locking facility.} + \seemodule{os}{If the locking flags \constant{O_SHLOCK} and + \constant{O_EXLOCK} are present in the \module{os} module, + the \function{os.open()} function provides a more + platform-independent alternative to the \function{lockf()} + and \function{flock()} functions.} \end{seealso} From anthonybaxter at users.sourceforge.net Wed Jun 8 06:35:30 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Tue, 07 Jun 2005 21:35:30 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_dumbdbm.py, 1.10.10.1, 1.10.10.2 test_posixpath.py, 1.12.2.1, 1.12.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14610/test Modified Files: Tag: release24-maint test_dumbdbm.py test_posixpath.py Log Message: Tools/scripts/reindent.py is your friend Index: test_dumbdbm.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_dumbdbm.py,v retrieving revision 1.10.10.1 retrieving revision 1.10.10.2 diff -u -d -r1.10.10.1 -r1.10.10.2 --- test_dumbdbm.py 7 Jun 2005 19:57:01 -0000 1.10.10.1 +++ test_dumbdbm.py 8 Jun 2005 04:35:28 -0000 1.10.10.2 @@ -86,12 +86,12 @@ data = open(_fname + '.dir').read() data = data.replace('\n', '\r\n') open(_fname + '.dir', 'wb').write(data) - + f = dumbdbm.open(_fname) self.assertEqual(f['1'], 'hello') self.assertEqual(f['2'], 'hello2') - - + + def read_helper(self, f): keys = self.keys_helper(f) for key in self._dict: Index: test_posixpath.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_posixpath.py,v retrieving revision 1.12.2.1 retrieving revision 1.12.2.2 diff -u -d -r1.12.2.1 -r1.12.2.2 --- test_posixpath.py 3 Jun 2005 14:31:49 -0000 1.12.2.1 +++ test_posixpath.py 8 Jun 2005 04:35:28 -0000 1.12.2.2 @@ -476,7 +476,7 @@ self.safe_rmdir(ABSTFN + "/k/y") self.safe_rmdir(ABSTFN + "/k") self.safe_rmdir(ABSTFN) - + def test_realpath_resolve_first(self): # Bug #1213894: The first component of the path, if not absolute, # must be resolved too. @@ -487,7 +487,7 @@ os.mkdir(ABSTFN + "/k") os.symlink(ABSTFN, ABSTFN + "link") os.chdir(dirname(ABSTFN)) - + base = basename(ABSTFN) self.assertEqual(realpath(base + "link"), ABSTFN) self.assertEqual(realpath(base + "link/k"), ABSTFN + "/k") From anthonybaxter at users.sourceforge.net Wed Jun 8 06:35:52 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Tue, 07 Jun 2005 21:35:52 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_dumbdbm.py, 1.11, 1.12 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14707/test Modified Files: test_dumbdbm.py Log Message: Tools/scripts/reindent.py _is_ your friend Index: test_dumbdbm.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_dumbdbm.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- test_dumbdbm.py 7 Jun 2005 19:36:10 -0000 1.11 +++ test_dumbdbm.py 8 Jun 2005 04:35:50 -0000 1.12 @@ -86,12 +86,12 @@ data = open(_fname + '.dir').read() data = data.replace('\n', '\r\n') open(_fname + '.dir', 'wb').write(data) - + f = dumbdbm.open(_fname) self.assertEqual(f['1'], 'hello') self.assertEqual(f['2'], 'hello2') - - + + def read_helper(self, f): keys = self.keys_helper(f) for key in self._dict: From anthonybaxter at users.sourceforge.net Wed Jun 8 06:35:53 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Tue, 07 Jun 2005 21:35:53 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_associate.py, 1.12, 1.13 test_basics.py, 1.14, 1.15 test_compare.py, 1.3, 1.4 test_recno.py, 1.9, 1.10 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14707/bsddb/test Modified Files: test_associate.py test_basics.py test_compare.py test_recno.py Log Message: Tools/scripts/reindent.py _is_ your friend Index: test_associate.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_associate.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- test_associate.py 6 Jun 2005 18:12:24 -0000 1.12 +++ test_associate.py 8 Jun 2005 04:35:50 -0000 1.13 @@ -94,12 +94,12 @@ homeDir = os.path.join(os.path.dirname(sys.argv[0]), 'db_home') self.homeDir = homeDir try: - os.mkdir(homeDir) + os.mkdir(homeDir) except os.error: - import glob - files = glob.glob(os.path.join(self.homeDir, '*')) - for file in files: - os.remove(file) + import glob + files = glob.glob(os.path.join(self.homeDir, '*')) + for file in files: + os.remove(file) self.env = db.DBEnv() self.env.open(homeDir, db.DB_CREATE | db.DB_INIT_MPOOL) @@ -126,17 +126,17 @@ # dupDB has been configured to allow duplicates, it can't # associate with a secondary. BerkeleyDB will return an error. - try: - def f(a,b): return a+b - dupDB.associate(secDB, f) - except db.DBError: - # good - secDB.close() - dupDB.close() - else: - secDB.close() - dupDB.close() - self.fail("DBError exception was expected") + try: + def f(a,b): return a+b + dupDB.associate(secDB, f) + except db.DBError: + # good + secDB.close() + dupDB.close() + else: + secDB.close() + dupDB.close() + self.fail("DBError exception was expected") @@ -153,12 +153,12 @@ homeDir = os.path.join(os.path.dirname(sys.argv[0]), 'db_home') self.homeDir = homeDir try: - os.mkdir(homeDir) + os.mkdir(homeDir) except os.error: - import glob - files = glob.glob(os.path.join(self.homeDir, '*')) - for file in files: - os.remove(file) + import glob + files = glob.glob(os.path.join(self.homeDir, '*')) + for file in files: + os.remove(file) self.env = db.DBEnv() self.env.open(homeDir, db.DB_CREATE | db.DB_INIT_MPOOL | db.DB_INIT_LOCK | db.DB_THREAD | self.envFlags) @@ -178,22 +178,22 @@ d.put(key, string.join(value, '|'), txn=txn) def createDB(self, txn=None): - self.cur = None - self.secDB = None + self.cur = None + self.secDB = None self.primary = db.DB(self.env) self.primary.set_get_returns_none(2) - if db.version() >= (4, 1): - self.primary.open(self.filename, "primary", self.dbtype, + if db.version() >= (4, 1): + self.primary.open(self.filename, "primary", self.dbtype, db.DB_CREATE | db.DB_THREAD | self.dbFlags, txn=txn) - else: - self.primary.open(self.filename, "primary", self.dbtype, + else: + self.primary.open(self.filename, "primary", self.dbtype, db.DB_CREATE | db.DB_THREAD | self.dbFlags) def closeDB(self): - if self.cur: - self.cur.close() - if self.secDB: - self.secDB.close() + if self.cur: + self.cur.close() + if self.secDB: + self.secDB.close() self.primary.close() def getDB(self): @@ -294,7 +294,7 @@ # all items accounted for EXCEPT for 1 with "Blues" genre assert count == len(musicdata)-1 - self.cur = None + self.cur = None def getGenre(self, priKey, priData): assert type(priData) == type("") @@ -327,14 +327,14 @@ dbFlags = 0 def txn_finish_test(self, sDB, txn): - try: - self.finish_test(sDB, txn=txn) - finally: - if self.cur: - self.cur.close() - self.cur = None - if txn: - txn.commit() + try: + self.finish_test(sDB, txn=txn) + finally: + if self.cur: + self.cur.close() + self.cur = None + if txn: + txn.commit() def test13_associate_in_transaction(self): if verbose: @@ -342,26 +342,26 @@ print "Running %s.test13_associateAutoCommit..." % \ self.__class__.__name__ - txn = self.env.txn_begin() - try: - self.createDB(txn=txn) + txn = self.env.txn_begin() + try: + self.createDB(txn=txn) - self.secDB = db.DB(self.env) - self.secDB.set_flags(db.DB_DUP) - self.secDB.set_get_returns_none(2) - self.secDB.open(self.filename, "secondary", db.DB_BTREE, - db.DB_CREATE | db.DB_THREAD, txn=txn) - if db.version() >= (4,1): - self.getDB().associate(self.secDB, self.getGenre, txn=txn) - else: - self.getDB().associate(self.secDB, self.getGenre) + self.secDB = db.DB(self.env) + self.secDB.set_flags(db.DB_DUP) + self.secDB.set_get_returns_none(2) + self.secDB.open(self.filename, "secondary", db.DB_BTREE, + db.DB_CREATE | db.DB_THREAD, txn=txn) + if db.version() >= (4,1): + self.getDB().associate(self.secDB, self.getGenre, txn=txn) + else: + self.getDB().associate(self.secDB, self.getGenre) - self.addDataToDB(self.getDB(), txn=txn) - except: - txn.abort() - raise + self.addDataToDB(self.getDB(), txn=txn) + except: + txn.abort() + raise - self.txn_finish_test(self.secDB, txn=txn) + self.txn_finish_test(self.secDB, txn=txn) #---------------------------------------------------------------------- @@ -454,8 +454,8 @@ suite.addTest(unittest.makeSuite(AssociateBTreeTestCase)) suite.addTest(unittest.makeSuite(AssociateRecnoTestCase)) - if db.version() >= (4, 1): - suite.addTest(unittest.makeSuite(AssociateBTreeTxnTestCase)) + if db.version() >= (4, 1): + suite.addTest(unittest.makeSuite(AssociateBTreeTxnTestCase)) suite.addTest(unittest.makeSuite(ShelveAssociateHashTestCase)) suite.addTest(unittest.makeSuite(ShelveAssociateBTreeTestCase)) Index: test_basics.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_basics.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- test_basics.py 4 Jun 2005 06:46:58 -0000 1.14 +++ test_basics.py 8 Jun 2005 04:35:50 -0000 1.15 @@ -397,14 +397,14 @@ try: rec = c.current() except db.DBKeyEmptyError, val: - if get_raises_error: - assert val[0] == db.DB_KEYEMPTY - if verbose: print val - else: - self.fail("unexpected DBKeyEmptyError") + if get_raises_error: + assert val[0] == db.DB_KEYEMPTY + if verbose: print val + else: + self.fail("unexpected DBKeyEmptyError") else: - if get_raises_error: - self.fail('DBKeyEmptyError exception expected') + if get_raises_error: + self.fail('DBKeyEmptyError exception expected') c.next() c2 = c.dup(db.DB_POSITION) Index: test_compare.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_compare.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- test_compare.py 6 Jun 2005 17:30:22 -0000 1.3 +++ test_compare.py 8 Jun 2005 04:35:50 -0000 1.4 @@ -164,21 +164,21 @@ self.finishTest () def verifyStderr(self, method, successRe): - """ - Call method() while capturing sys.stderr output internally and - call self.fail() if successRe.search() does not match the stderr - output. This is used to test for uncatchable exceptions. - """ - stdErr = sys.stderr - sys.stderr = StringIO() - try: - method() - finally: - temp = sys.stderr - sys.stderr = stdErr - errorOut = temp.getvalue() - if not successRe.search(errorOut): - self.fail("unexpected stderr output:\n"+errorOut) + """ + Call method() while capturing sys.stderr output internally and + call self.fail() if successRe.search() does not match the stderr + output. This is used to test for uncatchable exceptions. + """ + stdErr = sys.stderr + sys.stderr = StringIO() + try: + method() + finally: + temp = sys.stderr + sys.stderr = stdErr + errorOut = temp.getvalue() + if not successRe.search(errorOut): + self.fail("unexpected stderr output:\n"+errorOut) def _test_compare_function_exception (self): self.startTest () @@ -188,15 +188,15 @@ return 0 raise RuntimeError, "i'm a naughty comparison function" self.createDB (bad_comparator) - #print "\n*** test should print 2 uncatchable tracebacks ***" - self.addDataToDB (['a', 'b', 'c']) # this should raise, but... - self.finishTest () + #print "\n*** test should print 2 uncatchable tracebacks ***" + self.addDataToDB (['a', 'b', 'c']) # this should raise, but... + self.finishTest () def test_compare_function_exception(self): - self.verifyStderr( - self._test_compare_function_exception, - re.compile('(^RuntimeError:.* naughty.*){2}', re.M|re.S) - ) + self.verifyStderr( + self._test_compare_function_exception, + re.compile('(^RuntimeError:.* naughty.*){2}', re.M|re.S) + ) def _test_compare_function_bad_return (self): self.startTest () @@ -211,10 +211,10 @@ self.finishTest () def test_compare_function_bad_return(self): - self.verifyStderr( - self._test_compare_function_bad_return, - re.compile('(^TypeError:.* return an int.*){2}', re.M|re.S) - ) + self.verifyStderr( + self._test_compare_function_bad_return, + re.compile('(^TypeError:.* return an int.*){2}', re.M|re.S) + ) def test_cannot_assign_twice (self): Index: test_recno.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_recno.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- test_recno.py 4 Jun 2005 06:46:58 -0000 1.9 +++ test_recno.py 8 Jun 2005 04:35:50 -0000 1.10 @@ -35,8 +35,8 @@ def test01_basic(self): d = db.DB() - get_returns_none = d.set_get_returns_none(2) - d.set_get_returns_none(get_returns_none) + get_returns_none = d.set_get_returns_none(2) + d.set_get_returns_none(get_returns_none) d.open(self.filename, db.DB_RECNO, db.DB_CREATE) @@ -69,13 +69,13 @@ else: self.fail("expected exception") - # test that has_key raises DB exceptions (fixed in pybsddb 4.3.2) - try: - d.has_key(0) - except db.DBError, val: - pass - else: - self.fail("has_key did not raise a proper exception") + # test that has_key raises DB exceptions (fixed in pybsddb 4.3.2) + try: + d.has_key(0) + except db.DBError, val: + pass + else: + self.fail("has_key did not raise a proper exception") try: data = d[100] @@ -84,13 +84,13 @@ else: self.fail("expected exception") - try: - data = d.get(100) - except db.DBNotFoundError, val: - if get_returns_none: - self.fail("unexpected exception") - else: - assert data == None + try: + data = d.get(100) + except db.DBNotFoundError, val: + if get_returns_none: + self.fail("unexpected exception") + else: + assert data == None keys = d.keys() if verbose: @@ -178,14 +178,14 @@ try: d.get(99) except db.DBKeyEmptyError, val: - if get_returns_none: - self.fail("unexpected DBKeyEmptyError exception") - else: - assert val[0] == db.DB_KEYEMPTY - if verbose: print val + if get_returns_none: + self.fail("unexpected DBKeyEmptyError exception") + else: + assert val[0] == db.DB_KEYEMPTY + if verbose: print val else: - if not get_returns_none: - self.fail("expected exception") + if not get_returns_none: + self.fail("expected exception") rec = c.set(40) while rec: From anthonybaxter at users.sourceforge.net Wed Jun 8 06:55:53 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Tue, 07 Jun 2005 21:55:53 -0700 Subject: [Python-checkins] python/dist/src/Lib/email/test test_email.py, 1.68, 1.69 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/email/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25246/email/test Modified Files: test_email.py Log Message: fix broken (unexecuted) test Index: test_email.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/email/test/test_email.py,v retrieving revision 1.68 retrieving revision 1.69 diff -u -d -r1.68 -r1.69 --- test_email.py 5 Dec 2004 03:45:42 -0000 1.68 +++ test_email.py 8 Jun 2005 04:55:50 -0000 1.69 @@ -892,9 +892,9 @@ payload = self._au.get_payload() self.assertEqual(base64.decodestring(payload), self._audiodata) - def checkSetMinor(self): + def test_checkSetMinor(self): au = MIMEAudio(self._audiodata, 'fish') - self.assertEqual(im.get_type(), 'audio/fish') + self.assertEqual(au.get_type(), 'audio/fish') def test_add_header(self): eq = self.assertEqual @@ -935,7 +935,7 @@ payload = self._im.get_payload() self.assertEqual(base64.decodestring(payload), self._imgdata) - def checkSetMinor(self): + def test_checkSetMinor(self): im = MIMEImage(self._imgdata, 'fish') self.assertEqual(im.get_type(), 'image/fish') From anthonybaxter at users.sourceforge.net Wed Jun 8 06:56:44 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Tue, 07 Jun 2005 21:56:44 -0700 Subject: [Python-checkins] python/dist/src/Lib/email/test test_email.py, 1.67.2.1, 1.67.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/email/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25755/email/test Modified Files: Tag: release24-maint test_email.py Log Message: fix broken (unexecuted) test Index: test_email.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/email/test/test_email.py,v retrieving revision 1.67.2.1 retrieving revision 1.67.2.2 diff -u -d -r1.67.2.1 -r1.67.2.2 --- test_email.py 5 Dec 2004 03:34:14 -0000 1.67.2.1 +++ test_email.py 8 Jun 2005 04:56:41 -0000 1.67.2.2 @@ -892,9 +892,9 @@ payload = self._au.get_payload() self.assertEqual(base64.decodestring(payload), self._audiodata) - def checkSetMinor(self): + def test_checkSetMinor(self): au = MIMEAudio(self._audiodata, 'fish') - self.assertEqual(im.get_type(), 'audio/fish') + self.assertEqual(au.get_type(), 'audio/fish') def test_add_header(self): eq = self.assertEqual @@ -935,7 +935,7 @@ payload = self._im.get_payload() self.assertEqual(base64.decodestring(payload), self._imgdata) - def checkSetMinor(self): + def test_checkSetMinor(self): im = MIMEImage(self._imgdata, 'fish') self.assertEqual(im.get_type(), 'image/fish') From akuchling at users.sourceforge.net Wed Jun 8 23:51:31 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 08 Jun 2005 14:51:31 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libbsddb.tex,1.20,1.21 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29475 Modified Files: libbsddb.tex Log Message: [Bug #1149413] 'psize' argument should be 'pgsize' Index: libbsddb.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libbsddb.tex,v retrieving revision 1.20 retrieving revision 1.21 diff -u -d -r1.20 -r1.21 --- libbsddb.tex 1 Jan 2005 00:28:34 -0000 1.20 +++ libbsddb.tex 8 Jun 2005 21:51:28 -0000 1.21 @@ -56,7 +56,7 @@ \begin{funcdesc}{btopen}{filename\optional{, flag\optional{, mode\optional{, btflags\optional{, cachesize\optional{, maxkeypage\optional{, -minkeypage\optional{, psize\optional{, lorder}}}}}}}}} +minkeypage\optional{, pgsize\optional{, lorder}}}}}}}}} Open the btree format file named \var{filename}. Files never intended to be preserved on disk may be created by passing \code{None} as the @@ -71,7 +71,7 @@ \end{funcdesc} \begin{funcdesc}{rnopen}{filename\optional{, flag\optional{, mode\optional{, -rnflags\optional{, cachesize\optional{, psize\optional{, lorder\optional{, +rnflags\optional{, cachesize\optional{, pgsize\optional{, lorder\optional{, reclen\optional{, bval\optional{, bfname}}}}}}}}}} Open a DB record format file named \var{filename}. Files never intended From akuchling at users.sourceforge.net Wed Jun 8 23:52:01 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 08 Jun 2005 14:52:01 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libbsddb.tex, 1.18.2.2, 1.18.2.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29907 Modified Files: Tag: release24-maint libbsddb.tex Log Message: [Bug #1149413] 'psize' argument should be 'pgsize' Index: libbsddb.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libbsddb.tex,v retrieving revision 1.18.2.2 retrieving revision 1.18.2.3 diff -u -d -r1.18.2.2 -r1.18.2.3 --- libbsddb.tex 1 Jan 2005 00:34:53 -0000 1.18.2.2 +++ libbsddb.tex 8 Jun 2005 21:51:58 -0000 1.18.2.3 @@ -56,7 +56,7 @@ \begin{funcdesc}{btopen}{filename\optional{, flag\optional{, mode\optional{, btflags\optional{, cachesize\optional{, maxkeypage\optional{, -minkeypage\optional{, psize\optional{, lorder}}}}}}}}} +minkeypage\optional{, pgsize\optional{, lorder}}}}}}}}} Open the btree format file named \var{filename}. Files never intended to be preserved on disk may be created by passing \code{None} as the @@ -71,7 +71,7 @@ \end{funcdesc} \begin{funcdesc}{rnopen}{filename\optional{, flag\optional{, mode\optional{, -rnflags\optional{, cachesize\optional{, psize\optional{, lorder\optional{, +rnflags\optional{, cachesize\optional{, pgsize\optional{, lorder\optional{, reclen\optional{, bval\optional{, bfname}}}}}}}}}} Open a DB record format file named \var{filename}. Files never intended From akuchling at users.sourceforge.net Thu Jun 9 00:51:41 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 08 Jun 2005 15:51:41 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_base64.py,1.7,1.8 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29282/Lib/test Modified Files: test_base64.py Log Message: [Patch #1171487, bug #1170331] Fix error in base64.b32decode when encoding a single null byte; test a null byte in all encodings to be sure it works Index: test_base64.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_base64.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- test_base64.py 4 Jan 2004 01:13:02 -0000 1.7 +++ test_base64.py 8 Jun 2005 22:51:38 -0000 1.8 @@ -60,6 +60,7 @@ eq = self.assertEqual # Test default alphabet eq(base64.b64encode("www.python.org"), "d3d3LnB5dGhvbi5vcmc=") + eq(base64.b64encode('\x00'), 'AA==') eq(base64.b64encode("a"), "YQ==") eq(base64.b64encode("ab"), "YWI=") eq(base64.b64encode("abc"), "YWJj") @@ -90,6 +91,7 @@ def test_b64decode(self): eq = self.assertEqual eq(base64.b64decode("d3d3LnB5dGhvbi5vcmc="), "www.python.org") + eq(base64.b64decode('AA=='), '\x00') eq(base64.b64decode("YQ=="), "a") eq(base64.b64decode("YWI="), "ab") eq(base64.b64decode("YWJj"), "abc") @@ -123,6 +125,7 @@ def test_b32encode(self): eq = self.assertEqual eq(base64.b32encode(''), '') + eq(base64.b32encode('\x00'), 'AA======') eq(base64.b32encode('a'), 'ME======') eq(base64.b32encode('ab'), 'MFRA====') eq(base64.b32encode('abc'), 'MFRGG===') @@ -132,6 +135,7 @@ def test_b32decode(self): eq = self.assertEqual eq(base64.b32decode(''), '') + eq(base64.b32decode('AA======'), '\x00') eq(base64.b32decode('ME======'), 'a') eq(base64.b32decode('MFRA===='), 'ab') eq(base64.b32decode('MFRGG==='), 'abc') @@ -166,10 +170,12 @@ def test_b16encode(self): eq = self.assertEqual eq(base64.b16encode('\x01\x02\xab\xcd\xef'), '0102ABCDEF') + eq(base64.b16encode('\x00'), '00') def test_b16decode(self): eq = self.assertEqual eq(base64.b16decode('0102ABCDEF'), '\x01\x02\xab\xcd\xef') + eq(base64.b16decode('00'), '\x00') # Lower case is not allowed without a flag self.assertRaises(TypeError, base64.b16decode, '0102abcdef') # Case fold From akuchling at users.sourceforge.net Thu Jun 9 00:51:41 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 08 Jun 2005 15:51:41 -0700 Subject: [Python-checkins] python/dist/src/Lib base64.py,1.16,1.17 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29282/Lib Modified Files: base64.py Log Message: [Patch #1171487, bug #1170331] Fix error in base64.b32decode when encoding a single null byte; test a null byte in all encodings to be sure it works Index: base64.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/base64.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- base64.py 12 Feb 2004 17:35:05 -0000 1.16 +++ base64.py 8 Jun 2005 22:51:38 -0000 1.17 @@ -221,12 +221,14 @@ acc += _b32rev[c] << shift shift -= 5 if shift < 0: - parts.append(binascii.unhexlify(hex(acc)[2:-1])) + parts.append(binascii.unhexlify('%010x' % acc)) acc = 0 shift = 35 # Process the last, partial quanta - last = binascii.unhexlify(hex(acc)[2:-1]) - if padchars == 1: + last = binascii.unhexlify('%010x' % acc) + if padchars == 0: + last = '' # No characters + elif padchars == 1: last = last[:-1] elif padchars == 3: last = last[:-2] @@ -234,7 +236,7 @@ last = last[:-3] elif padchars == 6: last = last[:-4] - elif padchars <> 0: + else: raise TypeError('Incorrect padding') parts.append(last) return EMPTYSTRING.join(parts) From akuchling at users.sourceforge.net Thu Jun 9 00:53:03 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 08 Jun 2005 15:53:03 -0700 Subject: [Python-checkins] python/dist/src/Lib base64.py,1.16,1.16.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29883/Lib Modified Files: Tag: release24-maint base64.py Log Message: [Patch #1171487, bug #1170331] Fix error in base64.b32decode when encoding a single null byte; test a null byte in all encodings to be sure it works Index: base64.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/base64.py,v retrieving revision 1.16 retrieving revision 1.16.4.1 diff -u -d -r1.16 -r1.16.4.1 --- base64.py 12 Feb 2004 17:35:05 -0000 1.16 +++ base64.py 8 Jun 2005 22:53:00 -0000 1.16.4.1 @@ -221,12 +221,14 @@ acc += _b32rev[c] << shift shift -= 5 if shift < 0: - parts.append(binascii.unhexlify(hex(acc)[2:-1])) + parts.append(binascii.unhexlify('%010x' % acc)) acc = 0 shift = 35 # Process the last, partial quanta - last = binascii.unhexlify(hex(acc)[2:-1]) - if padchars == 1: + last = binascii.unhexlify('%010x' % acc) + if padchars == 0: + last = '' # No characters + elif padchars == 1: last = last[:-1] elif padchars == 3: last = last[:-2] @@ -234,7 +236,7 @@ last = last[:-3] elif padchars == 6: last = last[:-4] - elif padchars <> 0: + else: raise TypeError('Incorrect padding') parts.append(last) return EMPTYSTRING.join(parts) From akuchling at users.sourceforge.net Thu Jun 9 00:53:03 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 08 Jun 2005 15:53:03 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_base64.py, 1.7, 1.7.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29883/Lib/test Modified Files: Tag: release24-maint test_base64.py Log Message: [Patch #1171487, bug #1170331] Fix error in base64.b32decode when encoding a single null byte; test a null byte in all encodings to be sure it works Index: test_base64.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_base64.py,v retrieving revision 1.7 retrieving revision 1.7.4.1 diff -u -d -r1.7 -r1.7.4.1 --- test_base64.py 4 Jan 2004 01:13:02 -0000 1.7 +++ test_base64.py 8 Jun 2005 22:53:01 -0000 1.7.4.1 @@ -60,6 +60,7 @@ eq = self.assertEqual # Test default alphabet eq(base64.b64encode("www.python.org"), "d3d3LnB5dGhvbi5vcmc=") + eq(base64.b64encode('\x00'), 'AA==') eq(base64.b64encode("a"), "YQ==") eq(base64.b64encode("ab"), "YWI=") eq(base64.b64encode("abc"), "YWJj") @@ -90,6 +91,7 @@ def test_b64decode(self): eq = self.assertEqual eq(base64.b64decode("d3d3LnB5dGhvbi5vcmc="), "www.python.org") + eq(base64.b64decode('AA=='), '\x00') eq(base64.b64decode("YQ=="), "a") eq(base64.b64decode("YWI="), "ab") eq(base64.b64decode("YWJj"), "abc") @@ -123,6 +125,7 @@ def test_b32encode(self): eq = self.assertEqual eq(base64.b32encode(''), '') + eq(base64.b32encode('\x00'), 'AA======') eq(base64.b32encode('a'), 'ME======') eq(base64.b32encode('ab'), 'MFRA====') eq(base64.b32encode('abc'), 'MFRGG===') @@ -132,6 +135,7 @@ def test_b32decode(self): eq = self.assertEqual eq(base64.b32decode(''), '') + eq(base64.b32decode('AA======'), '\x00') eq(base64.b32decode('ME======'), 'a') eq(base64.b32decode('MFRA===='), 'ab') eq(base64.b32decode('MFRGG==='), 'abc') @@ -166,10 +170,12 @@ def test_b16encode(self): eq = self.assertEqual eq(base64.b16encode('\x01\x02\xab\xcd\xef'), '0102ABCDEF') + eq(base64.b16encode('\x00'), '00') def test_b16decode(self): eq = self.assertEqual eq(base64.b16decode('0102ABCDEF'), '\x01\x02\xab\xcd\xef') + eq(base64.b16decode('00'), '\x00') # Lower case is not allowed without a flag self.assertRaises(TypeError, base64.b16decode, '0102abcdef') # Case fold From greg at users.sourceforge.net Thu Jun 9 09:11:46 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Thu, 09 Jun 2005 00:11:46 -0700 Subject: [Python-checkins] python/dist/src/Lib/bsddb/test test_compare.py, 1.4, 1.5 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/bsddb/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28172 Modified Files: test_compare.py Log Message: fix import to work with either module name. Index: test_compare.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/test/test_compare.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- test_compare.py 8 Jun 2005 04:35:50 -0000 1.4 +++ test_compare.py 9 Jun 2005 07:11:43 -0000 1.5 @@ -7,7 +7,12 @@ from cStringIO import StringIO import unittest -from bsddb3 import db +try: + # For Pythons w/distutils pybsddb + from bsddb3 import db, dbshelve +except ImportError: + # For Python 2.3 + from bsddb import db, dbshelve lexical_cmp = cmp From akuchling at users.sourceforge.net Thu Jun 9 16:12:40 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 07:12:40 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_gzip.py,1.13,1.14 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27123/Lib/test Modified Files: test_gzip.py Log Message: Convert gzip test suite to use unittest Index: test_gzip.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_gzip.py,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- test_gzip.py 29 Jul 2004 03:55:56 -0000 1.13 +++ test_gzip.py 9 Jun 2005 14:12:36 -0000 1.14 @@ -1,8 +1,12 @@ -from test.test_support import verify, TESTFN +#! /usr/bin/env python +"""Test script for the gzip module. +""" + +import unittest +from test import test_support import sys, os import gzip -filename = TESTFN data1 = """ int length=DEFAULTALLOC, err = Z_OK; PyObject *RetVal; @@ -16,75 +20,107 @@ /* See http://www.winimage.com/zLibDll for Windows */ """ -f = gzip.GzipFile(filename, 'wb') ; f.write(data1 * 50) -# Try flush and fileno. -f.flush() -f.fileno() -if hasattr(os, 'fsync'): - os.fsync(f.fileno()) -f.close() +class TestGzip(unittest.TestCase): + filename = test_support.TESTFN -# Try reading. -f = gzip.GzipFile(filename, 'r') ; d = f.read() ; f.close() -verify(d == data1*50) + def setUp (self): + pass -# Append to the previous file -f = gzip.GzipFile(filename, 'ab') ; f.write(data2 * 15) ; f.close() + def tearDown (self): + try: + os.unlink(self.filename) + except os.error: + pass -f = gzip.GzipFile(filename, 'rb') ; d = f.read() ; f.close() -verify(d == (data1*50) + (data2*15)) -# Try .readline() with varying line lengths + def test_write (self): + f = gzip.GzipFile(self.filename, 'wb') ; f.write(data1 * 50) -f = gzip.GzipFile(filename, 'rb') -line_length = 0 -while 1: - L = f.readline(line_length) - if L == "" and line_length != 0: break - verify(len(L) <= line_length) - line_length = (line_length + 1) % 50 -f.close() + # Try flush and fileno. + f.flush() + f.fileno() + if hasattr(os, 'fsync'): + os.fsync(f.fileno()) + f.close() -# Try .readlines() + def test_read(self): + self.test_write() + # Try reading. + f = gzip.GzipFile(self.filename, 'r') ; d = f.read() ; f.close() + self.assertEqual(d, data1*50) -f = gzip.GzipFile(filename, 'rb') -L = f.readlines() -f.close() + def test_append(self): + self.test_write() + # Append to the previous file + f = gzip.GzipFile(self.filename, 'ab') ; f.write(data2 * 15) ; f.close() -f = gzip.GzipFile(filename, 'rb') -while 1: - L = f.readlines(150) - if L == []: break -f.close() + f = gzip.GzipFile(self.filename, 'rb') ; d = f.read() ; f.close() + self.assertEqual(d, (data1*50) + (data2*15)) -# Try seek, read test + def test_readline(self): + self.test_write() + # Try .readline() with varying line lengths -f = gzip.GzipFile(filename) -while 1: - oldpos = f.tell() - line1 = f.readline() - if not line1: break - newpos = f.tell() - f.seek(oldpos) # negative seek - if len(line1)>10: - amount = 10 - else: - amount = len(line1) - line2 = f.read(amount) - verify(line1[:amount] == line2) - f.seek(newpos) # positive seek -f.close() + f = gzip.GzipFile(self.filename, 'rb') + line_length = 0 + while 1: + L = f.readline(line_length) + if L == "" and line_length != 0: break + self.assert_(len(L) <= line_length) + line_length = (line_length + 1) % 50 + f.close() -# Try seek, write test -f = gzip.GzipFile(filename, 'w') -for pos in range(0, 256, 16): - f.seek(pos) - f.write('GZ\n') -f.close() + def test_readlines(self): + self.test_write() + # Try .readlines() -f = gzip.GzipFile(filename, 'r') -verify(f.myfileobj.mode == 'rb') -f.close() + f = gzip.GzipFile(self.filename, 'rb') + L = f.readlines() + f.close() -os.unlink(filename) + f = gzip.GzipFile(self.filename, 'rb') + while 1: + L = f.readlines(150) + if L == []: break + f.close() + + def test_seek_read(self): + self.test_write() + # Try seek, read test + + f = gzip.GzipFile(self.filename) + while 1: + oldpos = f.tell() + line1 = f.readline() + if not line1: break + newpos = f.tell() + f.seek(oldpos) # negative seek + if len(line1)>10: + amount = 10 + else: + amount = len(line1) + line2 = f.read(amount) + self.assertEqual(line1[:amount], line2) + f.seek(newpos) # positive seek + f.close() + + def test_seek_write(self): + # Try seek, write test + f = gzip.GzipFile(self.filename, 'w') + for pos in range(0, 256, 16): + f.seek(pos) + f.write('GZ\n') + f.close() + + def test_mode(self): + self.test_write() + f = gzip.GzipFile(self.filename, 'r') + self.assertEqual(f.myfileobj.mode, 'rb') + f.close() + +def test_main(verbose=None): + test_support.run_unittest(TestGzip) + +if __name__ == "__main__": + test_main(verbose=True) From akuchling at users.sourceforge.net Thu Jun 9 16:19:35 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 07:19:35 -0700 Subject: [Python-checkins] python/dist/src/Lib gzip.py,1.44,1.45 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30527 Modified Files: gzip.py Log Message: [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Index: gzip.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/gzip.py,v retrieving revision 1.44 retrieving revision 1.45 diff -u -d -r1.44 -r1.45 --- gzip.py 28 Mar 2005 01:08:02 -0000 1.44 +++ gzip.py 9 Jun 2005 14:19:32 -0000 1.45 @@ -55,6 +55,7 @@ """ myfileobj = None + max_read_chunk = 10 * 1024 * 1024 # 10Mb def __init__(self, filename=None, mode=None, compresslevel=9, fileobj=None): @@ -215,14 +216,14 @@ try: while True: self._read(readsize) - readsize = readsize * 2 + readsize = min(self.max_read_chunk, readsize * 2) except EOFError: size = self.extrasize else: # just get some more of it try: while size > self.extrasize: self._read(readsize) - readsize = readsize * 2 + readsize = min(self.max_read_chunk, readsize * 2) except EOFError: if size > self.extrasize: size = self.extrasize From akuchling at users.sourceforge.net Thu Jun 9 16:19:35 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 07:19:35 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_gzip.py,1.14,1.15 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30527/test Modified Files: test_gzip.py Log Message: [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Index: test_gzip.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_gzip.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- test_gzip.py 9 Jun 2005 14:12:36 -0000 1.14 +++ test_gzip.py 9 Jun 2005 14:19:32 -0000 1.15 @@ -58,6 +58,29 @@ f = gzip.GzipFile(self.filename, 'rb') ; d = f.read() ; f.close() self.assertEqual(d, (data1*50) + (data2*15)) + def test_many_append(self): + # Bug #1074261 was triggered when reading a file that contained + # many, many members. Create such a file and verify that reading it + # works. + f = gzip.open(self.filename, 'wb', 9) + f.write('a') + f.close() + for i in range(0,200): + f = gzip.open(self.filename, "ab", 9) # append + f.write('a') + f.close() + + # Try reading the file + zgfile = gzip.open(self.filename, "rb") + contents = "" + while 1: + ztxt = zgfile.read(8192) + contents += ztxt + if not ztxt: break + zgfile.close() + self.assertEquals(contents, 'a'*201) + + def test_readline(self): self.test_write() # Try .readline() with varying line lengths From akuchling at users.sourceforge.net Thu Jun 9 16:22:10 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 07:22:10 -0700 Subject: [Python-checkins] python/dist/src/Lib gzip.py,1.42,1.42.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32057 Modified Files: Tag: release24-maint gzip.py Log Message: [Bug #1074261, patch #1074381] Restrict the size of chunks read from the file in order to avoid overflow or huge memory consumption. Patch by Mark Eichin Index: gzip.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/gzip.py,v retrieving revision 1.42 retrieving revision 1.42.4.1 diff -u -d -r1.42 -r1.42.4.1 --- gzip.py 27 Jul 2004 21:05:21 -0000 1.42 +++ gzip.py 9 Jun 2005 14:22:07 -0000 1.42.4.1 @@ -55,6 +55,7 @@ """ myfileobj = None + max_read_chunk = 10 * 1024 * 1024 def __init__(self, filename=None, mode=None, compresslevel=9, fileobj=None): @@ -215,14 +216,14 @@ try: while True: self._read(readsize) - readsize = readsize * 2 + readsize = min(self.max_read_chunk, readsize * 2) except EOFError: size = self.extrasize else: # just get some more of it try: while size > self.extrasize: self._read(readsize) - readsize = readsize * 2 + readsize = min(self.max_read_chunk, readsize * 2) except EOFError: if size > self.extrasize: size = self.extrasize From akuchling at users.sourceforge.net Thu Jun 9 16:56:34 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 07:56:34 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_asynchat.py, 1.5, 1.6 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17416 Modified Files: test_asynchat.py Log Message: Convert asynchat test to unittest; exercise the client using a numeric value as the terminator Index: test_asynchat.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_asynchat.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- test_asynchat.py 12 Feb 2004 17:35:11 -0000 1.5 +++ test_asynchat.py 9 Jun 2005 14:56:31 -0000 1.6 @@ -2,6 +2,8 @@ import thread # If this fails, we can't test this module import asyncore, asynchat, socket, threading, time +import unittest +from test import test_support HOST = "127.0.0.1" PORT = 54321 @@ -16,7 +18,7 @@ conn, client = sock.accept() buffer = "" while "\n" not in buffer: - data = conn.recv(10) + data = conn.recv(1) if not data: break buffer = buffer + data @@ -28,31 +30,61 @@ class echo_client(asynchat.async_chat): - def __init__(self): + def __init__(self, terminator): asynchat.async_chat.__init__(self) + self.contents = None self.create_socket(socket.AF_INET, socket.SOCK_STREAM) self.connect((HOST, PORT)) - self.set_terminator("\n") + self.set_terminator(terminator) self.buffer = "" def handle_connect(self): - print "Connected" + pass + ##print "Connected" def collect_incoming_data(self, data): self.buffer = self.buffer + data def found_terminator(self): - print "Received:", repr(self.buffer) + #print "Received:", repr(self.buffer) + self.contents = self.buffer self.buffer = "" self.close() -def main(): - s = echo_server() - s.start() - time.sleep(1) # Give server time to initialize - c = echo_client() - c.push("hello ") - c.push("world\n") - asyncore.loop() -main() +class TestAsynchat(unittest.TestCase): + def setUp (self): + pass + + def tearDown (self): + pass + + def test_line_terminator(self): + s = echo_server() + s.start() + time.sleep(1) # Give server time to initialize + c = echo_client('\n') + c.push("hello ") + c.push("world\n") + asyncore.loop() + + self.assertEqual(c.contents, 'hello world') + + def test_numeric_terminator(self): + # Try reading a fixed number of bytes + s = echo_server() + s.start() + time.sleep(1) # Give server time to initialize + c = echo_client(6L) + c.push("hello ") + c.push("world\n") + asyncore.loop() + + self.assertEqual(c.contents, 'hello ') + + +def test_main(verbose=None): + test_support.run_unittest(TestAsynchat) + +if __name__ == "__main__": + test_main(verbose=True) From akuchling at users.sourceforge.net Thu Jun 9 16:59:47 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 07:59:47 -0700 Subject: [Python-checkins] python/dist/src/Lib asynchat.py,1.25,1.26 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19209 Modified Files: asynchat.py Log Message: [Patch #1002763] Allow long ints as terminator values; also, treat a terminator of 0 like the empty string or None Index: asynchat.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/asynchat.py,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- asynchat.py 27 Sep 2004 17:49:00 -0000 1.25 +++ asynchat.py 9 Jun 2005 14:59:45 -0000 1.26 @@ -101,11 +101,11 @@ while self.ac_in_buffer: lb = len(self.ac_in_buffer) terminator = self.get_terminator() - if terminator is None or terminator == '': + if not terminator: # no terminator, collect it all self.collect_incoming_data (self.ac_in_buffer) self.ac_in_buffer = '' - elif isinstance(terminator, int): + elif isinstance(terminator, int) or isinstance(terminator, long): # numeric terminator n = terminator if lb < n: From akuchling at users.sourceforge.net Thu Jun 9 17:57:35 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 08:57:35 -0700 Subject: [Python-checkins] python/dist/src/Doc/whatsnew whatsnew24.tex, 1.128, 1.129 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/whatsnew In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16226 Modified Files: whatsnew24.tex Log Message: [Bug #1079134] Mention change to date comparisons; bump version info and release date to 2.4.1 Index: whatsnew24.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/whatsnew/whatsnew24.tex,v retrieving revision 1.128 retrieving revision 1.129 diff -u -d -r1.128 -r1.129 --- whatsnew24.tex 7 Jan 2005 14:34:41 -0000 1.128 +++ whatsnew24.tex 9 Jun 2005 15:57:32 -0000 1.129 @@ -18,8 +18,8 @@ \maketitle \tableofcontents -This article explains the new features in Python 2.4, released on -November~30, 2004. +This article explains the new features in Python 2.4.1, released on +March~30, 2005. Python 2.4 is a medium-sized release. It doesn't introduce as many changes as the radical Python 2.2, but introduces more features than @@ -1713,6 +1713,11 @@ now return an empty list instead of raising a \exception{TypeError} exception if called with no arguments. +\item You can no longer compare the \class{date} and \class{datetime} + instances provided by the \module{datetime} module. Two + instances of different classes will now always be unequal, and + relative comparisons (\code{<}, \code{>}) will raise a \exception{TypeError}. + \item \function{dircache.listdir()} now passes exceptions to the caller instead of returning empty lists. @@ -1745,7 +1750,8 @@ The author would like to thank the following people for offering suggestions, corrections and assistance with various drafts of this -article: Koray Can, Hye-Shik Chang, Michael Dyck, Raymond Hettinger, -Brian Hurt, Hamish Lawson, Fredrik Lundh, Sean Reifschneider. +article: Koray Can, Hye-Shik Chang, Michael Dyck, Raymond Hettinger, +Brian Hurt, Hamish Lawson, Fredrik Lundh, Sean Reifschneider, +Sadruddin Rejeb. \end{document} From akuchling at users.sourceforge.net Thu Jun 9 19:53:32 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 10:53:32 -0700 Subject: [Python-checkins] python/dist/src/Modules _cursesmodule.c, 2.73, 2.74 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12117 Modified Files: _cursesmodule.c Log Message: [Bug #1200134] Fix buffer overflow by constraining size of .getstr(), .instr() to size of allocated buffer Index: _cursesmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_cursesmodule.c,v retrieving revision 2.73 retrieving revision 2.74 diff -u -d -r2.73 -r2.74 --- _cursesmodule.c 4 Aug 2004 14:33:28 -0000 2.73 +++ _cursesmodule.c 9 Jun 2005 17:53:27 -0000 2.74 @@ -162,6 +162,10 @@ "must call start_color() first"); \ return 0; } +#ifndef MIN +#define MIN(x,y) ((x) < (y) ? (x) : (y)) +#endif + /* Utility Functions */ /* @@ -801,21 +805,21 @@ switch (PyTuple_Size(args)) { case 0: Py_BEGIN_ALLOW_THREADS - rtn2 = wgetstr(self->win,rtn); + rtn2 = wgetnstr(self->win,rtn, 1023); Py_END_ALLOW_THREADS break; case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; Py_BEGIN_ALLOW_THREADS - rtn2 = wgetnstr(self->win,rtn,n); + rtn2 = wgetnstr(self->win,rtn,MIN(n, 1023)); Py_END_ALLOW_THREADS break; case 2: if (!PyArg_ParseTuple(args,"ii;y,x",&y,&x)) return NULL; Py_BEGIN_ALLOW_THREADS - rtn2 = mvwgetstr(self->win,y,x,rtn); + rtn2 = mvwgetnstr(self->win,y,x,rtn, 1023); Py_END_ALLOW_THREADS break; case 3: @@ -825,11 +829,11 @@ /* Untested */ Py_BEGIN_ALLOW_THREADS rtn2 = wmove(self->win,y,x)==ERR ? ERR : - wgetnstr(self->win, rtn, n); + wgetnstr(self->win, rtn, MIN(n, 1023)); Py_END_ALLOW_THREADS #else Py_BEGIN_ALLOW_THREADS - rtn2 = mvwgetnstr(self->win, y, x, rtn, n); + rtn2 = mvwgetnstr(self->win, y, x, rtn, MIN(n, 1023)); Py_END_ALLOW_THREADS #endif break; @@ -962,22 +966,22 @@ switch (PyTuple_Size(args)) { case 0: - rtn2 = winstr(self->win,rtn); + rtn2 = winnstr(self->win,rtn, 1023); break; case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; - rtn2 = winnstr(self->win,rtn,n); + rtn2 = winnstr(self->win,rtn,MIN(n,1023)); break; case 2: if (!PyArg_ParseTuple(args,"ii;y,x",&y,&x)) return NULL; - rtn2 = mvwinstr(self->win,y,x,rtn); + rtn2 = mvwinnstr(self->win,y,x,rtn,1023); break; case 3: if (!PyArg_ParseTuple(args, "iii;y,x,n", &y, &x, &n)) return NULL; - rtn2 = mvwinnstr(self->win, y, x, rtn, n); + rtn2 = mvwinnstr(self->win, y, x, rtn, MIN(n,1023)); break; default: PyErr_SetString(PyExc_TypeError, "instr requires 0 or 3 arguments"); From akuchling at users.sourceforge.net Thu Jun 9 19:55:36 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Thu, 09 Jun 2005 10:55:36 -0700 Subject: [Python-checkins] python/dist/src/Modules _cursesmodule.c, 2.73, 2.73.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13351 Modified Files: Tag: release24-maint _cursesmodule.c Log Message: [Bug #1200134] Fix buffer overflow by constraining size of .getstr(), .instr() to size of allocated buffer Index: _cursesmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_cursesmodule.c,v retrieving revision 2.73 retrieving revision 2.73.2.1 diff -u -d -r2.73 -r2.73.2.1 --- _cursesmodule.c 4 Aug 2004 14:33:28 -0000 2.73 +++ _cursesmodule.c 9 Jun 2005 17:55:34 -0000 2.73.2.1 @@ -162,6 +162,10 @@ "must call start_color() first"); \ return 0; } +#ifndef MIN +#define MIN(x,y) ((x) < (y) ? (x) : (y)) +#endif + /* Utility Functions */ /* @@ -801,21 +805,21 @@ switch (PyTuple_Size(args)) { case 0: Py_BEGIN_ALLOW_THREADS - rtn2 = wgetstr(self->win,rtn); + rtn2 = wgetnstr(self->win,rtn, 1023); Py_END_ALLOW_THREADS break; case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; Py_BEGIN_ALLOW_THREADS - rtn2 = wgetnstr(self->win,rtn,n); + rtn2 = wgetnstr(self->win,rtn,MIN(n, 1023)); Py_END_ALLOW_THREADS break; case 2: if (!PyArg_ParseTuple(args,"ii;y,x",&y,&x)) return NULL; Py_BEGIN_ALLOW_THREADS - rtn2 = mvwgetstr(self->win,y,x,rtn); + rtn2 = mvwgetnstr(self->win,y,x,rtn, 1023); Py_END_ALLOW_THREADS break; case 3: @@ -825,11 +829,11 @@ /* Untested */ Py_BEGIN_ALLOW_THREADS rtn2 = wmove(self->win,y,x)==ERR ? ERR : - wgetnstr(self->win, rtn, n); + wgetnstr(self->win, rtn, MIN(n, 1023)); Py_END_ALLOW_THREADS #else Py_BEGIN_ALLOW_THREADS - rtn2 = mvwgetnstr(self->win, y, x, rtn, n); + rtn2 = mvwgetnstr(self->win, y, x, rtn, MIN(n, 1023)); Py_END_ALLOW_THREADS #endif break; @@ -962,22 +966,22 @@ switch (PyTuple_Size(args)) { case 0: - rtn2 = winstr(self->win,rtn); + rtn2 = winnstr(self->win,rtn, 1023); break; case 1: if (!PyArg_ParseTuple(args,"i;n", &n)) return NULL; - rtn2 = winnstr(self->win,rtn,n); + rtn2 = winnstr(self->win,rtn,MIN(n,1023)); break; case 2: if (!PyArg_ParseTuple(args,"ii;y,x",&y,&x)) return NULL; - rtn2 = mvwinstr(self->win,y,x,rtn); + rtn2 = mvwinnstr(self->win,y,x,rtn,1023); break; case 3: if (!PyArg_ParseTuple(args, "iii;y,x,n", &y, &x, &n)) return NULL; - rtn2 = mvwinnstr(self->win, y, x, rtn, n); + rtn2 = mvwinnstr(self->win, y, x, rtn, MIN(n,1023)); break; default: PyErr_SetString(PyExc_TypeError, "instr requires 0 or 3 arguments"); From anthonybaxter at users.sourceforge.net Fri Jun 10 08:41:41 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:41:41 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor - New directory Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22127/distutils_refactor Log Message: Directory /cvsroot/python/python/nondist/sandbox/distutils_refactor added to the repository From anthonybaxter at users.sourceforge.net Fri Jun 10 08:42:15 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:42:15 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils - New directory Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22357/distutils Log Message: Directory /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils added to the repository From anthonybaxter at users.sourceforge.net Fri Jun 10 08:42:30 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:42:30 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils/command - New directory Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22497/command Log Message: Directory /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/command added to the repository From anthonybaxter at users.sourceforge.net Fri Jun 10 08:42:31 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:42:31 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils/tests - New directory Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22497/tests Log Message: Directory /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/tests added to the repository From anthonybaxter at users.sourceforge.net Fri Jun 10 08:43:25 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:43:25 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils/tests __init__.py, NONE, 1.1 support.py, NONE, 1.1 test_build_py.py, NONE, 1.1 test_build_scripts.py, NONE, 1.1 test_dist.py, NONE, 1.1 test_install.py, NONE, 1.1 test_install_scripts.py, NONE, 1.1 test_versionpredicate.py, NONE, 1.1 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22974/distutils_refactor/distutils/tests Added Files: __init__.py support.py test_build_py.py test_build_scripts.py test_dist.py test_install.py test_install_scripts.py test_versionpredicate.py Log Message: checking into the sandbox --- NEW FILE: __init__.py --- """Test suite for distutils. This test suite consists of a collection of test modules in the distutils.tests package. Each test module has a name starting with 'test' and contains a function test_suite(). The function is expected to return an initialized unittest.TestSuite instance. Tests for the command classes in the distutils.command package are included in distutils.tests as well, instead of using a separate distutils.command.tests package, since command identification is done by import rather than matching pre-defined names. """ import os import sys import unittest here = os.path.dirname(__file__) def test_suite(): suite = unittest.TestSuite() for fn in os.listdir(here): if fn.startswith("test") and fn.endswith(".py"): modname = "distutils.tests." + fn[:-3] __import__(modname) module = sys.modules[modname] suite.addTest(module.test_suite()) return suite if __name__ == "__main__": unittest.main(defaultTest="test_suite") --- NEW FILE: support.py --- """Support code for distutils test cases.""" import shutil import tempfile from distutils import log class LoggingSilencer(object): def setUp(self): super(LoggingSilencer, self).setUp() self.threshold = log.set_threshold(log.FATAL) def tearDown(self): log.set_threshold(self.threshold) super(LoggingSilencer, self).tearDown() class TempdirManager(object): """Mix-in class that handles temporary directories for test cases. This is intended to be used with unittest.TestCase. """ def setUp(self): super(TempdirManager, self).setUp() self.tempdirs = [] def tearDown(self): super(TempdirManager, self).tearDown() while self.tempdirs: d = self.tempdirs.pop() shutil.rmtree(d) def mkdtemp(self): """Create a temporary directory that will be cleaned up. Returns the path of the directory. """ d = tempfile.mkdtemp() self.tempdirs.append(d) return d class DummyCommand: """Class to store options for retrieval via set_undefined_options().""" def __init__(self, **kwargs): for kw, val in kwargs.items(): setattr(self, kw, val) def ensure_finalized(self): pass --- NEW FILE: test_build_py.py --- """Tests for distutils.command.build_py.""" import os import unittest from distutils.command.build_py import build_py from distutils.core import Distribution from distutils.tests import support class BuildPyTestCase(support.TempdirManager, support.LoggingSilencer, unittest.TestCase): def test_package_data(self): sources = self.mkdtemp() f = open(os.path.join(sources, "__init__.py"), "w") f.write("# Pretend this is a package.") f.close() f = open(os.path.join(sources, "README.txt"), "w") f.write("Info about this package") f.close() destination = self.mkdtemp() dist = Distribution({"packages": ["pkg"], "package_dir": {"pkg": sources}}) # script_name need not exist, it just need to be initialized dist.script_name = os.path.join(sources, "setup.py") dist.command_obj["build"] = support.DummyCommand( force=0, build_lib=destination) dist.packages = ["pkg"] dist.package_data = {"pkg": ["README.txt"]} dist.package_dir = {"pkg": sources} cmd = build_py(dist) cmd.compile = 1 cmd.ensure_finalized() self.assertEqual(cmd.package_data, dist.package_data) cmd.run() # This makes sure the list of outputs includes byte-compiled # files for Python modules but not for package data files # (there shouldn't *be* byte-code files for those!). # self.assertEqual(len(cmd.get_outputs()), 3) pkgdest = os.path.join(destination, "pkg") files = os.listdir(pkgdest) self.assert_("__init__.py" in files) self.assert_("__init__.pyc" in files) self.assert_("README.txt" in files) def test_suite(): return unittest.makeSuite(BuildPyTestCase) if __name__ == "__main__": unittest.main(defaultTest="test_suite") --- NEW FILE: test_build_scripts.py --- """Tests for distutils.command.build_scripts.""" import os import unittest from distutils.command.build_scripts import build_scripts from distutils.core import Distribution from distutils.tests import support class BuildScriptsTestCase(support.TempdirManager, support.LoggingSilencer, unittest.TestCase): def test_default_settings(self): cmd = self.get_build_scripts_cmd("/foo/bar", []) self.assert_(not cmd.force) self.assert_(cmd.build_dir is None) cmd.finalize_options() self.assert_(cmd.force) self.assertEqual(cmd.build_dir, "/foo/bar") def test_build(self): source = self.mkdtemp() target = self.mkdtemp() expected = self.write_sample_scripts(source) cmd = self.get_build_scripts_cmd(target, [os.path.join(source, fn) for fn in expected]) cmd.finalize_options() cmd.run() built = os.listdir(target) for name in expected: self.assert_(name in built) def get_build_scripts_cmd(self, target, scripts): import sys dist = Distribution() dist.scripts = scripts dist.command_obj["build"] = support.DummyCommand( build_scripts=target, force=1, executable=sys.executable ) return build_scripts(dist) def write_sample_scripts(self, dir): expected = [] expected.append("script1.py") self.write_script(dir, "script1.py", ("#! /usr/bin/env python2.3\n" "# bogus script w/ Python sh-bang\n" "pass\n")) expected.append("script2.py") self.write_script(dir, "script2.py", ("#!/usr/bin/python\n" "# bogus script w/ Python sh-bang\n" "pass\n")) expected.append("shell.sh") self.write_script(dir, "shell.sh", ("#!/bin/sh\n" "# bogus shell script w/ sh-bang\n" "exit 0\n")) return expected def write_script(self, dir, name, text): f = open(os.path.join(dir, name), "w") f.write(text) f.close() def test_suite(): return unittest.makeSuite(BuildScriptsTestCase) if __name__ == "__main__": unittest.main(defaultTest="test_suite") --- NEW FILE: test_dist.py --- """Tests for distutils.dist.""" import distutils.cmd import distutils.dist import os import shutil import StringIO import sys import tempfile import unittest from test.test_support import TESTFN class test_dist(distutils.cmd.Command): """Sample distutils extension command.""" user_options = [ ("sample-option=", "S", "help text"), ] def initialize_options(self): self.sample_option = None class TestDistribution(distutils.dist.Distribution): """Distribution subclasses that avoids the default search for configuration files. The ._config_files attribute must be set before .parse_config_files() is called. """ def find_config_files(self): return self._config_files class DistributionTestCase(unittest.TestCase): def setUp(self): self.argv = sys.argv[:] del sys.argv[1:] def tearDown(self): sys.argv[:] = self.argv def create_distribution(self, configfiles=()): d = TestDistribution() d._config_files = configfiles d.parse_config_files() d.parse_command_line() return d def test_command_packages_unspecified(self): sys.argv.append("build") d = self.create_distribution() self.assertEqual(d.get_command_packages(), ["distutils.command"]) def test_command_packages_cmdline(self): sys.argv.extend(["--command-packages", "foo.bar,distutils.tests", "test_dist", "-Ssometext", ]) d = self.create_distribution() # let's actually try to load our test command: self.assertEqual(d.get_command_packages(), ["distutils.command", "foo.bar", "distutils.tests"]) cmd = d.get_command_obj("test_dist") self.assert_(isinstance(cmd, test_dist)) self.assertEqual(cmd.sample_option, "sometext") def test_command_packages_configfile(self): sys.argv.append("build") f = open(TESTFN, "w") try: print >>f, "[global]" print >>f, "command_packages = foo.bar, splat" f.close() d = self.create_distribution([TESTFN]) self.assertEqual(d.get_command_packages(), ["distutils.command", "foo.bar", "splat"]) # ensure command line overrides config: sys.argv[1:] = ["--command-packages", "spork", "build"] d = self.create_distribution([TESTFN]) self.assertEqual(d.get_command_packages(), ["distutils.command", "spork"]) # Setting --command-packages to '' should cause the default to # be used even if a config file specified something else: sys.argv[1:] = ["--command-packages", "", "build"] d = self.create_distribution([TESTFN]) self.assertEqual(d.get_command_packages(), ["distutils.command"]) finally: os.unlink(TESTFN) class MetadataTestCase(unittest.TestCase): def test_simple_metadata(self): attrs = {"name": "package", "version": "1.0"} dist = distutils.dist.Distribution(attrs) meta = self.format_metadata(dist) self.assert_("Metadata-Version: 1.0" in meta) self.assert_("provides:" not in meta.lower()) self.assert_("requires:" not in meta.lower()) self.assert_("obsoletes:" not in meta.lower()) def test_provides(self): attrs = {"name": "package", "version": "1.0", "provides": ["package", "package.sub"]} dist = distutils.dist.Distribution(attrs) self.assertEqual(dist.metadata.get_provides(), ["package", "package.sub"]) self.assertEqual(dist.get_provides(), ["package", "package.sub"]) meta = self.format_metadata(dist) self.assert_("Metadata-Version: 1.1" in meta) self.assert_("requires:" not in meta.lower()) self.assert_("obsoletes:" not in meta.lower()) def test_provides_illegal(self): self.assertRaises(ValueError, distutils.dist.Distribution, {"name": "package", "version": "1.0", "provides": ["my.pkg (splat)"]}) def test_requires(self): attrs = {"name": "package", "version": "1.0", "requires": ["other", "another (==1.0)"]} dist = distutils.dist.Distribution(attrs) self.assertEqual(dist.metadata.get_requires(), ["other", "another (==1.0)"]) self.assertEqual(dist.get_requires(), ["other", "another (==1.0)"]) meta = self.format_metadata(dist) self.assert_("Metadata-Version: 1.1" in meta) self.assert_("provides:" not in meta.lower()) self.assert_("Requires: other" in meta) self.assert_("Requires: another (==1.0)" in meta) self.assert_("obsoletes:" not in meta.lower()) def test_requires_illegal(self): self.assertRaises(ValueError, distutils.dist.Distribution, {"name": "package", "version": "1.0", "requires": ["my.pkg (splat)"]}) def test_obsoletes(self): attrs = {"name": "package", "version": "1.0", "obsoletes": ["other", "another (<1.0)"]} dist = distutils.dist.Distribution(attrs) self.assertEqual(dist.metadata.get_obsoletes(), ["other", "another (<1.0)"]) self.assertEqual(dist.get_obsoletes(), ["other", "another (<1.0)"]) meta = self.format_metadata(dist) self.assert_("Metadata-Version: 1.1" in meta) self.assert_("provides:" not in meta.lower()) self.assert_("requires:" not in meta.lower()) self.assert_("Obsoletes: other" in meta) self.assert_("Obsoletes: another (<1.0)" in meta) def test_obsoletes_illegal(self): self.assertRaises(ValueError, distutils.dist.Distribution, {"name": "package", "version": "1.0", "obsoletes": ["my.pkg (splat)"]}) def format_metadata(self, dist): sio = StringIO.StringIO() dist.metadata.write_pkg_file(sio) return sio.getvalue() def test_suite(): suite = unittest.TestSuite() suite.addTest(unittest.makeSuite(DistributionTestCase)) suite.addTest(unittest.makeSuite(MetadataTestCase)) return suite --- NEW FILE: test_install.py --- """Tests for distutils.command.install.""" import os import unittest from distutils.command.install import install from distutils.core import Distribution from distutils.tests import support class InstallTestCase(support.TempdirManager, unittest.TestCase): def test_home_installation_scheme(self): # This ensure two things: # - that --home generates the desired set of directory names # - test --home is supported on all platforms builddir = self.mkdtemp() destination = os.path.join(builddir, "installation") dist = Distribution({"name": "foopkg"}) # script_name need not exist, it just need to be initialized dist.script_name = os.path.join(builddir, "setup.py") dist.command_obj["build"] = support.DummyCommand( build_base=builddir, build_lib=os.path.join(builddir, "lib"), ) cmd = install(dist) cmd.home = destination cmd.ensure_finalized() self.assertEqual(cmd.install_base, destination) self.assertEqual(cmd.install_platbase, destination) def check_path(got, expected): got = os.path.normpath(got) expected = os.path.normpath(expected) self.assertEqual(got, expected) libdir = os.path.join(destination, "lib", "python") check_path(cmd.install_lib, libdir) check_path(cmd.install_platlib, libdir) check_path(cmd.install_purelib, libdir) check_path(cmd.install_headers, os.path.join(destination, "include", "python", "foopkg")) check_path(cmd.install_scripts, os.path.join(destination, "bin")) check_path(cmd.install_data, destination) def test_suite(): return unittest.makeSuite(InstallTestCase) if __name__ == "__main__": unittest.main(defaultTest="test_suite") --- NEW FILE: test_install_scripts.py --- """Tests for distutils.command.install_scripts.""" import os import unittest from distutils.command.install_scripts import install_scripts from distutils.core import Distribution from distutils.tests import support class InstallScriptsTestCase(support.TempdirManager, support.LoggingSilencer, unittest.TestCase): def test_default_settings(self): dist = Distribution() dist.command_obj["build"] = support.DummyCommand( build_scripts="/foo/bar") dist.command_obj["install"] = support.DummyCommand( install_scripts="/splat/funk", force=1, skip_build=1, ) cmd = install_scripts(dist) self.assert_(not cmd.force) self.assert_(not cmd.skip_build) self.assert_(cmd.build_dir is None) self.assert_(cmd.install_dir is None) cmd.finalize_options() self.assert_(cmd.force) self.assert_(cmd.skip_build) self.assertEqual(cmd.build_dir, "/foo/bar") self.assertEqual(cmd.install_dir, "/splat/funk") def test_installation(self): source = self.mkdtemp() expected = [] def write_script(name, text): expected.append(name) f = open(os.path.join(source, name), "w") f.write(text) f.close() write_script("script1.py", ("#! /usr/bin/env python2.3\n" "# bogus script w/ Python sh-bang\n" "pass\n")) write_script("script2.py", ("#!/usr/bin/python\n" "# bogus script w/ Python sh-bang\n" "pass\n")) write_script("shell.sh", ("#!/bin/sh\n" "# bogus shell script w/ sh-bang\n" "exit 0\n")) target = self.mkdtemp() dist = Distribution() dist.command_obj["build"] = support.DummyCommand(build_scripts=source) dist.command_obj["install"] = support.DummyCommand( install_scripts=target, force=1, skip_build=1, ) cmd = install_scripts(dist) cmd.finalize_options() cmd.run() installed = os.listdir(target) for name in expected: self.assert_(name in installed) def test_suite(): return unittest.makeSuite(InstallScriptsTestCase) if __name__ == "__main__": unittest.main(defaultTest="test_suite") --- NEW FILE: test_versionpredicate.py --- """Tests harness for distutils.versionpredicate. """ import distutils.versionpredicate import doctest def test_suite(): return doctest.DocTestSuite(distutils.versionpredicate) From anthonybaxter at users.sourceforge.net Fri Jun 10 08:43:25 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:43:25 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils/command __init__.py, NONE, 1.1 bdist.py, NONE, 1.1 bdist_dumb.py, NONE, 1.1 bdist_rpm.py, NONE, 1.1 bdist_wininst.py, NONE, 1.1 build.py, NONE, 1.1 build_clib.py, NONE, 1.1 build_ext.py, NONE, 1.1 build_py.py, NONE, 1.1 build_scripts.py, NONE, 1.1 clean.py, NONE, 1.1 config.py, NONE, 1.1 install.py, NONE, 1.1 install_data.py, NONE, 1.1 install_headers.py, NONE, 1.1 install_lib.py, NONE, 1.1 install_scripts.py, NONE, 1.1 register.py, NONE, 1.1 sdist.py, NONE, 1.1 upload.py, NONE, 1.1 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22974/distutils_refactor/distutils/command Added Files: __init__.py bdist.py bdist_dumb.py bdist_rpm.py bdist_wininst.py build.py build_clib.py build_ext.py build_py.py build_scripts.py clean.py config.py install.py install_data.py install_headers.py install_lib.py install_scripts.py register.py sdist.py upload.py Log Message: checking into the sandbox --- NEW FILE: __init__.py --- """distutils.command Package containing implementation of all the standard Distutils commands.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: __init__.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" __all__ = ['build', 'build_py', 'build_ext', 'build_clib', 'build_scripts', 'clean', 'install', 'install_lib', 'install_headers', 'install_scripts', 'install_data', 'sdist', 'register', 'bdist', 'bdist_dumb', 'bdist_rpm', 'bdist_wininst', # These two are reserved for future use: #'bdist_sdux', #'bdist_pkgtool', # Note: # bdist_packager is not included because it only provides # an abstract base class ] --- NEW FILE: bdist.py --- """distutils.command.bdist Implements the Distutils 'bdist' command (create a built [binary] distribution).""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: bdist.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os, string from types import * from distutils.core import Command from distutils.errors import * from distutils.util import get_platform def show_formats (): """Print list of available formats (arguments to "--format" option). """ from distutils.fancy_getopt import FancyGetopt formats=[] for format in bdist.format_commands: formats.append(("formats=" + format, None, bdist.format_command[format][1])) pretty_printer = FancyGetopt(formats) pretty_printer.print_help("List of available distribution formats:") class bdist (Command): description = "create a built (binary) distribution" user_options = [('bdist-base=', 'b', "temporary directory for creating built distributions"), ('plat-name=', 'p', "platform name to embed in generated filenames " "(default: %s)" % get_platform()), ('formats=', None, "formats for distribution (comma-separated list)"), ('dist-dir=', 'd', "directory to put final built distributions in " "[default: dist]"), ('skip-build', None, "skip rebuilding everything (for testing/debugging)"), ] boolean_options = ['skip-build'] help_options = [ ('help-formats', None, "lists available distribution formats", show_formats), ] # The following commands do not take a format option from bdist no_format_option = ('bdist_rpm', #'bdist_sdux', 'bdist_pkgtool' ) # This won't do in reality: will need to distinguish RPM-ish Linux, # Debian-ish Linux, Solaris, FreeBSD, ..., Windows, Mac OS. default_format = { 'posix': 'gztar', 'nt': 'zip', 'os2': 'zip', } # Establish the preferred order (for the --help-formats option). format_commands = ['rpm', 'gztar', 'bztar', 'ztar', 'tar', 'wininst', 'zip', #'pkgtool', 'sdux' ] # And the real information. format_command = { 'rpm': ('bdist_rpm', "RPM distribution"), 'zip': ('bdist_dumb', "ZIP file"), 'gztar': ('bdist_dumb', "gzip'ed tar file"), 'bztar': ('bdist_dumb', "bzip2'ed tar file"), 'ztar': ('bdist_dumb', "compressed tar file"), 'tar': ('bdist_dumb', "tar file"), 'wininst': ('bdist_wininst', "Windows executable installer"), 'zip': ('bdist_dumb', "ZIP file"), #'pkgtool': ('bdist_pkgtool', # "Solaris pkgtool distribution"), #'sdux': ('bdist_sdux', "HP-UX swinstall depot"), } def initialize_options (self): self.bdist_base = None self.plat_name = None self.formats = None self.dist_dir = None self.skip_build = 0 # initialize_options() def finalize_options (self): # have to finalize 'plat_name' before 'bdist_base' if self.plat_name is None: self.plat_name = get_platform() # 'bdist_base' -- parent of per-built-distribution-format # temporary directories (eg. we'll probably have # "build/bdist./dumb", "build/bdist./rpm", etc.) if self.bdist_base is None: build_base = self.get_finalized_command('build').build_base self.bdist_base = os.path.join(build_base, 'bdist.' + self.plat_name) self.ensure_string_list('formats') if self.formats is None: try: self.formats = [self.default_format[os.name]] except KeyError: raise DistutilsPlatformError, \ "don't know how to create built distributions " + \ "on platform %s" % os.name if self.dist_dir is None: self.dist_dir = "dist" # finalize_options() def run (self): # Figure out which sub-commands we need to run. commands = [] for format in self.formats: try: commands.append(self.format_command[format][0]) except KeyError: raise DistutilsOptionError, "invalid format '%s'" % format # Reinitialize and run each command. for i in range(len(self.formats)): cmd_name = commands[i] sub_cmd = self.reinitialize_command(cmd_name) if cmd_name not in self.no_format_option: sub_cmd.format = self.formats[i] # If we're going to need to run this command again, tell it to # keep its temporary files around so subsequent runs go faster. if cmd_name in commands[i+1:]: sub_cmd.keep_temp = 1 self.run_command(cmd_name) # run() # class bdist --- NEW FILE: bdist_dumb.py --- """distutils.command.bdist_dumb Implements the Distutils 'bdist_dumb' command (create a "dumb" built distribution -- i.e., just an archive to be unpacked under $prefix or $exec_prefix).""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: bdist_dumb.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os from distutils.core import Command from distutils.util import get_platform from distutils.dir_util import create_tree, remove_tree, ensure_relative from distutils.errors import * from distutils.sysconfig import get_python_version from distutils import log class bdist_dumb (Command): description = "create a \"dumb\" built distribution" user_options = [('bdist-dir=', 'd', "temporary directory for creating the distribution"), ('plat-name=', 'p', "platform name to embed in generated filenames " "(default: %s)" % get_platform()), ('format=', 'f', "archive format to create (tar, ztar, gztar, zip)"), ('keep-temp', 'k', "keep the pseudo-installation tree around after " + "creating the distribution archive"), ('dist-dir=', 'd', "directory to put final built distributions in"), ('skip-build', None, "skip rebuilding everything (for testing/debugging)"), ('relative', None, "build the archive using relative paths" "(default: false)"), ] boolean_options = ['keep-temp', 'skip-build', 'relative'] default_format = { 'posix': 'gztar', 'nt': 'zip', 'os2': 'zip' } def initialize_options (self): self.bdist_dir = None self.plat_name = None self.format = None self.keep_temp = 0 self.dist_dir = None self.skip_build = 0 self.relative = 0 # initialize_options() def finalize_options (self): if self.bdist_dir is None: bdist_base = self.get_finalized_command('bdist').bdist_base self.bdist_dir = os.path.join(bdist_base, 'dumb') if self.format is None: try: self.format = self.default_format[os.name] except KeyError: raise DistutilsPlatformError, \ ("don't know how to create dumb built distributions " + "on platform %s") % os.name self.set_undefined_options('bdist', ('dist_dir', 'dist_dir'), ('plat_name', 'plat_name')) # finalize_options() def run (self): if not self.skip_build: self.run_command('build') install = self.reinitialize_command('install', reinit_subcommands=1) install.root = self.bdist_dir install.skip_build = self.skip_build install.warn_dir = 0 log.info("installing to %s" % self.bdist_dir) self.run_command('install') # And make an archive relative to the root of the # pseudo-installation tree. archive_basename = "%s.%s" % (self.distribution.get_fullname(), self.plat_name) # OS/2 objects to any ":" characters in a filename (such as when # a timestamp is used in a version) so change them to hyphens. if os.name == "os2": archive_basename = archive_basename.replace(":", "-") pseudoinstall_root = os.path.join(self.dist_dir, archive_basename) if not self.relative: archive_root = self.bdist_dir else: if (self.distribution.has_ext_modules() and (install.install_base != install.install_platbase)): raise DistutilsPlatformError, \ ("can't make a dumb built distribution where " "base and platbase are different (%s, %s)" % (repr(install.install_base), repr(install.install_platbase))) else: archive_root = os.path.join(self.bdist_dir, ensure_relative(install.install_base)) # Make the archive filename = self.make_archive(pseudoinstall_root, self.format, root_dir=archive_root) if self.distribution.has_ext_modules(): pyversion = get_python_version() else: pyversion = 'any' self.distribution.dist_files.append(('bdist_dumb', pyversion, filename)) if not self.keep_temp: remove_tree(self.bdist_dir, dry_run=self.dry_run) # run() # class bdist_dumb --- NEW FILE: bdist_rpm.py --- """distutils.command.bdist_rpm Implements the Distutils 'bdist_rpm' command (create RPM source and binary distributions).""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: bdist_rpm.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string import glob from types import * from distutils.core import Command from distutils.debug import DEBUG from distutils.util import get_platform from distutils.file_util import write_file from distutils.errors import * from distutils.sysconfig import get_python_version from distutils import log class bdist_rpm (Command): description = "create an RPM distribution" user_options = [ ('bdist-base=', None, "base directory for creating built distributions"), ('rpm-base=', None, "base directory for creating RPMs (defaults to \"rpm\" under " "--bdist-base; must be specified for RPM 2)"), ('dist-dir=', 'd', "directory to put final RPM files in " "(and .spec files if --spec-only)"), ('python=', None, "path to Python interpreter to hard-code in the .spec file " "(default: \"python\")"), ('fix-python', None, "hard-code the exact path to the current Python interpreter in " "the .spec file"), ('spec-only', None, "only regenerate spec file"), ('source-only', None, "only generate source RPM"), ('binary-only', None, "only generate binary RPM"), ('use-bzip2', None, "use bzip2 instead of gzip to create source distribution"), # More meta-data: too RPM-specific to put in the setup script, # but needs to go in the .spec file -- so we make these options # to "bdist_rpm". The idea is that packagers would put this # info in setup.cfg, although they are of course free to # supply it on the command line. ('distribution-name=', None, "name of the (Linux) distribution to which this " "RPM applies (*not* the name of the module distribution!)"), ('group=', None, "package classification [default: \"Development/Libraries\"]"), ('release=', None, "RPM release number"), ('serial=', None, "RPM serial number"), ('vendor=', None, "RPM \"vendor\" (eg. \"Joe Blow \") " "[default: maintainer or author from setup script]"), ('packager=', None, "RPM packager (eg. \"Jane Doe \")" "[default: vendor]"), ('doc-files=', None, "list of documentation files (space or comma-separated)"), ('changelog=', None, "RPM changelog"), ('icon=', None, "name of icon file"), ('provides=', None, "capabilities provided by this package"), ('requires=', None, "capabilities required by this package"), ('conflicts=', None, "capabilities which conflict with this package"), ('build-requires=', None, "capabilities required to build this package"), ('obsoletes=', None, "capabilities made obsolete by this package"), ('no-autoreq', None, "do not automatically calculate dependencies"), # Actions to take when building RPM ('keep-temp', 'k', "don't clean up RPM build directory"), ('no-keep-temp', None, "clean up RPM build directory [default]"), ('use-rpm-opt-flags', None, "compile with RPM_OPT_FLAGS when building from source RPM"), ('no-rpm-opt-flags', None, "do not pass any RPM CFLAGS to compiler"), ('rpm3-mode', None, "RPM 3 compatibility mode (default)"), ('rpm2-mode', None, "RPM 2 compatibility mode"), # Add the hooks necessary for specifying custom scripts ('prep-script=', None, "Specify a script for the PREP phase of RPM building"), ('build-script=', None, "Specify a script for the BUILD phase of RPM building"), ('pre-install=', None, "Specify a script for the pre-INSTALL phase of RPM building"), ('install-script=', None, "Specify a script for the INSTALL phase of RPM building"), ('post-install=', None, "Specify a script for the post-INSTALL phase of RPM building"), ('pre-uninstall=', None, "Specify a script for the pre-UNINSTALL phase of RPM building"), ('post-uninstall=', None, "Specify a script for the post-UNINSTALL phase of RPM building"), ('clean-script=', None, "Specify a script for the CLEAN phase of RPM building"), ('verify-script=', None, "Specify a script for the VERIFY phase of the RPM build"), # Allow a packager to explicitly force an architecture ('force-arch=', None, "Force an architecture onto the RPM build process"), ] boolean_options = ['keep-temp', 'use-rpm-opt-flags', 'rpm3-mode', 'no-autoreq'] negative_opt = {'no-keep-temp': 'keep-temp', 'no-rpm-opt-flags': 'use-rpm-opt-flags', 'rpm2-mode': 'rpm3-mode'} def initialize_options (self): self.bdist_base = None self.rpm_base = None self.dist_dir = None self.python = None self.fix_python = None self.spec_only = None self.binary_only = None self.source_only = None self.use_bzip2 = None self.distribution_name = None self.group = None self.release = None self.serial = None self.vendor = None self.packager = None self.doc_files = None self.changelog = None self.icon = None self.prep_script = None self.build_script = None self.install_script = None self.clean_script = None self.verify_script = None self.pre_install = None self.post_install = None self.pre_uninstall = None self.post_uninstall = None self.prep = None self.provides = None self.requires = None self.conflicts = None self.build_requires = None self.obsoletes = None self.keep_temp = 0 self.use_rpm_opt_flags = 1 self.rpm3_mode = 1 self.no_autoreq = 0 self.force_arch = None # initialize_options() def finalize_options (self): self.set_undefined_options('bdist', ('bdist_base', 'bdist_base')) if self.rpm_base is None: if not self.rpm3_mode: raise DistutilsOptionError, \ "you must specify --rpm-base in RPM 2 mode" self.rpm_base = os.path.join(self.bdist_base, "rpm") if self.python is None: if self.fix_python: self.python = sys.executable else: self.python = "python" elif self.fix_python: raise DistutilsOptionError, \ "--python and --fix-python are mutually exclusive options" if os.name != 'posix': raise DistutilsPlatformError, \ ("don't know how to create RPM " "distributions on platform %s" % os.name) if self.binary_only and self.source_only: raise DistutilsOptionError, \ "cannot supply both '--source-only' and '--binary-only'" # don't pass CFLAGS to pure python distributions if not self.distribution.has_ext_modules(): self.use_rpm_opt_flags = 0 self.set_undefined_options('bdist', ('dist_dir', 'dist_dir')) self.finalize_package_data() # finalize_options() def finalize_package_data (self): self.ensure_string('group', "Development/Libraries") self.ensure_string('vendor', "%s <%s>" % (self.distribution.get_contact(), self.distribution.get_contact_email())) self.ensure_string('packager') self.ensure_string_list('doc_files') if type(self.doc_files) is ListType: for readme in ('README', 'README.txt'): if os.path.exists(readme) and readme not in self.doc_files: self.doc_files.append(readme) self.ensure_string('release', "1") self.ensure_string('serial') # should it be an int? self.ensure_string('distribution_name') self.ensure_string('changelog') # Format changelog correctly self.changelog = self._format_changelog(self.changelog) self.ensure_filename('icon') self.ensure_filename('prep_script') self.ensure_filename('build_script') self.ensure_filename('install_script') self.ensure_filename('clean_script') self.ensure_filename('verify_script') self.ensure_filename('pre_install') self.ensure_filename('post_install') self.ensure_filename('pre_uninstall') self.ensure_filename('post_uninstall') # XXX don't forget we punted on summaries and descriptions -- they # should be handled here eventually! # Now *this* is some meta-data that belongs in the setup script... self.ensure_string_list('provides') self.ensure_string_list('requires') self.ensure_string_list('conflicts') self.ensure_string_list('build_requires') self.ensure_string_list('obsoletes') self.ensure_string('force_arch') # finalize_package_data () def run (self): if DEBUG: print "before _get_package_data():" print "vendor =", self.vendor print "packager =", self.packager print "doc_files =", self.doc_files print "changelog =", self.changelog # make directories if self.spec_only: spec_dir = self.dist_dir self.mkpath(spec_dir) else: rpm_dir = {} for d in ('SOURCES', 'SPECS', 'BUILD', 'RPMS', 'SRPMS'): rpm_dir[d] = os.path.join(self.rpm_base, d) self.mkpath(rpm_dir[d]) spec_dir = rpm_dir['SPECS'] # Spec file goes into 'dist_dir' if '--spec-only specified', # build/rpm. otherwise. spec_path = os.path.join(spec_dir, "%s.spec" % self.distribution.get_name()) self.execute(write_file, (spec_path, self._make_spec_file()), "writing '%s'" % spec_path) if self.spec_only: # stop if requested return # Make a source distribution and copy to SOURCES directory with # optional icon. saved_dist_files = self.distribution.dist_files[:] sdist = self.reinitialize_command('sdist') if self.use_bzip2: sdist.formats = ['bztar'] else: sdist.formats = ['gztar'] self.run_command('sdist') self.distribution.dist_files = saved_dist_files source = sdist.get_archive_files()[0] source_dir = rpm_dir['SOURCES'] self.copy_file(source, source_dir) if self.icon: if os.path.exists(self.icon): self.copy_file(self.icon, source_dir) else: raise DistutilsFileError, \ "icon file '%s' does not exist" % self.icon # build package log.info("building RPMs") rpm_cmd = ['rpm'] if os.path.exists('/usr/bin/rpmbuild') or \ os.path.exists('/bin/rpmbuild'): rpm_cmd = ['rpmbuild'] if self.source_only: # what kind of RPMs? rpm_cmd.append('-bs') elif self.binary_only: rpm_cmd.append('-bb') else: rpm_cmd.append('-ba') if self.rpm3_mode: rpm_cmd.extend(['--define', '_topdir %s' % os.path.abspath(self.rpm_base)]) if not self.keep_temp: rpm_cmd.append('--clean') rpm_cmd.append(spec_path) self.spawn(rpm_cmd) # XXX this is a nasty hack -- we really should have a proper way to # find out the names of the RPM files created; also, this assumes # that RPM creates exactly one source and one binary RPM. if not self.dry_run: if not self.binary_only: srpms = glob.glob(os.path.join(rpm_dir['SRPMS'], "*.rpm")) assert len(srpms) == 1, \ "unexpected number of SRPM files found: %s" % srpms dist_file = ('bdist_rpm', 'any', self._dist_path(srpms[0])) self.distribution.dist_files.append(dist_file) self.move_file(srpms[0], self.dist_dir) if not self.source_only: rpms = glob.glob(os.path.join(rpm_dir['RPMS'], "*/*.rpm")) debuginfo = glob.glob(os.path.join(rpm_dir['RPMS'], "*/*debuginfo*.rpm")) if debuginfo: rpms.remove(debuginfo[0]) assert len(rpms) == 1, \ "unexpected number of RPM files found: %s" % rpms dist_file = ('bdist_rpm', get_python_version(), self._dist_path(rpms[0])) self.distribution.dist_files.append(dist_file) self.move_file(rpms[0], self.dist_dir) if debuginfo: dist_file = ('bdist_rpm', get_python_version(), self._dist_path(debuginfo[0])) self.move_file(debuginfo[0], self.dist_dir) # run() def _dist_path(self, path): return os.path.join(self.dist_dir, os.path.basename(path)) def _make_spec_file(self): """Generate the text of an RPM spec file and return it as a list of strings (one per line). """ # definitions and headers spec_file = [ '%define name ' + self.distribution.get_name(), '%define version ' + self.distribution.get_version().replace('-','_'), '%define release ' + self.release.replace('-','_'), '', 'Summary: ' + self.distribution.get_description(), ] # put locale summaries into spec file # XXX not supported for now (hard to put a dictionary # in a config file -- arg!) #for locale in self.summaries.keys(): # spec_file.append('Summary(%s): %s' % (locale, # self.summaries[locale])) spec_file.extend([ 'Name: %{name}', 'Version: %{version}', 'Release: %{release}',]) # XXX yuck! this filename is available from the "sdist" command, # but only after it has run: and we create the spec file before # running "sdist", in case of --spec-only. if self.use_bzip2: spec_file.append('Source0: %{name}-%{version}.tar.bz2') else: spec_file.append('Source0: %{name}-%{version}.tar.gz') spec_file.extend([ 'License: ' + self.distribution.get_license(), 'Group: ' + self.group, 'BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-buildroot', 'Prefix: %{_prefix}', ]) if not self.force_arch: # noarch if no extension modules if not self.distribution.has_ext_modules(): spec_file.append('BuildArch: noarch') else: spec_file.append( 'BuildArch: %s' % self.force_arch ) for field in ('Vendor', 'Packager', 'Provides', 'Requires', 'Conflicts', 'Obsoletes', ): val = getattr(self, string.lower(field)) if type(val) is ListType: spec_file.append('%s: %s' % (field, string.join(val))) elif val is not None: spec_file.append('%s: %s' % (field, val)) if self.distribution.get_url() != 'UNKNOWN': spec_file.append('Url: ' + self.distribution.get_url()) if self.distribution_name: spec_file.append('Distribution: ' + self.distribution_name) if self.build_requires: spec_file.append('BuildRequires: ' + string.join(self.build_requires)) if self.icon: spec_file.append('Icon: ' + os.path.basename(self.icon)) if self.no_autoreq: spec_file.append('AutoReq: 0') spec_file.extend([ '', '%description', self.distribution.get_long_description() ]) # put locale descriptions into spec file # XXX again, suppressed because config file syntax doesn't # easily support this ;-( #for locale in self.descriptions.keys(): # spec_file.extend([ # '', # '%description -l ' + locale, # self.descriptions[locale], # ]) # rpm scripts # figure out default build script def_build = "%s setup.py build" % self.python if self.use_rpm_opt_flags: def_build = 'env CFLAGS="$RPM_OPT_FLAGS" ' + def_build # insert contents of files # XXX this is kind of misleading: user-supplied options are files # that we open and interpolate into the spec file, but the defaults # are just text that we drop in as-is. Hmmm. script_options = [ ('prep', 'prep_script', "%setup"), ('build', 'build_script', def_build), ('install', 'install_script', ("%s setup.py install " "--root=$RPM_BUILD_ROOT " "--record=INSTALLED_FILES") % self.python), ('clean', 'clean_script', "rm -rf $RPM_BUILD_ROOT"), ('verifyscript', 'verify_script', None), ('pre', 'pre_install', None), ('post', 'post_install', None), ('preun', 'pre_uninstall', None), ('postun', 'post_uninstall', None), ] for (rpm_opt, attr, default) in script_options: # Insert contents of file referred to, if no file is referred to # use 'default' as contents of script val = getattr(self, attr) if val or default: spec_file.extend([ '', '%' + rpm_opt,]) if val: spec_file.extend(string.split(open(val, 'r').read(), '\n')) else: spec_file.append(default) # files section spec_file.extend([ '', '%files -f INSTALLED_FILES', '%defattr(-,root,root)', ]) if self.doc_files: spec_file.append('%doc ' + string.join(self.doc_files)) if self.changelog: spec_file.extend([ '', '%changelog',]) spec_file.extend(self.changelog) return spec_file # _make_spec_file () def _format_changelog(self, changelog): """Format the changelog correctly and convert it to a list of strings """ if not changelog: return changelog new_changelog = [] for line in string.split(string.strip(changelog), '\n'): line = string.strip(line) if line[0] == '*': new_changelog.extend(['', line]) elif line[0] == '-': new_changelog.append(line) else: new_changelog.append(' ' + line) # strip trailing newline inserted by first changelog entry if not new_changelog[0]: del new_changelog[0] return new_changelog # _format_changelog() # class bdist_rpm --- NEW FILE: bdist_wininst.py --- """distutils.command.bdist_wininst Implements the Distutils 'bdist_wininst' command: create a windows installer exe-program.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: bdist_wininst.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string from distutils.core import Command from distutils.util import get_platform from distutils.dir_util import create_tree, remove_tree from distutils.errors import * from distutils.sysconfig import get_python_version from distutils import log class bdist_wininst (Command): description = "create an executable installer for MS Windows" user_options = [('bdist-dir=', None, "temporary directory for creating the distribution"), ('keep-temp', 'k', "keep the pseudo-installation tree around after " + "creating the distribution archive"), ('target-version=', None, "require a specific python version" + " on the target system"), ('no-target-compile', 'c', "do not compile .py to .pyc on the target system"), ('no-target-optimize', 'o', "do not compile .py to .pyo (optimized)" "on the target system"), ('dist-dir=', 'd', "directory to put final built distributions in"), ('bitmap=', 'b', "bitmap to use for the installer instead of python-powered logo"), ('title=', 't', "title to display on the installer background instead of default"), ('skip-build', None, "skip rebuilding everything (for testing/debugging)"), ('install-script=', None, "basename of installation script to be run after" "installation or before deinstallation"), ('pre-install-script=', None, "Fully qualified filename of a script to be run before " "any files are installed. This script need not be in the " "distribution"), ] boolean_options = ['keep-temp', 'no-target-compile', 'no-target-optimize', 'skip-build'] def initialize_options (self): self.bdist_dir = None self.keep_temp = 0 self.no_target_compile = 0 self.no_target_optimize = 0 self.target_version = None self.dist_dir = None self.bitmap = None self.title = None self.skip_build = 0 self.install_script = None self.pre_install_script = None # initialize_options() def finalize_options (self): if self.bdist_dir is None: bdist_base = self.get_finalized_command('bdist').bdist_base self.bdist_dir = os.path.join(bdist_base, 'wininst') if not self.target_version: self.target_version = "" if not self.skip_build and self.distribution.has_ext_modules(): short_version = get_python_version() if self.target_version and self.target_version != short_version: raise DistutilsOptionError, \ "target version can only be %s, or the '--skip_build'" \ " option must be specified" % (short_version,) self.target_version = short_version self.set_undefined_options('bdist', ('dist_dir', 'dist_dir')) if self.install_script: for script in self.distribution.scripts: if self.install_script == os.path.basename(script): break else: raise DistutilsOptionError, \ "install_script '%s' not found in scripts" % \ self.install_script # finalize_options() def run (self): if (sys.platform != "win32" and (self.distribution.has_ext_modules() or self.distribution.has_c_libraries())): raise DistutilsPlatformError \ ("distribution contains extensions and/or C libraries; " "must be compiled on a Windows 32 platform") if not self.skip_build: self.run_command('build') install = self.reinitialize_command('install', reinit_subcommands=1) install.root = self.bdist_dir install.skip_build = self.skip_build install.warn_dir = 0 install_lib = self.reinitialize_command('install_lib') # we do not want to include pyc or pyo files install_lib.compile = 0 install_lib.optimize = 0 if self.distribution.has_ext_modules(): # If we are building an installer for a Python version other # than the one we are currently running, then we need to ensure # our build_lib reflects the other Python version rather than ours. # Note that for target_version!=sys.version, we must have skipped the # build step, so there is no issue with enforcing the build of this # version. target_version = self.target_version if not target_version: assert self.skip_build, "Should have already checked this" target_version = sys.version[0:3] plat_specifier = ".%s-%s" % (get_platform(), target_version) build = self.get_finalized_command('build') build.build_lib = os.path.join(build.build_base, 'lib' + plat_specifier) # Use a custom scheme for the zip-file, because we have to decide # at installation time which scheme to use. for key in ('purelib', 'platlib', 'headers', 'scripts', 'data'): value = string.upper(key) if key == 'headers': value = value + '/Include/$dist_name' setattr(install, 'install_' + key, value) log.info("installing to %s", self.bdist_dir) install.ensure_finalized() # avoid warning of 'install_lib' about installing # into a directory not in sys.path sys.path.insert(0, os.path.join(self.bdist_dir, 'PURELIB')) install.run() del sys.path[0] # And make an archive relative to the root of the # pseudo-installation tree. from tempfile import mktemp archive_basename = mktemp() fullname = self.distribution.get_fullname() arcname = self.make_archive(archive_basename, "zip", root_dir=self.bdist_dir) # create an exe containing the zip-file self.create_exe(arcname, fullname, self.bitmap) if self.distribution.has_ext_modules(): pyversion = get_python_version() else: pyversion = 'any' self.distribution.dist_files.append(('bdist_wininst', pyversion, self.get_installer_filename(fullname))) # remove the zip-file again log.debug("removing temporary file '%s'", arcname) os.remove(arcname) if not self.keep_temp: remove_tree(self.bdist_dir, dry_run=self.dry_run) # run() def get_inidata (self): # Return data describing the installation. lines = [] metadata = self.distribution.metadata # Write the [metadata] section. lines.append("[metadata]") # 'info' will be displayed in the installer's dialog box, # describing the items to be installed. info = (metadata.long_description or '') + '\n' # Escape newline characters def escape(s): return string.replace(s, "\n", "\\n") for name in ["author", "author_email", "description", "maintainer", "maintainer_email", "name", "url", "version"]: data = getattr(metadata, name, "") if data: info = info + ("\n %s: %s" % \ (string.capitalize(name), escape(data))) lines.append("%s=%s" % (name, escape(data))) # The [setup] section contains entries controlling # the installer runtime. lines.append("\n[Setup]") if self.install_script: lines.append("install_script=%s" % self.install_script) lines.append("info=%s" % escape(info)) lines.append("target_compile=%d" % (not self.no_target_compile)) lines.append("target_optimize=%d" % (not self.no_target_optimize)) if self.target_version: lines.append("target_version=%s" % self.target_version) title = self.title or self.distribution.get_fullname() lines.append("title=%s" % escape(title)) import time import distutils build_info = "Built %s with distutils-%s" % \ (time.ctime(time.time()), distutils.__version__) lines.append("build_info=%s" % build_info) return string.join(lines, "\n") # get_inidata() def create_exe (self, arcname, fullname, bitmap=None): import struct self.mkpath(self.dist_dir) cfgdata = self.get_inidata() installer_name = self.get_installer_filename(fullname) self.announce("creating %s" % installer_name) if bitmap: bitmapdata = open(bitmap, "rb").read() bitmaplen = len(bitmapdata) else: bitmaplen = 0 file = open(installer_name, "wb") file.write(self.get_exe_bytes()) if bitmap: file.write(bitmapdata) # Convert cfgdata from unicode to ascii, mbcs encoded try: unicode except NameError: pass else: if isinstance(cfgdata, unicode): cfgdata = cfgdata.encode("mbcs") # Append the pre-install script cfgdata = cfgdata + "\0" if self.pre_install_script: script_data = open(self.pre_install_script, "r").read() cfgdata = cfgdata + script_data + "\n\0" else: # empty pre-install script cfgdata = cfgdata + "\0" file.write(cfgdata) # The 'magic number' 0x1234567B is used to make sure that the # binary layout of 'cfgdata' is what the wininst.exe binary # expects. If the layout changes, increment that number, make # the corresponding changes to the wininst.exe sources, and # recompile them. header = struct.pack(" cur_version: bv = get_build_version() else: if self.target_version < "2.4": bv = "6" else: bv = "7.1" else: # for current version - use authoritative check. bv = get_build_version() # wininst-x.y.exe is in the same directory as this file directory = os.path.dirname(__file__) # we must use a wininst-x.y.exe built with the same C compiler # used for python. XXX What about mingw, borland, and so on? filename = os.path.join(directory, "wininst-%s.exe" % bv) return open(filename, "rb").read() # class bdist_wininst --- NEW FILE: build.py --- """distutils.command.build Implements the Distutils 'build' command.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: build.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os from distutils.core import Command from distutils.util import get_platform def show_compilers (): from distutils.ccompiler import show_compilers show_compilers() class build (Command): description = "build everything needed to install" user_options = [ ('build-base=', 'b', "base directory for build library"), ('build-purelib=', None, "build directory for platform-neutral distributions"), ('build-platlib=', None, "build directory for platform-specific distributions"), ('build-lib=', None, "build directory for all distribution (defaults to either " + "build-purelib or build-platlib"), ('build-scripts=', None, "build directory for scripts"), ('build-temp=', 't', "temporary build directory"), ('compiler=', 'c', "specify the compiler type"), ('debug', 'g', "compile extensions and libraries with debugging information"), ('force', 'f', "forcibly build everything (ignore file timestamps)"), ('executable=', 'e', "specify final destination interpreter path (build.py)"), ] boolean_options = ['debug', 'force'] help_options = [ ('help-compiler', None, "list available compilers", show_compilers), ] def initialize_options (self): self.build_base = 'build' # these are decided only after 'build_base' has its final value # (unless overridden by the user or client) self.build_purelib = None self.build_platlib = None self.build_lib = None self.build_temp = None self.build_scripts = None self.compiler = None self.debug = None self.force = 0 self.executable = None def finalize_options (self): plat_specifier = ".%s-%s" % (get_platform(), sys.version[0:3]) # 'build_purelib' and 'build_platlib' just default to 'lib' and # 'lib.' under the base build directory. We only use one of # them for a given distribution, though -- if self.build_purelib is None: self.build_purelib = os.path.join(self.build_base, 'lib') if self.build_platlib is None: self.build_platlib = os.path.join(self.build_base, 'lib' + plat_specifier) # 'build_lib' is the actual directory that we will use for this # particular module distribution -- if user didn't supply it, pick # one of 'build_purelib' or 'build_platlib'. if self.build_lib is None: if self.distribution.ext_modules: self.build_lib = self.build_platlib else: self.build_lib = self.build_purelib # 'build_temp' -- temporary directory for compiler turds, # "build/temp." if self.build_temp is None: self.build_temp = os.path.join(self.build_base, 'temp' + plat_specifier) if self.build_scripts is None: self.build_scripts = os.path.join(self.build_base, 'scripts-' + sys.version[0:3]) if self.executable is None: self.executable = os.path.normpath(sys.executable) # finalize_options () def run (self): # Run all relevant sub-commands. This will be some subset of: # - build_py - pure Python modules # - build_clib - standalone C libraries # - build_ext - Python extensions # - build_scripts - (Python) scripts for cmd_name in self.get_sub_commands(): self.run_command(cmd_name) # -- Predicates for the sub-command list --------------------------- def has_pure_modules (self): return self.distribution.has_pure_modules() def has_c_libraries (self): return self.distribution.has_c_libraries() def has_ext_modules (self): return self.distribution.has_ext_modules() def has_scripts (self): return self.distribution.has_scripts() sub_commands = [('build_py', has_pure_modules), ('build_clib', has_c_libraries), ('build_ext', has_ext_modules), ('build_scripts', has_scripts), ] # class build --- NEW FILE: build_clib.py --- """distutils.command.build_clib Implements the Distutils 'build_clib' command, to build a C/C++ library that is included in the module distribution and needed by an extension module.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: build_clib.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" # XXX this module has *lots* of code ripped-off quite transparently from # build_ext.py -- not surprisingly really, as the work required to build # a static library from a collection of C source files is not really all # that different from what's required to build a shared object file from # a collection of C source files. Nevertheless, I haven't done the # necessary refactoring to account for the overlap in code between the # two modules, mainly because a number of subtle details changed in the # cut 'n paste. Sigh. import os, string from types import * from distutils.core import Command from distutils.errors import * from distutils.sysconfig import customize_compiler from distutils import log def show_compilers (): from distutils.ccompiler import show_compilers show_compilers() class build_clib (Command): description = "build C/C++ libraries used by Python extensions" user_options = [ ('build-clib', 'b', "directory to build C/C++ libraries to"), ('build-temp', 't', "directory to put temporary build by-products"), ('debug', 'g', "compile with debugging information"), ('force', 'f', "forcibly build everything (ignore file timestamps)"), ('compiler=', 'c', "specify the compiler type"), ] boolean_options = ['debug', 'force'] help_options = [ ('help-compiler', None, "list available compilers", show_compilers), ] def initialize_options (self): self.build_clib = None self.build_temp = None # List of libraries to build self.libraries = None # Compilation options for all libraries self.include_dirs = None self.define = None self.undef = None self.debug = None self.force = 0 self.compiler = None # initialize_options() def finalize_options (self): # This might be confusing: both build-clib and build-temp default # to build-temp as defined by the "build" command. This is because # I think that C libraries are really just temporary build # by-products, at least from the point of view of building Python # extensions -- but I want to keep my options open. self.set_undefined_options('build', ('build_temp', 'build_clib'), ('build_temp', 'build_temp'), ('compiler', 'compiler'), ('debug', 'debug'), ('force', 'force')) self.libraries = self.distribution.libraries if self.libraries: self.check_library_list(self.libraries) if self.include_dirs is None: self.include_dirs = self.distribution.include_dirs or [] if type(self.include_dirs) is StringType: self.include_dirs = string.split(self.include_dirs, os.pathsep) # XXX same as for build_ext -- what about 'self.define' and # 'self.undef' ? # finalize_options() def run (self): if not self.libraries: return # Yech -- this is cut 'n pasted from build_ext.py! from distutils.ccompiler import new_compiler self.compiler = new_compiler(compiler=self.compiler, dry_run=self.dry_run, force=self.force) customize_compiler(self.compiler) if self.include_dirs is not None: self.compiler.set_include_dirs(self.include_dirs) if self.define is not None: # 'define' option is a list of (name,value) tuples for (name,value) in self.define: self.compiler.define_macro(name, value) if self.undef is not None: for macro in self.undef: self.compiler.undefine_macro(macro) self.build_libraries(self.libraries) # run() def check_library_list (self, libraries): """Ensure that the list of libraries (presumably provided as a command option 'libraries') is valid, i.e. it is a list of 2-tuples, where the tuples are (library_name, build_info_dict). Raise DistutilsSetupError if the structure is invalid anywhere; just returns otherwise.""" # Yechh, blecch, ackk: this is ripped straight out of build_ext.py, # with only names changed to protect the innocent! if type(libraries) is not ListType: raise DistutilsSetupError, \ "'libraries' option must be a list of tuples" for lib in libraries: if type(lib) is not TupleType and len(lib) != 2: raise DistutilsSetupError, \ "each element of 'libraries' must a 2-tuple" if type(lib[0]) is not StringType: raise DistutilsSetupError, \ "first element of each tuple in 'libraries' " + \ "must be a string (the library name)" if '/' in lib[0] or (os.sep != '/' and os.sep in lib[0]): raise DistutilsSetupError, \ ("bad library name '%s': " + "may not contain directory separators") % \ lib[0] if type(lib[1]) is not DictionaryType: raise DistutilsSetupError, \ "second element of each tuple in 'libraries' " + \ "must be a dictionary (build info)" # for lib # check_library_list () def get_library_names (self): # Assume the library list is valid -- 'check_library_list()' is # called from 'finalize_options()', so it should be! if not self.libraries: return None lib_names = [] for (lib_name, build_info) in self.libraries: lib_names.append(lib_name) return lib_names # get_library_names () def get_source_files (self): self.check_library_list(self.libraries) filenames = [] for (lib_name, build_info) in self.libraries: sources = build_info.get('sources') if (sources is None or type(sources) not in (ListType, TupleType) ): raise DistutilsSetupError, \ ("in 'libraries' option (library '%s'), " "'sources' must be present and must be " "a list of source filenames") % lib_name filenames.extend(sources) return filenames # get_source_files () def build_libraries (self, libraries): for (lib_name, build_info) in libraries: sources = build_info.get('sources') if sources is None or type(sources) not in (ListType, TupleType): raise DistutilsSetupError, \ ("in 'libraries' option (library '%s'), " + "'sources' must be present and must be " + "a list of source filenames") % lib_name sources = list(sources) log.info("building '%s' library", lib_name) # First, compile the source code to object files in the library # directory. (This should probably change to putting object # files in a temporary build directory.) macros = build_info.get('macros') include_dirs = build_info.get('include_dirs') objects = self.compiler.compile(sources, output_dir=self.build_temp, macros=macros, include_dirs=include_dirs, debug=self.debug) # Now "link" the object files together into a static library. # (On Unix at least, this isn't really linking -- it just # builds an archive. Whatever.) self.compiler.create_static_lib(objects, lib_name, output_dir=self.build_clib, debug=self.debug) # for libraries # build_libraries () # class build_lib --- NEW FILE: build_ext.py --- """distutils.command.build_ext Implements the Distutils 'build_ext' command, for building extension modules (currently limited to C extensions, should accommodate C++ extensions ASAP).""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: build_ext.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string, re from types import * from distutils.core import Command from distutils.errors import * from distutils.sysconfig import customize_compiler, get_python_version from distutils.dep_util import newer_group from distutils.extension import Extension from distutils import log # An extension name is just a dot-separated list of Python NAMEs (ie. # the same as a fully-qualified module name). extension_name_re = re.compile \ (r'^[a-zA-Z_][a-zA-Z_0-9]*(\.[a-zA-Z_][a-zA-Z_0-9]*)*$') def show_compilers (): from distutils.ccompiler import show_compilers show_compilers() class build_ext (Command): description = "build C/C++ extensions (compile/link to build directory)" # XXX thoughts on how to deal with complex command-line options like # these, i.e. how to make it so fancy_getopt can suck them off the # command line and make it look like setup.py defined the appropriate # lists of tuples of what-have-you. # - each command needs a callback to process its command-line options # - Command.__init__() needs access to its share of the whole # command line (must ultimately come from # Distribution.parse_command_line()) # - it then calls the current command class' option-parsing # callback to deal with weird options like -D, which have to # parse the option text and churn out some custom data # structure # - that data structure (in this case, a list of 2-tuples) # will then be present in the command object by the time # we get to finalize_options() (i.e. the constructor # takes care of both command-line and client options # in between initialize_options() and finalize_options()) sep_by = " (separated by '%s')" % os.pathsep user_options = [ ('build-lib=', 'b', "directory for compiled extension modules"), ('build-temp=', 't', "directory for temporary files (build by-products)"), ('inplace', 'i', "ignore build-lib and put compiled extensions into the source " + "directory alongside your pure Python modules"), ('include-dirs=', 'I', "list of directories to search for header files" + sep_by), ('define=', 'D', "C preprocessor macros to define"), ('undef=', 'U', "C preprocessor macros to undefine"), ('libraries=', 'l', "external C libraries to link with"), ('library-dirs=', 'L', "directories to search for external C libraries" + sep_by), ('rpath=', 'R', "directories to search for shared C libraries at runtime"), ('link-objects=', 'O', "extra explicit link objects to include in the link"), ('debug', 'g', "compile/link with debugging information"), ('force', 'f', "forcibly build everything (ignore file timestamps)"), ('compiler=', 'c', "specify the compiler type"), ('swig-cpp', None, "make SWIG create C++ files (default is C)"), ('swig-opts=', None, "list of SWIG command line options"), ('swig=', None, "path to the SWIG executable"), ] boolean_options = ['inplace', 'debug', 'force', 'swig-cpp'] help_options = [ ('help-compiler', None, "list available compilers", show_compilers), ] def initialize_options (self): self.extensions = None self.build_lib = None self.build_temp = None self.inplace = 0 self.package = None self.include_dirs = None self.define = None self.undef = None self.libraries = None self.library_dirs = None self.rpath = None self.link_objects = None self.debug = None self.force = None self.compiler = None self.swig = None self.swig_cpp = None self.swig_opts = None def finalize_options (self): from distutils import sysconfig self.set_undefined_options('build', ('build_lib', 'build_lib'), ('build_temp', 'build_temp'), ('compiler', 'compiler'), ('debug', 'debug'), ('force', 'force')) if self.package is None: self.package = self.distribution.ext_package self.extensions = self.distribution.ext_modules # Make sure Python's include directories (for Python.h, pyconfig.h, # etc.) are in the include search path. py_include = sysconfig.get_python_inc() plat_py_include = sysconfig.get_python_inc(plat_specific=1) if self.include_dirs is None: self.include_dirs = self.distribution.include_dirs or [] if type(self.include_dirs) is StringType: self.include_dirs = string.split(self.include_dirs, os.pathsep) # Put the Python "system" include dir at the end, so that # any local include dirs take precedence. self.include_dirs.append(py_include) if plat_py_include != py_include: self.include_dirs.append(plat_py_include) if type(self.libraries) is StringType: self.libraries = [self.libraries] # Life is easier if we're not forever checking for None, so # simplify these options to empty lists if unset if self.libraries is None: self.libraries = [] if self.library_dirs is None: self.library_dirs = [] elif type(self.library_dirs) is StringType: self.library_dirs = string.split(self.library_dirs, os.pathsep) if self.rpath is None: self.rpath = [] elif type(self.rpath) is StringType: self.rpath = string.split(self.rpath, os.pathsep) # for extensions under windows use different directories # for Release and Debug builds. # also Python's library directory must be appended to library_dirs if os.name == 'nt': self.library_dirs.append(os.path.join(sys.exec_prefix, 'libs')) if self.debug: self.build_temp = os.path.join(self.build_temp, "Debug") else: self.build_temp = os.path.join(self.build_temp, "Release") # Append the source distribution include and library directories, # this allows distutils on windows to work in the source tree self.include_dirs.append(os.path.join(sys.exec_prefix, 'PC')) self.library_dirs.append(os.path.join(sys.exec_prefix, 'PCBuild')) # OS/2 (EMX) doesn't support Debug vs Release builds, but has the # import libraries in its "Config" subdirectory if os.name == 'os2': self.library_dirs.append(os.path.join(sys.exec_prefix, 'Config')) # for extensions under Cygwin and AtheOS Python's library directory must be # appended to library_dirs if sys.platform[:6] == 'cygwin' or sys.platform[:6] == 'atheos': if string.find(sys.executable, sys.exec_prefix) != -1: # building third party extensions self.library_dirs.append(os.path.join(sys.prefix, "lib", "python" + get_python_version(), "config")) else: # building python standard extensions self.library_dirs.append('.') # The argument parsing will result in self.define being a string, but # it has to be a list of 2-tuples. All the preprocessor symbols # specified by the 'define' option will be set to '1'. Multiple # symbols can be separated with commas. if self.define: defines = string.split(self.define, ',') self.define = map(lambda symbol: (symbol, '1'), defines) # The option for macros to undefine is also a string from the # option parsing, but has to be a list. Multiple symbols can also # be separated with commas here. if self.undef: self.undef = string.split(self.undef, ',') if self.swig_opts is None: self.swig_opts = [] else: self.swig_opts = self.swig_opts.split(' ') # finalize_options () def run (self): from distutils.ccompiler import new_compiler # 'self.extensions', as supplied by setup.py, is a list of # Extension instances. See the documentation for Extension (in # distutils.extension) for details. # # For backwards compatibility with Distutils 0.8.2 and earlier, we # also allow the 'extensions' list to be a list of tuples: # (ext_name, build_info) # where build_info is a dictionary containing everything that # Extension instances do except the name, with a few things being # differently named. We convert these 2-tuples to Extension # instances as needed. if not self.extensions: return # If we were asked to build any C/C++ libraries, make sure that the # directory where we put them is in the library search path for # linking extensions. if self.distribution.has_c_libraries(): build_clib = self.get_finalized_command('build_clib') self.libraries.extend(build_clib.get_library_names() or []) self.library_dirs.append(build_clib.build_clib) # Setup the CCompiler object that we'll use to do all the # compiling and linking self.compiler = new_compiler(compiler=self.compiler, verbose=self.verbose, dry_run=self.dry_run, force=self.force) customize_compiler(self.compiler) # And make sure that any compile/link-related options (which might # come from the command-line or from the setup script) are set in # that CCompiler object -- that way, they automatically apply to # all compiling and linking done here. if self.include_dirs is not None: self.compiler.set_include_dirs(self.include_dirs) if self.define is not None: # 'define' option is a list of (name,value) tuples for (name,value) in self.define: self.compiler.define_macro(name, value) if self.undef is not None: for macro in self.undef: self.compiler.undefine_macro(macro) if self.libraries is not None: self.compiler.set_libraries(self.libraries) if self.library_dirs is not None: self.compiler.set_library_dirs(self.library_dirs) if self.rpath is not None: self.compiler.set_runtime_library_dirs(self.rpath) if self.link_objects is not None: self.compiler.set_link_objects(self.link_objects) # Now actually compile and link everything. self.build_extensions() # run () def check_extensions_list (self, extensions): """Ensure that the list of extensions (presumably provided as a command option 'extensions') is valid, i.e. it is a list of Extension objects. We also support the old-style list of 2-tuples, where the tuples are (ext_name, build_info), which are converted to Extension instances here. Raise DistutilsSetupError if the structure is invalid anywhere; just returns otherwise. """ if type(extensions) is not ListType: raise DistutilsSetupError, \ "'ext_modules' option must be a list of Extension instances" for i in range(len(extensions)): ext = extensions[i] if isinstance(ext, Extension): continue # OK! (assume type-checking done # by Extension constructor) (ext_name, build_info) = ext log.warn(("old-style (ext_name, build_info) tuple found in " "ext_modules for extension '%s'" "-- please convert to Extension instance" % ext_name)) if type(ext) is not TupleType and len(ext) != 2: raise DistutilsSetupError, \ ("each element of 'ext_modules' option must be an " "Extension instance or 2-tuple") if not (type(ext_name) is StringType and extension_name_re.match(ext_name)): raise DistutilsSetupError, \ ("first element of each tuple in 'ext_modules' " "must be the extension name (a string)") if type(build_info) is not DictionaryType: raise DistutilsSetupError, \ ("second element of each tuple in 'ext_modules' " "must be a dictionary (build info)") # OK, the (ext_name, build_info) dict is type-safe: convert it # to an Extension instance. ext = Extension(ext_name, build_info['sources']) # Easy stuff: one-to-one mapping from dict elements to # instance attributes. for key in ('include_dirs', 'library_dirs', 'libraries', 'extra_objects', 'extra_compile_args', 'extra_link_args'): val = build_info.get(key) if val is not None: setattr(ext, key, val) # Medium-easy stuff: same syntax/semantics, different names. ext.runtime_library_dirs = build_info.get('rpath') if build_info.has_key('def_file'): log.warn("'def_file' element of build info dict " "no longer supported") # Non-trivial stuff: 'macros' split into 'define_macros' # and 'undef_macros'. macros = build_info.get('macros') if macros: ext.define_macros = [] ext.undef_macros = [] for macro in macros: if not (type(macro) is TupleType and 1 <= len(macro) <= 2): raise DistutilsSetupError, \ ("'macros' element of build info dict " "must be 1- or 2-tuple") if len(macro) == 1: ext.undef_macros.append(macro[0]) elif len(macro) == 2: ext.define_macros.append(macro) extensions[i] = ext # for extensions # check_extensions_list () def get_source_files (self): self.check_extensions_list(self.extensions) filenames = [] # Wouldn't it be neat if we knew the names of header files too... for ext in self.extensions: filenames.extend(ext.sources) return filenames def get_outputs (self): # Sanity check the 'extensions' list -- can't assume this is being # done in the same run as a 'build_extensions()' call (in fact, we # can probably assume that it *isn't*!). self.check_extensions_list(self.extensions) # And build the list of output (built) filenames. Note that this # ignores the 'inplace' flag, and assumes everything goes in the # "build" tree. outputs = [] for ext in self.extensions: fullname = self.get_ext_fullname(ext.name) outputs.append(os.path.join(self.build_lib, self.get_ext_filename(fullname))) return outputs # get_outputs () def build_extensions(self): # First, sanity-check the 'extensions' list self.check_extensions_list(self.extensions) for ext in self.extensions: self.build_extension(ext) def build_extension(self, ext): sources = ext.sources if sources is None or type(sources) not in (ListType, TupleType): raise DistutilsSetupError, \ ("in 'ext_modules' option (extension '%s'), " + "'sources' must be present and must be " + "a list of source filenames") % ext.name sources = list(sources) fullname = self.get_ext_fullname(ext.name) if self.inplace: # ignore build-lib -- put the compiled extension into # the source tree along with pure Python modules modpath = string.split(fullname, '.') package = string.join(modpath[0:-1], '.') base = modpath[-1] build_py = self.get_finalized_command('build_py') package_dir = build_py.get_package_dir(package) ext_filename = os.path.join(package_dir, self.get_ext_filename(base)) else: ext_filename = os.path.join(self.build_lib, self.get_ext_filename(fullname)) depends = sources + ext.depends if not (self.force or newer_group(depends, ext_filename, 'newer')): log.debug("skipping '%s' extension (up-to-date)", ext.name) return else: log.info("building '%s' extension", ext.name) # First, scan the sources for SWIG definition files (.i), run # SWIG on 'em to create .c files, and modify the sources list # accordingly. sources = self.swig_sources(sources, ext) # Next, compile the source code to object files. # XXX not honouring 'define_macros' or 'undef_macros' -- the # CCompiler API needs to change to accommodate this, and I # want to do one thing at a time! # Two possible sources for extra compiler arguments: # - 'extra_compile_args' in Extension object # - CFLAGS environment variable (not particularly # elegant, but people seem to expect it and I # guess it's useful) # The environment variable should take precedence, and # any sensible compiler will give precedence to later # command line args. Hence we combine them in order: extra_args = ext.extra_compile_args or [] macros = ext.define_macros[:] for undef in ext.undef_macros: macros.append((undef,)) objects = self.compiler.compile(sources, output_dir=self.build_temp, macros=macros, include_dirs=ext.include_dirs, debug=self.debug, extra_postargs=extra_args, depends=ext.depends) # XXX -- this is a Vile HACK! # # The setup.py script for Python on Unix needs to be able to # get this list so it can perform all the clean up needed to # avoid keeping object files around when cleaning out a failed # build of an extension module. Since Distutils does not # track dependencies, we have to get rid of intermediates to # ensure all the intermediates will be properly re-built. # self._built_objects = objects[:] # Now link the object files together into a "shared object" -- # of course, first we have to figure out all the other things # that go into the mix. if ext.extra_objects: objects.extend(ext.extra_objects) extra_args = ext.extra_link_args or [] # Detect target language, if not provided language = ext.language or self.compiler.detect_language(sources) self.compiler.link_shared_object( objects, ext_filename, libraries=self.get_libraries(ext), library_dirs=ext.library_dirs, runtime_library_dirs=ext.runtime_library_dirs, extra_postargs=extra_args, export_symbols=self.get_export_symbols(ext), debug=self.debug, build_temp=self.build_temp, target_lang=language) def swig_sources (self, sources, extension): """Walk the list of source files in 'sources', looking for SWIG interface (.i) files. Run SWIG on all that are found, and return a modified 'sources' list with SWIG source files replaced by the generated C (or C++) files. """ new_sources = [] swig_sources = [] swig_targets = {} # XXX this drops generated C/C++ files into the source tree, which # is fine for developers who want to distribute the generated # source -- but there should be an option to put SWIG output in # the temp dir. if self.swig_cpp: log.warn("--swig-cpp is deprecated - use --swig-opts=-c++") if self.swig_cpp or ('-c++' in self.swig_opts): target_ext = '.cpp' else: target_ext = '.c' for source in sources: (base, ext) = os.path.splitext(source) if ext == ".i": # SWIG interface file new_sources.append(base + '_wrap' + target_ext) swig_sources.append(source) swig_targets[source] = new_sources[-1] else: new_sources.append(source) if not swig_sources: return new_sources swig = self.swig or self.find_swig() swig_cmd = [swig, "-python"] swig_cmd.extend(self.swig_opts) if self.swig_cpp: swig_cmd.append("-c++") # Do not override commandline arguments if not self.swig_opts: for o in extension.swig_opts: swig_cmd.append(o) for source in swig_sources: target = swig_targets[source] log.info("swigging %s to %s", source, target) self.spawn(swig_cmd + ["-o", target, source]) return new_sources # swig_sources () def find_swig (self): """Return the name of the SWIG executable. On Unix, this is just "swig" -- it should be in the PATH. Tries a bit harder on Windows. """ if os.name == "posix": return "swig" elif os.name == "nt": # Look for SWIG in its standard installation directory on # Windows (or so I presume!). If we find it there, great; # if not, act like Unix and assume it's in the PATH. for vers in ("1.3", "1.2", "1.1"): fn = os.path.join("c:\\swig%s" % vers, "swig.exe") if os.path.isfile(fn): return fn else: return "swig.exe" elif os.name == "os2": # assume swig available in the PATH. return "swig.exe" else: raise DistutilsPlatformError, \ ("I don't know how to find (much less run) SWIG " "on platform '%s'") % os.name # find_swig () # -- Name generators ----------------------------------------------- # (extension names, filenames, whatever) def get_ext_fullname (self, ext_name): if self.package is None: return ext_name else: return self.package + '.' + ext_name def get_ext_filename (self, ext_name): r"""Convert the name of an extension (eg. "foo.bar") into the name of the file from which it will be loaded (eg. "foo/bar.so", or "foo\bar.pyd"). """ from distutils.sysconfig import get_config_var ext_path = string.split(ext_name, '.') # OS/2 has an 8 character module (extension) limit :-( if os.name == "os2": ext_path[len(ext_path) - 1] = ext_path[len(ext_path) - 1][:8] # extensions in debug_mode are named 'module_d.pyd' under windows so_ext = get_config_var('SO') if os.name == 'nt' and self.debug: return apply(os.path.join, ext_path) + '_d' + so_ext return apply(os.path.join, ext_path) + so_ext def get_export_symbols (self, ext): """Return the list of symbols that a shared extension has to export. This either uses 'ext.export_symbols' or, if it's not provided, "init" + module_name. Only relevant on Windows, where the .pyd file (DLL) must export the module "init" function. """ initfunc_name = "init" + string.split(ext.name,'.')[-1] if initfunc_name not in ext.export_symbols: ext.export_symbols.append(initfunc_name) return ext.export_symbols def get_libraries (self, ext): """Return the list of libraries to link against when building a shared extension. On most platforms, this is just 'ext.libraries'; on Windows and OS/2, we add the Python library (eg. python20.dll). """ # The python library is always needed on Windows. For MSVC, this # is redundant, since the library is mentioned in a pragma in # pyconfig.h that MSVC groks. The other Windows compilers all seem # to need it mentioned explicitly, though, so that's what we do. # Append '_d' to the python import library on debug builds. if sys.platform == "win32": from distutils.msvccompiler import MSVCCompiler if not isinstance(self.compiler, MSVCCompiler): template = "python%d%d" if self.debug: template = template + '_d' pythonlib = (template % (sys.hexversion >> 24, (sys.hexversion >> 16) & 0xff)) # don't extend ext.libraries, it may be shared with other # extensions, it is a reference to the original list return ext.libraries + [pythonlib] else: return ext.libraries elif sys.platform == "os2emx": # EMX/GCC requires the python library explicitly, and I # believe VACPP does as well (though not confirmed) - AIM Apr01 template = "python%d%d" # debug versions of the main DLL aren't supported, at least # not at this time - AIM Apr01 #if self.debug: # template = template + '_d' pythonlib = (template % (sys.hexversion >> 24, (sys.hexversion >> 16) & 0xff)) # don't extend ext.libraries, it may be shared with other # extensions, it is a reference to the original list return ext.libraries + [pythonlib] elif sys.platform[:6] == "cygwin": template = "python%d.%d" pythonlib = (template % (sys.hexversion >> 24, (sys.hexversion >> 16) & 0xff)) # don't extend ext.libraries, it may be shared with other # extensions, it is a reference to the original list return ext.libraries + [pythonlib] elif sys.platform[:6] == "atheos": from distutils import sysconfig template = "python%d.%d" pythonlib = (template % (sys.hexversion >> 24, (sys.hexversion >> 16) & 0xff)) # Get SHLIBS from Makefile extra = [] for lib in sysconfig.get_config_var('SHLIBS').split(): if lib.startswith('-l'): extra.append(lib[2:]) else: extra.append(lib) # don't extend ext.libraries, it may be shared with other # extensions, it is a reference to the original list return ext.libraries + [pythonlib, "m"] + extra else: return ext.libraries # class build_ext --- NEW FILE: build_py.py --- """distutils.command.build_py Implements the Distutils 'build_py' command.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: build_py.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, string, os from types import * from glob import glob from distutils.core import Command from distutils.errors import * from distutils.util import convert_path from distutils import log class build_py (Command): description = "\"build\" pure Python modules (copy to build directory)" user_options = [ ('build-lib=', 'd', "directory to \"build\" (copy) to"), ('compile', 'c', "compile .py to .pyc"), ('no-compile', None, "don't compile .py files [default]"), ('optimize=', 'O', "also compile with optimization: -O1 for \"python -O\", " "-O2 for \"python -OO\", and -O0 to disable [default: -O0]"), ('force', 'f', "forcibly build everything (ignore file timestamps)"), ] boolean_options = ['compile', 'force'] negative_opt = {'no-compile' : 'compile'} def initialize_options (self): self.build_lib = None self.py_modules = None self.package = None self.package_data = None self.package_dir = None self.compile = 0 self.optimize = 0 self.force = None def finalize_options (self): self.set_undefined_options('build', ('build_lib', 'build_lib'), ('force', 'force')) # Get the distribution options that are aliases for build_py # options -- list of packages and list of modules. self.packages = self.distribution.packages self.py_modules = self.distribution.py_modules self.package_data = self.distribution.package_data self.package_dir = {} if self.distribution.package_dir: for name, path in self.distribution.package_dir.items(): self.package_dir[name] = convert_path(path) self.data_files = self.get_data_files() # Ick, copied straight from install_lib.py (fancy_getopt needs a # type system! Hell, *everything* needs a type system!!!) if type(self.optimize) is not IntType: try: self.optimize = int(self.optimize) assert 0 <= self.optimize <= 2 except (ValueError, AssertionError): raise DistutilsOptionError, "optimize must be 0, 1, or 2" def run (self): # XXX copy_file by default preserves atime and mtime. IMHO this is # the right thing to do, but perhaps it should be an option -- in # particular, a site administrator might want installed files to # reflect the time of installation rather than the last # modification time before the installed release. # XXX copy_file by default preserves mode, which appears to be the # wrong thing to do: if a file is read-only in the working # directory, we want it to be installed read/write so that the next # installation of the same module distribution can overwrite it # without problems. (This might be a Unix-specific issue.) Thus # we turn off 'preserve_mode' when copying to the build directory, # since the build directory is supposed to be exactly what the # installation will look like (ie. we preserve mode when # installing). # Two options control which modules will be installed: 'packages' # and 'py_modules'. The former lets us work with whole packages, not # specifying individual modules at all; the latter is for # specifying modules one-at-a-time. if self.py_modules: self.build_modules() if self.packages: self.build_packages() self.build_package_data() self.byte_compile(self.get_outputs(include_bytecode=0)) # run () def get_data_files (self): """Generate list of '(package,src_dir,build_dir,filenames)' tuples""" data = [] if not self.packages: return data for package in self.packages: # Locate package source directory src_dir = self.get_package_dir(package) # Compute package build directory build_dir = os.path.join(*([self.build_lib] + package.split('.'))) # Length of path to strip from found files plen = len(src_dir)+1 # Strip directory from globbed filenames filenames = [ file[plen:] for file in self.find_data_files(package, src_dir) ] data.append((package, src_dir, build_dir, filenames)) return data def find_data_files (self, package, src_dir): """Return filenames for package's data files in 'src_dir'""" globs = (self.package_data.get('', []) + self.package_data.get(package, [])) files = [] for pattern in globs: # Each pattern has to be converted to a platform-specific path filelist = glob(os.path.join(src_dir, convert_path(pattern))) # Files that match more than one pattern are only added once files.extend([fn for fn in filelist if fn not in files]) return files def build_package_data (self): """Copy data files into build directory""" lastdir = None for package, src_dir, build_dir, filenames in self.data_files: for filename in filenames: target = os.path.join(build_dir, filename) self.mkpath(os.path.dirname(target)) self.copy_file(os.path.join(src_dir, filename), target, preserve_mode=False) def get_package_dir (self, package): """Return the directory, relative to the top of the source distribution, where package 'package' should be found (at least according to the 'package_dir' option, if any).""" path = string.split(package, '.') if not self.package_dir: if path: return apply(os.path.join, path) else: return '' else: tail = [] while path: try: pdir = self.package_dir[string.join(path, '.')] except KeyError: tail.insert(0, path[-1]) del path[-1] else: tail.insert(0, pdir) return apply(os.path.join, tail) else: # Oops, got all the way through 'path' without finding a # match in package_dir. If package_dir defines a directory # for the root (nameless) package, then fallback on it; # otherwise, we might as well have not consulted # package_dir at all, as we just use the directory implied # by 'tail' (which should be the same as the original value # of 'path' at this point). pdir = self.package_dir.get('') if pdir is not None: tail.insert(0, pdir) if tail: return apply(os.path.join, tail) else: return '' # get_package_dir () def check_package (self, package, package_dir): # Empty dir name means current directory, which we can probably # assume exists. Also, os.path.exists and isdir don't know about # my "empty string means current dir" convention, so we have to # circumvent them. if package_dir != "": if not os.path.exists(package_dir): raise DistutilsFileError, \ "package directory '%s' does not exist" % package_dir if not os.path.isdir(package_dir): raise DistutilsFileError, \ ("supposed package directory '%s' exists, " + "but is not a directory") % package_dir # Require __init__.py for all but the "root package" if package: init_py = os.path.join(package_dir, "__init__.py") if os.path.isfile(init_py): return init_py else: log.warn(("package init file '%s' not found " + "(or not a regular file)"), init_py) # Either not in a package at all (__init__.py not expected), or # __init__.py doesn't exist -- so don't return the filename. return None # check_package () def check_module (self, module, module_file): if not os.path.isfile(module_file): log.warn("file %s (for module %s) not found", module_file, module) return 0 else: return 1 # check_module () def find_package_modules (self, package, package_dir): self.check_package(package, package_dir) module_files = glob(os.path.join(package_dir, "*.py")) modules = [] setup_script = os.path.abspath(self.distribution.script_name) for f in module_files: abs_f = os.path.abspath(f) if abs_f != setup_script: module = os.path.splitext(os.path.basename(f))[0] modules.append((package, module, f)) else: self.debug_print("excluding %s" % setup_script) return modules def find_modules (self): """Finds individually-specified Python modules, ie. those listed by module name in 'self.py_modules'. Returns a list of tuples (package, module_base, filename): 'package' is a tuple of the path through package-space to the module; 'module_base' is the bare (no packages, no dots) module name, and 'filename' is the path to the ".py" file (relative to the distribution root) that implements the module. """ # Map package names to tuples of useful info about the package: # (package_dir, checked) # package_dir - the directory where we'll find source files for # this package # checked - true if we have checked that the package directory # is valid (exists, contains __init__.py, ... ?) packages = {} # List of (package, module, filename) tuples to return modules = [] # We treat modules-in-packages almost the same as toplevel modules, # just the "package" for a toplevel is empty (either an empty # string or empty list, depending on context). Differences: # - don't check for __init__.py in directory for empty package for module in self.py_modules: path = string.split(module, '.') package = string.join(path[0:-1], '.') module_base = path[-1] try: (package_dir, checked) = packages[package] except KeyError: package_dir = self.get_package_dir(package) checked = 0 if not checked: init_py = self.check_package(package, package_dir) packages[package] = (package_dir, 1) if init_py: modules.append((package, "__init__", init_py)) # XXX perhaps we should also check for just .pyc files # (so greedy closed-source bastards can distribute Python # modules too) module_file = os.path.join(package_dir, module_base + ".py") if not self.check_module(module, module_file): continue modules.append((package, module_base, module_file)) return modules # find_modules () def find_all_modules (self): """Compute the list of all modules that will be built, whether they are specified one-module-at-a-time ('self.py_modules') or by whole packages ('self.packages'). Return a list of tuples (package, module, module_file), just like 'find_modules()' and 'find_package_modules()' do.""" modules = [] if self.py_modules: modules.extend(self.find_modules()) if self.packages: for package in self.packages: package_dir = self.get_package_dir(package) m = self.find_package_modules(package, package_dir) modules.extend(m) return modules # find_all_modules () def get_source_files (self): modules = self.find_all_modules() filenames = [] for module in modules: filenames.append(module[-1]) return filenames def get_module_outfile (self, build_dir, package, module): outfile_path = [build_dir] + list(package) + [module + ".py"] return apply(os.path.join, outfile_path) def get_outputs (self, include_bytecode=1): modules = self.find_all_modules() outputs = [] for (package, module, module_file) in modules: package = string.split(package, '.') filename = self.get_module_outfile(self.build_lib, package, module) outputs.append(filename) if include_bytecode: if self.compile: outputs.append(filename + "c") if self.optimize > 0: outputs.append(filename + "o") outputs += [ os.path.join(build_dir, filename) for package, src_dir, build_dir, filenames in self.data_files for filename in filenames ] return outputs def build_module (self, module, module_file, package): if type(package) is StringType: package = string.split(package, '.') elif type(package) not in (ListType, TupleType): raise TypeError, \ "'package' must be a string (dot-separated), list, or tuple" # Now put the module source file into the "build" area -- this is # easy, we just copy it somewhere under self.build_lib (the build # directory for Python source). outfile = self.get_module_outfile(self.build_lib, package, module) dir = os.path.dirname(outfile) self.mkpath(dir) return self.copy_file(module_file, outfile, preserve_mode=0) def build_modules (self): modules = self.find_modules() for (package, module, module_file) in modules: # Now "build" the module -- ie. copy the source file to # self.build_lib (the build directory for Python source). # (Actually, it gets copied to the directory for this package # under self.build_lib.) self.build_module(module, module_file, package) # build_modules () def build_packages (self): for package in self.packages: # Get list of (package, module, module_file) tuples based on # scanning the package directory. 'package' is only included # in the tuple so that 'find_modules()' and # 'find_package_tuples()' have a consistent interface; it's # ignored here (apart from a sanity check). Also, 'module' is # the *unqualified* module name (ie. no dots, no package -- we # already know its package!), and 'module_file' is the path to # the .py file, relative to the current directory # (ie. including 'package_dir'). package_dir = self.get_package_dir(package) modules = self.find_package_modules(package, package_dir) # Now loop over the modules we found, "building" each one (just # copy it to self.build_lib). for (package_, module, module_file) in modules: assert package == package_ self.build_module(module, module_file, package) # build_packages () def byte_compile (self, files): from distutils.util import byte_compile prefix = self.build_lib if prefix[-1] != os.sep: prefix = prefix + os.sep # XXX this code is essentially the same as the 'byte_compile() # method of the "install_lib" command, except for the determination # of the 'prefix' string. Hmmm. if self.compile: byte_compile(files, optimize=0, force=self.force, prefix=prefix, dry_run=self.dry_run) if self.optimize > 0: byte_compile(files, optimize=self.optimize, force=self.force, prefix=prefix, dry_run=self.dry_run) # class build_py --- NEW FILE: build_scripts.py --- """distutils.command.build_scripts Implements the Distutils 'build_scripts' command.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: build_scripts.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, re from stat import ST_MODE from distutils import sysconfig from distutils.core import Command from distutils.dep_util import newer from distutils.util import convert_path from distutils import log # check if Python is called on the first line with this expression first_line_re = re.compile('^#!.*python[0-9.]*([ \t].*)?$') class build_scripts (Command): description = "\"build\" scripts (copy and fixup #! line)" user_options = [ ('build-dir=', 'd', "directory to \"build\" (copy) to"), ('force', 'f', "forcibly build everything (ignore file timestamps"), ('executable=', 'e', "specify final destination interpreter path"), ] boolean_options = ['force'] def initialize_options (self): self.build_dir = None self.scripts = None self.force = None self.executable = None self.outfiles = None def finalize_options (self): self.set_undefined_options('build', ('build_scripts', 'build_dir'), ('force', 'force'), ('executable', 'executable')) self.scripts = self.distribution.scripts def get_source_files(self): return self.scripts def run (self): if not self.scripts: return self.copy_scripts() def copy_scripts (self): """Copy each script listed in 'self.scripts'; if it's marked as a Python script in the Unix way (first line matches 'first_line_re', ie. starts with "\#!" and contains "python"), then adjust the first line to refer to the current Python interpreter as we copy. """ self.mkpath(self.build_dir) outfiles = [] for script in self.scripts: adjust = 0 script = convert_path(script) outfile = os.path.join(self.build_dir, os.path.basename(script)) outfiles.append(outfile) if not self.force and not newer(script, outfile): log.debug("not copying %s (up-to-date)", script) continue # Always open the file, but ignore failures in dry-run mode -- # that way, we'll get accurate feedback if we can read the # script. try: f = open(script, "r") except IOError: if not self.dry_run: raise f = None else: first_line = f.readline() if not first_line: self.warn("%s is an empty file (skipping)" % script) continue match = first_line_re.match(first_line) if match: adjust = 1 post_interp = match.group(1) or '' if adjust: log.info("copying and adjusting %s -> %s", script, self.build_dir) if not self.dry_run: outf = open(outfile, "w") if not sysconfig.python_build: outf.write("#!%s%s\n" % (self.executable, post_interp)) else: outf.write("#!%s%s\n" % (os.path.join( sysconfig.get_config_var("BINDIR"), "python" + sysconfig.get_config_var("EXE")), post_interp)) outf.writelines(f.readlines()) outf.close() if f: f.close() else: f.close() self.copy_file(script, outfile) if os.name == 'posix': for file in outfiles: if self.dry_run: log.info("changing mode of %s", file) else: oldmode = os.stat(file)[ST_MODE] & 07777 newmode = (oldmode | 0555) & 07777 if newmode != oldmode: log.info("changing mode of %s from %o to %o", file, oldmode, newmode) os.chmod(file, newmode) # copy_scripts () # class build_scripts --- NEW FILE: clean.py --- """distutils.command.clean Implements the Distutils 'clean' command.""" # contributed by Bastian Kleineidam , added 2000-03-18 # This module should be kept compatible with Python 2.1. __revision__ = "$Id: clean.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os from distutils.core import Command from distutils.dir_util import remove_tree from distutils import log class clean (Command): description = "clean up temporary files from 'build' command" user_options = [ ('build-base=', 'b', "base build directory (default: 'build.build-base')"), ('build-lib=', None, "build directory for all modules (default: 'build.build-lib')"), ('build-temp=', 't', "temporary build directory (default: 'build.build-temp')"), ('build-scripts=', None, "build directory for scripts (default: 'build.build-scripts')"), ('bdist-base=', None, "temporary directory for built distributions"), ('all', 'a', "remove all build output, not just temporary by-products") ] boolean_options = ['all'] def initialize_options(self): self.build_base = None self.build_lib = None self.build_temp = None self.build_scripts = None self.bdist_base = None self.all = None def finalize_options(self): self.set_undefined_options('build', ('build_base', 'build_base'), ('build_lib', 'build_lib'), ('build_scripts', 'build_scripts'), ('build_temp', 'build_temp')) self.set_undefined_options('bdist', ('bdist_base', 'bdist_base')) def run(self): # remove the build/temp. directory (unless it's already # gone) if os.path.exists(self.build_temp): remove_tree(self.build_temp, dry_run=self.dry_run) else: log.debug("'%s' does not exist -- can't clean it", self.build_temp) if self.all: # remove build directories for directory in (self.build_lib, self.bdist_base, self.build_scripts): if os.path.exists(directory): remove_tree(directory, dry_run=self.dry_run) else: log.warn("'%s' does not exist -- can't clean it", directory) # just for the heck of it, try to remove the base build directory: # we might have emptied it right now, but if not we don't care if not self.dry_run: try: os.rmdir(self.build_base) log.info("removing '%s'", self.build_base) except OSError: pass # class clean --- NEW FILE: config.py --- """distutils.command.config Implements the Distutils 'config' command, a (mostly) empty command class that exists mainly to be sub-classed by specific module distributions and applications. The idea is that while every "config" command is different, at least they're all named the same, and users always see "config" in the list of standard commands. Also, this is a good place to put common configure-like tasks: "try to compile this C code", or "figure out where this header file lives". """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: config.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string, re from types import * from distutils.core import Command from distutils.errors import DistutilsExecError from distutils.sysconfig import customize_compiler from distutils import log LANG_EXT = {'c': '.c', 'c++': '.cxx'} class config (Command): description = "prepare to build" user_options = [ ('compiler=', None, "specify the compiler type"), ('cc=', None, "specify the compiler executable"), ('include-dirs=', 'I', "list of directories to search for header files"), ('define=', 'D', "C preprocessor macros to define"), ('undef=', 'U', "C preprocessor macros to undefine"), ('libraries=', 'l', "external C libraries to link with"), ('library-dirs=', 'L', "directories to search for external C libraries"), ('noisy', None, "show every action (compile, link, run, ...) taken"), ('dump-source', None, "dump generated source files before attempting to compile them"), ] # The three standard command methods: since the "config" command # does nothing by default, these are empty. def initialize_options (self): self.compiler = None self.cc = None self.include_dirs = None #self.define = None #self.undef = None self.libraries = None self.library_dirs = None # maximal output for now self.noisy = 1 self.dump_source = 1 # list of temporary files generated along-the-way that we have # to clean at some point self.temp_files = [] def finalize_options (self): if self.include_dirs is None: self.include_dirs = self.distribution.include_dirs or [] elif type(self.include_dirs) is StringType: self.include_dirs = string.split(self.include_dirs, os.pathsep) if self.libraries is None: self.libraries = [] elif type(self.libraries) is StringType: self.libraries = [self.libraries] if self.library_dirs is None: self.library_dirs = [] elif type(self.library_dirs) is StringType: self.library_dirs = string.split(self.library_dirs, os.pathsep) def run (self): pass # Utility methods for actual "config" commands. The interfaces are # loosely based on Autoconf macros of similar names. Sub-classes # may use these freely. def _check_compiler (self): """Check that 'self.compiler' really is a CCompiler object; if not, make it one. """ # We do this late, and only on-demand, because this is an expensive # import. from distutils.ccompiler import CCompiler, new_compiler if not isinstance(self.compiler, CCompiler): self.compiler = new_compiler(compiler=self.compiler, dry_run=self.dry_run, force=1) customize_compiler(self.compiler) if self.include_dirs: self.compiler.set_include_dirs(self.include_dirs) if self.libraries: self.compiler.set_libraries(self.libraries) if self.library_dirs: self.compiler.set_library_dirs(self.library_dirs) def _gen_temp_sourcefile (self, body, headers, lang): filename = "_configtest" + LANG_EXT[lang] file = open(filename, "w") if headers: for header in headers: file.write("#include <%s>\n" % header) file.write("\n") file.write(body) if body[-1] != "\n": file.write("\n") file.close() return filename def _preprocess (self, body, headers, include_dirs, lang): src = self._gen_temp_sourcefile(body, headers, lang) out = "_configtest.i" self.temp_files.extend([src, out]) self.compiler.preprocess(src, out, include_dirs=include_dirs) return (src, out) def _compile (self, body, headers, include_dirs, lang): src = self._gen_temp_sourcefile(body, headers, lang) if self.dump_source: dump_file(src, "compiling '%s':" % src) (obj,) = self.compiler.object_filenames([src]) self.temp_files.extend([src, obj]) self.compiler.compile([src], include_dirs=include_dirs) return (src, obj) def _link (self, body, headers, include_dirs, libraries, library_dirs, lang): (src, obj) = self._compile(body, headers, include_dirs, lang) prog = os.path.splitext(os.path.basename(src))[0] self.compiler.link_executable([obj], prog, libraries=libraries, library_dirs=library_dirs, target_lang=lang) if self.compiler.exe_extension is not None: prog = prog + self.compiler.exe_extension self.temp_files.append(prog) return (src, obj, prog) def _clean (self, *filenames): if not filenames: filenames = self.temp_files self.temp_files = [] log.info("removing: %s", string.join(filenames)) for filename in filenames: try: os.remove(filename) except OSError: pass # XXX these ignore the dry-run flag: what to do, what to do? even if # you want a dry-run build, you still need some sort of configuration # info. My inclination is to make it up to the real config command to # consult 'dry_run', and assume a default (minimal) configuration if # true. The problem with trying to do it here is that you'd have to # return either true or false from all the 'try' methods, neither of # which is correct. # XXX need access to the header search path and maybe default macros. def try_cpp (self, body=None, headers=None, include_dirs=None, lang="c"): """Construct a source file from 'body' (a string containing lines of C/C++ code) and 'headers' (a list of header files to include) and run it through the preprocessor. Return true if the preprocessor succeeded, false if there were any errors. ('body' probably isn't of much use, but what the heck.) """ from distutils.ccompiler import CompileError self._check_compiler() ok = 1 try: self._preprocess(body, headers, include_dirs, lang) except CompileError: ok = 0 self._clean() return ok def search_cpp (self, pattern, body=None, headers=None, include_dirs=None, lang="c"): """Construct a source file (just like 'try_cpp()'), run it through the preprocessor, and return true if any line of the output matches 'pattern'. 'pattern' should either be a compiled regex object or a string containing a regex. If both 'body' and 'headers' are None, preprocesses an empty file -- which can be useful to determine the symbols the preprocessor and compiler set by default. """ self._check_compiler() (src, out) = self._preprocess(body, headers, include_dirs, lang) if type(pattern) is StringType: pattern = re.compile(pattern) file = open(out) match = 0 while 1: line = file.readline() if line == '': break if pattern.search(line): match = 1 break file.close() self._clean() return match def try_compile (self, body, headers=None, include_dirs=None, lang="c"): """Try to compile a source file built from 'body' and 'headers'. Return true on success, false otherwise. """ from distutils.ccompiler import CompileError self._check_compiler() try: self._compile(body, headers, include_dirs, lang) ok = 1 except CompileError: ok = 0 log.info(ok and "success!" or "failure.") self._clean() return ok def try_link (self, body, headers=None, include_dirs=None, libraries=None, library_dirs=None, lang="c"): """Try to compile and link a source file, built from 'body' and 'headers', to executable form. Return true on success, false otherwise. """ from distutils.ccompiler import CompileError, LinkError self._check_compiler() try: self._link(body, headers, include_dirs, libraries, library_dirs, lang) ok = 1 except (CompileError, LinkError): ok = 0 log.info(ok and "success!" or "failure.") self._clean() return ok def try_run (self, body, headers=None, include_dirs=None, libraries=None, library_dirs=None, lang="c"): """Try to compile, link to an executable, and run a program built from 'body' and 'headers'. Return true on success, false otherwise. """ from distutils.ccompiler import CompileError, LinkError self._check_compiler() try: src, obj, exe = self._link(body, headers, include_dirs, libraries, library_dirs, lang) self.spawn([exe]) ok = 1 except (CompileError, LinkError, DistutilsExecError): ok = 0 log.info(ok and "success!" or "failure.") self._clean() return ok # -- High-level methods -------------------------------------------- # (these are the ones that are actually likely to be useful # when implementing a real-world config command!) def check_func (self, func, headers=None, include_dirs=None, libraries=None, library_dirs=None, decl=0, call=0): """Determine if function 'func' is available by constructing a source file that refers to 'func', and compiles and links it. If everything succeeds, returns true; otherwise returns false. The constructed source file starts out by including the header files listed in 'headers'. If 'decl' is true, it then declares 'func' (as "int func()"); you probably shouldn't supply 'headers' and set 'decl' true in the same call, or you might get errors about a conflicting declarations for 'func'. Finally, the constructed 'main()' function either references 'func' or (if 'call' is true) calls it. 'libraries' and 'library_dirs' are used when linking. """ self._check_compiler() body = [] if decl: body.append("int %s ();" % func) body.append("int main () {") if call: body.append(" %s();" % func) else: body.append(" %s;" % func) body.append("}") body = string.join(body, "\n") + "\n" return self.try_link(body, headers, include_dirs, libraries, library_dirs) # check_func () def check_lib (self, library, library_dirs=None, headers=None, include_dirs=None, other_libraries=[]): """Determine if 'library' is available to be linked against, without actually checking that any particular symbols are provided by it. 'headers' will be used in constructing the source file to be compiled, but the only effect of this is to check if all the header files listed are available. Any libraries listed in 'other_libraries' will be included in the link, in case 'library' has symbols that depend on other libraries. """ self._check_compiler() return self.try_link("int main (void) { }", headers, include_dirs, [library]+other_libraries, library_dirs) def check_header (self, header, include_dirs=None, library_dirs=None, lang="c"): """Determine if the system header file named by 'header_file' exists and can be found by the preprocessor; return true if so, false otherwise. """ return self.try_cpp(body="/* No body */", headers=[header], include_dirs=include_dirs) # class config def dump_file (filename, head=None): if head is None: print filename + ":" else: print head file = open(filename) sys.stdout.write(file.read()) file.close() --- NEW FILE: install.py --- """distutils.command.install Implements the Distutils 'install' command.""" from distutils import log # This module should be kept compatible with Python 2.1. __revision__ = "$Id: install.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string from types import * from distutils.core import Command from distutils.debug import DEBUG from distutils.sysconfig import get_config_vars from distutils.errors import DistutilsPlatformError from distutils.file_util import write_file from distutils.util import convert_path, subst_vars, change_root from distutils.errors import DistutilsOptionError from glob import glob if sys.version < "2.2": WINDOWS_SCHEME = { 'purelib': '$base', 'platlib': '$base', 'headers': '$base/Include/$dist_name', 'scripts': '$base/Scripts', 'data' : '$base', } else: WINDOWS_SCHEME = { 'purelib': '$base/Lib/site-packages', 'platlib': '$base/Lib/site-packages', 'headers': '$base/Include/$dist_name', 'scripts': '$base/Scripts', 'data' : '$base', } INSTALL_SCHEMES = { 'unix_prefix': { 'purelib': '$base/lib/python$py_version_short/site-packages', 'platlib': '$platbase/lib/python$py_version_short/site-packages', 'headers': '$base/include/python$py_version_short/$dist_name', 'scripts': '$base/bin', 'data' : '$base', }, 'unix_home': { 'purelib': '$base/lib/python', 'platlib': '$base/lib/python', 'headers': '$base/include/python/$dist_name', 'scripts': '$base/bin', 'data' : '$base', }, 'nt': WINDOWS_SCHEME, 'mac': { 'purelib': '$base/Lib/site-packages', 'platlib': '$base/Lib/site-packages', 'headers': '$base/Include/$dist_name', 'scripts': '$base/Scripts', 'data' : '$base', }, 'os2': { 'purelib': '$base/Lib/site-packages', 'platlib': '$base/Lib/site-packages', 'headers': '$base/Include/$dist_name', 'scripts': '$base/Scripts', 'data' : '$base', } } # The keys to an installation scheme; if any new types of files are to be # installed, be sure to add an entry to every installation scheme above, # and to SCHEME_KEYS here. SCHEME_KEYS = ('purelib', 'platlib', 'headers', 'scripts', 'data') class install (Command): description = "install everything from build directory" user_options = [ # Select installation scheme and set base director(y|ies) ('prefix=', None, "installation prefix"), ('exec-prefix=', None, "(Unix only) prefix for platform-specific files"), ('home=', None, "(Unix only) home directory to install under"), # Or, just set the base director(y|ies) ('install-base=', None, "base installation directory (instead of --prefix or --home)"), ('install-platbase=', None, "base installation directory for platform-specific files " + "(instead of --exec-prefix or --home)"), ('root=', None, "install everything relative to this alternate root directory"), # Or, explicitly set the installation scheme ('install-purelib=', None, "installation directory for pure Python module distributions"), ('install-platlib=', None, "installation directory for non-pure module distributions"), ('install-lib=', None, "installation directory for all module distributions " + "(overrides --install-purelib and --install-platlib)"), ('install-headers=', None, "installation directory for C/C++ headers"), ('install-scripts=', None, "installation directory for Python scripts"), ('install-data=', None, "installation directory for data files"), # Byte-compilation options -- see install_lib.py for details, as # these are duplicated from there (but only install_lib does # anything with them). ('compile', 'c', "compile .py to .pyc [default]"), ('no-compile', None, "don't compile .py files"), ('optimize=', 'O', "also compile with optimization: -O1 for \"python -O\", " "-O2 for \"python -OO\", and -O0 to disable [default: -O0]"), # Miscellaneous control options ('force', 'f', "force installation (overwrite any existing files)"), ('skip-build', None, "skip rebuilding everything (for testing/debugging)"), # Where to install documentation (eventually!) #('doc-format=', None, "format of documentation to generate"), #('install-man=', None, "directory for Unix man pages"), #('install-html=', None, "directory for HTML documentation"), #('install-info=', None, "directory for GNU info files"), ('record=', None, "filename in which to record list of installed files"), ] boolean_options = ['compile', 'force', 'skip-build'] negative_opt = {'no-compile' : 'compile'} def initialize_options (self): # High-level options: these select both an installation base # and scheme. self.prefix = None self.exec_prefix = None self.home = None # These select only the installation base; it's up to the user to # specify the installation scheme (currently, that means supplying # the --install-{platlib,purelib,scripts,data} options). self.install_base = None self.install_platbase = None self.root = None # These options are the actual installation directories; if not # supplied by the user, they are filled in using the installation # scheme implied by prefix/exec-prefix/home and the contents of # that installation scheme. self.install_purelib = None # for pure module distributions self.install_platlib = None # non-pure (dists w/ extensions) self.install_headers = None # for C/C++ headers self.install_lib = None # set to either purelib or platlib self.install_scripts = None self.install_data = None self.compile = None self.optimize = None # These two are for putting non-packagized distributions into their # own directory and creating a .pth file if it makes sense. # 'extra_path' comes from the setup file; 'install_path_file' can # be turned off if it makes no sense to install a .pth file. (But # better to install it uselessly than to guess wrong and not # install it when it's necessary and would be used!) Currently, # 'install_path_file' is always true unless some outsider meddles # with it. self.extra_path = None self.install_path_file = 1 # 'force' forces installation, even if target files are not # out-of-date. 'skip_build' skips running the "build" command, # handy if you know it's not necessary. 'warn_dir' (which is *not* # a user option, it's just there so the bdist_* commands can turn # it off) determines whether we warn about installing to a # directory not in sys.path. self.force = 0 self.skip_build = 0 self.warn_dir = 1 # These are only here as a conduit from the 'build' command to the # 'install_*' commands that do the real work. ('build_base' isn't # actually used anywhere, but it might be useful in future.) They # are not user options, because if the user told the install # command where the build directory is, that wouldn't affect the # build command. self.build_base = None self.build_lib = None # Not defined yet because we don't know anything about # documentation yet. #self.install_man = None #self.install_html = None #self.install_info = None self.record = None # -- Option finalizing methods ------------------------------------- # (This is rather more involved than for most commands, # because this is where the policy for installing third- # party Python modules on various platforms given a wide # array of user input is decided. Yes, it's quite complex!) def finalize_options (self): # This method (and its pliant slaves, like 'finalize_unix()', # 'finalize_other()', and 'select_scheme()') is where the default # installation directories for modules, extension modules, and # anything else we care to install from a Python module # distribution. Thus, this code makes a pretty important policy # statement about how third-party stuff is added to a Python # installation! Note that the actual work of installation is done # by the relatively simple 'install_*' commands; they just take # their orders from the installation directory options determined # here. # Check for errors/inconsistencies in the options; first, stuff # that's wrong on any platform. if ((self.prefix or self.exec_prefix or self.home) and (self.install_base or self.install_platbase)): raise DistutilsOptionError, \ ("must supply either prefix/exec-prefix/home or " + "install-base/install-platbase -- not both") if self.home and (self.prefix or self.exec_prefix): raise DistutilsOptionError, \ "must supply either home or prefix/exec-prefix -- not both" # Next, stuff that's wrong (or dubious) only on certain platforms. if os.name != "posix": if self.exec_prefix: self.warn("exec-prefix option ignored on this platform") self.exec_prefix = None # Now the interesting logic -- so interesting that we farm it out # to other methods. The goal of these methods is to set the final # values for the install_{lib,scripts,data,...} options, using as # input a heady brew of prefix, exec_prefix, home, install_base, # install_platbase, user-supplied versions of # install_{purelib,platlib,lib,scripts,data,...}, and the # INSTALL_SCHEME dictionary above. Phew! self.dump_dirs("pre-finalize_{unix,other}") if os.name == 'posix': self.finalize_unix() else: self.finalize_other() self.dump_dirs("post-finalize_{unix,other}()") # Expand configuration variables, tilde, etc. in self.install_base # and self.install_platbase -- that way, we can use $base or # $platbase in the other installation directories and not worry # about needing recursive variable expansion (shudder). py_version = (string.split(sys.version))[0] (prefix, exec_prefix) = get_config_vars('prefix', 'exec_prefix') self.config_vars = {'dist_name': self.distribution.get_name(), 'dist_version': self.distribution.get_version(), 'dist_fullname': self.distribution.get_fullname(), 'py_version': py_version, 'py_version_short': py_version[0:3], 'sys_prefix': prefix, 'prefix': prefix, 'sys_exec_prefix': exec_prefix, 'exec_prefix': exec_prefix, } self.expand_basedirs() self.dump_dirs("post-expand_basedirs()") # Now define config vars for the base directories so we can expand # everything else. self.config_vars['base'] = self.install_base self.config_vars['platbase'] = self.install_platbase if DEBUG: from pprint import pprint print "config vars:" pprint(self.config_vars) # Expand "~" and configuration variables in the installation # directories. self.expand_dirs() self.dump_dirs("post-expand_dirs()") # Pick the actual directory to install all modules to: either # install_purelib or install_platlib, depending on whether this # module distribution is pure or not. Of course, if the user # already specified install_lib, use their selection. if self.install_lib is None: if self.distribution.ext_modules: # has extensions: non-pure self.install_lib = self.install_platlib else: self.install_lib = self.install_purelib # Convert directories from Unix /-separated syntax to the local # convention. self.convert_paths('lib', 'purelib', 'platlib', 'scripts', 'data', 'headers') # Well, we're not actually fully completely finalized yet: we still # have to deal with 'extra_path', which is the hack for allowing # non-packagized module distributions (hello, Numerical Python!) to # get their own directories. self.handle_extra_path() self.install_libbase = self.install_lib # needed for .pth file self.install_lib = os.path.join(self.install_lib, self.extra_dirs) # If a new root directory was supplied, make all the installation # dirs relative to it. if self.root is not None: self.change_roots('libbase', 'lib', 'purelib', 'platlib', 'scripts', 'data', 'headers') self.dump_dirs("after prepending root") # Find out the build directories, ie. where to install from. self.set_undefined_options('build', ('build_base', 'build_base'), ('build_lib', 'build_lib')) # Punt on doc directories for now -- after all, we're punting on # documentation completely! # finalize_options () def dump_dirs (self, msg): if DEBUG: from distutils.fancy_getopt import longopt_xlate print msg + ":" for opt in self.user_options: opt_name = opt[0] if opt_name[-1] == "=": opt_name = opt_name[0:-1] if self.negative_opt.has_key(opt_name): opt_name = string.translate(self.negative_opt[opt_name], longopt_xlate) val = not getattr(self, opt_name) else: opt_name = string.translate(opt_name, longopt_xlate) val = getattr(self, opt_name) print " %s: %s" % (opt_name, val) def finalize_unix (self): if self.install_base is not None or self.install_platbase is not None: if ((self.install_lib is None and self.install_purelib is None and self.install_platlib is None) or self.install_headers is None or self.install_scripts is None or self.install_data is None): raise DistutilsOptionError, \ ("install-base or install-platbase supplied, but " "installation scheme is incomplete") return if self.home is not None: self.install_base = self.install_platbase = self.home self.select_scheme("unix_home") else: if self.prefix is None: if self.exec_prefix is not None: raise DistutilsOptionError, \ "must not supply exec-prefix without prefix" self.prefix = os.path.normpath(sys.prefix) self.exec_prefix = os.path.normpath(sys.exec_prefix) else: if self.exec_prefix is None: self.exec_prefix = self.prefix self.install_base = self.prefix self.install_platbase = self.exec_prefix self.select_scheme("unix_prefix") # finalize_unix () def finalize_other (self): # Windows and Mac OS for now if self.home is not None: self.install_base = self.install_platbase = self.home self.select_scheme("unix_home") else: if self.prefix is None: self.prefix = os.path.normpath(sys.prefix) self.install_base = self.install_platbase = self.prefix try: self.select_scheme(os.name) except KeyError: raise DistutilsPlatformError, \ "I don't know how to install stuff on '%s'" % os.name # finalize_other () def select_scheme (self, name): # it's the caller's problem if they supply a bad name! scheme = INSTALL_SCHEMES[name] for key in SCHEME_KEYS: attrname = 'install_' + key if getattr(self, attrname) is None: setattr(self, attrname, scheme[key]) def _expand_attrs (self, attrs): for attr in attrs: val = getattr(self, attr) if val is not None: if os.name == 'posix': val = os.path.expanduser(val) val = subst_vars(val, self.config_vars) setattr(self, attr, val) def expand_basedirs (self): self._expand_attrs(['install_base', 'install_platbase', 'root']) def expand_dirs (self): self._expand_attrs(['install_purelib', 'install_platlib', 'install_lib', 'install_headers', 'install_scripts', 'install_data',]) def convert_paths (self, *names): for name in names: attr = "install_" + name setattr(self, attr, convert_path(getattr(self, attr))) def handle_extra_path (self): if self.extra_path is None: self.extra_path = self.distribution.extra_path if self.extra_path is not None: if type(self.extra_path) is StringType: self.extra_path = string.split(self.extra_path, ',') if len(self.extra_path) == 1: path_file = extra_dirs = self.extra_path[0] elif len(self.extra_path) == 2: (path_file, extra_dirs) = self.extra_path else: raise DistutilsOptionError, \ ("'extra_path' option must be a list, tuple, or " "comma-separated string with 1 or 2 elements") # convert to local form in case Unix notation used (as it # should be in setup scripts) extra_dirs = convert_path(extra_dirs) else: path_file = None extra_dirs = '' # XXX should we warn if path_file and not extra_dirs? (in which # case the path file would be harmless but pointless) self.path_file = path_file self.extra_dirs = extra_dirs # handle_extra_path () def change_roots (self, *names): for name in names: attr = "install_" + name setattr(self, attr, change_root(self.root, getattr(self, attr))) # -- Command execution methods ------------------------------------- def run (self): # Obviously have to build before we can install if not self.skip_build: self.run_command('build') # Run all sub-commands (at least those that need to be run) for cmd_name in self.get_sub_commands(): self.run_command(cmd_name) if self.path_file: self.create_path_file() # write list of installed files, if requested. if self.record: outputs = self.get_outputs() if self.root: # strip any package prefix root_len = len(self.root) for counter in xrange(len(outputs)): outputs[counter] = outputs[counter][root_len:] self.execute(write_file, (self.record, outputs), "writing list of installed files to '%s'" % self.record) sys_path = map(os.path.normpath, sys.path) sys_path = map(os.path.normcase, sys_path) install_lib = os.path.normcase(os.path.normpath(self.install_lib)) if (self.warn_dir and not (self.path_file and self.install_path_file) and install_lib not in sys_path): log.debug(("modules installed to '%s', which is not in " "Python's module search path (sys.path) -- " "you'll have to change the search path yourself"), self.install_lib) # run () def create_path_file (self): filename = os.path.join(self.install_libbase, self.path_file + ".pth") if self.install_path_file: self.execute(write_file, (filename, [self.extra_dirs]), "creating %s" % filename) else: self.warn("path file '%s' not created" % filename) # -- Reporting methods --------------------------------------------- def get_outputs (self): # Assemble the outputs of all the sub-commands. outputs = [] for cmd_name in self.get_sub_commands(): cmd = self.get_finalized_command(cmd_name) # Add the contents of cmd.get_outputs(), ensuring # that outputs doesn't contain duplicate entries for filename in cmd.get_outputs(): if filename not in outputs: outputs.append(filename) if self.path_file and self.install_path_file: outputs.append(os.path.join(self.install_libbase, self.path_file + ".pth")) return outputs def get_inputs (self): # XXX gee, this looks familiar ;-( inputs = [] for cmd_name in self.get_sub_commands(): cmd = self.get_finalized_command(cmd_name) inputs.extend(cmd.get_inputs()) return inputs # -- Predicates for sub-command list ------------------------------- def has_lib (self): """Return true if the current distribution has any Python modules to install.""" return (self.distribution.has_pure_modules() or self.distribution.has_ext_modules()) def has_headers (self): return self.distribution.has_headers() def has_scripts (self): return self.distribution.has_scripts() def has_data (self): return self.distribution.has_data_files() # 'sub_commands': a list of commands this command might have to run to # get its work done. See cmd.py for more info. sub_commands = [('install_lib', has_lib), ('install_headers', has_headers), ('install_scripts', has_scripts), ('install_data', has_data), ] # class install --- NEW FILE: install_data.py --- """distutils.command.install_data Implements the Distutils 'install_data' command, for installing platform-independent data files.""" # contributed by Bastian Kleineidam # This module should be kept compatible with Python 2.1. __revision__ = "$Id: install_data.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os from types import StringType from distutils.core import Command from distutils.util import change_root, convert_path class install_data (Command): description = "install data files" user_options = [ ('install-dir=', 'd', "base directory for installing data files " "(default: installation base dir)"), ('root=', None, "install everything relative to this alternate root directory"), ('force', 'f', "force installation (overwrite existing files)"), ] boolean_options = ['force'] def initialize_options (self): self.install_dir = None self.outfiles = [] self.root = None self.force = 0 self.data_files = self.distribution.data_files self.warn_dir = 1 def finalize_options (self): self.set_undefined_options('install', ('install_data', 'install_dir'), ('root', 'root'), ('force', 'force'), ) def run (self): self.mkpath(self.install_dir) for f in self.data_files: if type(f) is StringType: # it's a simple file, so copy it f = convert_path(f) if self.warn_dir: self.warn("setup script did not provide a directory for " "'%s' -- installing right in '%s'" % (f, self.install_dir)) (out, _) = self.copy_file(f, self.install_dir) self.outfiles.append(out) else: # it's a tuple with path to install to and a list of files dir = convert_path(f[0]) if not os.path.isabs(dir): dir = os.path.join(self.install_dir, dir) elif self.root: dir = change_root(self.root, dir) self.mkpath(dir) if f[1] == []: # If there are no files listed, the user must be # trying to create an empty directory, so add the # directory to the list of output files. self.outfiles.append(dir) else: # Copy files, adding them to the list of output files. for data in f[1]: data = convert_path(data) (out, _) = self.copy_file(data, dir) self.outfiles.append(out) def get_inputs (self): return self.data_files or [] def get_outputs (self): return self.outfiles --- NEW FILE: install_headers.py --- """distutils.command.install_headers Implements the Distutils 'install_headers' command, to install C/C++ header files to the Python include directory.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: install_headers.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os from distutils.core import Command class install_headers (Command): description = "install C/C++ header files" user_options = [('install-dir=', 'd', "directory to install header files to"), ('force', 'f', "force installation (overwrite existing files)"), ] boolean_options = ['force'] def initialize_options (self): self.install_dir = None self.force = 0 self.outfiles = [] def finalize_options (self): self.set_undefined_options('install', ('install_headers', 'install_dir'), ('force', 'force')) def run (self): headers = self.distribution.headers if not headers: return self.mkpath(self.install_dir) for header in headers: (out, _) = self.copy_file(header, self.install_dir) self.outfiles.append(out) def get_inputs (self): return self.distribution.headers or [] def get_outputs (self): return self.outfiles # class install_headers --- NEW FILE: install_lib.py --- # This module should be kept compatible with Python 2.1. __revision__ = "$Id: install_lib.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string from types import IntType from distutils.core import Command from distutils.errors import DistutilsOptionError # Extension for Python source files. if hasattr(os, 'extsep'): PYTHON_SOURCE_EXTENSION = os.extsep + "py" else: PYTHON_SOURCE_EXTENSION = ".py" class install_lib (Command): description = "install all Python modules (extensions and pure Python)" # The byte-compilation options are a tad confusing. Here are the # possible scenarios: # 1) no compilation at all (--no-compile --no-optimize) # 2) compile .pyc only (--compile --no-optimize; default) # 3) compile .pyc and "level 1" .pyo (--compile --optimize) # 4) compile "level 1" .pyo only (--no-compile --optimize) # 5) compile .pyc and "level 2" .pyo (--compile --optimize-more) # 6) compile "level 2" .pyo only (--no-compile --optimize-more) # # The UI for this is two option, 'compile' and 'optimize'. # 'compile' is strictly boolean, and only decides whether to # generate .pyc files. 'optimize' is three-way (0, 1, or 2), and # decides both whether to generate .pyo files and what level of # optimization to use. user_options = [ ('install-dir=', 'd', "directory to install to"), ('build-dir=','b', "build directory (where to install from)"), ('force', 'f', "force installation (overwrite existing files)"), ('compile', 'c', "compile .py to .pyc [default]"), ('no-compile', None, "don't compile .py files"), ('optimize=', 'O', "also compile with optimization: -O1 for \"python -O\", " "-O2 for \"python -OO\", and -O0 to disable [default: -O0]"), ('skip-build', None, "skip the build steps"), ] boolean_options = ['force', 'compile', 'skip-build'] negative_opt = {'no-compile' : 'compile'} def initialize_options (self): # let the 'install' command dictate our installation directory self.install_dir = None self.build_dir = None self.force = 0 self.compile = None self.optimize = None self.skip_build = None def finalize_options (self): # Get all the information we need to install pure Python modules # from the umbrella 'install' command -- build (source) directory, # install (target) directory, and whether to compile .py files. self.set_undefined_options('install', ('build_lib', 'build_dir'), ('install_lib', 'install_dir'), ('force', 'force'), ('compile', 'compile'), ('optimize', 'optimize'), ('skip_build', 'skip_build'), ) if self.compile is None: self.compile = 1 if self.optimize is None: self.optimize = 0 if type(self.optimize) is not IntType: try: self.optimize = int(self.optimize) assert 0 <= self.optimize <= 2 except (ValueError, AssertionError): raise DistutilsOptionError, "optimize must be 0, 1, or 2" def run (self): # Make sure we have built everything we need first self.build() # Install everything: simply dump the entire contents of the build # directory to the installation directory (that's the beauty of # having a build directory!) outfiles = self.install() # (Optionally) compile .py to .pyc if outfiles is not None and self.distribution.has_pure_modules(): self.byte_compile(outfiles) # run () # -- Top-level worker functions ------------------------------------ # (called from 'run()') def build (self): if not self.skip_build: if self.distribution.has_pure_modules(): self.run_command('build_py') if self.distribution.has_ext_modules(): self.run_command('build_ext') def install (self): if os.path.isdir(self.build_dir): outfiles = self.copy_tree(self.build_dir, self.install_dir) else: self.warn("'%s' does not exist -- no Python modules to install" % self.build_dir) return return outfiles def byte_compile (self, files): from distutils.util import byte_compile # Get the "--root" directory supplied to the "install" command, # and use it as a prefix to strip off the purported filename # encoded in bytecode files. This is far from complete, but it # should at least generate usable bytecode in RPM distributions. install_root = self.get_finalized_command('install').root if self.compile: byte_compile(files, optimize=0, force=self.force, prefix=install_root, dry_run=self.dry_run) if self.optimize > 0: byte_compile(files, optimize=self.optimize, force=self.force, prefix=install_root, verbose=self.verbose, dry_run=self.dry_run) # -- Utility methods ----------------------------------------------- def _mutate_outputs (self, has_any, build_cmd, cmd_option, output_dir): if not has_any: return [] build_cmd = self.get_finalized_command(build_cmd) build_files = build_cmd.get_outputs() build_dir = getattr(build_cmd, cmd_option) prefix_len = len(build_dir) + len(os.sep) outputs = [] for file in build_files: outputs.append(os.path.join(output_dir, file[prefix_len:])) return outputs # _mutate_outputs () def _bytecode_filenames (self, py_filenames): bytecode_files = [] for py_file in py_filenames: # Since build_py handles package data installation, the # list of outputs can contain more than just .py files. # Make sure we only report bytecode for the .py files. ext = os.path.splitext(os.path.normcase(py_file))[1] if ext != PYTHON_SOURCE_EXTENSION: continue if self.compile: bytecode_files.append(py_file + "c") if self.optimize > 0: bytecode_files.append(py_file + "o") return bytecode_files # -- External interface -------------------------------------------- # (called by outsiders) def get_outputs (self): """Return the list of files that would be installed if this command were actually run. Not affected by the "dry-run" flag or whether modules have actually been built yet. """ pure_outputs = \ self._mutate_outputs(self.distribution.has_pure_modules(), 'build_py', 'build_lib', self.install_dir) if self.compile: bytecode_outputs = self._bytecode_filenames(pure_outputs) else: bytecode_outputs = [] ext_outputs = \ self._mutate_outputs(self.distribution.has_ext_modules(), 'build_ext', 'build_lib', self.install_dir) return pure_outputs + bytecode_outputs + ext_outputs # get_outputs () def get_inputs (self): """Get the list of files that are input to this command, ie. the files that get installed as they are named in the build tree. The files in this list correspond one-to-one to the output filenames returned by 'get_outputs()'. """ inputs = [] if self.distribution.has_pure_modules(): build_py = self.get_finalized_command('build_py') inputs.extend(build_py.get_outputs()) if self.distribution.has_ext_modules(): build_ext = self.get_finalized_command('build_ext') inputs.extend(build_ext.get_outputs()) return inputs # class install_lib --- NEW FILE: install_scripts.py --- """distutils.command.install_scripts Implements the Distutils 'install_scripts' command, for installing Python scripts.""" # contributed by Bastian Kleineidam # This module should be kept compatible with Python 2.1. __revision__ = "$Id: install_scripts.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os from distutils.core import Command from distutils import log from stat import ST_MODE class install_scripts (Command): description = "install scripts (Python or otherwise)" user_options = [ ('install-dir=', 'd', "directory to install scripts to"), ('build-dir=','b', "build directory (where to install from)"), ('force', 'f', "force installation (overwrite existing files)"), ('skip-build', None, "skip the build steps"), ] boolean_options = ['force', 'skip-build'] def initialize_options (self): self.install_dir = None self.force = 0 self.build_dir = None self.skip_build = None def finalize_options (self): self.set_undefined_options('build', ('build_scripts', 'build_dir')) self.set_undefined_options('install', ('install_scripts', 'install_dir'), ('force', 'force'), ('skip_build', 'skip_build'), ) def run (self): if not self.skip_build: self.run_command('build_scripts') self.outfiles = self.copy_tree(self.build_dir, self.install_dir) if os.name == 'posix': # Set the executable bits (owner, group, and world) on # all the scripts we just installed. for file in self.get_outputs(): if self.dry_run: log.info("changing mode of %s", file) else: mode = ((os.stat(file)[ST_MODE]) | 0555) & 07777 log.info("changing mode of %s to %o", file, mode) os.chmod(file, mode) def get_inputs (self): return self.distribution.scripts or [] def get_outputs(self): return self.outfiles or [] # class install_scripts --- NEW FILE: register.py --- """distutils.command.register Implements the Distutils 'register' command (register with the repository). """ # created 2002/10/21, Richard Jones __revision__ = "$Id: register.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string, urllib2, getpass, urlparse import StringIO, ConfigParser from distutils.core import Command from distutils.errors import * class register(Command): description = ("register the distribution with the Python package index") DEFAULT_REPOSITORY = 'http://www.python.org/pypi' user_options = [ ('repository=', 'r', "url of repository [default: %s]"%DEFAULT_REPOSITORY), ('list-classifiers', None, 'list the valid Trove classifiers'), ('show-response', None, 'display full response text from server'), ] boolean_options = ['verify', 'show-response', 'list-classifiers'] def initialize_options(self): self.repository = None self.show_response = 0 self.list_classifiers = 0 def finalize_options(self): if self.repository is None: self.repository = self.DEFAULT_REPOSITORY def run(self): self.check_metadata() if self.dry_run: self.verify_metadata() elif self.list_classifiers: self.classifiers() else: self.send_metadata() def check_metadata(self): """Ensure that all required elements of meta-data (name, version, URL, (author and author_email) or (maintainer and maintainer_email)) are supplied by the Distribution object; warn if any are missing. """ metadata = self.distribution.metadata missing = [] for attr in ('name', 'version', 'url'): if not (hasattr(metadata, attr) and getattr(metadata, attr)): missing.append(attr) if missing: self.warn("missing required meta-data: " + string.join(missing, ", ")) if metadata.author: if not metadata.author_email: self.warn("missing meta-data: if 'author' supplied, " + "'author_email' must be supplied too") elif metadata.maintainer: if not metadata.maintainer_email: self.warn("missing meta-data: if 'maintainer' supplied, " + "'maintainer_email' must be supplied too") else: self.warn("missing meta-data: either (author and author_email) " + "or (maintainer and maintainer_email) " + "must be supplied") def classifiers(self): ''' Fetch the list of classifiers from the server. ''' response = urllib2.urlopen(self.repository+'?:action=list_classifiers') print response.read() def verify_metadata(self): ''' Send the metadata to the package index server to be checked. ''' # send the info to the server and report the result (code, result) = self.post_to_server(self.build_post_data('verify')) print 'Server response (%s): %s'%(code, result) def send_metadata(self): ''' Send the metadata to the package index server. Well, do the following: 1. figure who the user is, and then 2. send the data as a Basic auth'ed POST. First we try to read the username/password from $HOME/.pypirc, which is a ConfigParser-formatted file with a section [server-login] containing username and password entries (both in clear text). Eg: [server-login] username: fred password: sekrit Otherwise, to figure who the user is, we offer the user three choices: 1. use existing login, 2. register as a new user, or 3. set the password to a random string and email the user. ''' choice = 'x' username = password = '' # see if we can short-cut and get the username/password from the # config config = None if os.environ.has_key('HOME'): rc = os.path.join(os.environ['HOME'], '.pypirc') if os.path.exists(rc): print 'Using PyPI login from %s'%rc config = ConfigParser.ConfigParser() config.read(rc) username = config.get('server-login', 'username') password = config.get('server-login', 'password') choice = '1' # get the user's login info choices = '1 2 3 4'.split() while choice not in choices: print '''We need to know who you are, so please choose either: 1. use your existing login, 2. register as a new user, 3. have the server generate a new password for you (and email it to you), or 4. quit Your selection [default 1]: ''', choice = raw_input() if not choice: choice = '1' elif choice not in choices: print 'Please choose one of the four options!' if choice == '1': # get the username and password while not username: username = raw_input('Username: ') while not password: password = getpass.getpass('Password: ') # set up the authentication auth = urllib2.HTTPPasswordMgr() host = urlparse.urlparse(self.repository)[1] auth.add_password('pypi', host, username, password) # send the info to the server and report the result code, result = self.post_to_server(self.build_post_data('submit'), auth) print 'Server response (%s): %s'%(code, result) # possibly save the login if os.environ.has_key('HOME') and config is None and code == 200: rc = os.path.join(os.environ['HOME'], '.pypirc') print 'I can store your PyPI login so future submissions will be faster.' print '(the login will be stored in %s)'%rc choice = 'X' while choice.lower() not in 'yn': choice = raw_input('Save your login (y/N)?') if not choice: choice = 'n' if choice.lower() == 'y': f = open(rc, 'w') f.write('[server-login]\nusername:%s\npassword:%s\n'%( username, password)) f.close() try: os.chmod(rc, 0600) except: pass elif choice == '2': data = {':action': 'user'} data['name'] = data['password'] = data['email'] = '' data['confirm'] = None while not data['name']: data['name'] = raw_input('Username: ') while data['password'] != data['confirm']: while not data['password']: data['password'] = getpass.getpass('Password: ') while not data['confirm']: data['confirm'] = getpass.getpass(' Confirm: ') if data['password'] != data['confirm']: data['password'] = '' data['confirm'] = None print "Password and confirm don't match!" while not data['email']: data['email'] = raw_input(' EMail: ') code, result = self.post_to_server(data) if code != 200: print 'Server response (%s): %s'%(code, result) else: print 'You will receive an email shortly.' print 'Follow the instructions in it to complete registration.' elif choice == '3': data = {':action': 'password_reset'} data['email'] = '' while not data['email']: data['email'] = raw_input('Your email address: ') code, result = self.post_to_server(data) print 'Server response (%s): %s'%(code, result) def build_post_data(self, action): # figure the data to send - the metadata plus some additional # information used by the package server meta = self.distribution.metadata data = { ':action': action, 'metadata_version' : '1.0', 'name': meta.get_name(), 'version': meta.get_version(), 'summary': meta.get_description(), 'home_page': meta.get_url(), 'author': meta.get_contact(), 'author_email': meta.get_contact_email(), 'license': meta.get_licence(), 'description': meta.get_long_description(), 'keywords': meta.get_keywords(), 'platform': meta.get_platforms(), 'classifiers': meta.get_classifiers(), 'download_url': meta.get_download_url(), # PEP 314 'provides': meta.get_provides(), 'requires': meta.get_requires(), 'obsoletes': meta.get_obsoletes(), } if data['provides'] or data['requires'] or data['obsoletes']: data['metadata_version'] = '1.1' return data def post_to_server(self, data, auth=None): ''' Post a query to the server, and return a string response. ''' # Build up the MIME payload for the urllib2 POST data boundary = '--------------GHSKFJDLGDS7543FJKLFHRE75642756743254' sep_boundary = '\n--' + boundary end_boundary = sep_boundary + '--' body = StringIO.StringIO() for key, value in data.items(): # handle multiple entries for the same name if type(value) != type([]): value = [value] for value in value: value = unicode(value).encode("utf-8") body.write(sep_boundary) body.write('\nContent-Disposition: form-data; name="%s"'%key) body.write("\n\n") body.write(value) if value and value[-1] == '\r': body.write('\n') # write an extra newline (lurve Macs) body.write(end_boundary) body.write("\n") body = body.getvalue() # build the Request headers = { 'Content-type': 'multipart/form-data; boundary=%s; charset=utf-8'%boundary, 'Content-length': str(len(body)) } req = urllib2.Request(self.repository, body, headers) # handle HTTP and include the Basic Auth handler opener = urllib2.build_opener( urllib2.HTTPBasicAuthHandler(password_mgr=auth) ) data = '' try: result = opener.open(req) except urllib2.HTTPError, e: if self.show_response: data = e.fp.read() result = e.code, e.msg except urllib2.URLError, e: result = 500, str(e) else: if self.show_response: data = result.read() result = 200, 'OK' if self.show_response: print '-'*75, data, '-'*75 return result --- NEW FILE: sdist.py --- """distutils.command.sdist Implements the Distutils 'sdist' command (create a source distribution).""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: sdist.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string from types import * from glob import glob from distutils.core import Command from distutils import dir_util, dep_util, file_util, archive_util from distutils.text_file import TextFile from distutils.errors import * from distutils.filelist import FileList from distutils import log def show_formats (): """Print all possible values for the 'formats' option (used by the "--help-formats" command-line option). """ from distutils.fancy_getopt import FancyGetopt from distutils.archive_util import ARCHIVE_FORMATS formats=[] for format in ARCHIVE_FORMATS.keys(): formats.append(("formats=" + format, None, ARCHIVE_FORMATS[format][2])) formats.sort() pretty_printer = FancyGetopt(formats) pretty_printer.print_help( "List of available source distribution formats:") class sdist (Command): description = "create a source distribution (tarball, zip file, etc.)" user_options = [ ('template=', 't', "name of manifest template file [default: MANIFEST.in]"), ('manifest=', 'm', "name of manifest file [default: MANIFEST]"), ('use-defaults', None, "include the default file set in the manifest " "[default; disable with --no-defaults]"), ('no-defaults', None, "don't include the default file set"), ('prune', None, "specifically exclude files/directories that should not be " "distributed (build tree, RCS/CVS dirs, etc.) " "[default; disable with --no-prune]"), ('no-prune', None, "don't automatically exclude anything"), ('manifest-only', 'o', "just regenerate the manifest and then stop " "(implies --force-manifest)"), ('force-manifest', 'f', "forcibly regenerate the manifest and carry on as usual"), ('formats=', None, "formats for source distribution (comma-separated list)"), ('keep-temp', 'k', "keep the distribution tree around after creating " + "archive file(s)"), ('dist-dir=', 'd', "directory to put the source distribution archive(s) in " "[default: dist]"), ] boolean_options = ['use-defaults', 'prune', 'manifest-only', 'force-manifest', 'keep-temp'] help_options = [ ('help-formats', None, "list available distribution formats", show_formats), ] negative_opt = {'no-defaults': 'use-defaults', 'no-prune': 'prune' } default_format = { 'posix': 'gztar', 'nt': 'zip' } def initialize_options (self): # 'template' and 'manifest' are, respectively, the names of # the manifest template and manifest file. self.template = None self.manifest = None # 'use_defaults': if true, we will include the default file set # in the manifest self.use_defaults = 1 self.prune = 1 self.manifest_only = 0 self.force_manifest = 0 self.formats = None self.keep_temp = 0 self.dist_dir = None self.archive_files = None def finalize_options (self): if self.manifest is None: self.manifest = "MANIFEST" if self.template is None: self.template = "MANIFEST.in" self.ensure_string_list('formats') if self.formats is None: try: self.formats = [self.default_format[os.name]] except KeyError: raise DistutilsPlatformError, \ "don't know how to create source distributions " + \ "on platform %s" % os.name bad_format = archive_util.check_archive_formats(self.formats) if bad_format: raise DistutilsOptionError, \ "unknown archive format '%s'" % bad_format if self.dist_dir is None: self.dist_dir = "dist" def run (self): # 'filelist' contains the list of files that will make up the # manifest self.filelist = FileList() # Ensure that all required meta-data is given; warn if not (but # don't die, it's not *that* serious!) self.check_metadata() # Do whatever it takes to get the list of files to process # (process the manifest template, read an existing manifest, # whatever). File list is accumulated in 'self.filelist'. self.get_file_list() # If user just wanted us to regenerate the manifest, stop now. if self.manifest_only: return # Otherwise, go ahead and create the source distribution tarball, # or zipfile, or whatever. self.make_distribution() def check_metadata (self): """Ensure that all required elements of meta-data (name, version, URL, (author and author_email) or (maintainer and maintainer_email)) are supplied by the Distribution object; warn if any are missing. """ metadata = self.distribution.metadata missing = [] for attr in ('name', 'version', 'url'): if not (hasattr(metadata, attr) and getattr(metadata, attr)): missing.append(attr) if missing: self.warn("missing required meta-data: " + string.join(missing, ", ")) if metadata.author: if not metadata.author_email: self.warn("missing meta-data: if 'author' supplied, " + "'author_email' must be supplied too") elif metadata.maintainer: if not metadata.maintainer_email: self.warn("missing meta-data: if 'maintainer' supplied, " + "'maintainer_email' must be supplied too") else: self.warn("missing meta-data: either (author and author_email) " + "or (maintainer and maintainer_email) " + "must be supplied") # check_metadata () def get_file_list (self): """Figure out the list of files to include in the source distribution, and put it in 'self.filelist'. This might involve reading the manifest template (and writing the manifest), or just reading the manifest, or just using the default file set -- it all depends on the user's options and the state of the filesystem. """ # If we have a manifest template, see if it's newer than the # manifest; if so, we'll regenerate the manifest. template_exists = os.path.isfile(self.template) if template_exists: template_newer = dep_util.newer(self.template, self.manifest) # The contents of the manifest file almost certainly depend on the # setup script as well as the manifest template -- so if the setup # script is newer than the manifest, we'll regenerate the manifest # from the template. (Well, not quite: if we already have a # manifest, but there's no template -- which will happen if the # developer elects to generate a manifest some other way -- then we # can't regenerate the manifest, so we don't.) self.debug_print("checking if %s newer than %s" % (self.distribution.script_name, self.manifest)) setup_newer = dep_util.newer(self.distribution.script_name, self.manifest) # cases: # 1) no manifest, template exists: generate manifest # (covered by 2a: no manifest == template newer) # 2) manifest & template exist: # 2a) template or setup script newer than manifest: # regenerate manifest # 2b) manifest newer than both: # do nothing (unless --force or --manifest-only) # 3) manifest exists, no template: # do nothing (unless --force or --manifest-only) # 4) no manifest, no template: generate w/ warning ("defaults only") manifest_outofdate = (template_exists and (template_newer or setup_newer)) force_regen = self.force_manifest or self.manifest_only manifest_exists = os.path.isfile(self.manifest) neither_exists = (not template_exists and not manifest_exists) # Regenerate the manifest if necessary (or if explicitly told to) if manifest_outofdate or neither_exists or force_regen: if not template_exists: self.warn(("manifest template '%s' does not exist " + "(using default file list)") % self.template) self.filelist.findall() if self.use_defaults: self.add_defaults() if template_exists: self.read_template() if self.prune: self.prune_file_list() self.filelist.sort() self.filelist.remove_duplicates() self.write_manifest() # Don't regenerate the manifest, just read it in. else: self.read_manifest() # get_file_list () def add_defaults (self): """Add all the default files to self.filelist: - README or README.txt - setup.py - test/test*.py - all pure Python modules mentioned in setup script - all C sources listed as part of extensions or C libraries in the setup script (doesn't catch C headers!) Warns if (README or README.txt) or setup.py are missing; everything else is optional. """ standards = [('README', 'README.txt'), self.distribution.script_name] for fn in standards: if type(fn) is TupleType: alts = fn got_it = 0 for fn in alts: if os.path.exists(fn): got_it = 1 self.filelist.append(fn) break if not got_it: self.warn("standard file not found: should have one of " + string.join(alts, ', ')) else: if os.path.exists(fn): self.filelist.append(fn) else: self.warn("standard file '%s' not found" % fn) optional = ['test/test*.py', 'setup.cfg'] for pattern in optional: files = filter(os.path.isfile, glob(pattern)) if files: self.filelist.extend(files) if self.distribution.has_pure_modules(): build_py = self.get_finalized_command('build_py') self.filelist.extend(build_py.get_source_files()) if self.distribution.has_ext_modules(): build_ext = self.get_finalized_command('build_ext') self.filelist.extend(build_ext.get_source_files()) if self.distribution.has_c_libraries(): build_clib = self.get_finalized_command('build_clib') self.filelist.extend(build_clib.get_source_files()) if self.distribution.has_scripts(): build_scripts = self.get_finalized_command('build_scripts') self.filelist.extend(build_scripts.get_source_files()) # add_defaults () def read_template (self): """Read and parse manifest template file named by self.template. (usually "MANIFEST.in") The parsing and processing is done by 'self.filelist', which updates itself accordingly. """ log.info("reading manifest template '%s'", self.template) template = TextFile(self.template, strip_comments=1, skip_blanks=1, join_lines=1, lstrip_ws=1, rstrip_ws=1, collapse_join=1) while 1: line = template.readline() if line is None: # end of file break try: self.filelist.process_template_line(line) except DistutilsTemplateError, msg: self.warn("%s, line %d: %s" % (template.filename, template.current_line, msg)) # read_template () def prune_file_list (self): """Prune off branches that might slip into the file list as created by 'read_template()', but really don't belong there: * the build tree (typically "build") * the release tree itself (only an issue if we ran "sdist" previously with --keep-temp, or it aborted) * any RCS, CVS and .svn directories """ build = self.get_finalized_command('build') base_dir = self.distribution.get_fullname() self.filelist.exclude_pattern(None, prefix=build.build_base) self.filelist.exclude_pattern(None, prefix=base_dir) self.filelist.exclude_pattern(r'/(RCS|CVS|\.svn)/.*', is_regex=1) def write_manifest (self): """Write the file list in 'self.filelist' (presumably as filled in by 'add_defaults()' and 'read_template()') to the manifest file named by 'self.manifest'. """ self.execute(file_util.write_file, (self.manifest, self.filelist.files), "writing manifest file '%s'" % self.manifest) # write_manifest () def read_manifest (self): """Read the manifest file (named by 'self.manifest') and use it to fill in 'self.filelist', the list of files to include in the source distribution. """ log.info("reading manifest file '%s'", self.manifest) manifest = open(self.manifest) while 1: line = manifest.readline() if line == '': # end of file break if line[-1] == '\n': line = line[0:-1] self.filelist.append(line) # read_manifest () def make_release_tree (self, base_dir, files): """Create the directory tree that will become the source distribution archive. All directories implied by the filenames in 'files' are created under 'base_dir', and then we hard link or copy (if hard linking is unavailable) those files into place. Essentially, this duplicates the developer's source tree, but in a directory named after the distribution, containing only the files to be distributed. """ # Create all the directories under 'base_dir' necessary to # put 'files' there; the 'mkpath()' is just so we don't die # if the manifest happens to be empty. self.mkpath(base_dir) dir_util.create_tree(base_dir, files, dry_run=self.dry_run) # And walk over the list of files, either making a hard link (if # os.link exists) to each one that doesn't already exist in its # corresponding location under 'base_dir', or copying each file # that's out-of-date in 'base_dir'. (Usually, all files will be # out-of-date, because by default we blow away 'base_dir' when # we're done making the distribution archives.) if hasattr(os, 'link'): # can make hard links on this system link = 'hard' msg = "making hard links in %s..." % base_dir else: # nope, have to copy link = None msg = "copying files to %s..." % base_dir if not files: log.warn("no files to distribute -- empty manifest?") else: log.info(msg) for file in files: if not os.path.isfile(file): log.warn("'%s' not a regular file -- skipping" % file) else: dest = os.path.join(base_dir, file) self.copy_file(file, dest, link=link) self.distribution.metadata.write_pkg_info(base_dir) # make_release_tree () def make_distribution (self): """Create the source distribution(s). First, we create the release tree with 'make_release_tree()'; then, we create all required archive files (according to 'self.formats') from the release tree. Finally, we clean up by blowing away the release tree (unless 'self.keep_temp' is true). The list of archive files created is stored so it can be retrieved later by 'get_archive_files()'. """ # Don't warn about missing meta-data here -- should be (and is!) # done elsewhere. base_dir = self.distribution.get_fullname() base_name = os.path.join(self.dist_dir, base_dir) self.make_release_tree(base_dir, self.filelist.files) archive_files = [] # remember names of files we create for fmt in self.formats: file = self.make_archive(base_name, fmt, base_dir=base_dir) archive_files.append(file) self.distribution.dist_files.append(('sdist', '', file)) self.archive_files = archive_files if not self.keep_temp: dir_util.remove_tree(base_dir, dry_run=self.dry_run) def get_archive_files (self): """Return the list of archive files created when the command was run, or None if the command hasn't run yet. """ return self.archive_files # class sdist --- NEW FILE: upload.py --- """distutils.command.upload Implements the Distutils 'upload' subcommand (upload package to PyPI).""" from distutils.errors import * from distutils.core import Command from distutils.spawn import spawn from distutils import log from md5 import md5 import os import socket import platform import ConfigParser import httplib import base64 import urlparse import cStringIO as StringIO class upload(Command): description = "upload binary package to PyPI" DEFAULT_REPOSITORY = 'http://www.python.org/pypi' user_options = [ ('repository=', 'r', "url of repository [default: %s]" % DEFAULT_REPOSITORY), ('show-response', None, 'display full response text from server'), ('sign', 's', 'sign files to upload using gpg'), ] boolean_options = ['show-response', 'sign'] def initialize_options(self): self.username = '' self.password = '' self.repository = '' self.show_response = 0 self.sign = False def finalize_options(self): if os.environ.has_key('HOME'): rc = os.path.join(os.environ['HOME'], '.pypirc') if os.path.exists(rc): self.announce('Using PyPI login from %s' % rc) config = ConfigParser.ConfigParser({ 'username':'', 'password':'', 'repository':''}) config.read(rc) if not self.repository: self.repository = config.get('server-login', 'repository') if not self.username: self.username = config.get('server-login', 'username') if not self.password: self.password = config.get('server-login', 'password') if not self.repository: self.repository = self.DEFAULT_REPOSITORY def run(self): if not self.distribution.dist_files: raise DistutilsOptionError("No dist file created in earlier command") for command, pyversion, filename in self.distribution.dist_files: self.upload_file(command, pyversion, filename) def upload_file(self, command, pyversion, filename): # Sign if requested if self.sign: spawn(("gpg", "--detach-sign", "-a", filename), dry_run=self.dry_run) # Fill in the data content = open(filename).read() data = { ':action':'file_upload', 'protcol_version':'1', 'name':self.distribution.get_name(), 'version':self.distribution.get_version(), 'content':(os.path.basename(filename),content), 'filetype':command, 'pyversion':pyversion, 'md5_digest':md5(content).hexdigest(), } comment = '' if command == 'bdist_rpm': dist, version, id = platform.dist() if dist: comment = 'built for %s %s' % (dist, version) elif command == 'bdist_dumb': comment = 'built for %s' % platform.platform(terse=1) data['comment'] = comment if self.sign: data['gpg_signature'] = (os.path.basename(filename) + ".asc", open(filename+".asc").read()) # set up the authentication auth = "Basic " + base64.encodestring(self.username + ":" + self.password).strip() # Build up the MIME payload for the POST data boundary = '--------------GHSKFJDLGDS7543FJKLFHRE75642756743254' sep_boundary = '\n--' + boundary end_boundary = sep_boundary + '--' body = StringIO.StringIO() for key, value in data.items(): # handle multiple entries for the same name if type(value) != type([]): value = [value] for value in value: if type(value) is tuple: fn = ';filename="%s"' % value[0] value = value[1] else: fn = "" value = str(value) body.write(sep_boundary) body.write('\nContent-Disposition: form-data; name="%s"'%key) body.write(fn) body.write("\n\n") body.write(value) if value and value[-1] == '\r': body.write('\n') # write an extra newline (lurve Macs) body.write(end_boundary) body.write("\n") body = body.getvalue() self.announce("Submitting %s to %s" % (filename, self.repository), log.INFO) # build the Request # We can't use urllib2 since we need to send the Basic # auth right with the first request schema, netloc, url, params, query, fragments = \ urlparse.urlparse(self.repository) assert not params and not query and not fragments if schema == 'http': http = httplib.HTTPConnection(netloc) elif schema == 'https': http = httplib.HTTPSConnection(netloc) else: raise AssertionError, "unsupported schema "+schema data = '' loglevel = log.INFO try: http.connect() http.putrequest("POST", url) http.putheader('Content-type', 'multipart/form-data; boundary=%s'%boundary) http.putheader('Content-length', str(len(body))) http.putheader('Authorization', auth) http.endheaders() http.send(body) except socket.error, e: self.announce(e.msg, log.ERROR) return r = http.getresponse() if r.status == 200: self.announce('Server response (%s): %s' % (r.status, r.reason), log.INFO) else: self.announce('Upload failed (%s): %s' % (r.status, r.reason), log.ERROR) if self.show_response: print '-'*75, r.read(), '-'*75 From anthonybaxter at users.sourceforge.net Fri Jun 10 08:43:25 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:43:25 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils .#dist.py.1.69, NONE, 1.1 .#sysconfig.py.1.60, NONE, 1.1 README, NONE, 1.1 __init__.py, NONE, 1.1 archive_util.py, NONE, 1.1 bcppcompiler.py, NONE, 1.1 ccompiler.py, NONE, 1.1 cmd.py, NONE, 1.1 core.py, NONE, 1.1 cygwinccompiler.py, NONE, 1.1 debug.py, NONE, 1.1 dep_util.py, NONE, 1.1 dir_util.py, NONE, 1.1 dist.py, NONE, 1.1 emxccompiler.py, NONE, 1.1 errors.py, NONE, 1.1 extension.py, NONE, 1.1 fancy_getopt.py, NONE, 1.1 file_util.py, NONE, 1.1 filelist.py, NONE, 1.1 log.py, NONE, 1.1 msvccompiler.py, NONE, 1.1 mwerkscompiler.py, NONE, 1.1 spawn.py, NONE, 1.1 sysconfig.py, NONE, 1.1 text_file.py, NONE, 1.1 unixccompiler.py, NONE, 1.1 util.py, NONE, 1.1 version.py, NONE, 1.1 versionpredicate.py, NONE, 1.1 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22974/distutils_refactor/distutils Added Files: .#dist.py.1.69 .#sysconfig.py.1.60 README __init__.py archive_util.py bcppcompiler.py ccompiler.py cmd.py core.py cygwinccompiler.py debug.py dep_util.py dir_util.py dist.py emxccompiler.py errors.py extension.py fancy_getopt.py file_util.py filelist.py log.py msvccompiler.py mwerkscompiler.py spawn.py sysconfig.py text_file.py unixccompiler.py util.py version.py versionpredicate.py Log Message: checking into the sandbox --- NEW FILE: .#dist.py.1.69 --- """distutils.dist Provides the Distribution class, which represents the module distribution being built/installed/distributed. """ # This module should be kept compatible with Python 1.5.2. __revision__ = "$Id: .#dist.py.1.69,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string, re from types import * from copy import copy try: import warnings except ImportError: warnings = None [...1162 lines suppressed...] def get_provides(self): return self.provides # class DistributionMetadata def fix_help_options (options): """Convert a 4-tuple 'help_options' list as found in various command classes to the 3-tuple form required by FancyGetopt. """ new_options = [] for help_tuple in options: new_options.append(help_tuple[0:3]) return new_options if __name__ == "__main__": dist = Distribution() print "ok" --- NEW FILE: .#sysconfig.py.1.60 --- """Provide access to Python's configuration information. The specific configuration variables available depend heavily on the platform and configuration. The values may be retrieved using get_config_var(name), and the list of variables is available via get_config_vars().keys(). Additional convenience functions are also available. Written by: Fred L. Drake, Jr. Email: """ __revision__ = "$Id: .#sysconfig.py.1.60,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os import re import string import sys from errors import DistutilsPlatformError # These are needed in a couple of spots, so just compute them once. PREFIX = os.path.normpath(sys.prefix) EXEC_PREFIX = os.path.normpath(sys.exec_prefix) # python_build: (Boolean) if true, we're either building Python or # building an extension with an un-installed Python, so we use # different (hard-wired) directories. argv0_path = os.path.dirname(os.path.abspath(sys.executable)) landmark = os.path.join(argv0_path, "Modules", "Setup") python_build = os.path.isfile(landmark) del argv0_path, landmark def get_python_version (): """Return a string containing the major and minor Python version, leaving off the patchlevel. Sample return values could be '1.5' or '2.2'. """ return sys.version[:3] def get_python_inc(plat_specific=0, prefix=None): """Return the directory containing installed Python header files. If 'plat_specific' is false (the default), this is the path to the non-platform-specific header files, i.e. Python.h and so on; otherwise, this is the path to platform-specific header files (namely pyconfig.h). If 'prefix' is supplied, use it instead of sys.prefix or sys.exec_prefix -- i.e., ignore 'plat_specific'. """ if prefix is None: prefix = plat_specific and EXEC_PREFIX or PREFIX if os.name == "posix": if python_build: base = os.path.dirname(os.path.abspath(sys.executable)) if plat_specific: inc_dir = base else: inc_dir = os.path.join(base, "Include") if not os.path.exists(inc_dir): inc_dir = os.path.join(os.path.dirname(base), "Include") return inc_dir return os.path.join(prefix, "include", "python" + sys.version[:3]) elif os.name == "nt": return os.path.join(prefix, "include") elif os.name == "mac": if plat_specific: return os.path.join(prefix, "Mac", "Include") else: return os.path.join(prefix, "Include") elif os.name == "os2": return os.path.join(prefix, "Include") else: raise DistutilsPlatformError( "I don't know where Python installs its C header files " "on platform '%s'" % os.name) def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): """Return the directory containing the Python library (standard or site additions). If 'plat_specific' is true, return the directory containing platform-specific modules, i.e. any module from a non-pure-Python module distribution; otherwise, return the platform-shared library directory. If 'standard_lib' is true, return the directory containing standard Python library modules; otherwise, return the directory for site-specific modules. If 'prefix' is supplied, use it instead of sys.prefix or sys.exec_prefix -- i.e., ignore 'plat_specific'. """ if prefix is None: prefix = plat_specific and EXEC_PREFIX or PREFIX if os.name == "posix": libpython = os.path.join(prefix, "lib", "python" + get_python_version()) if standard_lib: return libpython else: return os.path.join(libpython, "site-packages") elif os.name == "nt": if standard_lib: return os.path.join(prefix, "Lib") else: if sys.version < "2.2": return prefix else: return os.path.join(PREFIX, "Lib", "site-packages") elif os.name == "mac": if plat_specific: if standard_lib: return os.path.join(prefix, "Lib", "lib-dynload") else: return os.path.join(prefix, "Lib", "site-packages") else: if standard_lib: return os.path.join(prefix, "Lib") else: return os.path.join(prefix, "Lib", "site-packages") elif os.name == "os2": if standard_lib: return os.path.join(PREFIX, "Lib") else: return os.path.join(PREFIX, "Lib", "site-packages") else: raise DistutilsPlatformError( "I don't know where Python installs its library " "on platform '%s'" % os.name) def customize_compiler(compiler): """Do any platform-specific customization of a CCompiler instance. Mainly needed on Unix, so we can plug in the information that varies across Unices and is stored in Python's Makefile. """ if compiler.compiler_type == "unix": (cc, cxx, opt, basecflags, ccshared, ldshared, so_ext) = \ get_config_vars('CC', 'CXX', 'OPT', 'BASECFLAGS', 'CCSHARED', 'LDSHARED', 'SO') if os.environ.has_key('CC'): cc = os.environ['CC'] if os.environ.has_key('CXX'): cxx = os.environ['CXX'] if os.environ.has_key('LDSHARED'): ldshared = os.environ['LDSHARED'] if os.environ.has_key('CPP'): cpp = os.environ['CPP'] else: cpp = cc + " -E" # not always if os.environ.has_key('LDFLAGS'): ldshared = ldshared + ' ' + os.environ['LDFLAGS'] if basecflags: opt = basecflags + ' ' + opt if os.environ.has_key('CFLAGS'): opt = opt + ' ' + os.environ['CFLAGS'] ldshared = ldshared + ' ' + os.environ['CFLAGS'] if os.environ.has_key('CPPFLAGS'): cpp = cpp + ' ' + os.environ['CPPFLAGS'] opt = opt + ' ' + os.environ['CPPFLAGS'] ldshared = ldshared + ' ' + os.environ['CPPFLAGS'] cc_cmd = cc + ' ' + opt compiler.set_executables( preprocessor=cpp, compiler=cc_cmd, compiler_so=cc_cmd + ' ' + ccshared, compiler_cxx=cxx, linker_so=ldshared, linker_exe=cc) compiler.shared_lib_extension = so_ext def get_config_h_filename(): """Return full pathname of installed pyconfig.h file.""" if python_build: inc_dir = os.curdir else: inc_dir = get_python_inc(plat_specific=1) if sys.version < '2.2': config_h = 'config.h' else: # The name of the config.h file changed in 2.2 config_h = 'pyconfig.h' return os.path.join(inc_dir, config_h) def get_makefile_filename(): """Return full pathname of installed Makefile from the Python build.""" if python_build: return os.path.join(os.path.dirname(sys.executable), "Makefile") lib_dir = get_python_lib(plat_specific=1, standard_lib=1) return os.path.join(lib_dir, "config", "Makefile") def parse_config_h(fp, g=None): """Parse a config.h-style file. A dictionary containing name/value pairs is returned. If an optional dictionary is passed in as the second argument, it is used instead of a new dictionary. """ if g is None: g = {} define_rx = re.compile("#define ([A-Z][A-Z0-9_]+) (.*)\n") undef_rx = re.compile("/[*] #undef ([A-Z][A-Z0-9_]+) [*]/\n") # while 1: line = fp.readline() if not line: break m = define_rx.match(line) if m: n, v = m.group(1, 2) try: v = int(v) except ValueError: pass g[n] = v else: m = undef_rx.match(line) if m: g[m.group(1)] = 0 return g # Regexes needed for parsing Makefile (and similar syntaxes, # like old-style Setup files). _variable_rx = re.compile("([a-zA-Z][a-zA-Z0-9_]+)\s*=\s*(.*)") _findvar1_rx = re.compile(r"\$\(([A-Za-z][A-Za-z0-9_]*)\)") _findvar2_rx = re.compile(r"\${([A-Za-z][A-Za-z0-9_]*)}") def parse_makefile(fn, g=None): """Parse a Makefile-style file. A dictionary containing name/value pairs is returned. If an optional dictionary is passed in as the second argument, it is used instead of a new dictionary. """ from distutils.text_file import TextFile fp = TextFile(fn, strip_comments=1, skip_blanks=1, join_lines=1) if g is None: g = {} done = {} notdone = {} while 1: line = fp.readline() if line is None: # eof break m = _variable_rx.match(line) if m: n, v = m.group(1, 2) v = string.strip(v) if "$" in v: notdone[n] = v else: try: v = int(v) except ValueError: pass done[n] = v # do variable interpolation here while notdone: for name in notdone.keys(): value = notdone[name] m = _findvar1_rx.search(value) or _findvar2_rx.search(value) if m: n = m.group(1) if done.has_key(n): after = value[m.end():] value = value[:m.start()] + str(done[n]) + after if "$" in after: notdone[name] = value else: try: value = int(value) except ValueError: done[name] = string.strip(value) else: done[name] = value del notdone[name] elif notdone.has_key(n): # get it on a subsequent round pass else: done[n] = "" after = value[m.end():] value = value[:m.start()] + after if "$" in after: notdone[name] = value else: try: value = int(value) except ValueError: done[name] = string.strip(value) else: done[name] = value del notdone[name] else: # bogus variable reference; just drop it since we can't deal del notdone[name] fp.close() # save the results in the global dictionary g.update(done) return g def expand_makefile_vars(s, vars): """Expand Makefile-style variables -- "${foo}" or "$(foo)" -- in 'string' according to 'vars' (a dictionary mapping variable names to values). Variables not present in 'vars' are silently expanded to the empty string. The variable values in 'vars' should not contain further variable expansions; if 'vars' is the output of 'parse_makefile()', you're fine. Returns a variable-expanded version of 's'. """ # This algorithm does multiple expansion, so if vars['foo'] contains # "${bar}", it will expand ${foo} to ${bar}, and then expand # ${bar}... and so forth. This is fine as long as 'vars' comes from # 'parse_makefile()', which takes care of such expansions eagerly, # according to make's variable expansion semantics. while 1: m = _findvar1_rx.search(s) or _findvar2_rx.search(s) if m: (beg, end) = m.span() s = s[0:beg] + vars.get(m.group(1)) + s[end:] else: break return s _config_vars = None def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" g = {} # load the installed Makefile: try: filename = get_makefile_filename() parse_makefile(filename, g) except IOError, msg: my_msg = "invalid Python installation: unable to open %s" % filename if hasattr(msg, "strerror"): my_msg = my_msg + " (%s)" % msg.strerror raise DistutilsPlatformError(my_msg) # On MacOSX we need to check the setting of the environment variable # MACOSX_DEPLOYMENT_TARGET: configure bases some choices on it so # it needs to be compatible. # An alternative would be to force MACOSX_DEPLOYMENT_TARGET to be # the same as during configure. if sys.platform == 'darwin' and g.has_key('CONFIGURE_MACOSX_DEPLOYMENT_TARGET'): cfg_target = g['CONFIGURE_MACOSX_DEPLOYMENT_TARGET'] cur_target = os.getenv('MACOSX_DEPLOYMENT_TARGET', '') if cfg_target != cur_target: my_msg = ('$MACOSX_DEPLOYMENT_TARGET mismatch: now "%s" but "%s" during configure' % (cur_target, cfg_target)) raise DistutilsPlatformError(my_msg) # On AIX, there are wrong paths to the linker scripts in the Makefile # -- these paths are relative to the Python source, but when installed # the scripts are in another directory. if python_build: g['LDSHARED'] = g['BLDSHARED'] elif sys.version < '2.1': # The following two branches are for 1.5.2 compatibility. if sys.platform == 'aix4': # what about AIX 3.x ? # Linker script is in the config directory, not in Modules as the # Makefile says. python_lib = get_python_lib(standard_lib=1) ld_so_aix = os.path.join(python_lib, 'config', 'ld_so_aix') python_exp = os.path.join(python_lib, 'config', 'python.exp') g['LDSHARED'] = "%s %s -bI:%s" % (ld_so_aix, g['CC'], python_exp) elif sys.platform == 'beos': # Linker script is in the config directory. In the Makefile it is # relative to the srcdir, which after installation no longer makes # sense. python_lib = get_python_lib(standard_lib=1) linkerscript_path = string.split(g['LDSHARED'])[0] linkerscript_name = os.path.basename(linkerscript_path) linkerscript = os.path.join(python_lib, 'config', linkerscript_name) # XXX this isn't the right place to do this: adding the Python # library to the link, if needed, should be in the "build_ext" # command. (It's also needed for non-MS compilers on Windows, and # it's taken care of for them by the 'build_ext.get_libraries()' # method.) g['LDSHARED'] = ("%s -L%s/lib -lpython%s" % (linkerscript, PREFIX, sys.version[0:3])) global _config_vars _config_vars = g def _init_nt(): """Initialize the module as appropriate for NT""" g = {} # set basic install directories g['LIBDEST'] = get_python_lib(plat_specific=0, standard_lib=1) g['BINLIBDEST'] = get_python_lib(plat_specific=1, standard_lib=1) # XXX hmmm.. a normal install puts include files here g['INCLUDEPY'] = get_python_inc(plat_specific=0) g['SO'] = '.pyd' g['EXE'] = ".exe" global _config_vars _config_vars = g def _init_mac(): """Initialize the module as appropriate for Macintosh systems""" g = {} # set basic install directories g['LIBDEST'] = get_python_lib(plat_specific=0, standard_lib=1) g['BINLIBDEST'] = get_python_lib(plat_specific=1, standard_lib=1) # XXX hmmm.. a normal install puts include files here g['INCLUDEPY'] = get_python_inc(plat_specific=0) import MacOS if not hasattr(MacOS, 'runtimemodel'): g['SO'] = '.ppc.slb' else: g['SO'] = '.%s.slb' % MacOS.runtimemodel # XXX are these used anywhere? g['install_lib'] = os.path.join(EXEC_PREFIX, "Lib") g['install_platlib'] = os.path.join(EXEC_PREFIX, "Mac", "Lib") # These are used by the extension module build g['srcdir'] = ':' global _config_vars _config_vars = g def _init_os2(): """Initialize the module as appropriate for OS/2""" g = {} # set basic install directories g['LIBDEST'] = get_python_lib(plat_specific=0, standard_lib=1) g['BINLIBDEST'] = get_python_lib(plat_specific=1, standard_lib=1) # XXX hmmm.. a normal install puts include files here g['INCLUDEPY'] = get_python_inc(plat_specific=0) g['SO'] = '.pyd' g['EXE'] = ".exe" global _config_vars _config_vars = g def get_config_vars(*args): """With no arguments, return a dictionary of all configuration variables relevant for the current platform. Generally this includes everything needed to build extensions and install both pure modules and extensions. On Unix, this means every variable defined in Python's installed Makefile; on Windows and Mac OS it's a much smaller set. With arguments, return a list of values that result from looking up each argument in the configuration variable dictionary. """ global _config_vars if _config_vars is None: func = globals().get("_init_" + os.name) if func: func() else: _config_vars = {} # Normalized versions of prefix and exec_prefix are handy to have; # in fact, these are the standard versions used most places in the # Distutils. _config_vars['prefix'] = PREFIX _config_vars['exec_prefix'] = EXEC_PREFIX if args: vals = [] for name in args: vals.append(_config_vars.get(name)) return vals else: return _config_vars def get_config_var(name): """Return the value of a single variable using the dictionary returned by 'get_config_vars()'. Equivalent to get_config_vars().get(name) """ return get_config_vars().get(name) --- NEW FILE: README --- This directory contains only a subset of the Distutils, specifically the Python modules in the 'distutils' and 'distutils.command' packages. This is all you need to distribute and install Python modules using the Distutils. There is also a separately packaged standalone version of the Distutils available for people who want to upgrade the Distutils without upgrading Python, available from the Distutils web page: http://www.python.org/sigs/distutils-sig/ The standalone version includes all of the code in this directory, plus documentation, test scripts, examples, etc. The Distutils documentation is divided into two documents, "Installing Python Modules", which explains how to install Python packages, and "Distributing Python Modules", which explains how to write setup.py files. Both documents are part of the standard Python documentation set, and are available from http://www.python.org/doc/current/ . Greg Ward (gward at python.net) $Id: README,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $ --- NEW FILE: __init__.py --- """distutils The main package for the Python Module Distribution Utilities. Normally used from a setup script as from distutils.core import setup setup (...) """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: __init__.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" __version__ = "2.4.0" --- NEW FILE: archive_util.py --- """distutils.archive_util Utility functions for creating archive files (tarballs, zip files, that sort of thing).""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: archive_util.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os from distutils.errors import DistutilsExecError from distutils.spawn import spawn from distutils.dir_util import mkpath from distutils import log def make_tarball (base_name, base_dir, compress="gzip", verbose=0, dry_run=0): """Create a (possibly compressed) tar file from all the files under 'base_dir'. 'compress' must be "gzip" (the default), "compress", "bzip2", or None. Both "tar" and the compression utility named by 'compress' must be on the default program search path, so this is probably Unix-specific. The output tar file will be named 'base_dir' + ".tar", possibly plus the appropriate compression extension (".gz", ".bz2" or ".Z"). Return the output filename. """ # XXX GNU tar 1.13 has a nifty option to add a prefix directory. # It's pretty new, though, so we certainly can't require it -- # but it would be nice to take advantage of it to skip the # "create a tree of hardlinks" step! (Would also be nice to # detect GNU tar to use its 'z' option and save a step.) compress_ext = { 'gzip': ".gz", 'bzip2': '.bz2', 'compress': ".Z" } # flags for compression program, each element of list will be an argument compress_flags = {'gzip': ["-f9"], 'compress': ["-f"], 'bzip2': ['-f9']} if compress is not None and compress not in compress_ext.keys(): raise ValueError, \ "bad value for 'compress': must be None, 'gzip', or 'compress'" archive_name = base_name + ".tar" mkpath(os.path.dirname(archive_name), dry_run=dry_run) cmd = ["tar", "-cf", archive_name, base_dir] spawn(cmd, dry_run=dry_run) if compress: spawn([compress] + compress_flags[compress] + [archive_name], dry_run=dry_run) return archive_name + compress_ext[compress] else: return archive_name # make_tarball () def make_zipfile (base_name, base_dir, verbose=0, dry_run=0): """Create a zip file from all the files under 'base_dir'. The output zip file will be named 'base_dir' + ".zip". Uses either the "zipfile" Python module (if available) or the InfoZIP "zip" utility (if installed and found on the default search path). If neither tool is available, raises DistutilsExecError. Returns the name of the output zip file. """ try: import zipfile except ImportError: zipfile = None zip_filename = base_name + ".zip" mkpath(os.path.dirname(zip_filename), dry_run=dry_run) # If zipfile module is not available, try spawning an external # 'zip' command. if zipfile is None: if verbose: zipoptions = "-r" else: zipoptions = "-rq" try: spawn(["zip", zipoptions, zip_filename, base_dir], dry_run=dry_run) except DistutilsExecError: # XXX really should distinguish between "couldn't find # external 'zip' command" and "zip failed". raise DistutilsExecError, \ ("unable to create zip file '%s': " "could neither import the 'zipfile' module nor " "find a standalone zip utility") % zip_filename else: log.info("creating '%s' and adding '%s' to it", zip_filename, base_dir) def visit (z, dirname, names): for name in names: path = os.path.normpath(os.path.join(dirname, name)) if os.path.isfile(path): z.write(path, path) log.info("adding '%s'" % path) if not dry_run: z = zipfile.ZipFile(zip_filename, "w", compression=zipfile.ZIP_DEFLATED) os.path.walk(base_dir, visit, z) z.close() return zip_filename # make_zipfile () ARCHIVE_FORMATS = { 'gztar': (make_tarball, [('compress', 'gzip')], "gzip'ed tar-file"), 'bztar': (make_tarball, [('compress', 'bzip2')], "bzip2'ed tar-file"), 'ztar': (make_tarball, [('compress', 'compress')], "compressed tar file"), 'tar': (make_tarball, [('compress', None)], "uncompressed tar file"), 'zip': (make_zipfile, [],"ZIP file") } def check_archive_formats (formats): for format in formats: if not ARCHIVE_FORMATS.has_key(format): return format else: return None def make_archive (base_name, format, root_dir=None, base_dir=None, verbose=0, dry_run=0): """Create an archive file (eg. zip or tar). 'base_name' is the name of the file to create, minus any format-specific extension; 'format' is the archive format: one of "zip", "tar", "ztar", or "gztar". 'root_dir' is a directory that will be the root directory of the archive; ie. we typically chdir into 'root_dir' before creating the archive. 'base_dir' is the directory where we start archiving from; ie. 'base_dir' will be the common prefix of all files and directories in the archive. 'root_dir' and 'base_dir' both default to the current directory. Returns the name of the archive file. """ save_cwd = os.getcwd() if root_dir is not None: log.debug("changing into '%s'", root_dir) base_name = os.path.abspath(base_name) if not dry_run: os.chdir(root_dir) if base_dir is None: base_dir = os.curdir kwargs = { 'dry_run': dry_run } try: format_info = ARCHIVE_FORMATS[format] except KeyError: raise ValueError, "unknown archive format '%s'" % format func = format_info[0] for (arg,val) in format_info[1]: kwargs[arg] = val filename = apply(func, (base_name, base_dir), kwargs) if root_dir is not None: log.debug("changing back to '%s'", save_cwd) os.chdir(save_cwd) return filename # make_archive () --- NEW FILE: bcppcompiler.py --- """distutils.bcppcompiler Contains BorlandCCompiler, an implementation of the abstract CCompiler class for the Borland C++ compiler. """ # This implementation by Lyle Johnson, based on the original msvccompiler.py # module and using the directions originally published by Gordon Williams. # XXX looks like there's a LOT of overlap between these two classes: # someone should sit down and factor out the common code as # WindowsCCompiler! --GPW # This module should be kept compatible with Python 2.1. __revision__ = "$Id: bcppcompiler.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os from distutils.errors import \ DistutilsExecError, DistutilsPlatformError, \ CompileError, LibError, LinkError, UnknownFileError from distutils.ccompiler import \ CCompiler, gen_preprocess_options, gen_lib_options from distutils.file_util import write_file from distutils.dep_util import newer from distutils import log class BCPPCompiler(CCompiler) : """Concrete class that implements an interface to the Borland C/C++ compiler, as defined by the CCompiler abstract class. """ compiler_type = 'bcpp' # Just set this so CCompiler's constructor doesn't barf. We currently # don't use the 'set_executables()' bureaucracy provided by CCompiler, # as it really isn't necessary for this sort of single-compiler class. # Would be nice to have a consistent interface with UnixCCompiler, # though, so it's worth thinking about. executables = {} # Private class data (need to distinguish C from C++ source for compiler) _c_extensions = ['.c'] _cpp_extensions = ['.cc', '.cpp', '.cxx'] # Needed for the filename generation methods provided by the # base class, CCompiler. src_extensions = _c_extensions + _cpp_extensions obj_extension = '.obj' static_lib_extension = '.lib' shared_lib_extension = '.dll' static_lib_format = shared_lib_format = '%s%s' exe_extension = '.exe' def __init__ (self, verbose=0, dry_run=0, force=0): CCompiler.__init__ (self, verbose, dry_run, force) # These executables are assumed to all be in the path. # Borland doesn't seem to use any special registry settings to # indicate their installation locations. self.cc = "bcc32.exe" self.linker = "ilink32.exe" self.lib = "tlib.exe" self.preprocess_options = None self.compile_options = ['/tWM', '/O2', '/q', '/g0'] self.compile_options_debug = ['/tWM', '/Od', '/q', '/g0'] self.ldflags_shared = ['/Tpd', '/Gn', '/q', '/x'] self.ldflags_shared_debug = ['/Tpd', '/Gn', '/q', '/x'] self.ldflags_static = [] self.ldflags_exe = ['/Gn', '/q', '/x'] self.ldflags_exe_debug = ['/Gn', '/q', '/x','/r'] # -- Worker methods ------------------------------------------------ def compile(self, sources, output_dir=None, macros=None, include_dirs=None, debug=0, extra_preargs=None, extra_postargs=None, depends=None): macros, objects, extra_postargs, pp_opts, build = \ self._setup_compile(output_dir, macros, include_dirs, sources, depends, extra_postargs) compile_opts = extra_preargs or [] compile_opts.append ('-c') if debug: compile_opts.extend (self.compile_options_debug) else: compile_opts.extend (self.compile_options) for obj in objects: try: src, ext = build[obj] except KeyError: continue # XXX why do the normpath here? src = os.path.normpath(src) obj = os.path.normpath(obj) # XXX _setup_compile() did a mkpath() too but before the normpath. # Is it possible to skip the normpath? self.mkpath(os.path.dirname(obj)) if ext == '.res': # This is already a binary file -- skip it. continue # the 'for' loop if ext == '.rc': # This needs to be compiled to a .res file -- do it now. try: self.spawn (["brcc32", "-fo", obj, src]) except DistutilsExecError, msg: raise CompileError, msg continue # the 'for' loop # The next two are both for the real compiler. if ext in self._c_extensions: input_opt = "" elif ext in self._cpp_extensions: input_opt = "-P" else: # Unknown file type -- no extra options. The compiler # will probably fail, but let it just in case this is a # file the compiler recognizes even if we don't. input_opt = "" output_opt = "-o" + obj # Compiler command line syntax is: "bcc32 [options] file(s)". # Note that the source file names must appear at the end of # the command line. try: self.spawn ([self.cc] + compile_opts + pp_opts + [input_opt, output_opt] + extra_postargs + [src]) except DistutilsExecError, msg: raise CompileError, msg return objects # compile () def create_static_lib (self, objects, output_libname, output_dir=None, debug=0, target_lang=None): (objects, output_dir) = self._fix_object_args (objects, output_dir) output_filename = \ self.library_filename (output_libname, output_dir=output_dir) if self._need_link (objects, output_filename): lib_args = [output_filename, '/u'] + objects if debug: pass # XXX what goes here? try: self.spawn ([self.lib] + lib_args) except DistutilsExecError, msg: raise LibError, msg else: log.debug("skipping %s (up-to-date)", output_filename) # create_static_lib () def link (self, target_desc, objects, output_filename, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None): # XXX this ignores 'build_temp'! should follow the lead of # msvccompiler.py (objects, output_dir) = self._fix_object_args (objects, output_dir) (libraries, library_dirs, runtime_library_dirs) = \ self._fix_lib_args (libraries, library_dirs, runtime_library_dirs) if runtime_library_dirs: log.warn("I don't know what to do with 'runtime_library_dirs': %s", str(runtime_library_dirs)) if output_dir is not None: output_filename = os.path.join (output_dir, output_filename) if self._need_link (objects, output_filename): # Figure out linker args based on type of target. if target_desc == CCompiler.EXECUTABLE: startup_obj = 'c0w32' if debug: ld_args = self.ldflags_exe_debug[:] else: ld_args = self.ldflags_exe[:] else: startup_obj = 'c0d32' if debug: ld_args = self.ldflags_shared_debug[:] else: ld_args = self.ldflags_shared[:] # Create a temporary exports file for use by the linker if export_symbols is None: def_file = '' else: head, tail = os.path.split (output_filename) modname, ext = os.path.splitext (tail) temp_dir = os.path.dirname(objects[0]) # preserve tree structure def_file = os.path.join (temp_dir, '%s.def' % modname) contents = ['EXPORTS'] for sym in (export_symbols or []): contents.append(' %s=_%s' % (sym, sym)) self.execute(write_file, (def_file, contents), "writing %s" % def_file) # Borland C++ has problems with '/' in paths objects2 = map(os.path.normpath, objects) # split objects in .obj and .res files # Borland C++ needs them at different positions in the command line objects = [startup_obj] resources = [] for file in objects2: (base, ext) = os.path.splitext(os.path.normcase(file)) if ext == '.res': resources.append(file) else: objects.append(file) for l in library_dirs: ld_args.append("/L%s" % os.path.normpath(l)) ld_args.append("/L.") # we sometimes use relative paths # list of object files ld_args.extend(objects) # XXX the command-line syntax for Borland C++ is a bit wonky; # certain filenames are jammed together in one big string, but # comma-delimited. This doesn't mesh too well with the # Unix-centric attitude (with a DOS/Windows quoting hack) of # 'spawn()', so constructing the argument list is a bit # awkward. Note that doing the obvious thing and jamming all # the filenames and commas into one argument would be wrong, # because 'spawn()' would quote any filenames with spaces in # them. Arghghh!. Apparently it works fine as coded... # name of dll/exe file ld_args.extend([',',output_filename]) # no map file and start libraries ld_args.append(',,') for lib in libraries: # see if we find it and if there is a bcpp specific lib # (xxx_bcpp.lib) libfile = self.find_library_file(library_dirs, lib, debug) if libfile is None: ld_args.append(lib) # probably a BCPP internal library -- don't warn else: # full name which prefers bcpp_xxx.lib over xxx.lib ld_args.append(libfile) # some default libraries ld_args.append ('import32') ld_args.append ('cw32mt') # def file for export symbols ld_args.extend([',',def_file]) # add resource files ld_args.append(',') ld_args.extend(resources) if extra_preargs: ld_args[:0] = extra_preargs if extra_postargs: ld_args.extend(extra_postargs) self.mkpath (os.path.dirname (output_filename)) try: self.spawn ([self.linker] + ld_args) except DistutilsExecError, msg: raise LinkError, msg else: log.debug("skipping %s (up-to-date)", output_filename) # link () # -- Miscellaneous methods ----------------------------------------- def find_library_file (self, dirs, lib, debug=0): # List of effective library names to try, in order of preference: # xxx_bcpp.lib is better than xxx.lib # and xxx_d.lib is better than xxx.lib if debug is set # # The "_bcpp" suffix is to handle a Python installation for people # with multiple compilers (primarily Distutils hackers, I suspect # ;-). The idea is they'd have one static library for each # compiler they care about, since (almost?) every Windows compiler # seems to have a different format for static libraries. if debug: dlib = (lib + "_d") try_names = (dlib + "_bcpp", lib + "_bcpp", dlib, lib) else: try_names = (lib + "_bcpp", lib) for dir in dirs: for name in try_names: libfile = os.path.join(dir, self.library_filename(name)) if os.path.exists(libfile): return libfile else: # Oops, didn't find it in *any* of 'dirs' return None # overwrite the one from CCompiler to support rc and res-files def object_filenames (self, source_filenames, strip_dir=0, output_dir=''): if output_dir is None: output_dir = '' obj_names = [] for src_name in source_filenames: # use normcase to make sure '.rc' is really '.rc' and not '.RC' (base, ext) = os.path.splitext (os.path.normcase(src_name)) if ext not in (self.src_extensions + ['.rc','.res']): raise UnknownFileError, \ "unknown file type '%s' (from '%s')" % \ (ext, src_name) if strip_dir: base = os.path.basename (base) if ext == '.res': # these can go unchanged obj_names.append (os.path.join (output_dir, base + ext)) elif ext == '.rc': # these need to be compiled to .res-files obj_names.append (os.path.join (output_dir, base + '.res')) else: obj_names.append (os.path.join (output_dir, base + self.obj_extension)) return obj_names # object_filenames () def preprocess (self, source, output_file=None, macros=None, include_dirs=None, extra_preargs=None, extra_postargs=None): (_, macros, include_dirs) = \ self._fix_compile_args(None, macros, include_dirs) pp_opts = gen_preprocess_options(macros, include_dirs) pp_args = ['cpp32.exe'] + pp_opts if output_file is not None: pp_args.append('-o' + output_file) if extra_preargs: pp_args[:0] = extra_preargs if extra_postargs: pp_args.extend(extra_postargs) pp_args.append(source) # We need to preprocess: either we're being forced to, or the # source file is newer than the target (or the target doesn't # exist). if self.force or output_file is None or newer(source, output_file): if output_file: self.mkpath(os.path.dirname(output_file)) try: self.spawn(pp_args) except DistutilsExecError, msg: print msg raise CompileError, msg # preprocess() --- NEW FILE: ccompiler.py --- """distutils.ccompiler Contains CCompiler, an abstract base class that defines the interface for the Distutils compiler abstraction model.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: ccompiler.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, re from types import * from copy import copy from distutils.errors import * from distutils.spawn import spawn from distutils.file_util import move_file from distutils.dir_util import mkpath from distutils.dep_util import newer_pairwise, newer_group from distutils.sysconfig import python_build from distutils.util import split_quoted, execute [...1231 lines suppressed...] # sometimes you really do have to say "-lfoo -lbar -lfoo" in order to # resolve all symbols. I just hope we never have to say "-lfoo obj.o # -lbar" to get things to work -- that's certainly a possibility, but a # pretty nasty way to arrange your C code. for lib in libraries: (lib_dir, lib_name) = os.path.split (lib) if lib_dir: lib_file = compiler.find_library_file ([lib_dir], lib_name) if lib_file: lib_opts.append (lib_file) else: compiler.warn ("no library file corresponding to " "'%s' found (skipping)" % lib) else: lib_opts.append (compiler.library_option (lib)) return lib_opts # gen_lib_options () --- NEW FILE: cmd.py --- """distutils.cmd Provides the Command class, the base class for the command classes in the distutils.command package. """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: cmd.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string, re from types import * from distutils.errors import * from distutils import util, dir_util, file_util, archive_util, dep_util from distutils import log class Command: """Abstract base class for defining command classes, the "worker bees" of the Distutils. A useful analogy for command classes is to think of them as subroutines with local variables called "options". The options are "declared" in 'initialize_options()' and "defined" (given their final values, aka "finalized") in 'finalize_options()', both of which must be defined by every command class. The distinction between the two is necessary because option values might come from the outside world (command line, config file, ...), and any options dependent on other options must be computed *after* these outside influences have been processed -- hence 'finalize_options()'. The "body" of the subroutine, where it does all its work based on the values of its options, is the 'run()' method, which must also be implemented by every command class. """ # 'sub_commands' formalizes the notion of a "family" of commands, # eg. "install" as the parent with sub-commands "install_lib", # "install_headers", etc. The parent of a family of commands # defines 'sub_commands' as a class attribute; it's a list of # (command_name : string, predicate : unbound_method | string | None) # tuples, where 'predicate' is a method of the parent command that # determines whether the corresponding command is applicable in the # current situation. (Eg. we "install_headers" is only applicable if # we have any C header files to install.) If 'predicate' is None, # that command is always applicable. # # 'sub_commands' is usually defined at the *end* of a class, because # predicates can be unbound methods, so they must already have been # defined. The canonical example is the "install" command. sub_commands = [] # -- Creation/initialization methods ------------------------------- def __init__ (self, dist): """Create and initialize a new Command object. Most importantly, invokes the 'initialize_options()' method, which is the real initializer and depends on the actual command being instantiated. """ # late import because of mutual dependence between these classes from distutils.dist import Distribution if not isinstance(dist, Distribution): raise TypeError, "dist must be a Distribution instance" if self.__class__ is Command: raise RuntimeError, "Command is an abstract class" self.distribution = dist self.initialize_options() # Per-command versions of the global flags, so that the user can # customize Distutils' behaviour command-by-command and let some # commands fall back on the Distribution's behaviour. None means # "not defined, check self.distribution's copy", while 0 or 1 mean # false and true (duh). Note that this means figuring out the real # value of each flag is a touch complicated -- hence "self._dry_run" # will be handled by __getattr__, below. # XXX This needs to be fixed. self._dry_run = None # verbose is largely ignored, but needs to be set for # backwards compatibility (I think)? self.verbose = dist.verbose # Some commands define a 'self.force' option to ignore file # timestamps, but methods defined *here* assume that # 'self.force' exists for all commands. So define it here # just to be safe. self.force = None # The 'help' flag is just used for command-line parsing, so # none of that complicated bureaucracy is needed. self.help = 0 # 'finalized' records whether or not 'finalize_options()' has been # called. 'finalize_options()' itself should not pay attention to # this flag: it is the business of 'ensure_finalized()', which # always calls 'finalize_options()', to respect/update it. self.finalized = 0 # __init__ () # XXX A more explicit way to customize dry_run would be better. def __getattr__ (self, attr): if attr == 'dry_run': myval = getattr(self, "_" + attr) if myval is None: return getattr(self.distribution, attr) else: return myval else: raise AttributeError, attr def ensure_finalized (self): if not self.finalized: self.finalize_options() self.finalized = 1 # Subclasses must define: # initialize_options() # provide default values for all options; may be customized by # setup script, by options from config file(s), or by command-line # options # finalize_options() # decide on the final values for all options; this is called # after all possible intervention from the outside world # (command-line, option file, etc.) has been processed # run() # run the command: do whatever it is we're here to do, # controlled by the command's various option values def initialize_options (self): """Set default values for all the options that this command supports. Note that these defaults may be overridden by other commands, by the setup script, by config files, or by the command-line. Thus, this is not the place to code dependencies between options; generally, 'initialize_options()' implementations are just a bunch of "self.foo = None" assignments. This method must be implemented by all command classes. """ raise RuntimeError, \ "abstract method -- subclass %s must override" % self.__class__ def finalize_options (self): """Set final values for all the options that this command supports. This is always called as late as possible, ie. after any option assignments from the command-line or from other commands have been done. Thus, this is the place to code option dependencies: if 'foo' depends on 'bar', then it is safe to set 'foo' from 'bar' as long as 'foo' still has the same value it was assigned in 'initialize_options()'. This method must be implemented by all command classes. """ raise RuntimeError, \ "abstract method -- subclass %s must override" % self.__class__ def dump_options (self, header=None, indent=""): from distutils.fancy_getopt import longopt_xlate if header is None: header = "command options for '%s':" % self.get_command_name() print indent + header indent = indent + " " for (option, _, _) in self.user_options: option = string.translate(option, longopt_xlate) if option[-1] == "=": option = option[:-1] value = getattr(self, option) print indent + "%s = %s" % (option, value) def run (self): """A command's raison d'etre: carry out the action it exists to perform, controlled by the options initialized in 'initialize_options()', customized by other commands, the setup script, the command-line, and config files, and finalized in 'finalize_options()'. All terminal output and filesystem interaction should be done by 'run()'. This method must be implemented by all command classes. """ raise RuntimeError, \ "abstract method -- subclass %s must override" % self.__class__ def announce (self, msg, level=1): """If the current verbosity level is of greater than or equal to 'level' print 'msg' to stdout. """ log.log(level, msg) def debug_print (self, msg): """Print 'msg' to stdout if the global DEBUG (taken from the DISTUTILS_DEBUG environment variable) flag is true. """ from distutils.debug import DEBUG if DEBUG: print msg sys.stdout.flush() # -- Option validation methods ------------------------------------- # (these are very handy in writing the 'finalize_options()' method) # # NB. the general philosophy here is to ensure that a particular option # value meets certain type and value constraints. If not, we try to # force it into conformance (eg. if we expect a list but have a string, # split the string on comma and/or whitespace). If we can't force the # option into conformance, raise DistutilsOptionError. Thus, command # classes need do nothing more than (eg.) # self.ensure_string_list('foo') # and they can be guaranteed that thereafter, self.foo will be # a list of strings. def _ensure_stringlike (self, option, what, default=None): val = getattr(self, option) if val is None: setattr(self, option, default) return default elif type(val) is not StringType: raise DistutilsOptionError, \ "'%s' must be a %s (got `%s`)" % (option, what, val) return val def ensure_string (self, option, default=None): """Ensure that 'option' is a string; if not defined, set it to 'default'. """ self._ensure_stringlike(option, "string", default) def ensure_string_list (self, option): """Ensure that 'option' is a list of strings. If 'option' is currently a string, we split it either on /,\s*/ or /\s+/, so "foo bar baz", "foo,bar,baz", and "foo, bar baz" all become ["foo", "bar", "baz"]. """ val = getattr(self, option) if val is None: return elif type(val) is StringType: setattr(self, option, re.split(r',\s*|\s+', val)) else: if type(val) is ListType: types = map(type, val) ok = (types == [StringType] * len(val)) else: ok = 0 if not ok: raise DistutilsOptionError, \ "'%s' must be a list of strings (got %r)" % \ (option, val) def _ensure_tested_string (self, option, tester, what, error_fmt, default=None): val = self._ensure_stringlike(option, what, default) if val is not None and not tester(val): raise DistutilsOptionError, \ ("error in '%s' option: " + error_fmt) % (option, val) def ensure_filename (self, option): """Ensure that 'option' is the name of an existing file.""" self._ensure_tested_string(option, os.path.isfile, "filename", "'%s' does not exist or is not a file") def ensure_dirname (self, option): self._ensure_tested_string(option, os.path.isdir, "directory name", "'%s' does not exist or is not a directory") # -- Convenience methods for commands ------------------------------ def get_command_name (self): if hasattr(self, 'command_name'): return self.command_name else: return self.__class__.__name__ def set_undefined_options (self, src_cmd, *option_pairs): """Set the values of any "undefined" options from corresponding option values in some other command object. "Undefined" here means "is None", which is the convention used to indicate that an option has not been changed between 'initialize_options()' and 'finalize_options()'. Usually called from 'finalize_options()' for options that depend on some other command rather than another option of the same command. 'src_cmd' is the other command from which option values will be taken (a command object will be created for it if necessary); the remaining arguments are '(src_option,dst_option)' tuples which mean "take the value of 'src_option' in the 'src_cmd' command object, and copy it to 'dst_option' in the current command object". """ # Option_pairs: list of (src_option, dst_option) tuples src_cmd_obj = self.distribution.get_command_obj(src_cmd) src_cmd_obj.ensure_finalized() for (src_option, dst_option) in option_pairs: if getattr(self, dst_option) is None: setattr(self, dst_option, getattr(src_cmd_obj, src_option)) def get_finalized_command (self, command, create=1): """Wrapper around Distribution's 'get_command_obj()' method: find (create if necessary and 'create' is true) the command object for 'command', call its 'ensure_finalized()' method, and return the finalized command object. """ cmd_obj = self.distribution.get_command_obj(command, create) cmd_obj.ensure_finalized() return cmd_obj # XXX rename to 'get_reinitialized_command()'? (should do the # same in dist.py, if so) def reinitialize_command (self, command, reinit_subcommands=0): return self.distribution.reinitialize_command( command, reinit_subcommands) def run_command (self, command): """Run some other command: uses the 'run_command()' method of Distribution, which creates and finalizes the command object if necessary and then invokes its 'run()' method. """ self.distribution.run_command(command) def get_sub_commands (self): """Determine the sub-commands that are relevant in the current distribution (ie., that need to be run). This is based on the 'sub_commands' class attribute: each tuple in that list may include a method that we call to determine if the subcommand needs to be run for the current distribution. Return a list of command names. """ commands = [] for (cmd_name, method) in self.sub_commands: if method is None or method(self): commands.append(cmd_name) return commands # -- External world manipulation ----------------------------------- def warn (self, msg): sys.stderr.write("warning: %s: %s\n" % (self.get_command_name(), msg)) def execute (self, func, args, msg=None, level=1): util.execute(func, args, msg, dry_run=self.dry_run) def mkpath (self, name, mode=0777): dir_util.mkpath(name, mode, dry_run=self.dry_run) def copy_file (self, infile, outfile, preserve_mode=1, preserve_times=1, link=None, level=1): """Copy a file respecting verbose, dry-run and force flags. (The former two default to whatever is in the Distribution object, and the latter defaults to false for commands that don't define it.)""" return file_util.copy_file( infile, outfile, preserve_mode, preserve_times, not self.force, link, dry_run=self.dry_run) def copy_tree (self, infile, outfile, preserve_mode=1, preserve_times=1, preserve_symlinks=0, level=1): """Copy an entire directory tree respecting verbose, dry-run, and force flags. """ return dir_util.copy_tree( infile, outfile, preserve_mode,preserve_times,preserve_symlinks, not self.force, dry_run=self.dry_run) def move_file (self, src, dst, level=1): """Move a file respectin dry-run flag.""" return file_util.move_file(src, dst, dry_run = self.dry_run) def spawn (self, cmd, search_path=1, level=1): """Spawn an external command respecting dry-run flag.""" from distutils.spawn import spawn spawn(cmd, search_path, dry_run= self.dry_run) def make_archive (self, base_name, format, root_dir=None, base_dir=None): return archive_util.make_archive( base_name, format, root_dir, base_dir, dry_run=self.dry_run) def make_file (self, infiles, outfile, func, args, exec_msg=None, skip_msg=None, level=1): """Special case of 'execute()' for operations that process one or more input files and generate one output file. Works just like 'execute()', except the operation is skipped and a different message printed if 'outfile' already exists and is newer than all files listed in 'infiles'. If the command defined 'self.force', and it is true, then the command is unconditionally run -- does no timestamp checks. """ if exec_msg is None: exec_msg = "generating %s from %s" % \ (outfile, string.join(infiles, ', ')) if skip_msg is None: skip_msg = "skipping %s (inputs unchanged)" % outfile # Allow 'infiles' to be a single string if type(infiles) is StringType: infiles = (infiles,) elif type(infiles) not in (ListType, TupleType): raise TypeError, \ "'infiles' must be a string, or a list or tuple of strings" # If 'outfile' must be regenerated (either because it doesn't # exist, is out-of-date, or the 'force' flag is true) then # perform the action that presumably regenerates it if self.force or dep_util.newer_group (infiles, outfile): self.execute(func, args, exec_msg, level) # Otherwise, print the "skip" message else: log.debug(skip_msg) # make_file () # class Command # XXX 'install_misc' class not currently used -- it was the base class for # both 'install_scripts' and 'install_data', but they outgrew it. It might # still be useful for 'install_headers', though, so I'm keeping it around # for the time being. class install_misc (Command): """Common base class for installing some files in a subdirectory. Currently used by install_data and install_scripts. """ user_options = [('install-dir=', 'd', "directory to install the files to")] def initialize_options (self): self.install_dir = None self.outfiles = [] def _install_dir_from (self, dirname): self.set_undefined_options('install', (dirname, 'install_dir')) def _copy_files (self, filelist): self.outfiles = [] if not filelist: return self.mkpath(self.install_dir) for f in filelist: self.copy_file(f, self.install_dir) self.outfiles.append(os.path.join(self.install_dir, f)) def get_outputs (self): return self.outfiles if __name__ == "__main__": print "ok" --- NEW FILE: core.py --- """distutils.core The only module that needs to be imported to use the Distutils; provides the 'setup' function (which is to be called from the setup script). Also indirectly provides the Distribution and Command classes, although they are really defined in distutils.dist and distutils.cmd. """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: core.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os from types import * from distutils.debug import DEBUG from distutils.errors import * from distutils.util import grok_environment_error # Mainly import these so setup scripts can "from distutils.core import" them. from distutils.dist import Distribution from distutils.cmd import Command from distutils.extension import Extension # This is a barebones help message generated displayed when the user # runs the setup script with no arguments at all. More useful help # is generated with various --help options: global help, list commands, # and per-command help. USAGE = """\ usage: %(script)s [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: %(script)s --help [cmd1 cmd2 ...] or: %(script)s --help-commands or: %(script)s cmd --help """ def gen_usage (script_name): script = os.path.basename(script_name) return USAGE % vars() # Some mild magic to control the behaviour of 'setup()' from 'run_setup()'. _setup_stop_after = None _setup_distribution = None # Legal keyword arguments for the setup() function setup_keywords = ('distclass', 'script_name', 'script_args', 'options', 'name', 'version', 'author', 'author_email', 'maintainer', 'maintainer_email', 'url', 'license', 'description', 'long_description', 'keywords', 'platforms', 'classifiers', 'download_url', 'requires', 'provides', 'obsoletes', ) # Legal keyword arguments for the Extension constructor extension_keywords = ('name', 'sources', 'include_dirs', 'define_macros', 'undef_macros', 'library_dirs', 'libraries', 'runtime_library_dirs', 'extra_objects', 'extra_compile_args', 'extra_link_args', 'swig_opts', 'export_symbols', 'depends', 'language') def setup (**attrs): """The gateway to the Distutils: do everything your setup script needs to do, in a highly flexible and user-driven way. Briefly: create a Distribution instance; find and parse config files; parse the command line; run each Distutils command found there, customized by the options supplied to 'setup()' (as keyword arguments), in config files, and on the command line. The Distribution instance might be an instance of a class supplied via the 'distclass' keyword argument to 'setup'; if no such class is supplied, then the Distribution class (in dist.py) is instantiated. All other arguments to 'setup' (except for 'cmdclass') are used to set attributes of the Distribution instance. The 'cmdclass' argument, if supplied, is a dictionary mapping command names to command classes. Each command encountered on the command line will be turned into a command class, which is in turn instantiated; any class found in 'cmdclass' is used in place of the default, which is (for command 'foo_bar') class 'foo_bar' in module 'distutils.command.foo_bar'. The command class must provide a 'user_options' attribute which is a list of option specifiers for 'distutils.fancy_getopt'. Any command-line options between the current and the next command are used to set attributes of the current command object. When the entire command-line has been successfully parsed, calls the 'run()' method on each command object in turn. This method will be driven entirely by the Distribution object (which each command object has a reference to, thanks to its constructor), and the command-specific options that became attributes of each command object. """ global _setup_stop_after, _setup_distribution # Determine the distribution class -- either caller-supplied or # our Distribution (see below). klass = attrs.get('distclass') if klass: del attrs['distclass'] else: klass = Distribution if not attrs.has_key('script_name'): attrs['script_name'] = os.path.basename(sys.argv[0]) if not attrs.has_key('script_args'): attrs['script_args'] = sys.argv[1:] # Create the Distribution instance, using the remaining arguments # (ie. everything except distclass) to initialize it try: _setup_distribution = dist = klass(attrs) except DistutilsSetupError, msg: if attrs.has_key('name'): raise SystemExit, "error in %s setup command: %s" % \ (attrs['name'], msg) else: raise SystemExit, "error in setup command: %s" % msg if _setup_stop_after == "init": return dist # Find and parse the config file(s): they will override options from # the setup script, but be overridden by the command line. dist.parse_config_files() if DEBUG: print "options (after parsing config files):" dist.dump_option_dicts() if _setup_stop_after == "config": return dist # Parse the command line; any command-line errors are the end user's # fault, so turn them into SystemExit to suppress tracebacks. try: ok = dist.parse_command_line() except DistutilsArgError, msg: raise SystemExit, gen_usage(dist.script_name) + "\nerror: %s" % msg if DEBUG: print "options (after parsing command line):" dist.dump_option_dicts() if _setup_stop_after == "commandline": return dist # And finally, run all the commands found on the command line. if ok: try: dist.run_commands() except KeyboardInterrupt: raise SystemExit, "interrupted" except (IOError, os.error), exc: error = grok_environment_error(exc) if DEBUG: sys.stderr.write(error + "\n") raise else: raise SystemExit, error except (DistutilsError, CCompilerError), msg: if DEBUG: raise else: raise SystemExit, "error: " + str(msg) return dist # setup () def run_setup (script_name, script_args=None, stop_after="run"): """Run a setup script in a somewhat controlled environment, and return the Distribution instance that drives things. This is useful if you need to find out the distribution meta-data (passed as keyword args from 'script' to 'setup()', or the contents of the config files or command-line. 'script_name' is a file that will be run with 'execfile()'; 'sys.argv[0]' will be replaced with 'script' for the duration of the call. 'script_args' is a list of strings; if supplied, 'sys.argv[1:]' will be replaced by 'script_args' for the duration of the call. 'stop_after' tells 'setup()' when to stop processing; possible values: init stop after the Distribution instance has been created and populated with the keyword arguments to 'setup()' config stop after config files have been parsed (and their data stored in the Distribution instance) commandline stop after the command-line ('sys.argv[1:]' or 'script_args') have been parsed (and the data stored in the Distribution) run [default] stop after all commands have been run (the same as if 'setup()' had been called in the usual way Returns the Distribution instance, which provides all information used to drive the Distutils. """ if stop_after not in ('init', 'config', 'commandline', 'run'): raise ValueError, "invalid value for 'stop_after': %r" % (stop_after,) global _setup_stop_after, _setup_distribution _setup_stop_after = stop_after save_argv = sys.argv g = {} l = {} try: try: sys.argv[0] = script_name if script_args is not None: sys.argv[1:] = script_args execfile(script_name, g, l) finally: sys.argv = save_argv _setup_stop_after = None except SystemExit: # Hmm, should we do something if exiting with a non-zero code # (ie. error)? pass except: raise if _setup_distribution is None: raise RuntimeError, \ ("'distutils.core.setup()' was never called -- " "perhaps '%s' is not a Distutils setup script?") % \ script_name # I wonder if the setup script's namespace -- g and l -- would be of # any interest to callers? #print "_setup_distribution:", _setup_distribution return _setup_distribution # run_setup () --- NEW FILE: cygwinccompiler.py --- """distutils.cygwinccompiler Provides the CygwinCCompiler class, a subclass of UnixCCompiler that handles the Cygwin port of the GNU C compiler to Windows. It also contains the Mingw32CCompiler class which handles the mingw32 port of GCC (same as cygwin in no-cygwin mode). """ # problems: # # * if you use a msvc compiled python version (1.5.2) # 1. you have to insert a __GNUC__ section in its config.h # 2. you have to generate a import library for its dll # - create a def-file for python??.dll # - create a import library using # dlltool --dllname python15.dll --def python15.def \ # --output-lib libpython15.a # # see also http://starship.python.net/crew/kernr/mingw32/Notes.html # # * We put export_symbols in a def-file, and don't use # --export-all-symbols because it doesn't worked reliable in some # tested configurations. And because other windows compilers also # need their symbols specified this no serious problem. # # tested configurations: # # * cygwin gcc 2.91.57/ld 2.9.4/dllwrap 0.2.4 works # (after patching python's config.h and for C++ some other include files) # see also http://starship.python.net/crew/kernr/mingw32/Notes.html # * mingw32 gcc 2.95.2/ld 2.9.4/dllwrap 0.2.4 works # (ld doesn't support -shared, so we use dllwrap) # * cygwin gcc 2.95.2/ld 2.10.90/dllwrap 2.10.90 works now # - its dllwrap doesn't work, there is a bug in binutils 2.10.90 # see also http://sources.redhat.com/ml/cygwin/2000-06/msg01274.html # - using gcc -mdll instead dllwrap doesn't work without -static because # it tries to link against dlls instead their import libraries. (If # it finds the dll first.) # By specifying -static we force ld to link against the import libraries, # this is windows standard and there are normally not the necessary symbols # in the dlls. # *** only the version of June 2000 shows these problems # * cygwin gcc 3.2/ld 2.13.90 works # (ld supports -shared) # * mingw gcc 3.2/ld 2.13 works # (ld supports -shared) # This module should be kept compatible with Python 2.1. __revision__ = "$Id: cygwinccompiler.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os,sys,copy from distutils.ccompiler import gen_preprocess_options, gen_lib_options from distutils.unixccompiler import UnixCCompiler from distutils.file_util import write_file from distutils.errors import DistutilsExecError, CompileError, UnknownFileError from distutils import log class CygwinCCompiler (UnixCCompiler): compiler_type = 'cygwin' obj_extension = ".o" static_lib_extension = ".a" shared_lib_extension = ".dll" static_lib_format = "lib%s%s" shared_lib_format = "%s%s" exe_extension = ".exe" def __init__ (self, verbose=0, dry_run=0, force=0): UnixCCompiler.__init__ (self, verbose, dry_run, force) (status, details) = check_config_h() self.debug_print("Python's GCC status: %s (details: %s)" % (status, details)) if status is not CONFIG_H_OK: self.warn( "Python's pyconfig.h doesn't seem to support your compiler. " "Reason: %s. " "Compiling may fail because of undefined preprocessor macros." % details) self.gcc_version, self.ld_version, self.dllwrap_version = \ get_versions() self.debug_print(self.compiler_type + ": gcc %s, ld %s, dllwrap %s\n" % (self.gcc_version, self.ld_version, self.dllwrap_version) ) # ld_version >= "2.10.90" and < "2.13" should also be able to use # gcc -mdll instead of dllwrap # Older dllwraps had own version numbers, newer ones use the # same as the rest of binutils ( also ld ) # dllwrap 2.10.90 is buggy if self.ld_version >= "2.10.90": self.linker_dll = "gcc" else: self.linker_dll = "dllwrap" # ld_version >= "2.13" support -shared so use it instead of # -mdll -static if self.ld_version >= "2.13": shared_option = "-shared" else: shared_option = "-mdll -static" # Hard-code GCC because that's what this is all about. # XXX optimization, warnings etc. should be customizable. self.set_executables(compiler='gcc -mcygwin -O -Wall', compiler_so='gcc -mcygwin -mdll -O -Wall', compiler_cxx='g++ -mcygwin -O -Wall', linker_exe='gcc -mcygwin', linker_so=('%s -mcygwin %s' % (self.linker_dll, shared_option))) # cygwin and mingw32 need different sets of libraries if self.gcc_version == "2.91.57": # cygwin shouldn't need msvcrt, but without the dlls will crash # (gcc version 2.91.57) -- perhaps something about initialization self.dll_libraries=["msvcrt"] self.warn( "Consider upgrading to a newer version of gcc") else: self.dll_libraries=[] # Include the appropriate MSVC runtime library if Python was built # with MSVC 7.0 or 7.1. msc_pos = sys.version.find('MSC v.') if msc_pos != -1: msc_ver = sys.version[msc_pos+6:msc_pos+10] if msc_ver == '1300': # MSVC 7.0 self.dll_libraries = ['msvcr70'] elif msc_ver == '1310': # MSVC 7.1 self.dll_libraries = ['msvcr71'] # __init__ () def _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts): if ext == '.rc' or ext == '.res': # gcc needs '.res' and '.rc' compiled to object files !!! try: self.spawn(["windres", "-i", src, "-o", obj]) except DistutilsExecError, msg: raise CompileError, msg else: # for other files use the C-compiler try: self.spawn(self.compiler_so + cc_args + [src, '-o', obj] + extra_postargs) except DistutilsExecError, msg: raise CompileError, msg def link (self, target_desc, objects, output_filename, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None): # use separate copies, so we can modify the lists extra_preargs = copy.copy(extra_preargs or []) libraries = copy.copy(libraries or []) objects = copy.copy(objects or []) # Additional libraries libraries.extend(self.dll_libraries) # handle export symbols by creating a def-file # with executables this only works with gcc/ld as linker if ((export_symbols is not None) and (target_desc != self.EXECUTABLE or self.linker_dll == "gcc")): # (The linker doesn't do anything if output is up-to-date. # So it would probably better to check if we really need this, # but for this we had to insert some unchanged parts of # UnixCCompiler, and this is not what we want.) # we want to put some files in the same directory as the # object files are, build_temp doesn't help much # where are the object files temp_dir = os.path.dirname(objects[0]) # name of dll to give the helper files the same base name (dll_name, dll_extension) = os.path.splitext( os.path.basename(output_filename)) # generate the filenames for these files def_file = os.path.join(temp_dir, dll_name + ".def") lib_file = os.path.join(temp_dir, 'lib' + dll_name + ".a") # Generate .def file contents = [ "LIBRARY %s" % os.path.basename(output_filename), "EXPORTS"] for sym in export_symbols: contents.append(sym) self.execute(write_file, (def_file, contents), "writing %s" % def_file) # next add options for def-file and to creating import libraries # dllwrap uses different options than gcc/ld if self.linker_dll == "dllwrap": extra_preargs.extend(["--output-lib", lib_file]) # for dllwrap we have to use a special option extra_preargs.extend(["--def", def_file]) # we use gcc/ld here and can be sure ld is >= 2.9.10 else: # doesn't work: bfd_close build\...\libfoo.a: Invalid operation #extra_preargs.extend(["-Wl,--out-implib,%s" % lib_file]) # for gcc/ld the def-file is specified as any object files objects.append(def_file) #end: if ((export_symbols is not None) and # (target_desc != self.EXECUTABLE or self.linker_dll == "gcc")): # who wants symbols and a many times larger output file # should explicitly switch the debug mode on # otherwise we let dllwrap/ld strip the output file # (On my machine: 10KB < stripped_file < ??100KB # unstripped_file = stripped_file + XXX KB # ( XXX=254 for a typical python extension)) if not debug: extra_preargs.append("-s") UnixCCompiler.link(self, target_desc, objects, output_filename, output_dir, libraries, library_dirs, runtime_library_dirs, None, # export_symbols, we do this in our def-file debug, extra_preargs, extra_postargs, build_temp, target_lang) # link () # -- Miscellaneous methods ----------------------------------------- # overwrite the one from CCompiler to support rc and res-files def object_filenames (self, source_filenames, strip_dir=0, output_dir=''): if output_dir is None: output_dir = '' obj_names = [] for src_name in source_filenames: # use normcase to make sure '.rc' is really '.rc' and not '.RC' (base, ext) = os.path.splitext (os.path.normcase(src_name)) if ext not in (self.src_extensions + ['.rc','.res']): raise UnknownFileError, \ "unknown file type '%s' (from '%s')" % \ (ext, src_name) if strip_dir: base = os.path.basename (base) if ext == '.res' or ext == '.rc': # these need to be compiled to object files obj_names.append (os.path.join (output_dir, base + ext + self.obj_extension)) else: obj_names.append (os.path.join (output_dir, base + self.obj_extension)) return obj_names # object_filenames () # class CygwinCCompiler # the same as cygwin plus some additional parameters class Mingw32CCompiler (CygwinCCompiler): compiler_type = 'mingw32' def __init__ (self, verbose=0, dry_run=0, force=0): CygwinCCompiler.__init__ (self, verbose, dry_run, force) # ld_version >= "2.13" support -shared so use it instead of # -mdll -static if self.ld_version >= "2.13": shared_option = "-shared" else: shared_option = "-mdll -static" # A real mingw32 doesn't need to specify a different entry point, # but cygwin 2.91.57 in no-cygwin-mode needs it. if self.gcc_version <= "2.91.57": entry_point = '--entry _DllMain at 12' else: entry_point = '' self.set_executables(compiler='gcc -mno-cygwin -O -Wall', compiler_so='gcc -mno-cygwin -mdll -O -Wall', compiler_cxx='g++ -mno-cygwin -O -Wall', linker_exe='gcc -mno-cygwin', linker_so='%s -mno-cygwin %s %s' % (self.linker_dll, shared_option, entry_point)) # Maybe we should also append -mthreads, but then the finished # dlls need another dll (mingwm10.dll see Mingw32 docs) # (-mthreads: Support thread-safe exception handling on `Mingw32') # no additional libraries needed self.dll_libraries=[] # Include the appropriate MSVC runtime library if Python was built # with MSVC 7.0 or 7.1. msc_pos = sys.version.find('MSC v.') if msc_pos != -1: msc_ver = sys.version[msc_pos+6:msc_pos+10] if msc_ver == '1300': # MSVC 7.0 self.dll_libraries = ['msvcr70'] elif msc_ver == '1310': # MSVC 7.1 self.dll_libraries = ['msvcr71'] # __init__ () # class Mingw32CCompiler # Because these compilers aren't configured in Python's pyconfig.h file by # default, we should at least warn the user if he is using a unmodified # version. CONFIG_H_OK = "ok" CONFIG_H_NOTOK = "not ok" CONFIG_H_UNCERTAIN = "uncertain" def check_config_h(): """Check if the current Python installation (specifically, pyconfig.h) appears amenable to building extensions with GCC. Returns a tuple (status, details), where 'status' is one of the following constants: CONFIG_H_OK all is well, go ahead and compile CONFIG_H_NOTOK doesn't look good CONFIG_H_UNCERTAIN not sure -- unable to read pyconfig.h 'details' is a human-readable string explaining the situation. Note there are two ways to conclude "OK": either 'sys.version' contains the string "GCC" (implying that this Python was built with GCC), or the installed "pyconfig.h" contains the string "__GNUC__". """ # XXX since this function also checks sys.version, it's not strictly a # "pyconfig.h" check -- should probably be renamed... from distutils import sysconfig import string # if sys.version contains GCC then python was compiled with # GCC, and the pyconfig.h file should be OK if string.find(sys.version,"GCC") >= 0: return (CONFIG_H_OK, "sys.version mentions 'GCC'") fn = sysconfig.get_config_h_filename() try: # It would probably better to read single lines to search. # But we do this only once, and it is fast enough f = open(fn) s = f.read() f.close() except IOError, exc: # if we can't read this file, we cannot say it is wrong # the compiler will complain later about this file as missing return (CONFIG_H_UNCERTAIN, "couldn't read '%s': %s" % (fn, exc.strerror)) else: # "pyconfig.h" contains an "#ifdef __GNUC__" or something similar if string.find(s,"__GNUC__") >= 0: return (CONFIG_H_OK, "'%s' mentions '__GNUC__'" % fn) else: return (CONFIG_H_NOTOK, "'%s' does not mention '__GNUC__'" % fn) def get_versions(): """ Try to find out the versions of gcc, ld and dllwrap. If not possible it returns None for it. """ from distutils.version import StrictVersion from distutils.spawn import find_executable import re gcc_exe = find_executable('gcc') if gcc_exe: out = os.popen(gcc_exe + ' -dumpversion','r') out_string = out.read() out.close() result = re.search('(\d+\.\d+(\.\d+)*)',out_string) if result: gcc_version = StrictVersion(result.group(1)) else: gcc_version = None else: gcc_version = None ld_exe = find_executable('ld') if ld_exe: out = os.popen(ld_exe + ' -v','r') out_string = out.read() out.close() result = re.search('(\d+\.\d+(\.\d+)*)',out_string) if result: ld_version = StrictVersion(result.group(1)) else: ld_version = None else: ld_version = None dllwrap_exe = find_executable('dllwrap') if dllwrap_exe: out = os.popen(dllwrap_exe + ' --version','r') out_string = out.read() out.close() result = re.search(' (\d+\.\d+(\.\d+)*)',out_string) if result: dllwrap_version = StrictVersion(result.group(1)) else: dllwrap_version = None else: dllwrap_version = None return (gcc_version, ld_version, dllwrap_version) --- NEW FILE: debug.py --- import os # This module should be kept compatible with Python 2.1. __revision__ = "$Id: debug.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" # If DISTUTILS_DEBUG is anything other than the empty string, we run in # debug mode. DEBUG = os.environ.get('DISTUTILS_DEBUG') --- NEW FILE: dep_util.py --- """distutils.dep_util Utility functions for simple, timestamp-based dependency of files and groups of files; also, function based entirely on such timestamp dependency analysis.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: dep_util.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os from distutils.errors import DistutilsFileError def newer (source, target, dry_run=0): """Return true if 'source' exists and is more recently modified than 'target', or if 'source' exists and 'target' doesn't. Return false if both exist and 'target' is the same age or younger than 'source'. Raise DistutilsFileError if 'source' does not exist. """ if not os.path.exists(source): if dry_run: return 1 raise DistutilsFileError, "file '%s' does not exist" % source if not os.path.exists(target): return 1 from stat import ST_MTIME mtime1 = os.stat(source)[ST_MTIME] mtime2 = os.stat(target)[ST_MTIME] return mtime1 > mtime2 # newer () def newer_pairwise (sources, targets): """Walk two filename lists in parallel, testing if each source is newer than its corresponding target. Return a pair of lists (sources, targets) where source is newer than target, according to the semantics of 'newer()'. """ if len(sources) != len(targets): raise ValueError, "'sources' and 'targets' must be same length" # build a pair of lists (sources, targets) where source is newer n_sources = [] n_targets = [] for i in range(len(sources)): if newer(sources[i], targets[i]): n_sources.append(sources[i]) n_targets.append(targets[i]) return (n_sources, n_targets) # newer_pairwise () def newer_group (sources, target, missing='error'): """Return true if 'target' is out-of-date with respect to any file listed in 'sources'. In other words, if 'target' exists and is newer than every file in 'sources', return false; otherwise return true. 'missing' controls what we do when a source file is missing; the default ("error") is to blow up with an OSError from inside 'stat()'; if it is "ignore", we silently drop any missing source files; if it is "newer", any missing source files make us assume that 'target' is out-of-date (this is handy in "dry-run" mode: it'll make you pretend to carry out commands that wouldn't work because inputs are missing, but that doesn't matter because you're not actually going to run the commands). """ # If the target doesn't even exist, then it's definitely out-of-date. if not os.path.exists(target): return 1 # Otherwise we have to find out the hard way: if *any* source file # is more recent than 'target', then 'target' is out-of-date and # we can immediately return true. If we fall through to the end # of the loop, then 'target' is up-to-date and we return false. from stat import ST_MTIME target_mtime = os.stat(target)[ST_MTIME] for source in sources: if not os.path.exists(source): if missing == 'error': # blow up when we stat() the file pass elif missing == 'ignore': # missing source dropped from continue # target's dependency list elif missing == 'newer': # missing source means target is return 1 # out-of-date source_mtime = os.stat(source)[ST_MTIME] if source_mtime > target_mtime: return 1 else: return 0 # newer_group () --- NEW FILE: dir_util.py --- """distutils.dir_util Utility functions for manipulating directories and directory trees.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: dir_util.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os, sys from types import * from distutils.errors import DistutilsFileError, DistutilsInternalError from distutils import log # cache for by mkpath() -- in addition to cheapening redundant calls, # eliminates redundant "creating /foo/bar/baz" messages in dry-run mode _path_created = {} # I don't use os.makedirs because a) it's new to Python 1.5.2, and # b) it blows up if the directory already exists (I want to silently # succeed in that case). def mkpath (name, mode=0777, verbose=0, dry_run=0): """Create a directory and any missing ancestor directories. If the directory already exists (or if 'name' is the empty string, which means the current directory, which of course exists), then do nothing. Raise DistutilsFileError if unable to create some directory along the way (eg. some sub-path exists, but is a file rather than a directory). If 'verbose' is true, print a one-line summary of each mkdir to stdout. Return the list of directories actually created.""" global _path_created # Detect a common bug -- name is None if type(name) is not StringType: raise DistutilsInternalError, \ "mkpath: 'name' must be a string (got %r)" % (name,) # XXX what's the better way to handle verbosity? print as we create # each directory in the path (the current behaviour), or only announce # the creation of the whole path? (quite easy to do the latter since # we're not using a recursive algorithm) name = os.path.normpath(name) created_dirs = [] if os.path.isdir(name) or name == '': return created_dirs if _path_created.get(os.path.abspath(name)): return created_dirs (head, tail) = os.path.split(name) tails = [tail] # stack of lone dirs to create while head and tail and not os.path.isdir(head): #print "splitting '%s': " % head, (head, tail) = os.path.split(head) #print "to ('%s','%s')" % (head, tail) tails.insert(0, tail) # push next higher dir onto stack #print "stack of tails:", tails # now 'head' contains the deepest directory that already exists # (that is, the child of 'head' in 'name' is the highest directory # that does *not* exist) for d in tails: #print "head = %s, d = %s: " % (head, d), head = os.path.join(head, d) abs_head = os.path.abspath(head) if _path_created.get(abs_head): continue log.info("creating %s", head) if not dry_run: try: os.mkdir(head) created_dirs.append(head) except OSError, exc: raise DistutilsFileError, \ "could not create '%s': %s" % (head, exc[-1]) _path_created[abs_head] = 1 return created_dirs # mkpath () def create_tree (base_dir, files, mode=0777, verbose=0, dry_run=0): """Create all the empty directories under 'base_dir' needed to put 'files' there. 'base_dir' is just the a name of a directory which doesn't necessarily exist yet; 'files' is a list of filenames to be interpreted relative to 'base_dir'. 'base_dir' + the directory portion of every file in 'files' will be created if it doesn't already exist. 'mode', 'verbose' and 'dry_run' flags are as for 'mkpath()'.""" # First get the list of directories to create need_dir = {} for file in files: need_dir[os.path.join(base_dir, os.path.dirname(file))] = 1 need_dirs = need_dir.keys() need_dirs.sort() # Now create them for dir in need_dirs: mkpath(dir, mode, dry_run=dry_run) # create_tree () def copy_tree (src, dst, preserve_mode=1, preserve_times=1, preserve_symlinks=0, update=0, verbose=0, dry_run=0): """Copy an entire directory tree 'src' to a new location 'dst'. Both 'src' and 'dst' must be directory names. If 'src' is not a directory, raise DistutilsFileError. If 'dst' does not exist, it is created with 'mkpath()'. The end result of the copy is that every file in 'src' is copied to 'dst', and directories under 'src' are recursively copied to 'dst'. Return the list of files that were copied or might have been copied, using their output name. The return value is unaffected by 'update' or 'dry_run': it is simply the list of all files under 'src', with the names changed to be under 'dst'. 'preserve_mode' and 'preserve_times' are the same as for 'copy_file'; note that they only apply to regular files, not to directories. If 'preserve_symlinks' is true, symlinks will be copied as symlinks (on platforms that support them!); otherwise (the default), the destination of the symlink will be copied. 'update' and 'verbose' are the same as for 'copy_file'.""" from distutils.file_util import copy_file if not dry_run and not os.path.isdir(src): raise DistutilsFileError, \ "cannot copy tree '%s': not a directory" % src try: names = os.listdir(src) except os.error, (errno, errstr): if dry_run: names = [] else: raise DistutilsFileError, \ "error listing files in '%s': %s" % (src, errstr) if not dry_run: mkpath(dst) outputs = [] for n in names: src_name = os.path.join(src, n) dst_name = os.path.join(dst, n) if preserve_symlinks and os.path.islink(src_name): link_dest = os.readlink(src_name) log.info("linking %s -> %s", dst_name, link_dest) if not dry_run: os.symlink(link_dest, dst_name) outputs.append(dst_name) elif os.path.isdir(src_name): outputs.extend( copy_tree(src_name, dst_name, preserve_mode, preserve_times, preserve_symlinks, update, dry_run=dry_run)) else: copy_file(src_name, dst_name, preserve_mode, preserve_times, update, dry_run=dry_run) outputs.append(dst_name) return outputs # copy_tree () # Helper for remove_tree() def _build_cmdtuple(path, cmdtuples): for f in os.listdir(path): real_f = os.path.join(path,f) if os.path.isdir(real_f) and not os.path.islink(real_f): _build_cmdtuple(real_f, cmdtuples) else: cmdtuples.append((os.remove, real_f)) cmdtuples.append((os.rmdir, path)) def remove_tree (directory, verbose=0, dry_run=0): """Recursively remove an entire directory tree. Any errors are ignored (apart from being reported to stdout if 'verbose' is true). """ from distutils.util import grok_environment_error global _path_created log.info("removing '%s' (and everything under it)", directory) if dry_run: return cmdtuples = [] _build_cmdtuple(directory, cmdtuples) for cmd in cmdtuples: try: apply(cmd[0], (cmd[1],)) # remove dir from cache if it's already there abspath = os.path.abspath(cmd[1]) if _path_created.has_key(abspath): del _path_created[abspath] except (IOError, OSError), exc: log.warn(grok_environment_error( exc, "error removing %s: " % directory)) def ensure_relative (path): """Take the full path 'path', and make it a relative path so it can be the second argument to os.path.join(). """ drive, path = os.path.splitdrive(path) if sys.platform == 'mac': return os.sep + path else: if path[0:1] == os.sep: path = drive + path[1:] return path --- NEW FILE: dist.py --- """distutils.dist Provides the Distribution class, which represents the module distribution being built/installed/distributed. """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: dist.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string, re from types import * from copy import copy try: import warnings except ImportError: warnings = None [...1183 lines suppressed...] for v in value: distutils.versionpredicate.VersionPredicate(v) self.obsoletes = value # class DistributionMetadata def fix_help_options (options): """Convert a 4-tuple 'help_options' list as found in various command classes to the 3-tuple form required by FancyGetopt. """ new_options = [] for help_tuple in options: new_options.append(help_tuple[0:3]) return new_options if __name__ == "__main__": dist = Distribution() print "ok" --- NEW FILE: emxccompiler.py --- """distutils.emxccompiler Provides the EMXCCompiler class, a subclass of UnixCCompiler that handles the EMX port of the GNU C compiler to OS/2. """ # issues: # # * OS/2 insists that DLLs can have names no longer than 8 characters # We put export_symbols in a def-file, as though the DLL can have # an arbitrary length name, but truncate the output filename. # # * only use OMF objects and use LINK386 as the linker (-Zomf) # # * always build for multithreading (-Zmt) as the accompanying OS/2 port # of Python is only distributed with threads enabled. # # tested configurations: # # * EMX gcc 2.81/EMX 0.9d fix03 __revision__ = "$Id: emxccompiler.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os,sys,copy from distutils.ccompiler import gen_preprocess_options, gen_lib_options from distutils.unixccompiler import UnixCCompiler from distutils.file_util import write_file from distutils.errors import DistutilsExecError, CompileError, UnknownFileError from distutils import log class EMXCCompiler (UnixCCompiler): compiler_type = 'emx' obj_extension = ".obj" static_lib_extension = ".lib" shared_lib_extension = ".dll" static_lib_format = "%s%s" shared_lib_format = "%s%s" res_extension = ".res" # compiled resource file exe_extension = ".exe" def __init__ (self, verbose=0, dry_run=0, force=0): UnixCCompiler.__init__ (self, verbose, dry_run, force) (status, details) = check_config_h() self.debug_print("Python's GCC status: %s (details: %s)" % (status, details)) if status is not CONFIG_H_OK: self.warn( "Python's pyconfig.h doesn't seem to support your compiler. " + ("Reason: %s." % details) + "Compiling may fail because of undefined preprocessor macros.") (self.gcc_version, self.ld_version) = \ get_versions() self.debug_print(self.compiler_type + ": gcc %s, ld %s\n" % (self.gcc_version, self.ld_version) ) # Hard-code GCC because that's what this is all about. # XXX optimization, warnings etc. should be customizable. self.set_executables(compiler='gcc -Zomf -Zmt -O3 -fomit-frame-pointer -mprobe -Wall', compiler_so='gcc -Zomf -Zmt -O3 -fomit-frame-pointer -mprobe -Wall', linker_exe='gcc -Zomf -Zmt -Zcrtdll', linker_so='gcc -Zomf -Zmt -Zcrtdll -Zdll') # want the gcc library statically linked (so that we don't have # to distribute a version dependent on the compiler we have) self.dll_libraries=["gcc"] # __init__ () def _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts): if ext == '.rc': # gcc requires '.rc' compiled to binary ('.res') files !!! try: self.spawn(["rc", "-r", src]) except DistutilsExecError, msg: raise CompileError, msg else: # for other files use the C-compiler try: self.spawn(self.compiler_so + cc_args + [src, '-o', obj] + extra_postargs) except DistutilsExecError, msg: raise CompileError, msg def link (self, target_desc, objects, output_filename, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None): # use separate copies, so we can modify the lists extra_preargs = copy.copy(extra_preargs or []) libraries = copy.copy(libraries or []) objects = copy.copy(objects or []) # Additional libraries libraries.extend(self.dll_libraries) # handle export symbols by creating a def-file # with executables this only works with gcc/ld as linker if ((export_symbols is not None) and (target_desc != self.EXECUTABLE)): # (The linker doesn't do anything if output is up-to-date. # So it would probably better to check if we really need this, # but for this we had to insert some unchanged parts of # UnixCCompiler, and this is not what we want.) # we want to put some files in the same directory as the # object files are, build_temp doesn't help much # where are the object files temp_dir = os.path.dirname(objects[0]) # name of dll to give the helper files the same base name (dll_name, dll_extension) = os.path.splitext( os.path.basename(output_filename)) # generate the filenames for these files def_file = os.path.join(temp_dir, dll_name + ".def") # Generate .def file contents = [ "LIBRARY %s INITINSTANCE TERMINSTANCE" % \ os.path.splitext(os.path.basename(output_filename))[0], "DATA MULTIPLE NONSHARED", "EXPORTS"] for sym in export_symbols: contents.append(' "%s"' % sym) self.execute(write_file, (def_file, contents), "writing %s" % def_file) # next add options for def-file and to creating import libraries # for gcc/ld the def-file is specified as any other object files objects.append(def_file) #end: if ((export_symbols is not None) and # (target_desc != self.EXECUTABLE or self.linker_dll == "gcc")): # who wants symbols and a many times larger output file # should explicitly switch the debug mode on # otherwise we let dllwrap/ld strip the output file # (On my machine: 10KB < stripped_file < ??100KB # unstripped_file = stripped_file + XXX KB # ( XXX=254 for a typical python extension)) if not debug: extra_preargs.append("-s") UnixCCompiler.link(self, target_desc, objects, output_filename, output_dir, libraries, library_dirs, runtime_library_dirs, None, # export_symbols, we do this in our def-file debug, extra_preargs, extra_postargs, build_temp, target_lang) # link () # -- Miscellaneous methods ----------------------------------------- # override the object_filenames method from CCompiler to # support rc and res-files def object_filenames (self, source_filenames, strip_dir=0, output_dir=''): if output_dir is None: output_dir = '' obj_names = [] for src_name in source_filenames: # use normcase to make sure '.rc' is really '.rc' and not '.RC' (base, ext) = os.path.splitext (os.path.normcase(src_name)) if ext not in (self.src_extensions + ['.rc']): raise UnknownFileError, \ "unknown file type '%s' (from '%s')" % \ (ext, src_name) if strip_dir: base = os.path.basename (base) if ext == '.rc': # these need to be compiled to object files obj_names.append (os.path.join (output_dir, base + self.res_extension)) else: obj_names.append (os.path.join (output_dir, base + self.obj_extension)) return obj_names # object_filenames () # override the find_library_file method from UnixCCompiler # to deal with file naming/searching differences def find_library_file(self, dirs, lib, debug=0): shortlib = '%s.lib' % lib longlib = 'lib%s.lib' % lib # this form very rare # get EMX's default library directory search path try: emx_dirs = os.environ['LIBRARY_PATH'].split(';') except KeyError: emx_dirs = [] for dir in dirs + emx_dirs: shortlibp = os.path.join(dir, shortlib) longlibp = os.path.join(dir, longlib) if os.path.exists(shortlibp): return shortlibp elif os.path.exists(longlibp): return longlibp # Oops, didn't find it in *any* of 'dirs' return None # class EMXCCompiler # Because these compilers aren't configured in Python's pyconfig.h file by # default, we should at least warn the user if he is using a unmodified # version. CONFIG_H_OK = "ok" CONFIG_H_NOTOK = "not ok" CONFIG_H_UNCERTAIN = "uncertain" def check_config_h(): """Check if the current Python installation (specifically, pyconfig.h) appears amenable to building extensions with GCC. Returns a tuple (status, details), where 'status' is one of the following constants: CONFIG_H_OK all is well, go ahead and compile CONFIG_H_NOTOK doesn't look good CONFIG_H_UNCERTAIN not sure -- unable to read pyconfig.h 'details' is a human-readable string explaining the situation. Note there are two ways to conclude "OK": either 'sys.version' contains the string "GCC" (implying that this Python was built with GCC), or the installed "pyconfig.h" contains the string "__GNUC__". """ # XXX since this function also checks sys.version, it's not strictly a # "pyconfig.h" check -- should probably be renamed... from distutils import sysconfig import string # if sys.version contains GCC then python was compiled with # GCC, and the pyconfig.h file should be OK if string.find(sys.version,"GCC") >= 0: return (CONFIG_H_OK, "sys.version mentions 'GCC'") fn = sysconfig.get_config_h_filename() try: # It would probably better to read single lines to search. # But we do this only once, and it is fast enough f = open(fn) s = f.read() f.close() except IOError, exc: # if we can't read this file, we cannot say it is wrong # the compiler will complain later about this file as missing return (CONFIG_H_UNCERTAIN, "couldn't read '%s': %s" % (fn, exc.strerror)) else: # "pyconfig.h" contains an "#ifdef __GNUC__" or something similar if string.find(s,"__GNUC__") >= 0: return (CONFIG_H_OK, "'%s' mentions '__GNUC__'" % fn) else: return (CONFIG_H_NOTOK, "'%s' does not mention '__GNUC__'" % fn) def get_versions(): """ Try to find out the versions of gcc and ld. If not possible it returns None for it. """ from distutils.version import StrictVersion from distutils.spawn import find_executable import re gcc_exe = find_executable('gcc') if gcc_exe: out = os.popen(gcc_exe + ' -dumpversion','r') out_string = out.read() out.close() result = re.search('(\d+\.\d+\.\d+)',out_string) if result: gcc_version = StrictVersion(result.group(1)) else: gcc_version = None else: gcc_version = None # EMX ld has no way of reporting version number, and we use GCC # anyway - so we can link OMF DLLs ld_version = None return (gcc_version, ld_version) --- NEW FILE: errors.py --- """distutils.errors Provides exceptions used by the Distutils modules. Note that Distutils modules may raise standard exceptions; in particular, SystemExit is usually raised for errors that are obviously the end-user's fault (eg. bad command-line arguments). This module is safe to use in "from ... import *" mode; it only exports symbols whose names start with "Distutils" and end with "Error".""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: errors.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" class DistutilsError (Exception): """The root of all Distutils evil.""" pass class DistutilsModuleError (DistutilsError): """Unable to load an expected module, or to find an expected class within some module (in particular, command modules and classes).""" pass class DistutilsClassError (DistutilsError): """Some command class (or possibly distribution class, if anyone feels a need to subclass Distribution) is found not to be holding up its end of the bargain, ie. implementing some part of the "command "interface.""" pass class DistutilsGetoptError (DistutilsError): """The option table provided to 'fancy_getopt()' is bogus.""" pass class DistutilsArgError (DistutilsError): """Raised by fancy_getopt in response to getopt.error -- ie. an error in the command line usage.""" pass class DistutilsFileError (DistutilsError): """Any problems in the filesystem: expected file not found, etc. Typically this is for problems that we detect before IOError or OSError could be raised.""" pass class DistutilsOptionError (DistutilsError): """Syntactic/semantic errors in command options, such as use of mutually conflicting options, or inconsistent options, badly-spelled values, etc. No distinction is made between option values originating in the setup script, the command line, config files, or what-have-you -- but if we *know* something originated in the setup script, we'll raise DistutilsSetupError instead.""" pass class DistutilsSetupError (DistutilsError): """For errors that can be definitely blamed on the setup script, such as invalid keyword arguments to 'setup()'.""" pass class DistutilsPlatformError (DistutilsError): """We don't know how to do something on the current platform (but we do know how to do it on some platform) -- eg. trying to compile C files on a platform not supported by a CCompiler subclass.""" pass class DistutilsExecError (DistutilsError): """Any problems executing an external program (such as the C compiler, when compiling C files).""" pass class DistutilsInternalError (DistutilsError): """Internal inconsistencies or impossibilities (obviously, this should never be seen if the code is working!).""" pass class DistutilsTemplateError (DistutilsError): """Syntax error in a file list template.""" # Exception classes used by the CCompiler implementation classes class CCompilerError (Exception): """Some compile/link operation failed.""" class PreprocessError (CCompilerError): """Failure to preprocess one or more C/C++ files.""" class CompileError (CCompilerError): """Failure to compile one or more C/C++ source files.""" class LibError (CCompilerError): """Failure to create a static library from one or more C/C++ object files.""" class LinkError (CCompilerError): """Failure to link one or more C/C++ object files into an executable or shared library file.""" class UnknownFileError (CCompilerError): """Attempt to process an unknown file type.""" --- NEW FILE: extension.py --- """distutils.extension Provides the Extension class, used to describe C/C++ extension modules in setup scripts.""" __revision__ = "$Id: extension.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os, string, sys from types import * try: import warnings except ImportError: warnings = None # This class is really only used by the "build_ext" command, so it might # make sense to put it in distutils.command.build_ext. However, that # module is already big enough, and I want to make this class a bit more # complex to simplify some common cases ("foo" module in "foo.c") and do # better error-checking ("foo.c" actually exists). # # Also, putting this in build_ext.py means every setup script would have to # import that large-ish module (indirectly, through distutils.core) in # order to do anything. class Extension: """Just a collection of attributes that describes an extension module and everything needed to build it (hopefully in a portable way, but there are hooks that let you be as unportable as you need). Instance attributes: name : string the full name of the extension, including any packages -- ie. *not* a filename or pathname, but Python dotted name sources : [string] list of source filenames, relative to the distribution root (where the setup script lives), in Unix form (slash-separated) for portability. Source files may be C, C++, SWIG (.i), platform-specific resource files, or whatever else is recognized by the "build_ext" command as source for a Python extension. include_dirs : [string] list of directories to search for C/C++ header files (in Unix form for portability) define_macros : [(name : string, value : string|None)] list of macros to define; each macro is defined using a 2-tuple, where 'value' is either the string to define it to or None to define it without a particular value (equivalent of "#define FOO" in source or -DFOO on Unix C compiler command line) undef_macros : [string] list of macros to undefine explicitly library_dirs : [string] list of directories to search for C/C++ libraries at link time libraries : [string] list of library names (not filenames or paths) to link against runtime_library_dirs : [string] list of directories to search for C/C++ libraries at run time (for shared extensions, this is when the extension is loaded) extra_objects : [string] list of extra files to link with (eg. object files not implied by 'sources', static library that must be explicitly specified, binary resource files, etc.) extra_compile_args : [string] any extra platform- and compiler-specific information to use when compiling the source files in 'sources'. For platforms and compilers where "command line" makes sense, this is typically a list of command-line arguments, but for other platforms it could be anything. extra_link_args : [string] any extra platform- and compiler-specific information to use when linking object files together to create the extension (or to create a new static Python interpreter). Similar interpretation as for 'extra_compile_args'. export_symbols : [string] list of symbols to be exported from a shared extension. Not used on all platforms, and not generally necessary for Python extensions, which typically export exactly one symbol: "init" + extension_name. swig_opts : [string] any extra options to pass to SWIG if a source file has the .i extension. depends : [string] list of files that the extension depends on language : string extension language (i.e. "c", "c++", "objc"). Will be detected from the source extensions if not provided. """ # When adding arguments to this constructor, be sure to update # setup_keywords in core.py. def __init__ (self, name, sources, include_dirs=None, define_macros=None, undef_macros=None, library_dirs=None, libraries=None, runtime_library_dirs=None, extra_objects=None, extra_compile_args=None, extra_link_args=None, export_symbols=None, swig_opts = None, depends=None, language=None, **kw # To catch unknown keywords ): assert type(name) is StringType, "'name' must be a string" assert (type(sources) is ListType and map(type, sources) == [StringType]*len(sources)), \ "'sources' must be a list of strings" self.name = name self.sources = sources self.include_dirs = include_dirs or [] self.define_macros = define_macros or [] self.undef_macros = undef_macros or [] self.library_dirs = library_dirs or [] self.libraries = libraries or [] self.runtime_library_dirs = runtime_library_dirs or [] self.extra_objects = extra_objects or [] self.extra_compile_args = extra_compile_args or [] self.extra_link_args = extra_link_args or [] self.export_symbols = export_symbols or [] self.swig_opts = swig_opts or [] self.depends = depends or [] self.language = language # If there are unknown keyword options, warn about them if len(kw): L = kw.keys() ; L.sort() L = map(repr, L) msg = "Unknown Extension options: " + string.join(L, ', ') if warnings is not None: warnings.warn(msg) else: sys.stderr.write(msg + '\n') # class Extension def read_setup_file (filename): from distutils.sysconfig import \ parse_makefile, expand_makefile_vars, _variable_rx from distutils.text_file import TextFile from distutils.util import split_quoted # First pass over the file to gather "VAR = VALUE" assignments. vars = parse_makefile(filename) # Second pass to gobble up the real content: lines of the form # ... [ ...] [ ...] [ ...] file = TextFile(filename, strip_comments=1, skip_blanks=1, join_lines=1, lstrip_ws=1, rstrip_ws=1) extensions = [] while 1: line = file.readline() if line is None: # eof break if _variable_rx.match(line): # VAR=VALUE, handled in first pass continue if line[0] == line[-1] == "*": file.warn("'%s' lines not handled yet" % line) continue #print "original line: " + line line = expand_makefile_vars(line, vars) words = split_quoted(line) #print "expanded line: " + line # NB. this parses a slightly different syntax than the old # makesetup script: here, there must be exactly one extension per # line, and it must be the first word of the line. I have no idea # why the old syntax supported multiple extensions per line, as # they all wind up being the same. module = words[0] ext = Extension(module, []) append_next_word = None for word in words[1:]: if append_next_word is not None: append_next_word.append(word) append_next_word = None continue suffix = os.path.splitext(word)[1] switch = word[0:2] ; value = word[2:] if suffix in (".c", ".cc", ".cpp", ".cxx", ".c++", ".m", ".mm"): # hmm, should we do something about C vs. C++ sources? # or leave it up to the CCompiler implementation to # worry about? ext.sources.append(word) elif switch == "-I": ext.include_dirs.append(value) elif switch == "-D": equals = string.find(value, "=") if equals == -1: # bare "-DFOO" -- no value ext.define_macros.append((value, None)) else: # "-DFOO=blah" ext.define_macros.append((value[0:equals], value[equals+2:])) elif switch == "-U": ext.undef_macros.append(value) elif switch == "-C": # only here 'cause makesetup has it! ext.extra_compile_args.append(word) elif switch == "-l": ext.libraries.append(value) elif switch == "-L": ext.library_dirs.append(value) elif switch == "-R": ext.runtime_library_dirs.append(value) elif word == "-rpath": append_next_word = ext.runtime_library_dirs elif word == "-Xlinker": append_next_word = ext.extra_link_args elif word == "-Xcompiler": append_next_word = ext.extra_compile_args elif switch == "-u": ext.extra_link_args.append(word) if not value: append_next_word = ext.extra_link_args elif suffix in (".a", ".so", ".sl", ".o", ".dylib"): # NB. a really faithful emulation of makesetup would # append a .o file to extra_objects only if it # had a slash in it; otherwise, it would s/.o/.c/ # and append it to sources. Hmmmm. ext.extra_objects.append(word) else: file.warn("unrecognized argument '%s'" % word) extensions.append(ext) #print "module:", module #print "source files:", source_files #print "cpp args:", cpp_args #print "lib args:", library_args #extensions[module] = { 'sources': source_files, # 'cpp_args': cpp_args, # 'lib_args': library_args } return extensions # read_setup_file () --- NEW FILE: fancy_getopt.py --- """distutils.fancy_getopt Wrapper around the standard getopt module that provides the following additional features: * short and long options are tied together * options have help strings, so fancy_getopt could potentially create a complete usage summary * options set attributes of a passed-in object """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: fancy_getopt.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, string, re from types import * import getopt from distutils.errors import * # Much like command_re in distutils.core, this is close to but not quite # the same as a Python NAME -- except, in the spirit of most GNU # utilities, we use '-' in place of '_'. (The spirit of LISP lives on!) # The similarities to NAME are again not a coincidence... longopt_pat = r'[a-zA-Z](?:[a-zA-Z0-9-]*)' longopt_re = re.compile(r'^%s$' % longopt_pat) # For recognizing "negative alias" options, eg. "quiet=!verbose" neg_alias_re = re.compile("^(%s)=!(%s)$" % (longopt_pat, longopt_pat)) # This is used to translate long options to legitimate Python identifiers # (for use as attributes of some object). longopt_xlate = string.maketrans('-', '_') class FancyGetopt: """Wrapper around the standard 'getopt()' module that provides some handy extra functionality: * short and long options are tied together * options have help strings, and help text can be assembled from them * options set attributes of a passed-in object * boolean options can have "negative aliases" -- eg. if --quiet is the "negative alias" of --verbose, then "--quiet" on the command line sets 'verbose' to false """ def __init__ (self, option_table=None): # The option table is (currently) a list of tuples. The # tuples may have 3 or four values: # (long_option, short_option, help_string [, repeatable]) # if an option takes an argument, its long_option should have '=' # appended; short_option should just be a single character, no ':' # in any case. If a long_option doesn't have a corresponding # short_option, short_option should be None. All option tuples # must have long options. self.option_table = option_table # 'option_index' maps long option names to entries in the option # table (ie. those 3-tuples). self.option_index = {} if self.option_table: self._build_index() # 'alias' records (duh) alias options; {'foo': 'bar'} means # --foo is an alias for --bar self.alias = {} # 'negative_alias' keeps track of options that are the boolean # opposite of some other option self.negative_alias = {} # These keep track of the information in the option table. We # don't actually populate these structures until we're ready to # parse the command-line, since the 'option_table' passed in here # isn't necessarily the final word. self.short_opts = [] self.long_opts = [] self.short2long = {} self.attr_name = {} self.takes_arg = {} # And 'option_order' is filled up in 'getopt()'; it records the # original order of options (and their values) on the command-line, # but expands short options, converts aliases, etc. self.option_order = [] # __init__ () def _build_index (self): self.option_index.clear() for option in self.option_table: self.option_index[option[0]] = option def set_option_table (self, option_table): self.option_table = option_table self._build_index() def add_option (self, long_option, short_option=None, help_string=None): if self.option_index.has_key(long_option): raise DistutilsGetoptError, \ "option conflict: already an option '%s'" % long_option else: option = (long_option, short_option, help_string) self.option_table.append(option) self.option_index[long_option] = option def has_option (self, long_option): """Return true if the option table for this parser has an option with long name 'long_option'.""" return self.option_index.has_key(long_option) def get_attr_name (self, long_option): """Translate long option name 'long_option' to the form it has as an attribute of some object: ie., translate hyphens to underscores.""" return string.translate(long_option, longopt_xlate) def _check_alias_dict (self, aliases, what): assert type(aliases) is DictionaryType for (alias, opt) in aliases.items(): if not self.option_index.has_key(alias): raise DistutilsGetoptError, \ ("invalid %s '%s': " "option '%s' not defined") % (what, alias, alias) if not self.option_index.has_key(opt): raise DistutilsGetoptError, \ ("invalid %s '%s': " "aliased option '%s' not defined") % (what, alias, opt) def set_aliases (self, alias): """Set the aliases for this option parser.""" self._check_alias_dict(alias, "alias") self.alias = alias def set_negative_aliases (self, negative_alias): """Set the negative aliases for this option parser. 'negative_alias' should be a dictionary mapping option names to option names, both the key and value must already be defined in the option table.""" self._check_alias_dict(negative_alias, "negative alias") self.negative_alias = negative_alias def _grok_option_table (self): """Populate the various data structures that keep tabs on the option table. Called by 'getopt()' before it can do anything worthwhile. """ self.long_opts = [] self.short_opts = [] self.short2long.clear() self.repeat = {} for option in self.option_table: if len(option) == 3: long, short, help = option repeat = 0 elif len(option) == 4: long, short, help, repeat = option else: # the option table is part of the code, so simply # assert that it is correct raise ValueError, "invalid option tuple: %r" % (option,) # Type- and value-check the option names if type(long) is not StringType or len(long) < 2: raise DistutilsGetoptError, \ ("invalid long option '%s': " "must be a string of length >= 2") % long if (not ((short is None) or (type(short) is StringType and len(short) == 1))): raise DistutilsGetoptError, \ ("invalid short option '%s': " "must a single character or None") % short self.repeat[long] = repeat self.long_opts.append(long) if long[-1] == '=': # option takes an argument? if short: short = short + ':' long = long[0:-1] self.takes_arg[long] = 1 else: # Is option is a "negative alias" for some other option (eg. # "quiet" == "!verbose")? alias_to = self.negative_alias.get(long) if alias_to is not None: if self.takes_arg[alias_to]: raise DistutilsGetoptError, \ ("invalid negative alias '%s': " "aliased option '%s' takes a value") % \ (long, alias_to) self.long_opts[-1] = long # XXX redundant?! self.takes_arg[long] = 0 else: self.takes_arg[long] = 0 # If this is an alias option, make sure its "takes arg" flag is # the same as the option it's aliased to. alias_to = self.alias.get(long) if alias_to is not None: if self.takes_arg[long] != self.takes_arg[alias_to]: raise DistutilsGetoptError, \ ("invalid alias '%s': inconsistent with " "aliased option '%s' (one of them takes a value, " "the other doesn't") % (long, alias_to) # Now enforce some bondage on the long option name, so we can # later translate it to an attribute name on some object. Have # to do this a bit late to make sure we've removed any trailing # '='. if not longopt_re.match(long): raise DistutilsGetoptError, \ ("invalid long option name '%s' " + "(must be letters, numbers, hyphens only") % long self.attr_name[long] = self.get_attr_name(long) if short: self.short_opts.append(short) self.short2long[short[0]] = long # for option_table # _grok_option_table() def getopt (self, args=None, object=None): """Parse command-line options in args. Store as attributes on object. If 'args' is None or not supplied, uses 'sys.argv[1:]'. If 'object' is None or not supplied, creates a new OptionDummy object, stores option values there, and returns a tuple (args, object). If 'object' is supplied, it is modified in place and 'getopt()' just returns 'args'; in both cases, the returned 'args' is a modified copy of the passed-in 'args' list, which is left untouched. """ if args is None: args = sys.argv[1:] if object is None: object = OptionDummy() created_object = 1 else: created_object = 0 self._grok_option_table() short_opts = string.join(self.short_opts) try: opts, args = getopt.getopt(args, short_opts, self.long_opts) except getopt.error, msg: raise DistutilsArgError, msg for opt, val in opts: if len(opt) == 2 and opt[0] == '-': # it's a short option opt = self.short2long[opt[1]] else: assert len(opt) > 2 and opt[:2] == '--' opt = opt[2:] alias = self.alias.get(opt) if alias: opt = alias if not self.takes_arg[opt]: # boolean option? assert val == '', "boolean option can't have value" alias = self.negative_alias.get(opt) if alias: opt = alias val = 0 else: val = 1 attr = self.attr_name[opt] # The only repeating option at the moment is 'verbose'. # It has a negative option -q quiet, which should set verbose = 0. if val and self.repeat.get(attr) is not None: val = getattr(object, attr, 0) + 1 setattr(object, attr, val) self.option_order.append((opt, val)) # for opts if created_object: return args, object else: return args # getopt() def get_option_order (self): """Returns the list of (option, value) tuples processed by the previous run of 'getopt()'. Raises RuntimeError if 'getopt()' hasn't been called yet. """ if self.option_order is None: raise RuntimeError, "'getopt()' hasn't been called yet" else: return self.option_order def generate_help (self, header=None): """Generate help text (a list of strings, one per suggested line of output) from the option table for this FancyGetopt object. """ # Blithely assume the option table is good: probably wouldn't call # 'generate_help()' unless you've already called 'getopt()'. # First pass: determine maximum length of long option names max_opt = 0 for option in self.option_table: long = option[0] short = option[1] l = len(long) if long[-1] == '=': l = l - 1 if short is not None: l = l + 5 # " (-x)" where short == 'x' if l > max_opt: max_opt = l opt_width = max_opt + 2 + 2 + 2 # room for indent + dashes + gutter # Typical help block looks like this: # --foo controls foonabulation # Help block for longest option looks like this: # --flimflam set the flim-flam level # and with wrapped text: # --flimflam set the flim-flam level (must be between # 0 and 100, except on Tuesdays) # Options with short names will have the short name shown (but # it doesn't contribute to max_opt): # --foo (-f) controls foonabulation # If adding the short option would make the left column too wide, # we push the explanation off to the next line # --flimflam (-l) # set the flim-flam level # Important parameters: # - 2 spaces before option block start lines # - 2 dashes for each long option name # - min. 2 spaces between option and explanation (gutter) # - 5 characters (incl. space) for short option name # Now generate lines of help text. (If 80 columns were good enough # for Jesus, then 78 columns are good enough for me!) line_width = 78 text_width = line_width - opt_width big_indent = ' ' * opt_width if header: lines = [header] else: lines = ['Option summary:'] for option in self.option_table: long, short, help = option[:3] text = wrap_text(help, text_width) if long[-1] == '=': long = long[0:-1] # Case 1: no short option at all (makes life easy) if short is None: if text: lines.append(" --%-*s %s" % (max_opt, long, text[0])) else: lines.append(" --%-*s " % (max_opt, long)) # Case 2: we have a short option, so we have to include it # just after the long option else: opt_names = "%s (-%s)" % (long, short) if text: lines.append(" --%-*s %s" % (max_opt, opt_names, text[0])) else: lines.append(" --%-*s" % opt_names) for l in text[1:]: lines.append(big_indent + l) # for self.option_table return lines # generate_help () def print_help (self, header=None, file=None): if file is None: file = sys.stdout for line in self.generate_help(header): file.write(line + "\n") # class FancyGetopt def fancy_getopt (options, negative_opt, object, args): parser = FancyGetopt(options) parser.set_negative_aliases(negative_opt) return parser.getopt(args, object) WS_TRANS = string.maketrans(string.whitespace, ' ' * len(string.whitespace)) def wrap_text (text, width): """wrap_text(text : string, width : int) -> [string] Split 'text' into multiple lines of no more than 'width' characters each, and return the list of strings that results. """ if text is None: return [] if len(text) <= width: return [text] text = string.expandtabs(text) text = string.translate(text, WS_TRANS) chunks = re.split(r'( +|-+)', text) chunks = filter(None, chunks) # ' - ' results in empty strings lines = [] while chunks: cur_line = [] # list of chunks (to-be-joined) cur_len = 0 # length of current line while chunks: l = len(chunks[0]) if cur_len + l <= width: # can squeeze (at least) this chunk in cur_line.append(chunks[0]) del chunks[0] cur_len = cur_len + l else: # this line is full # drop last chunk if all space if cur_line and cur_line[-1][0] == ' ': del cur_line[-1] break if chunks: # any chunks left to process? # if the current line is still empty, then we had a single # chunk that's too big too fit on a line -- so we break # down and break it up at the line width if cur_len == 0: cur_line.append(chunks[0][0:width]) chunks[0] = chunks[0][width:] # all-whitespace chunks at the end of a line can be discarded # (and we know from the re.split above that if a chunk has # *any* whitespace, it is *all* whitespace) if chunks[0][0] == ' ': del chunks[0] # and store this line in the list-of-all-lines -- as a single # string, of course! lines.append(string.join(cur_line, '')) # while chunks return lines # wrap_text () def translate_longopt (opt): """Convert a long option name to a valid Python identifier by changing "-" to "_". """ return string.translate(opt, longopt_xlate) class OptionDummy: """Dummy class just used as a place to hold command-line option values as instance attributes.""" def __init__ (self, options=[]): """Create a new OptionDummy instance. The attributes listed in 'options' will be initialized to None.""" for opt in options: setattr(self, opt, None) # class OptionDummy if __name__ == "__main__": text = """\ Tra-la-la, supercalifragilisticexpialidocious. How *do* you spell that odd word, anyways? (Someone ask Mary -- she'll know [or she'll say, "How should I know?"].)""" for w in (10, 20, 30, 40): print "width: %d" % w print string.join(wrap_text(text, w), "\n") print --- NEW FILE: file_util.py --- """distutils.file_util Utility functions for operating on single files. """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: file_util.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os from distutils.errors import DistutilsFileError from distutils import log # for generating verbose output in 'copy_file()' _copy_action = { None: 'copying', 'hard': 'hard linking', 'sym': 'symbolically linking' } def _copy_file_contents (src, dst, buffer_size=16*1024): """Copy the file 'src' to 'dst'; both must be filenames. Any error opening either file, reading from 'src', or writing to 'dst', raises DistutilsFileError. Data is read/written in chunks of 'buffer_size' bytes (default 16k). No attempt is made to handle anything apart from regular files. """ # Stolen from shutil module in the standard library, but with # custom error-handling added. fsrc = None fdst = None try: try: fsrc = open(src, 'rb') except os.error, (errno, errstr): raise DistutilsFileError, \ "could not open '%s': %s" % (src, errstr) if os.path.exists(dst): try: os.unlink(dst) except os.error, (errno, errstr): raise DistutilsFileError, \ "could not delete '%s': %s" % (dst, errstr) try: fdst = open(dst, 'wb') except os.error, (errno, errstr): raise DistutilsFileError, \ "could not create '%s': %s" % (dst, errstr) while 1: try: buf = fsrc.read(buffer_size) except os.error, (errno, errstr): raise DistutilsFileError, \ "could not read from '%s': %s" % (src, errstr) if not buf: break try: fdst.write(buf) except os.error, (errno, errstr): raise DistutilsFileError, \ "could not write to '%s': %s" % (dst, errstr) finally: if fdst: fdst.close() if fsrc: fsrc.close() # _copy_file_contents() def copy_file (src, dst, preserve_mode=1, preserve_times=1, update=0, link=None, verbose=0, dry_run=0): """Copy a file 'src' to 'dst'. If 'dst' is a directory, then 'src' is copied there with the same name; otherwise, it must be a filename. (If the file exists, it will be ruthlessly clobbered.) If 'preserve_mode' is true (the default), the file's mode (type and permission bits, or whatever is analogous on the current platform) is copied. If 'preserve_times' is true (the default), the last-modified and last-access times are copied as well. If 'update' is true, 'src' will only be copied if 'dst' does not exist, or if 'dst' does exist but is older than 'src'. 'link' allows you to make hard links (os.link) or symbolic links (os.symlink) instead of copying: set it to "hard" or "sym"; if it is None (the default), files are copied. Don't set 'link' on systems that don't support it: 'copy_file()' doesn't check if hard or symbolic linking is available. Under Mac OS, uses the native file copy function in macostools; on other systems, uses '_copy_file_contents()' to copy file contents. Return a tuple (dest_name, copied): 'dest_name' is the actual name of the output file, and 'copied' is true if the file was copied (or would have been copied, if 'dry_run' true). """ # XXX if the destination file already exists, we clobber it if # copying, but blow up if linking. Hmmm. And I don't know what # macostools.copyfile() does. Should definitely be consistent, and # should probably blow up if destination exists and we would be # changing it (ie. it's not already a hard/soft link to src OR # (not update) and (src newer than dst). from distutils.dep_util import newer from stat import ST_ATIME, ST_MTIME, ST_MODE, S_IMODE if not os.path.isfile(src): raise DistutilsFileError, \ "can't copy '%s': doesn't exist or not a regular file" % src if os.path.isdir(dst): dir = dst dst = os.path.join(dst, os.path.basename(src)) else: dir = os.path.dirname(dst) if update and not newer(src, dst): log.debug("not copying %s (output up-to-date)", src) return dst, 0 try: action = _copy_action[link] except KeyError: raise ValueError, \ "invalid value '%s' for 'link' argument" % link if os.path.basename(dst) == os.path.basename(src): log.info("%s %s -> %s", action, src, dir) else: log.info("%s %s -> %s", action, src, dst) if dry_run: return (dst, 1) # On Mac OS, use the native file copy routine if os.name == 'mac': import macostools try: macostools.copy(src, dst, 0, preserve_times) except os.error, exc: raise DistutilsFileError, \ "could not copy '%s' to '%s': %s" % (src, dst, exc[-1]) # If linking (hard or symbolic), use the appropriate system call # (Unix only, of course, but that's the caller's responsibility) elif link == 'hard': if not (os.path.exists(dst) and os.path.samefile(src, dst)): os.link(src, dst) elif link == 'sym': if not (os.path.exists(dst) and os.path.samefile(src, dst)): os.symlink(src, dst) # Otherwise (non-Mac, not linking), copy the file contents and # (optionally) copy the times and mode. else: _copy_file_contents(src, dst) if preserve_mode or preserve_times: st = os.stat(src) # According to David Ascher , utime() should be done # before chmod() (at least under NT). if preserve_times: os.utime(dst, (st[ST_ATIME], st[ST_MTIME])) if preserve_mode: os.chmod(dst, S_IMODE(st[ST_MODE])) return (dst, 1) # copy_file () # XXX I suspect this is Unix-specific -- need porting help! def move_file (src, dst, verbose=0, dry_run=0): """Move a file 'src' to 'dst'. If 'dst' is a directory, the file will be moved into it with the same name; otherwise, 'src' is just renamed to 'dst'. Return the new full name of the file. Handles cross-device moves on Unix using 'copy_file()'. What about other systems??? """ from os.path import exists, isfile, isdir, basename, dirname import errno log.info("moving %s -> %s", src, dst) if dry_run: return dst if not isfile(src): raise DistutilsFileError, \ "can't move '%s': not a regular file" % src if isdir(dst): dst = os.path.join(dst, basename(src)) elif exists(dst): raise DistutilsFileError, \ "can't move '%s': destination '%s' already exists" % \ (src, dst) if not isdir(dirname(dst)): raise DistutilsFileError, \ "can't move '%s': destination '%s' not a valid path" % \ (src, dst) copy_it = 0 try: os.rename(src, dst) except os.error, (num, msg): if num == errno.EXDEV: copy_it = 1 else: raise DistutilsFileError, \ "couldn't move '%s' to '%s': %s" % (src, dst, msg) if copy_it: copy_file(src, dst) try: os.unlink(src) except os.error, (num, msg): try: os.unlink(dst) except os.error: pass raise DistutilsFileError, \ ("couldn't move '%s' to '%s' by copy/delete: " + "delete '%s' failed: %s") % \ (src, dst, src, msg) return dst # move_file () def write_file (filename, contents): """Create a file with the specified name and write 'contents' (a sequence of strings without line terminators) to it. """ f = open(filename, "w") for line in contents: f.write(line + "\n") f.close() --- NEW FILE: filelist.py --- """distutils.filelist Provides the FileList class, used for poking about the filesystem and building lists of files. """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: filelist.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os, string, re import fnmatch from types import * from glob import glob from distutils.util import convert_path from distutils.errors import DistutilsTemplateError, DistutilsInternalError from distutils import log class FileList: """A list of files built by on exploring the filesystem and filtered by applying various patterns to what we find there. Instance attributes: dir directory from which files will be taken -- only used if 'allfiles' not supplied to constructor files list of filenames currently being built/filtered/manipulated allfiles complete list of files under consideration (ie. without any filtering applied) """ def __init__(self, warn=None, debug_print=None): # ignore argument to FileList, but keep them for backwards # compatibility self.allfiles = None self.files = [] def set_allfiles (self, allfiles): self.allfiles = allfiles def findall (self, dir=os.curdir): self.allfiles = findall(dir) def debug_print (self, msg): """Print 'msg' to stdout if the global DEBUG (taken from the DISTUTILS_DEBUG environment variable) flag is true. """ from distutils.debug import DEBUG if DEBUG: print msg # -- List-like methods --------------------------------------------- def append (self, item): self.files.append(item) def extend (self, items): self.files.extend(items) def sort (self): # Not a strict lexical sort! sortable_files = map(os.path.split, self.files) sortable_files.sort() self.files = [] for sort_tuple in sortable_files: self.files.append(apply(os.path.join, sort_tuple)) # -- Other miscellaneous utility methods --------------------------- def remove_duplicates (self): # Assumes list has been sorted! for i in range(len(self.files) - 1, 0, -1): if self.files[i] == self.files[i - 1]: del self.files[i] # -- "File template" methods --------------------------------------- def _parse_template_line (self, line): words = string.split(line) action = words[0] patterns = dir = dir_pattern = None if action in ('include', 'exclude', 'global-include', 'global-exclude'): if len(words) < 2: raise DistutilsTemplateError, \ "'%s' expects ..." % action patterns = map(convert_path, words[1:]) elif action in ('recursive-include', 'recursive-exclude'): if len(words) < 3: raise DistutilsTemplateError, \ "'%s' expects ..." % action dir = convert_path(words[1]) patterns = map(convert_path, words[2:]) elif action in ('graft', 'prune'): if len(words) != 2: raise DistutilsTemplateError, \ "'%s' expects a single " % action dir_pattern = convert_path(words[1]) else: raise DistutilsTemplateError, "unknown action '%s'" % action return (action, patterns, dir, dir_pattern) # _parse_template_line () def process_template_line (self, line): # Parse the line: split it up, make sure the right number of words # is there, and return the relevant words. 'action' is always # defined: it's the first word of the line. Which of the other # three are defined depends on the action; it'll be either # patterns, (dir and patterns), or (dir_pattern). (action, patterns, dir, dir_pattern) = self._parse_template_line(line) # OK, now we know that the action is valid and we have the # right number of words on the line for that action -- so we # can proceed with minimal error-checking. if action == 'include': self.debug_print("include " + string.join(patterns)) for pattern in patterns: if not self.include_pattern(pattern, anchor=1): log.warn("warning: no files found matching '%s'", pattern) elif action == 'exclude': self.debug_print("exclude " + string.join(patterns)) for pattern in patterns: if not self.exclude_pattern(pattern, anchor=1): log.warn(("warning: no previously-included files " "found matching '%s'"), pattern) elif action == 'global-include': self.debug_print("global-include " + string.join(patterns)) for pattern in patterns: if not self.include_pattern(pattern, anchor=0): log.warn(("warning: no files found matching '%s' " + "anywhere in distribution"), pattern) elif action == 'global-exclude': self.debug_print("global-exclude " + string.join(patterns)) for pattern in patterns: if not self.exclude_pattern(pattern, anchor=0): log.warn(("warning: no previously-included files matching " "'%s' found anywhere in distribution"), pattern) elif action == 'recursive-include': self.debug_print("recursive-include %s %s" % (dir, string.join(patterns))) for pattern in patterns: if not self.include_pattern(pattern, prefix=dir): log.warn(("warning: no files found matching '%s' " + "under directory '%s'"), pattern, dir) elif action == 'recursive-exclude': self.debug_print("recursive-exclude %s %s" % (dir, string.join(patterns))) for pattern in patterns: if not self.exclude_pattern(pattern, prefix=dir): log.warn(("warning: no previously-included files matching " "'%s' found under directory '%s'"), pattern, dir) elif action == 'graft': self.debug_print("graft " + dir_pattern) if not self.include_pattern(None, prefix=dir_pattern): log.warn("warning: no directories found matching '%s'", dir_pattern) elif action == 'prune': self.debug_print("prune " + dir_pattern) if not self.exclude_pattern(None, prefix=dir_pattern): log.warn(("no previously-included directories found " + "matching '%s'"), dir_pattern) else: raise DistutilsInternalError, \ "this cannot happen: invalid action '%s'" % action # process_template_line () # -- Filtering/selection methods ----------------------------------- def include_pattern (self, pattern, anchor=1, prefix=None, is_regex=0): """Select strings (presumably filenames) from 'self.files' that match 'pattern', a Unix-style wildcard (glob) pattern. Patterns are not quite the same as implemented by the 'fnmatch' module: '*' and '?' match non-special characters, where "special" is platform- dependent: slash on Unix; colon, slash, and backslash on DOS/Windows; and colon on Mac OS. If 'anchor' is true (the default), then the pattern match is more stringent: "*.py" will match "foo.py" but not "foo/bar.py". If 'anchor' is false, both of these will match. If 'prefix' is supplied, then only filenames starting with 'prefix' (itself a pattern) and ending with 'pattern', with anything in between them, will match. 'anchor' is ignored in this case. If 'is_regex' is true, 'anchor' and 'prefix' are ignored, and 'pattern' is assumed to be either a string containing a regex or a regex object -- no translation is done, the regex is just compiled and used as-is. Selected strings will be added to self.files. Return 1 if files are found. """ files_found = 0 pattern_re = translate_pattern(pattern, anchor, prefix, is_regex) self.debug_print("include_pattern: applying regex r'%s'" % pattern_re.pattern) # delayed loading of allfiles list if self.allfiles is None: self.findall() for name in self.allfiles: if pattern_re.search(name): self.debug_print(" adding " + name) self.files.append(name) files_found = 1 return files_found # include_pattern () def exclude_pattern (self, pattern, anchor=1, prefix=None, is_regex=0): """Remove strings (presumably filenames) from 'files' that match 'pattern'. Other parameters are the same as for 'include_pattern()', above. The list 'self.files' is modified in place. Return 1 if files are found. """ files_found = 0 pattern_re = translate_pattern(pattern, anchor, prefix, is_regex) self.debug_print("exclude_pattern: applying regex r'%s'" % pattern_re.pattern) for i in range(len(self.files)-1, -1, -1): if pattern_re.search(self.files[i]): self.debug_print(" removing " + self.files[i]) del self.files[i] files_found = 1 return files_found # exclude_pattern () # class FileList # ---------------------------------------------------------------------- # Utility functions def findall (dir = os.curdir): """Find all files under 'dir' and return the list of full filenames (relative to 'dir'). """ from stat import ST_MODE, S_ISREG, S_ISDIR, S_ISLNK list = [] stack = [dir] pop = stack.pop push = stack.append while stack: dir = pop() names = os.listdir(dir) for name in names: if dir != os.curdir: # avoid the dreaded "./" syndrome fullname = os.path.join(dir, name) else: fullname = name # Avoid excess stat calls -- just one will do, thank you! stat = os.stat(fullname) mode = stat[ST_MODE] if S_ISREG(mode): list.append(fullname) elif S_ISDIR(mode) and not S_ISLNK(mode): push(fullname) return list def glob_to_re (pattern): """Translate a shell-like glob pattern to a regular expression; return a string containing the regex. Differs from 'fnmatch.translate()' in that '*' does not match "special characters" (which are platform-specific). """ pattern_re = fnmatch.translate(pattern) # '?' and '*' in the glob pattern become '.' and '.*' in the RE, which # IMHO is wrong -- '?' and '*' aren't supposed to match slash in Unix, # and by extension they shouldn't match such "special characters" under # any OS. So change all non-escaped dots in the RE to match any # character except the special characters. # XXX currently the "special characters" are just slash -- i.e. this is # Unix-only. pattern_re = re.sub(r'(^|[^\\])\.', r'\1[^/]', pattern_re) return pattern_re # glob_to_re () def translate_pattern (pattern, anchor=1, prefix=None, is_regex=0): """Translate a shell-like wildcard pattern to a compiled regular expression. Return the compiled regex. If 'is_regex' true, then 'pattern' is directly compiled to a regex (if it's a string) or just returned as-is (assumes it's a regex object). """ if is_regex: if type(pattern) is StringType: return re.compile(pattern) else: return pattern if pattern: pattern_re = glob_to_re(pattern) else: pattern_re = '' if prefix is not None: prefix_re = (glob_to_re(prefix))[0:-1] # ditch trailing $ pattern_re = "^" + os.path.join(prefix_re, ".*" + pattern_re) else: # no prefix -- respect anchor flag if anchor: pattern_re = "^" + pattern_re return re.compile(pattern_re) # translate_pattern () --- NEW FILE: log.py --- """A simple log mechanism styled after PEP 282.""" # This module should be kept compatible with Python 2.1. # The class here is styled after PEP 282 so that it could later be # replaced with a standard Python logging implementation. DEBUG = 1 INFO = 2 WARN = 3 ERROR = 4 FATAL = 5 import sys class Log: def __init__(self, threshold=WARN): self.threshold = threshold def _log(self, level, msg, args): if level >= self.threshold: print msg % args sys.stdout.flush() def log(self, level, msg, *args): self._log(level, msg, args) def debug(self, msg, *args): self._log(DEBUG, msg, args) def info(self, msg, *args): self._log(INFO, msg, args) def warn(self, msg, *args): self._log(WARN, msg, args) def error(self, msg, *args): self._log(ERROR, msg, args) def fatal(self, msg, *args): self._log(FATAL, msg, args) _global_log = Log() log = _global_log.log debug = _global_log.debug info = _global_log.info warn = _global_log.warn error = _global_log.error fatal = _global_log.fatal def set_threshold(level): # return the old threshold for use from tests old = _global_log.threshold _global_log.threshold = level return old def set_verbosity(v): if v <= 0: set_threshold(WARN) elif v == 1: set_threshold(INFO) elif v >= 2: set_threshold(DEBUG) --- NEW FILE: msvccompiler.py --- """distutils.msvccompiler Contains MSVCCompiler, an implementation of the abstract CCompiler class for the Microsoft Visual Studio. """ # Written by Perry Stoll # hacked by Robin Becker and Thomas Heller to do a better job of # finding DevStudio (through the registry) # This module should be kept compatible with Python 2.1. __revision__ = "$Id: msvccompiler.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string from distutils.errors import \ DistutilsExecError, DistutilsPlatformError, \ CompileError, LibError, LinkError from distutils.ccompiler import \ CCompiler, gen_preprocess_options, gen_lib_options from distutils import log _can_read_reg = 0 try: import _winreg _can_read_reg = 1 hkey_mod = _winreg RegOpenKeyEx = _winreg.OpenKeyEx RegEnumKey = _winreg.EnumKey RegEnumValue = _winreg.EnumValue RegError = _winreg.error except ImportError: try: import win32api import win32con _can_read_reg = 1 hkey_mod = win32con RegOpenKeyEx = win32api.RegOpenKeyEx RegEnumKey = win32api.RegEnumKey RegEnumValue = win32api.RegEnumValue RegError = win32api.error except ImportError: log.info("Warning: Can't read registry to find the " "necessary compiler setting\n" "Make sure that Python modules _winreg, " "win32api or win32con are installed.") pass if _can_read_reg: HKEYS = (hkey_mod.HKEY_USERS, hkey_mod.HKEY_CURRENT_USER, hkey_mod.HKEY_LOCAL_MACHINE, hkey_mod.HKEY_CLASSES_ROOT) def read_keys(base, key): """Return list of registry keys.""" try: handle = RegOpenKeyEx(base, key) except RegError: return None L = [] i = 0 while 1: try: k = RegEnumKey(handle, i) except RegError: break L.append(k) i = i + 1 return L def read_values(base, key): """Return dict of registry keys and values. All names are converted to lowercase. """ try: handle = RegOpenKeyEx(base, key) except RegError: return None d = {} i = 0 while 1: try: name, value, type = RegEnumValue(handle, i) except RegError: break name = name.lower() d[convert_mbcs(name)] = convert_mbcs(value) i = i + 1 return d def convert_mbcs(s): enc = getattr(s, "encode", None) if enc is not None: try: s = enc("mbcs") except UnicodeError: pass return s class MacroExpander: def __init__(self, version): self.macros = {} self.load_macros(version) def set_macro(self, macro, path, key): for base in HKEYS: d = read_values(base, path) if d: self.macros["$(%s)" % macro] = d[key] break def load_macros(self, version): vsbase = r"Software\Microsoft\VisualStudio\%0.1f" % version self.set_macro("VCInstallDir", vsbase + r"\Setup\VC", "productdir") self.set_macro("VSInstallDir", vsbase + r"\Setup\VS", "productdir") net = r"Software\Microsoft\.NETFramework" self.set_macro("FrameworkDir", net, "installroot") try: if version > 7.0: self.set_macro("FrameworkSDKDir", net, "sdkinstallrootv1.1") else: self.set_macro("FrameworkSDKDir", net, "sdkinstallroot") except KeyError, exc: # raise DistutilsPlatformError, \ ("The .NET Framework SDK needs to be installed before " "building extensions for Python.") p = r"Software\Microsoft\NET Framework Setup\Product" for base in HKEYS: try: h = RegOpenKeyEx(base, p) except RegError: continue key = RegEnumKey(h, 0) d = read_values(base, r"%s\%s" % (p, key)) self.macros["$(FrameworkVersion)"] = d["version"] def sub(self, s): for k, v in self.macros.items(): s = string.replace(s, k, v) return s def get_build_version(): """Return the version of MSVC that was used to build Python. For Python 2.3 and up, the version number is included in sys.version. For earlier versions, assume the compiler is MSVC 6. """ prefix = "MSC v." i = string.find(sys.version, prefix) if i == -1: return 6 i = i + len(prefix) s, rest = sys.version[i:].split(" ", 1) majorVersion = int(s[:-2]) - 6 minorVersion = int(s[2:3]) / 10.0 # I don't think paths are affected by minor version in version 6 if majorVersion == 6: minorVersion = 0 if majorVersion >= 6: return majorVersion + minorVersion # else we don't know what version of the compiler this is return None class MSVCCompiler (CCompiler) : """Concrete class that implements an interface to Microsoft Visual C++, as defined by the CCompiler abstract class.""" compiler_type = 'msvc' # Just set this so CCompiler's constructor doesn't barf. We currently # don't use the 'set_executables()' bureaucracy provided by CCompiler, # as it really isn't necessary for this sort of single-compiler class. # Would be nice to have a consistent interface with UnixCCompiler, # though, so it's worth thinking about. executables = {} # Private class data (need to distinguish C from C++ source for compiler) _c_extensions = ['.c'] _cpp_extensions = ['.cc', '.cpp', '.cxx'] _rc_extensions = ['.rc'] _mc_extensions = ['.mc'] # Needed for the filename generation methods provided by the # base class, CCompiler. src_extensions = (_c_extensions + _cpp_extensions + _rc_extensions + _mc_extensions) res_extension = '.res' obj_extension = '.obj' static_lib_extension = '.lib' shared_lib_extension = '.dll' static_lib_format = shared_lib_format = '%s%s' exe_extension = '.exe' def __init__ (self, verbose=0, dry_run=0, force=0): CCompiler.__init__ (self, verbose, dry_run, force) self.__version = get_build_version() if self.__version >= 7: self.__root = r"Software\Microsoft\VisualStudio" self.__macros = MacroExpander(self.__version) else: self.__root = r"Software\Microsoft\Devstudio" self.initialized = False def initialize(self): self.__paths = self.get_msvc_paths("path") if len (self.__paths) == 0: raise DistutilsPlatformError, \ ("Python was built with version %s of Visual Studio, " "and extensions need to be built with the same " "version of the compiler, but it isn't installed." % self.__version) self.cc = self.find_exe("cl.exe") self.linker = self.find_exe("link.exe") self.lib = self.find_exe("lib.exe") self.rc = self.find_exe("rc.exe") # resource compiler self.mc = self.find_exe("mc.exe") # message compiler self.set_path_env_var('lib') self.set_path_env_var('include') # extend the MSVC path with the current path try: for p in string.split(os.environ['path'], ';'): self.__paths.append(p) except KeyError: pass os.environ['path'] = string.join(self.__paths, ';') self.preprocess_options = None self.compile_options = [ '/nologo', '/Ox', '/MD', '/W3', '/GX' , '/DNDEBUG'] self.compile_options_debug = ['/nologo', '/Od', '/MDd', '/W3', '/GX', '/Z7', '/D_DEBUG'] self.ldflags_shared = ['/DLL', '/nologo', '/INCREMENTAL:NO'] if self.__version >= 7: self.ldflags_shared_debug = [ '/DLL', '/nologo', '/INCREMENTAL:no', '/DEBUG' ] else: self.ldflags_shared_debug = [ '/DLL', '/nologo', '/INCREMENTAL:no', '/pdb:None', '/DEBUG' ] self.ldflags_static = [ '/nologo'] self.initialized = True # -- Worker methods ------------------------------------------------ def object_filenames (self, source_filenames, strip_dir=0, output_dir=''): # Copied from ccompiler.py, extended to return .res as 'object'-file # for .rc input file if output_dir is None: output_dir = '' obj_names = [] for src_name in source_filenames: (base, ext) = os.path.splitext (src_name) if ext not in self.src_extensions: # Better to raise an exception instead of silently continuing # and later complain about sources and targets having # different lengths raise CompileError ("Don't know how to compile %s" % src_name) if strip_dir: base = os.path.basename (base) if ext in self._rc_extensions: obj_names.append (os.path.join (output_dir, base + self.res_extension)) elif ext in self._mc_extensions: obj_names.append (os.path.join (output_dir, base + self.res_extension)) else: obj_names.append (os.path.join (output_dir, base + self.obj_extension)) return obj_names # object_filenames () def compile(self, sources, output_dir=None, macros=None, include_dirs=None, debug=0, extra_preargs=None, extra_postargs=None, depends=None): if not self.initialized: self.initialize() macros, objects, extra_postargs, pp_opts, build = \ self._setup_compile(output_dir, macros, include_dirs, sources, depends, extra_postargs) compile_opts = extra_preargs or [] compile_opts.append ('/c') if debug: compile_opts.extend(self.compile_options_debug) else: compile_opts.extend(self.compile_options) for obj in objects: try: src, ext = build[obj] except KeyError: continue if debug: # pass the full pathname to MSVC in debug mode, # this allows the debugger to find the source file # without asking the user to browse for it src = os.path.abspath(src) if ext in self._c_extensions: input_opt = "/Tc" + src elif ext in self._cpp_extensions: input_opt = "/Tp" + src elif ext in self._rc_extensions: # compile .RC to .RES file input_opt = src output_opt = "/fo" + obj try: self.spawn ([self.rc] + pp_opts + [output_opt] + [input_opt]) except DistutilsExecError, msg: raise CompileError, msg continue elif ext in self._mc_extensions: # Compile .MC to .RC file to .RES file. # * '-h dir' specifies the directory for the # generated include file # * '-r dir' specifies the target directory of the # generated RC file and the binary message resource # it includes # # For now (since there are no options to change this), # we use the source-directory for the include file and # the build directory for the RC file and message # resources. This works at least for win32all. h_dir = os.path.dirname (src) rc_dir = os.path.dirname (obj) try: # first compile .MC to .RC and .H file self.spawn ([self.mc] + ['-h', h_dir, '-r', rc_dir] + [src]) base, _ = os.path.splitext (os.path.basename (src)) rc_file = os.path.join (rc_dir, base + '.rc') # then compile .RC to .RES file self.spawn ([self.rc] + ["/fo" + obj] + [rc_file]) except DistutilsExecError, msg: raise CompileError, msg continue else: # how to handle this file? raise CompileError ( "Don't know how to compile %s to %s" % \ (src, obj)) output_opt = "/Fo" + obj try: self.spawn ([self.cc] + compile_opts + pp_opts + [input_opt, output_opt] + extra_postargs) except DistutilsExecError, msg: raise CompileError, msg return objects # compile () def create_static_lib (self, objects, output_libname, output_dir=None, debug=0, target_lang=None): if not self.initialized: self.initialize() (objects, output_dir) = self._fix_object_args (objects, output_dir) output_filename = \ self.library_filename (output_libname, output_dir=output_dir) if self._need_link (objects, output_filename): lib_args = objects + ['/OUT:' + output_filename] if debug: pass # XXX what goes here? try: self.spawn ([self.lib] + lib_args) except DistutilsExecError, msg: raise LibError, msg else: log.debug("skipping %s (up-to-date)", output_filename) # create_static_lib () def link (self, target_desc, objects, output_filename, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None): if not self.initialized: self.initialize() (objects, output_dir) = self._fix_object_args (objects, output_dir) (libraries, library_dirs, runtime_library_dirs) = \ self._fix_lib_args (libraries, library_dirs, runtime_library_dirs) if runtime_library_dirs: self.warn ("I don't know what to do with 'runtime_library_dirs': " + str (runtime_library_dirs)) lib_opts = gen_lib_options (self, library_dirs, runtime_library_dirs, libraries) if output_dir is not None: output_filename = os.path.join (output_dir, output_filename) if self._need_link (objects, output_filename): if target_desc == CCompiler.EXECUTABLE: if debug: ldflags = self.ldflags_shared_debug[1:] else: ldflags = self.ldflags_shared[1:] else: if debug: ldflags = self.ldflags_shared_debug else: ldflags = self.ldflags_shared export_opts = [] for sym in (export_symbols or []): export_opts.append("/EXPORT:" + sym) ld_args = (ldflags + lib_opts + export_opts + objects + ['/OUT:' + output_filename]) # The MSVC linker generates .lib and .exp files, which cannot be # suppressed by any linker switches. The .lib files may even be # needed! Make sure they are generated in the temporary build # directory. Since they have different names for debug and release # builds, they can go into the same directory. if export_symbols is not None: (dll_name, dll_ext) = os.path.splitext( os.path.basename(output_filename)) implib_file = os.path.join( os.path.dirname(objects[0]), self.library_filename(dll_name)) ld_args.append ('/IMPLIB:' + implib_file) if extra_preargs: ld_args[:0] = extra_preargs if extra_postargs: ld_args.extend(extra_postargs) self.mkpath (os.path.dirname (output_filename)) try: self.spawn ([self.linker] + ld_args) except DistutilsExecError, msg: raise LinkError, msg else: log.debug("skipping %s (up-to-date)", output_filename) # link () # -- Miscellaneous methods ----------------------------------------- # These are all used by the 'gen_lib_options() function, in # ccompiler.py. def library_dir_option (self, dir): return "/LIBPATH:" + dir def runtime_library_dir_option (self, dir): raise DistutilsPlatformError, \ "don't know how to set runtime library search path for MSVC++" def library_option (self, lib): return self.library_filename (lib) def find_library_file (self, dirs, lib, debug=0): # Prefer a debugging library if found (and requested), but deal # with it if we don't have one. if debug: try_names = [lib + "_d", lib] else: try_names = [lib] for dir in dirs: for name in try_names: libfile = os.path.join(dir, self.library_filename (name)) if os.path.exists(libfile): return libfile else: # Oops, didn't find it in *any* of 'dirs' return None # find_library_file () # Helper methods for using the MSVC registry settings def find_exe(self, exe): """Return path to an MSVC executable program. Tries to find the program in several places: first, one of the MSVC program search paths from the registry; next, the directories in the PATH environment variable. If any of those work, return an absolute path that is known to exist. If none of them work, just return the original program name, 'exe'. """ for p in self.__paths: fn = os.path.join(os.path.abspath(p), exe) if os.path.isfile(fn): return fn # didn't find it; try existing path for p in string.split(os.environ['Path'],';'): fn = os.path.join(os.path.abspath(p),exe) if os.path.isfile(fn): return fn return exe def get_msvc_paths(self, path, platform='x86'): """Get a list of devstudio directories (include, lib or path). Return a list of strings. The list will be empty if unable to access the registry or appropriate registry keys not found. """ if not _can_read_reg: return [] path = path + " dirs" if self.__version >= 7: key = (r"%s\%0.1f\VC\VC_OBJECTS_PLATFORM_INFO\Win32\Directories" % (self.__root, self.__version)) else: key = (r"%s\6.0\Build System\Components\Platforms" r"\Win32 (%s)\Directories" % (self.__root, platform)) for base in HKEYS: d = read_values(base, key) if d: if self.__version >= 7: return string.split(self.__macros.sub(d[path]), ";") else: return string.split(d[path], ";") # MSVC 6 seems to create the registry entries we need only when # the GUI is run. if self.__version == 6: for base in HKEYS: if read_values(base, r"%s\6.0" % self.__root) is not None: self.warn("It seems you have Visual Studio 6 installed, " "but the expected registry settings are not present.\n" "You must at least run the Visual Studio GUI once " "so that these entries are created.") break return [] def set_path_env_var(self, name): """Set environment variable 'name' to an MSVC path type value. This is equivalent to a SET command prior to execution of spawned commands. """ if name == "lib": p = self.get_msvc_paths("library") else: p = self.get_msvc_paths(name) if p: os.environ[name] = string.join(p, ';') --- NEW FILE: mwerkscompiler.py --- """distutils.mwerkscompiler Contains MWerksCompiler, an implementation of the abstract CCompiler class for MetroWerks CodeWarrior on the Macintosh. Needs work to support CW on Windows.""" # This module should be kept compatible with Python 2.1. __revision__ = "$Id: mwerkscompiler.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string from types import * from distutils.errors import \ DistutilsExecError, DistutilsPlatformError, \ CompileError, LibError, LinkError from distutils.ccompiler import \ CCompiler, gen_preprocess_options, gen_lib_options import distutils.util import distutils.dir_util from distutils import log import mkcwproject class MWerksCompiler (CCompiler) : """Concrete class that implements an interface to MetroWerks CodeWarrior, as defined by the CCompiler abstract class.""" compiler_type = 'mwerks' # Just set this so CCompiler's constructor doesn't barf. We currently # don't use the 'set_executables()' bureaucracy provided by CCompiler, # as it really isn't necessary for this sort of single-compiler class. # Would be nice to have a consistent interface with UnixCCompiler, # though, so it's worth thinking about. executables = {} # Private class data (need to distinguish C from C++ source for compiler) _c_extensions = ['.c'] _cpp_extensions = ['.cc', '.cpp', '.cxx'] _rc_extensions = ['.r'] _exp_extension = '.exp' # Needed for the filename generation methods provided by the # base class, CCompiler. src_extensions = (_c_extensions + _cpp_extensions + _rc_extensions) res_extension = '.rsrc' obj_extension = '.obj' # Not used, really static_lib_extension = '.lib' shared_lib_extension = '.slb' static_lib_format = shared_lib_format = '%s%s' exe_extension = '' def __init__ (self, verbose=0, dry_run=0, force=0): CCompiler.__init__ (self, verbose, dry_run, force) def compile (self, sources, output_dir=None, macros=None, include_dirs=None, debug=0, extra_preargs=None, extra_postargs=None, depends=None): (output_dir, macros, include_dirs) = \ self._fix_compile_args (output_dir, macros, include_dirs) self.__sources = sources self.__macros = macros self.__include_dirs = include_dirs # Don't need extra_preargs and extra_postargs for CW return [] def link (self, target_desc, objects, output_filename, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None): # First fixup. (objects, output_dir) = self._fix_object_args (objects, output_dir) (libraries, library_dirs, runtime_library_dirs) = \ self._fix_lib_args (libraries, library_dirs, runtime_library_dirs) # First examine a couple of options for things that aren't implemented yet if not target_desc in (self.SHARED_LIBRARY, self.SHARED_OBJECT): raise DistutilsPlatformError, 'Can only make SHARED_LIBRARY or SHARED_OBJECT targets on the Mac' if runtime_library_dirs: raise DistutilsPlatformError, 'Runtime library dirs not implemented yet' if extra_preargs or extra_postargs: raise DistutilsPlatformError, 'Runtime library dirs not implemented yet' if len(export_symbols) != 1: raise DistutilsPlatformError, 'Need exactly one export symbol' # Next there are various things for which we need absolute pathnames. # This is because we (usually) create the project in a subdirectory of # where we are now, and keeping the paths relative is too much work right # now. sources = map(self._filename_to_abs, self.__sources) include_dirs = map(self._filename_to_abs, self.__include_dirs) if objects: objects = map(self._filename_to_abs, objects) else: objects = [] if build_temp: build_temp = self._filename_to_abs(build_temp) else: build_temp = os.curdir() if output_dir: output_filename = os.path.join(output_dir, output_filename) # The output filename needs special handling: splitting it into dir and # filename part. Actually I'm not sure this is really needed, but it # can't hurt. output_filename = self._filename_to_abs(output_filename) output_dir, output_filename = os.path.split(output_filename) # Now we need the short names of a couple of things for putting them # into the project. if output_filename[-8:] == '.ppc.slb': basename = output_filename[:-8] elif output_filename[-11:] == '.carbon.slb': basename = output_filename[:-11] else: basename = os.path.strip(output_filename)[0] projectname = basename + '.mcp' targetname = basename xmlname = basename + '.xml' exportname = basename + '.mcp.exp' prefixname = 'mwerks_%s_config.h'%basename # Create the directories we need distutils.dir_util.mkpath(build_temp, dry_run=self.dry_run) distutils.dir_util.mkpath(output_dir, dry_run=self.dry_run) # And on to filling in the parameters for the project builder settings = {} settings['mac_exportname'] = exportname settings['mac_outputdir'] = output_dir settings['mac_dllname'] = output_filename settings['mac_targetname'] = targetname settings['sysprefix'] = sys.prefix settings['mac_sysprefixtype'] = 'Absolute' sourcefilenames = [] sourcefiledirs = [] for filename in sources + objects: dirname, filename = os.path.split(filename) sourcefilenames.append(filename) if not dirname in sourcefiledirs: sourcefiledirs.append(dirname) settings['sources'] = sourcefilenames settings['libraries'] = libraries settings['extrasearchdirs'] = sourcefiledirs + include_dirs + library_dirs if self.dry_run: print 'CALLING LINKER IN', os.getcwd() for key, value in settings.items(): print '%20.20s %s'%(key, value) return # Build the export file exportfilename = os.path.join(build_temp, exportname) log.debug("\tCreate export file %s", exportfilename) fp = open(exportfilename, 'w') fp.write('%s\n'%export_symbols[0]) fp.close() # Generate the prefix file, if needed, and put it in the settings if self.__macros: prefixfilename = os.path.join(os.getcwd(), os.path.join(build_temp, prefixname)) fp = open(prefixfilename, 'w') fp.write('#include "mwerks_shcarbon_config.h"\n') for name, value in self.__macros: if value is None: fp.write('#define %s\n'%name) else: fp.write('#define %s %s\n'%(name, value)) fp.close() settings['prefixname'] = prefixname # Build the XML file. We need the full pathname (only lateron, really) # because we pass this pathname to CodeWarrior in an AppleEvent, and CW # doesn't have a clue about our working directory. xmlfilename = os.path.join(os.getcwd(), os.path.join(build_temp, xmlname)) log.debug("\tCreate XML file %s", xmlfilename) xmlbuilder = mkcwproject.cwxmlgen.ProjectBuilder(settings) xmlbuilder.generate() xmldata = settings['tmp_projectxmldata'] fp = open(xmlfilename, 'w') fp.write(xmldata) fp.close() # Generate the project. Again a full pathname. projectfilename = os.path.join(os.getcwd(), os.path.join(build_temp, projectname)) log.debug('\tCreate project file %s', projectfilename) mkcwproject.makeproject(xmlfilename, projectfilename) # And build it log.debug('\tBuild project') mkcwproject.buildproject(projectfilename) def _filename_to_abs(self, filename): # Some filenames seem to be unix-like. Convert to Mac names. ## if '/' in filename and ':' in filename: ## raise DistutilsPlatformError, 'Filename may be Unix or Mac style: %s'%filename ## if '/' in filename: ## filename = macurl2path(filename) filename = distutils.util.convert_path(filename) if not os.path.isabs(filename): curdir = os.getcwd() filename = os.path.join(curdir, filename) # Finally remove .. components components = string.split(filename, ':') for i in range(1, len(components)): if components[i] == '..': components[i] = '' return string.join(components, ':') def library_dir_option (self, dir): """Return the compiler option to add 'dir' to the list of directories searched for libraries. """ return # XXXX Not correct... def runtime_library_dir_option (self, dir): """Return the compiler option to add 'dir' to the list of directories searched for runtime libraries. """ # Nothing needed or Mwerks/Mac. return def library_option (self, lib): """Return the compiler option to add 'dir' to the list of libraries linked into the shared library or executable. """ return def find_library_file (self, dirs, lib, debug=0): """Search the specified list of directories for a static or shared library file 'lib' and return the full path to that file. If 'debug' true, look for a debugging version (if that makes sense on the current platform). Return None if 'lib' wasn't found in any of the specified directories. """ return 0 --- NEW FILE: spawn.py --- """distutils.spawn Provides the 'spawn()' function, a front-end to various platform- specific functions for launching another program in a sub-process. Also provides the 'find_executable()' to search the path for a given executable name. """ # This module should be kept compatible with Python 2.1. __revision__ = "$Id: spawn.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string from distutils.errors import * from distutils import log def spawn (cmd, search_path=1, verbose=0, dry_run=0): """Run another program, specified as a command list 'cmd', in a new process. 'cmd' is just the argument list for the new process, ie. cmd[0] is the program to run and cmd[1:] are the rest of its arguments. There is no way to run a program with a name different from that of its executable. If 'search_path' is true (the default), the system's executable search path will be used to find the program; otherwise, cmd[0] must be the exact path to the executable. If 'dry_run' is true, the command will not actually be run. Raise DistutilsExecError if running the program fails in any way; just return on success. """ if os.name == 'posix': _spawn_posix(cmd, search_path, dry_run=dry_run) elif os.name == 'nt': _spawn_nt(cmd, search_path, dry_run=dry_run) elif os.name == 'os2': _spawn_os2(cmd, search_path, dry_run=dry_run) else: raise DistutilsPlatformError, \ "don't know how to spawn programs on platform '%s'" % os.name # spawn () def _nt_quote_args (args): """Quote command-line arguments for DOS/Windows conventions: just wraps every argument which contains blanks in double quotes, and returns a new argument list. """ # XXX this doesn't seem very robust to me -- but if the Windows guys # say it'll work, I guess I'll have to accept it. (What if an arg # contains quotes? What other magic characters, other than spaces, # have to be escaped? Is there an escaping mechanism other than # quoting?) for i in range(len(args)): if string.find(args[i], ' ') != -1: args[i] = '"%s"' % args[i] return args def _spawn_nt (cmd, search_path=1, verbose=0, dry_run=0): executable = cmd[0] cmd = _nt_quote_args(cmd) if search_path: # either we find one or it stays the same executable = find_executable(executable) or executable log.info(string.join([executable] + cmd[1:], ' ')) if not dry_run: # spawn for NT requires a full path to the .exe try: rc = os.spawnv(os.P_WAIT, executable, cmd) except OSError, exc: # this seems to happen when the command isn't found raise DistutilsExecError, \ "command '%s' failed: %s" % (cmd[0], exc[-1]) if rc != 0: # and this reflects the command running but failing raise DistutilsExecError, \ "command '%s' failed with exit status %d" % (cmd[0], rc) def _spawn_os2 (cmd, search_path=1, verbose=0, dry_run=0): executable = cmd[0] #cmd = _nt_quote_args(cmd) if search_path: # either we find one or it stays the same executable = find_executable(executable) or executable log.info(string.join([executable] + cmd[1:], ' ')) if not dry_run: # spawnv for OS/2 EMX requires a full path to the .exe try: rc = os.spawnv(os.P_WAIT, executable, cmd) except OSError, exc: # this seems to happen when the command isn't found raise DistutilsExecError, \ "command '%s' failed: %s" % (cmd[0], exc[-1]) if rc != 0: # and this reflects the command running but failing print "command '%s' failed with exit status %d" % (cmd[0], rc) raise DistutilsExecError, \ "command '%s' failed with exit status %d" % (cmd[0], rc) def _spawn_posix (cmd, search_path=1, verbose=0, dry_run=0): log.info(string.join(cmd, ' ')) if dry_run: return exec_fn = search_path and os.execvp or os.execv pid = os.fork() if pid == 0: # in the child try: #print "cmd[0] =", cmd[0] #print "cmd =", cmd exec_fn(cmd[0], cmd) except OSError, e: sys.stderr.write("unable to execute %s: %s\n" % (cmd[0], e.strerror)) os._exit(1) sys.stderr.write("unable to execute %s for unknown reasons" % cmd[0]) os._exit(1) else: # in the parent # Loop until the child either exits or is terminated by a signal # (ie. keep waiting if it's merely stopped) while 1: try: (pid, status) = os.waitpid(pid, 0) except OSError, exc: import errno if exc.errno == errno.EINTR: continue raise DistutilsExecError, \ "command '%s' failed: %s" % (cmd[0], exc[-1]) if os.WIFSIGNALED(status): raise DistutilsExecError, \ "command '%s' terminated by signal %d" % \ (cmd[0], os.WTERMSIG(status)) elif os.WIFEXITED(status): exit_status = os.WEXITSTATUS(status) if exit_status == 0: return # hey, it succeeded! else: raise DistutilsExecError, \ "command '%s' failed with exit status %d" % \ (cmd[0], exit_status) elif os.WIFSTOPPED(status): continue else: raise DistutilsExecError, \ "unknown error executing '%s': termination status %d" % \ (cmd[0], status) # _spawn_posix () def find_executable(executable, path=None): """Try to find 'executable' in the directories listed in 'path' (a string listing directories separated by 'os.pathsep'; defaults to os.environ['PATH']). Returns the complete filename or None if not found. """ if path is None: path = os.environ['PATH'] paths = string.split(path, os.pathsep) (base, ext) = os.path.splitext(executable) if (sys.platform == 'win32' or os.name == 'os2') and (ext != '.exe'): executable = executable + '.exe' if not os.path.isfile(executable): for p in paths: f = os.path.join(p, executable) if os.path.isfile(f): # the file exists, we have a shot at spawn working return f return None else: return executable # find_executable() --- NEW FILE: sysconfig.py --- """Provide access to Python's configuration information. The specific configuration variables available depend heavily on the platform and configuration. The values may be retrieved using get_config_var(name), and the list of variables is available via get_config_vars().keys(). Additional convenience functions are also available. Written by: Fred L. Drake, Jr. Email: """ __revision__ = "$Id: sysconfig.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os import re import string import sys from errors import DistutilsPlatformError # These are needed in a couple of spots, so just compute them once. PREFIX = os.path.normpath(sys.prefix) EXEC_PREFIX = os.path.normpath(sys.exec_prefix) # python_build: (Boolean) if true, we're either building Python or # building an extension with an un-installed Python, so we use # different (hard-wired) directories. argv0_path = os.path.dirname(os.path.abspath(sys.executable)) landmark = os.path.join(argv0_path, "Modules", "Setup") python_build = os.path.isfile(landmark) del argv0_path, landmark def get_python_version(): """Return a string containing the major and minor Python version, leaving off the patchlevel. Sample return values could be '1.5' or '2.2'. """ return sys.version[:3] def get_python_inc(plat_specific=0, prefix=None): """Return the directory containing installed Python header files. If 'plat_specific' is false (the default), this is the path to the non-platform-specific header files, i.e. Python.h and so on; otherwise, this is the path to platform-specific header files (namely pyconfig.h). If 'prefix' is supplied, use it instead of sys.prefix or sys.exec_prefix -- i.e., ignore 'plat_specific'. """ if prefix is None: prefix = plat_specific and EXEC_PREFIX or PREFIX if os.name == "posix": if python_build: base = os.path.dirname(os.path.abspath(sys.executable)) if plat_specific: inc_dir = base else: inc_dir = os.path.join(base, "Include") if not os.path.exists(inc_dir): inc_dir = os.path.join(os.path.dirname(base), "Include") return inc_dir return os.path.join(prefix, "include", "python" + get_python_version()) elif os.name == "nt": return os.path.join(prefix, "include") elif os.name == "mac": if plat_specific: return os.path.join(prefix, "Mac", "Include") else: return os.path.join(prefix, "Include") elif os.name == "os2": return os.path.join(prefix, "Include") else: raise DistutilsPlatformError( "I don't know where Python installs its C header files " "on platform '%s'" % os.name) def get_python_lib(plat_specific=0, standard_lib=0, prefix=None): """Return the directory containing the Python library (standard or site additions). If 'plat_specific' is true, return the directory containing platform-specific modules, i.e. any module from a non-pure-Python module distribution; otherwise, return the platform-shared library directory. If 'standard_lib' is true, return the directory containing standard Python library modules; otherwise, return the directory for site-specific modules. If 'prefix' is supplied, use it instead of sys.prefix or sys.exec_prefix -- i.e., ignore 'plat_specific'. """ if prefix is None: prefix = plat_specific and EXEC_PREFIX or PREFIX if os.name == "posix": libpython = os.path.join(prefix, "lib", "python" + get_python_version()) if standard_lib: return libpython else: return os.path.join(libpython, "site-packages") elif os.name == "nt": if standard_lib: return os.path.join(prefix, "Lib") else: if get_python_version() < "2.2": return prefix else: return os.path.join(PREFIX, "Lib", "site-packages") elif os.name == "mac": if plat_specific: if standard_lib: return os.path.join(prefix, "Lib", "lib-dynload") else: return os.path.join(prefix, "Lib", "site-packages") else: if standard_lib: return os.path.join(prefix, "Lib") else: return os.path.join(prefix, "Lib", "site-packages") elif os.name == "os2": if standard_lib: return os.path.join(PREFIX, "Lib") else: return os.path.join(PREFIX, "Lib", "site-packages") else: raise DistutilsPlatformError( "I don't know where Python installs its library " "on platform '%s'" % os.name) def customize_compiler(compiler): """Do any platform-specific customization of a CCompiler instance. Mainly needed on Unix, so we can plug in the information that varies across Unices and is stored in Python's Makefile. """ if compiler.compiler_type == "unix": (cc, cxx, opt, cflags, ccshared, ldshared, so_ext) = \ get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS', 'CCSHARED', 'LDSHARED', 'SO') if os.environ.has_key('CC'): cc = os.environ['CC'] if os.environ.has_key('CXX'): cxx = os.environ['CXX'] if os.environ.has_key('LDSHARED'): ldshared = os.environ['LDSHARED'] if os.environ.has_key('CPP'): cpp = os.environ['CPP'] else: cpp = cc + " -E" # not always if os.environ.has_key('LDFLAGS'): ldshared = ldshared + ' ' + os.environ['LDFLAGS'] if os.environ.has_key('CFLAGS'): cflags = opt + ' ' + os.environ['CFLAGS'] ldshared = ldshared + ' ' + os.environ['CFLAGS'] if os.environ.has_key('CPPFLAGS'): cpp = cpp + ' ' + os.environ['CPPFLAGS'] cflags = cflags + ' ' + os.environ['CPPFLAGS'] ldshared = ldshared + ' ' + os.environ['CPPFLAGS'] cc_cmd = cc + ' ' + cflags compiler.set_executables( preprocessor=cpp, compiler=cc_cmd, compiler_so=cc_cmd + ' ' + ccshared, compiler_cxx=cxx, linker_so=ldshared, linker_exe=cc) compiler.shared_lib_extension = so_ext def get_config_h_filename(): """Return full pathname of installed pyconfig.h file.""" if python_build: inc_dir = os.curdir else: inc_dir = get_python_inc(plat_specific=1) if get_python_version() < '2.2': config_h = 'config.h' else: # The name of the config.h file changed in 2.2 config_h = 'pyconfig.h' return os.path.join(inc_dir, config_h) def get_makefile_filename(): """Return full pathname of installed Makefile from the Python build.""" if python_build: return os.path.join(os.path.dirname(sys.executable), "Makefile") lib_dir = get_python_lib(plat_specific=1, standard_lib=1) return os.path.join(lib_dir, "config", "Makefile") def parse_config_h(fp, g=None): """Parse a config.h-style file. A dictionary containing name/value pairs is returned. If an optional dictionary is passed in as the second argument, it is used instead of a new dictionary. """ if g is None: g = {} define_rx = re.compile("#define ([A-Z][A-Z0-9_]+) (.*)\n") undef_rx = re.compile("/[*] #undef ([A-Z][A-Z0-9_]+) [*]/\n") # while 1: line = fp.readline() if not line: break m = define_rx.match(line) if m: n, v = m.group(1, 2) try: v = int(v) except ValueError: pass g[n] = v else: m = undef_rx.match(line) if m: g[m.group(1)] = 0 return g # Regexes needed for parsing Makefile (and similar syntaxes, # like old-style Setup files). _variable_rx = re.compile("([a-zA-Z][a-zA-Z0-9_]+)\s*=\s*(.*)") _findvar1_rx = re.compile(r"\$\(([A-Za-z][A-Za-z0-9_]*)\)") _findvar2_rx = re.compile(r"\${([A-Za-z][A-Za-z0-9_]*)}") def parse_makefile(fn, g=None): """Parse a Makefile-style file. A dictionary containing name/value pairs is returned. If an optional dictionary is passed in as the second argument, it is used instead of a new dictionary. """ from distutils.text_file import TextFile fp = TextFile(fn, strip_comments=1, skip_blanks=1, join_lines=1) if g is None: g = {} done = {} notdone = {} while 1: line = fp.readline() if line is None: # eof break m = _variable_rx.match(line) if m: n, v = m.group(1, 2) v = string.strip(v) if "$" in v: notdone[n] = v else: try: v = int(v) except ValueError: pass done[n] = v # do variable interpolation here while notdone: for name in notdone.keys(): value = notdone[name] m = _findvar1_rx.search(value) or _findvar2_rx.search(value) if m: n = m.group(1) found = True if done.has_key(n): item = str(done[n]) elif notdone.has_key(n): # get it on a subsequent round found = False elif os.environ.has_key(n): # do it like make: fall back to environment item = os.environ[n] else: done[n] = item = "" if found: after = value[m.end():] value = value[:m.start()] + item + after if "$" in after: notdone[name] = value else: try: value = int(value) except ValueError: done[name] = string.strip(value) else: done[name] = value del notdone[name] else: # bogus variable reference; just drop it since we can't deal del notdone[name] fp.close() # save the results in the global dictionary g.update(done) return g def expand_makefile_vars(s, vars): """Expand Makefile-style variables -- "${foo}" or "$(foo)" -- in 'string' according to 'vars' (a dictionary mapping variable names to values). Variables not present in 'vars' are silently expanded to the empty string. The variable values in 'vars' should not contain further variable expansions; if 'vars' is the output of 'parse_makefile()', you're fine. Returns a variable-expanded version of 's'. """ # This algorithm does multiple expansion, so if vars['foo'] contains # "${bar}", it will expand ${foo} to ${bar}, and then expand # ${bar}... and so forth. This is fine as long as 'vars' comes from # 'parse_makefile()', which takes care of such expansions eagerly, # according to make's variable expansion semantics. while 1: m = _findvar1_rx.search(s) or _findvar2_rx.search(s) if m: (beg, end) = m.span() s = s[0:beg] + vars.get(m.group(1)) + s[end:] else: break return s _config_vars = None def _init_posix(): """Initialize the module as appropriate for POSIX systems.""" g = {} # load the installed Makefile: try: filename = get_makefile_filename() parse_makefile(filename, g) except IOError, msg: my_msg = "invalid Python installation: unable to open %s" % filename if hasattr(msg, "strerror"): my_msg = my_msg + " (%s)" % msg.strerror raise DistutilsPlatformError(my_msg) # On MacOSX we need to check the setting of the environment variable # MACOSX_DEPLOYMENT_TARGET: configure bases some choices on it so # it needs to be compatible. # If it isn't set we set it to the configure-time value if sys.platform == 'darwin' and g.has_key('CONFIGURE_MACOSX_DEPLOYMENT_TARGET'): cfg_target = g['CONFIGURE_MACOSX_DEPLOYMENT_TARGET'] cur_target = os.getenv('MACOSX_DEPLOYMENT_TARGET', '') if cur_target == '': cur_target = cfg_target os.putenv('MACOSX_DEPLOYMENT_TARGET', cfg_target) if cfg_target != cur_target: my_msg = ('$MACOSX_DEPLOYMENT_TARGET mismatch: now "%s" but "%s" during configure' % (cur_target, cfg_target)) raise DistutilsPlatformError(my_msg) # On AIX, there are wrong paths to the linker scripts in the Makefile # -- these paths are relative to the Python source, but when installed # the scripts are in another directory. if python_build: g['LDSHARED'] = g['BLDSHARED'] elif get_python_version() < '2.1': # The following two branches are for 1.5.2 compatibility. if sys.platform == 'aix4': # what about AIX 3.x ? # Linker script is in the config directory, not in Modules as the # Makefile says. python_lib = get_python_lib(standard_lib=1) ld_so_aix = os.path.join(python_lib, 'config', 'ld_so_aix') python_exp = os.path.join(python_lib, 'config', 'python.exp') g['LDSHARED'] = "%s %s -bI:%s" % (ld_so_aix, g['CC'], python_exp) elif sys.platform == 'beos': # Linker script is in the config directory. In the Makefile it is # relative to the srcdir, which after installation no longer makes # sense. python_lib = get_python_lib(standard_lib=1) linkerscript_path = string.split(g['LDSHARED'])[0] linkerscript_name = os.path.basename(linkerscript_path) linkerscript = os.path.join(python_lib, 'config', linkerscript_name) # XXX this isn't the right place to do this: adding the Python # library to the link, if needed, should be in the "build_ext" # command. (It's also needed for non-MS compilers on Windows, and # it's taken care of for them by the 'build_ext.get_libraries()' # method.) g['LDSHARED'] = ("%s -L%s/lib -lpython%s" % (linkerscript, PREFIX, get_python_version())) global _config_vars _config_vars = g def _init_nt(): """Initialize the module as appropriate for NT""" g = {} # set basic install directories g['LIBDEST'] = get_python_lib(plat_specific=0, standard_lib=1) g['BINLIBDEST'] = get_python_lib(plat_specific=1, standard_lib=1) # XXX hmmm.. a normal install puts include files here g['INCLUDEPY'] = get_python_inc(plat_specific=0) g['SO'] = '.pyd' g['EXE'] = ".exe" global _config_vars _config_vars = g def _init_mac(): """Initialize the module as appropriate for Macintosh systems""" g = {} # set basic install directories g['LIBDEST'] = get_python_lib(plat_specific=0, standard_lib=1) g['BINLIBDEST'] = get_python_lib(plat_specific=1, standard_lib=1) # XXX hmmm.. a normal install puts include files here g['INCLUDEPY'] = get_python_inc(plat_specific=0) import MacOS if not hasattr(MacOS, 'runtimemodel'): g['SO'] = '.ppc.slb' else: g['SO'] = '.%s.slb' % MacOS.runtimemodel # XXX are these used anywhere? g['install_lib'] = os.path.join(EXEC_PREFIX, "Lib") g['install_platlib'] = os.path.join(EXEC_PREFIX, "Mac", "Lib") # These are used by the extension module build g['srcdir'] = ':' global _config_vars _config_vars = g def _init_os2(): """Initialize the module as appropriate for OS/2""" g = {} # set basic install directories g['LIBDEST'] = get_python_lib(plat_specific=0, standard_lib=1) g['BINLIBDEST'] = get_python_lib(plat_specific=1, standard_lib=1) # XXX hmmm.. a normal install puts include files here g['INCLUDEPY'] = get_python_inc(plat_specific=0) g['SO'] = '.pyd' g['EXE'] = ".exe" global _config_vars _config_vars = g def get_config_vars(*args): """With no arguments, return a dictionary of all configuration variables relevant for the current platform. Generally this includes everything needed to build extensions and install both pure modules and extensions. On Unix, this means every variable defined in Python's installed Makefile; on Windows and Mac OS it's a much smaller set. With arguments, return a list of values that result from looking up each argument in the configuration variable dictionary. """ global _config_vars if _config_vars is None: func = globals().get("_init_" + os.name) if func: func() else: _config_vars = {} # Normalized versions of prefix and exec_prefix are handy to have; # in fact, these are the standard versions used most places in the # Distutils. _config_vars['prefix'] = PREFIX _config_vars['exec_prefix'] = EXEC_PREFIX if args: vals = [] for name in args: vals.append(_config_vars.get(name)) return vals else: return _config_vars def get_config_var(name): """Return the value of a single variable using the dictionary returned by 'get_config_vars()'. Equivalent to get_config_vars().get(name) """ return get_config_vars().get(name) --- NEW FILE: text_file.py --- """text_file provides the TextFile class, which gives an interface to text files that (optionally) takes care of stripping comments, ignoring blank lines, and joining lines with backslashes.""" __revision__ = "$Id: text_file.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" from types import * import sys, os, string class TextFile: """Provides a file-like object that takes care of all the things you commonly want to do when processing a text file that has some line-by-line syntax: strip comments (as long as "#" is your comment character), skip blank lines, join adjacent lines by escaping the newline (ie. backslash at end of line), strip leading and/or trailing whitespace. All of these are optional and independently controllable. Provides a 'warn()' method so you can generate warning messages that report physical line number, even if the logical line in question spans multiple physical lines. Also provides 'unreadline()' for implementing line-at-a-time lookahead. Constructor is called as: TextFile (filename=None, file=None, **options) It bombs (RuntimeError) if both 'filename' and 'file' are None; 'filename' should be a string, and 'file' a file object (or something that provides 'readline()' and 'close()' methods). It is recommended that you supply at least 'filename', so that TextFile can include it in warning messages. If 'file' is not supplied, TextFile creates its own using the 'open()' builtin. The options are all boolean, and affect the value returned by 'readline()': strip_comments [default: true] strip from "#" to end-of-line, as well as any whitespace leading up to the "#" -- unless it is escaped by a backslash lstrip_ws [default: false] strip leading whitespace from each line before returning it rstrip_ws [default: true] strip trailing whitespace (including line terminator!) from each line before returning it skip_blanks [default: true} skip lines that are empty *after* stripping comments and whitespace. (If both lstrip_ws and rstrip_ws are false, then some lines may consist of solely whitespace: these will *not* be skipped, even if 'skip_blanks' is true.) join_lines [default: false] if a backslash is the last non-newline character on a line after stripping comments and whitespace, join the following line to it to form one "logical line"; if N consecutive lines end with a backslash, then N+1 physical lines will be joined to form one logical line. collapse_join [default: false] strip leading whitespace from lines that are joined to their predecessor; only matters if (join_lines and not lstrip_ws) Note that since 'rstrip_ws' can strip the trailing newline, the semantics of 'readline()' must differ from those of the builtin file object's 'readline()' method! In particular, 'readline()' returns None for end-of-file: an empty string might just be a blank line (or an all-whitespace line), if 'rstrip_ws' is true but 'skip_blanks' is not.""" default_options = { 'strip_comments': 1, 'skip_blanks': 1, 'lstrip_ws': 0, 'rstrip_ws': 1, 'join_lines': 0, 'collapse_join': 0, } def __init__ (self, filename=None, file=None, **options): """Construct a new TextFile object. At least one of 'filename' (a string) and 'file' (a file-like object) must be supplied. They keyword argument options are described above and affect the values returned by 'readline()'.""" if filename is None and file is None: raise RuntimeError, \ "you must supply either or both of 'filename' and 'file'" # set values for all options -- either from client option hash # or fallback to default_options for opt in self.default_options.keys(): if options.has_key (opt): setattr (self, opt, options[opt]) else: setattr (self, opt, self.default_options[opt]) # sanity check client option hash for opt in options.keys(): if not self.default_options.has_key (opt): raise KeyError, "invalid TextFile option '%s'" % opt if file is None: self.open (filename) else: self.filename = filename self.file = file self.current_line = 0 # assuming that file is at BOF! # 'linebuf' is a stack of lines that will be emptied before we # actually read from the file; it's only populated by an # 'unreadline()' operation self.linebuf = [] def open (self, filename): """Open a new file named 'filename'. This overrides both the 'filename' and 'file' arguments to the constructor.""" self.filename = filename self.file = open (self.filename, 'r') self.current_line = 0 def close (self): """Close the current file and forget everything we know about it (filename, current line number).""" self.file.close () self.file = None self.filename = None self.current_line = None def gen_error (self, msg, line=None): outmsg = [] if line is None: line = self.current_line outmsg.append(self.filename + ", ") if type (line) in (ListType, TupleType): outmsg.append("lines %d-%d: " % tuple (line)) else: outmsg.append("line %d: " % line) outmsg.append(str(msg)) return string.join(outmsg, "") def error (self, msg, line=None): raise ValueError, "error: " + self.gen_error(msg, line) def warn (self, msg, line=None): """Print (to stderr) a warning message tied to the current logical line in the current file. If the current logical line in the file spans multiple physical lines, the warning refers to the whole range, eg. "lines 3-5". If 'line' supplied, it overrides the current line number; it may be a list or tuple to indicate a range of physical lines, or an integer for a single physical line.""" sys.stderr.write("warning: " + self.gen_error(msg, line) + "\n") def readline (self): """Read and return a single logical line from the current file (or from an internal buffer if lines have previously been "unread" with 'unreadline()'). If the 'join_lines' option is true, this may involve reading multiple physical lines concatenated into a single string. Updates the current line number, so calling 'warn()' after 'readline()' emits a warning about the physical line(s) just read. Returns None on end-of-file, since the empty string can occur if 'rstrip_ws' is true but 'strip_blanks' is not.""" # If any "unread" lines waiting in 'linebuf', return the top # one. (We don't actually buffer read-ahead data -- lines only # get put in 'linebuf' if the client explicitly does an # 'unreadline()'. if self.linebuf: line = self.linebuf[-1] del self.linebuf[-1] return line buildup_line = '' while 1: # read the line, make it None if EOF line = self.file.readline() if line == '': line = None if self.strip_comments and line: # Look for the first "#" in the line. If none, never # mind. If we find one and it's the first character, or # is not preceded by "\", then it starts a comment -- # strip the comment, strip whitespace before it, and # carry on. Otherwise, it's just an escaped "#", so # unescape it (and any other escaped "#"'s that might be # lurking in there) and otherwise leave the line alone. pos = string.find (line, "#") if pos == -1: # no "#" -- no comments pass # It's definitely a comment -- either "#" is the first # character, or it's elsewhere and unescaped. elif pos == 0 or line[pos-1] != "\\": # Have to preserve the trailing newline, because it's # the job of a later step (rstrip_ws) to remove it -- # and if rstrip_ws is false, we'd better preserve it! # (NB. this means that if the final line is all comment # and has no trailing newline, we will think that it's # EOF; I think that's OK.) eol = (line[-1] == '\n') and '\n' or '' line = line[0:pos] + eol # If all that's left is whitespace, then skip line # *now*, before we try to join it to 'buildup_line' -- # that way constructs like # hello \\ # # comment that should be ignored # there # result in "hello there". if string.strip(line) == "": continue else: # it's an escaped "#" line = string.replace (line, "\\#", "#") # did previous line end with a backslash? then accumulate if self.join_lines and buildup_line: # oops: end of file if line is None: self.warn ("continuation line immediately precedes " "end-of-file") return buildup_line if self.collapse_join: line = string.lstrip (line) line = buildup_line + line # careful: pay attention to line number when incrementing it if type (self.current_line) is ListType: self.current_line[1] = self.current_line[1] + 1 else: self.current_line = [self.current_line, self.current_line+1] # just an ordinary line, read it as usual else: if line is None: # eof return None # still have to be careful about incrementing the line number! if type (self.current_line) is ListType: self.current_line = self.current_line[1] + 1 else: self.current_line = self.current_line + 1 # strip whitespace however the client wants (leading and # trailing, or one or the other, or neither) if self.lstrip_ws and self.rstrip_ws: line = string.strip (line) elif self.lstrip_ws: line = string.lstrip (line) elif self.rstrip_ws: line = string.rstrip (line) # blank line (whether we rstrip'ed or not)? skip to next line # if appropriate if (line == '' or line == '\n') and self.skip_blanks: continue if self.join_lines: if line[-1] == '\\': buildup_line = line[:-1] continue if line[-2:] == '\\\n': buildup_line = line[0:-2] + '\n' continue # well, I guess there's some actual content there: return it return line # readline () def readlines (self): """Read and return the list of all logical lines remaining in the current file.""" lines = [] while 1: line = self.readline() if line is None: return lines lines.append (line) def unreadline (self, line): """Push 'line' (a string) onto an internal buffer that will be checked by future 'readline()' calls. Handy for implementing a parser with line-at-a-time lookahead.""" self.linebuf.append (line) if __name__ == "__main__": test_data = """# test file line 3 \\ # intervening comment continues on next line """ # result 1: no fancy options result1 = map (lambda x: x + "\n", string.split (test_data, "\n")[0:-1]) # result 2: just strip comments result2 = ["\n", "line 3 \\\n", " continues on next line\n"] # result 3: just strip blank lines result3 = ["# test file\n", "line 3 \\\n", "# intervening comment\n", " continues on next line\n"] # result 4: default, strip comments, blank lines, and trailing whitespace result4 = ["line 3 \\", " continues on next line"] # result 5: strip comments and blanks, plus join lines (but don't # "collapse" joined lines result5 = ["line 3 continues on next line"] # result 6: strip comments and blanks, plus join lines (and # "collapse" joined lines result6 = ["line 3 continues on next line"] def test_input (count, description, file, expected_result): result = file.readlines () # result = string.join (result, '') if result == expected_result: print "ok %d (%s)" % (count, description) else: print "not ok %d (%s):" % (count, description) print "** expected:" print expected_result print "** received:" print result filename = "test.txt" out_file = open (filename, "w") out_file.write (test_data) out_file.close () in_file = TextFile (filename, strip_comments=0, skip_blanks=0, lstrip_ws=0, rstrip_ws=0) test_input (1, "no processing", in_file, result1) in_file = TextFile (filename, strip_comments=1, skip_blanks=0, lstrip_ws=0, rstrip_ws=0) test_input (2, "strip comments", in_file, result2) in_file = TextFile (filename, strip_comments=0, skip_blanks=1, lstrip_ws=0, rstrip_ws=0) test_input (3, "strip blanks", in_file, result3) in_file = TextFile (filename) test_input (4, "default processing", in_file, result4) in_file = TextFile (filename, strip_comments=1, skip_blanks=1, join_lines=1, rstrip_ws=1) test_input (5, "join lines without collapsing", in_file, result5) in_file = TextFile (filename, strip_comments=1, skip_blanks=1, join_lines=1, rstrip_ws=1, collapse_join=1) test_input (6, "join lines with collapsing", in_file, result6) os.remove (filename) --- NEW FILE: unixccompiler.py --- """distutils.unixccompiler Contains the UnixCCompiler class, a subclass of CCompiler that handles the "typical" Unix-style command-line C compiler: * macros defined with -Dname[=value] * macros undefined with -Uname * include search directories specified with -Idir * libraries specified with -lllib * library search directories specified with -Ldir * compile handled by 'cc' (or similar) executable with -c option: compiles .c to .o * link static library handled by 'ar' command (possibly with 'ranlib') * link shared library handled by 'cc -shared' """ __revision__ = "$Id: unixccompiler.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import os, sys from types import StringType, NoneType from copy import copy from distutils import sysconfig from distutils.dep_util import newer from distutils.ccompiler import \ CCompiler, gen_preprocess_options, gen_lib_options from distutils.errors import \ DistutilsExecError, CompileError, LibError, LinkError from distutils import log # XXX Things not currently handled: # * optimization/debug/warning flags; we just use whatever's in Python's # Makefile and live with it. Is this adequate? If not, we might # have to have a bunch of subclasses GNUCCompiler, SGICCompiler, # SunCCompiler, and I suspect down that road lies madness. # * even if we don't know a warning flag from an optimization flag, # we need some way for outsiders to feed preprocessor/compiler/linker # flags in to us -- eg. a sysadmin might want to mandate certain flags # via a site config file, or a user might want to set something for # compiling this module distribution only via the setup.py command # line, whatever. As long as these options come from something on the # current system, they can be as system-dependent as they like, and we # should just happily stuff them into the preprocessor/compiler/linker # options and carry on. class UnixCCompiler(CCompiler): compiler_type = 'unix' # These are used by CCompiler in two places: the constructor sets # instance attributes 'preprocessor', 'compiler', etc. from them, and # 'set_executable()' allows any of these to be set. The defaults here # are pretty generic; they will probably have to be set by an outsider # (eg. using information discovered by the sysconfig about building # Python extensions). executables = {'preprocessor' : None, 'compiler' : ["cc"], 'compiler_so' : ["cc"], 'compiler_cxx' : ["cc"], 'linker_so' : ["cc", "-shared"], 'linker_exe' : ["cc"], 'archiver' : ["ar", "-cr"], 'ranlib' : None, } if sys.platform[:6] == "darwin": executables['ranlib'] = ["ranlib"] # Needed for the filename generation methods provided by the base # class, CCompiler. NB. whoever instantiates/uses a particular # UnixCCompiler instance should set 'shared_lib_ext' -- we set a # reasonable common default here, but it's not necessarily used on all # Unices! src_extensions = [".c",".C",".cc",".cxx",".cpp",".m"] obj_extension = ".o" static_lib_extension = ".a" shared_lib_extension = ".so" dylib_lib_extension = ".dylib" static_lib_format = shared_lib_format = dylib_lib_format = "lib%s%s" if sys.platform == "cygwin": exe_extension = ".exe" def preprocess(self, source, output_file=None, macros=None, include_dirs=None, extra_preargs=None, extra_postargs=None): ignore, macros, include_dirs = \ self._fix_compile_args(None, macros, include_dirs) pp_opts = gen_preprocess_options(macros, include_dirs) pp_args = self.preprocessor + pp_opts if output_file: pp_args.extend(['-o', output_file]) if extra_preargs: pp_args[:0] = extra_preargs if extra_postargs: pp_args.extend(extra_postargs) pp_args.append(source) # We need to preprocess: either we're being forced to, or we're # generating output to stdout, or there's a target output file and # the source file is newer than the target (or the target doesn't # exist). if self.force or output_file is None or newer(source, output_file): if output_file: self.mkpath(os.path.dirname(output_file)) try: self.spawn(pp_args) except DistutilsExecError, msg: raise CompileError, msg def _compile(self, obj, src, ext, cc_args, extra_postargs, pp_opts): try: self.spawn(self.compiler_so + cc_args + [src, '-o', obj] + extra_postargs) except DistutilsExecError, msg: raise CompileError, msg def create_static_lib(self, objects, output_libname, output_dir=None, debug=0, target_lang=None): objects, output_dir = self._fix_object_args(objects, output_dir) output_filename = \ self.library_filename(output_libname, output_dir=output_dir) if self._need_link(objects, output_filename): self.mkpath(os.path.dirname(output_filename)) self.spawn(self.archiver + [output_filename] + objects + self.objects) # Not many Unices required ranlib anymore -- SunOS 4.x is, I # think the only major Unix that does. Maybe we need some # platform intelligence here to skip ranlib if it's not # needed -- or maybe Python's configure script took care of # it for us, hence the check for leading colon. if self.ranlib: try: self.spawn(self.ranlib + [output_filename]) except DistutilsExecError, msg: raise LibError, msg else: log.debug("skipping %s (up-to-date)", output_filename) def link(self, target_desc, objects, output_filename, output_dir=None, libraries=None, library_dirs=None, runtime_library_dirs=None, export_symbols=None, debug=0, extra_preargs=None, extra_postargs=None, build_temp=None, target_lang=None): objects, output_dir = self._fix_object_args(objects, output_dir) libraries, library_dirs, runtime_library_dirs = \ self._fix_lib_args(libraries, library_dirs, runtime_library_dirs) lib_opts = gen_lib_options(self, library_dirs, runtime_library_dirs, libraries) if type(output_dir) not in (StringType, NoneType): raise TypeError, "'output_dir' must be a string or None" if output_dir is not None: output_filename = os.path.join(output_dir, output_filename) if self._need_link(objects, output_filename): ld_args = (objects + self.objects + lib_opts + ['-o', output_filename]) if debug: ld_args[:0] = ['-g'] if extra_preargs: ld_args[:0] = extra_preargs if extra_postargs: ld_args.extend(extra_postargs) self.mkpath(os.path.dirname(output_filename)) try: if target_desc == CCompiler.EXECUTABLE: linker = self.linker_exe[:] else: linker = self.linker_so[:] if target_lang == "c++" and self.compiler_cxx: linker[0] = self.compiler_cxx[0] self.spawn(linker + ld_args) except DistutilsExecError, msg: raise LinkError, msg else: log.debug("skipping %s (up-to-date)", output_filename) # -- Miscellaneous methods ----------------------------------------- # These are all used by the 'gen_lib_options() function, in # ccompiler.py. def library_dir_option(self, dir): return "-L" + dir def runtime_library_dir_option(self, dir): # XXX Hackish, at the very least. See Python bug #445902: # http://sourceforge.net/tracker/index.php # ?func=detail&aid=445902&group_id=5470&atid=105470 # Linkers on different platforms need different options to # specify that directories need to be added to the list of # directories searched for dependencies when a dynamic library # is sought. GCC has to be told to pass the -R option through # to the linker, whereas other compilers just know this. # Other compilers may need something slightly different. At # this time, there's no way to determine this information from # the configuration data stored in the Python installation, so # we use this hack. compiler = os.path.basename(sysconfig.get_config_var("CC")) if sys.platform[:6] == "darwin": # MacOSX's linker doesn't understand the -R flag at all return "-L" + dir elif sys.platform[:5] == "hp-ux": return "+s -L" + dir elif sys.platform[:7] == "irix646" or sys.platform[:6] == "osf1V5": return ["-rpath", dir] elif compiler[:3] == "gcc" or compiler[:3] == "g++": return "-Wl,-R" + dir else: return "-R" + dir def library_option(self, lib): return "-l" + lib def find_library_file(self, dirs, lib, debug=0): shared_f = self.library_filename(lib, lib_type='shared') dylib_f = self.library_filename(lib, lib_type='dylib') static_f = self.library_filename(lib, lib_type='static') for dir in dirs: shared = os.path.join(dir, shared_f) dylib = os.path.join(dir, dylib_f) static = os.path.join(dir, static_f) # We're second-guessing the linker here, with not much hard # data to go on: GCC seems to prefer the shared library, so I'm # assuming that *all* Unix C compilers do. And of course I'm # ignoring even GCC's "-static" option. So sue me. if os.path.exists(dylib): return dylib elif os.path.exists(shared): return shared elif os.path.exists(static): return static # Oops, didn't find it in *any* of 'dirs' return None --- NEW FILE: util.py --- """distutils.util Miscellaneous utility functions -- anything that doesn't fit into one of the other *util.py modules. """ __revision__ = "$Id: util.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $" import sys, os, string, re from distutils.errors import DistutilsPlatformError from distutils.dep_util import newer from distutils.spawn import spawn from distutils import log def get_platform (): """Return a string that identifies the current platform. This is used mainly to distinguish platform-specific build directories and platform-specific built distributions. Typically includes the OS name and version and the architecture (as supplied by 'os.uname()'), although the exact information included depends on the OS; eg. for IRIX the architecture isn't particularly important (IRIX only runs on SGI hardware), but for Linux the kernel version isn't particularly important. Examples of returned values: linux-i586 linux-alpha (?) solaris-2.6-sun4u irix-5.3 irix64-6.2 For non-POSIX platforms, currently just returns 'sys.platform'. """ if os.name != "posix" or not hasattr(os, 'uname'): # XXX what about the architecture? NT is Intel or Alpha, # Mac OS is M68k or PPC, etc. return sys.platform # Try to distinguish various flavours of Unix (osname, host, release, version, machine) = os.uname() # Convert the OS name to lowercase, remove '/' characters # (to accommodate BSD/OS), and translate spaces (for "Power Macintosh") osname = string.lower(osname) osname = string.replace(osname, '/', '') machine = string.replace(machine, ' ', '_') if osname[:5] == "linux": # At least on Linux/Intel, 'machine' is the processor -- # i386, etc. # XXX what about Alpha, SPARC, etc? return "%s-%s" % (osname, machine) elif osname[:5] == "sunos": if release[0] >= "5": # SunOS 5 == Solaris 2 osname = "solaris" release = "%d.%s" % (int(release[0]) - 3, release[2:]) # fall through to standard osname-release-machine representation elif osname[:4] == "irix": # could be "irix64"! return "%s-%s" % (osname, release) elif osname[:3] == "aix": return "%s-%s.%s" % (osname, version, release) elif osname[:6] == "cygwin": osname = "cygwin" rel_re = re.compile (r'[\d.]+') m = rel_re.match(release) if m: release = m.group() return "%s-%s-%s" % (osname, release, machine) # get_platform () def convert_path (pathname): """Return 'pathname' as a name that will work on the native filesystem, i.e. split it on '/' and put it back together again using the current directory separator. Needed because filenames in the setup script are always supplied in Unix style, and have to be converted to the local convention before we can actually use them in the filesystem. Raises ValueError on non-Unix-ish systems if 'pathname' either starts or ends with a slash. """ if os.sep == '/': return pathname if not pathname: return pathname if pathname[0] == '/': raise ValueError, "path '%s' cannot be absolute" % pathname if pathname[-1] == '/': raise ValueError, "path '%s' cannot end with '/'" % pathname paths = string.split(pathname, '/') while '.' in paths: paths.remove('.') if not paths: return os.curdir return apply(os.path.join, paths) # convert_path () def change_root (new_root, pathname): """Return 'pathname' with 'new_root' prepended. If 'pathname' is relative, this is equivalent to "os.path.join(new_root,pathname)". Otherwise, it requires making 'pathname' relative and then joining the two, which is tricky on DOS/Windows and Mac OS. """ if os.name == 'posix': if not os.path.isabs(pathname): return os.path.join(new_root, pathname) else: return os.path.join(new_root, pathname[1:]) elif os.name == 'nt': (drive, path) = os.path.splitdrive(pathname) if path[0] == '\\': path = path[1:] return os.path.join(new_root, path) elif os.name == 'os2': (drive, path) = os.path.splitdrive(pathname) if path[0] == os.sep: path = path[1:] return os.path.join(new_root, path) elif os.name == 'mac': if not os.path.isabs(pathname): return os.path.join(new_root, pathname) else: # Chop off volume name from start of path elements = string.split(pathname, ":", 1) pathname = ":" + elements[1] return os.path.join(new_root, pathname) else: raise DistutilsPlatformError, \ "nothing known about platform '%s'" % os.name _environ_checked = 0 def check_environ (): """Ensure that 'os.environ' has all the environment variables we guarantee that users can use in config files, command-line options, etc. Currently this includes: HOME - user's home directory (Unix only) PLAT - description of the current platform, including hardware and OS (see 'get_platform()') """ global _environ_checked if _environ_checked: return if os.name == 'posix' and not os.environ.has_key('HOME'): import pwd os.environ['HOME'] = pwd.getpwuid(os.getuid())[5] if not os.environ.has_key('PLAT'): os.environ['PLAT'] = get_platform() _environ_checked = 1 def subst_vars (s, local_vars): """Perform shell/Perl-style variable substitution on 'string'. Every occurrence of '$' followed by a name is considered a variable, and variable is substituted by the value found in the 'local_vars' dictionary, or in 'os.environ' if it's not in 'local_vars'. 'os.environ' is first checked/augmented to guarantee that it contains certain values: see 'check_environ()'. Raise ValueError for any variables not found in either 'local_vars' or 'os.environ'. """ check_environ() def _subst (match, local_vars=local_vars): var_name = match.group(1) if local_vars.has_key(var_name): return str(local_vars[var_name]) else: return os.environ[var_name] try: return re.sub(r'\$([a-zA-Z_][a-zA-Z_0-9]*)', _subst, s) except KeyError, var: raise ValueError, "invalid variable '$%s'" % var # subst_vars () def grok_environment_error (exc, prefix="error: "): """Generate a useful error message from an EnvironmentError (IOError or OSError) exception object. Handles Python 1.5.1 and 1.5.2 styles, and does what it can to deal with exception objects that don't have a filename (which happens when the error is due to a two-file operation, such as 'rename()' or 'link()'. Returns the error message as a string prefixed with 'prefix'. """ # check for Python 1.5.2-style {IO,OS}Error exception objects if hasattr(exc, 'filename') and hasattr(exc, 'strerror'): if exc.filename: error = prefix + "%s: %s" % (exc.filename, exc.strerror) else: # two-argument functions in posix module don't # include the filename in the exception object! error = prefix + "%s" % exc.strerror else: error = prefix + str(exc[-1]) return error # Needed by 'split_quoted()' _wordchars_re = _squote_re = _dquote_re = None def _init_regex(): global _wordchars_re, _squote_re, _dquote_re _wordchars_re = re.compile(r'[^\\\'\"%s ]*' % string.whitespace) _squote_re = re.compile(r"'(?:[^'\\]|\\.)*'") _dquote_re = re.compile(r'"(?:[^"\\]|\\.)*"') def split_quoted (s): """Split a string up according to Unix shell-like rules for quotes and backslashes. In short: words are delimited by spaces, as long as those spaces are not escaped by a backslash, or inside a quoted string. Single and double quotes are equivalent, and the quote characters can be backslash-escaped. The backslash is stripped from any two-character escape sequence, leaving only the escaped character. The quote characters are stripped from any quoted string. Returns a list of words. """ # This is a nice algorithm for splitting up a single string, since it # doesn't require character-by-character examination. It was a little # bit of a brain-bender to get it working right, though... if _wordchars_re is None: _init_regex() s = string.strip(s) words = [] pos = 0 while s: m = _wordchars_re.match(s, pos) end = m.end() if end == len(s): words.append(s[:end]) break if s[end] in string.whitespace: # unescaped, unquoted whitespace: now words.append(s[:end]) # we definitely have a word delimiter s = string.lstrip(s[end:]) pos = 0 elif s[end] == '\\': # preserve whatever is being escaped; # will become part of the current word s = s[:end] + s[end+1:] pos = end+1 else: if s[end] == "'": # slurp singly-quoted string m = _squote_re.match(s, end) elif s[end] == '"': # slurp doubly-quoted string m = _dquote_re.match(s, end) else: raise RuntimeError, \ "this can't happen (bad char '%c')" % s[end] if m is None: raise ValueError, \ "bad string (mismatched %s quotes?)" % s[end] (beg, end) = m.span() s = s[:beg] + s[beg+1:end-1] + s[end:] pos = m.end() - 2 if pos >= len(s): words.append(s) break return words # split_quoted () def execute (func, args, msg=None, verbose=0, dry_run=0): """Perform some action that affects the outside world (eg. by writing to the filesystem). Such actions are special because they are disabled by the 'dry_run' flag. This method takes care of all that bureaucracy for you; all you have to do is supply the function to call and an argument tuple for it (to embody the "external action" being performed), and an optional message to print. """ if msg is None: msg = "%s%r" % (func.__name__, args) if msg[-2:] == ',)': # correct for singleton tuple msg = msg[0:-2] + ')' log.info(msg) if not dry_run: apply(func, args) def strtobool (val): """Convert a string representation of truth to true (1) or false (0). True values are 'y', 'yes', 't', 'true', 'on', and '1'; false values are 'n', 'no', 'f', 'false', 'off', and '0'. Raises ValueError if 'val' is anything else. """ val = string.lower(val) if val in ('y', 'yes', 't', 'true', 'on', '1'): return 1 elif val in ('n', 'no', 'f', 'false', 'off', '0'): return 0 else: raise ValueError, "invalid truth value %r" % (val,) def byte_compile (py_files, optimize=0, force=0, prefix=None, base_dir=None, verbose=1, dry_run=0, direct=None): """Byte-compile a collection of Python source files to either .pyc or .pyo files in the same directory. 'py_files' is a list of files to compile; any files that don't end in ".py" are silently skipped. 'optimize' must be one of the following: 0 - don't optimize (generate .pyc) 1 - normal optimization (like "python -O") 2 - extra optimization (like "python -OO") If 'force' is true, all files are recompiled regardless of timestamps. The source filename encoded in each bytecode file defaults to the filenames listed in 'py_files'; you can modify these with 'prefix' and 'basedir'. 'prefix' is a string that will be stripped off of each source filename, and 'base_dir' is a directory name that will be prepended (after 'prefix' is stripped). You can supply either or both (or neither) of 'prefix' and 'base_dir', as you wish. If 'dry_run' is true, doesn't actually do anything that would affect the filesystem. Byte-compilation is either done directly in this interpreter process with the standard py_compile module, or indirectly by writing a temporary script and executing it. Normally, you should let 'byte_compile()' figure out to use direct compilation or not (see the source for details). The 'direct' flag is used by the script generated in indirect mode; unless you know what you're doing, leave it set to None. """ # First, if the caller didn't force us into direct or indirect mode, # figure out which mode we should be in. We take a conservative # approach: choose direct mode *only* if the current interpreter is # in debug mode and optimize is 0. If we're not in debug mode (-O # or -OO), we don't know which level of optimization this # interpreter is running with, so we can't do direct # byte-compilation and be certain that it's the right thing. Thus, # always compile indirectly if the current interpreter is in either # optimize mode, or if either optimization level was requested by # the caller. if direct is None: direct = (__debug__ and optimize == 0) # "Indirect" byte-compilation: write a temporary script and then # run it with the appropriate flags. if not direct: try: from tempfile import mkstemp (script_fd, script_name) = mkstemp(".py") except ImportError: from tempfile import mktemp (script_fd, script_name) = None, mktemp(".py") log.info("writing byte-compilation script '%s'", script_name) if not dry_run: if script_fd is not None: script = os.fdopen(script_fd, "w") else: script = open(script_name, "w") script.write("""\ from distutils.util import byte_compile files = [ """) # XXX would be nice to write absolute filenames, just for # safety's sake (script should be more robust in the face of # chdir'ing before running it). But this requires abspath'ing # 'prefix' as well, and that breaks the hack in build_lib's # 'byte_compile()' method that carefully tacks on a trailing # slash (os.sep really) to make sure the prefix here is "just # right". This whole prefix business is rather delicate -- the # problem is that it's really a directory, but I'm treating it # as a dumb string, so trailing slashes and so forth matter. #py_files = map(os.path.abspath, py_files) #if prefix: # prefix = os.path.abspath(prefix) script.write(string.join(map(repr, py_files), ",\n") + "]\n") script.write(""" byte_compile(files, optimize=%r, force=%r, prefix=%r, base_dir=%r, verbose=%r, dry_run=0, direct=1) """ % (optimize, force, prefix, base_dir, verbose)) script.close() cmd = [sys.executable, script_name] if optimize == 1: cmd.insert(1, "-O") elif optimize == 2: cmd.insert(1, "-OO") spawn(cmd, dry_run=dry_run) execute(os.remove, (script_name,), "removing %s" % script_name, dry_run=dry_run) # "Direct" byte-compilation: use the py_compile module to compile # right here, right now. Note that the script generated in indirect # mode simply calls 'byte_compile()' in direct mode, a weird sort of # cross-process recursion. Hey, it works! else: from py_compile import compile for file in py_files: if file[-3:] != ".py": # This lets us be lazy and not filter filenames in # the "install_lib" command. continue # Terminology from the py_compile module: # cfile - byte-compiled file # dfile - purported source filename (same as 'file' by default) cfile = file + (__debug__ and "c" or "o") dfile = file if prefix: if file[:len(prefix)] != prefix: raise ValueError, \ ("invalid prefix: filename %r doesn't start with %r" % (file, prefix)) dfile = dfile[len(prefix):] if base_dir: dfile = os.path.join(base_dir, dfile) cfile_base = os.path.basename(cfile) if direct: if force or newer(file, cfile, dry_run): log.info("byte-compiling %s to %s", file, cfile_base) if not dry_run: compile(file, cfile, dfile) else: log.debug("skipping byte-compilation of %s to %s", file, cfile_base) # byte_compile () def rfc822_escape (header): """Return a version of the string escaped for inclusion in an RFC-822 header, by ensuring there are 8 spaces space after each newline. """ lines = string.split(header, '\n') lines = map(string.strip, lines) header = string.join(lines, '\n' + 8*' ') return header --- NEW FILE: version.py --- # # distutils/version.py # # Implements multiple version numbering conventions for the # Python Module Distribution Utilities. # # $Id: version.py,v 1.1 2005/06/10 06:43:22 anthonybaxter Exp $ # """Provides classes to represent module version numbers (one class for each style of version numbering). There are currently two such classes implemented: StrictVersion and LooseVersion. Every version number class implements the following interface: * the 'parse' method takes a string and parses it to some internal representation; if the string is an invalid version number, 'parse' raises a ValueError exception * the class constructor takes an optional string argument which, if supplied, is passed to 'parse' * __str__ reconstructs the string that was passed to 'parse' (or an equivalent string -- ie. one that will generate an equivalent version number instance) * __repr__ generates Python code to recreate the version number instance * __cmp__ compares the current instance with either another instance of the same class or a string (which will be parsed to an instance of the same class, thus must follow the same rules) """ import string, re from types import StringType class Version: """Abstract base class for version numbering classes. Just provides constructor (__init__) and reproducer (__repr__), because those seem to be the same for all version numbering classes. """ def __init__ (self, vstring=None): if vstring: self.parse(vstring) def __repr__ (self): return "%s ('%s')" % (self.__class__.__name__, str(self)) # Interface for version-number classes -- must be implemented # by the following classes (the concrete ones -- Version should # be treated as an abstract class). # __init__ (string) - create and take same action as 'parse' # (string parameter is optional) # parse (string) - convert a string representation to whatever # internal representation is appropriate for # this style of version numbering # __str__ (self) - convert back to a string; should be very similar # (if not identical to) the string supplied to parse # __repr__ (self) - generate Python code to recreate # the instance # __cmp__ (self, other) - compare two version numbers ('other' may # be an unparsed version string, or another # instance of your version class) class StrictVersion (Version): """Version numbering for anal retentives and software idealists. Implements the standard interface for version number classes as described above. A version number consists of two or three dot-separated numeric components, with an optional "pre-release" tag on the end. The pre-release tag consists of the letter 'a' or 'b' followed by a number. If the numeric components of two version numbers are equal, then one with a pre-release tag will always be deemed earlier (lesser) than one without. The following are valid version numbers (shown in the order that would be obtained by sorting according to the supplied cmp function): 0.4 0.4.0 (these two are equivalent) 0.4.1 0.5a1 0.5b3 0.5 0.9.6 1.0 1.0.4a3 1.0.4b1 1.0.4 The following are examples of invalid version numbers: 1 2.7.2.2 1.3.a4 1.3pl1 1.3c4 The rationale for this version numbering system will be explained in the distutils documentation. """ version_re = re.compile(r'^(\d+) \. (\d+) (\. (\d+))? ([ab](\d+))?$', re.VERBOSE) def parse (self, vstring): match = self.version_re.match(vstring) if not match: raise ValueError, "invalid version number '%s'" % vstring (major, minor, patch, prerelease, prerelease_num) = \ match.group(1, 2, 4, 5, 6) if patch: self.version = tuple(map(string.atoi, [major, minor, patch])) else: self.version = tuple(map(string.atoi, [major, minor]) + [0]) if prerelease: self.prerelease = (prerelease[0], string.atoi(prerelease_num)) else: self.prerelease = None def __str__ (self): if self.version[2] == 0: vstring = string.join(map(str, self.version[0:2]), '.') else: vstring = string.join(map(str, self.version), '.') if self.prerelease: vstring = vstring + self.prerelease[0] + str(self.prerelease[1]) return vstring def __cmp__ (self, other): if isinstance(other, StringType): other = StrictVersion(other) compare = cmp(self.version, other.version) if (compare == 0): # have to compare prerelease # case 1: neither has prerelease; they're equal # case 2: self has prerelease, other doesn't; other is greater # case 3: self doesn't have prerelease, other does: self is greater # case 4: both have prerelease: must compare them! if (not self.prerelease and not other.prerelease): return 0 elif (self.prerelease and not other.prerelease): return -1 elif (not self.prerelease and other.prerelease): return 1 elif (self.prerelease and other.prerelease): return cmp(self.prerelease, other.prerelease) else: # numeric versions don't match -- return compare # prerelease stuff doesn't matter # end class StrictVersion # The rules according to Greg Stein: # 1) a version number has 1 or more numbers separate by a period or by # sequences of letters. If only periods, then these are compared # left-to-right to determine an ordering. # 2) sequences of letters are part of the tuple for comparison and are # compared lexicographically # 3) recognize the numeric components may have leading zeroes # # The LooseVersion class below implements these rules: a version number # string is split up into a tuple of integer and string components, and # comparison is a simple tuple comparison. This means that version # numbers behave in a predictable and obvious way, but a way that might # not necessarily be how people *want* version numbers to behave. There # wouldn't be a problem if people could stick to purely numeric version # numbers: just split on period and compare the numbers as tuples. # However, people insist on putting letters into their version numbers; # the most common purpose seems to be: # - indicating a "pre-release" version # ('alpha', 'beta', 'a', 'b', 'pre', 'p') # - indicating a post-release patch ('p', 'pl', 'patch') # but of course this can't cover all version number schemes, and there's # no way to know what a programmer means without asking him. # # The problem is what to do with letters (and other non-numeric # characters) in a version number. The current implementation does the # obvious and predictable thing: keep them as strings and compare # lexically within a tuple comparison. This has the desired effect if # an appended letter sequence implies something "post-release": # eg. "0.99" < "0.99pl14" < "1.0", and "5.001" < "5.001m" < "5.002". # # However, if letters in a version number imply a pre-release version, # the "obvious" thing isn't correct. Eg. you would expect that # "1.5.1" < "1.5.2a2" < "1.5.2", but under the tuple/lexical comparison # implemented here, this just isn't so. # # Two possible solutions come to mind. The first is to tie the # comparison algorithm to a particular set of semantic rules, as has # been done in the StrictVersion class above. This works great as long # as everyone can go along with bondage and discipline. Hopefully a # (large) subset of Python module programmers will agree that the # particular flavour of bondage and discipline provided by StrictVersion # provides enough benefit to be worth using, and will submit their # version numbering scheme to its domination. The free-thinking # anarchists in the lot will never give in, though, and something needs # to be done to accommodate them. # # Perhaps a "moderately strict" version class could be implemented that # lets almost anything slide (syntactically), and makes some heuristic # assumptions about non-digits in version number strings. This could # sink into special-case-hell, though; if I was as talented and # idiosyncratic as Larry Wall, I'd go ahead and implement a class that # somehow knows that "1.2.1" < "1.2.2a2" < "1.2.2" < "1.2.2pl3", and is # just as happy dealing with things like "2g6" and "1.13++". I don't # think I'm smart enough to do it right though. # # In any case, I've coded the test suite for this module (see # ../test/test_version.py) specifically to fail on things like comparing # "1.2a2" and "1.2". That's not because the *code* is doing anything # wrong, it's because the simple, obvious design doesn't match my # complicated, hairy expectations for real-world version numbers. It # would be a snap to fix the test suite to say, "Yep, LooseVersion does # the Right Thing" (ie. the code matches the conception). But I'd rather # have a conception that matches common notions about version numbers. class LooseVersion (Version): """Version numbering for anarchists and software realists. Implements the standard interface for version number classes as described above. A version number consists of a series of numbers, separated by either periods or strings of letters. When comparing version numbers, the numeric components will be compared numerically, and the alphabetic components lexically. The following are all valid version numbers, in no particular order: 1.5.1 1.5.2b2 161 3.10a 8.02 3.4j 1996.07.12 3.2.pl0 3.1.1.6 2g6 11g 0.960923 2.2beta29 1.13++ 5.5.kw 2.0b1pl0 In fact, there is no such thing as an invalid version number under this scheme; the rules for comparison are simple and predictable, but may not always give the results you want (for some definition of "want"). """ component_re = re.compile(r'(\d+ | [a-z]+ | \.)', re.VERBOSE) def __init__ (self, vstring=None): if vstring: self.parse(vstring) def parse (self, vstring): # I've given up on thinking I can reconstruct the version string # from the parsed tuple -- so I just store the string here for # use by __str__ self.vstring = vstring components = filter(lambda x: x and x != '.', self.component_re.split(vstring)) for i in range(len(components)): try: components[i] = int(components[i]) except ValueError: pass self.version = components def __str__ (self): return self.vstring def __repr__ (self): return "LooseVersion ('%s')" % str(self) def __cmp__ (self, other): if isinstance(other, StringType): other = LooseVersion(other) return cmp(self.version, other.version) # end class LooseVersion --- NEW FILE: versionpredicate.py --- """Module for parsing and testing package version predicate strings. """ import re import distutils.version import operator re_validPackage = re.compile(r"(?i)^\s*([a-z_]\w*(?:\.[a-z_]\w*)*)(.*)") # (package) (rest) re_paren = re.compile(r"^\s*\((.*)\)\s*$") # (list) inside of parentheses re_splitComparison = re.compile(r"^\s*(<=|>=|<|>|!=|==)\s*([^\s,]+)\s*$") # (comp) (version) def splitUp(pred): """Parse a single version comparison. Return (comparison string, StrictVersion) """ res = re_splitComparison.match(pred) if not res: raise ValueError("bad package restriction syntax: %r" % pred) comp, verStr = res.groups() return (comp, distutils.version.StrictVersion(verStr)) compmap = {"<": operator.lt, "<=": operator.le, "==": operator.eq, ">": operator.gt, ">=": operator.ge, "!=": operator.ne} class VersionPredicate: """Parse and test package version predicates. >>> v = VersionPredicate('pyepat.abc (>1.0, <3333.3a1, !=1555.1b3)') The `name` attribute provides the full dotted name that is given:: >>> v.name 'pyepat.abc' The str() of a `VersionPredicate` provides a normalized human-readable version of the expression:: >>> print v pyepat.abc (> 1.0, < 3333.3a1, != 1555.1b3) The `satisfied_by()` method can be used to determine with a given version number is included in the set described by the version restrictions:: >>> v.satisfied_by('1.1') True >>> v.satisfied_by('1.4') True >>> v.satisfied_by('1.0') False >>> v.satisfied_by('4444.4') False >>> v.satisfied_by('1555.1b3') False `VersionPredicate` is flexible in accepting extra whitespace:: >>> v = VersionPredicate(' pat( == 0.1 ) ') >>> v.name 'pat' >>> v.satisfied_by('0.1') True >>> v.satisfied_by('0.2') False If any version numbers passed in do not conform to the restrictions of `StrictVersion`, a `ValueError` is raised:: >>> v = VersionPredicate('p1.p2.p3.p4(>=1.0, <=1.3a1, !=1.2zb3)') Traceback (most recent call last): ... ValueError: invalid version number '1.2zb3' It the module or package name given does not conform to what's allowed as a legal module or package name, `ValueError` is raised:: >>> v = VersionPredicate('foo-bar') Traceback (most recent call last): ... ValueError: expected parenthesized list: '-bar' >>> v = VersionPredicate('foo bar (12.21)') Traceback (most recent call last): ... ValueError: expected parenthesized list: 'bar (12.21)' """ def __init__(self, versionPredicateStr): """Parse a version predicate string. """ # Fields: # name: package name # pred: list of (comparison string, StrictVersion) versionPredicateStr = versionPredicateStr.strip() if not versionPredicateStr: raise ValueError("empty package restriction") match = re_validPackage.match(versionPredicateStr) if not match: raise ValueError("bad package name in %r" % versionPredicateStr) self.name, paren = match.groups() paren = paren.strip() if paren: match = re_paren.match(paren) if not match: raise ValueError("expected parenthesized list: %r" % paren) str = match.groups()[0] self.pred = [splitUp(aPred) for aPred in str.split(",")] if not self.pred: raise ValueError("empty parenthesized list in %r" % versionPredicateStr) else: self.pred = [] def __str__(self): if self.pred: seq = [cond + " " + str(ver) for cond, ver in self.pred] return self.name + " (" + ", ".join(seq) + ")" else: return self.name def satisfied_by(self, version): """True if version is compatible with all the predicates in self. The parameter version must be acceptable to the StrictVersion constructor. It may be either a string or StrictVersion. """ for cond, ver in self.pred: if not compmap[cond](version, ver): return False return True _provision_rx = None def split_provision(value): """Return the name and optional version number of a provision. The version number, if given, will be returned as a `StrictVersion` instance, otherwise it will be `None`. >>> split_provision('mypkg') ('mypkg', None) >>> split_provision(' mypkg( 1.2 ) ') ('mypkg', StrictVersion ('1.2')) """ global _provision_rx if _provision_rx is None: _provision_rx = re.compile( "([a-zA-Z_]\w*(?:\.[a-zA-Z_]\w*)*)(?:\s*\(\s*([^)\s]+)\s*\))?$") value = value.strip() m = _provision_rx.match(value) if not m: raise ValueError("illegal provides specification: %r" % value) ver = m.group(2) or None if ver: ver = distutils.version.StrictVersion(ver) return m.group(1), ver From anthonybaxter at users.sourceforge.net Fri Jun 10 08:48:16 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:48:16 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils/command bdist.py, 1.1, 1.2 install.py, 1.1, 1.2 sdist.py, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25579/distutils/command Modified Files: bdist.py install.py sdist.py Log Message: first bit of getting rid of fancy_getopt Index: bdist.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/command/bdist.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- bdist.py 10 Jun 2005 06:43:22 -0000 1.1 +++ bdist.py 10 Jun 2005 06:48:14 -0000 1.2 @@ -17,12 +17,12 @@ def show_formats (): """Print list of available formats (arguments to "--format" option). """ - from distutils.fancy_getopt import FancyGetopt + from distutils.util import OptionPrettyPrinter formats=[] for format in bdist.format_commands: formats.append(("formats=" + format, None, bdist.format_command[format][1])) - pretty_printer = FancyGetopt(formats) + pretty_printer = OptionPrettyPrinter(formats) pretty_printer.print_help("List of available distribution formats:") Index: install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/command/install.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- install.py 10 Jun 2005 06:43:22 -0000 1.1 +++ install.py 10 Jun 2005 06:48:14 -0000 1.2 @@ -346,18 +346,16 @@ def dump_dirs (self, msg): if DEBUG: - from distutils.fancy_getopt import longopt_xlate print msg + ":" for opt in self.user_options: opt_name = opt[0] if opt_name[-1] == "=": opt_name = opt_name[0:-1] if self.negative_opt.has_key(opt_name): - opt_name = string.translate(self.negative_opt[opt_name], - longopt_xlate) + opt_name = self.negative_opt[opt_name].replace('-','_') val = not getattr(self, opt_name) else: - opt_name = string.translate(opt_name, longopt_xlate) + opt_name = opt_name.replace('-','_') val = getattr(self, opt_name) print " %s: %s" % (opt_name, val) Index: sdist.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/command/sdist.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- sdist.py 10 Jun 2005 06:43:22 -0000 1.1 +++ sdist.py 10 Jun 2005 06:48:14 -0000 1.2 @@ -21,14 +21,14 @@ """Print all possible values for the 'formats' option (used by the "--help-formats" command-line option). """ - from distutils.fancy_getopt import FancyGetopt + from distutils.util import OptionPrettyPrinter from distutils.archive_util import ARCHIVE_FORMATS formats=[] for format in ARCHIVE_FORMATS.keys(): formats.append(("formats=" + format, None, ARCHIVE_FORMATS[format][2])) formats.sort() - pretty_printer = FancyGetopt(formats) + pretty_printer = OptionPrettyPrinter(formats) pretty_printer.print_help( "List of available source distribution formats:") From anthonybaxter at users.sourceforge.net Fri Jun 10 08:48:16 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Thu, 09 Jun 2005 23:48:16 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils ccompiler.py, 1.1, 1.2 cmd.py, 1.1, 1.2 dep_util.py, 1.1, 1.2 dist.py, 1.1, 1.2 util.py, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25579/distutils Modified Files: ccompiler.py cmd.py dep_util.py dist.py util.py Log Message: first bit of getting rid of fancy_getopt Index: ccompiler.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/ccompiler.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- ccompiler.py 10 Jun 2005 06:43:22 -0000 1.1 +++ ccompiler.py 10 Jun 2005 06:48:14 -0000 1.2 @@ -1115,16 +1115,13 @@ """Print list of available compilers (used by the "--help-compiler" options to "build", "build_ext", "build_clib"). """ - # XXX this "knows" that the compiler option it's describing is - # "--compiler", which just happens to be the case for the three - # commands that use it. - from distutils.fancy_getopt import FancyGetopt + from distutils.util import OptionPrettyPrinter compilers = [] for compiler in compiler_class.keys(): compilers.append(("compiler="+compiler, None, compiler_class[compiler][2])) compilers.sort() - pretty_printer = FancyGetopt(compilers) + pretty_printer = OptionPrettyPrinter(compilers) pretty_printer.print_help("List of available compilers:") Index: cmd.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/cmd.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- cmd.py 10 Jun 2005 06:43:22 -0000 1.1 +++ cmd.py 10 Jun 2005 06:48:14 -0000 1.2 @@ -160,13 +160,12 @@ def dump_options (self, header=None, indent=""): - from distutils.fancy_getopt import longopt_xlate if header is None: header = "command options for '%s':" % self.get_command_name() print indent + header indent = indent + " " for (option, _, _) in self.user_options: - option = string.translate(option, longopt_xlate) + option = option.replace('-', '_') if option[-1] == "=": option = option[:-1] value = getattr(self, option) Index: dep_util.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/dep_util.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- dep_util.py 10 Jun 2005 06:43:22 -0000 1.1 +++ dep_util.py 10 Jun 2005 06:48:14 -0000 1.2 @@ -12,15 +12,13 @@ from distutils.errors import DistutilsFileError -def newer (source, target, dry_run=0): +def newer (source, target): """Return true if 'source' exists and is more recently modified than 'target', or if 'source' exists and 'target' doesn't. Return false if both exist and 'target' is the same age or younger than 'source'. Raise DistutilsFileError if 'source' does not exist. """ if not os.path.exists(source): - if dry_run: - return 1 raise DistutilsFileError, "file '%s' does not exist" % source if not os.path.exists(target): return 1 Index: dist.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/dist.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- dist.py 10 Jun 2005 06:43:22 -0000 1.1 +++ dist.py 10 Jun 2005 06:48:14 -0000 1.2 @@ -18,7 +18,7 @@ warnings = None from distutils.errors import * -from distutils.fancy_getopt import FancyGetopt, translate_longopt +from distutils.fancy_getopt import FancyGetopt from distutils.util import check_environ, strtobool, rfc822_escape from distutils import log from distutils.debug import DEBUG @@ -113,8 +113,7 @@ ('obsoletes', None, "print the list of packages/modules made obsolete") ] - display_option_names = map(lambda x: translate_longopt(x[0]), - display_options) + display_option_names = [x[0].replace('-','_') for x in display_options] # negative options are options that exclude other options negative_opt = {'quiet': 'verbose'} @@ -444,6 +443,9 @@ # until we know what the command is. self.commands = [] + print "toplevel_options", toplevel_options + print "display_options", self.display_options + print "negative_opt", self.negative_opt parser = FancyGetopt(toplevel_options + self.display_options) parser.set_negative_aliases(self.negative_opt) parser.set_aliases({'licence': 'license'}) @@ -692,7 +694,7 @@ for (opt, val) in option_order: if val and is_display_option.get(opt): - opt = translate_longopt(opt) + opt = opt.replace('-','_') value = getattr(self.metadata, "get_"+opt)() if opt in ['keywords', 'platforms']: print string.join(value, ',') @@ -897,7 +899,8 @@ for (option, (source, value)) in option_dict.items(): if DEBUG: print " %s = %s (from %s)" % (option, value, source) try: - bool_opts = map(translate_longopt, command_obj.boolean_options) + bool_opts = [ x.replace('-','_') + for x in command_obj.boolean_options] except AttributeError: bool_opts = [] try: Index: util.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/util.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- util.py 10 Jun 2005 06:43:22 -0000 1.1 +++ util.py 10 Jun 2005 06:48:14 -0000 1.2 @@ -444,7 +444,7 @@ cfile_base = os.path.basename(cfile) if direct: - if force or newer(file, cfile, dry_run): + if force or newer(file, cfile): log.info("byte-compiling %s to %s", file, cfile_base) if not dry_run: compile(file, cfile, dfile) @@ -462,3 +462,18 @@ lines = map(string.strip, lines) header = string.join(lines, '\n' + 8*' ') return header + +class OptionPrettyPrinter: + def __init__(self, options): + self.options = options + + def print_help(self, header, file=None): + if file is None: + file = sys.stdout + file.write(header+'\n') + # i18n? + maxlen = max([len(x[0]) for x in self.options]) + fmtstr = ' --%%-%ds %%s\n'%(maxlen+1) + for arg, ign, help in self.options: + file.write(fmtstr%(arg,help)) + From anthonybaxter at users.sourceforge.net Fri Jun 10 09:19:06 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 10 Jun 2005 00:19:06 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils/command command_template, NONE, 1.1 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8308/distutils/command Added Files: command_template Log Message: --- NEW FILE: command_template --- """distutils.command.x Implements the Distutils 'x' command. """ # created 2000/mm/dd, John Doe __revision__ = "$Id: command_template,v 1.1 2005/06/10 07:19:04 anthonybaxter Exp $" from distutils.core import Command class x (Command): # Brief (40-50 characters) description of the command description = "" # List of option tuples: long name, short name (None if no short # name), and help string. user_options = [('', '', ""), ] def initialize_options (self): self. = None self. = None self. = None # initialize_options() def finalize_options (self): if self.x is None: self.x = # finalize_options() def run (self): # run() # class x From jackjansen at users.sourceforge.net Fri Jun 10 12:46:42 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Fri, 10 Jun 2005 03:46:42 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenGenerator.py, 1.16, 1.17 bgenObjectDefinition.py, 1.27, 1.28 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25929 Modified Files: bgenGenerator.py bgenObjectDefinition.py Log Message: Minor tweaks, to allow some (out-of-tree, until successful) tinkering with C++. Index: bgenGenerator.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenGenerator.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- bgenGenerator.py 18 Jul 2004 06:02:01 -0000 1.16 +++ bgenGenerator.py 10 Jun 2005 10:46:40 -0000 1.17 @@ -18,6 +18,7 @@ def __init__(self, name, condition=None): if DEBUG: print "<--", name self.name = name + self.callname = name self.prefix = name self.objecttype = "PyObject" # Type of _self argument to function self.condition = condition @@ -202,7 +203,7 @@ def callit(self): args = "" if self.rv: - s = "%s = %s(" % (self.rv.name, self.name) + s = "%s = %s(" % (self.rv.name, self.callname) else: s = "%s(" % self.name sep = ",\n" + ' '*len(s) @@ -214,9 +215,9 @@ args = args + s if self.rv: Output("%s = %s(%s);", - self.rv.name, self.name, args) + self.rv.name, self.callname, args) else: - Output("%s(%s);", self.name, args) + Output("%s(%s);", self.callname, args) def checkit(self): for arg in self.argumentList: @@ -255,8 +256,7 @@ self.itself = Variable(t0, "_self->ob_itself", SelfMode) self.argumentList.append(self.itself) FunctionGenerator.parseArgumentList(self, args) - - + def _test(): void = None eggs = FunctionGenerator(void, "eggs", Index: bgenObjectDefinition.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenObjectDefinition.py,v retrieving revision 1.27 retrieving revision 1.28 diff -u -d -r1.27 -r1.28 --- bgenObjectDefinition.py 18 Jul 2004 06:02:01 -0000 1.27 +++ bgenObjectDefinition.py 10 Jun 2005 10:46:40 -0000 1.28 @@ -44,12 +44,8 @@ OutHeader2("Object type " + self.name) - sf = self.static and "static " - Output("%sPyTypeObject %s;", sf, self.typename) - Output() - Output("#define %s_Check(x) ((x)->ob_type == &%s || PyObject_TypeCheck((x), &%s))", - self.prefix, self.typename, self.typename) - Output() + self.outputCheck() + Output("typedef struct %s {", self.objecttype) IndentLevel() Output("PyObject_HEAD") @@ -84,6 +80,14 @@ OutHeader2("End object type " + self.name) + def outputCheck(self): + sf = self.static and "static " + Output("%sPyTypeObject %s;", sf, self.typename) + Output() + Output("#define %s_Check(x) ((x)->ob_type == &%s || PyObject_TypeCheck((x), &%s))", + self.prefix, self.typename, self.typename) + Output() + def outputMethodChain(self): Output("%sPyMethodChain %s_chain = { %s_methods, %s };", self.static, self.prefix, self.prefix, self.basechain) From rhettinger at users.sourceforge.net Fri Jun 10 13:05:20 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 10 Jun 2005 04:05:20 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libtokenize.tex,1.5,1.6 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2829/Doc/lib Modified Files: libtokenize.tex Log Message: Add untokenize() function to allow full round-trip tokenization. Should significantly enhance the utility of the module by supporting the creation of tools that modify the token stream and writeback the modified result. Index: libtokenize.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libtokenize.tex,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- libtokenize.tex 29 Jun 2001 23:51:07 -0000 1.5 +++ libtokenize.tex 10 Jun 2005 11:05:18 -0000 1.6 @@ -45,6 +45,9 @@ provides the same interface as the \method{readline()} method of built-in file objects (see section~\ref{bltin-file-objects}). Each call to the function should return one line of input as a string. + Alternately, \var{readline} may be a callable object that signals + completion by raising \exception{StopIteration}. + \versionchanged[Added StopIteration support]{2.5} The second parameter, \var{tokeneater}, must also be a callable object. It is called once for each token, with five arguments, @@ -65,3 +68,52 @@ are generated when a logical line of code is continued over multiple physical lines. \end{datadesc} + +Another function is provided to reverse the tokenization process. +This is useful for creating tools that tokenize a script, modify +the token stream, and write back the modified script. + +\begin{funcdesc}{untokenize}{iterable} + Converts tokens back into Python source code. The \variable{iterable} + must return sequences with at least two elements, the token type and + the token string. Any additional sequence elements are ignored. + + The reconstructed script is returned as a single string. The + result is guaranteed to tokenize back to match the input so that + the conversion is lossless and round-trips are assured. The + guarantee applies only to the token type and token string as + the spacing between tokens (column positions) may change. + \versionadded{2.5} +\end{funcdesc} + +Example of a script re-writer that transforms float literals into +Decimal objects: +\begin{verbatim} +def decistmt(s): + """Substitute Decimals for floats in a string of statements. + + >>> from decimal import Decimal + >>> s = 'print +21.3e-5*-.1234/81.7' + >>> decistmt(s) + "print +Decimal ('21.3e-5')*-Decimal ('.1234')/Decimal ('81.7')" + + >>> exec(s) + -3.21716034272e-007 + >>> exec(decistmt(s)) + -3.217160342717258261933904529E-7 + + """ + result = [] + g = generate_tokens(StringIO(s).readline) # tokenize the string + for toknum, tokval, _, _, _ in g: + if toknum == NUMBER and '.' in tokval: # replace NUMBER tokens + result.extend([ + (NAME, 'Decimal'), + (OP, '('), + (STRING, repr(tokval)), + (OP, ')') + ]) + else: + result.append((toknum, tokval)) + return untokenize(result) +\end{verbatim} From rhettinger at users.sourceforge.net Fri Jun 10 13:05:20 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 10 Jun 2005 04:05:20 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1299,1.1300 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2829/Misc Modified Files: NEWS Log Message: Add untokenize() function to allow full round-trip tokenization. Should significantly enhance the utility of the module by supporting the creation of tools that modify the token stream and writeback the modified result. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1299 retrieving revision 1.1300 diff -u -d -r1.1299 -r1.1300 --- NEWS 4 Jun 2005 09:20:03 -0000 1.1299 +++ NEWS 10 Jun 2005 11:05:16 -0000 1.1300 @@ -141,6 +141,11 @@ Library ------- +- The tokenize module has a new untokenize() function to support a full + roundtrip from lexed tokens back to Python sourcecode. In addition, + the generate_tokens() function now accepts a callable argument that + terminates by raising StopIteration. + - Bug #1196315: fix weakref.WeakValueDictionary constructor. - Bug #1213894: os.path.realpath didn't resolve symlinks that were the first From rhettinger at users.sourceforge.net Fri Jun 10 13:05:21 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 10 Jun 2005 04:05:21 -0700 Subject: [Python-checkins] python/dist/src/Lib tokenize.py,1.36,1.37 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2829/Lib Modified Files: tokenize.py Log Message: Add untokenize() function to allow full round-trip tokenization. Should significantly enhance the utility of the module by supporting the creation of tools that modify the token stream and writeback the modified result. Index: tokenize.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/tokenize.py,v retrieving revision 1.36 retrieving revision 1.37 diff -u -d -r1.36 -r1.37 --- tokenize.py 2 Aug 2004 06:09:53 -0000 1.36 +++ tokenize.py 10 Jun 2005 11:05:18 -0000 1.37 @@ -31,7 +31,7 @@ import token __all__ = [x for x in dir(token) if x[0] != '_'] + ["COMMENT", "tokenize", - "generate_tokens", "NL"] + "generate_tokens", "NL", "untokenize"] del x del token @@ -159,12 +159,55 @@ for token_info in generate_tokens(readline): tokeneater(*token_info) + +def untokenize(iterable): + """Transform tokens back into Python source code. + + Each element returned by the iterable must be a token sequence + with at least two elements, a token number and token value. + + Round-trip invariant: + # Output text will tokenize the back to the input + t1 = [tok[:2] for tok in generate_tokens(f.readline)] + newcode = untokenize(t1) + readline = iter(newcode.splitlines(1)).next + t2 = [tok[:2] for tokin generate_tokens(readline)] + assert t1 == t2 + """ + + startline = False + indents = [] + toks = [] + toks_append = toks.append + for tok in iterable: + toknum, tokval = tok[:2] + + if toknum == NAME: + tokval += ' ' + + if toknum == INDENT: + indents.append(tokval) + continue + elif toknum == DEDENT: + indents.pop() + continue + elif toknum in (NEWLINE, COMMENT, NL): + startline = True + elif startline and indents: + toks_append(indents[-1]) + startline = False + toks_append(tokval) + return ''.join(toks) + + def generate_tokens(readline): """ The generate_tokens() generator requires one argment, readline, which must be a callable object which provides the same interface as the readline() method of built-in file objects. Each call to the function - should return one line of input as a string. + should return one line of input as a string. Alternately, readline + can be a callable function terminating with StopIteration: + readline = open(myfile).next # Example of alternate readline The generator produces 5-tuples with these members: the token type; the token string; a 2-tuple (srow, scol) of ints specifying the row and @@ -180,7 +223,10 @@ indents = [0] while 1: # loop over lines in stream - line = readline() + try: + line = readline() + except StopIteration: + line = '' lnum = lnum + 1 pos, max = 0, len(line) From rhettinger at users.sourceforge.net Fri Jun 10 13:05:21 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 10 Jun 2005 04:05:21 -0700 Subject: [Python-checkins] python/dist/src/Lib/test regrtest.py, 1.167, 1.168 test_tokenize.py, 1.9, 1.10 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2829/Lib/test Modified Files: regrtest.py test_tokenize.py Log Message: Add untokenize() function to allow full round-trip tokenization. Should significantly enhance the utility of the module by supporting the creation of tools that modify the token stream and writeback the modified result. Index: regrtest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/regrtest.py,v retrieving revision 1.167 retrieving revision 1.168 diff -u -d -r1.167 -r1.168 --- regrtest.py 28 Mar 2005 01:08:03 -0000 1.167 +++ regrtest.py 10 Jun 2005 11:05:19 -0000 1.168 @@ -91,7 +91,9 @@ compiler - Test the compiler package by compiling all the source in the standard library and test suite. This takes - a long time. + a long time. Enabling this resource also allows + test_tokenize to verify round-trip lexing on every + file in the test library. subprocess Run all tests for the subprocess module. Index: test_tokenize.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_tokenize.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- test_tokenize.py 12 May 2003 19:42:04 -0000 1.9 +++ test_tokenize.py 10 Jun 2005 11:05:19 -0000 1.10 @@ -1,12 +1,82 @@ -from test.test_support import verbose, findfile -import tokenize, os, sys +from test.test_support import verbose, findfile, is_resource_enabled +import os, glob, random +from tokenize import (tokenize, generate_tokens, untokenize, + NUMBER, NAME, OP, STRING) if verbose: print 'starting...' f = file(findfile('tokenize_tests' + os.extsep + 'txt')) -tokenize.tokenize(f.readline) +tokenize(f.readline) f.close() + + +###### Test roundtrip for untokenize ########################## + +def test_roundtrip(f): + ## print 'Testing:', f + f = file(f) + try: + fulltok = list(generate_tokens(f.readline)) + finally: + f.close() + + t1 = [tok[:2] for tok in fulltok] + newtext = untokenize(t1) + readline = iter(newtext.splitlines(1)).next + t2 = [tok[:2] for tok in generate_tokens(readline)] + assert t1 == t2 + + +f = findfile('tokenize_tests' + os.extsep + 'txt') +test_roundtrip(f) + +testdir = os.path.dirname(f) or os.curdir +testfiles = glob.glob(testdir + os.sep + 'test*.py') +if not is_resource_enabled('compiler'): + testfiles = random.sample(testfiles, 10) + +for f in testfiles: + test_roundtrip(f) + + + +###### Test example in the docs ############################### + +from decimal import Decimal +from cStringIO import StringIO + +def decistmt(s): + """Substitute Decimals for floats in a string of statements. + + >>> from decimal import Decimal + >>> s = 'print +21.3e-5*-.1234/81.7' + >>> decistmt(s) + "print +Decimal ('21.3e-5')*-Decimal ('.1234')/Decimal ('81.7')" + + >>> exec(s) + -3.21716034272e-007 + >>> exec(decistmt(s)) + -3.217160342717258261933904529E-7 + + """ + result = [] + g = generate_tokens(StringIO(s).readline) # tokenize the string + for toknum, tokval, _, _, _ in g: + if toknum == NUMBER and '.' in tokval: # replace NUMBER tokens + result.extend([ + (NAME, 'Decimal'), + (OP, '('), + (STRING, repr(tokval)), + (OP, ')') + ]) + else: + result.append((toknum, tokval)) + return untokenize(result) + +import doctest +doctest.testmod() + if verbose: print 'finished' From arigo at users.sourceforge.net Fri Jun 10 17:29:24 2005 From: arigo at users.sourceforge.net (arigo@users.sourceforge.net) Date: Fri, 10 Jun 2005 08:29:24 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libaudioop.tex, 1.20, 1.21 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13951/lib Modified Files: libaudioop.tex Log Message: The functions audioop.lin2adpcm3() and audioop.adpcm32lin() were removed from this module in Jan 1994, but still documented. Index: libaudioop.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libaudioop.tex,v retrieving revision 1.20 retrieving revision 1.21 diff -u -d -r1.20 -r1.21 --- libaudioop.tex 6 Dec 2001 23:16:09 -0000 1.20 +++ libaudioop.tex 10 Jun 2005 15:29:22 -0000 1.21 @@ -42,11 +42,6 @@ has the width specified in \var{width}. \end{funcdesc} -\begin{funcdesc}{adpcm32lin}{adpcmfragment, width, state} -Decode an alternative 3-bit ADPCM code. See \function{lin2adpcm3()} -for details. -\end{funcdesc} - \begin{funcdesc}{avg}{fragment, width} Return the average over all samples in the fragment. \end{funcdesc} @@ -122,13 +117,6 @@ packed 2 4-bit values per byte. \end{funcdesc} -\begin{funcdesc}{lin2adpcm3}{fragment, width, state} -This is an alternative ADPCM coder that uses only 3 bits per sample. -It is not compatible with the Intel/DVI ADPCM coder and its output is -not packed (due to laziness on the side of the author). Its use is -discouraged. -\end{funcdesc} - \begin{funcdesc}{lin2ulaw}{fragment, width} Convert samples in the audio fragment to u-LAW encoding and return this as a Python string. u-LAW is an audio encoding format whereby From birkenfeld at users.sourceforge.net Fri Jun 10 19:15:20 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 10 Jun 2005 10:15:20 -0700 Subject: [Python-checkins] python/dist/src/Lib py_compile.py,1.26,1.27 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4367/Lib Modified Files: py_compile.py Log Message: Patch #1171150: add a newline to py_compile error output Index: py_compile.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/py_compile.py,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- py_compile.py 27 Feb 2003 23:18:46 -0000 1.26 +++ py_compile.py 10 Jun 2005 17:15:18 -0000 1.27 @@ -128,7 +128,7 @@ if doraise: raise py_exc else: - sys.stderr.write(py_exc.msg) + sys.stderr.write(py_exc.msg + '\n') return if cfile is None: cfile = file + (__debug__ and 'c' or 'o') From anthonybaxter at users.sourceforge.net Fri Jun 10 19:35:26 2005 From: anthonybaxter at users.sourceforge.net (anthonybaxter@users.sourceforge.net) Date: Fri, 10 Jun 2005 10:35:26 -0700 Subject: [Python-checkins] python/nondist/sandbox/distutils_refactor/distutils/tests test_options.py, NONE, 1.1 test_dist.py, 1.1, 1.2 test_versionpredicate.py, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/tests In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15190/tests Modified Files: test_dist.py test_versionpredicate.py Added Files: test_options.py Log Message: make tests runnable, add more options tests --- NEW FILE: test_options.py --- """Tests for distutils.command.build_py.""" import os import unittest from distutils.core import Distribution from distutils.tests import support class OptionsParsingTestCase(support.LoggingSilencer, unittest.TestCase): def test_basic_options(self): ae = self.assertEquals d = Distribution() ae(d.verbose, 1) # Why doesn't Distribution have a default commands = [] ?? ae(hasattr(d, 'commands'), False) d.script_name = 'test' d.script_args = ['--quiet', 'build', '--force', '--build-temp=/tem/por/rary', 'install','--prefix=/foo/bar/baz'] d.parse_command_line() ae(d.verbose, 0) ae(d.commands, ['build','install',]) io = d.get_option_dict('install') ae(io.keys(), ['prefix']) ae(io['prefix'], ('command line', '/foo/bar/baz')) bo = d.get_option_dict('build') ae(sorted(bo.keys()), ['build_temp', 'force',]) ae(bo['force'], ('command line', 1)) ae(bo['build_temp'], ('command line', '/tem/por/rary')) def test_aliases_and_negatives(self): ae = self.assertEquals d = Distribution() ae(d.verbose, 1) d.script_name = 'test' d.script_args = ['--quiet', '--licence', 'install'] d.parse_command_line() ae(d.verbose, 0) d = Distribution() ae(d.verbose, 1) d.script_name = 'test' d.script_args = ['--verbose', '--license', 'install'] d.parse_command_line() ae(d.verbose, 2) d = Distribution() ae(d.verbose, 1) d.script_name = 'test' d.script_args = ['--verbose', '--verbose', '--verbose', 'install'] d.parse_command_line() ae(d.verbose, 4) def test_command_packages(self): ae = self.assertEquals d = Distribution() d.script_name = 'test' d.script_args = ['--command-packages=bar,baz', 'install'] d.parse_command_line() ae(d.get_command_packages(), ['distutils.command', 'bar', 'baz']) def test_suite(): return unittest.makeSuite(OptionsParsingTestCase) if __name__ == "__main__": unittest.main(defaultTest="test_suite") Index: test_dist.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/tests/test_dist.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- test_dist.py 10 Jun 2005 06:43:22 -0000 1.1 +++ test_dist.py 10 Jun 2005 17:35:23 -0000 1.2 @@ -67,6 +67,9 @@ self.assertEqual(d.get_command_packages(), ["distutils.command", "foo.bar", "distutils.tests"]) cmd = d.get_command_obj("test_dist") + # Nasty. If this is run directly, test_dist is __main__.test_dist + if __name__ == "__main__": + from distutils.tests.test_dist import test_dist self.assert_(isinstance(cmd, test_dist)) self.assertEqual(cmd.sample_option, "sometext") @@ -187,3 +190,7 @@ suite.addTest(unittest.makeSuite(DistributionTestCase)) suite.addTest(unittest.makeSuite(MetadataTestCase)) return suite + +if __name__ == "__main__": + unittest.main(defaultTest="test_suite") + Index: test_versionpredicate.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/distutils_refactor/distutils/tests/test_versionpredicate.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- test_versionpredicate.py 10 Jun 2005 06:43:22 -0000 1.1 +++ test_versionpredicate.py 10 Jun 2005 17:35:24 -0000 1.2 @@ -7,3 +7,7 @@ def test_suite(): return doctest.DocTestSuite(distutils.versionpredicate) + +if __name__ == "__main__": + unittest.main(defaultTest="test_suite") + From birkenfeld at users.sourceforge.net Fri Jun 10 21:55:42 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 10 Jun 2005 12:55:42 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.155,1.156 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22979/Doc/lib Modified Files: libos.tex Log Message: Clarify docs about os.popen[234]. Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.155 retrieving revision 1.156 diff -u -d -r1.155 -r1.156 --- libos.tex 2 Jun 2005 13:09:30 -0000 1.155 +++ libos.tex 10 Jun 2005 19:55:35 -0000 1.156 @@ -354,7 +354,7 @@ \end{funcdesc} -For each of these \function{popen()} variants, if \var{bufsize} is +For each of the following \function{popen()} variants, if \var{bufsize} is specified, it specifies the buffer size for the I/O pipes. \var{mode}, if provided, should be the string \code{'b'} or \code{'t'}; on Windows this is needed to determine whether the file @@ -366,7 +366,7 @@ intervention (as with \function{os.spawnv()}). If \var{cmd} is a string it will be passed to the shell (as with \function{os.system()}). -These methods do not make it possible to retrieve the return code from +These methods do not make it possible to retrieve the exit status from the child processes. The only way to control the input and output streams and also retrieve the return codes is to use the \class{Popen3} and \class{Popen4} classes from the \refmodule{popen2} From birkenfeld at users.sourceforge.net Fri Jun 10 21:55:48 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Fri, 10 Jun 2005 12:55:48 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex, 1.146.2.2, 1.146.2.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23007/Doc/lib Modified Files: Tag: release24-maint libos.tex Log Message: Clarify docs about os.popen[234]. Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.146.2.2 retrieving revision 1.146.2.3 diff -u -d -r1.146.2.2 -r1.146.2.3 --- libos.tex 13 Feb 2005 22:56:40 -0000 1.146.2.2 +++ libos.tex 10 Jun 2005 19:55:46 -0000 1.146.2.3 @@ -354,7 +354,7 @@ \end{funcdesc} -For each of these \function{popen()} variants, if \var{bufsize} is +For each of the following \function{popen()} variants, if \var{bufsize} is specified, it specifies the buffer size for the I/O pipes. \var{mode}, if provided, should be the string \code{'b'} or \code{'t'}; on Windows this is needed to determine whether the file @@ -366,7 +366,7 @@ intervention (as with \function{os.spawnv()}). If \var{cmd} is a string it will be passed to the shell (as with \function{os.system()}). -These methods do not make it possible to retrieve the return code from +These methods do not make it possible to retrieve the exit status from the child processes. The only way to control the input and output streams and also retrieve the return codes is to use the \class{Popen3} and \class{Popen4} classes from the \refmodule{popen2} From pje at users.sourceforge.net Sun Jun 12 03:12:36 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 11 Jun 2005 18:12:36 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools archive_util.py, NONE, 1.1 sandbox.py, NONE, 1.1 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4594/setuptools Added Files: archive_util.py sandbox.py Log Message: Split setup-running and archive-extraction utilities into separate modules, for easy use by tools other than EasyInstall. --- NEW FILE: archive_util.py --- """Utilities for extracting common archive formats""" __all__ = [ "unpack_archive", "unpack_zipfile", "unpack_tarfile", "default_filter", "UnrecognizedFormat", "extraction_drivers" ] import zipfile, tarfile, os from pkg_resources import ensure_directory class UnrecognizedFormat(RuntimeError): """Couldn't recognize the archive type""" def default_filter(src,dst): """The default progress/filter callback; returns True for all files""" return True def unpack_archive(filename, extract_dir, progress_filter=default_filter, drivers=None ): """Unpack `filename` to `extract_dir`, or raise ``UnrecognizedFormat`` `progress_filter` is a function taking two arguments: a source path internal to the archive ('/'-separated), and a filesystem path where it will be extracted. The callback must return a true value, or else that file or directory will be skipped. The callback can thus be used to report on the progress of the extraction, as well as to filter the items extracted. `drivers`, if supplied, must be a non-empty sequence of functions with the same signature as this function (minus the `drivers` argument), that raise ``UnrecognizedFormat`` if they do not support extracting the designated archive type. The `drivers` are tried in sequence until one is found that does not raise an error, or until all are exhausted (in which case ``UnrecognizedFormat`` is raised). If you do not supply a sequence of drivers, the module's ``extraction_drivers`` constant will be used, which means that ``unpack_zipfile`` and ``unpack_tarfile`` will be tried, in that order. """ for driver in drivers or extraction_drivers: try: driver(filename, extract_dir, progress_filter) except UnrecognizedFormat: continue else: return else: raise UnrecognizedFormat( "Not a recognized archive type: %s" % filename ) def unpack_zipfile(filename, extract_dir, progress_filter=default_filter): """Unpack zip `filename` to `extract_dir` Raises ``UnrecognizedFormat`` if `filename` is not a zipfile (as determined by ``zipfile.is_zipfile()``). See ``unpack_archive()`` for an explanation of the `progress_filter` argument. """ if not zipfile.is_zipfile(filename): raise UnrecognizedFormat("%s is not a zip file" % (filename,)) z = zipfile.ZipFile(filename) try: for info in z.infolist(): name = info.filename # don't extract absolute paths or ones with .. in them if name.startswith('/') or '..' in name: continue target = os.path.join(extract_dir,name) if not progress_filter(name,target): continue if name.endswith('/'): # directory ensure_directory(target) else: # file ensure_directory(target) data = z.read(info.filename) f = open(target,'wb') try: f.write(data) finally: f.close() del data finally: z.close() def unpack_tarfile(filename, extract_dir, progress_filter=default_filter): """Unpack tar/tar.gz/tar.bz2 `filename` to `extract_dir` Raises ``UnrecognizedFormat`` if `filename` is not a tarfile (as determined by ``tarfile.open()``). See ``unpack_archive()`` for an explanation of the `progress_filter` argument. """ try: tarobj = tarfile.open(filename) except tarfile.TarError: raise UnrecognizedFormat( "%s is not a compressed or uncompressed tar file" % (filename,) ) try: tarobj.chown = lambda *args: None # don't do any chowning! for member in tarobj: if member.isfile() or member.isdir(): name = member.name # don't extract absolute paths or ones with .. in them if not name.startswith('/') and '..' not in name: dst = os.path.join(extract_dir, *name.split('/')) if progress_filter(name, dst): tarobj.extract(member,extract_dir) return True finally: tarobj.close() extraction_drivers = unpack_zipfile, unpack_tarfile --- NEW FILE: sandbox.py --- import os, sys, __builtin__ _os = sys.modules[os.name] _open = open __all__ = [ "AbstractSandbox", "DirectorySandbox", "SandboxViolation", "run_setup", ] def run_setup(setup_script, *args): """Run a distutils setup script, sandboxed in its directory""" old_dir = os.getcwd() save_argv = sys.argv[:] save_path = sys.path[:] setup_dir = os.path.abspath(os.path.dirname(setup_script)) try: os.chdir(setup_dir) try: sys.argv[:] = [setup_script]+list(args) sys.path.insert(0, setup_dir) DirectorySandbox(setup_dir).run( lambda: execfile( "setup.py", {'__file__':setup_script, '__name__':'__main__'} ) ) except SystemExit, v: if v.args and v.args[0]: raise # Normal exit, just return finally: os.chdir(old_dir) sys.path[:] = save_path sys.argv[:] = save_argv class AbstractSandbox: """Wrap 'os' module and 'open()' builtin for virtualizing setup scripts""" _active = False def __init__(self): self._attrs = [ name for name in dir(_os) if not name.startswith('_') and hasattr(self,name) ] def _copy(self, source): for name in self._attrs: setattr(os, name, getattr(source,name)) def run(self, func): """Run 'func' under os sandboxing""" try: self._copy(self) __builtin__.open = __builtin__.file = self._open self._active = True return func() finally: self._active = False __builtin__.open = __builtin__.file = _open self._copy(_os) def _mk_dual_path_wrapper(name): original = getattr(_os,name) def wrap(self,src,dst,*args,**kw): if self._active: src,dst = self._remap_pair(name,src,dst,*args,**kw) return original(src,dst,*args,**kw) return wrap for name in ["rename", "link", "symlink"]: if hasattr(_os,name): locals()[name] = _mk_dual_path_wrapper(name) def _mk_single_path_wrapper(name, original=None): original = original or getattr(_os,name) def wrap(self,path,*args,**kw): if self._active: path = self._remap_input(name,path,*args,**kw) return original(path,*args,**kw) return wrap _open = _mk_single_path_wrapper('file', _open) for name in [ "stat", "listdir", "chdir", "open", "chmod", "chown", "mkdir", "remove", "unlink", "rmdir", "utime", "lchown", "chroot", "lstat", "startfile", "mkfifo", "mknod", "pathconf", "access" ]: if hasattr(_os,name): locals()[name] = _mk_single_path_wrapper(name) def _mk_single_with_return(name): original = getattr(_os,name) def wrap(self,path,*args,**kw): if self._active: path = self._remap_input(name,path,*args,**kw) return self._remap_output(name, original(path,*args,**kw)) return original(path,*args,**kw) return wrap for name in ['readlink', 'tempnam']: if hasattr(_os,name): locals()[name] = _mk_single_with_return(name) def _mk_query(name): original = getattr(_os,name) def wrap(self,*args,**kw): retval = original(*args,**kw) if self._active: return self._remap_output(name, retval) return retval return wrap for name in ['getcwd', 'tmpnam']: if hasattr(_os,name): locals()[name] = _mk_query(name) def _validate_path(self,path): """Called to remap or validate any path, whether input or output""" return path def _remap_input(self,operation,path,*args,**kw): """Called for path inputs""" return self._validate_path(path) def _remap_output(self,operation,path): """Called for path outputs""" return self._validate_path(path) def _remap_pair(self,operation,src,dst,*args,**kw): """Called for path pairs like rename, link, and symlink operations""" return ( self._remap_input(operation+'-from',src,*args,**kw), self._remap_input(operation+'-to',dst,*args,**kw) ) class DirectorySandbox(AbstractSandbox): """Restrict operations to a single subdirectory - pseudo-chroot""" write_ops = dict.fromkeys([ "open", "chmod", "chown", "mkdir", "remove", "unlink", "rmdir", "utime", "lchown", "chroot", "mkfifo", "mknod", "tempnam", ]) def __init__(self,sandbox): self._sandbox = os.path.realpath(sandbox) self._prefix = os.path.join(self._sandbox,'') AbstractSandbox.__init__(self) def _violation(self, operation, *args, **kw): raise SandboxViolation(operation, args, kw) def _open(self, path, mode='r', *args, **kw): if mode not in ('r', 'rt', 'rb', 'rU') and not self._ok(path): self._violation("open", path, mode, *args, **kw) return _open(path,mode,*args,**kw) def tmpnam(self): self._violation("tmpnam") def _ok(self,path): active = self._active try: self._active = False realpath = os.path.realpath(path) if realpath==self._sandbox or realpath.startswith(self._prefix): return True finally: self._active = active def _remap_input(self,operation,path,*args,**kw): """Called for path inputs""" if operation in self.write_ops and not self._ok(path): self._violation(operation, os.path.realpath(path), *args, **kw) return path def _remap_pair(self,operation,src,dst,*args,**kw): """Called for path pairs like rename, link, and symlink operations""" if not self._ok(src) or not self._ok(dst): self._violation(operation, src, dst, *args, **kw) return (src,dst) class SandboxViolation(RuntimeError): """A setup script attempted to modify the filesystem outside the sandbox""" def __str__(self): return """SandboxViolation: %s%r %s The package setup script has attempted to modify files on your system that are not within the EasyInstall build area, and has been aborted. This package cannot be safely installed by EasyInstall, and may not support alternate installation locations even if you run its setup script by hand. Please inform the package's author and the EasyInstall maintainers to find out if a fix or workaround is available.""" % self.args From pje at users.sourceforge.net Sun Jun 12 03:12:36 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 11 Jun 2005 18:12:36 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.5, 1.6 easy_install.py, 1.13, 1.14 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4594 Modified Files: EasyInstall.txt easy_install.py Log Message: Split setup-running and archive-extraction utilities into separate modules, for easy use by tools other than EasyInstall. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- EasyInstall.txt 5 Jun 2005 21:55:41 -0000 1.5 +++ EasyInstall.txt 12 Jun 2005 01:12:33 -0000 1.6 @@ -270,6 +270,10 @@ Release Notes/Change History ============================ +Known Issues + * There's no automatic retry for borked Sourceforge mirrors, which can easily + time out or be missing a file. + 0.4a1 * Added ``--scan-url`` and ``--index-url`` options, to scan download pages and search PyPI for needed packages. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- easy_install.py 5 Jun 2005 21:33:51 -0000 1.13 +++ easy_install.py 12 Jun 2005 01:12:33 -0000 1.14 @@ -17,19 +17,17 @@ import pkg_resources import re import zipimport -import zipfile -import tarfile import shutil import urlparse import urllib import tempfile -import __builtin__ +from setuptools.sandbox import run_setup +from setuptools.archive_util import unpack_archive from distutils.sysconfig import get_python_lib from shutil import rmtree # must have, because it can be called from __del__ from pkg_resources import * -_os = sys.modules[os.name] -_open = open + class Opener(urllib.FancyURLopener): def http_error_default(self, url, fp, errcode, errmsg, headers): @@ -39,6 +37,8 @@ return info opener = Opener() + + HREF = re.compile(r"""href\s*=\s*['"]?([^'"> ]+)""", re.I) EXTENSIONS = ".tar.gz .tar.bz2 .tar .zip .tgz".split() @@ -251,7 +251,7 @@ # Anything else, try to extract and build if os.path.isfile(dist_filename): - self._extract_file(dist_filename) + unpack_archive(dist_filename, self.tmpdir) # XXX add progress log # Find the setup.py file from glob import glob @@ -268,7 +268,14 @@ ) setup_script = setups[0] - self._run_setup(setup_script) + from setuptools.command import bdist_egg + sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) + try: + run_setup(setup_script, '-q', 'bdist_egg') + except SystemExit, v: + raise RuntimeError( + "Setup script exited with %s" % (v.args[0],) + ) eggs = [] for egg in glob( @@ -278,95 +285,6 @@ return eggs - - - - - - - - def _extract_zip(self,zipname,extract_dir): - z = zipfile.ZipFile(zipname) - try: - for info in z.infolist(): - name = info.filename - - # don't extract absolute paths or ones with .. in them - if name.startswith('/') or '..' in name: - continue - - target = os.path.join(extract_dir,name) - if name.endswith('/'): - # directory - ensure_directory(target) - else: - # file - ensure_directory(target) - data = z.read(info.filename) - f = open(target,'wb') - try: - f.write(data) - finally: - f.close() - del data - finally: - z.close() - - def _extract_tar(self,tarobj): - try: - tarobj.chown = lambda *args: None # don't do any chowning! - for member in tarobj: - if member.isfile() or member.isdir(): - name = member.name - # don't extract absolute paths or ones with .. in them - if not name.startswith('/') and '..' not in name: - tarobj.extract(member,self.tmpdir) - finally: - tarobj.close() - - - - def _run_setup(self, setup_script): - from setuptools.command import bdist_egg - sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) - old_dir = os.getcwd() - save_argv = sys.argv[:] - save_path = sys.path[:] - try: - os.chdir(os.path.dirname(setup_script)) - try: - sys.argv[:] = [setup_script, '-q', 'bdist_egg'] - sys.path.insert(0,os.getcwd()) - DirectorySandbox(self.tmpdir).run( - lambda: execfile( - "setup.py", - {'__file__':setup_script, '__name__':'__main__'} - ) - ) - except SystemExit, v: - if v.args and v.args[0]: - raise RuntimeError( - "Setup script exited with %s" % (v.args[0],) - ) - finally: - os.chdir(old_dir) - sys.path[:] = save_path - sys.argv[:] = save_argv - - - - - - - - - - - - - - - def install_egg(self, egg_path, zip_ok): destination = os.path.join(self.instdir, os.path.basename(egg_path)) @@ -389,7 +307,7 @@ else: os.mkdir(destination) - self._extract_zip(egg_path, destination) + unpack_archive(egg_path, destination) # XXX add progress?? if os.path.isdir(destination): dist = Distribution.from_filename( @@ -449,20 +367,6 @@ - def _extract_file(self, dist_filename): - if zipfile.is_zipfile(dist_filename): - self._extract_zip(dist_filename, self.tmpdir) - else: - try: - tar = tarfile.open(dist_filename) - except tarfile.TarError: - raise RuntimeError( - "Not a valid tar or zip archive: %s" % dist_filename - ) - else: - self._extract_tar(tar) - - def _download_html(self, url, headers, filename): # Check for a sourceforge URL sf_url = url.startswith('http://prdownloads.') @@ -494,6 +398,16 @@ os.system("svn checkout -q %s %s" % (url, filename)) return filename + + + + + + + + + + def _download_sourceforge(self, source_url, sf_page): """Download package from randomly-selected SourceForge mirror""" @@ -531,6 +445,10 @@ + + + + def installation_report(self, dist): """Helpful installation message for display to package users""" @@ -572,170 +490,6 @@ -class AbstractSandbox: - """Wrap 'os' module and 'open()' builtin for virtualizing setup scripts""" - - _active = False - - def __init__(self): - self._attrs = [ - name for name in dir(_os) - if not name.startswith('_') and hasattr(self,name) - ] - - def _copy(self, source): - for name in self._attrs: - setattr(os, name, getattr(source,name)) - - def run(self, func): - """Run 'func' under os sandboxing""" - try: - self._copy(self) - __builtin__.open = __builtin__.file = self._open - self._active = True - return func() - finally: - self._active = False - __builtin__.open = __builtin__.file = _open - self._copy(_os) - - - def _mk_dual_path_wrapper(name): - original = getattr(_os,name) - def wrap(self,src,dst,*args,**kw): - if self._active: - src,dst = self._remap_pair(name,src,dst,*args,**kw) - return original(src,dst,*args,**kw) - return wrap - - - for name in ["rename", "link", "symlink"]: - if hasattr(_os,name): locals()[name] = _mk_dual_path_wrapper(name) - - - def _mk_single_path_wrapper(name, original=None): - original = original or getattr(_os,name) - def wrap(self,path,*args,**kw): - if self._active: - path = self._remap_input(name,path,*args,**kw) - return original(path,*args,**kw) - return wrap - - _open = _mk_single_path_wrapper('file', _open) - for name in [ - "stat", "listdir", "chdir", "open", "chmod", "chown", "mkdir", - "remove", "unlink", "rmdir", "utime", "lchown", "chroot", "lstat", - "startfile", "mkfifo", "mknod", "pathconf", "access" - ]: - if hasattr(_os,name): locals()[name] = _mk_single_path_wrapper(name) - - - def _mk_single_with_return(name): - original = getattr(_os,name) - def wrap(self,path,*args,**kw): - if self._active: - path = self._remap_input(name,path,*args,**kw) - return self._remap_output(name, original(path,*args,**kw)) - return original(path,*args,**kw) - return wrap - - for name in ['readlink', 'tempnam']: - if hasattr(_os,name): locals()[name] = _mk_single_with_return(name) - - def _mk_query(name): - original = getattr(_os,name) - def wrap(self,*args,**kw): - retval = original(*args,**kw) - if self._active: - return self._remap_output(name, retval) - return retval - return wrap - - for name in ['getcwd', 'tmpnam']: - if hasattr(_os,name): locals()[name] = _mk_query(name) - - def _validate_path(self,path): - """Called to remap or validate any path, whether input or output""" - return path - - def _remap_input(self,operation,path,*args,**kw): - """Called for path inputs""" - return self._validate_path(path) - - def _remap_output(self,operation,path): - """Called for path outputs""" - return self._validate_path(path) - - def _remap_pair(self,operation,src,dst,*args,**kw): - """Called for path pairs like rename, link, and symlink operations""" - return ( - self._remap_input(operation+'-from',src,*args,**kw), - self._remap_input(operation+'-to',dst,*args,**kw) - ) - - -class DirectorySandbox(AbstractSandbox): - """Restrict operations to a single subdirectory - pseudo-chroot""" - - write_ops = dict.fromkeys([ - "open", "chmod", "chown", "mkdir", "remove", "unlink", "rmdir", - "utime", "lchown", "chroot", "mkfifo", "mknod", "tempnam", - ]) - - def __init__(self,sandbox): - self._sandbox = os.path.realpath(sandbox) - self._prefix = os.path.join(self._sandbox,'') - AbstractSandbox.__init__(self) - - def _violation(self, operation, *args, **kw): - raise SandboxViolation(operation, args, kw) - - def _open(self, path, mode='r', *args, **kw): - if mode not in ('r', 'rt', 'rb', 'rU') and not self._ok(path): - self._violation("open", path, mode, *args, **kw) - return _open(path,mode,*args,**kw) - - def tmpnam(self): - self._violation("tmpnam") - - def _ok(self,path): - active = self._active - try: - self._active = False - realpath = os.path.realpath(path) - if realpath==self._sandbox or realpath.startswith(self._prefix): - return True - finally: - self._active = active - - def _remap_input(self,operation,path,*args,**kw): - """Called for path inputs""" - if operation in self.write_ops and not self._ok(path): - self._violation(operation, os.path.realpath(path), *args, **kw) - return path - - def _remap_pair(self,operation,src,dst,*args,**kw): - """Called for path pairs like rename, link, and symlink operations""" - if not self._ok(src) or not self._ok(dst): - self._violation(operation, src, dst, *args, **kw) - return (src,dst) - - -class SandboxViolation(RuntimeError): - """A setup script attempted to modify the filesystem outside the sandbox""" - - def __str__(self): - return """SandboxViolation: %s%r %s - -The package setup script has attempted to modify files on your system -that are not within the EasyInstall build area, and has been aborted. - -This package cannot be safely installed by EasyInstall, and may not -support alternate installation locations even if you run its setup -script by hand. Please inform the package's author and the EasyInstall -maintainers to find out if a fix or workaround is available.""" % self.args - - class PthDistributions(AvailableDistributions): """A .pth file with Distribution paths in it""" From pje at users.sourceforge.net Sun Jun 12 05:07:55 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 11 Jun 2005 20:07:55 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools sandbox.py, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31950/setuptools Modified Files: sandbox.py Log Message: Rebalance responsibilities between PackageIndex, Installer, and main() so that PackageIndex handles all downloading of any kind, Installers can be reused for multiple packages, and main() manages temporary directories and all communication between PackageIndex and Installer. Also, change run_setup to take an argument sequence, because later we will probably need other arguments to control other aspects of run_setup's behavior. Index: sandbox.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/sandbox.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- sandbox.py 12 Jun 2005 01:12:34 -0000 1.1 +++ sandbox.py 12 Jun 2005 03:07:53 -0000 1.2 @@ -8,7 +8,7 @@ ] -def run_setup(setup_script, *args): +def run_setup(setup_script, args): """Run a distutils setup script, sandboxed in its directory""" old_dir = os.getcwd() From pje at users.sourceforge.net Sun Jun 12 05:07:55 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 11 Jun 2005 20:07:55 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.6, 1.7 easy_install.py, 1.14, 1.15 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31950 Modified Files: EasyInstall.txt easy_install.py Log Message: Rebalance responsibilities between PackageIndex, Installer, and main() so that PackageIndex handles all downloading of any kind, Installers can be reused for multiple packages, and main() manages temporary directories and all communication between PackageIndex and Installer. Also, change run_setup to take an argument sequence, because later we will probably need other arguments to control other aspects of run_setup's behavior. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- EasyInstall.txt 12 Jun 2005 01:12:33 -0000 1.6 +++ EasyInstall.txt 12 Jun 2005 03:07:36 -0000 1.7 @@ -274,6 +274,23 @@ * There's no automatic retry for borked Sourceforge mirrors, which can easily time out or be missing a file. +0.4a2 + * Use ``urllib2`` instead of ``urllib``, to allow use of ``https:`` URLs if + Python includes SSL support. + + * All downloads are now managed by the ``PackageIndex`` class (which is now + subclassable and replaceable), so that embedders can more easily override + download logic, give download progress reports, etc. + + * The ``Installer`` class no longer handles downloading, manages a temporary + directory, or tracks the ``zip_ok`` option. Downloading is now handled + by ``PackageIndex``, and the latter two are now managed by ``main()``. + + * There is a new ``setuptools.sandbox.run_setup()`` API to invoke a setup + script in a directory sandbox, and a new ``setuptools.archive_util`` module + with an ``unpack_archive()`` API. These were split out of EasyInstall to + allow reuse by other tools and applications. + 0.4a1 * Added ``--scan-url`` and ``--index-url`` options, to scan download pages and search PyPI for needed packages. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- easy_install.py 12 Jun 2005 01:12:33 -0000 1.14 +++ easy_install.py 12 Jun 2005 03:07:37 -0000 1.15 @@ -19,23 +19,23 @@ import zipimport import shutil import urlparse -import urllib +import urllib2 import tempfile from setuptools.sandbox import run_setup from setuptools.archive_util import unpack_archive from distutils.sysconfig import get_python_lib -from shutil import rmtree # must have, because it can be called from __del__ from pkg_resources import * -class Opener(urllib.FancyURLopener): - def http_error_default(self, url, fp, errcode, errmsg, headers): - """Default error handling -- don't raise an exception.""" - info = urllib.addinfourl(fp, headers, "http:" + url) - info.status, info.reason = errcode, errmsg - return info -opener = Opener() + + + + + + + + @@ -46,7 +46,7 @@ """Yield egg or source distribution objects that might be found at a URL""" path = urlparse.urlparse(url)[2] - base = urllib.unquote(path.split('/')[-1]) + base = urllib2.unquote(path.split('/')[-1]) if base.endswith('.egg'): dist = Distribution.from_filename(base, metadata) @@ -71,7 +71,7 @@ # compare lower than any numeric version number, and is therefore unlikely # to match a request for it. It's still a potential problem, though, and # in the long run PyPI and the distutils should go for "safe" names and - # versions in source distribution names. + # versions in distribution archive names (sdist and bdist). parts = base.split('-') for p in range(1,len(parts)+1): @@ -105,7 +105,7 @@ # don't need the actual page return - f = opener.open(url) + f = self.open_url(url) self.fetched_urls[url] = self.fetched_urls[f.url] = True if 'html' not in f.headers['content-type'].lower(): f.close() # not html, we can't process it @@ -121,7 +121,7 @@ link = urlparse.urljoin(base, match.group(1)) self.process_url(link) - def find_packages(self,requirement): + def find_packages(self,requirement): self.scan_url(self.index_url + requirement.distname) if not self.package_pages.get(requirement.key): # We couldn't find the target package, so search the index page too @@ -134,13 +134,13 @@ def scan(link): if link.startswith(self.index_url): parts = map( - urllib.unquote, link[len(self.index_url):].split('/') + urllib2.unquote, link[len(self.index_url):].split('/') ) if len(parts)==2: # it's a package page, sanitize and index it pkg = safe_name(parts[0]) ver = safe_version(parts[1]) - self.package_pages.setdefault(pkg.lower(),{})[link] = True + self.package_pages.setdefault(pkg.lower(),{})[link] = True if url==self.index_url or 'Index of Packages' in page: # process an index page into the package-page index for match in HREF.finditer(page): @@ -162,69 +162,25 @@ if dist in requirement: return dist -class Installer: - """Manage a download/build/install process""" - - pth_file = None - cleanup = False - - def __init__(self, - instdir=None, zip_ok=False, multi=None, tmpdir=None, index=None - ): - if index is None: - index = AvailableDistributions() - if tmpdir is None: - tmpdir = tempfile.mkdtemp(prefix="easy_install-") - self.cleanup = True - elif not os.path.isdir(tmpdir): - os.makedirs(tmpdir) - self.tmpdir = os.path.realpath(tmpdir) - - site_packages = get_python_lib() - if instdir is None or self.samefile(site_packages,instdir): - instdir = site_packages - self.pth_file = PthDistributions( - os.path.join(instdir,'easy-install.pth') - ) - elif multi is None: - multi = True - - elif not multi: - # explicit false, raise an error - raise RuntimeError( - "Can't do single-version installs outside site-packages" - ) - self.index = index - self.instdir = instdir - self.zip_ok = zip_ok - self.multi = multi - - def close(self): - if self.cleanup and os.path.isdir(self.tmpdir): - rmtree(self.tmpdir,True) + def download(self, spec, tmpdir): + """Locate and/or download `spec`, returning a local filename - def __del__(self): - self.close() + `spec` may be a ``Requirement`` object, or a string containing a URL, + an existing local filename, or a package/version requirement spec + (i.e. the string form of a ``Requirement`` object). - def samefile(self,p1,p2): - if hasattr(os.path,'samefile') and ( - os.path.exists(p1) and os.path.exists(p2) - ): - return os.path.samefile(p1,p2) - return ( - os.path.normpath(os.path.normcase(p1)) == - os.path.normpath(os.path.normcase(p2)) - ) + If necessary, the requirement is searched for in the package index. + If the download is successful, the return value is a local file path, + and it is a subpath of `tmpdir` if the distribution had to be + downloaded. If no matching distribution is found, return ``None``. + Various errors may be raised if a problem occurs during downloading. + """ - def download(self, spec): - """Locate and/or download or `spec`, returning a local filename""" - if isinstance(spec,Requirement): - pass - else: + if not isinstance(spec,Requirement): scheme = URL_SCHEME(spec) if scheme: - # It's a url, download it to self.tmpdir - return self._download_url(scheme.group(1), spec) + # It's a url, download it to tmpdir + return self._download_url(scheme.group(1), spec, tmpdir) elif os.path.exists(spec): # Existing file or directory, just return it @@ -239,127 +195,89 @@ ) # process a Requirement - dist = self.index.best_match(spec,[]) + dist = self.best_match(spec,[]) if dist is not None: - return self.download(dist.path) + return self.download(dist.path, tmpdir) + return None - def install_eggs(self, dist_filename): - # .egg dirs or files are already built, so just return them - if dist_filename.lower().endswith('.egg'): - return [self.install_egg(dist_filename,True)] - # Anything else, try to extract and build - if os.path.isfile(dist_filename): - unpack_archive(dist_filename, self.tmpdir) # XXX add progress log - # Find the setup.py file - from glob import glob - setup_script = os.path.join(self.tmpdir, 'setup.py') - if not os.path.exists(setup_script): - setups = glob(os.path.join(self.tmpdir, '*', 'setup.py')) - if not setups: - raise RuntimeError( - "Couldn't find a setup script in %s" % dist_filename - ) - if len(setups)>1: + dl_blocksize = 8192 + + def _download_to(self, url, filename): + # Download the file + fp, tfp = None, None + try: + fp = self.open_url(url) + if isinstance(fp, urllib2.HTTPError): raise RuntimeError( - "Multiple setup scripts in %s" % dist_filename + "Can't download %s: %s %s" % (url, fp.code,fp.msg) ) - setup_script = setups[0] - - from setuptools.command import bdist_egg - sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) - try: - run_setup(setup_script, '-q', 'bdist_egg') - except SystemExit, v: - raise RuntimeError( - "Setup script exited with %s" % (v.args[0],) - ) - eggs = [] - for egg in glob( - os.path.join(os.path.dirname(setup_script),'dist','*.egg') - ): - eggs.append(self.install_egg(egg, self.zip_ok)) + headers = fp.info() + blocknum = 0 + bs = self.dl_blocksize + size = -1 - return eggs + if "content-length" in headers: + size = int(headers["Content-Length"]) + self.reporthook(url, filename, blocknum, bs, size) - def install_egg(self, egg_path, zip_ok): + tfp = open(filename,'wb') + while True: + block = fp.read(bs) + if block: + tfp.write(block) + blocknum += 1 + self.reporthook(url, filename, blocknum, bs, size) + else: + break + return headers - destination = os.path.join(self.instdir, os.path.basename(egg_path)) - ensure_directory(destination) + finally: + if fp: fp.close() + if tfp: tfp.close() - if not self.samefile(egg_path, destination): - if os.path.isdir(destination): - shutil.rmtree(destination) - elif os.path.isfile(destination): - os.unlink(destination) + def reporthook(self, url, filename, blocknum, blksize, size): + pass # no-op - if zip_ok: - if egg_path.startswith(self.tmpdir): - shutil.move(egg_path, destination) - else: - shutil.copy2(egg_path, destination) - elif os.path.isdir(egg_path): - shutil.move(egg_path, destination) - else: - os.mkdir(destination) - unpack_archive(egg_path, destination) # XXX add progress?? + def open_url(self, url): + try: + return urllib2.urlopen(url) + except urllib2.HTTPError, v: + return v + except urllib2.URLError, v: + raise RuntimeError("Download error: %s" % v.reason) - if os.path.isdir(destination): - dist = Distribution.from_filename( - destination, metadata=PathMetadata( - destination, os.path.join(destination,'EGG-INFO') - ) - ) - else: - metadata = EggMetadata(zipimport.zipimporter(destination)) - dist = Distribution.from_filename(destination,metadata=metadata) - self.index.add(dist) - if self.pth_file is not None: - map(self.pth_file.remove, self.pth_file.get(dist.key,())) # drop old - if not self.multi: - self.pth_file.add(dist) # add new - self.pth_file.save() - return dist - def _download_url(self, scheme, url): + def _download_url(self, scheme, url, tmpdir): # Determine download filename - name = filter(None,urlparse.urlparse(url)[2].split('/'))[-1] - - while '..' in name: - name = name.replace('..','.').replace('\\','_') + # + name = filter(None,urlparse.urlparse(url)[2].split('/')) + if name: + name = name[-1] + while '..' in name: + name = name.replace('..','.').replace('\\','_') + else: + name = "__downloaded__" # default if URL has no path contents - filename = os.path.join(self.tmpdir,name) + filename = os.path.join(tmpdir,name) + # Download the file + # if scheme=='svn' or scheme.startswith('svn+'): return self._download_svn(url, filename) - - # Download the file - class _opener(urllib.FancyURLopener): - http_error_default = urllib.URLopener.http_error_default - - try: - filename,headers = _opener().retrieve( - url,filename - ) - except IOError,v: - if v.args and v.args[0]=='http error': - raise RuntimeError( - "Download error: %s %s" % v.args[1:3] - ) + else: + headers = self._download_to(url, filename) + if 'html' in headers['content-type'].lower(): + return self._download_html(url, headers, filename, tmpdir) else: - raise - - if 'html' in headers['content-type'].lower(): - return self._download_html(url, headers, filename) + return filename - # and return its filename - return filename @@ -367,7 +285,7 @@ - def _download_html(self, url, headers, filename): + def _download_html(self, url, headers, filename, tmpdir): # Check for a sourceforge URL sf_url = url.startswith('http://prdownloads.') file = open(filename) @@ -388,7 +306,7 @@ page = file.read() file.close() os.unlink(filename) - return self._download_sourceforge(url, page) + return self._download_sourceforge(url, page, tmpdir) break # not an index page file.close() raise RuntimeError("Unexpected HTML page found at "+url) @@ -408,7 +326,7 @@ - def _download_sourceforge(self, source_url, sf_page): + def _download_sourceforge(self, source_url, sf_page, tmpdir): """Download package from randomly-selected SourceForge mirror""" mirror_regex = re.compile(r'HREF=(/.*?\?use_mirror=[^>]*)') @@ -420,7 +338,7 @@ import random url = urlparse.urljoin(source_url, random.choice(urls)) - f = urllib.urlopen(url) + f = self.open_url(url) match = re.search( r'1: + raise RuntimeError( + "Multiple setup scripts in %s" % dist_filename + ) + setup_script = setups[0] + + from setuptools.command import bdist_egg + sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) + try: + run_setup(setup_script, ['-q', 'bdist_egg']) + except SystemExit, v: + raise RuntimeError( + "Setup script exited with %s" % (v.args[0],) + ) + + eggs = [] + for egg in glob( + os.path.join(os.path.dirname(setup_script),'dist','*.egg') + ): + eggs.append(self.install_egg(egg, zip_ok, tmpdir)) + + return eggs + + def install_egg(self, egg_path, zip_ok, tmpdir): + + destination = os.path.join(self.instdir, os.path.basename(egg_path)) + ensure_directory(destination) + + if not self.samefile(egg_path, destination): + if os.path.isdir(destination): + shutil.rmtree(destination) + elif os.path.isfile(destination): + os.unlink(destination) + + if zip_ok: + if egg_path.startswith(tmpdir): + shutil.move(egg_path, destination) + else: + shutil.copy2(egg_path, destination) + + elif os.path.isdir(egg_path): + shutil.move(egg_path, destination) + + else: + os.mkdir(destination) + unpack_archive(egg_path, destination) # XXX add progress?? + + if os.path.isdir(destination): + dist = Distribution.from_filename( + destination, metadata=PathMetadata( + destination, os.path.join(destination,'EGG-INFO') + ) + ) + else: + metadata = EggMetadata(zipimport.zipimporter(destination)) + dist = Distribution.from_filename(destination,metadata=metadata) + + self.update_pth(dist) + return dist + + + + + def installation_report(self, dist): """Helpful installation message for display to package users""" @@ -478,14 +519,14 @@ version = dist.version return msg % locals() - - - - - - - - + def update_pth(self,dist): + if self.pth_file is not None: + remove = self.pth_file.remove + for d in self.pth_file.get(dist.key,()): # drop old entries + remove(d) + if not self.multi: + self.pth_file.add(dist) # add new entry + self.pth_file.save() @@ -533,7 +574,7 @@ URL_SCHEME = re.compile('([-+.a-z0-9]{2,}):',re.I).match -def main(argv, factory=Installer): +def main(argv, installer_type=Installer, index_type=PackageIndex): from optparse import OptionParser @@ -572,44 +613,44 @@ + def alloc_tmp(): + if options.tmpdir is None: + return tempfile.mkdtemp(prefix="easy_install-") + elif not os.path.isdir(options.tmpdir): + os.makedirs(options.tmpdir) + return os.path.realpath(options.tmpdir) + try: - index = PackageIndex(options.index_url) + index = index_type(options.index_url) + inst = installer_type(options.instdir, options.multi) + if options.scan_urls: for url in options.scan_urls: index.scan_url(url) for spec in args: - inst = factory( - options.instdir, options.zip_ok, options.multi, options.tmpdir, - index - ) + tmpdir = alloc_tmp() try: print "Downloading", spec - downloaded = inst.download(spec) - if downloaded is None: + download = index.download(spec, tmpdir) + if download is None: raise RuntimeError( "Could not find distribution for %r" % spec ) - print "Installing", os.path.basename(downloaded) - for dist in inst.install_eggs(downloaded): + + print "Installing", os.path.basename(download) + for dist in inst.install_eggs(download,options.zip_ok, tmpdir): + index.add(dist) print inst.installation_report(dist) + finally: - inst.close() + if options.tmpdir is None: + shutil.rmtree(tmpdir) except RuntimeError, v: print >>sys.stderr,"error:",v sys.exit(1) - if __name__ == '__main__': main(sys.argv[1:]) - - - - - - - - - From pje at users.sourceforge.net Sun Jun 12 05:44:09 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 11 Jun 2005 20:44:09 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools package_index.py, NONE, 1.1 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15558/setuptools Added Files: package_index.py Log Message: Move package index/downloading stuff to setuptools.package_index module. --- NEW FILE: package_index.py --- """PyPI and direct package downloading""" import sys, os.path, re, urlparse, urllib2 from pkg_resources import * HREF = re.compile(r"""href\s*=\s*['"]?([^'"> ]+)""", re.I) URL_SCHEME = re.compile('([-+.a-z0-9]{2,}):',re.I).match EXTENSIONS = ".tar.gz .tar.bz2 .tar .zip .tgz".split() __all__ = [ 'PackageIndex', 'distros_for_url', ] def distros_for_url(url, metadata=None): """Yield egg or source distribution objects that might be found at a URL""" path = urlparse.urlparse(url)[2] base = urllib2.unquote(path.split('/')[-1]) if base.endswith('.egg'): dist = Distribution.from_filename(base, metadata) dist.path = url yield dist return # only one, unambiguous interpretation for ext in EXTENSIONS: if base.endswith(ext): base = base[:-len(ext)] break else: return # no extension matched # Generate alternative interpretations of a source distro name # Because some packages are ambiguous as to name/versions split # e.g. "adns-python-1.1.0", "egenix-mx-commercial", etc. # So, we generate each possible interepretation (e.g. "adns, python-1.1.0" # "adns-python, 1.1.0", and "adns-python-1.1.0, no version"). In practice, # the spurious interpretations should be ignored, because in the event # there's also an "adns" package, the spurious "python-1.1.0" version will # compare lower than any numeric version number, and is therefore unlikely # to match a request for it. It's still a potential problem, though, and # in the long run PyPI and the distutils should go for "safe" names and # versions in distribution archive names (sdist and bdist). parts = base.split('-') for p in range(1,len(parts)+1): yield Distribution( url, metadata, '-'.join(parts[:p]), '-'.join(parts[p:]), distro_type = SOURCE_DIST ) class PackageIndex(AvailableDistributions): """A distribution index that scans web pages for download URLs""" def __init__(self,index_url="http://www.python.org/pypi",*args,**kw): AvailableDistributions.__init__(self,*args,**kw) self.index_url = index_url + "/"[:not index_url.endswith('/')] self.scanned_urls = {} self.fetched_urls = {} self.package_pages = {} def scan_url(self, url): self.process_url(url, True) def process_url(self, url, retrieve=False): if url in self.scanned_urls and not retrieve: return self.scanned_urls[url] = True dists = list(distros_for_url(url)) map(self.add, dists) if dists or not retrieve or url in self.fetched_urls: # don't need the actual page return f = self.open_url(url) self.fetched_urls[url] = self.fetched_urls[f.url] = True if 'html' not in f.headers['content-type'].lower(): f.close() # not html, we can't process it return base = f.url # handle redirects page = f.read() f.close() if url.startswith(self.index_url): self.process_index(url, page) else: for match in HREF.finditer(page): link = urlparse.urljoin(base, match.group(1)) self.process_url(link) def find_packages(self,requirement): self.scan_url(self.index_url + requirement.distname) if not self.package_pages.get(requirement.key): # We couldn't find the target package, so search the index page too self.scan_url(self.index_url) for url in self.package_pages.get(requirement.key,()): # scan each page that might be related to the desired package self.scan_url(url) def process_index(self,url,page): def scan(link): if link.startswith(self.index_url): parts = map( urllib2.unquote, link[len(self.index_url):].split('/') ) if len(parts)==2: # it's a package page, sanitize and index it pkg = safe_name(parts[0]) ver = safe_version(parts[1]) self.package_pages.setdefault(pkg.lower(),{})[link] = True if url==self.index_url or 'Index of Packages' in page: # process an index page into the package-page index for match in HREF.finditer(page): scan( urlparse.urljoin(url, match.group(1)) ) else: scan(url) # ensure this page is in the page index # process individual package page for tag in ("Home Page", "Download URL"): pos = page.find(tag) if pos!=-1: match = HREF.search(page,pos) if match: # Process the found URL self.scan_url(urlparse.urljoin(url, match.group(1))) def obtain(self,requirement): self.find_packages(requirement) for dist in self.get(requirement.key, ()): if dist in requirement: return dist def download(self, spec, tmpdir): """Locate and/or download `spec`, returning a local filename `spec` may be a ``Requirement`` object, or a string containing a URL, an existing local filename, or a package/version requirement spec (i.e. the string form of a ``Requirement`` object). If necessary, the requirement is searched for in the package index. If the download is successful, the return value is a local file path, and it is a subpath of `tmpdir` if the distribution had to be downloaded. If no matching distribution is found, return ``None``. Various errors may be raised if a problem occurs during downloading. """ if not isinstance(spec,Requirement): scheme = URL_SCHEME(spec) if scheme: # It's a url, download it to tmpdir return self._download_url(scheme.group(1), spec, tmpdir) elif os.path.exists(spec): # Existing file or directory, just return it return spec else: try: spec = Requirement.parse(spec) except ValueError: raise RuntimeError( "Not a URL, existing file, or requirement spec: %r" % (spec,) ) # process a Requirement dist = self.best_match(spec,[]) if dist is not None: return self.download(dist.path, tmpdir) return None dl_blocksize = 8192 def _download_to(self, url, filename): # Download the file fp, tfp = None, None try: fp = self.open_url(url) if isinstance(fp, urllib2.HTTPError): raise RuntimeError( "Can't download %s: %s %s" % (url, fp.code,fp.msg) ) headers = fp.info() blocknum = 0 bs = self.dl_blocksize size = -1 if "content-length" in headers: size = int(headers["Content-Length"]) self.reporthook(url, filename, blocknum, bs, size) tfp = open(filename,'wb') while True: block = fp.read(bs) if block: tfp.write(block) blocknum += 1 self.reporthook(url, filename, blocknum, bs, size) else: break return headers finally: if fp: fp.close() if tfp: tfp.close() def reporthook(self, url, filename, blocknum, blksize, size): pass # no-op def open_url(self, url): try: return urllib2.urlopen(url) except urllib2.HTTPError, v: return v except urllib2.URLError, v: raise RuntimeError("Download error: %s" % v.reason) def _download_url(self, scheme, url, tmpdir): # Determine download filename # name = filter(None,urlparse.urlparse(url)[2].split('/')) if name: name = name[-1] while '..' in name: name = name.replace('..','.').replace('\\','_') else: name = "__downloaded__" # default if URL has no path contents filename = os.path.join(tmpdir,name) # Download the file # if scheme=='svn' or scheme.startswith('svn+'): return self._download_svn(url, filename) else: headers = self._download_to(url, filename) if 'html' in headers['content-type'].lower(): return self._download_html(url, headers, filename, tmpdir) else: return filename def _download_html(self, url, headers, filename, tmpdir): # Check for a sourceforge URL sf_url = url.startswith('http://prdownloads.') file = open(filename) for line in file: if line.strip(): # Check for a subversion index page if re.search(r'Revision \d+:', line): # it's a subversion index page: file.close() os.unlink(filename) return self._download_svn(url, filename) # Check for a SourceForge header elif sf_url: if re.search(r'^<HTML><HEAD>', line, re.I): continue # skip first line elif re.search(r'<TITLE>Select a Mirror for File:',line): # Sourceforge mirror page page = file.read() file.close() os.unlink(filename) return self._download_sourceforge(url, page, tmpdir) break # not an index page file.close() raise RuntimeError("Unexpected HTML page found at "+url) def _download_svn(self, url, filename): os.system("svn checkout -q %s %s" % (url, filename)) return filename def _download_sourceforge(self, source_url, sf_page, tmpdir): """Download package from randomly-selected SourceForge mirror""" mirror_regex = re.compile(r'HREF=(/.*?\?use_mirror=[^>]*)') urls = [m.group(1) for m in mirror_regex.finditer(sf_page)] if not urls: raise RuntimeError( "URL looks like a Sourceforge mirror page, but no URLs found" ) import random url = urlparse.urljoin(source_url, random.choice(urls)) f = self.open_url(url) match = re.search( r'<META HTTP-EQUIV="refresh" content=".*?URL=(.*?)"', f.read() ) f.close() if match: download_url = match.group(1) scheme = URL_SCHEME(download_url) return self._download_url(scheme.group(1), download_url, tmpdir) else: raise RuntimeError( 'No META HTTP-EQUIV="refresh" found in Sourceforge page at %s' % url ) From pje at users.sourceforge.net Sun Jun 12 05:44:09 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 11 Jun 2005 20:44:09 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.7, 1.8 easy_install.py, 1.15, 1.16 Message-ID: <E1DhJOL-00045i-Ca@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15558 Modified Files: EasyInstall.txt easy_install.py Log Message: Move package index/downloading stuff to setuptools.package_index module. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- EasyInstall.txt 12 Jun 2005 03:07:36 -0000 1.7 +++ EasyInstall.txt 12 Jun 2005 03:44:07 -0000 1.8 @@ -280,7 +280,8 @@ * All downloads are now managed by the ``PackageIndex`` class (which is now subclassable and replaceable), so that embedders can more easily override - download logic, give download progress reports, etc. + download logic, give download progress reports, etc. The class has also + been moved to the new ``setuptools.package_index`` module. * The ``Installer`` class no longer handles downloading, manages a temporary directory, or tracks the ``zip_ok`` option. Downloading is now handled Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- easy_install.py 12 Jun 2005 03:07:37 -0000 1.15 +++ easy_install.py 12 Jun 2005 03:44:07 -0000 1.16 @@ -12,348 +12,20 @@ """ -import sys -import os.path -import pkg_resources -import re -import zipimport -import shutil -import urlparse -import urllib2 -import tempfile +import sys, os.path, zipimport, shutil, tempfile from setuptools.sandbox import run_setup -from setuptools.archive_util import unpack_archive from distutils.sysconfig import get_python_lib -from pkg_resources import * - - - - - - - - - - - - - -HREF = re.compile(r"""href\s*=\s*['"]?([^'"> ]+)""", re.I) -EXTENSIONS = ".tar.gz .tar.bz2 .tar .zip .tgz".split() - -def distros_for_url(url, metadata=None): - """Yield egg or source distribution objects that might be found at a URL""" - - path = urlparse.urlparse(url)[2] - base = urllib2.unquote(path.split('/')[-1]) - - if base.endswith('.egg'): - dist = Distribution.from_filename(base, metadata) - dist.path = url - yield dist - return # only one, unambiguous interpretation - - for ext in EXTENSIONS: - if base.endswith(ext): - base = base[:-len(ext)] - break - else: - return # no extension matched - - # Generate alternative interpretations of a source distro name - # Because some packages are ambiguous as to name/versions split - # e.g. "adns-python-1.1.0", "egenix-mx-commercial", etc. - # So, we generate each possible interepretation (e.g. "adns, python-1.1.0" - # "adns-python, 1.1.0", and "adns-python-1.1.0, no version"). In practice, - # the spurious interpretations should be ignored, because in the event - # there's also an "adns" package, the spurious "python-1.1.0" version will - # compare lower than any numeric version number, and is therefore unlikely - # to match a request for it. It's still a potential problem, though, and - # in the long run PyPI and the distutils should go for "safe" names and - # versions in distribution archive names (sdist and bdist). - - parts = base.split('-') - for p in range(1,len(parts)+1): - yield Distribution( - url, metadata, '-'.join(parts[:p]), '-'.join(parts[p:]), - distro_type = SOURCE_DIST - ) - -class PackageIndex(AvailableDistributions): - """A distribution index that scans web pages for download URLs""" - - def __init__(self,index_url="http://www.python.org/pypi",*args,**kw): - AvailableDistributions.__init__(self,*args,**kw) - self.index_url = index_url + "/"[:not index_url.endswith('/')] - self.scanned_urls = {} - self.fetched_urls = {} - self.package_pages = {} - - def scan_url(self, url): - self.process_url(url, True) - - def process_url(self, url, retrieve=False): - if url in self.scanned_urls and not retrieve: - return - - self.scanned_urls[url] = True - dists = list(distros_for_url(url)) - map(self.add, dists) - - if dists or not retrieve or url in self.fetched_urls: - # don't need the actual page - return - - f = self.open_url(url) - self.fetched_urls[url] = self.fetched_urls[f.url] = True - if 'html' not in f.headers['content-type'].lower(): - f.close() # not html, we can't process it - return - - base = f.url # handle redirects - page = f.read() - f.close() - if url.startswith(self.index_url): - self.process_index(url, page) - else: - for match in HREF.finditer(page): - link = urlparse.urljoin(base, match.group(1)) - self.process_url(link) - - def find_packages(self,requirement): - self.scan_url(self.index_url + requirement.distname) - if not self.package_pages.get(requirement.key): - # We couldn't find the target package, so search the index page too - self.scan_url(self.index_url) - for url in self.package_pages.get(requirement.key,()): - # scan each page that might be related to the desired package - self.scan_url(url) - - def process_index(self,url,page): - def scan(link): - if link.startswith(self.index_url): - parts = map( - urllib2.unquote, link[len(self.index_url):].split('/') - ) - if len(parts)==2: - # it's a package page, sanitize and index it - pkg = safe_name(parts[0]) - ver = safe_version(parts[1]) - self.package_pages.setdefault(pkg.lower(),{})[link] = True - if url==self.index_url or 'Index of Packages' in page: - # process an index page into the package-page index - for match in HREF.finditer(page): - scan( urlparse.urljoin(url, match.group(1)) ) - else: - scan(url) # ensure this page is in the page index - # process individual package page - for tag in ("Home Page", "Download URL"): - pos = page.find(tag) - if pos!=-1: - match = HREF.search(page,pos) - if match: - # Process the found URL - self.scan_url(urlparse.urljoin(url, match.group(1))) - - def obtain(self,requirement): - self.find_packages(requirement) - for dist in self.get(requirement.key, ()): - if dist in requirement: - return dist - - def download(self, spec, tmpdir): - """Locate and/or download `spec`, returning a local filename - - `spec` may be a ``Requirement`` object, or a string containing a URL, - an existing local filename, or a package/version requirement spec - (i.e. the string form of a ``Requirement`` object). - - If necessary, the requirement is searched for in the package index. - If the download is successful, the return value is a local file path, - and it is a subpath of `tmpdir` if the distribution had to be - downloaded. If no matching distribution is found, return ``None``. - Various errors may be raised if a problem occurs during downloading. - """ - - if not isinstance(spec,Requirement): - scheme = URL_SCHEME(spec) - if scheme: - # It's a url, download it to tmpdir - return self._download_url(scheme.group(1), spec, tmpdir) - - elif os.path.exists(spec): - # Existing file or directory, just return it - return spec - else: - try: - spec = Requirement.parse(spec) - except ValueError: - raise RuntimeError( - "Not a URL, existing file, or requirement spec: %r" % - (spec,) - ) - - # process a Requirement - dist = self.best_match(spec,[]) - if dist is not None: - return self.download(dist.path, tmpdir) - - return None - - - - dl_blocksize = 8192 - - def _download_to(self, url, filename): - # Download the file - fp, tfp = None, None - try: - fp = self.open_url(url) - if isinstance(fp, urllib2.HTTPError): - raise RuntimeError( - "Can't download %s: %s %s" % (url, fp.code,fp.msg) - ) - - headers = fp.info() - blocknum = 0 - bs = self.dl_blocksize - size = -1 - - if "content-length" in headers: - size = int(headers["Content-Length"]) - self.reporthook(url, filename, blocknum, bs, size) - - tfp = open(filename,'wb') - while True: - block = fp.read(bs) - if block: - tfp.write(block) - blocknum += 1 - self.reporthook(url, filename, blocknum, bs, size) - else: - break - return headers - - finally: - if fp: fp.close() - if tfp: tfp.close() - - def reporthook(self, url, filename, blocknum, blksize, size): - pass # no-op - - - - def open_url(self, url): - try: - return urllib2.urlopen(url) - except urllib2.HTTPError, v: - return v - except urllib2.URLError, v: - raise RuntimeError("Download error: %s" % v.reason) - - - def _download_url(self, scheme, url, tmpdir): - - # Determine download filename - # - name = filter(None,urlparse.urlparse(url)[2].split('/')) - if name: - name = name[-1] - while '..' in name: - name = name.replace('..','.').replace('\\','_') - else: - name = "__downloaded__" # default if URL has no path contents - - filename = os.path.join(tmpdir,name) - - # Download the file - # - if scheme=='svn' or scheme.startswith('svn+'): - return self._download_svn(url, filename) - else: - headers = self._download_to(url, filename) - if 'html' in headers['content-type'].lower(): - return self._download_html(url, headers, filename, tmpdir) - else: - return filename - - - - - - - - - def _download_html(self, url, headers, filename, tmpdir): - # Check for a sourceforge URL - sf_url = url.startswith('http://prdownloads.') - file = open(filename) - for line in file: - if line.strip(): - # Check for a subversion index page - if re.search(r'Revision \d+:', line): - # it's a subversion index page: - file.close() - os.unlink(filename) - return self._download_svn(url, filename) - # Check for a SourceForge header - elif sf_url: - if re.search(r'^<HTML><HEAD>', line, re.I): - continue # skip first line - elif re.search(r'<TITLE>Select a Mirror for File:',line): - # Sourceforge mirror page - page = file.read() - file.close() - os.unlink(filename) - return self._download_sourceforge(url, page, tmpdir) - break # not an index page - file.close() - raise RuntimeError("Unexpected HTML page found at "+url) - - - def _download_svn(self, url, filename): - os.system("svn checkout -q %s %s" % (url, filename)) - return filename - - - - - - - +from setuptools.archive_util import unpack_archive +from setuptools.package_index import PackageIndex +from pkg_resources import * - def _download_sourceforge(self, source_url, sf_page, tmpdir): - """Download package from randomly-selected SourceForge mirror""" - mirror_regex = re.compile(r'HREF=(/.*?\?use_mirror=[^>]*)') - urls = [m.group(1) for m in mirror_regex.finditer(sf_page)] - if not urls: - raise RuntimeError( - "URL looks like a Sourceforge mirror page, but no URLs found" - ) - import random - url = urlparse.urljoin(source_url, random.choice(urls)) - f = self.open_url(url) - match = re.search( - r'<META HTTP-EQUIV="refresh" content=".*?URL=(.*?)"', - f.read() - ) - f.close() - if match: - download_url = match.group(1) - scheme = URL_SCHEME(download_url) - return self._download_url(scheme.group(1), download_url, tmpdir) - else: - raise RuntimeError( - 'No META HTTP-EQUIV="refresh" found in Sourceforge page at %s' - % url - ) @@ -572,8 +244,6 @@ -URL_SCHEME = re.compile('([-+.a-z0-9]{2,}):',re.I).match - def main(argv, installer_type=Installer, index_type=PackageIndex): from optparse import OptionParser @@ -613,6 +283,8 @@ + + def alloc_tmp(): if options.tmpdir is None: return tempfile.mkdtemp(prefix="easy_install-") From kbk at users.sourceforge.net Sun Jun 12 06:33:33 2005 From: kbk at users.sourceforge.net (kbk@users.sourceforge.net) Date: Sat, 11 Jun 2005 21:33:33 -0700 Subject: [Python-checkins] python/dist/src/Lib/idlelib CREDITS.txt, 1.10, 1.11 ColorDelegator.py, 1.14, 1.15 EditorWindow.py, 1.67, 1.68 NEWS.txt, 1.58, 1.59 help.txt, 1.12, 1.13 Message-ID: <E1DhKA9-00087f-07@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Lib/idlelib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30052 Modified Files: CREDITS.txt ColorDelegator.py EditorWindow.py NEWS.txt help.txt Log Message: 1. Patch 1196895 Jeff Shute: New files are colorized by default, and colorizing is removed when saving as non-Python files. Patch 1196895 Jeff Shute Closes Python Bugs 775012 and 800432, partial fix IDLEfork 763524 2. Update help.txt for left/right word delete M CREDITS.txt M ColorDelegator.py M EditorWindow.py M NEWS.txt M help.txt Index: CREDITS.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/CREDITS.txt,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- CREDITS.txt 16 Jul 2003 03:10:43 -0000 1.10 +++ CREDITS.txt 12 Jun 2005 04:33:30 -0000 1.11 @@ -23,8 +23,8 @@ integration and persistent breakpoints). Scott David Daniels, Hernan Foffani, Christos Georgiou, Martin v. Löwis, -Jason Orendorff, Noam Raphael, Josh Robb, Nigel Rowe, and Bruce Sherwood have -submitted useful patches. Thanks, guys! +Jason Orendorff, Noam Raphael, Josh Robb, Nigel Rowe, Bruce Sherwood, and +Jeff Shute have submitted useful patches. Thanks, guys! For additional details refer to NEWS.txt and Changelog. Index: ColorDelegator.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/ColorDelegator.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- ColorDelegator.py 15 Mar 2004 04:26:37 -0000 1.14 +++ ColorDelegator.py 12 Jun 2005 04:33:30 -0000 1.15 @@ -237,6 +237,9 @@ if DEBUG: print "colorizing stopped" return + def removecolors(self): + for tag in self.tagdefs.keys(): + self.tag_remove(tag, "1.0", "end") def main(): from Percolator import Percolator Index: EditorWindow.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/EditorWindow.py,v retrieving revision 1.67 retrieving revision 1.68 diff -u -d -r1.67 -r1.68 --- EditorWindow.py 31 Jan 2005 03:34:26 -0000 1.67 +++ EditorWindow.py 12 Jun 2005 04:33:30 -0000 1.68 @@ -491,7 +491,7 @@ self.center() def ispythonsource(self, filename): - if not filename: + if not filename or os.path.isdir(filename): return True base, ext = os.path.splitext(os.path.basename(filename)) if os.path.normcase(ext) in (".py", ".pyw"): @@ -532,6 +532,7 @@ def rmcolorizer(self): if not self.color: return + self.color.removecolors() self.per.removefilter(self.undo) self.per.removefilter(self.color) self.color = None Index: NEWS.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/NEWS.txt,v retrieving revision 1.58 retrieving revision 1.59 diff -u -d -r1.58 -r1.59 --- NEWS.txt 10 May 2005 03:44:24 -0000 1.58 +++ NEWS.txt 12 Jun 2005 04:33:30 -0000 1.59 @@ -3,6 +3,10 @@ *Release date: XX-XXX-2005* +- New files are colorized by default, and colorizing is removed when + saving as non-Python files. Patch 1196895 Jeff Shute + Closes Python Bugs 775012 and 800432, partial fix IDLEfork 763524 + - Improve subprocess link error notification. - run.py: use Queue's blocking feature instead of sleeping in the main Index: help.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/help.txt,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- help.txt 24 Apr 2004 03:08:13 -0000 1.12 +++ help.txt 12 Jun 2005 04:33:30 -0000 1.13 @@ -132,7 +132,8 @@ Basic editing and navigation: - Backspace deletes to the left; DEL deletes to the right. + Backspace deletes char to the left; DEL deletes char to the right. + Control-backspace deletes word left, Control-DEL deletes word right. Arrow keys and Page Up/Down move around. Control-left/right Arrow moves by words in a strange but useful way. Home/End go to begin/end of line. From kbk at users.sourceforge.net Sun Jun 12 07:19:26 2005 From: kbk at users.sourceforge.net (kbk@users.sourceforge.net) Date: Sat, 11 Jun 2005 22:19:26 -0700 Subject: [Python-checkins] python/dist/src/Lib/idlelib EditorWindow.py, 1.68, 1.69 NEWS.txt, 1.59, 1.60 ScriptBinding.py, 1.29, 1.30 Message-ID: <E1DhKsY-0003cF-CL@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Lib/idlelib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12490 Modified Files: EditorWindow.py NEWS.txt ScriptBinding.py Log Message: 1. Clarify "tab/space" Error Dialog and "Tab Width" Dialog associated with the Untabify command. 2. Corrected "tab/space" Error Dialog to show correct menu for Untabify. Patch 1196980 Jeff Shute M EditorWindow.py M NEWS.txt M ScriptBinding.py Index: EditorWindow.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/EditorWindow.py,v retrieving revision 1.68 retrieving revision 1.69 diff -u -d -r1.68 -r1.69 --- EditorWindow.py 12 Jun 2005 04:33:30 -0000 1.68 +++ EditorWindow.py 12 Jun 2005 05:19:23 -0000 1.69 @@ -1271,7 +1271,7 @@ def _asktabwidth(self): return self.askinteger( "Tab width", - "Spaces per tab? (2-16)", + "Columns per tab? (2-16)", parent=self.text, initialvalue=self.indentwidth, minvalue=2, Index: NEWS.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/NEWS.txt,v retrieving revision 1.59 retrieving revision 1.60 diff -u -d -r1.59 -r1.60 --- NEWS.txt 12 Jun 2005 04:33:30 -0000 1.59 +++ NEWS.txt 12 Jun 2005 05:19:23 -0000 1.60 @@ -3,6 +3,12 @@ *Release date: XX-XXX-2005* +- Clarify "tab/space" Error Dialog and "Tab Width" Dialog associated with + the Untabify command. + +- Corrected "tab/space" Error Dialog to show correct menu for Untabify. + Patch 1196980 Jeff Shute + - New files are colorized by default, and colorizing is removed when saving as non-Python files. Patch 1196895 Jeff Shute Closes Python Bugs 775012 and 800432, partial fix IDLEfork 763524 Index: ScriptBinding.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/ScriptBinding.py,v retrieving revision 1.29 retrieving revision 1.30 diff -u -d -r1.29 -r1.30 --- ScriptBinding.py 19 Jan 2005 00:22:59 -0000 1.29 +++ ScriptBinding.py 12 Jun 2005 05:19:23 -0000 1.30 @@ -31,16 +31,13 @@ indent_message = """Error: Inconsistent indentation detected! -This means that either: - -1) your indentation is outright incorrect (easy to fix), or - -2) your indentation mixes tabs and spaces in a way that depends on \ -how many spaces a tab is worth. +1) Your indentation is outright incorrect (easy to fix), OR -To fix case 2, change all tabs to spaces by using Select All followed \ -by Untabify Region (both in the Edit menu).""" +2) Your indentation mixes tabs and spaces. +To fix case 2, change all tabs to spaces by using Edit->Select All followed \ +by Format->Untabify Region and specify the number of columns used by each tab. +""" class ScriptBinding: From pje at users.sourceforge.net Sun Jun 12 17:49:56 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 12 Jun 2005 08:49:56 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools __init__.py, 1.2, 1.3 dist.py, 1.5, 1.6 Message-ID: <E1DhUii-0002Z5-97@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7441/setuptools Modified Files: __init__.py dist.py Log Message: Restructure easy_install as a distutils "Command" object, so that it can access the distutils configuration and logging infrastructure, and can "inherit" options from a distutils setup script that wants to use it to install its own dependencies. Index: __init__.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/__init__.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- __init__.py 17 Jun 2004 20:37:44 -0000 1.2 +++ __init__.py 12 Jun 2005 15:49:53 -0000 1.3 @@ -4,7 +4,7 @@ from setuptools.dist import Distribution, Feature from setuptools.extension import Extension from setuptools.depends import Require -from distutils.core import Command +from distutils.core import Command as _Command from distutils.util import convert_path import os.path @@ -37,6 +37,8 @@ return out + + def setup(**attrs): """Do package setup @@ -47,3 +49,34 @@ """ attrs.setdefault("distclass",Distribution) return distutils.core.setup(**attrs) + + +class Command(_Command): + __doc__ = _Command.__doc__ + + command_consumes_arguments = False + + def reinitialize_command(self, command, reinit_subcommands=0, **kw): + cmd = _Command.reinitialize_command(self, command, reinit_subcommands) + for k,v in kw.items(): + setattr(cmd,k,v) # update command with keywords + return cmd + + + + + + + + + + + + + + + + + + + Index: dist.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/dist.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- dist.py 15 Jun 2004 19:00:03 -0000 1.5 +++ dist.py 12 Jun 2005 15:49:53 -0000 1.6 @@ -1,4 +1,5 @@ __all__ = ['Distribution', 'Feature'] + from distutils.core import Distribution as _Distribution from distutils.core import Extension from setuptools.depends import Require @@ -7,8 +8,37 @@ from setuptools.command.install_lib import install_lib from distutils.errors import DistutilsOptionError, DistutilsPlatformError from distutils.errors import DistutilsSetupError + sequence = tuple, list + + + + + + + + + + + + + + + + + + + + + + + + + + + + class Distribution(_Distribution): """Distribution with support for features, tests, and package data @@ -54,7 +84,6 @@ the distribution. They are used by the feature subsystem to configure the distribution for the included and excluded features. """ - def __init__ (self, attrs=None): have_package_data = hasattr(self, "package_data") if not have_package_data: @@ -83,6 +112,15 @@ """Convert feature name to corresponding option attribute name""" return 'with_'+name.replace('-','_') + + + + + + + + + def _set_global_opts_from_features(self): """Add --with-X/--without-X options based on optional features""" @@ -277,17 +315,58 @@ ) map(self.exclude_package, packages) + + + + + + + + + + + def _parse_command_opts(self, parser, args): # Remove --with-X/--without-X options when processing command args self.global_options = self.__class__.global_options self.negative_opt = self.__class__.negative_opt - return _Distribution._parse_command_opts(self, parser, args) + + # Handle commands that want to consume all remaining arguments + command = args[0] + nargs = _Distribution._parse_command_opts(self, parser, args) + + cmd_class = self.get_command_class(command) + if getattr(cmd_class,'command_consumes_arguments',None): + self.get_option_dict(command)['args'] = ("command line", nargs) + if nargs is not None: + return [] + + return nargs + def has_dependencies(self): return not not self.requires + + + + + + + + + + + + + + + + + + def get_cmdline_options(self): """Return a '{cmd: {opt:val}}' map of all command-line options From pje at users.sourceforge.net Sun Jun 12 17:49:56 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 12 Jun 2005 08:49:56 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.14, 1.15 Message-ID: <E1DhUii-0002Z8-If@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7441/setuptools/command Modified Files: bdist_egg.py Log Message: Restructure easy_install as a distutils "Command" object, so that it can access the distutils configuration and logging infrastructure, and can "inherit" options from a distutils setup script that wants to use it to install its own dependencies. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- bdist_egg.py 7 Jun 2005 04:41:51 -0000 1.14 +++ bdist_egg.py 12 Jun 2005 15:49:54 -0000 1.15 @@ -4,7 +4,7 @@ # This module should be kept compatible with Python 2.3 import os -from distutils.core import Command +from setuptools import Command from distutils.util import get_platform from distutils.dir_util import create_tree, remove_tree, ensure_relative,mkpath from distutils.sysconfig import get_python_version, get_python_lib @@ -234,16 +234,16 @@ return match.group(1) def call_command(self,cmdname,**kw): - cmd = self.reinitialize_command(cmdname) + """Invoke reinitialized command `cmdname` with keyword args""" for dirname in INSTALL_DIRECTORY_ATTRS: - if dirname in cmd.__dict__: # don't overwrite methods! - setattr(cmd,dirname,self.bdist_dir) - cmd.skip_build = self.skip_build - for k,v in kw.items(): - setattr(cmd,k,v) + kw.setdefault(dirname,self.bdist_dir) + kw.setdefault('skip_build',self.skip_build) + + cmd = self.reinitialize_command(cmdname, **kw) self.run_command(cmdname) return cmd + # Attribute names of options for commands that might need to be convinced to # install to the egg build directory From pje at users.sourceforge.net Sun Jun 12 17:49:56 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 12 Jun 2005 08:49:56 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.8, 1.9 easy_install.py, 1.16, 1.17 Message-ID: <E1DhUii-0002Z1-4K@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7441 Modified Files: EasyInstall.txt easy_install.py Log Message: Restructure easy_install as a distutils "Command" object, so that it can access the distutils configuration and logging infrastructure, and can "inherit" options from a distutils setup script that wants to use it to install its own dependencies. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- EasyInstall.txt 12 Jun 2005 03:44:07 -0000 1.8 +++ EasyInstall.txt 12 Jun 2005 15:49:53 -0000 1.9 @@ -179,11 +179,39 @@ Reference Manual ================ +Configuration Files +------------------- + +(New in 0.4a2) + +You may specify default options for EasyInstall using the standard +distutils configuration files, under the command heading ``easy_install``. +EasyInstall will look first for a ``setup.cfg`` file in the current directory, +then a ``~/.pydistutils.cfg`` or ``$HOME\\pydistutils.cfg`` (on Unix-like OSes +and Windows, respectively), and finally a ``distutils.cfg`` file in the +``distutils`` package directory. Here's a simple example:: + + [easy_install] + + # set the default location to install packages + install_dir = /home/me/lib/python + + # Notice that indentation can be used to continue an option + # value; this is especially useful for the "--find-links" + # option, which tells easy_install to use download links on + # these pages before consulting PyPI: + # + find_links = http://sqlobject.org/ + http://peak.telecommunity.com/dist/ + +See also the current Python documentation on the `use and location of distutils +configuration files <http://docs.python.org/inst/config-syntax.html>`_. + Command-Line Options -------------------- -``--zip, -z`` +``--zip-ok, -z`` Enable installing the package as a zip file. This can significantly increase Python's overall import performance if you're installing to ``site-packages`` and not using the ``--multi`` option, because Python @@ -221,11 +249,13 @@ which will put the latest installed version of the specified packages on ``sys.path`` for you. (For more advanced uses, like selecting specific versions and enabling optional dependencies, see the ``pkg_resources`` API - doc.) Note that if you install to a directory other than ``site-packages``, + doc.) + + Note that if you install to a directory other than ``site-packages``, this option is automatically in effect, because ``.pth`` files can only be used in ``site-packages`` (at least in Python 2.3 and 2.4). So, if you use - the ``--install-dir`` or ``-i`` options, you must also use ``require()`` to - enable packages at runtime + the ``--install-dir`` or ``-d`` option (or they are set via configuration + file(s)) you must also use ``require()`` to enable packages at runtime. ``--install-dir=DIR, -d DIR`` Set the installation directory. It is up to you to ensure that this @@ -233,39 +263,53 @@ ``pkg_resources.require()`` to enable the installed package(s) that you need. -``--build-directory=DIR, -b DIR`` (New in 0.3a3) - Set the directory used to download, extract, and install the package. The - directory is not cleared before or after installation, so the downloaded - packages and extracted contents will remain there afterwards, allowing you - to read any documentation, examples, scripts, etc. that may have been - included with the source distribution (if any). - - This option can only be used when you are specifying a single installation - URL or filename, so that the installer will not be confused by the presence - of multiple ``setup.py`` files in the build directory. + (New in 0.4a2) If this option is not directly specified on the command line + or in a distutils configuration file, the distutils default installation + location is used. Normally, this would be the ``site-packages`` directory, + but if you are using distutils configuration files, setting things like + ``--prefix`` or ``--install-lib``, then those settings are taken into + account when computing the default directory. -``--scan-url=URL, -s URL`` (New in 0.4a1) - Scan the specified "download page" for direct links to downloadable eggs or - source distributions. Any usable packages will be downloaded if they are - required by a command line argument. For example, this:: +``--find-links=URL, -f URL`` (Option renamed in 0.4a2) + Scan the specified "download pages" for direct links to downloadable eggs + or source distributions. Any usable packages will be downloaded if they + are required by a command line argument. For example, this:: - easy_install -s http://peak.telecommunity.com/dist PyProtocols + easy_install -f http://peak.telecommunity.com/dist PyProtocols will download and install the latest version of PyProtocols linked from the PEAK downloads page, but ignore the other download links on that page. + If all requested packages can be found using links on the specified + download pages, the Python Package Index will *not* be consulted. You can + use ``file:`` URLs to reference a local filename. - You may use this option more than once, to list multiple download pages. - If all requested packages can be found using the specified download pages, - the Python Package Index will *not* be consulted. + You may specify multiple URLs with this option, separated by whitespace. + Note that on the command line, you will probably have to surround the URLs + with quotes, so that they are recognized as a single option value. You can + also specify URLs in a configuration file; see `Configuration Files`_, + above; but note that this means the specified pages will be downloaded + every time you use EasyInstall (unless overridden on the command line) and + thus may make startup slower. ``--index-url=URL, -u URL`` (New in 0.4a1) Specifies the base URL of the Python Package Index. The default is http://www.python.org/pypi if not specified. When a package is requested - that is not locally available or linked from a ``--scan-url`` download + that is not locally available or linked from a ``--find-links`` download page, the package index will be searched for download pages for the needed package, and those download pages will be searched for links to download an egg or source distribution. +``--build-directory=DIR, -b DIR`` (New in 0.3a3) + Set the directory used to download, extract, and install the package. The + directory is not cleared before or after installation, so the downloaded + packages and extracted contents will remain there afterwards, allowing you + to read any documentation, examples, scripts, etc. that may have been + included with the source distribution (if any). + + This option can only be used when you are specifying a single installation + URL or filename, so that the installer will not be confused by the presence + of multiple ``setup.py`` files in the build directory. + Release Notes/Change History ============================ @@ -275,6 +319,11 @@ time out or be missing a file. 0.4a2 + * Added support for setting options via distutils configuration files + + * Renamed ``--scan-url/-s`` to ``--find-links/-f`` to free up ``-s`` for the + script installation directory option. + * Use ``urllib2`` instead of ``urllib``, to allow use of ``https:`` URLs if Python includes SSL support. @@ -292,6 +341,11 @@ with an ``unpack_archive()`` API. These were split out of EasyInstall to allow reuse by other tools and applications. + * ``setuptools.Command`` now supports reinitializing commands using keyword + arguments to set/reset options. Also, ``Command`` subclasses can now set + their ``command_consumes_arguments`` attribute to ``True`` in order to + receive an ``args`` option containing the rest of the command line. + 0.4a1 * Added ``--scan-url`` and ``--index-url`` options, to scan download pages and search PyPI for needed packages. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- easy_install.py 12 Jun 2005 03:44:07 -0000 1.16 +++ easy_install.py 12 Jun 2005 15:49:53 -0000 1.17 @@ -14,6 +14,7 @@ import sys, os.path, zipimport, shutil, tempfile +from setuptools import Command from setuptools.sandbox import run_setup from distutils.sysconfig import get_python_lib @@ -22,6 +23,15 @@ from pkg_resources import * +def samefile(p1,p2): + if hasattr(os.path,'samefile') and ( + os.path.exists(p1) and os.path.exists(p2) + ): + return os.path.samefile(p1,p2) + return ( + os.path.normpath(os.path.normcase(p1)) == + os.path.normpath(os.path.normcase(p2)) + ) @@ -29,51 +39,123 @@ +class easy_install(Command): + """Manage a download/build/install process""" + description = "Find/get/install Python packages" + command_consumes_arguments = True + + user_options = [ + ("zip-ok", "z", "install package as a zipfile"), + ("multi-version", "m", "make apps have to require() a version"), + ("install-dir=", "d", "install package to DIR"), + ("index-url=", "i", "base URL of Python Package Index"), + ("find-links=", "f", "additional URL(s) to search for packages"), + ("build-directory=", "b", + "download/extract/build in DIR; keep the results"), + ] + boolean_options = [ 'zip-ok', 'multi-version' ] + create_index = PackageIndex + + def initialize_options(self): + self.zip_ok = None + self.multi_version = None + self.install_dir = None + self.index_url = None + self.find_links = None + self.build_directory = None + self.args = None + # Options not specifiable via command line + self.package_index = None + self.pth_file = None + + def alloc_tmp(self): + if self.build_directory is None: + return tempfile.mkdtemp(prefix="easy_install-") + tmpdir = os.path.realpath(self.build_directory) + if not os.path.isdir(tmpdir): + os.makedirs(tmpdir) + return tmpdir + + def finalize_options(self): + # Let install_lib get set by install_lib command, which in turn + # gets its info from the install command, and takes into account + # --prefix and --home and all that other crud. + # + self.set_undefined_options('install_lib',('install_dir','install_dir')) + site_packages = get_python_lib() + instdir = self.install_dir + + if instdir is None or samefile(site_packages,instdir): + instdir = site_packages + if self.pth_file is None: + self.pth_file = PthDistributions( + os.path.join(instdir,'easy-install.pth') + ) + self.install_dir = instdir + + elif self.multi_version is None: + self.multi_version = True + + elif not self.multi_version: + # explicit false set from Python code; raise an error + raise RuntimeError( + "Can't do single-version installs outside site-packages" + ) + + self.index_url = self.index_url or "http://www.python.org/pypi" + if self.package_index is None: + self.package_index = self.create_index(self.index_url) + + if self.find_links is not None: + if isinstance(self.find_links, basestring): + self.find_links = self.find_links.split() + for link in self.find_links: + self.package_index.scan_url(link) + + if not self.args: + parser.error("No urls, filenames, or requirements specified") + elif len(self.args)>1 and self.build_directory is not None: + parser.error("Build directory can only be set when using one URL") + + def run(self): + for spec in self.args: + self.easy_install(spec) + + + def easy_install(self, spec): + tmpdir = self.alloc_tmp() + try: + print "Downloading", spec + download = self.package_index.download(spec, tmpdir) + if download is None: + raise RuntimeError( + "Could not find distribution for %r" % spec + ) + + print "Installing", os.path.basename(download) + for dist in self.install_eggs(download, self.zip_ok, tmpdir): + self.package_index.add(dist) + print self.installation_report(dist) + + finally: + if self.build_directory is None: + shutil.rmtree(tmpdir) -class Installer: - """Manage a download/build/install process""" - pth_file = None - cleanup = False - def __init__(self, instdir=None, multi=None): - site_packages = get_python_lib() - if instdir is None or self.samefile(site_packages,instdir): - instdir = site_packages - self.pth_file = PthDistributions( - os.path.join(instdir,'easy-install.pth') - ) - elif multi is None: - multi = True - elif not multi: - # explicit false, raise an error - raise RuntimeError( - "Can't do single-version installs outside site-packages" - ) - self.instdir = instdir - self.multi = multi - def samefile(self,p1,p2): - if hasattr(os.path,'samefile') and ( - os.path.exists(p1) and os.path.exists(p2) - ): - return os.path.samefile(p1,p2) - return ( - os.path.normpath(os.path.normcase(p1)) == - os.path.normpath(os.path.normcase(p2)) - ) @@ -123,10 +205,10 @@ def install_egg(self, egg_path, zip_ok, tmpdir): - destination = os.path.join(self.instdir, os.path.basename(egg_path)) + destination = os.path.join(self.install_dir,os.path.basename(egg_path)) ensure_directory(destination) - if not self.samefile(egg_path, destination): + if not samefile(egg_path, destination): if os.path.isdir(destination): shutil.rmtree(destination) elif os.path.isfile(destination): @@ -166,7 +248,7 @@ """Helpful installation message for display to package users""" msg = "Installed %(eggloc)s to %(instdir)s" - if self.multi: + if self.multi_version: msg += """ Because this distribution was installed --multi-version or --install-dir, @@ -178,7 +260,7 @@ pkg_resources.require("%(name)s==%(version)s") # this exact version pkg_resources.require("%(name)s>=%(version)s") # this version or higher """ - if not self.samefile(get_python_lib(),self.instdir): + if not samefile(get_python_lib(),self.install_dir): msg += """ Note also that the installation directory must be on sys.path at runtime for @@ -186,7 +268,7 @@ PYTHONPATH, or by being added to sys.path by your code.) """ eggloc = os.path.basename(dist.path) - instdir = os.path.realpath(self.instdir) + instdir = os.path.realpath(self.install_dir) name = dist.name version = dist.version return msg % locals() @@ -196,7 +278,7 @@ remove = self.pth_file.remove for d in self.pth_file.get(dist.key,()): # drop old entries remove(d) - if not self.multi: + if not self.multi_version: self.pth_file.add(dist) # add new entry self.pth_file.save() @@ -244,40 +326,32 @@ -def main(argv, installer_type=Installer, index_type=PackageIndex): +def main(argv, cmds={'easy_install':easy_install}): + from setuptools import setup + try: + setup(cmdclass = cmds, script_args = ['easy_install']+argv) + except RuntimeError, v: + print >>sys.stderr,"error:",v + sys.exit(1) + + +if __name__ == '__main__': + main(sys.argv[1:]) + + + + + + - from optparse import OptionParser - parser = OptionParser(usage = "usage: %prog [options] url [url...]") - parser.add_option("-d", "--install-dir", dest="instdir", default=None, - help="install package to DIR", metavar="DIR") - parser.add_option("-z", "--zip", - action="store_true", dest="zip_ok", default=False, - help="install package as a zipfile") - parser.add_option("-m", "--multi-version", - action="store_true", dest="multi", default=None, - help="make apps have to require() a version") - parser.add_option("-b", "--build-directory", dest="tmpdir", metavar="DIR", - default=None, - help="download/extract/build in DIR; keep the results") - parser.add_option("-u", "--index-url", dest="index_url", metavar="URL", - default="http://www.python.org/pypi", - help="base URL of Python Package Index") - parser.add_option("-s", "--scan-url", dest="scan_urls", metavar="URL", - action="append", - help="additional URL(s) to search for packages") - (options, args) = parser.parse_args() - if not args: - parser.error("No urls, filenames, or requirements specified") - elif len(args)>1 and options.tmpdir is not None: - parser.error("Build directory can only be set when using one URL") @@ -285,44 +359,11 @@ - def alloc_tmp(): - if options.tmpdir is None: - return tempfile.mkdtemp(prefix="easy_install-") - elif not os.path.isdir(options.tmpdir): - os.makedirs(options.tmpdir) - return os.path.realpath(options.tmpdir) - try: - index = index_type(options.index_url) - inst = installer_type(options.instdir, options.multi) - if options.scan_urls: - for url in options.scan_urls: - index.scan_url(url) - for spec in args: - tmpdir = alloc_tmp() - try: - print "Downloading", spec - download = index.download(spec, tmpdir) - if download is None: - raise RuntimeError( - "Could not find distribution for %r" % spec - ) - print "Installing", os.path.basename(download) - for dist in inst.install_eggs(download,options.zip_ok, tmpdir): - index.add(dist) - print inst.installation_report(dist) - finally: - if options.tmpdir is None: - shutil.rmtree(tmpdir) - except RuntimeError, v: - print >>sys.stderr,"error:",v - sys.exit(1) -if __name__ == '__main__': - main(sys.argv[1:]) From rhettinger at users.sourceforge.net Sun Jun 12 20:23:03 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sun, 12 Jun 2005 11:23:03 -0700 Subject: [Python-checkins] python/dist/src/Lib decimal.py, 1.31.2.3, 1.31.2.4 Message-ID: <E1DhX6t-0005wi-G2@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22827 Modified Files: Tag: release24-maint decimal.py Log Message: Backport 1.37 namespace cleanup. Index: decimal.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/decimal.py,v retrieving revision 1.31.2.3 retrieving revision 1.31.2.4 diff -u -d -r1.31.2.3 -r1.31.2.4 --- decimal.py 27 Mar 2005 10:55:26 -0000 1.31.2.3 +++ decimal.py 12 Jun 2005 18:23:01 -0000 1.31.2.4 @@ -134,7 +134,7 @@ 'setcontext', 'getcontext' ] -import copy +import copy as _copy #Rounding ROUND_DOWN = 'ROUND_DOWN' @@ -2210,7 +2210,7 @@ del s for name, val in locals().items(): if val is None: - setattr(self, name, copy.copy(getattr(DefaultContext, name))) + setattr(self, name, _copy.copy(getattr(DefaultContext, name))) else: setattr(self, name, val) del self.self From rhettinger at users.sourceforge.net Sun Jun 12 20:25:32 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sun, 12 Jun 2005 11:25:32 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libdecimal.tex, 1.24, 1.24.2.1 Message-ID: <E1DhX9I-0006HP-2b@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24118 Modified Files: Tag: release24-maint libdecimal.tex Log Message: Backport 1.26 Decimal FAQ. Index: libdecimal.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libdecimal.tex,v retrieving revision 1.24 retrieving revision 1.24.2.1 diff -u -d -r1.24 -r1.24.2.1 --- libdecimal.tex 25 Nov 2004 05:35:32 -0000 1.24 +++ libdecimal.tex 12 Jun 2005 18:25:29 -0000 1.24.2.1 @@ -525,11 +525,11 @@ large number of methods for doing arithmetic directly in a given context. \begin{methoddesc}{clear_flags}{} - Sets all of the flags to \constant{0}. + Resets all of the flags to \constant{0}. \end{methoddesc} \begin{methoddesc}{copy}{} - Returns a duplicate of the context. + Return a duplicate of the context. \end{methoddesc} \begin{methoddesc}{create_decimal}{num} @@ -1118,3 +1118,156 @@ return +s \end{verbatim} + + + +\subsection{Decimal FAQ \label{decimal-faq}} + + +Q. It is cumbersome to type \code{decimal.Decimal('1234.5')}. Is there a way +to minimize typing when using the interactive interpreter? + +A. Some users abbreviate the constructor to just a single letter: + +\begin{verbatim} +>>> D = decimal.Decimal +>>> D('1.23') + D('3.45') +Decimal("4.68") +\end{verbatim} + + +Q. In a fixed-point application to two decimal places, some inputs +have many places and need to be rounded. Others are not supposed to have +excess digits and need to be validated. What methods should be used? + +A. The \method{quantize()} method rounds to a fixed number of decimal places. +If the \constant{Inexact} trap is set, it is also useful for validation: + +\begin{verbatim} +>>> TWOPLACES = Decimal(10) ** -2 # same as Decimal('0.01') + +>>> # Round to two places +>>> Decimal("3.214").quantize(TWOPLACES) +Decimal("3.21") + +>>> # Validate that a number does not exceed two places +>>> Decimal("3.21").quantize(TWOPLACES, context=Context(traps=[Inexact])) +Decimal("3.21") + +>>> Decimal("3.214").quantize(TWOPLACES, context=Context(traps=[Inexact])) +Traceback (most recent call last): + ... +Inexact: Changed in rounding +\end{verbatim} + + +Q. Once I have valid two place inputs, how do I maintain that invariant +throughout an application? + +A. Some operations like addition and subtraction automatically preserve fixed +point. Others, like multiplication and division, change the number of decimal +places and need to be followed-up with a \method{quantize()} step. + + +Q. There are many ways to write express the same value. The numbers +\constant{200}, \constant{200.000}, \constant{2E2}, and \constant{.02E+4} all +have the same value at various precisions. Is there a way to transform them to +a single recognizable canonical value? + +A. The \method{normalize()} method maps all equivalent values to a single +representive: + +\begin{verbatim} +>>> values = map(Decimal, '200 200.000 2E2 .02E+4'.split()) +>>> [v.normalize() for v in values] +[Decimal("2E+2"), Decimal("2E+2"), Decimal("2E+2"), Decimal("2E+2")] +\end{verbatim} + + +Q. Is there a way to convert a regular float to a \class{Decimal}? + +A. Yes, all binary floating point numbers can be exactly expressed as a +Decimal. An exact conversion may take more precision than intuition would +suggest, so trapping \constant{Inexact} will signal a need for more precision: + +\begin{verbatim} +def floatToDecimal(f): + "Convert a floating point number to a Decimal with no loss of information" + # Transform (exactly) a float to a mantissa (0.5 <= abs(m) < 1.0) and an + # exponent. Double the mantissa until it is an integer. Use the integer + # mantissa and exponent to compute an equivalent Decimal. If this cannot + # be done exactly, then retry with more precision. + + mantissa, exponent = math.frexp(f) + while mantissa != int(mantissa): + mantissa *= 2.0 + exponent -= 1 + mantissa = int(mantissa) + oldcontext = getcontext() + setcontext(Context(traps=[Inexact])) + try: + while True: + try: + return mantissa * Decimal(2) ** exponent + except Inexact: + getcontext().prec += 1 + finally: + setcontext(oldcontext) +\end{verbatim} + + +Q. Why isn't the \function{floatToDecimal()} routine included in the module? + +A. There is some question about whether it is advisable to mix binary and +decimal floating point. Also, its use requires some care to avoid the +representation issues associated with binary floating point: + +\begin{verbatim} +>>> floatToDecimal(1.1) +Decimal("1.100000000000000088817841970012523233890533447265625") +\end{verbatim} + + +Q. Within a complex calculation, how can I make sure that I haven't gotten a +spurious result because of insufficient precision or rounding anomalies. + +A. The decimal module makes it easy to test results. A best practice is to +re-run calculations using greater precision and with various rounding modes. +Widely differing results indicate insufficient precision, rounding mode +issues, ill-conditioned inputs, or a numerically unstable algorithm. + + +Q. I noticed that context precision is applied to the results of operations +but not to the inputs. Is there anything to watch out for when mixing +values of different precisions? + +A. Yes. The principle is that all values are considered to be exact and so +is the arithmetic on those values. Only the results are rounded. The +advantage for inputs is that ``what you type is what you get''. A +disadvantage is that the results can look odd if you forget that the inputs +haven't been rounded: + +\begin{verbatim} +>>> getcontext().prec = 3 +>>> Decimal('3.104') + D('2.104') +Decimal("5.21") +>>> Decimal('3.104') + D('0.000') + D('2.104') +Decimal("5.20") +\end{verbatim} + +The solution is either to increase precision or to force rounding of inputs +using the unary plus operation: + +\begin{verbatim} +>>> getcontext().prec = 3 +>>> +Decimal('1.23456789') # unary plus triggers rounding +Decimal("1.23") +\end{verbatim} + +Alternatively, inputs can be rounded upon creation using the +\method{Context.create_decimal()} method: + +\begin{verbatim} +>>> Context(prec=5, rounding=ROUND_DOWN).create_decimal('1.2345678') +Decimal("1.2345") +\end{verbatim} From pje at users.sourceforge.net Sun Jun 12 21:26:48 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 12 Jun 2005 12:26:48 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.9, 1.10 easy_install.py, 1.17, 1.18 pkg_resources.py, 1.27, 1.28 setup.py, 1.10, 1.11 Message-ID: <E1DhY6a-0006ca-0c@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24242 Modified Files: EasyInstall.txt easy_install.py pkg_resources.py setup.py Log Message: Add script installation support. Use distutils' exceptions for option errors. Include Python version in setuptools' egg name for compatibility w/installs via easy_install. Add isdir/listdir facilities for metadata, along with support for running scripts from eggs. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- EasyInstall.txt 12 Jun 2005 15:49:53 -0000 1.9 +++ EasyInstall.txt 12 Jun 2005 19:26:45 -0000 1.10 @@ -65,10 +65,10 @@ easy_install SQLObject -**Example 2**. Install a package by name and version from a given -"download page":: +**Example 2**. Install or upgrade a package by name and version by finding +links on a given "download page":: - easy_install -s http://peak.telecommunity.com/dist "setuptools>=0.4a1" + easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.4a1" **Example 3**. Download a source distribution from a specified URL, automatically building and installing it:: @@ -90,17 +90,26 @@ By default, packages are installed to the running Python installation's ``site-packages`` directory, unless you provide the ``-d`` or ``--install-dir`` -option to specify an alternative directory. +option to specify an alternative directory, or specify an alternate location +using distutils configuration files. (See `Configuration Files`_, below.) + +By default, any scripts included with the package are installed to the running +Python installation's standard script installation location. However, if you +specify an installation directory via the command line or a config file, then +the default directory for installing scripts will be the same as the package +installation directory, to ensure that the script will have access to the +installed package. You can override this using the ``-s`` or ``--script-dir`` +option. Packages installed to ``site-packages`` are added to an ``easy-install.pth`` -file, so that Python will be able to import the package by default. If you do -not want this to happen, you should use the ``-m`` or ``--multi`` option, which -allows multiple versions of the same package to be selected at runtime. +file, so that Python will always use the most-recently-installed version of +the package. If you would like to be able to select which version to use at +runtime, you should use the ``-m`` or ``--multi-version`` option. -Note that installing to a directory other than ``site-packages`` already -implies the ``-m`` option, so if you cannot install to ``site-packages``, -please see the `Command-Line Options`_ section below (under ``--multi``) to -find out how to select packages at runtime. +Note, however, that installing to a directory other than ``site-packages`` +already implies the ``-m`` option, so if you cannot install to +``site-packages``, please see the `Command-Line Options`_ section below (under +``--multi-version``) to find out how to select packages at runtime. Upgrading a Package @@ -133,6 +142,11 @@ file, so that Python will import the most-recently installed version by default. +If you haven't suppressed script installation (using ``--exclude-scripts`` or +``-x``), then the upgraded version's scripts will be installed, and they will +be automatically patched to ``require()`` the corresponding version of the +package, so that you can use them even if not installing to ``site-packages``. + ``easy_install`` never actually deletes packages (unless you're installing a package with the same name and version number as an existing package), so if you want to get rid of older versions of a package, please see `Uninstalling @@ -148,15 +162,22 @@ easy_install PackageName==1.2.3 Where ``1.2.3`` is replaced by the exact version number you wish to switch to. -Note that the named package and version must already have been installed to -``site-packages``. +If a package matching the requested name and version is not already installed +in a directory on ``sys.path``, it will be located via PyPI and installed. -If you'd like to switch to the latest version of ``PackageName``, you can do so -like this:: +If you'd like to switch to the latest installed version of ``PackageName``, you +can do so like this:: easy_install PackageName -This will activate the latest installed version. +This will activate the latest installed version. (Note: if you have set any +``find_links`` via distutils configuration files, those download pages will be +checked for the latest available version of the package, and it will be +downloaded and installed if it is newer than your current version.) + +Note that changing the active version of a package will install the newly +active version's scripts, unless the ``--exclude-scripts`` or ``-x`` option is +specified. Uninstalling Packages @@ -173,7 +194,45 @@ This will ensure that Python doesn't continue to search for a package you're planning to remove. After you've done this, you can safely delete the .egg -files or directories. +files or directories, along with any scripts you wish to remove. + + +Managing Scripts +---------------- + +Whenever you install, upgrade, or change versions of a package, EasyInstall +automatically installs the scripts for the selected package version, unless +you tell it not to with ``-x`` or ``--exclude-scripts``. If any scripts in +the script directory have the same name, they are overwritten. + +Thus, you do not normally need to manually delete scripts for older versions of +a package, unless the newer version of the package does not include a script +of the same name. However, if you are completely uninstalling a package, you +may wish to manually delete its scripts. + +EasyInstall's default behavior means that you can normally only run scripts +from one version of a package at a time. If you want to keep multiple versions +of a script available, however, you can simply use the ``--multi-version`` or +``-m`` option, and rename the scripts that EasyInstall creates. This works +because EasyInstall installs scripts as short code stubs that ``require()`` the +matching version of the package the script came from, so renaming the script +has no effect on what it executes. + +For example, suppose you want to use two versions of the ``rst2html`` tool +provided by the `docutils <http://docutils.sf.net/>`_ package. You might +first install one version:: + + easy_install -m docutils==0.3.9 + +then rename the ``rst2html.py`` to ``r2h_039``, and install another version:: + + easy_install -m docutils==0.3.10 + +This will create another ``rst2html.py`` script, this one using docutils +version 0.3.10 instead of 0.3.9. You now have two scripts, each using a +different version of the package. (Notice that we used ``-m`` for both +installations, so that Python won't lock us out of using anything but the most +recently-installed version of the package.) Reference Manual @@ -267,8 +326,22 @@ or in a distutils configuration file, the distutils default installation location is used. Normally, this would be the ``site-packages`` directory, but if you are using distutils configuration files, setting things like - ``--prefix`` or ``--install-lib``, then those settings are taken into - account when computing the default directory. + ``prefix`` or ``install_lib``, then those settings are taken into + account when computing the default installation directory. + +``--script-dir=DIR, -s DIR`` + Set the script installation directory. If you don't supply this option + (via the command line or a configuration file), but you *have* supplied + an ``--install-dir`` (via command line or config file), then this option + defaults to the same directory, so that the scripts will be able to find + their associated package installation. Otherwise, this setting defaults + to the location where the distutils would normally install scripts, taking + any distutils configuration file settings into account. + +``--exclude-scripts, -x`` + Don't install scripts. This is useful if you need to install multiple + versions of a package, but do not want to reset the version that will be + run by scripts that are already installed. ``--find-links=URL, -f URL`` (Option renamed in 0.4a2) Scan the specified "download pages" for direct links to downloadable eggs @@ -319,6 +392,8 @@ time out or be missing a file. 0.4a2 + * Added support for installing scripts + * Added support for setting options via distutils configuration files * Renamed ``--scan-url/-s`` to ``--find-links/-f`` to free up ``-s`` for the @@ -405,9 +480,9 @@ Future Plans ============ -* Support packages that include scripts * Log progress to a logger, with -v and -q options to control verbosity * Process the installed package's dependencies as well as the base package * Additional utilities to list/remove/verify packages * Signature checking? SSL? Ability to suppress PyPI search? +* Support installation from bdist_wininst packages? Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- easy_install.py 12 Jun 2005 15:49:53 -0000 1.17 +++ easy_install.py 12 Jun 2005 19:26:45 -0000 1.18 @@ -17,7 +17,7 @@ from setuptools import Command from setuptools.sandbox import run_setup from distutils.sysconfig import get_python_lib - +from distutils.errors import DistutilsArgError from setuptools.archive_util import unpack_archive from setuptools.package_index import PackageIndex from pkg_resources import * @@ -43,31 +43,31 @@ """Manage a download/build/install process""" description = "Find/get/install Python packages" - command_consumes_arguments = True - user_options = [ ("zip-ok", "z", "install package as a zipfile"), ("multi-version", "m", "make apps have to require() a version"), ("install-dir=", "d", "install package to DIR"), + ("script-dir=", "s", "install scripts to DIR"), + ("exclude-scripts", "x", "Don't install scripts"), ("index-url=", "i", "base URL of Python Package Index"), ("find-links=", "f", "additional URL(s) to search for packages"), ("build-directory=", "b", "download/extract/build in DIR; keep the results"), ] - boolean_options = [ 'zip-ok', 'multi-version' ] + boolean_options = [ 'zip-ok', 'multi-version', 'exclude-scripts' ] create_index = PackageIndex def initialize_options(self): self.zip_ok = None self.multi_version = None - self.install_dir = None + self.install_dir = self.script_dir = self.exclude_scripts = None self.index_url = None self.find_links = None self.build_directory = None self.args = None - + # Options not specifiable via command line self.package_index = None self.pth_file = None @@ -81,11 +81,22 @@ return tmpdir def finalize_options(self): - # Let install_lib get set by install_lib command, which in turn + # If a non-default installation directory was specified, default the + # script directory to match it. + if self.script_dir is None: + self.script_dir = self.install_dir + + # Let install_dir get set by install_lib command, which in turn # gets its info from the install command, and takes into account # --prefix and --home and all that other crud. - # - self.set_undefined_options('install_lib',('install_dir','install_dir')) + self.set_undefined_options('install_lib', + ('install_dir','install_dir') + ) + # Likewise, set default script_dir from 'install_scripts.install_dir' + self.set_undefined_options('install_scripts', + ('install_dir', 'script_dir') + ) + site_packages = get_python_lib() instdir = self.install_dir @@ -102,10 +113,10 @@ elif not self.multi_version: # explicit false set from Python code; raise an error - raise RuntimeError( + raise DistutilsArgError( "Can't do single-version installs outside site-packages" ) - + self.index_url = self.index_url or "http://www.python.org/pypi" if self.package_index is None: self.package_index = self.create_index(self.index_url) @@ -117,10 +128,13 @@ self.package_index.scan_url(link) if not self.args: - parser.error("No urls, filenames, or requirements specified") + raise DistutilsArgError( + "No urls, filenames, or requirements specified (see --help)") elif len(self.args)>1 and self.build_directory is not None: - parser.error("Build directory can only be set when using one URL") - + raise DistutilsArgError( + "Build directory can only be set when using one URL" + ) + def run(self): for spec in self.args: self.easy_install(spec) @@ -139,6 +153,7 @@ print "Installing", os.path.basename(download) for dist in self.install_eggs(download, self.zip_ok, tmpdir): self.package_index.add(dist) + self.install_egg_scripts(dist) print self.installation_report(dist) finally: @@ -147,16 +162,42 @@ + def install_egg_scripts(self, dist): + metadata = dist.metadata + if self.exclude_scripts or not metadata.metadata_isdir('scripts'): + return + from distutils.command.build_scripts import first_line_re + for script_name in metadata.metadata_listdir('scripts'): + target = os.path.join(self.script_dir, script_name) + print "Installing", script_name, "to", target + script_text = metadata.get_metadata('scripts/'+script_name) + script_text = script_text.replace('\r','\n') + first, rest = script_text.split('\n',1) + match = first_line_re.match(first) + options = '' + if match: + options = match.group(1) or '' + if options: + options = ' '+options + spec = '%s==%s' % (dist.name,dist.version) + script_text = '\n'.join([ + "#!%s%s" % (os.path.normpath(sys.executable),options), + "# EASY-INSTALL-SCRIPT: %r,%r" % (spec, script_name), + "import pkg_resources", + "pkg_resources.run_main(%r, %r)" % (spec, script_name) + ]) - - + f = open(target,"w") + f.write(script_text) + f.close() + @@ -329,7 +370,7 @@ def main(argv, cmds={'easy_install':easy_install}): from setuptools import setup try: - setup(cmdclass = cmds, script_args = ['easy_install']+argv) + setup(cmdclass = cmds, script_args = ['-q','easy_install', '-v']+argv) except RuntimeError, v: print >>sys.stderr,"error:",v sys.exit(1) Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.27 retrieving revision 1.28 diff -u -d -r1.27 -r1.28 --- pkg_resources.py 5 Jun 2005 21:33:51 -0000 1.27 +++ pkg_resources.py 12 Jun 2005 19:26:45 -0000 1.28 @@ -22,7 +22,7 @@ 'InvalidOption', 'Distribution', 'Requirement', 'yield_lines', 'get_importer', 'find_distributions', 'find_on_path', 'register_finder', 'split_sections', 'declare_namespace', 'register_namespace_handler', - 'safe_name', 'safe_version' + 'safe_name', 'safe_version', 'run_main', ] import sys, os, zipimport, time, re, imp @@ -102,12 +102,12 @@ # XXX all the tricky cases go here return False - - - - - - +def run_main(dist_spec, script_name): + """Locate distribution `dist_spec` and run its `script_name` script""" + import __main__ + __main__.__dict__.clear() + __main__.__dict__.update({'__name__':'__main__'}) + require(dist_spec)[0].metadata.run_script(script_name, __main__.__dict__) @@ -135,6 +135,33 @@ Leading and trailing whitespace is stripped from each line, and lines with ``#`` as the first non-blank character are omitted.""" + def metadata_isdir(name): + """Is the named metadata a directory? (like ``os.path.isdir()``)""" + + def metadata_listdir(name): + """List of metadata names in the directory (like ``os.listdir()``)""" + + def run_script(script_name, namespace): + """Execute the named script in the supplied namespace dictionary""" + + + + + + + + + + + + + + + + + + + class IResourceProvider(IMetadataProvider): """An object that provides access to package resources""" @@ -162,6 +189,20 @@ def resource_listdir(resource_name): """List of resource names in the directory (like ``os.listdir()``)""" + + + + + + + + + + + + + + class AvailableDistributions(object): """Searchable snapshot of distributions on a search path""" @@ -460,9 +501,10 @@ """ requirements = parse_requirements(requirements) - - for dist in AvailableDistributions().resolve(requirements): + to_install = AvailableDistributions().resolve(requirements) + for dist in to_install: dist.install_on(sys.path) + return to_install def safe_name(name): @@ -489,7 +531,6 @@ - class NullProvider: """Try to implement resources and metadata for arbitrary PEP 302 loaders""" @@ -502,40 +543,79 @@ self.module_path = os.path.dirname(getattr(module, '__file__', '')) def get_resource_filename(self, manager, resource_name): - return self._fn(resource_name) + return self._fn(self.module_path, resource_name) def get_resource_stream(self, manager, resource_name): - return open(self._fn(resource_name), 'rb') + return open(self._fn(self.module_path, resource_name), 'rb') def get_resource_string(self, manager, resource_name): - return self._get(self._fn(resource_name)) + return self._get(self._fn(self.module_path, resource_name)) def has_resource(self, resource_name): - return self._has(self._fn(resource_name)) + return self._has(self._fn(self.module_path, resource_name)) def has_metadata(self, name): - if not self.egg_info: - raise NotImplementedError("Only .egg supports metadata") - return self._has(os.path.join(self.egg_info, *name.split('/'))) + return self.egg_info and self._has(self._fn(self.egg_info,name)) def get_metadata(self, name): if not self.egg_info: - raise NotImplementedError("Only .egg supports metadata") - return self._get(os.path.join(self.egg_info, *name.split('/'))) + return "" + return self._get(self._fn(self.egg_info,name)) def get_metadata_lines(self, name): return yield_lines(self.get_metadata(name)) - def resource_isdir(self,name): return False + def resource_isdir(self,name): + return self._isdir(self._fn(self.module_path, resource_name)) + + def metadata_isdir(self,name): + return self.egg_info and self._isdir(self._fn(self.egg_info,name)) + def resource_listdir(self,name): + return self._listdir(self._fn(self.egg_info,name)) + + def metadata_listdir(self,name): + if self.egg_info: + return self._listdir(self._fn(self.egg_info,name)) return [] + def run_script(self,script_name,namespace): + script = 'scripts/'+script_name + if not self.has_metadata(script): + raise ResolutionError("No script named %r" % script_name) + script_text = self.get_metadata(script).replace('\r\n','\n') + script_text = script_text.replace('\r','\n') + script_filename = self._fn(self.egg_info,script) + + if os.path.exists(script_filename): + execfile(script_filename, namespace, namespace) + else: + from linecache import cache + cache[script_filename] = ( + len(script_text), 0, script_text.split('\n'), script_filename + ) + script_code = compile(script_text,script_filename,'exec') + exec script_code in namespace, namespace + def _has(self, path): raise NotImplementedError( "Can't perform this operation for unregistered loader type" ) + def _isdir(self, path): + raise NotImplementedError( + "Can't perform this operation for unregistered loader type" + ) + + def _listdir(self, path): + raise NotImplementedError( + "Can't perform this operation for unregistered loader type" + ) + + def _fn(self, base, resource_name): + return os.path.join(base, *resource_name.split('/')) + def _get(self, path): if hasattr(self.loader, 'get_data'): return self.loader.get_data(path) @@ -543,9 +623,6 @@ "Can't perform this operation for loaders without 'get_data()'" ) - def _fn(self, resource_name): - return os.path.join(self.module_path, *resource_name.split('/')) - register_loader_type(object, NullProvider) @@ -572,13 +649,18 @@ + + + + + class DefaultProvider(NullProvider): """Provides access to package resources in the filesystem""" def __init__(self,module): NullProvider.__init__(self,module) self._setup_prefix() - + def _setup_prefix(self): # we assume here that our metadata may be nested inside a "basket" # of multiple eggs; that's why we use module_path instead of .archive @@ -597,11 +679,11 @@ def _has(self, path): return os.path.exists(path) - def resource_isdir(self,name): - return os.path.isdir(self._fn(name)) + def _isdir(self,path): + return os.path.isdir(path) - def resource_listdir(self,name): - return os.listdir(self._fn(name)) + def _listdir(self,path): + return os.listdir(path) def _get(self, path): stream = open(path, 'rb') @@ -628,10 +710,6 @@ return path[len(self.zip_pre):] return path - def _has(self, path): return self._short_name(path) in self.zipinfo or self.resource_isdir(path) - - def _get(self, path): return self.loader.get_data(path) - def get_resource_stream(self, manager, resource_name): return StringIO(self.get_resource_string(manager, resource_name)) @@ -649,27 +727,19 @@ return self._extract_resource(manager, resource_name) - def resource_isdir(self, resource_name): - if resource_name.endswith('/'): - resource_name = resource_name[:-1] - return resource_name in self._index() - - def resource_listdir(self, resource_name): - if resource_name.endswith('/'): - resource_name = resource_name[:-1] - return list(self._index().get(resource_name, ())) - def _extract_directory(self, manager, resource_name): if resource_name.endswith('/'): resource_name = resource_name[:-1] for resource in self.resource_listdir(resource_name): last = self._extract_resource(manager, resource_name+'/'+resource) return os.path.dirname(last) # return the directory path - + + + def _extract_resource(self, manager, resource_name): if self.resource_isdir(resource_name): return self._extract_dir(resource_name) - + parts = resource_name.split('/') zip_path = os.path.join(self.module_path, *parts) zip_stat = self.zipinfo[os.path.join(*self.prefix+parts)] @@ -704,6 +774,9 @@ self.eagers = eagers return self.eagers + + + def _index(self): try: return self._dirindex @@ -724,14 +797,23 @@ self._dirindex = ind return ind + def _has(self, path): + return self._short_name(path) in self.zipinfo or self._isdir(path) -register_loader_type(zipimport.zipimporter, ZipProvider) - - + def _isdir(self,path): + path = self._short_name(path).replace(os.sep, '/') + if path.endswith('/'): path = path[:-1] + return path in self._index() + def _listdir(self,path): + path = self._short_name(path).replace(os.sep, '/') + if path.endswith('/'): path = path[:-1] + return list(self._index().get(path, ())) + _get = NullProvider._get +register_loader_type(zipimport.zipimporter, ZipProvider) @@ -886,7 +968,7 @@ subpath = os.path.join(path_item, subitem) for dist in find_in_zip(zipimport.zipimporter(subpath), subpath): yield dist - + register_finder(zipimport.zipimporter,find_in_zip) @@ -1148,7 +1230,7 @@ class Distribution(object): """Wrap an actual or potential sys.path entry w/metadata""" - + def __init__(self, path_str, metadata=None, name=None, version=None, py_version=PY_MAJOR, platform=None, distro_type = EGG_DIST Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- setup.py 6 Jun 2005 03:30:48 -0000 1.10 +++ setup.py 12 Jun 2005 19:26:45 -0000 1.11 @@ -2,8 +2,9 @@ """Distutils setup file, used to install or test 'setuptools'""" VERSION = "0.4a1" +import sys from setuptools import setup, find_packages, Require -from distutils.version import LooseVersion +PYVER = sys.version[:3] setup( name="setuptools", @@ -38,11 +39,10 @@ Require('PyUnit', None, 'unittest', "http://pyunit.sf.net/"), ], - packages = find_packages(), py_modules = ['pkg_resources', 'easy_install'], scripts = ['easy_install.py'], - extra_path = ('setuptools', 'setuptools-%s.egg' % VERSION), + extra_path = ('setuptools', 'setuptools-%s-py%s.egg' % (VERSION,PYVER)), classifiers = [f.strip() for f in """ Development Status :: 3 - Alpha From pje at users.sourceforge.net Sun Jun 12 23:47:35 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 12 Jun 2005 14:47:35 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools ez_setup.py, NONE, 1.1 EasyInstall.txt, 1.10, 1.11 easy_install.py, 1.18, 1.19 pkg_resources.py, 1.28, 1.29 setup.py, 1.11, 1.12 Message-ID: <E1DhaIp-0007lB-Ry@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29239 Modified Files: EasyInstall.txt easy_install.py pkg_resources.py setup.py Added Files: ez_setup.py Log Message: Add 'ez_setup' bootstrap installer. Prep for 0.4a2 release. --- NEW FILE: ez_setup.py --- #!python """Bootstrap setuptools installation If you want to use setuptools in your package's setup.py, just include this file in the same directory with it, and add this to the top of your setup.py:: from ez_setup import use_setuptools use_setuptools() If you want to require a specific version of setuptools, set a download mirror, or use an alternate download directory, you can do so by supplying the appropriate options to ``use_setuptools()``. This file can also be run as a script to install or upgrade setuptools. """ DEFAULT_VERSION = "0.4a2" DEFAULT_URL = "http://peak.telecommunity.com/dist/" import sys, os def use_setuptools( version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir ): """Automatically find/download setuptools and make it available on sys.path `version` should be a valid setuptools version number that is available as an egg for download under the `download_base` URL (which should end with a '/'). `to_dir` is the directory where setuptools will be downloaded, if it is not already available. If an older version of setuptools is installed, this will print a message to ``sys.stderr`` and raise SystemExit in an attempt to abort the calling script. """ try: import setuptools if setuptools.__version__ == '0.0.1': print >>sys.stderr, ( "You have an obsolete version of setuptools installed. Please\n" "remove it from your system entirely before rerunning this script." ) sys.exit(2) except ImportError: sys.path.insert(0, download_setuptools(version, download_base, to_dir)) import pkg_resources try: pkg_resources.require("setuptools>="+version) except pkg_resources.VersionConflict: # XXX could we install in a subprocess here? print >>sys.stderr, ( "The required version of setuptools (>=%s) is not available, and\n" "can't be installed while this script is running. Please install\n" " a more recent version first." ) % version sys.exit(2) def download_setuptools( version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir ): """Download setuptools from a specified location and return its filename `version` should be a valid setuptools version number that is available as an egg for download under the `download_base` URL (which should end with a '/'). `to_dir` is the directory where the egg will be downloaded. """ import urllib2, shutil egg_name = "setuptools-%s-py%s.egg" % (version,sys.version[:3]) url = download_base + egg_name saveto = os.path.join(to_dir, egg_name) src = dst = None try: src = urllib2.urlopen(url) dst = open(saveto,"wb") shutil.copyfileobj(src,dst) finally: if src: src.close() if dst: dst.close() return os.path.realpath(saveto) def main(argv, version=DEFAULT_VERSION): """Install or upgrade setuptools and EasyInstall""" try: import setuptools except ImportError: import tempfile, shutil tmpdir = tempfile.mkdtemp(prefix="easy_install-") try: egg = download_setuptools(version, to_dir=tmpdir) sys.path.insert(0,egg) from easy_install import main main(list(argv)+[egg]) finally: shutil.rmtree(tmpdir) else: if setuptools.__version__ == '0.0.1': # tell the user to uninstall obsolete version use_setuptools(version) req = "setuptools>="+version import pkg_resources try: pkg_resources.require(req) except pkg_resources.VersionConflict: from easy_install import main main(list(argv)+[req]) sys.exit(0) # try to force an exit else: if argv: from easy_install import main main(argv) else: print "Setuptools successfully installed or upgraded." if __name__=='__main__': main(sys.argv[1:]) Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- EasyInstall.txt 12 Jun 2005 19:26:45 -0000 1.10 +++ EasyInstall.txt 12 Jun 2005 21:47:32 -0000 1.11 @@ -22,34 +22,28 @@ Installing "Easy Install" ------------------------- -Unix-like Systems (including Mac OS X and Cygwin) - Download either the `Python 2.3 easy_install shell script - <http://peak.telecommunity.com/dist/ez_install-0.4a1-py2.3-unix.sh>`_ or the - `Python 2.4 easy_install shell script - <http://peak.telecommunity.com/dist/ez_install-0.4a1-py2.4-unix.sh>`_. Place - the file somewhere on your PATH, after renaming it to ``easy_install``. Note - that these scripts assume you have ``python2.3`` or ``python2.4`` accessible - via the ``PATH`` environment variable. Then, you can use ``easy_install`` to - finish its own installation, by running one of the following, depending on - your Python version:: - - # Python 2.3 - easy_install http://peak.telecommunity.com/dist/setuptools-0.4a1-py2.3.egg - - # Python 2.4 - easy_install http://peak.telecommunity.com/dist/setuptools-0.4a1-py2.4.egg - +Windows users can just download and run the `setuptools binary installer for +Windows <http://peak.telecommunity.com/dist/setuptools-0.4a2.win32.exe>`_. +All others should just download `ez_setup.py +<http://peak.telecommunity.com/dist/ez_setup.py>`_, and run it; this will +download and install the correct version of ``setuptools`` for your Python +version. You may receive a message telling you about an obsolete version of +setuptools being present; if so, you must be sure to delete it entirely, along +with the old ``pkg_resources`` module if it's present on ``sys.path``. + +An ``easy_install.py`` script will be installed in the normal location for +Python scripts on your platform. In the examples below, you'll need to replace +references to ``easy_install`` with the correct invocation to run +``easy_install.py`` on your system. If you have Python 2.4 or better, you can +also use ``python -m easy_install``, which will have the same effect, but which +may be easier for you to type. -All Other Systems - Download the `easy_install (aka setuptools) source distribution - <http://peak.telecommunity.com/dist/setuptools-0.4a1.zip>`_, and follow the - normal procedure for installing a source package with distutils. An - ``easy_install.py`` script will be installed in the normal location for - Python scripts on your platform. In the examples below, you'll need to - replace references to ``easy_install`` with the correct invocation to run - ``easy_install.py`` on your system. If you have Python 2.4 or better, you - can also use ``python -m easy_install``, which will have the same effect, - but which may be easier for you to type. +(Note: the ``ez_setup.py`` script accepts the same `Command-Line Options`_ and +`Configuration Files`_ as ``easy_install`` itself, so you can use them to +control its behavior. However, you should avoid using a custom installation +directory or doing multi-version installs of the setuptools package, because +this may make it impossible for scripts installed with EasyInstall to access +it afterwards.) Downloading and Installing a Package @@ -68,7 +62,7 @@ **Example 2**. Install or upgrade a package by name and version by finding links on a given "download page":: - easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.4a1" + easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.4a2" **Example 3**. Download a source distribution from a specified URL, automatically building and installing it:: @@ -391,14 +385,22 @@ * There's no automatic retry for borked Sourceforge mirrors, which can easily time out or be missing a file. + * EasyInstall does not yet respect the distutils "verbose/quite" and "dry-run" + options, even though it accepts them. + 0.4a2 * Added support for installing scripts - * Added support for setting options via distutils configuration files + * Added support for setting options via distutils configuration files, and + using distutils' default options as a basis for EasyInstall's defaults. * Renamed ``--scan-url/-s`` to ``--find-links/-f`` to free up ``-s`` for the script installation directory option. + * Added ``ez_setup.py`` installer/bootstrap script to make initial setuptools + installation easier, and to allow distributions using setuptools to avoid + having to include setuptools in their source distribution. + * Use ``urllib2`` instead of ``urllib``, to allow use of ``https:`` URLs if Python includes SSL support. @@ -409,7 +411,8 @@ * The ``Installer`` class no longer handles downloading, manages a temporary directory, or tracks the ``zip_ok`` option. Downloading is now handled - by ``PackageIndex``, and the latter two are now managed by ``main()``. + by ``PackageIndex``, and ``Installer`` has become an ``easy_install`` + command class based on ``setuptools.Command``. * There is a new ``setuptools.sandbox.run_setup()`` API to invoke a setup script in a directory sandbox, and a new ``setuptools.archive_util`` module @@ -476,13 +479,13 @@ Initial release. - Future Plans ============ * Log progress to a logger, with -v and -q options to control verbosity +* Display more information about downloads and progress when being verbose * Process the installed package's dependencies as well as the base package +* Support installation from bdist_wininst packages? * Additional utilities to list/remove/verify packages * Signature checking? SSL? Ability to suppress PyPI search? -* Support installation from bdist_wininst packages? Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.18 retrieving revision 1.19 diff -u -d -r1.18 -r1.19 --- easy_install.py 12 Jun 2005 19:26:45 -0000 1.18 +++ easy_install.py 12 Jun 2005 21:47:32 -0000 1.19 @@ -247,6 +247,7 @@ def install_egg(self, egg_path, zip_ok, tmpdir): destination = os.path.join(self.install_dir,os.path.basename(egg_path)) + destination = os.path.abspath(destination) ensure_directory(destination) if not samefile(egg_path, destination): @@ -284,7 +285,6 @@ - def installation_report(self, dist): """Helpful installation message for display to package users""" Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.28 retrieving revision 1.29 diff -u -d -r1.28 -r1.29 --- pkg_resources.py 12 Jun 2005 19:26:45 -0000 1.28 +++ pkg_resources.py 12 Jun 2005 21:47:33 -0000 1.29 @@ -1307,7 +1307,7 @@ ) version = property(version) - + #@property @@ -1351,6 +1351,47 @@ fixup_namespace_packages(self.path) map(declare_namespace, self._get_metadata('namespace_packages.txt')) + def egg_name(self): + """Return what this distribution's standard .egg filename should be""" + filename = "%s-%s-py%s" % ( + self.name.replace('-','_'), self.version.replace('-','_'), + self.py_version or PY_MAJOR + ) + + if self.platform: + filename += '-'+self.platform + + return filename + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + def parse_requirements(strs): """Yield ``Requirement`` objects for each specification in `strs` Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- setup.py 12 Jun 2005 19:26:45 -0000 1.11 +++ setup.py 12 Jun 2005 21:47:33 -0000 1.12 @@ -1,10 +1,8 @@ #!/usr/bin/env python """Distutils setup file, used to install or test 'setuptools'""" -VERSION = "0.4a1" -import sys +VERSION = "0.4a2" from setuptools import setup, find_packages, Require -PYVER = sys.version[:3] setup( name="setuptools", @@ -26,7 +24,7 @@ "download mirrors, or from Python Eggs. Been looking for a CPAN " "clone for Python? When combined with PyPI, this gets pretty darn " "close. See the home page and download page for details and docs.", - + keywords = "CPAN PyPI distutils eggs package management", url = "http://peak.telecommunity.com/PythonEggs", download_url = "http://peak.telecommunity.com/DevCenter/EasyInstall", @@ -39,10 +37,12 @@ Require('PyUnit', None, 'unittest', "http://pyunit.sf.net/"), ], + + packages = find_packages(), py_modules = ['pkg_resources', 'easy_install'], scripts = ['easy_install.py'], - extra_path = ('setuptools', 'setuptools-%s-py%s.egg' % (VERSION,PYVER)), + extra_path = ('setuptools', 'setuptools-%s.egg' % VERSION), classifiers = [f.strip() for f in """ Development Status :: 3 - Alpha From pje at users.sourceforge.net Sun Jun 12 23:47:45 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 12 Jun 2005 14:47:45 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools __init__.py, 1.3, 1.4 Message-ID: <E1DhaIz-0007lI-7a@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29239/setuptools Modified Files: __init__.py Log Message: Add 'ez_setup' bootstrap installer. Prep for 0.4a2 release. Index: __init__.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/__init__.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- __init__.py 12 Jun 2005 15:49:53 -0000 1.3 +++ __init__.py 12 Jun 2005 21:47:33 -0000 1.4 @@ -8,7 +8,7 @@ from distutils.util import convert_path import os.path -__version__ = '0.0.1' +__version__ = '0.4a2' __all__ = [ 'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require', From pje at users.sourceforge.net Sun Jun 12 23:47:45 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 12 Jun 2005 14:47:45 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.15, 1.16 Message-ID: <E1DhaIz-0007lK-7h@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29239/setuptools/command Modified Files: bdist_egg.py Log Message: Add 'ez_setup' bootstrap installer. Prep for 0.4a2 release. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- bdist_egg.py 12 Jun 2005 15:49:54 -0000 1.15 +++ bdist_egg.py 12 Jun 2005 21:47:34 -0000 1.16 @@ -10,7 +10,8 @@ from distutils.sysconfig import get_python_version, get_python_lib from distutils.errors import * from distutils import log -from pkg_resources import parse_requirements, get_platform, safe_name, safe_version +from pkg_resources import parse_requirements, get_platform, safe_name, \ + safe_version, Distribution class bdist_egg(Command): description = "create an \"egg\" distribution" @@ -34,7 +35,6 @@ ('skip-build', None, "skip rebuilding everything (for testing/debugging)"), ] - boolean_options = [ 'keep-temp', 'skip-build', 'relative','tag-date','tag-svn-revision' ] @@ -133,14 +133,14 @@ ) to_compile = [] - - for ext_name in ext_outputs: + for (p,ext_name) in enumerate(ext_outputs): filename,ext = os.path.splitext(ext_name) pyfile = os.path.join(self.bdist_dir, filename + '.py') log.info("creating stub loader for %s" % ext_name) if not self.dry_run: self.write_stub(os.path.basename(ext_name), pyfile) to_compile.append(pyfile) + ext_outputs[p] = ext_name.replace(os.sep,'/') if to_compile: cmd.byte_compile(to_compile) @@ -155,18 +155,10 @@ # And make an archive relative to the root of the # pseudo-installation tree. - archive_basename = "%s-%s-py%s" % ( self.egg_name.replace('-','_'), - self.egg_version.replace('-','_'), get_python_version()) - - if ext_outputs: - archive_basename += "-" + self.plat_name - ext_outputs = [out.replace(os.sep,'/') for out in ext_outputs] - - # OS/2 objects to any ":" characters in a filename (such as when - # a timestamp is used in a version) so change them to underscores. - if os.name == "os2": - archive_basename = archive_basename.replace(":", "_") - + archive_basename = Distribution( + None, None, self.egg_name, self.egg_version, get_python_version(), + ext_outputs and self.plat_name + ).egg_name() pseudoinstall_root = os.path.join(self.dist_dir, archive_basename) archive_root = self.bdist_dir @@ -174,6 +166,7 @@ egg_info = os.path.join(archive_root,'EGG-INFO') self.mkpath(egg_info) self.mkpath(self.egg_info) + log.info("writing %s" % os.path.join(self.egg_info,'PKG-INFO')) if not self.dry_run: metadata = self.distribution.metadata @@ -244,6 +237,13 @@ return cmd + + + + + + + # Attribute names of options for commands that might need to be convinced to # install to the egg build directory From rhettinger at users.sourceforge.net Mon Jun 13 03:10:23 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sun, 12 Jun 2005 18:10:23 -0700 Subject: [Python-checkins] python/dist/src/Lib copy.py,1.45,1.46 Message-ID: <E1DhdT5-0000su-Oa@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3305 Modified Files: copy.py Log Message: SF bug #1219361 Fix typo Index: copy.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/copy.py,v retrieving revision 1.45 retrieving revision 1.46 diff -u -d -r1.45 -r1.46 --- copy.py 8 Mar 2005 15:02:59 -0000 1.45 +++ copy.py 13 Jun 2005 01:10:15 -0000 1.46 @@ -14,7 +14,7 @@ class instances). - A shallow copy constructs a new compound object and then (to the - extent possible) inserts *the same objects* into in that the + extent possible) inserts *the same objects* into it that the original contains. - A deep copy constructs a new compound object and then, recursively, From rhettinger at users.sourceforge.net Mon Jun 13 03:14:51 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sun, 12 Jun 2005 18:14:51 -0700 Subject: [Python-checkins] python/dist/src/Lib copy.py,1.43.4.1,1.43.4.2 Message-ID: <E1DhdXP-0001S2-Gk@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5556 Modified Files: Tag: release24-maint copy.py Log Message: SF bug #1219361 Fix typo Index: copy.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/copy.py,v retrieving revision 1.43.4.1 retrieving revision 1.43.4.2 diff -u -d -r1.43.4.1 -r1.43.4.2 --- copy.py 7 Feb 2005 12:39:54 -0000 1.43.4.1 +++ copy.py 13 Jun 2005 01:14:34 -0000 1.43.4.2 @@ -14,7 +14,7 @@ class instances). - A shallow copy constructs a new compound object and then (to the - extent possible) inserts *the same objects* into in that the + extent possible) inserts *the same objects* into it that the original contains. - A deep copy constructs a new compound object and then, recursively, From mwh at users.sourceforge.net Mon Jun 13 19:50:22 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 13 Jun 2005 10:50:22 -0700 Subject: [Python-checkins] python/dist/src/Python marshal.c,1.84,1.85 Message-ID: <E1Dht4o-0006pR-Ok@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25670 Modified Files: marshal.c Log Message: Remove extraneous format character from PyArg_ParseTuple call in marshal_loads. Bugfix candidate. Index: marshal.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/marshal.c,v retrieving revision 1.84 retrieving revision 1.85 diff -u -d -r1.84 -r1.85 --- marshal.c 3 Jun 2005 15:17:16 -0000 1.84 +++ marshal.c 13 Jun 2005 17:50:18 -0000 1.85 @@ -1081,7 +1081,7 @@ char *s; int n; PyObject* result; - if (!PyArg_ParseTuple(args, "s#|i:loads", &s, &n)) + if (!PyArg_ParseTuple(args, "s#:loads", &s, &n)) return NULL; rf.fp = NULL; rf.ptr = s; From mwh at users.sourceforge.net Mon Jun 13 20:28:48 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 13 Jun 2005 11:28:48 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_marshal.py, 1.11, 1.12 Message-ID: <E1Dhtg0-0003gf-Nv@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13610/Lib/test Modified Files: test_marshal.py Log Message: Fix bug [ 1180997 ] lax error-checking in new-in-2.4 marshal stuff which I'd assigned to Martin, but actually turned out to be easy to fix. Also, a test. Index: test_marshal.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_marshal.py,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- test_marshal.py 4 Jun 2005 12:55:32 -0000 1.11 +++ test_marshal.py 13 Jun 2005 18:28:42 -0000 1.12 @@ -211,6 +211,15 @@ self.assertEquals(marshal.loads(marshal.dumps(5, 0)), 5) self.assertEquals(marshal.loads(marshal.dumps(5, 1)), 5) + def test_fuzz(self): + # simple test that it's at least not *totally* trivial to + # crash from bad marshal data + for c in [chr(i) for i in range(256)]: + try: + marshal.loads(c) + except Exception: + pass + def test_main(): test_support.run_unittest(IntTestCase, FloatTestCase, From mwh at users.sourceforge.net Mon Jun 13 20:28:48 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 13 Jun 2005 11:28:48 -0700 Subject: [Python-checkins] python/dist/src/Python marshal.c,1.85,1.86 Message-ID: <E1Dhtg0-0003gh-OC@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13610/Python Modified Files: marshal.c Log Message: Fix bug [ 1180997 ] lax error-checking in new-in-2.4 marshal stuff which I'd assigned to Martin, but actually turned out to be easy to fix. Also, a test. Index: marshal.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/marshal.c,v retrieving revision 1.85 retrieving revision 1.86 diff -u -d -r1.85 -r1.86 --- marshal.c 13 Jun 2005 17:50:18 -0000 1.85 +++ marshal.c 13 Jun 2005 18:28:46 -0000 1.86 @@ -648,6 +648,10 @@ case TYPE_STRINGREF: n = r_long(p); + if (n < 0 || n >= PyList_GET_SIZE(p->strings)) { + PyErr_SetString(PyExc_ValueError, "bad marshal data"); + return NULL; + } v = PyList_GET_ITEM(p->strings, n); Py_INCREF(v); return v; From doerwalter at users.sourceforge.net Mon Jun 13 23:44:50 2005 From: doerwalter at users.sourceforge.net (doerwalter@users.sourceforge.net) Date: Mon, 13 Jun 2005 14:44:50 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_long.py,1.25,1.26 Message-ID: <E1Dhwji-0006DR-He@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23690/Lib/test Modified Files: test_long.py Log Message: Port test_long.py to unittest. Index: test_long.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_long.py,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- test_long.py 23 Sep 2004 08:06:37 -0000 1.25 +++ test_long.py 13 Jun 2005 21:44:48 -0000 1.26 @@ -1,6 +1,16 @@ -from test.test_support import verify, verbose, TestFailed, fcmp -from string import join -from random import random, randint +import unittest +from test import test_support + +import random + +# Used for lazy formatting of failure messages +class Frm(object): + def __init__(self, format, *args): + self.format = format + self.args = args + + def __str__(self): + return self.format % self.args # SHIFT should match the value in longintrepr.h for best testing. SHIFT = 15 @@ -26,518 +36,451 @@ special = special + map(lambda x: ~x, special) + \ map(lambda x: -x, special) -# ------------------------------------------------------------ utilities - -# Use check instead of assert so the test still does something -# under -O. - -def check(ok, *args): - if not ok: - raise TestFailed, join(map(str, args), " ") - -# Get quasi-random long consisting of ndigits digits (in base BASE). -# quasi == the most-significant digit will not be 0, and the number -# is constructed to contain long strings of 0 and 1 bits. These are -# more likely than random bits to provoke digit-boundary errors. -# The sign of the number is also random. - -def getran(ndigits): - verify(ndigits > 0) - nbits_hi = ndigits * SHIFT - nbits_lo = nbits_hi - SHIFT + 1 - answer = 0L - nbits = 0 - r = int(random() * (SHIFT * 2)) | 1 # force 1 bits to start - while nbits < nbits_lo: - bits = (r >> 1) + 1 - bits = min(bits, nbits_hi - nbits) - verify(1 <= bits <= SHIFT) - nbits = nbits + bits - answer = answer << bits - if r & 1: - answer = answer | ((1 << bits) - 1) - r = int(random() * (SHIFT * 2)) - verify(nbits_lo <= nbits <= nbits_hi) - if random() < 0.5: - answer = -answer - return answer - -# Get random long consisting of ndigits random digits (relative to base -# BASE). The sign bit is also random. -def getran2(ndigits): - answer = 0L - for i in range(ndigits): - answer = (answer << SHIFT) | randint(0, MASK) - if random() < 0.5: - answer = -answer - return answer +class LongTest(unittest.TestCase): -# --------------------------------------------------------------- divmod + # Get quasi-random long consisting of ndigits digits (in base BASE). + # quasi == the most-significant digit will not be 0, and the number + # is constructed to contain long strings of 0 and 1 bits. These are + # more likely than random bits to provoke digit-boundary errors. + # The sign of the number is also random. -def test_division_2(x, y): - q, r = divmod(x, y) - q2, r2 = x//y, x%y - pab, pba = x*y, y*x - check(pab == pba, "multiplication does not commute for", x, y) - check(q == q2, "divmod returns different quotient than / for", x, y) - check(r == r2, "divmod returns different mod than % for", x, y) - check(x == q*y + r, "x != q*y + r after divmod on", x, y) - if y > 0: - check(0 <= r < y, "bad mod from divmod on", x, y) - else: - check(y < r <= 0, "bad mod from divmod on", x, y) + def getran(self, ndigits): + self.assert_(ndigits > 0) + nbits_hi = ndigits * SHIFT + nbits_lo = nbits_hi - SHIFT + 1 + answer = 0L + nbits = 0 + r = int(random.random() * (SHIFT * 2)) | 1 # force 1 bits to start + while nbits < nbits_lo: + bits = (r >> 1) + 1 + bits = min(bits, nbits_hi - nbits) + self.assert_(1 <= bits <= SHIFT) + nbits = nbits + bits + answer = answer << bits + if r & 1: + answer = answer | ((1 << bits) - 1) + r = int(random.random() * (SHIFT * 2)) + self.assert_(nbits_lo <= nbits <= nbits_hi) + if random.random() < 0.5: + answer = -answer + return answer -def test_division(maxdigits=MAXDIGITS): - if verbose: - print "long / * % divmod" - digits = range(1, maxdigits+1) + range(KARATSUBA_CUTOFF, - KARATSUBA_CUTOFF + 14) - digits.append(KARATSUBA_CUTOFF * 3) - for lenx in digits: - x = getran(lenx) - for leny in digits: - y = getran(leny) or 1L - test_division_2(x, y) -# ------------------------------------------------------------ karatsuba + # Get random long consisting of ndigits random digits (relative to base + # BASE). The sign bit is also random. -def test_karatsuba(): + def getran2(ndigits): + answer = 0L + for i in xrange(ndigits): + answer = (answer << SHIFT) | random.randint(0, MASK) + if random.random() < 0.5: + answer = -answer + return answer - if verbose: - print "Karatsuba" + def check_division(self, x, y): + eq = self.assertEqual + q, r = divmod(x, y) + q2, r2 = x//y, x%y + pab, pba = x*y, y*x + eq(pab, pba, Frm("multiplication does not commute for %r and %r", x, y)) + eq(q, q2, Frm("divmod returns different quotient than / for %r and %r", x, y)) + eq(r, r2, Frm("divmod returns different mod than %% for %r and %r", x, y)) + eq(x, q*y + r, Frm("x != q*y + r after divmod on x=%r, y=%r", x, y)) + if y > 0: + self.assert_(0 <= r < y, Frm("bad mod from divmod on %r and %r", x, y)) + else: + self.assert_(y < r <= 0, Frm("bad mod from divmod on %r and %r", x, y)) - digits = range(1, 5) + range(KARATSUBA_CUTOFF, KARATSUBA_CUTOFF + 10) - digits.extend([KARATSUBA_CUTOFF * 10, KARATSUBA_CUTOFF * 100]) + def test_division(self): + digits = range(1, MAXDIGITS+1) + range(KARATSUBA_CUTOFF, + KARATSUBA_CUTOFF + 14) + digits.append(KARATSUBA_CUTOFF * 3) + for lenx in digits: + x = self.getran(lenx) + for leny in digits: + y = self.getran(leny) or 1L + self.check_division(x, y) - bits = [digit * SHIFT for digit in digits] + def test_karatsuba(self): + digits = range(1, 5) + range(KARATSUBA_CUTOFF, KARATSUBA_CUTOFF + 10) + digits.extend([KARATSUBA_CUTOFF * 10, KARATSUBA_CUTOFF * 100]) - # Test products of long strings of 1 bits -- (2**x-1)*(2**y-1) == - # 2**(x+y) - 2**x - 2**y + 1, so the proper result is easy to check. - for abits in bits: - a = (1L << abits) - 1 - for bbits in bits: - if bbits < abits: - continue - b = (1L << bbits) - 1 - x = a * b - y = ((1L << (abits + bbits)) - - (1L << abits) - - (1L << bbits) + - 1) - check(x == y, "bad result for", a, "*", b, x, y) -# -------------------------------------------------------------- ~ & | ^ + bits = [digit * SHIFT for digit in digits] -def test_bitop_identities_1(x): - check(x & 0 == 0, "x & 0 != 0 for", x) - check(x | 0 == x, "x | 0 != x for", x) - check(x ^ 0 == x, "x ^ 0 != x for", x) - check(x & -1 == x, "x & -1 != x for", x) - check(x | -1 == -1, "x | -1 != -1 for", x) - check(x ^ -1 == ~x, "x ^ -1 != ~x for", x) - check(x == ~~x, "x != ~~x for", x) - check(x & x == x, "x & x != x for", x) - check(x | x == x, "x | x != x for", x) - check(x ^ x == 0, "x ^ x != 0 for", x) - check(x & ~x == 0, "x & ~x != 0 for", x) - check(x | ~x == -1, "x | ~x != -1 for", x) - check(x ^ ~x == -1, "x ^ ~x != -1 for", x) - check(-x == 1 + ~x == ~(x-1), "not -x == 1 + ~x == ~(x-1) for", x) - for n in range(2*SHIFT): - p2 = 2L ** n - check(x << n >> n == x, "x << n >> n != x for", x, n) - check(x // p2 == x >> n, "x // p2 != x >> n for x n p2", x, n, p2) - check(x * p2 == x << n, "x * p2 != x << n for x n p2", x, n, p2) - check(x & -p2 == x >> n << n == x & ~(p2 - 1), - "not x & -p2 == x >> n << n == x & ~(p2 - 1) for x n p2", - x, n, p2) + # Test products of long strings of 1 bits -- (2**x-1)*(2**y-1) == + # 2**(x+y) - 2**x - 2**y + 1, so the proper result is easy to check. + for abits in bits: + a = (1L << abits) - 1 + for bbits in bits: + if bbits < abits: + continue + b = (1L << bbits) - 1 + x = a * b + y = ((1L << (abits + bbits)) - + (1L << abits) - + (1L << bbits) + + 1) + self.assertEqual(x, y, + Frm("bad result for a*b: a=%r, b=%r, x=%r, y=%r", a, b, x, y)) -def test_bitop_identities_2(x, y): - check(x & y == y & x, "x & y != y & x for", x, y) - check(x | y == y | x, "x | y != y | x for", x, y) - check(x ^ y == y ^ x, "x ^ y != y ^ x for", x, y) - check(x ^ y ^ x == y, "x ^ y ^ x != y for", x, y) - check(x & y == ~(~x | ~y), "x & y != ~(~x | ~y) for", x, y) - check(x | y == ~(~x & ~y), "x | y != ~(~x & ~y) for", x, y) - check(x ^ y == (x | y) & ~(x & y), - "x ^ y != (x | y) & ~(x & y) for", x, y) - check(x ^ y == (x & ~y) | (~x & y), - "x ^ y == (x & ~y) | (~x & y) for", x, y) - check(x ^ y == (x | y) & (~x | ~y), - "x ^ y == (x | y) & (~x | ~y) for", x, y) + def check_bitop_identities_1(self, x): + eq = self.assertEqual + eq(x & 0, 0, Frm("x & 0 != 0 for x=%r", x)) + eq(x | 0, x, Frm("x | 0 != x for x=%r", x)) + eq(x ^ 0, x, Frm("x ^ 0 != x for x=%r", x)) + eq(x & -1, x, Frm("x & -1 != x for x=%r", x)) + eq(x | -1, -1, Frm("x | -1 != -1 for x=%r", x)) + eq(x ^ -1, ~x, Frm("x ^ -1 != ~x for x=%r", x)) + eq(x, ~~x, Frm("x != ~~x for x=%r", x)) + eq(x & x, x, Frm("x & x != x for x=%r", x)) + eq(x | x, x, Frm("x | x != x for x=%r", x)) + eq(x ^ x, 0, Frm("x ^ x != 0 for x=%r", x)) + eq(x & ~x, 0, Frm("x & ~x != 0 for x=%r", x)) + eq(x | ~x, -1, Frm("x | ~x != -1 for x=%r", x)) + eq(x ^ ~x, -1, Frm("x ^ ~x != -1 for x=%r", x)) + eq(-x, 1 + ~x, Frm("not -x == 1 + ~x for x=%r", x)) + eq(-x, ~(x-1), Frm("not -x == ~(x-1) forx =%r", x)) + for n in xrange(2*SHIFT): + p2 = 2L ** n + eq(x << n >> n, x, + Frm("x << n >> n != x for x=%r, n=%r", (x, n))) + eq(x // p2, x >> n, + Frm("x // p2 != x >> n for x=%r n=%r p2=%r", (x, n, p2))) + eq(x * p2, x << n, + Frm("x * p2 != x << n for x=%r n=%r p2=%r", (x, n, p2))) + eq(x & -p2, x >> n << n, + Frm("not x & -p2 == x >> n << n for x=%r n=%r p2=%r", (x, n, p2))) + eq(x & -p2, x & ~(p2 - 1), + Frm("not x & -p2 == x & ~(p2 - 1) for x=%r n=%r p2=%r", (x, n, p2))) -def test_bitop_identities_3(x, y, z): - check((x & y) & z == x & (y & z), - "(x & y) & z != x & (y & z) for", x, y, z) - check((x | y) | z == x | (y | z), - "(x | y) | z != x | (y | z) for", x, y, z) - check((x ^ y) ^ z == x ^ (y ^ z), - "(x ^ y) ^ z != x ^ (y ^ z) for", x, y, z) - check(x & (y | z) == (x & y) | (x & z), - "x & (y | z) != (x & y) | (x & z) for", x, y, z) - check(x | (y & z) == (x | y) & (x | z), - "x | (y & z) != (x | y) & (x | z) for", x, y, z) + def check_bitop_identities_2(self, x, y): + eq = self.assertEqual + eq(x & y, y & x, Frm("x & y != y & x for x=%r, y=%r", (x, y))) + eq(x | y, y | x, Frm("x | y != y | x for x=%r, y=%r", (x, y))) + eq(x ^ y, y ^ x, Frm("x ^ y != y ^ x for x=%r, y=%r", (x, y))) + eq(x ^ y ^ x, y, Frm("x ^ y ^ x != y for x=%r, y=%r", (x, y))) + eq(x & y, ~(~x | ~y), Frm("x & y != ~(~x | ~y) for x=%r, y=%r", (x, y))) + eq(x | y, ~(~x & ~y), Frm("x | y != ~(~x & ~y) for x=%r, y=%r", (x, y))) + eq(x ^ y, (x | y) & ~(x & y), + Frm("x ^ y != (x | y) & ~(x & y) for x=%r, y=%r", (x, y))) + eq(x ^ y, (x & ~y) | (~x & y), + Frm("x ^ y == (x & ~y) | (~x & y) for x=%r, y=%r", (x, y))) + eq(x ^ y, (x | y) & (~x | ~y), + Frm("x ^ y == (x | y) & (~x | ~y) for x=%r, y=%r", (x, y))) -def test_bitop_identities(maxdigits=MAXDIGITS): - if verbose: - print "long bit-operation identities" - for x in special: - test_bitop_identities_1(x) - digits = range(1, maxdigits+1) - for lenx in digits: - x = getran(lenx) - test_bitop_identities_1(x) - for leny in digits: - y = getran(leny) - test_bitop_identities_2(x, y) - test_bitop_identities_3(x, y, getran((lenx + leny)//2)) + def check_bitop_identities_3(self, x, y, z): + eq = self.assertEqual + eq((x & y) & z, x & (y & z), + Frm("(x & y) & z != x & (y & z) for x=%r, y=%r, z=%r", (x, y, z))) + eq((x | y) | z, x | (y | z), + Frm("(x | y) | z != x | (y | z) for x=%r, y=%r, z=%r", (x, y, z))) + eq((x ^ y) ^ z, x ^ (y ^ z), + Frm("(x ^ y) ^ z != x ^ (y ^ z) for x=%r, y=%r, z=%r", (x, y, z))) + eq(x & (y | z), (x & y) | (x & z), + Frm("x & (y | z) != (x & y) | (x & z) for x=%r, y=%r, z=%r", (x, y, z))) + eq(x | (y & z), (x | y) & (x | z), + Frm("x | (y & z) != (x | y) & (x | z) for x=%r, y=%r, z=%r", (x, y, z))) -# ------------------------------------------------- hex oct repr str atol + def test_bitop_identities(self): + for x in special: + self.check_bitop_identities_1(x) + digits = xrange(1, MAXDIGITS+1) + for lenx in digits: + x = self.getran(lenx) + self.check_bitop_identities_1(x) + for leny in digits: + y = self.getran(leny) + self.check_bitop_identities_2(x, y) + self.check_bitop_identities_3(x, y, self.getran((lenx + leny)//2)) -def slow_format(x, base): - if (x, base) == (0, 8): - # this is an oddball! - return "0L" - digits = [] - sign = 0 - if x < 0: - sign, x = 1, -x - while x: - x, r = divmod(x, base) - digits.append(int(r)) - digits.reverse() - digits = digits or [0] - return '-'[:sign] + \ - {8: '0', 10: '', 16: '0x'}[base] + \ - join(map(lambda i: "0123456789ABCDEF"[i], digits), '') + \ - "L" + def slow_format(self, x, base): + if (x, base) == (0, 8): + # this is an oddball! + return "0L" + digits = [] + sign = 0 + if x < 0: + sign, x = 1, -x + while x: + x, r = divmod(x, base) + digits.append(int(r)) + digits.reverse() + digits = digits or [0] + return '-'[:sign] + \ + {8: '0', 10: '', 16: '0x'}[base] + \ + "".join(map(lambda i: "0123456789ABCDEF"[i], digits)) + "L" -def test_format_1(x): - from string import atol - for base, mapper in (8, oct), (10, repr), (16, hex): - got = mapper(x) - expected = slow_format(x, base) - check(got == expected, mapper.__name__, "returned", - got, "but expected", expected, "for", x) - check(atol(got, 0) == x, 'atol("%s", 0) !=' % got, x) - # str() has to be checked a little differently since there's no - # trailing "L" - got = str(x) - expected = slow_format(x, 10)[:-1] - check(got == expected, mapper.__name__, "returned", - got, "but expected", expected, "for", x) + def check_format_1(self, x): + for base, mapper in (8, oct), (10, repr), (16, hex): + got = mapper(x) + expected = self.slow_format(x, base) + msg = Frm("%s returned %r but expected %r for %r", + mapper.__name__, got, expected, x) + self.assertEqual(got, expected, msg) + self.assertEqual(long(got, 0), x, Frm('long("%s", 0) != %r', got, x)) + # str() has to be checked a little differently since there's no + # trailing "L" + got = str(x) + expected = self.slow_format(x, 10)[:-1] + msg = Frm("%s returned %r but expected %r for %r", + mapper.__name__, got, expected, x) + self.assertEqual(got, expected, msg) -def test_format(maxdigits=MAXDIGITS): - if verbose: - print "long str/hex/oct/atol" - for x in special: - test_format_1(x) - for i in range(10): - for lenx in range(1, maxdigits+1): - x = getran(lenx) - test_format_1(x) + def test_format(self): + for x in special: + self.check_format_1(x) + for i in xrange(10): + for lenx in xrange(1, MAXDIGITS+1): + x = self.getran(lenx) + self.check_format_1(x) -# ----------------------------------------------------------------- misc + def test_misc(self): + import sys -def test_misc(maxdigits=MAXDIGITS): - if verbose: - print "long miscellaneous operations" - import sys + # check the extremes in int<->long conversion + hugepos = sys.maxint + hugeneg = -hugepos - 1 + hugepos_aslong = long(hugepos) + hugeneg_aslong = long(hugeneg) + self.assertEqual(hugepos, hugepos_aslong, "long(sys.maxint) != sys.maxint") + self.assertEqual(hugeneg, hugeneg_aslong, + "long(-sys.maxint-1) != -sys.maxint-1") - # check the extremes in int<->long conversion - hugepos = sys.maxint - hugeneg = -hugepos - 1 - hugepos_aslong = long(hugepos) - hugeneg_aslong = long(hugeneg) - check(hugepos == hugepos_aslong, "long(sys.maxint) != sys.maxint") - check(hugeneg == hugeneg_aslong, - "long(-sys.maxint-1) != -sys.maxint-1") + # long -> int should not fail for hugepos_aslong or hugeneg_aslong + try: + self.assertEqual(int(hugepos_aslong), hugepos, + "converting sys.maxint to long and back to int fails") + except OverflowError: + self.fail("int(long(sys.maxint)) overflowed!") + try: + self.assertEqual(int(hugeneg_aslong), hugeneg, + "converting -sys.maxint-1 to long and back to int fails") + except OverflowError: + self.fail("int(long(-sys.maxint-1)) overflowed!") - # long -> int should not fail for hugepos_aslong or hugeneg_aslong - try: - check(int(hugepos_aslong) == hugepos, - "converting sys.maxint to long and back to int fails") - except OverflowError: - raise TestFailed, "int(long(sys.maxint)) overflowed!" - try: - check(int(hugeneg_aslong) == hugeneg, - "converting -sys.maxint-1 to long and back to int fails") - except OverflowError: - raise TestFailed, "int(long(-sys.maxint-1)) overflowed!" + # but long -> int should overflow for hugepos+1 and hugeneg-1 + x = hugepos_aslong + 1 + try: + y = int(x) + except OverflowError: + self.fail("int(long(sys.maxint) + 1) mustn't overflow") + self.assert_(isinstance(y, long), + "int(long(sys.maxint) + 1) should have returned long") - # but long -> int should overflow for hugepos+1 and hugeneg-1 - x = hugepos_aslong + 1 - try: - y = int(x) - except OverflowError: - raise TestFailed, "int(long(sys.maxint) + 1) mustn't overflow" - if not isinstance(y, long): - raise TestFailed("int(long(sys.maxint) + 1) should have returned long") + x = hugeneg_aslong - 1 + try: + y = int(x) + except OverflowError: + self.fail("int(long(-sys.maxint-1) - 1) mustn't overflow") + self.assert_(isinstance(y, long), + "int(long(-sys.maxint-1) - 1) should have returned long") - x = hugeneg_aslong - 1 - try: + class long2(long): + pass + x = long2(1L<<100) y = int(x) - except OverflowError: - raise TestFailed, "int(long(-sys.maxint-1) - 1) mustn't overflow" - if not isinstance(y, long): - raise TestFailed("int(long(-sys.maxint-1) - 1) should have returned long") + self.assert_(type(y) is long, + "overflowing int conversion must return long not long subtype") - class long2(long): - pass - x = long2(1L<<100) - y = int(x) - if type(y) is not long: - raise TestFailed("overflowing int conversion must return long not long subtype") # ----------------------------------- tests of auto int->long conversion -def test_auto_overflow(): - import math, sys - - if verbose: - print "auto-convert int->long on overflow" + def test_auto_overflow(self): + import math, sys - special = [0, 1, 2, 3, sys.maxint-1, sys.maxint, sys.maxint+1] - sqrt = int(math.sqrt(sys.maxint)) - special.extend([sqrt-1, sqrt, sqrt+1]) - special.extend([-i for i in special]) + special = [0, 1, 2, 3, sys.maxint-1, sys.maxint, sys.maxint+1] + sqrt = int(math.sqrt(sys.maxint)) + special.extend([sqrt-1, sqrt, sqrt+1]) + special.extend([-i for i in special]) - def checkit(*args): - # Heavy use of nested scopes here! - verify(got == expected, "for %r expected %r got %r" % - (args, expected, got)) + def checkit(*args): + # Heavy use of nested scopes here! + self.assertEqual(got, expected, + Frm("for %r expected %r got %r", args, expected, got)) - for x in special: - longx = long(x) + for x in special: + longx = long(x) - expected = -longx - got = -x - checkit('-', x) + expected = -longx + got = -x + checkit('-', x) - for y in special: - longy = long(y) + for y in special: + longy = long(y) - expected = longx + longy - got = x + y - checkit(x, '+', y) + expected = longx + longy + got = x + y + checkit(x, '+', y) - expected = longx - longy - got = x - y - checkit(x, '-', y) + expected = longx - longy + got = x - y + checkit(x, '-', y) - expected = longx * longy - got = x * y - checkit(x, '*', y) + expected = longx * longy + got = x * y + checkit(x, '*', y) - if y: - expected = longx / longy - got = x / y - checkit(x, '/', y) + if y: + expected = longx / longy + got = x / y + checkit(x, '/', y) - expected = longx // longy - got = x // y - checkit(x, '//', y) + expected = longx // longy + got = x // y + checkit(x, '//', y) - expected = divmod(longx, longy) - got = divmod(longx, longy) - checkit(x, 'divmod', y) + expected = divmod(longx, longy) + got = divmod(longx, longy) + checkit(x, 'divmod', y) - if abs(y) < 5 and not (x == 0 and y < 0): - expected = longx ** longy - got = x ** y - checkit(x, '**', y) + if abs(y) < 5 and not (x == 0 and y < 0): + expected = longx ** longy + got = x ** y + checkit(x, '**', y) - for z in special: - if z != 0 : - if y >= 0: - expected = pow(longx, longy, long(z)) - got = pow(x, y, z) - checkit('pow', x, y, '%', z) - else: - try: - pow(longx, longy, long(z)) - except TypeError: - pass + for z in special: + if z != 0 : + if y >= 0: + expected = pow(longx, longy, long(z)) + got = pow(x, y, z) + checkit('pow', x, y, '%', z) else: - raise TestFailed("pow%r should have raised " - "TypeError" % ((longx, longy, long(z)),)) - -# ---------------------------------------- tests of long->float overflow - -def test_float_overflow(): - import math - - if verbose: - print "long->float overflow" - - for x in -2.0, -1.0, 0.0, 1.0, 2.0: - verify(float(long(x)) == x) - - shuge = '12345' * 120 - huge = 1L << 30000 - mhuge = -huge - namespace = {'huge': huge, 'mhuge': mhuge, 'shuge': shuge, 'math': math} - for test in ["float(huge)", "float(mhuge)", - "complex(huge)", "complex(mhuge)", - "complex(huge, 1)", "complex(mhuge, 1)", - "complex(1, huge)", "complex(1, mhuge)", - "1. + huge", "huge + 1.", "1. + mhuge", "mhuge + 1.", - "1. - huge", "huge - 1.", "1. - mhuge", "mhuge - 1.", - "1. * huge", "huge * 1.", "1. * mhuge", "mhuge * 1.", - "1. // huge", "huge // 1.", "1. // mhuge", "mhuge // 1.", - "1. / huge", "huge / 1.", "1. / mhuge", "mhuge / 1.", - "1. ** huge", "huge ** 1.", "1. ** mhuge", "mhuge ** 1.", - "math.sin(huge)", "math.sin(mhuge)", - "math.sqrt(huge)", "math.sqrt(mhuge)", # should do better - "math.floor(huge)", "math.floor(mhuge)"]: - - try: - eval(test, namespace) - except OverflowError: - pass - else: - raise TestFailed("expected OverflowError from %s" % test) - - # XXX Perhaps float(shuge) can raise OverflowError on some box? - # The comparison should not. - if float(shuge) == int(shuge): - raise TestFailed("float(shuge) should not equal int(shuge)") - -# ---------------------------------------------- test huge log and log10 + self.assertRaises(TypeError, pow,longx, longy, long(z)) -def test_logs(): - import math + def test_float_overflow(self): + import math - if verbose: - print "log and log10" + for x in -2.0, -1.0, 0.0, 1.0, 2.0: + self.assertEqual(float(long(x)), x) - LOG10E = math.log10(math.e) + shuge = '12345' * 120 + huge = 1L << 30000 + mhuge = -huge + namespace = {'huge': huge, 'mhuge': mhuge, 'shuge': shuge, 'math': math} + for test in ["float(huge)", "float(mhuge)", + "complex(huge)", "complex(mhuge)", + "complex(huge, 1)", "complex(mhuge, 1)", + "complex(1, huge)", "complex(1, mhuge)", + "1. + huge", "huge + 1.", "1. + mhuge", "mhuge + 1.", + "1. - huge", "huge - 1.", "1. - mhuge", "mhuge - 1.", + "1. * huge", "huge * 1.", "1. * mhuge", "mhuge * 1.", + "1. // huge", "huge // 1.", "1. // mhuge", "mhuge // 1.", + "1. / huge", "huge / 1.", "1. / mhuge", "mhuge / 1.", + "1. ** huge", "huge ** 1.", "1. ** mhuge", "mhuge ** 1.", + "math.sin(huge)", "math.sin(mhuge)", + "math.sqrt(huge)", "math.sqrt(mhuge)", # should do better + "math.floor(huge)", "math.floor(mhuge)"]: - for exp in range(10) + [100, 1000, 10000]: - value = 10 ** exp - log10 = math.log10(value) - verify(fcmp(log10, exp) == 0) + self.assertRaises(OverflowError, eval, test, namespace) - # log10(value) == exp, so log(value) == log10(value)/log10(e) == - # exp/LOG10E - expected = exp / LOG10E - log = math.log(value) - verify(fcmp(log, expected) == 0) + # XXX Perhaps float(shuge) can raise OverflowError on some box? + # The comparison should not. + self.assertNotEqual(float(shuge), int(shuge), + "float(shuge) should not equal int(shuge)") - for bad in -(1L << 10000), -2L, 0L: - try: - math.log(bad) - raise TestFailed("expected ValueError from log(<= 0)") - except ValueError: - pass + def test_logs(self): + import math - try: - math.log10(bad) - raise TestFailed("expected ValueError from log10(<= 0)") - except ValueError: - pass + LOG10E = math.log10(math.e) -# ----------------------------------------------- test mixed comparisons + for exp in range(10) + [100, 1000, 10000]: + value = 10 ** exp + log10 = math.log10(value) + self.assertAlmostEqual(log10, exp) -def test_mixed_compares(): - import math - import sys + # log10(value) == exp, so log(value) == log10(value)/log10(e) == + # exp/LOG10E + expected = exp / LOG10E + log = math.log(value) + self.assertAlmostEqual(log, expected) - if verbose: - print "mixed comparisons" + for bad in -(1L << 10000), -2L, 0L: + self.assertRaises(ValueError, math.log, bad) + self.assertRaises(ValueError, math.log10, bad) - # We're mostly concerned with that mixing floats and longs does the - # right stuff, even when longs are too large to fit in a float. - # The safest way to check the results is to use an entirely different - # method, which we do here via a skeletal rational class (which - # represents all Python ints, longs and floats exactly). - class Rat: - def __init__(self, value): - if isinstance(value, (int, long)): - self.n = value - self.d = 1 + def test_mixed_compares(self): + eq = self.assertEqual + import math + import sys - elif isinstance(value, float): - # Convert to exact rational equivalent. - f, e = math.frexp(abs(value)) - assert f == 0 or 0.5 <= f < 1.0 - # |value| = f * 2**e exactly + # We're mostly concerned with that mixing floats and longs does the + # right stuff, even when longs are too large to fit in a float. + # The safest way to check the results is to use an entirely different + # method, which we do here via a skeletal rational class (which + # represents all Python ints, longs and floats exactly). + class Rat: + def __init__(self, value): + if isinstance(value, (int, long)): + self.n = value + self.d = 1 + elif isinstance(value, float): + # Convert to exact rational equivalent. + f, e = math.frexp(abs(value)) + assert f == 0 or 0.5 <= f < 1.0 + # |value| = f * 2**e exactly - # Suck up CHUNK bits at a time; 28 is enough so that we suck - # up all bits in 2 iterations for all known binary double- - # precision formats, and small enough to fit in an int. - CHUNK = 28 - top = 0 - # invariant: |value| = (top + f) * 2**e exactly - while f: - f = math.ldexp(f, CHUNK) - digit = int(f) - assert digit >> CHUNK == 0 - top = (top << CHUNK) | digit - f -= digit - assert 0.0 <= f < 1.0 - e -= CHUNK + # Suck up CHUNK bits at a time; 28 is enough so that we suck + # up all bits in 2 iterations for all known binary double- + # precision formats, and small enough to fit in an int. + CHUNK = 28 + top = 0 + # invariant: |value| = (top + f) * 2**e exactly + while f: + f = math.ldexp(f, CHUNK) + digit = int(f) + assert digit >> CHUNK == 0 + top = (top << CHUNK) | digit + f -= digit + assert 0.0 <= f < 1.0 + e -= CHUNK - # Now |value| = top * 2**e exactly. - if e >= 0: - n = top << e - d = 1 + # Now |value| = top * 2**e exactly. + if e >= 0: + n = top << e + d = 1 + else: + n = top + d = 1 << -e + if value < 0: + n = -n + self.n = n + self.d = d + assert float(n) / float(d) == value else: - n = top - d = 1 << -e - if value < 0: - n = -n - self.n = n - self.d = d - assert float(n) / float(d) == value - - else: - raise TypeError("can't deal with %r" % val) + raise TypeError("can't deal with %r" % val) - def __cmp__(self, other): - if not isinstance(other, Rat): - other = Rat(other) - return cmp(self.n * other.d, self.d * other.n) + def __cmp__(self, other): + if not isinstance(other, Rat): + other = Rat(other) + return cmp(self.n * other.d, self.d * other.n) - cases = [0, 0.001, 0.99, 1.0, 1.5, 1e20, 1e200] - # 2**48 is an important boundary in the internals. 2**53 is an - # important boundary for IEEE double precision. - for t in 2.0**48, 2.0**50, 2.0**53: - cases.extend([t - 1.0, t - 0.3, t, t + 0.3, t + 1.0, - long(t-1), long(t), long(t+1)]) - cases.extend([0, 1, 2, sys.maxint, float(sys.maxint)]) - # 1L<<20000 should exceed all double formats. long(1e200) is to - # check that we get equality with 1e200 above. - t = long(1e200) - cases.extend([0L, 1L, 2L, 1L << 20000, t-1, t, t+1]) - cases.extend([-x for x in cases]) - for x in cases: - Rx = Rat(x) - for y in cases: - Ry = Rat(y) - Rcmp = cmp(Rx, Ry) - xycmp = cmp(x, y) - if Rcmp != xycmp: - raise TestFailed('%r %r %d %d' % (x, y, Rcmp, xycmp)) - if (x == y) != (Rcmp == 0): - raise TestFailed('%r == %r %d' % (x, y, Rcmp)) - if (x != y) != (Rcmp != 0): - raise TestFailed('%r != %r %d' % (x, y, Rcmp)) - if (x < y) != (Rcmp < 0): - raise TestFailed('%r < %r %d' % (x, y, Rcmp)) - if (x <= y) != (Rcmp <= 0): - raise TestFailed('%r <= %r %d' % (x, y, Rcmp)) - if (x > y) != (Rcmp > 0): - raise TestFailed('%r > %r %d' % (x, y, Rcmp)) - if (x >= y) != (Rcmp >= 0): - raise TestFailed('%r >= %r %d' % (x, y, Rcmp)) + cases = [0, 0.001, 0.99, 1.0, 1.5, 1e20, 1e200] + # 2**48 is an important boundary in the internals. 2**53 is an + # important boundary for IEEE double precision. + for t in 2.0**48, 2.0**50, 2.0**53: + cases.extend([t - 1.0, t - 0.3, t, t + 0.3, t + 1.0, + long(t-1), long(t), long(t+1)]) + cases.extend([0, 1, 2, sys.maxint, float(sys.maxint)]) + # 1L<<20000 should exceed all double formats. long(1e200) is to + # check that we get equality with 1e200 above. + t = long(1e200) + cases.extend([0L, 1L, 2L, 1L << 20000, t-1, t, t+1]) + cases.extend([-x for x in cases]) + for x in cases: + Rx = Rat(x) + for y in cases: + Ry = Rat(y) + Rcmp = cmp(Rx, Ry) + xycmp = cmp(x, y) + eq(Rcmp, xycmp, Frm("%r %r %d %d", x, y, Rcmp, xycmp)) + eq(x == y, Rcmp == 0, Frm("%r == %r %d", x, y, Rcmp)) + eq(x != y, Rcmp != 0, Frm("%r != %r %d", x, y, Rcmp)) + eq(x < y, Rcmp < 0, Frm("%r < %r %d", x, y, Rcmp)) + eq(x <= y, Rcmp <= 0, Frm("%r <= %r %d", x, y, Rcmp)) + eq(x > y, Rcmp > 0, Frm("%r > %r %d", x, y, Rcmp)) + eq(x >= y, Rcmp >= 0, Frm("%r >= %r %d", x, y, Rcmp)) -# ---------------------------------------------------------------- do it +def test_main(): + test_support.run_unittest(LongTest) -test_division() -test_karatsuba() -test_bitop_identities() -test_format() -test_misc() -test_auto_overflow() -test_float_overflow() -test_logs() -test_mixed_compares() +if __name__ == "__main__": + test_main() From pje at users.sourceforge.net Tue Jun 14 03:25:50 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 13 Jun 2005 18:25:50 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.29, 1.30 Message-ID: <E1Di0Ba-0001AP-Cm@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4382 Modified Files: pkg_resources.py Log Message: Fix a bug in resource_isdir(), introduced in 0.4a2. Add str/repr of Distribution objects. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.29 retrieving revision 1.30 diff -u -d -r1.29 -r1.30 --- pkg_resources.py 12 Jun 2005 21:47:33 -0000 1.29 +++ pkg_resources.py 14 Jun 2005 01:25:48 -0000 1.30 @@ -565,15 +565,15 @@ def get_metadata_lines(self, name): return yield_lines(self.get_metadata(name)) - def resource_isdir(self,name): + def resource_isdir(self,resource_name): return self._isdir(self._fn(self.module_path, resource_name)) def metadata_isdir(self,name): return self.egg_info and self._isdir(self._fn(self.egg_info,name)) - def resource_listdir(self,name): - return self._listdir(self._fn(self.egg_info,name)) + def resource_listdir(self,resource_name): + return self._listdir(self._fn(self.egg_info,resource_name)) def metadata_listdir(self,name): if self.egg_info: @@ -1237,7 +1237,7 @@ ): if name: self.name = safe_name(name) - if version: + if version is not None: self._version = safe_version(version) self.py_version = py_version self.platform = platform @@ -1363,12 +1363,12 @@ return filename + def __repr__(self): + return "%s (%s)" % (self,self.path) - - - - - + def __str__(self): + version = getattr(self,'version',None) or "[unknown version]" + return "%s %s" % (self.name,version) From pje at users.sourceforge.net Tue Jun 14 03:26:29 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 13 Jun 2005 18:26:29 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools package_index.py, 1.1, 1.2 Message-ID: <E1Di0CD-0001Cw-FX@sc8-pr-cvs1.sourceforge.net> Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4594/setuptools Modified Files: package_index.py Log Message: Add lots of progress messages, so people know what the package search is doing. Index: package_index.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/package_index.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- package_index.py 12 Jun 2005 03:44:07 -0000 1.1 +++ package_index.py 14 Jun 2005 01:26:26 -0000 1.2 @@ -90,21 +90,21 @@ self.fetched_urls = {} self.package_pages = {} - def scan_url(self, url): - self.process_url(url, True) - def process_url(self, url, retrieve=False): if url in self.scanned_urls and not retrieve: return self.scanned_urls[url] = True dists = list(distros_for_url(url)) - map(self.add, dists) + if dists: self.debug("Found link: %s", url) - if dists or not retrieve or url in self.fetched_urls: + if dists or not retrieve or url in self.fetched_urls: + for dist in dists: + self.add(dist) # don't need the actual page return + self.info("Reading %s", url) f = self.open_url(url) self.fetched_urls[url] = self.fetched_urls[f.url] = True if 'html' not in f.headers['content-type'].lower(): @@ -121,17 +121,11 @@ link = urlparse.urljoin(base, match.group(1)) self.process_url(link) - def find_packages(self,requirement): - self.scan_url(self.index_url + requirement.distname) - if not self.package_pages.get(requirement.key): - # We couldn't find the target package, so search the index page too - self.scan_url(self.index_url) - for url in self.package_pages.get(requirement.key,()): - # scan each page that might be related to the desired package - self.scan_url(url) - def process_index(self,url,page): + """Process the contents of a PyPI page""" + def scan(link): + # Process a URL to see if it's for a package page if link.startswith(self.index_url): parts = map( urllib2.unquote, link[len(self.index_url):].split('/') @@ -141,10 +135,12 @@ pkg = safe_name(parts[0]) ver = safe_version(parts[1]) self.package_pages.setdefault(pkg.lower(),{})[link] = True + if url==self.index_url or 'Index of Packages' in page: # process an index page into the package-page index for match in HREF.finditer(page): scan( urlparse.urljoin(url, match.group(1)) ) + else: scan(url) # ensure this page is in the page index # process individual package page @@ -156,11 +152,56 @@ # Process the found URL self.scan_url(urlparse.urljoin(url, match.group(1))) + + + + + + + + + + + def find_packages(self,requirement): + self.scan_url(self.index_url + requirement.distname+'/') + if not self.package_pages.get(requirement.key): + # We couldn't find the target package, so search the index page too + self.warn( + "Couldn't find index page for %r (maybe misspelled?)", + requirement.distname + ) + if self.index_url not in self.fetched_urls: + self.warn( + "Scanning index of all packages (this may take a while)" + ) + self.scan_url(self.index_url) + + for url in self.package_pages.get(requirement.key,()): + # scan each page that might be related to the desired package + self.scan_url(url) + def obtain(self,requirement): self.find_packages(requirement) for dist in self.get(requirement.key, ()): if dist in requirement: return dist + self.debug("%s does not match %s", requirement, dist) + + + + + + + + + + + + + + + + def download(self, spec, tmpdir): """Locate and/or download `spec`, returning a local filename @@ -193,19 +234,21 @@ "Not a URL, existing file, or requirement spec: %r" % (spec,) ) - # process a Requirement + self.info("Searching for %s", spec) dist = self.best_match(spec,[]) if dist is not None: + self.info("Best match: %s", dist) return self.download(dist.path, tmpdir) + self.warn("No local packages or download links found for %s", spec) return None - - dl_blocksize = 8192 def _download_to(self, url, filename): + self.info("Downloading %s", url) + # Download the file fp, tfp = None, None try: @@ -242,8 +285,6 @@ def reporthook(self, url, filename, blocknum, blksize, size): pass # no-op - - def open_url(self, url): try: return urllib2.urlopen(url) @@ -278,8 +319,8 @@ else: return filename - - + def scan_url(self, url): + self.process_url(url, True) @@ -313,22 +354,24 @@ def _download_svn(self, url, filename): + self.info("Doing subversion checkout from %s to %s", url, filename) os.system("svn checkout -q %s %s" % (url, filename)) return filename + def debug(self, msg, *args): + pass #print msg % args # XXX - - - - - - - - + def info(self, msg, *args): + print msg % args # XXX + + def warn(self, msg, *args): + print msg % args # XXX def _download_sourceforge(self, source_url, sf_page, tmpdir): """Download package from randomly-selected SourceForge mirror""" + self.debug("Processing SourceForge mirror page") + mirror_regex = re.compile(r'HREF=(/.*?\?use_mirror=[^>]*)') urls = [m.group(1) for m in mirror_regex.finditer(sf_page)] if not urls: @@ -338,6 +381,12 @@ import random url = urlparse.urljoin(source_url, random.choice(urls)) + + self.info( + "Requesting redirect to (randomly selected) %r mirror", + url.split('=',1)[-1] + ) + f = self.open_url(url) match = re.search( r' Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4879 Modified Files: easy_install.py Log Message: Cosmetic improvements to progress messages. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.19 retrieving revision 1.20 diff -u -d -r1.19 -r1.20 --- easy_install.py 12 Jun 2005 21:47:32 -0000 1.19 +++ easy_install.py 14 Jun 2005 01:27:12 -0000 1.20 @@ -143,14 +143,13 @@ def easy_install(self, spec): tmpdir = self.alloc_tmp() try: - print "Downloading", spec download = self.package_index.download(spec, tmpdir) if download is None: raise RuntimeError( "Could not find distribution for %r" % spec ) - print "Installing", os.path.basename(download) + print "Processing", os.path.basename(download) for dist in self.install_eggs(download, self.zip_ok, tmpdir): self.package_index.add(dist) self.install_egg_scripts(dist) @@ -162,6 +161,7 @@ + def install_egg_scripts(self, dist): metadata = dist.metadata if self.exclude_scripts or not metadata.metadata_isdir('scripts'): @@ -172,7 +172,7 @@ for script_name in metadata.metadata_listdir('scripts'): target = os.path.join(self.script_dir, script_name) - print "Installing", script_name, "to", target + print "Installing", script_name, "script to", self.script_dir script_text = metadata.get_metadata('scripts/'+script_name) script_text = script_text.replace('\r','\n') @@ -226,10 +226,10 @@ "Multiple setup scripts in %s" % dist_filename ) setup_script = setups[0] - from setuptools.command import bdist_egg sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) try: + print "Running", setup_script[len(tmpdir)+1:] run_setup(setup_script, ['-q', 'bdist_egg']) except SystemExit, v: raise RuntimeError( From pje at users.sourceforge.net Tue Jun 14 03:28:47 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 13 Jun 2005 18:28:47 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.30, 1.31 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5704 Modified Files: pkg_resources.py Log Message: Fix missing '__file__' when running scripts. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.30 retrieving revision 1.31 diff -u -d -r1.30 -r1.31 --- pkg_resources.py 14 Jun 2005 01:25:48 -0000 1.30 +++ pkg_resources.py 14 Jun 2005 01:28:44 -0000 1.31 @@ -587,7 +587,7 @@ script_text = self.get_metadata(script).replace('\r\n','\n') script_text = script_text.replace('\r','\n') script_filename = self._fn(self.egg_info,script) - + namespace['__file__'] = script_filename if os.path.exists(script_filename): execfile(script_filename, namespace, namespace) else: From pje at users.sourceforge.net Tue Jun 14 03:30:01 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 13 Jun 2005 18:30:01 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.11, 1.12 setup.py, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6326 Modified Files: EasyInstall.txt setup.py Log Message: Update to version 0.4a3 Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.11 retrieving revision 1.12 diff -u -d -r1.11 -r1.12 --- EasyInstall.txt 12 Jun 2005 21:47:32 -0000 1.11 +++ EasyInstall.txt 14 Jun 2005 01:29:59 -0000 1.12 @@ -62,7 +62,7 @@ **Example 2**. Install or upgrade a package by name and version by finding links on a given "download page":: - easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.4a2" + easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.4a3" **Example 3**. Download a source distribution from a specified URL, automatically building and installing it:: @@ -385,9 +385,20 @@ * There's no automatic retry for borked Sourceforge mirrors, which can easily time out or be missing a file. - * EasyInstall does not yet respect the distutils "verbose/quite" and "dry-run" + * EasyInstall does not yet respect the distutils "verbose/quiet" and "dry-run" options, even though it accepts them. +0.4a3 + * Fixed scripts not being able to see a ``__file__`` variable in ``__main__`` + + * Fixed a problem with ``resource_isdir()`` implementation that was introduced + in 0.4a2. + + * Add progress messages to the search/download process so that you can tell + what URLs it's reading to find download links. (Hopefully, this will help + people report out-of-date and broken links to package authors, and to tell + when they've asked for a package that doesn't exist.) + 0.4a2 * Added support for installing scripts Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- setup.py 12 Jun 2005 21:47:33 -0000 1.12 +++ setup.py 14 Jun 2005 01:29:59 -0000 1.13 @@ -1,7 +1,7 @@ #!/usr/bin/env python """Distutils setup file, used to install or test 'setuptools'""" -VERSION = "0.4a2" +VERSION = "0.4a3" from setuptools import setup, find_packages, Require setup( From pje at users.sourceforge.net Tue Jun 14 03:30:01 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 13 Jun 2005 18:30:01 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools __init__.py, 1.4, 1.5 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6326/setuptools Modified Files: __init__.py Log Message: Update to version 0.4a3 Index: __init__.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/__init__.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- __init__.py 12 Jun 2005 21:47:33 -0000 1.4 +++ __init__.py 14 Jun 2005 01:29:59 -0000 1.5 @@ -8,7 +8,7 @@ from distutils.util import convert_path import os.path -__version__ = '0.4a2' +__version__ = '0.4a3' __all__ = [ 'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require', From gvanrossum at users.sourceforge.net Tue Jun 14 03:30:59 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Mon, 13 Jun 2005 18:30:59 -0700 Subject: [Python-checkins] python/nondist/peps pep-0342.txt,1.2,1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7086 Modified Files: pep-0342.txt Log Message: Update: at this point I'm leaning towards preferring next() over __next__() again, but I've no time to update the PEP right now. I've changed the title to Coroutines via Enhanced Generators at Timothy Delaney's suggestion. Index: pep-0342.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0342.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0342.txt 11 May 2005 22:09:37 -0000 1.2 +++ pep-0342.txt 14 Jun 2005 01:30:57 -0000 1.3 @@ -1,5 +1,5 @@ PEP: 342 -Title: Enhanced Iterators +Title: Coroutines via Enhanced Iterators Version: $Revision$ Last-Modified: $Date$ Author: Guido van Rossum @@ -22,6 +22,11 @@ pretty much orthogonal from the anonymous block statement discussion. Thanks to Steven Bethard for doing the editing. + Update: at this point I'm leaning towards preferring next() over + __next__() again, but I've no time to update the PEP right now. + I've changed the title to Coroutines via Enhanced Generators at + Timothy Delaney's suggestion. + Motivation and Summary TBD. From pje at users.sourceforge.net Tue Jun 14 03:31:21 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Mon, 13 Jun 2005 18:31:21 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools ez_setup.py, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7361 Modified Files: ez_setup.py Log Message: Oops, forgot to bump a version number. Index: ez_setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/ez_setup.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- ez_setup.py 12 Jun 2005 21:47:33 -0000 1.1 +++ ez_setup.py 14 Jun 2005 01:31:19 -0000 1.2 @@ -14,7 +14,7 @@ This file can also be run as a script to install or upgrade setuptools. """ -DEFAULT_VERSION = "0.4a2" +DEFAULT_VERSION = "0.4a3" DEFAULT_URL = "http://peak.telecommunity.com/dist/" import sys, os From goodger at users.sourceforge.net Tue Jun 14 04:21:39 2005 From: goodger at users.sourceforge.net (goodger@users.sourceforge.net) Date: Mon, 13 Jun 2005 19:21:39 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt,1.312,1.313 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29829 Modified Files: pep-0000.txt Log Message: updated PEP 342 title Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.312 retrieving revision 1.313 diff -u -d -r1.312 -r1.313 --- pep-0000.txt 1 Jun 2005 15:16:33 -0000 1.312 +++ pep-0000.txt 14 Jun 2005 02:21:33 -0000 1.313 @@ -118,7 +118,7 @@ S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan S 341 Unifying try-except and try-finally Birkenfeld - S 342 Enhanced Iterators GvR + S 342 Coroutines via Enhanced Generators GvR S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones @@ -383,7 +383,7 @@ I 339 How to Change CPython's Bytecode Cannon SR 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld - S 342 Enhanced Iterators GvR + S 342 Coroutines via Enhanced Generators GvR S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones From rhettinger at users.sourceforge.net Tue Jun 14 10:45:46 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 14 Jun 2005 01:45:46 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libstdtypes.tex, 1.180, 1.181 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11133 Modified Files: libstdtypes.tex Log Message: SF bug #1215887: String and list methods documentation deeply hidden Make doc on specific types easier to find by listing them in the headings for the abstract types. Index: libstdtypes.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libstdtypes.tex,v retrieving revision 1.180 retrieving revision 1.181 diff -u -d -r1.180 -r1.181 --- libstdtypes.tex 31 May 2005 11:03:59 -0000 1.180 +++ libstdtypes.tex 14 Jun 2005 08:45:43 -0000 1.181 @@ -72,7 +72,9 @@ \index{False} \index{True} -\subsection{Boolean Operations \label{boolean}} +\subsection{Boolean Operations --- + \keyword{and}, \keyword{or}, \keyword{not} + \label{boolean}} These are the Boolean operations, ordered by ascending priority: \indexii{Boolean}{operations} @@ -173,7 +175,9 @@ only by sequence types (below). -\subsection{Numeric Types \label{typesnumeric}} +\subsection{Numeric Types --- + \class{int}, \class{float}, \class{long}, \class{complex} + \label{typesnumeric}} There are four distinct numeric types: \dfn{plain integers}, \dfn{long integers}, @@ -405,7 +409,10 @@ supplying the \method{__iter__()} and \method{next()} methods. -\subsection{Sequence Types \label{typesseq}} +\subsection{Sequence Types --- + \class{str}, \class{unicode}, \class{list}, + \class{tuple}, \class{buffer}, \class{xrange} + \label{typesseq}} There are six sequence types: strings, Unicode strings, lists, tuples, buffers, and xrange objects. @@ -1157,7 +1164,9 @@ that the list has been mutated during a sort. \end{description} -\subsection{Set Types \label{types-set}} +\subsection{Set Types --- + \class{set}, \class{frozenset} + \label{types-set}} \obindex{set} A \dfn{set} object is an unordered collection of immutable values. @@ -1283,7 +1292,7 @@ as an argument. -\subsection{Mapping Types \label{typesmapping}} +\subsection{Mapping Types --- class{dict} \label{typesmapping}} \obindex{mapping} \obindex{dictionary} From rhettinger at users.sourceforge.net Tue Jun 14 10:47:21 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 14 Jun 2005 01:47:21 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libstdtypes.tex, 1.170.2.9, 1.170.2.10 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12360 Modified Files: Tag: release24-maint libstdtypes.tex Log Message: SF bug #1215887: String and list methods documentation deeply hidden Make doc on specific types easier to find by listing them in the headings for the abstract types. Index: libstdtypes.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libstdtypes.tex,v retrieving revision 1.170.2.9 retrieving revision 1.170.2.10 diff -u -d -r1.170.2.9 -r1.170.2.10 --- libstdtypes.tex 31 May 2005 10:28:07 -0000 1.170.2.9 +++ libstdtypes.tex 14 Jun 2005 08:47:19 -0000 1.170.2.10 @@ -72,7 +72,9 @@ \index{False} \index{True} -\subsection{Boolean Operations \label{boolean}} +\subsection{Boolean Operations --- + \keyword{and}, \keyword{or}, \keyword{not} + \label{boolean}} These are the Boolean operations, ordered by ascending priority: \indexii{Boolean}{operations} @@ -173,7 +175,9 @@ only by sequence types (below). -\subsection{Numeric Types \label{typesnumeric}} +\subsection{Numeric Types --- + \class{int}, \class{float}, \class{long}, \class{complex} + \label{typesnumeric}} There are four distinct numeric types: \dfn{plain integers}, \dfn{long integers}, @@ -256,7 +260,7 @@ \lineiii{float(\var{x})}{\var{x} converted to floating point}{} \lineiii{complex(\var{re},\var{im})}{a complex number with real part \var{re}, imaginary part \var{im}. \var{im} defaults to zero.}{} \lineiii{\var{c}.conjugate()}{conjugate of the complex number \var{c}}{} - \lineiii{divmod(\var{x}, \var{y})}{the pair \code{(\var{x} / \var{y}, \var{x} \%{} \var{y})}}{(3)(4)} + \lineiii{divmod(\var{x}, \var{y})}{the pair \code{(\var{x} // \var{y}, \var{x} \%{} \var{y})}}{(3)(4)} \lineiii{pow(\var{x}, \var{y})}{\var{x} to the power \var{y}}{} \lineiii{\var{x} ** \var{y}}{\var{x} to the power \var{y}}{} \end{tableiii} @@ -405,7 +409,10 @@ supplying the \method{__iter__()} and \method{next()} methods. -\subsection{Sequence Types \label{typesseq}} +\subsection{Sequence Types --- + \class{str}, \class{unicode}, \class{list}, + \class{tuple}, \class{buffer}, \class{xrange} + \label{typesseq}} There are six sequence types: strings, Unicode strings, lists, tuples, buffers, and xrange objects. @@ -1157,7 +1164,9 @@ that the list has been mutated during a sort. \end{description} -\subsection{Set Types \label{types-set}} +\subsection{Set Types --- + \class{set}, \class{frozenset} + \label{types-set}} \obindex{set} A \dfn{set} object is an unordered collection of immutable values. @@ -1283,7 +1292,7 @@ as an argument. -\subsection{Mapping Types \label{typesmapping}} +\subsection{Mapping Types --- class{dict} \label{typesmapping}} \obindex{mapping} \obindex{dictionary} From rhettinger at users.sourceforge.net Tue Jun 14 10:57:31 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 14 Jun 2005 01:57:31 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut tut.tex,1.270,1.271 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17856 Modified Files: tut.tex Log Message: SF bug #1219448: small output bug Index: tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.270 retrieving revision 1.271 diff -u -d -r1.270 -r1.271 --- tut.tex 27 Apr 2005 09:41:23 -0000 1.270 +++ tut.tex 14 Jun 2005 08:57:28 -0000 1.271 @@ -2640,7 +2640,7 @@ >>> import fibo >>> fib = fibo.fib >>> dir() -['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib'] +['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib', 'fibo', 'sys'] \end{verbatim} Note that it lists all types of names: variables, modules, functions, etc. From rhettinger at users.sourceforge.net Tue Jun 14 10:58:29 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 14 Jun 2005 01:58:29 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut tut.tex, 1.261.2.5, 1.261.2.6 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18628 Modified Files: Tag: release24-maint tut.tex Log Message: SF bug #1219448: small output bug Index: tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.261.2.5 retrieving revision 1.261.2.6 diff -u -d -r1.261.2.5 -r1.261.2.6 --- tut.tex 28 Apr 2005 00:21:16 -0000 1.261.2.5 +++ tut.tex 14 Jun 2005 08:58:27 -0000 1.261.2.6 @@ -2640,7 +2640,7 @@ >>> import fibo >>> fib = fibo.fib >>> dir() -['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib'] +['__builtins__', '__doc__', '__file__', '__name__', 'a', 'fib', 'fibo', 'sys'] \end{verbatim} Note that it lists all types of names: variables, modules, functions, etc. From mwh at users.sourceforge.net Tue Jun 14 11:31:30 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Tue, 14 Jun 2005 02:31:30 -0700 Subject: [Python-checkins] python/dist/src/Lib/test regrtest.py, 1.168, 1.169 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3612 Modified Files: regrtest.py Log Message: yet another cache to clear when leak hunting. Index: regrtest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/regrtest.py,v retrieving revision 1.168 retrieving revision 1.169 diff -u -d -r1.168 -r1.169 --- regrtest.py 10 Jun 2005 11:05:19 -0000 1.168 +++ regrtest.py 14 Jun 2005 09:31:28 -0000 1.169 @@ -491,6 +491,7 @@ import gc def cleanup(): import _strptime, urlparse, warnings, dircache + import linecache from distutils.dir_util import _path_created _path_created.clear() warnings.filters[:] = fs @@ -503,6 +504,7 @@ sys.path_importer_cache.clear() sys.path_importer_cache.update(pic) dircache.reset() + linecache.clearcache() if indirect_test: def run_the_test(): indirect_test() From pje at users.sourceforge.net Tue Jun 14 14:44:46 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 05:44:46 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.31, 1.32 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7644 Modified Files: pkg_resources.py Log Message: Fix typos found by Ryan Tomayko. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.31 retrieving revision 1.32 diff -u -d -r1.31 -r1.32 --- pkg_resources.py 14 Jun 2005 01:28:44 -0000 1.31 +++ pkg_resources.py 14 Jun 2005 12:44:43 -0000 1.32 @@ -573,7 +573,7 @@ def resource_listdir(self,resource_name): - return self._listdir(self._fn(self.egg_info,resource_name)) + return self._listdir(self._fn(self.module_path,resource_name)) def metadata_listdir(self,name): if self.egg_info: @@ -738,7 +738,7 @@ def _extract_resource(self, manager, resource_name): if self.resource_isdir(resource_name): - return self._extract_dir(resource_name) + return self._extract_directory(resource_name) parts = resource_name.split('/') zip_path = os.path.join(self.module_path, *parts) From pje at users.sourceforge.net Tue Jun 14 14:46:38 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 05:46:38 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools package_index.py, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8315/setuptools Modified Files: package_index.py Log Message: Support downloading packages that were uploaded to PyPI (by scanning all links on package pages, not just the homepage/download links). Index: package_index.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/package_index.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- package_index.py 14 Jun 2005 01:26:26 -0000 1.2 +++ package_index.py 14 Jun 2005 12:46:35 -0000 1.3 @@ -116,10 +116,10 @@ f.close() if url.startswith(self.index_url): self.process_index(url, page) - else: - for match in HREF.finditer(page): - link = urlparse.urljoin(base, match.group(1)) - self.process_url(link) + + for match in HREF.finditer(page): + link = urlparse.urljoin(base, match.group(1)) + self.process_url(link) def process_index(self,url,page): """Process the contents of a PyPI page""" From akuchling at users.sourceforge.net Tue Jun 14 16:45:26 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Tue, 14 Jun 2005 07:45:26 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libcurses.tex,1.51,1.52 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv5858 Modified Files: libcurses.tex Log Message: [Bug #1219862] Document correct argument range for pair_content() Index: libcurses.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libcurses.tex,v retrieving revision 1.51 retrieving revision 1.52 diff -u -d -r1.51 -r1.52 --- libcurses.tex 1 Jun 2005 23:31:18 -0000 1.51 +++ libcurses.tex 14 Jun 2005 14:45:24 -0000 1.52 @@ -391,7 +391,7 @@ \begin{funcdesc}{pair_content}{pair_number} Returns a tuple \code{(\var{fg}, \var{bg})} containing the colors for the requested color pair. The value of \var{pair_number} must be -between \code{0} and \code{\constant{COLOR_PAIRS} - 1}. +between \code{1} and \code{\constant{COLOR_PAIRS} - 1}. \end{funcdesc} \begin{funcdesc}{pair_number}{attr} From akuchling at users.sourceforge.net Tue Jun 14 16:46:09 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Tue, 14 Jun 2005 07:46:09 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libcurses.tex, 1.48.2.3, 1.48.2.4 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6090 Modified Files: Tag: release24-maint libcurses.tex Log Message: [Bug #1219862] Document correct argument range for pair_content() Index: libcurses.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libcurses.tex,v retrieving revision 1.48.2.3 retrieving revision 1.48.2.4 diff -u -d -r1.48.2.3 -r1.48.2.4 --- libcurses.tex 1 Jun 2005 23:32:31 -0000 1.48.2.3 +++ libcurses.tex 14 Jun 2005 14:46:06 -0000 1.48.2.4 @@ -391,7 +391,7 @@ \begin{funcdesc}{pair_content}{pair_number} Returns a tuple \code{(\var{fg}, \var{bg})} containing the colors for the requested color pair. The value of \var{pair_number} must be -between \code{0} and \code{\constant{COLOR_PAIRS} - 1}. +between \code{1} and \code{\constant{COLOR_PAIRS} - 1}. \end{funcdesc} \begin{funcdesc}{pair_number}{attr} From gvanrossum at users.sourceforge.net Tue Jun 14 17:14:20 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Tue, 14 Jun 2005 08:14:20 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.313, 1.314 pep-0342.txt, 1.3, 1.4 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21023 Modified Files: pep-0000.txt pep-0342.txt Log Message: Fix/unify PEP 342's title. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.313 retrieving revision 1.314 diff -u -d -r1.313 -r1.314 --- pep-0000.txt 14 Jun 2005 02:21:33 -0000 1.313 +++ pep-0000.txt 14 Jun 2005 15:14:01 -0000 1.314 @@ -118,7 +118,7 @@ S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan S 341 Unifying try-except and try-finally Birkenfeld - S 342 Coroutines via Enhanced Generators GvR + S 342 Coroutines via Enhanced Iterators GvR S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones @@ -383,7 +383,7 @@ I 339 How to Change CPython's Bytecode Cannon SR 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld - S 342 Coroutines via Enhanced Generators GvR + S 342 Coroutines via Enhanced Iterators GvR S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones Index: pep-0342.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0342.txt,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- pep-0342.txt 14 Jun 2005 01:30:57 -0000 1.3 +++ pep-0342.txt 14 Jun 2005 15:14:01 -0000 1.4 @@ -24,7 +24,7 @@ Update: at this point I'm leaning towards preferring next() over __next__() again, but I've no time to update the PEP right now. - I've changed the title to Coroutines via Enhanced Generators at + I've changed the title to Coroutines via Enhanced Iterators at Timothy Delaney's suggestion. Motivation and Summary From pje at users.sourceforge.net Tue Jun 14 17:29:43 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 08:29:43 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.32, 1.33 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29224 Modified Files: pkg_resources.py Log Message: Fix more zipped-egg directory resource bugs reported by Ryan Tomayko. Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.32 retrieving revision 1.33 diff -u -d -r1.32 -r1.33 --- pkg_resources.py 14 Jun 2005 12:44:43 -0000 1.32 +++ pkg_resources.py 14 Jun 2005 15:29:41 -0000 1.33 @@ -738,7 +738,7 @@ def _extract_resource(self, manager, resource_name): if self.resource_isdir(resource_name): - return self._extract_directory(resource_name) + return self._extract_directory(manager, resource_name) parts = resource_name.split('/') zip_path = os.path.join(self.module_path, *parts) @@ -801,23 +801,23 @@ return self._short_name(path) in self.zipinfo or self._isdir(path) def _isdir(self,path): - path = self._short_name(path).replace(os.sep, '/') - if path.endswith('/'): path = path[:-1] - return path in self._index() + return self._dir_name(path) in self._index() def _listdir(self,path): - path = self._short_name(path).replace(os.sep, '/') + return list(self._index().get(self._dir_name(path), ())) + + def _dir_name(self,path): + if path.startswith(self.module_path+os.sep): + path = path[len(self.module_path+os.sep):] + path = path.replace(os.sep,'/') if path.endswith('/'): path = path[:-1] - return list(self._index().get(path, ())) + return path _get = NullProvider._get - register_loader_type(zipimport.zipimporter, ZipProvider) - - class PathMetadata(DefaultProvider): """Metadata provider for egg directories From pje at users.sourceforge.net Tue Jun 14 17:30:34 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 08:30:34 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools package_index.py, 1.3, 1.4 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29928/setuptools Modified Files: package_index.py Log Message: Add support for quiet/verbose/dry-run/optimize flags. Index: package_index.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/package_index.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- package_index.py 14 Jun 2005 12:46:35 -0000 1.3 +++ package_index.py 14 Jun 2005 15:30:31 -0000 1.4 @@ -2,6 +2,7 @@ import sys, os.path, re, urlparse, urllib2 from pkg_resources import * +from distutils import log HREF = re.compile(r"""href\s*=\s*['"]?([^'"> ]+)""", re.I) URL_SCHEME = re.compile('([-+.a-z0-9]{2,}):',re.I).match @@ -38,7 +39,6 @@ - def distros_for_url(url, metadata=None): """Yield egg or source distribution objects that might be found at a URL""" @@ -359,13 +359,13 @@ return filename def debug(self, msg, *args): - pass #print msg % args # XXX + log.debug(msg, *args) def info(self, msg, *args): - print msg % args # XXX + log.info(msg, *args) def warn(self, msg, *args): - print msg % args # XXX + log.warn(msg, *args) def _download_sourceforge(self, source_url, sf_page, tmpdir): """Download package from randomly-selected SourceForge mirror""" From pje at users.sourceforge.net Tue Jun 14 17:30:34 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 08:30:34 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.16, 1.17 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29928/setuptools/command Modified Files: bdist_egg.py Log Message: Add support for quiet/verbose/dry-run/optimize flags. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- bdist_egg.py 12 Jun 2005 21:47:34 -0000 1.16 +++ bdist_egg.py 14 Jun 2005 15:30:32 -0000 1.17 @@ -190,7 +190,7 @@ if not self.dry_run: os.unlink(native_libs) - if self.egg_info: + if self.egg_info and os.path.exists(self.egg_info): for filename in os.listdir(self.egg_info): path = os.path.join(self.egg_info,filename) if os.path.isfile(path): @@ -231,7 +231,7 @@ for dirname in INSTALL_DIRECTORY_ATTRS: kw.setdefault(dirname,self.bdist_dir) kw.setdefault('skip_build',self.skip_build) - + kw.setdefault('dry_run', self.dry_run) cmd = self.reinitialize_command(cmdname, **kw) self.run_command(cmdname) return cmd @@ -262,24 +262,24 @@ import zipfile mkpath(os.path.dirname(zip_filename), dry_run=dry_run) - # If zipfile module is not available, try spawning an external - # 'zip' command. - log.info("creating '%s' and adding '%s' to it", - zip_filename, base_dir) + log.info("creating '%s' and adding '%s' to it", zip_filename, base_dir) def visit (z, dirname, names): for name in names: path = os.path.normpath(os.path.join(dirname, name)) if os.path.isfile(path): p = path[len(base_dir)+1:] - z.write(path, p) - log.info("adding '%s'" % p) + if not dry_run: + z.write(path, p) + log.debug("adding '%s'" % p) if not dry_run: z = zipfile.ZipFile(zip_filename, "w", compression=zipfile.ZIP_DEFLATED) os.path.walk(base_dir, visit, z) z.close() + else: + os.path.walk(base_dir, visit, None) return zip_filename From pje at users.sourceforge.net Tue Jun 14 17:30:34 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 08:30:34 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.12, 1.13 easy_install.py, 1.20, 1.21 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29928 Modified Files: EasyInstall.txt easy_install.py Log Message: Add support for quiet/verbose/dry-run/optimize flags. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- EasyInstall.txt 14 Jun 2005 01:29:59 -0000 1.12 +++ EasyInstall.txt 14 Jun 2005 15:30:31 -0000 1.13 @@ -23,7 +23,7 @@ ------------------------- Windows users can just download and run the `setuptools binary installer for -Windows `_. +Windows `_. All others should just download `ez_setup.py `_, and run it; this will download and install the correct version of ``setuptools`` for your Python @@ -62,7 +62,7 @@ **Example 2**. Install or upgrade a package by name and version by finding links on a given "download page":: - easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.4a3" + easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.4a4" **Example 3**. Download a source distribution from a specified URL, automatically building and installing it:: @@ -229,6 +229,26 @@ recently-installed version of the package.) +Controlling Build Options +------------------------- + +EasyInstall respects standard distutils `Configuration Files`_, so you can use +them to configure build options for packages that it installs from source. For +example, if you are on Windows using the MinGW compiler, you can configure the +default compiler by putting something like this:: + + [build] + compiler = mingw32 + +into the appropriate distutils configuration file. In fact, since this is just +normal distutils configuration, it will affect any builds using that config +file, not just ones done by EasyInstall. For example, if you add those lines +to ``distutils.cfg`` in the ``distutils`` package directory, it will be the +default compiler for *all* packages you build. See `Configuration Files`_ +below for a list of the standard configuration file locations, and links to +more documentation on using distutils configuration files. + + Reference Manual ================ @@ -257,8 +277,17 @@ find_links = http://sqlobject.org/ http://peak.telecommunity.com/dist/ -See also the current Python documentation on the `use and location of distutils -configuration files `_. +In addition to accepting configuration for its own options under +``[easy_install]``, EasyInstall also respects defaults specified for other +distutils commands. For example, if you don't set an ``install_dir`` for +``[easy_install]``, but *have* set an ``install_lib`` for the ``[install]`` +command, this will become EasyInstall's default installation directory. Thus, +if you are already using distutils configuration files to set default install +locations, build options, etc., EasyInstall will respect your existing settings +until and unless you override them explicitly in an ``[easy_install]`` section. + +For more information, see also the current Python documentation on the `use and +location of distutils configuration files `_. Command-Line Options @@ -377,6 +406,34 @@ URL or filename, so that the installer will not be confused by the presence of multiple ``setup.py`` files in the build directory. +``--verbose, -v, --quiet, -q`` (New in 0.4a4) + Control the level of detail of EasyInstall's progress messages. The + default detail level is "info", which prints information only about + relatively time-consuming operations like running a setup script, unpacking + an archive, or retrieving a URL. Using ``-q`` or ``--quiet`` drops the + detail level to "warn", which will only display installation reports, + warnings, and errors. Using ``-v`` or ``--verbose`` increases the detail + level to include individual file-level operations, link analysis messages, + and distutils messages from any setup scripts that get run. If you include + the ``-v`` option more than once, the second and subsequent uses are passed + down to any setup scripts, increasing the verbosity of their reporting as + well. + +``--dry-run, -n`` (New in 0.4a4) + Don't actually install the package or scripts. This option is passed down + to any setup scripts run, so packages should not actually build either. + This does *not* skip downloading, nor does it skip extracting source + distributions to a temporary/build directory. + +``--optimize=LEVEL``, ``-O LEVEL`` (New in 0.4a4) + If you are installing from a source distribution, and are *not* using the + ``--zip-ok`` option, this option controls the optimization level for + compiling installed ``.py`` files to ``.pyo`` files. It does not affect + the compilation of modules contained in ``.egg`` files, only those in + ``.egg`` directories. The optimization level can be set to 0, 1, or 2; + the default is 0 (unless it's set under ``install`` or ``install_lib`` in + one of your distutils configuration files). + Release Notes/Change History ============================ @@ -385,8 +442,15 @@ * There's no automatic retry for borked Sourceforge mirrors, which can easily time out or be missing a file. - * EasyInstall does not yet respect the distutils "verbose/quiet" and "dry-run" - options, even though it accepts them. +0.4a4 + * Added support for the distutils "verbose/quiet" and "dry-run" options, as + well as the "optimize" flag. + + * Support downloading packages that were uploaded to PyPI (by scanning all + links on package pages, not just the homepage/download links). + + * Fix problems with ``resource_listdir()``, ``resource_isdir()`` and resource + directory extraction for zipped eggs. 0.4a3 * Fixed scripts not being able to see a ``__file__`` variable in ``__main__`` @@ -493,10 +557,13 @@ Future Plans ============ -* Log progress to a logger, with -v and -q options to control verbosity -* Display more information about downloads and progress when being verbose * Process the installed package's dependencies as well as the base package +* Support "self-installation" - bootstrapping setuptools install into another + package's installation process (copy egg, write setuptools.pth) * Support installation from bdist_wininst packages? * Additional utilities to list/remove/verify packages * Signature checking? SSL? Ability to suppress PyPI search? +* Display byte progress meter when downloading distributions and long pages? +* Redirect stdout/stderr to log during run_setup? + Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.20 retrieving revision 1.21 diff -u -d -r1.20 -r1.21 --- easy_install.py 14 Jun 2005 01:27:12 -0000 1.20 +++ easy_install.py 14 Jun 2005 15:30:31 -0000 1.21 @@ -16,8 +16,9 @@ from setuptools import Command from setuptools.sandbox import run_setup +from distutils import log, dir_util from distutils.sysconfig import get_python_lib -from distutils.errors import DistutilsArgError +from distutils.errors import DistutilsArgError, DistutilsOptionError from setuptools.archive_util import unpack_archive from setuptools.package_index import PackageIndex from pkg_resources import * @@ -38,7 +39,6 @@ - class easy_install(Command): """Manage a download/build/install process""" @@ -54,11 +54,14 @@ ("find-links=", "f", "additional URL(s) to search for packages"), ("build-directory=", "b", "download/extract/build in DIR; keep the results"), + ('optimize=', 'O', + "also compile with optimization: -O1 for \"python -O\", " + "-O2 for \"python -OO\", and -O0 to disable [default: -O0]"), ] boolean_options = [ 'zip-ok', 'multi-version', 'exclude-scripts' ] create_index = PackageIndex - + def initialize_options(self): self.zip_ok = None self.multi_version = None @@ -67,19 +70,16 @@ self.find_links = None self.build_directory = None self.args = None - + self.optimize = None + # Options not specifiable via command line self.package_index = None self.pth_file = None - def alloc_tmp(self): - if self.build_directory is None: - return tempfile.mkdtemp(prefix="easy_install-") - tmpdir = os.path.realpath(self.build_directory) - if not os.path.isdir(tmpdir): - os.makedirs(tmpdir) - return tmpdir - + + + + def finalize_options(self): # If a non-default installation directory was specified, default the # script directory to match it. @@ -91,13 +91,13 @@ # --prefix and --home and all that other crud. self.set_undefined_options('install_lib', ('install_dir','install_dir') - ) + ) # Likewise, set default script_dir from 'install_scripts.install_dir' self.set_undefined_options('install_scripts', ('install_dir', 'script_dir') ) - site_packages = get_python_lib() + site_packages = get_python_lib() instdir = self.install_dir if instdir is None or samefile(site_packages,instdir): @@ -106,7 +106,7 @@ self.pth_file = PthDistributions( os.path.join(instdir,'easy-install.pth') ) - self.install_dir = instdir + self.install_dir = instdir elif self.multi_version is None: self.multi_version = True @@ -124,23 +124,66 @@ if self.find_links is not None: if isinstance(self.find_links, basestring): self.find_links = self.find_links.split() - for link in self.find_links: - self.package_index.scan_url(link) + else: + self.find_links = [] + + self.set_undefined_options('install_lib', ('optimize','optimize')) + + if not isinstance(self.optimize,int): + try: + self.optimize = int(self.optimize) + if not (0 <= self.optimize <= 2): raise ValueError + except ValueError: + raise DistutilsOptionError("--optimize must be 0, 1, or 2") if not self.args: raise DistutilsArgError( "No urls, filenames, or requirements specified (see --help)") + elif len(self.args)>1 and self.build_directory is not None: raise DistutilsArgError( - "Build directory can only be set when using one URL" + "Build directory can only be set when using one URL" ) + + + + + + + + + + + + + + + + + + def alloc_tmp(self): + if self.build_directory is None: + return tempfile.mkdtemp(prefix="easy_install-") + tmpdir = os.path.realpath(self.build_directory) + if not os.path.isdir(tmpdir): + os.makedirs(tmpdir) + return tmpdir + + def run(self): - for spec in self.args: - self.easy_install(spec) + if self.verbose<>self.distribution.verbose: + log.set_verbosity(self.verbose) + try: + for link in self.find_links: + self.package_index.scan_url(link) + for spec in self.args: + self.easy_install(spec) + finally: + log.set_verbosity(self.distribution.verbose) - def easy_install(self, spec): + def easy_install(self, spec): tmpdir = self.alloc_tmp() try: download = self.package_index.download(spec, tmpdir) @@ -149,19 +192,17 @@ "Could not find distribution for %r" % spec ) - print "Processing", os.path.basename(download) + log.info("Processing %s", os.path.basename(download)) for dist in self.install_eggs(download, self.zip_ok, tmpdir): self.package_index.add(dist) self.install_egg_scripts(dist) - print self.installation_report(dist) + log.warn(self.installation_report(dist)) finally: if self.build_directory is None: shutil.rmtree(tmpdir) - - def install_egg_scripts(self, dist): metadata = dist.metadata if self.exclude_scripts or not metadata.metadata_isdir('scripts'): @@ -172,7 +213,7 @@ for script_name in metadata.metadata_listdir('scripts'): target = os.path.join(self.script_dir, script_name) - print "Installing", script_name, "script to", self.script_dir + log.info("Installing %s script to %s", script_name,self.script_dir) script_text = metadata.get_metadata('scripts/'+script_name) script_text = script_text.replace('\r','\n') @@ -193,11 +234,11 @@ "import pkg_resources", "pkg_resources.run_main(%r, %r)" % (spec, script_name) ]) + if not self.dry_run: + f = open(target,"w") + f.write(script_text) + f.close() - f = open(target,"w") - f.write(script_text) - f.close() - @@ -210,7 +251,7 @@ # Anything else, try to extract and build if os.path.isfile(dist_filename): - unpack_archive(dist_filename, tmpdir) # XXX add progress log + unpack_archive(dist_filename, tmpdir, self.unpack_progress) # Find the setup.py file from glob import glob @@ -226,48 +267,51 @@ "Multiple setup scripts in %s" % dist_filename ) setup_script = setups[0] - from setuptools.command import bdist_egg - sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) - try: - print "Running", setup_script[len(tmpdir)+1:] - run_setup(setup_script, ['-q', 'bdist_egg']) - except SystemExit, v: - raise RuntimeError( - "Setup script exited with %s" % (v.args[0],) - ) + + self.build_egg(tmpdir, setup_script) + dist_dir = os.path.join(os.path.dirname(setup_script),'dist') eggs = [] - for egg in glob( - os.path.join(os.path.dirname(setup_script),'dist','*.egg') - ): + for egg in glob(os.path.join(dist_dir,'*.egg')): eggs.append(self.install_egg(egg, zip_ok, tmpdir)) + if not eggs and not self.dry_run: + log.warn("No eggs found in %s (setup script problem?)", dist_dir) + return eggs - def install_egg(self, egg_path, zip_ok, tmpdir): + + + + + def install_egg(self, egg_path, zip_ok, tmpdir): destination = os.path.join(self.install_dir,os.path.basename(egg_path)) destination = os.path.abspath(destination) - ensure_directory(destination) + if not self.dry_run: + ensure_directory(destination) if not samefile(egg_path, destination): if os.path.isdir(destination): - shutil.rmtree(destination) + dir_util.remove_tree(destination, dry_run=self.dry_run) + elif os.path.isfile(destination): - os.unlink(destination) + self.execute(os.unlink,(destination,),"Removing "+destination) if zip_ok: if egg_path.startswith(tmpdir): - shutil.move(egg_path, destination) + f,m = shutil.move, "Moving" else: - shutil.copy2(egg_path, destination) - + f,m = shutil.copy2, "Copying" elif os.path.isdir(egg_path): - shutil.move(egg_path, destination) - + f,m = shutil.move, "Moving" else: - os.mkdir(destination) - unpack_archive(egg_path, destination) # XXX add progress?? + self.mkpath(destination) + f,m = self.unpack_and_compile, "Extracting" + + self.execute(f, (egg_path, destination), + (m+" %s to %s") % + (os.path.basename(egg_path),os.path.dirname(destination))) if os.path.isdir(destination): dist = Distribution.from_filename( @@ -282,13 +326,10 @@ self.update_pth(dist) return dist - - - def installation_report(self, dist): """Helpful installation message for display to package users""" - msg = "Installed %(eggloc)s to %(instdir)s" + msg = "\nInstalled %(eggloc)s to %(instdir)s" if self.multi_version: msg += """ @@ -318,12 +359,94 @@ if self.pth_file is not None: remove = self.pth_file.remove for d in self.pth_file.get(dist.key,()): # drop old entries + log.info("Removing %s from .pth file", d) remove(d) if not self.multi_version: + log.info("Adding %s to .pth file", dist) self.pth_file.add(dist) # add new entry self.pth_file.save() + def build_egg(self, tmpdir, setup_script): + from setuptools.command import bdist_egg + sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) + + args = ['bdist_egg'] + if self.verbose>2: + v = 'v' * self.verbose - 1 + args.insert(0,'-'+v) + elif self.verbose<2: + args.insert(0,'-q') + if self.dry_run: + args.insert(0,'-n') + + log.info("Running %s %s", setup_script[len(tmpdir)+1:], ' '.join(args)) + try: + try: + run_setup(setup_script, args) + except SystemExit, v: + raise RuntimeError( + "Setup script exited with %s" % (v.args[0],) + ) + finally: + log.set_verbosity(self.verbose) # restore our log verbosity + + + + + + + + + + + + + + + + + + + def unpack_progress(self, src, dst): + # Progress filter for unpacking + log.debug("Unpacking %s to %s", src, dst) + return True # only unpack-and-compile skips files for dry run + + + def unpack_and_compile(self, egg_path, destination): + to_compile = [] + + def pf(src,dst): + if dst.endswith('.py'): + to_compile.append(dst) + self.unpack_progress(src,dst) + return not self.dry_run + + unpack_archive(egg_path, destination, pf) + + from distutils.util import byte_compile + try: + # try to make the byte compile messages quieter + log.set_verbosity(self.verbose - 1) + + byte_compile(to_compile, optimize=0, force=1, dry_run=self.dry_run) + if self.optimize: + byte_compile( + to_compile, optimize=self.optimize, force=1, + dry_run=self.dry_run + ) + finally: + log.set_verbosity(self.verbose) # restore original verbosity + + + + + + + + + class PthDistributions(AvailableDistributions): From pje at users.sourceforge.net Tue Jun 14 17:48:47 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 08:48:47 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools ez_setup.py, 1.2, 1.3 setup.py, 1.13, 1.14 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8097 Modified Files: ez_setup.py setup.py Log Message: Bump version # for release. Index: ez_setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/ez_setup.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- ez_setup.py 14 Jun 2005 01:31:19 -0000 1.2 +++ ez_setup.py 14 Jun 2005 15:48:44 -0000 1.3 @@ -14,7 +14,7 @@ This file can also be run as a script to install or upgrade setuptools. """ -DEFAULT_VERSION = "0.4a3" +DEFAULT_VERSION = "0.4a4" DEFAULT_URL = "http://peak.telecommunity.com/dist/" import sys, os Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- setup.py 14 Jun 2005 01:29:59 -0000 1.13 +++ setup.py 14 Jun 2005 15:48:44 -0000 1.14 @@ -1,7 +1,7 @@ #!/usr/bin/env python """Distutils setup file, used to install or test 'setuptools'""" -VERSION = "0.4a3" +VERSION = "0.4a4" from setuptools import setup, find_packages, Require setup( From pje at users.sourceforge.net Tue Jun 14 17:48:47 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 08:48:47 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools __init__.py, 1.5, 1.6 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8097/setuptools Modified Files: __init__.py Log Message: Bump version # for release. Index: __init__.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/__init__.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- __init__.py 14 Jun 2005 01:29:59 -0000 1.5 +++ __init__.py 14 Jun 2005 15:48:44 -0000 1.6 @@ -8,7 +8,7 @@ from distutils.util import convert_path import os.path -__version__ = '0.4a3' +__version__ = '0.4a4' __all__ = [ 'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require', From jackjansen at users.sourceforge.net Tue Jun 14 23:32:53 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Tue, 14 Jun 2005 14:32:53 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenType.py, 1.12, 1.13 bgenVariable.py, 1.2, 1.3 scantools.py, 1.35, 1.36 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29410 Modified Files: bgenType.py bgenVariable.py scantools.py Log Message: More tweaks for C++ support. Still doesn't seem to break anything:-) Index: bgenType.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenType.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- bgenType.py 18 Jul 2004 06:02:01 -0000 1.12 +++ bgenType.py 14 Jun 2005 21:32:51 -0000 1.13 @@ -19,12 +19,15 @@ self.typeName = typeName self.fmt = fmt - def declare(self, name): + def declare(self, name, reference=False): """Declare a variable of the type with a given name. Example: int.declare('spam') prints "int spam;" """ - Output("%s %s;", self.typeName, name) + if reference: + Output("%s& %s;", self.typeName, name) + else: + Output("%s %s;", self.typeName, name) def getargs(self): return self.getargsFormat(), self.getargsArgs() @@ -64,6 +67,11 @@ """ return "&" + name + def passReference(self, name): + """Return an argument for C++ pass-by-reference. + Default is to call passInput(). + """ + return self.passInput(name) def errorCheck(self, name): """Check for an error returned in the variable. @@ -172,7 +180,7 @@ self.substitute = substitute self.typeName = None # Don't show this argument in __doc__ string - def declare(self, name): + def declare(self, name, reference=False): pass def getargsFormat(self): @@ -236,6 +244,25 @@ def mkvalueArgs(self, name): return "%s, %s" % (self.new, name) + +class OpaqueByRefType(OpaqueType): + """An opaque object type, passed by reference. + + Instantiate with the type name, and optionally an object type name whose + New/Convert functions will be used. + """ + + def passInput(self, name): + return name + +# def passOutput(self, name): +# return name + + def mkvalueFormat(self): + return "O" + + def mkvalueArgs(self, name): + return "%s(%s)" % (self.new, name) class OpaqueByValueStructType(OpaqueByValueType): """Similar to OpaqueByValueType, but we also pass this to mkvalue by Index: bgenVariable.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenVariable.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- bgenVariable.py 19 Jan 2003 21:53:57 -0000 1.2 +++ bgenVariable.py 14 Jun 2005 21:32:51 -0000 1.3 @@ -13,7 +13,7 @@ SelfMode = 4+InMode # this is 'self' -- don't declare it ReturnMode = 8+OutMode # this is the function return value ErrorMode = 16+OutMode # this is an error status -- turn it into an exception - +RefMode = 32 class Variable: @@ -39,7 +39,9 @@ If it is "self", it is not declared. """ - if self.flags != SelfMode: + if self.flags == ReturnMode+RefMode: + self.type.declare(self.name, reference=True) + elif self.flags != SelfMode: self.type.declare(self.name) def getargsFormat(self): @@ -62,6 +64,8 @@ """ if self.mode == InMode: return self.type.passInput(self.name) + if self.mode & RefMode: + return self.type.passReference(self.name) if self.mode in (OutMode, InOutMode): return self.type.passOutput(self.name) # XXX Shouldn't get here Index: scantools.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/scantools.py,v retrieving revision 1.35 retrieving revision 1.36 diff -u -d -r1.35 -r1.36 --- scantools.py 18 Jul 2004 06:02:01 -0000 1.35 +++ scantools.py 14 Jun 2005 21:32:51 -0000 1.36 @@ -466,6 +466,7 @@ if self.debug: self.report("* WHOLE LINE: %r" % (raw,)) self.processrawspec(raw) + return raw def processrawspec(self, raw): match = self.whole.search(raw) @@ -478,8 +479,8 @@ self.report("(but type matched)") return type, name, args = match.group('type', 'name', 'args') - type = re.sub("\*", " ptr", type) - type = re.sub("[ \t]+", "_", type) + type = self.pythonizename(type) + name = self.pythonizename(name) if name in self.alreadydone: self.report("Name has already been defined: %r", name) return @@ -500,6 +501,12 @@ self.alreadydone.append(name) self.generate(type, name, arglist) + def pythonizename(self, name): + name = re.sub("\*", " ptr", name) + name = name.strip() + name = re.sub("[ \t]+", "_", name) + return name + def extractarglist(self, args): args = args.strip() if not args or args == "void": @@ -522,9 +529,7 @@ if array: # array matches an optional [] after the argument name type = type + " ptr " - type = re.sub("\*", " ptr ", type) - type = type.strip() - type = re.sub("[ \t]+", "_", type) + type = self.pythonizename(type) return self.modifyarg(type, name, mode) def modifyarg(self, type, name, mode): @@ -590,6 +595,7 @@ def generate(self, type, name, arglist): self.typeused(type, 'return') classname, listname = self.destination(type, name, arglist) + if not classname or not listname: return if not self.specfile: return self.specfile.write("f = %s(%s, %r,\n" % (classname, type, name)) for atype, aname, amode in arglist: From montanaro at users.sourceforge.net Wed Jun 15 03:33:49 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Tue, 14 Jun 2005 18:33:49 -0700 Subject: [Python-checkins] python/dist/src/Modules _csv.c,1.36,1.37 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24661 Modified Files: _csv.c Log Message: Leak fix from Michael Hudson. Fix memory leak when dialect doesn't validate. Closes 1220242. Index: _csv.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_csv.c,v retrieving revision 1.36 retrieving revision 1.37 diff -u -d -r1.36 -r1.37 --- _csv.c 13 Jan 2005 11:30:53 -0000 1.36 +++ _csv.c 15 Jun 2005 01:33:30 -0000 1.37 @@ -416,7 +416,9 @@ } ret = (PyObject *)self; + Py_INCREF(self); err: + Py_XDECREF(self); Py_XDECREF(dialect); Py_XDECREF(delimiter); Py_XDECREF(doublequote); From pje at users.sourceforge.net Wed Jun 15 04:11:11 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 19:11:11 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools archive_util.py, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8734/setuptools Modified Files: archive_util.py Log Message: Enhance unpack_* utilities to allow on-the-fly redirection of where files are extracted to. Index: archive_util.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/archive_util.py,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- archive_util.py 12 Jun 2005 01:12:34 -0000 1.1 +++ archive_util.py 15 Jun 2005 02:11:09 -0000 1.2 @@ -13,9 +13,9 @@ """Couldn't recognize the archive type""" def default_filter(src,dst): - """The default progress/filter callback; returns True for all files""" - - return True + """The default progress/filter callback; returns True for all files""" + return dst + @@ -46,10 +46,11 @@ `progress_filter` is a function taking two arguments: a source path internal to the archive ('/'-separated), and a filesystem path where it - will be extracted. The callback must return a true value, or else that - file or directory will be skipped. The callback can thus be used to - report on the progress of the extraction, as well as to filter the items - extracted. + will be extracted. The callback must return the desired extract path + (which may be the same as the one passed in), or else ``None`` to skip + that file or directory. The callback can thus be used to report on the + progress of the extraction, as well as to filter the items extracted or + alter their extraction paths. `drivers`, if supplied, must be a non-empty sequence of functions with the same signature as this function (minus the `drivers` argument), that raise @@ -79,7 +80,6 @@ - def unpack_zipfile(filename, extract_dir, progress_filter=default_filter): """Unpack zip `filename` to `extract_dir` @@ -100,8 +100,9 @@ if name.startswith('/') or '..' in name: continue - target = os.path.join(extract_dir,name) - if not progress_filter(name,target): + target = os.path.join(extract_dir, *name.split('/')) + target = progress_filter(name, target) + if not target: continue if name.endswith('/'): # directory @@ -120,7 +121,6 @@ z.close() - def unpack_tarfile(filename, extract_dir, progress_filter=default_filter): """Unpack tar/tar.gz/tar.bz2 `filename` to `extract_dir` @@ -144,8 +144,11 @@ # don't extract absolute paths or ones with .. in them if not name.startswith('/') and '..' not in name: dst = os.path.join(extract_dir, *name.split('/')) - if progress_filter(name, dst): - tarobj.extract(member,extract_dir) + dst = progress_filter(name, dst) + if dst: + if dst.endswith(os.sep): + dst = dst[:-1] + tarobj._extract_member(member,dst) # XXX Ugh return True finally: tarobj.close() @@ -159,6 +162,3 @@ - - - From pje at users.sourceforge.net Wed Jun 15 04:12:51 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 19:12:51 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.17, 1.18 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9282/setuptools/command Modified Files: bdist_egg.py Log Message: Make write_stub() a function, so easy_install can use it too. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- bdist_egg.py 14 Jun 2005 15:30:32 -0000 1.17 +++ bdist_egg.py 15 Jun 2005 02:12:49 -0000 1.18 @@ -13,6 +13,32 @@ from pkg_resources import parse_requirements, get_platform, safe_name, \ safe_version, Distribution + +def write_stub(resource, pyfile): + f = open(pyfile,'w') + f.write('\n'.join([ + "def __bootstrap__():", + " global __bootstrap__, __loader__, __file__", + " import sys, pkg_resources, imp", + " __file__ = pkg_resources.resource_filename(__name__,%r)" + % resource, + " del __bootstrap__, __loader__", + " imp.load_dynamic(__name__,__file__)", + "__bootstrap__()", + "" # terminal \n + ])) + f.close() + + + + + + + + + + + class bdist_egg(Command): description = "create an \"egg\" distribution" user_options = [ @@ -53,6 +79,7 @@ self.tag_svn_revision = 0 self.tag_date = 0 + def finalize_options (self): self.egg_name = safe_name(self.distribution.get_name()) self.egg_version = self.tagged_version() @@ -80,20 +107,19 @@ self.plat_name = get_platform() self.set_undefined_options('bdist',('dist_dir', 'dist_dir')) - def write_stub(self, resource, pyfile): - f = open(pyfile,'w') - f.write('\n'.join([ - "def __bootstrap__():", - " global __bootstrap__, __loader__, __file__", - " import sys, pkg_resources, imp", - " __file__ = pkg_resources.resource_filename(__name__,%r)" - % resource, - " del __bootstrap__, __loader__", - " imp.load_dynamic(__name__,__file__)", - "__bootstrap__()", - "" # terminal \n - ])) - f.close() + + + + + + + + + + + + + def do_install_data(self): # Hack for packages that install data to install's --install-lib @@ -121,6 +147,21 @@ self.distribution.data_files = old + + + + + + + + + + + + + + + def run(self): # We run install_lib before install_data, because some data hacks # pull their data path from the install_lib command. @@ -138,7 +179,7 @@ pyfile = os.path.join(self.bdist_dir, filename + '.py') log.info("creating stub loader for %s" % ext_name) if not self.dry_run: - self.write_stub(os.path.basename(ext_name), pyfile) + write_stub(os.path.basename(ext_name), pyfile) to_compile.append(pyfile) ext_outputs[p] = ext_name.replace(os.sep,'/') From pje at users.sourceforge.net Wed Jun 15 04:16:06 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 19:16:06 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools pkg_resources.py, 1.33, 1.34 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9909 Modified Files: pkg_resources.py Log Message: Fix incorrect sorting of packages by string version instead of parsed version info. Don't set a default Python version for distribution objects. Add enum for binary distributions (like bdist_wininst). Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.33 retrieving revision 1.34 diff -u -d -r1.33 -r1.34 --- pkg_resources.py 14 Jun 2005 15:29:41 -0000 1.33 +++ pkg_resources.py 15 Jun 2005 02:16:03 -0000 1.34 @@ -22,7 +22,7 @@ 'InvalidOption', 'Distribution', 'Requirement', 'yield_lines', 'get_importer', 'find_distributions', 'find_on_path', 'register_finder', 'split_sections', 'declare_namespace', 'register_namespace_handler', - 'safe_name', 'safe_version', 'run_main', + 'safe_name', 'safe_version', 'run_main', 'BINARY_DIST', ] import sys, os, zipimport, time, re, imp @@ -54,7 +54,8 @@ _provider_factories = {} PY_MAJOR = sys.version[:3] -EGG_DIST = 2 +EGG_DIST = 3 +BINARY_DIST = 2 SOURCE_DIST = 1 def register_loader_type(loader_type, provider_factory): @@ -79,7 +80,6 @@ - def get_platform(): """Return this platform's string for platform-specific distributions @@ -231,7 +231,8 @@ def can_add(self, dist): """Is distribution `dist` acceptable for this collection?""" - return (self.python is None or dist.py_version==self.python) \ + return (self.python is None or dist.py_version is None + or dist.py_version==self.python) \ and compatible_platforms(dist.platform,self.platform) def __iter__(self): @@ -242,7 +243,6 @@ """Has a distribution named `name` ever been added to this map?""" return name.lower() in self._distmap - def __len__(self): return len(self._distmap) def get(self,key,default=None): """Return ``self[key]`` if `key` in self, otherwise return `default`""" @@ -361,11 +361,11 @@ return to_install # return list of distros to install - def obtain(self, requirement): """Obtain a distro that matches requirement (e.g. via download)""" return None # override this in subclasses + def __len__(self): return len(self._distmap) class ResourceManager: """Manage resource extraction and packages""" @@ -373,7 +373,7 @@ extraction_path = None def __init__(self): - self.cached_files = [] + self.cached_files = {} def resource_exists(self, package_name, resource_name): """Does the named resource exist in the named package?""" @@ -425,7 +425,7 @@ extract_path = extract_path or os.path.expanduser('~/.python-eggs') target_path = os.path.join(extract_path, archive_name, *names) ensure_directory(target_path) - self.cached_files.append(target_path) + self.cached_files[target_path] = 1 return target_path @@ -1263,7 +1263,7 @@ ) return cls( filename, metadata, name=name, version=version, - py_version=py_version or PY_MAJOR, platform=platform + py_version=py_version, platform=platform ) from_filename = classmethod(from_filename) @@ -1453,7 +1453,7 @@ def _sort_dists(dists): - tmp = [(dist.version,dist.distro_type,dist) for dist in dists] + tmp = [(dist.parsed_version,dist.distro_type,dist) for dist in dists] tmp.sort() dists[::-1] = [d for v,t,d in tmp] From pje at users.sourceforge.net Wed Jun 15 04:19:44 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 19:19:44 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools ez_setup.py, 1.3, 1.4 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11977 Modified Files: ez_setup.py Log Message: Add bootstrap installation support that "hitches a ride" on other packages being installed via the normal distutils "setup.py install". Also, don't repeatedly download the setuptools egg if it's already in the target location. Index: ez_setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/ez_setup.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- ez_setup.py 14 Jun 2005 15:48:44 -0000 1.3 +++ ez_setup.py 15 Jun 2005 02:19:42 -0000 1.4 @@ -63,8 +63,10 @@ sys.exit(2) except ImportError: - sys.path.insert(0, download_setuptools(version, download_base, to_dir)) - + egg = download_setuptools(version, download_base, to_dir) + sys.path.insert(0, egg) + import setuptools; setuptools.bootstrap_install_from = egg + import pkg_resources try: pkg_resources.require("setuptools>="+version) @@ -78,8 +80,6 @@ ) % version sys.exit(2) - - def download_setuptools( version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir ): @@ -95,13 +95,19 @@ saveto = os.path.join(to_dir, egg_name) src = dst = None - try: - src = urllib2.urlopen(url) - dst = open(saveto,"wb") - shutil.copyfileobj(src,dst) - finally: - if src: src.close() - if dst: dst.close() + if not os.path.exists(saveto): # Avoid repeated downloads + try: + from distutils import log + log.warn("Downloading %s", url) + src = urllib2.urlopen(url) + # Read/write all in one block, so we don't create a corrupt file + # if the download is interrupted. + data = src.read() + dst = open(saveto,"wb") + dst.write(data) + finally: + if src: src.close() + if dst: dst.close() return os.path.realpath(saveto) @@ -115,12 +121,6 @@ - - - - - - def main(argv, version=DEFAULT_VERSION): """Install or upgrade setuptools and EasyInstall""" From pje at users.sourceforge.net Wed Jun 15 04:19:46 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 19:19:46 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools dist.py, 1.6, 1.7 __init__.py, 1.6, 1.7 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11977/setuptools Modified Files: dist.py __init__.py Log Message: Add bootstrap installation support that "hitches a ride" on other packages being installed via the normal distutils "setup.py install". Also, don't repeatedly download the setuptools egg if it's already in the target location. Index: dist.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/dist.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- dist.py 12 Jun 2005 15:49:53 -0000 1.6 +++ dist.py 15 Jun 2005 02:19:42 -0000 1.7 @@ -8,6 +8,7 @@ from setuptools.command.install_lib import install_lib from distutils.errors import DistutilsOptionError, DistutilsPlatformError from distutils.errors import DistutilsSetupError +import setuptools sequence = tuple, list @@ -38,7 +39,6 @@ - class Distribution(_Distribution): """Distribution with support for features, tests, and package data @@ -348,18 +348,18 @@ return not not self.requires + def run_commands(self): + if setuptools.bootstrap_install_from and 'install' in self.commands: + # Bootstrap self-installation of setuptools + from easy_install import easy_install + cmd = easy_install( + self, args=[setuptools.bootstrap_install_from], zip_ok=1 + ) + cmd.ensure_finalized() + cmd.run() + setuptools.bootstrap_install_from = None - - - - - - - - - - - + _Distribution.run_commands(self) Index: __init__.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/__init__.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- __init__.py 14 Jun 2005 15:48:44 -0000 1.6 +++ __init__.py 15 Jun 2005 02:19:42 -0000 1.7 @@ -15,6 +15,7 @@ 'find_packages' ] +bootstrap_install_from = None def find_packages(where='.'): """Return a list all Python packages found within directory 'where' @@ -38,7 +39,6 @@ - def setup(**attrs): """Do package setup @@ -56,6 +56,12 @@ command_consumes_arguments = False + def __init__(self, dist, **kw): + # Add support for keyword arguments + _Command.__init__(self,dist) + for k,v in kw.items(): + setattr(self,k,v) + def reinitialize_command(self, command, reinit_subcommands=0, **kw): cmd = _Command.reinitialize_command(self, command, reinit_subcommands) for k,v in kw.items(): @@ -74,9 +80,3 @@ - - - - - - From pje at users.sourceforge.net Wed Jun 15 04:23:50 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 19:23:50 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools __init__.py, 1.7, 1.8 package_index.py, 1.4, 1.5 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14136/setuptools Modified Files: __init__.py package_index.py Log Message: Add support for installing from .win32.exe's created by distutils (by converting them to eggs). Bump version to 0.5a1. Index: __init__.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/__init__.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- __init__.py 15 Jun 2005 02:19:42 -0000 1.7 +++ __init__.py 15 Jun 2005 02:23:48 -0000 1.8 @@ -8,7 +8,7 @@ from distutils.util import convert_path import os.path -__version__ = '0.4a4' +__version__ = '0.5a1' __all__ = [ 'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require', Index: package_index.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/package_index.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- package_index.py 14 Jun 2005 15:30:31 -0000 1.4 +++ package_index.py 15 Jun 2005 02:23:48 -0000 1.5 @@ -9,13 +9,25 @@ EXTENSIONS = ".tar.gz .tar.bz2 .tar .zip .tgz".split() __all__ = [ - 'PackageIndex', 'distros_for_url', + 'PackageIndex', 'distros_for_url', 'parse_bdist_wininst', + 'interpret_distro_name', ] +def parse_bdist_wininst(name): + """Return (base,pyversion) or (None,None) for possible .exe name""" + lower = name.lower() + base, py_ver = None, None + if lower.endswith('.exe'): + if lower.endswith('.win32.exe'): + base = name[:-10] + elif lower[-16:].startswith('.win32-py'): + py_ver = base[-7:-4] + base = name[:-16] + return base,py_ver @@ -27,8 +39,32 @@ +def distros_for_url(url, metadata=None): + """Yield egg or source distribution objects that might be found at a URL""" + path = urlparse.urlparse(url)[2] + base = urllib2.unquote(path.split('/')[-1]) + + if base.endswith('.egg'): + dist = Distribution.from_filename(base, metadata) + dist.path = url + return [dist] # only one, unambiguous interpretation + if base.endswith('.exe'): + win_base, py_ver = parse_bdist_wininst(name) + if win_base is not None: + return interpret_distro_name( + url, win_base, metadata, py_ver, BINARY_DIST, "win32" + ) + + # Try source distro extensions (.zip, .tgz, etc.) + # + for ext in EXTENSIONS: + if base.endswith(ext): + base = base[:-len(ext)] + return interpret_distro_name(url, base, metadata) + + return [] # no extension matched @@ -39,24 +75,14 @@ -def distros_for_url(url, metadata=None): - """Yield egg or source distribution objects that might be found at a URL""" - path = urlparse.urlparse(url)[2] - base = urllib2.unquote(path.split('/')[-1]) - if base.endswith('.egg'): - dist = Distribution.from_filename(base, metadata) - dist.path = url - yield dist - return # only one, unambiguous interpretation - for ext in EXTENSIONS: - if base.endswith(ext): - base = base[:-len(ext)] - break - else: - return # no extension matched + + +def interpret_distro_name(url, base, metadata, + py_version=None, distro_type=SOURCE_DIST, platform=None +): # Generate alternative interpretations of a source distro name # Because some packages are ambiguous as to name/versions split @@ -74,12 +100,27 @@ for p in range(1,len(parts)+1): yield Distribution( url, metadata, '-'.join(parts[:p]), '-'.join(parts[p:]), - distro_type = SOURCE_DIST + py_version=py_version, distro_type = distro_type, + platform = platform ) + + + + + + + + + + + + + + class PackageIndex(AvailableDistributions): """A distribution index that scans web pages for download URLs""" From pje at users.sourceforge.net Wed Jun 15 04:23:50 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 19:23:50 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.13, 1.14 easy_install.py, 1.21, 1.22 ez_setup.py, 1.4, 1.5 setup.py, 1.14, 1.15 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14136 Modified Files: EasyInstall.txt easy_install.py ez_setup.py setup.py Log Message: Add support for installing from .win32.exe's created by distutils (by converting them to eggs). Bump version to 0.5a1. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- EasyInstall.txt 14 Jun 2005 15:30:31 -0000 1.13 +++ EasyInstall.txt 15 Jun 2005 02:23:47 -0000 1.14 @@ -23,14 +23,14 @@ ------------------------- Windows users can just download and run the `setuptools binary installer for -Windows `_. +Windows `_. All others should just download `ez_setup.py `_, and run it; this will download and install the correct version of ``setuptools`` for your Python version. You may receive a message telling you about an obsolete version of setuptools being present; if so, you must be sure to delete it entirely, along with the old ``pkg_resources`` module if it's present on ``sys.path``. - + An ``easy_install.py`` script will be installed in the normal location for Python scripts on your platform. In the examples below, you'll need to replace references to ``easy_install`` with the correct invocation to run @@ -62,7 +62,7 @@ **Example 2**. Install or upgrade a package by name and version by finding links on a given "download page":: - easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.4a4" + easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a1" **Example 3**. Download a source distribution from a specified URL, automatically building and installing it:: @@ -78,9 +78,9 @@ attempt to locate the latest available version that meets your criteria. When downloading or processing downloaded files, Easy Install recognizes -distutils *source* (not binary) distribution files with extensions of .tgz, -.tar, .tar.gz, .tar.bz2, or .zip. And of course it handles already-built .egg -distributions as well. +distutils source distribution files with extensions of .tgz, .tar, .tar.gz, +.tar.bz2, or .zip. And of course it handles already-built .egg +distributions as well as ``.win32.exe`` installers built using distutils. By default, packages are installed to the running Python installation's ``site-packages`` directory, unless you provide the ``-d`` or ``--install-dir`` @@ -268,7 +268,7 @@ # set the default location to install packages install_dir = /home/me/lib/python - + # Notice that indentation can be used to continue an option # value; this is especially useful for the "--find-links" # option, which tells easy_install to use download links on @@ -442,6 +442,24 @@ * There's no automatic retry for borked Sourceforge mirrors, which can easily time out or be missing a file. +0.5a1 + * Added support for converting ``.win32.exe`` installers to eggs on the fly. + EasyInstall will now recognize such files by name and install them. + + * Added support for "self-installation" bootstrapping. Packages can now + include ``ez_setup.py`` in their source distribution, and add the following + to their ``setup.py``, in order to automatically bootstrap installation of + setuptools as part of their setup process:: + + from ez_setup import use_setuptools + use_setuptools() + + from setuptools import setup + # etc... + + * Fixed a problem with picking the "best" version to install (versions were + being sorted as strings, rather than as parsed values) + 0.4a4 * Added support for the distutils "verbose/quiet" and "dry-run" options, as well as the "optimize" flag. @@ -465,7 +483,7 @@ 0.4a2 * Added support for installing scripts - + * Added support for setting options via distutils configuration files, and using distutils' default options as a basis for EasyInstall's defaults. @@ -558,9 +576,6 @@ ============ * Process the installed package's dependencies as well as the base package -* Support "self-installation" - bootstrapping setuptools install into another - package's installation process (copy egg, write setuptools.pth) -* Support installation from bdist_wininst packages? * Additional utilities to list/remove/verify packages * Signature checking? SSL? Ability to suppress PyPI search? * Display byte progress meter when downloading distributions and long pages? Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.21 retrieving revision 1.22 diff -u -d -r1.21 -r1.22 --- easy_install.py 14 Jun 2005 15:30:31 -0000 1.21 +++ easy_install.py 15 Jun 2005 02:23:47 -0000 1.22 @@ -12,7 +12,7 @@ """ -import sys, os.path, zipimport, shutil, tempfile +import sys, os.path, zipimport, shutil, tempfile, zipfile from setuptools import Command from setuptools.sandbox import run_setup @@ -20,9 +20,14 @@ from distutils.sysconfig import get_python_lib from distutils.errors import DistutilsArgError, DistutilsOptionError from setuptools.archive_util import unpack_archive -from setuptools.package_index import PackageIndex +from setuptools.package_index import PackageIndex, parse_bdist_wininst +from setuptools.command import bdist_egg from pkg_resources import * +__all__ = [ + 'samefile', 'easy_install', 'PthDistributions', 'extract_wininst_cfg', + 'main', 'get_exe_prefixes', +] def samefile(p1,p2): if hasattr(os.path,'samefile') and ( @@ -34,11 +39,6 @@ os.path.normpath(os.path.normcase(p2)) ) - - - - - class easy_install(Command): """Manage a download/build/install process""" @@ -249,6 +249,9 @@ if dist_filename.lower().endswith('.egg'): return [self.install_egg(dist_filename, True, tmpdir)] + if dist_filename.lower().endswith('.exe'): + return [self.install_exe(dist_filename, tmpdir)] + # Anything else, try to extract and build if os.path.isfile(dist_filename): unpack_archive(dist_filename, tmpdir, self.unpack_progress) @@ -282,9 +285,6 @@ - - - def install_egg(self, egg_path, zip_ok, tmpdir): destination = os.path.join(self.install_dir,os.path.basename(egg_path)) destination = os.path.abspath(destination) @@ -326,6 +326,88 @@ self.update_pth(dist) return dist + def install_exe(self, dist_filename, tmpdir): + # See if it's valid, get data + cfg = extract_wininst_cfg(dist_filename) + if cfg is None: + raise RuntimeError( + "%s is not a valid distutils Windows .exe" % dist_filename + ) + + # Create a dummy distribution object until we build the real distro + dist = Distribution(None, + name=cfg.get('metadata','name'), + version=cfg.get('metadata','version'), + platform="win32" + ) + + # Convert the .exe to an unpacked egg + egg_path = dist.path = os.path.join(tmpdir, dist.egg_name()+'.egg') + egg_tmp = egg_path+'.tmp' + self.exe_to_egg(dist_filename, egg_tmp) + + # Write EGG-INFO/PKG-INFO + pkg_inf = os.path.join(egg_tmp, 'EGG-INFO', 'PKG-INFO') + f = open(pkg_inf,'w') + f.write('Metadata-Version: 1.0\n') + for k,v in cfg.items('metadata'): + if k<>'target_version': + f.write('%s: %s\n' % (k.replace('_','-').title(), v)) + f.close() + + # Build .egg file from tmpdir + bdist_egg.make_zipfile( + egg_path, egg_tmp, + verbose=self.verbose, dry_run=self.dry_run + ) + + # install the .egg + return self.install_egg(egg_path, self.zip_ok, tmpdir) + + + + + def exe_to_egg(self, dist_filename, egg_tmp): + """Extract a bdist_wininst to the directories an egg would use""" + + # Check for .pth file and set up prefix translations + prefixes = get_exe_prefixes(dist_filename) + to_compile = [] + native_libs = [] + + def process(src,dst): + for old,new in prefixes: + if src.startswith(old): + src = new+src[len(old):] + dst = os.path.join(egg_tmp, *src.split('/')) + dl = dst.lower() + if dl.endswith('.pyd') or dl.endswith('.dll'): + native_libs.append(src) + elif dl.endswith('.py') and old!='SCRIPTS/': + to_compile.append(dst) + return dst + if not src.endswith('.pth'): + log.warn("WARNING: can't process %s", src) + return None + + # extract, tracking .pyd/.dll->native_libs and .py -> to_compile + unpack_archive(dist_filename, egg_tmp, process) + + for res in native_libs: + if res.lower().endswith('.pyd'): # create stubs for .pyd's + parts = res.split('/') + resource, parts[-1] = parts[-1], parts[-1][:-1] + pyfile = os.path.join(egg_tmp, *parts) + to_compile.append(pyfile) + bdist_egg.write_stub(resource, pyfile) + + self.byte_compile(to_compile) # compile .py's + + if native_libs: # write EGG-INFO/native_libs.txt + nl_txt = os.path.join(egg_tmp, 'EGG-INFO', 'native_libs.txt') + ensure_directory(nl_txt) + open(nl_txt,'w').write('\n'.join(native_libs)+'\n') + def installation_report(self, dist): """Helpful installation message for display to package users""" @@ -355,20 +437,19 @@ version = dist.version return msg % locals() - def update_pth(self,dist): - if self.pth_file is not None: - remove = self.pth_file.remove - for d in self.pth_file.get(dist.key,()): # drop old entries - log.info("Removing %s from .pth file", d) - remove(d) - if not self.multi_version: - log.info("Adding %s to .pth file", dist) - self.pth_file.add(dist) # add new entry - self.pth_file.save() + + + + + + + + + + def build_egg(self, tmpdir, setup_script): - from setuptools.command import bdist_egg sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) args = ['bdist_egg'] @@ -391,27 +472,28 @@ finally: log.set_verbosity(self.verbose) # restore our log verbosity + def update_pth(self,dist): + if self.pth_file is not None: + remove = self.pth_file.remove + for d in self.pth_file.get(dist.key,()): # drop old entries + log.info("Removing %s from .pth file", d) + remove(d) + if not self.multi_version: + log.info("Adding %s to .pth file", dist) + self.pth_file.add(dist) # add new entry + self.pth_file.save() - - - - - - - - - - - - - - + if dist.name=='setuptools': + # Ensure that setuptools itself never becomes unavailable! + f = open(os.path.join(self.install_dir,'setuptools.pth'), 'w') + f.write(dist.path+'\n') + f.close() def unpack_progress(self, src, dst): # Progress filter for unpacking log.debug("Unpacking %s to %s", src, dst) - return True # only unpack-and-compile skips files for dry run + return dst # only unpack-and-compile skips files for dry run def unpack_and_compile(self, egg_path, destination): @@ -421,10 +503,13 @@ if dst.endswith('.py'): to_compile.append(dst) self.unpack_progress(src,dst) - return not self.dry_run + return not self.dry_run and dst or None unpack_archive(egg_path, destination, pf) + self.byte_compile(to_compile) + + def byte_compile(self, to_compile): from distutils.util import byte_compile try: # try to make the byte compile messages quieter @@ -446,6 +531,85 @@ +def extract_wininst_cfg(dist_filename): + """Extract configuration data from a bdist_wininst .exe + + Returns a ConfigParser.RawConfigParser, or None + """ + f = open(dist_filename,'rb') + try: + endrec = zipfile._EndRecData(f) + if endrec is None: + return None + + prepended = (endrec[9] - endrec[5]) - endrec[6] + if prepended < 12: # no wininst data here + return None + f.seek(prepended-12) + + import struct, StringIO, ConfigParser + tag, cfglen, bmlen = struct.unpack("0x1234567A: + return None # not a valid tag + + f.seek(prepended-(12+cfglen+bmlen)) + cfg = ConfigParser.RawConfigParser({'version':'','target_version':''}) + try: + cfg.readfp(StringIO.StringIO(f.read(cfglen))) + except ConfigParser.Error: + return None + if not cfg.has_section('metadata') or not cfg.has_section('Setup'): + return None + return cfg + + finally: + f.close() + + + + + + + + +def get_exe_prefixes(exe_filename): + """Get exe->egg path translations for a given .exe file""" + + prefixes = [ + ('PURELIB/', ''), + ('PLATLIB/', ''), + ('SCRIPTS/', 'EGG-INFO/scripts/') + ] + z = zipfile.ZipFile(exe_filename) + try: + for info in z.infolist(): + name = info.filename + if not name.endswith('.pth'): + continue + parts = name.split('/') + if len(parts)<>2: + continue + if parts[0] in ('PURELIB','PLATLIB'): + pth = z.read(name).strip() + prefixes[0] = ('PURELIB/%s/' % pth), '' + prefixes[1] = ('PLATLIB/%s/' % pth), '' + break + finally: + z.close() + + return prefixes + + + + + + + + + + + + Index: ez_setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/ez_setup.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- ez_setup.py 15 Jun 2005 02:19:42 -0000 1.4 +++ ez_setup.py 15 Jun 2005 02:23:48 -0000 1.5 @@ -14,7 +14,7 @@ This file can also be run as a script to install or upgrade setuptools. """ -DEFAULT_VERSION = "0.4a4" +DEFAULT_VERSION = "0.5a1" DEFAULT_URL = "http://peak.telecommunity.com/dist/" import sys, os Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- setup.py 14 Jun 2005 15:48:44 -0000 1.14 +++ setup.py 15 Jun 2005 02:23:48 -0000 1.15 @@ -1,7 +1,7 @@ #!/usr/bin/env python """Distutils setup file, used to install or test 'setuptools'""" -VERSION = "0.4a4" +VERSION = "0.5a1" from setuptools import setup, find_packages, Require setup( From pje at users.sourceforge.net Wed Jun 15 04:39:12 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Tue, 14 Jun 2005 19:39:12 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools package_index.py, 1.5, 1.6 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21770/setuptools Modified Files: package_index.py Log Message: Fix stupid typos. Index: package_index.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/package_index.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- package_index.py 15 Jun 2005 02:23:48 -0000 1.5 +++ package_index.py 15 Jun 2005 02:39:10 -0000 1.6 @@ -24,7 +24,7 @@ if lower.endswith('.win32.exe'): base = name[:-10] elif lower[-16:].startswith('.win32-py'): - py_ver = base[-7:-4] + py_ver = name[-7:-4] base = name[:-16] return base,py_ver @@ -51,7 +51,7 @@ return [dist] # only one, unambiguous interpretation if base.endswith('.exe'): - win_base, py_ver = parse_bdist_wininst(name) + win_base, py_ver = parse_bdist_wininst(base) if win_base is not None: return interpret_distro_name( url, win_base, metadata, py_ver, BINARY_DIST, "win32" From mwh at users.sourceforge.net Wed Jun 15 13:38:04 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Wed, 15 Jun 2005 04:38:04 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libmarshal.tex, 1.25, 1.26 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6295/Doc/lib Modified Files: libmarshal.tex Log Message: Missed this documentation change about my marshal-floats-binarywise change. Index: libmarshal.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libmarshal.tex,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- libmarshal.tex 20 Dec 2004 12:25:56 -0000 1.25 +++ libmarshal.tex 15 Jun 2005 11:38:01 -0000 1.26 @@ -109,8 +109,9 @@ \begin{datadesc}{version} Indicates the format that the module uses. Version 0 is the - historical format, version 1 (added in Python 2.4) shares - interned strings. The current version is 1. + historical format, version 1 (added in Python 2.4) shares interned + strings and version 2 (added in Python 2.5) uses a binary format for + floating point numbers. The current version is 2. \versionadded{2.4} -\end{datadesc} \ No newline at end of file +\end{datadesc} From mwh at users.sourceforge.net Wed Jun 15 13:46:29 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Wed, 15 Jun 2005 04:46:29 -0700 Subject: [Python-checkins] python/dist/src/Demo/embed Makefile,1.12,1.13 Message-ID: Update of /cvsroot/python/python/dist/src/Demo/embed In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11763 Modified Files: Makefile Log Message: Update version. I wonder if there's a way to fix this Makefile to be slightly less totally unreliable. Index: Makefile =================================================================== RCS file: /cvsroot/python/python/dist/src/Demo/embed/Makefile,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- Makefile 25 Jul 2002 16:23:21 -0000 1.12 +++ Makefile 15 Jun 2005 11:46:27 -0000 1.13 @@ -10,7 +10,7 @@ srcdir= ../.. # Python version -VERSION= 2.3 +VERSION= 2.5 # Compiler flags OPT= -g From mwh at users.sourceforge.net Wed Jun 15 13:47:49 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Wed, 15 Jun 2005 04:47:49 -0700 Subject: [Python-checkins] python/dist/src/Demo/embed Makefile, 1.12, 1.12.18.1 Message-ID: Update of /cvsroot/python/python/dist/src/Demo/embed In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12828 Modified Files: Tag: release24-maint Makefile Log Message: Update version here, too. Index: Makefile =================================================================== RCS file: /cvsroot/python/python/dist/src/Demo/embed/Makefile,v retrieving revision 1.12 retrieving revision 1.12.18.1 diff -u -d -r1.12 -r1.12.18.1 --- Makefile 25 Jul 2002 16:23:21 -0000 1.12 +++ Makefile 15 Jun 2005 11:47:46 -0000 1.12.18.1 @@ -10,7 +10,7 @@ srcdir= ../.. # Python version -VERSION= 2.3 +VERSION= 2.4 # Compiler flags OPT= -g From mwh at users.sourceforge.net Wed Jun 15 14:25:22 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Wed, 15 Jun 2005 05:25:22 -0700 Subject: [Python-checkins] python/dist/src/Modules threadmodule.c, 2.61, 2.62 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1917 Modified Files: threadmodule.c Log Message: Add a missing incref. Backport candidate. Index: threadmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/threadmodule.c,v retrieving revision 2.61 retrieving revision 2.62 diff -u -d -r2.61 -r2.62 --- threadmodule.c 2 Jun 2005 17:07:11 -0000 2.61 +++ threadmodule.c 15 Jun 2005 12:25:20 -0000 2.62 @@ -651,6 +651,7 @@ Py_INCREF(&Locktype); PyDict_SetItemString(d, "LockType", (PyObject *)&Locktype); + Py_INCREF(&localtype); if (PyModule_AddObject(m, "_local", (PyObject *)&localtype) < 0) return; From mwh at users.sourceforge.net Wed Jun 15 14:48:43 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Wed, 15 Jun 2005 05:48:43 -0700 Subject: [Python-checkins] python/dist/src/Modules threadmodule.c, 2.62, 2.63 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14610 Modified Files: threadmodule.c Log Message: Consistently use hard tabs for indentation. Slightly de-Fultonize two bits of C layout. No semantic changes. Index: threadmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/threadmodule.c,v retrieving revision 2.62 retrieving revision 2.63 diff -u -d -r2.62 -r2.63 --- threadmodule.c 15 Jun 2005 12:25:20 -0000 2.62 +++ threadmodule.c 15 Jun 2005 12:48:40 -0000 2.63 @@ -158,11 +158,11 @@ #include "structmember.h" typedef struct { - PyObject_HEAD - PyObject *key; - PyObject *args; - PyObject *kw; - PyObject *dict; + PyObject_HEAD + PyObject *key; + PyObject *args; + PyObject *kw; + PyObject *dict; } localobject; static PyTypeObject localtype; @@ -170,91 +170,87 @@ static PyObject * local_new(PyTypeObject *type, PyObject *args, PyObject *kw) { - localobject *self; - PyObject *tdict; + localobject *self; + PyObject *tdict; - if (type->tp_init == PyBaseObject_Type.tp_init - && ((args && PyObject_IsTrue(args)) - || - (kw && PyObject_IsTrue(kw)) - ) - ) { - PyErr_SetString(PyExc_TypeError, - "Initialization arguments are not supported"); - return NULL; - } + if (type->tp_init == PyBaseObject_Type.tp_init + && ((args && PyObject_IsTrue(args)) + || (kw && PyObject_IsTrue(kw)))) { + PyErr_SetString(PyExc_TypeError, + "Initialization arguments are not supported"); + return NULL; + } - self = (localobject *)type->tp_alloc(type, 0); - if (self == NULL) - return NULL; + self = (localobject *)type->tp_alloc(type, 0); + if (self == NULL) + return NULL; - Py_XINCREF(args); - self->args = args; - Py_XINCREF(kw); - self->kw = kw; - self->dict = NULL; /* making sure */ - self->key = PyString_FromFormat("thread.local.%p", self); - if (self->key == NULL) - goto err; + Py_XINCREF(args); + self->args = args; + Py_XINCREF(kw); + self->kw = kw; + self->dict = NULL; /* making sure */ + self->key = PyString_FromFormat("thread.local.%p", self); + if (self->key == NULL) + goto err; - self->dict = PyDict_New(); - if (self->dict == NULL) - goto err; + self->dict = PyDict_New(); + if (self->dict == NULL) + goto err; - tdict = PyThreadState_GetDict(); - if (tdict == NULL) { - PyErr_SetString(PyExc_SystemError, - "Couldn't get thread-state dictionary"); - goto err; - } + tdict = PyThreadState_GetDict(); + if (tdict == NULL) { + PyErr_SetString(PyExc_SystemError, + "Couldn't get thread-state dictionary"); + goto err; + } - if (PyDict_SetItem(tdict, self->key, self->dict) < 0) - goto err; - - return (PyObject *)self; + if (PyDict_SetItem(tdict, self->key, self->dict) < 0) + goto err; - err: - Py_DECREF(self); - return NULL; + return (PyObject *)self; + + err: + Py_DECREF(self); + return NULL; } static int local_traverse(localobject *self, visitproc visit, void *arg) { - Py_VISIT(self->args); - Py_VISIT(self->kw); - Py_VISIT(self->dict); + Py_VISIT(self->args); + Py_VISIT(self->kw); + Py_VISIT(self->dict); return 0; } static int local_clear(localobject *self) { - Py_CLEAR(self->key); - Py_CLEAR(self->args); - Py_CLEAR(self->kw); - Py_CLEAR(self->dict); - return 0; + Py_CLEAR(self->key); + Py_CLEAR(self->args); + Py_CLEAR(self->kw); + Py_CLEAR(self->dict); + return 0; } static void local_dealloc(localobject *self) { - PyThreadState *tstate; - if (self->key - && (tstate = PyThreadState_Get()) - && tstate->interp) { - for(tstate = PyInterpreterState_ThreadHead(tstate->interp); - tstate; - tstate = PyThreadState_Next(tstate) - ) - if (tstate->dict && - PyDict_GetItem(tstate->dict, self->key)) - PyDict_DelItem(tstate->dict, self->key); - } + PyThreadState *tstate; + if (self->key + && (tstate = PyThreadState_Get()) + && tstate->interp) { + for(tstate = PyInterpreterState_ThreadHead(tstate->interp); + tstate; + tstate = PyThreadState_Next(tstate)) + if (tstate->dict && + PyDict_GetItem(tstate->dict, self->key)) + PyDict_DelItem(tstate->dict, self->key); + } - local_clear(self); - self->ob_type->tp_free((PyObject*)self); + local_clear(self); + self->ob_type->tp_free((PyObject*)self); } static PyObject * @@ -263,48 +259,47 @@ PyObject *tdict, *ldict; tdict = PyThreadState_GetDict(); - if (tdict == NULL) { - PyErr_SetString(PyExc_SystemError, - "Couldn't get thread-state dictionary"); - return NULL; - } + if (tdict == NULL) { + PyErr_SetString(PyExc_SystemError, + "Couldn't get thread-state dictionary"); + return NULL; + } - ldict = PyDict_GetItem(tdict, self->key); - if (ldict == NULL) { - ldict = PyDict_New(); /* we own ldict */ + ldict = PyDict_GetItem(tdict, self->key); + if (ldict == NULL) { + ldict = PyDict_New(); /* we own ldict */ - if (ldict == NULL) - return NULL; - else { - int i = PyDict_SetItem(tdict, self->key, ldict); - Py_DECREF(ldict); /* now ldict is borowed */ - if (i < 0) - return NULL; - } + if (ldict == NULL) + return NULL; + else { + int i = PyDict_SetItem(tdict, self->key, ldict); + Py_DECREF(ldict); /* now ldict is borowed */ + if (i < 0) + return NULL; + } - Py_CLEAR(self->dict); - Py_INCREF(ldict); - self->dict = ldict; /* still borrowed */ + Py_CLEAR(self->dict); + Py_INCREF(ldict); + self->dict = ldict; /* still borrowed */ - if (self->ob_type->tp_init != PyBaseObject_Type.tp_init && - self->ob_type->tp_init((PyObject*)self, - self->args, self->kw) < 0 - ) { - /* we need to get rid of ldict from thread so - we create a new one the next time we do an attr - acces */ - PyDict_DelItem(tdict, self->key); - return NULL; - } - - } - else if (self->dict != ldict) { - Py_CLEAR(self->dict); - Py_INCREF(ldict); - self->dict = ldict; - } + if (self->ob_type->tp_init != PyBaseObject_Type.tp_init && + self->ob_type->tp_init((PyObject*)self, + self->args, self->kw) < 0) { + /* we need to get rid of ldict from thread so + we create a new one the next time we do an attr + acces */ + PyDict_DelItem(tdict, self->key); + return NULL; + } + + } + else if (self->dict != ldict) { + Py_CLEAR(self->dict); + Py_INCREF(ldict); + self->dict = ldict; + } - return ldict; + return ldict; } static PyObject * @@ -312,54 +307,52 @@ { PyObject *ldict, *value; - ldict = _ldict(self); - if (ldict == NULL) - return NULL; + ldict = _ldict(self); + if (ldict == NULL) + return NULL; - if (self->ob_type != &localtype) - /* use generic lookup for subtypes */ - return PyObject_GenericGetAttr((PyObject *)self, name); + if (self->ob_type != &localtype) + /* use generic lookup for subtypes */ + return PyObject_GenericGetAttr((PyObject *)self, name); - /* Optimization: just look in dict ourselves */ - value = PyDict_GetItem(ldict, name); - if (value == NULL) - /* Fall back on generic to get __class__ and __dict__ */ - return PyObject_GenericGetAttr((PyObject *)self, name); + /* Optimization: just look in dict ourselves */ + value = PyDict_GetItem(ldict, name); + if (value == NULL) + /* Fall back on generic to get __class__ and __dict__ */ + return PyObject_GenericGetAttr((PyObject *)self, name); - Py_INCREF(value); - return value; + Py_INCREF(value); + return value; } static int local_setattro(localobject *self, PyObject *name, PyObject *v) { PyObject *ldict; - - ldict = _ldict(self); - if (ldict == NULL) - return -1; + + ldict = _ldict(self); + if (ldict == NULL) + return -1; - return PyObject_GenericSetAttr((PyObject *)self, name, v); + return PyObject_GenericSetAttr((PyObject *)self, name, v); } static PyObject * local_getdict(localobject *self, void *closure) { - if (self->dict == NULL) { - PyErr_SetString(PyExc_AttributeError, "__dict__"); - return NULL; - } + if (self->dict == NULL) { + PyErr_SetString(PyExc_AttributeError, "__dict__"); + return NULL; + } - Py_INCREF(self->dict); - return self->dict; + Py_INCREF(self->dict); + return self->dict; } static PyGetSetDef local_getset[] = { - {"__dict__", - (getter)local_getdict, (setter)0, - "Local-data dictionary", - NULL}, - {NULL} /* Sentinel */ + {"__dict__", (getter)local_getdict, (setter)NULL, + "Local-data dictionary", NULL}, + {NULL} /* Sentinel */ }; static PyTypeObject localtype = { @@ -380,28 +373,28 @@ /* tp_hash */ (hashfunc)0, /* tp_call */ (ternaryfunc)0, /* tp_str */ (reprfunc)0, - /* tp_getattro */ (getattrofunc)local_getattro, - /* tp_setattro */ (setattrofunc)local_setattro, - /* tp_as_buffer */ 0, - /* tp_flags */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, + /* tp_getattro */ (getattrofunc)local_getattro, + /* tp_setattro */ (setattrofunc)local_setattro, + /* tp_as_buffer */ 0, + /* tp_flags */ Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE, /* tp_doc */ "Thread-local data", - /* tp_traverse */ (traverseproc)local_traverse, - /* tp_clear */ (inquiry)local_clear, - /* tp_richcompare */ (richcmpfunc)0, - /* tp_weaklistoffset */ (long)0, - /* tp_iter */ (getiterfunc)0, - /* tp_iternext */ (iternextfunc)0, - /* tp_methods */ 0, - /* tp_members */ 0, - /* tp_getset */ local_getset, - /* tp_base */ 0, - /* tp_dict */ 0, /* internal use */ - /* tp_descr_get */ (descrgetfunc)0, - /* tp_descr_set */ (descrsetfunc)0, - /* tp_dictoffset */ offsetof(localobject, dict), - /* tp_init */ (initproc)0, - /* tp_alloc */ (allocfunc)0, - /* tp_new */ (newfunc)local_new, + /* tp_traverse */ (traverseproc)local_traverse, + /* tp_clear */ (inquiry)local_clear, + /* tp_richcompare */ (richcmpfunc)0, + /* tp_weaklistoffset */ (long)0, + /* tp_iter */ (getiterfunc)0, + /* tp_iternext */ (iternextfunc)0, + /* tp_methods */ 0, + /* tp_members */ 0, + /* tp_getset */ local_getset, + /* tp_base */ 0, + /* tp_dict */ 0, /* internal use */ + /* tp_descr_get */ (descrgetfunc)0, + /* tp_descr_set */ (descrsetfunc)0, + /* tp_dictoffset */ offsetof(localobject, dict), + /* tp_init */ (initproc)0, + /* tp_alloc */ (allocfunc)0, + /* tp_new */ (newfunc)local_new, /* tp_free */ 0, /* Low-level free-mem routine */ /* tp_is_gc */ (inquiry)0, /* For PyObject_IS_GC */ }; @@ -635,10 +628,10 @@ initthread(void) { PyObject *m, *d; - - /* Initialize types: */ - if (PyType_Ready(&localtype) < 0) - return; + + /* Initialize types: */ + if (PyType_Ready(&localtype) < 0) + return; /* Create the module and add the functions */ m = Py_InitModule3("thread", thread_methods, thread_doc); @@ -652,8 +645,8 @@ PyDict_SetItemString(d, "LockType", (PyObject *)&Locktype); Py_INCREF(&localtype); - if (PyModule_AddObject(m, "_local", (PyObject *)&localtype) < 0) - return; + if (PyModule_AddObject(m, "_local", (PyObject *)&localtype) < 0) + return; /* Initialize the C thread library */ PyThread_init_thread(); From montanaro at users.sourceforge.net Wed Jun 15 15:35:10 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Wed, 15 Jun 2005 06:35:10 -0700 Subject: [Python-checkins] python/dist/src/Modules _csv.c,1.37,1.38 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7496 Modified Files: _csv.c Log Message: Michael Hudson pointed out that the Dialect_Type object isn't INCREF'd. Why this worked is a bit mystical. Perhaps it never gets freed because the object just happens never to be DECREF'd (but that seems unlikely). Index: _csv.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_csv.c,v retrieving revision 1.37 retrieving revision 1.38 diff -u -d -r1.37 -r1.38 --- _csv.c 15 Jun 2005 01:33:30 -0000 1.37 +++ _csv.c 15 Jun 2005 13:35:08 -0000 1.38 @@ -1587,6 +1587,7 @@ } /* Add the Dialect type */ + Py_INCREF(&Dialect_Type); if (PyModule_AddObject(module, "Dialect", (PyObject *)&Dialect_Type)) return; From montanaro at users.sourceforge.net Wed Jun 15 15:38:10 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Wed, 15 Jun 2005 06:38:10 -0700 Subject: [Python-checkins] python/dist/src/Modules _csv.c,1.15,1.15.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8778 Modified Files: Tag: release24-maint _csv.c Log Message: backport Index: _csv.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_csv.c,v retrieving revision 1.15 retrieving revision 1.15.2.1 diff -u -d -r1.15 -r1.15.2.1 --- _csv.c 15 Aug 2004 12:23:10 -0000 1.15 +++ _csv.c 15 Jun 2005 13:38:07 -0000 1.15.2.1 @@ -1545,6 +1545,7 @@ } /* Add the Dialect type */ + Py_INCREF(&Dialect_Type); if (PyModule_AddObject(module, "Dialect", (PyObject *)&Dialect_Type)) return; From rhettinger at users.sourceforge.net Wed Jun 15 18:34:05 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Wed, 15 Jun 2005 09:34:05 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.314, 1.315 pep-0288.txt, 1.5, 1.6 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4613 Modified Files: pep-0000.txt pep-0288.txt Log Message: Withdraw 288 in favor of 343. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.314 retrieving revision 1.315 diff -u -d -r1.314 -r1.315 --- pep-0000.txt 14 Jun 2005 15:14:01 -0000 1.314 +++ pep-0000.txt 15 Jun 2005 16:34:01 -0000 1.315 @@ -92,7 +92,6 @@ S 284 Integer for-loops Eppstein, Ewing S 286 Enhanced Argument Tuples von Loewis I 287 reStructuredText Docstring Format Goodger - S 288 Generators Attributes and Exceptions Hettinger S 294 Type Names in the types Module Tirosh S 297 Support for System Upgrades Lemburg S 298 The Locked Buffer Interface Heller @@ -205,6 +204,7 @@ SD 269 Pgen Module for Python Riehl SR 270 uniq method for list objects Petrone SR 271 Prefixing sys.path by command line option Giacometti + SR 288 Generators Attributes and Exceptions Hettinger SR 295 Interpretation of multiline string constants Koltsov SR 296 Adding a bytes Object Type Gilbert SR 308 If-then-else expression GvR, Hettinger @@ -330,7 +330,7 @@ SF 285 Adding a bool type GvR S 286 Enhanced Argument Tuples von Loewis I 287 reStructuredText Docstring Format Goodger - S 288 Generators Attributes and Exceptions Hettinger + SR 288 Generators Attributes and Exceptions Hettinger SF 289 Generator Expressions Hettinger I 290 Code Migration and Modernization Hettinger I 291 Backward Compatibility for Standard Library Norwitz Index: pep-0288.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0288.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- pep-0288.txt 3 Jan 2005 04:59:21 -0000 1.5 +++ pep-0288.txt 15 Jun 2005 16:34:03 -0000 1.6 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: python at rcn.com (Raymond D. Hettinger) -Status: Draft +Status: Withdrawn Type: Standards Track Created: 21-Mar-2002 Python-Version: 2.5 @@ -16,6 +16,14 @@ raising exceptions and sharing data with running generators. +Status + + This PEP is withdrawn. The exception raising mechanism was extended + and subsumed into PEP 343. The attribute passing capability + never built a following, did not have a clear implementation, + and did not have a clean way for the running generator to access + its own namespace. + Rationale @@ -49,7 +57,7 @@ the reference. For example: def mygen(filename): - self = mygen.get_instance() + self = sys.get_generator() myfile = open(filename) for line in myfile: if len(line) < 10: From rhettinger at users.sourceforge.net Wed Jun 15 18:53:33 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Wed, 15 Jun 2005 09:53:33 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libdecimal.tex, 1.26, 1.27 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16636 Modified Files: libdecimal.tex Log Message: Fix nits. Index: libdecimal.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libdecimal.tex,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- libdecimal.tex 7 Jun 2005 18:50:56 -0000 1.26 +++ libdecimal.tex 15 Jun 2005 16:53:31 -0000 1.27 @@ -1121,9 +1121,9 @@ +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \subsection{Decimal FAQ \label{decimal-faq}} - Q. It is cumbersome to type \code{decimal.Decimal('1234.5')}. Is there a way to minimize typing when using the interactive interpreter? @@ -1169,7 +1169,7 @@ places and need to be followed-up with a \method{quantize()} step. -Q. There are many ways to write express the same value. The numbers +Q. There are many ways to express the same value. The numbers \constant{200}, \constant{200.000}, \constant{2E2}, and \constant{.02E+4} all have the same value at various precisions. Is there a way to transform them to a single recognizable canonical value? @@ -1203,6 +1203,7 @@ mantissa *= 2.0 exponent -= 1 mantissa = int(mantissa) + oldcontext = getcontext() setcontext(Context(traps=[Inexact])) try: From rhettinger at users.sourceforge.net Wed Jun 15 18:54:49 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Wed, 15 Jun 2005 09:54:49 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libdecimal.tex, 1.24.2.1, 1.24.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17437 Modified Files: Tag: release24-maint libdecimal.tex Log Message: Fix nits. Index: libdecimal.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libdecimal.tex,v retrieving revision 1.24.2.1 retrieving revision 1.24.2.2 diff -u -d -r1.24.2.1 -r1.24.2.2 --- libdecimal.tex 12 Jun 2005 18:25:29 -0000 1.24.2.1 +++ libdecimal.tex 15 Jun 2005 16:54:30 -0000 1.24.2.2 @@ -1121,9 +1121,9 @@ +%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \subsection{Decimal FAQ \label{decimal-faq}} - Q. It is cumbersome to type \code{decimal.Decimal('1234.5')}. Is there a way to minimize typing when using the interactive interpreter? @@ -1169,7 +1169,7 @@ places and need to be followed-up with a \method{quantize()} step. -Q. There are many ways to write express the same value. The numbers +Q. There are many ways to express the same value. The numbers \constant{200}, \constant{200.000}, \constant{2E2}, and \constant{.02E+4} all have the same value at various precisions. Is there a way to transform them to a single recognizable canonical value? @@ -1203,6 +1203,7 @@ mantissa *= 2.0 exponent -= 1 mantissa = int(mantissa) + oldcontext = getcontext() setcontext(Context(traps=[Inexact])) try: From akuchling at users.sourceforge.net Wed Jun 15 20:44:25 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 15 Jun 2005 11:44:25 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_curses.py,1.8,1.9 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15879 Modified Files: test_curses.py Log Message: [Patch #1005892 from Alexandr Zamaraev] Fix two errors in the curses test suite Index: test_curses.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_curses.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- test_curses.py 7 Aug 2004 15:27:16 -0000 1.8 +++ test_curses.py 15 Jun 2005 18:44:23 -0000 1.9 @@ -107,6 +107,7 @@ stdscr.scroll(2) stdscr.scroll(-3) + stdscr.move(12, 2) stdscr.setscrreg(10,15) win3 = stdscr.subwin(10,10) win3 = stdscr.subwin(10,10, 5,5) @@ -179,7 +180,7 @@ curses.init_pair(2, 1,1) curses.color_content(1) curses.color_pair(2) - curses.pair_content(curses.COLOR_PAIRS) + curses.pair_content(curses.COLOR_PAIRS - 1) curses.pair_number(0) if hasattr(curses, 'use_default_colors'): From akuchling at users.sourceforge.net Wed Jun 15 20:45:37 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 15 Jun 2005 11:45:37 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_curses.py, 1.8, 1.8.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16177 Modified Files: Tag: release24-maint test_curses.py Log Message: [Patch #1005892 from Alexandr Zamaraev] Fix two errors in the curses test suite Index: test_curses.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_curses.py,v retrieving revision 1.8 retrieving revision 1.8.2.1 diff -u -d -r1.8 -r1.8.2.1 --- test_curses.py 7 Aug 2004 15:27:16 -0000 1.8 +++ test_curses.py 15 Jun 2005 18:45:27 -0000 1.8.2.1 @@ -107,6 +107,7 @@ stdscr.scroll(2) stdscr.scroll(-3) + stdscr.move(12, 2) stdscr.setscrreg(10,15) win3 = stdscr.subwin(10,10) win3 = stdscr.subwin(10,10, 5,5) @@ -179,7 +180,7 @@ curses.init_pair(2, 1,1) curses.color_content(1) curses.color_pair(2) - curses.pair_content(curses.COLOR_PAIRS) + curses.pair_content(curses.COLOR_PAIRS - 1) curses.pair_number(0) if hasattr(curses, 'use_default_colors'): From akuchling at users.sourceforge.net Wed Jun 15 20:46:53 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 15 Jun 2005 11:46:53 -0700 Subject: [Python-checkins] python/dist/src/Modules _cursesmodule.c, 2.74, 2.75 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16843 Modified Files: _cursesmodule.c Log Message: Fix check for error condition Index: _cursesmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_cursesmodule.c,v retrieving revision 2.74 retrieving revision 2.75 diff -u -d -r2.74 -r2.75 --- _cursesmodule.c 9 Jun 2005 17:53:27 -0000 2.74 +++ _cursesmodule.c 15 Jun 2005 18:46:50 -0000 2.75 @@ -2107,7 +2107,7 @@ return NULL; } - if (!pair_content(pair, &f, &b)) { + if (pair_content(pair, &f, &b)==ERR) { PyErr_SetString(PyCursesError, "Argument 1 was out of range. (1..COLOR_PAIRS-1)"); return NULL; From akuchling at users.sourceforge.net Wed Jun 15 20:48:31 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Wed, 15 Jun 2005 11:48:31 -0700 Subject: [Python-checkins] python/dist/src/Modules _cursesmodule.c, 2.73.2.1, 2.73.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17695 Modified Files: Tag: release24-maint _cursesmodule.c Log Message: Fix check for error condition Index: _cursesmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_cursesmodule.c,v retrieving revision 2.73.2.1 retrieving revision 2.73.2.2 diff -u -d -r2.73.2.1 -r2.73.2.2 --- _cursesmodule.c 9 Jun 2005 17:55:34 -0000 2.73.2.1 +++ _cursesmodule.c 15 Jun 2005 18:48:16 -0000 2.73.2.2 @@ -2107,7 +2107,7 @@ return NULL; } - if (!pair_content(pair, &f, &b)) { + if (pair_content(pair, &f, &b)==ERR) { PyErr_SetString(PyCursesError, "Argument 1 was out of range. (1..COLOR_PAIRS-1)"); return NULL; From mwh at users.sourceforge.net Thu Jun 16 13:35:15 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Thu, 16 Jun 2005 04:35:15 -0700 Subject: [Python-checkins] python/dist/src/Python pystate.c,2.40,2.41 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2668 Modified Files: pystate.c Log Message: You can have more than one thread state for a thread if they correspond to different interpreters (I hope, please revert if this is wrong :). Index: pystate.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/pystate.c,v retrieving revision 2.40 retrieving revision 2.41 diff -u -d -r2.40 -r2.41 --- pystate.c 18 Apr 2005 08:46:17 -0000 2.40 +++ pystate.c 16 Jun 2005 11:35:00 -0000 2.41 @@ -289,7 +289,7 @@ #if defined(Py_DEBUG) && defined(WITH_THREAD) if (new) { PyThreadState *check = PyGILState_GetThisThreadState(); - if (check && check != new) + if (check && check->interp == new->interp && check != new) Py_FatalError("Invalid thread state for this thread"); } #endif From mwh at users.sourceforge.net Thu Jun 16 16:53:25 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Thu, 16 Jun 2005 07:53:25 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_class.py, 1.12, 1.12.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv26229 Modified Files: Tag: release24-maint test_class.py Log Message: backport Lib/test/test_class.py revision 1.13: Don't use 'is not' to compare strings. (spotted by reading pypy-svn :) Index: test_class.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_class.py,v retrieving revision 1.12 retrieving revision 1.12.4.1 diff -u -d -r1.12 -r1.12.4.1 --- test_class.py 19 Jul 2004 16:29:15 -0000 1.12 +++ test_class.py 16 Jun 2005 14:53:22 -0000 1.12.4.1 @@ -353,7 +353,7 @@ try: A().a # Raised AttributeError: A instance has no attribute 'a' except AttributeError, x: - if str(x) is not "booh": + if str(x) != "booh": print "attribute error for A().a got masked:", str(x) class E: From mwh at users.sourceforge.net Thu Jun 16 17:51:22 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Thu, 16 Jun 2005 08:51:22 -0700 Subject: [Python-checkins] python/dist/src README,1.188,1.189 Message-ID: Update of /cvsroot/python/python/dist/src In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22318 Modified Files: README Log Message: Update URL. Index: README =================================================================== RCS file: /cvsroot/python/python/dist/src/README,v retrieving revision 1.188 retrieving revision 1.189 diff -u -d -r1.188 -r1.189 --- README 24 Apr 2005 22:26:37 -0000 1.188 +++ README 16 Jun 2005 15:51:20 -0000 1.189 @@ -68,7 +68,7 @@ as a list of other introductions, and reference documentation. There's a quickly growing set of books on Python. See -www.python.org/moin/PythonBooks for a list. +http://wiki.python.org/moin/PythonBooks for a list. Documentation From greg at users.sourceforge.net Thu Jun 16 21:01:45 2005 From: greg at users.sourceforge.net (greg@users.sourceforge.net) Date: Thu, 16 Jun 2005 12:01:45 -0700 Subject: [Python-checkins] python/dist/src/Modules _bsddb.c,1.45,1.46 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6945/extsrc Modified Files: _bsddb.c Log Message: remove c++ style comment Index: _bsddb.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/_bsddb.c,v retrieving revision 1.45 retrieving revision 1.46 diff -u -d -r1.45 -r1.46 --- _bsddb.c 6 Jun 2005 17:31:32 -0000 1.45 +++ _bsddb.c 16 Jun 2005 19:01:42 -0000 1.46 @@ -2027,7 +2027,7 @@ result = PyEval_CallObject (self->btCompareCallback, args); if (result == 0) { /* we're in a callback within the DB code, we can't raise */ - PyErr_Print (); // XXX-gps or can we? either way the DB is screwed + PyErr_Print (); res = _default_cmp (leftKey, rightKey); } else if (PyInt_Check (result)) { From jackjansen at users.sourceforge.net Thu Jun 16 23:26:33 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Thu, 16 Jun 2005 14:26:33 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenType.py, 1.13, 1.14 bgenVariable.py, 1.3, 1.4 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20173 Modified Files: bgenType.py bgenVariable.py Log Message: More factorization. Index: bgenType.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenType.py,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- bgenType.py 14 Jun 2005 21:32:51 -0000 1.13 +++ bgenType.py 16 Jun 2005 21:26:23 -0000 1.14 @@ -24,11 +24,16 @@ Example: int.declare('spam') prints "int spam;" """ + Output("%s;", self.getDeclaration(name, reference)) + + def getDeclaration(self, name, reference=False): + """Return a string declaring a variable or argument, without + any syntactic adornment""" if reference: - Output("%s& %s;", self.typeName, name) + return "%s& %s" % (self.typeName, name) else: - Output("%s %s;", self.typeName, name) - + return "%s %s" % (self.typeName, name) + def getargs(self): return self.getargsFormat(), self.getargsArgs() @@ -72,6 +77,7 @@ Default is to call passInput(). """ return self.passInput(name) + def errorCheck(self, name): """Check for an error returned in the variable. Index: bgenVariable.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenVariable.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- bgenVariable.py 14 Jun 2005 21:32:51 -0000 1.3 +++ bgenVariable.py 16 Jun 2005 21:26:24 -0000 1.4 @@ -43,6 +43,11 @@ self.type.declare(self.name, reference=True) elif self.flags != SelfMode: self.type.declare(self.name) + + def getDeclaration(self): + """Return the unadorned declaration of the variable, + suitable for use in a formal parameter list.""" + return self.type.getDeclaration(self.name) def getargsFormat(self): """Call the type's getargsFormatmethod.""" From montanaro at users.sourceforge.net Fri Jun 17 03:14:53 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Thu, 16 Jun 2005 18:14:53 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_posix.py,1.7,1.8 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv365 Modified Files: test_posix.py Log Message: Add tests for posix O_SHLOCK & O_EXLOCK. Missed checking this in with posixmodule.c 2.335. Really should be considered part of patch #1103951. Index: test_posix.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_posix.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- test_posix.py 6 Jun 2004 20:27:05 -0000 1.7 +++ test_posix.py 17 Jun 2005 01:14:49 -0000 1.8 @@ -94,6 +94,37 @@ self.fdopen_helper('r') self.fdopen_helper('r', 100) + def test_osexlock(self): + if hasattr(posix, "O_EXLOCK"): + fd = os.open(test_support.TESTFN, + os.O_WRONLY|os.O_EXLOCK|os.O_CREAT) + self.assertRaises(OSError, os.open, test_support.TESTFN, + os.O_WRONLY|os.O_EXLOCK|os.O_NONBLOCK) + os.close(fd) + + if hasattr(posix, "O_SHLOCK"): + fd = os.open(test_support.TESTFN, + os.O_WRONLY|os.O_SHLOCK|os.O_CREAT) + self.assertRaises(OSError, os.open, test_support.TESTFN, + os.O_WRONLY|os.O_EXLOCK|os.O_NONBLOCK) + os.close(fd) + + def test_osshlock(self): + if hasattr(posix, "O_SHLOCK"): + fd1 = os.open(test_support.TESTFN, + os.O_WRONLY|os.O_SHLOCK|os.O_CREAT) + fd2 = os.open(test_support.TESTFN, + os.O_WRONLY|os.O_SHLOCK|os.O_CREAT) + os.close(fd2) + os.close(fd1) + + if hasattr(posix, "O_EXLOCK"): + fd = os.open(test_support.TESTFN, + os.O_WRONLY|os.O_SHLOCK|os.O_CREAT) + self.assertRaises(OSError, os.open, test_support.TESTFN, + os.O_RDONLY|os.O_EXLOCK|os.O_NONBLOCK) + os.close(fd) + def test_fstat(self): if hasattr(posix, 'fstat'): fp = open(test_support.TESTFN) From rhettinger at users.sourceforge.net Fri Jun 17 04:13:13 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Thu, 16 Jun 2005 19:13:13 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.315, 1.316 pep-0274.txt, 1.5, 1.6 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27569 Modified Files: pep-0000.txt pep-0274.txt Log Message: With Barry's blessing, marking PEP 274 as withdrawn. After genexps were introduced, it was no longer needed. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.315 retrieving revision 1.316 diff -u -d -r1.315 -r1.316 --- pep-0000.txt 15 Jun 2005 16:34:01 -0000 1.315 +++ pep-0000.txt 17 Jun 2005 02:13:11 -0000 1.316 @@ -84,7 +84,6 @@ S 266 Optimizing Global Variable/Attribute Access Montanaro S 267 Optimized Access to Module Namespaces Hylton S 268 Extended HTTP functionality and WebDAV Stein - S 274 Dict Comprehensions Warsaw S 275 Switching on Multiple Values Lemburg S 276 Simple Iterator for ints Althoff S 280 Optimizing access to globals GvR @@ -204,6 +203,7 @@ SD 269 Pgen Module for Python Riehl SR 270 uniq method for list objects Petrone SR 271 Prefixing sys.path by command line option Giacometti + SR 274 Dict Comprehensions Warsaw SR 288 Generators Attributes and Exceptions Hettinger SR 295 Interpretation of multiline string constants Koltsov SR 296 Adding a bytes Object Type Gilbert @@ -316,7 +316,7 @@ SR 271 Prefixing sys.path by command line option Giacometti IF 272 API for Block Encryption Algorithms v1.0 Kuchling SF 273 Import Modules from Zip Archives Ahlstrom - S 274 Dict Comprehensions Warsaw + SR 274 Dict Comprehensions Warsaw S 275 Switching on Multiple Values Lemburg S 276 Simple Iterator for ints Althoff SF 277 Unicode file name support for Windows NT Hodgson Index: pep-0274.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0274.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- pep-0274.txt 22 Sep 2003 04:51:50 -0000 1.5 +++ pep-0274.txt 17 Jun 2005 02:13:11 -0000 1.6 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: barry at python.org (Barry A. Warsaw) -Status: Draft +Status: Withdrawn Type: Standards Track Created: 25-Oct-2001 Python-Version: 2.3 @@ -19,6 +19,11 @@ very similar to list comprehensions, except that they produce Python dictionary objects instead of list objects. +Resolution + + This PEP is withdrawn. Substantially all of its benefits were + subsumed by generator expressions coupled with the dict() constructor. + Proposed Solution From rhettinger at users.sourceforge.net Fri Jun 17 06:30:43 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Thu, 16 Jun 2005 21:30:43 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.316, 1.317 pep-0265.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28394 Modified Files: pep-0000.txt pep-0265.txt Log Message: Record the rejection of PEP 265. The requested functionality was largely fulfilled by Py2.4's sorted() function. See Guido's 6/17/2005 note on python-dev. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.316 retrieving revision 1.317 diff -u -d -r1.316 -r1.317 --- pep-0000.txt 17 Jun 2005 02:13:11 -0000 1.316 +++ pep-0000.txt 17 Jun 2005 04:30:41 -0000 1.317 @@ -80,7 +80,6 @@ S 256 Docstring Processing System Framework Goodger S 258 Docutils Design Specification Goodger SD 262 Database of Installed Python Packages Kuchling - S 265 Sorting Dictionaries by Value Griffin S 266 Optimizing Global Variable/Attribute Access Montanaro S 267 Optimized Access to Module Namespaces Hylton S 268 Extended HTTP functionality and WebDAV Stein @@ -200,6 +199,7 @@ SR 242 Numeric Kinds Dubois SR 244 The `directive' Statement von Loewis SR 259 Omit printing newline after newline GvR + SR 265 Sorting Dictionaries by Value Griffin SD 269 Pgen Module for Python Riehl SR 270 uniq method for list objects Petrone SR 271 Prefixing sys.path by command line option Giacometti @@ -307,7 +307,7 @@ SD 262 Database of Installed Python Packages Kuchling SF 263 Defining Python Source Code Encodings Lemburg SF 264 Future statements in simulated shells Hudson - S 265 Sorting Dictionaries by Value Griffin + SR 265 Sorting Dictionaries by Value Griffin S 266 Optimizing Global Variable/Attribute Access Montanaro S 267 Optimized Access to Module Namespaces Hylton S 268 Extended HTTP functionality and WebDAV Stein Index: pep-0265.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0265.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0265.txt 14 Aug 2001 23:07:17 -0000 1.1 +++ pep-0265.txt 17 Jun 2005 04:30:41 -0000 1.2 @@ -2,7 +2,7 @@ Title: Sorting Dictionaries by Value Version: $Revision$ Author: g2 at iowegian.com (Grant Griffin) -Status: Draft +Status: Rejected Type: Standards Track Created: 8-Aug-2001 Python-Version: 2.2 @@ -17,6 +17,25 @@ both difficult for beginners to understand and cumbersome for all to implement. +BDFL Pronouncement + + This PEP is rejected because the need for it has been largely + fulfilled by Py2.4's sorted() builtin function: + + >>> sorted(d.iteritems(), key=itemgetter(1), reverse=True) + [('b', 23), ('d', 17), ('c', 5), ('a', 2), ('e', 1)] + + or for just the keys: + + sorted(d, key=d.__getitem__, reverse=True) + ['b', 'd', 'c', 'a', 'e'] + + Also, Python 2.5's heapq.nlargest() function addresses the common use + case of finding only a few of the highest valued items: + + >>> nlargest(2, d.iteritems(), itemgetter(1)) + [('b', 23), ('d', 17)] + Motivation From rhettinger at users.sourceforge.net Fri Jun 17 06:57:09 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Thu, 16 Jun 2005 21:57:09 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.317, 1.318 pep-0281.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6881 Modified Files: pep-0000.txt pep-0281.txt Log Message: Record the rejection of PEP 281. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.317 retrieving revision 1.318 diff -u -d -r1.317 -r1.318 --- pep-0000.txt 17 Jun 2005 04:30:41 -0000 1.317 +++ pep-0000.txt 17 Jun 2005 04:57:07 -0000 1.318 @@ -86,7 +86,6 @@ S 275 Switching on Multiple Values Lemburg S 276 Simple Iterator for ints Althoff S 280 Optimizing access to globals GvR - S 281 Loop Counter Iteration with range and xrange Hetland S 284 Integer for-loops Eppstein, Ewing S 286 Enhanced Argument Tuples von Loewis I 287 reStructuredText Docstring Format Goodger @@ -160,6 +159,7 @@ SF 277 Unicode file name support for Windows NT Hodgson SF 278 Universal Newline Support Jansen SF 279 The enumerate() built-in function Hettinger + SR 281 Loop Counter Iteration with range and xrange Hetland SF 282 A Logging System Sajip, Mick IF 283 Python 2.3 Release Schedule GvR SF 285 Adding a bool type GvR @@ -323,7 +323,7 @@ SF 278 Universal Newline Support Jansen SF 279 The enumerate() built-in function Hettinger S 280 Optimizing access to globals GvR - S 281 Loop Counter Iteration with range and xrange Hetland + SR 281 Loop Counter Iteration with range and xrange Hetland SF 282 A Logging System Sajip, Mick IF 283 Python 2.3 Release Schedule GvR S 284 Integer for-loops Eppstein, Ewing Index: pep-0281.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0281.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0281.txt 11 Feb 2002 16:43:16 -0000 1.1 +++ pep-0281.txt 17 Jun 2005 04:57:07 -0000 1.2 @@ -16,6 +16,19 @@ function indices() from PEP 212 [1] be included in the existing functions range() and xrange(). +Pronouncement + + In commenting on PEP 279's enumerate() function, this PEP's author + offered, "I'm quite happy to have it make PEP 281 obsolete." + Subsequently, PEP 279 was accepted into Python 2.3. + + On 17 June 2005, the BDFL concurred with it being obsolete and + hereby rejected the PEP. For the record, he found some of the + examples to somewhat jarring in appearance: + + >>> range(range(5), range(10), range(2)) + [5, 7, 9] + Motivation From rhettinger at users.sourceforge.net Fri Jun 17 09:09:22 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 00:09:22 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.318, 1.319 pep-0276.txt, 1.4, 1.5 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2456 Modified Files: pep-0000.txt pep-0276.txt Log Message: Record the rejection of PEP 276. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.318 retrieving revision 1.319 diff -u -d -r1.318 -r1.319 --- pep-0000.txt 17 Jun 2005 04:57:07 -0000 1.318 +++ pep-0000.txt 17 Jun 2005 07:09:04 -0000 1.319 @@ -84,7 +84,6 @@ S 267 Optimized Access to Module Namespaces Hylton S 268 Extended HTTP functionality and WebDAV Stein S 275 Switching on Multiple Values Lemburg - S 276 Simple Iterator for ints Althoff S 280 Optimizing access to globals GvR S 284 Integer for-loops Eppstein, Ewing S 286 Enhanced Argument Tuples von Loewis @@ -204,6 +203,7 @@ SR 270 uniq method for list objects Petrone SR 271 Prefixing sys.path by command line option Giacometti SR 274 Dict Comprehensions Warsaw + SR 276 Simple Iterator for ints Althoff SR 288 Generators Attributes and Exceptions Hettinger SR 295 Interpretation of multiline string constants Koltsov SR 296 Adding a bytes Object Type Gilbert @@ -318,7 +318,7 @@ SF 273 Import Modules from Zip Archives Ahlstrom SR 274 Dict Comprehensions Warsaw S 275 Switching on Multiple Values Lemburg - S 276 Simple Iterator for ints Althoff + SR 276 Simple Iterator for ints Althoff SF 277 Unicode file name support for Windows NT Hodgson SF 278 Universal Newline Support Jansen SF 279 The enumerate() built-in function Hettinger Index: pep-0276.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0276.txt,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- pep-0276.txt 5 Apr 2002 19:42:56 -0000 1.4 +++ pep-0276.txt 17 Jun 2005 07:09:04 -0000 1.5 @@ -22,6 +22,24 @@ builtin type int (types.IntType). Such an iterator would simplify the coding of certain for-loops in Python. +BDFL Pronouncement + + This PEP was rejected on 17 June 2005 with a note to python-dev. + + Much of the original need was met by the enumerate() function which + was accepted for Python 2.3. + + Also, the proposal both allowed and encouraged misuses such as: + + >>> for i in 3: print i + 0 + 1 + 2 + + Likewise, it was not helpful that the proposal would disable the + syntax error in statements like: + + x, = 1 Specification From rhettinger at users.sourceforge.net Fri Jun 17 10:40:52 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 01:40:52 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt,1.319,1.320 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15505 Modified Files: pep-0000.txt Log Message: Move a deferred PEP to the correct section. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.319 retrieving revision 1.320 diff -u -d -r1.319 -r1.320 --- pep-0000.txt 17 Jun 2005 07:09:04 -0000 1.319 +++ pep-0000.txt 17 Jun 2005 08:40:50 -0000 1.320 @@ -79,7 +79,6 @@ S 254 Making Classes Look More Like Types GvR S 256 Docstring Processing System Framework Goodger S 258 Docutils Design Specification Goodger - SD 262 Database of Installed Python Packages Kuchling S 266 Optimizing Global Variable/Attribute Access Montanaro S 267 Optimized Access to Module Namespaces Hylton S 268 Extended HTTP functionality and WebDAV Stein @@ -198,6 +197,7 @@ SR 242 Numeric Kinds Dubois SR 244 The `directive' Statement von Loewis SR 259 Omit printing newline after newline GvR + SD 262 Database of Installed Python Packages Kuchling SR 265 Sorting Dictionaries by Value Griffin SD 269 Pgen Module for Python Riehl SR 270 uniq method for list objects Petrone From rhettinger at users.sourceforge.net Fri Jun 17 12:25:36 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 03:25:36 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut tut.tex,1.271,1.272 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2194 Modified Files: tut.tex Log Message: Fix typo. Index: tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.271 retrieving revision 1.272 diff -u -d -r1.271 -r1.272 --- tut.tex 14 Jun 2005 08:57:28 -0000 1.271 +++ tut.tex 17 Jun 2005 10:25:33 -0000 1.272 @@ -2024,7 +2024,7 @@ ((12345, 54321, 'hello!'), (1, 2, 3, 4, 5)) \end{verbatim} -As you see, on output tuples are alway enclosed in parentheses, so +As you see, on output tuples are always enclosed in parentheses, so that nested tuples are interpreted correctly; they may be input with or without surrounding parentheses, although often parentheses are necessary anyway (if the tuple is part of a larger expression). From rhettinger at users.sourceforge.net Fri Jun 17 12:27:52 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 03:27:52 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut tut.tex, 1.261.2.6, 1.261.2.7 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3391 Modified Files: Tag: release24-maint tut.tex Log Message: Fix typo. Index: tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.261.2.6 retrieving revision 1.261.2.7 diff -u -d -r1.261.2.6 -r1.261.2.7 --- tut.tex 14 Jun 2005 08:58:27 -0000 1.261.2.6 +++ tut.tex 17 Jun 2005 10:27:50 -0000 1.261.2.7 @@ -2024,7 +2024,7 @@ ((12345, 54321, 'hello!'), (1, 2, 3, 4, 5)) \end{verbatim} -As you see, on output tuples are alway enclosed in parentheses, so +As you see, on output tuples are always enclosed in parentheses, so that nested tuples are interpreted correctly; they may be input with or without surrounding parentheses, although often parentheses are necessary anyway (if the tuple is part of a larger expression). From rhettinger at users.sourceforge.net Fri Jun 17 19:33:20 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 10:33:20 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.320, 1.321 pep-0313.txt, 1.4, 1.5 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv853 Modified Files: pep-0000.txt pep-0313.txt Log Message: Record the rejection for PEP 313. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.320 retrieving revision 1.321 diff -u -d -r1.320 -r1.321 --- pep-0000.txt 17 Jun 2005 08:40:50 -0000 1.320 +++ pep-0000.txt 17 Jun 2005 17:33:17 -0000 1.321 @@ -96,7 +96,6 @@ S 304 Controlling Generation of Bytecode Files Montanaro S 310 Reliable Acquisition/Release Pairs Hudson, Moore S 312 Simple Implicit Lambda Suzi, Martelli - S 313 Adding Roman Numeral Literals to Python Meyer S 314 Metadata for Python Software Packages v1.1 Kuchling S 315 Enhanced While Loop Carroll S 319 Python Synchronize/Asynchronize Block Pelletier @@ -208,6 +207,7 @@ SR 295 Interpretation of multiline string constants Koltsov SR 296 Adding a bytes Object Type Gilbert SR 308 If-then-else expression GvR, Hettinger + SR 313 Adding Roman Numeral Literals to Python Meyer SD 316 Programming by Contract for Python Way SR 317 Eliminate Implicit Exception Instantiation Taschuk SR 326 A Case for Top and Bottom Values Carlson, Reedy @@ -354,7 +354,7 @@ S 310 Reliable Acquisition/Release Pairs Hudson, Moore SF 311 Simplified GIL Acquisition for Extensions Hammond S 312 Simple Implicit Lambda Suzi, Martelli - S 313 Adding Roman Numeral Literals to Python Meyer + SR 313 Adding Roman Numeral Literals to Python Meyer S 314 Metadata for Python Software Packages v1.1 Kuchling S 315 Enhanced While Loop Carroll SD 316 Programming by Contract for Python Way Index: pep-0313.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0313.txt,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- pep-0313.txt 4 Apr 2003 21:21:38 -0000 1.4 +++ pep-0313.txt 17 Jun 2005 17:33:18 -0000 1.5 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Mike Meyer -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/plain Created: 01-Apr-2003 @@ -19,6 +19,17 @@ converts the integer to a string that is the Roman numeral literal equivalent to the integer. +BDFL Pronouncement + + This PEP is rejected. While the majority of Python users deemed this + to be a nice-to-have feature, the community was unable to reach a + concensus on whether nine should be represented as IX, the modern + form, or VIIII, the classic form. Likewise, no agreement was + reached on whether MXM or MCMXC would be considered a well-formed + representation of 1990. A vocal minority of users has also requested + support for lower-cased numerals for use in (i) powerpoint slides, + (ii) academic work, and (iii) Perl documentation. + Rationale From rhettinger at users.sourceforge.net Fri Jun 17 19:43:02 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 10:43:02 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.321, 1.322 pep-0336.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6282 Modified Files: pep-0000.txt pep-0336.txt Log Message: Record the rejection of PEP 336 Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.321 retrieving revision 1.322 diff -u -d -r1.321 -r1.322 --- pep-0000.txt 17 Jun 2005 17:33:17 -0000 1.321 +++ pep-0000.txt 17 Jun 2005 17:43:00 -0000 1.322 @@ -107,7 +107,6 @@ S 332 Byte vectors and String/Unicode Unification Montanaro S 334 Simple Coroutines via SuspendIteration Evans S 335 Overloadable Boolean Operators Ewing - S 336 Make None Callable McClelland S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan S 341 Unifying try-except and try-finally Birkenfeld @@ -212,6 +211,7 @@ SR 317 Eliminate Implicit Exception Instantiation Taschuk SR 326 A Case for Top and Bottom Values Carlson, Reedy SR 329 Treating Builtins as Constants in the Standard Library Hettinger + SR 336 Make None Callable McClelland SR 340 Anonymous Block Statements GvR SR 346 User Defined ("with") Statements Coghlan SR 666 Reject Foolish Indentation Creighton @@ -377,7 +377,7 @@ I 333 Python Web Server Gateway Interface v1.0 Eby S 334 Simple Coroutines via SuspendIteration Evans S 335 Overloadable Boolean Operators Ewing - S 336 Make None Callable McClelland + SR 336 Make None Callable McClelland S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan I 339 How to Change CPython's Bytecode Cannon Index: pep-0336.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0336.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0336.txt 3 Nov 2004 16:58:30 -0000 1.1 +++ pep-0336.txt 17 Jun 2005 17:43:00 -0000 1.2 @@ -15,6 +15,14 @@ None should be a callable object that when called with any arguments has no side effect and returns None. +BDFL Pronouncement + + This PEP is rejected. It is considered a feature that None raises + an error when called. The proposal falls short in tests for + obviousness, clarity, explictness, and necessity. The provided Switch + example is nice but easily handled by a simple lambda definition. + See python-dev discussion on 17 June 2005. + Motivation From rhettinger at users.sourceforge.net Fri Jun 17 20:27:17 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 11:27:17 -0700 Subject: [Python-checkins] python/nondist/peps pep-0336.txt,1.2,1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv27825 Modified Files: pep-0336.txt Log Message: Record the rejection of PEP 336 Index: pep-0336.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0336.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0336.txt 17 Jun 2005 17:43:00 -0000 1.2 +++ pep-0336.txt 17 Jun 2005 18:27:15 -0000 1.3 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Andrew McClelland -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/plain Created: 28-Oct-2004 From rhettinger at users.sourceforge.net Fri Jun 17 20:28:02 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 11:28:02 -0700 Subject: [Python-checkins] python/nondist/peps pep-0276.txt,1.5,1.6 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28095 Modified Files: pep-0276.txt Log Message: Record the rejection of PEP 276 Index: pep-0276.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0276.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- pep-0276.txt 17 Jun 2005 07:09:04 -0000 1.5 +++ pep-0276.txt 17 Jun 2005 18:28:00 -0000 1.6 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: james_althoff at i2.com (Jim Althoff) -Status: Active +Status: Rejected Type: Standards Track Created: 12-Nov-2001 Python-Version: 2.3 From rhettinger at users.sourceforge.net Fri Jun 17 20:28:51 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 11:28:51 -0700 Subject: [Python-checkins] python/nondist/peps pep-0281.txt,1.2,1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv28517 Modified Files: pep-0281.txt Log Message: Record the rejection of PEP 281 Index: pep-0281.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0281.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0281.txt 17 Jun 2005 04:57:07 -0000 1.2 +++ pep-0281.txt 17 Jun 2005 18:28:49 -0000 1.3 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: magnus at hetland.org (Magnus Lie Hetland) -Status: Draft +Status: Rejected Type: Standards Track Created: 11-Feb-2002 Python-Version: 2.3 From rhettinger at users.sourceforge.net Fri Jun 17 22:19:28 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 13:19:28 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.322, 1.323 pep-0303.txt, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17764 Modified Files: pep-0000.txt pep-0303.txt Log Message: Reject PEP 303. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.322 retrieving revision 1.323 diff -u -d -r1.322 -r1.323 --- pep-0000.txt 17 Jun 2005 17:43:00 -0000 1.322 +++ pep-0000.txt 17 Jun 2005 20:19:25 -0000 1.323 @@ -92,7 +92,6 @@ S 298 The Locked Buffer Interface Heller S 299 Special __main__() function in modules Epler S 302 New Import Hooks JvR - S 303 Extend divmod() for Multiple Divisors Bellman S 304 Controlling Generation of Bytecode Files Montanaro S 310 Reliable Acquisition/Release Pairs Hudson, Moore S 312 Simple Implicit Lambda Suzi, Martelli @@ -205,6 +204,7 @@ SR 288 Generators Attributes and Exceptions Hettinger SR 295 Interpretation of multiline string constants Koltsov SR 296 Adding a bytes Object Type Gilbert + SR 303 Extend divmod() for Multiple Divisors Bellman SR 308 If-then-else expression GvR, Hettinger SR 313 Adding Roman Numeral Literals to Python Meyer SD 316 Programming by Contract for Python Way @@ -344,7 +344,7 @@ S 299 Special __main__() function in modules Epler SF 301 Package Index and Metadata for Distutils Jones S 302 New Import Hooks JvR - S 303 Extend divmod() for Multiple Divisors Bellman + SR 303 Extend divmod() for Multiple Divisors Bellman S 304 Controlling Generation of Bytecode Files Montanaro SF 305 CSV File API Montanaro, et al I 306 How to Change Python's Grammar Hudson Index: pep-0303.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0303.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0303.txt 31 Dec 2002 16:02:49 -0000 1.2 +++ pep-0303.txt 17 Jun 2005 20:19:25 -0000 1.3 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Thomas Bellman -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/plain Created: 31-Dec-2002 @@ -17,6 +17,21 @@ allowing it to take multiple divisors, chaining several calls to divmod() into one. +Pronouncement + + This PEP is rejected. Most uses for chained divmod() involve a + constant modulus (in radix conversions for example) and are more + properly coded as a loop. The example of splitting seconds + into days/hours/minutes/seconds does not generalize to months + and years; rather, the whole use case is handled more flexibly and + robustly by date and time modules. The other use cases mentioned + in the PEP are somewhat rare in real code. The proposal is also + problematic in terms of clarity and obviousness. In the examples, + it is not immediately clear that the argument order is correct or + that the target tuple is of the right length. Users from other + languages are more likely to understand the standard two argument + form without having to re-read the documentation. See python-dev + discussion on 17 June 2005. Specification From rhettinger at users.sourceforge.net Fri Jun 17 23:38:05 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 14:38:05 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.323, 1.324 pep-0239.txt, 1.7, 1.8 pep-0240.txt, 1.7, 1.8 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12350 Modified Files: pep-0000.txt pep-0239.txt pep-0240.txt Log Message: Mark PEPs 239 and 240 as rejected. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.323 retrieving revision 1.324 diff -u -d -r1.323 -r1.324 --- pep-0000.txt 17 Jun 2005 20:19:25 -0000 1.323 +++ pep-0000.txt 17 Jun 2005 21:38:02 -0000 1.324 @@ -71,8 +71,6 @@ S 209 Adding Multidimensional Arrays Barrett, Oliphant S 228 Reworking Python's Numeric Model Zadka, GvR S 237 Unifying Long Integers and Integers Zadka, GvR - S 239 Adding a Rational Type to Python Craig, Zadka - S 240 Adding a Rational Literal to Python Craig, Zadka S 243 Module Repository Upload Mechanism Reifschneider S 245 Python Interface Syntax Pelletier S 246 Object Adaptation Evans @@ -191,6 +189,8 @@ SD 225 Elementwise/Objectwise Operators Zhu, Lielens SR 231 __findattr__() Warsaw SD 233 Python Online Help Prescod + SR 239 Adding a Rational Type to Python Craig, Zadka + SR 240 Adding a Rational Literal to Python Craig, Zadka SR 242 Numeric Kinds Dubois SR 244 The `directive' Statement von Loewis SR 259 Omit printing newline after newline GvR @@ -281,8 +281,8 @@ SF 236 Back to the __future__ Peters S 237 Unifying Long Integers and Integers Zadka, GvR SF 238 Changing the Division Operator Zadka, GvR - S 239 Adding a Rational Type to Python Craig, Zadka - S 240 Adding a Rational Literal to Python Craig, Zadka + SR 239 Adding a Rational Type to Python Craig, Zadka + SR 240 Adding a Rational Literal to Python Craig, Zadka SF 241 Metadata for Python Software Packages Kuchling SR 242 Numeric Kinds Dubois S 243 Module Repository Upload Mechanism Reifschneider Index: pep-0239.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0239.txt,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- pep-0239.txt 20 Sep 2002 14:10:51 -0000 1.7 +++ pep-0239.txt 17 Jun 2005 21:38:02 -0000 1.8 @@ -3,7 +3,7 @@ Version: $Revision$ Author: Christopher A. Craig , Moshe Zadka -Status: Draft +Status: Rejected Type: Standards Track Created: 11-Mar-2001 Python-Version: 2.2 @@ -18,6 +18,13 @@ support such a type. This PEP suggests no literals for rational numbers; that is left for another PEP[1]. +BDFL Pronouncement + + This PEP is rejected. The needs outlined in the rationale section + have been addressed to some extent by the acceptance of PEP 327 + for decimal arithmetic. Guido also noted, "Rational arithmetic + was the default 'exact' arithmetic in ABC and it did not work out as + expected". See the python-dev discussion on 17 June 2005. Rationale Index: pep-0240.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0240.txt,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- pep-0240.txt 18 Jan 2003 00:59:04 -0000 1.7 +++ pep-0240.txt 17 Jun 2005 21:38:02 -0000 1.8 @@ -3,7 +3,7 @@ Version: $Revision$ Author: Christopher A. Craig , Moshe Zadka -Status: Draft +Status: Rejected Type: Standards Track Created: 11-Mar-2001 Python-Version: 2.2 @@ -17,6 +17,13 @@ rational in Python, and modifying non-integer division to return it. +BDFL Pronouncement + + This PEP is rejected. The needs outlined in the rationale section + have been addressed to some extent by the acceptance of PEP 327 + for decimal arithmetic. Guido also noted that "Rational arithmetic + was the default 'exact' arithmetic in ABC and it did not work out as + expected. See the python-dev discussion on 17 June 2005. Rationale From rhettinger at users.sourceforge.net Fri Jun 17 23:39:21 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 14:39:21 -0700 Subject: [Python-checkins] python/nondist/peps pep-0240.txt,1.8,1.9 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13103 Modified Files: pep-0240.txt Log Message: Fix quotation. Index: pep-0240.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0240.txt,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- pep-0240.txt 17 Jun 2005 21:38:02 -0000 1.8 +++ pep-0240.txt 17 Jun 2005 21:39:18 -0000 1.9 @@ -21,9 +21,9 @@ This PEP is rejected. The needs outlined in the rationale section have been addressed to some extent by the acceptance of PEP 327 - for decimal arithmetic. Guido also noted that "Rational arithmetic + for decimal arithmetic. Guido also noted, "Rational arithmetic was the default 'exact' arithmetic in ABC and it did not work out as - expected. See the python-dev discussion on 17 June 2005. + expected". See the python-dev discussion on 17 June 2005. Rationale From rhettinger at users.sourceforge.net Fri Jun 17 23:59:12 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 14:59:12 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.24,1.25 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23023 Modified Files: pep-0343.txt Log Message: Optional arguments for throw() to match raise syntax Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.24 retrieving revision 1.25 diff -u -d -r1.24 -r1.25 --- pep-0343.txt 11 Jun 2005 05:14:20 -0000 1.24 +++ pep-0343.txt 17 Jun 2005 21:58:59 -0000 1.25 @@ -167,10 +167,10 @@ log it; the generator is not allowed to yield another value, since the with-statement should not be usable as a loop (raising a different exception is marginally acceptable). To enable this, a - new throw() method for generators is proposed, which takes three - arguments representing an exception in the usual fashion (type, - value, traceback) and raises it at the point where the generator - is suspended. + new throw() method for generators is proposed, which takes one to + three arguments representing an exception in the usual fashion + (type, value, traceback) and raises it at the point where the + generator is suspended. Once we have this, it is a small step to proposing another generator method, close(), which calls throw() with a special @@ -273,7 +273,7 @@ The syntax for generator functions is extended to allow a yield-statement inside a try-finally statement. - New generator method: throw(type, value, traceback) + New generator method: throw(type, value=None, traceback=None) g.throw(type, value, traceback) causes the specified exception to be thrown at the point where the generator g is currently From rhettinger at users.sourceforge.net Sat Jun 18 00:19:58 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 15:19:58 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.324, 1.325 pep-0312.txt, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1661 Modified Files: pep-0000.txt pep-0312.txt Log Message: Mark PEP 312 as Deferred -- it needs a more Pythonic syntax) Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.324 retrieving revision 1.325 diff -u -d -r1.324 -r1.325 --- pep-0000.txt 17 Jun 2005 21:38:02 -0000 1.324 +++ pep-0000.txt 17 Jun 2005 22:19:41 -0000 1.325 @@ -92,7 +92,6 @@ S 302 New Import Hooks JvR S 304 Controlling Generation of Bytecode Files Montanaro S 310 Reliable Acquisition/Release Pairs Hudson, Moore - S 312 Simple Implicit Lambda Suzi, Martelli S 314 Metadata for Python Software Packages v1.1 Kuchling S 315 Enhanced While Loop Carroll S 319 Python Synchronize/Asynchronize Block Pelletier @@ -206,6 +205,7 @@ SR 296 Adding a bytes Object Type Gilbert SR 303 Extend divmod() for Multiple Divisors Bellman SR 308 If-then-else expression GvR, Hettinger + SD 312 Simple Implicit Lambda Suzi, Martelli SR 313 Adding Roman Numeral Literals to Python Meyer SD 316 Programming by Contract for Python Way SR 317 Eliminate Implicit Exception Instantiation Taschuk @@ -353,7 +353,7 @@ SF 309 Partial Function Application Harris S 310 Reliable Acquisition/Release Pairs Hudson, Moore SF 311 Simplified GIL Acquisition for Extensions Hammond - S 312 Simple Implicit Lambda Suzi, Martelli + SD 312 Simple Implicit Lambda Suzi, Martelli SR 313 Adding Roman Numeral Literals to Python Meyer S 314 Metadata for Python Software Packages v1.1 Kuchling S 315 Enhanced While Loop Carroll Index: pep-0312.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0312.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0312.txt 14 Feb 2003 14:51:27 -0000 1.2 +++ pep-0312.txt 17 Jun 2005 22:19:41 -0000 1.3 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Roman Suzi , Alex Martelli -Status: Draft +Status: Deferred Type: Standards Track Content-Type: text/plain Created: 11-Feb-2003 @@ -16,6 +16,18 @@ This PEP proposes to make argumentless lambda keyword optional in some cases where it is not grammatically ambiguous. +Deferral + + The BDFL hates the unary colon syntax. This PEP needs to go back + to the drawing board and find a more Pythonic syntax (perhaps an + alternative unary operator). See python-dev discussion on + 17 June 2005. + + Also, it is probably a good idea to eliminate the alternative + propositions which have no chance at all. The examples section + is good and highlights the readability improvements. It would + carry more weight with additional examples and with real-world + referrents (instead of the abstracted dummy calls to :A and :B). Motivation From rhettinger at users.sourceforge.net Sat Jun 18 08:21:39 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Fri, 17 Jun 2005 23:21:39 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.325, 1.326 pep-0315.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16944 Modified Files: pep-0000.txt pep-0315.txt Log Message: Pick-up responsibility for PEP 315 (per agreement with Isaac Carroll). Correct the semantics for continue statements found in the do suite. The previous semantics did not match that for do-while statements in other languages. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.325 retrieving revision 1.326 diff -u -d -r1.325 -r1.326 --- pep-0000.txt 17 Jun 2005 22:19:41 -0000 1.325 +++ pep-0000.txt 18 Jun 2005 06:21:36 -0000 1.326 @@ -93,7 +93,7 @@ S 304 Controlling Generation of Bytecode Files Montanaro S 310 Reliable Acquisition/Release Pairs Hudson, Moore S 314 Metadata for Python Software Packages v1.1 Kuchling - S 315 Enhanced While Loop Carroll + S 315 Enhanced While Loop Carroll, Hettinger S 319 Python Synchronize/Asynchronize Block Pelletier S 321 Date/Time Parsing and Formatting Kuchling S 323 Copyable Iterators Martelli @@ -356,7 +356,7 @@ SD 312 Simple Implicit Lambda Suzi, Martelli SR 313 Adding Roman Numeral Literals to Python Meyer S 314 Metadata for Python Software Packages v1.1 Kuchling - S 315 Enhanced While Loop Carroll + S 315 Enhanced While Loop Carroll, Hettinger SD 316 Programming by Contract for Python Way SR 317 Eliminate Implicit Exception Instantiation Taschuk SF 318 Decorators for Functions and Methods Smith, et al Index: pep-0315.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0315.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0315.txt 2 May 2003 22:53:32 -0000 1.1 +++ pep-0315.txt 18 Jun 2005 06:21:36 -0000 1.2 @@ -2,12 +2,13 @@ Title: Enhanced While Loop Version: $Revision$ Last-Modified: $Date$ -Author: W Isaac Carroll +Author: W Isaac Carroll + Raymond Hettinger Status: Draft Type: Standards Track Content-Type: text/plain Created: 25-Apr-2003 -Python-Version: 2.4 +Python-Version: 2.5 Post-History: @@ -90,11 +91,16 @@ without evaluating the loop condition or executing the else clause. - A continue statement in the do-while loop will behave somewhat - differently than in the standard while loop. Instead of jumping - back to the loop condition, it will jump to the beginning of the - first suite of the loop. This is to ensure that the setup code - has a chance to do its job before the condition is evaluated. + A continue statement in the do-while loop jumps to the while + condition check. + + In general, when the while suite is empty (a pass statement), + the do-while loop and break and continue statements should match + the semantics of do-while in other languages. + + Likewise, when the do suite is empty, the do-while loop and + break and continue statements should match behavior found + in regular while loops. Future Statement From montanaro at users.sourceforge.net Sat Jun 18 15:04:06 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Sat, 18 Jun 2005 06:04:06 -0700 Subject: [Python-checkins] python/nondist/peps pep-0304.txt,1.12,1.13 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv12312 Modified Files: pep-0304.txt Log Message: Minor tweakage. Been sitting in my sandbox for awhile. Index: pep-0304.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0304.txt,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- pep-0304.txt 29 May 2003 13:46:46 -0000 1.12 +++ pep-0304.txt 18 Jun 2005 13:04:03 -0000 1.13 @@ -34,8 +34,8 @@ - If not defined, Python bytecode is generated in exactly the same way as is currently done. sys.bytecodebase is set to the root directory - (either / on Unix or the root directory of the startup drive -- - typically ``C:\`` -- on Windows). + (either / on Unix and Mac OSX or the root directory of the startup + (installation???) drive -- typically ``C:\`` -- on Windows). - If defined and it refers to an existing directory to which the user has write permission, sys.bytecodebase is set to that directory and @@ -76,6 +76,9 @@ - If no byte-compiled file is found, an attempt to read a byte-compiled file from the augmented directory is made. +- If bytecode generation is required, the generated bytecode is wrtten + to the augmented directory if possible. + Note that this PEP is explicitly *not* about providing module-by-module or directory-by-directory control over the disposition of bytecode files. @@ -146,22 +149,22 @@ to the sys module, effectively:: pcb = os.path.abspath(os.environ["PYTHONBYTECODEBASE"]) + probe = os.path.join(pcb, "foo") try: - probe = os.path.join(pcb, "foo") open(probe, "w") - os.unlink(probe) - sys.bytecodebase = pcb except IOError: sys.bytecodebase = None + else: + os.unlink(probe) + sys.bytecodebase = pcb This allows the user to specify the bytecode base as a relative path, -but not have it subject to changes to the current working directory. -(I can't imagine you'd want it to move around during program -execution.) +but not have it subject to changes to the current working directory +during program execution. (I can't imagine you'd want it to move +around during program execution.) -There is nothing special about sys.bytecodebase. The user may -change it at runtime if she so chooses, but normally it will not be -modified. +There is nothing special about sys.bytecodebase. The user may change +it at runtime if desired, but normally it will not be modified. Rationale @@ -194,7 +197,7 @@ altogether. This proposal subsumes that. Adding a command-line option is certainly possible, but is probably not sufficient, as the interpreter's command line is not readily available during -installation. +installation (early during program startup???). Issues @@ -208,12 +211,11 @@ - Security - What if root has PYTHONBYTECODEBASE set? Yes, this can present a security risk, but so can many other things the root user does. The root user should probably not set PYTHONBYTECODEBASE - except during installation. Still, perhaps this problem can be - minimized. When running as root the interpreter should check to see - if PYTHONBYTECODEBASE refers to a directory which is writable by - anyone other than root. If so, it could raise an exception or - warning and set sys.bytecodebase to None. Or, see the next - item. + except possibly during installation. Still, perhaps this problem + can be minimized. When running as root the interpreter should check + to see if PYTHONBYTECODEBASE refers to a directory which is writable + by anyone other than root. If so, it could raise an exception or + warning and set sys.bytecodebase to None. Or, see the next item. - More security - What if PYTHONBYTECODEBASE refers to a general directory (say, /tmp)? In this case, perhaps loading of a From nascheme at users.sourceforge.net Sat Jun 18 19:37:09 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Sat, 18 Jun 2005 10:37:09 -0700 Subject: [Python-checkins] python/dist/src/Modules gcmodule.c,2.80,2.81 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14221/Modules Modified Files: gcmodule.c Log Message: Add missing INCREF. Backport candidate. Index: gcmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/gcmodule.c,v retrieving revision 2.80 retrieving revision 2.81 diff -u -d -r2.80 -r2.81 --- gcmodule.c 1 Nov 2004 16:39:57 -0000 2.80 +++ gcmodule.c 18 Jun 2005 17:37:06 -0000 2.81 @@ -1166,6 +1166,7 @@ if (garbage == NULL) return; } + Py_INCREF(garbage); if (PyModule_AddObject(m, "garbage", garbage) < 0) return; #define ADD_INT(NAME) if (PyModule_AddIntConstant(m, #NAME, NAME) < 0) return From nascheme at users.sourceforge.net Sat Jun 18 19:54:15 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Sat, 18 Jun 2005 10:54:15 -0700 Subject: [Python-checkins] python/dist/src/Doc/api concrete.tex, 1.60, 1.61 newtypes.tex, 1.35, 1.36 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/api In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22799/Doc/api Modified Files: concrete.tex newtypes.tex Log Message: Fix grammar (it's -> its). Index: concrete.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/api/concrete.tex,v retrieving revision 1.60 retrieving revision 1.61 diff -u -d -r1.60 -r1.61 --- concrete.tex 17 Feb 2005 05:17:17 -0000 1.60 +++ concrete.tex 18 Jun 2005 17:54:13 -0000 1.61 @@ -2200,7 +2200,7 @@ \begin{cfuncdesc}{PyObject*}{PyInstance_NewRaw}{PyObject *class, PyObject *dict} - Create a new instance of a specific class without calling it's + Create a new instance of a specific class without calling its constructor. \var{class} is the class of new object. The \var{dict} parameter will be used as the object's \member{__dict__}; if \NULL{}, a new dictionary will be created for the instance. Index: newtypes.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/api/newtypes.tex,v retrieving revision 1.35 retrieving revision 1.36 diff -u -d -r1.35 -r1.36 --- newtypes.tex 24 Mar 2005 07:45:53 -0000 1.35 +++ newtypes.tex 18 Jun 2005 17:54:13 -0000 1.36 @@ -191,7 +191,7 @@ int ob_size; \end{verbatim} Note that \csimplemacro{PyObject_HEAD} is part of the expansion, and - that it's own expansion varies depending on the definition of + that its own expansion varies depending on the definition of \csimplemacro{Py_TRACE_REFS}. \end{csimplemacrodesc} From nascheme at users.sourceforge.net Sat Jun 18 19:54:15 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Sat, 18 Jun 2005 10:54:15 -0700 Subject: [Python-checkins] python/dist/src/Doc/doc doc.tex,1.92,1.93 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/doc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22799/Doc/doc Modified Files: doc.tex Log Message: Fix grammar (it's -> its). Index: doc.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/doc/doc.tex,v retrieving revision 1.92 retrieving revision 1.93 diff -u -d -r1.92 -r1.93 --- doc.tex 3 Mar 2005 23:07:21 -0000 1.92 +++ doc.tex 18 Jun 2005 17:54:13 -0000 1.93 @@ -617,7 +617,7 @@ Documentation for a ``simple'' macro. Simple macros are macros which are used for code expansion, but which do not take arguments so cannot be described as functions. This is not to - be used for simple constant definitions. Examples of it's use + be used for simple constant definitions. Examples of its use in the Python documentation include \csimplemacro{PyObject_HEAD} and \csimplemacro{Py_BEGIN_ALLOW_THREADS}. @@ -812,7 +812,7 @@ The name of a ``simple'' macro. Simple macros are macros which are used for code expansion, but which do not take arguments so cannot be described as functions. This is not to - be used for simple constant definitions. Examples of it's use + be used for simple constant definitions. Examples of its use in the Python documentation include \csimplemacro{PyObject_HEAD} and \csimplemacro{Py_BEGIN_ALLOW_THREADS}. From nascheme at users.sourceforge.net Sat Jun 18 19:54:15 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Sat, 18 Jun 2005 10:54:15 -0700 Subject: [Python-checkins] python/dist/src/Doc/ref ref2.tex,1.57,1.58 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/ref In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22799/Doc/ref Modified Files: ref2.tex Log Message: Fix grammar (it's -> its). Index: ref2.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/ref/ref2.tex,v retrieving revision 1.57 retrieving revision 1.58 diff -u -d -r1.57 -r1.58 --- ref2.tex 25 May 2005 05:29:17 -0000 1.57 +++ ref2.tex 18 Jun 2005 17:54:13 -0000 1.58 @@ -349,7 +349,7 @@ \item[\code{__*__}] System-defined names. These names are defined by the interpreter - and it's implementation (including the standard library); + and its implementation (including the standard library); applications should not expect to define additional names using this convention. The set of names of this class defined by Python may be extended in future versions. From nascheme at users.sourceforge.net Sat Jun 18 19:54:15 2005 From: nascheme at users.sourceforge.net (nascheme@users.sourceforge.net) Date: Sat, 18 Jun 2005 10:54:15 -0700 Subject: [Python-checkins] python/dist/src/Doc/mac libframework.tex, 1.13, 1.14 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/mac In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22799/Doc/mac Modified Files: libframework.tex Log Message: Fix grammar (it's -> its). Index: libframework.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/mac/libframework.tex,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- libframework.tex 13 Feb 2005 22:50:03 -0000 1.13 +++ libframework.tex 18 Jun 2005 17:54:13 -0000 1.14 @@ -29,8 +29,8 @@ instance, uses a different way to enable/disable menus and that plugs right in leaving the rest intact. The weak points of \module{FrameWork} are that it has no abstract command interface (but -that shouldn't be difficult), that it's dialog support is minimal and -that it's control/toolbar support is non-existent. +that shouldn't be difficult), that its dialog support is minimal and +that its control/toolbar support is non-existent. \end{quotation} From birkenfeld at users.sourceforge.net Sat Jun 18 22:06:17 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 18 Jun 2005 13:06:17 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib lib.tex,1.238,1.239 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17016/Doc/lib Modified Files: lib.tex Log Message: Patch #1180012: add documentation for modulefinder Index: lib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/lib.tex,v retrieving revision 1.238 retrieving revision 1.239 diff -u -d -r1.238 -r1.239 --- lib.tex 28 Feb 2005 19:39:23 -0000 1.238 +++ lib.tex 18 Jun 2005 20:06:15 -0000 1.239 @@ -93,6 +93,7 @@ \input{libimp} \input{libzipimport} \input{libpkgutil} +\input{libmodulefinder} \input{libcode} \input{libcodeop} \input{libpprint} From birkenfeld at users.sourceforge.net Sat Jun 18 22:06:33 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 18 Jun 2005 13:06:33 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib lib.tex, 1.231.2.1, 1.231.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17095/Doc/lib Modified Files: Tag: release24-maint lib.tex Log Message: Patch #1180012: add documentation for modulefinder (backport) Index: lib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/lib.tex,v retrieving revision 1.231.2.1 retrieving revision 1.231.2.2 diff -u -d -r1.231.2.1 -r1.231.2.2 --- lib.tex 19 Jan 2005 05:54:36 -0000 1.231.2.1 +++ lib.tex 18 Jun 2005 20:06:30 -0000 1.231.2.2 @@ -93,6 +93,7 @@ \input{libimp} \input{libzipimport} \input{libpkgutil} +\input{libmodulefinder} \input{libcode} \input{libcodeop} \input{libpprint} From birkenfeld at users.sourceforge.net Sat Jun 18 22:11:30 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 18 Jun 2005 13:11:30 -0700 Subject: [Python-checkins] python/dist/src/Doc/dist dist.tex, 1.86.2.4, 1.86.2.5 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/dist In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19557/Doc/dist Modified Files: Tag: release24-maint dist.tex Log Message: Bug #1209560: spurious blank page in dist.pdf (backport) Index: dist.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/dist/dist.tex,v retrieving revision 1.86.2.4 retrieving revision 1.86.2.5 diff -u -d -r1.86.2.4 -r1.86.2.5 --- dist.tex 7 Jun 2005 19:05:06 -0000 1.86.2.4 +++ dist.tex 18 Jun 2005 20:11:26 -0000 1.86.2.5 @@ -25,6 +25,9 @@ \begin{document} \maketitle + +\input{copyright} + \begin{abstract} \noindent This document describes the Python Distribution Utilities From birkenfeld at users.sourceforge.net Sat Jun 18 22:11:43 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 18 Jun 2005 13:11:43 -0700 Subject: [Python-checkins] python/dist/src/Doc/dist dist.tex,1.94,1.95 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/dist In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19611/Doc/dist Modified Files: dist.tex Log Message: Bug #1209560: spurious blank page in dist.pdf Index: dist.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/dist/dist.tex,v retrieving revision 1.94 retrieving revision 1.95 diff -u -d -r1.94 -r1.95 --- dist.tex 7 Jun 2005 18:51:42 -0000 1.94 +++ dist.tex 18 Jun 2005 20:11:40 -0000 1.95 @@ -25,6 +25,9 @@ \begin{document} \maketitle + +\input{copyright} + \begin{abstract} \noindent This document describes the Python Distribution Utilities From rhettinger at users.sourceforge.net Sat Jun 18 22:36:31 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 18 Jun 2005 13:36:31 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.326, 1.327 pep-0284.txt, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32360 Modified Files: pep-0000.txt pep-0284.txt Log Message: Reject pep-0284.txt. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.326 retrieving revision 1.327 diff -u -d -r1.326 -r1.327 --- pep-0000.txt 18 Jun 2005 06:21:36 -0000 1.326 +++ pep-0000.txt 18 Jun 2005 20:36:28 -0000 1.327 @@ -82,7 +82,6 @@ S 268 Extended HTTP functionality and WebDAV Stein S 275 Switching on Multiple Values Lemburg S 280 Optimizing access to globals GvR - S 284 Integer for-loops Eppstein, Ewing S 286 Enhanced Argument Tuples von Loewis I 287 reStructuredText Docstring Format Goodger S 294 Type Names in the types Module Tirosh @@ -200,6 +199,7 @@ SR 271 Prefixing sys.path by command line option Giacometti SR 274 Dict Comprehensions Warsaw SR 276 Simple Iterator for ints Althoff + SR 284 Integer for-loops Eppstein, Ewing SR 288 Generators Attributes and Exceptions Hettinger SR 295 Interpretation of multiline string constants Koltsov SR 296 Adding a bytes Object Type Gilbert @@ -326,7 +326,7 @@ SR 281 Loop Counter Iteration with range and xrange Hetland SF 282 A Logging System Sajip, Mick IF 283 Python 2.3 Release Schedule GvR - S 284 Integer for-loops Eppstein, Ewing + SR 284 Integer for-loops Eppstein, Ewing SF 285 Adding a bool type GvR S 286 Enhanced Argument Tuples von Loewis I 287 reStructuredText Docstring Format Goodger Index: pep-0284.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0284.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0284.txt 6 Mar 2002 20:53:47 -0000 1.2 +++ pep-0284.txt 18 Jun 2005 20:36:28 -0000 1.3 @@ -27,6 +27,17 @@ values of var that make the comparison true, starting from the left endpoint of the given interval. +Pronouncement + + This PEP is rejected. There were a number of fixable issues with + the proposal (see the fixups listed in Raymond Hettinger's + python-dev post on 18 June 2005). However, even with the fixups the + proposal did not garner support. Specifically, Guido did not buy + the premise that the range() format needed fixing, "The whole point + (15 years ago) of range() was to *avoid* needing syntax to specify a + loop over numbers. I think it's worked out well and there's nothing + that needs to be fixed (except range() needs to become an interator, + which it will in Python 3.0)." Rationale From rhettinger at users.sourceforge.net Sat Jun 18 23:00:28 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 18 Jun 2005 14:00:28 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_set.py,1.16,1.17 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8799/Lib/test Modified Files: test_set.py Log Message: SF patch #1200018: Restore GC support to set objects Reverts 1.26 and 1.27. And adds cycle testing. Index: test_set.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_set.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- test_set.py 9 Nov 2004 07:25:31 -0000 1.16 +++ test_set.py 18 Jun 2005 21:00:26 -0000 1.17 @@ -202,6 +202,16 @@ self.assertNotEqual(id(t), id(newt)) self.assertEqual(t.value + 1, newt.value) + def test_gc(self): + # Create a nest of cycles to exercise overall ref count check + class A: + pass + s = set(A() for i in xrange(1000)) + for elem in s: + elem.cycle = s + elem.sub = elem + elem.set = set([elem]) + class TestSet(TestJointOps): thetype = set From rhettinger at users.sourceforge.net Sat Jun 18 23:00:43 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 18 Jun 2005 14:00:43 -0700 Subject: [Python-checkins] python/dist/src/Objects setobject.c,1.31,1.32 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8799/Objects Modified Files: setobject.c Log Message: SF patch #1200018: Restore GC support to set objects Reverts 1.26 and 1.27. And adds cycle testing. Index: setobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/setobject.c,v retrieving revision 1.31 retrieving revision 1.32 diff -u -d -r1.31 -r1.32 --- setobject.c 9 Nov 2004 07:25:28 -0000 1.31 +++ setobject.c 18 Jun 2005 21:00:25 -0000 1.32 @@ -114,12 +114,21 @@ static void set_dealloc(PySetObject *so) { + PyObject_GC_UnTrack(so); if (so->weakreflist != NULL) PyObject_ClearWeakRefs((PyObject *) so); Py_XDECREF(so->data); so->ob_type->tp_free(so); } +static int +set_traverse(PySetObject *so, visitproc visit, void *arg) +{ + if (so->data) + return visit(so->data, arg); + return 0; +} + static PyObject * set_iter(PySetObject *so) { @@ -757,6 +766,14 @@ PyDoc_STRVAR(clear_doc, "Remove all elements from this set."); +static int +set_tp_clear(PySetObject *so) +{ + PyDict_Clear(so->data); + so->hash = -1; + return 0; +} + static PyObject * set_add(PySetObject *so, PyObject *item) { @@ -1007,11 +1024,11 @@ PyObject_GenericGetAttr, /* tp_getattro */ 0, /* tp_setattro */ 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_CHECKTYPES | + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_CHECKTYPES | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_WEAKREFS, /* tp_flags */ set_doc, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ + (traverseproc)set_traverse, /* tp_traverse */ + (inquiry)set_tp_clear, /* tp_clear */ (richcmpfunc)set_richcompare, /* tp_richcompare */ offsetof(PySetObject, weakreflist), /* tp_weaklistoffset */ (getiterfunc)set_iter, /* tp_iter */ @@ -1027,7 +1044,7 @@ (initproc)set_init, /* tp_init */ PyType_GenericAlloc, /* tp_alloc */ set_new, /* tp_new */ - PyObject_Del, /* tp_free */ + PyObject_GC_Del, /* tp_free */ }; /* frozenset object ********************************************************/ @@ -1102,10 +1119,10 @@ PyObject_GenericGetAttr, /* tp_getattro */ 0, /* tp_setattro */ 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_CHECKTYPES | + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_CHECKTYPES | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_WEAKREFS, /* tp_flags */ frozenset_doc, /* tp_doc */ - 0, /* tp_traverse */ + (traverseproc)set_traverse, /* tp_traverse */ 0, /* tp_clear */ (richcmpfunc)set_richcompare, /* tp_richcompare */ offsetof(PySetObject, weakreflist), /* tp_weaklistoffset */ @@ -1122,5 +1139,5 @@ 0, /* tp_init */ PyType_GenericAlloc, /* tp_alloc */ frozenset_new, /* tp_new */ - PyObject_Del, /* tp_free */ + PyObject_GC_Del, /* tp_free */ }; From rhettinger at users.sourceforge.net Sun Jun 19 07:53:17 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 18 Jun 2005 22:53:17 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_set.py, 1.16, 1.16.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11766/Lib/test Modified Files: Tag: release24-maint test_set.py Log Message: SF patch #1200018: Restore GC support to set objects Reverts 1.26 and 1.27. And adds cycle testing. Index: test_set.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_set.py,v retrieving revision 1.16 retrieving revision 1.16.2.1 diff -u -d -r1.16 -r1.16.2.1 --- test_set.py 9 Nov 2004 07:25:31 -0000 1.16 +++ test_set.py 19 Jun 2005 05:53:14 -0000 1.16.2.1 @@ -202,6 +202,16 @@ self.assertNotEqual(id(t), id(newt)) self.assertEqual(t.value + 1, newt.value) + def test_gc(self): + # Create a nest of cycles to exercise overall ref count check + class A: + pass + s = set(A() for i in xrange(1000)) + for elem in s: + elem.cycle = s + elem.sub = elem + elem.set = set([elem]) + class TestSet(TestJointOps): thetype = set From rhettinger at users.sourceforge.net Sun Jun 19 07:53:17 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 18 Jun 2005 22:53:17 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.53, 1.1193.2.54 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11766/Misc Modified Files: Tag: release24-maint NEWS Log Message: SF patch #1200018: Restore GC support to set objects Reverts 1.26 and 1.27. And adds cycle testing. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1193.2.53 retrieving revision 1.1193.2.54 diff -u -d -r1.1193.2.53 -r1.1193.2.54 --- NEWS 4 Jun 2005 09:20:12 -0000 1.1193.2.53 +++ NEWS 19 Jun 2005 05:53:14 -0000 1.1193.2.54 @@ -12,6 +12,8 @@ Core and builtins ----------------- +- Fix garbage collection for set and frozenset objects. SF patch #1200018. + - It is now safe to call PyGILState_Release() before PyEval_InitThreads() (note that if there is reason to believe there are multiple threads around you still must call PyEval_InitThreads() From rhettinger at users.sourceforge.net Sun Jun 19 07:53:17 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sat, 18 Jun 2005 22:53:17 -0700 Subject: [Python-checkins] python/dist/src/Objects setobject.c, 1.31, 1.31.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11766/Objects Modified Files: Tag: release24-maint setobject.c Log Message: SF patch #1200018: Restore GC support to set objects Reverts 1.26 and 1.27. And adds cycle testing. Index: setobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/setobject.c,v retrieving revision 1.31 retrieving revision 1.31.2.1 diff -u -d -r1.31 -r1.31.2.1 --- setobject.c 9 Nov 2004 07:25:28 -0000 1.31 +++ setobject.c 19 Jun 2005 05:53:15 -0000 1.31.2.1 @@ -114,12 +114,21 @@ static void set_dealloc(PySetObject *so) { + PyObject_GC_UnTrack(so); if (so->weakreflist != NULL) PyObject_ClearWeakRefs((PyObject *) so); Py_XDECREF(so->data); so->ob_type->tp_free(so); } +static int +set_traverse(PySetObject *so, visitproc visit, void *arg) +{ + if (so->data) + return visit(so->data, arg); + return 0; +} + static PyObject * set_iter(PySetObject *so) { @@ -757,6 +766,14 @@ PyDoc_STRVAR(clear_doc, "Remove all elements from this set."); +static int +set_tp_clear(PySetObject *so) +{ + PyDict_Clear(so->data); + so->hash = -1; + return 0; +} + static PyObject * set_add(PySetObject *so, PyObject *item) { @@ -1007,11 +1024,11 @@ PyObject_GenericGetAttr, /* tp_getattro */ 0, /* tp_setattro */ 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_CHECKTYPES | + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_CHECKTYPES | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_WEAKREFS, /* tp_flags */ set_doc, /* tp_doc */ - 0, /* tp_traverse */ - 0, /* tp_clear */ + (traverseproc)set_traverse, /* tp_traverse */ + (inquiry)set_tp_clear, /* tp_clear */ (richcmpfunc)set_richcompare, /* tp_richcompare */ offsetof(PySetObject, weakreflist), /* tp_weaklistoffset */ (getiterfunc)set_iter, /* tp_iter */ @@ -1027,7 +1044,7 @@ (initproc)set_init, /* tp_init */ PyType_GenericAlloc, /* tp_alloc */ set_new, /* tp_new */ - PyObject_Del, /* tp_free */ + PyObject_GC_Del, /* tp_free */ }; /* frozenset object ********************************************************/ @@ -1102,10 +1119,10 @@ PyObject_GenericGetAttr, /* tp_getattro */ 0, /* tp_setattro */ 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_CHECKTYPES | + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_CHECKTYPES | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_WEAKREFS, /* tp_flags */ frozenset_doc, /* tp_doc */ - 0, /* tp_traverse */ + (traverseproc)set_traverse, /* tp_traverse */ 0, /* tp_clear */ (richcmpfunc)set_richcompare, /* tp_richcompare */ offsetof(PySetObject, weakreflist), /* tp_weaklistoffset */ @@ -1122,5 +1139,5 @@ 0, /* tp_init */ PyType_GenericAlloc, /* tp_alloc */ frozenset_new, /* tp_new */ - PyObject_Del, /* tp_free */ + PyObject_GC_Del, /* tp_free */ }; From rhettinger at users.sourceforge.net Sun Jun 19 09:37:06 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sun, 19 Jun 2005 00:37:06 -0700 Subject: [Python-checkins] python/nondist/peps pep-0042.txt,1.77,1.78 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24848 Modified Files: pep-0042.txt Log Message: Miscellaneous updates to PEP 42 * Builtin that determines an object's size in bytes. This is much more intractable than it would seem. Allocating a single object may result in a large block being requested from the O/S and never being returned until shutdown. * Test suite. This was a useless entry and more applicable before huge efforts were expended to improve testing coverage. * Four-space tabs. This was fixed-up a long time ago. Since then, a whitespace czar routinely comes along correct subsequent transgressions. * Generalized eval. This was done for Py2.4. * Generalizing builtins to accept UserDict, UserList, sequences etc. This is somewhat out-of-date with the introduction of subclassable builtin types and the iterator interface. Index: pep-0042.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0042.txt,v retrieving revision 1.77 retrieving revision 1.78 diff -u -d -r1.77 -r1.78 --- pep-0042.txt 26 May 2004 16:17:39 -0000 1.77 +++ pep-0042.txt 19 Jun 2005 07:37:03 -0000 1.78 @@ -52,12 +52,6 @@ Core Language / Builtins - - A builtin function that returns the number of bytes an object - uses internally. Apparently mxTools has a sizeof function that - returns the size of the object struct itself. - - http://www.python.org/sf/210835 - - The parser should handle more deeply nested parse trees. The following will fail -- eval("["*50 + "]"*50) -- because the @@ -95,12 +89,6 @@ Standard Library - - The test suite is incomplete (and probably always will be). - This is a reminder to people that more regression tests are - needed. - - http://www.python.org/sf/210819 - - The urllib module should support proxies which require authenication. See SourceForge bug #210619 for information: @@ -122,14 +110,6 @@ http://www.python.org/sf/210599 - - Ensure that all .py files in the std library use 4-space indents and - no hard tabs. This was actually a PEP200 precondition for the - release of 2.0b1, but got misinterpreted as the weaker condition that - tabnanny not complain. Tim Peters will do this now, but, since about - 250 files are affected, will wait until after 2.0final is released. - - http://www.python.org/sf/214557 - - Extend Windows utime to accept directory paths. http://www.python.org/sf/214245 @@ -142,10 +122,6 @@ http://www.python.org/sf/214754 - - Generalize eval to accept any mapping objects for locals and globals. - - http://www.python.org/sf/215126 - - rfc822.py should be more lenient than the spec in the types of address fields it parses. Specifically, an invalid address of the form "From: Amazon.com " should @@ -207,13 +183,6 @@ http://www.python.org/sf/210631 - - every built-in function or method (including all core - extensions) that accepts a string, dict, or list, should also - accept a UserString, UserDict, or UserList. (The latter two - should more generally accept all mappings, all sequences.) - - http://www.python.org/sf/232493 - - Jim Fulton suggested the following: I wonder if it would be a good idea to have a new kind of From rhettinger at users.sourceforge.net Sun Jun 19 10:42:25 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Sun, 19 Jun 2005 01:42:25 -0700 Subject: [Python-checkins] python/dist/src/Objects classobject.c, 2.177, 2.178 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv10111 Modified Files: classobject.c Log Message: Insert missing flag. Index: classobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/classobject.c,v retrieving revision 2.177 retrieving revision 2.178 diff -u -d -r2.177 -r2.178 --- classobject.c 30 Mar 2005 16:32:10 -0000 2.177 +++ classobject.c 19 Jun 2005 08:42:20 -0000 2.178 @@ -2486,7 +2486,7 @@ (getattrofunc)instancemethod_getattro, /* tp_getattro */ PyObject_GenericSetAttr, /* tp_setattro */ 0, /* tp_as_buffer */ - Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */ + Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_HAVE_WEAKREFS, /* tp_flags */ instancemethod_doc, /* tp_doc */ (traverseproc)instancemethod_traverse, /* tp_traverse */ 0, /* tp_clear */ From kbk at users.sourceforge.net Sun Jun 19 20:56:18 2005 From: kbk at users.sourceforge.net (kbk@users.sourceforge.net) Date: Sun, 19 Jun 2005 11:56:18 -0700 Subject: [Python-checkins] python/dist/src/Lib/idlelib NEWS.txt, 1.60, 1.61 PyShell.py, 1.96, 1.97 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/idlelib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32688 Modified Files: NEWS.txt PyShell.py Log Message: when cursor is on a previous command retrieves that command. Instead of replacing the input line, the previous command is now appended to the input line. Indentation is preserved, and undo is enabled. Patch 1196917 Jeff Shute Modified Files: NEWS.txt PyShell.py Index: NEWS.txt =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/NEWS.txt,v retrieving revision 1.60 retrieving revision 1.61 diff -u -d -r1.60 -r1.61 --- NEWS.txt 12 Jun 2005 05:19:23 -0000 1.60 +++ NEWS.txt 19 Jun 2005 18:56:15 -0000 1.61 @@ -3,6 +3,11 @@ *Release date: XX-XXX-2005* +- when cursor is on a previous command retrieves that command. Instead + of replacing the input line, the previous command is now appended to the + input line. Indentation is preserved, and undo is enabled. + Patch 1196917 Jeff Shute + - Clarify "tab/space" Error Dialog and "Tab Width" Dialog associated with the Untabify command. Index: PyShell.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/PyShell.py,v retrieving revision 1.96 retrieving revision 1.97 diff -u -d -r1.96 -r1.97 --- PyShell.py 10 May 2005 03:44:24 -0000 1.96 +++ PyShell.py 19 Jun 2005 18:56:15 -0000 1.97 @@ -1074,7 +1074,7 @@ sel = self.text.get("sel.first", "sel.last") if sel: if self.text.compare("sel.last", "<=", "iomark"): - self.recall(sel) + self.recall(sel, event) return "break" except: pass @@ -1085,18 +1085,18 @@ # Check if there's a relevant stdin range -- if so, use it prev = self.text.tag_prevrange("stdin", "insert") if prev and self.text.compare("insert", "<", prev[1]): - self.recall(self.text.get(prev[0], prev[1])) + self.recall(self.text.get(prev[0], prev[1]), event) return "break" next = self.text.tag_nextrange("stdin", "insert") if next and self.text.compare("insert lineend", ">=", next[0]): - self.recall(self.text.get(next[0], next[1])) + self.recall(self.text.get(next[0], next[1]), event) return "break" # No stdin mark -- just get the current line, less any prompt line = self.text.get("insert linestart", "insert lineend") last_line_of_prompt = sys.ps1.split('\n')[-1] if line.startswith(last_line_of_prompt): line = line[len(last_line_of_prompt):] - self.recall(line) + self.recall(line, event) return "break" # If we're between the beginning of the line and the iomark, i.e. # in the prompt area, move to the end of the prompt @@ -1127,9 +1127,29 @@ self.runit() return "break" - def recall(self, s): - if self.history: - self.history.recall(s) + def recall(self, s, event): + self.text.undo_block_start() + try: + self.text.tag_remove("sel", "1.0", "end") + self.text.mark_set("insert", "end-1c") + s = s.strip() + lines = s.split('\n') + if lines: + prefix = self.text.get("insert linestart","insert").rstrip() + if prefix and prefix[-1]==':': + self.newline_and_indent_event(event) + + self.text.insert("insert",lines[0].strip()) + if len(lines) > 1: + self.newline_and_indent_event(event) + for line in lines[1:]: + self.text.insert("insert", line.strip()) + self.newline_and_indent_event(event) + else: + self.text.insert("insert", s) + finally: + self.text.see("insert") + self.text.undo_block_stop() def runit(self): line = self.text.get("iomark", "end-1c") From goodger at users.sourceforge.net Mon Jun 20 04:27:40 2005 From: goodger at users.sourceforge.net (goodger@users.sourceforge.net) Date: Sun, 19 Jun 2005 19:27:40 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt,1.327,1.328 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv26486 Modified Files: pep-0000.txt Log Message: corrected classification Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.327 retrieving revision 1.328 diff -u -d -r1.327 -r1.328 --- pep-0000.txt 18 Jun 2005 20:36:28 -0000 1.327 +++ pep-0000.txt 20 Jun 2005 02:27:22 -0000 1.328 @@ -150,7 +150,6 @@ SF 277 Unicode file name support for Windows NT Hodgson SF 278 Universal Newline Support Jansen SF 279 The enumerate() built-in function Hettinger - SR 281 Loop Counter Iteration with range and xrange Hetland SF 282 A Logging System Sajip, Mick IF 283 Python 2.3 Release Schedule GvR SF 285 Adding a bool type GvR @@ -199,6 +198,7 @@ SR 271 Prefixing sys.path by command line option Giacometti SR 274 Dict Comprehensions Warsaw SR 276 Simple Iterator for ints Althoff + SR 281 Loop Counter Iteration with range and xrange Hetland SR 284 Integer for-loops Eppstein, Ewing SR 288 Generators Attributes and Exceptions Hettinger SR 295 Interpretation of multiline string constants Koltsov From managed at forex-day-trading.com Mon Jun 20 02:03:26 2005 From: managed at forex-day-trading.com (Forex Managed Accounts) Date: Sun, 19 Jun 2005 20:03:26 -0400 Subject: [Python-checkins] Why Warren Buffett Chose FX Over Real Estate - Live Meeting June 21st on Alternative Investments Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20050619/59c1b523/attachment.htm From rhettinger at users.sourceforge.net Mon Jun 20 11:49:45 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Mon, 20 Jun 2005 02:49:45 -0700 Subject: [Python-checkins] python/dist/src/Lib decimal.py,1.37,1.38 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv26385 Modified Files: decimal.py Log Message: Apply the _is_special guard. Index: decimal.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/decimal.py,v retrieving revision 1.37 retrieving revision 1.38 diff -u -d -r1.37 -r1.38 --- decimal.py 7 Jun 2005 18:52:34 -0000 1.37 +++ decimal.py 20 Jun 2005 09:49:42 -0000 1.38 @@ -760,18 +760,19 @@ Captures all of the information in the underlying representation. """ - if self._isnan(): - minus = '-'*self._sign - if self._int == (0,): - info = '' - else: - info = ''.join(map(str, self._int)) - if self._isnan() == 2: - return minus + 'sNaN' + info - return minus + 'NaN' + info - if self._isinfinity(): - minus = '-'*self._sign - return minus + 'Infinity' + if self._is_special: + if self._isnan(): + minus = '-'*self._sign + if self._int == (0,): + info = '' + else: + info = ''.join(map(str, self._int)) + if self._isnan() == 2: + return minus + 'sNaN' + info + return minus + 'NaN' + info + if self._isinfinity(): + minus = '-'*self._sign + return minus + 'Infinity' if context is None: context = getcontext() From mwh at users.sourceforge.net Mon Jun 20 14:12:47 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 20 Jun 2005 05:12:47 -0700 Subject: [Python-checkins] python/dist/src/Doc/api init.tex,1.23,1.24 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/api In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7251 Modified Files: init.tex Log Message: fix some silly whitespace stuff (must have hit M-q or something?) Index: init.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/api/init.tex,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- init.tex 13 Feb 2005 22:50:02 -0000 1.23 +++ init.tex 20 Jun 2005 12:12:45 -0000 1.24 @@ -471,10 +471,10 @@ PyGILState_Release(gstate); \end{verbatim} -Note that the \cfunction{PyGILState_*()} functions assume there is only -one global interpreter (created automatically by +Note that the \cfunction{PyGILState_*()} functions assume there is +only one global interpreter (created automatically by \cfunction{Py_Initialize()}). Python still supports the creation of -additional interpreters (using \cfunction{Py_NewInterpreter()}), but +additional interpreters (using \cfunction{Py_NewInterpreter()}), but mixing multiple interpreters and the \cfunction{PyGILState_*()} API is unsupported. @@ -697,16 +697,16 @@ \end{cfuncdesc} \begin{cfuncdesc}{PyGILState_STATE}{PyGILState_Ensure}{} -Ensure that the current thread is ready to call the Python -C API regardless of the current state of Python, or of its -thread lock. This may be called as many times as desired -by a thread as long as each call is matched with a call to -\cfunction{PyGILState_Release()}. -In general, other thread-related APIs may -be used between \cfunction{PyGILState_Ensure()} and \cfunction{PyGILState_Release()} calls as long as the -thread state is restored to its previous state before the Release(). -For example, normal usage of the \csimplemacro{Py_BEGIN_ALLOW_THREADS} -and \csimplemacro{Py_END_ALLOW_THREADS} macros is acceptable. +Ensure that the current thread is ready to call the Python C API +regardless of the current state of Python, or of its thread lock. +This may be called as many times as desired by a thread as long as +each call is matched with a call to \cfunction{PyGILState_Release()}. +In general, other thread-related APIs may be used between +\cfunction{PyGILState_Ensure()} and \cfunction{PyGILState_Release()} +calls as long as the thread state is restored to its previous state +before the Release(). For example, normal usage of the +\csimplemacro{Py_BEGIN_ALLOW_THREADS} and +\csimplemacro{Py_END_ALLOW_THREADS} macros is acceptable. The return value is an opaque "handle" to the thread state when \cfunction{PyGILState_Acquire()} was called, and must be passed to From mwh at users.sourceforge.net Mon Jun 20 15:45:35 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 20 Jun 2005 06:45:35 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_asynchat.py, 1.6, 1.7 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22075 Modified Files: test_asynchat.py Log Message: test_asynchat is no longer expected to produce output. also, wait for threads to finish before proceeding. Index: test_asynchat.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_asynchat.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- test_asynchat.py 9 Jun 2005 14:56:31 -0000 1.6 +++ test_asynchat.py 20 Jun 2005 13:45:33 -0000 1.7 @@ -6,7 +6,7 @@ from test import test_support HOST = "127.0.0.1" -PORT = 54321 +PORT = 54322 class echo_server(threading.Thread): @@ -67,6 +67,7 @@ c.push("hello ") c.push("world\n") asyncore.loop() + s.join() self.assertEqual(c.contents, 'hello world') @@ -79,6 +80,7 @@ c.push("hello ") c.push("world\n") asyncore.loop() + s.join() self.assertEqual(c.contents, 'hello ') From mwh at users.sourceforge.net Mon Jun 20 15:45:36 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 20 Jun 2005 06:45:36 -0700 Subject: [Python-checkins] python/dist/src/Lib/test/output test_asynchat, 1.1, NONE Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test/output In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22075/output Removed Files: test_asynchat Log Message: test_asynchat is no longer expected to produce output. also, wait for threads to finish before proceeding. --- test_asynchat DELETED --- From mwh at users.sourceforge.net Mon Jun 20 18:37:06 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 20 Jun 2005 09:37:06 -0700 Subject: [Python-checkins] python/dist/src/Doc/api init.tex,1.24,1.25 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/api In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18045 Modified Files: init.tex Log Message: Add a(nother) warning about mixing Py_NewInterpreter and the PyGILState_* functions. Index: init.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/api/init.tex,v retrieving revision 1.24 retrieving revision 1.25 diff -u -d -r1.24 -r1.25 --- init.tex 20 Jun 2005 12:12:45 -0000 1.24 +++ init.tex 20 Jun 2005 16:37:03 -0000 1.25 @@ -131,6 +131,12 @@ objects may affect the wrong (sub-)interpreter's dictionary of loaded modules. (XXX This is a hard-to-fix bug that will be addressed in a future release.) + + Also note that the use of this functionality is incompatible with + extension modules such as PyObjC and ctypes that use the + \cfunction{PyGILState_*} APIs (and this is inherent in the way the + \cfunction{PyGILState_*} functions work). Simple things may work, + but confusing behavior will always be near. \end{cfuncdesc} \begin{cfuncdesc}{void}{Py_EndInterpreter}{PyThreadState *tstate} From mwh at users.sourceforge.net Mon Jun 20 18:52:58 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 20 Jun 2005 09:52:58 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1300,1.1301 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24783/Misc Modified Files: NEWS Log Message: Fix bug: [ 1163563 ] Sub threads execute in restricted mode basically by fixing bug 1010677 in a non-broken way. Backport candidate. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1300 retrieving revision 1.1301 diff -u -d -r1.1300 -r1.1301 --- NEWS 10 Jun 2005 11:05:16 -0000 1.1300 +++ NEWS 20 Jun 2005 16:52:54 -0000 1.1301 @@ -12,6 +12,12 @@ Core and builtins ----------------- +- SF bug #1163563: the original fix for bug #1010677 ("thread Module + Breaks PyGILState_Ensure()") broke badly in the case of multiple + interpreter states; back out that fix and do a better job (see + http://mail.python.org/pipermail/python-dev/2005-June/054258.html + for a longer write-up of the problem). + - SF patch #1180995: marshal now uses a binary format by default when serializing floats. From mwh at users.sourceforge.net Mon Jun 20 18:52:59 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 20 Jun 2005 09:52:59 -0700 Subject: [Python-checkins] python/dist/src/Modules threadmodule.c, 2.63, 2.64 Message-ID: Update of /cvsroot/python/python/dist/src/Modules In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24783/Modules Modified Files: threadmodule.c Log Message: Fix bug: [ 1163563 ] Sub threads execute in restricted mode basically by fixing bug 1010677 in a non-broken way. Backport candidate. Index: threadmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Modules/threadmodule.c,v retrieving revision 2.63 retrieving revision 2.64 diff -u -d -r2.63 -r2.64 --- threadmodule.c 15 Jun 2005 12:48:40 -0000 2.63 +++ threadmodule.c 20 Jun 2005 16:52:56 -0000 2.64 @@ -413,10 +413,12 @@ t_bootstrap(void *boot_raw) { struct bootstate *boot = (struct bootstate *) boot_raw; - PyGILState_STATE gstate; + PyThreadState *tstate; PyObject *res; - gstate = PyGILState_Ensure(); + tstate = PyThreadState_New(boot->interp); + + PyEval_AcquireThread(tstate); res = PyEval_CallObjectWithKeywords( boot->func, boot->args, boot->keyw); if (res == NULL) { @@ -441,7 +443,8 @@ Py_DECREF(boot->args); Py_XDECREF(boot->keyw); PyMem_DEL(boot_raw); - PyGILState_Release(gstate); + PyThreadState_Clear(tstate); + PyThreadState_DeleteCurrent(); PyThread_exit_thread(); } From mwh at users.sourceforge.net Mon Jun 20 18:53:04 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Mon, 20 Jun 2005 09:53:04 -0700 Subject: [Python-checkins] python/dist/src/Python pystate.c,2.41,2.42 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24783/Python Modified Files: pystate.c Log Message: Fix bug: [ 1163563 ] Sub threads execute in restricted mode basically by fixing bug 1010677 in a non-broken way. Backport candidate. Index: pystate.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/pystate.c,v retrieving revision 2.41 retrieving revision 2.42 diff -u -d -r2.41 -r2.42 --- pystate.c 16 Jun 2005 11:35:00 -0000 2.41 +++ pystate.c 20 Jun 2005 16:52:57 -0000 2.42 @@ -36,6 +36,12 @@ #define HEAD_INIT() (void)(head_mutex || (head_mutex = PyThread_allocate_lock())) #define HEAD_LOCK() PyThread_acquire_lock(head_mutex, WAIT_LOCK) #define HEAD_UNLOCK() PyThread_release_lock(head_mutex) + +/* The single PyInterpreterState used by this process' + GILState implementation +*/ +static PyInterpreterState *autoInterpreterState = NULL; +static int autoTLSkey = 0; #else #define HEAD_INIT() /* Nothing */ #define HEAD_LOCK() /* Nothing */ @@ -47,6 +53,8 @@ PyThreadState *_PyThreadState_Current = NULL; PyThreadFrameGetter _PyThreadState_GetFrame = NULL; +static void _PyGILState_NoteThreadState(PyThreadState* tstate); + PyInterpreterState * PyInterpreterState_New(void) @@ -180,6 +188,8 @@ tstate->c_profileobj = NULL; tstate->c_traceobj = NULL; + _PyGILState_NoteThreadState(tstate); + HEAD_LOCK(); tstate->next = interp->tstate_head; interp->tstate_head = tstate; @@ -261,6 +271,8 @@ "PyThreadState_DeleteCurrent: no current tstate"); _PyThreadState_Current = NULL; tstate_delete_common(tstate); + if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate) + PyThread_delete_key_value(autoTLSkey); PyEval_ReleaseLock(); } #endif /* WITH_THREAD */ @@ -393,12 +405,6 @@ return tstate == _PyThreadState_Current; } -/* The single PyInterpreterState used by this process' - GILState implementation -*/ -static PyInterpreterState *autoInterpreterState = NULL; -static int autoTLSkey = 0; - /* Internal initialization/finalization functions called by Py_Initialize/Py_Finalize */ @@ -408,12 +414,10 @@ assert(i && t); /* must init with valid states */ autoTLSkey = PyThread_create_key(); autoInterpreterState = i; - /* Now stash the thread state for this thread in TLS */ assert(PyThread_get_key_value(autoTLSkey) == NULL); - if (PyThread_set_key_value(autoTLSkey, (void *)t) < 0) - Py_FatalError("Couldn't create autoTLSkey mapping"); - assert(t->gilstate_counter == 0); /* must be a new thread state */ - t->gilstate_counter = 1; + assert(t->gilstate_counter == 0); + + _PyGILState_NoteThreadState(t); } void @@ -424,6 +428,41 @@ autoInterpreterState = NULL;; } +/* When a thread state is created for a thread by some mechanism other than + PyGILState_Ensure, it's important that the GILState machinery knows about + it so it doesn't try to create another thread state for the thread (this is + a better fix for SF bug #1010677 than the first one attempted). +*/ +void +_PyGILState_NoteThreadState(PyThreadState* tstate) +{ + /* If autoTLSkey is 0, this must be the very first threadstate created + in Py_Initialize(). Don't do anything for now (we'll be back here + when _PyGILState_Init is called). */ + if (!autoTLSkey) + return; + + /* Stick the thread state for this thread in thread local storage. + + The only situation where you can legitimately have more than one + thread state for an OS level thread is when there are multiple + interpreters, when: + + a) You shouldn't really be using the PyGILState_ APIs anyway, + and: + + b) The slightly odd way PyThread_set_key_value works (see + comments by its implementation) means that the first thread + state created for that given OS level thread will "win", + which seems reasonable behaviour. + */ + if (PyThread_set_key_value(autoTLSkey, (void *)tstate) < 0) + Py_FatalError("Couldn't create autoTLSkey mapping"); + + /* PyGILState_Release must not try to delete this thread state. */ + tstate->gilstate_counter = 1; +} + /* The public functions */ PyThreadState * PyGILState_GetThisThreadState(void) @@ -450,8 +489,9 @@ tcur = PyThreadState_New(autoInterpreterState); if (tcur == NULL) Py_FatalError("Couldn't create thread-state for new thread"); - if (PyThread_set_key_value(autoTLSkey, (void *)tcur) < 0) - Py_FatalError("Couldn't create autoTLSkey mapping"); + /* This is our thread state! We'll need to delete it in the + matching call to PyGILState_Release(). */ + tcur->gilstate_counter = 0; current = 0; /* new thread state is never current */ } else @@ -498,8 +538,6 @@ * habit of coming back). */ PyThreadState_DeleteCurrent(); - /* Delete this thread from our TLS. */ - PyThread_delete_key_value(autoTLSkey); } /* Release the lock if necessary */ else if (oldstate == PyGILState_UNLOCKED) From kbk at users.sourceforge.net Tue Jun 21 04:42:19 2005 From: kbk at users.sourceforge.net (kbk@users.sourceforge.net) Date: Mon, 20 Jun 2005 19:42:19 -0700 Subject: [Python-checkins] python/dist/src/Lib/idlelib IdleHistory.py, 1.5, 1.6 PyShell.py, 1.97, 1.98 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/idlelib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19290 Modified Files: IdleHistory.py PyShell.py Log Message: Remove dead code M IdleHistory.py M PyShell.py Index: IdleHistory.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/IdleHistory.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- IdleHistory.py 3 Feb 2005 01:37:13 -0000 1.5 +++ IdleHistory.py 21 Jun 2005 02:42:17 -0000 1.6 @@ -86,11 +86,3 @@ self.history.append(source) self.history_pointer = None self.history_prefix = None - - def recall(self, s): - s = s.strip() - self.text.tag_remove("sel", "1.0", "end") - self.text.delete("iomark", "end-1c") - self.text.mark_set("insert", "end-1c") - self.text.insert("insert", s) - self.text.see("insert") Index: PyShell.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/idlelib/PyShell.py,v retrieving revision 1.97 retrieving revision 1.98 diff -u -d -r1.97 -r1.98 --- PyShell.py 19 Jun 2005 18:56:15 -0000 1.97 +++ PyShell.py 21 Jun 2005 02:42:17 -0000 1.98 @@ -1134,19 +1134,15 @@ self.text.mark_set("insert", "end-1c") s = s.strip() lines = s.split('\n') - if lines: - prefix = self.text.get("insert linestart","insert").rstrip() - if prefix and prefix[-1]==':': - self.newline_and_indent_event(event) - - self.text.insert("insert",lines[0].strip()) - if len(lines) > 1: + prefix = self.text.get("insert linestart","insert").rstrip() + if prefix and prefix[-1]==':': + self.newline_and_indent_event(event) + self.text.insert("insert",lines[0].strip()) + if len(lines) > 1: + self.newline_and_indent_event(event) + for line in lines[1:]: + self.text.insert("insert", line.strip()) self.newline_and_indent_event(event) - for line in lines[1:]: - self.text.insert("insert", line.strip()) - self.newline_and_indent_event(event) - else: - self.text.insert("insert", s) finally: self.text.see("insert") self.text.undo_block_stop() From rhettinger at users.sourceforge.net Tue Jun 21 09:44:00 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 21 Jun 2005 00:44:00 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1301,1.1302 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25473/Misc Modified Files: NEWS Log Message: SF bug #1224621: tokenize module does not detect inconsistent dedents Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1301 retrieving revision 1.1302 diff -u -d -r1.1301 -r1.1302 --- NEWS 20 Jun 2005 16:52:54 -0000 1.1301 +++ NEWS 21 Jun 2005 07:43:56 -0000 1.1302 @@ -147,6 +147,9 @@ Library ------- +- The tokenize module now detects and reports indentation errors. + Bug #1224621. + - The tokenize module has a new untokenize() function to support a full roundtrip from lexed tokens back to Python sourcecode. In addition, the generate_tokens() function now accepts a callable argument that From rhettinger at users.sourceforge.net Tue Jun 21 09:44:15 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 21 Jun 2005 00:44:15 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_tokenize.py, 1.10, 1.11 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25473/Lib/test Modified Files: test_tokenize.py Log Message: SF bug #1224621: tokenize module does not detect inconsistent dedents Index: test_tokenize.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_tokenize.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- test_tokenize.py 10 Jun 2005 11:05:19 -0000 1.10 +++ test_tokenize.py 21 Jun 2005 07:43:58 -0000 1.11 @@ -1,4 +1,4 @@ -from test.test_support import verbose, findfile, is_resource_enabled +from test.test_support import verbose, findfile, is_resource_enabled, TestFailed import os, glob, random from tokenize import (tokenize, generate_tokens, untokenize, NUMBER, NAME, OP, STRING) @@ -41,6 +41,24 @@ test_roundtrip(f) +###### Test detecton of IndentationError ###################### + +from cStringIO import StringIO + +sampleBadText = """ +def foo(): + bar + baz +""" + +try: + for tok in generate_tokens(StringIO(sampleBadText).readline): + pass +except IndentationError: + pass +else: + raise TestFailed("Did not detect IndentationError:") + ###### Test example in the docs ############################### From rhettinger at users.sourceforge.net Tue Jun 21 09:44:15 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 21 Jun 2005 00:44:15 -0700 Subject: [Python-checkins] python/dist/src/Lib tokenize.py,1.37,1.38 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25473/Lib Modified Files: tokenize.py Log Message: SF bug #1224621: tokenize module does not detect inconsistent dedents Index: tokenize.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/tokenize.py,v retrieving revision 1.37 retrieving revision 1.38 diff -u -d -r1.37 -r1.38 --- tokenize.py 10 Jun 2005 11:05:18 -0000 1.37 +++ tokenize.py 21 Jun 2005 07:43:57 -0000 1.38 @@ -271,6 +271,9 @@ indents.append(column) yield (INDENT, line[:pos], (lnum, 0), (lnum, pos), line) while column < indents[-1]: + if column not in indents: + raise IndentationError( + "unindent does not match any outer indentation level") indents = indents[:-1] yield (DEDENT, '', (lnum, pos), (lnum, pos), line) From rhettinger at users.sourceforge.net Tue Jun 21 09:53:58 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 21 Jun 2005 00:53:58 -0700 Subject: [Python-checkins] python/dist/src/Lib tokenize.py,1.36,1.36.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31482/Lib Modified Files: Tag: release24-maint tokenize.py Log Message: SF bug #1224621: tokenize module does not detect inconsistent dedents Index: tokenize.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/tokenize.py,v retrieving revision 1.36 retrieving revision 1.36.4.1 diff -u -d -r1.36 -r1.36.4.1 --- tokenize.py 2 Aug 2004 06:09:53 -0000 1.36 +++ tokenize.py 21 Jun 2005 07:53:56 -0000 1.36.4.1 @@ -225,6 +225,9 @@ indents.append(column) yield (INDENT, line[:pos], (lnum, 0), (lnum, pos), line) while column < indents[-1]: + if column not in indents: + raise IndentationError( + "unindent does not match any outer indentation level") indents = indents[:-1] yield (DEDENT, '', (lnum, pos), (lnum, pos), line) From rhettinger at users.sourceforge.net Tue Jun 21 09:53:58 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 21 Jun 2005 00:53:58 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.54, 1.1193.2.55 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31482/Misc Modified Files: Tag: release24-maint NEWS Log Message: SF bug #1224621: tokenize module does not detect inconsistent dedents Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1193.2.54 retrieving revision 1.1193.2.55 diff -u -d -r1.1193.2.54 -r1.1193.2.55 --- NEWS 19 Jun 2005 05:53:14 -0000 1.1193.2.54 +++ NEWS 21 Jun 2005 07:53:55 -0000 1.1193.2.55 @@ -39,6 +39,8 @@ Library ------- +- Bug #1224621: tokenize module does not detect inconsistent dedents + - Bug #1196315: fix weakref.WeakValueDictionary constructor. - Bug #1213894: os.path.realpath didn't resolve symlinks that were the first From rhettinger at users.sourceforge.net Tue Jun 21 09:53:58 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 21 Jun 2005 00:53:58 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_tokenize.py, 1.9, 1.9.12.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31482/Lib/test Modified Files: Tag: release24-maint test_tokenize.py Log Message: SF bug #1224621: tokenize module does not detect inconsistent dedents Index: test_tokenize.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_tokenize.py,v retrieving revision 1.9 retrieving revision 1.9.12.1 diff -u -d -r1.9 -r1.9.12.1 --- test_tokenize.py 12 May 2003 19:42:04 -0000 1.9 +++ test_tokenize.py 21 Jun 2005 07:53:56 -0000 1.9.12.1 @@ -1,4 +1,4 @@ -from test.test_support import verbose, findfile +from test.test_support import verbose, findfile, TestFailed import tokenize, os, sys if verbose: @@ -10,3 +10,21 @@ if verbose: print 'finished' + +###### Test detecton of IndentationError ###################### + +from cStringIO import StringIO + +sampleBadText = """ +def foo(): + bar + baz +""" + +try: + for tok in tokenize.generate_tokens(StringIO(sampleBadText).readline): + pass +except IndentationError: + pass +else: + raise TestFailed("Did not detect IndentationError:") From jackjansen at users.sourceforge.net Tue Jun 21 22:54:53 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Tue, 21 Jun 2005 13:54:53 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenVariable.py, 1.4, 1.5 scantools.py, 1.36, 1.37 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25268 Modified Files: bgenVariable.py scantools.py Log Message: Added support for optional modifiers to functions/methods (such as C++ const, static for methods, inline, etc). Index: bgenVariable.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenVariable.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- bgenVariable.py 16 Jun 2005 21:26:24 -0000 1.4 +++ bgenVariable.py 21 Jun 2005 20:54:51 -0000 1.5 @@ -14,6 +14,7 @@ ReturnMode = 8+OutMode # this is the function return value ErrorMode = 16+OutMode # this is an error status -- turn it into an exception RefMode = 32 +ConstMode = 64 class Variable: @@ -47,7 +48,8 @@ def getDeclaration(self): """Return the unadorned declaration of the variable, suitable for use in a formal parameter list.""" - return self.type.getDeclaration(self.name) + refmode = (self.flags & RefMode) + return self.type.getDeclaration(self.name, reference=refmode) def getargsFormat(self): """Call the type's getargsFormatmethod.""" Index: scantools.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/scantools.py,v retrieving revision 1.36 retrieving revision 1.37 diff -u -d -r1.36 -r1.37 --- scantools.py 14 Jun 2005 21:32:51 -0000 1.36 +++ scantools.py 21 Jun 2005 20:54:51 -0000 1.37 @@ -479,6 +479,7 @@ self.report("(but type matched)") return type, name, args = match.group('type', 'name', 'args') + modifiers = self.getmodifiers(match) type = self.pythonizename(type) name = self.pythonizename(name) if name in self.alreadydone: @@ -499,8 +500,14 @@ self.report("*** %s %s unmanageable", type, name) return self.alreadydone.append(name) - self.generate(type, name, arglist) + if modifiers: + self.generate(type, name, arglist, modifiers) + else: + self.generate(type, name, arglist) + def getmodifiers(self, match): + return [] + def pythonizename(self, name): name = re.sub("\*", " ptr", name) name = name.strip() @@ -592,12 +599,16 @@ ##self.report("new: %r", new) return new - def generate(self, type, name, arglist): - self.typeused(type, 'return') - classname, listname = self.destination(type, name, arglist) + def generate(self, tp, name, arglist, modifiers=[]): + + self.typeused(tp, 'return') + if modifiers: + classname, listname = self.destination(tp, name, arglist, modifiers) + else: + classname, listname = self.destination(tp, name, arglist) if not classname or not listname: return if not self.specfile: return - self.specfile.write("f = %s(%s, %r,\n" % (classname, type, name)) + self.specfile.write("f = %s(%s, %r,\n" % (classname, tp, name)) for atype, aname, amode in arglist: self.typeused(atype, amode) self.specfile.write(" (%s, %r, %s),\n" % From jackjansen at users.sourceforge.net Wed Jun 22 22:35:26 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Wed, 22 Jun 2005 13:35:26 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenBuffer.py, 1.9, 1.10 bgenHeapBuffer.py, 1.5, 1.6 bgenStackBuffer.py, 1.3, 1.4 bgenStringBuffer.py, 1.3, 1.4 bgenType.py, 1.14, 1.15 bgenVariable.py, 1.5, 1.6 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25145 Modified Files: bgenBuffer.py bgenHeapBuffer.py bgenStackBuffer.py bgenStringBuffer.py bgenType.py bgenVariable.py Log Message: Revamped type declaration so the basic routines return a list of strings. This allows variables to be declared as formal arguments. The bgenType.declare method now simply outputs all declarations on separate lines ending in semicolons. Index: bgenBuffer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenBuffer.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- bgenBuffer.py 18 Jul 2004 06:02:01 -0000 1.9 +++ bgenBuffer.py 22 Jun 2005 20:35:23 -0000 1.10 @@ -38,23 +38,25 @@ self.sizeformat = sizeformat or type2format[sizetype] self.label_needed = 0 - def declare(self, name): - self.declareBuffer(name) - self.declareSize(name) + def getDeclarations(self, name, reference=False): + if reference: + raise RuntimeError, "Cannot pass buffer types by reference" + return self.getBufferDeclarations(name) + self.getSizeDeclarations(name) - def declareBuffer(self, name): - self.declareInputBuffer(name) - self.declareOutputBuffer(name) + def getBufferDeclarations(self, name): + return self.getInputBufferDeclarations(name) + self.getOutputBufferDeclarations(name) - def declareInputBuffer(self, name): - Output("%s *%s__in__;", self.datatype, name) + def getInputBufferDeclarations(self, name): + return ["%s *%s__in__" % (self.datatype, name)] - def declareOutputBuffer(self, name): - Output("%s %s__out__[%s];", self.datatype, name, self.size) + def getOutputBufferDeclarations(self, name): + return ["%s %s__out__[%s]" % (self.datatype, name, self.size)] - def declareSize(self, name): - Output("%s %s__len__;", self.sizetype, name) - Output("int %s__in_len__;", name) + def getSizeDeclarations(self, name): + return [ + "%s %s__len__" %(self.sizetype, name), + "int %s__in_len__" %(name) + ] def getargsFormat(self): return "s#" @@ -102,14 +104,14 @@ class InputOnlyBufferMixIn(InputOnlyMixIn): - def declareOutputBuffer(self, name): - pass + def getOutputBufferDeclarations(self, name): + return [] class OutputOnlyBufferMixIn(OutputOnlyMixIn): - def declareInputBuffer(self, name): - pass + def getInputBufferDeclarations(self, name): + return [] class OptionalInputBufferMixIn: @@ -183,14 +185,14 @@ FixedInputOutputBufferType.__init__(self, "sizeof(%s)" % type) self.typeName = self.type = type - def declareInputBuffer(self, name): - Output("%s *%s__in__;", self.type, name) + def getInputBufferDeclarations(self, name): + return ["%s *%s__in__" % (self.type, name)] - def declareSize(self, name): - Output("int %s__in_len__;", name) + def getSizeDeclarations(self, name): + return ["int %s__in_len__" % (name)] - def declareOutputBuffer(self, name): - Output("%s %s__out__;", self.type, name) + def getOutputBufferDeclarations(self, name): + return ["%s %s__out__" % (self.type, name)] def getargsArgs(self, name): return "(char **)&%s__in__, &%s__in_len__" % (name, name) @@ -243,8 +245,8 @@ Instantiate with the struct type as parameter. """ - def declareSize(self, name): - pass + def getSizeDeclarations(self, name): + return [] def passOutput(self, name): return "&%s__out__" % name @@ -257,8 +259,8 @@ Instantiate with the struct type as parameter. """ - def declareSize(self, name): - pass + def getSizeDeclarations(self, name): + return [] def passOutput(self, name): return "%s__out__" % name Index: bgenHeapBuffer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenHeapBuffer.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- bgenHeapBuffer.py 18 Jul 2004 06:02:01 -0000 1.5 +++ bgenHeapBuffer.py 22 Jun 2005 20:35:23 -0000 1.6 @@ -16,8 +16,8 @@ def __init__(self, datatype = 'char', sizetype = 'int', sizeformat = None): FixedInputOutputBufferType.__init__(self, "0", datatype, sizetype, sizeformat) - def declareOutputBuffer(self, name): - Output("%s *%s__out__;", self.datatype, name) + def getOutputBufferDeclarations(self, name): + return ["%s *%s__out__" % (self.datatype, name)] def getargsCheck(self, name): Output("if ((%s__out__ = malloc(%s__in_len__)) == NULL)", name, name) @@ -74,8 +74,8 @@ Call from Python with buffer size. """ - def declareInputBuffer(self, name): - pass + def getInputBufferDeclarations(self, name): + return [] def getargsFormat(self): return "i" Index: bgenStackBuffer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenStackBuffer.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- bgenStackBuffer.py 19 Jan 2003 21:53:57 -0000 1.3 +++ bgenStackBuffer.py 22 Jun 2005 20:35:23 -0000 1.4 @@ -22,8 +22,8 @@ Instantiate with the buffer size as parameter. """ - def declareSize(self, name): - Output("int %s__len__ = %s;", name, self.size) + def getSizeDeclarations(self, name): + return ["int %s__len__ = %s" % (name, self.size)] def passOutput(self, name): return "%s__out__, &%s__len__" % (name, name) Index: bgenStringBuffer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenStringBuffer.py,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- bgenStringBuffer.py 18 Jul 2004 06:02:01 -0000 1.3 +++ bgenStringBuffer.py 22 Jun 2005 20:35:23 -0000 1.4 @@ -23,8 +23,8 @@ less common. I'll write the classes when there is demand.) """ - def declareSize(self, name): - pass + def getSizeDeclarations(self, name): + return [] def getargsFormat(self): return "s" Index: bgenType.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenType.py,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- bgenType.py 16 Jun 2005 21:26:23 -0000 1.14 +++ bgenType.py 22 Jun 2005 20:35:23 -0000 1.15 @@ -24,15 +24,16 @@ Example: int.declare('spam') prints "int spam;" """ - Output("%s;", self.getDeclaration(name, reference)) + for decl in self.getDeclarations(name, reference): + Output("%s;", decl) - def getDeclaration(self, name, reference=False): + def getDeclarations(self, name, reference=False): """Return a string declaring a variable or argument, without any syntactic adornment""" if reference: - return "%s& %s" % (self.typeName, name) + return ["%s& %s" % (self.typeName, name)] else: - return "%s %s" % (self.typeName, name) + return ["%s %s" % (self.typeName, name)] def getargs(self): return self.getargsFormat(), self.getargsArgs() @@ -185,9 +186,9 @@ def __init__(self, substitute): self.substitute = substitute self.typeName = None # Don't show this argument in __doc__ string - - def declare(self, name, reference=False): - pass + + def getDeclarations(self, name, reference=False): + return [] def getargsFormat(self): return "" Index: bgenVariable.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenVariable.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- bgenVariable.py 21 Jun 2005 20:54:51 -0000 1.5 +++ bgenVariable.py 22 Jun 2005 20:35:23 -0000 1.6 @@ -45,11 +45,11 @@ elif self.flags != SelfMode: self.type.declare(self.name) - def getDeclaration(self): + def getDeclarations(self): """Return the unadorned declaration of the variable, suitable for use in a formal parameter list.""" refmode = (self.flags & RefMode) - return self.type.getDeclaration(self.name, reference=refmode) + return self.type.getDeclarations(self.name, reference=refmode) def getargsFormat(self): """Call the type's getargsFormatmethod.""" From montanaro at users.sourceforge.net Thu Jun 23 00:56:10 2005 From: montanaro at users.sourceforge.net (montanaro@users.sourceforge.net) Date: Wed, 22 Jun 2005 15:56:10 -0700 Subject: [Python-checkins] python/nondist/peps pep-0304.txt,1.13,1.14 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8181 Modified Files: pep-0304.txt Log Message: update post history Index: pep-0304.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0304.txt,v retrieving revision 1.13 retrieving revision 1.14 diff -u -d -r1.13 -r1.14 --- pep-0304.txt 18 Jun 2005 13:04:03 -0000 1.13 +++ pep-0304.txt 22 Jun 2005 22:56:06 -0000 1.14 @@ -7,7 +7,7 @@ Type: Standards Track Content-Type: text/x-rst Created: 22-Jan-2003 -Post-History: 27-Jan-2003, 31-Jan-2003 +Post-History: 27-Jan-2003, 31-Jan-2003, 17-Jun-2005 Abstract From jackjansen at users.sourceforge.net Fri Jun 24 00:33:02 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Thu, 23 Jun 2005 15:33:02 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenObjectDefinition.py, 1.28, 1.29 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv9716 Modified Files: bgenObjectDefinition.py Log Message: Added a missing newline Output(). Index: bgenObjectDefinition.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenObjectDefinition.py,v retrieving revision 1.28 retrieving revision 1.29 diff -u -d -r1.28 -r1.29 --- bgenObjectDefinition.py 10 Jun 2005 10:46:40 -0000 1.28 +++ bgenObjectDefinition.py 23 Jun 2005 22:32:59 -0000 1.29 @@ -117,6 +117,7 @@ "Override this method to apply additional checks/conversions" def outputConvert(self): + Output() Output("%sint %s_Convert(PyObject *v, %s *p_itself)", self.static, self.prefix, self.itselftype) OutLbrace() From tim_one at users.sourceforge.net Fri Jun 24 21:46:56 2005 From: tim_one at users.sourceforge.net (tim_one@users.sourceforge.net) Date: Fri, 24 Jun 2005 12:46:56 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenGenerator.py, 1.17, 1.18 bgenObjectDefinition.py, 1.29, 1.30 bgenType.py, 1.15, 1.16 bgenVariable.py, 1.6, 1.7 scantools.py, 1.37, 1.38 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18095/Tools/bgen/bgen Modified Files: bgenGenerator.py bgenObjectDefinition.py bgenType.py bgenVariable.py scantools.py Log Message: Normalize whitespace to avoid offending Bug Day volunteers. Index: bgenGenerator.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenGenerator.py,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- bgenGenerator.py 10 Jun 2005 10:46:40 -0000 1.17 +++ bgenGenerator.py 24 Jun 2005 19:46:36 -0000 1.18 @@ -256,7 +256,7 @@ self.itself = Variable(t0, "_self->ob_itself", SelfMode) self.argumentList.append(self.itself) FunctionGenerator.parseArgumentList(self, args) - + def _test(): void = None eggs = FunctionGenerator(void, "eggs", Index: bgenObjectDefinition.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenObjectDefinition.py,v retrieving revision 1.29 retrieving revision 1.30 diff -u -d -r1.29 -r1.30 --- bgenObjectDefinition.py 23 Jun 2005 22:32:59 -0000 1.29 +++ bgenObjectDefinition.py 24 Jun 2005 19:46:52 -0000 1.30 @@ -45,7 +45,7 @@ OutHeader2("Object type " + self.name) self.outputCheck() - + Output("typedef struct %s {", self.objecttype) IndentLevel() Output("PyObject_HEAD") Index: bgenType.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenType.py,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- bgenType.py 22 Jun 2005 20:35:23 -0000 1.15 +++ bgenType.py 24 Jun 2005 19:46:53 -0000 1.16 @@ -34,7 +34,7 @@ return ["%s& %s" % (self.typeName, name)] else: return ["%s %s" % (self.typeName, name)] - + def getargs(self): return self.getargsFormat(), self.getargsArgs() @@ -78,7 +78,7 @@ Default is to call passInput(). """ return self.passInput(name) - + def errorCheck(self, name): """Check for an error returned in the variable. @@ -186,7 +186,7 @@ def __init__(self, substitute): self.substitute = substitute self.typeName = None # Don't show this argument in __doc__ string - + def getDeclarations(self, name, reference=False): return [] @@ -251,23 +251,23 @@ def mkvalueArgs(self, name): return "%s, %s" % (self.new, name) - + class OpaqueByRefType(OpaqueType): """An opaque object type, passed by reference. Instantiate with the type name, and optionally an object type name whose New/Convert functions will be used. """ - + def passInput(self, name): return name - + # def passOutput(self, name): # return name - + def mkvalueFormat(self): return "O" - + def mkvalueArgs(self, name): return "%s(%s)" % (self.new, name) Index: bgenVariable.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenVariable.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- bgenVariable.py 22 Jun 2005 20:35:23 -0000 1.6 +++ bgenVariable.py 24 Jun 2005 19:46:53 -0000 1.7 @@ -44,7 +44,7 @@ self.type.declare(self.name, reference=True) elif self.flags != SelfMode: self.type.declare(self.name) - + def getDeclarations(self): """Return the unadorned declaration of the variable, suitable for use in a formal parameter list.""" Index: scantools.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/scantools.py,v retrieving revision 1.37 retrieving revision 1.38 diff -u -d -r1.37 -r1.38 --- scantools.py 21 Jun 2005 20:54:51 -0000 1.37 +++ scantools.py 24 Jun 2005 19:46:53 -0000 1.38 @@ -507,7 +507,7 @@ def getmodifiers(self, match): return [] - + def pythonizename(self, name): name = re.sub("\*", " ptr", name) name = name.strip() @@ -600,7 +600,7 @@ return new def generate(self, tp, name, arglist, modifiers=[]): - + self.typeused(tp, 'return') if modifiers: classname, listname = self.destination(tp, name, arglist, modifiers) From bcannon at users.sourceforge.net Sat Jun 25 09:07:37 2005 From: bcannon at users.sourceforge.net (bcannon@users.sourceforge.net) Date: Sat, 25 Jun 2005 00:07:37 -0700 Subject: [Python-checkins] python/dist/src/Include pystate.h,2.29,2.30 Message-ID: Update of /cvsroot/python/python/dist/src/Include In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31041/Include Modified Files: pystate.h Log Message: Add comments about PyThreadState and the usage of its fields. Index: pystate.h =================================================================== RCS file: /cvsroot/python/python/dist/src/Include/pystate.h,v retrieving revision 2.29 retrieving revision 2.30 diff -u -d -r2.29 -r2.30 --- pystate.h 8 Jun 2004 08:17:42 -0000 2.29 +++ pystate.h 25 Jun 2005 07:07:34 -0000 2.30 @@ -53,12 +53,16 @@ #define PyTrace_C_RETURN 6 typedef struct _ts { + /* See Python/ceval.c for comments explaining most fields */ struct _ts *next; PyInterpreterState *interp; struct _frame *frame; int recursion_depth; + /* 'tracing' keeps track of the execution depth when tracing/profiling. + This is to prevent the actual trace/profile code from being recorded in + the trace/profile. */ int tracing; int use_tracing; @@ -75,7 +79,7 @@ PyObject *exc_value; PyObject *exc_traceback; - PyObject *dict; + PyObject *dict; /* Stores per-thread state */ /* tick_counter is incremented whenever the check_interval ticker * reaches zero. The purpose is to give a useful measure of the number From bcannon at users.sourceforge.net Sat Jun 25 09:07:38 2005 From: bcannon at users.sourceforge.net (bcannon@users.sourceforge.net) Date: Sat, 25 Jun 2005 00:07:38 -0700 Subject: [Python-checkins] python/dist/src/Python ceval.c,2.422,2.423 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31041/Python Modified Files: ceval.c Log Message: Add comments about PyThreadState and the usage of its fields. Index: ceval.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v retrieving revision 2.422 retrieving revision 2.423 diff -u -d -r2.422 -r2.423 --- ceval.c 4 Apr 2005 15:49:02 -0000 2.422 +++ ceval.c 25 Jun 2005 07:07:35 -0000 2.423 @@ -3284,10 +3284,12 @@ Py_XINCREF(arg); tstate->c_profilefunc = NULL; tstate->c_profileobj = NULL; + /* Must make sure that tracing is not ignored if 'temp' is freed */ tstate->use_tracing = tstate->c_tracefunc != NULL; Py_XDECREF(temp); tstate->c_profilefunc = func; tstate->c_profileobj = arg; + /* Flag that tracing or profiling is turned on */ tstate->use_tracing = (func != NULL) || (tstate->c_tracefunc != NULL); } @@ -3299,10 +3301,12 @@ Py_XINCREF(arg); tstate->c_tracefunc = NULL; tstate->c_traceobj = NULL; + /* Must make sure that profiling is not ignored if 'temp' is freed */ tstate->use_tracing = tstate->c_profilefunc != NULL; Py_XDECREF(temp); tstate->c_tracefunc = func; tstate->c_traceobj = arg; + /* Flag that tracing or profiling is turned on */ tstate->use_tracing = ((func != NULL) || (tstate->c_profilefunc != NULL)); } From bcannon at users.sourceforge.net Sat Jun 25 10:23:44 2005 From: bcannon at users.sourceforge.net (bcannon@users.sourceforge.net) Date: Sat, 25 Jun 2005 01:23:44 -0700 Subject: [Python-checkins] python/dist/src/Python ceval.c, 2.423, 2.424 compile.c, 2.349, 2.350 marshal.c, 1.86, 1.87 Message-ID: Update of /cvsroot/python/python/dist/src/Python In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2413/Python Modified Files: ceval.c compile.c marshal.c Log Message: Fix signedness of various char variables to stop causing a warning under gcc 4. Index: ceval.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/ceval.c,v retrieving revision 2.423 retrieving revision 2.424 diff -u -d -r2.423 -r2.424 --- ceval.c 25 Jun 2005 07:07:35 -0000 2.423 +++ ceval.c 25 Jun 2005 08:23:40 -0000 2.424 @@ -721,7 +721,7 @@ consts = co->co_consts; fastlocals = f->f_localsplus; freevars = f->f_localsplus + f->f_nlocals; - first_instr = PyString_AS_STRING(co->co_code); + first_instr = (unsigned char*) PyString_AS_STRING(co->co_code); /* An explanation is in order for the next line. f->f_lasti now refers to the index of the last instruction Index: compile.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/compile.c,v retrieving revision 2.349 retrieving revision 2.350 diff -u -d -r2.349 -r2.350 --- compile.c 5 Mar 2005 06:47:57 -0000 2.349 +++ compile.c 25 Jun 2005 08:23:41 -0000 2.350 @@ -662,7 +662,7 @@ /* Bypass optimization when the lineno table is too complex */ assert(PyString_Check(lineno_obj)); - lineno = PyString_AS_STRING(lineno_obj); + lineno = (unsigned char*)PyString_AS_STRING(lineno_obj); tabsiz = PyString_GET_SIZE(lineno_obj); if (memchr(lineno, 255, tabsiz) != NULL) goto exitUnchanged; Index: marshal.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/marshal.c,v retrieving revision 1.86 retrieving revision 1.87 diff -u -d -r1.86 -r1.87 --- marshal.c 13 Jun 2005 18:28:46 -0000 1.86 +++ marshal.c 25 Jun 2005 08:23:41 -0000 1.87 @@ -169,14 +169,14 @@ } else if (PyFloat_Check(v)) { if (p->version > 1) { - char buf[8]; + unsigned char buf[8]; if (_PyFloat_Pack8(PyFloat_AsDouble(v), buf, 1) < 0) { p->error = 1; return; } w_byte(TYPE_BINARY_FLOAT, p); - w_string(buf, 8, p); + w_string((char*)buf, 8, p); } else { char buf[256]; /* Plenty to format any double */ @@ -190,20 +190,20 @@ #ifndef WITHOUT_COMPLEX else if (PyComplex_Check(v)) { if (p->version > 1) { - char buf[8]; + unsigned char buf[8]; if (_PyFloat_Pack8(PyComplex_RealAsDouble(v), buf, 1) < 0) { p->error = 1; return; } w_byte(TYPE_BINARY_COMPLEX, p); - w_string(buf, 8, p); + w_string((char*)buf, 8, p); if (_PyFloat_Pack8(PyComplex_ImagAsDouble(v), buf, 1) < 0) { p->error = 1; return; } - w_string(buf, 8, p); + w_string((char*)buf, 8, p); } else { char buf[256]; /* Plenty to format any double */ @@ -556,9 +556,9 @@ case TYPE_BINARY_FLOAT: { - char buf[8]; + unsigned char buf[8]; double x; - if (r_string(buf, 8, p) != 8) { + if (r_string((char*)buf, 8, p) != 8) { PyErr_SetString(PyExc_EOFError, "EOF read where object expected"); return NULL; @@ -600,9 +600,9 @@ case TYPE_BINARY_COMPLEX: { - char buf[8]; + unsigned char buf[8]; Py_complex c; - if (r_string(buf, 8, p) != 8) { + if (r_string((char*)buf, 8, p) != 8) { PyErr_SetString(PyExc_EOFError, "EOF read where object expected"); return NULL; @@ -611,7 +611,7 @@ if (c.real == -1.0 && PyErr_Occurred()) { return NULL; } - if (r_string(buf, 8, p) != 8) { + if (r_string((char*)buf, 8, p) != 8) { PyErr_SetString(PyExc_EOFError, "EOF read where object expected"); return NULL; From birkenfeld at users.sourceforge.net Sat Jun 25 20:24:06 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 11:24:06 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libsmtplib.tex, 1.26, 1.27 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2706/Doc/lib Modified Files: libsmtplib.tex Log Message: Patch #1227442: smtplib.SMTP.sendmail() accepts a string or list as to_addrs. Index: libsmtplib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libsmtplib.tex,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- libsmtplib.tex 31 Dec 2003 18:37:28 -0000 1.26 +++ libsmtplib.tex 25 Jun 2005 18:24:03 -0000 1.27 @@ -190,13 +190,14 @@ \begin{methoddesc}{sendmail}{from_addr, to_addrs, msg\optional{, mail_options, rcpt_options}} Send mail. The required arguments are an \rfc{822} from-address -string, a list of \rfc{822} to-address strings, and a message string. -The caller may pass a list of ESMTP options (such as \samp{8bitmime}) -to be used in \samp{MAIL FROM} commands as \var{mail_options}. ESMTP -options (such as \samp{DSN} commands) that should be used with all -\samp{RCPT} commands can be passed as \var{rcpt_options}. (If you -need to use different ESMTP options to different recipients you have -to use the low-level methods such as \method{mail}, \method{rcpt} and +string, a list of \rfc{822} to-address strings (a bare string will be +treated as a list with 1 address), and a message string. The caller +may pass a list of ESMTP options (such as \samp{8bitmime}) to be used +in \samp{MAIL FROM} commands as \var{mail_options}. ESMTP options +(such as \samp{DSN} commands) that should be used with all \samp{RCPT} +commands can be passed as \var{rcpt_options}. (If you need to use +different ESMTP options to different recipients you have to use the +low-level methods such as \method{mail}, \method{rcpt} and \method{data} to send the message.) \note{The \var{from_addr} and \var{to_addrs} parameters are From birkenfeld at users.sourceforge.net Sat Jun 25 20:24:31 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 11:24:31 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libsmtplib.tex, 1.26, 1.26.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2761 Modified Files: Tag: release24-maint libsmtplib.tex Log Message: backport Patch #1227442: smtplib.SMTP.sendmail() accepts list or string for to_addrs. Index: libsmtplib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libsmtplib.tex,v retrieving revision 1.26 retrieving revision 1.26.4.1 diff -u -d -r1.26 -r1.26.4.1 --- libsmtplib.tex 31 Dec 2003 18:37:28 -0000 1.26 +++ libsmtplib.tex 25 Jun 2005 18:24:28 -0000 1.26.4.1 @@ -190,13 +190,14 @@ \begin{methoddesc}{sendmail}{from_addr, to_addrs, msg\optional{, mail_options, rcpt_options}} Send mail. The required arguments are an \rfc{822} from-address -string, a list of \rfc{822} to-address strings, and a message string. -The caller may pass a list of ESMTP options (such as \samp{8bitmime}) -to be used in \samp{MAIL FROM} commands as \var{mail_options}. ESMTP -options (such as \samp{DSN} commands) that should be used with all -\samp{RCPT} commands can be passed as \var{rcpt_options}. (If you -need to use different ESMTP options to different recipients you have -to use the low-level methods such as \method{mail}, \method{rcpt} and +string, a list of \rfc{822} to-address strings (a bare string will be +treated as a list with 1 address), and a message string. The caller +may pass a list of ESMTP options (such as \samp{8bitmime}) to be used +in \samp{MAIL FROM} commands as \var{mail_options}. ESMTP options +(such as \samp{DSN} commands) that should be used with all \samp{RCPT} +commands can be passed as \var{rcpt_options}. (If you need to use +different ESMTP options to different recipients you have to use the +low-level methods such as \method{mail}, \method{rcpt} and \method{data} to send the message.) \note{The \var{from_addr} and \var{to_addrs} parameters are From birkenfeld at users.sourceforge.net Sat Jun 25 20:44:51 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 11:44:51 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.156,1.157 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13058/Doc/lib Modified Files: libos.tex Log Message: Bug [ 1225705 ] os.environ documentation should mention unsetenv Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.156 retrieving revision 1.157 diff -u -d -r1.156 -r1.157 --- libos.tex 10 Jun 2005 19:55:35 -0000 1.156 +++ libos.tex 25 Jun 2005 18:44:49 -0000 1.157 @@ -106,9 +106,15 @@ \code{environ} may cause memory leaks. Refer to the system documentation for \cfunction{putenv()}.} -If \function{putenv()} is not provided, this mapping may be passed to -the appropriate process-creation functions to cause child processes to -use a modified environment. +If \function{putenv()} is not provided, a modified copy of this mapping +may be passed to the appropriate process-creation functions to cause +child processes to use a modified environment. + +If the platform supports the \function{unsetenv()} function, you can +delete items in this mapping to unset environment variables. +\function{unsetenv()} will be called automatically when an item is +deleted from \code{os.environ}. + \end{datadesc} \begin{funcdescni}{chdir}{path} @@ -307,7 +313,20 @@ Availability: recent flavors of \UNIX. \end{funcdesc} +\begin{funcdesc}{unsetenv}{varname} +\index{environment variables!deleting} +Unset (delete) the environment variable named \var{varname}. Such +changes to the environment affect subprocesses started with +\function{os.system()}, \function{popen()} or \function{fork()} and +\function{execv()}. Availability: most flavors of \UNIX, Windows. +When \function{unsetenv()} is +supported, deletion of items in \code{os.environ} is automatically +translated into a corresponding call to \function{unsetenv()}; however, +calls to \function{unsetenv()} don't update \code{os.environ}, so it is +actually preferable to delete items of \code{os.environ}. +\end{funcdesc} +\end{funcdesc} \subsection{File Object Creation \label{os-newstreams}} From birkenfeld at users.sourceforge.net Sat Jun 25 20:46:12 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 11:46:12 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.157,1.158 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13873/Doc/lib Modified Files: libos.tex Log Message: correcting duplicate TeX markup Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.157 retrieving revision 1.158 diff -u -d -r1.157 -r1.158 --- libos.tex 25 Jun 2005 18:44:49 -0000 1.157 +++ libos.tex 25 Jun 2005 18:46:09 -0000 1.158 @@ -326,7 +326,6 @@ calls to \function{unsetenv()} don't update \code{os.environ}, so it is actually preferable to delete items of \code{os.environ}. \end{funcdesc} -\end{funcdesc} \subsection{File Object Creation \label{os-newstreams}} From birkenfeld at users.sourceforge.net Sat Jun 25 20:47:31 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 11:47:31 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex, 1.146.2.3, 1.146.2.4 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14211/Doc/lib Modified Files: Tag: release24-maint libos.tex Log Message: backport of bug [ 1225705 ] os.environ documentation should mention unsetenv Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.146.2.3 retrieving revision 1.146.2.4 diff -u -d -r1.146.2.3 -r1.146.2.4 --- libos.tex 10 Jun 2005 19:55:46 -0000 1.146.2.3 +++ libos.tex 25 Jun 2005 18:47:28 -0000 1.146.2.4 @@ -106,9 +106,15 @@ \code{environ} may cause memory leaks. Refer to the system documentation for \cfunction{putenv()}.} -If \function{putenv()} is not provided, this mapping may be passed to -the appropriate process-creation functions to cause child processes to -use a modified environment. +If \function{putenv()} is not provided, a modified copy of this mapping +may be passed to the appropriate process-creation functions to cause +child processes to use a modified environment. + +If the platform supports the \function{unsetenv()} function, you can +delete items in this mapping to unset environment variables. +\function{unsetenv()} will be called automatically when an item is +deleted from \code{os.environ}. + \end{datadesc} \begin{funcdescni}{chdir}{path} @@ -307,7 +313,19 @@ Availability: recent flavors of \UNIX. \end{funcdesc} +\begin{funcdesc}{unsetenv}{varname} +\index{environment variables!deleting} +Unset (delete) the environment variable named \var{varname}. Such +changes to the environment affect subprocesses started with +\function{os.system()}, \function{popen()} or \function{fork()} and +\function{execv()}. Availability: most flavors of \UNIX, Windows. +When \function{unsetenv()} is +supported, deletion of items in \code{os.environ} is automatically +translated into a corresponding call to \function{unsetenv()}; however, +calls to \function{unsetenv()} don't update \code{os.environ}, so it is +actually preferable to delete items of \code{os.environ}. +\end{funcdesc} \subsection{File Object Creation \label{os-newstreams}} From birkenfeld at users.sourceforge.net Sat Jun 25 20:52:27 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 11:52:27 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.158,1.159 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16864/Doc/lib Modified Files: libos.tex Log Message: Patch [ 1213031 ] note that os.chown can have -1 as an argument Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.158 retrieving revision 1.159 diff -u -d -r1.158 -r1.159 --- libos.tex 25 Jun 2005 18:46:09 -0000 1.158 +++ libos.tex 25 Jun 2005 18:52:24 -0000 1.159 @@ -763,7 +763,7 @@ \begin{funcdesc}{chown}{path, uid, gid} Change the owner and group id of \var{path} to the numeric \var{uid} -and \var{gid}. +and \var{gid}. To leave one of the ids unchanged, set it to -1. Availability: Macintosh, \UNIX. \end{funcdesc} From birkenfeld at users.sourceforge.net Sat Jun 25 20:53:26 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 11:53:26 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex, 1.146.2.4, 1.146.2.5 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17391/Doc/lib Modified Files: Tag: release24-maint libos.tex Log Message: backport patch [ 1213031 ] note that os.chown can have -1 as an argument Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.146.2.4 retrieving revision 1.146.2.5 diff -u -d -r1.146.2.4 -r1.146.2.5 --- libos.tex 25 Jun 2005 18:47:28 -0000 1.146.2.4 +++ libos.tex 25 Jun 2005 18:53:24 -0000 1.146.2.5 @@ -750,7 +750,7 @@ \begin{funcdesc}{chown}{path, uid, gid} Change the owner and group id of \var{path} to the numeric \var{uid} -and \var{gid}. +and \var{gid}. To leave one of the ids unchanged, set it to -1. Availability: Macintosh, \UNIX. \end{funcdesc} From birkenfeld at users.sourceforge.net Sat Jun 25 21:15:50 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 12:15:50 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libhttplib.tex, 1.38, 1.39 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv29772/Doc/lib Modified Files: libhttplib.tex Log Message: bug [ 1202475 ] httplib docs mentioning HTTPConnection.getreply Index: libhttplib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libhttplib.tex,v retrieving revision 1.38 retrieving revision 1.39 diff -u -d -r1.38 -r1.39 --- libhttplib.tex 18 Sep 2004 09:03:48 -0000 1.38 +++ libhttplib.tex 25 Jun 2005 19:15:48 -0000 1.39 @@ -304,6 +304,8 @@ \begin{methoddesc}{getresponse}{} Should be called after a request is sent to get the response from the server. Returns an \class{HTTPResponse} instance. +\note{Note that you must have read the whole response before you can send a new +request to the server.} \end{methoddesc} \begin{methoddesc}{set_debuglevel}{level} @@ -320,11 +322,9 @@ Close the connection to the server. \end{methoddesc} -\begin{methoddesc}{send}{data} -Send data to the server. This should be used directly only after the -\method{endheaders()} method has been called and before -\method{getreply()} has been called. -\end{methoddesc} +As an alternative to using the \method{request()} method described above, +you can also send your request step by step, by using the four functions +below. \begin{methoddesc}{putrequest}{request, selector\optional{, skip\_host\optional{, skip_accept_encoding}}} @@ -349,6 +349,11 @@ Send a blank line to the server, signalling the end of the headers. \end{methoddesc} +\begin{methoddesc}{send}{data} +Send data to the server. This should be used directly only after the +\method{endheaders()} method has been called and before +\method{getresponse()} is called. +\end{methoddesc} \subsection{HTTPResponse Objects \label{httpresponse-objects}} From birkenfeld at users.sourceforge.net Sat Jun 25 21:17:00 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 12:17:00 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libhttplib.tex, 1.38, 1.38.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30350/Doc/lib Modified Files: Tag: release24-maint libhttplib.tex Log Message: backport bug [ 1202475 ] httplib docs mentioning HTTPConnection.getreply Index: libhttplib.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libhttplib.tex,v retrieving revision 1.38 retrieving revision 1.38.2.1 diff -u -d -r1.38 -r1.38.2.1 --- libhttplib.tex 18 Sep 2004 09:03:48 -0000 1.38 +++ libhttplib.tex 25 Jun 2005 19:16:58 -0000 1.38.2.1 @@ -304,6 +304,8 @@ \begin{methoddesc}{getresponse}{} Should be called after a request is sent to get the response from the server. Returns an \class{HTTPResponse} instance. +\note{Note that you must have read the whole response before you can send a new +request to the server.} \end{methoddesc} \begin{methoddesc}{set_debuglevel}{level} @@ -320,11 +322,9 @@ Close the connection to the server. \end{methoddesc} -\begin{methoddesc}{send}{data} -Send data to the server. This should be used directly only after the -\method{endheaders()} method has been called and before -\method{getreply()} has been called. -\end{methoddesc} +As an alternative to using the \method{request()} method described above, +you can also send your request step by step, by using the four functions +below. \begin{methoddesc}{putrequest}{request, selector\optional{, skip\_host\optional{, skip_accept_encoding}}} @@ -349,6 +349,11 @@ Send a blank line to the server, signalling the end of the headers. \end{methoddesc} +\begin{methoddesc}{send}{data} +Send data to the server. This should be used directly only after the +\method{endheaders()} method has been called and before +\method{getresponse()} is called. +\end{methoddesc} \subsection{HTTPResponse Objects \label{httpresponse-objects}} From pje at users.sourceforge.net Sat Jun 25 21:33:08 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 25 Jun 2005 12:33:08 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools __init__.py, 1.8, 1.9 package_index.py, 1.6, 1.7 sandbox.py, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7438/setuptools Modified Files: __init__.py package_index.py sandbox.py Log Message: 0.5a3 bugfix release Index: __init__.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/__init__.py,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- __init__.py 15 Jun 2005 02:23:48 -0000 1.8 +++ __init__.py 25 Jun 2005 19:33:06 -0000 1.9 @@ -8,7 +8,7 @@ from distutils.util import convert_path import os.path -__version__ = '0.5a1' +__version__ = '0.5a3' __all__ = [ 'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require', Index: package_index.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/package_index.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- package_index.py 15 Jun 2005 02:39:10 -0000 1.6 +++ package_index.py 25 Jun 2005 19:33:06 -0000 1.7 @@ -23,7 +23,7 @@ if lower.endswith('.exe'): if lower.endswith('.win32.exe'): base = name[:-10] - elif lower[-16:].startswith('.win32-py'): + elif lower.startswith('.win32-py',-16): py_ver = name[-7:-4] base = name[:-16] Index: sandbox.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/sandbox.py,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- sandbox.py 12 Jun 2005 03:07:53 -0000 1.2 +++ sandbox.py 25 Jun 2005 19:33:06 -0000 1.3 @@ -1,13 +1,11 @@ -import os, sys, __builtin__ +import os, sys, __builtin__, tempfile _os = sys.modules[os.name] _open = open - __all__ = [ "AbstractSandbox", "DirectorySandbox", "SandboxViolation", "run_setup", ] - def run_setup(setup_script, args): """Run a distutils setup script, sandboxed in its directory""" @@ -15,8 +13,12 @@ save_argv = sys.argv[:] save_path = sys.path[:] setup_dir = os.path.abspath(os.path.dirname(setup_script)) + temp_dir = os.path.join(setup_dir,'temp') + if not os.path.isdir(temp_dir): os.makedirs(temp_dir) + save_tmp = tempfile.tempdir try: + tempfile.tempdir = temp_dir os.chdir(setup_dir) try: sys.argv[:] = [setup_script]+list(args) @@ -35,9 +37,7 @@ os.chdir(old_dir) sys.path[:] = save_path sys.argv[:] = save_argv - - - + tempfile.tempdir = save_tmp class AbstractSandbox: """Wrap 'os' module and 'open()' builtin for virtualizing setup scripts""" From pje at users.sourceforge.net Sat Jun 25 21:33:08 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 25 Jun 2005 12:33:08 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.14, 1.15 easy_install.py, 1.22, 1.23 ez_setup.py, 1.5, 1.6 setup.py, 1.15, 1.16 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7438 Modified Files: EasyInstall.txt easy_install.py ez_setup.py setup.py Log Message: 0.5a3 bugfix release Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.14 retrieving revision 1.15 diff -u -d -r1.14 -r1.15 --- EasyInstall.txt 15 Jun 2005 02:23:47 -0000 1.14 +++ EasyInstall.txt 25 Jun 2005 19:33:06 -0000 1.15 @@ -23,7 +23,7 @@ ------------------------- Windows users can just download and run the `setuptools binary installer for -Windows `_. +Windows `_. All others should just download `ez_setup.py `_, and run it; this will download and install the correct version of ``setuptools`` for your Python @@ -62,7 +62,7 @@ **Example 2**. Install or upgrade a package by name and version by finding links on a given "download page":: - easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a1" + easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a3" **Example 3**. Download a source distribution from a specified URL, automatically building and installing it:: @@ -442,6 +442,15 @@ * There's no automatic retry for borked Sourceforge mirrors, which can easily time out or be missing a file. +0.5a3 + * Fixed not setting script permissions to allow execution. + + * Improved sandboxing so that setup scripts that want a temporary directory + (e.g. pychecker) can still run in the sandbox. + +0.5a2 + * Fix stupid stupid refactoring-at-the-last-minute typos. :( + 0.5a1 * Added support for converting ``.win32.exe`` installers to eggs on the fly. EasyInstall will now recognize such files by name and install them. Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.22 retrieving revision 1.23 diff -u -d -r1.22 -r1.23 --- easy_install.py 15 Jun 2005 02:23:47 -0000 1.22 +++ easy_install.py 25 Jun 2005 19:33:06 -0000 1.23 @@ -238,10 +238,10 @@ f = open(target,"w") f.write(script_text) f.close() - - - - + try: + os.chmod(target,0755) + except (AttributeError, os.error): + pass def install_eggs(self, dist_filename, zip_ok, tmpdir): Index: ez_setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/ez_setup.py,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- ez_setup.py 15 Jun 2005 02:23:48 -0000 1.5 +++ ez_setup.py 25 Jun 2005 19:33:06 -0000 1.6 @@ -14,7 +14,7 @@ This file can also be run as a script to install or upgrade setuptools. """ -DEFAULT_VERSION = "0.5a1" +DEFAULT_VERSION = "0.5a3" DEFAULT_URL = "http://peak.telecommunity.com/dist/" import sys, os Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- setup.py 15 Jun 2005 02:23:48 -0000 1.15 +++ setup.py 25 Jun 2005 19:33:06 -0000 1.16 @@ -1,7 +1,7 @@ #!/usr/bin/env python """Distutils setup file, used to install or test 'setuptools'""" -VERSION = "0.5a1" +VERSION = "0.5a3" from setuptools import setup, find_packages, Require setup( @@ -26,7 +26,7 @@ "close. See the home page and download page for details and docs.", keywords = "CPAN PyPI distutils eggs package management", - url = "http://peak.telecommunity.com/PythonEggs", + url = "http://peak.telecommunity.com/DevCenter/PythonEggs", download_url = "http://peak.telecommunity.com/DevCenter/EasyInstall", test_suite = 'setuptools.tests.test_suite', From birkenfeld at users.sourceforge.net Sat Jun 25 21:55:07 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 12:55:07 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.159,1.160 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18256/Doc/lib Modified Files: libos.tex Log Message: Bug [ 1190563 ] os.waitpid docs don't specify return value for WNOHANG Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.159 retrieving revision 1.160 diff -u -d -r1.159 -r1.160 --- libos.tex 25 Jun 2005 18:52:24 -0000 1.159 +++ libos.tex 25 Jun 2005 19:55:04 -0000 1.160 @@ -1680,8 +1680,9 @@ \end{funcdesc} \begin{datadesc}{WNOHANG} -The option for \function{waitpid()} to avoid hanging if no child -process status is available immediately. +The option for \function{waitpid()} to return immediately if no child +process status is available immediately. The function returns +\code{(0, 0)} in this case. Availability: Macintosh, \UNIX. \end{datadesc} From birkenfeld at users.sourceforge.net Sat Jun 25 21:55:40 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 12:55:40 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex, 1.146.2.5, 1.146.2.6 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18436/Doc/lib Modified Files: Tag: release24-maint libos.tex Log Message: backport [ 1190563 ] os.waitpid docs don't specify return value for WNOHANG Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.146.2.5 retrieving revision 1.146.2.6 diff -u -d -r1.146.2.5 -r1.146.2.6 --- libos.tex 25 Jun 2005 18:53:24 -0000 1.146.2.5 +++ libos.tex 25 Jun 2005 19:55:37 -0000 1.146.2.6 @@ -1657,8 +1657,9 @@ \end{funcdesc} \begin{datadesc}{WNOHANG} -The option for \function{waitpid()} to avoid hanging if no child -process status is available immediately. +The option for \function{waitpid()} to return immediately if no child +process status is available immediately. The function returns +\code{(0, 0)} in this case. Availability: Macintosh, \UNIX. \end{datadesc} From birkenfeld at users.sourceforge.net Sat Jun 25 22:07:39 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 13:07:39 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libfuncs.tex, 1.183, 1.184 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23986/Doc/lib Modified Files: libfuncs.tex Log Message: bug [ 1175022 ] property example code error Index: libfuncs.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libfuncs.tex,v retrieving revision 1.183 retrieving revision 1.184 diff -u -d -r1.183 -r1.184 --- libfuncs.tex 31 May 2005 11:04:00 -0000 1.183 +++ libfuncs.tex 25 Jun 2005 20:07:36 -0000 1.184 @@ -755,6 +755,7 @@ \begin{verbatim} class C(object): + def __init__(self): self.__x = None def getx(self): return self.__x def setx(self, value): self.__x = value def delx(self): del self.__x From birkenfeld at users.sourceforge.net Sat Jun 25 22:07:41 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 13:07:41 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libfuncs.tex, 1.175.2.3, 1.175.2.4 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23926/Doc/lib Modified Files: Tag: release24-maint libfuncs.tex Log Message: backport bug [ 1175022 ] property example code error Index: libfuncs.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libfuncs.tex,v retrieving revision 1.175.2.3 retrieving revision 1.175.2.4 diff -u -d -r1.175.2.3 -r1.175.2.4 --- libfuncs.tex 25 May 2005 05:42:51 -0000 1.175.2.3 +++ libfuncs.tex 25 Jun 2005 20:07:39 -0000 1.175.2.4 @@ -717,6 +717,7 @@ \begin{verbatim} class C(object): + def __init__(self): self.__x = None def getx(self): return self.__x def setx(self, value): self.__x = value def delx(self): del self.__x From birkenfeld at users.sourceforge.net Sat Jun 25 22:44:13 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 13:44:13 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.160,1.161 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11347/lib Modified Files: libos.tex Log Message: bug [ 1186072 ] tempnam doc doesn't include link to tmpfile Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.160 retrieving revision 1.161 diff -u -d -r1.160 -r1.161 --- libos.tex 25 Jun 2005 19:55:04 -0000 1.160 +++ libos.tex 25 Jun 2005 20:44:10 -0000 1.161 @@ -1101,8 +1101,8 @@ behavior of this function depends on the C library implementation; some aspects are underspecified in system documentation. \warning{Use of \function{tempnam()} is vulnerable to symlink attacks; -consider using \function{tmpfile()} instead.} -Availability: Macintosh, \UNIX, Windows. +consider using \function{tmpfile()} (section \ref{os-newstreams}) +instead.} Availability: Macintosh, \UNIX, Windows. \end{funcdesc} \begin{funcdesc}{tmpnam}{} @@ -1113,12 +1113,13 @@ paths returned by \function{tmpnam()}; no automatic cleanup is provided. \warning{Use of \function{tmpnam()} is vulnerable to symlink attacks; -consider using \function{tmpfile()} instead.} -Availability: \UNIX, Windows. This function probably shouldn't be used -on Windows, though: Microsoft's implementation of \function{tmpnam()} -always creates a name in the root directory of the current drive, and -that's generally a poor location for a temp file (depending on -privileges, you may not even be able to open a file using this name). +consider using \function{tmpfile()} (section \ref{os-newstreams}) +instead.} Availability: \UNIX, Windows. This function probably +shouldn't be used on Windows, though: Microsoft's implementation of +\function{tmpnam()} always creates a name in the root directory of the +current drive, and that's generally a poor location for a temp file +(depending on privileges, you may not even be able to open a file +using this name). \end{funcdesc} \begin{datadesc}{TMP_MAX} From birkenfeld at users.sourceforge.net Sat Jun 25 22:44:15 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 13:44:15 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex, 1.146.2.6, 1.146.2.7 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv11414/Doc/lib Modified Files: Tag: release24-maint libos.tex Log Message: backport bug [ 1186072 ] tempnam doc doesn't include link to tmpfile Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.146.2.6 retrieving revision 1.146.2.7 diff -u -d -r1.146.2.6 -r1.146.2.7 --- libos.tex 25 Jun 2005 19:55:37 -0000 1.146.2.6 +++ libos.tex 25 Jun 2005 20:44:12 -0000 1.146.2.7 @@ -1078,8 +1078,8 @@ behavior of this function depends on the C library implementation; some aspects are underspecified in system documentation. \warning{Use of \function{tempnam()} is vulnerable to symlink attacks; -consider using \function{tmpfile()} instead.} -Availability: Macintosh, \UNIX, Windows. +consider using \function{tmpfile()} (section \ref{os-newstreams}) +instead.} Availability: Macintosh, \UNIX, Windows. \end{funcdesc} \begin{funcdesc}{tmpnam}{} @@ -1090,12 +1090,13 @@ paths returned by \function{tmpnam()}; no automatic cleanup is provided. \warning{Use of \function{tmpnam()} is vulnerable to symlink attacks; -consider using \function{tmpfile()} instead.} -Availability: \UNIX, Windows. This function probably shouldn't be used -on Windows, though: Microsoft's implementation of \function{tmpnam()} -always creates a name in the root directory of the current drive, and -that's generally a poor location for a temp file (depending on -privileges, you may not even be able to open a file using this name). +consider using \function{tmpfile()} (section \ref{os-newstreams}) +instead.} Availability: \UNIX, Windows. This function probably +shouldn't be used on Windows, though: Microsoft's implementation of +\function{tmpnam()} always creates a name in the root directory of the +current drive, and that's generally a poor location for a temp file +(depending on privileges, you may not even be able to open a file +using this name). \end{funcdesc} \begin{datadesc}{TMP_MAX} From birkenfeld at users.sourceforge.net Sat Jun 25 23:03:54 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 14:03:54 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libuserdict.tex, 1.26, 1.27 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22192/Doc/lib Modified Files: libuserdict.tex Log Message: bug [ 1166582 ] IterableUserDict not in docs Index: libuserdict.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libuserdict.tex,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- libuserdict.tex 4 Jan 2005 21:25:00 -0000 1.26 +++ libuserdict.tex 25 Jun 2005 21:03:52 -0000 1.27 @@ -21,18 +21,24 @@ The \module{UserDict} module defines the \class{UserDict} class and \class{DictMixin}: -\begin{classdesc}{UserDict}{\optional{initialdata}} -Class that simulates a dictionary. The instance's -contents are kept in a regular dictionary, which is accessible via the -\member{data} attribute of \class{UserDict} instances. If -\var{initialdata} is provided, \member{data} is initialized with its -contents; note that a reference to \var{initialdata} will not be kept, -allowing it be used for other purposes. +\begin{classdesc}{UserDict}{\optional{initialdata}} +Class that simulates a dictionary. The instance's contents are kept +in a regular dictionary, which is accessible via the \member{data} +attribute of \class{UserDict} instances. If \var{initialdata} is +provided, \member{data} is initialized with its contents; note that a +reference to \var{initialdata} will not be kept, allowing it be used +for other purposes. \note{For backward compatibility, instances of +\class{UserDict} are not iterable.} +\end{classdesc} + +\begin{classdesc}{IterableUserDict}{\optional{initialdata}} +Subclass of \class{UserDict} that supports direct iteration (e.g. +\code{for key in myDict}). \end{classdesc} In addition to supporting the methods and operations of mappings (see -section \ref{typesmapping}), \class{UserDict} instances provide the -following attribute: +section \ref{typesmapping}), \class{UserDict} and +\class{IterableUserDict} instances provide the following attribute: \begin{memberdesc}{data} A real dictionary used to store the contents of the \class{UserDict} From birkenfeld at users.sourceforge.net Sat Jun 25 23:04:00 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 14:04:00 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libuserdict.tex, 1.24.4.2, 1.24.4.3 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22219/Doc/lib Modified Files: Tag: release24-maint libuserdict.tex Log Message: backport bug [ 1166582 ] IterableUserDict not in docs Index: libuserdict.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libuserdict.tex,v retrieving revision 1.24.4.2 retrieving revision 1.24.4.3 diff -u -d -r1.24.4.2 -r1.24.4.3 --- libuserdict.tex 4 Jan 2005 21:28:48 -0000 1.24.4.2 +++ libuserdict.tex 25 Jun 2005 21:03:57 -0000 1.24.4.3 @@ -27,12 +27,18 @@ \member{data} attribute of \class{UserDict} instances. If \var{initialdata} is provided, \member{data} is initialized with its contents; note that a reference to \var{initialdata} will not be kept, -allowing it be used for other purposes. +allowing it be used for other purposes. \note{For backward compatibility, +instances of \class{UserDict} are not iterable.} +\end{classdesc} + +\begin{classdesc}{IterableUserDict}{\optional{initialdata}} +Subclass of \class{UserDict} that supports direct iteration (e.g. +\code{for key in myDict}). \end{classdesc} In addition to supporting the methods and operations of mappings (see -section \ref{typesmapping}), \class{UserDict} instances provide the -following attribute: +section \ref{typesmapping}), \class{UserDict} and +\class{IterableUserDict} instances provide the following attribute: \begin{memberdesc}{data} A real dictionary used to store the contents of the \class{UserDict} From birkenfeld at users.sourceforge.net Sat Jun 25 23:08:49 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sat, 25 Jun 2005 14:08:49 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1302,1.1303 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv24181/Misc Modified Files: NEWS Log Message: NEWS entry for last bugfix Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1302 retrieving revision 1.1303 diff -u -d -r1.1302 -r1.1303 --- NEWS 21 Jun 2005 07:43:56 -0000 1.1302 +++ NEWS 25 Jun 2005 21:08:46 -0000 1.1303 @@ -370,6 +370,9 @@ Documentation ------------- +- Document the IterableUserDict class in the UserDict module. + Closes bug #1166582. + - Remove all latent references for "Macintosh" that referred to semantics for Mac OS 9 and change to reflect the state for OS X. Closes patch #1095802. Thanks Jack Jansen. From pje at users.sourceforge.net Sun Jun 26 04:18:43 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 25 Jun 2005 19:18:43 -0700 Subject: [Python-checkins] python/nondist/peps pep-0342.txt,1.4,1.5 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv16919 Modified Files: pep-0342.txt Log Message: PEP 342: Update and assume co-authorship, as directed by Guido in private email, merging the implemented features from PEP 343 and dropping most of the PEP 340 holdover features such as __next__, the next() builtin, and 'continue EXPR'. Also, added more motivation and examples, giving lots of credit to PEPs 288 and 325, where most of these ideas were first thought of and initially fleshed out. Index: pep-0342.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0342.txt,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- pep-0342.txt 14 Jun 2005 15:14:01 -0000 1.4 +++ pep-0342.txt 26 Jun 2005 02:18:40 -0000 1.5 @@ -1,8 +1,8 @@ PEP: 342 -Title: Coroutines via Enhanced Iterators +Title: Coroutines via Enhanced Generators Version: $Revision$ Last-Modified: $Date$ -Author: Guido van Rossum +Author: Guido van Rossum, Phillip J. Eby Status: Draft Type: Standards Track Content-Type: text/plain @@ -11,118 +11,150 @@ Introduction - This PEP proposes a new iterator API that allows values to be - passed into an iterator using "continue EXPR". These values are - received in the iterator as an argument to the new __next__ - method, and can be accessed in a generator with a - yield-expression. - - The content of this PEP is derived from the original content of - PEP 340, broken off into its own PEP as the new iterator API is - pretty much orthogonal from the anonymous block statement - discussion. Thanks to Steven Bethard for doing the editing. - - Update: at this point I'm leaning towards preferring next() over - __next__() again, but I've no time to update the PEP right now. - I've changed the title to Coroutines via Enhanced Iterators at - Timothy Delaney's suggestion. - -Motivation and Summary + This PEP proposes some enhancements to the API and syntax of + generators, to make them usable as simple coroutines. It is + basically a combination of ideas from these two PEPs, which + may be considered redundant if this PEP is accepted: - TBD. + - PEP 288, Generators Attributes and Exceptions. The current PEP + covers its second half, generator exceptions (in fact the + throw() method name was taken from PEP 288). PEP 342 replaces + generator attributes, however, with a concept from an earlier + revision of PEP 288, the "yield expression". -Use Cases + - PEP 325, Resource-Release Support for Generators. PEP 342 + ties up a few loose ends in the PEP 325 spec, to make it suitable + for actual implementation. - See the Examples section near the end. +Motivation -Specification: the __next__() Method + Coroutines are a natural way of expressing many algorithms, such as + simulations, games, asynchronous I/O, and other forms of event- + driven programming or co-operative multitasking. Python's generator + functions are almost coroutines -- but not quite -- in that they + allow pausing execution to produce a value, but do not provide for + values or exceptions to be passed in when execution resumes. They + also do not allow execution to be paused within the "try" portion of + try/finally blocks, and therefore make it difficult for an aborted + coroutine to clean up after itself. - A new method for iterators is proposed, called __next__(). It - takes one optional argument, which defaults to None. Calling the - __next__() method without argument or with None is equivalent to - using the old iterator API, next(). For backwards compatibility, - it is recommended that iterators also implement a next() method as - an alias for calling the __next__() method without an argument. + Generators also cannot yield control while other functions are + executing, unless those functions are also expressed as generators, + and the outer generator is written to yield in response to values + yielded by the inner generator, which complicates the implementation + of even relatively simple use cases like asynchronous + communications, because calling any functions either requires the + generator to "block" (i.e. be unable to yield control), or else a + lot of boilerplate looping code around every needed function call. - The argument to the __next__() method may be used by the iterator - as a hint on what to do next. + However, if it were possible to pass values or exceptions into a + generator at the point where it was suspended, a simple co-routine + scheduler or "trampoline function" would let coroutines "call" each + other without blocking -- a tremendous boon for asynchronous + applications. Such applications could then write co-routines to + do non-blocking socket I/O by yielding control to an I/O scheduler + until data has been sent or becomes available. Meanwhile, code that + performs the I/O would simply do something like: -Specification: the next() Built-in Function + data = (yield nonblocking_read(my_socket, nbytes)) - This is a built-in function defined as follows: + In order to pause execution until the nonblocking_read() coroutine + produced a value. - def next(itr, arg=None): - nxt = getattr(itr, "__next__", None) - if nxt is not None: - return nxt(arg) - if arg is None: - return itr.next() - raise TypeError("next() with arg for old-style iterator") + In other words, with a few relatively minor enhancements to the + language and the implementation of the generator-iterator type, + Python will be able to support writing asynchronous applications + without needing to write entire applications as a series of + callbacks, and without requiring the use resource-intensive threads + for programs that need hundreds or even thousands of co-operatively + multitasking pseudothreads. In a sense, these enhancements will + give Python many of the benefits of the Stackless Python fork, + without requiring any significant modification to the CPython core + or its APIs. In addition, these enhancements should be readily + implementable by any Python implementation (such as Jython) that + already supports generators. - This function is proposed because there is often a need to call - the next() method outside a for-loop; the new API, and the - backwards compatibility code, is too ugly to have to repeat in - user code. -Specification: a Change to the 'for' Loop +Specification Summary - A small change in the translation of the for-loop is proposed. - The statement + By adding a few simple methods to the generator-iterator type, and + with two minor syntax adjustments, Python developers will be able + to use generator functions to implement co-routines and other forms + of co-operative multitasking. These method and adjustments are: - for VAR1 in EXPR1: - BLOCK1 - else: - BLOCK2 + 1. Redefine "yield" to be an expression, rather than a statement. + The current yield statement would become a yield expression + whose value is thrown away. A yield expression's value is + None if the generator is resumed by a normal next() call. - will be translated as follows: + 2. Add a new send() method for generator-iterators, which resumes + the generator and "sends" a value that becomes the result of the + current "yield expression". The send() method returns the next + value yielded by the generator, or raises StopIteration if the + generator exits without yielding another value. - itr = iter(EXPR1) - arg = None # Set by "continue EXPR2", see below - brk = False - while True: - try: - VAR1 = next(itr, arg) - except StopIteration: - brk = True - break - arg = None - BLOCK1 - if brk: - BLOCK2 + 3. Add a new throw() method for generator-iterators, which raises + an exception at the point where the generator was paused, and + which returns the next value yielded by the generator, raising + StopIteration if the generator exits without yielding another + value. (If the generator does not catch the passed-in exception, + or raises a different exception, then that exception propagates + to the caller. - (However, the variables 'itr' etc. are not user-visible and the - built-in names used cannot be overridden by the user.) + 4. Add a close() method for generator-iterators, which raises + GeneratorExit at the point where the generator was paused. If + the generator then raises StopIteration (by exiting normally, or + due to already being closed) or GeneratorExit (by not catching + the exception), close() returns to its caller. If the generator + yields a value, a RuntimeError is raised. If the generator + raises any other exception, it is propagated to the caller. + close() does nothing if the generator has already exited due to + an exception or normal exit. -Specification: the Extended 'continue' Statement + 5. Adding support to ensure that close() is called when a generator + iterator is garbage-collected. - In the translation of the for-loop, inside BLOCK1, the new syntax + 6. Allowing "yield" to be used in try/finally blocks, since garbage + collection or an explicit close() call allows the finally clause + to execute. - continue EXPR2 + A prototype patch implementing all of these changes against the + current Python CVS HEAD is available as SourceForge patch #1223381 + (http://python.org/sf/1223381). - is legal and is translated into - arg = EXPR2 - continue +Specification: Sending Values into Generators - (Where 'arg' references the corresponding hidden variable from the - previous section.) + New generator method: send(value) - This is also the case in the body of the block-statement proposed - below. + A new method for generator-iterators is proposed, called send(). It + takes exactly one argument, which is the value that should be "sent + in" to the generator. Calling send(None) is exactly equivalent to + calling a generator's next() method. Calling send() with any other + value is the same, except that the value produced by the generator's + current yield expression will be different. - EXPR2 may contain commas; "continue 1, 2, 3" is equivalent to - "continue (1, 2, 3)". + Because generator-iterators begin execution at the top of the + generator's function body, there is no yield expression to receive + a value when the generator has just been created. Therefore, + calling send() with a non-None argument is prohibited when the + generator iterator has just started, and a TypeError is raised if + this occurs (presumably due to a logic error of some kind). Thus, + before you can communicate with a coroutine you must call first + call next() or send(None) to advance its execution to its first + yield expression. -Specification: Generators and Yield-Expressions + As with the next() method, the send() method returns the next value + yielded by the generator-iterator, or raises StopIteration if the + generator exits normally, or has already exited. If the generator + raises an exception, it is propagated to send()'s caller. - Generators will implement the new __next__() method API, as well - as the old argument-less next() method which becomes an alias for - calling __next__() without an argument. + New syntax: Yield Expressions The yield-statement will be allowed to be used on the right-hand side of an assignment; in that case it is referred to as yield-expression. The value of this yield-expression is None - unless __next__() was called with an argument; see below. + unless send() was called with a non-None argument; see below. A yield-expression must always be parenthesized except when it occurs at the top-level expression on the right-hand side of an @@ -151,57 +183,318 @@ yield without passing an explicit value ("yield" is of course equivalent to "yield None"). - When __next__() is called with an argument that is not None, the - yield-expression that it resumes will return the argument. If it - resumes a yield-statement, the value is ignored (this is similar - to ignoring the value returned by a function call). When the - *initial* call to __next__() receives an argument that is not - None, TypeError is raised; this is likely caused by some logic - error. When __next__() is called without an argument or with None - as argument, and a yield-expression is resumed, the - yield-expression returns None. + When send(value) is called, the yield-expression that it resumes + will return the passed-in value. When next() is called, the resumed + yield-expression will return None. If the yield-expression is a + yield-statement, this returned value is ignored, similar to ignoring + the value returned by a function call used as a statement. + + In effect, a yield-expression is like an inverted function call; the + argument to yield is in fact returned (yielded) from the currently + executing function, and the "return value" of yield is the argument + passed in via send(). Note: the syntactic extensions to yield make its use very similar to that in Ruby. This is intentional. Do note that in Python the - block passes a value to the generator using "continue EXPR" rather + block passes a value to the generator using "send(EXPR)" rather than "return EXPR", and the underlying mechanism whereby control is passed between the generator and the block is completely different. Blocks in Python are not compiled into thunks; rather, yield suspends execution of the generator's frame. Some edge cases work differently; in Python, you cannot save the block for later use, and you cannot test whether there is a block or not. + (XXX - this stuff about blocks seems out of place now, perhaps + Guido can edit to clarify.) -Alternative +Specification: Exceptions and Cleanup - An alternative proposal is still under consideration, where - instead of adding a __next__() method, the existing next() method - is given an optional argument. The next() built-in function is - then unnecessary. The only line that changes in the translation is - the line + Let a generator object be the iterator produced by calling a + generator function. Below, 'g' always refers to a generator + object. - VAR1 = next(itr, arg) + New syntax: yield allowed inside try-finally - which will be replaced by this + The syntax for generator functions is extended to allow a + yield-statement inside a try-finally statement. - if arg is None: - VAR1 = itr.next() - else: - VAR1 = itr.next(arg) + New generator method: throw(type, value=None, traceback=None) - If "continue EXPR2" is used and EXPR2 does not evaluate to None, - and the iterator's next() method does not support the optional - argument, a TypeError exception will be raised, which is the same - behavior as above. + g.throw(type, value, traceback) causes the specified exception to + be thrown at the point where the generator g is currently + suspended (i.e. at a yield-statement, or at the start of its + function body if next() has not been called yet). If the + generator catches the exception and yields another value, that is + the return value of g.throw(). If it doesn't catch the exception, + the throw() appears to raise the same exception passed it (it + "falls through"). If the generator raises another exception (this + includes the StopIteration produced when it returns) that + exception is raised by the throw() call. In summary, throw() + behaves like next() or send(), except it raises an exception at the + suspension point. If the generator is already in the closed + state, throw() just raises the exception it was passed without + executing any of the generator's code. - This proposal is more compatible (no new method name, no new - built-in needed) but less future-proof; in some sense it was a - mistake to call this method next() instead of __next__(), since - *all* other operations corresponding to function pointers in the C - type structure have names with leading and trailing underscores. + The effect of raising the exception is exactly as if the + statement: + + raise type, value, traceback + + was executed at the suspension point. The type argument must + not be None, and the type and value must be compatible. If the + value is not an instance of the type, a new exception instance + is created, with the value passed in as argument(s), following + the same rules that the raise statement uses to create an + exception instance. The traceback, if supplied, must be a valid + Python traceback object, or a TypeError occurs. + + New standard exception: GeneratorExit + + A new standard exception is defined, GeneratorExit, inheriting + from Exception. A generator should handle this by re-raising it + or by raising StopIteration. + + New generator method: close() + + g.close() is defined by the following pseudo-code: + + def close(self): + try: + self.throw(GeneratorExit) + except (GeneratorExit, StopIteration): + pass + else: + raise RuntimeError("generator ignored GeneratorExit") + # Other exceptions are not caught + + New generator method: __del__() + + g.__del__() is a wrapper for g.close(). This will be called when + the generator object is garbage-collected (in CPython, this is + when its reference count goes to zero). If close() raises an + exception, a traceback for the exception is printed to sys.stderr + and further ignored; it is not propagated back to the place that + triggered the garbage collection. This is consistent with the + handling of exceptions in __del__() methods on class instances. + + If the generator object participates in a cycle, g.__del__() may + not be called. This is the behavior of CPython's current garbage + collector. The reason for the restriction is that the GC code + needs to "break" a cycle at an arbitrary point in order to collect + it, and from then on no Python code should be allowed to see the + objects that formed the cycle, as they may be in an invalid state. + Objects "hanging off" a cycle are not subject to this restriction. + Note that it is unlikely to see a generator object participate in + a cycle in practice. However, storing a generator object in a + global variable creates a cycle via the generator frame's + f_globals pointer. Another way to create a cycle would be to + store a reference to the generator object in a data structure that + is passed to the generator as an argument. Neither of these cases + are very likely given the typical pattern of generator use. + + Also, in the CPython implementation of this PEP, the frame object + used by the generator should be released whenever its execution is + terminated due to an error or normal exit. This will ensure that + generators that cannot be resumed do not remain part of an + uncollectable reference cycle. This allows other code to + potentially use close() in a try/finally or "with" block (per PEP + 343) to ensure that a given generator is properly finalized. + +Optional Extensions + + The Extended 'continue' Statement + + An earlier draft of this PEP proposed a new "continue EXPR" + syntax for use in for-loops (carried over from PEP 340), that + would pass the value of EXPR into the iterator being looped over. + This feature has been withdrawn for the time being, because the + scope of this PEP has been narrowed to focus only on passing values + into generator-iterators, and not other kinds of iterators. It + was also felt by some on the Python-Dev list that adding new syntax + for this particular feature would be premature at best. + +Open Issues + + Discussion on python-dev has revealed some open issues. I list + them here, with my preferred resolution and its motivation. The + PEP as currently written reflects this preferred resolution. + + 1. What exception should be raised by close() when the generator + yields another value as a response to the GeneratorExit + exception? + + I originally chose TypeError because it represents gross + misbehavior of the generator function, which should be fixed by + changing the code. But the with_template decorator class in + PEP 343 uses RuntimeError for similar offenses. Arguably they + should all use the same exception. I'd rather not introduce a + new exception class just for this purpose, since it's not an + exception that I want people to catch: I want it to turn into a + traceback which is seen by the programmer who then fixes the + code. So now I believe they should both raise RuntimeError. + There are some precedents for that: it's raised by the core + Python code in situations where endless recursion is detected, + and for uninitialized objects (and for a variety of + miscellaneous conditions). + + 2. Oren Tirosh has proposed renaming the send() method to feed(), + for compatibility with the "consumer interface" (see + http://effbot.org/zone/consumer.htm for the specification.) + + However, looking more closely at the consumer interface, it seems + that the desired semantics for feed() are different than for + send(), because send() can't be meaningfully called on a just- + started generator. Also, the consumer interface as currently + defined doesn't include handling for StopIteration. + + Therefore, it seems like it would probably be more useful to + create a simple decorator that wraps a generator function to make + it conform to the consumer interface. For example, it could + "warm up" the generator with an initial next() call, trap + StopIteration, and perhaps even provide reset() by re-invoking + the generator function. + +Examples + + 1. A simple co-routine scheduler or "trampoline" that lets + coroutines "call" other coroutines by yielding the coroutine + they wish to invoke. Any non-generator value yielded by + a coroutine is returned to the coroutine that "called" the + one yielding the value. Similarly, if a coroutine raises an + exception, the exception is propagated to its "caller". In + effect, this example emulates simple tasklets as are used + in Stackless Python, as long as you use a yield expression to + invoke routines that would otherwise "block". This is only + a very simple example, and far more sophisticated schedulers + are possible. (For example, the existing GTasklet framework + for Python (http://www.gnome.org/~gjc/gtasklet/gtasklets.html) + and the peak.events framework (http://peak.telecommunity.com/) + already implement similar scheduling capabilities, but must + currently use awkward workarounds for the inability to pass + values or exceptions into generators.) + + import collections + + class Trampoline: + """Manage communications between coroutines until + + running = False + + def __init__(self): + self.queue = collections.deque + + def add(self, coroutine): + """Request that a coroutine be executed""" + self.schedule(coroutine) + + def run(self): + result = None + self.running = True + try: + while self.running and self.queue: + func = self.queue.popleft() + result = func() + return result + finally: + self.running = False + + def stop(self): + self.running = False + + def schedule(self, coroutine, stack=(), value=None, *exc): + def resume(): + try: + if exc: + value = coroutine.throw(value,*exc) + else: + value = coroutine.send(value) + except: + if stack: + # send the error back to the "caller" + self.schedule( + stack[0], stack[1], *sys.exc_info() + ) + else: + # Nothing left in this pseudothread to + # handle it, let it propagate to the + # run loop + raise + + if isinstance(value, types.GeneratorType): + # Yielded to a specific coroutine, push the + # current one on the stack, and call the new + # one with no args + self.schedule(value, (coroutine,stack)) + + elif stack: + # Yielded a result, pop the stack and send the + # value to the caller + self.schedule(stack[0], stack[1], value) + + # else: this pseudothread has ended + + self.queue.append(resume) + + 2. A simple "echo" server, and code to run it using a trampoline + (presumes the existence of "nonblocking_read", + "nonblocking_write", and other I/O coroutines, that e.g. raise + ConnectionLost if the connection is closed): + + # coroutine function that echos data back on a connected + # socket + # + def echo_handler(sock): + while True: + try: + data = yield nonblocking_read(sock) + yield nonblocking_write(sock, data) + except ConnectionLost: + pass # exit normally if connection lost + + # coroutine function that listens for connections on a + # socket, and then launches a service "handler" coroutine + # to service the connection + # + def listen_on(trampoline, sock, handler): + while True: + # get the next incoming connection + connected_socket = yield nonblocking_accept(sock) + + # start another coroutine to handle the connection + trampoline.add( handler(connected_socket) ) + + # Create a scheduler to manage all our coroutines + t = Trampoline() + + # Create a coroutine instance to run the echo_handler on + # incoming connections + # + server = listen_on( + t, listening_socket("localhost","echo"), echo_handler + ) + + # Add the coroutine to the scheduler + t.add(server) + + # loop forever, accepting connections and servicing them + # "in parallel" + # + t.run() + + +Reference Implementation + + A prototype patch implementing all of the features described in this + PEP is available as SourceForge patch #1223381 + (http://python.org/sf/1223381). Acknowledgements - See Acknowledgements of PEP 340. + Raymond Hettinger (PEP 288) and Samuele Pedroni (PEP 325) first + formally proposed the ideas of communicating values or exceptions + into generators, and the ability to "close" generators. Timothy + Delaney suggested the title of this PEP, and Steven Bethard helped + edit a previous version. See also the Acknowledgements section + of PEP 340. References From pje at users.sourceforge.net Sun Jun 26 04:21:24 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sat, 25 Jun 2005 19:21:24 -0700 Subject: [Python-checkins] python/nondist/peps pep-0343.txt,1.25,1.26 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19319 Modified Files: pep-0343.txt Log Message: PEP 343: Remove functionality now covered under PEP 342. Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.25 retrieving revision 1.26 diff -u -d -r1.25 -r1.26 --- pep-0343.txt 17 Jun 2005 21:58:59 -0000 1.25 +++ pep-0343.txt 26 Jun 2005 02:21:21 -0000 1.26 @@ -26,19 +26,9 @@ If this PEP is approved, the following PEPs will be rejected due to overlap: - - PEP 288, Generators Attributes and Exceptions. The current PEP - covers its second half, generator exceptions (in fact the - throw() method name was taken from this PEP). I'm not in favor - of generator attributes, since they can easily be handled by - making the generator a method or by passing a mutable argument. - - PEP 310, Reliable Acquisition/Release Pairs. This is the original with-statement proposal. - - PEP 325, Resource-Release Support for Generators. The current - PEP covers this (in fact the close() method name was taken from - this PEP). - - PEP 319, Python Synchronize/Asynchronize Block. Its use cases can be covered by the current PEP by providing suitable with-statement controllers: for 'synchronize' we can use the @@ -262,91 +252,11 @@ non-local goto should be considered unexceptional for the purposes of a database transaction roll-back decision. -Specification: Generator Enhancements - - Let a generator object be the iterator produced by calling a - generator function. Below, 'g' always refers to a generator - object. - - New syntax: yield allowed inside try-finally - - The syntax for generator functions is extended to allow a - yield-statement inside a try-finally statement. - - New generator method: throw(type, value=None, traceback=None) - - g.throw(type, value, traceback) causes the specified exception to - be thrown at the point where the generator g is currently - suspended (i.e. at a yield-statement, or at the start of its - function body if next() has not been called yet). If the - generator catches the exception and yields another value, that is - the return value of g.throw(). If it doesn't catch the exception, - the throw() appears to raise the same exception passed it (it - "falls through"). If the generator raises another exception (this - includes the StopIteration produced when it returns) that - exception is raised by the throw() call. In summary, throw() - behaves like next() except it raises an exception at the - suspension point. If the generator is already in the closed - state, throw() just raises the exception it was passed without - executing any of the generator's code. - - The effect of raising the exception is exactly as if the - statement: - - raise type, value, traceback - - was executed at the suspension point. The type argument should - not be None. - - New standard exception: GeneratorExit - - A new standard exception is defined, GeneratorExit, inheriting - from Exception. A generator should handle this by re-raising it - or by raising StopIteration. - - New generator method: close() - - g.close() is defined by the following pseudo-code: - - def close(self): - try: - self.throw(GeneratorExit, GeneratorExit(), None) - except (GeneratorExit, StopIteration): - pass - else: - raise RuntimeError("generator ignored GeneratorExit") - # Other exceptions are not caught - - New generator method: __del__() - - g.__del__() is an alias for g.close(). This will be called when - the generator object is garbage-collected (in CPython, this is - when its reference count goes to zero). If close() raises an - exception, a traceback for the exception is printed to sys.stderr - and further ignored; it is not propagated back to the place that - triggered the garbage collection. This is consistent with the - handling of exceptions in __del__() methods on class instances. - - If the generator object participates in a cycle, g.__del__() may - not be called. This is the behavior of CPython's current garbage - collector. The reason for the restriction is that the GC code - needs to "break" a cycle at an arbitrary point in order to collect - it, and from then on no Python code should be allowed to see the - objects that formed the cycle, as they may be in an invalid state. - Objects "hanging off" a cycle are not subject to this restriction. - Note that it is unlikely to see a generator object participate in - a cycle in practice. However, storing a generator object in a - global variable creates a cycle via the generator frame's - f_globals pointer. Another way to create a cycle would be to - store a reference to the generator object in a data structure that - is passed to the generator as an argument. Neither of these cases - are very likely given the typical pattern of generator use. - Generator Decorator - It is possible to write a decorator that makes it possible to use - a generator that yields exactly once to control a with-statement. - Here's a sketch of such a decorator: + If PEP 342 is accepted, it will be possible to write a decorator + that makes it possible to use a generator that yields exactly once + to control a with-statement. Here's a sketch of such a decorator: class Wrapper(object): @@ -431,28 +341,9 @@ them here, with my preferred resolution and its motivation. The PEP as currently written reflects this preferred resolution. - 1. What exception should be raised by close() when the generator - yields another value as a response to the GeneratorExit - exception? - - I originally chose TypeError because it represents gross - misbehavior of the generator function, which should be fixed by - changing the code. But the with_template decorator class uses - RuntimeError for similar offenses. Arguably they should all - use the same exception. I'd rather not introduce a new - exception class just for this purpose, since it's not an - exception that I want people to catch: I want it to turn into a - traceback which is seen by the programmer who then fixes the - code. So now I believe they should both raise RuntimeError. - There are some precedents for that: it's raised by the core - Python code in situations where endless recursion is detected, - and for uninitialized objects (and for a variety of - miscellaneous conditions). - - 2. Both the generator close() method and the __exit__() method of - the with_template decorator class catch StopIteration and - consider it equivalent to re-raising the exception passed to - throw(). Is allowing StopIteration right here? + 1. The __exit__() method of the with_template decorator class catches + StopIteration and considers it equivalent to re-raising the exception + passed to throw(). Is allowing StopIteration right here? This is so that a generator doing cleanup depending on the exception thrown (like the transactional() example below) can From deheeskyjjnyo at ms72.url.com.tw Sun Jun 26 01:17:32 2005 From: deheeskyjjnyo at ms72.url.com.tw (´¿­è¼w ) Date: Sat, 25 Jun 2005 19:17:32 -0400 Subject: [Python-checkins] ?W??Mail?j?M?Q?? Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-checkins/attachments/20050625/3c341bda/attachment.htm From gvanrossum at users.sourceforge.net Sun Jun 26 16:10:54 2005 From: gvanrossum at users.sourceforge.net (gvanrossum@users.sourceforge.net) Date: Sun, 26 Jun 2005 07:10:54 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt,1.328,1.329 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv13531 Modified Files: pep-0000.txt Log Message: Correct title (again!) and authorship of PEP 342. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.328 retrieving revision 1.329 diff -u -d -r1.328 -r1.329 --- pep-0000.txt 20 Jun 2005 02:27:22 -0000 1.328 +++ pep-0000.txt 26 Jun 2005 14:10:44 -0000 1.329 @@ -105,7 +105,7 @@ S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan S 341 Unifying try-except and try-finally Birkenfeld - S 342 Coroutines via Enhanced Iterators GvR + S 342 Coroutines via Enhanced Generators GvR, Eby S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones @@ -383,7 +383,7 @@ I 339 How to Change CPython's Bytecode Cannon SR 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld - S 342 Coroutines via Enhanced Iterators GvR + S 342 Coroutines via Enhanced Generators GvR, Eby S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones From pje at users.sourceforge.net Sun Jun 26 17:20:56 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 26 Jun 2005 08:20:56 -0700 Subject: [Python-checkins] python/nondist/peps pep-0342.txt,1.5,1.6 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17813 Modified Files: pep-0342.txt Log Message: PEP 342: Fix misc. typos and grammar abuses I committed while short of sleep. :) Index: pep-0342.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0342.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- pep-0342.txt 26 Jun 2005 02:18:40 -0000 1.5 +++ pep-0342.txt 26 Jun 2005 15:20:53 -0000 1.6 @@ -38,58 +38,58 @@ try/finally blocks, and therefore make it difficult for an aborted coroutine to clean up after itself. - Generators also cannot yield control while other functions are - executing, unless those functions are also expressed as generators, - and the outer generator is written to yield in response to values - yielded by the inner generator, which complicates the implementation - of even relatively simple use cases like asynchronous + Also, generators cannot yield control while other functions are + executing, unless those functions are themselves expressed as + generators, and the outer generator is written to yield in response + to values yielded by the inner generator. This complicates the + implementation of even relatively simple use cases like asynchronous communications, because calling any functions either requires the generator to "block" (i.e. be unable to yield control), or else a - lot of boilerplate looping code around every needed function call. + lot of boilerplate looping code must be added around every needed + function call. - However, if it were possible to pass values or exceptions into a + However, if it were possible to pass values or exceptions *into* a generator at the point where it was suspended, a simple co-routine scheduler or "trampoline function" would let coroutines "call" each other without blocking -- a tremendous boon for asynchronous applications. Such applications could then write co-routines to do non-blocking socket I/O by yielding control to an I/O scheduler until data has been sent or becomes available. Meanwhile, code that - performs the I/O would simply do something like: + performs the I/O would simply do something like this: data = (yield nonblocking_read(my_socket, nbytes)) - In order to pause execution until the nonblocking_read() coroutine + in order to pause execution until the nonblocking_read() coroutine produced a value. In other words, with a few relatively minor enhancements to the - language and the implementation of the generator-iterator type, - Python will be able to support writing asynchronous applications - without needing to write entire applications as a series of - callbacks, and without requiring the use resource-intensive threads + language and to the implementation of the generator-iterator type, + Python will be able to support performing asynchronous operations + without needing to write the entire application as a series of + callbacks, and without requiring the use of resource-intensive threads for programs that need hundreds or even thousands of co-operatively - multitasking pseudothreads. In a sense, these enhancements will - give Python many of the benefits of the Stackless Python fork, + multitasking pseudothreads. Thus, these enhancements will give + standard Python many of the benefits of the Stackless Python fork, without requiring any significant modification to the CPython core or its APIs. In addition, these enhancements should be readily implementable by any Python implementation (such as Jython) that already supports generators. - Specification Summary By adding a few simple methods to the generator-iterator type, and with two minor syntax adjustments, Python developers will be able to use generator functions to implement co-routines and other forms - of co-operative multitasking. These method and adjustments are: + of co-operative multitasking. These methods and adjustments are: 1. Redefine "yield" to be an expression, rather than a statement. The current yield statement would become a yield expression whose value is thrown away. A yield expression's value is - None if the generator is resumed by a normal next() call. + None whenever the generator is resumed by a normal next() call. 2. Add a new send() method for generator-iterators, which resumes the generator and "sends" a value that becomes the result of the - current "yield expression". The send() method returns the next + current yield-expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value. @@ -99,7 +99,7 @@ StopIteration if the generator exits without yielding another value. (If the generator does not catch the passed-in exception, or raises a different exception, then that exception propagates - to the caller. + to the caller.) 4. Add a close() method for generator-iterators, which raises GeneratorExit at the point where the generator was paused. If @@ -111,12 +111,12 @@ close() does nothing if the generator has already exited due to an exception or normal exit. - 5. Adding support to ensure that close() is called when a generator + 5. Add support to ensure that close() is called when a generator iterator is garbage-collected. - 6. Allowing "yield" to be used in try/finally blocks, since garbage - collection or an explicit close() call allows the finally clause - to execute. + 6. Allow "yield" to be used in try/finally blocks, since garbage + collection or an explicit close() call would now allow the + finally clause to execute. A prototype patch implementing all of these changes against the current Python CVS HEAD is available as SourceForge patch #1223381 @@ -140,14 +140,14 @@ calling send() with a non-None argument is prohibited when the generator iterator has just started, and a TypeError is raised if this occurs (presumably due to a logic error of some kind). Thus, - before you can communicate with a coroutine you must call first - call next() or send(None) to advance its execution to its first - yield expression. + before you can communicate with a coroutine you must first call + next() or send(None) to advance its execution to the first yield + expression. As with the next() method, the send() method returns the next value yielded by the generator-iterator, or raises StopIteration if the generator exits normally, or has already exited. If the generator - raises an exception, it is propagated to send()'s caller. + raises an uncaught exception, it is propagated to send()'s caller. New syntax: Yield Expressions @@ -242,16 +242,16 @@ was executed at the suspension point. The type argument must not be None, and the type and value must be compatible. If the value is not an instance of the type, a new exception instance - is created, with the value passed in as argument(s), following - the same rules that the raise statement uses to create an - exception instance. The traceback, if supplied, must be a valid - Python traceback object, or a TypeError occurs. + is created using the value, following the same rules that the raise + statement uses to create an exception instance. The traceback, if + supplied, must be a valid Python traceback object, or a TypeError + occurs. New standard exception: GeneratorExit A new standard exception is defined, GeneratorExit, inheriting from Exception. A generator should handle this by re-raising it - or by raising StopIteration. + (or just not catching it) or by raising StopIteration. New generator method: close() @@ -283,13 +283,16 @@ it, and from then on no Python code should be allowed to see the objects that formed the cycle, as they may be in an invalid state. Objects "hanging off" a cycle are not subject to this restriction. + Note that it is unlikely to see a generator object participate in a cycle in practice. However, storing a generator object in a global variable creates a cycle via the generator frame's f_globals pointer. Another way to create a cycle would be to store a reference to the generator object in a data structure that - is passed to the generator as an argument. Neither of these cases - are very likely given the typical pattern of generator use. + is passed to the generator as an argument (e.g., if an object has + a method that's a generator, and keeps a reference to a running + iterator created by that method). Neither of these cases + are very likely given the typical patterns of generator use. Also, in the CPython implementation of this PEP, the frame object used by the generator should be released whenever its execution is @@ -375,12 +378,12 @@ import collections class Trampoline: - """Manage communications between coroutines until + """Manage communications between coroutines""" running = False def __init__(self): - self.queue = collections.deque + self.queue = collections.deque() def add(self, coroutine): """Request that a coroutine be executed""" From pje at users.sourceforge.net Sun Jun 26 18:29:17 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 26 Jun 2005 09:29:17 -0700 Subject: [Python-checkins] python/nondist/peps pep-0342.txt,1.6,1.7 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20371 Modified Files: pep-0342.txt Log Message: PEP 342: Add more examples based on ideas stolen from an early version of PEP 288. Also add some more explanation of the 'throw()' name, also stolen from PEP 288. :) Index: pep-0342.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0342.txt,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- pep-0342.txt 26 Jun 2005 15:20:53 -0000 1.6 +++ pep-0342.txt 26 Jun 2005 16:29:14 -0000 1.7 @@ -247,6 +247,18 @@ supplied, must be a valid Python traceback object, or a TypeError occurs. + Note: The name of the throw() method was selected for several + reasons. Raise is a keyword and so cannot be used as a method + name. Unlike raise (which immediately raises an exception from the + current execution point), throw() first resumes the generator, and + only then raises the exception. The word throw is suggestive of + putting the exception in another location, and is already associated + with exceptions in other languages. + + Alternative method names were considered: resolve(), signal(), + genraise(), raiseinto(), and flush(). None of these seem to fit + as well as throw(). + New standard exception: GeneratorExit A new standard exception is defined, GeneratorExit, inheriting @@ -358,7 +370,76 @@ Examples - 1. A simple co-routine scheduler or "trampoline" that lets + 1. A simple "consumer" decorator that makes a generator function + automatically advance to its first yield point when initially + called: + + def consumer(func): + def wrapper(*args,**kw): + gen = func(*args, **kw) + gen.next() + return gen + wrapper.__name__ = func.__name__ + wrapper.__dict__ = func.__dict__ + wrapper.__doc__ = func.__doc__ + return wrapper + + 2. An example of using the "consumer" decorator to create a + "reverse generator" that receives images and creates thumbnail + pages, sending them on to another consumer. Functions like + this can be chained together to form efficient processing + pipelines of "consumers" that each can have complex internal + state: + + @consumer + def thumbnail_pager(pagesize, thumbsize, destination): + while True: + page = new_image(pagesize) + rows, columns = pagesize / thumbsize + pending = False + try: + for row in xrange(rows): + for column in xrange(columns): + thumb = create_thumbnail((yield), thumbsize) + page.write( + thumb, col*thumbsize.x, row*thumbsize.y + ) + pending = True + except GeneratorExit: + # close() was called, so flush any pending output + if pending: + destination.send(page) + + # then close the downstream consumer, and exit + destination.close() + return + else: + # we finished a page full of thumbnails, so send it + # downstream and keep on looping + destination.send(page) + + @consumer + def jpeg_writer(dirname): + fileno = 1 + while True: + filename = os.path.join(dirname,"page%04d.jpg" % fileno) + write_jpeg((yield), filename) + + + # Put them together to make a function that makes thumbnail + # pages from a list of images and other parameters. + # + def write_thumbnails(pagesize, thumbsize, images, output_dir): + pipeline = paginate( + pagesize, thumbsize, jpeg_writer(output_dir) + ) + + for image in images: + pipeline.send(image) + + pipeline.close() + + 3. A simple co-routine scheduler or "trampoline" that lets coroutines "call" other coroutines by yielding the coroutine they wish to invoke. Any non-generator value yielded by a coroutine is returned to the coroutine that "called" the @@ -437,7 +518,7 @@ self.queue.append(resume) - 2. A simple "echo" server, and code to run it using a trampoline + 4. A simple "echo" server, and code to run it using a trampoline (presumes the existence of "nonblocking_read", "nonblocking_write", and other I/O coroutines, that e.g. raise ConnectionLost if the connection is closed): From pje at users.sourceforge.net Sun Jun 26 18:51:39 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 26 Jun 2005 09:51:39 -0700 Subject: [Python-checkins] python/nondist/peps pep-0342.txt,1.7,1.8 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv596 Modified Files: pep-0342.txt Log Message: Oops, fix more typos. Guess I'm still short on sleep. :) Index: pep-0342.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0342.txt,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- pep-0342.txt 26 Jun 2005 16:29:14 -0000 1.7 +++ pep-0342.txt 26 Jun 2005 16:51:37 -0000 1.8 @@ -424,13 +424,14 @@ while True: filename = os.path.join(dirname,"page%04d.jpg" % fileno) write_jpeg((yield), filename) + fileno += 1 # Put them together to make a function that makes thumbnail # pages from a list of images and other parameters. # def write_thumbnails(pagesize, thumbsize, images, output_dir): - pipeline = paginate( + pipeline = thumbnail_pager( pagesize, thumbsize, jpeg_writer(output_dir) ) From kbk at users.sourceforge.net Sun Jun 26 20:27:39 2005 From: kbk at users.sourceforge.net (kbk@users.sourceforge.net) Date: Sun, 26 Jun 2005 11:27:39 -0700 Subject: [Python-checkins] python/dist/src/Lib smtplib.py,1.69,1.70 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18713 Modified Files: smtplib.py Log Message: SMTP.help() was returning a tuple instead of the promised text. Index: smtplib.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/smtplib.py,v retrieving revision 1.69 retrieving revision 1.70 diff -u -d -r1.69 -r1.70 --- smtplib.py 6 Feb 2005 06:57:08 -0000 1.69 +++ smtplib.py 26 Jun 2005 18:27:36 -0000 1.70 @@ -439,7 +439,7 @@ """SMTP 'help' command. Returns help text from server.""" self.putcmd("help", args) - return self.getreply() + return self.getreply()[1] def rset(self): """SMTP 'rset' command -- resets session.""" From birkenfeld at users.sourceforge.net Sun Jun 26 22:20:50 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 13:20:50 -0700 Subject: [Python-checkins] python/dist/src/Tools/scripts pindent.py, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/scripts In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14449/Tools/scripts Modified Files: pindent.py Log Message: bug [ 1072853 ] thisid not intialized in pindent.py script Index: pindent.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/scripts/pindent.py,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- pindent.py 13 May 2003 18:14:25 -0000 1.12 +++ pindent.py 26 Jun 2005 20:20:48 -0000 1.13 @@ -238,6 +238,7 @@ self.indentsize = 1 stack = [] todo = [] + thisid = '' current, firstkw, lastkw, topid = 0, '', '', '' while 1: line = self.getline() From birkenfeld at users.sourceforge.net Sun Jun 26 22:22:48 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 13:22:48 -0700 Subject: [Python-checkins] python/dist/src/Tools/scripts pindent.py, 1.12, 1.12.12.1 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/scripts In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15567/Tools/scripts Modified Files: Tag: release24-maint pindent.py Log Message: backport bug [ 1072853 ] thisid not intialized in pindent.py script Index: pindent.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/scripts/pindent.py,v retrieving revision 1.12 retrieving revision 1.12.12.1 diff -u -d -r1.12 -r1.12.12.1 --- pindent.py 13 May 2003 18:14:25 -0000 1.12 +++ pindent.py 26 Jun 2005 20:22:46 -0000 1.12.12.1 @@ -238,6 +238,7 @@ self.indentsize = 1 stack = [] todo = [] + thisid = '' current, firstkw, lastkw, topid = 0, '', '', '' while 1: line = self.getline() From birkenfeld at users.sourceforge.net Sun Jun 26 23:02:51 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 14:02:51 -0700 Subject: [Python-checkins] python/dist/src/Lib Cookie.py,1.17,1.18 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv4891/Lib Modified Files: Cookie.py Log Message: bug [ 1108948 ] Cookie.py produces invalid code Index: Cookie.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/Cookie.py,v retrieving revision 1.17 retrieving revision 1.18 diff -u -d -r1.17 -r1.18 --- Cookie.py 20 Oct 2003 14:01:49 -0000 1.17 +++ Cookie.py 26 Jun 2005 21:02:49 -0000 1.18 @@ -470,9 +470,9 @@ def js_output(self, attrs=None): # Print javascript return """ - """ % ( self.OutputString(attrs), ) From birkenfeld at users.sourceforge.net Sun Jun 26 23:09:41 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 14:09:41 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1303,1.1304 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8342/Misc Modified Files: NEWS Log Message: Add NEWS entries for bugs closed at Bug Day. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1303 retrieving revision 1.1304 diff -u -d -r1.1303 -r1.1304 --- NEWS 25 Jun 2005 21:08:46 -0000 1.1303 +++ NEWS 26 Jun 2005 21:09:38 -0000 1.1304 @@ -147,6 +147,8 @@ Library ------- +- Bug #1108948: Cookie.py produced invalid JavaScript code. + - The tokenize module now detects and reports indentation errors. Bug #1224621. @@ -370,6 +372,14 @@ Documentation ------------- +- Patch #1180012: The documentation for modulefinder is now in the library reference. + +- Patch #1213031: Document that os.chown() accepts argument values of -1. + +- Bug #1190563: Document os.waitpid() return value with WNOHANG flag. + +- Bug #1175022: Correct the example code for property(). + - Document the IterableUserDict class in the UserDict module. Closes bug #1166582. @@ -385,6 +395,8 @@ Tools/Demos ----------- +- Bug #1072853: pindent.py used an uninitialized variable. + - Patch #1177597: Correct Complex.__init__. From birkenfeld at users.sourceforge.net Sun Jun 26 23:33:24 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 14:33:24 -0700 Subject: [Python-checkins] python/dist/src/Lib BaseHTTPServer.py,1.29,1.30 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21516/Lib Modified Files: BaseHTTPServer.py Log Message: bug [ 1100201 ] Cross-site scripting on BaseHTTPServer Index: BaseHTTPServer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/BaseHTTPServer.py,v retrieving revision 1.29 retrieving revision 1.30 diff -u -d -r1.29 -r1.30 --- BaseHTTPServer.py 12 Feb 2004 17:35:05 -0000 1.29 +++ BaseHTTPServer.py 26 Jun 2005 21:33:14 -0000 1.30 @@ -89,6 +89,8 @@ """ +def _quote_html(html): + return html.replace("&", "&").replace("<", "<").replace(">", ">") class HTTPServer(SocketServer.TCPServer): @@ -336,8 +338,9 @@ message = short explain = long self.log_error("code %d, message %s", code, message) + # using _quote_html to prevent Cross Site Scripting attacks (see bug #1100201) content = (self.error_message_format % - {'code': code, 'message': message, 'explain': explain}) + {'code': code, 'message': _quote_html(message), 'explain': explain}) self.send_response(code, message) self.send_header("Content-Type", "text/html") self.send_header('Connection', 'close') From birkenfeld at users.sourceforge.net Sun Jun 26 23:34:00 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 14:34:00 -0700 Subject: [Python-checkins] python/dist/src/Lib BaseHTTPServer.py, 1.29, 1.29.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21620/Lib Modified Files: Tag: release24-maint BaseHTTPServer.py Log Message: backport bug [ 1100201 ] Cross-site scripting on BaseHTTPServer Index: BaseHTTPServer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/BaseHTTPServer.py,v retrieving revision 1.29 retrieving revision 1.29.4.1 diff -u -d -r1.29 -r1.29.4.1 --- BaseHTTPServer.py 12 Feb 2004 17:35:05 -0000 1.29 +++ BaseHTTPServer.py 26 Jun 2005 21:33:54 -0000 1.29.4.1 @@ -89,6 +89,8 @@ """ +def _quote_html(html): + return html.replace("&", "&").replace("<", "<").replace(">", ">") class HTTPServer(SocketServer.TCPServer): @@ -336,8 +338,9 @@ message = short explain = long self.log_error("code %d, message %s", code, message) + # using _quote_html to prevent Cross Site Scripting attacks (see bug #1100201) content = (self.error_message_format % - {'code': code, 'message': message, 'explain': explain}) + {'code': code, 'message': _quote_html(message), 'explain': explain}) self.send_response(code, message) self.send_header("Content-Type", "text/html") self.send_header('Connection', 'close') From birkenfeld at users.sourceforge.net Sun Jun 26 23:36:28 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 14:36:28 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1304,1.1305 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23201/Misc Modified Files: NEWS Log Message: - Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via error messages. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1304 retrieving revision 1.1305 diff -u -d -r1.1304 -r1.1305 --- NEWS 26 Jun 2005 21:09:38 -0000 1.1304 +++ NEWS 26 Jun 2005 21:36:25 -0000 1.1305 @@ -147,6 +147,9 @@ Library ------- +- Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via + error messages. + - Bug #1108948: Cookie.py produced invalid JavaScript code. - The tokenize module now detects and reports indentation errors. From birkenfeld at users.sourceforge.net Sun Jun 26 23:50:42 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 14:50:42 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.55, 1.1193.2.56 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30680/Misc Modified Files: Tag: release24-maint NEWS Log Message: - Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via error messages. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1193.2.55 retrieving revision 1.1193.2.56 diff -u -d -r1.1193.2.55 -r1.1193.2.56 --- NEWS 21 Jun 2005 07:53:55 -0000 1.1193.2.55 +++ NEWS 26 Jun 2005 21:50:34 -0000 1.1193.2.56 @@ -39,6 +39,9 @@ Library ------- +- Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via + error messages. + - Bug #1224621: tokenize module does not detect inconsistent dedents - Bug #1196315: fix weakref.WeakValueDictionary constructor. @@ -55,6 +58,22 @@ line ending. Remove the special handling of a "\r\n" that has been split between two lines. +Documentation +------------- + +- Patch #1180012: The documentation for modulefinder is now in the library reference. + +- Patch #1213031: Document that os.chown() accepts argument values of -1. + +- Bug #1190563: Document os.waitpid() return value with WNOHANG flag. + +- Bug #1175022: Correct the example code for property(). + +Tools/Demos +----------- + +- Bug #1072853: pindent.py used an uninitialized variable. + What's New in Python 2.4.1 final? ================================= From birkenfeld at users.sourceforge.net Sun Jun 26 23:57:57 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 14:57:57 -0700 Subject: [Python-checkins] python/dist/src/Lib cgitb.py,1.16,1.17 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1941/Lib Modified Files: cgitb.py Log Message: Prevent creating a HTML link to file://?/ Index: cgitb.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/cgitb.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- cgitb.py 6 Feb 2005 06:57:07 -0000 1.16 +++ cgitb.py 26 Jun 2005 21:57:55 -0000 1.17 @@ -112,8 +112,11 @@ frames = [] records = inspect.getinnerframes(etb, context) for frame, file, lnum, func, lines, index in records: - file = file and os.path.abspath(file) or '?' - link = '%s' % (file, pydoc.html.escape(file)) + if file: + file = os.path.abspath(file) + link = '%s' % (file, pydoc.html.escape(file)) + else: + file = link = '?' args, varargs, varkw, locals = inspect.getargvalues(frame) call = '' if func != '?': From birkenfeld at users.sourceforge.net Sun Jun 26 23:59:36 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 14:59:36 -0700 Subject: [Python-checkins] python/dist/src/Lib cgitb.py,1.15,1.15.4.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2907/Lib Modified Files: Tag: release24-maint cgitb.py Log Message: Backport: Prevent creating a HTML link to file://?/ Index: cgitb.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/cgitb.py,v retrieving revision 1.15 retrieving revision 1.15.4.1 diff -u -d -r1.15 -r1.15.4.1 --- cgitb.py 18 Jul 2004 06:14:41 -0000 1.15 +++ cgitb.py 26 Jun 2005 21:59:34 -0000 1.15.4.1 @@ -112,8 +112,11 @@ frames = [] records = inspect.getinnerframes(etb, context) for frame, file, lnum, func, lines, index in records: - file = file and os.path.abspath(file) or '?' - link = '%s' % (file, pydoc.html.escape(file)) + if file: + file = os.path.abspath(file) + link = '%s' % (file, pydoc.html.escape(file)) + else: + file = link = '?' args, varargs, varkw, locals = inspect.getargvalues(frame) call = '' if func != '?': From birkenfeld at users.sourceforge.net Mon Jun 27 00:01:52 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:01:52 -0700 Subject: [Python-checkins] python/dist/src/Lib urllib2.py,1.81,1.82 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv3820/Lib Modified Files: urllib2.py Log Message: bug [ 1175848 ] poorly named variable in urllib2.py Index: urllib2.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/urllib2.py,v retrieving revision 1.81 retrieving revision 1.82 diff -u -d -r1.81 -r1.82 --- urllib2.py 6 Feb 2005 06:57:08 -0000 1.81 +++ urllib2.py 26 Jun 2005 22:01:35 -0000 1.82 @@ -277,8 +277,8 @@ class OpenerDirector: def __init__(self): - server_version = "Python-urllib/%s" % __version__ - self.addheaders = [('User-agent', server_version)] + client_version = "Python-urllib/%s" % __version__ + self.addheaders = [('User-agent', client_version)] # manage the individual handlers self.handlers = [] self.handle_open = {} From birkenfeld at users.sourceforge.net Mon Jun 27 00:06:57 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:06:57 -0700 Subject: [Python-checkins] python/dist/src/Lib httplib.py,1.94,1.95 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6274/Lib Modified Files: httplib.py Log Message: bug [ 1155638 ] self.length shield exception in httplib Index: httplib.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/httplib.py,v retrieving revision 1.94 retrieving revision 1.95 diff -u -d -r1.94 -r1.95 --- httplib.py 7 Nov 2004 16:13:09 -0000 1.94 +++ httplib.py 26 Jun 2005 22:06:54 -0000 1.95 @@ -353,6 +353,7 @@ raise UnknownProtocol(version) if self.version == 9: + self.length = None self.chunked = 0 self.will_close = 1 self.msg = HTTPMessage(StringIO()) From birkenfeld at users.sourceforge.net Mon Jun 27 00:07:13 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:07:13 -0700 Subject: [Python-checkins] python/dist/src/Lib httplib.py,1.94,1.94.2.1 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6280/Lib Modified Files: Tag: release24-maint httplib.py Log Message: backport bug [ 1155638 ] self.length shield exception in httplib Index: httplib.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/httplib.py,v retrieving revision 1.94 retrieving revision 1.94.2.1 diff -u -d -r1.94 -r1.94.2.1 --- httplib.py 7 Nov 2004 16:13:09 -0000 1.94 +++ httplib.py 26 Jun 2005 22:06:56 -0000 1.94.2.1 @@ -353,6 +353,7 @@ raise UnknownProtocol(version) if self.version == 9: + self.length = None self.chunked = 0 self.will_close = 1 self.msg = HTTPMessage(StringIO()) From birkenfeld at users.sourceforge.net Mon Jun 27 00:09:13 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:09:13 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1305,1.1306 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7283/Misc Modified Files: NEWS Log Message: - Bug #1155638: Fix a bug which affected HTTP 0.9 responses in httplib. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1305 retrieving revision 1.1306 diff -u -d -r1.1305 -r1.1306 --- NEWS 26 Jun 2005 21:36:25 -0000 1.1305 +++ NEWS 26 Jun 2005 22:09:06 -0000 1.1306 @@ -147,6 +147,8 @@ Library ------- +- Bug #1155638: Fix a bug which affected HTTP 0.9 responses in httplib. + - Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via error messages. From birkenfeld at users.sourceforge.net Mon Jun 27 00:09:15 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:09:15 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.56, 1.1193.2.57 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7420/Misc Modified Files: Tag: release24-maint NEWS Log Message: - Bug #1155638: Fix a bug which affected HTTP 0.9 responses in httplib. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1193.2.56 retrieving revision 1.1193.2.57 diff -u -d -r1.1193.2.56 -r1.1193.2.57 --- NEWS 26 Jun 2005 21:50:34 -0000 1.1193.2.56 +++ NEWS 26 Jun 2005 22:09:12 -0000 1.1193.2.57 @@ -39,6 +39,8 @@ Library ------- +- Bug #1155638: Fix a bug which affected HTTP 0.9 responses in httplib. + - Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via error messages. From birkenfeld at users.sourceforge.net Mon Jun 27 00:22:34 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:22:34 -0700 Subject: [Python-checkins] python/dist/src/Lib doctest.py,1.122,1.123 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14574/Lib Modified Files: doctest.py Log Message: bug [ 1172785 ] doctest.script_from_examples() result sometimes un-exec-able Index: doctest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/doctest.py,v retrieving revision 1.122 retrieving revision 1.123 diff -u -d -r1.122 -r1.123 --- doctest.py 28 Mar 2005 23:50:54 -0000 1.122 +++ doctest.py 26 Jun 2005 22:22:31 -0000 1.123 @@ -2498,7 +2498,8 @@ while output and output[0] == '#': output.pop(0) # Combine the output, and return it. - return '\n'.join(output) + # Add a courtesy newline to prevent exec from choking (see bug #1172785) + return '\n'.join(output) + '\n' def testsource(module, name): """Extract the test sources from a doctest docstring as a script. From birkenfeld at users.sourceforge.net Mon Jun 27 00:23:46 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:23:46 -0700 Subject: [Python-checkins] python/dist/src/Lib doctest.py, 1.120.2.1, 1.120.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15362/Lib Modified Files: Tag: release24-maint doctest.py Log Message: backport bug [ 1172785 ] doctest.script_from_examples() result sometimes un-exec-able Index: doctest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/doctest.py,v retrieving revision 1.120.2.1 retrieving revision 1.120.2.2 diff -u -d -r1.120.2.1 -r1.120.2.2 --- doctest.py 29 Mar 2005 12:35:54 -0000 1.120.2.1 +++ doctest.py 26 Jun 2005 22:23:44 -0000 1.120.2.2 @@ -2494,7 +2494,8 @@ while output and output[0] == '#': output.pop(0) # Combine the output, and return it. - return '\n'.join(output) + # Add a courtesy newline to prevent exec from choking (see bug #1172785) + return '\n'.join(output) + '\n' def testsource(module, name): """Extract the test sources from a doctest docstring as a script. From birkenfeld at users.sourceforge.net Mon Jun 27 00:53:31 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:53:31 -0700 Subject: [Python-checkins] python/dist/src/Lib warnings.py,1.26,1.27 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31176/Lib Modified Files: warnings.py Log Message: bug [ 839151 ] attempt to access sys.argv when it doesn't exist Index: warnings.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/warnings.py,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- warnings.py 6 Feb 2005 06:57:08 -0000 1.26 +++ warnings.py 26 Jun 2005 22:53:29 -0000 1.27 @@ -50,7 +50,11 @@ filename = filename[:-1] else: if module == "__main__": - filename = sys.argv[0] + try: + filename = sys.argv[0] + except AttributeError: + # embedded interpreters don't have sys.argv, see bug #839151 + filename = '__main__' if not filename: filename = module registry = globals.setdefault("__warningregistry__", {}) From birkenfeld at users.sourceforge.net Mon Jun 27 00:53:45 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:53:45 -0700 Subject: [Python-checkins] python/dist/src/Lib warnings.py, 1.24.2.1, 1.24.2.2 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31208/Lib Modified Files: Tag: release24-maint warnings.py Log Message: backport bug [ 839151 ] attempt to access sys.argv when it doesn't exist Index: warnings.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/warnings.py,v retrieving revision 1.24.2.1 retrieving revision 1.24.2.2 diff -u -d -r1.24.2.1 -r1.24.2.2 --- warnings.py 29 Dec 2004 15:28:29 -0000 1.24.2.1 +++ warnings.py 26 Jun 2005 22:53:43 -0000 1.24.2.2 @@ -50,7 +50,11 @@ filename = filename[:-1] else: if module == "__main__": - filename = sys.argv[0] + try: + filename = sys.argv[0] + except AttributeError: + # embedded interpreters don't have sys.argv, see bug #839151 + filename = '__main__' if not filename: filename = module registry = globals.setdefault("__warningregistry__", {}) From birkenfeld at users.sourceforge.net Mon Jun 27 00:55:00 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:55:00 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1306,1.1307 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31745/Misc Modified Files: NEWS Log Message: - Bug #839151: Fix an attempt to access sys.argv in the warnings module though this can be missing in embedded interpreters Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1306 retrieving revision 1.1307 diff -u -d -r1.1306 -r1.1307 --- NEWS 26 Jun 2005 22:09:06 -0000 1.1306 +++ NEWS 26 Jun 2005 22:54:58 -0000 1.1307 @@ -147,6 +147,9 @@ Library ------- +- Bug #839151: Fix an attempt to access sys.argv in the warnings module + though this can be missing in embedded interpreters + - Bug #1155638: Fix a bug which affected HTTP 0.9 responses in httplib. - Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via From birkenfeld at users.sourceforge.net Mon Jun 27 00:55:08 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 15:55:08 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS, 1.1193.2.57, 1.1193.2.58 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31761/Misc Modified Files: Tag: release24-maint NEWS Log Message: - Bug #839151: Fix an attempt to access sys.argv in the warnings module though this can be missing in embedded interpreters Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1193.2.57 retrieving revision 1.1193.2.58 diff -u -d -r1.1193.2.57 -r1.1193.2.58 --- NEWS 26 Jun 2005 22:09:12 -0000 1.1193.2.57 +++ NEWS 26 Jun 2005 22:55:05 -0000 1.1193.2.58 @@ -39,6 +39,9 @@ Library ------- +- Bug #839151: Fix an attempt to access sys.argv in the warnings module + though this can be missing in embedded interpreters + - Bug #1155638: Fix a bug which affected HTTP 0.9 responses in httplib. - Bug #1100201: Cross-site scripting was possible on BaseHTTPServer via From birkenfeld at users.sourceforge.net Mon Jun 27 01:09:53 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 16:09:53 -0700 Subject: [Python-checkins] python/dist/src/Lib doctest.py,1.123,1.124 Message-ID: Update of /cvsroot/python/python/dist/src/Lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6155/Lib Modified Files: doctest.py Log Message: Fix test cases for doctest. Index: doctest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/doctest.py,v retrieving revision 1.123 retrieving revision 1.124 diff -u -d -r1.123 -r1.124 --- doctest.py 26 Jun 2005 22:22:31 -0000 1.123 +++ doctest.py 26 Jun 2005 23:09:50 -0000 1.124 @@ -2476,6 +2476,7 @@ blah # # Ho hum + """ output = [] for piece in DocTestParser().parse(s): From birkenfeld at users.sourceforge.net Mon Jun 27 01:09:54 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 16:09:54 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_doctest.py, 1.53, 1.54 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6155/Lib/test Modified Files: test_doctest.py Log Message: Fix test cases for doctest. Index: test_doctest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_doctest.py,v retrieving revision 1.53 retrieving revision 1.54 diff -u -d -r1.53 -r1.54 --- test_doctest.py 21 Dec 2004 23:46:27 -0000 1.53 +++ test_doctest.py 26 Jun 2005 23:09:51 -0000 1.54 @@ -1517,6 +1517,7 @@ ## 44 # # Yee ha! + >>> name = 'test.test_doctest.SampleNewStyleClass' >>> print doctest.testsource(test.test_doctest, name) @@ -1525,6 +1526,7 @@ ## 1 ## 2 ## 3 + >>> name = 'test.test_doctest.SampleClass.a_classmethod' >>> print doctest.testsource(test.test_doctest, name) @@ -1534,6 +1536,7 @@ print SampleClass(0).a_classmethod(10) # Expected: ## 12 + """ def test_debug(): r""" From pje at users.sourceforge.net Mon Jun 27 02:31:05 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 26 Jun 2005 17:31:05 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools __init__.py, 1.9, 1.10 dist.py, 1.7, 1.8 package_index.py, 1.7, 1.8 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15697/setuptools Modified Files: __init__.py dist.py package_index.py Log Message: EasyInstall/setuptools 0.5a4: significant new features, including automatic installation of dependencies, the ability to specify dependencies in a setup script, and several new options to control EasyInstall's behavior. Index: __init__.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/__init__.py,v retrieving revision 1.9 retrieving revision 1.10 diff -u -d -r1.9 -r1.10 --- __init__.py 25 Jun 2005 19:33:06 -0000 1.9 +++ __init__.py 27 Jun 2005 00:31:02 -0000 1.10 @@ -8,7 +8,7 @@ from distutils.util import convert_path import os.path -__version__ = '0.5a3' +__version__ = '0.5a4' __all__ = [ 'setup', 'Distribution', 'Feature', 'Command', 'Extension', 'Require', Index: dist.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/dist.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- dist.py 15 Jun 2005 02:19:42 -0000 1.7 +++ dist.py 27 Jun 2005 00:31:02 -0000 1.8 @@ -8,50 +8,44 @@ from setuptools.command.install_lib import install_lib from distutils.errors import DistutilsOptionError, DistutilsPlatformError from distutils.errors import DistutilsSetupError -import setuptools +import setuptools, pkg_resources sequence = tuple, list - - - - - - - - - - - - - - - - - - - - - - - - - - - class Distribution(_Distribution): """Distribution with support for features, tests, and package data This is an enhanced version of 'distutils.dist.Distribution' that effectively adds the following new optional keyword arguments to 'setup()': + 'install_requires' -- a string or sequence of strings specifying project + versions that the distribution requires when installed, in the format + used by 'pkg_resources.require()'. They will be installed + automatically when the package is installed. If you wish to use + packages that are not available in PyPI, or want to give your users an + alternate download location, you can add a 'find_links' option to the + '[easy_install]' section of your project's 'setup.cfg' file, and then + setuptools will scan the listed web pages for links that satisfy the + requirements. + + 'extras_require' -- a dictionary mapping names of optional "extras" to the + additional requirement(s) that using those extras incurs. For example, + this:: + + extras_require = dict(reST = ["docutils>=0.3", "reSTedit"]) + + indicates that the distribution can optionally provide an extra + capability called "reST", but it can only be used if docutils and + reSTedit are installed. If the user installs your package using + EasyInstall and requests one of your extras, the corresponding + additional requirements will be installed if needed. + 'features' -- a dictionary mapping option names to 'setuptools.Feature' objects. Features are a portion of the distribution that can be included or excluded based on user options, inter-feature dependencies, and availability on the current system. Excluded features are omitted from all setup commands, including source and binary distributions, so you can create multiple distributions from the same source tree. - Feature names should be valid Python identifiers, except that they may contain the '-' (minus) sign. Features can be included or excluded via the command line options '--with-X' and '--without-X', where 'X' is @@ -84,6 +78,8 @@ the distribution. They are used by the feature subsystem to configure the distribution for the included and excluded features. """ + + def __init__ (self, attrs=None): have_package_data = hasattr(self, "package_data") if not have_package_data: @@ -91,16 +87,68 @@ self.features = {} self.test_suite = None self.requires = [] + self.install_requires = [] + self.extras_require = {} _Distribution.__init__(self,attrs) if not have_package_data: from setuptools.command.build_py import build_py self.cmdclass.setdefault('build_py',build_py) + self.cmdclass.setdefault('build_ext',build_ext) self.cmdclass.setdefault('install',install) self.cmdclass.setdefault('install_lib',install_lib) + + + + + + + + + + + + + + + + + + + + + + + + + def finalize_options(self): + _Distribution.finalize_options(self) + if self.features: self._set_global_opts_from_features() + if self.extra_path: + raise DistutilsSetupError( + "The 'extra_path' parameter is not needed when using " + "setuptools. Please remove it from your setup script." + ) + try: + list(pkg_resources.parse_requirements(self.install_requires)) + except (TypeError,ValueError): + raise DistutilsSetupError( + "'install_requires' must be a string or list of strings " + "containing valid project/version requirement specifiers" + ) + try: + for k,v in self.extras_require.items(): + list(pkg_resources.parse_requirements(v)) + except (TypeError,ValueError,AttributeError): + raise DistutilsSetupError( + "'extras_require' must be a dictionary whose values are " + "strings or lists of strings containing valid project/version " + "requirement specifiers." + ) + def parse_command_line(self): """Process features after parsing command line options""" result = _Distribution.parse_command_line(self) @@ -108,19 +156,12 @@ self._finalize_features() return result + def _feature_attrname(self,name): """Convert feature name to corresponding option attribute name""" return 'with_'+name.replace('-','_') - - - - - - - - def _set_global_opts_from_features(self): """Add --with-X/--without-X options based on optional features""" @@ -343,29 +384,29 @@ return nargs - def has_dependencies(self): return not not self.requires - def run_commands(self): - if setuptools.bootstrap_install_from and 'install' in self.commands: - # Bootstrap self-installation of setuptools - from easy_install import easy_install - cmd = easy_install( - self, args=[setuptools.bootstrap_install_from], zip_ok=1 - ) - cmd.ensure_finalized() - cmd.run() - setuptools.bootstrap_install_from = None - - _Distribution.run_commands(self) - - - - - + for cmd in self.commands: + if cmd=='install' and not cmd in self.have_run: + self.install_eggs() + else: + self.run_command(cmd) + def install_eggs(self): + from easy_install import easy_install + cmd = easy_install(self, args="x") + cmd.ensure_finalized() # finalize before bdist_egg munges install cmd + self.run_command('bdist_egg') + args = [self.get_command_obj('bdist_egg').egg_output] + if setuptools.bootstrap_install_from: + # Bootstrap self-installation of setuptools + args.insert(0, setuptools.bootstrap_install_from) + cmd.args = args + cmd.run() + self.have_run['install'] = 1 + setuptools.bootstrap_install_from = None def get_cmdline_options(self): """Return a '{cmd: {opt:val}}' map of all command-line options Index: package_index.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/package_index.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- package_index.py 25 Jun 2005 19:33:06 -0000 1.7 +++ package_index.py 27 Jun 2005 00:31:03 -0000 1.8 @@ -203,7 +203,7 @@ - def find_packages(self,requirement): + def find_packages(self, requirement): self.scan_url(self.index_url + requirement.distname+'/') if not self.package_pages.get(requirement.key): # We couldn't find the target package, so search the index page too @@ -221,13 +221,13 @@ # scan each page that might be related to the desired package self.scan_url(url) - def obtain(self,requirement): + def obtain(self, requirement, installer=None): self.find_packages(requirement) for dist in self.get(requirement.key, ()): if dist in requirement: return dist self.debug("%s does not match %s", requirement, dist) - + return super(PackageIndex, self).obtain(requirement,installer) @@ -245,19 +245,20 @@ def download(self, spec, tmpdir): - """Locate and/or download `spec`, returning a local filename + """Locate and/or download `spec` to `tmpdir`, returning a local path `spec` may be a ``Requirement`` object, or a string containing a URL, - an existing local filename, or a package/version requirement spec + an existing local filename, or a project/version requirement spec (i.e. the string form of a ``Requirement`` object). - If necessary, the requirement is searched for in the package index. - If the download is successful, the return value is a local file path, - and it is a subpath of `tmpdir` if the distribution had to be - downloaded. If no matching distribution is found, return ``None``. - Various errors may be raised if a problem occurs during downloading. + If `spec` is a ``Requirement`` object or a string containing a + project/version requirement spec, this method is equivalent to + the ``fetch()`` method. If `spec` is a local, existing file or + directory name, it is simply returned unchanged. If `spec` is a URL, + it is downloaded to a subpath of `tmpdir`, and the local filename is + returned. Various errors may be raised if a problem occurs during + downloading. """ - if not isinstance(spec,Requirement): scheme = URL_SCHEME(spec) if scheme: @@ -275,16 +276,56 @@ "Not a URL, existing file, or requirement spec: %r" % (spec,) ) + + return self.fetch(spec, tmpdir, force_scan) + + + + + + + + def fetch(self, requirement, tmpdir, force_scan=False): + """Obtain a file suitable for fulfilling `requirement` + + `requirement` must be a ``pkg_resources.Requirement`` instance. + If necessary, or if the `force_scan` flag is set, the requirement is + searched for in the (online) package index as well as the locally + installed packages. If a distribution matching `requirement` is found, + the return value is the same as if you had called the ``download()`` + method with the matching distribution's URL. If no matching + distribution is found, returns ``None``. + """ + # process a Requirement - self.info("Searching for %s", spec) - dist = self.best_match(spec,[]) + self.info("Searching for %s", requirement) + + if force_scan: + self.find_packages(requirement) + + dist = self.best_match(requirement, []) # XXX + if dist is not None: self.info("Best match: %s", dist) return self.download(dist.path, tmpdir) - self.warn("No local packages or download links found for %s", spec) + self.warn( + "No local packages or download links found for %s", requirement + ) return None + + + + + + + + + + + + dl_blocksize = 8192 def _download_to(self, url, filename): From pje at users.sourceforge.net Mon Jun 27 02:31:05 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 26 Jun 2005 17:31:05 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools/setuptools/command bdist_egg.py, 1.18, 1.19 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15697/setuptools/command Modified Files: bdist_egg.py Log Message: EasyInstall/setuptools 0.5a4: significant new features, including automatic installation of dependencies, the ability to specify dependencies in a setup script, and several new options to control EasyInstall's behavior. Index: bdist_egg.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setuptools/command/bdist_egg.py,v retrieving revision 1.18 retrieving revision 1.19 diff -u -d -r1.18 -r1.19 --- bdist_egg.py 15 Jun 2005 02:12:49 -0000 1.18 +++ bdist_egg.py 27 Jun 2005 00:31:03 -0000 1.19 @@ -11,7 +11,7 @@ from distutils.errors import * from distutils import log from pkg_resources import parse_requirements, get_platform, safe_name, \ - safe_version, Distribution + safe_version, Distribution, yield_lines def write_stub(resource, pyfile): @@ -78,7 +78,7 @@ self.tag_build = None self.tag_svn_revision = 0 self.tag_date = 0 - + self.egg_output = None def finalize_options (self): self.egg_name = safe_name(self.distribution.get_name()) @@ -105,19 +105,19 @@ self.bdist_dir = os.path.join(bdist_base, 'egg') if self.plat_name is None: self.plat_name = get_platform() - self.set_undefined_options('bdist',('dist_dir', 'dist_dir')) - - - - - - - - + self.set_undefined_options('bdist',('dist_dir', 'dist_dir')) + if self.egg_output is None: + # Compute filename of the output egg + basename = Distribution( + None, None, self.egg_name, self.egg_version, + get_python_version(), + self.distribution.has_ext_modules() and self.plat_name + ).egg_name() + self.egg_output = os.path.join(self.dist_dir, basename+'.egg') @@ -146,22 +146,22 @@ finally: self.distribution.data_files = old + def get_outputs(self): + return [self.egg_output] - - - - - - - - - - - - - - - + def write_requirements(self): + dist = self.distribution + if not getattr(dist,'install_requires',None) and \ + not getattr(dist,'extras_require',None): return + requires = os.path.join(self.egg_info,"requires.txt") + log.info("writing %s", requires) + if not self.dry_run: + f = open(requires, 'wt') + f.write('\n'.join(yield_lines(dist.install_requires))) + for extra,reqs in dist.extras_require.items(): + f.write('\n\n[%s]\n%s' % (extra, '\n'.join(yield_lines(reqs)))) + f.close() + def run(self): # We run install_lib before install_data, because some data hacks # pull their data path from the install_lib command. @@ -189,24 +189,19 @@ if self.distribution.data_files: self.do_install_data() + # Make the EGG-INFO directory + archive_root = self.bdist_dir + egg_info = os.path.join(archive_root,'EGG-INFO') + self.mkpath(egg_info) + self.mkpath(self.egg_info) + if self.distribution.scripts: - script_dir = os.path.join(self.bdist_dir,'EGG-INFO','scripts') + script_dir = os.path.join(egg_info, 'scripts') log.info("installing scripts to %s" % script_dir) self.call_command('install_scripts', install_dir=script_dir) - # And make an archive relative to the root of the - # pseudo-installation tree. - archive_basename = Distribution( - None, None, self.egg_name, self.egg_version, get_python_version(), - ext_outputs and self.plat_name - ).egg_name() - pseudoinstall_root = os.path.join(self.dist_dir, archive_basename) - archive_root = self.bdist_dir + self.write_requirements() - # Make the EGG-INFO directory - egg_info = os.path.join(archive_root,'EGG-INFO') - self.mkpath(egg_info) - self.mkpath(self.egg_info) log.info("writing %s" % os.path.join(self.egg_info,'PKG-INFO')) if not self.dry_run: @@ -231,15 +226,20 @@ if not self.dry_run: os.unlink(native_libs) - if self.egg_info and os.path.exists(self.egg_info): - for filename in os.listdir(self.egg_info): - path = os.path.join(self.egg_info,filename) - if os.path.isfile(path): - self.copy_file(path,os.path.join(egg_info,filename)) + for filename in os.listdir(self.egg_info): + path = os.path.join(self.egg_info,filename) + if os.path.isfile(path): + self.copy_file(path,os.path.join(egg_info,filename)) + if os.path.exists(os.path.join(self.egg_info,'depends.txt')): + log.warn( + "WARNING: 'depends.txt' will not be used by setuptools 0.6!" + ) + log.warn( + "Use the install_requires/extras_require setup() args instead." + ) # Make the archive - make_zipfile(pseudoinstall_root+'.egg', - archive_root, verbose=self.verbose, + make_zipfile(self.egg_output, archive_root, verbose=self.verbose, dry_run=self.dry_run) if not self.keep_temp: remove_tree(self.bdist_dir, dry_run=self.dry_run) From pje at users.sourceforge.net Mon Jun 27 02:31:05 2005 From: pje at users.sourceforge.net (pje@users.sourceforge.net) Date: Sun, 26 Jun 2005 17:31:05 -0700 Subject: [Python-checkins] python/nondist/sandbox/setuptools EasyInstall.txt, 1.15, 1.16 easy_install.py, 1.23, 1.24 ez_setup.py, 1.6, 1.7 pkg_resources.py, 1.34, 1.35 setup.py, 1.16, 1.17 Message-ID: Update of /cvsroot/python/python/nondist/sandbox/setuptools In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv15697 Modified Files: EasyInstall.txt easy_install.py ez_setup.py pkg_resources.py setup.py Log Message: EasyInstall/setuptools 0.5a4: significant new features, including automatic installation of dependencies, the ability to specify dependencies in a setup script, and several new options to control EasyInstall's behavior. Index: EasyInstall.txt =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/EasyInstall.txt,v retrieving revision 1.15 retrieving revision 1.16 diff -u -d -r1.15 -r1.16 --- EasyInstall.txt 25 Jun 2005 19:33:06 -0000 1.15 +++ EasyInstall.txt 27 Jun 2005 00:31:02 -0000 1.16 @@ -23,7 +23,7 @@ ------------------------- Windows users can just download and run the `setuptools binary installer for -Windows `_. +Windows `_. All others should just download `ez_setup.py `_, and run it; this will download and install the correct version of ``setuptools`` for your Python @@ -62,7 +62,7 @@ **Example 2**. Install or upgrade a package by name and version by finding links on a given "download page":: - easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a3" + easy_install -f http://peak.telecommunity.com/dist "setuptools>=0.5a4" **Example 3**. Download a source distribution from a specified URL, automatically building and installing it:: @@ -73,6 +73,11 @@ easy_install /my_downloads/OtherPackage-3.2.1-py2.3.egg +**Example 5**. Upgrade an already-installed package to the latest version +listed on PyPI: + + easy_install --upgrade PyProtocols + Easy Install accepts URLs, filenames, PyPI package names (i.e., ``distutils`` "distribution" names), and package+version specifiers. In each case, it will attempt to locate the latest available version that meets your criteria. @@ -118,23 +123,29 @@ easy_install "SomePackage>2.0" +using the upgrade flag, to find the latest available version on PyPI:: + + easy_install --upgrade SomePackage + or by using a download page, direct download URL, or package filename:: - easy_install -s http://example.com/downloads ExamplePackage + easy_install -f http://example.com/downloads ExamplePackage easy_install http://example.com/downloads/ExamplePackage-2.0-py2.4.egg easy_install my_downloads/ExamplePackage-2.0.tgz If you're using ``-m`` or ``--multi`` (or installing outside of -``site-packages``), the ``require()`` function automatically selects the newest -available version of a package that meets your version criteria at runtime, so -installation is the only step needed. +``site-packages``), using the ``require()`` function at runtime automatically +selects the newest installed version of a package that meets your version +criteria. So, installing a newer version is the only step needed to upgrade +such packages. -If you're installing to ``site-packages`` and not using ``-m``, installing a -package automatically replaces any previous version in the ``easy-install.pth`` -file, so that Python will import the most-recently installed version by -default. +If you're installing to Python's ``site-packages`` directory (and not +using ``-m``), installing a package automatically replaces any previous version +in the ``easy-install.pth`` file, so that Python will import the most-recently +installed version by default. So, again, installing the newer version is the +only upgrade step needed. If you haven't suppressed script installation (using ``--exclude-scripts`` or ``-x``), then the upgraded version's scripts will be installed, and they will @@ -339,6 +350,16 @@ the ``--install-dir`` or ``-d`` option (or they are set via configuration file(s)) you must also use ``require()`` to enable packages at runtime. +``--upgrade, -U`` (New in 0.5a4) + By default, EasyInstall only searches the Python Package Index if a + project/version requirement can't be met by distributions already installed + on sys.path or the installation directory. However, if you supply the + ``--upgrade`` or ``-U`` flag, EasyInstall will always check the package + index before selecting a version to install. In this way, you can force + EasyInstall to use the latest available version of any package it installs + (subject to any version requirements that might exclude such later + versions). + ``--install-dir=DIR, -d DIR`` Set the installation directory. It is up to you to ensure that this directory is on ``sys.path`` at runtime, and to use @@ -366,6 +387,14 @@ versions of a package, but do not want to reset the version that will be run by scripts that are already installed. +``--always-copy, -a`` (New in 0.5a4) + Copy all needed distributions to the installation directory, even if they + are already present in a directory on sys.path. In older versions of + EasyInstall, this was the default behavior, but now you must explicitly + request it. By default, EasyInstall will no longer copy such distributions + from other sys.path directories to the installation directory, unless you + explicitly gave the distribution's filename on the command line. + ``--find-links=URL, -f URL`` (Option renamed in 0.4a2) Scan the specified "download pages" for direct links to downloadable eggs or source distributions. Any usable packages will be downloaded if they @@ -434,6 +463,12 @@ the default is 0 (unless it's set under ``install`` or ``install_lib`` in one of your distutils configuration files). +``--record=FILENAME`` (New in 0.5a4) + Write a record of all installed files to FILENAME. This is basically the + same as the same option for the standard distutils "install" command, and + is included for compatibility with tools that expect to pass this option + to "setup.py install". + Release Notes/Change History ============================ @@ -442,6 +477,46 @@ * There's no automatic retry for borked Sourceforge mirrors, which can easily time out or be missing a file. +0.5a4 + * Added ``--always-copy/-a`` option to always copy needed packages to the + installation directory, even if they're already present elsewhere on + sys.path. (In previous versions, this was the default behavior, but now + you must request it.) + + * Added ``--upgrade/-U`` option to force checking PyPI for latest available + version(s) of all packages requested by name and version, even if a matching + version is available locally. + + * Setup scripts using setuptools can now list their dependencies directly in + the setup.py file, without having to manually create a ``depends.txt`` file. + The ``install_requires`` and ``extras_require`` arguments to ``setup()`` + are used to create a dependencies file automatically. If you are manually + creating ``depends.txt`` right now, please switch to using these setup + arguments as soon as practical, because ``depends.txt`` support will be + removed in the 0.6 release cycle. For documentation on the new arguments, + see the ``setuptools.dist.Distribution`` class. + + * Added automatic installation of dependencies declared by a distribution + being installed. These dependencies must be listed in the distribution's + ``EGG-INFO`` directory, so the distribution has to have declared its + dependencies by using setuptools. If a package has requirements it didn't + declare, you'll still have to deal with them yourself. (E.g., by asking + EasyInstall to find and install them.) + + * Setup scripts using setuptools now always install using ``easy_install`` + internally, for ease of uninstallation and upgrading. Note: you *must* + remove any ``extra_path`` argument from your setup script, as it conflicts + with the proper functioning of the ``easy_install`` command. (Also, added + the ``--record`` option to ``easy_install`` for the benefit of tools that + run ``setup.py install --record=filename`` on behalf of another packaging + system.) + + * ``pkg_resources.AvailableDistributions.resolve()`` and related methods now + accept an ``installer`` argument: a callable taking one argument, a + ``Requirement`` instance. The callable must return a ``Distribution`` + object, or ``None`` if no distribution is found. This feature is used by + EasyInstall to resolve dependencies by recursively invoking itself. + 0.5a3 * Fixed not setting script permissions to allow execution. @@ -584,10 +659,8 @@ Future Plans ============ -* Process the installed package's dependencies as well as the base package * Additional utilities to list/remove/verify packages * Signature checking? SSL? Ability to suppress PyPI search? * Display byte progress meter when downloading distributions and long pages? * Redirect stdout/stderr to log during run_setup? - Index: easy_install.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/easy_install.py,v retrieving revision 1.23 retrieving revision 1.24 diff -u -d -r1.23 -r1.24 --- easy_install.py 25 Jun 2005 19:33:06 -0000 1.23 +++ easy_install.py 27 Jun 2005 00:31:02 -0000 1.24 @@ -9,7 +9,6 @@ file, or visit the `EasyInstall home page`__. __ http://peak.telecommunity.com/DevCenter/EasyInstall - """ import sys, os.path, zipimport, shutil, tempfile, zipfile @@ -21,6 +20,7 @@ from distutils.errors import DistutilsArgError, DistutilsOptionError from setuptools.archive_util import unpack_archive from setuptools.package_index import PackageIndex, parse_bdist_wininst +from setuptools.package_index import URL_SCHEME from setuptools.command import bdist_egg from pkg_resources import * @@ -47,9 +47,11 @@ user_options = [ ("zip-ok", "z", "install package as a zipfile"), ("multi-version", "m", "make apps have to require() a version"), + ("upgrade", "U", "force upgrade (searches PyPI for latest versions)"), ("install-dir=", "d", "install package to DIR"), ("script-dir=", "s", "install scripts to DIR"), ("exclude-scripts", "x", "Don't install scripts"), + ("always-copy", "a", "Copy all needed packages to install dir"), ("index-url=", "i", "base URL of Python Package Index"), ("find-links=", "f", "additional URL(s) to search for packages"), ("build-directory=", "b", @@ -57,29 +59,27 @@ ('optimize=', 'O', "also compile with optimization: -O1 for \"python -O\", " "-O2 for \"python -OO\", and -O0 to disable [default: -O0]"), + ('record=', None, + "filename in which to record list of installed files"), + ] + boolean_options = [ + 'zip-ok', 'multi-version', 'exclude-scripts', 'upgrade', 'always-copy' ] - - boolean_options = [ 'zip-ok', 'multi-version', 'exclude-scripts' ] create_index = PackageIndex def initialize_options(self): self.zip_ok = None - self.multi_version = None self.install_dir = self.script_dir = self.exclude_scripts = None self.index_url = None self.find_links = None self.build_directory = None self.args = None - self.optimize = None - + self.optimize = self.record = None + self.upgrade = self.always_copy = self.multi_version = None # Options not specifiable via command line self.package_index = None self.pth_file = None - - - - def finalize_options(self): # If a non-default installation directory was specified, default the # script directory to match it. @@ -96,6 +96,8 @@ self.set_undefined_options('install_scripts', ('install_dir', 'script_dir') ) + # default --record from the install command + self.set_undefined_options('install', ('record', 'record')) site_packages = get_python_lib() instdir = self.install_dir @@ -118,8 +120,19 @@ ) self.index_url = self.index_url or "http://www.python.org/pypi" + + self.shadow_path = sys.path[:] + + for path_item in self.install_dir, self.script_dir: + if path_item not in self.shadow_path: + self.shadow_path.insert(0, self.install_dir) + if self.package_index is None: - self.package_index = self.create_index(self.index_url) + self.package_index = self.create_index( + self.index_url, search_path = self.shadow_path + ) + + self.local_index = AvailableDistributions(self.shadow_path) if self.find_links is not None: if isinstance(self.find_links, basestring): @@ -145,20 +158,7 @@ "Build directory can only be set when using one URL" ) - - - - - - - - - - - - - - + self.outputs = [] @@ -179,30 +179,112 @@ self.package_index.scan_url(link) for spec in self.args: self.easy_install(spec) + if self.record: + from distutils import file_util + self.execute( + file_util.write_file, (self.record, self.outputs), + "writing list of installed files to '%s'" % + self.record + ) finally: log.set_verbosity(self.distribution.verbose) + def add_output(self, path): + if os.path.isdir(path): + for base, dirs, files in walk(path): + for filename in files: + self.outputs.append(os.path.join(base,filename)) + else: + self.outputs.append(path) + + + + + + def easy_install(self, spec): tmpdir = self.alloc_tmp() - try: - download = self.package_index.download(spec, tmpdir) + download = None + + try: + if not isinstance(spec,Requirement): + if URL_SCHEME(spec): + # It's a url, download it to tmpdir and process + download = self.package_index.download(spec, tmpdir) + return self.install_item(None, download, tmpdir, True) + + elif os.path.exists(spec): + # Existing file or directory, just process it directly + return self.install_item(None, spec, tmpdir, True) + else: + try: + spec = Requirement.parse(spec) + except ValueError: + raise RuntimeError( + "Not a URL, existing file, or requirement spec: %r" + % (spec,) + ) + + if isinstance(spec, Requirement): + download = self.package_index.fetch(spec, tmpdir, self.upgrade) + else: + spec = None + if download is None: raise RuntimeError( "Could not find distribution for %r" % spec ) - log.info("Processing %s", os.path.basename(download)) - for dist in self.install_eggs(download, self.zip_ok, tmpdir): - self.package_index.add(dist) - self.install_egg_scripts(dist) - log.warn(self.installation_report(dist)) + return self.install_item(spec, download, tmpdir) finally: if self.build_directory is None: shutil.rmtree(tmpdir) + + def install_item(self, spec, download, tmpdir, install_needed=False): + # Installation is also needed if file in tmpdir or is not an egg + install_needed = install_needed or os.path.dirname(download) == tmpdir + install_needed = install_needed or not download.endswith('.egg') + log.info("Processing %s", os.path.basename(download)) + if install_needed or self.always_copy: + dists = self.install_eggs(download, self.zip_ok, tmpdir) + for dist in dists: + self.process_distribution(spec, dist) + else: + dists = [self.egg_distribution(download)] + self.process_distribution(spec, dists[0], "Using") + if spec is not None: + for dist in dists: + if dist in spec: + return dist + + def process_distribution(self, requirement, dist, *info): + self.update_pth(dist) + self.package_index.add(dist) + self.local_index.add(dist) + self.install_egg_scripts(dist) + log.warn(self.installation_report(dist, *info)) + if requirement is None: + requirement = Requirement.parse('%s==%s'%(dist.name,dist.version)) + if dist in requirement: + log.info("Processing dependencies for %s", requirement) + try: + self.local_index.resolve( + [requirement], self.shadow_path, self.easy_install + ) + except DistributionNotFound, e: + raise RuntimeError( + "Could not find required distribution %s" % e.args + ) + except VersionConflict, e: + raise RuntimeError( + "Installed distribution %s conflicts with requirement %s" + % e.args + ) + def install_egg_scripts(self, dist): metadata = dist.metadata if self.exclude_scripts or not metadata.metadata_isdir('scripts'): @@ -285,26 +367,36 @@ + def egg_distribution(self, egg_path): + if os.path.isdir(egg_path): + metadata = PathMetadata(egg_path,os.path.join(egg_path,'EGG-INFO')) + else: + metadata = EggMetadata(zipimport.zipimporter(egg_path)) + return Distribution.from_filename(egg_path,metadata=metadata) + def install_egg(self, egg_path, zip_ok, tmpdir): destination = os.path.join(self.install_dir,os.path.basename(egg_path)) destination = os.path.abspath(destination) + if not self.dry_run: ensure_directory(destination) if not samefile(egg_path, destination): if os.path.isdir(destination): dir_util.remove_tree(destination, dry_run=self.dry_run) - elif os.path.isfile(destination): self.execute(os.unlink,(destination,),"Removing "+destination) - if zip_ok: + if os.path.isdir(egg_path): + if egg_path.startswith(tmpdir): + f,m = shutil.move, "Moving" + else: + f,m = shutil.copytree, "Copying" + elif zip_ok: if egg_path.startswith(tmpdir): f,m = shutil.move, "Moving" else: f,m = shutil.copy2, "Copying" - elif os.path.isdir(egg_path): - f,m = shutil.move, "Moving" else: self.mkpath(destination) f,m = self.unpack_and_compile, "Extracting" @@ -313,18 +405,8 @@ (m+" %s to %s") % (os.path.basename(egg_path),os.path.dirname(destination))) - if os.path.isdir(destination): - dist = Distribution.from_filename( - destination, metadata=PathMetadata( - destination, os.path.join(destination,'EGG-INFO') - ) - ) - else: - metadata = EggMetadata(zipimport.zipimporter(destination)) - dist = Distribution.from_filename(destination,metadata=metadata) - - self.update_pth(dist) - return dist + self.add_output(destination) + return self.egg_distribution(destination) def install_exe(self, dist_filename, tmpdir): # See if it's valid, get data @@ -408,10 +490,10 @@ ensure_directory(nl_txt) open(nl_txt,'w').write('\n'.join(native_libs)+'\n') - def installation_report(self, dist): + def installation_report(self, dist, what="Installed"): """Helpful installation message for display to package users""" - msg = "\nInstalled %(eggloc)s to %(instdir)s" + msg = "\n%(what)s %(eggloc)s" if self.multi_version: msg += """ @@ -431,8 +513,7 @@ this to work. (e.g. by being the application's script directory, by being on PYTHONPATH, or by being added to sys.path by your code.) """ - eggloc = os.path.basename(dist.path) - instdir = os.path.realpath(self.install_dir) + eggloc = dist.path name = dist.name version = dist.version return msg % locals() @@ -449,6 +530,7 @@ + def build_egg(self, tmpdir, setup_script): sys.modules.setdefault('distutils.command.bdist_egg', bdist_egg) @@ -472,22 +554,54 @@ finally: log.set_verbosity(self.verbose) # restore our log verbosity + + + + + + + + + + + + + + + + + + def update_pth(self,dist): - if self.pth_file is not None: - remove = self.pth_file.remove - for d in self.pth_file.get(dist.key,()): # drop old entries - log.info("Removing %s from .pth file", d) - remove(d) - if not self.multi_version: - log.info("Adding %s to .pth file", dist) + if self.pth_file is None: + return + + for d in self.pth_file.get(dist.key,()): # drop old entries + if self.multi_version or d.path != dist.path: + log.info("Removing %s from easy-install.pth file", d) + self.pth_file.remove(d) + if d.path in self.shadow_path: + self.shadow_path.remove(d.path) + + if not self.multi_version: + if dist.path in self.pth_file.paths: + log.info( + "%s is already the active version in easy-install.pth", + dist + ) + else: + log.info("Adding %s to easy-install.pth file", dist) self.pth_file.add(dist) # add new entry - self.pth_file.save() + if dist.path not in self.shadow_path: + self.shadow_path.append(dist.path) - if dist.name=='setuptools': - # Ensure that setuptools itself never becomes unavailable! - f = open(os.path.join(self.install_dir,'setuptools.pth'), 'w') - f.write(dist.path+'\n') - f.close() + self.pth_file.save() + + if dist.name=='setuptools': + # Ensure that setuptools itself never becomes unavailable! + f = open(os.path.join(self.install_dir,'setuptools.pth'), 'w') + f.write(dist.path+'\n') + f.close() def unpack_progress(self, src, dst): @@ -496,6 +610,9 @@ return dst # only unpack-and-compile skips files for dry run + + + def unpack_and_compile(self, egg_path, destination): to_compile = [] @@ -531,6 +648,12 @@ + + + + + + def extract_wininst_cfg(dist_filename): """Extract configuration data from a bdist_wininst .exe Index: ez_setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/ez_setup.py,v retrieving revision 1.6 retrieving revision 1.7 diff -u -d -r1.6 -r1.7 --- ez_setup.py 25 Jun 2005 19:33:06 -0000 1.6 +++ ez_setup.py 27 Jun 2005 00:31:02 -0000 1.7 @@ -14,7 +14,7 @@ This file can also be run as a script to install or upgrade setuptools. """ -DEFAULT_VERSION = "0.5a3" +DEFAULT_VERSION = "0.5a4" DEFAULT_URL = "http://peak.telecommunity.com/dist/" import sys, os Index: pkg_resources.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/pkg_resources.py,v retrieving revision 1.34 retrieving revision 1.35 diff -u -d -r1.34 -r1.35 --- pkg_resources.py 15 Jun 2005 02:16:03 -0000 1.34 +++ pkg_resources.py 27 Jun 2005 00:31:02 -0000 1.35 @@ -296,7 +296,7 @@ """Remove `dist` from the distribution map""" self._distmap[dist.key].remove(dist) - def best_match(self,requirement,path=None): + def best_match(self, requirement, path=None, installer=None): """Find distribution best matching `requirement` and usable on `path` If a distribution that's already installed on `path` is unsuitable, @@ -324,9 +324,9 @@ for dist in distros: if dist in requirement: return dist - return self.obtain(requirement) # try and download + return self.obtain(requirement, installer) # try and download/install - def resolve(self, requirements, path=None): + def resolve(self, requirements, path=None, installer=None): """List all distributions needed to (recursively) meet requirements""" if path is None: @@ -344,26 +344,26 @@ continue dist = best.get(req.key) - if dist is None: # Find the best distribution and add it to the map - dist = best[req.key] = self.best_match(req,path) + dist = best[req.key] = self.best_match(req, path, installer) if dist is None: raise DistributionNotFound(req) # XXX put more info here to_install.append(dist) elif dist not in req: # Oops, the "best" so far conflicts with a dependency - raise VersionConflict(req,dist) # XXX put more info here + raise VersionConflict(dist,req) # XXX put more info here requirements.extend(dist.depends(req.options)[::-1]) processed[req] = True return to_install # return list of distros to install - def obtain(self, requirement): + def obtain(self, requirement, installer=None): """Obtain a distro that matches requirement (e.g. via download)""" - return None # override this in subclasses + if installer is not None: + return installer(requirement) def __len__(self): return len(self._distmap) @@ -1316,10 +1316,9 @@ return self.__dep_map except AttributeError: dm = self.__dep_map = {None: []} - for section,contents in split_sections( - self._get_metadata('depends.txt') - ): - dm[section] = list(parse_requirements(contents)) + for name in 'requires.txt', 'depends.txt': + for extra,reqs in split_sections(self._get_metadata(name)): + dm.setdefault(extra,[]).extend(parse_requirements(reqs)) return dm _dep_map = property(_dep_map) @@ -1351,6 +1350,7 @@ fixup_namespace_packages(self.path) map(declare_namespace, self._get_metadata('namespace_packages.txt')) + def egg_name(self): """Return what this distribution's standard .egg filename should be""" filename = "%s-%s-py%s" % ( Index: setup.py =================================================================== RCS file: /cvsroot/python/python/nondist/sandbox/setuptools/setup.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- setup.py 25 Jun 2005 19:33:06 -0000 1.16 +++ setup.py 27 Jun 2005 00:31:02 -0000 1.17 @@ -1,7 +1,7 @@ #!/usr/bin/env python """Distutils setup file, used to install or test 'setuptools'""" -VERSION = "0.5a3" +VERSION = "0.5a4" from setuptools import setup, find_packages, Require setup( @@ -42,7 +42,6 @@ packages = find_packages(), py_modules = ['pkg_resources', 'easy_install'], scripts = ['easy_install.py'], - extra_path = ('setuptools', 'setuptools-%s.egg' % VERSION), classifiers = [f.strip() for f in """ Development Status :: 3 - Alpha @@ -80,3 +79,4 @@ + From birkenfeld at users.sourceforge.net Mon Jun 27 07:51:11 2005 From: birkenfeld at users.sourceforge.net (birkenfeld@users.sourceforge.net) Date: Sun, 26 Jun 2005 22:51:11 -0700 Subject: [Python-checkins] python/dist/src/Lib/test/output test_cookie, 1.7, 1.8 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test/output In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18398/Lib/test/output Modified Files: test_cookie Log Message: Adapt output file to new Cookie JS output. Index: test_cookie =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/output/test_cookie,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- test_cookie 13 May 2001 00:19:31 -0000 1.7 +++ test_cookie 27 Jun 2005 05:51:07 -0000 1.8 @@ -16,16 +16,16 @@ Set-Cookie: keebler=E=mc2; Set-Cookie: Customer="WILE_E_COYOTE"; Path=/acme; - - From rhettinger at users.sourceforge.net Mon Jun 27 21:50:17 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Mon, 27 Jun 2005 12:50:17 -0700 Subject: [Python-checkins] python/nondist/peps pep-0327.txt,1.8,1.9 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv31728 Modified Files: pep-0327.txt Log Message: Update the list of exceptional conditions to match version 1.45 of the spec and to match the actual Python implementation of the decimal module. Index: pep-0327.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0327.txt,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- pep-0327.txt 18 May 2005 06:07:07 -0000 1.8 +++ pep-0327.txt 27 Jun 2005 19:50:05 -0000 1.9 @@ -307,12 +307,8 @@ Condition Signal Result ==================== ================= =================================== Clamped clamped see spec [2]_ -Conversion syntax invalid-operation [0,qNaN] Division by zero division-by-zero [sign,inf] -Division impossible invalid-operation [0,qNaN] -Division undefined invalid-operation [0,qNaN] Inexact inexact unchanged -Invalid context invalid-operation [0,qNaN] Invalid operation invalid-operation [0,qNaN] (or [s,qNaN] or [s,qNaN,d] when the cause is a signaling NaN) Overflow overflow depends on the rounding mode From akuchling at users.sourceforge.net Mon Jun 27 22:32:47 2005 From: akuchling at users.sourceforge.net (akuchling@users.sourceforge.net) Date: Mon, 27 Jun 2005 13:32:47 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.329, 1.330 pep-0206.txt, 1.12, 1.13 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv17543 Modified Files: pep-0000.txt pep-0206.txt Log Message: Take over PEP 206, renaming it to 'Python Advanced Library' Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.329 retrieving revision 1.330 diff -u -d -r1.329 -r1.330 --- pep-0000.txt 26 Jun 2005 14:10:44 -0000 1.329 +++ pep-0000.txt 27 Jun 2005 20:31:42 -0000 1.330 @@ -67,7 +67,7 @@ Open PEPs (under consideration) - I 206 2.0 Batteries Included Zadka + I 206 Python Advanced Library Kuchling S 209 Adding Multidimensional Arrays Barrett, Oliphant S 228 Reworking Python's Numeric Model Zadka, GvR S 237 Unifying Long Integers and Integers Zadka, GvR @@ -248,7 +248,7 @@ SF 203 Augmented Assignments Wouters SR 204 Range Literals Wouters S 205 Weak References Drake - I 206 2.0 Batteries Included Zadka + I 206 Python Advanced Library Kuchling SF 207 Rich Comparisons GvR, Ascher SF 208 Reworking the Coercion Model Schemenauer, Lemburg S 209 Adding Multidimensional Arrays Barrett, Oliphant Index: pep-0206.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0206.txt,v retrieving revision 1.12 retrieving revision 1.13 diff -u -d -r1.12 -r1.13 --- pep-0206.txt 23 Jan 2001 11:33:04 -0000 1.12 +++ pep-0206.txt 27 Jun 2005 20:31:59 -0000 1.13 @@ -1,133 +1,109 @@ PEP: 206 -Title: 2.0 Batteries Included +Title: Python Advanced Library Version: $Revision$ -Author: moshez at zadka.site.co.il (Moshe Zadka) -Python-Version: 2.0 +Author: A.M. Kuchling Status: Draft Introduction - This PEP describes the `batteries included' proposal for Python - 2.0, the fat distribution containing commonly used third party - extension modules. This PEP tracks the status and ownership of - this proposal, slated for introduction in Python 2.0. It contains - a description of the proposal and outlines how to support it. - + This PEP describes the Python Advanced Library, a collection of + high-quality and frequently-used third party extension modules. Batteries Included Philosophy - The Python source distribution always maintained the philosophy of - "batteries included" -- having a rich and versatile standard + The Python source distribution has long maintained the philosophy + of "batteries included" -- having a rich and versatile standard library which is immediately available, without making the user download separate packages. This gives the Python language a head - start in many projects. However, the Python standard library - often relies on important Open Source libraries which might be - unavailable on many computers, so that the user has to separately - download and compile those. Some examples are the zlib - compression library, and the gmp number manipulation library. + start in many projects. - The original coinage of the term "batteries included" can be found - here: + However, the standard library modules aren't always the best + choices for a job. Some library modules were quick hacks + (e.g. calendar, commands), some were designed poorly and are now + near-impossible to fix (cgi), and some have been rendered obsolete + by other, more complete modules (binascii offers the same features + as the binhex, uu, base64 modules). This PEP describes a list of + third-party modules that make Python more competitive for various + application domains, forming the Python Advanced Library. - http://www.uk.research.att.com/~fms/ipc7/tr-1998-9.html + The deliverable is a set of scripts that will retrieve, build, and + install the packages for a particular application domain. The + Python Package Index now contains enough information to let + software automatically find packages and download them, so the + time is ripe to implement this. + + Currently this document doesn't suggest *removing* modules from + the standard library that are superseded by a third-party module. + That's difficult to do because it entails many backward-compatibility + problems, so it's not worth bothering with now. + Please suggest additional domains of interest. -The Proposed Solution - The proposed solution is to maintain an alternate form of distribution, - nicknamed the "sumo" interpreter. It will be structured as a core - interpreter + extra useful libraries. In addition, the build procedure - will be changed to build those libraries by default, and build the Python - modules which rely on them linked against those libraries. Individual - users will still be able to link the Python modules against already - installed libraries, or disable them completely. +Domain: Web tasks - Additionally, some Open Source third-party Python modules will - also be distributed in the "sumo" distribution. The - difference between those and external libraries is that the former - are distributed in order to make the Python distribution self - contained, and the latter are added so there will be new - functionality in out-of-the-box Python. + XML parsing: ElementTree + SAX. -Python Advanced Library + URL retrieval: libcurl? other possibilities? - Since it is decided that the additions to the interpreter will live - in a seperate directory, they can even be distributed seperately, - as an additional library. This is called the "Python Advanced Library", - or PAL. This will also solve the problem of an advanced library module - failing -- the core interpreter will still be built, tested and installed. - Since the compilation of the other modules be helped by Python scripts, - it is quite possible that it will even install as many modules as it - can. + HTML parsing: mxTidy? HTMLParser? -Suggested Libraries and Modules + Async network I/O: Twisted - Here is the tentative list of libraries which are proposed to be - dealt with in this manner, and where they can be downloaded: + RDF parser: ??? - zlib -- http://www.info-zip.org/pub/infozip/zlib/zlib.tar.gz - expat -- ftp://ftp.jclark.com/pub/xml/expat.zip - Tcl -- http://dev.scriptics.com:80/download/tcl/tcl8_3/tcl8.3.1.tar.gz - Tk -- http://dev.scriptics.com:80/download/tcl/tcl8_3/tk8.3.1.tar.gz - PIL -- http://www.pythonware.com/downloads/Imaging-1.1.tar.gz - libjpeg -- ftp://ftp.uu.net/graphics/jpeg/jpegsrc.v6b.tar.gz - ncurses -- ftp://dickey.his.com/ncurses/ncurses.tar.gz + HTTP serving: ??? - TBD, the following: + HTTP cookie processing: ??? - NumPy -- http://download.sourceforge.net/numpy/Numerical-15.3.tgz - Pmw -- ftp://ftp.dscpl.com.au/pub/pmw/Pmw.0.8.4.tar.gz - BLT -- ftp://ftp.tcltk.com/aa004735/pub/blt/BLT2.4u.tar.gz - piddle -- http://download.sourceforge.net/piddle/piddle-1.0b7.tgz + Web framework: A WSGI gateway, perhaps? Paste? + Graphics: PIL, Chaco. -Connection to Distutils - Since Python 2.0 comes with a new mechanism of distributing Python - modules, called ``distutils'', we should be able to use it. Specificially, - since distutils has the capability or building Python modules in an - automatic fashion, it is hoped that some of the work will consist in - creating a new directory with the source, and having the ``super-make'' - run "./setup --install" in those directories. This should at least take - care of PIL, NumPy and Pmw. In addition, it is hoped that this mechanism - will be easy enough to add other Python modules as user requests direct - us. - -Software covered by the GNU General Public License +Domain: Scientific Programming - While many Python modules rely on software distributed under the - GNU General Public License and the GNU Lesser General Public - License, no such sources are proposed here to be added to the - sumo interpreter. Currently, the Python interpreter is - distributed under a closed-source-friendly license, which means - that it is possible to include it in a closed source product. - Making redistributers worry about which parts they need to remove - to make closed-source redistribution legal might cost more then - the benefits. + Numeric: Numeric, SciPy -How It Is Implemented + Graphics: PIL, Chaco. - Sumo-Python is basically be a set of scripts, which download - a Python source tarball, and source tarballs for all the modules. - Then it unpacks everything, modifies Setup.in (or whatever mechanism - will be in place by Python 2.1, and puts in a little - shell script which builds everything. Then it will repack it - into the huge tarball which will be a sumo-python-version.tar.gz, - which users can then download and install. - Inside the tarball, running the script +Domain: Application Development - build-everything [--prefix ...] + GUI toolkit: ??? - Will compile and install a sumo interpreter into specified prefix. + Graphics: Reportlab for PDF generation. + + +Domain: Education + + Graphics: PyGame + + +Software covered by the GNU General Public License + + Some of these third-party modules are covered by the GNU General + Public License and the GNU Lesser General Public License. + Providing a script to download and install such packages, or even + assembling all these packages into a single tarball or CD-ROM, + shouldn't cause any difficulties with the GPL, under the "mere + aggregation" clause of the license. + Open Issues - ESR also mentioned libpng, but I don't know of a Python module - that uses it. + What other application domains are important? - How to make PAL as easy to build as the whole Sumo Python? + Should this just be a set of Ubuntu or Debian packages? Compiling + things such as PyGame can be very complicated and may be too + difficult to automate. + + +Acknowledgements + + The PEP is based on an earlier draft PEP by Moshe Zadka, titled + "2.0 Batteries Included." Local Variables: From fdrake at users.sourceforge.net Tue Jun 28 01:22:25 2005 From: fdrake at users.sourceforge.net (fdrake@users.sourceforge.net) Date: Mon, 27 Jun 2005 16:22:25 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex, 1.146.2.7, 1.146.2.8 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv22945/lib Modified Files: Tag: release24-maint libos.tex Log Message: add note that os.EX_* constants are subject to availability on the host platform Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.146.2.7 retrieving revision 1.146.2.8 diff -u -d -r1.146.2.7 -r1.146.2.8 --- libos.tex 25 Jun 2005 20:44:12 -0000 1.146.2.7 +++ libos.tex 27 Jun 2005 23:22:22 -0000 1.146.2.8 @@ -1304,6 +1304,9 @@ \function{_exit()}, although they are not required. These are typically used for system programs written in Python, such as a mail server's external command delivery program. +\note{Some of these may not be available on all \UNIX{} platforms, +since there is some variation. These constants are defined where they +are defined by the underlying platform.} \begin{datadesc}{EX_OK} Exit code that means no error occurred. From fdrake at users.sourceforge.net Tue Jun 28 01:23:51 2005 From: fdrake at users.sourceforge.net (fdrake@users.sourceforge.net) Date: Mon, 27 Jun 2005 16:23:51 -0700 Subject: [Python-checkins] python/dist/src/Doc/lib libos.tex,1.161,1.162 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/lib In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv23635/lib Modified Files: libos.tex Log Message: add note that os.EX_* constants are subject to availability on the host platform Index: libos.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/lib/libos.tex,v retrieving revision 1.161 retrieving revision 1.162 diff -u -d -r1.161 -r1.162 --- libos.tex 25 Jun 2005 20:44:10 -0000 1.161 +++ libos.tex 27 Jun 2005 23:23:43 -0000 1.162 @@ -1327,6 +1327,9 @@ \function{_exit()}, although they are not required. These are typically used for system programs written in Python, such as a mail server's external command delivery program. +\note{Some of these may not be available on all \UNIX{} platforms, +since there is some variation. These constants are defined where they +are defined by the underlying platform.} \begin{datadesc}{EX_OK} Exit code that means no error occurred. From rhettinger at users.sourceforge.net Tue Jun 28 01:36:50 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Mon, 27 Jun 2005 16:36:50 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut tut.tex,1.272,1.273 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv30414 Modified Files: tut.tex Log Message: * Show the keyword argument form of dict(). * Note that dict works with the "in" keyword. Index: tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.272 retrieving revision 1.273 diff -u -d -r1.272 -r1.273 --- tut.tex 17 Jun 2005 10:25:33 -0000 1.272 +++ tut.tex 27 Jun 2005 23:36:47 -0000 1.273 @@ -2146,8 +2146,8 @@ The \method{keys()} method of a dictionary object returns a list of all the keys used in the dictionary, in arbitrary order (if you want it sorted, just apply the \method{sort()} method to the list of keys). To -check whether a single key is in the dictionary, use the -\method{has_key()} method of the dictionary. +check whether a single key is in the dictionary, either use the dictionary's +\method{has_key()} method or the \keyword{in} keyword. Here is a small example using a dictionary: @@ -2166,6 +2166,8 @@ ['guido', 'irv', 'jack'] >>> tel.has_key('guido') True +>>> 'guido' in tel +True \end{verbatim} The \function{dict()} constructor builds dictionaries directly from @@ -2183,6 +2185,14 @@ which are even better suited for the task of supplying key-values pairs to the \function{dict()} constructor. +When the keys are simple strings, it is sometimes easier to specify +pairs using keyword arguments: + +\begin{verbatim} +>>> dict(sape=4139, guido=4127, jack=4098) +{'sape': 4139, 'jack': 4098, 'guido': 4127} +\end{verbatim} + \section{Looping Techniques \label{loopidioms}} From rhettinger at users.sourceforge.net Tue Jun 28 01:39:00 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Mon, 27 Jun 2005 16:39:00 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut tut.tex, 1.261.2.7, 1.261.2.8 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv32348 Modified Files: Tag: release24-maint tut.tex Log Message: * Show the keyword argument form of dict(). * Note that dict works with the "in" keyword. Index: tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.261.2.7 retrieving revision 1.261.2.8 diff -u -d -r1.261.2.7 -r1.261.2.8 --- tut.tex 17 Jun 2005 10:27:50 -0000 1.261.2.7 +++ tut.tex 27 Jun 2005 23:38:47 -0000 1.261.2.8 @@ -2146,8 +2146,8 @@ The \method{keys()} method of a dictionary object returns a list of all the keys used in the dictionary, in arbitrary order (if you want it sorted, just apply the \method{sort()} method to the list of keys). To -check whether a single key is in the dictionary, use the -\method{has_key()} method of the dictionary. +check whether a single key is in the dictionary, either use the dictionary's +\method{has_key()} method or the \keyword{in} keyword. Here is a small example using a dictionary: @@ -2166,6 +2166,8 @@ ['guido', 'irv', 'jack'] >>> tel.has_key('guido') True +>>> 'guido' in tel +True \end{verbatim} The \function{dict()} constructor builds dictionaries directly from @@ -2183,6 +2185,14 @@ which are even better suited for the task of supplying key-values pairs to the \function{dict()} constructor. +When the keys are simple strings, it is sometimes easier to specify +pairs using keyword arguments: + +\begin{verbatim} +>>> dict(sape=4139, guido=4127, jack=4098) +{'sape': 4139, 'jack': 4098, 'guido': 4127} +\end{verbatim} + \section{Looping Techniques \label{loopidioms}} From rhettinger at users.sourceforge.net Tue Jun 28 02:16:12 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Mon, 27 Jun 2005 17:16:12 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut tut.tex,1.273,1.274 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv19720 Modified Files: tut.tex Log Message: Note that files are iterable. Index: tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.273 retrieving revision 1.274 diff -u -d -r1.273 -r1.274 --- tut.tex 27 Jun 2005 23:36:47 -0000 1.273 +++ tut.tex 28 Jun 2005 00:16:08 -0000 1.274 @@ -3166,6 +3166,21 @@ ['This is the first line of the file.\n', 'Second line of the file\n'] \end{verbatim} +An alternate approach to reading lines is to loop over the file object. +This is memory efficient, fast, and leads to simpler code: + +\begin{verbatim} +>>> for line in f: + print line, + +This is the first line of the file. +Second line of the file +\end{verbatim} + +The alternative approach is simpler but does not provide as fine-grained +control. Since the two approaches manage line buffering differently, +they should not be mixed. + \code{f.write(\var{string})} writes the contents of \var{string} to the file, returning \code{None}. From rhettinger at users.sourceforge.net Tue Jun 28 02:18:14 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Mon, 27 Jun 2005 17:18:14 -0700 Subject: [Python-checkins] python/dist/src/Doc/tut tut.tex, 1.261.2.8, 1.261.2.9 Message-ID: Update of /cvsroot/python/python/dist/src/Doc/tut In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv20790 Modified Files: Tag: release24-maint tut.tex Log Message: Note that file objects are iterable. Index: tut.tex =================================================================== RCS file: /cvsroot/python/python/dist/src/Doc/tut/tut.tex,v retrieving revision 1.261.2.8 retrieving revision 1.261.2.9 diff -u -d -r1.261.2.8 -r1.261.2.9 --- tut.tex 27 Jun 2005 23:38:47 -0000 1.261.2.8 +++ tut.tex 28 Jun 2005 00:18:10 -0000 1.261.2.9 @@ -3166,6 +3166,21 @@ ['This is the first line of the file.\n', 'Second line of the file\n'] \end{verbatim} +An alternate approach to reading lines is to loop over the file object. +This is memory efficient, fast, and leads to simpler code: + +\begin{verbatim} +>>> for line in f: + print line, + +This is the first line of the file. +Second line of the file +\end{verbatim} + +The alternative approach is simpler but does not provide as fine-grained +control. Since the two approaches manage line buffering differently, +they should not be mixed. + \code{f.write(\var{string})} writes the contents of \var{string} to the file, returning \code{None}. From rhettinger at users.sourceforge.net Tue Jun 28 09:39:11 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 28 Jun 2005 00:39:11 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.330, 1.331 pep-0254.txt, 1.1, 1.2 pep-0342.txt, 1.8, 1.9 pep-0343.txt, 1.26, 1.27 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv21657 Modified Files: pep-0000.txt pep-0254.txt pep-0342.txt pep-0343.txt Log Message: Mark 342 and 343 as accepted. Mark 254 as abandoned. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.330 retrieving revision 1.331 diff -u -d -r1.330 -r1.331 --- pep-0000.txt 27 Jun 2005 20:31:42 -0000 1.330 +++ pep-0000.txt 28 Jun 2005 07:39:09 -0000 1.331 @@ -64,6 +64,8 @@ Accepted PEPs (accepted; may not be implemented yet) SA 328 Imports: Multi-Line and Absolute/Relative Aahz + SA 342 Coroutines via Enhanced Generators GvR, Eby + SA 343 Anonymous Block Redux and Generator Enhancements GvR Open PEPs (under consideration) @@ -74,7 +76,6 @@ S 243 Module Repository Upload Mechanism Reifschneider S 245 Python Interface Syntax Pelletier S 246 Object Adaptation Evans - S 254 Making Classes Look More Like Types GvR S 256 Docstring Processing System Framework Goodger S 258 Docutils Design Specification Goodger S 266 Optimizing Global Variable/Attribute Access Montanaro @@ -105,8 +106,6 @@ S 337 Logging Usage in the Standard Library Dubner S 338 Executing modules inside packages with '-m' Coghlan S 341 Unifying try-except and try-finally Birkenfeld - S 342 Coroutines via Enhanced Generators GvR, Eby - S 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones S 754 IEEE 754 Floating Point Special Values Warnes @@ -190,6 +189,7 @@ SR 240 Adding a Rational Literal to Python Craig, Zadka SR 242 Numeric Kinds Dubois SR 244 The `directive' Statement von Loewis + SR 254 Making Classes Look More Like Types GvR SR 259 Omit printing newline after newline GvR SD 262 Database of Installed Python Packages Kuchling SR 265 Sorting Dictionaries by Value Griffin @@ -296,7 +296,7 @@ IF 251 Python 2.2 Release Schedule Warsaw, GvR SF 252 Making Types Look More Like Classes GvR SF 253 Subtyping Built-in Types GvR - S 254 Making Classes Look More Like Types GvR + SR 254 Making Classes Look More Like Types GvR SF 255 Simple Generators Schemenauer, et al S 256 Docstring Processing System Framework Goodger I 257 Docstring Conventions Goodger, GvR @@ -383,8 +383,8 @@ I 339 How to Change CPython's Bytecode Cannon SR 340 Anonymous Block Statements GvR S 341 Unifying try-except and try-finally Birkenfeld - S 342 Coroutines via Enhanced Generators GvR, Eby - S 343 Anonymous Block Redux and Generator Enhancements GvR + SA 342 Coroutines via Enhanced Generators GvR, Eby + SA 343 Anonymous Block Redux and Generator Enhancements GvR S 344 Exception Chaining and Embedded Tracebacks Yee S 345 Metadata for Python Software Packages 1.2 Jones SR 346 User Defined ("with") Statements Coghlan Index: pep-0254.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0254.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0254.txt 18 Jun 2001 18:13:07 -0000 1.1 +++ pep-0254.txt 28 Jun 2005 07:39:09 -0000 1.2 @@ -2,7 +2,7 @@ Title: Making Classes Look More Like Types Version: $Revision$ Author: guido at python.org (Guido van Rossum) -Status: Draft +Status: Rejected Type: Standards Track Python-Version: 2.2 Created: 18-June-2001 @@ -12,6 +12,12 @@ This PEP has not been written yet. Watch this space! +Status + + This PEP was a stub entry and eventually abandoned without having + been filled-out. Substantially most of the intended functionality + was implemented in Py2.2 with new-style types and classes. + Copyright Index: pep-0342.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0342.txt,v retrieving revision 1.8 retrieving revision 1.9 diff -u -d -r1.8 -r1.9 --- pep-0342.txt 26 Jun 2005 16:51:37 -0000 1.8 +++ pep-0342.txt 28 Jun 2005 07:39:09 -0000 1.9 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Guido van Rossum, Phillip J. Eby -Status: Draft +Status: Accepted Type: Standards Track Content-Type: text/plain Created: 10-May-2005 Index: pep-0343.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0343.txt,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- pep-0343.txt 26 Jun 2005 02:21:21 -0000 1.26 +++ pep-0343.txt 28 Jun 2005 07:39:09 -0000 1.27 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Guido van Rossum -Status: Draft +Status: Accepted Type: Standards Track Content-Type: text/plain Created: 13-May-2005 From rhettinger at users.sourceforge.net Tue Jun 28 10:00:15 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 28 Jun 2005 01:00:15 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.331, 1.332 pep-0294.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1698 Modified Files: pep-0000.txt pep-0294.txt Log Message: Reject 294's proposed updates to the "types" and "new" modules. Instead, missing type names can be added to __builtin__ or sys. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.331 retrieving revision 1.332 diff -u -d -r1.331 -r1.332 --- pep-0000.txt 28 Jun 2005 07:39:09 -0000 1.331 +++ pep-0000.txt 28 Jun 2005 08:00:12 -0000 1.332 @@ -85,7 +85,6 @@ S 280 Optimizing access to globals GvR S 286 Enhanced Argument Tuples von Loewis I 287 reStructuredText Docstring Format Goodger - S 294 Type Names in the types Module Tirosh S 297 Support for System Upgrades Lemburg S 298 The Locked Buffer Interface Heller S 299 Special __main__() function in modules Epler @@ -201,6 +200,7 @@ SR 281 Loop Counter Iteration with range and xrange Hetland SR 284 Integer for-loops Eppstein, Ewing SR 288 Generators Attributes and Exceptions Hettinger + SR 294 Type Names in the types Module Tirosh SR 295 Interpretation of multiline string constants Koltsov SR 296 Adding a bytes Object Type Gilbert SR 303 Extend divmod() for Multiple Divisors Bellman @@ -336,7 +336,7 @@ I 291 Backward Compatibility for Standard Library Norwitz SF 292 Simpler String Substitutions Warsaw SF 293 Codec Error Handling Callbacks Dörwald - S 294 Type Names in the types Module Tirosh + SR 294 Type Names in the types Module Tirosh SR 295 Interpretation of multiline string constants Koltsov SR 296 Adding a bytes Object Type Gilbert S 297 Support for System Upgrades Lemburg Index: pep-0294.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0294.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0294.txt 23 Jun 2002 23:52:19 -0000 1.1 +++ pep-0294.txt 28 Jun 2005 08:00:12 -0000 1.2 @@ -3,10 +3,10 @@ Version: $Revision$ Last-Modified: $Date$ Author: oren at hishome.net (Oren Tirosh) -Status: Draft +Status: Rejected Type: Standards track Created: 19-Jun-2002 -Python-Version: 2.3 +Python-Version: 2.5 Post-History: @@ -29,6 +29,21 @@ PEP 4. +Pronouncement + + A centralized repository of type names was a mistake. Neither the + "types" nor "new" modules should be carried forward to Python 3.0. + + In the meantime, it does not make sense to make the proposed updates + to the modules. This would cause disruption without any compensating + benefit. + + Instead, the problem that some internal types (frames, functions, + etc.) don't live anywhere outside those modules may be addressed by + either adding them to __builtin__ or sys. This will provide a + smoother transition to Python 3.0. + + Rationale Using two sets of names for the same objects is redundant and From rhettinger at users.sourceforge.net Tue Jun 28 10:14:48 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 28 Jun 2005 01:14:48 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.332, 1.333 pep-0330.txt, 1.1, 1.2 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv8753 Modified Files: pep-0000.txt pep-0330.txt Log Message: Closing PEP 330 -- Bytecode Verification. If someone wants to write and test some code, no PEP is required. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.332 retrieving revision 1.333 diff -u -d -r1.332 -r1.333 --- pep-0000.txt 28 Jun 2005 08:00:12 -0000 1.332 +++ pep-0000.txt 28 Jun 2005 08:14:45 -0000 1.333 @@ -97,7 +97,6 @@ S 321 Date/Time Parsing and Formatting Kuchling S 323 Copyable Iterators Martelli S 325 Resource-Release Support for Generators Pedroni - S 330 Python Bytecode Verification Pelletier S 331 Locale-Independent Float/String conversions Reis S 332 Byte vectors and String/Unicode Unification Montanaro S 334 Simple Coroutines via SuspendIteration Evans @@ -211,6 +210,7 @@ SR 317 Eliminate Implicit Exception Instantiation Taschuk SR 326 A Case for Top and Bottom Values Carlson, Reedy SR 329 Treating Builtins as Constants in the Standard Library Hettinger + SR 330 Python Bytecode Verification Pelletier SR 336 Make None Callable McClelland SR 340 Anonymous Block Statements GvR SR 346 User Defined ("with") Statements Coghlan @@ -371,7 +371,7 @@ SF 327 Decimal Data Type Batista SA 328 Imports: Multi-Line and Absolute/Relative Aahz SR 329 Treating Builtins as Constants in the Standard Library Hettinger - S 330 Python Bytecode Verification Pelletier + SR 330 Python Bytecode Verification Pelletier S 331 Locale-Independent Float/String conversions Reis S 332 Byte vectors and String/Unicode Unification Montanaro I 333 Python Web Server Gateway Interface v1.0 Eby Index: pep-0330.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0330.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0330.txt 28 May 2004 01:45:34 -0000 1.1 +++ pep-0330.txt 28 Jun 2005 08:14:46 -0000 1.2 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Michel Pelletier -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/plain Created: 17-Jun-2004 @@ -24,6 +24,17 @@ of Python Virtual Machine (PVM) bytecode and provides an implementation in Python of this verification process. +Pronouncement + + Guido believes that a verification tool has some value. If + someone wants to add it to Tools/scripts, no PEP is required. + + Such a tool may have value for validating the output from + "bytecodehacks" or from direct edits of PYC files. As security + measure, its value is somewhat limited because perfectly valid + bytecode can still do horrible things. That situation could + change if the concept of restricted execution were to be + successfully resurrected. Motivation From rhettinger at users.sourceforge.net Tue Jun 28 10:31:11 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 28 Jun 2005 01:31:11 -0700 Subject: [Python-checkins] python/nondist/peps pep-0000.txt, 1.334, 1.335 pep-0310.txt, 1.5, 1.6 pep-0319.txt, 1.1, 1.2 pep-0325.txt, 1.2, 1.3 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv18231 Modified Files: pep-0000.txt pep-0310.txt pep-0319.txt pep-0325.txt Log Message: Reject 310 and 319 in favor of 343. Index: pep-0000.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v retrieving revision 1.334 retrieving revision 1.335 diff -u -d -r1.334 -r1.335 --- pep-0000.txt 28 Jun 2005 08:20:44 -0000 1.334 +++ pep-0000.txt 28 Jun 2005 08:31:08 -0000 1.335 @@ -90,10 +90,8 @@ S 299 Special __main__() function in modules Epler S 302 New Import Hooks JvR S 304 Controlling Generation of Bytecode Files Montanaro - S 310 Reliable Acquisition/Release Pairs Hudson, Moore S 314 Metadata for Python Software Packages v1.1 Kuchling S 315 Enhanced While Loop Carroll, Hettinger - S 319 Python Synchronize/Asynchronize Block Pelletier S 321 Date/Time Parsing and Formatting Kuchling S 323 Copyable Iterators Martelli S 331 Locale-Independent Float/String conversions Reis @@ -203,10 +201,12 @@ SR 296 Adding a bytes Object Type Gilbert SR 303 Extend divmod() for Multiple Divisors Bellman SR 308 If-then-else expression GvR, Hettinger + SR 310 Reliable Acquisition/Release Pairs Hudson, Moore SD 312 Simple Implicit Lambda Suzi, Martelli SR 313 Adding Roman Numeral Literals to Python Meyer SD 316 Programming by Contract for Python Way SR 317 Eliminate Implicit Exception Instantiation Taschuk + SR 319 Python Synchronize/Asynchronize Block Pelletier SR 325 Resource-Release Support for Generators Pedroni SR 326 A Case for Top and Bottom Values Carlson, Reedy SR 329 Treating Builtins as Constants in the Standard Library Hettinger @@ -360,7 +360,7 @@ SD 316 Programming by Contract for Python Way SR 317 Eliminate Implicit Exception Instantiation Taschuk SF 318 Decorators for Functions and Methods Smith, et al - S 319 Python Synchronize/Asynchronize Block Pelletier + SR 319 Python Synchronize/Asynchronize Block Pelletier IF 320 Python 2.4 Release Schedule Warsaw, et al S 321 Date/Time Parsing and Formatting Kuchling SF 322 Reverse Iteration Hettinger Index: pep-0310.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0310.txt,v retrieving revision 1.5 retrieving revision 1.6 diff -u -d -r1.5 -r1.6 --- pep-0310.txt 22 Apr 2005 08:52:49 -0000 1.5 +++ pep-0310.txt 28 Jun 2005 08:31:09 -0000 1.6 @@ -4,7 +4,7 @@ Last-Modified: $Date$ Author: Michael Hudson , Paul Moore -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/plain Created: 18-Dec-2002 @@ -25,6 +25,9 @@ This PEP proposes a piece of syntax (a 'with' block) and a "small-i" interface that generalizes the above. +Pronouncement + + This PEP is rejected in favor of PEP 343. Rationale Index: pep-0319.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0319.txt,v retrieving revision 1.1 retrieving revision 1.2 diff -u -d -r1.1 -r1.2 --- pep-0319.txt 14 Jun 2003 22:40:36 -0000 1.1 +++ pep-0319.txt 28 Jun 2005 08:31:09 -0000 1.2 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Michel Pelletier -Status: Draft +Status: Rejected Type: Standards Track Created: 24-Feb-2003 Python-Version: 2.4? @@ -15,6 +15,9 @@ This PEP proposes adding two new keywords to Python, `synchronize' and 'asynchronize'. +Pronouncement + + This PEP is rejected in favor of PEP 343. The `synchronize' Keyword Index: pep-0325.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0325.txt,v retrieving revision 1.2 retrieving revision 1.3 diff -u -d -r1.2 -r1.3 --- pep-0325.txt 28 Jun 2005 08:20:44 -0000 1.2 +++ pep-0325.txt 28 Jun 2005 08:31:09 -0000 1.3 @@ -3,7 +3,7 @@ Version: $Revision$ Last-Modified: $Date$ Author: Samuele Pedroni -Status: Draft +Status: Rejected Type: Standards Track Content-Type: text/plain Created: 25-Aug-2003 From rhettinger at users.sourceforge.net Tue Jun 28 10:46:42 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Tue, 28 Jun 2005 01:46:42 -0700 Subject: [Python-checkins] python/nondist/peps pep-0323.txt,1.3,1.4 Message-ID: Update of /cvsroot/python/python/nondist/peps In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv25380 Modified Files: pep-0323.txt Log Message: Add comments to 323. Index: pep-0323.txt =================================================================== RCS file: /cvsroot/python/python/nondist/peps/pep-0323.txt,v retrieving revision 1.3 retrieving revision 1.4 diff -u -d -r1.3 -r1.4 --- pep-0323.txt 2 Jan 2004 09:15:55 -0000 1.3 +++ pep-0323.txt 28 Jun 2005 08:46:39 -0000 1.4 @@ -7,7 +7,7 @@ Type: Standards Track Content-Type: text/plain Created: 25-Oct-2003 -Python-Version: 2.4 +Python-Version: 2.5 Post-History: 29-Oct-2003 @@ -19,6 +19,18 @@ might exploit such a __copy__ method when present. +Update and Comments + + Support for __copy__ was included in Py2.4's itertools.tee(). + + Adding __copy__ methods to existing iterators will change the + behavior under tee(). Currently, the copied iterators remain + tied to the original iterator. If the original advances, then + so do all of the copies. Good practice is to overwrite the + original so that anamolies don't result: a,b=tee(a). + Code that doesn't follow that practice may observe a semantic + change if a __copy__ method is added to an iterator. + Motivation In Python up to 2.3, most built-in iterator types don't let the user From jackjansen at users.sourceforge.net Tue Jun 28 17:14:40 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Tue, 28 Jun 2005 08:14:40 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenBuffer.py, 1.10, 1.11 bgenStackBuffer.py, 1.4, 1.5 bgenStringBuffer.py, 1.4, 1.5 bgenType.py, 1.16, 1.17 bgenVariable.py, 1.7, 1.8 scantools.py, 1.38, 1.39 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1216 Modified Files: bgenBuffer.py bgenStackBuffer.py bgenStringBuffer.py bgenType.py bgenVariable.py scantools.py Log Message: More factorization to help C++ support. Index: bgenBuffer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenBuffer.py,v retrieving revision 1.10 retrieving revision 1.11 diff -u -d -r1.10 -r1.11 --- bgenBuffer.py 22 Jun 2005 20:35:23 -0000 1.10 +++ bgenBuffer.py 28 Jun 2005 15:14:34 -0000 1.11 @@ -38,10 +38,11 @@ self.sizeformat = sizeformat or type2format[sizetype] self.label_needed = 0 - def getDeclarations(self, name, reference=False): + def getArgDeclarations(self, name, reference=False): if reference: raise RuntimeError, "Cannot pass buffer types by reference" - return self.getBufferDeclarations(name) + self.getSizeDeclarations(name) + return (self.getBufferDeclarations(name) + + self.getSizeDeclarations(name)) def getBufferDeclarations(self, name): return self.getInputBufferDeclarations(name) + self.getOutputBufferDeclarations(name) @@ -53,10 +54,10 @@ return ["%s %s__out__[%s]" % (self.datatype, name, self.size)] def getSizeDeclarations(self, name): - return [ - "%s %s__len__" %(self.sizetype, name), - "int %s__in_len__" %(name) - ] + return ["%s %s__len__" %(self.sizetype, name)] + + def getAuxDeclarations(self, name): + return ["int %s__in_len__" %(name)] def getargsFormat(self): return "s#" @@ -189,6 +190,9 @@ return ["%s *%s__in__" % (self.type, name)] def getSizeDeclarations(self, name): + return [] + + def getAuxDeclarations(self, name): return ["int %s__in_len__" % (name)] def getOutputBufferDeclarations(self, name): @@ -248,6 +252,9 @@ def getSizeDeclarations(self, name): return [] + def getAuxDeclarations(self, name): + return [] + def passOutput(self, name): return "&%s__out__" % name @@ -262,5 +269,8 @@ def getSizeDeclarations(self, name): return [] + def getAuxDeclarations(self, name): + return [] + def passOutput(self, name): return "%s__out__" % name Index: bgenStackBuffer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenStackBuffer.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- bgenStackBuffer.py 22 Jun 2005 20:35:23 -0000 1.4 +++ bgenStackBuffer.py 28 Jun 2005 15:14:35 -0000 1.5 @@ -23,6 +23,9 @@ """ def getSizeDeclarations(self, name): + return [] + + def getAuxDeclarations(self, name): return ["int %s__len__ = %s" % (name, self.size)] def passOutput(self, name): Index: bgenStringBuffer.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenStringBuffer.py,v retrieving revision 1.4 retrieving revision 1.5 diff -u -d -r1.4 -r1.5 --- bgenStringBuffer.py 22 Jun 2005 20:35:23 -0000 1.4 +++ bgenStringBuffer.py 28 Jun 2005 15:14:35 -0000 1.5 @@ -26,6 +26,9 @@ def getSizeDeclarations(self, name): return [] + def getAuxDeclarations(self, name): + return [] + def getargsFormat(self): return "s" Index: bgenType.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenType.py,v retrieving revision 1.16 retrieving revision 1.17 diff -u -d -r1.16 -r1.17 --- bgenType.py 24 Jun 2005 19:46:53 -0000 1.16 +++ bgenType.py 28 Jun 2005 15:14:35 -0000 1.17 @@ -24,17 +24,25 @@ Example: int.declare('spam') prints "int spam;" """ - for decl in self.getDeclarations(name, reference): + for decl in self.getArgDeclarations(name, reference): + Output("%s;", decl) + for decl in self.getAuxDeclarations(name): Output("%s;", decl) - def getDeclarations(self, name, reference=False): - """Return a string declaring a variable or argument, without - any syntactic adornment""" + def getArgDeclarations(self, name, reference=False): + """Return the main part of the declarations for this type: the items + that will be passed as arguments in the C/C++ function call.""" if reference: return ["%s& %s" % (self.typeName, name)] else: return ["%s %s" % (self.typeName, name)] + def getAuxDeclarations(self, name): + """Return any auxiliary declarations needed for implementing this + type, such as helper variables used to hold sizes, etc. These declarations + are not part of the C/C++ function call interface.""" + return [] + def getargs(self): return self.getargsFormat(), self.getargsArgs() @@ -187,7 +195,10 @@ self.substitute = substitute self.typeName = None # Don't show this argument in __doc__ string - def getDeclarations(self, name, reference=False): + def getArgDeclarations(self, name, reference=False): + return [] + + def getAuxDeclarations(self, name, reference=False): return [] def getargsFormat(self): Index: bgenVariable.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenVariable.py,v retrieving revision 1.7 retrieving revision 1.8 diff -u -d -r1.7 -r1.8 --- bgenVariable.py 24 Jun 2005 19:46:53 -0000 1.7 +++ bgenVariable.py 28 Jun 2005 15:14:35 -0000 1.8 @@ -45,11 +45,12 @@ elif self.flags != SelfMode: self.type.declare(self.name) - def getDeclarations(self): - """Return the unadorned declaration of the variable, - suitable for use in a formal parameter list.""" + def getArgDeclarations(self): refmode = (self.flags & RefMode) - return self.type.getDeclarations(self.name, reference=refmode) + return self.type.getArgDeclarations(self.name, reference=refmode) + + def getAuxDeclarations(self): + return self.type.getAuxDeclarations(self.name) def getargsFormat(self): """Call the type's getargsFormatmethod.""" Index: scantools.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/scantools.py,v retrieving revision 1.38 retrieving revision 1.39 diff -u -d -r1.38 -r1.39 --- scantools.py 24 Jun 2005 19:46:53 -0000 1.38 +++ scantools.py 28 Jun 2005 15:14:35 -0000 1.39 @@ -482,8 +482,7 @@ modifiers = self.getmodifiers(match) type = self.pythonizename(type) name = self.pythonizename(name) - if name in self.alreadydone: - self.report("Name has already been defined: %r", name) + if self.checkduplicate(name): return self.report("==> %s %s <==", type, name) if self.blacklisted(type, name): @@ -499,7 +498,6 @@ ## self.report(" %r", arg) self.report("*** %s %s unmanageable", type, name) return - self.alreadydone.append(name) if modifiers: self.generate(type, name, arglist, modifiers) else: @@ -508,6 +506,13 @@ def getmodifiers(self, match): return [] + def checkduplicate(self, name): + if name in self.alreadydone: + self.report("Name has already been defined: %r", name) + return True + self.alreadydone.append(name) + return False + def pythonizename(self, name): name = re.sub("\*", " ptr", name) name = name.strip() From tycoon200 at dslextreme.com Wed Jun 29 00:57:58 2005 From: tycoon200 at dslextreme.com (tycoon200@dslextreme.com) Date: Tue, 28 Jun 2005 15:57:58 -0700 (PDT) Subject: [Python-checkins] Python Tutorial Problem Message-ID: <35454ac5bf2a9d63ea7eb82a.20050628155758.glpbba200@www.dslextreme.com> Hey i have python 2.4.1. I seem i have a problem with this tutorial http://www.freenetpages.co.uk/hp/alan.gauld/ When im done writing my scripys in 2.4.1 i can not seem to find Run Script. I have look under File,Edit,Windows this is my first time doing codeing. Is this the right tutorial for 2.4.1 and if not are all python the same . From jackjansen at users.sourceforge.net Wed Jun 29 16:17:09 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Wed, 29 Jun 2005 07:17:09 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenGenerator.py, 1.18, 1.19 bgenObjectDefinition.py, 1.30, 1.31 scantools.py, 1.39, 1.40 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv6541 Modified Files: bgenGenerator.py bgenObjectDefinition.py scantools.py Log Message: Added optional suppport for storage modifiers (virtual/static/inline/etc) and conditional generation of objects and methods. Index: bgenGenerator.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenGenerator.py,v retrieving revision 1.18 retrieving revision 1.19 diff -u -d -r1.18 -r1.19 --- bgenGenerator.py 24 Jun 2005 19:46:36 -0000 1.18 +++ bgenGenerator.py 29 Jun 2005 14:17:04 -0000 1.19 @@ -15,18 +15,27 @@ class BaseFunctionGenerator: - def __init__(self, name, condition=None): + def __init__(self, name, condition=None, callname=None, modifiers=None): if DEBUG: print "<--", name self.name = name - self.callname = name + if callname: + self.callname = callname + else: + self.callname = name self.prefix = name self.objecttype = "PyObject" # Type of _self argument to function self.condition = condition + self.modifiers = modifiers def setprefix(self, prefix): self.prefix = prefix + def checkgenerate(self): + return True + def generate(self): + if not self.checkgenerate(): + return if DEBUG: print "-->", self.name if self.condition: Output() @@ -51,6 +60,8 @@ OutRbrace() def reference(self, name = None): + if not self.checkgenerate(): + return if name is None: name = self.name docstring = self.docstring() Index: bgenObjectDefinition.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenObjectDefinition.py,v retrieving revision 1.30 retrieving revision 1.31 diff -u -d -r1.30 -r1.31 --- bgenObjectDefinition.py 24 Jun 2005 19:46:52 -0000 1.30 +++ bgenObjectDefinition.py 29 Jun 2005 14:17:05 -0000 1.31 @@ -219,6 +219,9 @@ Output("if (PyType_Ready(&%s) < 0) return;", self.typename) Output("""Py_INCREF(&%s);""", self.typename) Output("PyModule_AddObject(m, \"%s\", (PyObject *)&%s);", self.name, self.typename); + self.outputTypeObjectInitializerCompat() + + def outputTypeObjectInitializerCompat(self): Output("/* Backward-compatible name */") Output("""Py_INCREF(&%s);""", self.typename); Output("PyModule_AddObject(m, \"%sType\", (PyObject *)&%s);", self.name, self.typename); Index: scantools.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/scantools.py,v retrieving revision 1.39 retrieving revision 1.40 diff -u -d -r1.39 -r1.40 --- scantools.py 28 Jun 2005 15:14:35 -0000 1.39 +++ scantools.py 29 Jun 2005 14:17:05 -0000 1.40 @@ -620,11 +620,15 @@ (atype, aname, amode)) if self.greydictnames.has_key(name): self.specfile.write(" condition=%r,\n"%(self.greydictnames[name],)) + self.generatemodifiers(classname, name, modifiers) self.specfile.write(")\n") self.specfile.write("%s.append(f)\n\n" % listname) def destination(self, type, name, arglist): return "FunctionGenerator", "functions" + + def generatemodifiers(self, classname, name, modifiers): + pass def blacklisted(self, type, name): if type in self.blacklisttypes: From rhettinger at users.sourceforge.net Thu Jun 30 01:29:59 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Wed, 29 Jun 2005 16:29:59 -0700 Subject: [Python-checkins] python/dist/src/Lib/test test_long.py,1.26,1.27 Message-ID: Update of /cvsroot/python/python/dist/src/Lib/test In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14804/Lib/test Modified Files: test_long.py Log Message: SF bug #1224347: int/long unification and hex() Hex longs now print with lowercase letters like their int counterparts. Index: test_long.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/test/test_long.py,v retrieving revision 1.26 retrieving revision 1.27 diff -u -d -r1.26 -r1.27 --- test_long.py 13 Jun 2005 21:44:48 -0000 1.26 +++ test_long.py 29 Jun 2005 23:29:56 -0000 1.27 @@ -208,7 +208,7 @@ digits = digits or [0] return '-'[:sign] + \ {8: '0', 10: '', 16: '0x'}[base] + \ - "".join(map(lambda i: "0123456789ABCDEF"[i], digits)) + "L" + "".join(map(lambda i: "0123456789abcdef"[i], digits)) + "L" def check_format_1(self, x): for base, mapper in (8, oct), (10, repr), (16, hex): From rhettinger at users.sourceforge.net Thu Jun 30 01:29:59 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Wed, 29 Jun 2005 16:29:59 -0700 Subject: [Python-checkins] python/dist/src/Objects stringobject.c, 2.229, 2.230 longobject.c, 1.167, 1.168 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14804/Objects Modified Files: stringobject.c longobject.c Log Message: SF bug #1224347: int/long unification and hex() Hex longs now print with lowercase letters like their int counterparts. Index: stringobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/stringobject.c,v retrieving revision 2.229 retrieving revision 2.230 diff -u -d -r2.229 -r2.230 --- stringobject.c 20 Feb 2005 09:54:52 -0000 2.229 +++ stringobject.c 29 Jun 2005 23:29:54 -0000 2.230 @@ -3753,18 +3753,12 @@ } /* Fix up case for hex conversions. */ - switch (type) { - case 'x': - /* Need to convert all upper case letters to lower case. */ + if (type == 'X') { + /* Need to convert all lower case letters to upper case. + and need to convert 0x to 0X (and -0x to -0X). */ for (i = 0; i < len; i++) - if (buf[i] >= 'A' && buf[i] <= 'F') - buf[i] += 'a'-'A'; - break; - case 'X': - /* Need to convert 0x to 0X (and -0x to -0X). */ - if (buf[sign + 1] == 'x') - buf[sign + 1] = 'X'; - break; + if (buf[i] >= 'a' && buf[i] <= 'x') + buf[i] -= 'a'-'A'; } *pbuf = buf; *plen = len; Index: longobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/longobject.c,v retrieving revision 1.167 retrieving revision 1.168 diff -u -d -r1.167 -r1.168 --- longobject.c 26 Apr 2005 03:45:25 -0000 1.167 +++ longobject.c 29 Jun 2005 23:29:55 -0000 1.168 @@ -1090,7 +1090,7 @@ assert(accumbits >= basebits); do { char cdigit = (char)(accum & (base - 1)); - cdigit += (cdigit < 10) ? '0' : 'A'-10; + cdigit += (cdigit < 10) ? '0' : 'a'-10; assert(p > PyString_AS_STRING(str)); *--p = cdigit; accumbits -= basebits; @@ -1144,7 +1144,7 @@ digit nextrem = (digit)(rem / base); char c = (char)(rem - nextrem * base); assert(p > PyString_AS_STRING(str)); - c += (c < 10) ? '0' : 'A'-10; + c += (c < 10) ? '0' : 'a'-10; *--p = c; rem = nextrem; --ntostore; From rhettinger at users.sourceforge.net Thu Jun 30 01:30:33 2005 From: rhettinger at users.sourceforge.net (rhettinger@users.sourceforge.net) Date: Wed, 29 Jun 2005 16:30:33 -0700 Subject: [Python-checkins] python/dist/src/Misc NEWS,1.1307,1.1308 Message-ID: Update of /cvsroot/python/python/dist/src/Misc In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv14804/Misc Modified Files: NEWS Log Message: SF bug #1224347: int/long unification and hex() Hex longs now print with lowercase letters like their int counterparts. Index: NEWS =================================================================== RCS file: /cvsroot/python/python/dist/src/Misc/NEWS,v retrieving revision 1.1307 retrieving revision 1.1308 diff -u -d -r1.1307 -r1.1308 --- NEWS 26 Jun 2005 22:54:58 -0000 1.1307 +++ NEWS 29 Jun 2005 23:29:43 -0000 1.1308 @@ -12,6 +12,9 @@ Core and builtins ----------------- +- SF bug #1224347: hex longs now print with lowercase letters just + like their int counterparts. + - SF bug #1163563: the original fix for bug #1010677 ("thread Module Breaks PyGILState_Ensure()") broke badly in the case of multiple interpreter states; back out that fix and do a better job (see From mwh at users.sourceforge.net Thu Jun 30 02:02:34 2005 From: mwh at users.sourceforge.net (mwh@users.sourceforge.net) Date: Wed, 29 Jun 2005 17:02:34 -0700 Subject: [Python-checkins] python/dist/src/Objects floatobject.c, 2.136, 2.137 Message-ID: Update of /cvsroot/python/python/dist/src/Objects In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv1671 Modified Files: floatobject.c Log Message: Apparently some compiler gives a warning on float y = x; when x is a double. Go figure. Index: floatobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/floatobject.c,v retrieving revision 2.136 retrieving revision 2.137 diff -u -d -r2.136 -r2.137 --- floatobject.c 27 May 2005 15:23:14 -0000 2.136 +++ floatobject.c 30 Jun 2005 00:02:26 -0000 2.137 @@ -1427,7 +1427,7 @@ return -1; } else { - float y = x; + float y = (float)x; const char *s = (char*)&y; int i, incr = 1; From jackjansen at users.sourceforge.net Thu Jun 30 17:00:18 2005 From: jackjansen at users.sourceforge.net (jackjansen@users.sourceforge.net) Date: Thu, 30 Jun 2005 08:00:18 -0700 Subject: [Python-checkins] python/dist/src/Tools/bgen/bgen bgenGenerator.py, 1.19, 1.20 Message-ID: Update of /cvsroot/python/python/dist/src/Tools/bgen/bgen In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv2704 Modified Files: bgenGenerator.py Log Message: More factorization: added a method getrvforcallit(). This allows a C++ bridge to combine declaration and assignment to the return value temporary, allowing us to handle functions returning const values. Index: bgenGenerator.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Tools/bgen/bgen/bgenGenerator.py,v retrieving revision 1.19 retrieving revision 1.20 diff -u -d -r1.19 -r1.20 --- bgenGenerator.py 29 Jun 2005 14:17:04 -0000 1.19 +++ bgenGenerator.py 30 Jun 2005 15:00:13 -0000 1.20 @@ -213,10 +213,7 @@ def callit(self): args = "" - if self.rv: - s = "%s = %s(" % (self.rv.name, self.callname) - else: - s = "%s(" % self.name + s = "%s%s(" % (self.getrvforcallit(), self.callname) sep = ",\n" + ' '*len(s) for arg in self.argumentList: if arg is self.rv: @@ -224,12 +221,15 @@ s = arg.passArgument() if args: s = sep + s args = args + s + Output("%s%s(%s);", + self.getrvforcallit(), self.callname, args) + + def getrvforcallit(self): if self.rv: - Output("%s = %s(%s);", - self.rv.name, self.callname, args) + return "%s = " % self.rv.name else: - Output("%s(%s);", self.callname, args) - + return "" + def checkit(self): for arg in self.argumentList: arg.errorCheck()