From jack@performancedrivers.com Fri Aug 1 00:14:53 2003 From: jack@performancedrivers.com (Jack Diederich) Date: Thu, 31 Jul 2003 19:14:53 -0400 Subject: [Python-Dev] Re: A syntax for function attributes? In-Reply-To: ; from pf_moore@yahoo.co.uk on Thu, Jul 31, 2003 at 07:58:40PM +0100 References: <3F292D75.1030508@llnl.gov> <5.1.1.6.0.20030731115256.0201e180@telecommunity.com> Message-ID: <20030731191453.A972@bitch.nowhere.blah> On Thu, Jul 31, 2003 at 07:58:40PM +0100, Paul Moore wrote: > 1. It doesn't help with properties. To be honest, I don't see this as > much of an issue, as properties are a very different beast (they > effectively combine multiple functions in one). People have > demonstrated clever hacks to make properties possible, which leads > me nicely to... I agree > 2. It's possibly *too* flexible. The temptation to define clever hacks > may be just a little high. The example of attrs() mentioned above > is a good example. It satisfies a real need, it's simple, and it's > easy to implement via the [...] syntax. But is it really the best > way of handling function attributes? I really don't know, without > trying it out. And that leads me onto... This doesn't worry me too much. I think it will clear out more hacks by providing a generally readable syntax that takes the place of everyone's personal hacks. > > My canonical case for function attributes comes from the parser Spark, > which (mis-)uses docstrings to contain the grammar rule for which the > function is an action: > > def p_expr_term(self, args): > ''' > expr ::= expr + term > term ::= term * factor > ''' > return AST(type=args[1], > left=args[0], > right=args[2]) This is an abuse of doc strings any way you slice it, the SPARC folks wanted a mini-language and decided to use python as the interpreter. They found a way to define _most_ of the info through other tricks, but had to cram a little into docstrings. Their mini-language isn't working around the lack of method decorators, it is working around the lack of macros. A more pythonic way to write what SPARC wants would be to write out the python that SPARC translates the above into. something more like def anonfunc1(self, args): return AST(type=args[1],left=args[0],right=args[2]) g = Grammar(Rule('expr', 'expr + term', anonfunc1), Rule('term', 'term * factor', anonfunc1), ) So method decorators might make this kind of hack easier, but they don't change its hackiness. It only bothers me a little the way SPARC does it, the abuse of docstrings makes for more readable code once you know that it is a sparc file. The fully written out way requires reading around too much. So the fact that they might use decorators instead of docstrings to implement their mini-language is a non-issue for me. I'm still dying to write (python3k rules!): class MyClass: # knows I meant 'class MyClass(object)' def func1(arg1) [staticmethod, memoized]: # memoized defined elsewhere # do stuff return answer > > It's also interesting to note that classmethod/staticmethod and > properties get used in spite of the lack of syntactic support, whereas > function attributes generally don't seem to. I'm not quite sure what > this implies about the usefulness of function attributes... I'm not exactly sure what function attributes are, does this just mean: def func1(a, b): return a + b func.priority = 1 def func2(a, b): return a * b func.priority = 2 If so I'm not sure what I would use that for except overloading the interpreter to write mini-languages, which I don't do much. -jackdied From patmiller@llnl.gov Fri Aug 1 00:34:25 2003 From: patmiller@llnl.gov (Pat Miller) Date: Thu, 31 Jul 2003 16:34:25 -0700 Subject: [Python-Dev] Re: A syntax for function attributes? References: Message-ID: <3F29A781.4090007@llnl.gov> Mark writes: > def heres_my_function_name(args): > heres_my_function_name.my_attribute = "Foo" > > Without a special syntax, I think we'll see more and more abuses of __doc__, like > > def heres_my_function_name(args): > """my_attribute = "Foo" """ Worse than just being yucky, it is also dangerous to count on a function keeping the same name or being available in its context. Another argument for keeping the sytax tightly bound to the declaration. def heres_my_function_name(args): heres_my_function_name.my_attribute = "Foo" callbacks = [ heres_my_function_name, something_else, and_some_more] del heres_my_function_name Now, heres_my_function_name fails because the name "heres_my_function_name" is no longer in the global space. I actually do sometimes overwrite functions (e.g. replacing a generic function with a specialized one). -- Patrick Miller | (925) 423-0309 | http://www.llnl.gov/CASC/people/pmiller I don't want yes men around me. I want everyone to tell the truth, even if it costs them their jobs. -- Samuel Goldwyn From staschuk@telusplanet.net Fri Aug 1 01:30:21 2003 From: staschuk@telusplanet.net (Steven Taschuk) Date: Thu, 31 Jul 2003 18:30:21 -0600 Subject: [Python-Dev] deadlock problem In-Reply-To: <200307311514.52891.harri.pasanen@trema.com>; from harri.pasanen@trema.com on Thu, Jul 31, 2003 at 03:14:52PM +0200 References: <200307311153.39665.harri.pasanen@trema.com> <200307311514.52891.harri.pasanen@trema.com> Message-ID: <20030731183021.A22513@tibia.amotlpaa.bogus> Quoth Harri Pasanen: [...] > It is actually Python 2.3b2, Win2000 under vmWare, linux (Mandrake > 9.1 host). That python passes all tests, except test_httplib > fails with socket.error: (10060, 'Operation timed out') I don't know about the deadlock problem, but this test_httplib failure sounds much like bug 764095, which was fixed in 2.3c1. -- Steven Taschuk staschuk@telusplanet.net "Its force is immeasurable. Even Computer cannot determine it." -- _Space: 1999_ episode "Black Sun" From greg@cosc.canterbury.ac.nz Fri Aug 1 01:26:43 2003 From: greg@cosc.canterbury.ac.nz (Greg Ewing) Date: Fri, 01 Aug 2003 12:26:43 +1200 (NZST) Subject: [Python-Dev] Re: A syntax for function attributes? In-Reply-To: Message-ID: <200308010026.h710Qh321752@oma.cosc.canterbury.ac.nz> Paul Moore : > 2. It's possibly *too* flexible. The temptation to define clever hacks > may be just a little high. The example of attrs() mentioned above > is a good example. It satisfies a real need, it's simple, and it's > easy to implement via the [...] syntax. But is it really the best > way of handling function attributes? Perhaps provision could be made for both: def spam(self) [classmethod, foo = "blarg"]: ... > 1. It doesn't help with properties. To be honest, I don't see this as > much of an issue, as properties are a very different beast I agree. We should *not* try to shoehorn properties into the function modifier syntax, as the resulting contortions would be at least as bad as what we have now, and probably worse! Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From aahz@pythoncraft.com Fri Aug 1 03:31:21 2003 From: aahz@pythoncraft.com (Aahz) Date: Thu, 31 Jul 2003 22:31:21 -0400 Subject: [Python-Dev] Looking for programs using regular expressions In-Reply-To: <3F296B2E.6070203@ocf.berkeley.edu> References: <3F296B2E.6070203@ocf.berkeley.edu> Message-ID: <20030801023121.GA12057@panix.com> On Thu, Jul 31, 2003, Brett C. wrote: > Scott A Crosby wrote: >> >>I'm working on techniques to automatically identify problematic >>regular expressions where carefully chosen inputs can cause a matcher >>to run for a long time. >> >>I need testcases, so I'm looking for Python software that that is >>widely used and also uses lots of regular expressions. Can anyone >>offer any suggestions of what I should look at? I'm also looking for >>Perl software. > > Scott, this is the wrong place to be asking this. Python-dev is used to > discuss the development of the Python language. A better place to look > for an answer to your question is on comp.lang.python (which can also be > reached through python-list@python.org ). Brett, your trigger finger is a tiny bit itchy. I believe this is a followup on the earlier thread about denial-of-service against hashing functions, where Uncle Timmy pointed out that regexes are far more troublesome. Scott's more likely to get good testcases out of c.l.py, IMO, but this is certainly an appropriate place for discussion if he's planning to roll code back into the system. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From skip@pobox.com Fri Aug 1 05:03:32 2003 From: skip@pobox.com (Skip Montanaro) Date: Thu, 31 Jul 2003 23:03:32 -0500 Subject: [Python-Dev] python-mode category? Message-ID: <16169.59028.85893.460880@montanaro.dyndns.org> Can the powers-that-be maybe add a python-mode category to the bug/patch tracker? That will make it easier to locate those items which are still open (once I've had a chance to update them). Thanks, Skip From mhammond@skippinet.com.au Fri Aug 1 07:32:14 2003 From: mhammond@skippinet.com.au (Mark Hammond) Date: Fri, 1 Aug 2003 16:32:14 +1000 Subject: [Python-Dev] python-mode category? In-Reply-To: <16169.59028.85893.460880@montanaro.dyndns.org> Message-ID: <00c901c357f6$a6aef100$f502a8c0@eden> > Can the powers-that-be maybe add a python-mode category to > the bug/patch > tracker? That will make it easier to locate those items > which are still > open (once I've had a chance to update them). I wasn't 100% sure what you wanted, so I did the next best thing - I made you one of the "powers that be" :) Mark. From mzaratc603@rogers.com Fri Aug 1 08:03:03 2003 From: mzaratc603@rogers.com (Martin Zarate) Date: Fri, 1 Aug 2003 03:03:03 -0400 Subject: [Python-Dev] A syntax for function attributes? References: <20030731211501.21809.58646.Mailman@mail.python.org> Message-ID: <013501c357fa$f3baec20$8001a8c0@PXTL> This is my first post to this thing, but don't be gentle. If I should go away, say so. Well, an ugly reverse could be made, with only a minor modification to classmethod (the need for a callable member - it should be there anyways). class foo(object): bar = classmethod() #this doesn't work right now - it needs a parameter - changing that is trivial. def myfunc(self,x,y): pass bar.callable = myfunc #callable isn't in members right now, but changing that is even more trivial. del myfunc not perfect, but then the classmethod thing is mentioned at the top. Having to create and delete myfunc just to avoid having some pointless thing left behind in the class seems a waste - but its a problem I run into often in modules and classes. Some sort of "temp" variables would be nice, that get left behind on localstofast - but they'd require a new statement, similar to 'global'. I remember the 2.2 class tutorial mention that using the actual type-object to make the thing was thought of as a temporary hack anyways, but I don't know if there are specific plans for the future. Anyhow, the "as" keyword would seem to be better then the [] idea.... >>>def myfunctionname(x=5,y=5,z) as classmethod: or maybe just two words, C style. >>>def classmethod myfunctionname(x=5,y=5,z): I've got a much more fleshed-out description of this concept (expanded to include classes) posted at http://members.rogers.com/pxtl/stuff/Pythondefidea.txt rather then taking up space here. I can't contribute to the discussion of rules, tho, as I have no clue what you're talking about. From martin@v.loewis.de Fri Aug 1 08:37:06 2003 From: martin@v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: 01 Aug 2003 09:37:06 +0200 Subject: [Python-Dev] Switch to 2.4a0 on trunk? In-Reply-To: <1059661508.1171.4.camel@yyz> References: <1059661508.1171.4.camel@yyz> Message-ID: Barry Warsaw writes: > Well, here's the problem. The branch was already created with > release23-branch, and there doesn't appear to be a way to rename a > branch tag. Fred's going to dig into the docs to see if he can find a > clue, but if anybody else has direct experience with renaming branch > tags, please speak up! I fail to see the problem. Python 2.2 also has release22-fork (a tag), release22-branch (a branch), release22-maint (a branch), and release22 (a tag). I don't know whether this was intentional or not, but I would interpret it this way: - release22-fork: Point where release procedure for 2.2 started - release22-branch: Branch on which 2.2.0 was developed - release22: the actual release - release22-maint: Maintenance (2.2.1, etc) I suggest to do the same for 2.3, i.e. cvs co -r release23-fork cvs tag -b release23-maint You will then need to merge the changes from release23-branch to release23-maint, which, I believe, is done by cvs co -r release23-maint cvs up -j release23-branch cvs commit In the future, I recommend to drop the release branch altogether. Instead, when RC1 is going to be released, create a -maint branch, and release RC1 off that, then release RC2, and the final release also from the -maint branch. Changes made to the -maint branch must then get forwarded to HEAD manually, but there shouldn't be many. Regards, Martin From Paul.Moore@atosorigin.com Fri Aug 1 09:50:27 2003 From: Paul.Moore@atosorigin.com (Moore, Paul) Date: Fri, 1 Aug 2003 09:50:27 +0100 Subject: [Python-Dev] Re: A syntax for function attributes? Message-ID: <16E1010E4581B049ABC51D4975CEDB88619AE2@UKDCX001.uk.int.atosorigin.com> From: Eric S. Raymond [mailto:esr@thyrsus.com] > Zack Weinberg : >> I.e. allow a dictionary constructor expression between the argument >> list and the colon; this initializes the property dictionary of the >> code object. One could stick a call to spark.rule() in there, or = have >> a metaclass do it automatically. >>=20 >> This allows the square bracket notation to be reserved for >> [classmethod] and the like. I suppose there's nothing stopping = square >> brackets from being used for both, but I like having consistency with >> other dictionary constructor expressions. > > I like this argument. +1. All of the notations involving "stuff" between the definition and the colon look reasonable when the "stuff" is small, but start to look odd when it gets bigger (because it usually means that the colon is not on the same line as the "def"). How about leaving the [...] construct between the args and the colon, on the assumption that "normal use" is for short stuff, like = [classmethod]. But for the dictionary option for function attributes, locate it after = the docstring. The we have def p_expr_term(self,args): """docstring here""" { 'rule' : """expr ::=3D expr + term term ::=3D term * factor""" } pass It may get (slightly) hairy to parse, but should be OK, as docstrings = are a precedent for bare literals at the start of a function definition = having special meaning. Paul. From barry@python.org Fri Aug 1 13:25:14 2003 From: barry@python.org (Barry Warsaw) Date: 01 Aug 2003 08:25:14 -0400 Subject: [Python-Dev] Switch to 2.4a0 on trunk? In-Reply-To: References: <1059661508.1171.4.camel@yyz> Message-ID: <1059740713.25229.5.camel@anthem> On Fri, 2003-08-01 at 03:37, Martin v. L=F6wis wrote: > I suggest to do the same for 2.3, i.e. >=20 > cvs co -r release23-fork > cvs tag -b release23-maint >=20 > You will then need to merge the changes from release23-branch to > release23-maint, which, I believe, is done by >=20 > cvs co -r release23-maint > cvs up -j release23-branch > cvs commit I think this is the right (and easiest) suggestion. While Fred's approach works, stewing on it overnight, it does seem like overkill. Right now, Jack owns the -branch, so I'm going to wait until Jack is done with the MacOS9 build and then I'll do this. > In the future, I recommend to drop the release branch > altogether. Instead, when RC1 is going to be released, create a -maint > branch, and release RC1 off that, then release RC2, and the final > release also from the -maint branch. Changes made to the -maint branch > must then get forwarded to HEAD manually, but there shouldn't be many. PEP 101 was written at a time when we had plenty of spare cycles to do releases. That isn't true any more so I've been slowly modifying it to today's reality. Right now, it makes no mention of -branch tags, cutting all releases (except final) from the trunk. When we're about to release the final, we cut a -maint branch and do the final release from there. rc's still get released from the trunk. I'm inclined to leave it this way. Orlijn will probably tweak it when he releases Python 2.4 . -Barry From barry@python.org Fri Aug 1 13:37:48 2003 From: barry@python.org (Barry Warsaw) Date: 01 Aug 2003 08:37:48 -0400 Subject: [Python-Dev] Re: A syntax for function attributes? In-Reply-To: <200308010026.h710Qh321752@oma.cosc.canterbury.ac.nz> References: <200308010026.h710Qh321752@oma.cosc.canterbury.ac.nz> Message-ID: <1059741467.25229.12.camel@anthem> On Thu, 2003-07-31 at 20:26, Greg Ewing wrote: > I agree. We should *not* try to shoehorn properties into the > function modifier syntax, as the resulting contortions would > be at least as bad as what we have now, and probably worse! Agreed, although I'll point out that properties are only one type of descriptor. I have an application which uses non-data descriptors (i.e. define __get__() only) all over the place, and this syntactic extension would definitely clean that code up. -Barry From barry@python.org Fri Aug 1 13:49:15 2003 From: barry@python.org (Barry Warsaw) Date: 01 Aug 2003 08:49:15 -0400 Subject: [Python-Dev] Looking for a couple Emacs Lisp helper/guru types In-Reply-To: <16168.40375.103671.374290@montanaro.dyndns.org> References: <16168.40375.103671.374290@montanaro.dyndns.org> Message-ID: <1059742155.25229.18.camel@anthem> On Thu, 2003-07-31 at 00:40, Skip Montanaro wrote: > Barry & Ken have run completely out of round tuits on which are written > "python-mode". I believe Barry indicated privately that he is going to set > up a SF project to allow other folks to contribute (maybe he's just going to > give Python CVS developer privileges to anyone, he may be just that giddy > after the 2.3 release). Heh, all it takes is a $10k donation to the PSF. Or you can make the checks out to me, if you prefer. Sshhhh! Don't tell Guido! Naw, seriously. I have created a SF project for python-mode development and about all I've done so far is to make Skip and Ken co-admins. I'd like to move python-mode.el to its CVS (and probably "disable" updates to Misc/python-mode.el) but I won't have time for that today. Skip's been making some checkins to python-mode.el, so he can feel free to set up the SF project's CVS . As far as I'm concerned, anybody who wants check-in privileges on the python-mode project is welcome to it, although I'd hope we'd still discuss some of the controversial or bigger changes on some mailing list. To that effect, there is currently a python-mode@python.org alias which just goes to me and Ken right now. I'm willing to turn that into a Mailman list and use that as the main focal point for mode development. -Barry From mal@lemburg.com Fri Aug 1 14:06:06 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 01 Aug 2003 15:06:06 +0200 Subject: [Python-Dev] Stack frames Message-ID: <3F2A65BE.7060400@lemburg.com> I'm currently developing an application that needs to execute code from a library in a way that gives each call into that system access to a set of call-specific variables. The problem is that I can't change the library all that much, e.g. add some context parameter to all calls, so I have to use some other technique for accessing the call-specific variables. I've come up with a trick that uses Python call stacks which works quite well. My only concern is whether this technique will continue to work in future versions of Python. Here's the helper I'm using: import sys def acquire_from_call_stack(varname): # Start from the caller of the function calling this helper frame = sys._getframe(2) while frame is not None: if frame.f_locals.has_key(varname): value = frame.f_locals[varname] frame = None return value frame = frame.f_back raise AttributeError, varname Any idea as to what might happen to stack frames in the future ? -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Aug 01 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ From Jack.Jansen@cwi.nl Fri Aug 1 15:23:08 2003 From: Jack.Jansen@cwi.nl (Jack Jansen) Date: Fri, 1 Aug 2003 16:23:08 +0200 Subject: [Python-Dev] Stack frames In-Reply-To: <3F2A65BE.7060400@lemburg.com> Message-ID: Everything you use appears to be documented in the Python Language Reference, ("Frame Objects" in "Internal Types"), and the heading of that section explicitly states that these may change in future versions of the interpreter. So, while you are theoretically not safe in practice you are probably reasonably safe. sys.exc_traceback is also still there, even though sys.exc_info() was added 5 years ago as the preferred interface. On Friday, Aug 1, 2003, at 15:06 Europe/Amsterdam, M.-A. Lemburg wrote: > I'm currently developing an application that needs to execute > code from a library in a way that gives each call into that > system access to a set of call-specific variables. > > The problem is that I can't change the library all that much, > e.g. add some context parameter to all calls, so I have to use > some other technique for accessing the call-specific variables. > > I've come up with a trick that uses Python call stacks which > works quite well. My only concern is whether this technique will > continue to work in future versions of Python. Here's the > helper I'm using: > > import sys > > def acquire_from_call_stack(varname): > > # Start from the caller of the function calling this helper > frame = sys._getframe(2) > while frame is not None: > if frame.f_locals.has_key(varname): > value = frame.f_locals[varname] > frame = None > return value > frame = frame.f_back > raise AttributeError, varname > > Any idea as to what might happen to stack frames in the > future ? > > -- > Marc-Andre Lemburg > eGenix.com > > Professional Python Software directly from the Source (#1, Aug 01 > 2003) > >>> Python/Zope Products & Consulting ... > http://www.egenix.com/ > >>> mxODBC, mxDateTime, mxTextTools ... > http://python.egenix.com/ > _______________________________________________________________________ > _ > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From jeremy@alum.mit.edu Fri Aug 1 16:00:18 2003 From: jeremy@alum.mit.edu (Jeremy Hylton) Date: Fri, 1 Aug 2003 11:00:18 -0400 Subject: [Python-Dev] Looking for programs using regular expressions In-Reply-To: Message-ID: > I'm working on techniques to automatically identify problematic > regular expressions where carefully chosen inputs can cause a matcher > to run for a long time. > > I need testcases, so I'm looking for Python software that that is > widely used and also uses lots of regular expressions. Can anyone > offer any suggestions of what I should look at? I'm also looking for > Perl software. The first two things that come to mind are the HTMLParser and urllib modules, which both use regular expressions. I have no idea if those libraries have regexps of the sort you're looking for. Jeremy From martin@v.loewis.de Fri Aug 1 20:11:35 2003 From: martin@v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: 01 Aug 2003 21:11:35 +0200 Subject: [Python-Dev] Stack frames In-Reply-To: <3F2A65BE.7060400@lemburg.com> References: <3F2A65BE.7060400@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > The problem is that I can't change the library all that much, > e.g. add some context parameter to all calls, so I have to use > some other technique for accessing the call-specific variables. I recommend to use thread-local variables. Regards, Martin From mal@lemburg.com Fri Aug 1 20:29:49 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 01 Aug 2003 21:29:49 +0200 Subject: [Python-Dev] Stack frames In-Reply-To: References: <3F2A65BE.7060400@lemburg.com> Message-ID: <3F2ABFAD.6040207@lemburg.com> Martin v. L=F6wis wrote: > "M.-A. Lemburg" writes: >=20 >>The problem is that I can't change the library all that much, >>e.g. add some context parameter to all calls, so I have to use >>some other technique for accessing the call-specific variables. >=20 > I recommend to use thread-local variables. Thanks, but that only works in multi-threaded environments where you process each call in a different thread. This application is single-threaded (even though it may turn into a multi-threaded one at some later stage). --=20 Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Aug 01 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ From mal@lemburg.com Fri Aug 1 20:31:43 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Fri, 01 Aug 2003 21:31:43 +0200 Subject: [Python-Dev] Stack frames In-Reply-To: References: Message-ID: <3F2AC01F.8040107@lemburg.com> Jack Jansen wrote: > Everything you use appears to be documented in the Python Language > Reference, > ("Frame Objects" in "Internal Types"), and the heading of that section > explicitly > states that these may change in future versions of the interpreter. So, > while > you are theoretically not safe in practice you are probably reasonably > safe. > sys.exc_traceback is also still there, even though sys.exc_info() was > added > 5 years ago as the preferred interface. Thanks (perhaps we should add more such notes to the spec, to make sure they still exist another 5 years down the road :-). -- Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Aug 01 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ From martin@v.loewis.de Fri Aug 1 21:54:33 2003 From: martin@v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: 01 Aug 2003 22:54:33 +0200 Subject: [Python-Dev] Stack frames In-Reply-To: <3F2ABFAD.6040207@lemburg.com> References: <3F2A65BE.7060400@lemburg.com> <3F2ABFAD.6040207@lemburg.com> Message-ID: "M.-A. Lemburg" writes: > > I recommend to use thread-local variables. > > Thanks, but that only works in multi-threaded environments where > you process each call in a different thread. This application is > single-threaded (even though it may turn into a multi-threaded > one at some later stage). That is not true. In the single-threaded case, just use global variables. Regards, Martin From mal@lemburg.com Fri Aug 1 23:49:27 2003 From: mal@lemburg.com (M.-A. Lemburg) Date: Sat, 02 Aug 2003 00:49:27 +0200 Subject: [Python-Dev] Stack frames In-Reply-To: References: <3F2A65BE.7060400@lemburg.com> <3F2ABFAD.6040207@lemburg.com> Message-ID: <3F2AEE77.6030909@lemburg.com> Martin v. L=F6wis wrote: > "M.-A. Lemburg" writes: >=20 >>>I recommend to use thread-local variables. >> >>Thanks, but that only works in multi-threaded environments where >>you process each call in a different thread. This application is >>single-threaded (even though it may turn into a multi-threaded >>one at some later stage). >=20 > That is not true. In the single-threaded case, just use global > variables. ... and then manage access to these global variables ? No thanks. I always try to keep global state in objects so that a later transition to a multi-threaded application remains easily possible. I'll stick with my solution for the time being. --=20 Marc-Andre Lemburg eGenix.com Professional Python Software directly from the Source (#1, Aug 02 2003) >>> Python/Zope Products & Consulting ... http://www.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/ ________________________________________________________________________ From mhammond@skippinet.com.au Sat Aug 2 01:48:09 2003 From: mhammond@skippinet.com.au (Mark Hammond) Date: Sat, 2 Aug 2003 10:48:09 +1000 Subject: [Python-Dev] python-mode category? In-Reply-To: <00c901c357f6$a6aef100$f502a8c0@eden> Message-ID: <04c101c3588f$bed789b0$f502a8c0@eden> [Me, responding to Skip] > > Can the powers-that-be maybe add a python-mode category to > > the bug/patch > > tracker? That will make it easier to locate those items > > which are still > > open (once I've had a chance to update them). > > I wasn't 100% sure what you wanted, so I did the next best > thing - I made > you one of the "powers that be" :) Um - well I did on the SpamBayes project! I should read the origin of the mails better! Mark. From michal@sabren.com Sat Aug 2 04:09:06 2003 From: michal@sabren.com (Michal Wallace) Date: Fri, 1 Aug 2003 23:09:06 -0400 (EDT) Subject: [Python-Dev] pirate (python+parrot) Message-ID: Hey all, Earlier this week, I picked up amk's old crack at generating parrot bytecode, and have got it working again with imcc, the new(er) high(er)-level parrot assembler. Between imcc and the python compiler package, it's actually moving pretty rapidly. Currently, it supports: - print - lists, strings, ints (but no methods on them) - assignment and multi-assignment ( x,y=1,2 ) - while, for, if/elif/else, break, continue - math operations (+, -, *, /, %) - boolean logic (and, or, not) - comparison operators - function calls - lambda You can browse the code here: http://pirate.versionhost.com/viewcvs.cgi/pirate/ (See the README file for more info) To run it, you'll need python and also to check out and build parrot from cvs: http://dev.perl.org/cvs/ I'm just posting here to see if anyone's interested in helping out. There's a to-do list and help wanted section in the README, but one thing I could *really* use some help with is figuring out how to get the core C types working as PMCs. A PMC is supposedly very similar to a PyObject, so I'm wondering if it's possible to bridge the two without doing a lot of extra work. I just don't know enough C to judge. Anyway, cheers. :) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From michal@sabren.com Sat Aug 2 04:13:16 2003 From: michal@sabren.com (Michal Wallace) Date: Fri, 1 Aug 2003 23:13:16 -0400 (EDT) Subject: [Python-Dev] cvs hosting Message-ID: Oh, and... I saw in the archives that there was talk of moving off sourceforge. I do commercial CVS pserver hosting: http://www.versionhost.com/ Other than CVS itself, it's entirely powered by python, and I'd be honored to host the official python repository. (For free, of course!) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From tjreedy@udel.edu Sat Aug 2 07:07:59 2003 From: tjreedy@udel.edu (Terry Reedy) Date: Sat, 2 Aug 2003 02:07:59 -0400 Subject: [Python-Dev] Re: cvs hosting References: Message-ID: "Michal Wallace" wrote in message news:Pine.LNX.4.44.0308012309120.20222-100000@hydrogen.sabren.com... > > Oh, and... I saw in the archives that there > was talk of moving off sourceforge. I do > commercial CVS pserver hosting: > > http://www.versionhost.com/ > > Other than CVS itself, it's entirely powered > by python, and I'd be honored to host the > official python repository. (For free, of > course!) FWIW, Michal's other post gave a link to his pirate project http://pirate.versionhost.com/viewcvs.cgi/pirate/ I checked it out and noted that he is running a newer version of ViewCVS than SourceForge (0.9.2 vs 0.8 -- main visible diff is an explicit 'text' download link) and appears to be running against the actual repository rather than a 'backup up to 24 hours old'. (I could get 2 hour old revision). In a short and inadequate test, it seems to be as least as snappy as SF. Terry J. Reedy From barry at python.org Sun Aug 3 05:31:21 2003 From: barry at python.org (Barry Warsaw) Date: Sun Aug 3 00:31:23 2003 Subject: [Python-Dev] New python-mode project at SourceForge Message-ID: <1059885046.1109.21.camel@geddy> I no longer have time to properly maintain python-mode.el, so in the fine open source tradition, I've started a SourceForge project and a mailing list for python-mode. Skip and Ken are co-admins of the project, so this way we can give people access to python-mode without going through the rigamarole of making them committers on the Python project. python-mode@python.org is the mailing list (it used to be just an alias). http://sf.net/projects/python-mode is the project. Both are only minimally set up at the moment (I couldn't access cvs tonight so I haven't installed python-mode.el yet). We still need to decide what to do about Misc/python-mode.el. I propose we do imports from the python-mode project at appropriate times. Anyone hacking on python-mode.el should stop doing that in the Python project. Feel free to subscribe to the mailingl ist if you want to help. -Barry From martin at v.loewis.de Sun Aug 3 08:53:56 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sun Aug 3 03:53:57 2003 Subject: [Python-Dev] New python-mode project at SourceForge In-Reply-To: <1059885046.1109.21.camel@geddy> References: <1059885046.1109.21.camel@geddy> Message-ID: Barry Warsaw writes: > We still need to decide what to do about Misc/python-mode.el. I propose > we do imports from the python-mode project at appropriate times. That is easily forgotten. I propose that we remove Misc/python-mode.el, then people reviewing python-mode patches will remember to redirect submitters. IMO, it would be better if python-mode.el was released together with Emacs/XEmacs, instead of being released together with Python. Regards, Martin From skip at mojam.com Sun Aug 3 08:00:25 2003 From: skip at mojam.com (Skip Montanaro) Date: Sun Aug 3 08:00:29 2003 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200308031200.h73C0PR08400@manatee.mojam.com> Bug/Patch Summary ----------------- 428 open / 3965 total bugs (+28) 172 open / 2300 total patches (+3) New Bugs -------- Malformed HTML in the FAQ Wizard (2003-07-27) http://python.org/sf/778558 struct does not pad to alignment (2003-07-27) http://python.org/sf/778681 scripts destination directory not on default path (2003-07-28) http://python.org/sf/778799 CHIHTTPServer connat manage cgi in sub directories (2003-07-28) http://python.org/sf/778804 parameter confusion in 2.3c2 (2003-07-28) http://python.org/sf/778964 FutureWarning in Carbon/Controls.py (2003-07-28) http://python.org/sf/779147 bgenlocations.py only works for OS9 users and jack ;) (2003-07-28) http://python.org/sf/779151 bgen requires Universal Headers, not OS X dev headers (2003-07-28) http://python.org/sf/779153 PackageManager maintainer missing documentation (2003-07-28) http://python.org/sf/779154 urllib proxy handling incomplete for MacOSX (2003-07-28) http://python.org/sf/779167 BasicModuleLoader behaviour in Python 2.3c2 (2003-07-28) http://python.org/sf/779191 2.3c2 test_pwd fails on Mac OS X 10.2.6 (2003-07-28) http://python.org/sf/779218 Carbon Event ReceiveNextEvent (2003-07-28) http://python.org/sf/779285 sgmllib incorrectly handles entities in attributes (2003-07-28) http://python.org/sf/779388 plistlib should be moved out of plat-mac (2003-07-29) http://python.org/sf/779460 plistlib and bundlebuilder not in the documentation (2003-07-29) http://python.org/sf/779825 Missing DEFS in Makefile.pre.in breakage (2003-07-29) http://python.org/sf/779973 docs missing 'trace' module (2003-07-29) http://python.org/sf/779976 Makefile.pre.in ignores CPPFLAGS from environment (2003-07-29) http://python.org/sf/780024 Bug in "Build and C API Changes" section of what's new doc? (2003-07-30) http://python.org/sf/780231 pyexpat LexicalHandler swaps system_id and public_id (2003-07-30) http://python.org/sf/780300 socket.makefile() isn't compatible with marshal/cPickle/etc (2003-07-30) http://python.org/sf/780354 cast from pointer to integer of different size (2003-07-30) http://python.org/sf/780407 IDLE fails to launch if 2.3 is installed to "Program Files" (2003-07-30) http://python.org/sf/780451 MacOS.Error for platform.mac_ver under OS X (2003-07-30) http://python.org/sf/780461 test_ioctl fails (2003-07-30) http://python.org/sf/780576 infinite __str__ recursion in thread causes seg fault (2003-07-31) http://python.org/sf/780714 Compile error messages and PEP-263 (2003-07-31) http://python.org/sf/780725 PEP-263 and 278 (2003-07-31) http://python.org/sf/780730 time.strptime() 1,200 times slower in python2.3 (2003-07-31) http://python.org/sf/780807 recent-files defaulted to strange place and bad permissions (2003-07-31) http://python.org/sf/780887 pynche does not start (2003-07-31) http://python.org/sf/780996 test_normalization fails (2003-07-31) http://python.org/sf/781065 2.3 web page link to 'New style classes' changes to 2.2.2 (2003-07-31) http://python.org/sf/781285 MacOS module doesn't compile on 10.1 (2003-08-01) http://python.org/sf/781433 Applets don't work on 10.1 (2003-08-01) http://python.org/sf/781445 Windows _bsddb link warnings (2003-08-01) http://python.org/sf/781614 py-2.3 socket.inet_pton() segfault. (2003-08-01) http://python.org/sf/781722 Listbox itemconfigure leaks memory (2003-08-02) http://python.org/sf/781883 file.write() incorrectly tests remaining space (2003-08-02) http://python.org/sf/781891 New Patches ----------- python-mode py-execute-buffer bug (2003-07-29) http://python.org/sf/779839 Fix zip file header format in zipfile.py (2003-07-30) http://python.org/sf/780595 More robust MSVC6 finder (2003-07-31) http://python.org/sf/780821 fix gettext NullTranslation.add_fallback typo (2003-07-31) http://python.org/sf/781126 Closed Bugs ----------- python-mode.el disables raw_input() (2002-08-13) http://python.org/sf/594833 python-mode.el replaces function on f1 (2002-09-06) http://python.org/sf/605818 python-mode kills arrow in gdb (gud.el) (2002-09-08) http://python.org/sf/606250 elisp: doesn't recognize comment-syntax (2002-09-08) http://python.org/sf/606251 py-electric-colon & delete-selection-mod (2002-09-08) http://python.org/sf/606254 socket.sendto(SOCK_DGRAM) very slow on OSX! (2003-05-31) http://python.org/sf/746895 IDLE/MAC: Command / Alt Bindings Don't Work (2003-07-09) http://python.org/sf/768469 right button context 'edit with idle' (2003-07-16) http://python.org/sf/772787 MacOS9: test_urllib fails (2003-07-23) http://python.org/sf/776203 MacOS9: test_strptime fails (2003-07-23) http://python.org/sf/776205 MacOS9: test_posixpath fails (2003-07-23) http://python.org/sf/776207 MacOS9: test_csv fails (2003-07-23) http://python.org/sf/776209 readline should be in PackageManager database (2003-07-23) http://python.org/sf/776602 IDLE hangs when selecting "Edit with IDLE" from explorer (2003-07-27) http://python.org/sf/778400 Closed Patches -------------- add SIGRTMIN, SIGRTMAX to signalmodule.c (2003-07-23) http://python.org/sf/776725 From allison at sumeru.stanford.EDU Sun Aug 3 20:18:27 2003 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Sun Aug 3 22:18:37 2003 Subject: [Python-Dev] RH9 build does not seem to autodetect properly Message-ID: I'ver just tried to build the new 2.3 release under RH9. It builds but with some problems.... 228 tests OK. 27 tests skipped: test_aepack test_al test_bsddb185 test_bsddb3 test_bz2 test_cd test_cl test_curses test_email_codecs test_gl test_imgfile test_linuxaudiodev test_macfs test_macostools test_nis test_normalization test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_socket_ssl test_socketserver test_sunaudiodev test_timeout test_urllibnet test_winreg test_winsound 1 skip unexpected on linux2: test_bz2 test_urllibnet test_urllibnet skipped -- Use of the `network' resource not enabled and similar network related modules are not detected. When I did the same thing with Python 2.1.3 I get similar results, but things fall apart a bit more. The network modules don't get detected, large files support does not get detected, etc. Building under RH 7.3 seems to functions as I expect which leads me to suspect RH 9.0 (updated to the latest RH 9 updates). Any thoughts as to where to look for the problem--or is there some cockpit error? From skip at pobox.com Sun Aug 3 23:18:12 2003 From: skip at pobox.com (Skip Montanaro) Date: Sun Aug 3 23:18:24 2003 Subject: [Python-Dev] RH9 build does not seem to autodetect properly In-Reply-To: References: Message-ID: <16173.53364.616906.22330@montanaro.dyndns.org> Dennis> I'ver just tried to build the new 2.3 release under RH9. It builds but Dennis> with some problems.... Dennis> 228 tests OK. ... Dennis> 1 skip unexpected on linux2: Dennis> test_bz2 Can you confirm that the necessary libraries and header files are installed on your system? There are probably two rpms, named something like bz2 and bz2-devel which must be installed. Dennis> test_urllibnet Dennis> test_urllibnet skipped -- Use of the `network' resource not enabled Dennis> and similar network related modules are not detected. Network tests are disabled by default because they assume a connection to the net. You can run them like ./python Lib/regrtest.py -uall This will enable all tests which are disabled by default. Skip From allison at sumeru.stanford.EDU Mon Aug 4 00:07:32 2003 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Mon Aug 4 02:07:43 2003 Subject: [Python-Dev] RH9 build does not seem to autodetect properly In-Reply-To: <16173.53364.616906.22330@montanaro.dyndns.org> Message-ID: Thanks Skip -- I'll blame the RH installer here--I'm sure I requested the development headers but they were missing. Installing them fixes the bzip problem. Running the regression tests (.python Lib/test/regrtest.py -uall) exposes other problems: > ./python Lib/test/regrtest.py -uall > junk1 Exception in thread writer 0: Traceback (most recent call last): File "/usr/local/src/python/Python-2.3/Lib/threading.py", line 436, in __bootstrap self.run() File "/usr/local/src/python/Python-2.3/Lib/threading.py", line 416, in run self.__target(*self.__args, **self.__kwargs) File "/usr/local/src/python/Python-2.3/Lib/bsddb/test/test_thread.py", line 254, in writerThread self.assertEqual(data, self.makeData(key)) File "/usr/local/src/python/Python-2.3/Lib/unittest.py", line 292, in failUnlessEqual raise self.failureException, \ AssertionError: None != '0003-0003-0003-0003-0003' /usr/local/src/python/Python-2.3/Lib/bsddb/dbutils.py:67: RuntimeWarning: DB_INCOMPLETE: Cache flush was unable to complete return function(*_args, **_kwargs) Exception in thread writer 1: Traceback (most recent call last): File "/usr/local/src/python/Python-2.3/Lib/threading.py", line 436, in __bootstrap self.run() File "/usr/local/src/python/Python-2.3/Lib/threading.py", line 416, in run self.__target(*self.__args, **self.__kwargs) File "/usr/local/src/python/Python-2.3/Lib/bsddb/test/test_thread.py", line 254, in writerThread self.assertEqual(data, self.makeData(key)) File "/usr/local/src/python/Python-2.3/Lib/unittest.py", line 292, in failUnlessEqual raise self.failureException, \ AssertionError: None != '1000-1000-1000-1000-1000' Exception in thread writer 2: Traceback (most recent call last): File "/usr/local/src/python/Python-2.3/Lib/threading.py", line 436, in __bootstrap self.run() File "/usr/local/src/python/Python-2.3/Lib/threading.py", line 416, in run self.__target(*self.__args, **self.__kwargs) File "/usr/local/src/python/Python-2.3/Lib/bsddb/test/test_thread.py", line 254, in writerThread self.assertEqual(data, self.makeData(key)) File "/usr/local/src/python/Python-2.3/Lib/unittest.py", line 292, in failUnlessEqual raise self.failureException, \ AssertionError: None != '2001-2001-2001-2001-2001' which comes from the bsddbB3 test. Deadlock and exceptions not withstanding, it appears to have been declared "correct". This is the latest version of RH9 with all the latest and greatest upgrades. The curses test output a bunch of stuff to the terminal. I'm not sure what's to be expected there, but I suspect that it isn't what I am seeing. The summary results reports: 235 tests OK. 20 tests skipped: test_aepack test_al test_bsddb185 test_cd test_cl test_email_codecs test_gl test_imgfile test_linuxaudiodev test_macfs test_macostools test_nis test_normalization test_ossaudiodev test_pep277 test_plistlib test_scriptpackages test_sunaudiodev test_winreg test_winsound Those skips are all expected on linux2. On Sun, 3 Aug 2003, Skip Montanaro wrote: > > Dennis> I'ver just tried to build the new 2.3 release under RH9. It builds but > Dennis> with some problems.... > > Dennis> 228 tests OK. > ... > Dennis> 1 skip unexpected on linux2: > Dennis> test_bz2 > > Can you confirm that the necessary libraries and header files are installed > on your system? There are probably two rpms, named something like bz2 and > bz2-devel which must be installed. > > Dennis> test_urllibnet > Dennis> test_urllibnet skipped -- Use of the `network' resource not enabled > > Dennis> and similar network related modules are not detected. > > Network tests are disabled by default because they assume a connection to > the net. You can run them like > > ./python Lib/regrtest.py -uall > > This will enable all tests which are disabled by default. > > Skip > From raymond.hettinger at verizon.net Mon Aug 4 03:51:29 2003 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Mon Aug 4 02:55:51 2003 Subject: [Python-Dev] Caching tuple hashes Message-ID: <007401c35a54$d6204e60$e841fea9@oemcomputer> Strings save their hash values to avoid recomputation in subsequent hashings. In contrast, tuples recompute on every call. I've googled around and cannot find the rationale for this. Some ideas are: * the time to initialize and check the hash fields isn't repaid on average * it isn't worth an extra structure field for storing the hash value * C code can mutate immutables so there is a safety risk * nobody ever thought of it (unlikely) or got around to it (more likely). Raymond Hettinger From allison at sumeru.stanford.EDU Mon Aug 4 02:12:31 2003 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Mon Aug 4 04:12:40 2003 Subject: [Python-Dev] RH9 build does not seem to autodetect properly In-Reply-To: Message-ID: This is probably old news, but I wanted to follow up on my earlier messages. Building Python 2.3 under RH9 works when all the header files are present. Running the full test suite turns up problems with the threading module. (see my earlier post for details)A I had significant problems building Python 2.1.3 under RH9. The SSL/Kerberos5 interface has changed and, at least on my system, there are dangling includes. The usual configure, make mantra does not do the job even after installing Kerberos. As I am not using Kerberos5, I got python 2.1.3 to compile after a bit of shameful brain surgery. I've not been able to figure out how to get large file support to work. From andrew-pythondev at puzzling.org Mon Aug 4 19:14:15 2003 From: andrew-pythondev at puzzling.org (Andrew Bennetts) Date: Mon Aug 4 04:14:21 2003 Subject: [Python-Dev] Caching tuple hashes In-Reply-To: <007401c35a54$d6204e60$e841fea9@oemcomputer> References: <007401c35a54$d6204e60$e841fea9@oemcomputer> Message-ID: <20030804081415.GJ636@frobozz> On Mon, Aug 04, 2003 at 02:51:29AM -0400, Raymond Hettinger wrote: > Strings save their hash values to avoid recomputation in subsequent > hashings. In contrast, tuples recompute on every call. I've googled > around and cannot find the rationale for this. Some ideas are: > > * the time to initialize and check the hash fields isn't repaid on average > * it isn't worth an extra structure field for storing the hash value > * C code can mutate immutables so there is a safety risk > * nobody ever thought of it (unlikely) or got around to it (more likely). Perhaps this: * Tuples can contain badly-behaved mutables, e.g.: >>> class X: ... x = 1 ... def __hash__(self): ... return self.x ... >>> x = X() >>> t = (x,) >>> hash(t) -1660579480 >>> X.x = 2 >>> hash(t) -1660579477 -Andrew. From jepler at unpythonic.net Mon Aug 4 08:03:20 2003 From: jepler at unpythonic.net (Jeff Epler) Date: Mon Aug 4 08:03:26 2003 Subject: [Python-Dev] Caching tuple hashes In-Reply-To: <007401c35a54$d6204e60$e841fea9@oemcomputer> References: <007401c35a54$d6204e60$e841fea9@oemcomputer> Message-ID: <20030804120316.GD10412@unpythonic.net> Well, something like the following rule could be used to detect tuples that can cache their hash values: All the entries in the a hash-safe tuple must be non-NULL, and of an immutable builtin type (int, long, float, string, unicode, and hash-safe tuple.) Jeff From amk at amk.ca Mon Aug 4 11:22:39 2003 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 4 10:23:48 2003 Subject: [Python-Dev] Re: pirate (python+parrot) References: Message-ID: On Fri, 1 Aug 2003 23:09:06 -0400 (EDT), Michal Wallace wrote: > in helping out. There's a to-do list and help wanted > section in the README, but one thing I could *really* > use some help with is figuring out how to get the core C types working as > PMCs. A PMC is supposedly very similar to a PyObject, so I'm wondering if > it's > possible to bridge the two without doing a lot of > extra work. I just don't know enough C to judge. Excellent work on pirate; you've already advanced the translation much further than I did. I don't believe there's much chance for PMC implementation beyond writing a new PythonString/Int/whatever PMC from scratch. Let me explain why I think that. PMCs and PyObjects are similar in the general sense. Both are collections of pointers to functions that do things for the specified type. PyObjects contain instances information for a type -- e.g. the contents of a string, the value of an int, the FILE * for a file. PyTypeObjects are PyObjects that contain a bunch of pointers to C functions that do things to a type: hash it, try to convert it to a number, get an attribute from it. For example, when Python encounters 'obj1 % obj2', it gets the type object for obj1 and calls type_obj->as_number->nb_remainder(obj1, obj2). Poke around Include/object.h for a full list. Parrot's PMCs are similar, but the list of operations they support is different. I can't find a source file or document that lists them all, so looking at classes/default.pmc may be the best way to get a list. For example, I was wondering how to implement Python's slicing in Parrot. In Python, there's a tp_as_sequence structure that has a bunch of sequence-related functions, one of which is sq_slice. This has the signature (PyObject *seq, int index_low, int index_high). Strings implement this function, as do lists, so: s = 'abcdef' print s[0:3] and s = [1,2,"string", {}] print s[0:3] both work. Parrot's PMCs have a substring method, but not a generic slice method, so I'm not sure what you would compile the above Python code to. You might end up having to compile it to PASM that was basically: if (s is a string): call substring(s, 0, 3) elif (s is an array): call subarray(s, 0, 3) You might have a base PythonSequence PMC and have PythonString/PythonList/PythonArray/etc. all inherit from it, which would simplify the generated code, but I don't know if I can just invent a new set of PMC operations -- sq_slice, sq_assign_slice, sq_length -- or if introducing new ones requires modifying Parrot itself. (In fact I can't figure out how you'd get a slice of an array in CVS Parrot: none of the methods in classes/array.pmc or classes/perlarray.pmc seems relevant. I may well be missing it, though.) Therefore, I don't think there's any simple way of taking the existing Python interpreter code in Objects/listobject.c, stringobject.c, and turning them into PMC code; the types and function signatures are just too different. IMHO, simply getting Python's sequence operations (slices, slice assignment) working will be a significant milestone for Parrot/Python work, because doing it will require figuring out how to support Python's different semantics within the currently rather Perl-centric set of PMC methods. (Plus, sequence operations are needed by most Python programs -- the very first function in pystone.py needs them, for example, which is why I started looking at them.) --amk After all, we are human beings, and not creatures of infinite possibilities. -- Robertson Davies, _Conversations_ From pedronis at bluewin.ch Mon Aug 4 18:57:29 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Mon Aug 4 11:56:13 2003 Subject: [Python-Dev] histerical math.log(zero) Message-ID: <5.2.1.1.0.20030804174907.02695f00@pop.bluewin.ch> math.log raises different unrelated exceptions depending on the type of a zero argument: >>> math.log(0) Traceback (most recent call last): File "", line 1, in -toplevel- math.log(0) OverflowError: math range error >>> math.log(0L) Traceback (most recent call last): File "", line 1, in -toplevel- math.log(0L) ValueError: math domain error >>> math.log(0.0) Traceback (most recent call last): File "", line 1, in -toplevel- math.log(0.0) OverflowError: math range error should this stay this way? notice that the (only) common ancestor of OverflowError and ValueError is StandardError. regards. From tim.one at comcast.net Mon Aug 4 13:31:28 2003 From: tim.one at comcast.net (Tim Peters) Date: Mon Aug 4 12:32:04 2003 Subject: [Python-Dev] histerical math.log(zero) In-Reply-To: <5.2.1.1.0.20030804174907.02695f00@pop.bluewin.ch> Message-ID: [Samuele Pedroni] > math.log raises different unrelated exceptions depending on the type > of a zero argument: It may, and probably does on most platforms. Which exception gets raised for a float 0.0, and even whether an exception gets raised at all in that case, depends on what the platform libm does. Standard C allows a lot of variation here. > >>> math.log(0) > > Traceback (most recent call last): > File "", line 1, in -toplevel- > math.log(0) > OverflowError: math range error > >>> math.log(0L) Python handles log(long) itself, so the exception in this case is platform-independent. > Traceback (most recent call last): > File "", line 1, in -toplevel- > math.log(0L) > ValueError: math domain error > > >>> math.log(0.0) > > Traceback (most recent call last): > File "", line 1, in -toplevel- > math.log(0.0) > OverflowError: math range error > > should this stay this way? Unless we write our own libm, consistency is hopeless. Note that the 2.3 docs (for the math module) address this up-front: Note: The math module consists mostly of thin wrappers around the platform C math library functions. Behavior in exceptional cases is loosely specified by the C standards, and Python inherits much of its math-function error-reporting behavior from the platform C implementation. As a result, the specific exceptions raised in error cases (and even whether some arguments are considered to be exceptional at all) are not defined in any useful cross-platform or cross-release way. For example, whether math.log(0) returns -Inf or raises ValueError or OverflowError isn't defined, and in cases where math.log(0) raises OverflowError, math.log(0L) may raise ValueError instead. CPython started life with "classic" (pre-754) libms, where ERANGE got mapped to OverflowError and EDOM to ValueError. I view 0 as not being in the domain of log(), so deliberately raised ValueError for a 0L argument. The best thing to do in an IEEE-754-ish system is probably to raise ZeroDivisionError (whose generalized meaning isn't literally division by 0, but "there's a singularity here"); and, indeed, the C99 standard requires that implementations choosing to support the 754 gimmicks must raise the 754 divide-by-0 exception (and return minus infinity) for log(+0) and log(-0). doesn't-matter-where-you-want-to-go-you-can't-get-there-from-here-ly y'rs - tim From pedronis at bluewin.ch Mon Aug 4 19:45:17 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Mon Aug 4 12:44:04 2003 Subject: [Python-Dev] histerical math.log(zero) In-Reply-To: References: <5.2.1.1.0.20030804174907.02695f00@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030804184153.02630518@pop.bluewin.ch> At 12:31 04.08.2003 -0400, Tim Peters wrote: >[Samuele Pedroni] > > math.log raises different unrelated exceptions depending on the type > > of a zero argument: > >It may, and probably does on most platforms. Which exception gets raised >for a float 0.0, and even whether an exception gets raised at all in that >case, depends on what the platform libm does. Standard C allows a lot of >variation here. > > > >>> math.log(0) > > > > Traceback (most recent call last): > > File "", line 1, in -toplevel- > > math.log(0) > > OverflowError: math range error > > >>> math.log(0L) > >Python handles log(long) itself, so the exception in this case is >platform-independent. I was asking this because I have just finished making jython handle log(long) itself too. And I noticed that test_long is picky about the ValueError thing. I should have read the manual, sorry. I was more wondering that there's no encompassing math exception. regards. From guido at python.net Mon Aug 4 14:07:54 2003 From: guido at python.net (Guido van Rossum) Date: Mon Aug 4 13:07:57 2003 Subject: [Python-Dev] Caching tuple hashes In-Reply-To: <007401c35a54$d6204e60$e841fea9@oemcomputer>; from raymond.hettinger@verizon.net on Mon, Aug 04, 2003 at 02:51:29AM -0400 References: <007401c35a54$d6204e60$e841fea9@oemcomputer> Message-ID: <20030804130754.A16524@starship.python.net> On Mon, Aug 04, 2003 at 02:51:29AM -0400, Raymond Hettinger wrote: > Strings save their hash values to avoid recomputation in subsequent > hashings. In contrast, tuples recompute on every call. I've googled > around and cannot find the rationale for this. Some ideas are: > > * the time to initialize and check the hash fields isn't repaid on average > * it isn't worth an extra structure field for storing the hash value > * C code can mutate immutables so there is a safety risk > * nobody ever thought of it (unlikely) or got around to it (more likely). Long ago, strings didn't cache their hash values either. The cache was added because strings are the singlemost common dict key type and the hash calculation was a significant part of the cost of dict lookup, and experiments showed that caching the hash significantly sped up almost any Python program, thus justifying the extra expense of space. I don't see tuples in the same situation -- their hash is rarely in the critical path of an application, and certainly not of all apps. --Guido van Rossum From guido at python.net Mon Aug 4 14:11:24 2003 From: guido at python.net (Guido van Rossum) Date: Mon Aug 4 13:11:26 2003 Subject: [Python-Dev] Caching tuple hashes In-Reply-To: <20030804081415.GJ636@frobozz>; from andrew-pythondev@puzzling.org on Mon, Aug 04, 2003 at 06:14:15PM +1000 References: <007401c35a54$d6204e60$e841fea9@oemcomputer> <20030804081415.GJ636@frobozz> Message-ID: <20030804131124.B16524@starship.python.net> On Mon, Aug 04, 2003 at 06:14:15PM +1000, Andrew Bennetts wrote: > > * Tuples can contain badly-behaved mutables, e.g.: > > >>> class X: > ... x = 1 > ... def __hash__(self): > ... return self.x > ... > >>> x = X() > >>> t = (x,) > >>> hash(t) > -1660579480 > >>> X.x = 2 > >>> hash(t) > -1660579477 No, that's not a reason not to do this. The problem is the responsibility of class X; a tuple can assume that objects have constant hashes (since dict lookup also makes this assumption). --Guido van Rossum From guido at python.net Mon Aug 4 14:21:52 2003 From: guido at python.net (Guido van Rossum) Date: Mon Aug 4 13:21:54 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: ; from amk@amk.ca on Mon, Aug 04, 2003 at 10:22:39AM -0400 References: Message-ID: <20030804132152.C16524@starship.python.net> [Andrew and Michal mused on compiling Python to Parrot assembler.] I'm not sure if you guys are nuts or brilliant. :-) If your efforts save Dan Sugalski time in implementing Python on Parrot, realize that it may cost me a round of drinks and a laundry run (if Dan has a good aim). OTOH there may be a devilish plan here to let Dan believe he won't have to work very hard, and then in the end he'll lose because your implementations are inefficcient... :-) --Guido From tim.one at comcast.net Mon Aug 4 14:23:26 2003 From: tim.one at comcast.net (Tim Peters) Date: Mon Aug 4 13:24:05 2003 Subject: [Python-Dev] histerical math.log(zero) In-Reply-To: <5.2.1.1.0.20030804184153.02630518@pop.bluewin.ch> Message-ID: [Samuele Pedroni] > I was asking this because I have just finished making jython handle > log(long) itself too. And I noticed that test_long is picky about the > ValueError thing. It could test for one of (ValueError, OverflowError, ZeroDivisionError) instead, and also count no exception as a success. We probably see all 4 of those behaviors in CPython across platforms for log(0.0). Python-the-language doesn't really define exceptional math behaviors. > I should have read the manual, sorry. I was more wondering that > there's no encompassing math exception. Ask Guido : CPython started life with "classic" (pre-754) libms, where ERANGE got mapped to OverflowError and EDOM to ValueError. IOW, there has never been an encompassing math exception. ArithmeticError may have been an attempt to create one, but ValueError doesn't derive from it so if that was the plan, it didn't work. From michal at sabren.com Mon Aug 4 14:56:24 2003 From: michal at sabren.com (Michal Wallace) Date: Mon Aug 4 13:56:30 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: <20030804132152.C16524@starship.python.net> Message-ID: On Mon, 4 Aug 2003, Guido van Rossum wrote: > If your efforts save Dan Sugalski time in implementing Python on > Parrot, realize that it may cost me a round of drinks and a laundry > run (if Dan has a good aim). > > OTOH there may be a devilish plan here to let Dan believe he won't > have to work very hard, and then in the end he'll lose because your > implementations are inefficcient... :-) Personally, I'm working for the highest bidder. :) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From dan at sidhe.org Mon Aug 4 15:07:28 2003 From: dan at sidhe.org (Dan Sugalski) Date: Mon Aug 4 14:07:44 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: <20030804132152.C16524@starship.python.net> References: <20030804132152.C16524@starship.python.net> Message-ID: At 1:21 PM -0400 8/4/03, Guido van Rossum wrote: >[Andrew and Michal mused on compiling Python to Parrot assembler.] > >I'm not sure if you guys are nuts or brilliant. :-) > >If your efforts save Dan Sugalski time in implementing Python on Parrot, >realize that it may cost me a round of drinks and a laundry run (if >Dan has a good aim). Nah, this isn't something to worry about, since I can't use it by the terms of the challenge--I have to use the bytecode, not the source. (Besides, if you really want to avoid the laundry charges, make sure someone who's a better target's standing closer :) >OTOH there may be a devilish plan here to let Dan believe he won't >have to work very hard, and then in the end he'll lose because your >implementations are inefficcient... :-) It's not like you have to worry anyway, right? It's all religion, smoke, mirrors, and wishful thinking... :-P -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From amk at amk.ca Mon Aug 4 15:13:34 2003 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 4 14:23:31 2003 Subject: [Python-Dev] Re: pirate (python+parrot) References: <20030804132152.C16524@starship.python.net> Message-ID: On Mon, 4 Aug 2003 13:21:52 -0400, Guido van Rossum wrote: > If your efforts save Dan Sugalski time in implementing Python on Parrot, > realize that it may cost me a round of drinks and a laundry run (if > Dan has a good aim). On the other hand, if Parrot provides stacklessness, JIT compilation, cross-language interop, and pushes some complex C-level maintenance burden onto other people, perhaps you'll consider consider the laundry run and drinks worth it in the end. No way to find out if Parrot is up to the task without trying... > OTOH there may be a devilish plan here to let Dan believe he won't > have to work very hard, and then in the end he'll lose because your > implementations are inefficcient... :-) For some reason the Parrot developers seem more focused on working with bytecode; see Dan Sugalski's weblog at http://www.sidhe.org/~dan/blog/archives/000205.html . (There's an amusing link in the comments for a Python-bytecode-to-SAX-event-stream translator; my mind is boggled.) Bytecode translation doesn't strike me as a very useful course to follow, because Python's bytecode is vulnerable to being changed between versions and because looking at bytecode provides less information than looking at an AST. --amk (www.amk.ca) MENELAUS: An odd man, lady? Every man is odd. -- _Troilus and Cressida_, IV, v From michal at sabren.com Mon Aug 4 15:37:54 2003 From: michal at sabren.com (Michal Wallace) Date: Mon Aug 4 14:38:05 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: Message-ID: On Mon, 4 Aug 2003, Dan Sugalski wrote: > >If your efforts save Dan Sugalski time in implementing Python on > >Parrot, realize that it may cost me a round of drinks and a laundry > >run (if Dan has a good aim). > Nah, this isn't something to worry about, since I can't use it by the > terms of the challenge--I have to use the bytecode, not the source. Okay, that's just silly. :) What do you guys think about changing the bet to allow using the source? In any case, there's a loophole in the "massage the bytecode as needed" clause, assuming decompyle works with 2.3 :) http://www.crazy-compilers.com/decompyle/ By the way, what are the chances of type declarations being in python a year from now? I thought there was a PEP for this, but I couldn't find it. :) There's talk of pirate becoming a generic code generator for parrot, and I plan to support type declarations in the backend. I don't really want to change the python syntax, so I was thinking up an evil kludge so that people could declare types and still use the code in cpython or jython. def f(x, y, *etc): if compile is open: arg.x is int arg.y is str var.z is int return int z = 5 # ... return z I guess that wraps up the "nuts or brilliant" question, at least for me... :) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From guido at python.net Mon Aug 4 15:48:22 2003 From: guido at python.net (Guido van Rossum) Date: Mon Aug 4 14:48:24 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: ; from amk@amk.ca on Mon, Aug 04, 2003 at 02:13:34PM -0400 References: <20030804132152.C16524@starship.python.net> Message-ID: <20030804144822.A25271@starship.python.net> > On the other hand, if Parrot provides stacklessness, JIT compilation, > cross-language interop, and pushes some complex C-level maintenance burden > onto other people, perhaps you'll consider consider the laundry run and > drinks worth it in the end. No way to find out if Parrot is up to the task > without trying... That's why I'm taking the challenge. But I'm skeptical, and that's why I'm not too worried... > > OTOH there may be a devilish plan here to let Dan believe he won't > > have to work very hard, and then in the end he'll lose because your > > implementations are inefficcient... :-) > > For some reason the Parrot developers seem more focused on working with > bytecode; see Dan Sugalski's weblog at > http://www.sidhe.org/~dan/blog/archives/000205.html . Interesting indeed. The focus on bytecode is understandable: let someone else write a parser, so Parrot won't have to. But from skimming his log, Dan seems to take the bytecode too seriously; it's really only meant to be a quick way for the parser to output something. In particular, bytecode hacks were never a goal. (There's an amusing > link in the comments for a Python-bytecode-to-SAX-event-stream translator; > my mind is boggled.) Bytecode translation doesn't strike me as a very > useful course to follow, because Python's bytecode is vulnerable to being > changed between versions and because looking at bytecode provides less > information than looking at an AST. Sure, but bytecode evolves slowly enough to be useful for Dan; given that the only parse tree format we have is *intricately* linked with the Python parser and (at least part of) its runtime, I can see why Dan would have nothing of it. --Guido From dan at sidhe.org Mon Aug 4 16:15:42 2003 From: dan at sidhe.org (Dan Sugalski) Date: Mon Aug 4 15:17:06 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: References: Message-ID: At 2:37 PM -0400 8/4/03, Michal Wallace wrote: >On Mon, 4 Aug 2003, Dan Sugalski wrote: > >> >If your efforts save Dan Sugalski time in implementing Python on >> >Parrot, realize that it may cost me a round of drinks and a laundry >> >run (if Dan has a good aim). > >> Nah, this isn't something to worry about, since I can't use it by the >> terms of the challenge--I have to use the bytecode, not the source. > >Okay, that's just silly. :) > >What do you guys think about changing the bet >to allow using the source? To be honest, I'd rather leave it as it is--not because I don't think that parsing python and compiling to parrot isn't worthwhile (I think it is) but because I can be reasonably assured that the conversion will take place (because I want it to happen and have wagered pie on that want, and thus will make it happen :). Besides, I'm not much of a parser guy, thus the parsing part's actually more difficult than the bytecode translation, which is reasonably straightforward. There's also the matter of effort at bet-time--when I get a good working python bytecode->parrot bytecode translation program, folks can use the current python install as a compiler, which gets a goal (running python programs on parrot) achieved faster, since there's less effort in writing a bytecode translator than there is writing a compiler. OTOH, the... 'interesting' task that is parsing perl may well have skewed by feelings on the difficulties of bytecode translation vs source compilation. ;-P -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From dan at sidhe.org Mon Aug 4 16:10:34 2003 From: dan at sidhe.org (Dan Sugalski) Date: Mon Aug 4 15:26:26 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: References: <20030804132152.C16524@starship.python.net> Message-ID: At 2:13 PM -0400 8/4/03, A.M. Kuchling wrote: >On Mon, 4 Aug 2003 13:21:52 -0400, Guido van Rossum wrote: >>OTOH there may be a devilish plan here to let Dan believe he won't >>have to work very hard, and then in the end he'll lose because your >>implementations are inefficcient... :-) > >For some reason the Parrot developers seem more focused on working >with bytecode; see Dan Sugalski's weblog at >http://www.sidhe.org/~dan/blog/archives/000205.html . (There's an >amusing link in the comments for a >Python-bytecode-to-SAX-event-stream translator; my mind is boggled.) >Bytecode translation doesn't strike me as a very useful course to >follow, because Python's bytecode is vulnerable to being changed >between versions and because looking at bytecode provides less >information than looking at an AST. The reason for the interest in bytecode for this goes back to the python-dev traffic that lead to the challenge starts at http://mail.python.org/pipermail/python-dev/2003-January/032598.html more or less (there are some messages previous to it). The quick summary is that it was posited that native compilation and/or JITting (depending on how you look at it) wouldn't be a win for Python. I said it would be and parrot was sufficiently far along for me to be comfortable with that, and put up $10 and a round of beer to emphasize I was willing to back it up. (Seemed an appropriate way to show I was serious) Guido took me up on it, and the bet escalated a bit afterwards. The bytecode emphasis is there mainly because of the bet. I put up that parrot'd be a faster engine for python than the CPython core, not a better parser of the python source. The best way to demonstrate that is to run some python bytecode--actually producing that bytecode is a matter for a compiler, and while that's certainly an important thing, it wasn't the main emphasis. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From pedronis at bluewin.ch Mon Aug 4 23:12:11 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Mon Aug 4 16:17:34 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: References: Message-ID: <5.2.1.1.0.20030804220705.026a76e8@pop.bluewin.ch> At 15:15 04.08.2003 -0400, Dan Sugalski wrote: >OTOH, the... 'interesting' task that is parsing perl may well have skewed >by feelings on the difficulties of bytecode translation vs source >compilation. ;-P with Python they are both easy, the meaty part of the task is writing a PMC hierarchy that embodies Python semantics or a non-trivial subset thereof. I imagine that the challenge code should touch on things like __getattribute__, descriptors, bound and unbound methods ... otherwise it would be rather unfair. regards. From michal at sabren.com Mon Aug 4 17:25:51 2003 From: michal at sabren.com (Michal Wallace) Date: Mon Aug 4 16:26:04 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: Message-ID: On Mon, 4 Aug 2003, Dan Sugalski wrote: > To be honest, I'd rather leave it as it is--not because I don't > think that parsing python and compiling to parrot isn't worthwhile > (I think it is) but because I can be reasonably assured that the > conversion will take place (because I want it to happen and have > wagered pie on that want, and thus will make it happen :). Besides, > I'm not much of a parser guy, thus the parsing part's actually more > difficult than the bytecode translation, which is reasonably > straightforward. Fair enough, but have you looked at pirate? The parser's done. Python exposes its own parser via the parser module. There's a python module for walking that tree, and all I'm doing is calling that and writing methods. The only potential snag in the scheme is the exec statement, because somehow parrot's got to have a python parser. But unless I'm missing something completely, you have the same problem with your bytecode interpreter because there is no way you can do exec without a python parser. :) So, since the rules as they are say nothing about not having exec in there, Guido can just build all kinds of dynamic python code strings and test out the exec statement, and you're either going to have to parse it or get pied. :) On the other hand, there's already at least one seemingly-working python parser written in python: http://codespeak.net/moin/pypy/moin.cgi/BytecodeCompiler To me, the shortest path looks like this: - use python's parser to build the tree - finish the pirate code generator - wrap or recode PyObjects as PMC for the built-in types - wrap the python parser library or use a pyPython parser so we can get eval working without the python source - focus on optimizing the generated bytecode I know you're more comfortable looking at bytecode, and yes, some of the translation stuff might be applicable to translating other stack-based bytecode to parrot, but it just seems like it's not really helping anyone. I read through Klaas-Jan's entire Lua->parrot paper yesterday: http://members.home.nl/joeijoei/parrot/report.pdf and except for the for statement (which has different semantics in python), our generated IMCC and the problems we're dealing with are almost identical. That's why we're talking about merging the code generators. I think compiling python source to parrot bytecode is going to take a couple weeks AT MOST. It would go a whole lot faster if someone like you or Leo were pairing with me. Then you could make sure it's spitting out good parrot bytecode (and I wouldn't have to spend hours hunting through the docs), and I could make sure the python semantics are doing the right thing (and you wouldn't have to worry about parsing python). We'd both move a whole lot faster. Really, pair with me for an hour or two and we can knock out function definitions or something. I think you'll see it's a piece of cake. We just both ssh to my server and use screen and an IRC or AIM window to chat, and I'll walk you through the python stuff and you can show me the IMC/PIL/PIR/PASM. It would fly by. Then the tricky part is just making the bytecode run as fast as possible, and dealing with the built-in types, which is what you probably really want to be doing, right? Honestly, the goal shouldn't be to be faster than the python virtual machine. Your goal should be to be to deliver something so compelling that the python team votes unanimously to dump the current VM in favor of parrot for python 3.0. :) Of course if they ever did that, an old-bytecode to parrot-bytecode interpreter wouldn't be of much use. On the OTHER hand, a good generic parrot compiler that's benefitted from Dan the Parrot God's expertise is going to be valuable to *anyone* who wants to compile their language down to parrot. It helps PHP, it helps Lua. It helps Ruby. It could even help Perl! :) Long term, I'm imagining a situation where people don't really have to deal with code generation at all, except to override the parts that need to be customized for their own language. It'll boil down to generating an AST and then transforming the tree to fit with the generic compiler's worldview, and then hitting "go". Everything from that point on down - peephole optimization, calling conventions, imcc, would be standard. Okay, I'm rambling here, so I'll stop... But take me up on my pairing offer. Give me an hour or two. Then, if you're not convinced, fine. And if you are, I'm definitely committed to seeing this through to the end. Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From pycon at python.org Mon Aug 4 17:35:22 2003 From: pycon at python.org (PyCon Committee) Date: Mon Aug 4 16:35:29 2003 Subject: [Python-Dev] PyCon DC 2004 Kickoff! Message-ID: <20030804203522.GA3565@panix.com> [Posted to comp.lang.python/python-list, comp.lang.python.announce, zope-announce, pycon-announce, and python-dev.] [Please repost to local Python mailing lists.] Planning for PyCon DC 2004, the Python community conference, is now underway. DC 2004 will be held March 24-26, 2004 in Washington, D.C. It will contain many of the features of this year's successful conference, including a development sprint and a similar cost structure. Registration will be open by October 1. A Call For Proposals will be issued in the next two weeks. Presentations will be required in electronic form for publication on the web. PyCon DC 2003 had a broad range of presentations, from reports on academic and commercial projects to tutorials and case studies, and these will all be included in the Call For Proposals. As long as the presentation is interesting and potentially useful to the Python community, it will be considered for inclusion in the program. There will again be a significant amount of Open Space to allow for informal and spur-of-the-moment presentations for which no formal submission is required. There will also be several Lightning Talk sessions. These informal sessions are opportunities for those who don't want to make a formal presentation. We're looking for volunteers to help run PyCon. If you're interested, subscribe to http://mail.python.org/mailman/listinfo/pycon-organizers Don't miss any PyCon announcements! Subscribe to http://mail.python.org/mailman/listinfo/pycon-announce You can discuss PyCon with other interested people by subscribing to http://mail.python.org/mailman/listinfo/pycon-interest The central resource for PyCon DC 2004 is http://www.python.org/pycon/dc2004/ -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From dan at sidhe.org Mon Aug 4 18:00:51 2003 From: dan at sidhe.org (Dan Sugalski) Date: Mon Aug 4 17:04:16 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: <5.2.1.1.0.20030804220705.026a76e8@pop.bluewin.ch> References: <5.2.1.1.0.20030804220705.026a76e8@pop.bluewin.ch> Message-ID: At 10:12 PM +0200 8/4/03, Samuele Pedroni wrote: >At 15:15 04.08.2003 -0400, Dan Sugalski wrote: >>OTOH, the... 'interesting' task that is parsing perl may well have >>skewed by feelings on the difficulties of bytecode translation vs >>source compilation. ;-P > >with Python they are both easy, the meaty part of the task is >writing a PMC hierarchy that embodies Python semantics or a >non-trivial subset thereof. I imagine that the challenge code should >touch on things like __getattribute__, descriptors, bound and >unbound methods ... otherwise it would be rather unfair. I fully expect a good chunk of the time to be taken up implementing the builtin behaviour of Python, since as you say it wouldn't be much of a challenge without that. Besides, I can't picture any reasonable python program that wouldn't use pretty much all of the engine internals other than the compiler. -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From amk at amk.ca Mon Aug 4 18:03:02 2003 From: amk at amk.ca (A.M. Kuchling) Date: Mon Aug 4 17:09:00 2003 Subject: [Python-Dev] Re: Re: pirate (python+parrot) References: <20030804132152.C16524@starship.python.net> <20030804144822.A25271@starship.python.net> Message-ID: On Mon, 4 Aug 2003 14:48:22 -0400, Guido van Rossum wrote: > Sure, but bytecode evolves slowly enough to be useful for Dan; > given that the only parse tree format we have is *intricately* linked > with the Python parser and (at least part of) its runtime, It is? Both translators are using the compiler package, not the parser module; can it be said to be intricately linked with the runtime? Admittedly it uses the parser module underneath, but the AST representation itself is pretty generic, isn't it? (Actually, that opens another possibility: use the compiler/ package to produce an AST and then dump it as S-expressions or XML which can be processing by some other AST-to-PASM translator.) --amk (www.amk.ca) AUTOLYCUS: How blessed are we that are not simple men! -- _The Winter's Tale_, IV, iv From barry at python.org Mon Aug 4 23:59:18 2003 From: barry at python.org (Barry Warsaw) Date: Mon Aug 4 18:59:19 2003 Subject: [Python-Dev] Python 2.3 maintenance branch is now open Message-ID: <1060037927.20338.38.camel@yyz> Jack turned over the release23-branch branch to me so I was able to create the Python 2.3 maintenance branch. Here's what I did: - checked out release23-fork - cvs tag -b release23-maint - cvs up -rrelease23-maint - cvs up -j release23-branch - cvs commit IOW, I basically branched release23-maint off of the release23-fork tag on the trunk, then I merged all the changes in the release23-branch to the release23-maint. The release23-maint branch is now open for patches for Python 2.3.1. The release23-branch branch is now closed. Cheers, -Barry From fdrake at acm.org Mon Aug 4 22:48:09 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon Aug 4 21:48:49 2003 Subject: [Python-Dev] Python 2.3 maintenance branch is now open In-Reply-To: <1060037927.20338.38.camel@yyz> References: <1060037927.20338.38.camel@yyz> Message-ID: <16175.3289.798448.651431@grendel.zope.com> Barry Warsaw writes: > The release23-maint branch is now open for patches for Python 2.3.1. > The release23-branch branch is now closed. Should we use something like branchctl to "close" declared-dead branches? We probably don't have enough legacy branches to make it worthwhile for Python, though I know some projects that could use it effectively. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From barry at python.org Tue Aug 5 03:01:24 2003 From: barry at python.org (Barry Warsaw) Date: Mon Aug 4 22:01:25 2003 Subject: [Python-Dev] Python 2.3 maintenance branch is now open In-Reply-To: <16175.3289.798448.651431@grendel.zope.com> References: <1060037927.20338.38.camel@yyz> <16175.3289.798448.651431@grendel.zope.com> Message-ID: <1060048851.1042.8.camel@geddy> On Mon, 2003-08-04 at 21:48, Fred L. Drake, Jr. wrote: > Barry Warsaw writes: > > The release23-maint branch is now open for patches for Python 2.3.1. > > The release23-branch branch is now closed. > > Should we use something like branchctl to "close" declared-dead > branches? We probably don't have enough legacy branches to make it > worthwhile for Python, though I know some projects that could use it > effectively. ;-) I'm not sure Python needs it. It's enough that any checkins on the dead branch will bit rot long before Guido gets a pie in the face . -Barry From bac at OCF.Berkeley.EDU Mon Aug 4 20:21:40 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 4 22:21:53 2003 Subject: [Python-Dev] Document Lib/platform.py ? Message-ID: <3F2F14B4.2070001@ocf.berkeley.edu> While skimming the "What's New" doc for 2.3 I noticed the mention of the platform module. After noticing that there were no docs for it, I checked out the module and noticed that the docstrings are pretty good as-is. So, the question becomes whether it is okay for me to write up some LaTeX docs for Fred to correct. The only reason I am asking instead of just writing the thing up is I know once it is documented it is considered "official" and thus require maintenance and support and I don't know if we want to bother with that. Marc-Andre has already given me the green light to write the docs but I wanted python-dev's go-ahead before I proceeded. -Brett From tim.one at comcast.net Mon Aug 4 23:29:05 2003 From: tim.one at comcast.net (Tim Peters) Date: Mon Aug 4 22:29:46 2003 Subject: [Python-Dev] Document Lib/platform.py ? In-Reply-To: <3F2F14B4.2070001@ocf.berkeley.edu> Message-ID: [Brett] > While skimming the "What's New" doc for 2.3 I noticed the mention of > the platform module. After noticing that there were no docs for > it, I checked out the module and noticed that the docstrings are > pretty good as-is. > > So, the question becomes whether it is okay for me to write up some > LaTeX docs for Fred to correct. The only reason I am asking instead > of just writing the thing up is I know once it is documented it is > considered "official" and thus require maintenance and support and I > don't know if we want to bother with that. Go for it! If Fred could make the time, he'd never let an undocumented module or change go out the door. The somtimes-lagging state of the docs is an unwanted concession to current development reality, not an intentional thing. platform.py became "official" the instant it was released in the standard library. Then again, it's all Guido's problem now . From fdrake at acm.org Mon Aug 4 23:41:14 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon Aug 4 22:41:55 2003 Subject: [Python-Dev] Document Lib/platform.py ? In-Reply-To: <3F2F14B4.2070001@ocf.berkeley.edu> References: <3F2F14B4.2070001@ocf.berkeley.edu> Message-ID: <16175.6474.908389.994365@grendel.zope.com> Brett C. writes: > While skimming the "What's New" doc for 2.3 I noticed the mention of the > platform module. After noticing that there were no docs for it, I > checked out the module and noticed that the docstrings are pretty good > as-is. Yep; they just need to be converted to LaTeX. > So, the question becomes whether it is okay for me to write up some > LaTeX docs for Fred to correct. The only reason I am asking instead of > just writing the thing up is I know once it is documented it is > considered "official" and thus require maintenance and support and I > don't know if we want to bother with that. Just because they're official doesn't mean we can't decide they're buggy, so they can be fixed should it be necessary. It's harder to change the interface as implemented than the interface as documented. > Marc-Andre has already given me the green light to write the docs but I > wanted python-dev's go-ahead before I proceeded. There's an open documentation bug report assigned to Marc-Andre for this; I'll re-assign it to you. ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From bac at OCF.Berkeley.EDU Mon Aug 4 20:52:32 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 4 22:52:40 2003 Subject: [Python-Dev] Document Lib/platform.py ? In-Reply-To: <16175.6474.908389.994365@grendel.zope.com> References: <3F2F14B4.2070001@ocf.berkeley.edu> <16175.6474.908389.994365@grendel.zope.com> Message-ID: <3F2F1BF0.8050209@ocf.berkeley.edu> Fred L. Drake, Jr. wrote: > Brett C. writes: > > While skimming the "What's New" doc for 2.3 I noticed the mention of the > > platform module. After noticing that there were no docs for it, I > > checked out the module and noticed that the docstrings are pretty good > > as-is. > > Yep; they just need to be converted to LaTeX. > Yeah, just a few clarifications and a small amount of cleanup. Shouldn't be a big deal. > There's an open documentation bug report assigned to Marc-Andre for > this; I'll re-assign it to you. ;-) > Gee, thanks, Fred. =) -Brett From fdrake at acm.org Mon Aug 4 23:53:19 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Mon Aug 4 22:53:59 2003 Subject: [Python-Dev] Document Lib/platform.py ? In-Reply-To: <3F2F1BF0.8050209@ocf.berkeley.edu> References: <3F2F14B4.2070001@ocf.berkeley.edu> <16175.6474.908389.994365@grendel.zope.com> <3F2F1BF0.8050209@ocf.berkeley.edu> Message-ID: <16175.7199.85215.482474@grendel.zope.com> Brett C. writes: > Gee, thanks, Fred. =) Surely you expected no less? ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From bac at OCF.Berkeley.EDU Mon Aug 4 21:00:33 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 4 23:00:43 2003 Subject: [Python-Dev] python-dev Summary for 2003-07-01 through 2003-07-31 [draft] Message-ID: <3F2F1DD1.8070603@ocf.berkeley.edu> Here is the rough draft of the Summary for the month of July (once again I combined two summaries into a single month-long summary since I couldn't get to the earlier summary soon enough to warrant a separate release). As usual you guys have a day or so to send in corrections. I would appreciate not just correctional feedback but just overall feedback as well for this summary. I have changed the format once again as well as the general philosophy behind the Summaries in preparation for when I start grad school and have to cut back on the breadth of the Summaries. You can read the Summary Announcements section to see what and why I have changed. Any feedback is appreciated. ------------- python-dev Summary for 2003-07-01 through 2003-07-31 ++++++++++++++++++++++++++++++++++++++++++++++++++++ This is a summary of traffic on the `python-dev mailing list`_ from July 1, 2003 through July 31, 2003. It is intended to inform the wider Python community of on-going developments on the list and to have some impact or hold interest to the wider Python community. To comment on anything mentioned here, just post to python-list@python.org or `comp.lang.python`_ with a subject line mentioning what you are discussing. All python-dev members are interested in seeing ideas discussed by the community, so don't hesitate to take a stance on something. And if all of this really interests you then get involved and join `python-dev`_! This is the twenty-first/twenty-second summary written by Brett Cannon (who is still learning how to count; gave the wrong summary count last month). All summaries are archived at http://www.python.org/dev/summary/ . Please note that this summary is written using reStructuredText_ which can be found at http://docutils.sf.net/rst.html . Any unfamiliar punctuation is probably markup for reST_ (otherwise it is probably regular expression syntax or a typo =); you can safely ignore it, although I suggest learning reST; its simple and is accepted for `PEP markup`_ and gives some perks for the HTML output. Also, because of the wonders of programs that like to reformat text, I cannot guarantee you will be able to run the text version of this summary through Docutils_ as-is unless it is from the original text file. .. _PEP Markup: http://www.python.org/peps/pep-0012.html The in-development version of the documentation for Python can be found at http://www.python.org/dev/doc/devel/ and should be used when looking up any documentation on something mentioned here. Python PEPs (Python Enhancement Proposals) are located at http://www.python.org/peps/ . To view files in the Python CVS online, go to http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ . .. _python-dev: http://www.python.org/dev/ .. _python-dev mailing list: http://mail.python.org/mailman/listinfo/python-dev .. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python .. _Docutils: http://docutils.sf.net/ .. _reST: .. _reStructuredText: http://docutils.sf.net/rst.html .. contents:: .. _last summary: http://www.python.org/dev/summary/2003-06-01_2003-06-30.html ===================== Summary Announcements ===================== It's experimentation time again! In preparation for having to cut down on the breadth of the summary thanks to my return to school I am starting this summary off as the inaugural release of the new format. Pretty much everything is going to change somewhat. First, though, I think I should explain my new viewpoint towards the summaries that I am taking. In order to continue to provide a certain level of quality and coverage while retaining my personal sanity and to not feel like the Summaries are a job instead of something I do for fun and to help others, I am going to start viewing the Summaries more from a journalistic perspective than a historical one. Before I felt more like a historian that was summarizing *everything* that happened on python-dev. Now, though, I am be more like a journalist, covering only the pertinent ideas. This also allows me to inject more of my personality into my writing. But what ramifications does this new perspective cause? Well, it will dictate what I do and do not summarize. I plan on several types of discussions to be covered in the Summaries. One is new features of the language. This is so people can know about them ASAP and thus comment on them. Since practically any feature mentioned here is experimental there is a chance to remove it if a community movement mounts. I will also cover any major thinking on the future of Python. I have always wished there was an easy way to see what pie-in-the-sky ideas python-dev was thinking of in the back of our collective mind; the summaries might as well fill this desire. And of course any random discussion that is large, important, and/or complex will continue to be covered. In terms of formatting and layout, that will also change along with the new viewpoint. I am planning to follow something more along the lines of the Perl's `This Week... `__; yes, I am taking a cue from Perl but AM Kuchling and Michael Hudson originally used this format as well. This means titles of summary sections will more reflect the topic being summarized rather than the specific thread(s). I will continue to list all of the threads that contributed to the summary, though, for reference purposes. I feel this will be beneficial since I know I have personally looked back into the Summary archives and had a hard time finding something specific since the subject of a thread does not always match the topic of discussion; if I can't find something and I know about what month it happened I can only imagine what everyone else has had to go through to find something! I am removing the Quickies section from the Summaries. This section was nice to have, but it just ate away at my time. This also goes along with the new viewpoint. I am sure not everyone will be happy with this change, that's fine. But if someone out there *truly* hates the direction I am going then they can step forward and take the reigns from me. But as I have stated before, the fine print on me giving up this post is that the person who takes over must be as thorough as I *used* to be; I can do the summaries in a half-assed fashion as well as anyone else. Enough of that topic. One thing I would like to mention is that the `Python Software Foundation`_ can now accept donations through PayPal. Specifics can be found at http://www.python.org/psf/donations.html . It would be great if people could donate since the PSF is more than just a non-profit to hold the Python intellectual property. One of the goals of the group is to get enough money to do something like the Perl Foundation does and sponsor someone to hack on Python for a living for a set amount of time. Right now we are a ways off from having that kind of revenue, but there is no reason why we can't. Heck, if we can just get enough to sponsor one person to do full-time Python hacking for a week it would be beneficial. Or better yet, the PSF could sponsor one of their own members to write a summary of the python-dev mailing list for the betterment of the community. =) .. _PSF: .. _Python Software Foundation: http://www.python.org/psf/ .. _PayPal: http://www.paypal.com/ ========= Summaries ========= -------------------------- Python 2.3 final released! -------------------------- In case for some odd reason you didn't know, Python 2.3 final has gone out the door. The team managed to meet the goal of getting it out by the end of the month so that Apple_ can use a final release instead of a release candidate in OS X 10.3 (a.k.a. Panther). The goal of this release was to continue to improve Python without adding any new keywords. This was met while speeding things up, fixing bugs, adding to the standard library, and adding some new built-ins. To find out what is new or has changed look at the changelog at http://www.python.org/2.3/highlights.html or `What's New in Python 2.3`_. As with all software, there are bound to be bugs. What bugs that are known are listed at http://www.python.org/2.3/bugs.html . If you find bugs not listed there, please report them on the SourceForge bug tracker at http://sourceforge.net/tracker/?group_id=5470&atid=105470 . Running the regression test suite (details on how to do this is covered in the documentation for the 'test' package as found at http://www.python.org/doc/current/lib/module-test.html) is always appreciated. Contributing threads: - `Problem with 2.3b2 tarfile?`__ - `MacPython 2.3b2 binary installer for MacOSX`__ - `Running tests on freebsd5`__ - `Python 2.3 release schedule update`__ - `Vacation and possibly a new bug`__ - `Doc/ tree closes at noon, US/Eastern`__ - `I've tagged the tree`__ - `cygwin errors`__ - `test_strptime; test_logging; test_time failure`__ - `2.3rc1 delayed`__ - `Python 2.3 release candidate 1`__ - `post-release festivities`__ - `New branch for r23c2 work`__ - `Serious CallTip Bug in IDLE`__ - `MacPython-OS9 test failures`__ - `Doc/ tree on trunk closed`__ - `Cutting the 2.3c2 tonight 8pm EDT`__ - `Re: [Python-checkins] python/dist/src/Misc NEWS,1.826,1.827`__ - `RELEASED Python 2.3c2`__ - `2.3rc2 IDLE problems on a Win2000sp3 box`__ - `Draft: 2.3 download page`__ - `Last priority >= 7 bug for Python 2.3`__ - `CVS Freeze`__ - `RELEASED Python 2.3 (final)`__ __ http://mail.python.org/pipermail/python-dev/2003-July/036669.html __ http://mail.python.org/pipermail/python-dev/2003-July/036732.html __ http://mail.python.org/pipermail/python-dev/2003-July/036762.html __ http://mail.python.org/pipermail/python-dev/2003-July/036783.html __ http://mail.python.org/pipermail/python-dev/2003-July/036845.html __ http://mail.python.org/pipermail/python-dev/2003-July/036892.html __ http://mail.python.org/pipermail/python-dev/2003-July/036899.html __ http://mail.python.org/pipermail/python-dev/2003-July/036900.html __ http://mail.python.org/pipermail/python-dev/2003-July/037117.html __ http://mail.python.org/pipermail/python-dev/2003-July/036921.html __ http://mail.python.org/pipermail/python-dev/2003-July/036940.html __ http://mail.python.org/pipermail/python-dev/2003-July/036941.html __ http://mail.python.org/pipermail/python-dev/2003-July/037024.html __ http://mail.python.org/pipermail/python-dev/2003-July/037134.html __ http://mail.python.org/pipermail/python-dev/2003-July/037144.html __ http://mail.python.org/pipermail/python-dev/2003-July/037193.html __ http://mail.python.org/pipermail/python-dev/2003-July/037200.html __ http://mail.python.org/pipermail/python-dev/2003-July/037207.html __ http://mail.python.org/pipermail/python-dev/2003-July/037208.html __ http://mail.python.org/pipermail/python-dev/2003-July/037237.html __ http://mail.python.org/pipermail/python-dev/2003-July/037284.html __ http://mail.python.org/pipermail/python-dev/2003-July/037311.html __ http://mail.python.org/pipermail/python-dev/2003-July/037315.html __ http://mail.python.org/pipermail/python-dev/2003-July/037319.html .. _Apple: http://www.apple.com/ .. _What's New in Python 2.3: http://www.python.org/doc/2.3/whatsnew/ ------------------------------------------- String Exceptions Going Extinct, Eventually ------------------------------------------- It has been brought several times before, but things are now moving along to eventually deprecated string exceptions (e.g., ``raise "this string exception"``). they have now been removed from the tutorial to prevent people new to the language from picking up habits that will eventually be impossible to perform. As for when the impending deprecation will occur only Guido knows. The chances of it coming before Python 3, though, is most likely small. Contributing threads: - `String exceptions in the Tutorial`__ __ http://mail.python.org/pipermail/python-dev/2003-July/036665.html -------------------------------------------------------- Someday, My CVS updates Will Succeed on the First Try... -------------------------------------------------------- We all know that SourceForge_ does not have the snappiest or most reliable CVS_ servers in the world; Python 2.3 final's release was targeted a few days earlier than needed for Apple_ in case SF's CVS went down. Solutions to how to deal with this problem have come up. One specific problem that people are trying to solve is the poor anonymous pserver access. SF has their CVS servers set up so that authorized CVS commands take priority over anonymous ones and thus when there is heavy traffic anonymous CVS basically shuts down. Martin v. L?wis put up a link to a nightly tarball at http://www.dcl.hpi.uni-potsdam.de/home/loewis/python.tgz of the CVS HEAD. perl.org has provided a nightly tarball as well at http://cvs.perl.org/snapshots/python/ for quite some time now. The suggestion of having a read-only version of the CVS tree on python.org was came up, but how to properly do it and who would set it up was not settled. And of course there is always the option of moving CVS off of SF entirely. There is one offer on the table with the caveat that we need to know what kind of bandwidth demands we would have. Another offer sprung up this month from Michal Wallace to host our CVS at http://www.versionhost.com/ with the hitch that the server would be pserver only which would thus most likely only work for anonymous checkouts Contributing threads: - `alternate pserver access for python CVS?`__ - `[fwd] SourceForge status`__ - `Volunteering to help with pserver cvs on python.org`__ __ http://mail.python.org/pipermail/python-dev/2003-July/036699.html __ http://mail.python.org/pipermail/python-dev/2003-July/036708.html __ http://mail.python.org/pipermail/python-dev/2003-July/036860.html .. _SourceForge: http://www.sf.net/ .. _CVS: http://www.cvshome.org/ --------------------- Which License to Use? --------------------- Domenico Andreoli wanted to donate `python-crack`_ to the PSF_ and wondered what steps would be necessary to do so. He was first told that the module would not be included in Python since it is too specific. Secondly, it was suggested he not choose the PSF license. Instead, he was told that he should use the BSD or MIT license (both of which can be found at http://www.opensource.org/licenses/index.php); clean, simple licenses that are not viral like the GPL but are GPL-compatible. This is so that it can be included in any software that uses Python. Contributing threads: - `donating python-crack module`__ __ http://mail.python.org/pipermail/python-dev/2003-July/036712.html .. _python-crack: http://savannah.nongnu.org/projects/python-crack ---------------- Not Quite ANSI C ---------------- Every so often someone runs Python against a C code checker and notices failures that get reported to us. Often, though, we have to tell the person that we already know of the failures and that there is nothing to worry about. This time it was failures under Purify. We have also received reports of failures under Valgrind_ which are spurious. The language isn't even fully ANSI C compliant, but as Tim Peters said, "it's not possible to write a useful and portable memory allocator in 100% standard ANSI C" among other reasons why strict ANSI C conformity is broken. The only major violations in the code are casting a pointer to a unsigned integer type and assuming "that whatever memory protection gimmicks an OS implements are at page granularity" which is a minor sin at best. There is also some cases of accessing memory without proper casting. Contributing threads: - `Invalid memory read in PyObject_Free`__ - `strict aliasing and Python`__ __ http://mail.python.org/pipermail/python-dev/2003-July/036717.html __ http://mail.python.org/pipermail/python-dev/2003-July/036898.html .. _Valgrind: http://developer.kde.org/~sewardj/ --------------------------------------- Shadowing Built-ins is Becoming a No-No --------------------------------------- Before the code was removed, Python 2.3 raised a DeprectionWarning when you overshadowed a built-in in a module from outside the module's actual code. What this means is that doing something like:: import mod mod.reload = 42 would raise the warning. Now doing ``reload = 42`` in the module itself would still be allowed. The idea was (and still is) that the compiler can know when a built-in is shadowed in a module when it compiles the module's code. But allowing someone to overwrite a built-in externally means that the compiler cannot make any assumptions that built-ins will forever reference the actual built-in code. This is unfortunate since if shadowing externally was made illegal the compiler can reference the built-in code directly and skip some steps for a nice speed-up. This is why there was code to warn against this initially. But it didn't work well enough to be left in Python 2.3 so it has been removed. But this restriction will most likely be enforced at some point so it is something to keep in mind. Contributing threads: - `Deprecation warning 'assignment shadows builtin'`__... - `DeprecationWarning: assignment shadows builtin`__ __ http://mail.python.org/pipermail/python-dev/2003-July/036767.html __ http://mail.python.org/pipermail/python-dev/2003-July/036879.html ------------------------ Must...Start...Faster... ------------------------ While Python 2.3 sees up to a 30% speed increase over 2.2, it still starts up slower. Originally it was believed that it was because of the importation of the re module. Marc-Andr? Lemburg applied a patch that tried to minimize its possible importation at startup but it was still slow. Jeremy Hylton pointed out that 2.2.3 imported re so that probably was not the problem. One thing that is different, though, is the automatic inclusion of warnings, codecs, and the ability to do zip imports. These three things pull more modules and all of this leads to a longer startup time. There is also the importation of distutils thanks to the site.py module, but that only occurs if you execute in a build directory. Including zip imports also got attacked for the number of failed checks for files it can incur. But it is believed this is not as bad as all of the extra imports being done now and Just van Rossum said he would try to cut down the unneeded checks in the future. Contributing threads: - `2.3 startup speed?`__ __ http://mail.python.org/pipermail/python-dev/2003-July/036815.html --------------------- The World and Numbers --------------------- People might not be aware of it, but locale.setlocale never changes the LC_NUMERIC locale value. This is so that 'str' and 'float' can return and receive numbers in a consistent way in source code. This causes a problem, though, for people in locales that do not use a period for a decimal point. If you had your locale set to Brazilian and tried to pass the string "3,14" to 'float' it would fail even though that is a legitimate number representation for your locale. The suggestion become to have a locale-agnostic version of 'str' and 'float' in the locale module for people to use when they needed such functionality. It has now reached the stage of a rough draft PEP as found at http://mail.python.org/pipermail/python-dev/2003-July/036996.html . Contributing threads: - `LC_NUMERIC and C libraries`__ - `Be Honest about LC_NUMERIC`__ - `RE: [Spambayes] Question (or possibly a bug report)`__ __ http://mail.python.org/pipermail/python-dev/2003-July/036869.html __ http://mail.python.org/pipermail/python-dev/2003-July/036996.html __ http://mail.python.org/pipermail/python-dev/2003-July/037178.html --------------- Zippin' Nothin' --------------- The built-in 'zip' function in Python 2.3 raises TypeError when given no arguments. This causes problem when you do ``zip(*args)`` and args is empty. So for Python 2.4 'zip' with no arguments will return an empty list. Contributing threads: - `zip() in Py2.4`__ __ http://mail.python.org/pipermail/python-dev/2003-July/037332.html -------------------------------- A Syntax For Function Attributes -------------------------------- An old debate (or at least old in terms of it having popped up multiple times during the tenure of your author) has been ways of enhancing function definitions. Previously it has been over ways to make defining classmethod and its ilk in a cleaner fashion along with properties (past summaries on this can be found at http://www.python.org/dev/summary/2003-01-16_2003-01-31.html#extended-function-syntax and http://www.python.org/dev/summary/2003-02-01_2003-02-15.html#extended-function-syntax ). What made the discussion slightly different this time, though, was the focus was on function attributes specifically. One idea was to tweak Michael Hudson's bracket syntax for method annnotations to allow for function attributes; ``def foo() [bar=42]: pass``. Another suggestion was to take the dictionary-like connection even farther; ``def foo() {bar=42}: pass``. There was no clear winner in either case partially because this discussion flared up towards the end of the month. It was also pointed out that using Michael Hudson's method descriptors you could come up with a wrapping function that handled the situation:: def attrs(**kw): def _(func): for k in kw: setattr(func, k, kw[k]) return func return _ def foo() [attrs(bar=42)]: pass Perk of this is it does not lead to any special syntax just for function attributes. One drawback, though, as pointed out by Paul Moore is that this makes the method annotation idea a little to liberal and open to hackish solutions like this that will lead to abuse. This was not a huge worry, though, since almost anything can be abused and this is all already doable now, just without a nice, clean syntax. Contributing threads: - `A syntax for function attributes?`__ __ http://mail.python.org/pipermail/python-dev/2003-July/037338.html From michal at sabren.com Tue Aug 5 00:20:43 2003 From: michal at sabren.com (Michal Wallace) Date: Mon Aug 4 23:20:48 2003 Subject: [Python-Dev] python-dev Summary for 2003-07-01 through 2003-07-31 [draft] In-Reply-To: <3F2F1DD1.8070603@ocf.berkeley.edu> Message-ID: On Mon, 4 Aug 2003, Brett C. wrote: > Another offer sprung up this month from Michal Wallace to host our > CVS at http://www.versionhost.com/ with the hitch that the server > would be pserver only which would thus most likely only work for > anonymous checkouts Not at all! That's the whole point. :) And if you like using ssh, there's always stunnel, or: http://www.sabren.net/code/cvssh/ Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From bac at OCF.Berkeley.EDU Mon Aug 4 21:25:08 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 4 23:25:31 2003 Subject: [Python-Dev] python-dev Summary for 2003-07-01 through 2003-07-31 [draft] In-Reply-To: References: Message-ID: <3F2F2394.4070807@ocf.berkeley.edu> Michal Wallace wrote: > On Mon, 4 Aug 2003, Brett C. wrote: > > >>Another offer sprung up this month from Michal Wallace to host our >>CVS at http://www.versionhost.com/ with the hitch that the server >>would be pserver only which would thus most likely only work for >>anonymous checkouts > > > Not at all! That's the whole point. :) > > And if you like using ssh, there's always stunnel, or: > http://www.sabren.net/code/cvssh/ > Glad to hear that! I know I would not be the only one who would strongly object to moving the CVS over to anything that didn't have some form of SSH protection (and allowed for SSH keys since I know I am too damn lazy to type my password in constantly =) . -Brett From skip at pobox.com Mon Aug 4 23:49:02 2003 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 4 23:49:12 2003 Subject: [Python-Dev] New place for python-mode bug reports and patches Message-ID: <16175.10542.302239.792809@montanaro.dyndns.org> As many of you are already aware, Barry Warsaw recently created a new project on SourceForge to house the python-mode.el code which makes Emacs and XEmacs so delightful to edit Python code. I just completed migrating all open bugs and patches which mention python-mode from the python project to the new python-mode project. In the future, please submit all bugs and patches at http://sourceforge.net/projects/python-mode/ All new bugs and patches regarding python-mode should be submitted to the new project. There's also a mailing list, python-mode@python.org, to which you can subscribe in the usual Pythonic fashion: http://mail.python.org/mailman/listinfo/python-mode Skip From logistix at cathoderaymission.net Tue Aug 5 03:58:32 2003 From: logistix at cathoderaymission.net (logistix) Date: Tue Aug 5 02:52:51 2003 Subject: [Python-Dev] Re: Re: pirate (python+parrot) In-Reply-To: Message-ID: <000001c35b1e$fc938a60$20bba8c0@XP> [guido@python.net] > wrote: > > Sure, but bytecode evolves slowly enough to be useful for Dan; given > > that the only parse tree format we have is *intricately* linked with > > the Python parser and (at least part of) its runtime, > [AMK] > It is? Both translators are using the compiler package, not > the parser > module; can it be said to be intricately linked with the runtime? > Admittedly it uses the parser module underneath, but the AST > representation > itself is pretty generic, isn't it? > As Michal touched on earlier, I wrote a hand-coded pure-python parser that plugs into the compiler package. It takes about 1.5-2x as long to compile vs using the native parser module. The resultant tuples can also be shoved back into genuine parse trees via parser.sequence2ast(). So yeah, it's not that big of a deal to generate AST trees without the runtime (until some crazy function attribute syntax gets added to python, that is) http://www.cathoderaymission.net/~logistix/python/pparser.py -Grant From mwh at python.net Tue Aug 5 12:22:09 2003 From: mwh at python.net (Michael Hudson) Date: Tue Aug 5 06:22:14 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: <20030804144822.A25271@starship.python.net> (Guido van Rossum's message of "Mon, 4 Aug 2003 14:48:22 -0400") References: <20030804132152.C16524@starship.python.net> <20030804144822.A25271@starship.python.net> Message-ID: <2mllu8qu9a.fsf@starship.python.net> Guido van Rossum writes: > Interesting indeed. The focus on bytecode is understandable: let > someone else write a parser, so Parrot won't have to. But from > skimming his log, Dan seems to take the bytecode too seriously; > it's really only meant to be a quick way for the parser to output > something. In particular, bytecode hacks were never a goal. Really? You surprise me . Cheers, mwh -- 3. Syntactic sugar causes cancer of the semicolon. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From Jack.Jansen at cwi.nl Tue Aug 5 13:47:47 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Tue Aug 5 06:45:24 2003 Subject: [Python-Dev] Python 2.3 maintenance branch is now open In-Reply-To: <1060037927.20338.38.camel@yyz> Message-ID: <4053FFDF-C732-11D7-BB10-0030655234CE@cwi.nl> On Tuesday, Aug 5, 2003, at 00:58 Europe/Amsterdam, Barry Warsaw wrote: > Jack turned over the release23-branch branch to me so I was able to > create the Python 2.3 maintenance branch. Here's what I did: > > - checked out release23-fork > - cvs tag -b release23-maint > - cvs up -rrelease23-maint > - cvs up -j release23-branch > - cvs commit Barrt, how about porting things back to the trunk? Is that my responsibility for the stuff I changed between 2.3 and 2.3-mac? -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From mcherm at mcherm.com Tue Aug 5 05:30:28 2003 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 5 07:30:40 2003 Subject: [Python-Dev] RE: python-dev Summary for 2003-07-01 through 2003-07-31 [draft] Message-ID: <1060083028.3f2f95543a9a2@mcherm.com> Brett writes: > I would appreciate not just correctional feedback but just overall > feedback as well for this summary. I have changed the format once again > as well as the general philosophy behind the Summaries Brett, you're doing great as usual. Saving enough time to survive grad school may have been your primary motivation, but I actually think that this revised format is better than your exhaustive one. IMHO people who wanted to know every detail that went on would just subscribe to python-dev, and people who wanted historical accuracy would just consult the list archives. Your summaries made these tasks easier, but could never be as exhaustive as just reading the list. However, the "more journalistic" approach is really nice because it helps to separate out the relevent from the insignifigant. It is a very useful filtering of the information, providing something you CAN'T get by reading the list. Of course, it's *your* idea of what's important and what's not, but at this point I'd say you're as qualified as any to make that call, and more qualified than almost all. Keep up the great work! -- Michael Chermside From mwh at python.net Tue Aug 5 13:31:08 2003 From: mwh at python.net (Michael Hudson) Date: Tue Aug 5 07:31:14 2003 Subject: [Python-Dev] Python 2.3 maintenance branch is now open In-Reply-To: <1060037927.20338.38.camel@yyz> (Barry Warsaw's message of "04 Aug 2003 18:58:47 -0400") References: <1060037927.20338.38.camel@yyz> Message-ID: <2m8yq8qr2b.fsf@starship.python.net> Barry Warsaw writes: > Jack turned over the release23-branch branch to me so I was able to > create the Python 2.3 maintenance branch. Here's what I did: > > - checked out release23-fork > - cvs tag -b release23-maint > - cvs up -rrelease23-maint > - cvs up -j release23-branch > - cvs commit > > IOW, I basically branched release23-maint off of the release23-fork tag > on the trunk, then I merged all the changes in the release23-branch to > the release23-maint. Here's a recipe I use for branches that others might find useful: $ cd ~/src/python/dist/ $ cvs co -d src-23x -r release23-maint python/dist/src (This creates -- eventually -- a src-23x directory alongside the src directory which contains the trunk) Then hack CVS/Entries to contain a line D/src-23x//// Now running cvs up in python/dist/ will update both the trunk and the release23-maint code (and in my case release22-maint, too). Cheers, mwh -- Just point your web browser at http://www.python.org/search/ and look for "program", "doesn't", "work", or "my". Whenever you find someone else whose program didn't work, don't do what they did. Repeat as needed. -- Tim Peters, on python-help, 16 Jun 1998 From barry at python.org Tue Aug 5 13:45:31 2003 From: barry at python.org (Barry Warsaw) Date: Tue Aug 5 08:45:32 2003 Subject: [Python-Dev] Python 2.3 maintenance branch is now open In-Reply-To: <4053FFDF-C732-11D7-BB10-0030655234CE@cwi.nl> References: <4053FFDF-C732-11D7-BB10-0030655234CE@cwi.nl> Message-ID: <1060087498.8108.2.camel@anthem> On Tue, 2003-08-05 at 06:47, Jack Jansen wrote: > Barrt, > how about porting things back to the trunk? Is that my responsibility > for the stuff I changed between 2.3 and 2.3-mac? I wasn't planning on porting things back to the trunk, so it would be great if you could do it. -Barry From kbk at shore.net Tue Aug 5 12:54:50 2003 From: kbk at shore.net (Kurt B. Kaiser) Date: Tue Aug 5 11:54:54 2003 Subject: [Python-Dev] IDLE NEWS and Version Message-ID: IDLE has historically maintained its own NEWS file, and it is now accessible from IDLE's Help/About menu. Maintaining this file has the advantage that all the IDLE NEWS is kept in one place and is easily parsable for display. Do we want to continue this tradition or merge IDLE NEWS into Python NEWS on a go-forward basis? Second, do we want to continue the IDLE versioning (currently 1.0) or just drop it now that IDLE is no longer distributed separately? -- KBK From dan at sidhe.org Tue Aug 5 11:33:13 2003 From: dan at sidhe.org (Dan Sugalski) Date: Tue Aug 5 12:47:30 2003 Subject: [Python-Dev] Re: pirate (python+parrot) In-Reply-To: References: Message-ID: At 4:25 PM -0400 8/4/03, Michal Wallace wrote: >On Mon, 4 Aug 2003, Dan Sugalski wrote: > >> To be honest, I'd rather leave it as it is--not because I don't >> think that parsing python and compiling to parrot isn't worthwhile >> (I think it is) but because I can be reasonably assured that the >> conversion will take place (because I want it to happen and have >> wagered pie on that want, and thus will make it happen :). Besides, >> I'm not much of a parser guy, thus the parsing part's actually more >> difficult than the bytecode translation, which is reasonably >> straightforward. > >Fair enough, but have you looked at pirate? Yep. Checked it out earlier today. A quick scan of it looks pretty cool. >The >parser's done. Python exposes its own parser via >the parser module. There's a python module for >walking that tree, and all I'm doing is calling >that and writing methods. You do have a significant advantage in this as you've done a lot of python work already, so you're familiar with the language. I'm much less familiar, and as such a bytecode translation's easier, or at least I find it easier. None of that pesky syntax to deal with. :) >The only potential snag in the scheme is the exec >statement, because somehow parrot's got to have a >python parser. But unless I'm missing something >completely, you have the same problem with your >bytecode interpreter because there is no way >you can do exec without a python parser. :) True, but the original terms of the challenge precluded this, along with extensions written in C. I don't know if that made it into the final challenge--if not, I may have been outmaneuvered... >I know you're more comfortable looking at bytecode, >and yes, some of the translation stuff might be >applicable to translating other stack-based bytecode >to parrot, but it just seems like it's not really >helping anyone. Not strictly true. Besides the parrot infrastructure this challenge will force me to make sure is working, it'll be the easiest way to make sure that all the semantics that need to be there for Python will be there, so it's a good reality/design check before we go beta or final, depending on timing. Plus I get to pitch a pie at someone who may well be Guido. :-P >I read through Klaas-Jan's entire Lua->parrot paper >yesterday: http://members.home.nl/joeijoei/parrot/report.pdf >and except for the for statement (which has different >semantics in python), our generated IMCC and the problems >we're dealing with are almost identical. That's why >we're talking about merging the code generators. Which is a good project, and I'm digging to those messages now. I'll comment more on p6i, where it's more appropriate. >I think compiling python source to parrot bytecode is >going to take a couple weeks AT MOST. It would go a >whole lot faster if someone like you or Leo were >pairing with me. Talk to Leo, though there may be timezone issues. (He's GMT+100) I'd love to, though I'm pretty desperately short of large chunks of time. :( >Then the tricky part is just making the bytecode run as >fast as possible, and dealing with the built-in types, >which is what you probably really want to be doing, right? Absolutely. >Honestly, the goal shouldn't be to be faster than the >python virtual machine. Your goal should be to be to >deliver something so compelling that the python team votes >unanimously to dump the current VM in favor of parrot >for python 3.0. :) Oddly enough, that's not my goal, and never has been. While I wouldn't mind, it's a bit presumptuous on my part. Language designers, in my experience (granted, somewhat limited, I've only met a dozen or so) tend to want to fiddle with the software as they go as well, so even when Parrot runs things faster they'll still want to write the software to test out their ideas. As an example, if I handed Matz a Ruby on Parrot that ran an order of magnitude faster than plain ruby he'd be thrilled, and still work on his version of the ruby interpreter, and that's fine. So, while I'd be happy if Guido decided that Parrot was a sufficient win to switch over to it as the core for some version of Python, that's not my goal. I'm OK with Parrot having Jython status for any language I can get my hands on. >Long term, I'm imagining a situation where people don't >really have to deal with code generation at all, except >to override the parts that need to be customized for >their own language. It'll boil down to generating >an AST and then transforming the tree to fit with the >generic compiler's worldview, and then hitting "go". >Everything from that point on down - peephole optimization, >calling conventions, imcc, would be standard. That's always been the plan, though my dislike of parsing's kept it near the back of the list of things to do. Time to change that, I think. ;) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk From sanket at kelseus.com Tue Aug 5 18:52:39 2003 From: sanket at kelseus.com (Sanket Patel) Date: Tue Aug 5 12:52:25 2003 Subject: [Python-Dev] RE: Python-Dev Digest, Vol 1, Issue 3435 Message-ID: <4317512F97B4D311A7EA0050DA3DA510155BED@EXCHANGE> Stop spaming our server. Please remove all the email addresses containing @kelseus.com domain from your mailing list. Sanket Patel Network Engineer Kelseus Ltd DDI: +44 (0)1223 471235 Fax: +44 (0)1223 471220 http://www.kelseus.com -----Original Message----- From: python-dev-request@python.org [mailto:python-dev-request@python.org] Sent: Tuesday, August 05, 2003 5:01 PM To: python-dev@python.org Subject: Python-Dev Digest, Vol 1, Issue 3435 Send Python-Dev mailing list submissions to python-dev@python.org To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-dev or, via email, send a message with subject or body 'help' to python-dev-request@python.org You can reach the person managing the list at python-dev-owner@python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-Dev digest..." Today's Topics: 1. Re: python-dev Summary for 2003-07-01 through 2003-07-31 [draft] (Michal Wallace) 2. Re: python-dev Summary for 2003-07-01 through 2003-07-31 [draft] (Brett C.) 3. New place for python-mode bug reports and patches (Skip Montanaro) 4. RE: Re: Re: pirate (python+parrot) (logistix) 5. Re: Re: pirate (python+parrot) (Michael Hudson) 6. Re: Python 2.3 maintenance branch is now open (Jack Jansen) 7. RE: python-dev Summary for 2003-07-01 through 2003-07-31 [draft] (Michael Chermside) 8. Re: Python 2.3 maintenance branch is now open (Michael Hudson) 9. Re: Python 2.3 maintenance branch is now open (Barry Warsaw) 10. IDLE NEWS and Version (Kurt B. Kaiser) ---------------------------------------------------------------------- Message: 1 Date: Mon, 4 Aug 2003 23:20:43 -0400 (EDT) From: Michal Wallace Subject: Re: [Python-Dev] python-dev Summary for 2003-07-01 through 2003-07-31 [draft] To: brett@python.org Cc: python-dev@python.org Message-ID: Content-Type: TEXT/PLAIN; charset=US-ASCII On Mon, 4 Aug 2003, Brett C. wrote: > Another offer sprung up this month from Michal Wallace to host our > CVS at http://www.versionhost.com/ with the hitch that the server > would be pserver only which would thus most likely only work for > anonymous checkouts Not at all! That's the whole point. :) And if you like using ssh, there's always stunnel, or: http://www.sabren.net/code/cvssh/ Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- ------------------------------ Message: 2 Date: Mon, 04 Aug 2003 20:25:08 -0700 From: "Brett C." Subject: Re: [Python-Dev] python-dev Summary for 2003-07-01 through 2003-07-31 [draft] To: Michal Wallace Cc: brett@python.org, python-dev@python.org Message-ID: <3F2F2394.4070807@ocf.berkeley.edu> Content-Type: text/plain; charset=us-ascii; format=flowed Michal Wallace wrote: > On Mon, 4 Aug 2003, Brett C. wrote: > > >>Another offer sprung up this month from Michal Wallace to host our >>CVS at http://www.versionhost.com/ with the hitch that the server >>would be pserver only which would thus most likely only work for >>anonymous checkouts > > > Not at all! That's the whole point. :) > > And if you like using ssh, there's always stunnel, or: > http://www.sabren.net/code/cvssh/ > Glad to hear that! I know I would not be the only one who would strongly object to moving the CVS over to anything that didn't have some form of SSH protection (and allowed for SSH keys since I know I am too damn lazy to type my password in constantly =) . -Brett ------------------------------ Message: 3 Date: Mon, 4 Aug 2003 22:49:02 -0500 From: Skip Montanaro Subject: [Python-Dev] New place for python-mode bug reports and patches To: python-list@python.org, python-dev@python.org, python-mode@python.org Message-ID: <16175.10542.302239.792809@montanaro.dyndns.org> Content-Type: text/plain; charset=us-ascii As many of you are already aware, Barry Warsaw recently created a new project on SourceForge to house the python-mode.el code which makes Emacs and XEmacs so delightful to edit Python code. I just completed migrating all open bugs and patches which mention python-mode from the python project to the new python-mode project. In the future, please submit all bugs and patches at http://sourceforge.net/projects/python-mode/ All new bugs and patches regarding python-mode should be submitted to the new project. There's also a mailing list, python-mode@python.org, to which you can subscribe in the usual Pythonic fashion: http://mail.python.org/mailman/listinfo/python-mode Skip ------------------------------ Message: 4 Date: Tue, 5 Aug 2003 02:58:32 -0400 From: "logistix" Subject: RE: [Python-Dev] Re: Re: pirate (python+parrot) To: "'A.M. Kuchling'" , Message-ID: <000001c35b1e$fc938a60$20bba8c0@XP> Content-Type: text/plain; charset="us-ascii" [guido@python.net] > wrote: > > Sure, but bytecode evolves slowly enough to be useful for Dan; given > > that the only parse tree format we have is *intricately* linked with > > the Python parser and (at least part of) its runtime, > [AMK] > It is? Both translators are using the compiler package, not > the parser > module; can it be said to be intricately linked with the runtime? > Admittedly it uses the parser module underneath, but the AST > representation > itself is pretty generic, isn't it? > As Michal touched on earlier, I wrote a hand-coded pure-python parser that plugs into the compiler package. It takes about 1.5-2x as long to compile vs using the native parser module. The resultant tuples can also be shoved back into genuine parse trees via parser.sequence2ast(). So yeah, it's not that big of a deal to generate AST trees without the runtime (until some crazy function attribute syntax gets added to python, that is) http://www.cathoderaymission.net/~logistix/python/pparser.py -Grant ------------------------------ Message: 5 Date: Tue, 05 Aug 2003 11:22:09 +0100 From: Michael Hudson Subject: Re: [Python-Dev] Re: pirate (python+parrot) To: python-dev@python.org Message-ID: <2mllu8qu9a.fsf@starship.python.net> Content-Type: text/plain; charset=us-ascii Guido van Rossum writes: > Interesting indeed. The focus on bytecode is understandable: let > someone else write a parser, so Parrot won't have to. But from > skimming his log, Dan seems to take the bytecode too seriously; > it's really only meant to be a quick way for the parser to output > something. In particular, bytecode hacks were never a goal. Really? You surprise me . Cheers, mwh -- 3. Syntactic sugar causes cancer of the semicolon. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html ------------------------------ Message: 6 Date: Tue, 5 Aug 2003 12:47:47 +0200 From: Jack Jansen Subject: Re: [Python-Dev] Python 2.3 maintenance branch is now open To: Barry Warsaw Cc: LIST Python Developers Message-ID: <4053FFDF-C732-11D7-BB10-0030655234CE@cwi.nl> Content-Type: text/plain; charset=US-ASCII; format=flowed On Tuesday, Aug 5, 2003, at 00:58 Europe/Amsterdam, Barry Warsaw wrote: > Jack turned over the release23-branch branch to me so I was able to > create the Python 2.3 maintenance branch. Here's what I did: > > - checked out release23-fork > - cvs tag -b release23-maint > - cvs up -rrelease23-maint > - cvs up -j release23-branch > - cvs commit Barrt, how about porting things back to the trunk? Is that my responsibility for the stuff I changed between 2.3 and 2.3-mac? -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman ------------------------------ Message: 7 Date: Tue, 5 Aug 2003 04:30:28 -0700 From: Michael Chermside Subject: [Python-Dev] RE: python-dev Summary for 2003-07-01 through 2003-07-31 [draft] To: python-dev@python.org Message-ID: <1060083028.3f2f95543a9a2@mcherm.com> Content-Type: text/plain; charset=ISO-8859-1 Brett writes: > I would appreciate not just correctional feedback but just overall > feedback as well for this summary. I have changed the format once again > as well as the general philosophy behind the Summaries Brett, you're doing great as usual. Saving enough time to survive grad school may have been your primary motivation, but I actually think that this revised format is better than your exhaustive one. IMHO people who wanted to know every detail that went on would just subscribe to python-dev, and people who wanted historical accuracy would just consult the list archives. Your summaries made these tasks easier, but could never be as exhaustive as just reading the list. However, the "more journalistic" approach is really nice because it helps to separate out the relevent from the insignifigant. It is a very useful filtering of the information, providing something you CAN'T get by reading the list. Of course, it's *your* idea of what's important and what's not, but at this point I'd say you're as qualified as any to make that call, and more qualified than almost all. Keep up the great work! -- Michael Chermside ------------------------------ Message: 8 Date: Tue, 05 Aug 2003 12:31:08 +0100 From: Michael Hudson Subject: Re: [Python-Dev] Python 2.3 maintenance branch is now open To: python-dev@python.org Message-ID: <2m8yq8qr2b.fsf@starship.python.net> Content-Type: text/plain; charset=us-ascii Barry Warsaw writes: > Jack turned over the release23-branch branch to me so I was able to > create the Python 2.3 maintenance branch. Here's what I did: > > - checked out release23-fork > - cvs tag -b release23-maint > - cvs up -rrelease23-maint > - cvs up -j release23-branch > - cvs commit > > IOW, I basically branched release23-maint off of the release23-fork tag > on the trunk, then I merged all the changes in the release23-branch to > the release23-maint. Here's a recipe I use for branches that others might find useful: $ cd ~/src/python/dist/ $ cvs co -d src-23x -r release23-maint python/dist/src (This creates -- eventually -- a src-23x directory alongside the src directory which contains the trunk) Then hack CVS/Entries to contain a line D/src-23x//// Now running cvs up in python/dist/ will update both the trunk and the release23-maint code (and in my case release22-maint, too). Cheers, mwh -- Just point your web browser at http://www.python.org/search/ and look for "program", "doesn't", "work", or "my". Whenever you find someone else whose program didn't work, don't do what they did. Repeat as needed. -- Tim Peters, on python-help, 16 Jun 1998 ------------------------------ Message: 9 Date: 05 Aug 2003 08:44:58 -0400 From: Barry Warsaw Subject: Re: [Python-Dev] Python 2.3 maintenance branch is now open To: Jack Jansen Cc: LIST Python Developers Message-ID: <1060087498.8108.2.camel@anthem> Content-Type: text/plain On Tue, 2003-08-05 at 06:47, Jack Jansen wrote: > Barrt, > how about porting things back to the trunk? Is that my responsibility > for the stuff I changed between 2.3 and 2.3-mac? I wasn't planning on porting things back to the trunk, so it would be great if you could do it. -Barry ------------------------------ Message: 10 Date: Tue, 05 Aug 2003 11:54:50 -0400 From: kbk@shore.net (Kurt B. Kaiser) Subject: [Python-Dev] IDLE NEWS and Version To: python-dev@python.org Message-ID: Content-Type: text/plain; charset=us-ascii IDLE has historically maintained its own NEWS file, and it is now accessible from IDLE's Help/About menu. Maintaining this file has the advantage that all the IDLE NEWS is kept in one place and is easily parsable for display. Do we want to continue this tradition or merge IDLE NEWS into Python NEWS on a go-forward basis? Second, do we want to continue the IDLE versioning (currently 1.0) or just drop it now that IDLE is no longer distributed separately? -- KBK ------------------------------ _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev End of Python-Dev Digest, Vol 1, Issue 3435 ******************************************* From barry at python.org Tue Aug 5 17:53:13 2003 From: barry at python.org (Barry Warsaw) Date: Tue Aug 5 12:53:15 2003 Subject: [Python-Dev] IDLE NEWS and Version In-Reply-To: References: Message-ID: <1060102360.21919.65.camel@yyz> On Tue, 2003-08-05 at 11:54, Kurt B. Kaiser wrote: > IDLE has historically maintained its own NEWS file, and it is now > accessible from IDLE's Help/About menu. Maintaining this file has the > advantage that all the IDLE NEWS is kept in one place and is easily > parsable for display. > > Do we want to continue this tradition or merge IDLE NEWS into Python > NEWS on a go-forward basis? Whatever you want to do. I see no problem in having a separate IDLE news file, but adding entries in Python's main news file for the Really Big Stuff. > Second, do we want to continue the IDLE versioning (currently 1.0) > or just drop it now that IDLE is no longer distributed separately? I think IDLE should continue to have its own versioning scheme. -Barry From gjc at inescporto.pt Tue Aug 5 18:05:26 2003 From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro) Date: Tue Aug 5 13:05:27 2003 Subject: [Python-Dev] shared library Message-ID: <1060102840.24404.7.camel@spectrum.inescn.pt> Without meaning to criticize, is there any particular reason why the python shared library isn't enabled by default? -- Gustavo Jo?o Alves Marques Carneiro From bac at OCF.Berkeley.EDU Tue Aug 5 12:41:35 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Aug 5 14:41:51 2003 Subject: [Python-Dev] Re-introduce caching for _strptime.py for 2.3.1? Message-ID: <3F2FFA5F.8010609@ocf.berkeley.edu> Bug #780807 is a complaint that strptime is now 1,200 times slower than a C library version the user used to have. Now slowdown is going to happen since Python string code can't compete with C string code, let alone _strptime has to figure out the locale info it needs while a C version has direct access. And I am sure people who now have strptime on their platform are not going to complain about performance. =) But 1,200 times is a little high. The new version I checked into HEAD is supposedly only 3 times slower than the equivalant C version according to the bug report submitter. This is mainly because of caching. My question is whether I should backport any of this. The new version of _strptime is thread-safe and has caching (2.3 is already thread-safe thanks to the loss of caching for that version). If I were to re-introduce caching (which was in 2.3 until a day before 2.3.0c2) I would need to also tweak other code to keep it thread-safe. In other words I would have to do more work than just throw back in the caching code with the addition of a thread lock. Is this considered a bugfix? The only reason I question whether it is since it is not a pure "bugfix" is because I know at least Raymond thought the code should have gone back in before 2.3.0 final went out. Had I not been so panicked about fixing that one bug the caching code would still be in there and I would be patching 2.3.1 to make it thread-safe instead of sending out this email. -Brett From martin at v.loewis.de Tue Aug 5 20:26:39 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Tue Aug 5 15:26:41 2003 Subject: [Python-Dev] shared library In-Reply-To: <1060102840.24404.7.camel@spectrum.inescn.pt> References: <1060102840.24404.7.camel@spectrum.inescn.pt> Message-ID: "Gustavo J. A. M. Carneiro" writes: > Without meaning to criticize, is there any particular reason why > the python shared library isn't enabled by default? Because shared libraries suck, and the world would be a better place without them. I can elaborate if you want to :-) Regards, Martin From martin at v.loewis.de Tue Aug 5 20:38:26 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Tue Aug 5 15:38:27 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) Message-ID: On Win64, a number of tests fail as they expect that a Python integer can represent a platform pointer. For example, test_array expects that buffer_info returns an integer. Likewise, test_descr expects that id() and hash() return the same value by default. Is that a bug in the Win64 port, or in the tests? Regards, Martin From tim.one at comcast.net Tue Aug 5 16:58:00 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 5 15:58:37 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: Message-ID: [Martin v. Lowis] > On Win64, a number of tests fail as they expect that a Python integer > can represent a platform pointer. If by "Python integer" we mean the union of int and long, that's a good assumption. > For example, test_array expects that buffer_info returns an integer. That can't be right -- buffer_info returns a tuple on all platforms. The first element of the tuple is constructed via PyTuple_SET_ITEM(retval, 0, PyLong_FromVoidPtr(self->ob_item)); and PyLong_FromVoidPtr() is supposed to be smart enough to return a Python long if sizeof(void*) > sizeof(long) (by way of the platform C "long long" type, which should be __int64 on any flavor of Windows). Ah! You must mean this line in test_buffer_info(): self.assert_(isinstance(bi[0], int)) That's clearly a bug in the test; it should read self.assert_(isinstance(bi[0], (int, long))) > Likewise, test_descr expects that id() and hash() return the same value by > default. Sorry, I couldn't follow that one. Like, the id() and hash() of what? Certainly nobody expects, e.g., that hash("xyz") == id("xyz"). > Is that a bug in the Win64 port, or in the tests? I don't understand what problem(s) you're seeing -- showing tracebacks is always more useful than trying to paraphrase in English. Trent Mick did the Win64 port, and I believe all tests passed at the time he finished that. 'Twas quite a while ago, though, and I don't of anyone running tests on Win64 since then. Still, because they used to pass, I expect any problems that may exist now are shallow. From pedronis at bluewin.ch Tue Aug 5 23:06:27 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue Aug 5 16:05:10 2003 Subject: hash defaulting to id and id (was: [Python-Dev] sizeof(long) != sizeof(void*)) In-Reply-To: Message-ID: <5.2.1.1.0.20030805215752.02686778@pop.bluewin.ch> At 21:38 05.08.2003 +0200, Martin v. L?wis wrote: >On Win64, a number of tests fail as they expect that a Python integer >can represent a platform pointer. For example, test_array expects that >buffer_info returns an integer. Likewise, test_descr expects that id() >and hash() return the same value by default. hash() returning the same value as id() as fallback default is not a reasonable assumption. I'm about to fix the long standing broken id in Jython and keeping the default hash() equal to id() is a too expensive option. I even think that in the long run id() should be depracated in favor of offering identity mappings and weak identity mappings. id() is not reasonably implementable across the range of GC implementations. regards. From pedronis at bluewin.ch Tue Aug 5 23:16:52 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue Aug 5 16:15:33 2003 Subject: hash defaulting to id and id (was: [Python-Dev] sizeof(long) != sizeof(void*)) In-Reply-To: <5.2.1.1.0.20030805215752.02686778@pop.bluewin.ch> References: Message-ID: <5.2.1.1.0.20030805221554.00b05b60@pop.bluewin.ch> At 22:06 05.08.2003 +0200, Samuele Pedroni wrote: >At 21:38 05.08.2003 +0200, Martin v. L?wis wrote: >>On Win64, a number of tests fail as they expect that a Python integer >>can represent a platform pointer. For example, test_array expects that >>buffer_info returns an integer. Likewise, test_descr expects that id() >>and hash() return the same value by default. > >hash() returning the same value as id() as fallback default i.e. object.__hash__(x) == id(x) >is not a reasonable assumption. I'm about to fix the long standing broken >id in Jython and keeping the default hash() equal to id() is a too >expensive option. > >I even think that in the long run id() should be depracated in favor of >offering identity mappings and weak identity mappings. id() is not >reasonably implementable across the range of GC implementations. > >regards. > > > > > > >_______________________________________________ >Python-Dev mailing list >Python-Dev@python.org >http://mail.python.org/mailman/listinfo/python-dev From pinard at iro.umontreal.ca Tue Aug 5 21:27:45 2003 From: pinard at iro.umontreal.ca (Francois Pinard) Date: Tue Aug 5 16:27:46 2003 Subject: [Python-Dev] Re: shared library In-Reply-To: References: <1060102840.24404.7.camel@spectrum.inescn.pt> Message-ID: [Martin v. L?wis] > > Without meaning to criticize, is there any particular reason why > > the python shared library isn't enabled by default? > Because shared libraries suck, and the world would be a better place > without them. I can elaborate if you want to :-) No doubt that shared libraries have their own complexity, but I'm still not sure the world (or at least the Linux machines I work on!) would be a better place without them. I would surely need more disks and more memory to run everything, and probably more time to load the slightest program. But I might be missing the meaning of the original question. If by any chance you feel like elaborating a bit, that could be instructive for me. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From patmiller at llnl.gov Tue Aug 5 14:42:07 2003 From: patmiller at llnl.gov (Pat Miller) Date: Tue Aug 5 16:42:53 2003 Subject: [Python-Dev] shared library References: <1060102840.24404.7.camel@spectrum.inescn.pt> Message-ID: <3F30169F.1090201@llnl.gov> > "Gustavo J. A. M. Carneiro" writes: >> Without meaning to criticize, is there any particular reason why >> the python shared library isn't enabled by default? Martin pontificates: > Because shared libraries suck, and the world would be a better place > without them. > > I can elaborate if you want to :-) I agree with Martin that they can be problematic, but amongst our large Python community here, I've had several requests for exactly that. Typical use is something that embeds (or optionally embeds) Python. I understand there are portability issues with generating, but on platforms where they work well (e.g Linux, WinXX) why not? Pat -- Patrick Miller | (925) 423-0309 | http://www.llnl.gov/CASC/people/pmiller If it dies, it's biology. If it blows up, it's chemistry, and if it doesn't work, it's physics. From martin at v.loewis.de Tue Aug 5 21:46:25 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Tue Aug 5 16:46:26 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: References: Message-ID: "Tim Peters" writes: > Ah! You must mean this line in test_buffer_info(): > > self.assert_(isinstance(bi[0], int)) That's what I meant, indeed. > > Likewise, test_descr expects that id() and hash() return the same value by > > default. > > Sorry, I couldn't follow that one. Like, the id() and hash() of what? > Certainly nobody expects, e.g., that hash("xyz") == id("xyz"). I mean this: # Test the default behavior for static classes class C(object): def __getitem__(self, i): if 0 <= i < 10: return i raise IndexError c1 = C() vereq(hash(c1), id(c1)) > > > Is that a bug in the Win64 port, or in the tests? > > I don't understand what problem(s) you're seeing -- showing tracebacks is > always more useful than trying to paraphrase in English. Certainly, yes. Unfortunately, I don't have access to my Email account at the same computer I have access to a Win64 machine, so I thought paraphrasing might be sufficient - and indeed, in one case, you were guessing right. The third case is one of repr() failing, where it puts a "foo object at ABCDEFL" in one place, and "foo object at abcdef" in the other, for the test case i3 = ClassWithFailingRepr() eq(r(i3), (""%id(i3))) > Trent Mick did the Win64 port, and I believe all tests passed at the > time he finished that. 'Twas quite a while ago, though, and I don't > of anyone running tests on Win64 since then. Still, because they > used to pass, I expect any problems that may exist now are shallow. I feel this is deeper: Should there be a guarantee that the type used to represent is large enough to hold a void*, or not? You seem to think that Python should make no such guarantee, which would then indicate that three of the four failing tests are broken (I'm uncertain about test_queue at the moment). Regards, Martin From martin at v.loewis.de Tue Aug 5 21:56:24 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Tue Aug 5 16:56:25 2003 Subject: [Python-Dev] Re: shared library In-Reply-To: References: <1060102840.24404.7.camel@spectrum.inescn.pt> Message-ID: Francois Pinard writes: > But I might be missing the meaning of the original question. If by any > chance you feel like elaborating a bit, that could be instructive for me. The question is whether the code containing the Python implementation should be a shared library, i.e. libpythonxy.dll, which would give a very small stub python executable. I think this causes more bad than good because: 1. If there is a single /python executable, there is just about the same amount of sharing. Virtually all applications using Python use the single executable, and they all share that single executable, both on disk and in memory. The only significant exception might be mod_python, which currently needs to link its own copy of the Python implementation. The various run-time copies of mod_python (across different Apache processes) still get shared. 2. Having libpythonxy.so causes significant maintenance problems. In some installations, Python might fail to find libpython after installation, because /lib is not in the search path of the shared linker, or ldconfig might need to be run before it is found. 3. Having libpythonxy.so increases the start-up time, as an additional directory is added to the ld.so search path, which needs to be searched for any shared library loaded by Python, including all C extension modules which are not builtin. 4. Having libpythonxy.so decreases the run-time efficiency, as now all code needs to be compiled as PIC. 5. Having libpythonxy.so creates a maintenance cost, as, for any implementation of that feature, it fails on some system because of obscure compiler/linker/dynamic linker aspects that haven't been considered. I expect several years to pass until no new bug reports about that feature come in. So it has only disadvantages and no advantages, and is thus disabled by default. It is only implemented to decrease the maintenance burden of having to respond to people asking for the feature. Regards, Martin From tim.one at comcast.net Tue Aug 5 18:05:00 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 5 17:05:38 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: Message-ID: [Martin] > ... > I feel this is deeper: Should there be a guarantee that the type used > to represent is large enough to hold a void*, or not? The confusion over what you mean still persists for me. In the Python I use , there is no type that identifies itself as . There's a type that identifies itself as , and another as . When the Language Reference Manual uses the word "integer", it generally means the union of those two. It's certainly not intended that objects of can always hold a casted void*. It certainly is intended that objects of can always hold a casted void*. So it's intended that a Python integer be able to hold a casted void*, but "integer" doesn't take a stand on whether the concrete type is Python int or Python long. After int/long unification is complete, the distinction will be even more a case of academic irrelevance. > You seem to think that Python should make no such guarantee, Definitely yes, or definitely no, depending on what you mean by "integer". I explained what I mean above. > which would then indicate that three of the four failing tests are broken > (I'm uncertain about test_queue at the moment). It's easy to believe that tests have broken, since (a) AFAIK nobody runs the tests on Win64, and (b) Win64 may well be the only platform around where sizeof(void*) > sizeof(long) in C (meaning that a casted void* can't fit in a Python int -- although it can fit in a Python integer, and PyLong_FromVoidPtr already knows how to deal with this). From martin at v.loewis.de Tue Aug 5 23:07:36 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Tue Aug 5 18:07:37 2003 Subject: [Python-Dev] shared library In-Reply-To: <3F30169F.1090201@llnl.gov> References: <1060102840.24404.7.camel@spectrum.inescn.pt> <3F30169F.1090201@llnl.gov> Message-ID: Pat Miller writes: > I agree with Martin that they can be problematic, but amongst > our large Python community here, I've had several requests > for exactly that. Typical use is something that embeds > (or optionally embeds) Python. There is nothing wrong with requesting it; what stopped it for the last years that nobody was capable of implementing it correctly. Now, in Python 2.3, the feature is implemented. > I understand there are portability issues with generating, but on platforms > where they work well (e.g Linux, WinXX) why not? Why not what? Why not offer it as an option? It is an option in 2.3. Why not activate the option by default? Because most people won't need it, and because, if activated by default, it will cause problems to more people than people actually needing the feature. Regards, Martin From martin at v.loewis.de Tue Aug 5 23:09:25 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Tue Aug 5 18:09:26 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: References: Message-ID: "Tim Peters" writes: > The confusion over what you mean still persists for me. In the Python I use > , there is no type that identifies itself as . > There's a type that identifies itself as , and another as 'long'>. When the Language Reference Manual uses the word "integer", it > generally means the union of those two. Ah, and I mean , when I say :-) Regards, Martin From pinard at iro.umontreal.ca Tue Aug 5 23:13:44 2003 From: pinard at iro.umontreal.ca (Francois Pinard) Date: Tue Aug 5 18:13:45 2003 Subject: [Python-Dev] Re: shared library In-Reply-To: References: <1060102840.24404.7.camel@spectrum.inescn.pt> Message-ID: [Martin v. L?wis] > Francois Pinard writes: > > But I might be missing the meaning of the original question. If by any > > chance you feel like elaborating a bit, that could be instructive for me. > The question is whether the code containing the Python implementation > should be a shared library, i.e. libpythonxy.dll, which would give a > very small stub python executable. Thanks for having replied. > So it has only disadvantages and no advantages, [...] I would not go that far. It might have a few advantages. A few times in the past, not so long ago, I compiled C main programs using Python-written modules and functions, and with Pyrex-generated interfacing code. I surely did this once or twice for getting a static debugging environment, but I also got actual, real, production use for such things. The usual "rewrite-everything-in-Python" does not always apply for legacy code. I could try to defend the "add-new-parts-in-Python" stanza, but not always the idea to rewrite what already exists -- this would takes years, as if Python makes faster writing and easier maintenance, it does not necessarily transform the actual problems into simple things. However, each such generated executable is significantly bulkier as it holds a copy of the whole interpreter, and the final linking calls is neither evident to write, nor to explain to other team members. So, a kind of effect is that using Python from C is not used as often as it might be. I guess that if it was more "innocuous", using Python modules together with C programs will happen more often, and might become more a natural after a while, than still an extraordinary thing. But it is easier to hypothesise than ascertain, the truth is that I'm not so sure. Last week, I had to recompile `vim' on many machines to enable Python support, and this doubles the size of the installed binary. Here again, one might hypothesise that the Linux packagers (SuSE in this case) would have more naturally provided a Python-enabled `vim' if its size was not bloating so evidently. Similar considerations might apply each time Python is considered as an extension languages for a system written in another language, who knows. The system has to be pretty big to start with, for Python-as-an-extension to appear as something else than an enterprise. > I think this causes more bad than good because: > 1. If there is a single /python executable, there is just > about the same amount of sharing. Of course, if Python is never used as an extension, it is much simpler and better to merely share the whole executable. On the other hand, this is also encouraging people to _not_ use Python as an extension. The snake is eating its tail, here. :-) > 2. Having libpythonxy.so causes significant maintenance problems. At least on Linux, with the number of shared libraries already around, I would presume that the technology has some maturity. On this system, it is likely that it would work painlessly. However, there are many platforms where things are not so controlled, and since Python development aims a lot of platform, the is a significant maintenance problem indeed. > 3. Having libpythonxy.so increases the start-up time, Surely. Yet, whenever Python starts nowadays, there is already oodles and tons of disk accesses, and I'm often surprised that it starts so speedily nevertheless. Machines got fast, and the memory cache of the system is very effective. In that sea of disk accesses, I wonder if the increase of start-up time would be that significant. Optimisation should probably address other areas before this one. > 4. Having libpythonxy.so decreases the run-time efficiency, as now all > code needs to be compiled as PIC. This might account for a few percents of CPU, according to what I read (I did not benchmark those things myself). I've seen extra-ordinary efforts on python-dev for getting Python more speedy, so your argument may be a strong one for many people. No doubt that I like speed, but I would probably not crusade for a few percents. This was lengthily and hotly discussed when ELF was first introduced, and it seems that the overall consensus which emerged is that the community is ready to drop a few percents for the gains brought by shared libraries. Of course, now that many softwares accepted to move and resultantly are a bit slower, Python might gain a few points by standing still. :-) > 5. Having libpythonxy.so creates a maintenance cost, [...] This is more or less a repeat of point 2., and there is a maintenance cost, no doubt. This might kill the idea more than anything else, in my opinion. Would it be a reasonable compromise if Python was compiled twice? Once non-PIC for the usual static installation, and a another one for preparing a shared library? Both would be installed. The `python' stand-alone interpreter would continue to use static linking, and its executable would be shared among all currently running Python scripts. The library would be available for writing extensions of all kinds. System packagers would be much encouraged to provide both whenever it works for their system. Linux packagers among them, of course. Maybe that for the usual "configure-without-options" way of installing that most people use themselves, the status quo would apply for at least a long good while. -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From martin at v.loewis.de Wed Aug 6 01:37:21 2003 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Tue Aug 5 18:37:49 2003 Subject: [Python-Dev] Re: shared library In-Reply-To: References: <1060102840.24404.7.camel@spectrum.inescn.pt> Message-ID: <3F3031A1.6030305@v.loewis.de> Francois Pinard wrote: > Would it be a reasonable compromise if Python was compiled twice? Indeed, this is what Debian does, and anybody producing Python binary distributions might want to do. > System packagers would be much encouraged to provide both whenever it works > for their system. System packagers can do so today if they want to. I don't think they *need* anything beyond that: They would not even have needed the standard option to build a shared libpython - Debian has managed to produce a shared libpython for years without support for that feature inside Python itself. If packagers haven't done so in the past, I can only conclude that users haven't requested that feature from their system vendors. Regards, Martin From python at rcn.com Tue Aug 5 19:56:35 2003 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 5 19:00:28 2003 Subject: [Python-Dev] Re-introduce caching for _strptime.py for 2.3.1? References: <3F2FFA5F.8010609@ocf.berkeley.edu> Message-ID: <006e01c35ba4$d2898d60$0e27a044@oemcomputer> > Bug #780807 is a complaint that strptime is now 1,200 times slower than > a C library version the user used to have. Now slowdown is going to > happen since Python string code can't compete with C string code, let > alone _strptime has to figure out the locale info it needs while a C > version has direct access. And I am sure people who now have strptime > on their platform are not going to complain about performance. =) > > But 1,200 times is a little high. The new version I checked into HEAD > is supposedly only 3 times slower than the equivalant C version > according to the bug report submitter. This is mainly because of caching. > > My question is whether I should backport any of this. The new version > of _strptime is thread-safe and has caching (2.3 is already thread-safe > thanks to the loss of caching for that version). If I were to > re-introduce caching (which was in 2.3 until a day before 2.3.0c2) I > would need to also tweak other code to keep it thread-safe. In other > words I would have to do more work than just throw back in the caching > code with the addition of a thread lock. > > Is this considered a bugfix? The only reason I question whether it is > since it is not a pure "bugfix" is because I know at least Raymond > thought the code should have gone back in before 2.3.0 final went out. > Had I not been so panicked about fixing that one bug the caching code > would still be in there and I would be patching 2.3.1 to make it > thread-safe instead of sending out this email. I'm strongly in favor of re-introducing this code: * it is the way you designed it in the first place * it makes a huge performance difference * the code was tested with it and when through beta and rc1 without a problem (there was an unrelated bug elsewhere). * I'm more confident in the code before it was "ripped out" at the last minute. * There is no compelling reason to have to wait another 18 months to have the code behave reasonably * the is no real impact on the API as released I'm +0 on tweaking it further to make it thread-safe. IMO that is a separate issue (possibly important but having no bearing on whether caching should be included). Raymond Hettinger ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From tim.one at comcast.net Tue Aug 5 22:04:50 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 5 21:05:24 2003 Subject: [Python-Dev] Re-introduce caching for _strptime.py for 2.3.1? In-Reply-To: <3F2FFA5F.8010609@ocf.berkeley.edu> Message-ID: [Brett, on strptime changes] > ... > My question is whether I should backport any of this. I'm with Raymond: yes. Ripping out the cache for 2.3 final was a time-pressure bugfix that wouldn't have been needed had you had another day to repair it correctly. I'm not so keen on adding complications beyond that for "thread safety", though, as locale is inherently thread-insane in the absence of a user-implemented locking protocol (but since most programs will at worst change locale once at the start, worry about threads is mostly wasted regardless). From tim.one at comcast.net Tue Aug 5 22:19:41 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 5 21:20:15 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: Message-ID: [Martin] > Ah, and I mean , when I say :-) In that case, no, Python has no intent of claiming that a Python int can hold a machine address. It does intend that a machine address can be held a Python integer, but doesn't promise that such an integer won't be a Python long. So the array buffer_info test is making a bogus assumption (and the intended way to repair that was given before). The other test insisting that id() == hash() for an instance of a user-defined class that doesn't define __hash__ is also making a bogus assumption, since hash() results are specifically Python ints today while id() results are Python integers (ints or longs). I'm not sure what that test was *trying* to check; the test looked pretty useless to me. I expect that both these tests were introduced after Trent finished the Win64 port (test_descr.py certainly came after that port), so simply haven't been run on Win64 before. If Win64 has a future, I suppose it would be good to repair this stuff; but I don't expect any other platform in my lifetime to decide to make sizeof(void*) > sizeof(long), so it's tempting to take the lack of previous reports of Win64 test failures as evidence about that platform's viability. From bac at OCF.Berkeley.EDU Tue Aug 5 20:59:43 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Tue Aug 5 22:59:46 2003 Subject: [Python-Dev] Re-introduce caching for _strptime.py for 2.3.1? In-Reply-To: References: Message-ID: <3F306F1F.2010403@ocf.berkeley.edu> Tim Peters wrote: > [Brett, on strptime changes] > >>... >>My question is whether I should backport any of this. > > > I'm with Raymond: yes. Ripping out the cache for 2.3 final was a > time-pressure bugfix that wouldn't have been needed had you had another day > to repair it correctly. I'm not so keen on adding complications beyond that > for "thread safety", though, as locale is inherently thread-insane in the > absence of a user-implemented locking protocol (but since most programs will > at worst change locale once at the start, worry about threads is mostly > wasted regardless). > OK, having two people whose opinions I trust say I should add it back in I will. What I will do is diff back to the last version that had caching, re-implement the fix for the bug (it was like four lines of code that are brain-dead simple) and then apply that to the 2.3.x branch. I won't make any changes for the sake of thread-safety to LocaleTime and such. There is a possibility of some weird race condition since I cache an instance of TimeRE and a dict of compiled regexes that is dependent on TimeRE, but I am having a hard time coming up with an issue outside of locale funkiness so I won't worry about it with 2.3.x (dealt with in 2.4 as a safety precaution). -Brett From Jack.Jansen at cwi.nl Wed Aug 6 11:58:04 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Wed Aug 6 04:55:49 2003 Subject: [Python-Dev] Re: shared library In-Reply-To: Message-ID: <175D4FD5-C7EC-11D7-B696-0030655234CE@cwi.nl> On Wednesday, Aug 6, 2003, at 00:12 Europe/Amsterdam, Francois Pinard wrote: > Last week, I had to recompile `vim' on many machines to enable Python > support, and this doubles the size of the installed binary. Here > again, > one might hypothesise that the Linux packagers (SuSE in this case) > would > have more naturally provided a Python-enabled `vim' if its size was not > bloating so evidently. > > Similar considerations might apply each time Python is considered as an > extension languages for a system written in another language, who > knows. > The system has to be pretty big to start with, for > Python-as-an-extension > to appear as something else than an enterprise. This is the main point, I think. By not including the shared library by default people looking for an extension language may decide to look elsewhere. MacPython has always had a shared library architecture on both MacOS9 and MacOSX (I think we can consider frameworks shared libraries for this discussion). Actually, the fact that Apple didn't ship the shared library with the Python 2.2 they included with MacOSX 10.2 caused headaches: there's funky workarounds with Python scripts for the fact that applets can't be linked against the python shared libraries. And mixing ObjC and Python is pretty much impossible. This is all solved for 2.3, through the framework build. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From mhammond at skippinet.com.au Wed Aug 6 21:43:17 2003 From: mhammond at skippinet.com.au (Mark Hammond) Date: Wed Aug 6 06:43:40 2003 Subject: [Python-Dev] Re: shared library In-Reply-To: <175D4FD5-C7EC-11D7-B696-0030655234CE@cwi.nl> Message-ID: <009001c35c07$8c7992d0$f502a8c0@eden> [Jack] > This is the main point, I think. By not including the shared > library by defaultpeople looking for an extension language > may decide to look elsewhere. Not to mention the fact that Python on Windows has (effectively) never supported static linking, and complaints have been scant. When first moving the Python/Mozilla XPCOM binaries to Linux, I was flabbergasted to find dynamic linking was considered an esoteric feature. This certainly has been a significant barrier to getting PyXPCOM working in a reasonable way for standard distributions. My next significant paid job also looks like being a Linux based embedding job where dynamic linking is going to be a pre-requisite. Mark. From pinard at iro.umontreal.ca Wed Aug 6 13:46:00 2003 From: pinard at iro.umontreal.ca (Francois Pinard) Date: Wed Aug 6 08:46:01 2003 Subject: [Python-Dev] Re: shared library In-Reply-To: <009001c35c07$8c7992d0$f502a8c0@eden> References: <009001c35c07$8c7992d0$f502a8c0@eden> Message-ID: [Mark Hammond] > When first moving the Python/Mozilla XPCOM binaries to Linux, I was > flabbergasted to find dynamic linking was considered an esoteric feature. Indeed. Here is something which counter-weights these arguments, a bit, in favour of Martin's original statement. Christos Georgiou kindly communicated to me this pointer for an experiment he did: http://groups.google.com/groups?selm=k7clgvoqjur1f10mosue3os74vv7nd34hi%404ax.com Christos observed that using PIC may trigger a 25% slowdown. This looks to me like more than "a few" percent points. Hey, there are good arguments in either direction, it's difficult sustaining a strong opinion. Sigh! :-) -- Fran?ois Pinard http://www.iro.umontreal.ca/~pinard From mwh at python.net Wed Aug 6 14:54:19 2003 From: mwh at python.net (Michael Hudson) Date: Wed Aug 6 08:54:54 2003 Subject: [Python-Dev] CALL_ATTR patch (was: 2.3b1 release) In-Reply-To: <200304180022.h3I0Mu012443@pcp02138704pcs.reston01.va.comcast.net> Message-ID: <17E59550-C80D-11D7-978B-0003931DF95C@python.net> On Friday, Apr 18, 2003, at 01:22 Europe/London, Guido van Rossum wrote: Note that this is quite some time ago :-) I've been reading through some messages that have been ticked for rather too long in my python-dev mail folder... >> (Looking at PyObject_GenericGetAttr with that in mind, I wonder if >> there isn't a possible crash there. In the first MRO lookup, looking >> for descr's, if a non-data-descr is found, it is kept around but not >> INCREF'd until later, after the instance-dict is searched. Am I >> wrong in believing the PyDict_GetItem of the instance dict can call >> Python code ? > > It can, if there's a key whose type has a custom __eq__ or __cmp__. > So indeed, if this custom __eq__ is evil enough to delete the > corresponding key from the class dict, it could cause descr to point > to freed memory. I won't try to construct a case, but it's not > impossible. :-( Indeed not! Here's mine: import gc class Evil(object): def __hash__(self): return hash('attr') def __eq__(self, other): global foo foo = gc.get_referrers(C.attr) del C.attr return 0 class Descr(object): def __get__(self, ob, type=None): return 1 class C(object): attr = Descr() c = C() c.__dict__[Evil()] = 0 c.attr `foo` It's always a bit hard to be sure that stuff like this will actually crash the interpreter, which is what the gimmicks with gc.get_referrers & `foo` are in aid of. This crashes every Python I have lying around on my iBook -- the 2.2[.0] in /usr/bin, heads of release22-maint, release23-maint, the CVS trunk, a random build of 2.2.3c1, Jack's 2.3 build. Etc ;-) I'll check on linux when I get into work. > Fixing this would make the code even hairier though... :-( Pff, it's not so bad, just a little refcount code jiggling. I'll make a real patch including the above test case soon (once I'm reasonably confident that I'm not leaking references). And then I find out running test_descr in a loop leaks 166(!) references *anyway* (run the attached blah.py in a debug build, and some print test or other that I don't understand fails when run like this, so I fiddled it) Should I have been expecting this, or is it time for some ref-leak hunting? Actually, the process seems to grow -- slowly -- when you loop, so it's probably a real problem. I gather it's best to ask about this sort of thing when Tim is off sick from work :-) Cheers, mwh -------------- next part -------------- A non-text attachment was scrubbed... Name: blah.py Type: application/octet-stream Size: 814 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20030806/4e83b45b/blah.obj From aahz at pythoncraft.com Wed Aug 6 13:04:41 2003 From: aahz at pythoncraft.com (Aahz) Date: Wed Aug 6 12:04:45 2003 Subject: [Python-Dev] FWD: Re: Problem w/ IDLE on Win2000 Message-ID: <20030806160441.GA29620@panix.com> Is there an SF bug for this? Should I add it to bugs.html? ----- Forwarded message from deelan ----- > From: deelan > Newsgroups: comp.lang.python > Subject: Re: Problem w/ IDLE on Win2000 > Date: Wed, 06 Aug 2003 09:53:15 +0200 > Organization: IUnet > > David Lees wrote: > (...) > >I believe the problem is the same one previously discussed with respect > >to Windows XP. IDLE can not be in a path where any of the directory > >names have a space. Unfortunately, 'program files' has a space in it. > some problem on my win2k server sp4. installed py2.3 into "program > files", IDLE failed to start. reinstalled 2.3 into "c:\python", problem > fixed. > > bye > -- > goldicus1970 AT yahoo.it > http://www.deelan.com > > > > > > ----- End forwarded message ----- -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From oussoren at cistron.nl Wed Aug 6 21:09:17 2003 From: oussoren at cistron.nl (Ronald Oussoren) Date: Wed Aug 6 14:09:23 2003 Subject: [Python-Dev] Multiple levels of site-packages Message-ID: <18163A21-C839-11D7-9FDF-0003931CFE24@cistron.nl> Hi, This issue first surfaced on the MacOS SIG list, but seems to be a more cross-platform issue. The Python distribution, and distutils, knows of two types of packages/modules: those in the core distribution and the site-packages. This is fine when Python is locally installed anyway, but now that Python is distributed as part of the OS on some systems (e.g. Linux and MacOS X) there seem to be more levels at which you can install packages: between the core python packages and locally installed site-packages are packages installed by the system vendor. As Jack noticed on the MacOS Sig list there are actually even more levels of packages: 1) part of the python core 2) part of the system vendor packages 3) locally installed network-wide packages 4) locally installed system-wide packages 5) locally installed user packages Items 2 to 5 are currently lumped together in site-packages. IMO it would be usefull to define a mechanism to define these levels of package collections, and to teach distutils about them. This would make it easier for system vendors to keep the packages provided by the system seperate from locally installed packages. A related issue is that the site-packages is usually stored in the same part of the OS tree as the python executable, e.g. /usr/lib/python2.3 if the executable is in /usr/bin. This means that local python packages will be installed into "os vendor space" when python is part of the system software. I think it would be prudent to define a policy on how system vendors should deal with this. The Apple distribution of Python 2.3 in Panther solves this problem by moving site-packages to "system admin" space using a symlink. This is a less then perfect option because some Apple provided python packages are in the site-packages directory, and therefore in "system admin" space. Is this really a problem worth writing a PEP for, or am I seeing ghosts? Ronald P.S. As this is my first post to python-dev, I'd better introduce myself :-). My claim to fame is PyObjC, the bridge between Python and Cocoa on MacOS X. I've been a (happy) Python user from about release 1.2 (mostly on various Unix flavors and returning to the Mac with MacOS X) From guido at python.net Wed Aug 6 15:15:08 2003 From: guido at python.net (Guido van Rossum) Date: Wed Aug 6 14:15:11 2003 Subject: [Python-Dev] FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: <20030806160441.GA29620@panix.com>; from aahz@pythoncraft.com on Wed, Aug 06, 2003 at 12:04:41PM -0400 References: <20030806160441.GA29620@panix.com> Message-ID: <20030806141508.C32531@starship.python.net> On Wed, Aug 06, 2003 at 12:04:41PM -0400, Aahz wrote: > Is there an SF bug for this? Should I add it to bugs.html? What happened to your position on top-posting? :-) (please don't reply!) The default path is \python23 for a reason. Why do people ignore the default? I wouldn't call it a bug, at most a caveat. --Guido > ----- Forwarded message from deelan ----- > > From: deelan > > Newsgroups: comp.lang.python > > Subject: Re: Problem w/ IDLE on Win2000 > > Date: Wed, 06 Aug 2003 09:53:15 +0200 > > Organization: IUnet > > > > David Lees wrote: > > (...) > > >I believe the problem is the same one previously discussed with respect > > >to Windows XP. IDLE can not be in a path where any of the directory > > >names have a space. Unfortunately, 'program files' has a space in it. > > some problem on my win2k server sp4. installed py2.3 into "program > > files", IDLE failed to start. reinstalled 2.3 into "c:\python", problem > > fixed. From barry at python.org Wed Aug 6 19:20:00 2003 From: barry at python.org (Barry Warsaw) Date: Wed Aug 6 14:20:01 2003 Subject: [Python-Dev] Multiple levels of site-packages In-Reply-To: <18163A21-C839-11D7-9FDF-0003931CFE24@cistron.nl> References: <18163A21-C839-11D7-9FDF-0003931CFE24@cistron.nl> Message-ID: <1060193964.23433.51.camel@yyz> On Wed, 2003-08-06 at 14:09, Ronald Oussoren wrote: > 1) part of the python core > 2) part of the system vendor packages > 3) locally installed network-wide packages > 4) locally installed system-wide packages > 5) locally installed user packages > > Items 2 to 5 are currently lumped together in site-packages. IMO it > would be usefull to define a mechanism to define these levels of > package collections, and to teach distutils about them. This would make > it easier for system vendors to keep the packages provided by the > system seperate from locally installed packages. +1. I actually argued for a separate #5 when we were originally discussing site-packages. Now, I think we have better use cases to back it up, and I for one would really like to see this. -Barry From guido at python.net Wed Aug 6 15:23:30 2003 From: guido at python.net (Guido van Rossum) Date: Wed Aug 6 14:23:42 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: ; from tim.one@comcast.net on Tue, Aug 05, 2003 at 09:19:41PM -0400 References: Message-ID: <20030806142330.A32662@starship.python.net> On Tue, Aug 05, 2003 at 09:19:41PM -0400, Tim Peters wrote: > [Martin] > > Ah, and I mean , when I say :-) > > In that case, no, Python has no intent of claiming that a Python int can > hold a machine address. It does intend that a machine address can be held a > Python integer, but doesn't promise that such an integer won't be a Python > long. So the array buffer_info test is making a bogus assumption (and the > intended way to repair that was given before). OK, but in the past (when I was younger :-) *I* thought that that was a safe claim, otherwise I wouldn't have implemented id() the way it was. In that past, the difference between Python int and long was much greater, since much of the C library code wouldn't deal with longs at all. Unfortunately MS broke the assumption that sizeof(void*) <= sizeof(long) with its decision to keep long at 32 bits on Win64, and there's nothing we can do about it now (except switching int to use a long long internally, an idea with which I have played a bit in theory but which probably would suck in practice for various stupid reasons). > The other test insisting that id() == hash() for an instance of a > user-defined class that doesn't define __hash__ is also making a bogus > assumption, since hash() results are specifically Python ints today while > id() results are Python integers (ints or longs). I'm not sure what that > test was *trying* to check; the test looked pretty useless to me. WIthout reading the code, agreed. > I expect that both these tests were introduced after Trent finished the > Win64 port (test_descr.py certainly came after that port), so simply haven't > been run on Win64 before. > > If Win64 has a future, I suppose it would be good to repair this stuff; but > I don't expect any other platform in my lifetime to decide to make > sizeof(void*) > sizeof(long), so it's tempting to take the lack of previous > reports of Win64 test failures as evidence about that platform's viability. An alternative might be to make the type of C integer used to implement a Python (short) int a configuration option, using a 64 bit int on Win32. This unfortunately would suffer from various suckage too, as convenience functions like PyInt_AsLong() are screwed and can fail in ways they could never fail before, likely confusing code that doesn't expect them to fail. --Guido van Rossum From martin at v.loewis.de Wed Aug 6 19:25:54 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Wed Aug 6 14:25:55 2003 Subject: [Python-Dev] FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: <20030806160441.GA29620@panix.com> References: <20030806160441.GA29620@panix.com> Message-ID: Aahz writes: > Is there an SF bug for this? Should I add it to bugs.html? [ 784183 ] IDLE does not start for 2.3 on windows. Martin From martin at v.loewis.de Wed Aug 6 19:38:16 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Wed Aug 6 14:38:18 2003 Subject: [Python-Dev] Multiple levels of site-packages In-Reply-To: <18163A21-C839-11D7-9FDF-0003931CFE24@cistron.nl> References: <18163A21-C839-11D7-9FDF-0003931CFE24@cistron.nl> Message-ID: Ronald Oussoren writes: > The Python distribution, and distutils, knows of two types of > packages/modules: those in the core distribution and the > site-packages. This is incorrect. The Python distribution knows about four types of packages: core distribution, site-packages, site-python, and PYTHONPATH. Regards, Martin From james.kew at btinternet.com Wed Aug 6 20:48:49 2003 From: james.kew at btinternet.com (James Kew) Date: Wed Aug 6 14:49:00 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 References: <20030806160441.GA29620@panix.com> <20030806141508.C32531@starship.python.net> Message-ID: "Guido van Rossum" wrote in message news:20030806141508.C32531@starship.python.net... > The default path is \python23 for a reason. Why do people > ignore the default? I wouldn't call it a bug, at most a caveat. IMHO, it's a bug if: a) the user is allowed to change the installation default, but b) the installation (or part of it) fails to work correctly if he does. I find the \pythonxx default more useful for what I do, but installing to the Program Files directory is pretty idiomatic for Windows users; why discourage them? James From tim.one at comcast.net Wed Aug 6 15:48:31 2003 From: tim.one at comcast.net (Tim Peters) Date: Wed Aug 6 14:49:05 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <20030806142330.A32662@starship.python.net> Message-ID: [Guido] > OK, but in the past (when I was younger :-) *I* thought > that that was a safe claim, otherwise I wouldn't have implemented > id() the way it was. In that past, the difference between Python int > and long was much greater, since much of the C library code wouldn't > deal with longs at all. Well, that's all in the past, and id() isn't implemented that way anymore; at this time it's just a fact that a Python int isn't big enough to hold a machine address on all boxes. > Unfortunately MS broke the assumption that sizeof(void*) <= > sizeof(long) with its decision to keep long at 32 bits on Win64, and > there's nothing we can do about it now (except switching int to use a > long long internally, an idea with which I have played a bit in > theory but which probably would suck in practice for various stupid > reasons). At a time when we're trying to eradicate the user-visible distinction between ints and longs, making ints bigger seems like a distraction too. > ... > An alternative might be to make the type of C integer used to > implement a Python (short) int a configuration option, using a 64 bit > int on Win32. > > This unfortunately would suffer from various suckage too, as > convenience functions like PyInt_AsLong() are screwed and can fail in > ways they could never fail before, likely confusing code that doesn't > expect them to fail. Martin is seeing a handful of failing tests on Win84, and so far it sounds like they're shallow failures and easily fixed. If we were to fiddle Python's C code, I'd be more keen to get rid of the assumption that containers have a number of elements that fits in a C int. Either way probably leaks into the signatures in the Python C API. From tim.one at comcast.net Wed Aug 6 15:52:55 2003 From: tim.one at comcast.net (Tim Peters) Date: Wed Aug 6 14:53:26 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: Message-ID: [James Kew] > ... > I find the \pythonxx default more useful for what I do, but > installing to the Program Files directory is pretty idiomatic for > Windows users; why discourage them? Because they're making trouble for themselves if they do. Python discourages them from referencing uninitialized variables too . From oussoren at cistron.nl Wed Aug 6 22:01:51 2003 From: oussoren at cistron.nl (Ronald Oussoren) Date: Wed Aug 6 15:02:01 2003 Subject: [Python-Dev] Multiple levels of site-packages In-Reply-To: References: <18163A21-C839-11D7-9FDF-0003931CFE24@cistron.nl> Message-ID: <703138C5-C840-11D7-9FDF-0003931CFE24@cistron.nl> On Wednesday, 6 August, 2003, at 20:38, Martin v. L?wis wrote: > Ronald Oussoren writes: > >> The Python distribution, and distutils, knows of two types of >> packages/modules: those in the core distribution and the >> site-packages. > > This is incorrect. The Python distribution knows about four types of > packages: core distribution, site-packages, site-python, and > PYTHONPATH. I didn't know about site-python, but this seems to be a version-independent directory. The code in site.py adds os.path.join(prefix, "lib", "site-python"), that doesn't include a version number. Wouldn't that cause problems if you have multiple versions of Python installed (especially when you place python extensions in that directory)? Ronald From bac at OCF.Berkeley.EDU Wed Aug 6 13:14:29 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 6 15:14:35 2003 Subject: [Python-Dev] Do we even care about 2.1.x anymore? Message-ID: <3F315395.4070009@ocf.berkeley.edu> Bug #783348 discovered a bug in the email package for Python 2.1.2 . Do we even care about this branch anymore? I know Zope is based on it but that's it. If we do care what branch should be patched? -Brett From pedronis at bluewin.ch Wed Aug 6 22:24:15 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Wed Aug 6 15:23:15 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: References: <20030806142330.A32662@starship.python.net> Message-ID: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> At 14:48 06.08.2003 -0400, Tim Peters wrote: >[Guido] > > OK, but in the past (when I was younger :-) *I* thought > > that that was a safe claim, otherwise I wouldn't have implemented > > id() the way it was. In that past, the difference between Python int > > and long was much greater, since much of the C library code wouldn't > > deal with longs at all. > >Well, that's all in the past, and id() isn't implemented that way anymore; >at this time it's just a fact that a Python int isn't big enough to hold a >machine address on all boxes. but that something like id() can be cheaply offered/exposed is very much a characteristic of the underlying GC implementation that is being exploited (objects don't move), and the GC impl is an overall implementation detail. I have just gone through implementing a correct id() for Jython. For the serious usage id() would be better substituted by an identity mapping implementation which initially for CPython could simply be using id(). But the user would use that and not rely on id(). id() could maybe be kept around for debugging purposes. Now, a next step anyway for Jython is to modify copy.py (.deepcopy) to use an indenty mapping instead of id(), either having a Jython version or changing CPython one with approriate conditional logic. regards. From tim.one at comcast.net Wed Aug 6 16:30:08 2003 From: tim.one at comcast.net (Tim Peters) Date: Wed Aug 6 15:30:41 2003 Subject: [Python-Dev] Do we even care about 2.1.x anymore? In-Reply-To: <3F315395.4070009@ocf.berkeley.edu> Message-ID: [Brett] > Bug #783348 discovered a bug in the email package for Python 2.1.2 . > Do we even care about this branch anymore? Who's "we"? I don't care about it. > I know Zope is based on it Zope installations should be using 2.1.3. > but that's it. If we do care what branch should be patched? I doubt there will be a 2.1.4 release, but if somone wants it, work for it should occur on branch release21-maint. From tim.one at comcast.net Wed Aug 6 16:37:25 2003 From: tim.one at comcast.net (Tim Peters) Date: Wed Aug 6 15:37:56 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> Message-ID: [Samuele Pedroni] > but that something like id() can be cheaply offered/exposed is very > much a characteristic of the underlying GC implementation that is > being exploited (objects don't move), The docs for id() just promise a unique integer; it needn't be a memory address. > and the GC impl is an overall implementation detail. > > I have just gone through implementing a correct id() for Jython. > > For the serious usage id() would be better substituted by an identity > mapping implementation Sorry, I don't know what that means, and a unique integer seems to be what most users of id() are looking for. > which initially for CPython could simply be using id(). But the user would use > that and not rely on id(). As above, the identity-mapping business lost me. > ... From pedronis at bluewin.ch Wed Aug 6 22:56:44 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Wed Aug 6 16:59:51 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: References: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030806214558.0263ab30@pop.bluewin.ch> At 15:37 06.08.2003 -0400, Tim Peters wrote: >[Samuele Pedroni] > > but that something like id() can be cheaply offered/exposed is very > > much a characteristic of the underlying GC implementation that is > > being exploited (objects don't move), > >The docs for id() just promise a unique integer; it needn't be a memory >address. care to suggest a cheap way to get such a unique integer that's not the address. > > and the GC impl is an overall implementation detail. > > > > I have just gone through implementing a correct id() for Jython. > > > > For the serious usage id() would be better substituted by an identity > > mapping implementation > >Sorry, I don't know what that means, and a unique integer seems to be what >most users of id() are looking for. copy, pickle can use an identity mapping i.e a dictionary using identity instead of equality to do their business, this is more convenient if you cannot get a id() out of an address because otherwise I see no other way than keeping a weak identity mapping from objects to integral type values, the aforementioned unique integers. Likely using a type for which a counter to get fresh unique integers cheaply will not overflow too quickly. Or instead of the mapping attach such integrals value to each object. The latter is not an option for Jython, because we cannot attach things to general Java objects we have to deal with. regards. From martin at v.loewis.de Wed Aug 6 22:05:09 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Wed Aug 6 17:05:10 2003 Subject: [Python-Dev] Multiple levels of site-packages In-Reply-To: <703138C5-C840-11D7-9FDF-0003931CFE24@cistron.nl> References: <18163A21-C839-11D7-9FDF-0003931CFE24@cistron.nl> <703138C5-C840-11D7-9FDF-0003931CFE24@cistron.nl> Message-ID: Ronald Oussoren writes: > I didn't know about site-python, but this seems to be a > version-independent directory. The code in site.py adds > os.path.join(prefix, "lib", "site-python"), that doesn't include a > version number. Wouldn't that cause problems if you have multiple > versions of Python installed (especially when you place python > extensions in that directory)? It may cause problems, but it may also be a solution. If you have a package that is independent of a version, you only need to install it once. Regards, Martin From tim.one at comcast.net Wed Aug 6 17:20:55 2003 From: tim.one at comcast.net (Tim Peters) Date: Wed Aug 6 17:08:51 2003 Subject: [Python-Dev] CALL_ATTR patch (was: 2.3b1 release) In-Reply-To: <17E59550-C80D-11D7-978B-0003931DF95C@python.net> Message-ID: [Michael Hudson] > ... > And then I find out running test_descr in a loop leaks 166(!) > references *anyway* (run the attached blah.py in a debug build, and > some print test or other that I don't understand fails when run like > this, so I fiddled it) > > Should I have been expecting this, or is it time for some ref-leak > hunting? Actually, the process seems to grow -- slowly -- when you > loop, so it's probably a real problem. I gather it's best to ask > about this sort of thing when Tim is off sick from work :-) Only if you want help . Since Zope has worse leak problems than Python, Zope testing has more tools to dig into it: http://cvs.zope.org/Zope3/test.py That's the Zope3 unittest test driver. The TrackRefs class is a handy little beast, which uses a debug build's sys.getobjects() to keep track of how many objects of each *type* exist across test runs, and how many refcounts total they account for. After the first few iterations, TrackRefs doesn't itself distort the results it prints. It will tell you the types of the objects that are leaking, and also whether objects are actually leaking, or that merely refcounts to existing objects are leaking (e.g., a common kind of "leak" is to forget to decref Py_None, but memory doesn't actually grow then). Do use 2.3 if you try this. The "list of all objects" before 2.3 turned out to be missing many objects that tend to leak (None, booleans, any number of type objects, essentially all statically allocated objects), and far fewer are missing in 2.3. It's typically easy to find out which types of objects are leaking this way. Then it's typical to add more code to TraceRefs.update() to get more details. In Zope use, it's surprising how often that still left us scratching our heads! But Zope has lots of extension types coded in C, massive interconnections among them, and none of them originally participated in cyclic gc. Python leaks are usually easier to plug (for example, in Python, they're almost always in new code). From pedronis at bluewin.ch Wed Aug 6 23:47:26 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Wed Aug 6 17:09:21 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: References: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <20030806142330.A32662@starship.python.net> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> At 22:25 06.08.2003 +0200, Martin v. L?wis wrote: >Samuele Pedroni writes: > > > but that something like id() can be cheaply offered/exposed is very > > much a characteristic of the underlying GC implementation that is > > being exploited (objects don't move) > >That is not true. id() exposes the "identity" of an object, and an >run-time system that has the notion of object identities (as opposed >to all-things-with-equal-values-are-equal) should have no difficulties >exposing the object identity as a number. at each time objects have all different addresses, which does not mean that an object will not have the same address that some other object, even still alive, had at some time in the past. Identity distinguishability and identity as a number are not the same. If you want identity as a number you should do some work. > > I have just gone through implementing a correct id() for Jython. > >It sounds like you haven't been looking hard enough. The Java VM >certainly has a unique (among "life" objects) identification of >objects, which likely also fits into a fixed number of bytes. > Thanks, I did my homework. why do you think it certainly have to. It doesn't. From martin at v.loewis.de Wed Aug 6 23:27:12 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Wed Aug 6 18:27:13 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> References: <20030806142330.A32662@starship.python.net> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> Message-ID: Samuele Pedroni writes: > but that something like id() can be cheaply offered/exposed is very > much a characteristic of the underlying GC implementation that is > being exploited (objects don't move) That is not true. id() exposes the "identity" of an object, and an run-time system that has the notion of object identities (as opposed to all-things-with-equal-values-are-equal) should have no difficulties exposing the object identity as a number. > I have just gone through implementing a correct id() for Jython. It sounds like you haven't been looking hard enough. The Java VM certainly has a unique (among "life" objects) identification of objects, which likely also fits into a fixed number of bytes. Interpret these bytes as a number. Regards, Martin From martin at v.loewis.de Wed Aug 6 23:27:13 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Wed Aug 6 18:27:20 2003 Subject: [Python-Dev] FWD: Re: Problem w/ IDLE on Win2000 References: <20030806160441.GA29620@panix.com> Message-ID: martin@v.loewis.de (Martin v. L?wis) writes: > Aahz writes: > > > Is there an SF bug for this? Should I add it to bugs.html? > > [ 784183 ] IDLE does not start for 2.3 on windows. Also [ 780451 ] IDLE fails to launch if 2.3 is installed to "Program Files" Regards, Martin From martin at v.loewis.de Wed Aug 6 23:27:45 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Wed Aug 6 18:28:05 2003 Subject: [Python-Dev] FWD: Re: Problem w/ IDLE on Win2000 Message-ID: <200308062027.h76KRjAr003770@mira.informatik.hu-berlin.de> martin@v.loewis.de (Martin v. L?wis) writes: > Aahz writes: > > > Is there an SF bug for this? Should I add it to bugs.html? > > [ 784183 ] IDLE does not start for 2.3 on windows. Also [ 780451 ] IDLE fails to launch if 2.3 is installed to "Program Files" Regards, Martin From pedronis at bluewin.ch Thu Aug 7 01:53:48 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Wed Aug 6 18:52:41 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> References: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <20030806142330.A32662@starship.python.net> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030807005157.0263ab30@pop.bluewin.ch> At 22:47 06.08.2003 +0200, Samuele Pedroni wrote: >> > I have just gone through implementing a correct id() for Jython. >> >>It sounds like you haven't been looking hard enough. The Java VM >>certainly has a unique (among "life" objects) identification of >>objects, which likely also fits into a fixed number of bytes. > >Thanks, I did my homework. > >why do you think it certainly have to. It doesn't. btw, I'm amazed how I managed to answer so calmly,next time I won't. From pedronis at bluewin.ch Thu Aug 7 04:11:13 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Wed Aug 6 21:10:04 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> References: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <20030806142330.A32662@starship.python.net> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030807021257.026bba10@pop.bluewin.ch> At 22:47 06.08.2003 +0200, Samuele Pedroni wrote: >At 22:25 06.08.2003 +0200, Martin v. L?wis wrote: >>Samuele Pedroni writes: >> >> > but that something like id() can be cheaply offered/exposed is very >> > much a characteristic of the underlying GC implementation that is >> > being exploited (objects don't move) >> >>That is not true. id() exposes the "identity" of an object, and an >>run-time system that has the notion of object identities (as opposed >>to all-things-with-equal-values-are-equal) should have no difficulties >>exposing the object identity as a number. > >at each time objects have all different addresses, which does not mean >that an object will not have the same address that some other object, even >still alive, had at some time in the past. Identity distinguishability and >identity as a number are not the same. If you want identity as a number >you should >do some work. > Or put in another way, you obviously don't need id(.) to implement 'is'. Across the range of possible GC implementations there are trade-off wrt offering id. In the long run limiting id() usage to the production of generic <...> reprs, if x-substrate reimplementability of Python is a goal, is a reasonable course of action. Then uniqueness and to be constant become then nice-to-have properties, which can be costly but if the usage is for <...> reprs i.e. basically debugging, then the price can be more reasonable to pay anywhere, if users are informed of the trade-offs and know not to use id for some usage situations it is used even by the std lib in CPython right now (e.g. in pickle or copy.deepcopy) and have alternatives and the std lib use the alternatives too. Common Lisp for example has EQ which is equivalent to Python 'is', but nothing like id(.): then e.g. Xanalys LispWorks (here 4.2 under Windows) produces the equivalent of <...> reprs using memory addresses which can change under user nose: CL-USER 61 : 1 > (setq x (make-instance 'standard-object)) # ... CL-USER 75 : 1 > x # CL-USER 76 : 1 > x # CL-USER 77 : 1 > x # CL-USER 78 : 1 > x # I don't find this very user-friendly btw, but I understand the trade-offs. See also implementation specific object-address function docs: http://www.lispworks.com/reference/lww42/LWRM-W/html/lwref-w-426.htm#pgfId-886378 Post 1.2 Java has java.lang.System.identityHashCode(.) which is constant but not unique for a given object, which is what broken Jython id(.) was using. Now Jython sports a costly but correct id(.), which makes it so paradoxically that is should'nt be used for copy.deepcopy ... at least by Jython. Btw, the broken id(.) bug was a long standing one, and didn't try to address it for the fun of getting crappy remarks. Thanks. On a more serious note: I wonder if there is thirdy party code depending on copy.deepcopy/pickle memos being id keyed. I fear the answer is yes. regards. From tjreedy at udel.edu Wed Aug 6 22:13:48 2003 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Aug 6 21:13:55 2003 Subject: [Python-Dev] Re: sizeof(long) != sizeof(void*) References: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch><20030806142330.A32662@starship.python.net><5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> Message-ID: "Tim Peters" wrote in message news:BIEJKCLHCIOIHAGOKOLHMEMPGIAA.tim.one@comcast.net... > The docs for id() just promise a unique integer; it needn't be a memory > address. "Martin v. Löwis" wrote in message news:m3r83yfs8a.fsf@mira.informatik.hu-berlin.de... > The Java VM > certainly has a unique (among "life" objects) identification of > objects, which likely also fits into a fixed number of bytes. But does not need to. > Interpret these bytes as a number. Same idea I had. "Samuele Pedroni" wrote in message news:5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch... >at each time objects have all different addresses, which does not mean >that an object will not have the same address that some other object, even >still alive, had at some time in the past. Remove 'even still alive' and the same is true of CPython, leading some newbies to misunderstand why id([])==id([]) can be true. > Identity distinguishability and >identity as a number are not the same. If you want identity as a number you >should do some work. Can't parse this. (repeat of Lowis)>>The Java VM >>certainly has a unique (among "life" objects) identification of >>objects, which likely also fits into a fixed number of bytes. >why do you think it certainly have to. It doesn't. Which does it not have? unique id info or fixed # of bytes? When Jython (or any Python interpreter acting through physical means) binds a Py object to a 'target' (name or slot), it would naively seem that it must store *some* sort of permanent information, unique for each object, that enables the interpreter to reliably access that particular object. If so, Martin (and I) are suggesting to turn that info into an id integer. (The conversion need not even be invertible). An alternative, if there is *no* permanent access information for an object, because gc updates *all* refs after moving an object, is an id field as part of the object header. I can see that could be a nuisance for something that is almost not needed. Terry J. Reedy From tim.one at comcast.net Wed Aug 6 22:23:02 2003 From: tim.one at comcast.net (Tim Peters) Date: Wed Aug 6 21:23:34 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030806214558.0263ab30@pop.bluewin.ch> Message-ID: [Samuele Pedroni] > care to suggest a cheap way to get such a unique integer that's > not the address. I'd love to, but don't know enough about Java to guess. > ... > otherwise I see no other way than keeping a weak identity mapping > from objects to integral type values, the aforementioned unique > integers. Then is there a reason not to do that? Presumably only objects to which id() is actually applied need to become keys in such a mapping, and if so only id() users pay the price. > Likely using a type for which a counter to get fresh unique > integers cheaply will not overflow too quickly. If it's a weak-keyed dict you could also reuse the integer values associated with keys that go away. > Or instead of the mapping attach such integrals value to each > object. The latter is not an option for Jython, because we cannot > attach things to general Java objects we have to deal with. Then why bring it up ? Note that I don't object to introducing a mechanism that copy etc can use that's highly efficient under all implementations. But the introduction of such a mechanism isn't sufficient reason to get rid of the current id(), even if id() is horridly expensive in some implementations. From tjreedy at udel.edu Wed Aug 6 22:37:21 2003 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Aug 6 21:37:26 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 References: <20030806160441.GA29620@panix.com><20030806141508.C32531@starship.python.net> Message-ID: "James Kew" wrote in message news:bgrihu$83n$1@main.gmane.org... > "Guido van Rossum" wrote in message > news:20030806141508.C32531@starship.python.net... > > > The default path is \python23 for a reason. Why do people > > ignore the default? I wouldn't call it a bug, at most a caveat. > > IMHO, it's a bug if: > > a) the user is allowed to change the installation default, but > b) the installation (or part of it) fails to work correctly if he does. The problem is that most changes are ok but one (which people are too likely to make) is not. If \python23, etc, were required, people would complain about *that*. I suspect the Wise Installer does not have an easy option to check user entries for spaces. The root problem is Microsoft casually breaking working code for no decent reason. For perspective: A few years ago, a major company (Adaptec as I remember) automatically installed programs to \Prog Files\Adaptec\whatever that would not work there. Now *that* was an install bug. Terry J. Reedy From pedronis at bluewin.ch Thu Aug 7 04:50:07 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Wed Aug 6 21:48:47 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: References: <5.2.1.1.0.20030806214558.0263ab30@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030807033755.026b7730@pop.bluewin.ch> At 21:23 06.08.2003 -0400, Tim Peters wrote: >[Samuele Pedroni] > > care to suggest a cheap way to get such a unique integer that's > > not the address. > >I'd love to, but don't know enough about Java to guess. > > > ... > > otherwise I see no other way than keeping a weak identity mapping > > from objects to integral type values, the aforementioned unique > > integers. > >Then is there a reason not to do that? Presumably only objects to which >id() is actually applied need to become keys in such a mapping, and if so >only id() users pay the price. indeed is what I did, but that's why I want to reduce id(.) usage if possible > > Likely using a type for which a counter to get fresh unique > > integers cheaply will not overflow too quickly. > >If it's a weak-keyed dict you could also reuse the integer values associated >with keys that go away. for the moment I'm using 64 bits without reuse. I made the maybe silly computation that at the (for the moment) unrealist rate of a fresh id request per 1ns, it will need some years to overflow. > > Or instead of the mapping attach such integrals value to each > > object. The latter is not an option for Jython, because we cannot > > attach things to general Java objects we have to deal with. > >Then why bring it up ? > >Note that I don't object to introducing a mechanism that copy etc can use >that's highly efficient under all implementations. But the introduction of >such a mechanism isn't sufficient reason to get rid of the current id(), >even if id() is horridly expensive in some implementations. yes, that would be a reasonable compromise, basically as implemented the new Jython id is costly a bit in speed (not so relevant) and then memory if one asks the id of a lot of long lived objects, so users should be aware of this and in case stear away from id, having copy.deepcopy etc not using id(.) would help with that. Deprecating maybe is too much but at least making users aware that id() is horridly expensive in some implementations . regards. From tim.one at comcast.net Thu Aug 7 00:17:44 2003 From: tim.one at comcast.net (Tim Peters) Date: Wed Aug 6 23:18:15 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: Message-ID: [Terry Reedy] > The problem is that most changes are ok but one (which people are > too likely to make) is not. The real problem is that there shouldn't be a problem here at all. The shortcut the installer creates to launch IDLE is fine, spaces or not, and, e.g., the similar shortcut created to launch the Tk-based pydoc GUI works fine if you install Python into a path with embedded spaces. I don't know why IDLE fails to start up, just that the reason isn't obvious. IDLE in 2.2 did start up, spaces or not. IDLE in 2.3 is very different. I *suspect* 2.3 IDLE is getting screwed by building a command line for spawn that isn't robust against embedded path whitespace. > If \python23, etc, were required, people would complain about > *that*. Ya, people complain about everything. That's a wash. > I suspect the Wise Installer does not have an easy option to > check user entries for spaces. It's actually not hard to do that. It shouldn't be necessary to do that, though. > The root problem is Microsoft casually breaking working code for > no decent reason. I have no evidence of an MS bug here. My sisters like putting spaces in file names, which is why MS started allowing them. The backward compatibility problem then was severe, especially given system shells with weak and inconsistent quoting gimmicks. Python doesn't try to hide this on Windows, and using spawn() or system() successfully from Python on Windows requires that you deal with whitespace in the executable's path yourself. From martin at v.loewis.de Thu Aug 7 07:24:05 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Thu Aug 7 02:24:06 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> References: <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <20030806142330.A32662@starship.python.net> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> Message-ID: Samuele Pedroni writes: > at each time objects have all different addresses, which does not mean > that an object will not have the same address that some other object, even > still alive, had at some time in the past. So yes, you cannot use addresses. > why do you think it certainly have to. Because, in jvmpi.h, I find this: struct _jobjectID; typedef struct _jobjectID * jobjectID; /* type of object ids */ typedef struct { ... jobjectID (*jobject2jobjectID)(jobject jobj); ... } JVMPI_Interface; Regards, Martin From kbk at shore.net Thu Aug 7 03:29:36 2003 From: kbk at shore.net (Kurt B. Kaiser) Date: Thu Aug 7 02:29:40 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: ("Tim Peters"'s message of "Wed, 6 Aug 2003 23:17:44 -0400") References: Message-ID: "Tim Peters" writes: > The real problem is that there shouldn't be a problem here at all. > The shortcut the installer creates to launch IDLE is fine, spaces or not, > and, e.g., the similar shortcut created to launch the Tk-based pydoc GUI > works fine if you install Python into a path with embedded spaces. Yes, I installed Python23 in Program Files and it will start only if I change the shortcut by appending the -n switch, which causes IDLE to start w/o the subprocess, as in days of yore. > I don't know why IDLE fails to start up, just that the reason isn't > obvious. IDLE in 2.2 did start up, spaces or not. IDLE in 2.3 is > very different. I *suspect* 2.3 IDLE is getting screwed by building > a command line for spawn that isn't robust against embedded path > whitespace. Something like that, no doubt. [...] > I have no evidence of an MS bug here. :-) > My sisters like putting spaces in file names, which is why MS > started allowing them. The backward compatibility problem then was > severe, especially given system shells with weak and inconsistent > quoting gimmicks. C:\>"Progam Files"\Python23\python f:\"My Documents"\Python\animal.py Program Files\Python23\python: can't open file 'f:\My' C:\>"Progam Files"\Python23\python "f:\My Documents\Python\animal.py" bark bark bark :~P > Python doesn't try to hide this on Windows, and using spawn() or > system() successfully from Python on Windows requires that you deal > with whitespace in the executable's path yourself. >From a command window: C:\>"Program Files"\Python23\python "\Program Files\Python23\Lib\idlelib\idle.py" C:\Program: Can't open 'Files\Python23\python.exe' ::-P Quoting the entire first phase gives the same result. C:\>PROGRA~1\Python23\python "\Program Files\Python23\Lib\idlelib\idle.py" will start IDLE with the subprocess. In other universes this would be laughable. I assume the attempt to spawn is responsible for that error message, and the python executable can't be located, but I haven't got that far yet. It leaves a partially started (and stuck) user process behind. This is not an MS bug and nothing can go wrong go wrong go rwong -- KBK From pedronis at bluewin.ch Thu Aug 7 10:13:21 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Thu Aug 7 03:12:21 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: References: <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <20030806142330.A32662@starship.python.net> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030807090514.026b83f8@pop.bluewin.ch> At 08:23 07.08.2003 +0200, Martin v. L?wis wrote: > > why do you think it certainly have to. > >Because, in jvmpi.h, I find this: > >struct _jobjectID; >typedef struct _jobjectID * jobjectID; /* type of object ids */ >typedef struct { > ... > jobjectID (*jobject2jobjectID)(jobject jobj); > ... >} JVMPI_Interface; care to do you your homework better next time please, instead of a making a cheap point: http://java.sun.com/j2se/1.4.2/docs/guide/jvmpi/jvmpi.html#jvmpi_ids notice things like: A jobjectID is defined by an object alloc event, and remains valid in the arena in which the object is allocated until one of its undefining events arrive: ... An object move event is a special type of undefining events. Unlike other undefining events which signal the end-of-life of the corresponding entities, the object still exists, but its ID changes, and it may have been moved to a new arena. ... Since object IDs may be invalidated during GC,... See also: http://java.sun.com/j2se/1.4.2/docs/guide/jvmpi/jvmpi.html#obj_move Thanks. From mwh at python.net Thu Aug 7 15:08:52 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 7 09:08:55 2003 Subject: Ref counting fun (was: Re: [Python-Dev] CALL_ATTR patch) In-Reply-To: ("Tim Peters"'s message of "Wed, 6 Aug 2003 16:20:55 -0400") References: Message-ID: <2mr83xpqcb.fsf@starship.python.net> "Tim Peters" writes: > [Michael Hudson] >> ... >> And then I find out running test_descr in a loop leaks 166(!) >> references *anyway* (run the attached blah.py in a debug build, and >> some print test or other that I don't understand fails when run like >> this, so I fiddled it) >> >> Should I have been expecting this, or is it time for some ref-leak >> hunting? Actually, the process seems to grow -- slowly -- when you >> loop, so it's probably a real problem. I gather it's best to ask >> about this sort of thing when Tim is off sick from work :-) > > Only if you want help . Actually, I found the worst culprit this morning. Some dunderhead had got the decrefs wrong on error return from type_set_bases... I'll check my fix in momentarily. > Since Zope has worse leak problems than > Python, Zope testing has more tools to dig into it: > > http://cvs.zope.org/Zope3/test.py > > That's the Zope3 unittest test driver. The TrackRefs class is a handy > little beast, which uses a debug build's sys.getobjects() to keep track of > how many objects of each *type* exist across test runs, and how many > refcounts total they account for. After the first few iterations, TrackRefs > doesn't itself distort the results it prints. It will tell you the types of > the objects that are leaking, and also whether objects are actually leaking, > or that merely refcounts to existing objects are leaking (e.g., a common > kind of "leak" is to forget to decref Py_None, but memory doesn't actually > grow then). Cool! I'd blundered my way to something like an ad hoc version of this, next time I'll use something that someone's actually thought about :-) > Do use 2.3 if you try this. Well, yeah. Turned out the worst leaks weren't in 2.2 code anyway. > The "list of all objects" before 2.3 turned out to be missing many > objects that tend to leak (None, booleans, any number of type > objects, essentially all statically allocated objects), and far > fewer are missing in 2.3. > > It's typically easy to find out which types of objects are leaking this way. > Then it's typical to add more code to TraceRefs.update() to get more > details. In Zope use, it's surprising how often that still left us > scratching our heads! But Zope has lots of extension types coded in C, > massive interconnections among them, and none of them originally > participated in cyclic gc. Python leaks are usually easier to plug (for > example, in Python, they're almost always in new code). Bingo! I did all these tests incorporating my safety fixes to PyObject_GenericGetAttr so I think they're probably solid, too. Cheers, mwh -- The PROPER way to handle HTML postings is to cancel the article, then hire a hitman to kill the poster, his wife and kids, and fuck his dog and smash his computer into little bits. Anything more is just extremism. -- Paul Tomblin, asr From aahz at pythoncraft.com Thu Aug 7 11:42:39 2003 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 7 10:42:44 2003 Subject: [Python-Dev] SF tracker mini-form Message-ID: <20030807144239.GA16130@panix.com> Many of you are aware that using http://www.python.org/sf/##### will take you directly to the SourceForge item of #####, regardless of whether it's a patch or a bug. I've updated the underlying CGI script to also allow http://www.python.org/sf?##### which means that it can be used in a form. I've added a mini-form to http://www.python.org/dev/ that demonstrates. Hope some of y'all find it useful. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From goodger at python.org Thu Aug 7 12:12:22 2003 From: goodger at python.org (David Goodger) Date: Thu Aug 7 11:13:06 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <20030807144239.GA16130@panix.com> References: <20030807144239.GA16130@panix.com> Message-ID: <3F326C56.5000306@python.org> Aahz wrote: > which means that it can be used in a form. I've added a mini-form to > > http://www.python.org/dev/ That's cool. Would it be possible to provide a minimal form at (i.e., instead of the current "You did not provide a report number")? -- David Goodger From aahz at pythoncraft.com Thu Aug 7 12:17:25 2003 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 7 11:17:34 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <3F326C56.5000306@python.org> References: <20030807144239.GA16130@panix.com> <3F326C56.5000306@python.org> Message-ID: <20030807151725.GA28082@panix.com> On Thu, Aug 07, 2003, David Goodger wrote: > Aahz wrote: >> >>which means that it can be used in a form. I've added a mini-form to >> >> http://www.python.org/dev/ > > That's cool. Would it be possible to provide a minimal form at > (i.e., instead of the current "You > did not provide a report number")? Possible, but I don't feel like thinking about it. As always, patches are welcome. ;-) -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From mwh at python.net Thu Aug 7 17:34:57 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 7 11:35:00 2003 Subject: [Python-Dev] CALL_ATTR patch In-Reply-To: <17E59550-C80D-11D7-978B-0003931DF95C@python.net> (Michael Hudson's message of "Wed, 6 Aug 2003 13:54:19 +0100") References: <17E59550-C80D-11D7-978B-0003931DF95C@python.net> Message-ID: <2mbrv1pjku.fsf@starship.python.net> Michael Hudson writes: > On Friday, Apr 18, 2003, at 01:22 Europe/London, Guido van Rossum wrote: > > Note that this is quite some time ago :-) I've been reading through > some messages that have been ticked for rather too long in my > python-dev mail folder... > >>> (Looking at PyObject_GenericGetAttr with that in mind, I wonder if >>> there isn't a possible crash there. In the first MRO lookup, looking >>> for descr's, if a non-data-descr is found, it is kept around but not >>> INCREF'd until later, after the instance-dict is searched. Am I >>> wrong in believing the PyDict_GetItem of the instance dict can call >>> Python code ? [..] >> Fixing this would make the code even hairier though... :-( > > Pff, it's not so bad, just a little refcount code jiggling. I'll make > a real patch including the above test case soon (once I'm reasonably > confident that I'm not leaking references). http://www.python.org/sf/784825 Cheers, mwh -- ... with these conditions cam the realisation that ... nothing turned a perfectly normal healthy individual into a great political or military leader better than irreversible brain damage. -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From pedronis at bluewin.ch Fri Aug 8 01:02:13 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Thu Aug 7 18:01:01 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030807033755.026b7730@pop.bluewin.ch> References: <5.2.1.1.0.20030806214558.0263ab30@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030807235941.026c0518@pop.bluewin.ch> At 03:50 07.08.2003 +0200, Samuele Pedroni wrote: >At 21:23 06.08.2003 -0400, Tim Peters wrote: > >>Note that I don't object to introducing a mechanism that copy etc can use >>that's highly efficient under all implementations. But the introduction of >>such a mechanism isn't sufficient reason to get rid of the current id(), >>even if id() is horridly expensive in some implementations. > >yes, that would be a reasonable compromise, basically as implemented the >new Jython id is costly a bit in speed (not so relevant) and then memory >if one asks the id of a lot of long lived objects, so users should be >aware of this and in case stear away from id, having copy.deepcopy etc not >using id(.) would help with that. Deprecating maybe is too much but at >least making users aware that id() is horridly expensive in some >implementations . I will follow up with something more concrete to discuss about (some patches) at some point in the 2.4 timeframe. Now I want to focus on other things. At least the problems have been put on the table and misunderstandings cleared up. regards. From mwh at python.net Fri Aug 8 14:41:42 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 8 08:42:27 2003 Subject: Ref counting fun (was: Re: [Python-Dev] CALL_ATTR patch) In-Reply-To: <2mr83xpqcb.fsf@starship.python.net> Message-ID: On Thursday, Aug 7, 2003, at 14:08 Europe/London, Michael Hudson wrote: >> Since Zope has worse leak problems than >> Python, Zope testing has more tools to dig into it: >> >> http://cvs.zope.org/Zope3/test.py >> >> That's the Zope3 unittest test driver. The TrackRefs class is a handy >> little beast, which uses a debug build's sys.getobjects() to keep >> track of >> how many objects of each *type* exist across test runs, and how many >> refcounts total they account for. After the first few iterations, >> TrackRefs >> doesn't itself distort the results it prints. It will tell you the >> types of >> the objects that are leaking, and also whether objects are actually >> leaking, >> or that merely refcounts to existing objects are leaking (e.g., a >> common >> kind of "leak" is to forget to decref Py_None, but memory doesn't >> actually >> grow then). [I'll observe that trying to wrap running a test in a refcount neutral way is entertaining...] > Cool! I'd blundered my way to something like an ad hoc version of > this, next time I'll use something that someone's actually thought > about :-) OK, so I've now blundered my way to the attached, which when run produces this: ints seems to leak 1 references... 1 4 multi seems to leak 9 references... 1 5 1 4 1 4 1 3 0 4 0 1 slots seems to leak 10 references... 5 20 0 5 errors seems to leak 9 references... 4 16 0 5 subtype_resurrection seems to leak 2 references... (the names are all names of functions in test_descr). I might have a look at these over the next few days (and also might ponder other bits of the test suite). Cheers, mwh -------------- next part -------------- A non-text attachment was scrubbed... Name: leak2.py Type: application/octet-stream Size: 2055 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20030808/cb519fa1/leak2.obj -------------- next part -------------- From Jack.Jansen at cwi.nl Fri Aug 8 16:54:25 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Fri Aug 8 09:58:26 2003 Subject: [Python-Dev] Multiple levels of site-packages In-Reply-To: <18163A21-C839-11D7-9FDF-0003931CFE24@cistron.nl> Message-ID: On Wednesday, Aug 6, 2003, at 20:09 Europe/Amsterdam, Ronald Oussoren wrote: > 1) part of the python core > 2) part of the system vendor packages > 3) locally installed network-wide packages > 4) locally installed system-wide packages > 5) locally installed user packages Here's a suggestion for the PEP: come up with an extensible scheme. There are situations where people may want to add yet another level (think of a value added reseller who builds and customizes machines for a specific target market), and there some levels that are not needed on all platforms (such as the network-wide one). We know the core is there, and we know the end user is there, but the stuff in between should be customizable. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From mwh at python.net Fri Aug 8 16:08:00 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 8 10:08:03 2003 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects typeobject.c,2.242,2.243 In-Reply-To: (mwh@users.sourceforge.net's message of "Fri, 08 Aug 2003 06:57:24 -0700") References: Message-ID: <2mn0ekqm2n.fsf@starship.python.net> mwh@users.sourceforge.net writes: > Update of /cvsroot/python/python/dist/src/Objects > In directory sc8-pr-cvs1:/tmp/cvs-serv29176 > > Modified Files: > typeobject.c > Log Message: > /* XXX From here until type is allocated, "return NULL" leaks bases! */ > > Sure looks like it to me! > > When I run the leak2.py script I posted to python-dev, I only see > three reference leaks in all of test_descr. When I run > test_descr.test_main, I still see 46 leaks. This clearly demands > posting a yelp to python-dev :-) This is deeply weird. Watch: [mwh@pc150 build-debug]$ ./python.exe Python 2.4a0 (#25, Aug 8 2003, 14:17:07) [GCC 3.1 20020420 (prerelease)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from test import test_support [40915 refs] >>> test_support.verbose = 0 [41028 refs] >>> from test import test_descr [56312 refs] >>> import gc [56439 refs] >>> for i in range(5): # just to get things to stabilize ... test_descr.test_main(); gc.collect() ... 273 297 297 297 297 [66427 refs] ----------------------------------. >>> test_descr.test_main(); gc.collect() | leaks 46 refs 297 | [66473 refs] <---------------------------------: >>> test_descr.test_main(); gc.collect() | 297 | and again [66519 refs] <---------------------------------' >>> test_descr.test_main(); gc.collect() 32 [66523 refs] ----------------------------------. >>> test_descr.test_main(); gc.collect() | now we leak 32 | just three! [66526 refs] <---------------------------------' >>> gc.set_threshold(193, 10, 10) [66526 refs] >>> test_descr.test_main(); gc.collect() 32 [66571 refs] ----------------------------------. >>> test_descr.test_main(); gc.collect() | back to 46! 32 | [66617 refs] <---------------------------------' >>> Help!? Anyone!? (Neil!?) bewildered-ly y'rs, mwh -- Ya, ya, ya, except ... if I were built out of KSR chips, I'd be running at 25 or 50 MHz, and would be wrong about ALMOST EVERYTHING almost ALL THE TIME just due to being a computer! -- Tim Peters, 30 Apr 97 From mwh at python.net Fri Aug 8 18:44:03 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 8 12:44:06 2003 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects typeobject.c,2.242,2.243 In-Reply-To: <2mn0ekqm2n.fsf@starship.python.net> (Michael Hudson's message of "Fri, 08 Aug 2003 15:08:00 +0100") References: <2mn0ekqm2n.fsf@starship.python.net> Message-ID: <2mk79oqeuk.fsf@starship.python.net> Michael Hudson writes: > mwh@users.sourceforge.net writes: > >> Update of /cvsroot/python/python/dist/src/Objects >> In directory sc8-pr-cvs1:/tmp/cvs-serv29176 >> >> Modified Files: >> typeobject.c >> Log Message: >> /* XXX From here until type is allocated, "return NULL" leaks bases! */ >> >> Sure looks like it to me! >> >> When I run the leak2.py script I posted to python-dev, I only see >> three reference leaks in all of test_descr. When I run >> test_descr.test_main, I still see 46 leaks. This clearly demands >> posting a yelp to python-dev :-) > > This is deeply weird. Watch: > > [mwh@pc150 build-debug]$ ./python.exe > Python 2.4a0 (#25, Aug 8 2003, 14:17:07) > [GCC 3.1 20020420 (prerelease)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> from test import test_support > [40915 refs] >>>> test_support.verbose = 0 > [41028 refs] >>>> from test import test_descr > [56312 refs] >>>> import gc > [56439 refs] >>>> for i in range(5): # just to get things to stabilize > ... test_descr.test_main(); gc.collect() > ... > 273 > 297 > 297 > 297 > 297 > [66427 refs] ----------------------------------. >>>> test_descr.test_main(); gc.collect() | leaks 46 refs > 297 | > [66473 refs] <---------------------------------: >>>> test_descr.test_main(); gc.collect() | > 297 | and again > [66519 refs] <---------------------------------' I missed a "gc.set_threshold(193, 10, 10)" out here... >>>> test_descr.test_main(); gc.collect() > 32 > [66523 refs] ----------------------------------. >>>> test_descr.test_main(); gc.collect() | now we leak > 32 | just three! > [66526 refs] <---------------------------------' >>>> gc.set_threshold(193, 10, 10) > [66526 refs] >>>> test_descr.test_main(); gc.collect() > 32 > [66571 refs] ----------------------------------. >>>> test_descr.test_main(); gc.collect() | back to 46! > 32 | > [66617 refs] <---------------------------------' >>>> > > Help!? Anyone!? (Neil!?) > > bewildered-ly y'rs, > mwh > > -- > Ya, ya, ya, except ... if I were built out of KSR chips, I'd > be running at 25 or 50 MHz, and would be wrong about ALMOST > EVERYTHING almost ALL THE TIME just due to being a computer! > -- Tim Peters, 30 Apr 97 > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev -- You owe The Oracle a TV with an 'intelligence' control - I've tried 'brightness' but that didn't work. -- Internet Oracularity #1192-01 From theller at python.net Fri Aug 8 20:19:54 2003 From: theller at python.net (Thomas Heller) Date: Fri Aug 8 13:20:41 2003 Subject: [Python-Dev] hook for standalone executable Message-ID: There has been a discussion on comp.lang.python about a cross-platform py2exe-like tool: The basic idea is, now that the zipimport feature is in place, to make it easy to convert the Python interpreter itself into a standalone executable by a simple operation (which can well be platform dependend, but on some platforms, at least, a zipfile can be appended to a copy of the interpreter executable itself). Alex Martelli and Oren Tirosh also came up with ideas how to do this, and it seems a hook in Py_Main() would be able to do the trick. This hook could be triggered by examining the filename of the executable itself. If the basename starts (case-insensitive) with the characters 'python', everything continues as it is now. In the other case, the default command line processing would not be executed, instead the command line flags would be inserted into sys.argv. Also the default environment variables would not be used to set flags like Py_OptimizeFlag and so on, instead a special data section in the executable would be searched and used to set these. Some code would be run to insert the executable filename itself at the front of sys.path, and finally an import of a special module, maybe named '__boot__', would be done. Another possibility would be to execute some Python code with PyRun_SimpleString() found in this special data area. Thomas From nas-python at python.ca Fri Aug 8 12:44:22 2003 From: nas-python at python.ca (Neil Schemenauer) Date: Fri Aug 8 14:38:35 2003 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects typeobject.c, 2.242, 2.243 In-Reply-To: <2mn0ekqm2n.fsf@starship.python.net> References: <2mn0ekqm2n.fsf@starship.python.net> Message-ID: <20030808184422.GA4751@glacier.arctrix.com> Michael Hudson wrote: > [66427 refs] ----------------------------------. > >>> test_descr.test_main(); gc.collect() | leaks 46 refs > 297 | > [66473 refs] <---------------------------------: I can't reproduce this. It always leaks 3 references for me. BTW, it's probably best to disable automatic GC with gc.disable() when doing this kind of debugging. I vaguely recall running into a similar bug before. I think the problem in that case was that some tp_clear methods were not dropping all their references and so it took more than one GC run to completely destroy the cyclic structure. It's also possible that this behavior is caused by reference leaks. E.g. a reference count is too high and it takes more than one tp_clear to get it to zero. That shouldn't happen though since tp_clear methods set points to NULL and use Py_XDECREF. Neil From Jack.Jansen at cwi.nl Fri Aug 8 23:21:42 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Fri Aug 8 16:21:54 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: Message-ID: On vrijdag, aug 8, 2003, at 19:19 Europe/Amsterdam, Thomas Heller wrote: > This hook could be triggered by examining the filename of the > executable > itself. If the basename starts (case-insensitive) with the > characters 'python', everything continues as it is now. I don't think I like this. Having something that looks like a perfectly normal application turn into a Python interpreter by copying it and renaming it to "python", or worse, by making a symlink "python" to it (think setuid program here) is too dangerous. As you want to hack the dataspace anyway, to provide for setting flags and such, I would suggest also adding a flag Py_StandaloneProgram there. If the flag is false everything is normal. If the flag is true you do the special processing, but without the possibility of fallback to a normal interpreter. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From kbk at shore.net Fri Aug 8 20:37:22 2003 From: kbk at shore.net (Kurt B. Kaiser) Date: Fri Aug 8 19:37:42 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: References: Message-ID: On Wed, 6 Aug 2003 23:17:44 -0400, Tim Peters wrote: > I have no evidence of an MS bug here. My sisters like putting spaces in > file names, which is why MS started allowing them. The backward > compatibility problem then was severe, especially given system shells > with > weak and inconsistent quoting gimmicks. Python doesn't try to hide this > on > Windows, and using spawn() or system() successfully from Python on > Windows > requires that you deal with whitespace in the executable's path yourself. (I put a copy of python.exe at c:\ to allow the first example below to work.) Python 2.3 (#46, Jul 29 2003, 18:54:32) [MSC v.1200 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> >>> executable = 'c:\\python' >>> os.spawnv(os.P_NOWAIT, executable, (executable, '-V')) 1948 >>> Python 2.3 >>> >>> executable = 'c:\"Program Files"\Python23\python' >>> executable 'c:"Program Files"\\Python23\\python' >>> os.spawnv(os.P_NOWAIT, executable, (executable, '-V')) Traceback (most recent call last): File "", line 1, in ? OSError: [Errno 22] Invalid argument ===================================================== and that version of executable isn't an absolute path ===================================================== >>> executable = 'c:\\\"Program Files"\Python23\python' >>> executable 'c:\\"Program Files"\\Python23\\python' >>> os.spawnv(os.P_NOWAIT, executable, (executable, '-V')) Traceback (most recent call last): File "", line 1, in ? OSError: [Errno 22] Invalid argument >>> >>> executable = '"c:\Program Files\Python23\python"' >>> executable '"c:\\Program Files\\Python23\\python"' >>> os.spawnv(os.P_NOWAIT, executable, (executable, '-V')) Traceback (most recent call last): File "", line 1, in ? OSError: [Errno 22] Invalid argument >>> >>> executable = 'c:\PROGRA~1\Python23\python' >>> executable 'c:\\PROGRA~1\\Python23\\python' >>> os.spawnv(os.P_NOWAIT, executable, (executable, '-V')) 1944 >>> Python 2.3 ======================== Can someone offer advice on what I need to do to create a valid os.spawnv call involving "Program Files" that will work in the above examples? The posts I've been able to find essentially say, "just quote it," but I can't find a combination that works so I can fix the IDLE startup bug. -- KBK From tismer at tismer.com Sat Aug 9 05:59:27 2003 From: tismer at tismer.com (Christian Tismer) Date: Fri Aug 8 22:59:23 2003 Subject: [Python-Dev] Slogan: Getting Rich Overnight Message-ID: <3F34638F.9020201@tismer.com> Dear friends. During a conversation with good friends and newly acquired Pythonista, we were discussing Python, what it is in essence, and what it is giving to us. The people were Dinu Gherman, Giorgio Giacomazzi, a promizing newcomer in the Python noosphere, and myself. We were discussing how to advertize for Python, and Dinu spread some of the recent library enhancements, like - email package - XML parsers - distutils - add lots of other great stuff, here. Then, after a while of silence, Giorgio said something like """ Well, right. But despite of the libraries, I was hit by pure Python, by the following, simply by using it interactively: There are these lists, these tuples, and these dicts. They are immediately there, at my fingertips. And this is a feeling that I never had, before. Especially these dicts are incredible. This was a feeling like 'getting rich overnight'. """ I loved this statement very much, and I have to say, this is essentially my feeling for myself, since many years now. I could imagine that this might be a candidate for next year's Python congress' slogan. "Python makes you rich, overnight". Not by money, in the first place, but by multiplying your own capabilities, immediately. It needed the fresh experience of a newcomer to become aware of this, again. The ambiguity is obvious. On first reading, it will attract many. On second reading, those who are thinking "ahh, ohh, yes, not I understand" will remain. But that's ok for a good slogan! got rich overnight by Python! being rich since 1800 nights now - sincerely -- chris -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tim.one at comcast.net Sat Aug 9 00:44:15 2003 From: tim.one at comcast.net (Tim Peters) Date: Fri Aug 8 23:44:54 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: Message-ID: [Kurt B. Kaiser, struggles to get an embedded-space spawn to work] I think the trick is that you shouldn't quote the executable path at all. Under the covers, MS takes the executable name as-is, passing it as the lpApplicationName argument to the Win32 CreateProcess(), but squashes all your *arguments* into a single string, which gets passed on as the lpCommandLine argument to CreateProcess(). In the arguments, you should avoid the cleverness of trying to quote only the part(s) with a space(s). Stick a double quote on each end of each argument, and that's it(*). So, for example, I made a copy of a wc.exe into c:\Program Files\tmp\w c.exe and then this Python works (on Win98SE(*)) to run that program on itself: """ raw = r'C:\Program Files\tmp\w c.exe' import os print os.path.exists(raw) decorated = '"%s"' % raw os.spawnv(os.P_NOWAIT, raw, (decorated, decorated)) """ Here's the output: C:\Code\python\PCbuild>python temp2.py True 26 203 13824 C:\Program Files\tmp\w c.exe C:\Code\python\PCbuild> So I expect that IDLE is getting in trouble because it's not stuffing quotes around an *argument* to a program it's spawning. (*) There the fun begins. While os.system() spawns a command shell, I don't think spawnv() does. Which would be a real help here. Win9x use command.com by default, and Windows after that use cmd.exe. They have have different quoting rules, and cmd.exe actually has two different sets of quoting rules (triggered by an option to cmd.exe, and/or by a registry setting). Still, I haven't tried the program above on a Win2K (etc) system, and somebody should before you get optimistic. In one of cmd.exe's quoting nightmare modes, whether double quotes get stripped actually depends on the content of the quoted string ! From guido at python.org Fri Aug 8 21:53:08 2003 From: guido at python.org (Guido van Rossum) Date: Fri Aug 8 23:53:38 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: Your message of "Fri, 08 Aug 2003 23:44:15 EDT." References: Message-ID: <200308090353.h793r8e12155@12-236-84-31.client.attbi.com> > [Kurt B. Kaiser, struggles to get an embedded-space spawn to work] [Tim, can't belp helping] > I think the trick is that you shouldn't quote the executable path at all. > Under the covers, MS takes the executable name as-is, passing it as the > lpApplicationName argument to the Win32 CreateProcess(), but squashes all > your *arguments* into a single string, which gets passed on as the > lpCommandLine argument to CreateProcess(). > > In the arguments, you should avoid the cleverness of trying to quote only > the part(s) with a space(s). Stick a double quote on each end of each > argument, and that's it(*). > > So, for example, I made a copy of a wc.exe into > > c:\Program Files\tmp\w c.exe > > and then this Python works (on Win98SE(*)) to run that program on itself: > > """ > raw = r'C:\Program Files\tmp\w c.exe' > > import os > print os.path.exists(raw) > > decorated = '"%s"' % raw > os.spawnv(os.P_NOWAIT, raw, (decorated, decorated)) > """ > > Here's the output: > > C:\Code\python\PCbuild>python temp2.py > True > 26 203 13824 C:\Program Files\tmp\w c.exe > > C:\Code\python\PCbuild> > > So I expect that IDLE is getting in trouble because it's not stuffing quotes > around an *argument* to a program it's spawning. > > > (*) There the fun begins. While os.system() spawns a command shell, > I don't think spawnv() does. Which would be a real help here. > Win9x use command.com by default, and Windows after that use > cmd.exe. They have have different quoting rules, and cmd.exe > actually has two different sets of quoting rules (triggered by > an option to cmd.exe, and/or by a registry setting). > > Still, I haven't tried the program above on a Win2K (etc) > system, and somebody should before you get optimistic. In one > of cmd.exe's quoting nightmare modes, whether double quotes > get stripped actually depends on the content of the quoted > string ! Right. I'm pretty sure that the quoting rules are different on 9x/Me on the one hand and on NT/2K/XP OTOH (though the real reason may be the different default shell). For example, on my 98se laptop, in a DOS shell, cd \Program Files says Too any parameters - files while this works on the XP (still -- Red Hat Linux 9.0 coming soon :-) box at work. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one at comcast.net Sat Aug 9 01:55:45 2003 From: tim.one at comcast.net (Tim Peters) Date: Sat Aug 9 00:56:22 2003 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects typeobject.c, 2.242, 2.243 In-Reply-To: <2mn0ekqm2n.fsf@starship.python.net> Message-ID: [Michael Hudson] > This is deeply weird. Watch: > > [mwh@pc150 build-debug]$ ./python.exe > Python 2.4a0 (#25, Aug 8 2003, 14:17:07) > [GCC 3.1 20020420 (prerelease)] on darwin > Type "help", "copyright", "credits" or "license" for more information. >>>> from test import test_support > [40915 refs] >>>> test_support.verbose = 0 > [41028 refs] We're different already: my total refs stay at a comparatively slim 21426 after both initial lines. Maybe you're tracking a leak due to GNU readline? As for the rest, mine leaks 166 references per stab without variation: >>> import gc [40804 refs] >>> for i in range(5): ... test_descr.test_main(); gc.collect() ... 226 250 250 250 250 [49410 refs] >>> test_descr.test_main(); gc.collect() 250 [49576 refs] >>> test_descr.test_main(); gc.collect() 250 [49742 refs] >>> test_descr.test_main(); gc.collect() 250 [49908 refs] >>> test_descr.test_main(); gc.collect() 250 [50074 refs] >>> test_descr.test_main(); gc.collect() 250 [50240 refs] >>> test_descr.test_main(); gc.collect() 250 [50406 refs] >>> test_descr.test_main(); gc.collect() 250 [50572 refs] >>> test_descr.test_main(); gc.collect() 250 [50738 refs] >>> test_descr.test_main(); gc.collect() 250 [50904 refs] >>> etc. Hmm -- I'm not sure my 2.4 build is entirely up to date, though. From mhammond at skippinet.com.au Sat Aug 9 16:34:26 2003 From: mhammond at skippinet.com.au (Mark Hammond) Date: Sat Aug 9 01:34:27 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: Message-ID: <000701c35e37$e6145410$f502a8c0@eden> I'm with Jack to some extent - it seems a little hacky. But the problem is real. > but on some platforms, at least, a zipfile can be appended to > a copy of > the interpreter executable itself). > > Alex Martelli and Oren Tirosh also came up with ideas how to do this, > and it seems a hook in Py_Main() would be able to do the trick. How about splitting this out a little. * Make changes to Py_Main() necessary to better support "hard-embedders", which may include custom sys.argv/environ processing. * Propose a new executable that some Python platforms can choose to distribute - eg, 'python-package{.exe}'. This is really just identical to python.exe, but with the only 'if doesnt start with 'python'' parts of your proposal. This seems like less magic, but still more flexible for those people who *still* aren't happy with what we did Mark. From raymond.hettinger at verizon.net Sat Aug 9 02:54:55 2003 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat Aug 9 01:59:23 2003 Subject: [Python-Dev] Make it an error to use __slots__ with classic classes Message-ID: <000f01c35e3a$c2c7e460$e841fea9@oemcomputer> Using __slots__ with a classic class is an error that does not readily reveal itself. For Py2.3.1, I would like to issue a warning, and for Py2.4, I would like to raise an exception upon class creation: >>> class A: ... __slots__ = 'abc' # Would otherwise pass silently ... pass ... Traceback (most recent call last): File "", line 1, in ? TypeError: PyClass_New: __slots__ only works with new-style classes Do you guys agree? Raymond Hettinger From fdrake at acm.org Sat Aug 9 03:04:00 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Sat Aug 9 02:04:41 2003 Subject: [Python-Dev] Make it an error to use __slots__ with classic classes In-Reply-To: <000f01c35e3a$c2c7e460$e841fea9@oemcomputer> References: <000f01c35e3a$c2c7e460$e841fea9@oemcomputer> Message-ID: <16180.36560.265354.68464@grendel.zope.com> Raymond Hettinger writes: > Using __slots__ with a classic class is an error that does > not readily reveal itself. For Py2.3.1, I would like to > issue a warning, and for Py2.4, I would like to > raise an exception upon class creation: You'll need to explain your motivation a bit more. Why is it an error to define __slots__ on an old-style class? (Useless perhaps, but why an error?) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From python at rcn.com Sat Aug 9 04:10:47 2003 From: python at rcn.com (Raymond Hettinger) Date: Sat Aug 9 03:14:44 2003 Subject: [Python-Dev] Make it an error to use __slots__ with classic classes References: <000f01c35e3a$c2c7e460$e841fea9@oemcomputer> <16180.36560.265354.68464@grendel.zope.com> Message-ID: <001301c35e45$5becb620$e841fea9@oemcomputer> > Raymond Hettinger writes: > > Using __slots__ with a classic class is an error that does > > not readily reveal itself. For Py2.3.1, I would like to > > issue a warning, and for Py2.4, I would like to > > raise an exception upon class creation: [Mr. Fred] > You'll need to explain your motivation a bit more. Why is it an error > to define __slots__ on an old-style class? (Useless perhaps, but why > an error?) Everytime I've seen this occur, the author had intended for __slots__ to actually be working to save memory by having lighter weight objects. However, since memory usage is not immediately visible, the programming error would go unnoticed. The risk is even higher when subclassing is used. Quick, is __slots__ working in this code: import random class MarkovPair(random.Random): __slots__ = ['current_variate', 'previous_variate'] . . . In my PyZine article, my first draft included an erroneous __slots__ example which did not inherit from object. The code and tests ran fine -- luckily, one alert reviewer caught it. I would have appreciated an exception being raised. Likewise, a recent poster on comp.lang.python had the same experience. Whereever __slots__ are used with old style classes, there is a super high probability that the author intended the code to behave differently than it actually does. Raymond Hettinger P.S. The answer to the quick look question above is that __slots__ has no effect in Py2.2 but does in Py2.3 because one is an old-style class written in pure python and the other is a new-style class derived from the MersenneTwister type. From aleax at aleax.it Sat Aug 9 10:33:27 2003 From: aleax at aleax.it (Alex Martelli) Date: Sat Aug 9 03:33:54 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: <000701c35e37$e6145410$f502a8c0@eden> References: <000701c35e37$e6145410$f502a8c0@eden> Message-ID: <200308090933.27254.aleax@aleax.it> On Saturday 09 August 2003 07:34, Mark Hammond wrote: > I'm with Jack to some extent - it seems a little hacky. But the problem > is real. Jack's point -- that my half-baked idea about changing behavior depending on the executable's name is insecure due e.g. to the possibility of links to a setuid executable -- is perfectly well taken. I'm not sure that there is a "real need" for such an examination of the executable-name. Thus, rather than: > * Make changes to Py_Main() necessary to better support "hard-embedders", > which may include custom sys.argv/environ processing. > > * Propose a new executable that some Python platforms can choose to > distribute - eg, 'python-package{.exe}'. This is really just identical > to python.exe, but with the only 'if doesnt start with 'python'' parts of > your proposal. it seems to me that giving up on the "only if it doesn't start with python" specific tidbit. Rather, 'python-package.exe' could be distributed with the new parts of the code turned on (and perhaps with the parts only needed for interactive interpretation turned off -- optional, just to save some space & perhaps enhance security a bit), if that makes life any easier on some platforms; thus python.exe itself would remain pristine if desired (presumably all of this new stuff might depend on some kind of --with... switch to ./configure). Alex From pedronis at bluewin.ch Sat Aug 9 13:20:17 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Sat Aug 9 06:19:01 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030807090514.026b83f8@pop.bluewin.ch> References: <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <20030806142330.A32662@starship.python.net> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030809121720.026e5288@pop.bluewin.ch> At 09:13 07.08.2003 +0200, Samuele Pedroni wrote: >At 08:23 07.08.2003 +0200, Martin v. L?wis wrote: > >> > why do you think it certainly have to. >> >>Because, in jvmpi.h, I find this: >> >>struct _jobjectID; >>typedef struct _jobjectID * jobjectID; /* type of object ids */ >>typedef struct { >> ... >> jobjectID (*jobject2jobjectID)(jobject jobj); >> ... >>} JVMPI_Interface; > >care to do you your homework better next time please, instead of a making >a cheap point: > >http://java.sun.com/j2se/1.4.2/docs/guide/jvmpi/jvmpi.html#jvmpi_ids [snip] >Thanks. this whole exchange was really sad. :( From guido at python.org Sat Aug 9 07:48:54 2003 From: guido at python.org (Guido van Rossum) Date: Sat Aug 9 09:49:09 2003 Subject: [Python-Dev] Make it an error to use __slots__ with classic classes In-Reply-To: Your message of "Sat, 09 Aug 2003 03:10:47 EDT." <001301c35e45$5becb620$e841fea9@oemcomputer> References: <000f01c35e3a$c2c7e460$e841fea9@oemcomputer> <16180.36560.265354.68464@grendel.zope.com> <001301c35e45$5becb620$e841fea9@oemcomputer> Message-ID: <200308091348.h79DmtK12762@12-236-84-31.client.attbi.com> > > Raymond Hettinger writes: > > > Using __slots__ with a classic class is an error that does > > > not readily reveal itself. For Py2.3.1, I would like to > > > issue a warning, and for Py2.4, I would like to > > > raise an exception upon class creation: > > [Mr. Fred] > > You'll need to explain your motivation a bit more. Why is it an error > > to define __slots__ on an old-style class? (Useless perhaps, but why > > an error?) > > Everytime I've seen this occur, the author had intended for > __slots__ to actually be working to save memory by having > lighter weight objects. However, since memory usage is > not immediately visible, the programming error would go > unnoticed. Hm. There are numerous features of new-style classes that don't work as expected without giving a clear error when accidentally using a classic class: overriding __getattribute__, properties trapping write access, MRO dependencies... Why should __slots__ be special? Most of the times it is used in a new style class it is a subtle mistake just as much as when used in a classic class. __slots__ is a terrible hack with nasty, hard-to-fathom side effects that should only be used by programmers at grandmaster and wizard levels. Unfortunately it has gained an enormous undeserved popularity amongst the novices and apprentices, who should know better than to use this magic incantation casually. But since we don't have a declaration for programmer experience, there's no way to forbid or warn about __slots__ when it is misused. I'd say leave it alone. --Guido van Rossum (home page: http://www.python.org/~guido/) (Yes, I'm really back -- my Linux box survived the move and I have high-speed internet at my new home. No more posting from starship using mutt, yee hah!) From aahz at pythoncraft.com Sat Aug 9 10:49:38 2003 From: aahz at pythoncraft.com (Aahz) Date: Sat Aug 9 09:49:42 2003 Subject: [Python-Dev] Python 2.2.4 In-Reply-To: <16167.13822.82876.622001@grendel.zope.com> References: <16167.2834.754645.707376@gargle.gargle.HOWL> <3F2732E8.70205@ocf.berkeley.edu> <16167.13640.547146.10919@grendel.zope.com> <16167.13822.82876.622001@grendel.zope.com> Message-ID: <20030809134938.GA23431@panix.com> On Tue, Jul 29, 2003, Fred L. Drake, Jr. wrote: > Fred wrote: >> >> I'm inclined to be a little more conservative than that. I'd be glad >> to see a 2.2.4 release toward the end of the year to fix up any minor >> niggles, but we should be really careful about it. I know I have one >> memory leak patch I need to backport to the release22-maint branch. > > And I'll put my time where my mailer is: I'll volunteer to be the > release manager. This prompted me to check PEP 6, and I noticed that nobody has updated it to include any recent Patch Czars. Does anybody care? If yes, I'll track down all the other patch releases. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From sholden at holdenweb.com Sat Aug 9 11:42:46 2003 From: sholden at holdenweb.com (Steve Holden) Date: Sat Aug 9 10:44:33 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: <000701c35e37$e6145410$f502a8c0@eden> Message-ID: > -----Original Message----- > From: python-dev-bounces@python.org > [mailto:python-dev-bounces@python.org]On Behalf Of Mark Hammond > Sent: Saturday, August 09, 2003 1:34 AM > To: 'Thomas Heller'; python-dev@python.org > Subject: RE: [Python-Dev] hook for standalone executable > > > I'm with Jack to some extent - it seems a little hacky. But > the problem is > real. > > > but on some platforms, at least, a zipfile can be appended to > > a copy of > > the interpreter executable itself). > > > > Alex Martelli and Oren Tirosh also came up with ideas how > to do this, > > and it seems a hook in Py_Main() would be able to do the trick. > > How about splitting this out a little. > > * Make changes to Py_Main() necessary to better support > "hard-embedders", > which may include custom sys.argv/environ processing. > > * Propose a new executable that some Python platforms can choose to > distribute - eg, 'python-package{.exe}'. This is really just > identical to > python.exe, but with the only 'if doesnt start with 'python'' > parts of your > proposal. > > This seems like less magic, but still more flexible for those > people who > *still* aren't happy with what we did > I agree with [?Skip?] that the file's *name* is a poor way to determine whether magic behavior should be invoked. Seems to me that the interpreter should know it's own length as an executable. Then, if it gets loaded from a longer file, it can assume there are zipped modules to be imported and run by whatever mechanism is decided. Quite how to implement this without an extra pass is beyond me. regards -- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ > From skip at pobox.com Sat Aug 9 11:38:59 2003 From: skip at pobox.com (Skip Montanaro) Date: Sat Aug 9 11:39:13 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: References: <000701c35e37$e6145410$f502a8c0@eden> Message-ID: <16181.5523.50443.755048@montanaro.dyndns.org> Steve> I agree with [?Skip?] that the file's *name* is a poor way to Steve> determine whether magic behavior should be invoked. It wasn't me. I agree it seems a tad fragile, however it will probably be a lot easier than coming up with a "portable" way for the interpreter to know its size. I'd suggest a command line flag but I don't think that's going to fly either. Skip From martin at v.loewis.de Sat Aug 9 19:06:26 2003 From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=) Date: Sat Aug 9 12:06:51 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: <16181.5523.50443.755048@montanaro.dyndns.org> References: <000701c35e37$e6145410$f502a8c0@eden> <16181.5523.50443.755048@montanaro.dyndns.org> Message-ID: <3F351C02.4090903@v.loewis.de> Skip Montanaro wrote: > It wasn't me. I agree it seems a tad fragile, however it will probably be a > lot easier than coming up with a "portable" way for the interpreter to know > its size. I'd suggest a command line flag but I don't think that's going to > fly either. I recommend to drop the requirement that simple appending of .zip file needs to work, and suggest to use platform-specific mechanisms on each platform. These mechanisms should arrange to patch a variable, Py_EmbeddedZipfile. Normally, this variable is NULL. If a zipfile is appended, this variable changes its value to point to the zipfile (it should probably point to the zip directory, or some such). So in essence, a proper linking step is executed. There should be no dependence on native tools to perform this linking, instead, it should be implemented in pure Python. Regards, Martin From lalo at laranja.org Sat Aug 9 14:24:13 2003 From: lalo at laranja.org (Lalo Martins) Date: Sat Aug 9 12:23:44 2003 Subject: [Python-Dev] Slogan: Getting Rich Overnight In-Reply-To: <3F34638F.9020201@tismer.com> References: <3F34638F.9020201@tismer.com> Message-ID: <20030809162413.GJ29592@laranja.org> On Sat, Aug 09, 2003 at 04:59:27AM +0200, Christian Tismer wrote: > I loved this statement very much, and I have to say, this > is essentially my feeling for myself, since many years now. > I could imagine that this might be a candidate for next year's > Python congress' slogan. "Python makes you rich, overnight". > Not by money, in the first place, but by multiplying your > own capabilities, immediately. > > It needed the fresh experience of a newcomer to become aware > of this, again. > > The ambiguity is obvious. On first reading, it will attract > many. On second reading, those who are thinking "ahh, ohh, yes, > not I understand" will remain. But that's ok for a good slogan! of course all mail we send with this slogan will be held by spam filters ;-) yours only escaped because my spambayes seems to consider your name "ham". []s, |alo +---- -- Those who trade freedom for security lose both and deserve neither. -- http://www.laranja.org/ mailto:lalo@laranja.org pgp key: http://www.laranja.org/pessoal/pgp Eu jogo RPG! (I play RPG) http://www.eujogorpg.com.br/ GNU: never give up freedom http://www.gnu.org/ From terry at wayforward.net Sat Aug 9 13:42:03 2003 From: terry at wayforward.net (Terence Way) Date: Sat Aug 9 12:39:47 2003 Subject: [Python-Dev] Make it an error to use __slots__ with classic classes Message-ID: <67C6EFA4-CA88-11D7-B4F3-00039344A0EC@wayforward.net> On Saturday, August 9, 2003, at 03:10 AM, Raymond Hettinger wrote: > > P.S. The answer to the quick look question above is that > __slots__ has no effect in Py2.2 but does in Py2.3 > because one is an old-style class written in pure python > and the other is a new-style class derived from the > MersenneTwister type. That, to me, indicates that an exception is the wrong way to do this... it would be too easy to create non-backward compatible code, especially as people start moving their classes to new-style. A brief test shows that PyChecker checks for this, however: Warnings... testslot.py:2: Using __slots__ in classic class foo has no effect, consider deriving from object From gherron at islandtraining.com Sat Aug 9 10:50:45 2003 From: gherron at islandtraining.com (Gary Herron) Date: Sat Aug 9 12:50:57 2003 Subject: [Python-Dev] Slogan: Getting Rich Overnight In-Reply-To: <20030809162413.GJ29592@laranja.org> References: <3F34638F.9020201@tismer.com> <20030809162413.GJ29592@laranja.org> Message-ID: <200308090950.45953.gherron@islandtraining.com> On Saturday 09 August 2003 09:24 am, Lalo Martins wrote: > On Sat, Aug 09, 2003 at 04:59:27AM +0200, Christian Tismer wrote: > > I loved this statement very much, and I have to say, this > > is essentially my feeling for myself, since many years now. > > I could imagine that this might be a candidate for next year's > > Python congress' slogan. "Python makes you rich, overnight". > > Not by money, in the first place, but by multiplying your > > own capabilities, immediately. > > > > It needed the fresh experience of a newcomer to become aware > > of this, again. > > > > The ambiguity is obvious. On first reading, it will attract > > many. On second reading, those who are thinking "ahh, ohh, yes, > > not I understand" will remain. But that's ok for a good slogan! > > of course all mail we send with this slogan will be held by spam > filters ;-) yours only escaped because my spambayes seems to > consider your name "ham". That and things Python and Python related tipped the balance of my spambayes towards "ham", but it had to overcome some highly suspicious words, as can be seen from some of the individual word scores. python, 0.00198325253416 python's 0.00372208436725 url:starship 0.0067264573991 "python 0.0079086115993 christian 0.00884086444008 distutils 0.0110024449878 python 0.013485114594 tismer 0.0167286245353 money, 0.903491880007 rich, 0.908163265306 subject:Overnight 0.934782608696 overnight 0.960512038332 Gary Herron From skip at pobox.com Sat Aug 9 13:45:10 2003 From: skip at pobox.com (Skip Montanaro) Date: Sat Aug 9 13:45:15 2003 Subject: [Python-Dev] PyMapping_Check Bug? Message-ID: <16181.13094.559794.648236@montanaro.dyndns.org> Passing along a fragment from the psycopg list where Federico Di Gregorio (fog@initd.org) wrote: .... because on python 2.3 PyMapping_Check apperently return True for every new-type instance.... Is this true? If so, is it a bug? Skip From guido at python.org Sat Aug 9 11:52:21 2003 From: guido at python.org (Guido van Rossum) Date: Sat Aug 9 13:52:33 2003 Subject: [Python-Dev] PyMapping_Check Bug? In-Reply-To: Your message of "Sat, 09 Aug 2003 12:45:10 CDT." <16181.13094.559794.648236@montanaro.dyndns.org> References: <16181.13094.559794.648236@montanaro.dyndns.org> Message-ID: <200308091752.h79HqL313056@12-236-84-31.client.attbi.com> > Passing along a fragment from the psycopg list where Federico Di Gregorio > (fog@initd.org) wrote: > > .... because on python 2.3 PyMapping_Check apperently return True for > every new-type instance.... > > Is this true? If so, is it a bug? I don't see this. operator.isMappingType() maps directly to PyMapping_Check(); and I tried this: >>> import operator >>> operator.isMappingType(str) False >>> class C(object): pass ... >>> operator.isMappingType(C) False >>> operator.isMappingType(C()) False >>> operator.isMappingType({}) True >>> Looks okay to me. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From allison at sumeru.stanford.EDU Sat Aug 9 11:58:27 2003 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Sat Aug 9 13:58:36 2003 Subject: [Python-Dev] RH9 and Python 2.1.3, 2.2, and 2.3 Message-ID: What's the status of the thread and threading modules with respect to RH9's new NPTL (Native Posix Thread Library)? I am concerned with Python 2.1.3 (for Zope) and Python 2.3 (for everything else). From theller at python.net Sat Aug 9 21:40:40 2003 From: theller at python.net (Thomas Heller) Date: Sat Aug 9 14:40:57 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: <3F351C02.4090903@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sat, 09 Aug 2003 18:06:26 +0200") References: <000701c35e37$e6145410$f502a8c0@eden> <16181.5523.50443.755048@montanaro.dyndns.org> <3F351C02.4090903@v.loewis.de> Message-ID: <3cgaoes7.fsf@python.net> "Martin v. L?wis" writes: > I recommend to drop the requirement that simple appending of .zip file > needs to work, and suggest to use platform-specific mechanisms on each > platform. These mechanisms should arrange to patch a variable, > Py_EmbeddedZipfile. Normally, this variable is NULL. If a zipfile is > appended, this variable changes its value to point to the zipfile (it > should probably point to the zip directory, or some such). > So in essence, a proper linking step is executed. There should be no > dependence on native tools to perform this linking, instead, it should > be implemented in pure Python. I don't understand: Isn't the first sentence a contradiction of the second? How is this variable set, without calling native tools, in pure Python? And is it really a global variable accessible from C code? Thomas From theller at python.net Sat Aug 9 21:42:50 2003 From: theller at python.net (Thomas Heller) Date: Sat Aug 9 14:43:07 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: <200308090353.h793r8e12155@12-236-84-31.client.attbi.com> (Guido van Rossum's message of "Fri, 08 Aug 2003 20:53:08 -0700") References: <200308090353.h793r8e12155@12-236-84-31.client.attbi.com> Message-ID: Guido van Rossum writes: >> [Kurt B. Kaiser, struggles to get an embedded-space spawn to work] > > [Tim, can't belp helping] So, shouldn't the installer simply warn and offer to select a different path, or simply refuse to install, when the user selects a directory containing a space? Thomas From tim.one at comcast.net Sat Aug 9 16:05:26 2003 From: tim.one at comcast.net (Tim Peters) Date: Sat Aug 9 15:06:08 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: Message-ID: [Thomas Heller] > So, shouldn't the installer simply warn and offer to select a > different path, or simply refuse to install, when the user selects a > directory containing a space? More installer code == more chances to screw up, and any restriction will irritate some classes of users. IDLE in particular should be friendly to "determined idiots", in spite of themselves. By default, the installer suggests a path not only devoid of spaces, but a path equal to its 8.3 DOS name. "Long names" can create problems too, even if they're devoid of spaces (people using Python for CGI on Windows bump into that one). People unwilling to accept the experience that went into the defaults will have to learn the hard way; it's educational to let them . From kbk at shore.net Sat Aug 9 16:08:32 2003 From: kbk at shore.net (Kurt B. Kaiser) Date: Sat Aug 9 15:08:57 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: References: Message-ID: On Fri, 8 Aug 2003 23:44:15 -0400, Tim Peters wrote: > [Kurt B. Kaiser, struggles to get an embedded-space spawn to work] > > I think the trick is that you shouldn't quote the executable path at all. This is the key point. Thanks. I was trying to quote both the executable and the arguments. Now to try IDLE again.... > Under the covers, MS takes the executable name as-is, passing it as the > lpApplicationName argument to the Win32 CreateProcess(), but squashes all > your *arguments* into a single string, which gets passed on as the > lpCommandLine argument to CreateProcess(). > > In the arguments, you should avoid the cleverness of trying to quote only > the part(s) with a space(s). Stick a double quote on each end of each > argument, and that's it(*). [...] > ! Does it make sense to update the spawn* docs to include a comment that on Windows a spawn* /path/ with embedded spaces should not be quoted, but an arg with embedded spaces should be decorated with double quotes on each end? ============================================================ path with embedded space, XP: ============================================================ >>> executable = sys.executable >>> executable 'C:\\Program Files\\Python23\\python.exe' >>> decorated = '"%s"' % executable >>> decorated '"C:\\Program Files\\Python23\\python.exe"' >>> os.spawnv(os.P_NOWAIT, executable, (decorated, '-V')) 1900 >>> Python 2.3 >>> os.spawnv(os.P_NOWAIT, executable, (executable, '-V')) 1896 >>> C:\Program: can't open file 'Files\Python23\python.exe' >>> os.spawnv(os.P_NOWAIT, decorated, (decorated, '-V')) Traceback (most recent call last): File "", line 1, in ? OSError: [Errno 22] Invalid argument >>> =========================================================== ** On W98 the second version works! ** And, BTW, IDLE1.0 runs on W98 even when Python23 is installed in Program Files. Now a path w/o embedded space, XP: =========================================================== >>> executable = r'C:\python.exe' >>> executable 'C:\\python.exe' >>> decorated = '"%s"' % executable >>> decorated '"C:\\python.exe"' >>> os.spawnv(os.P_NOWAIT, executable, (decorated, '-V')) 1892 >>> Python 2.3 >>> os.spawnv(os.P_NOWAIT, executable, (executable, '-V')) 1888 >>> Python 2.3 >>> os.spawnv(os.P_NOWAIT, decorated, (decorated, '-V')) Traceback (most recent call last): File "", line 1, in ? OSError: [Errno 22] Invalid argument >>> =========================================================== Same results on W98. From tim.one at comcast.net Sat Aug 9 16:31:49 2003 From: tim.one at comcast.net (Tim Peters) Date: Sat Aug 9 15:32:24 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: Message-ID: [Kurt B. Kaiser] > ... > Does it make sense to update the spawn* docs to include a comment that > on Windows a spawn* /path/ with embedded spaces should not be quoted, > but an arg with embedded spaces should be decorated with double > quotes on each end? I don't know. We should document something that's sure to work, but I don't know what's sure to work on all versions of Windows, and can't make time to try to reverse-engineer it. The Windows docs are hopelessly ambiguous here, and all I guessed I reverse-engineered from studying the MS CRT source code supplied with MSVC 6. For all I know, it differs in critical ways in VC 7 (.NET), and I found nothing to explain observed differences in spawnv() behavior between 98SE and XP in VC6. Typical Windows docs: The reason that main and WinMain cannot return Unicode strings is that argc, argv, and lpCmdLine use the LPSTR data type for parameters, not the LPSTR data type. The GetCommandLine function can be used to access Unicode strings, because it uses the LPSTR data type. I suppose some of that might be correct . From skip at pobox.com Sat Aug 9 17:28:50 2003 From: skip at pobox.com (Skip Montanaro) Date: Sat Aug 9 17:29:00 2003 Subject: [Python-Dev] Frank was onto something Message-ID: <16181.26514.737409.678703@montanaro.dyndns.org> Frank Stajano was onto something way back when he used the term "batteries included" in reference to the Python distribution. I just noticed the TclTkAqua installer has a "batteries included" version. Skip From nhodgson at bigpond.net.au Sun Aug 10 09:38:57 2003 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Sat Aug 9 18:39:02 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 References: Message-ID: <004b01c35ec7$052cc9b0$3da48490@neil> Tim Peters: > behavior between 98SE and XP in VC6. Typical Windows docs: > > The reason that main and WinMain cannot return Unicode strings is > that argc, argv, and lpCmdLine use the LPSTR data type for > parameters, not the LPSTR data type. The GetCommandLine function > can be used to access Unicode strings, because it uses the LPSTR > data type. > > I suppose some of that might be correct . Well, if you'd just upgrade to VS .NET 2003, a mere 8 hour upgrade process over VS .NET 2002 for me: > The reason that main and WinMain cannot return Unicode strings is > that argc, argv, and lpCmdLine use the LPSTR data type for > parameters, not the LPTSTR data type. The GetCommandLine function > can be used to access Unicode strings, because it uses the LPTSTR > data type. Two extra easily-overlooked characters and it makes some more sense. Although it still contains errors, it at least touches the point it is trying to make. Neil From martin at v.loewis.de Sun Aug 10 06:53:04 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sun Aug 10 01:53:07 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: <3cgaoes7.fsf@python.net> References: <000701c35e37$e6145410$f502a8c0@eden> <16181.5523.50443.755048@montanaro.dyndns.org> <3F351C02.4090903@v.loewis.de> <3cgaoes7.fsf@python.net> Message-ID: Thomas Heller writes: > I don't understand: Isn't the first sentence a contradiction of the > second? > How is this variable set, without calling native tools, in pure Python? All that the native tool could do can also be done in pure Python. There are, typically no system calls required to a linker except for read() and write() (perhaps seek()). > And is it really a global variable accessible from C code? Yes, this is how I think it should be implemented. This would be a lot of work, no doubt. Regards, Martin From skip at mojam.com Sun Aug 10 08:00:22 2003 From: skip at mojam.com (Skip Montanaro) Date: Sun Aug 10 08:00:26 2003 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200308101200.h7AC0MM08485@manatee.mojam.com> Bug/Patch Summary ----------------- 431 open / 3995 total bugs (+5) 168 open / 2309 total patches (-4) New Bugs -------- IDE defaults to Mac linefeeds (2003-08-04) http://python.org/sf/782686 PyObject_Free unallocated memory read warning (2003-08-04) http://python.org/sf/782689 ImportError: cannot import name __all__ (2003-08-04) http://python.org/sf/782752 MacOS9: installer should test CarbonLib version (2003-08-04) http://python.org/sf/783095 test_ossaudiodev timing failure (2003-08-04) http://python.org/sf/783242 Inconsistent results with super and __getattribute__ (2003-08-05) http://python.org/sf/783528 SIGSEGV in _sre.c (IRIX 6.5.20) (2003-08-05) http://python.org/sf/783789 Tab / Space Configuration Does Not Work in IDLE (2003-08-05) http://python.org/sf/783887 test_strptime fails with duplicate DST timezone name (2003-08-05) http://python.org/sf/783952 test_re failure on 64bit targets (alpha, ia64) (2003-08-06) http://python.org/sf/783990 Byte-order bug in socket-module getaddrinfo.c (2003-08-06) http://python.org/sf/784031 Fatal Python error: unknown scope (2003-08-06) http://python.org/sf/784075 IDLE does not start for 2.3 on windows. (2003-08-06) http://python.org/sf/784183 test_repr failure on m68k-linux (2003-08-06) http://python.org/sf/784443 MacPython installer fails on UFS filesystem (2003-08-07) http://python.org/sf/785031 zlib monotonic test -- false assumptions or bug in zlib.lib (2003-08-08) http://python.org/sf/785222 urllib output: Worker thread.. (2003-08-08) http://python.org/sf/785584 posixmodule uses utimes, which is broken in glibc-2.3.2 (2003-08-10) http://python.org/sf/786194 New Patches ----------- Port tests to unittest (Part 2) (2003-05-13) http://python.org/sf/736962 pty.fork() compatibility code wrong? (2003-08-04) http://python.org/sf/783050 support for server side transactions in _ssl (2003-08-04) http://python.org/sf/783188 A program to scan python files and list those require coding (2003-08-06) http://python.org/sf/784089 getopt_long_only() (2003-08-06) http://python.org/sf/784231 fix obscure crash in descriptor handling (2003-08-07) http://python.org/sf/784825 pydoc's usage should use basename (2003-08-08) http://python.org/sf/785689 Documentation for platform module (2003-08-08) http://python.org/sf/785752 Closed Bugs ----------- python-mode and nested indents (2002-07-26) http://python.org/sf/587239 menues in emacs21 in transient mode (2002-10-10) http://python.org/sf/621554 python-mode loops on if/else (2003-02-12) http://python.org/sf/685465 python-mode.el: sexp commands don't understand Python (2003-03-17) http://python.org/sf/705005 cStringIO does not set closed attr (2003-07-13) http://python.org/sf/770485 Non-ASCII characters bugs (2003-07-20) http://python.org/sf/774680 change 0,1 to False,True in dict.has_key doc (2003-07-22) http://python.org/sf/775836 Solaris error doing a print (2003-07-22) http://python.org/sf/775985 HIDDEN in Tkconstants.py (2003-07-25) http://python.org/sf/777664 struct does not pad to alignment (2003-07-27) http://python.org/sf/778681 parameter confusion in 2.3c2 (2003-07-28) http://python.org/sf/778964 time.strptime() 1,200 times slower in python2.3 (2003-07-31) http://python.org/sf/780807 test_normalization fails (2003-07-31) http://python.org/sf/781065 py-2.3 socket.inet_pton() segfault. (2003-08-01) http://python.org/sf/781722 file.write() incorrectly tests remaining space (2003-08-02) http://python.org/sf/781891 Closed Patches -------------- test for patch #543865 & others (2002-04-14) http://python.org/sf/543867 A different patch for python-mode vs gdb (2002-06-11) http://python.org/sf/567468 Make python-mode.el use "jython" interp (2002-06-27) http://python.org/sf/574750 python-mode patch for ipython support (2002-06-30) http://python.org/sf/575774 More flexible py-shell interpreter selection (2003-01-26) http://python.org/sf/675034 python-mode.el: make C-c - work correctly after C-c | (2003-01-30) http://python.org/sf/677838 BaseHTTPServer doesn't need StringIO intermediary (2003-06-02) http://python.org/sf/747364 IRIX libmpc patch (2003-07-09) http://python.org/sf/768840 Fix for LD_LIBRARY_PATH inheritance on SunOS when -shared (2003-07-15) http://python.org/sf/771998 Updated 2.2.3 .spec file. (2003-07-16) http://python.org/sf/772696 python-mode py-execute-buffer bug (2003-07-29) http://python.org/sf/779839 fix gettext NullTranslation.add_fallback typo (2003-07-31) http://python.org/sf/781126 From oren-py-d at hishome.net Sun Aug 10 16:51:58 2003 From: oren-py-d at hishome.net (Oren Tirosh) Date: Sun Aug 10 15:52:02 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: <3F351C02.4090903@v.loewis.de> References: <000701c35e37$e6145410$f502a8c0@eden> <16181.5523.50443.755048@montanaro.dyndns.org> <3F351C02.4090903@v.loewis.de> Message-ID: <20030810195158.GA72674@hishome.net> On Sat, Aug 09, 2003 at 06:06:26PM +0200, "Martin v. Löwis" wrote: > Skip Montanaro wrote: > > >It wasn't me. I agree it seems a tad fragile, however it will probably be > >a > >lot easier than coming up with a "portable" way for the interpreter to know > >its size. I'd suggest a command line flag but I don't think that's going > >to > >fly either. > > I recommend to drop the requirement that simple appending of .zip file > needs to work, and suggest to use platform-specific mechanisms on each > platform. These mechanisms should arrange to patch a variable, > Py_EmbeddedZipfile. If the solution is patching a variable in the executable there is one variable that controls all aspects Python initialization: argv. A variable called Py_OverrideArgv will contain by default a unique signature. An external tool can search for this signature and replace it with a customized string. If not modified, the interpreter will use the real argv. If it is modified the interpreter will use it instead of argv and apply all standard argv processing including flags and -c. The only difference will be that PySys_SetArgv will still be called with the real argv to make the real arguments accessible to the script. I think the modification should be less than 20 lines of code. A tool like py2exe could take the python interpreter, copy it and append the zip archive. While copying it can search for the unique signature and replace it with something like: '-O -Whatever -c "import sys; sys.path.append(sys.executable); import boot' Oren From Jack.Jansen at cwi.nl Mon Aug 11 00:46:06 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Sun Aug 10 17:46:16 2003 Subject: [Python-Dev] Slogan: Getting Rich Overnight In-Reply-To: <200308090950.45953.gherron@islandtraining.com> Message-ID: <0BBE60D5-CB7C-11D7-889B-000A27B19B96@cwi.nl> On zaterdag, aug 9, 2003, at 18:50 Europe/Amsterdam, Gary Herron wrote: > That and things Python and Python related tipped the balance of my > spambayes towards "ham", Don't let the spammers hear this, or before you know we'll all be getting messages of the form "With our new pill your manhood can be as big[*] as a Python". [*] replace "big" with "long", "strong", "agile", "good-looking", "stamina-filled" or whatever adjective fits your personal problem with your male member. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From Jack.Jansen at cwi.nl Mon Aug 11 00:57:53 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Sun Aug 10 17:58:05 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: <000701c35e37$e6145410$f502a8c0@eden> Message-ID: On zaterdag, aug 9, 2003, at 07:34 Europe/Amsterdam, Mark Hammond wrote: > * Propose a new executable that some Python platforms can choose to > distribute - eg, 'python-package{.exe}'. This is really just > identical to > python.exe, but with the only 'if doesnt start with 'python'' parts of > your > proposal. Even though having the one Python interpreter fill a dual role is a hack I still think you should try to go for it, and I say this from experience. MacPython-OS9 has had applets (which is basically what you're after) since about 1.3 or 1.4. Initially, applets had a special main program. At some point I hacked something up (now that I think about it it may not have been me, it may have been Just who did it) so that the two main programs were united, and this really made life a lot simpler and less error-prone. On the Mac life was easy, though, because of the resource fork. The main program simply started with if (there's a 'PYC ' resource named '__main__') I'm an applet; else I'm a normal interpreter; But: now that I'm thinking this whole idea through a little bit: if this is all meant for packaging applications, how are you going to handle extension modules? These can't be put into zipfiles, or can they? (Again, MacOS had a neat hack for this: you could put multiple shared libraries and executables and more into a single file. But I don't think you can do that one most systems). -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From tdelaney at avaya.com Mon Aug 11 10:30:11 2003 From: tdelaney at avaya.com (Delaney, Timothy C (Timothy)) Date: Sun Aug 10 19:30:17 2003 Subject: [Python-Dev] Make it an error to use __slots__ with classicclasses Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DE9E00A7@au3010avexu1.global.avaya.com> > From: Terence Way [mailto:terry@wayforward.net] > > A brief test shows that PyChecker checks for this, however: > Warnings... > > testslot.py:2: Using __slots__ in classic class foo has no effect, > consider deriving from object That seems sufficient to me. I think the right approach is educating people to use PyChecker. Tim Delaney From tismer at tismer.com Mon Aug 11 04:19:13 2003 From: tismer at tismer.com (Christian Tismer) Date: Sun Aug 10 21:19:25 2003 Subject: [Python-Dev] Slogan: Getting Rich Overnight In-Reply-To: <0BBE60D5-CB7C-11D7-889B-000A27B19B96@cwi.nl> References: <0BBE60D5-CB7C-11D7-889B-000A27B19B96@cwi.nl> Message-ID: <3F36EF11.3080500@tismer.com> Jack Jansen wrote: ... > Don't let the spammers hear this, or before you know we'll all be getting > messages of the form "With our new pill your manhood can be as big[*] as a > Python". If we can't make a simple claim like this without being hosed by the spammers and *their* filters (since *we* allowed them to grow after them), then we are very badly hosed. Spammers are dictating our subject lines. Do you allow this? Really? I think we should stop spamming now. We can do that and we should do that. How? Don't ask these silly questions. We all know what to do, and the whole spamming problem could be over in, say, three weeks. Spamming is mega-out. Just let them feel this! -- Christian Tismer :^) Mission Impossible 5oftware : Have a break! Take a ride on Python's Johannes-Niemeyer-Weg 9a : *Starship* http://starship.python.net/ 14109 Berlin : PGP key -> http://wwwkeys.pgp.net/ work +49 30 89 09 53 34 home +49 30 802 86 56 pager +49 173 24 18 776 PGP 0x57F3BF04 9064 F4E1 D754 C2FF 1619 305B C09C 5A3B 57F3 BF04 whom do you want to sponsor today? http://www.stackless.com/ From tim.one at comcast.net Mon Aug 11 01:23:01 2003 From: tim.one at comcast.net (Tim Peters) Date: Mon Aug 11 00:23:36 2003 Subject: [Python-Dev] Outlook release on SourceForge Message-ID: [pyton-dev'ers, read the last paragraph and weep ] I created a "Outlook Addin" package on the spambayes file release page, and added Mark's 0.7 installer to it: http://sf.net/project/showfiles.php?group_id=61702 I also ripped off much of the text from Mark's Starship page (this is what you get to if you click on the "Version 0.7" on the page above): http://sf.net/project/shownotes.php?release_id=177282 Mark, if you object to any of this, don't be shy! Meaningless statistics : Project UNIX name: spambayes Registered: 2002-09-03 22:33 Activity Percentile (last week): 99.4015% "Activity ratings" on SF are computed by an arcane formula, but generally speaking the higher the percentile the more "active" a project is. With the percentile above, this is how many SF projects rank below spambayes: >>> round(66620 * 0.994015) 66221.0 >>> In particular, we rank higher than Python(!) now, which currently sits at 99.0583%. Python used to be among the 10 most active projects every week, but lost a lot when it stopped releasing files from SF (downloads count a lot toward the activity ranking). It's possible that downloads of the Outlook addin could boost spambayes to that lofty level. It's also possible that I'll get a job working on Python someday . From theller at python.net Mon Aug 11 11:22:20 2003 From: theller at python.net (Thomas Heller) Date: Mon Aug 11 04:23:06 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: (Jack Jansen's message of "Sun, 10 Aug 2003 23:57:53 +0200") References: Message-ID: Jack Jansen writes: > On zaterdag, aug 9, 2003, at 07:34 Europe/Amsterdam, Mark Hammond wrote: >> * Propose a new executable that some Python platforms can choose to >> distribute - eg, 'python-package{.exe}'. This is really just >> identical to >> python.exe, but with the only 'if doesnt start with 'python'' parts >> of your >> proposal. > > Even though having the one Python interpreter fill a dual role is a hack > I still think you should try to go for it, and I say this from > experience. > > MacPython-OS9 has had applets (which is basically what you're after) > since about 1.3 or 1.4. Initially, applets had a special main > program. At some point I hacked something up (now that I think about > it it may not have been me, it may have been Just who did it) so that > the two main programs were united, and this really made life a lot > simpler and less error-prone. > > On the Mac life was easy, though, because of the resource fork. The main > program simply started with > if (there's a 'PYC ' resource named '__main__') > I'm an applet; > else > I'm a normal interpreter; And, IMO, that's also the way to go on Windows. > But: now that I'm thinking this whole idea through a little bit: if > this is all meant for packaging applications, how are you going to > handle extension modules? These can't be put into zipfiles, or can > they? No, they cannot. py2exe currently only has a 'single directory' target - it creates a directory which must be distributed as whole, containing an app.exe, python22.dll, and zlib.pyd, _sre.pyd, and maybe other extension modules. Although the zipfile makes sure that the extensions are loaded from this directory and not from somewhere else. That's good enough for me at the moment, although single file executables would be nicer. Thomas From mwh at python.net Mon Aug 11 12:02:16 2003 From: mwh at python.net (Michael Hudson) Date: Mon Aug 11 06:02:19 2003 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects typeobject.c, 2.242, 2.243 In-Reply-To: ("Tim Peters"'s message of "Sat, 9 Aug 2003 00:55:45 -0400") References: Message-ID: <2mbruwqzpz.fsf@starship.python.net> "Tim Peters" writes: > [Michael Hudson] >> This is deeply weird. Watch: >> >> [mwh@pc150 build-debug]$ ./python.exe >> Python 2.4a0 (#25, Aug 8 2003, 14:17:07) >> [GCC 3.1 20020420 (prerelease)] on darwin >> Type "help", "copyright", "credits" or "license" for more information. >>>>> from test import test_support >> [40915 refs] >>>>> test_support.verbose = 0 >> [41028 refs] > > We're different already: my total refs stay at a comparatively slim 21426 > after both initial lines. Maybe you're tracking a leak due to GNU readline? Perhaps. I should try running under -E, I guess. But I don't think so. > As for the rest, mine leaks 166 references per stab without variation: [...] > Hmm -- I'm not sure my 2.4 build is entirely up to date, though. I don't think so -- I fixed a big bundle of leaks in my assigning-to-__bases__ code just a few days ago. I'll try and dig myself, but it's too bloody hot to think here at the moment... Cheers, mwh -- ZAPHOD: You know what I'm thinking? FORD: No. ZAPHOD: Neither do I. Frightening isn't it? -- The Hitch-Hikers Guide to the Galaxy, Episode 11 From mwh at python.net Mon Aug 11 12:28:07 2003 From: mwh at python.net (Michael Hudson) Date: Mon Aug 11 06:28:10 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: ("Tim Peters"'s message of "Fri, 8 Aug 2003 23:44:15 -0400") References: Message-ID: <2m7k5kqyiw.fsf@starship.python.net> "Tim Peters" writes: > (*) There the fun begins. While os.system() spawns a command shell, > I don't think spawnv() does. Which would be a real help here. > Win9x use command.com by default, and Windows after that use > cmd.exe. They have have different quoting rules, and cmd.exe > actually has two different sets of quoting rules (triggered by > an option to cmd.exe, and/or by a registry setting). > > Still, I haven't tried the program above on a Win2K (etc) > system, and somebody should before you get optimistic. In one > of cmd.exe's quoting nightmare modes, whether double quotes > get stripped actually depends on the content of the quoted > string ! And I thought bash was insane (actually bash quoting *is* insane -- windows just manages to be worse. I'm impressed). Cheers, mwh -- FORD: Just pust the fish in your ear, come on, it's only a little one. ARTHUR: Uuuuuuuuggh! -- The Hitch-Hikers Guide to the Galaxy, Episode 1 From mwh at python.net Mon Aug 11 12:34:22 2003 From: mwh at python.net (Michael Hudson) Date: Mon Aug 11 06:34:24 2003 Subject: [Python-Dev] Outlook release on SourceForge In-Reply-To: ("Tim Peters"'s message of "Mon, 11 Aug 2003 00:23:01 -0400") References: Message-ID: <2m4r0oqy8h.fsf@starship.python.net> "Tim Peters" writes: > In particular, we rank higher than Python(!) now, which currently sits at > 99.0583%. Python used to be among the 10 most active projects every week, > but lost a lot when it stopped releasing files from SF (downloads count a > lot toward the activity ranking). Yep. However, the pain of dealing with the SF file release thingy compared to scp-ing things to creosote is night and day. *I'm* pretty sure having a high activity rating on SF isn't worth it. > It's possible that downloads of the Outlook addin could boost > spambayes to that lofty level. It's also possible that I'll get a > job working on Python someday . I seem to recall Python got back into the top 6 during a week when Brett seemingly did nothing else but comment on bug reports... Cheers, mwh -- I'm not sure that the ability to create routing diagrams similar to pretzels with mad cow disease is actually a marketable skill. -- Steve Levin -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html From pedronis at bluewin.ch Mon Aug 11 15:38:16 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Mon Aug 11 08:37:18 2003 Subject: [Python-Dev] sizeof(long) != sizeof(void*) In-Reply-To: <5.2.1.1.0.20030807090514.026b83f8@pop.bluewin.ch> References: <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <20030806142330.A32662@starship.python.net> <5.2.1.1.0.20030806211139.02692178@pop.bluewin.ch> <5.2.1.1.0.20030806224144.02692178@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030811143720.02694200@pop.bluewin.ch> At 09:13 07.08.2003 +0200, Samuele Pedroni wrote: >At 08:23 07.08.2003 +0200, Martin v. L?wis wrote: I overreacted, sorry. Samuele. From mwh at python.net Mon Aug 11 14:41:05 2003 From: mwh at python.net (Michael Hudson) Date: Mon Aug 11 08:41:08 2003 Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects typeobject.c,2.242,2.243 In-Reply-To: <2mn0ekqm2n.fsf@starship.python.net> (Michael Hudson's message of "Fri, 08 Aug 2003 15:08:00 +0100") References: <2mn0ekqm2n.fsf@starship.python.net> Message-ID: <2mhe4opdsu.fsf@starship.python.net> Michael Hudson writes: > mwh@users.sourceforge.net writes: > >> Update of /cvsroot/python/python/dist/src/Objects >> In directory sc8-pr-cvs1:/tmp/cvs-serv29176 >> >> Modified Files: >> typeobject.c >> Log Message: >> /* XXX From here until type is allocated, "return NULL" leaks bases! */ >> >> Sure looks like it to me! >> >> When I run the leak2.py script I posted to python-dev, I only see >> three reference leaks in all of test_descr. When I run >> test_descr.test_main, I still see 46 leaks. This clearly demands >> posting a yelp to python-dev :-) This turns out to be at least something of a false alarm. The details are boring (and are due to test_descr doing funky things with sys.stdout...) Sorry for the noise. Cheers, mwh -- ARTHUR: Why should a rock hum? FORD: Maybe it feels good about being a rock. -- The Hitch-Hikers Guide to the Galaxy, Episode 8 From jepler at unpythonic.net Mon Aug 11 09:44:24 2003 From: jepler at unpythonic.net (Jeff Epler) Date: Mon Aug 11 09:44:31 2003 Subject: [Python-Dev] Re: FWD: Re: Problem w/ IDLE on Win2000 In-Reply-To: <2m7k5kqyiw.fsf@starship.python.net> References: <2m7k5kqyiw.fsf@starship.python.net> Message-ID: <20030811134424.GH6517@unpythonic.net> On Mon, Aug 11, 2003 at 11:28:07AM +0100, Michael Hudson wrote: > And I thought bash was insane (actually bash quoting *is* > insane -- windows just manages to be worse. I'm impressed). bash quoting is well-documented (and surely sh-quoting is posix standard), and with just a few good habits you'll rarely get tripped up, even on filenames and options with embedded whitespace (which nobody sane will use, anyway). I'd say the level of care is not too different from the level that tcl requires. And bash isn't involved in os.exec*/spawn*, thank god. Jeff From koenig at science-computing.de Mon Aug 11 16:58:29 2003 From: koenig at science-computing.de (Harald Koenig) Date: Mon Aug 11 09:58:35 2003 Subject: [Python-Dev] python 2.3 bug patch Message-ID: <20030811135829.GA8249@turtle.science-computing.de> Hi, compiling python-2.3 on various platforms I needed the attached patch. the first two patches to Makefile.pre.in and Modules/makesetup are needed if source dir and compile dir are not the same. Modules/resource.c patch fixes a problem for Solaris 2.5.1 which doesn't define _SC_PAGE_SIZE but _SC_PAGESIZE in unistd.h. other than those small patches, python-2.3 compiled fine on the following systems using either gcc-2.95.3 or gcc-3.3: alpha.OSF1-V4.0 hp9000s700.HP-UX-B.10.20 hp9000s700.HP-UX-B.11.00 i386-linux.RedHat-6.2 i386-linux.RedHat-7.1 i386-linux.suse62 i386-linux.suse72 iris4d.IRIX64-6.5 rs6000.AIX-4.3 sun4.SunOS-5.5.1 sun4.SunOS-5.8 Harald Koenig -- "I hope to die ___ _____ before I *have* to use Microsoft Word.", 0--,| /OOOOOOO\ Donald E. Knuth, 02-Oct-2001 in Tuebingen. <_/ / /OOOOOOOOOOO\ \ \/OOOOOOOOOOOOOOO\ \ OOOOOOOOOOOOOOOOO|// Harald Koenig \/\/\/\/\/\/\/\/\/ science+computing ag // / \\ \ koenig@science-computing.de ^^^^^ ^^^^^ -------------- next part -------------- --- Python-2.3/Makefile.pre.in~ Thu Aug 7 14:36:12 2003 +++ Python-2.3/Makefile.pre.in Thu Aug 7 14:35:44 2003 @@ -56,7 +56,7 @@ OPT= @OPT@ BASECFLAGS= @BASECFLAGS@ CFLAGS= $(BASECFLAGS) $(OPT) -CPPFLAGS= -I. -I$(srcdir)/Include +CPPFLAGS= -I. -IInclude -I$(srcdir)/Include LDFLAGS= @LDFLAGS@ LDLAST= @LDLAST@ SGI_ABI= @SGI_ABI@ @@ -432,6 +432,7 @@ $(GRAMMAR_H) $(GRAMMAR_C): $(PGEN) $(GRAMMAR_INPUT) + -@ mkdir Include -$(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C) $(PGEN): $(PGENOBJS) --- Python-2.3/Modules/makesetup~ Fri Mar 29 19:00:18 2002 +++ Python-2.3/Modules/makesetup Thu Aug 7 14:30:31 2003 @@ -238,7 +238,7 @@ no) SHAREDMODS="$SHAREDMODS $file";; esac rule="$file: $objs" - rule="$rule; \$(LDSHARED) $objs $libs $ExtraLibs -o $file" + rule="$rule; \$(BLDSHARED) $objs $libs $ExtraLibs -o $file" echo "$rule" >>$rulesf done done --- Python-2.3/Modules/resource.c~ Sun Mar 30 22:51:29 2003 +++ Python-2.3/Modules/resource.c Mon Aug 11 15:44:34 2003 @@ -202,7 +202,11 @@ #if defined(HAVE_GETPAGESIZE) pagesize = getpagesize(); #elif defined(HAVE_SYSCONF) +#if defined(_SC_PAGESIZE) /* for Solaris 2.5.1 */ + pagesize = sysconf(_SC_PAGESIZE); +#else pagesize = sysconf(_SC_PAGE_SIZE); +#endif #endif return Py_BuildValue("i", pagesize); From aahz at pythoncraft.com Mon Aug 11 11:28:12 2003 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 11 10:28:15 2003 Subject: [Python-Dev] python 2.3 bug patch In-Reply-To: <20030811135829.GA8249@turtle.science-computing.de> References: <20030811135829.GA8249@turtle.science-computing.de> Message-ID: <20030811142811.GA18428@panix.com> On Mon, Aug 11, 2003, Harald Koenig wrote: > > compiling python-2.3 on various platforms I needed the attached patch. Please submit a patch to SourceForge. We lose track of patches sent here. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From skip at pobox.com Mon Aug 11 10:43:07 2003 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 11 10:43:25 2003 Subject: [Python-Dev] python 2.3 bug patch In-Reply-To: <20030811135829.GA8249@turtle.science-computing.de> References: <20030811135829.GA8249@turtle.science-computing.de> Message-ID: <16183.43899.846057.884164@montanaro.dyndns.org> Harald, Thanks for the input. Could you file two bug reports for this on SourceForge though? I'm skeptical that the -IInclude change is necessary, and any discussion of that should be captured together so it's not lost. The _SC_PAGE(_)?SIZE change might be better handled by adding a new test in the configure script. Solaris 2.5.1 is a pretty minor platform these days. I'd prefer to clutter up the configure script instead of the C code. (Martin v. L?wis may have a comment on that though.) Skip From neal at metaslash.com Mon Aug 11 11:53:55 2003 From: neal at metaslash.com (Neal Norwitz) Date: Mon Aug 11 10:54:37 2003 Subject: [Python-Dev] python 2.3 bug patch In-Reply-To: <16183.43899.846057.884164@montanaro.dyndns.org> References: <20030811135829.GA8249@turtle.science-computing.de> <16183.43899.846057.884164@montanaro.dyndns.org> Message-ID: <20030811145354.GW13215@epoch.metaslash.com> On Mon, Aug 11, 2003 at 09:43:07AM -0500, Skip Montanaro wrote: > (Martin v. L?wis may have a comment on that though.) FYI, I think Martin has left for vacation. Neal From theller at python.net Mon Aug 11 18:17:34 2003 From: theller at python.net (Thomas Heller) Date: Mon Aug 11 11:18:22 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: <3F351C02.4090903@v.loewis.de> ( =?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sat, 09 Aug 2003 18:06:26 +0200") References: <000701c35e37$e6145410$f502a8c0@eden> <16181.5523.50443.755048@montanaro.dyndns.org> <3F351C02.4090903@v.loewis.de> Message-ID: "Martin v. L?wis" writes: > Skip Montanaro wrote: > >> It wasn't me. I agree it seems a tad fragile, however it will probably be a >> lot easier than coming up with a "portable" way for the interpreter to know >> its size. I'd suggest a command line flag but I don't think that's going to >> fly either. > > I recommend to drop the requirement that simple appending of .zip file > needs to work, and suggest to use platform-specific mechanisms on each > platform. These mechanisms should arrange to patch a variable, > Py_EmbeddedZipfile. Normally, this variable is NULL. If a zipfile is > appended, this variable changes its value to point to the zipfile (it > should probably point to the zip directory, or some such). > > So in essence, a proper linking step is executed. There should be no > dependence on native tools to perform this linking, instead, it should > be implemented in pure Python. Isn't it possible to parse the executable file format, looking for the location where the Py_EmbeddedZipfile variable is located, and change it's value? That's what I understand when you say 'patch', and I wouldn't call this 'linking'. Thomas From jepler at unpythonic.net Mon Aug 11 12:47:11 2003 From: jepler at unpythonic.net (Jeff Epler) Date: Mon Aug 11 12:47:56 2003 Subject: [Python-Dev] Slogan: Getting Rich Overnight In-Reply-To: <0BBE60D5-CB7C-11D7-889B-000A27B19B96@cwi.nl> References: <200308090950.45953.gherron@islandtraining.com> <0BBE60D5-CB7C-11D7-889B-000A27B19B96@cwi.nl> Message-ID: <20030811164709.GI6517@unpythonic.net> | Subject: urgent "len(dict)" doubt | | When you "import" our new pill, your "member function" can be as | "high-level" as "Python". If you're having problems with "rapid | development" when practicing "extreme programming" with your partner, we | "assert" that our product is also a "design pattern" you should consider. | "__All__" of our customers recieve, "PyObject_Free", a 3-month supply of | "rexec/bastion" "LaTeX" sheaths, a 100% effective "mutex" for people who | engauge in "threading". If you want to be "garbage collected" from this | "list", simply "Py_DECREF" this URL: -- hamming it up with spambayes Jeff From bac at OCF.Berkeley.EDU Mon Aug 11 13:16:00 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 11 15:16:17 2003 Subject: [Python-Dev] Outlook release on SourceForge In-Reply-To: <2m4r0oqy8h.fsf@starship.python.net> References: <2m4r0oqy8h.fsf@starship.python.net> Message-ID: <3F37EB70.5060504@ocf.berkeley.edu> Michael Hudson wrote: > "Tim Peters" writes: > > >>In particular, we rank higher than Python(!) now, which currently sits at >>99.0583%. Python used to be among the 10 most active projects every week, >>but lost a lot when it stopped releasing files from SF (downloads count a >>lot toward the activity ranking). > > > Yep. However, the pain of dealing with the SF file release thingy > compared to scp-ing things to creosote is night and day. *I'm* pretty > sure having a high activity rating on SF isn't worth it. > It is kind of nice just for the sake of catching people's attention and making it not seem like these other projects are busy while Python is trudging along at a snail's pace. I doubt very many people realize how the ratings work, let alone why ours is lower than it should be. But I agree with Michael that using scp instead of SF for releasing files is *so* much better. > >>It's possible that downloads of the Outlook addin could boost >>spambayes to that lofty level. It's also possible that I'll get a >>job working on Python someday . > > > I seem to recall Python got back into the top 6 during a week when > Brett seemingly did nothing else but comment on bug reports... > Yep, that's right. That was when I was going through the RFCs and bug reports. I think I plowed through 150 tracker items in a day or two. Still have to do the patches. Doubt those will go as fast, though. -Brett From tim.one at comcast.net Mon Aug 11 17:48:51 2003 From: tim.one at comcast.net (Tim Peters) Date: Mon Aug 11 16:49:20 2003 Subject: [Python-Dev] Outlook release on SourceForge In-Reply-To: <2m4r0oqy8h.fsf@starship.python.net> Message-ID: [Tim] >> Python used to be among the 10 most active projects every week, >> but lost a lot when it stopped releasing files from SF >> (downloads count a lot toward the activity ranking). [Michael Hudson] > Yep. However, the pain of dealing with the SF file release thingy > compared to scp-ing things to creosote is night and day. I find setting up an SF file release to be obscure but actually very quick and easy once you figure it out. I don't know how hard people find it to download from there now; it used to be dead in the water too often. > *I'm* pretty sure having a high activity rating on SF isn't worth it. A fine case of sour grapes if ever I heard one . the-spambayes-folks-trade-activity-points-for-fast-cars-mansions- and-recreational-drugs-ly y'rs - tim From mwh at python.net Tue Aug 12 12:02:26 2003 From: mwh at python.net (Michael Hudson) Date: Tue Aug 12 06:02:30 2003 Subject: [Python-Dev] Outlook release on SourceForge In-Reply-To: ("Tim Peters"'s message of "Mon, 11 Aug 2003 16:48:51 -0400") References: Message-ID: <2m8ypzry6l.fsf@starship.python.net> "Tim Peters" writes: > [Tim] >>> Python used to be among the 10 most active projects every week, >>> but lost a lot when it stopped releasing files from SF >>> (downloads count a lot toward the activity ranking). > > [Michael Hudson] >> Yep. However, the pain of dealing with the SF file release thingy >> compared to scp-ing things to creosote is night and day. > > I find setting up an SF file release to be obscure but actually very quick > and easy once you figure it out. Maybe I just never figured it out, then. Perhaps we should go back to putting full releases up there -- but I found dealing with that thing a serious hassle for 221c1 & 2. > I don't know how hard people find it to download from there now; it > used to be dead in the water too often. Well, *I* have a much faster connection to python.org than any of SFs servers, though this may be an accident of routing (ping from here to python.org is only 11 msec). >> *I'm* pretty sure having a high activity rating on SF isn't worth it. > > A fine case of sour grapes if ever I heard one . > > the-spambayes-folks-trade-activity-points-for-fast-cars-mansions- > and-recreational-drugs-ly y'rs - tim Whoa! That sounds like a better inducement than appearing on SF's front page :-) Cheers, mwh -- MGM will not get your whites whiter or your colors brighter. It will, however, sit there and look spiffy while sucking down a major honking wad of RAM. -- http://www.xiph.org/mgm/ From amk at amk.ca Tue Aug 12 07:56:42 2003 From: amk at amk.ca (A.M. Kuchling) Date: Tue Aug 12 07:47:54 2003 Subject: [Python-Dev] Updating PEP 308 Message-ID: PEP 308, the ternary operator PEP, doesn't specify the results of the vote and still has the status of "Draft". Anyone want to update it to include the vote results and change it to "Rejected"? --amk From guido at python.org Tue Aug 12 07:59:09 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 12 10:13:29 2003 Subject: [Python-Dev] Updating PEP 308 In-Reply-To: Your message of "Tue, 12 Aug 2003 06:56:42 EDT." References: Message-ID: <200308121359.h7CDx9T32148@12-236-84-31.client.attbi.com> > PEP 308, the ternary operator PEP, doesn't specify the results of the > vote and still > has the status of "Draft". Anyone want to update it to include the > vote results > and change it to "Rejected"? If Raymond still has the vote results he's welcome to do it. BTW the motivation was "when in doubt, don't change the status quo" or "there was no *overwhelming* majority for change". --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Aug 12 12:05:14 2003 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 12 11:09:18 2003 Subject: [Python-Dev] Updating PEP 308 References: <200308121359.h7CDx9T32148@12-236-84-31.client.attbi.com> Message-ID: <001801c360e3$23e5a0c0$89ac2c81@oemcomputer> > > PEP 308, the ternary operator PEP, doesn't specify the results of the > > vote and still > > has the status of "Draft". Anyone want to update it to include the > > vote results > > and change it to "Rejected"? > > If Raymond still has the vote results he's welcome to do it. BTW the > motivation was "when in doubt, don't change the status quo" or "there > was no *overwhelming* majority for change". Will update and close the PEP. Raymond From tree at basistech.com Tue Aug 12 16:05:15 2003 From: tree at basistech.com (Tom Emerson) Date: Tue Aug 12 15:10:37 2003 Subject: [Python-Dev] pyconfig.h installation issue Message-ID: <16185.14955.742957.657483@magrathea.basistech.com> Greetings, I am building Python 2.3 in an heterogenous environment where I want all of the platform-independent code to be shared and the platform specific code kept segregated. In particular I have a structure like /foo/bar/python/include /foo/bar/python/lib /foo/bar/python/man for the platform independent files, and /foo/bar/python/RH72-gcc-3.2 /foo/bar/python/sol-CC ... for platform-specific directories and files (bin/, include/, and lib/). Getting this to work is easily accomplished with --prefix and --exec-prefix when running configure on each platform: --prefix=/foo/bar/python --exec-prefix=/foo/bar/python/RH72-gcc-3.2 But... These builds are put under source control and are read-only once they are checked in. Hence the problem: If /foo/bar/python/include and its contents are read-only then following installs fail to put pyconfig.h (a platform-dependent file) into the platform-specific include directory because the $(INSTALL_DATA) in the 'inclinstall' target fails and the final line in that target (to install pyconfig.h) never runs. Clear as mud? The main thing I want to do is avoid reinstalling the shared files after the first time, since I'm building this on many different platforms. My proposed solution to this is to move the $(INSTALL_DATA) for pyconfig.h from the bottom of the inclinstall target to before the loop that installs the shared headers. Then that step will succeed and it doesn't matter whether or not the other INSTALL_DATA calls fail. Make sense? This begs another install target, which just installs the items that end up in the exec-prefix directories. Then on each platform I can just install the platform-specific code. Thoughts? Thanks in advance, tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From tim.one at comcast.net Tue Aug 12 23:08:54 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 12 22:09:27 2003 Subject: [Python-Dev] Outlook release on SourceForge In-Reply-To: <2m8ypzry6l.fsf@starship.python.net> Message-ID: [Tim] >> I find setting up an SF file release to be obscure but actually very >> quick and easy once you figure it out. [Michael Hudson] > Maybe I just never figured it out, then. Perhaps we should go back to > putting full releases up there -- but I found dealing with that thing > a serious hassle for 221c1 & 2. I'm not pushing for releasing Python on SF again -- while I think that would work OK now, it would be pointless thrashing (a change without good reason). OTOH, I think it helps spambayes to climb the SF rankings because that's a way to get some publicity for a project that isn't nearly as well known as Python. Yet . From info at kandidates.com Tue Aug 12 18:17:14 2003 From: info at kandidates.com (Kandidates) Date: Wed Aug 13 09:57:25 2003 Subject: [Python-Dev] JOBS: Python Technical Lead & Head of Data Quality Assurance, NYC Message-ID: <041801c36117$1a856e00$0300a8c0@nyc.rr.com> If you are interested in a position, please review the requirements and reply with your salary requirement. My clients main concern with these positions is Python. Elaborating on your knowledge of Python when replying will greatly increase your chances of landing an interview. JOB #1: Python Technical Lead, New York, NY DETAILS: http://www.nyc-search.com/jobs/python.html JOB #2: Head of Data Quality Assurance, New York, NY DETAILS: http://www.nyc-search.com/jobs/headQA.html Thank you, Beau J. Gould Kandidates.com Python Jobs Yahoo Group >> http://groups.yahoo.com/group/pythonjobs -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20030812/4117df6f/attachment.htm From info at kandidates.com Tue Aug 12 18:19:45 2003 From: info at kandidates.com (Kandidates) Date: Wed Aug 13 09:57:57 2003 Subject: [Python-Dev] JOBS: Python Technical Lead & Head of Data Quality Assurance, NYC Message-ID: <042f01c36117$7459f360$0300a8c0@nyc.rr.com> If you are interested in a position, please review the requirements and reply with your salary requirement. My clients main concern with these positions is Python. Elaborating on your knowledge of Python when replying will greatly increase your chances of landing an interview. JOB #1: Python Technical Lead, New York, NY DETAILS: http://www.nyc-search.com/jobs/python.html JOB #2: Head of Data Quality Assurance, New York, NY DETAILS: http://www.nyc-search.com/jobs/headQA.html Thank you, Beau J. Gould Kandidates.com Python Jobs Yahoo Group >> http://groups.yahoo.com/group/pythonjobs From aahz at pythoncraft.com Wed Aug 13 11:05:13 2003 From: aahz at pythoncraft.com (Aahz) Date: Wed Aug 13 10:05:18 2003 Subject: [Python-Dev] JOBS: Python Technical Lead & Head of Data Quality Assurance, NYC In-Reply-To: <041801c36117$1a856e00$0300a8c0@nyc.rr.com> References: <041801c36117$1a856e00$0300a8c0@nyc.rr.com> Message-ID: <20030813140513.GA13265@panix.com> python-dev is not an appropriate place to send job ads. Please desist. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From vivianwang at tcindex.com Wed Aug 13 13:17:26 2003 From: vivianwang at tcindex.com (Vivian Wang) Date: Wed Aug 13 12:17:43 2003 Subject: [Python-Dev] Runtime Error about fileinput.py Message-ID: <5.1.0.14.2.20030813120937.00b32c50@notes.tcindex.com> I got this error, what should I do? File "os.py", line 43, in ? prepdata(datapath+subcode+'.g1cp',datapath+subcode+'.out'); File "/usr/programs/python/osbatch/generic/func6.py", line 176, in prepdata for line in fileinput.input(g1cpin): File "/usr/lib/python2.2/fileinput.py", line 94, in input raise RuntimeError, "input() already active" RuntimeError: input() already active in os.py #! /usr/local/bin/python2.2 import string, re, time, fileinput, _mysql, os, posix, sys, ftplib, MySQLdb; execfile( '/usr/programs/python/osbatch/generic/func6.py') in /usr/programs/python/osbatch/generic/func6.py def prepdata(g1cpin,g1cpout): fo = open(g1cpout,'w') for line in fileinput.input(g1cpin): seqno = string.zfill(str(fileinput.filelineno()),8); zip1 = line[0] zip = line[0:5] fo.close() return From jepler at unpythonic.net Wed Aug 13 12:34:11 2003 From: jepler at unpythonic.net (Jeff Epler) Date: Wed Aug 13 12:34:15 2003 Subject: [Python-Dev] Runtime Error about fileinput.py In-Reply-To: <5.1.0.14.2.20030813120937.00b32c50@notes.tcindex.com> References: <5.1.0.14.2.20030813120937.00b32c50@notes.tcindex.com> Message-ID: <20030813163410.GA22290@unpythonic.net> it's not necessarily related to your problem, but it's a bad idea to give your program the name "XXX.py" when XXX is the name of a standard Python module. The error message is intended to indicate that you've called fileinput.input() before, but then you didn't consume all the lines available. For instance, this program shows a very similar traceback: #!/usr/bin/python import fileinput x = fileinput.input("/etc/fstab") x.readline() y = fileinput.input("/etc/fstab") $ python /tmp/vivian.py Traceback (most recent call last): File "/tmp/vivian.py", line 5, in ? y = fileinput.input("/etc/fstab") File "/usr/lib/python2.2/fileinput.py", line 101, in input raise RuntimeError, "input() already active" RuntimeError: input() already active Jeff From allison at sumeru.stanford.EDU Wed Aug 13 12:23:21 2003 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Wed Aug 13 14:23:34 2003 Subject: [Python-Dev] Building 2.1.3 on RH9 Message-ID: Seems to be broken -- at least _socket does not build correctly because of openssl changes. (Arggh... ) Where should this be reported? From info at kandidates.com Wed Aug 13 15:23:24 2003 From: info at kandidates.com (Kandidates) Date: Wed Aug 13 14:23:46 2003 Subject: [Python-Dev] Re: Python Jobs Message-ID: <016801c361c7$fc733260$0300a8c0@nyc.rr.com> Sorry for posting so many times. When I posted the job yesterday, it came back to me. It said the list was not active and I thought perhaps I should change the email format to plain text and I got the same message. Later on I posted again just to see if it was working and it was not. Today it seems all the jobs got sent to the list and I apologize for any inconvenience this may have caused the list members. On another note, if any Python people are looking for work I would like to get a copy of your resume and rate and/or salary requirement. I want to start marketing Python devs/programmers to the hiring community. I want to assure anyone who is interested that your name/contact information will never be revealed to prospective employers until you say it's ok. Thank you, Beau J. Gould Kandidates.com From skip at pobox.com Wed Aug 13 14:56:03 2003 From: skip at pobox.com (Skip Montanaro) Date: Wed Aug 13 14:56:14 2003 Subject: [Python-Dev] Building 2.1.3 on RH9 In-Reply-To: References: Message-ID: <16186.35267.394816.456702@montanaro.dyndns.org> Dennis> Seems to be broken -- at least _socket does not build correctly Dennis> because of openssl changes. (Arggh... ) Dennis> Where should this be reported? Well, presuming it will be fixed, the correct place to report it is on the Bugs page at http://sourceforge.net/projects/python/ However, the 2.1 series is getting to be pretty ancient history. Even the Python Business Forum, when scouting around for a "stable" version of Python to use as py-in-a-tie chose 2.2.x. There is almost certainly not going to be a 2.1.4 (unless you volunteer to be the release manager). 2.3 was released a couple weeks ago. You should probably consider upgrading to 2.3 (or at least 2.2.3). If that's not possible, you might want to look at the _socket code in 2.2.3 or 2.3 for hints about how you can get the 2.1.3 version to compile. Skip From skip at pobox.com Wed Aug 13 15:18:15 2003 From: skip at pobox.com (Skip Montanaro) Date: Wed Aug 13 15:18:25 2003 Subject: [Python-Dev] PyMapping_Check Bug? (fwd) Message-ID: <16186.36599.22737.157720@montanaro.dyndns.org> (muffed the first send...) >>>>> "Guido" == Guido van Rossum writes: >> Passing along a fragment from the psycopg list where Federico Di Gregorio >> (fog@initd.org) wrote: >> >> .... because on python 2.3 PyMapping_Check apperently return True for >> every new-type instance.... >> >> Is this true? If so, is it a bug? Guido> I don't see this. operator.isMappingType() maps directly to Guido> PyMapping_Check(); and I tried this: >>>> import operator >>>> operator.isMappingType(str) Guido> False >>>> class C(object): pass Guido> ... >>>> operator.isMappingType(C) Guido> False >>>> operator.isMappingType(C()) Guido> False >>>> operator.isMappingType({}) Guido> True >>>> Guido> Looks okay to me. :-) Guido> --Guido van Rossum (home page: http://www.python.org/~guido/) Good to hear, and straight from the horse's mouth! Nice to know about operator.isMappingType as well. I had no idea that it existed and didn't want to write C just to check this. Skip From allison at sumeru.stanford.EDU Wed Aug 13 13:22:33 2003 From: allison at sumeru.stanford.EDU (Dennis Allison) Date: Wed Aug 13 15:22:54 2003 Subject: [Python-Dev] Building 2.1.3 on RH9 In-Reply-To: <16186.35267.394816.456702@montanaro.dyndns.org> Message-ID: Skip -- I love the 2.3 release, but I need the 2.1.3 engine for Zope. Unfortunately, Zope is locked into 2.1.3. Apoparently this is because of incompatibilities with old vs. new classes, etc. etc. 2.2 and later is unsupported. All the docs say NO! but rumor has it that it works. RH9 seems to have introduced all sorts of grubby stuff into the mix including big changes in SSL and a threads library which breaks a lot of things. It's frustrating to see Zope come up and just stop.... I'll try running with 2.2 and/or 2.3 to see if they work--then I guess I'll have to reinstall RH7.3 which I know is stable.... I'll file a bug report Thanks. -dra On Wed, 13 Aug 2003, Skip Montanaro wrote: > > Dennis> Seems to be broken -- at least _socket does not build correctly > Dennis> because of openssl changes. (Arggh... ) > > Dennis> Where should this be reported? > > Well, presuming it will be fixed, the correct place to report it is on the > Bugs page at > > http://sourceforge.net/projects/python/ > > However, the 2.1 series is getting to be pretty ancient history. Even the > Python Business Forum, when scouting around for a "stable" version of Python > to use as py-in-a-tie chose 2.2.x. There is almost certainly not going to > be a 2.1.4 (unless you volunteer to be the release manager). 2.3 was > released a couple weeks ago. > > You should probably consider upgrading to 2.3 (or at least 2.2.3). If > that's not possible, you might want to look at the _socket code in 2.2.3 or > 2.3 for hints about how you can get the 2.1.3 version to compile. > > Skip > From jepler at unpythonic.net Wed Aug 13 15:41:41 2003 From: jepler at unpythonic.net (Jeff Epler) Date: Wed Aug 13 15:41:45 2003 Subject: [Python-Dev] Runtime Error about fileinput.py In-Reply-To: <20030813163410.GA22290@unpythonic.net> References: <5.1.0.14.2.20030813120937.00b32c50@notes.tcindex.com> <20030813163410.GA22290@unpythonic.net> Message-ID: <20030813194141.GD22290@unpythonic.net> On Wed, Aug 13, 2003 at 11:34:11AM -0500, Jeff Epler wrote: > it's not necessarily related to your problem, but it's a bad idea to > give your program the name "XXX.py" when XXX is the name of a standard > Python module. [...] I apologize for posting this followup on python-dev. This question (and its answers) are more appropriate for python-list / the comp.lang.python newsgroup. From tree at basistech.com Wed Aug 13 16:57:02 2003 From: tree at basistech.com (Tom Emerson) Date: Wed Aug 13 16:02:29 2003 Subject: [Python-Dev] Forward declaration of PyObject* values Message-ID: <16186.38926.784050.981822@magrathea.basistech.com> I'm working on an application (in C++) that conditionally embeds Python. Some of the (C++) classes include PyObject*'s only when embedding Python. In other cases these aren't here. I do not want to include Python.h in the header files for these classes because not all of the clients of these classes use the Python functionality. Hence I would like to forward declare PyObject so it can be used as an opaque type in these interfaces, and only have Python.h included in those source files that actually need it. This comes up because Python.h needs to be included before any system interfaces, lest you get various compiler warnings (cf. sf issue 637321 related to the redefinition of _POSIX_C_SOURCE) but struct PyObject; doesn't work, at least in GCC 3.2.2: ../include/bt_notice.h:31: conflicting types for `struct PyObject' ../../../third-party-tools/python/include/python2.3/object.h:104: previous declaration as `typedef struct _object PyObject' make: *** [obj_g/RH72-gcc-3.2/bt_notice.o] Error 1 Since I would rather not have to include Python.h in every source file that could possible include the headers that reference PyObject*'s, I'm looking for a way to avoid these warnings. I expect there is none, but... Thanks in advance. -tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From xscottg at yahoo.com Wed Aug 13 14:15:59 2003 From: xscottg at yahoo.com (Scott Gilbert) Date: Wed Aug 13 16:16:33 2003 Subject: [Python-Dev] Forward declaration of PyObject* values In-Reply-To: <16186.38926.784050.981822@magrathea.basistech.com> Message-ID: <20030813201559.4811.qmail@web40102.mail.yahoo.com> I used the following for a similar task: #ifndef PyObject_HEAD struct _object; typedef _object PyObject; #endif Some compilers won't like the incomplete type on the struct, but it looks like you're using GCC which should be fine. Since PyObject_HEAD is defined in object.h, you won't be redefining it if the actual file is really included. This is probably fragile, but so is using a leading underscore in a user defined type, or having a name as generic as _object at the global scope. It works for now at least. --- Tom Emerson wrote: > > [...] I would like to forward declare PyObject so it can be used as an > opaque type in these interfaces, and only have Python.h included in > those source files that actually need it. > From tree at basistech.com Wed Aug 13 17:14:46 2003 From: tree at basistech.com (Tom Emerson) Date: Wed Aug 13 16:20:10 2003 Subject: [Python-Dev] Forward declaration of PyObject* values In-Reply-To: <20030813201559.4811.qmail@web40102.mail.yahoo.com> References: <16186.38926.784050.981822@magrathea.basistech.com> <20030813201559.4811.qmail@web40102.mail.yahoo.com> Message-ID: <16186.39990.258071.567900@magrathea.basistech.com> Scott Gilbert writes: > I used the following for a similar task: > > #ifndef PyObject_HEAD > struct _object; > typedef _object PyObject; > #endif > > Some compilers won't like the incomplete type on the struct, but it looks > like you're using GCC which should be fine. Thanks Scott. I'll eventually be on upwards of 11 platforms with this code, so I'll soon find out what platform compilers don't like this. -tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From michel at dialnetwork.com Wed Aug 13 10:02:16 2003 From: michel at dialnetwork.com (Michel Pelletier) Date: Wed Aug 13 16:45:38 2003 Subject: [Python-Dev] Exposing socket and poll objects in the Concrete Object Layer Message-ID: <200308130902.17667.michel@dialnetwork.com> I'm sketching a C module for a client using new style classes. I'd like to use, and return to the user in some cases, Python socket and poll objects. I want to create socket objects for custom connection handlers and poll objects to poll a set of handlers periodically; all written in C and intended to be subclassed in Python to customize behavior. Unfortunately these objects cannot be created, as far as I can tell, in C without resorting to PyImport_* and PyObject_Call* hacks which are alarmingly ugly. I could of course just use the standard socket and select C libraries, but this means re-implementing all the hacks that make the Python implementations so rock solid across platforms, and it also means I can't return socket and poll objects so that *Python* subclasses of my code, and other Python code, can use those objects directly. It seems to me that socket and poll objects can be easily exposed to C programmers through the Concrete Object Layer API. There may be a good reason why this isn't so that I don't see. Does it makes sense to expose methods like PySocket_New, PySocket_Bind etc. and PyPoll_New, PyPoll_Poll, PyPoll_Register etc? One reason I can see is that these modules cannot be guaranteed to work consistently (if at all) across all platforms. In this case it is acceptable to my client to detect that condition and explicitly fail. We do not ever anticipate using platforms upon which these services are broken. Does this sound useful to anyone else? Now that new style classes make it possible for people to write classes in C, I think it will become a much more common practice, and that Python programmers moving "down" to C will expect to be able to use services in C that they take for granted in Python. -Michel From kiko at async.com.br Thu Aug 14 00:14:08 2003 From: kiko at async.com.br (Christian Reis) Date: Wed Aug 13 22:14:38 2003 Subject: [Python-Dev] g_ascii_strtod/snprintf and Python Message-ID: <20030814021408.GL3095@async.com.br> Hello Alex, how's it going? I'm assuming Johan has talked to you about this issue we're pursuing on python-dev to make Python LC_NUMERIC-safe (if not, the PEP I wrote is attached, and it has details and a number of references). Basically, we're changing Python to stop depending on having the C locale set for the LC_NUMERIC category -- the parser and Int/Float and String objects currently rely on glibc versions of strtod/atof and snprintf, and this is a problem specifically for C extensions that link to python in runtime (which depend on LC_NUMERIC being set correctly to be able to localize correctly). Gustavo Carneiro started work on a patch to implement this a while back. We'd really like to use (and his prototype patch indeed does) the LC_NUMERIC-safe versions that you wrote for glib, but python-dev has been clear that they really need the original author to contribute the code (and sign the PSF license). So I'm writing to ask if you'd be kind enough to help us out and comply with the steps required to contribute the code. Basically (AFAICT, and someone please correct me if I'm wrong) this means checking out http://www.python.org/psf/psf-contributor-agreement.html and filling out the Python Contributor Agreement, and mailing it in to the PSF. This would allow you to check in the code in question (at which point we would be a lot closer to having this fixed for good.) [PS: Does anyone on python-dev (Martin, Barry perhaps?) have any further comments on the process that we should go through to move on with this?] Thanks for any help you can provide us. Take care, -- Christian Reis, Senior Engineer, Async Open Source, Brazil. http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL From kiko at async.com.br Thu Aug 14 00:21:28 2003 From: kiko at async.com.br (Christian Reis) Date: Wed Aug 13 22:21:44 2003 Subject: [Python-Dev] Be Honest about LC_NUMERIC [REPOST] Message-ID: <20030814022128.GN3095@async.com.br> So, in an attempt to garner comments (now that we have 2.3 off the chopping block) I'm reposting my PEP proposal (with minor updates). Comments would be appreciated, of course (nudges Barry slightly after him getting me to write this on my only free Sunday in months ;) PEP: XXX Title: Be Honest about LC_NUMERIC (to the C library) Version: $Revision: 1.9 $ Last-Modified: $Date: 2002/08/26 16:29:31 $ Author: Christian R. Reis Status: Draft Type: Standards Track Content-Type: text/plain Created: 19-July-2003 Post-History: ------------------------------------------------------------------------ Abstract Support in Python for the LC_NUMERIC locale category is currently implemented only in Python-space, which causes inconsistent behavior and thread-safety issues for applications that use extension modules and libraries implemented in C. This document proposes a plan for removing this inconsistency by providing and using substitute locale-agnostic functions as necessary. Introduction Python currently provides generic localization services through the locale module, which among other things allows localizing the display and conversion process of numeric types. Locale categories, such as LC_TIME and LC_COLLATE, allow configuring precisely what aspects of the application are to be localized. The LC_NUMERIC category specifies formatting for non-monetary numeric information, such as the decimal separator in float and fixed-precision numbers. Localization of the LC_NUMERIC category is currently implemented in only in Python-space; the C libraries are unaware of the application's LC_NUMERIC setting. This is done to avoid changing the behavior of certain low-level functions that are used by the Python parser and related code [2]. However, this presents a problem for extension modules that wrap C libraries; applications that use these extension modules will inconsistently display and convert numeric values. James Henstridge, the author of PyGTK [3], has additionally pointed out that the setlocale() function also presents thread-safety issues, since a thread may call the C library setlocale() outside of the GIL, and cause Python to function incorrectly. Rationale The inconsistency between Python and C library localization for LC_NUMERIC is a problem for any localized application using C extensions. The exact nature of the problem will vary depending on the application, but it will most likely occur when parsing or formatting a numeric value. Example Problem The initial problem that motivated this PEP is related to the GtkSpinButton [4] widget in the GTK+ UI toolkit, wrapped by PyGTK. The widget can be set to numeric mode, and when this occurs, characters typed into it are evaluated as a number. Because LC_NUMERIC is not set in libc, float values are displayed incorrectly, and it is impossible to enter values using the localized decimal separator (for instance, `,' for the Brazilian locale pt_BR). This small example demonstrates reduced usability for localized applications using this toolkit when coded in Python. Proposal Martin v. L?wis commented on the initial constraints for an acceptable solution to the problem on python-dev: - LC_NUMERIC can be set at the C library level without breaking the parser. - float() and str() stay locale-unaware. The following seems to be the current practice: - locale-aware str() and float() [XXX: atof(), currently?] stay in the locale module. An analysis of the Python source suggests that the following functions currently depend on LC_NUMERIC being set to the C locale: - Python/compile.c:parsenumber() - Python/marshal.c:r_object() - Objects/complexobject.c:complex_to_buf() - Objects/complexobject.c:complex_subtype_from_string() - Objects/floatobject.c:PyFloat_FromString() - Objects/floatobject.c:format_float() - Modules/stropmodule.c:strop_atof() - Modules/cPickle.c:load_float() [XXX: still need to check if any other occurrences exist] The proposed approach is to implement LC_NUMERIC-agnostic functions for converting from (strtod()/atof()) and to (snprintf()) float formats, using these functions where the formatting should not vary according to the user-specified locale. This change should also solve the aforementioned thread-safety problems. Potential Code Contributions This problem was initially reported as a problem in the GTK+ libraries [5]; since then it has been correctly diagnosed as an inconsistency in Python's implementation. However, in a fortunate coincidence, the glib library implements a number of LC_NUMERIC-agnostic functions (for an example, see [6]) for reasons similar to those presented in this paper. In the same GTK+ problem report, Havoc Pennington has suggested that the glib authors would be willing to contribute this code to the PSF, which would simplify implementation of this PEP considerably. [I'm checking if Alex Larsson is willing to sign the PSF contributor agreement [7] to make sure the code is safe to integrate; XXX: what would be necessary to sign here?] Risks There may be cross-platform issues with the provided locale-agnostic functions. This needs to be tested further. Martin has pointed out potential copyright problems with the contributed code. I believe we will have no problems in this area as members of the GTK+ and glib teams have said they are fine with relicensing the code. Code An implementation is being developed by Gustavo Carneiro . It is currently attached to Sourceforge.net bug 744665 [8] [XXX: The SF.net tracker is horrible 8(] References [1] PEP 1, PEP Purpose and Guidelines, Warsaw, Hylton http://www.python.org/peps/pep-0001.html [2] Python locale documentation for embedding, http://www.python.org/doc/current/lib/embedding-locale.html [3] PyGTK homepage, http://www.daa.com.au/~james/pygtk/ [4] GtkSpinButton screenshot (demonstrating problem), http://www.async.com.br/~kiko/spin.png [5] GNOME bug report, http://bugzilla.gnome.org/show_bug.cgi?id=114132 [6] Code submission of g_ascii_strtod and g_ascii_dtostr (later renamed g_ascii_formatd) by Alex Larsson, http://mail.gnome.org/archives/gtk-devel-list/2001-October/msg00114.html [7] PSF Contributor Agreement, http://www.python.org/psf/psf-contributor-agreement.html [8] Python bug report, http://www.python.org/sf/774665 Copyright This document has been placed in the public domain. Take care, -- Christian Reis, Senior Engineer, Async Open Source, Brazil. http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL From misa at redhat.com Wed Aug 13 23:28:29 2003 From: misa at redhat.com (Mihai Ibanescu) Date: Wed Aug 13 22:28:29 2003 Subject: [Python-Dev] Building 2.1.3 on RH9 In-Reply-To: Message-ID: On Wed, 13 Aug 2003, Dennis Allison wrote: > Skip -- > > I love the 2.3 release, but I need the 2.1.3 engine for Zope. > Unfortunately, Zope is locked into 2.1.3. Apoparently this is because of > incompatibilities with old vs. new classes, etc. etc. 2.2 and later is > unsupported. All the docs say NO! but rumor has it that it works. > > RH9 seems to have introduced all sorts of grubby stuff into the mix > including big changes in SSL and a threads library which breaks a lot of > things. It's frustrating to see Zope come up and just stop.... Dennis, I do understand your frustration. However, 2.1 is quite old; guaranteeing backwards compatibility is generally difficult, having so many things changing so fast. I find increasing numbers of programs that no longer compile on my Red Hat 7.3 machine. Can you provide me with an example of something NPTL breaks? Is there something I can do to help? Did you try Skip's suggestion of applying the changes from the 2.2 or 2.3 tree to make 2.1.3 compile? Cheers, Misa > I'll try running with 2.2 and/or 2.3 to see if they work--then I guess > I'll have to reinstall RH7.3 which I know is stable.... > > I'll file a bug report > > Thanks. -dra > > On Wed, 13 Aug 2003, Skip Montanaro wrote: > > > > > Dennis> Seems to be broken -- at least _socket does not build correctly > > Dennis> because of openssl changes. (Arggh... ) > > > > Dennis> Where should this be reported? > > > > Well, presuming it will be fixed, the correct place to report it is on the > > Bugs page at > > > > http://sourceforge.net/projects/python/ > > > > However, the 2.1 series is getting to be pretty ancient history. Even the > > Python Business Forum, when scouting around for a "stable" version of Python > > to use as py-in-a-tie chose 2.2.x. There is almost certainly not going to > > be a 2.1.4 (unless you volunteer to be the release manager). 2.3 was > > released a couple weeks ago. > > > > You should probably consider upgrading to 2.3 (or at least 2.2.3). If > > that's not possible, you might want to look at the _socket code in 2.2.3 or > > 2.3 for hints about how you can get the 2.1.3 version to compile. > > > > Skip > > > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev > From kiko at async.com.br Thu Aug 14 00:35:15 2003 From: kiko at async.com.br (Christian Reis) Date: Wed Aug 13 22:36:28 2003 Subject: [spambayes-dev] RE: [Python-Dev] RE: [Spambayes] Question (orpossibly a bug report) In-Reply-To: References: <020901c35236$e5576f10$f502a8c0@eden> Message-ID: <20030814023515.GO3095@async.com.br> On Fri, Jul 25, 2003 at 07:25:48AM +0200, Martin v. L?wis wrote: > "Mark Hammond" writes: > > > The "best" solution to this probably involves removing Python being > > dependent on the locale - there is even an existing patch for that. > > While the feature is desirable, I don't like the patch it all. It > copies the relevant code of Gnome glib, and I > a) doubt it works on all systems we care about, and I'm sorry you don't like the patch, but if there's something that can be fixed, we will fix it :-) Well, glib is known to be quite portable, and we would make sure that it does run on the supported platforms before considering checking it in. (I'm betting it does.) > b) is too much code for us to maintain, and It's not *that* much code, and we can rely on fixes that are produced to glib being easily ported to us -- we get free maintenance of the code if we choose to do so, actually. > c) introduces yet another license (although the true authors > of that code would be willing to relicense it) Which means that c) is a non-issue? > It would be better if system functions could be found for a > locale-agnostic atof/strtod on all systems. For example, glibc > has a strtod_l function, which expects a locale_t in addition > to the char*. Yes, but if all we were worried about was glibc, then point a) would be a non-issue too. I imagine it's easier to make sure the code we *have* runs on multiple platforms than trying to find and call code that *may* exist on each given platform. > It would be good if something similar was discovered for VC. Using > undocumented or straight Win32 API functions would be fine. > Unfortunately, the "true" source of atof (i.e. from conv.obj) is not > shipped with MSVC :-( I don't understand this bit. You'd rather use an undocumented API function than an open source, well-tested, properly licensed set of functions? Take care, -- Christian Reis, Senior Engineer, Async Open Source, Brazil. http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL From kiko at async.com.br Thu Aug 14 00:38:49 2003 From: kiko at async.com.br (Christian Reis) Date: Wed Aug 13 22:38:56 2003 Subject: [spambayes-dev] RE: [Python-Dev] RE: [Spambayes] Question (orpossibly a bug report) In-Reply-To: References: Message-ID: <20030814023849.GP3095@async.com.br> On Fri, Jul 25, 2003 at 03:13:46AM -0400, Tim Peters wrote: > [martin@v.loewis.de] > > While the feature is desirable, I don't like the patch it all. It > > copies the relevant code of Gnome glib, and I > > a) doubt it works on all systems we care about, and > > b) is too much code for us to maintain, and > > c) introduces yet another license (although the true authors > > of that code would be willing to relicense it) > > OTOH, even assuming "C" locale, Python's float<->string story varies across > platforms anyway, due to different C libraries treating things like > infinities, NaNs, signed zeroes, and the number of digits displayed in an > exponent differently. This also has bad consequences, although one-platform > programmers usually don't notice them (Windows programmers do more than > most, because MS's C library can't read back the strings it produces for > NaNs and infinities -- which Python also produces and can't read back in > then). > > So it's not that the patch is too much code to maintain, it's not enough > code to do the whole job <0.9 wink>. My question, now, is if we would we be able to cobble something even more magical into the g_ascii_* functions that makes Python more robust to these changes (over time)? Take care, -- Christian Reis, Senior Engineer, Async Open Source, Brazil. http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL From guido at python.org Wed Aug 13 21:34:16 2003 From: guido at python.org (Guido van Rossum) Date: Wed Aug 13 23:35:24 2003 Subject: [spambayes-dev] RE: [Python-Dev] RE: [Spambayes] Question (orpossibly a bug report) In-Reply-To: Your message of "Wed, 13 Aug 2003 23:35:15 -0300." <20030814023515.GO3095@async.com.br> References: <020901c35236$e5576f10$f502a8c0@eden> <20030814023515.GO3095@async.com.br> Message-ID: <200308140334.h7E3YGa02755@12-236-84-31.client.attbi.com> > I don't understand this bit. You'd rather use an undocumented API > function than an open source, well-tested, properly licensed set of > functions? I don't know what the exact requirements of this license are, but I assure you that redistributing code that is not under the PSF license is a pain, even if it's an open source license. If we can get the original authors to contribute the code to the PSF without the requirement to include a license of any kind (beyond the PSF license) in redistributions, either by the PSF or downstream, even if those redistributions are commercial or contain proprietary code in addition to open source code. This is what's possible with the PSF license, and that needs to remain the case. In particular, the GPL is *not* acceptable for this purpose. --Guido van Rossum (home page: http://www.python.org/~guido/) From kiko at async.com.br Thu Aug 14 01:48:03 2003 From: kiko at async.com.br (Christian Reis) Date: Wed Aug 13 23:48:40 2003 Subject: [spambayes-dev] RE: [Python-Dev] RE: [Spambayes] Question (orpossibly a bug report) In-Reply-To: <200308140334.h7E3YGa02755@12-236-84-31.client.attbi.com> References: <020901c35236$e5576f10$f502a8c0@eden> <20030814023515.GO3095@async.com.br> <200308140334.h7E3YGa02755@12-236-84-31.client.attbi.com> Message-ID: <20030814034803.GB5693@async.com.br> On Wed, Aug 13, 2003 at 08:34:16PM -0700, Guido van Rossum wrote: > > I don't understand this bit. You'd rather use an undocumented API > > function than an open source, well-tested, properly licensed set of > > functions? > > I don't know what the exact requirements of this license are, but I > assure you that redistributing code that is not under the PSF license > is a pain, even if it's an open source license. If we can get the > original authors to contribute the code to the PSF without the > requirement to include a license of any kind (beyond the PSF license) > in redistributions, either by the PSF or downstream, even if those > redistributions are commercial or contain proprietary code in addition > to open source code. This is what's possible with the PSF license, You omit the predicate that follows this if clause, but I'm hoping you meant something positive like `we will gladly accept it' I'm waiting on Alex's answer on relicensing the code, but he's said on IRC that he'd be willing to do it, so barring any environmental disasters, that should be solved sometime soon. Take care, -- Christian Reis, Senior Engineer, Async Open Source, Brazil. http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL From aahz at pythoncraft.com Thu Aug 14 01:52:53 2003 From: aahz at pythoncraft.com (Aahz) Date: Thu Aug 14 00:52:57 2003 Subject: [Python-Dev] Be Honest about LC_NUMERIC [REPOST] In-Reply-To: <20030814022128.GN3095@async.com.br> References: <20030814022128.GN3095@async.com.br> Message-ID: <20030814045253.GA11360@panix.com> On Wed, Aug 13, 2003, Christian Reis wrote: > > So, in an attempt to garner comments (now that we have 2.3 off the > chopping block) I'm reposting my PEP proposal (with minor updates). > Comments would be appreciated, of course (nudges Barry slightly after > him getting me to write this on my only free Sunday in months ;) > > PEP: XXX > Title: Be Honest about LC_NUMERIC (to the C library) > Version: $Revision: 1.9 $ > Last-Modified: $Date: 2002/08/26 16:29:31 $ I'll repeat what I said on 7/24: Looks good to me (can't comment on the tech issues, but it seems clear enough). However, I'd recommend changing the title to something like "Locale-independent float conversions". -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From kiko at async.com.br Thu Aug 14 03:16:02 2003 From: kiko at async.com.br (Christian Reis) Date: Thu Aug 14 01:16:08 2003 Subject: [Python-Dev] Be Honest about LC_NUMERIC [REPOST] In-Reply-To: <20030814045253.GA11360@panix.com> References: <20030814022128.GN3095@async.com.br> <20030814045253.GA11360@panix.com> Message-ID: <20030814051602.GB7580@async.com.br> On Thu, Aug 14, 2003 at 12:52:53AM -0400, Aahz wrote: > On Wed, Aug 13, 2003, Christian Reis wrote: > > > > So, in an attempt to garner comments (now that we have 2.3 off the > > chopping block) I'm reposting my PEP proposal (with minor updates). > > Comments would be appreciated, of course (nudges Barry slightly after > > him getting me to write this on my only free Sunday in months ;) > > > > PEP: XXX > > Title: Be Honest about LC_NUMERIC (to the C library) > > Version: $Revision: 1.9 $ > > Last-Modified: $Date: 2002/08/26 16:29:31 $ > > I'll repeat what I said on 7/24: OT: I actually replied to you back then, but your MX denies mail from me (since our host happens to be in a pretty wide spamhaus block). > Looks good to me (can't comment on the tech issues, but it seems clear > enough). However, I'd recommend changing the title to something like > "Locale-independent float conversions". I suppose I'm okay with changing the title if it's too obscure to be helpful -- it was only meant to be precise in a light-hearted fashion. Take care, -- Christian Reis, Senior Engineer, Async Open Source, Brazil. http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL From alexl at redhat.com Thu Aug 14 07:28:48 2003 From: alexl at redhat.com (Alexander Larsson) Date: Thu Aug 14 02:28:50 2003 Subject: [Python-Dev] Re: g_ascii_strtod/snprintf and Python In-Reply-To: <20030814021408.GL3095@async.com.br> References: <20030814021408.GL3095@async.com.br> Message-ID: <1060842501.6956.45.camel@localhost.localdomain> On Thu, 2003-08-14 at 04:14, Christian Reis wrote: > Hello Alex, > > how's it going? I'm assuming Johan has talked to you about this > issue we're pursuing on python-dev to make Python LC_NUMERIC-safe (if > not, the PEP I wrote is attached, and it has details and a number of > references). > > Basically, we're changing Python to stop depending on having the C > locale set for the LC_NUMERIC category -- the parser and Int/Float and > String objects currently rely on glibc versions of strtod/atof and > snprintf, and this is a problem specifically for C extensions that link > to python in runtime (which depend on LC_NUMERIC being set correctly to > be able to localize correctly). > > Gustavo Carneiro started work on a patch to implement this a while back. > We'd really like to use (and his prototype patch indeed does) the > LC_NUMERIC-safe versions that you wrote for glib, but python-dev has > been clear that they really need the original author to contribute the > code (and sign the PSF license). So I'm writing to ask if you'd be kind > enough to help us out and comply with the steps required to contribute > the code. Sure, no problems. > Basically (AFAICT, and someone please correct me if I'm wrong) this > means checking out > > http://www.python.org/psf/psf-contributor-agreement.html > > and filling out the Python Contributor Agreement, and mailing it in to > the PSF. This would allow you to check in the code in question (at which > point we would be a lot closer to having this fixed for good.) Which form do I need to fill out? "Proposed Contributor Agreement" seems a bit much. I don't need a cvs account. Maybe "Proposed Contibution Licensing Agreement"? =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Alexander Larsson Red Hat, Inc alexl@redhat.com alla@lysator.liu.se He's a fast talking native American shaman with acid for blood. She's a psychotic belly-dancing single mother with only herself to blame. They fight crime! From mwh at python.net Thu Aug 14 14:29:57 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 14 08:30:00 2003 Subject: [Python-Dev] Exposing socket and poll objects in the Concrete Object Layer In-Reply-To: <200308130902.17667.michel@dialnetwork.com> (Michel Pelletier's message of "Wed, 13 Aug 2003 09:02:16 -0500") References: <200308130902.17667.michel@dialnetwork.com> Message-ID: <2msmo4pgl6.fsf@starship.python.net> Michel Pelletier writes: > I'm sketching a C module for a client using new style classes. I'd like to > use, and return to the user in some cases, Python socket and poll objects. I > want to create socket objects for custom connection handlers and poll objects > to poll a set of handlers periodically; all written in C and intended to be > subclassed in Python to customize behavior. Unfortunately these objects > cannot be created, as far as I can tell, in C without resorting to PyImport_* > and PyObject_Call* hacks which are alarmingly ugly. Is it that bad? You can wrap them up in utility functions & so on. I agree it's a bit vulgar. > I could of course just use the standard socket and select C libraries, but > this means re-implementing all the hacks that make the Python implementations > so rock solid across platforms, and it also means I can't return socket and > poll objects so that *Python* subclasses of my code, and other Python code, > can use those objects directly. > > It seems to me that socket and poll objects can be easily exposed to C > programmers through the Concrete Object Layer API. There may be a good > reason why this isn't so that I don't see. Does it makes sense to expose > methods like PySocket_New, PySocket_Bind etc. and PyPoll_New, PyPoll_Poll, > PyPoll_Register etc? Hmm. I think it would be best to implement these in a CObject-y style, like the way cPickle uses StringIO and the way (I presume, haven't actually looked) _socket uses _ssl. > One reason I can see is that these modules cannot be guaranteed to work > consistently (if at all) across all platforms. In this case it is acceptable > to my client to detect that condition and explicitly fail. We do not ever > anticipate using platforms upon which these services are broken. I guess this is easy enough with CObjects. > Does this sound useful to anyone else? It falls into the "yep, might be handy" category, but I can't say that *I* would find it intensely useful myself. > Now that new style classes make it possible for people to write > classes in C, I think it will become a much more common practice, > and that Python programmers moving "down" to C will expect to be > able to use services in C that they take for granted in Python. This would also please those people who try to sneak Python onto projects with cries of "it's just a library of useful C routines!" :-) Cheers, mwh -- M-x psych[TAB][RETURN] -- try it From mwh at python.net Thu Aug 14 14:41:01 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 14 08:41:03 2003 Subject: [Python-Dev] Re: g_ascii_strtod/snprintf and Python In-Reply-To: <1060842501.6956.45.camel@localhost.localdomain> (Alexander Larsson's message of "14 Aug 2003 08:28:21 +0200") References: <20030814021408.GL3095@async.com.br> <1060842501.6956.45.camel@localhost.localdomain> Message-ID: <2mptj8pg2q.fsf@starship.python.net> Alexander Larsson writes: >> Basically (AFAICT, and someone please correct me if I'm wrong) this >> means checking out >> >> http://www.python.org/psf/psf-contributor-agreement.html >> >> and filling out the Python Contributor Agreement, and mailing it in to >> the PSF. This would allow you to check in the code in question (at which >> point we would be a lot closer to having this fixed for good.) > > Which form do I need to fill out? "Proposed Contributor Agreement" seems > a bit much. I don't need a cvs account. Maybe "Proposed Contibution > Licensing Agreement"? Well, the situation is a bit confused at the moment. To my knowledge *noone* has signed one of these agreements yet -- I certainly haven't. Cheers, mwh -- ZAPHOD: Listen three eyes, don't try to outwierd me, I get stranger things than you free with my breakfast cereal. -- The Hitch-Hikers Guide to the Galaxy, Episode 7 From guido at python.org Thu Aug 14 08:54:19 2003 From: guido at python.org (Guido van Rossum) Date: Thu Aug 14 10:54:56 2003 Subject: [Python-Dev] Re: g_ascii_strtod/snprintf and Python In-Reply-To: Your message of "Thu, 14 Aug 2003 13:41:01 BST." <2mptj8pg2q.fsf@starship.python.net> References: <20030814021408.GL3095@async.com.br> <1060842501.6956.45.camel@localhost.localdomain> <2mptj8pg2q.fsf@starship.python.net> Message-ID: <200308141454.h7EEsJX03619@12-236-84-31.client.attbi.com> > > Which form do I need to fill out? "Proposed Contributor Agreement" seems > > a bit much. I don't need a cvs account. Maybe "Proposed Contibution > > Licensing Agreement"? > > Well, the situation is a bit confused at the moment. To my knowledge > *noone* has signed one of these agreements yet -- I certainly haven't. That's because we're still discussing them with lawyers (a year too long :-( ). Eventually we'll want one from you too. In this specific case of code that is known to be distributed as part of glibc as well, I think it's important to have an explicit paper trail from the original author (who better be the author of *all* lines he's contributing to us :-). I recommend that he sign the Proposed Contribution Licensing Agreement. --Guido van Rossum (home page: http://www.python.org/~guido/) From mwh at python.net Thu Aug 14 18:37:54 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 14 12:37:57 2003 Subject: [Python-Dev] refleak hunting fun! Message-ID: <2mk79gp53x.fsf@starship.python.net> Prompted by finding my own refcounting leaks in test_descr, I'm now looking to spread the misery :-) I've hacked regrtest (hacks attached) to do some very crude monitoring of sys.gettotalrefcount(). Here's the output of a recent run: test_exceptions leaked 17 references test_types leaked 22 references test_class leaked 66 references test_codeccallbacks leaked 1107 references test_coercion leaked 14 references test_compile leaked 69 references test_complex leaked 2 references test_cookie leaked 14 references test_copy leaked 82 references test_cpickle leaked 17 references test_descr leaked 1 references test_gc leaked 18 references test_global leaked 14 references test_hotshot leaked 111 references test_logging leaked 82 references test_minidom leaked 1 references test_mutants leaked -452 references test_new leaked 2 references test_optparse leaked 408 references test_os leaked 24 references test_pkg leaked 7 references test_pkgimport leaked 6 references test_regex leaked 14 references test_rotor leaked 14 references test_sax leaked 1762 references test_scope leaked 14 references test_socket leaked 1140 references test_sundry leaked 42 references test_threadedtempfile leaked 81 references test_threading leaked -24 references test_time leaked -24 references test_unicode leaked 29 references test_urlparse leaked -70 references test_xreadline leaked 14 references test_grammar leaked 14 references *some* of these are bogus -- caches in particular play merry hell with trying to instrument this kind of thing -- and indeed I've already found causes for some of the bogosities above but as my hacked regrtest runs each test five times, runs are rraatthheerr ssllooww. Some however, seem to be real. test_sax, test_socket and test_codeccallbacks seem to be the most alarming. The TrackRef class Tim posted a link to is likely to be indispensable in hunting these. Some tests are a bit unhappy at being run repeatedly -- notably test_threaded_import (which deadlocks) and all the doctest tests spew stuff to stdout. You'll want to add an '-x test_threaded_import' to your command line. I'd like to do some more work in aid of finding out exactly which test cases in unittest-based are leaking the references. Finding out which of the flat-file style tests in test_types is leaking is not going to be fun... (seems to be leaking a list of integers, oh dear hope that's not my fault again :-/). Cheers, mwh -------------- next part -------------- A non-text attachment was scrubbed... Name: regr-hacks.diff Type: text/x-patch Size: 7406 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20030814/1918ff9f/regr-hacks.bin -------------- next part -------------- -- We've had a lot of problems going from glibc 2.0 to glibc 2.1. People claim binary compatibility. Except for functions they don't like. -- Peter Van Eynde, comp.lang.lisp From mwh at python.net Thu Aug 14 18:54:45 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 14 12:54:47 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mk79gp53x.fsf@starship.python.net> (Michael Hudson's message of "Thu, 14 Aug 2003 17:37:54 +0100") References: <2mk79gp53x.fsf@starship.python.net> Message-ID: <2mhe4kp4bu.fsf@starship.python.net> Michael Hudson writes: > Finding out which of the flat-file style tests in test_types is > leaking is not going to be fun... Binary chop got there: >>> a = [] [25139 refs] >>> a[:] = a [25140 refs] >>> a[:] = a [25141 refs] Cheers, mwh -- 59. In English every word can be verbed. Would that it were so in our programming languages. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html From theller at python.net Thu Aug 14 20:29:02 2003 From: theller at python.net (Thomas Heller) Date: Thu Aug 14 13:29:43 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: (Thomas Heller's message of "Fri, 08 Aug 2003 19:19:54 +0200") References: Message-ID: <7k5gqhb5.fsf@python.net> Thomas Heller writes: > There has been a discussion on comp.lang.python about a cross-platform > py2exe-like tool: > > > The basic idea is, now that the zipimport feature is in place, to make > it easy to convert the Python interpreter itself into a standalone > executable by a simple operation (which can well be platform dependend, > but on some platforms, at least, a zipfile can be appended to a copy of > the interpreter executable itself). > > Alex Martelli and Oren Tirosh also came up with ideas how to do this, > and it seems a hook in Py_Main() would be able to do the trick. > > This hook could be triggered by examining the filename of the executable > itself. If the basename starts (case-insensitive) with the > characters 'python', everything continues as it is now. Here's another idea, based on the ancient history of Visual Smalltalk: Let the hook be triggered by the fact that in the same directory as the executable is another file present having the same name as the executable, with a certain extension appended (VS used '.bnd'). This file could contain information about how to run the exe: the values for the Py_Optimize and other flags, entries to use for sys.path, a pointer to or the code itself for the script to run. It was a very flexible mechanism for Visual Smalltalk, then. If anyone want further details how this worked, I can provide them. Thomas From nas at python.ca Thu Aug 14 12:10:58 2003 From: nas at python.ca (Neil Schemenauer) Date: Thu Aug 14 14:04:55 2003 Subject: [Python-Dev] filter(bool, ...) Message-ID: <20030814181058.GA1557@glacier.arctrix.com> I just realized that filter(bool, ...) is the same as filter(None, ...). Anyone object to optimizing the 'bool' case? I think using 'bool' is much clearer than 'None'. Neil Index: Python/bltinmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/bltinmodule.c,v retrieving revision 2.293 diff -u -r2.293 bltinmodule.c --- Python/bltinmodule.c 2 Aug 2003 07:42:56 -0000 2.293 +++ Python/bltinmodule.c 14 Aug 2003 17:49:26 -0000 @@ -197,7 +197,7 @@ break; } - if (func == Py_None) { + if (func == Py_None || func == (PyObject *)&PyBool_Type) { ok = PyObject_IsTrue(item); } else { From guido at python.org Thu Aug 14 12:27:17 2003 From: guido at python.org (Guido van Rossum) Date: Thu Aug 14 14:27:52 2003 Subject: [Python-Dev] filter(bool, ...) In-Reply-To: Your message of "Thu, 14 Aug 2003 11:10:58 PDT." <20030814181058.GA1557@glacier.arctrix.com> References: <20030814181058.GA1557@glacier.arctrix.com> Message-ID: <200308141827.h7EIRHo04040@12-236-84-31.client.attbi.com> > I just realized that filter(bool, ...) is the same as filter(None, ...). > Anyone object to optimizing the 'bool' case? I think using 'bool' is > much clearer than 'None'. > > Neil > > Index: Python/bltinmodule.c > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Python/bltinmodule.c,v > retrieving revision 2.293 > diff -u -r2.293 bltinmodule.c > --- Python/bltinmodule.c 2 Aug 2003 07:42:56 -0000 2.293 > +++ Python/bltinmodule.c 14 Aug 2003 17:49:26 -0000 > @@ -197,7 +197,7 @@ > break; > } > > - if (func == Py_None) { > + if (func == Py_None || func == (PyObject *)&PyBool_Type) { > ok = PyObject_IsTrue(item); > } > else { Cute. But I still think it's better to spell this as list(x) --Guido van Rossum (home page: http://www.python.org/~guido/) From fdrake at acm.org Thu Aug 14 15:30:29 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Thu Aug 14 14:31:10 2003 Subject: [Python-Dev] filter(bool, ...) In-Reply-To: <200308141827.h7EIRHo04040@12-236-84-31.client.attbi.com> References: <20030814181058.GA1557@glacier.arctrix.com> <200308141827.h7EIRHo04040@12-236-84-31.client.attbi.com> Message-ID: <16187.54597.641050.146871@grendel.zope.com> Guido van Rossum writes: > Cute. But I still think it's better to spell this as > > list(x) That doesn't do the same thing. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From walter at livinglogic.de Thu Aug 14 21:42:46 2003 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 14 14:43:22 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mk79gp53x.fsf@starship.python.net> References: <2mk79gp53x.fsf@starship.python.net> Message-ID: <3F3BD826.5060600@livinglogic.de> Michael Hudson wrote: > Prompted by finding my own refcounting leaks in test_descr, I'm now > looking to spread the misery :-) > > I've hacked regrtest (hacks attached) to do some very crude monitoring > of sys.gettotalrefcount(). > > Here's the output of a recent run: > > [...] > test_codeccallbacks leaked 1107 references > [...] > > Some however, seem to be real. test_sax, test_socket and > test_codeccallbacks seem to be the most alarming. The TrackRef class > Tim posted a link to is likely to be indispensable in hunting these. The number for test_codeccallbacks really looks scary. Some of it is the result of the callback registry, but there seem to be real leaks here. I've used the attached patch to unittest to get refcount diffs for individual test methods. Here are the results for test_codeccallbacks: 212 test_backslashescape 1 test_badandgoodbackslashreplaceexceptions 0 test_badandgoodignoreexceptions 0 test_badandgoodreplaceexceptions 0 test_badandgoodstrictexceptions 0 test_badandgoodxmlcharrefreplaceexceptions 271 test_badhandlerresults 0 test_badregistercall 153 test_callbacks 3 test_charmapencode 121 test_decodehelper 26 test_encodehelper 285 test_longstrings 0 test_lookup 9 test_relaxedutf8 1 test_translatehelper 11 test_unencodablereplacement 265 test_unicodedecodeerror 792 test_unicodeencodeerror 0 test_unicodetranslateerror 12 test_uninamereplace 0 test_unknownhandler 14 test_xmlcharnamereplace 0 test_xmlcharrefreplace 6 test_xmlcharrefvalues Bye, Walter D?rwald -------------- next part -------------- Index: Lib/unittest.py =================================================================== RCS file: /cvsroot/python/python/dist/src/Lib/unittest.py,v retrieving revision 1.24 diff -u -r1.24 unittest.py --- Lib/unittest.py 13 Jul 2003 15:18:12 -0000 1.24 +++ Lib/unittest.py 14 Aug 2003 18:37:19 -0000 @@ -376,7 +376,13 @@ for test in self._tests: if result.shouldStop: break + import gc, sys + gc.collect() + rc1 = sys.gettotalrefcount() test(result) + gc.collect() + rc2 = sys.gettotalrefcount() + print >>sys.stderr, "%5d %s" % (rc2-rc1, test) return result def debug(self): From jepler at unpythonic.net Thu Aug 14 14:43:47 2003 From: jepler at unpythonic.net (Jeff Epler) Date: Thu Aug 14 14:43:55 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: <7k5gqhb5.fsf@python.net> References: <7k5gqhb5.fsf@python.net> Message-ID: <20030814184346.GD16941@unpythonic.net> On Thu, Aug 14, 2003 at 07:29:02PM +0200, Thomas Heller wrote: > Here's another idea, based on the ancient history of Visual Smalltalk: > Let the hook be triggered by the fact that in the same directory as the > executable is another file present having the same name as the > executable, with a certain extension appended (VS used '.bnd'). -1. This may be a good answer on Windows and Mac, but this is very much not the Unix way. If a file is in a directory called "bin", it'd damned well better be an executable file. Jeff From walter at livinglogic.de Thu Aug 14 21:50:26 2003 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Thu Aug 14 14:51:00 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <3F3BD826.5060600@livinglogic.de> References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> Message-ID: <3F3BD9F2.4060003@livinglogic.de> > test_codeccallbacks leaked 1107 references Found the first leak: u"\uffff".encode("iso-8859-15", "backslashreplace") leaks one reference. I think I'll go hunting through the C code... Bye, Walter D?rwald From damien-morton at nyc.rr.com Thu Aug 14 15:55:37 2003 From: damien-morton at nyc.rr.com (Damien Morton) Date: Thu Aug 14 14:56:14 2003 Subject: [Python-Dev] Re: hook for standalone executable Message-ID: <001c01c36295$a6a646f0$6401a8c0@damien> Why not activate the hook if a magic cookie has been appended to the executable. This cookie would flag that a zip archive had been appended to the executable, and would incorporate information as to the length of the archive. The file layout would then be: To test for the existance of a zip archive, youd would do the following: If ((exe[-4:] == ) && (exe[exe[-8:-4]:] == )) From skip at pobox.com Thu Aug 14 15:23:16 2003 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 14 15:23:28 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: <001c01c36295$a6a646f0$6401a8c0@damien> References: <001c01c36295$a6a646f0$6401a8c0@damien> Message-ID: <16187.57764.894514.960478@montanaro.dyndns.org> Damien> Why not activate the hook if a magic cookie has been appended to Damien> the executable.... Damien> The file layout would then be: Damien> I don't think zipimport supports importing from zip files with appended stuff. I suspect it needs to know the last N bytes form the zip length. Skip From theller at python.net Thu Aug 14 22:29:07 2003 From: theller at python.net (Thomas Heller) Date: Thu Aug 14 15:29:49 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: <20030814184346.GD16941@unpythonic.net> (Jeff Epler's message of "Thu, 14 Aug 2003 13:43:47 -0500") References: <7k5gqhb5.fsf@python.net> <20030814184346.GD16941@unpythonic.net> Message-ID: Jeff Epler writes: > On Thu, Aug 14, 2003 at 07:29:02PM +0200, Thomas Heller wrote: >> Here's another idea, based on the ancient history of Visual Smalltalk: >> Let the hook be triggered by the fact that in the same directory as the >> executable is another file present having the same name as the >> executable, with a certain extension appended (VS used '.bnd'). > > -1. This may be a good answer on Windows and Mac, but this is very > much not the Unix way. If a file is in a directory called "bin", it'd > damned well better be an executable file. Ok. I don't know enough about Unix. What's the usual directory layout if an executable needs additional private files (library.zip, something.so and so on)? Thomas From nas-python at python.ca Thu Aug 14 13:42:32 2003 From: nas-python at python.ca (Neil Schemenauer) Date: Thu Aug 14 15:36:30 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: <001c01c36295$a6a646f0$6401a8c0@damien> References: <001c01c36295$a6a646f0$6401a8c0@damien> Message-ID: <20030814194231.GC1943@glacier.arctrix.com> Damien Morton wrote: > Why not activate the hook if a magic cookie has been appended to the > executable. This cookie would flag that a zip archive had been appended > to the executable, and would incorporate information as to the length of > the archive. I suppose some executable formats do not like extra data appended at the end. Patching a global variable with a linking step sounds like the best solution (as suggested by MvL). If someone would create a tool for Windows that would make most users happy. Neil From skip at pobox.com Thu Aug 14 15:38:19 2003 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 14 15:38:46 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: References: <7k5gqhb5.fsf@python.net> <20030814184346.GD16941@unpythonic.net> Message-ID: <16187.58667.69818.175657@montanaro.dyndns.org> Thomas> What's the usual directory layout if an executable needs Thomas> additional private files (library.zip, something.so and so on)? As with all things Unix there several different ways to skin the cat. One common installation scheme is Python's. The interpreter lives in /usr/local/bin/pythonM.m and all the version-specific stuff lives in places like /usr/local/lib/pythonM.m and /usr/local/include/pythonM.m where M and m are the major and minor versions. Another way is to create an application-specific directory in a prefix directory like /usr/local, then create bin, etc, lib and include directories underneath. Apache and Berkeley DB prefer this setup. Either way, there is a known relationship between the executable and the files it relies on. Skip From jepler at unpythonic.net Thu Aug 14 15:42:01 2003 From: jepler at unpythonic.net (Jeff Epler) Date: Thu Aug 14 15:42:05 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: References: <7k5gqhb5.fsf@python.net> <20030814184346.GD16941@unpythonic.net> Message-ID: <20030814194200.GF16941@unpythonic.net> If you install a program in $(prefix), anything executable as a command goes in $(prefix)/bin, anything used by the executable which is architecture-dependent goes in $(prefix)/lib, and anything used by the executable which is not architecture-dependent goes in $(prefix)/share (preferred) or $(prefix)/lib (accetable). If there are more than a few support files, then they should go in $(prefix)/{lib,share}/$(package_name) Python comes pretty close, but technically you'd want to put all the .py, .pyc and .pyo files in share/ So I guess that if you had $(prefix)/bin/xyzzy you'd want to look at $(prefix)/lib/xyzzy.zip and/or $(prefix)/share/xyzzy.zip to see if it's a (near-)standalone executable. From theller at python.net Thu Aug 14 22:48:59 2003 From: theller at python.net (Thomas Heller) Date: Thu Aug 14 15:49:40 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: <20030814194231.GC1943@glacier.arctrix.com> (Neil Schemenauer's message of "Thu, 14 Aug 2003 12:42:32 -0700") References: <001c01c36295$a6a646f0$6401a8c0@damien> <20030814194231.GC1943@glacier.arctrix.com> Message-ID: Neil Schemenauer writes: > Damien Morton wrote: >> Why not activate the hook if a magic cookie has been appended to the >> executable. This cookie would flag that a zip archive had been appended >> to the executable, and would incorporate information as to the length of >> the archive. > > I suppose some executable formats do not like extra data appended at the > end. Patching a global variable with a linking step sounds like the > best solution (as suggested by MvL). Right. I only didn't understand why he called this 'patching'. I would call it 'link with an object file defining this global variable'. I do not want to be nitpicking here, but a clear terminology avoids misunderstandings. > If someone would create a tool for > Windows that would make most users happy. On windows I would include this into the resources (which can be changed without any needs for a linker or so, at least on NT). Or do you mean that all this isn't needed for Unix? Thomas From nas-python at python.ca Thu Aug 14 14:08:19 2003 From: nas-python at python.ca (Neil Schemenauer) Date: Thu Aug 14 16:02:17 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: References: <001c01c36295$a6a646f0$6401a8c0@damien> <20030814194231.GC1943@glacier.arctrix.com> Message-ID: <20030814200819.GA2161@glacier.arctrix.com> Thomas Heller wrote: > Or do you mean that all this isn't needed for Unix? I mean that Unix users are less likely to want a stand-alone executable instead of a Python script. Also, stand-alone executables are usually intended for end users with little or no Python development skills. Almost all of those people are using windows. Neil From nas at debian.org Thu Aug 14 14:58:35 2003 From: nas at debian.org (Neil Schemenauer) Date: Thu Aug 14 16:52:36 2003 Subject: [Python-Dev] map(None, ...) in tutorial Message-ID: <20030814205835.GB2374@glacier.arctrix.com> I've noticed this section in the tutorial: [...], we see that map(None, list1, list2) is a convenient way of turning a pair of lists into a list of pairs. For example: >>> map(None, seq, map(square, seq)) [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25), (6, 36), (7, 49)] I think the example be changed to use zip() instead, ie: >>> zip(seq, map(square, seq)) [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25), (6, 36), (7, 49)] Any objections? Neil From skip at pobox.com Thu Aug 14 17:16:27 2003 From: skip at pobox.com (Skip Montanaro) Date: Thu Aug 14 17:16:36 2003 Subject: [Python-Dev] map(None, ...) in tutorial In-Reply-To: <20030814205835.GB2374@glacier.arctrix.com> References: <20030814205835.GB2374@glacier.arctrix.com> Message-ID: <16187.64555.181801.198405@montanaro.dyndns.org> Neil> I think the example be changed to use zip() instead, ie: Fire away. Hmmm... Do we have an insider at Debian now? Skip From python at rcn.com Thu Aug 14 18:13:31 2003 From: python at rcn.com (Raymond Hettinger) Date: Thu Aug 14 17:17:37 2003 Subject: [Python-Dev] map(None, ...) in tutorial References: <20030814205835.GB2374@glacier.arctrix.com> Message-ID: <002c01c362a8$eab89920$2a03a044@oemcomputer> > I've noticed this section in the tutorial: > > [...], we see that map(None, list1, list2) is a convenient way of > turning a pair of lists into a list of pairs. For example: > > >>> map(None, seq, map(square, seq)) > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25), (6, 36), (7, 49)] > > I think the example be changed to use zip() instead, ie: > > >>> zip(seq, map(square, seq)) > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25), (6, 36), (7, 49)] > > Any objections? No objections, just a couple of thoughts: * [(x, square(x) for x in seq] # this is much nicer * map(None, 'abc', range(5)) # demonstrates None fill-in # (which zip() doesn't have). * to unzip the above zip example: zip(*result) # turns the list of pairs back into a pair of lists Raymond Hettinger From nas at python.ca Thu Aug 14 16:04:47 2003 From: nas at python.ca (Neil Schemenauer) Date: Thu Aug 14 17:58:45 2003 Subject: [Python-Dev] map(None, ...) in tutorial In-Reply-To: <002c01c362a8$eab89920$2a03a044@oemcomputer> References: <20030814205835.GB2374@glacier.arctrix.com> <002c01c362a8$eab89920$2a03a044@oemcomputer> Message-ID: <20030814220447.GA2652@glacier.arctrix.com> Raymond Hettinger wrote: > No objections, just a couple of thoughts: > > * [(x, square(x) for x in seq] # this is much nicer That example is already in the listcomp section of the tutorial. Now but I'm wondering if the whole section on multiple arguments to map() should be removed from the tutorial. I don't think it's important for new users to learn it. > * map(None, 'abc', range(5)) # demonstrates None fill-in > # (which zip() doesn't have). The None fill-in behavior is pretty advanced for the tutorial. I don't think I've ever written a map(None, ...) expression that wasn't better expressed with the newer zip() builtin. Neil From guido at python.org Thu Aug 14 16:03:12 2003 From: guido at python.org (Guido van Rossum) Date: Thu Aug 14 18:03:54 2003 Subject: [Python-Dev] map(None, ...) in tutorial In-Reply-To: Your message of "Thu, 14 Aug 2003 13:58:35 PDT." <20030814205835.GB2374@glacier.arctrix.com> References: <20030814205835.GB2374@glacier.arctrix.com> Message-ID: <200308142203.h7EM3Cv04465@12-236-84-31.client.attbi.com> > I've noticed this section in the tutorial: > > [...], we see that map(None, list1, list2) is a convenient way of > turning a pair of lists into a list of pairs. For example: > > >>> map(None, seq, map(square, seq)) > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25), (6, 36), (7, 49)] > > I think the example be changed to use zip() instead, ie: > > >>> zip(seq, map(square, seq)) > [(0, 0), (1, 1), (2, 4), (3, 9), (4, 16), (5, 25), (6, 36), (7, 49)] > > Any objections? But then it's no longer an example of the usefulness of map(None, x, y)? (Well, map(None, ...) is never useful, so maybe that's okay...) --Guido van Rossum (home page: http://www.python.org/~guido/) From michel at dialnetwork.com Thu Aug 14 16:16:28 2003 From: michel at dialnetwork.com (Michel Pelletier) Date: Thu Aug 14 18:20:02 2003 Subject: [Python-Dev] Exposing socket and poll objects in the Concrete Object Layer In-Reply-To: <2msmo4pgl6.fsf@starship.python.net> References: <200308130902.17667.michel@dialnetwork.com> <2msmo4pgl6.fsf@starship.python.net> Message-ID: <200308141516.28364.michel@dialnetwork.com> On Thursday 14 August 2003 07:29, Michael Hudson wrote: > > Hmm. I think it would be best to implement these in a CObject-y > style, like the way cPickle uses StringIO and the way (I presume, > haven't actually looked) _socket uses _ssl. Ooohhhh I didn't know about CObject until you mentioned it. This is clearly the way to go (still ugly, but at least endorsed). > > Now that new style classes make it possible for people to write > > classes in C, I think it will become a much more common practice, > > and that Python programmers moving "down" to C will expect to be > > able to use services in C that they take for granted in Python. > > This would also please those people who try to sneak Python onto > projects with cries of "it's just a library of useful C routines!" :-) It's a good argument. I guess it depends on the goals of the AOL and COL APis. Personally I think it would be worth the effort trying to make the APIs close to the same richness that Python provides. This is a huge advantage for Jython; Java programmers do not need to jump through CObject-like hoops to use other Java objects. CObject may not be painful enough to warrent the extra work (It doesn't seem incredibly painful). Something in the docs caught my eye: Doc/html/ext/using-cobjects.html "At first sight this [providing extension module services to other extension modules] seems easy: just write the functions (without declaring them static, of course), provide an appropriate header file, and document the C API. And in fact this would work if all extension modules were always linked statically with the Python interpreter. When modules are used as shared libraries, however, the symbols defined in one module may not be visible to another module. The details of visibility depend on the operating system; ..." Are socketmodule and selectmodule always linked staticly with the Python interpreter? If this is the case I think its good to do as the first part of this paragraph says, write the functions, provide a header file, and document the API. The COL will only get more functional. -Michel From eric at enthought.com Thu Aug 14 19:00:21 2003 From: eric at enthought.com (eric jones) Date: Thu Aug 14 19:00:34 2003 Subject: [Python-Dev] ANN: Job posting -- developer position available Message-ID: <010701c362c0$3b54ec50$a47ba8c0@ericlaptop> Hey group, We have a developer position open here at Enthought, and I thought I'd see if anyone is interested. The posting is below. There is also a tech writer and HCI opening listed on the website: http://www.enthought.com/careers.html thanks! eric Position: Python Developer Type: Full-Time Enthought, Inc. is a technology startup in the Austin, TX area that is looking for a motivated Python Developer to join our development team. Responsibilities include developing and deploying scientific applications within the Fortune 500. Our size and prospects provide significant opportunity for growth and advancement for Enthought employees. Candidates should have the following qualifications: - BS or MS in Computer Science or Engineering or other scientific field - 3+ years experience in the software industry - Experience developing solutions in the following languages: - Python - C/C++ - User interface design experience - Developing and deploying custom solutions for clients - Excellent verbal and written communication skills Ideal candidates will also demonstrate the following skills: - Use of the wxPython user interface library - Experience with Geophysics applications - Experience in the fields of statistics, linear algebra and calculus Please send resumes to jobs@enthought.com. From tim.one at comcast.net Thu Aug 14 22:17:51 2003 From: tim.one at comcast.net (Tim Peters) Date: Thu Aug 14 21:18:25 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mk79gp53x.fsf@starship.python.net> Message-ID: [Michael Hudson] > Here's the output of a recent run: > ... > test_complex leaked 2 references For some reason that irritated me . It's fixed now -- really was a leak! From tim.one at comcast.net Thu Aug 14 22:29:34 2003 From: tim.one at comcast.net (Tim Peters) Date: Thu Aug 14 21:30:08 2003 Subject: [Python-Dev] New bug in listobject.c Message-ID: list_ass_slice() currently looks like: ... int n; /* Size of replacement list */ int d; /* Change in size */ int k; /* Loop index */ #define b ((PyListObject *)v) if (v == NULL) n = 0; else { char msg[256]; if (a == b) { /* Special case "a[i:j] = a" -- copy b first */ int ret; v = list_slice(b, 0, n); MSVC complains that n can be used before initialization in the last line. Looks like a fair complaint to me! SF ViewCVS doesn't show this code, so it must be very new. From bac at OCF.Berkeley.EDU Thu Aug 14 19:44:00 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Thu Aug 14 21:44:26 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mk79gp53x.fsf@starship.python.net> References: <2mk79gp53x.fsf@starship.python.net> Message-ID: <3F3C3AE0.70003@ocf.berkeley.edu> Michael Hudson wrote: > Prompted by finding my own refcounting leaks in test_descr, I'm now > looking to spread the misery :-) > > I've hacked regrtest (hacks attached) to do some very crude monitoring > of sys.gettotalrefcount(). > Some however, seem to be real. test_sax, test_socket and > test_codeccallbacks seem to be the most alarming. The TrackRef class > Tim posted a link to is likely to be indispensable in hunting these. > Is there any reason not to add Michael's code to regrtest.py (or at least get it to the point of where it could be added)? Finding leaks as part of running a test would be nice. We could even have a set in regrtest.py that stored tests known to throw the test off (because of caching or whatever). Obviously testing would be option that is off by default. How about Tim's code class? Or is finding leaks that much of a random black art and there is not good way of doing it? Part of the reason I ask is I have had a patch sitting on SF for a while now that implements the skips.txt file idea that was brought up back when Martin was complaining about expected skips on certain platforms and such. Is there a general view of not touching regrtest.py unless needed since breaking that would not be fun? -Brett From mhammond at skippinet.com.au Fri Aug 15 15:05:52 2003 From: mhammond at skippinet.com.au (Mark Hammond) Date: Fri Aug 15 00:05:49 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <3F3C3AE0.70003@ocf.berkeley.edu> Message-ID: <095701c362e2$85023070$f502a8c0@eden> > > > Is there any reason not to add Michael's code to regrtest.py (or at > least get it to the point of where it could be added)? > Finding leaks as > part of running a test would be nice. We could even have a set in > regrtest.py that stored tests known to throw the test off (because of > caching or whatever). Obviously testing would be option that > is off by I added some reference count leak tests to the pyxpcom test suite. The general strategy was to run a test once, count the references, then run the tests n (for a large n) more times. The first run before starting to count reduces the effect of caches. Then only a "few" leaked references is likely to be related to the environment, whereas >n leaked references is likely to be a real leak. It seems to be working fairly well, and trapped a number of old and new leaks. Mark. From mhammond at skippinet.com.au Fri Aug 15 15:11:21 2003 From: mhammond at skippinet.com.au (Mark Hammond) Date: Fri Aug 15 00:11:02 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: <20030814200819.GA2161@glacier.arctrix.com> Message-ID: <095801c362e3$494da770$f502a8c0@eden> > I mean that Unix users are less likely to want a stand-alone > executable > instead of a Python script. Also, stand-alone executables are usually > intended for end users with little or no Python development skills. > Almost all of those people are using windows. But isn't at least part of the Linux community hoping that this will change, and that Linux will become viable for such users? Mark. From bac at OCF.Berkeley.EDU Thu Aug 14 22:21:19 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 15 00:21:50 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <095701c362e2$85023070$f502a8c0@eden> References: <095701c362e2$85023070$f502a8c0@eden> Message-ID: <3F3C5FBF.4050403@ocf.berkeley.edu> Mark Hammond wrote: >> >> >>Is there any reason not to add Michael's code to regrtest.py (or at >>least get it to the point of where it could be added)? >>Finding leaks as >>part of running a test would be nice. We could even have a set in >>regrtest.py that stored tests known to throw the test off (because of >>caching or whatever). Obviously testing would be option that >>is off by > > > I added some reference count leak tests to the pyxpcom test suite. The > general strategy was to run a test once, count the references, then run the > tests n (for a large n) more times. The first run before starting to count > reduces the effect of caches. Then only a "few" leaked references is likely > to be related to the environment, whereas >n leaked references is likely to > be a real leak. > > It seems to be working fairly well, and trapped a number of old and new > leaks. > Sounds good. Could make the value of n part of the command-line option. Any interest of adding them to regrtest.py? -Brett From bac at OCF.Berkeley.EDU Thu Aug 14 23:45:34 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 15 01:45:43 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <20030807151725.GA28082@panix.com> References: <20030807144239.GA16130@panix.com> <3F326C56.5000306@python.org> <20030807151725.GA28082@panix.com> Message-ID: <3F3C737E.8050409@ocf.berkeley.edu> Aahz wrote: > On Thu, Aug 07, 2003, David Goodger wrote: > >>Aahz wrote: >> >>>which means that it can be used in a form. I've added a mini-form to >>> >>> http://www.python.org/dev/ >> >>That's cool. Would it be possible to provide a minimal form at >> (i.e., instead of the current "You >>did not provide a report number")? > > > Possible, but I don't feel like thinking about it. As always, patches > are welcome. ;-) Where is it kept? I have always wondered where the heck that script was. -Brett From doko at cs.tu-berlin.de Fri Aug 15 09:33:51 2003 From: doko at cs.tu-berlin.de (Matthias Klose) Date: Fri Aug 15 02:37:04 2003 Subject: [Python-Dev] autoconf's AM_CHECK_FUNCS breaks python2.3 on HPUX 11.11 Message-ID: <16188.32463.461438.334028@gargle.gargle.HOWL> Reported to the autoconf developers, but it badly breaks python2.3 on HPUX 11.11. The test for AC_CHECK_FUNCS(mktime, strftime, strptime, select) and some other time functions is broken on HPUX 11.11. The test snippet #ifdef __STDC__ # include #else # include #endif includes sys/time.h, which contains the declarations for the above functions (among others). Changing this to #if defined(__STDC) && !defined(__hpux__) fixes this (although I don't know, if autoconf may use the "defined" syntax). There is at least one more function not detected by this change: gettimeofday. I didn't investigate yet, but hand edited pyconfig.h. Maybe the generated configure could be hand-edited for releases or a note added for the platform specific notes. Matthias From mwh at python.net Fri Aug 15 12:48:42 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 15 06:48:45 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <3F3BD826.5060600@livinglogic.de> (Walter =?iso-8859-1?q?D=F6rwald's?= message of "Thu, 14 Aug 2003 20:42:46 +0200") References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> Message-ID: <2m65kzp56d.fsf@starship.python.net> Walter D?rwald writes: > Michael Hudson wrote: > >> Prompted by finding my own refcounting leaks in test_descr, I'm now >> looking to spread the misery :-) >> I've hacked regrtest (hacks attached) to do some very crude >> monitoring >> of sys.gettotalrefcount(). >> Here's the output of a recent run: >> > > [...] >> test_codeccallbacks leaked 1107 references > > [...] >> Some however, seem to be real. test_sax, test_socket and >> test_codeccallbacks seem to be the most alarming. The TrackRef class >> Tim posted a link to is likely to be indispensable in hunting these. > > The number for test_codeccallbacks really looks scary. Some of it is > the result of the callback registry, A ha. I found a lot of things like that that I didn't know existed in the last few days... some of the leaks I posted are due to use of warnings.filterwarnings, too. > but there seem to be real leaks here. In a perverse kind of way, phew :-) Wouldn't want to have gone to all this effort to uncover *nothing* but a bunch of false alarms... > I've used the attached patch to unittest to get refcount diffs for > individual test methods. Here are the results for test_codeccallbacks: > > 212 test_backslashescape > 1 test_badandgoodbackslashreplaceexceptions > 0 test_badandgoodignoreexceptions > 0 test_badandgoodreplaceexceptions > 0 test_badandgoodstrictexceptions > 0 test_badandgoodxmlcharrefreplaceexceptions > 271 test_badhandlerresults > 0 test_badregistercall > 153 test_callbacks > 3 test_charmapencode > 121 test_decodehelper > 26 test_encodehelper > 285 test_longstrings > 0 test_lookup > 9 test_relaxedutf8 > 1 test_translatehelper > 11 test_unencodablereplacement > 265 test_unicodedecodeerror > 792 test_unicodeencodeerror > 0 test_unicodetranslateerror > 12 test_uninamereplace > 0 test_unknownhandler > 14 test_xmlcharnamereplace > 0 test_xmlcharrefreplace > 6 test_xmlcharrefvalues > > Bye, > Walter D?rwald > > Index: Lib/unittest.py > =================================================================== > RCS file: /cvsroot/python/python/dist/src/Lib/unittest.py,v > retrieving revision 1.24 > diff -u -r1.24 unittest.py > --- Lib/unittest.py 13 Jul 2003 15:18:12 -0000 1.24 > +++ Lib/unittest.py 14 Aug 2003 18:37:19 -0000 > @@ -376,7 +376,13 @@ > for test in self._tests: > if result.shouldStop: > break > + import gc, sys > + gc.collect() > + rc1 = sys.gettotalrefcount() > test(result) > + gc.collect() > + rc2 = sys.gettotalrefcount() > + print >>sys.stderr, "%5d %s" % (rc2-rc1, test) > return result > > def debug(self): In general (not sure about these tests) you want to run each test a few time to let things settle down before measuring the effect on gettotalrefcount(). Cheers, mwh -- same software, different verbosity settings (this one goes to eleven) -- the effbot on the martellibot From mwh at python.net Fri Aug 15 12:50:41 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 15 06:50:44 2003 Subject: [Python-Dev] New bug in listobject.c In-Reply-To: ("Tim Peters"'s message of "Thu, 14 Aug 2003 21:29:34 -0400") References: Message-ID: <2mznibnqim.fsf@starship.python.net> "Tim Peters" writes: > list_ass_slice() currently looks like: > > ... > int n; /* Size of replacement list */ > int d; /* Change in size */ > int k; /* Loop index */ > #define b ((PyListObject *)v) > if (v == NULL) > n = 0; > else { > char msg[256]; > if (a == b) { > /* Special case "a[i:j] = a" -- copy b first */ > int ret; > v = list_slice(b, 0, n); > > MSVC complains that n can be used before initialization in the last line. > Looks like a fair complaint to me! SF ViewCVS doesn't show this code, so it > must be very new. My fault. Will fix momentarily. Cheers, mwh -- NUTRIMAT: That drink was individually tailored to meet your personal requirements for nutrition and pleasure. ARTHUR: Ah. So I'm a masochist on a diet am I? -- The Hitch-Hikers Guide to the Galaxy, Episode 9 From mwh at python.net Fri Aug 15 12:52:58 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 15 06:53:00 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: ("Tim Peters"'s message of "Thu, 14 Aug 2003 21:17:51 -0400") References: Message-ID: <2mwudfnqet.fsf@starship.python.net> "Tim Peters" writes: > [Michael Hudson] >> Here's the output of a recent run: >> ... >> test_complex leaked 2 references > > For some reason that irritated me . It's fixed now -- really was a > leak! Well, wasting everyone else's time was the point of that posting . Actually, finding these suckers requires a blend of knowledge, tool smithing, intuition and downright guesswork that's actually quite entertaining (in a screamingly geeky fashion, of course). Cheers, mwh -- You're posting to a Scheme group. Around here, arguing that Java is better than C++ is like arguing that grasshoppers taste better than tree bark. -- Thant Tessman, comp.lang.scheme From mwh at python.net Fri Aug 15 13:06:39 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 15 07:06:42 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <3F3C3AE0.70003@ocf.berkeley.edu> ("Brett C."'s message of "Thu, 14 Aug 2003 18:44:00 -0700") References: <2mk79gp53x.fsf@starship.python.net> <3F3C3AE0.70003@ocf.berkeley.edu> Message-ID: <2msmo3nps0.fsf@starship.python.net> "Brett C." writes: > Michael Hudson wrote: > >> Prompted by finding my own refcounting leaks in test_descr, I'm now >> looking to spread the misery :-) >> I've hacked regrtest (hacks attached) to do some very crude >> monitoring >> of sys.gettotalrefcount(). > > > >> Some however, seem to be real. test_sax, test_socket and >> test_codeccallbacks seem to be the most alarming. The TrackRef class >> Tim posted a link to is likely to be indispensable in hunting these. >> > > > > Is there any reason not to add Michael's code to regrtest.py Because it's at least something of a horrendous hack, at the moment. > (or at least get it to the point of where it could be added)? That's a better question . > Finding leaks as part of running a test would be nice. Certainly! Then we might have found these *before* a major release... though most of the leaks found so far are obscure enough not to overly concern me. > We could even have a set in regrtest.py that stored tests known to > throw the test off (because of caching or whatever). What my latest version does is run the test nine (!) times, and if there seems to be a refcount abnormality, prints the refcount deltas for the final four runs. This helps spot some faux leaks (e.g. test_mutants which does randomized testing). Also, it's possible & desrible to do much finer testing on the more structured tests (unittests & probably doctests, too). > Obviously testing would be option that is off by default. Yup, for several reasons... > How about Tim's code class? It should probably be floating around somewhere helpful, but I don't think there's a way of automating this kind of thing. If regrtest can optionally point you at a problem, that's good enough, IMHO. > Or is finding leaks that much of a random black art and there is not > good way of doing it? > > Part of the reason I ask is I have had a patch sitting on SF for a > while now that implements the skips.txt file idea that was brought up > back when Martin was complaining about expected skips on certain > platforms and such. Is there a general view of not touching > regrtest.py unless needed since breaking that would not be fun? I don't think so. Doing it a couple of weeks before 2.3 would have been insane, but I don't think there's any special reason to avoid enhancing regrtest. The Python test suite is a bit untidy with the various methods of testing (flat file, with test_main, unittests, doctests), but people have been gradually unittest-ifying the tests, and I don't think there's any reason to stop this or agitate to get it going faster. Cheers, mwh -- This is the fixed point problem again; since all some implementors do is implement the compiler and libraries for compiler writing, the language becomes good at writing compilers and not much else! -- Brian Rogoff, comp.lang.functional From mwh at python.net Fri Aug 15 13:12:11 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 15 07:12:13 2003 Subject: [Python-Dev] Exposing socket and poll objects in the Concrete Object Layer In-Reply-To: <200308141516.28364.michel@dialnetwork.com> (Michel Pelletier's message of "Thu, 14 Aug 2003 15:16:28 -0500") References: <200308130902.17667.michel@dialnetwork.com> <2msmo4pgl6.fsf@starship.python.net> <200308141516.28364.michel@dialnetwork.com> Message-ID: <2mptj7npis.fsf@starship.python.net> Michel Pelletier writes: > On Thursday 14 August 2003 07:29, Michael Hudson wrote: >> >> Hmm. I think it would be best to implement these in a CObject-y >> style, like the way cPickle uses StringIO and the way (I presume, >> haven't actually looked) _socket uses _ssl. > > Ooohhhh I didn't know about CObject until you mentioned it. This is clearly > the way to go (still ugly, but at least endorsed). And the ugliness can largely be hidden in the implementation, too. > Something in the docs caught my eye: > > Doc/html/ext/using-cobjects.html > > "At first sight this [providing extension module services to other extension > modules] seems easy: just write the functions (without declaring them static, > of course), provide an appropriate header file, and document the C API. And > in fact this would work if all extension modules were always linked > statically with the Python interpreter. When modules are used as shared > libraries, however, the symbols defined in one module may not be visible to > another module. The details of visibility depend on the operating system; > ..." > > Are socketmodule and selectmodule always linked staticly with the Python > interpreter? $ ls build/lib.linux-i686-2.4/_socket.so build/lib.linux-i686-2.4/_socket.so Doesn't look like it :-) Cheers, mwh -- > So what does "abc" / "ab" equal? cheese -- Steve Holden defends obscure semantics on comp.lang.python From sholden at holdenweb.com Fri Aug 15 08:11:30 2003 From: sholden at holdenweb.com (Steve Holden) Date: Fri Aug 15 07:13:25 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: <095801c362e3$494da770$f502a8c0@eden> Message-ID: [Mark] > [Neil] > > I mean that Unix users are less likely to want a stand-alone > > executable > > instead of a Python script. Also, stand-alone executables > are usually > > intended for end users with little or no Python development skills. > > Almost all of those people are using windows. > > But isn't at least part of the Linux community hoping that > this will change, > and that Linux will become viable for such users? > Besides which, stand-alone executable delivery means freedom from dependencies on the installed Python, which for a long time was a complaint on Red Hat (and may well become one for various platforms when people start trying to deliver 2.3-specific functionality). IMHO It would be a mistake to overlook the Unix world in delivering this ability to a waiting world. regards -- Steve Holden http://www.holdenweb.com/ Python Web Programming http://pydish.holdenweb.com/pwp/ From michal at sabren.com Fri Aug 15 09:13:24 2003 From: michal at sabren.com (Michal Wallace) Date: Fri Aug 15 08:13:36 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: Message-ID: On Fri, 15 Aug 2003, Steve Holden wrote: > Besides which, stand-alone executable delivery means freedom from > dependencies on the installed Python, which for a long time was a > complaint on Red Hat (and may well become one for various platforms when > people start trying to deliver 2.3-specific functionality). I'll second that. I still have to keep 1.5.2 around on all my servers just to run up2date. :) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From gmcm at hypernet.com Fri Aug 15 09:24:36 2003 From: gmcm at hypernet.com (Gordon McMillan) Date: Fri Aug 15 08:24:18 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: Message-ID: <3F3C98C4.19981.4C64CC0A@localhost> [Skip Montanaro] > I don't think zipimport supports importing from zip files > with appended stuff. I suspect it needs to know the last N > bytes form the zip length. Don't know about zipimport, but zip files are designed to be opened from the back. [Neil Schemenauer] > I suppose some executable formats do not like extra data > appended at the end. It's an ELF / COFF trick. [Neil Schemenauer again] > I mean that Unix users are less likely to want a stand-alone > executable instead of a Python script. Also, stand-alone > executables are usually intended for end users with little or > no Python development skills. Almost all of those people are > using windows. It may be a surprise to the developer community, but there are a lot of Unix users who are not developers and do not have root priveleges on their machines. These people want to install to ~/bin and don't particularly care what the install looks like. In this country it's usually analysts, statisticians and scientists. In other countries where MS is not so popular, it's anybody. [in general] The trick being discussed really doesn't cut the mustard on *nix anyway, because shared libs will be involved. -- Gordon http://www.mcmillan-inc.com/ From amk at amk.ca Fri Aug 15 09:31:07 2003 From: amk at amk.ca (A.M. Kuchling) Date: Fri Aug 15 08:31:13 2003 Subject: [Python-Dev] Re: Re: hook for standalone executable References: <20030814200819.GA2161@glacier.arctrix.com> <095801c362e3$494da770$f502a8c0@eden> Message-ID: On Fri, 15 Aug 2003 14:11:21 +1000, Mark Hammond wrote: > But isn't at least part of the Linux community hoping that this will > change, > and that Linux will become viable for such users? Any Linux distribution aimed at end users will use some packaging format and provide dependency checking. We'd need better support for building packages using Distutils; RPM support is there, but dpkg isn't. --amk From walter at livinglogic.de Fri Aug 15 15:38:20 2003 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Fri Aug 15 08:38:56 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2m65kzp56d.fsf@starship.python.net> References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> <2m65kzp56d.fsf@starship.python.net> Message-ID: <3F3CD43C.3090206@livinglogic.de> Michael Hudson wrote: > [...] >> >>>test_codeccallbacks leaked 1107 references > [...] >>but there seem to be real leaks here. > > In a perverse kind of way, phew :-) Wouldn't want to have gone to all > this effort to uncover *nothing* but a bunch of false alarms... I've fixed two of the leaks. > [...] > > In general (not sure about these tests) you want to run each test a > few time to let things settle down before measuring the effect on > gettotalrefcount(). I think I'll try that, but this will take ages to run. Meanwhile here is the result of my patch for the complete test suite: http://styx.livinglogic.de/~walter/reflog3.txt (This includes only unittest based tests) It would simplify hunting leaks if we separated tests that are known to change the total refcount from the rest by moving them to separate test methods or even test cases. Bye, Walter D?rwald From mwh at python.net Fri Aug 15 15:13:45 2003 From: mwh at python.net (Michael Hudson) Date: Fri Aug 15 09:13:55 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <3F3CD43C.3090206@livinglogic.de> (Walter =?iso-8859-1?q?D=F6rwald's?= message of "Fri, 15 Aug 2003 14:38:20 +0200") References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> <2m65kzp56d.fsf@starship.python.net> <3F3CD43C.3090206@livinglogic.de> Message-ID: <2mk79fnjw6.fsf@starship.python.net> Walter D?rwald writes: > Michael Hudson wrote: > >> [...] >>> >>>>test_codeccallbacks leaked 1107 references >> [...] >>>but there seem to be real leaks here. >> In a perverse kind of way, phew :-) Wouldn't want to have gone to all >> this effort to uncover *nothing* but a bunch of false alarms... > > I've fixed two of the leaks. Cool. Do you think that's it for real leaks in test_codeccallbacks? > > [...] >> In general (not sure about these tests) you want to run each test a >> few time to let things settle down before measuring the effect on >> gettotalrefcount(). > > I think I'll try that, but this will take ages to run. No kidding. > Meanwhile here is the result of my patch for the complete test > suite: > > http://styx.livinglogic.de/~walter/reflog3.txt > > (This includes only unittest based tests) Cool. Is this from CVS head? I thought a bunch of leaks in arrays had already been fixed. > It would simplify hunting leaks if we separated tests that are > known to change the total refcount from the rest by moving > them to separate test methods or even test cases. Sure would! Not sure that's a trivial proposition, though. Cheers, mwh -- I think perhaps we should have electoral collages and construct our representatives entirely of little bits of cloth and papier mache. -- Owen Dunn, ucam.chat, from his review of the year From skip at pobox.com Fri Aug 15 09:31:46 2003 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 15 09:32:04 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <3F3C737E.8050409@ocf.berkeley.edu> References: <20030807144239.GA16130@panix.com> <3F326C56.5000306@python.org> <20030807151725.GA28082@panix.com> <3F3C737E.8050409@ocf.berkeley.edu> Message-ID: <16188.57538.419799.220344@montanaro.dyndns.org> >>> That's cool. Would it be possible to provide a minimal form at >>> (i.e., instead of the current "You did >>> not provide a report number")? ... Brett> Where is it kept? I have always wondered where the heck that Brett> script was. It's in the cgi-bin directory on creosote. I modified it for the SpamBayes project and have it installed on the Musi-Cal staging server for the moment. You can use it here: http://staging.musi-cal.com/cgi-bin/sf It has the simple form modification. I've attached the code. Skip -------------- next part -------------- A non-text attachment was scrubbed... Name: sf Type: application/octet-stream Size: 2586 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20030815/cdc9728d/sf.obj From walter at livinglogic.de Fri Aug 15 17:36:30 2003 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Fri Aug 15 10:37:06 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mk79fnjw6.fsf@starship.python.net> References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> <2m65kzp56d.fsf@starship.python.net> <3F3CD43C.3090206@livinglogic.de> <2mk79fnjw6.fsf@starship.python.net> Message-ID: <3F3CEFEE.8020307@livinglogic.de> Michael Hudson wrote: > Walter D?rwald writes: > >> Michael Hudson wrote: >> >>> [...] >>> >>>> but there seem to be real leaks here. >>> >>> >>> In a perverse kind of way, phew :-) Wouldn't want to have gone to all >>> this effort to uncover *nothing* but a bunch of false alarms... >> >> >> I've fixed two of the leaks. > > > Cool. Do you think that's it for real leaks in test_codeccallbacks? I'll try to go through the list and see if I can find any other leaks. test_callbacks() from the test is the next candidate, but it *does* register callbacks, so the registry comes into play. >> > [...] >> >>> In general (not sure about these tests) you want to run each test a >>> few time to let things settle down before measuring the effect on >>> gettotalrefcount(). >> >> >> I think I'll try that, but this will take ages to run. > > > No kidding. A job for the weekend. This leads to an idea: Maybe we should set up a cronjob that runs the tests and publishes the results somewhere on the web? >> Meanwhile here is the result of my patch for the complete test >> suite: >> >> http://styx.livinglogic.de/~walter/reflog3.txt >> >> (This includes only unittest based tests) > > > Cool. Is this from CVS head? Yes. > I thought a bunch of leaks in arrays > had already been fixed. I'll do an update before I start the job for the weekend just to be sure. >> It would simplify hunting leaks if we separated tests that are >> known to change the total refcount from the rest by moving >> them to separate test methods or even test cases. > > > Sure would! > > Not sure that's a trivial proposition, though. It isn't, we should finish the unittest migration first. Bye, Walter D?rwald From skip at pobox.com Fri Aug 15 10:38:01 2003 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 15 10:38:22 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mk79fnjw6.fsf@starship.python.net> References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> <2m65kzp56d.fsf@starship.python.net> <3F3CD43C.3090206@livinglogic.de> <2mk79fnjw6.fsf@starship.python.net> Message-ID: <16188.61513.82220.861108@montanaro.dyndns.org> >> I think I'll try that, but this will take ages to run. Michael> No kidding. If this was available via regrtest using a command line flag, couldn't you do something like python regrtest.py -L test_codeccallbacks to perform leak testing on just the one test suite? That would make it go a lot faster when trying to squash particular leaks, as long as they weren't related to other tests. Skip From tim.one at comcast.net Fri Aug 15 11:55:45 2003 From: tim.one at comcast.net (Tim Peters) Date: Fri Aug 15 10:56:19 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mwudfnqet.fsf@starship.python.net> Message-ID: [Michael Hudson] > Well, wasting everyone else's time was the point of that posting > . > > Actually, finding these suckers requires a blend of knowledge, tool > smithing, intuition and downright guesswork that's actually quite > entertaining (in a screamingly geeky fashion, of course). Yup! I played with test_complex as a break from some crushingly boring "real work". I'm still not sure why leaks in Python are usually so much easier to nail than leaks in Zope. It could be just because a Zope test sucks in a hundred thousand lines of Zope stuff no matter simple it is. From tim.one at comcast.net Fri Aug 15 12:07:49 2003 From: tim.one at comcast.net (Tim Peters) Date: Fri Aug 15 11:08:23 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <16188.61513.82220.861108@montanaro.dyndns.org> Message-ID: [Skip] > If this was available via regrtest using a command line flag, > couldn't you do something like > > python regrtest.py -L test_codeccallbacks > > to perform leak testing on just the one test suite? That would make > it go a lot faster when trying to squash particular leaks, as long as > they weren't related to other tests. Again it's a good idea to look at Zope's test drivers. For example, http://cvs.zope.org/Zope3/test.py implements an -L switch: -L Keep running the selected tests in a loop. You may experience memory leakage. In a debug build, that also automatically uses the TrackRefs class to break down leaks by object type leaking, and by nature (leaking new objects vs merely accumulating excess references without leaking new objects). Beware overdoing it, though! Once the initial leaks found are plugged, new ones are almost certainly caused by recent changes, and so much easier to find. A ton of delicate leak-tracking framework becomes counterproductive then. I think the Zope3 test driver (above) strikes a good balance. From tree at basistech.com Fri Aug 15 13:37:21 2003 From: tree at basistech.com (Tom Emerson) Date: Fri Aug 15 12:42:58 2003 Subject: [Python-Dev] Embedded python module search path Message-ID: <16189.3137.778095.103611@magrathea.basistech.com> I have a question for those of you who have embedded Python into a large application concerning how you handle the module search path and related functions. Currently the module search path (and the related directory names set as a side effect) is determined by looking at the environment that the executable calling Py_Initialize() is running in. Hence if I've embedded Python 2.3 and also have Python 2.3 installed in (say /usr/local) it is going to use the Python paths in /usr/local/ over those in my customized embedded version. As far as I can tell, the only way I can control this behavior is to rewrite Py_GetPath and friends in my custom build. In my case the user of my application has a configuration file which specifies the pathnames for platform (in-)dependent files, both Python and other. But I cannot pass this information on to Py_Initialize() and on into Py_GetPath. Is it worth providing an alternative initialization API that allows these values to be specified explicitly instead of having them computed? Or is there a reason not to do this? I appreciate the insight. -tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From cyril.lombardo at wanadoo.fr Fri Aug 15 22:12:09 2003 From: cyril.lombardo at wanadoo.fr (Cyril Lombardo) Date: Fri Aug 15 15:12:51 2003 Subject: [Python-Dev] SoS on Tkinter Message-ID: <000001c36361$205d9f90$8d9afea9@pent> Hello everyone, First, sorry for my poor english (i am french..) I try to develop an program with Tkinter. This program is a attentional psychological test witch is used for evaluate attention in schizophrenic patients. Methodology: 3 references images are in top-center of window in middle of windows, a series of 30 images (3 row of 10 images). If image on the bottom is identical at the references images, the patient must click on the mouse. I have make all interface code, but I need help for obtain the results of this task If its possible, I want know the number of good responses (when a patient click on good image (same at reference image)) and the number of wrong responses(when patient has click on image who has different of reference image). I want to know too, the number of bottom which has been analysed by the patient in 30 sec. I am a newbie in python development, and i think with your help I will finish this program. Summary Python code: root = Tk() root.title("jeu de carte") larg, haut=root.winfo_screenwidth(), root.winfo_screenheight() root.overrideredirect(1) root.geometry("%dx%d+0+0" % (larg, haut)) frame1= Frame(root) frame2= Frame(root) frame1.grid(row=0, column=0) frame2.grid(row=1, column=0) #card list liste_cartes = ['c:/carte1.gif', 'c:/carte2.gif', 'c:/carte3.gif', 'c:/carte4.gif', 'c:/carte5.gif', 'c:/carte6.gif', 'c:/carte7.gif', 'c:/carte8.gif', 'c:/carte9.gif', 'c:/carte10.gif'] #tirage al?atoire de l'image de la carte mod?le temp= liste_cartes carte_mod?le= random.choice(temp) #affichage de l'image de la carte mod?le dans le frame1 carte_mod?le = PhotoImage (file=carte_mod?le) label1= Label(frame1, image=carte_mod?le, borderwidth=5, relief=RIDGE) label1.grid() #Choix al?atoire des images de carte sur les boutons en frame 2 class ImButton(Button): def __init__(self,master,liste,**args): f=random.choice(liste) self.p=PhotoImage(file=f) #Cr?ation des Boutons r=2 while r >= 0: c=8 while c>=0: b=ImButton(frame2,liste_cartes) b.grid(row=r, column=c, padx= 3, pady= 3, ipadx=5, ipady= 5 r= r-1 root.mainloop() ________________________________________________________________________ __ Thkx a lot for you r help -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20030815/c7ab6db7/attachment.htm From bac at OCF.Berkeley.EDU Fri Aug 15 13:55:37 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 15 15:55:54 2003 Subject: [Python-Dev] SoS on Tkinter In-Reply-To: <000001c36361$205d9f90$8d9afea9@pent> References: <000001c36361$205d9f90$8d9afea9@pent> Message-ID: <3F3D3AB9.30803@ocf.berkeley.edu> Cyril Lombardo wrote: > Hello everyone, > First, sorry for my poor english (i am french..) > I try to develop an program with Tkinter. This program is a attentional > psychological test witch is used for evaluate attention in schizophrenic > patients. > This is the wrong place to ask this, Cyril. python-dev is used for the discussion of the development of Python. For help using Python try posting your question on comp.lang.python (which can also be reached through python-list@python.org). -Brett From bac at OCF.Berkeley.EDU Fri Aug 15 13:58:18 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 15 15:58:36 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <16188.57538.419799.220344@montanaro.dyndns.org> References: <20030807144239.GA16130@panix.com> <3F326C56.5000306@python.org> <20030807151725.GA28082@panix.com> <3F3C737E.8050409@ocf.berkeley.edu> <16188.57538.419799.220344@montanaro.dyndns.org> Message-ID: <3F3D3B5A.8050603@ocf.berkeley.edu> Skip Montanaro wrote: > >>> That's cool. Would it be possible to provide a minimal form at > >>> (i.e., instead of the current "You did > >>> not provide a report number")? > ... > Brett> Where is it kept? I have always wondered where the heck that > Brett> script was. > > It's in the cgi-bin directory on creosote. I modified it for the SpamBayes > project and have it installed on the Musi-Cal staging server for the > moment. You can use it here: > > http://staging.musi-cal.com/cgi-bin/sf > > It has the simple form modification. I've attached the code. > In other words it isn't in the CVS tree, huh? OK. So is there any procedure for editing the CGI scripts beyond sending out an email saying you changed something? Should I add mention of this at /dev/pydotorg/ ? -Brett From Jack.Jansen at cwi.nl Sat Aug 16 01:25:42 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Fri Aug 15 18:26:52 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <16189.3137.778095.103611@magrathea.basistech.com> Message-ID: <67C75048-CF6F-11D7-A73A-000A27B19B96@cwi.nl> On vrijdag, aug 15, 2003, at 18:37 Europe/Amsterdam, Tom Emerson wrote: > I have a question for those of you who have embedded Python into a > large application concerning how you handle the module search path and > related functions. > > Currently the module search path (and the related directory names set > as a side effect) is determined by looking at the environment that the > executable calling Py_Initialize() is running in. Hence if I've > embedded Python 2.3 and also have Python 2.3 installed in (say > /usr/local) it is going to use the Python paths in /usr/local/ over > those in my customized embedded version. > > As far as I can tell, the only way I can control this behavior is to > rewrite Py_GetPath and friends in my custom build. > > In my case the user of my application has a configuration file which > specifies the pathnames for platform (in-)dependent files, both Python > and other. But I cannot pass this information on to Py_Initialize() > and on into Py_GetPath. > > Is it worth providing an alternative initialization API that allows > these values to be specified explicitly instead of having them > computed? Or is there a reason not to do this? +1. There is a hack you can use nowadays, but a hack it truly is: fiddle _environ before calling Py_Initialize(). -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From Jack.Jansen at cwi.nl Sat Aug 16 01:31:39 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Fri Aug 15 18:31:47 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <3F3D3B5A.8050603@ocf.berkeley.edu> Message-ID: <3C8F654E-CF70-11D7-A73A-000A27B19B96@cwi.nl> The original question asked about "a minimal form", but shouldn't we make it a bit less minimal than this one? At the very least I think there should be a link back into the "known universe", probably to a place where the procedure for submitting Python bugs is explained (and which I can't find right now:-(). -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From bac at OCF.Berkeley.EDU Fri Aug 15 16:40:05 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Fri Aug 15 18:40:28 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <3C8F654E-CF70-11D7-A73A-000A27B19B96@cwi.nl> References: <3C8F654E-CF70-11D7-A73A-000A27B19B96@cwi.nl> Message-ID: <3F3D6145.3000900@ocf.berkeley.edu> Jack Jansen wrote: > The original question asked about "a minimal form", but shouldn't we > make it > a bit less minimal than this one? > > At the very least I think there should be a link back into the "known > universe", > probably to a place where the procedure for submitting Python bugs is > explained > (and which I can't find right now:-(). If I can find out where the file is kept (I know Skip attached the file but I just want to understand how the whole CGI setup works on creosote before I do anything) I plan on coming up with a page that takes a number and has relevant links to the parts on SF and such. As for the lack of docs on submitting a bug and such, all that I know of is http://www.python.org/dev/devfaq.html#bugs . I just gave a talk about how Python is developed that was well-received that I plan to write up into an essay and get that put up on the site. It covers how to submit a bug (among other things). -Brett From fdrake at acm.org Fri Aug 15 19:43:37 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Fri Aug 15 18:44:16 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <3F3D6145.3000900@ocf.berkeley.edu> References: <3C8F654E-CF70-11D7-A73A-000A27B19B96@cwi.nl> <3F3D6145.3000900@ocf.berkeley.edu> Message-ID: <16189.25113.34410.62919@grendel.zope.com> Brett C. writes: > As for the lack of docs on submitting a bug and such, all that I know of > is http://www.python.org/dev/devfaq.html#bugs . I just gave a talk > about how Python is developed that was well-received that I plan to > write up into an essay and get that put up on the site. It covers how > to submit a bug (among other things). The Library Reference has a short section on submitting bug reports for Python as well. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From tree at basistech.com Fri Aug 15 21:52:31 2003 From: tree at basistech.com (Tom Emerson) Date: Fri Aug 15 21:03:54 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <67C75048-CF6F-11D7-A73A-000A27B19B96@cwi.nl> References: <16189.3137.778095.103611@magrathea.basistech.com> <67C75048-CF6F-11D7-A73A-000A27B19B96@cwi.nl> Message-ID: <16189.32847.817035.122283@magrathea.basistech.com> Jack Jansen writes: > There is a hack you can use nowadays, but a hack it truly is: fiddle > _environ before calling Py_Initialize(). Yes, but this is pretty ugly, and to get full coverage involves messing with PATH as well as PYTHONHOME et al. In the case of embedding I would expect that would usually does *not* want the standard search path to be constructed, or minimally only part of it. -tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From aahz at pythoncraft.com Fri Aug 15 22:58:20 2003 From: aahz at pythoncraft.com (Aahz) Date: Fri Aug 15 21:59:20 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <3F3D3B5A.8050603@ocf.berkeley.edu> References: <20030807144239.GA16130@panix.com> <3F326C56.5000306@python.org> <20030807151725.GA28082@panix.com> <3F3C737E.8050409@ocf.berkeley.edu> <16188.57538.419799.220344@montanaro.dyndns.org> <3F3D3B5A.8050603@ocf.berkeley.edu> Message-ID: <20030816015820.GA7605@panix.com> On Fri, Aug 15, 2003, Brett C. wrote: > > In other words it isn't in the CVS tree, huh? OK. So is there any > procedure for editing the CGI scripts beyond sending out an email saying > you changed something? Should I add mention of this at /dev/pydotorg/ ? It's supposed to be in the creosote RCS. Go ahead and add it to pydotorg; I'll be re-organizing the material Real Soon Now to make it more manageable. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From skip at pobox.com Fri Aug 15 22:28:42 2003 From: skip at pobox.com (Skip Montanaro) Date: Fri Aug 15 22:28:50 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <3C8F654E-CF70-11D7-A73A-000A27B19B96@cwi.nl> References: <3F3D3B5A.8050603@ocf.berkeley.edu> <3C8F654E-CF70-11D7-A73A-000A27B19B96@cwi.nl> Message-ID: <16189.38618.691298.471151@montanaro.dyndns.org> Jack> The original question asked about "a minimal form", but shouldn't Jack> we make it a bit less minimal than this one? I agree. It was just a 30-second change to do the minimum necessary to satisfy Aahz's requirements. ;-) Jack> At the very least I think there should be a link back into the Jack> "known universe", probably to a place where the procedure for Jack> submitting Python bugs is explained (and which I can't find right Jack> now:-(). I'll take a look at it this weekend. Skip From raymond.hettinger at verizon.net Sat Aug 16 04:52:37 2003 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sat Aug 16 03:57:45 2003 Subject: [Python-Dev] Optimization Clue Message-ID: <009e01c363cb$6d71eec0$e841fea9@oemcomputer> There is a 2:1 speed difference between dict.has_key() and dict.__contains__(): >>> setup = """ d = dict.fromkeys(range(1000)) hk = d.has_key cn = d.__contains__ """ >>> import timeit >>> min(timeit.Timer('hk(500)', setup).repeat(5)) 0.55875391810121755 >>> min(timeit.Timer('cn(500)', setup).repeat(5)) 1.0645585745654671 The implementations are almost identical except that has_key() is implemented as a METH_O PyCFunction attached to the method table and __contains__() is a regular C function attached to the sq_contains slot which, AFAICT, gets wrapped in a PyCFunction_Type. Then both has_key() and __contains__() get bundled in a method-wrapper object. I'm at the outer limits of my understanding here, but the above indicates to me that *all* of the slots with a (PyCFunction) signature could halve their access time, if instead of being wrapped in a PyCFunction_Type, they could just be added to the method table with the appropriate name for the slot, a METH_O flag, and a generic docstring. My first thought for an implementation is to append a forwarding function, and append it to the method definition table (manually at first, but using PyType_Ready later to do it automagically): static PyObject * directcontains(PyObject *mp, PyObject *key) { register long ok = mp->ob_type->tp_as_sequence->sq_contains(mp, key); return PyInt_FromLong(ok); /* Note this really should be PyBool */ } PyDoc_STRVAR(directcontains__doc__, "builtin __contains__ method"); static PyMethodDef mapp_methods[] = { {"__contains__", (PyCFunction)directcontains, METH_O, directcontains__doc__}, . . . In theory, this makes the two methods equally fast. In practice, the new function doesn't get run when d.__contains__() is called. Somewhere, I need to tell it that __contains__ refers to the new function instead of the old wrapper. If anyone knows where or has a better idea, please let me know. Raymond Hettinger ---------------------------------------------------------------------- P.S. Here's the source for dict.has_key() and dict.__contains__(): static PyObject * dict_has_key(register dictobject *mp, PyObject *key) { long hash; register long ok; if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { hash = PyObject_Hash(key); if (hash == -1) return NULL; } ok = (mp->ma_lookup)(mp, key, hash)->me_value != NULL; return PyBool_FromLong(ok); } static int dict_contains(dictobject *mp, PyObject *key) { long hash; if (!PyString_CheckExact(key) || (hash = ((PyStringObject *) key)->ob_shash) == -1) { hash = PyObject_Hash(key); if (hash == -1) return -1; } return (mp->ma_lookup)(mp, key, hash)->me_value != NULL; } From guido at python.org Sat Aug 16 08:53:07 2003 From: guido at python.org (Guido van Rossum) Date: Sat Aug 16 10:53:55 2003 Subject: [Python-Dev] Optimization Clue In-Reply-To: Your message of "Sat, 16 Aug 2003 03:52:37 EDT." <009e01c363cb$6d71eec0$e841fea9@oemcomputer> References: <009e01c363cb$6d71eec0$e841fea9@oemcomputer> Message-ID: <200308161453.h7GEr7b06879@12-236-84-31.client.attbi.com> > There is a 2:1 speed difference between dict.has_key() > and dict.__contains__(): > > >>> setup = """ > d = dict.fromkeys(range(1000)) > hk = d.has_key > cn = d.__contains__ > """ > >>> import timeit > >>> min(timeit.Timer('hk(500)', setup).repeat(5)) > 0.55875391810121755 > >>> min(timeit.Timer('cn(500)', setup).repeat(5)) > 1.0645585745654671 OTOH, on my machine, with the same setup: >>> # repeat the above for calibration >>> min(timeit.Timer('hk(500)', setup).repeat(5)) 0.86988496780395508 >>> min(timeit.Timer('cn(500)', setup).repeat(5)) 1.8778510093688965 >>> # try some other things >>> min(timeit.Timer('d.has_key(500)', setup).repeat(5)) 1.2841780185699463 >>> min(timeit.Timer('500 in d', setup).repeat(5)) 0.83904504776000977 >>> i.e. while explicitly calling __contains__ is slower than calling has_key, using the 'in' operator is faster than any of these. This is because when you explicitly use __contains__ in Python, you call a wrapper that adds extra overhead, while if the operator is used, everything stays at the C level. > The implementations are almost identical except that has_key() > is implemented as a METH_O PyCFunction attached to the method > table and __contains__() is a regular C function attached to > the sq_contains slot which, AFAICT, gets wrapped in a > PyCFunction_Type. Then both has_key() and __contains__() > get bundled in a method-wrapper object. > > I'm at the outer limits of my understanding here, but the above > indicates to me that *all* of the slots with a (PyCFunction) signature > could halve their access time, if instead of being wrapped in a > PyCFunction_Type, they could just be added to the method > table with the appropriate name for the slot, a METH_O flag, > and a generic docstring. My question is, is it worth it? As I showed above, when you call the operation directly rather than spelling out its special name in Python, you don't pay the wrapper overhead. > My first thought for an implementation is to append a forwarding > function, and append it to the method definition table (manually > at first, but using PyType_Ready later to do it automagically): The wrappers that are currently used are exactly what you propose: PyType_Ready automagically puts the right wrappers in the class dict. The wrappers are in typeobject.c; __contains__ uses wrap_objobjproc. But there's a second-tier wrapper used by the machinery that I can't be bothered to look up and describe right now; this probably causes the slowdown, and if you can find a way to avoid this, I'd be happy to consider it! [...] > static PyMethodDef mapp_methods[] = { > {"__contains__", (PyCFunction)directcontains, METH_O, > directcontains__doc__}, > . . . > > In theory, this makes the two methods equally fast. > In practice, the new function doesn't get run when d.__contains__() > is called. > > Somewhere, I need to tell it that __contains__ refers to the > new function instead of the old wrapper. If anyone knows > where or has a better idea, please let me know. That's because when there's a C slot, a wrapper gets stuffed into the class dict even if there is already something there. (There's an exception somewhere if the C slot is a wrapper for whatever's in the class dict.) --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Sat Aug 16 17:08:22 2003 From: barry at python.org (Barry Warsaw) Date: Sat Aug 16 12:08:23 2003 Subject: [Python-Dev] Re: SF tracker mini-form In-Reply-To: <3F3D6145.3000900@ocf.berkeley.edu> References: <3C8F654E-CF70-11D7-A73A-000A27B19B96@cwi.nl> <3F3D6145.3000900@ocf.berkeley.edu> Message-ID: <1061050067.18003.0.camel@anthem> On Fri, 2003-08-15 at 18:40, Brett C. wrote: > If I can find out where the file is kept (I know Skip attached the file > but I just want to understand how the whole CGI setup works on creosote > before I do anything) All cgi's are in /usr/local/apache/cgi-bin on creosote. Everything should be webmaster group readable and writable. -Barry From tim.one at comcast.net Sat Aug 16 13:25:41 2003 From: tim.one at comcast.net (Tim Peters) Date: Sat Aug 16 12:26:19 2003 Subject: [Python-Dev] World domination Message-ID: Following up on my Machiavellian plan to release the spambayes Outlook addin from SourceForge, the spambayes project ranked 99.9551% at SF last week, and is now on the front page as the 7th "most active" project (of 67,000) at SF last week. There have been at least 4,300 downloads of the OL addin from SF. Congratulations! I wish I could claim more credit for myself, but I've had little to do with it since last year. The credit belongs to the currently active developers, who've wrestled tirelessly with never-ending nightmares from Outlook to IMAP. Thanks to all -- great work! Neverthless, beatings will continue until it's #1 . To help brand recognition, I've changed the SF "public name" of the project from "Bayesian anti-spam classifier" to "SpamBayes anti-spam". taking-thrills-where-i-can-find-'em-ly y'rs - tim From bac at OCF.Berkeley.EDU Sat Aug 16 16:57:41 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sat Aug 16 18:57:47 2003 Subject: [Python-Dev] python-dev Summary for 2003-08-01 through 2003-08-15 [draft] Message-ID: <3F3EB6E5.2090609@ocf.berkeley.edu> This week I am under a time crunch. I am moving this Tuesday so in order for me to get this out to the rest of the world and up on to the archives I need corrections by Monday night. I am also a little unsure about the summary covering the standalone executable. Any help with that is appreciated. --------------------- python-dev Summary for 2003-08-01 through 2003-08-15 ++++++++++++++++++++++++++++++++++++++++++++++++++++ This is a summary of traffic on the `python-dev mailing list`_ from August 1, 2003 through August 15, 2003. It is intended to inform the wider Python community of on-going developments on the list. To comment on anything mentioned here, just post to python-list@python.org or `comp.lang.python`_ with a subject line mentioning what you are discussing. All python-dev members are interested in seeing ideas discussed by the community, so don't hesitate to take a stance on something. And if all of this really interests you then get involved and join `python-dev`_! This is the twenty-third summary written by Brett Cannon (about to move for the umpteenth time). All summaries are archived at http://www.python.org/dev/summary/ . Please note that this summary is written using reStructuredText_ which can be found at http://docutils.sf.net/rst.html . Any unfamiliar punctuation is probably markup for reST_ (otherwise it is probably regular expression syntax or a typo =); you can safely ignore it, although I suggest learning reST; its simple and is accepted for `PEP markup`_ and gives some perks for the HTML output. Also, because of the wonders of programs that like to reformat text, I cannot guarantee you will be able to run the text version of this summary through Docutils_ as-is unless it is from the original text file. .. _PEP Markup: http://www.python.org/peps/pep-0012.html The in-development version of the documentation for Python can be found at http://www.python.org/dev/doc/devel/ and should be used when looking up any documentation on something mentioned here. Python PEPs (Python Enhancement Proposals) are located at http://www.python.org/peps/ . To view files in the Python CVS online, go to http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ . .. _python-dev: http://www.python.org/dev/ .. _python-dev mailing list: http://mail.python.org/mailman/listinfo/python-dev .. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python .. _Docutils: http://docutils.sf.net/ .. _reST: .. _reStructuredText: http://docutils.sf.net/rst.html .. contents:: .. _last summary: http://www.python.org/dev/summary/2003-07-01_2003-07-31.html ===================== Summary Announcements ===================== Well, Michael Chermside responded to my question from the last summary about whether the new format and style of the Summaries was good. Once again a single person has led to how the summaries will be handled. You guys need to speak up (although I like the side that one this time =)! I am playing with layout once again. This time I am changing how the "contributing threads" lists are formatted. I know some of you hate inlined links in reST markup, but when these lists become huge it becomes really hard to keep track of the URIs when I have to list them away from the actual item on a separate line below the list of thread names. With `Python 2.3` out the door bug reports have started to come in. Work on 2.3.1 has begun. Please keep running the regression tests (easiest way is to run either ``make test`` or run regrtest.py in the test package; see the docs for the test package for help). On a personal note, if anyone knows of any Python users and such in the San Luis Obispo area of California, drop me a line at brett at python.org. ========= Summaries ========= ----------------------------------------- Python user helping Parrot? Treacherous! ----------------------------------------- Michal Wallace decided to get AM Kuchling's `previous work `__ on getting Python code to run on the Parrot_ virtual machine (which is what Perl 6 will use). Well, the rather nutty fellow managed to get pretty damn far with it as shown at http://pirate.versionhost.com/viewcvs.cgi/pirate/ . Michal was actually almost done with handling pure Python code and was getting ready to try to figure out how to get Parrot to handle C extension modules with that being the biggest sticking point. Since Parrot is not Python it does not have a parser for Python code; problem if your code has an exec statement. This turned out to not be a worry, though, since there are pure Python parsers out there. All of this has direct relevance to python-dev because of the bet between Guido and Dan Sugalski, developer of Parrot. The rules are outlined at http://www.sidhe.org/~dan/blog/archives/2003_07.html#000219 . What is going to happen at OSCON 2004 is a benchmark program written in pure Python will be run using a CVS checkout of Python against a Parrot (after converting the bytecode to Parrot's own bytecode) checkout; slowest implementation's author gets a pie in the face, buy the winner's dev team a round of beer, and $10. So why have this bet? This was discussed and basically came down to finding out whether Parrot really can run Python fast. Parrot wants to be the VM for as many languages as possible, including Python. This acts as a way to motivate people to see how feasible this is. And don't think that the CPython interpreter will disappear if Parrot wins. Dan pointed out that even if he did win the bet that Guido would probably want to keep CPython around since that is under his control and allows testing out new language features much easier then having to deal with Parrot and an external dev team. In other words, let other people worry about forcing a Python-shaped peg into a Parrot-sized hole. .. _Parrot: http://www.parrotcode.org/ Contributing threads: * `pirate (python+parrot) `__ ---------------------------------------------------- python-mode gets its own SF project; Vim users laugh ---------------------------------------------------- Barry Warsaw, citing lack of time to "properly maintain python-mode.el", has created a SourceForge project for the Emacs mode at http://sf.net/projects/python-mode . This means all bug reports, patches, etc. should be done at that project. Martin v. L?wis suggested removing `python-mode.el`_ from Python itself and to get it distributed with Emacs_ and XEmacs_. This way there does not have to be any synchronization between the new SF project and the Python CVS tree. As of right now, though, python-mode.el is still in the Python CVS. And to give equal billing to Vim_, my code editor of choice, since it does not get as much coverage on python-dev as XEmacs does, here are some scripts you might want to check out: taglist.vim : http://vim.sourceforge.net/scripts/script.php?script_id=273 "provides an overview of the structure of source code files" by splitting the window. python_fold.vim : http://vim.sourceforge.net/scripts/script.php?script_id=515 "This script uses the expr fold-method to create folds for python source code." vimDebug : http://vim.sourceforge.net/scripts/script.php?script_id=663 " integrates your favorite debugger with vim." .. _python-mode.el: http://sf.net/projects/python-mode .. _Emacs: http://www.gnu.org/software/emacs/emacs.html .. _XEmacs: http://www.xemacs.org/ .. _Vim: http://www.vim.org/ Contributing threads: * `New python-mode project at SourceForge `__ * New place for python-mode bug reports and patches `__ -------------------- Caching tuple hashes -------------------- Raymond Hettinger asked if there was any reason why tuples, being immutable, didn't cache their hash values. Strings cache their hashes and they are immutable, so it seem to make sense. It was pointed out, though, that tuples could contain an object that changed its hash value between hash calls. Guido said, though, that it was the responsibility of the object and not tuples to worry about keeping a consistent hash value. Guido also explained why strings cache their hashes. It turns out that since strings are used so often for keys in dicts that caching their hashes gave a large performance boost for almost any program, so the effort was felt justified. But Guido did not see this same justification for tuples. Thus tuples will not cache their hash values. Contributing threads: * `Caching tuple hashes `__ ------------------------------- PyCon 2004 is under development ------------------------------- Preparation for PyCon_ 2004 have now begun. With us getting the ball rolling much earlier this conference should turn out to be even better than last year (which, in my opinion, was great)! You can go to http://www.python.org/pycon/dc2004/ to find links to lists to get involved in organizing or just to be kept abreast of new developments. .. _PyCon: http://www.python.org/pycon/dc2004/ Contributing threads: * `PyCon DC 2004 Kickoff! `__ ---------------------------- Let the fixing of 2.3 begin! ---------------------------- The maintenance branch of Python 2.3 (named release23-maint) has been created in CVS. Several bugs have already been fixed. For instructions on how to check out a copy of the branch, read http://www.python.org/dev/devfaq.html#c10 . Contributing threads: * `Python 2.3 maintenance branch is now open `__ ------------------------------------ When a void doesn't equal an integer ------------------------------------ After clearing up the confusing issue of the difference between a Python int and a Python Integer (the former is while the latter is the union of and ), the discussion of how to handle a quirk of Win64 commenced. It turns out that Win64 thinks that ``sizeof(void *) > sizeof(long)`` is a reasonable thing to do. Well, all other platforms now and most likely for the rest of Tim Peter's life (at least according to Tim's guess) won't hold this to be true. As of right now Python claims that a void pointer can be held in a Python Integer, but this breaks that claim. Do we muck around with Python's internals for this one strange exception that does not make sense, such as making Python ints use long long? No, of course not. Code would break let alone the added complexity. So the needed changes to the failing tests were dealt with and all seems to be fine with the world again ... well, Win64 is still thinking on crack, but that is someone else's problem. Contributing threads: * `sizeof(long) != sizeof(void*) `__ ------------------------------------ Where do I put all of my packages?!? ------------------------------------ As it stands now there are four places where packages can reside: 1. standard library 2. site-packages 3. site-python (on UNIX this is the same directory where your python2.3 directory exists) 4. PYTHONPATH environment variable It was pointed out that this does not necessarily give one all the options one might want. The above covers the Python core easily, but there is not clear distinction for vendor packages, network-installed packages, system-wide packages or user-installed packages; they are covered by 2-4 above. The suggestion was to make more distinct places available for installation and to make distutils aware of them. A good way to see how this would be useful is to look at how OS X's Panther will handle Python 2.3 . It will have the standard library in the standard location of /usr/local/python2.4 . In that directory, the site-packages directory is a symlink to a more system-aware location. This is fine but what about a sysadmin who would rather avoid the possibility of breaking OS X by messing with the OS's Python installation? What about a user on that system who does not have root but wants to have their own place to put their packages? There is definitely room for adding more standard path locations for package installations. A PEP was being mentioned but appears to not have been written yet. Contributing threads: * `Multiple levels of site-packages `__ ------------------------- Be careful with __slots__ ------------------------- It seems people have been using __slots__ without fully understanding its purpose. It should be known that "__slots__ is a terrible hack with nasty, hard-to-fathom side effects that should only be used by programmers at grandmaster and wizard levels". So only use __slots__ if you can apply the labal of "programmer at grandmaster or wizard level" to yourself without your programming peers laughting their bums off; you have been warned. Contributing threads: * `Make it an error to use __slots__ with classic classes `__ -------------- Plugging leaks -------------- Michael Hudson thought he had discovered a leak somewhere and went off on a little hunt. It turned out to be a red herring more or less, but there was some discussion on the best way to find leaks through regression tests. The basic scheme that everyone seemed to use was to run the regression test once to get any caching out of the way and then run the test a set number of times and see if the reference counts seemed to grow. Michael Hudson posted a diff to add such a testing feature to test/regrtest.py at http://mail.python.org/pipermail/python-dev/2003-August/037617.html . Tim Peters also posted a link to Zope's test driver at http://cvs.zope.org/Zope3/test.py which includes a class named TrackRefs which can help keep track of the leaked references as well as leaked objects. Contributing threads: * `CALL_ATTR patch `__ * `refleak hunting fun! `__ ----------------------------------------- Making the interpreter its own executable ----------------------------------------- As it stands now the Python interpreter is distributed as a bunch of files mostly made up of the standard library. But wouldn't it be nice if you could make the interpreter just a single executable that was easy to distribute with your code? Well, that discussion cropped up on `comp.lang.python`_ at http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&safe=off&threadm=ptji8wgr.fsf%40python.net . The idea was to somehow introduce a hook into Py_Main() that could harness the new zipimport facility. The idea came up of appending the stdlib to the end of the Python interpreter and to have a flag set to signal that the appending had occurred. The problem is that this could cause unzipping problems. But setting the flag is not necessarily simple either. One suggestion was to literally patch the interpreter to set the flag. But there was some confusion over the use of the term "patch"; Thomas Heller thought more of "link with an object file defining this global variable". This thread was still going as of this writing and had not yet reached a clear solution. Contributing threads: * `hook for standalone executable `__ From nhodgson at bigpond.net.au Sun Aug 17 11:02:20 2003 From: nhodgson at bigpond.net.au (Neil Hodgson) Date: Sat Aug 16 20:02:21 2003 Subject: [Python-Dev] Searching SourceForge tracker Message-ID: <017901c36452$d411ad50$3da48490@neil> Is it possible to search the bug tracker at SourceForge? I want to see if the problem with os.access not allowing a Unicode file name (as reported on 12th August in c.l.py) has been reported or is being worked on before working on it myself. Neil From skip at pobox.com Sat Aug 16 20:45:16 2003 From: skip at pobox.com (Skip Montanaro) Date: Sat Aug 16 20:45:19 2003 Subject: [Python-Dev] python-dev Summary for 2003-08-01 through 2003-08-15 [draft] In-Reply-To: <3F3EB6E5.2090609@ocf.berkeley.edu> References: <3F3EB6E5.2090609@ocf.berkeley.edu> Message-ID: <16190.53276.61206.361694@montanaro.dyndns.org> Brett> After clearing up the confusing issue of the difference between a Brett> Python int and a Python Integer (the former is while Brett> the latter is the union of and ), .... Brett> As of right now Python claims that a void pointer can be held in Brett> a Python Integer, but this breaks that claim.... How so? If a Python Integer is the union of int and long, then when sizeof(void *) > sizeof(long), you just spill into Python longs. I didn't follow the discussion closely, but something about what you've said doesn't make a lot of sense to me. Maybe the wording just needs to be clarified. Skip From skip at pobox.com Sat Aug 16 20:48:55 2003 From: skip at pobox.com (Skip Montanaro) Date: Sat Aug 16 20:49:01 2003 Subject: [Python-Dev] Searching SourceForge tracker In-Reply-To: <017901c36452$d411ad50$3da48490@neil> References: <017901c36452$d411ad50$3da48490@neil> Message-ID: <16190.53495.155173.716929@montanaro.dyndns.org> Neil> Is it possible to search the bug tracker at SourceForge? I want to Neil> see if the problem with os.access not allowing a Unicode file name Neil> (as reported on 12th August in c.l.py) has been reported or is Neil> being worked on before working on it myself. Yup, though it's fairly crude. Go here: http://sourceforge.net/tracker/?group_id=5470&atid=105470 and use the search box in the left-hand margin. Skip From michal at sabren.com Sat Aug 16 22:15:56 2003 From: michal at sabren.com (Michal Wallace) Date: Sat Aug 16 21:16:00 2003 Subject: [Python-Dev] pirate 0.01 alpha! Message-ID: Annoucing pirate 0.01 alpha! http://pirate.tangentcode.com/ """ Right now, pirate supports a good deal of python. The two biggest things that are still missing are classes (because parrot classes aren't quite done yet) and exec/eval/import (because those things require a compiler -- and without classes, pirate can't compile itself). None of the builtin functions exist, either, and the types are borrowed from perl. :) BUT: there's still a lot you can do! For example, parrot is able to run this generator-based microtheads demo and correctly run over 50 small test cases covering much of the language. As a special bonus, it currently runs dog slow! The microthreads example runs about 8-9 times slower on pirate than on regular python. But that's not parrot's fault: the generated code is completely unoptimized. """ Comments, bug reports, patches, flames, etc welcome. :) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From mnot at mnot.net Sat Aug 16 22:41:34 2003 From: mnot at mnot.net (Mark Nottingham) Date: Sun Aug 17 00:41:38 2003 Subject: [Python-Dev] Re: A syntax for function attributes? Message-ID: <14B90FF1-D06D-11D7-B9A7-00039396E15A@mnot.net> I very much like this approach, and reiterate my willingness to help push it (or another solution) forward however I can (probably limited to writing a PEP, unfortunately). Cheers, Paul Moore wrote: > All of the notations involving "stuff" between the definition and the > colon look reasonable when the "stuff" is small, but start to look odd > when it gets bigger (because it usually means that the colon is not on > the same line as the "def"). > > How about leaving the [...] construct between the args and the colon, > on the assumption that "normal use" is for short stuff, like > [classmethod]. But for the dictionary option for function attributes, > locate it after the docstring. The we have > > def p_expr_term(self,args): > """docstring here""" > { 'rule' : > """expr ::=3D expr + term > term ::=3D term * factor""" > } > > pass > > It may get (slightly) hairy to parse, but should be OK, as docstrings > are a precedent for bare literals at the start of a function > definition having special meaning. > > Paul. From skip at mojam.com Sun Aug 17 08:00:27 2003 From: skip at mojam.com (Skip Montanaro) Date: Sun Aug 17 08:00:30 2003 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200308171200.h7HC0R532570@manatee.mojam.com> Bug/Patch Summary ----------------- 451 open / 4024 total bugs (+22) 173 open / 2319 total patches (+5) New Bugs -------- patch for build with read-only $srcdir (2003-08-11) http://python.org/sf/786737 Solaris 2.5.1 _SC_PAGESIZE vs. _SC_PAGE_SIZE (2003-08-11) http://python.org/sf/786743 IDLE starts with no menus (Cygwin build of python2.3) (2003-08-11) http://python.org/sf/786827 re doesn't like (^$)* (2003-08-11) http://python.org/sf/786970 copy_reg globals in cPickle (2003-08-11) http://python.org/sf/787077 zipimport on meta_path fails with mutual importers (2003-08-11) http://python.org/sf/787113 compiler.compileFile fails on csv.py (2003-08-13) http://python.org/sf/788011 missing universal newline support in os.popen & friends (2003-08-13) http://python.org/sf/788035 build fails on Tru64 Unix (osf1V5) (2003-08-13) http://python.org/sf/788183 Unsupported 'locale' setting causes Idle to crash on startup (2003-08-13) http://python.org/sf/788378 bsddb btree set_location() semantics changed (2003-08-13) http://python.org/sf/788421 Cannot decode/encode anything! (2003-08-13) http://python.org/sf/788457 Queue class has logic error when non-blocking (2003-08-13) http://python.org/sf/788520 Closing dbenv first bsddb doesn't release locks & segfaults (2003-08-13) http://python.org/sf/788526 resolving relative paths for external entities with xml.sax (2003-08-14) http://python.org/sf/788931 Minor floatobject.c bug (2003-08-15) http://python.org/sf/789159 __getattr__ and metaclasses (2003-08-15) http://python.org/sf/789262 Minor FP bug in object.c (2003-08-15) http://python.org/sf/789290 Solaris Forte 7 &8 bug in test_long (2003-08-15) http://python.org/sf/789294 Memory leak on open() only in 2.3? (2003-08-15) http://python.org/sf/789402 CSV reader does not parse Mac line endings (2003-08-15) http://python.org/sf/789519 zappyfiles.py absent from MacPython binary (2003-08-15) http://python.org/sf/789545 "SetFrontProcess failed,-606" error for Tkinter under OS X (2003-08-16) http://python.org/sf/789926 access fails on Windows with Unicode file name (2003-08-17) http://python.org/sf/789995 New Patches ----------- os.path.exists should use os.access when possible (2003-08-10) http://python.org/sf/786237 test_largefile cleanup patch (2003-08-11) http://python.org/sf/786591 termios module on IRIX (2003-08-11) http://python.org/sf/787189 unittest.py: Custom TestRunners and --verbose (2003-08-12) http://python.org/sf/787789 explicitly provide a buffer in PyFile_SetBufSize() (2003-08-13) http://python.org/sf/788249 ignore "b" and "t" mode modifiers in posix_popen (2003-08-13) http://python.org/sf/788404 Glossary (2003-08-13) http://python.org/sf/788509 Allow os.access to handle Unicode file name (2003-08-17) http://python.org/sf/790000 Closed Bugs ----------- IDLE fails to launch if 2.3 is installed to "Program Files" (2003-07-30) http://python.org/sf/780451 2.3 web page link to 'New style classes' changes to 2.2.2 (2003-07-31) http://python.org/sf/781285 test_strptime fails with duplicate DST timezone name (2003-08-05) http://python.org/sf/783952 IDLE does not start for 2.3 on windows. (2003-08-06) http://python.org/sf/784183 Closed Patches -------------- Add use_default_colors support to curses module. (2003-05-17) http://python.org/sf/739124 Cygwin test_netrc open mode patch (2003-07-22) http://python.org/sf/775777 fix obscure crash in descriptor handling (2003-08-07) http://python.org/sf/784825 From raymond.hettinger at verizon.net Sun Aug 17 15:11:06 2003 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Sun Aug 17 14:15:45 2003 Subject: [Python-Dev] Py2.3.1 Message-ID: <006001c364ea$ee0166e0$e841fea9@oemcomputer> Is it still the plan to have to rapid schedule for the first bugfix release (perhaps in early September)? I think it will increase the acceptance of 2.3 and help it to be viewed as stabilized . Raymond Hettinger From tim.one at comcast.net Sun Aug 17 15:58:16 2003 From: tim.one at comcast.net (Tim Peters) Date: Sun Aug 17 14:58:51 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <006001c364ea$ee0166e0$e841fea9@oemcomputer> Message-ID: [Raymond Hettinger] > Is it still the plan to have to rapid schedule for the > first bugfix release (perhaps in early September)? AFAIK there's nothing worthy of being called "a plan" for any future release(s). > I think it will increase the acceptance of 2.3 and help > it to be viewed as stabilized . Me too. I can't give non-trivial time to closing bugs or patches in the foreseeable future, but can volunteer a night to test and pump out a 3.1 Windows installer (unless Thomas Heller wants to do it). More important is that someone volunteer to be the 3.1 Release Czar (which, as former Release Czars can testify, requires real time and effort). From theller at python.net Sun Aug 17 22:06:03 2003 From: theller at python.net (Thomas Heller) Date: Sun Aug 17 15:06:15 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: (Tim Peters's message of "Sun, 17 Aug 2003 14:58:16 -0400") References: Message-ID: "Tim Peters" writes: > [Raymond Hettinger] >> Is it still the plan to have to rapid schedule for the >> first bugfix release (perhaps in early September)? > > AFAIK there's nothing worthy of being called "a plan" for any future > release(s). > >> I think it will increase the acceptance of 2.3 and help >> it to be viewed as stabilized . > > Me too. I can't give non-trivial time to closing bugs or patches in the > foreseeable future, but can volunteer a night to test and pump out a 3.1 > Windows installer (unless Thomas Heller wants to do it). I would do it, but I'm on vacation the first two weeks of September. Thomas From bac at OCF.Berkeley.EDU Sun Aug 17 13:25:17 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Sun Aug 17 15:25:28 2003 Subject: [Python-Dev] python-dev Summary for 2003-08-01 through 2003-08-15 [draft] In-Reply-To: <16190.53276.61206.361694@montanaro.dyndns.org> References: <3F3EB6E5.2090609@ocf.berkeley.edu> <16190.53276.61206.361694@montanaro.dyndns.org> Message-ID: <3F3FD69D.1080908@ocf.berkeley.edu> Skip Montanaro wrote: > Brett> After clearing up the confusing issue of the difference between a > Brett> Python int and a Python Integer (the former is while > Brett> the latter is the union of and ), .... > > Brett> As of right now Python claims that a void pointer can be held in > Brett> a Python Integer, but this breaks that claim.... > > How so? If a Python Integer is the union of int and long, then when > sizeof(void *) > sizeof(long), you just spill into Python longs. I didn't > follow the discussion closely, but something about what you've said doesn't > make a lot of sense to me. Maybe the wording just needs to be clarified. > That is a typo. I meant to write Python int. Guess I had the same problems as the thread did in terms of terms used. =) -Brett From aahz at pythoncraft.com Sun Aug 17 17:00:34 2003 From: aahz at pythoncraft.com (Aahz) Date: Sun Aug 17 16:00:37 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <006001c364ea$ee0166e0$e841fea9@oemcomputer> References: <006001c364ea$ee0166e0$e841fea9@oemcomputer> Message-ID: <20030817200034.GA2496@panix.com> On Sun, Aug 17, 2003, Raymond Hettinger wrote: > > Is it still the plan to have to rapid schedule for the > first bugfix release (perhaps in early September)? > > I think it will increase the acceptance of 2.3 and help > it to be viewed as stabilized . While I think this is a Good Idea, to my knowledge there isn't any particular driver for a speedy release. I also haven't heard that many of the bugs considered important/critical for 2.3.1 have been fixed; generally speaking, a patch release tends to be a collection of available fixes rather than driving the bugfix process. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From python at rcn.com Sun Aug 17 17:29:14 2003 From: python at rcn.com (Raymond Hettinger) Date: Sun Aug 17 16:33:23 2003 Subject: [Python-Dev] Py2.3.1 References: Message-ID: <007601c364fe$3a49ed20$e841fea9@oemcomputer> > > [Raymond Hettinger] > >> Is it still the plan to have to rapid schedule for the > >> first bugfix release (perhaps in early September)? [Tim] > > AFAIK there's nothing worthy of being called "a plan" for any future > > release(s). Let's make a plan so that it doesn't languish. [Raymond] > >> I think it will increase the acceptance of 2.3 and help > >> it to be viewed as stabilized . [Tim] > > Me too. I can't give non-trivial time to closing bugs or patches in the > > foreseeable future, but can volunteer a night to test and pump out a 3.1 > > Windows installer (unless Thomas Heller wants to do it). [Thomas Heller] > I would do it, but I'm on vacation the first two weeks of September. The third week in September would be a good target date if you could do it then. That will leave time for more fixes to get in and be exercised. Kurt still has a handful of open idle issues, Michael has a few leaks left to hunt down, and MvL has been on vacation and not had a chance to review the open bug reports. [Aahz] > While I think this is a Good Idea, to my knowledge there isn't any > particular driver for a speedy release. I also haven't heard that many > of the bugs considered important/critical for 2.3.1 have been fixed; A handful of small disasters were fixed: * Windows failures when installed on path names with spaces * Runaway leaks in the socket and array modules * strptime() running thousands of times slower without caching All of these are a barrier to 2.3.1 being accepted as stable. Raymond Hettinger ################################################################# ################################################################# ################################################################# ##### ##### ##### ################################################################# ################################################################# ################################################################# From aahz at pythoncraft.com Sun Aug 17 17:43:33 2003 From: aahz at pythoncraft.com (Aahz) Date: Sun Aug 17 16:44:20 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <007601c364fe$3a49ed20$e841fea9@oemcomputer> References: <007601c364fe$3a49ed20$e841fea9@oemcomputer> Message-ID: <20030817204333.GA9081@panix.com> On Sun, Aug 17, 2003, Raymond Hettinger wrote: > > A handful of small disasters were fixed: > * Windows failures when installed on path names with spaces Has this actually been fixed, or is this referring to something other than IDLE? (I haven't seen anything, so I'm just double-checking.) > All of these are a barrier to 2.3.1 being accepted as stable. Hmmm? That doesn't make sense. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From python at rcn.com Sun Aug 17 17:45:42 2003 From: python at rcn.com (Raymond Hettinger) Date: Sun Aug 17 16:49:52 2003 Subject: [Python-Dev] Py2.3.1 References: <007601c364fe$3a49ed20$e841fea9@oemcomputer> <20030817204333.GA9081@panix.com> Message-ID: <009901c36500$868e8b80$e841fea9@oemcomputer> > > All of these are a barrier to 2.3.1 being accepted as stable. > > Hmmm? That doesn't make sense. I think you know that I meant 2.3 Raymond From anthony at interlink.com.au Mon Aug 18 08:00:53 2003 From: anthony at interlink.com.au (Anthony Baxter) Date: Sun Aug 17 17:00:55 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: Message-ID: <200308172100.h7HL0rJP020838@localhost.localdomain> >>> "Tim Peters" wrote > More important is that someone volunteer to be the 3.1 Release Czar (which, > as former Release Czars can testify, requires real time and effort). I'm happy to take this on. Anthony -- Anthony Baxter It's never too late to have a happy childhood. From tim.one at comcast.net Sun Aug 17 18:00:41 2003 From: tim.one at comcast.net (Tim Peters) Date: Sun Aug 17 17:01:15 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <20030817204333.GA9081@panix.com> Message-ID: [Raymond] >> A handful of small disasters were fixed: >> * Windows failures when installed on path names with spaces [Aahz] > Has this actually been fixed, Yes, it's fine in current CVS. OTOH, I'd characterize it more minor nuisance than small disaster. > or is this referring to something other than IDLE? No. From python at rcn.com Sun Aug 17 18:14:11 2003 From: python at rcn.com (Raymond Hettinger) Date: Sun Aug 17 17:18:20 2003 Subject: [Python-Dev] Py2.3.1 References: <200308172100.h7HL0rJP020838@localhost.localdomain> Message-ID: <002c01c36504$815f8020$e841fea9@oemcomputer> > >>> "Tim Peters" wrote > > More important is that someone volunteer to be the 3.1 Release Czar (which, > > as former Release Czars can testify, requires real time and effort). [Anthony Baxter] > I'm happy to take this on. Since I need to learn to do this, I'm happy to assist as much as possible. Raymond Hettinger From guido at python.org Sun Aug 17 09:20:34 2003 From: guido at python.org (Guido van Rossum) Date: Sun Aug 17 18:21:31 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: Your message of "Sun, 17 Aug 2003 16:29:14 EDT." <007601c364fe$3a49ed20$e841fea9@oemcomputer> References: <007601c364fe$3a49ed20$e841fea9@oemcomputer> Message-ID: <200308171520.h7HFKYs10739@12-236-84-31.client.attbi.com> > The third week in September would be a good target date if you > could do it then. Excellent, assuming 2.3.1 will need me as much as 2.3 did -- I'll be on vacation that week. --Guido van Rossum (home page: http://www.python.org/~guido/) From aahz at pythoncraft.com Sun Aug 17 20:25:33 2003 From: aahz at pythoncraft.com (Aahz) Date: Sun Aug 17 19:25:36 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <009901c36500$868e8b80$e841fea9@oemcomputer> References: <20030817204333.GA9081@panix.com> <009901c36500$868e8b80$e841fea9@oemcomputer> Message-ID: <20030817232533.GA13528@panix.com> >>> All of these are a barrier to 2.3.1 being accepted as stable. >> >> Hmmm? That doesn't make sense. > > I think you know that I meant 2.3 That's what I *thought* you meant, but I've seen enough typos that I'm rarely certain anymore what was meant. "Explicit is better than implicit" -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From walter at livinglogic.de Mon Aug 18 03:12:18 2003 From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=) Date: Sun Aug 17 20:12:56 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mk79fnjw6.fsf@starship.python.net> References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> <2m65kzp56d.fsf@starship.python.net> <3F3CD43C.3090206@livinglogic.de> <2mk79fnjw6.fsf@starship.python.net> Message-ID: <3F4019E2.1080500@livinglogic.de> Michael Hudson wrote: > Walter D?rwald writes: > >>Michael Hudson wrote: >> >>>[...] >>> >>>>>test_codeccallbacks leaked 1107 references >>> >>>[...] >>> >>>>but there seem to be real leaks here. >>> >>>In a perverse kind of way, phew :-) Wouldn't want to have gone to all >>>this effort to uncover *nothing* but a bunch of false alarms... >> >>I've fixed two of the leaks. > > Cool. Do you think that's it for real leaks in test_codeccallbacks? All leaks in test_codeccallbacks are fixed now. Take a look at http://styx.livinglogic.de/~walter/refleakhunt/reflog3.txt which is the result of running the test suite with the patch at http://styx.livinglogic.de/~walter/refleakhunt/unittest.diff The only test_codeccallbacks test that seems to leak references is test_callbacks() and this is the result of calling codecs.register_error(), i.e. the refcount leak disappears when the call to register_error() is moved out of the test method. The other number that is worrying me is test_builtin.BuiltinTest.test_filter_subclasses() which is probably the result of the recent changes for tuple, str und unicode subclasses. I'm going to look into this tomorrow. > [...] >> >>http://styx.livinglogic.de/~walter/reflog3.txt >> >>(This includes only unittest based tests) > > Cool. Is this from CVS head? I thought a bunch of leaks in arrays > had already been fixed. Seems they have. The bad numbers from test_array are gone, so it *was* an older checkout. > [...] Bye, Walter D?rwald From T.A.Meyer at massey.ac.nz Mon Aug 18 13:35:27 2003 From: T.A.Meyer at massey.ac.nz (Meyer, Tony) Date: Sun Aug 17 20:41:02 2003 Subject: [Python-Dev] RE: [spambayes-dev] World domination Message-ID: <1ED4ECF91CDED24C8D012BCF2B034F1302D92599@its-xchg4.massey.ac.nz> > front page as the 7th "most active" project (of 67,000) at SF > last week. There have been at least 4,300 downloads of the > OL addin from SF. #6 now...maybe one fewer beating today? ;) Interestingly, the number of downloads fell quite a lot over the last couple of days, but the ranking went up. Maybe it was a slow day for everyone... > Congratulations! I wish I could claim more credit for > myself, but I've had little to do with it since last year. Well, to be fair, as nice as the Outlook plug-in is, I doubt many people would be recommending it if it didn't actually do the business of correctly classifying mail. You can have as much of that credit as you can get before Gary et. al. realise that there is credit to be taken and come for their share... :) > Nevertheless, beatings will continue until it's #1 . I don't know if I've ever seen the sf page without "Compiere ERP + CRM Business Solution", "Gaim", and "phpMyAdmin" in the top five. Pushing them out will be quite a feat, but then there's bound to be a surge with the next release of the plug-in, so maybe that would do it. =Tony Meyer From barry at python.org Mon Aug 18 03:35:54 2003 From: barry at python.org (Barry Warsaw) Date: Sun Aug 17 22:35:55 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <002c01c36504$815f8020$e841fea9@oemcomputer> References: <200308172100.h7HL0rJP020838@localhost.localdomain> <002c01c36504$815f8020$e841fea9@oemcomputer> Message-ID: <1061174117.29611.0.camel@anthem> On Sun, 2003-08-17 at 17:14, Raymond Hettinger wrote: > > >>> "Tim Peters" wrote > > > More important is that someone volunteer to be the 3.1 Release Czar (which, > > > as former Release Czars can testify, requires real time and effort). > > [Anthony Baxter] > > I'm happy to take this on. > > Since I need to learn to do this, I'm happy > to assist as much as possible. Since I've done it several times before, I'm available to help out if and where necessary. -Barry From skip at pobox.com Sun Aug 17 23:22:43 2003 From: skip at pobox.com (Skip Montanaro) Date: Sun Aug 17 23:22:46 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line Message-ID: <16192.18051.288852.969201@montanaro.dyndns.org> A bug was reported against the csv module, claiming (rightly so) that the csv module was not properly parsing files which use Mac line endings. I tracked the problem down to an apparent defiency in readahead_get_line_skip() in fileobject.c. It believes that only \n can terminate a line. The patch below fixes my csv module problem, but I wonder if it's the correct fix. Suppose you're using Mac line endings and encounter a \n before a \r? This function will return a too-short line. (Of course, it would without the patch as well.) I don't know how (or if) this should work with universal newline support. We expect files to be opened in binary mode, so I don't know if universal newline support applies. In short, does this look like the correct patch, closer to the correct behavior than the current setup, or no improvement at all? Thx, Skip cvs diff fileobject.c Index: fileobject.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Objects/fileobject.c,v retrieving revision 2.179 diff -c -r2.179 fileobject.c *** fileobject.c 18 May 2003 12:56:25 -0000 2.179 --- fileobject.c 18 Aug 2003 03:13:38 -0000 *************** *** 1803,1810 **** return (PyStringObject *) PyString_FromStringAndSize(NULL, skip); bufptr = memchr(f->f_bufptr, '\n', len); if (bufptr != NULL) { ! bufptr++; /* Count the '\n' */ len = bufptr - f->f_bufptr; s = (PyStringObject *) PyString_FromStringAndSize(NULL, skip+len); --- 1803,1812 ---- return (PyStringObject *) PyString_FromStringAndSize(NULL, skip); bufptr = memchr(f->f_bufptr, '\n', len); + if (bufptr == NULL) + bufptr = memchr(f->f_bufptr, '\r', len); if (bufptr != NULL) { ! bufptr++; /* Count the '\n' or '\r' */ len = bufptr - f->f_bufptr; s = (PyStringObject *) PyString_FromStringAndSize(NULL, skip+len); From aahz at pythoncraft.com Mon Aug 18 00:38:14 2003 From: aahz at pythoncraft.com (Aahz) Date: Sun Aug 17 23:38:17 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: <16192.18051.288852.969201@montanaro.dyndns.org> References: <16192.18051.288852.969201@montanaro.dyndns.org> Message-ID: <20030818033814.GA24174@panix.com> On Sun, Aug 17, 2003, Skip Montanaro wrote: > > I don't know how (or if) this should work with universal newline support. > We expect files to be opened in binary mode, so I don't know if universal > newline support applies. Why do you expect binary rather than text mode? I've always thought of CSV files as text and would expect the usual line ending issues that are supposed to be solved with universal newlines. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From guido at python.org Sun Aug 17 15:05:30 2003 From: guido at python.org (Guido van Rossum) Date: Mon Aug 18 00:06:23 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: Your message of "Sun, 17 Aug 2003 22:22:43 CDT." <16192.18051.288852.969201@montanaro.dyndns.org> References: <16192.18051.288852.969201@montanaro.dyndns.org> Message-ID: <200308172105.h7HL5UD10914@12-236-84-31.client.attbi.com> > The patch below fixes my csv module problem, but I wonder > if it's the correct fix. Suppose you're using Mac line endings and > encounter a \n before a \r? This function will return a too-short line. > (Of course, it would without the patch as well.) > > I don't know how (or if) this should work with universal newline support. > We expect files to be opened in binary mode, so I don't know if universal > newline support applies. > > In short, does this look like the correct patch, closer to the correct > behavior than the current setup, or no improvement at all? I'm not familiar with the csv module, but if it opens the file in binary mode, readline() and next() are *required* to only honor \n as the line end character. Maybe the csv module ought to open the file in universal newline mode (f = open(filename, "U")? The patch you give would make next() behave like universal newline mode in all modes, which is definitely wrong. --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one at comcast.net Mon Aug 18 01:11:11 2003 From: tim.one at comcast.net (Tim Peters) Date: Mon Aug 18 00:11:46 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <200308172100.h7HL0rJP020838@localhost.localdomain> Message-ID: [Anthony Baxter, re the 3.1 Release Czar position] > I'm happy to take this on. So where's the release 3.1 PEP already ? gratefully y'rs - tim From anthony at interlink.com.au Mon Aug 18 15:17:21 2003 From: anthony at interlink.com.au (Anthony Baxter) Date: Mon Aug 18 00:17:46 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: Message-ID: <200308180417.h7I4HLg6027413@localhost.localdomain> >>> "Tim Peters" wrote > [Anthony Baxter, re the 3.1 Release Czar position] > > I'm happy to take this on. > > So where's the release 3.1 PEP already ? > > gratefully y'rs - tim Working on a 2.3.1 release PEP at the moment. the-3.1-release-PEP-can-wait-you-should-see-the-deprecations-list y'rs, Anthony From cce at clarkevans.com Mon Aug 18 06:28:22 2003 From: cce at clarkevans.com (Clark C. Evans) Date: Mon Aug 18 01:25:43 2003 Subject: [Python-Dev] cooperative generators Message-ID: <20030818052822.GA89739@polya.axista.com> I would like to get some feedback on some thoughts that I've been having about cooperative multitasking using generators. After some attempts to change that 'flow' module [1] into a C module, I got thinking about how to better support my requirements with a relatively small change to the Python interpreter. Anyway, if you would humor me a bit, perhaps you can see what I'm getting at; perhaps the change to Python could be relatively localized and simple. And Python would be quite more useful in cooperative multitasking contexts. First, let me provide an application context. Suppose you are building a webpage in response to an HTTP request. Further suppose that the information for this web page comes from two sources, a PostgreSQL database and an OpenLDAP directory. Now assume that you want to make your page building modular, that is, broken into several operations with sub operations. Here is, perhaps a call graph of how the page would be done: buildPage | buildHeader | | writeTopOfHeader | | writeMetaKeywords | | queryLDAP* (returns iteratable keyword sequence) | | writeRestOfHeader | buildNavigator | | writeTopOfNavigator | | writeNavigatorRows | | queryLDAP* (to get items for the Navigator) | | writeRestOfNavigator | buildGrid | | writeTopOfGrid | | writeRows | | queryDatabase* (returns sequence of rows ) | writeFooter Assume that each of these items is a generator, and that queryLDAP, and queryDatabase generators either yield a value (in the sequence that they return) or return a special sentinel 'Cooperate'. Then, assume that you have a reactor which is building N pages at a time. To make this magic work, each one of the intermediate generators (buildPage, buildGrid, writeRows) on the way to a leaf generator (queryDatabase) must explicitly check for this 'Cooperate' special value, and then yield themselves with this same value. In this way, you can 'pause' the whole chain of generators. The flow module [1] tries to make this easy, and makes the logic for 'Cooperate' to work by scheduling each page to be build. Second, let me describe the problem (or irritant) Well, what happens in the flow module, is that each one of the intermediate generators needs to be 'wrapped' to handle exceptions and other concerns. This wrapper slows things down substantially, especially in tight loops. Furthermore, each intermediate generator must be made 'aware' that it may be paused (an unnecessary ickyness) for example, the following code: for x in parentGenerator: # do something with X has to be rewritten as: parentGenerator = flow.wrap(parentGenerator) yield parentGenerator for x in parentGenerator: # do something with X yeild parentGenerator It works, but it is just stuff that I think could be done much better in the guts of python itself. And without all of the tricks needed to coax exceptions into working as you'd expect. Third, let me describe what I 'think' would be a nice solution I'd like a special value, lets call it Pause, which can be given to a yield statement. When this is encountered by the Python interpreter, it bypasses all of the intermediate generators and goes to the most recent non-generator. For example, in the a queryDatabase yield of Pause would bypass the chain (buildPage, buildGrid, writeRows) and the caller of buildPage's next() method would receive the Pause object. Then, when the next call to buildPage's next() method is invoked, it would resume the top level generator (queryDatabase) and if that generator yields a non-Pause value, the stack would unwind as normal. So, in an attempt to be more explicit: - Let a generator chain C be composed of three generators, g, g', g'' which are calling each other. Let f be a function which is iterating over the generator g. - Let the generator g'' yield a value p, which is a subclass of a special 'Pause' built-in class. - At this point, the Python evaluator goes down the stack frame to find the first non-generator in the stack, in this case, f. - And then the evaluator creates a new instance zzz of a PauseIter object. Then, every instance of g in f is replaced with zzz. At this point, zzz is initialized with g and g'', as the head and tail of the paused generator. - The function f is given the value p as its result of g.next() - When the function f calls g.next() again (which actually calls zzz.next() instead), the generator g'' is resumed. - If g'' yield another Pause object, then this object is passed back to the function f, and things continue - If g'' yield a non-Pause object, then f() is rewritten to link to g again, and the non-Pause object is handed to g' so that normal processing proceeds - If g'' creates an exception, then f() is rewritten to link to g again, and the exception object is percolated down to g' for normal processing. Fourth, in conclusion I think with a chance similar to above, Cooperative Multitasking in Python with the ability to break-down generators into sub-generators becomes easy to manage (a function f can simply keep each micro-thread in a queue, etc.). I'm not sure if I understand the implications of all of this, and in particular, how hard a ceval.c would be, but I'd be very interested to hear your opinion. Kind Regards, Clark From bac at OCF.Berkeley.EDU Sun Aug 17 23:32:12 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Mon Aug 18 01:32:22 2003 Subject: [Python-Dev] Moving Tuesday; offline for 2 weeks Message-ID: <3F4064DC.9040504@ocf.berkeley.edu> I am moving Tuesday morning down to San Luis Obispo. The problem with this is that I am going to lose Net for two weeks thanks to SBC saying it will take 10 to 12 working days for me to get my DSL turned on; I ordered it this past Friday. So I have disabled delivery on all lists (pydotorg-related lists, PyCon-related lists) but python-dev, but that is just so that I have a copy for when I do the next Summary. There is a chance I will be able to check between now and and when I get DSL, but if I do it will only be python-dev and I have no clue if that will even happen. Hopefully I will be productive during my time "off". withdrawls-here-I-come-ly y'rs, Brett From bh at intevation.de Mon Aug 18 13:28:21 2003 From: bh at intevation.de (Bernhard Herzog) Date: Mon Aug 18 06:28:25 2003 Subject: [Python-Dev] PyMapping_Check Bug? In-Reply-To: <200308091752.h79HqL313056@12-236-84-31.client.attbi.com> (Guido van Rossum's message of "Sat, 09 Aug 2003 10:52:21 -0700") References: <16181.13094.559794.648236@montanaro.dyndns.org> <200308091752.h79HqL313056@12-236-84-31.client.attbi.com> Message-ID: <6qlltr2ray.fsf@salmakis.intevation.de> Guido van Rossum writes: > I don't see this. operator.isMappingType() maps directly to > PyMapping_Check(); and I tried this: > > >>> import operator > >>> operator.isMappingType(str) > False > >>> class C(object): pass > ... > >>> operator.isMappingType(C) > False > >>> operator.isMappingType(C()) > False > >>> operator.isMappingType({}) > True > >>> > > Looks okay to me. :-) Are these OK, though: >>> operator.isMappingType(()) True >>> operator.isMappingType([]) True >>> operator.isMappingType("") True In Python 2.2: >>> operator.isMappingType(()) 0 >>> operator.isMappingType([]) 0 >>> operator.isMappingType("") 0 I just found out that this change is the cause for a Bug in my shapelib bindings which at one point try to determine whether a method argument is a dictionary like object by testing whether PyMapping_Check returns true and that assume it's a sequence otherwise. As a work-around it seems I can use isSequenceType/PySequence_Check to effectively turn the test around and determine whether the object is a sequence and assume it's a mapping otherwise. Bernhard -- Intevation GmbH http://intevation.de/ Sketch http://sketch.sourceforge.net/ Thuban http://thuban.intevation.org/ From skip at pobox.com Mon Aug 18 09:16:21 2003 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 18 09:16:26 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: <200308172105.h7HL5UD10914@12-236-84-31.client.attbi.com> References: <16192.18051.288852.969201@montanaro.dyndns.org> <200308172105.h7HL5UD10914@12-236-84-31.client.attbi.com> Message-ID: <16192.53669.274955.99820@montanaro.dyndns.org> Guido> I'm not familiar with the csv module, but if it opens the file in Guido> binary mode, readline() and next() are *required* to only honor Guido> \n as the line end character. Maybe the csv module ought to open Guido> the file in universal newline mode (f = open(filename, "U")? That makes sense. I think the csv module will have to do its own end-of-line detection, since the end-of-line terminator is explicit. Guido> The patch you give would make next() behave like universal Guido> newline mode in all modes, which is definitely wrong. Good thing I asked. ;-) Skip From skip at pobox.com Mon Aug 18 09:18:59 2003 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 18 09:19:05 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: <20030818033814.GA24174@panix.com> References: <16192.18051.288852.969201@montanaro.dyndns.org> <20030818033814.GA24174@panix.com> Message-ID: <16192.53827.329638.239507@montanaro.dyndns.org> >> I don't know how (or if) this should work with universal newline >> support. We expect files to be opened in binary mode, so I don't >> know if universal newline support applies. aahz> Why do you expect binary rather than text mode? I've always aahz> thought of CSV files as text and would expect the usual line aahz> ending issues that are supposed to be solved with universal aahz> newlines. I think the csv module was started before universal newline mode was available. We allow the user to specify the line terminator, and I suppose it could be anything, hence binary mode. We'll have to consider whether universal newline mode - along with an implicit restriction in possible EOL sequences - is the way to go. Thanks for the feedback, Skip From mwh at python.net Mon Aug 18 15:25:04 2003 From: mwh at python.net (Michael Hudson) Date: Mon Aug 18 09:25:07 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <3F4019E2.1080500@livinglogic.de> (Walter =?iso-8859-1?q?D=F6rwald's?= message of "Mon, 18 Aug 2003 02:12:18 +0200") References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> <2m65kzp56d.fsf@starship.python.net> <3F3CD43C.3090206@livinglogic.de> <2mk79fnjw6.fsf@starship.python.net> <3F4019E2.1080500@livinglogic.de> Message-ID: <2mhe4fm72n.fsf@starship.python.net> Walter D?rwald writes: > Michael Hudson wrote: > >> Walter D?rwald writes: >> >>>Michael Hudson wrote: >>> >>>>[...] >>>> >>>>>>test_codeccallbacks leaked 1107 references >>>> >>>>[...] >>>> >>>>>but there seem to be real leaks here. >>>> >>>>In a perverse kind of way, phew :-) Wouldn't want to have gone to all >>>>this effort to uncover *nothing* but a bunch of false alarms... >>> >>>I've fixed two of the leaks. >> Cool. Do you think that's it for real leaks in test_codeccallbacks? > > All leaks in test_codeccallbacks are fixed now. Cool! > Take a look at > http://styx.livinglogic.de/~walter/refleakhunt/reflog3.txt which is > the result of running the test suite with the patch at > http://styx.livinglogic.de/~walter/refleakhunt/unittest.diff The > only test_codeccallbacks test that seems to leak references is > test_callbacks() and this is the result of calling > codecs.register_error(), i.e. the refcount leak disappears when the > call to register_error() is moved out of the test method. That makes sense. > The other number that is worrying me is > test_builtin.BuiltinTest.test_filter_subclasses() which is probably > the result of the recent changes for tuple, str und unicode > subclasses. I'm going to look into this tomorrow. Goody. I found that that test was a problem this morning but got confused looking at the function :-) I did notice that there are more than just leaks here, I'm afraid: >>> class BadSeq(tuple): ... def __getitem__(self, i): ... raise IndexError ... [25508 refs] >>> filter(None, BadSeq((1,))) Segmentation fault >>>http://styx.livinglogic.de/~walter/reflog3.txt >>> >>>(This includes only unittest based tests) >> Cool. Is this from CVS head? I thought a bunch of leaks in arrays >> had already been fixed. > > Seems they have. The bad numbers from test_array are gone, so > it *was* an older checkout. Excellent. Cheers, mwh -- I never realized it before, but having looked that over I'm certain I'd rather have my eyes burned out by zombies with flaming dung sticks than work on a conscientious Unicode regex engine. -- Tim Peters, 3 Dec 1998 From mwh at python.net Mon Aug 18 15:30:14 2003 From: mwh at python.net (Michael Hudson) Date: Mon Aug 18 09:30:27 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <007601c364fe$3a49ed20$e841fea9@oemcomputer> ("Raymond Hettinger"'s message of "Sun, 17 Aug 2003 16:29:14 -0400") References: <007601c364fe$3a49ed20$e841fea9@oemcomputer> Message-ID: <2mekzjm6u1.fsf@starship.python.net> "Raymond Hettinger" writes: > The third week in September would be a good target date if you > could do it then. So long as you mean "the end of the third week" (I'm away 6-14 Sept.), sounds good. Did I hear you volunteering to be RM ? Cheers, mwh -- The "of course, while I have no problem with this at all, it's surely too much for a lesser being" flavor of argument always rings hollow to me. -- Tim Peters, 29 Apr 1998 From neal at metaslash.com Mon Aug 18 10:31:59 2003 From: neal at metaslash.com (Neal Norwitz) Date: Mon Aug 18 09:32:36 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <2mhe4fm72n.fsf@starship.python.net> References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> <2m65kzp56d.fsf@starship.python.net> <3F3CD43C.3090206@livinglogic.de> <2mk79fnjw6.fsf@starship.python.net> <3F4019E2.1080500@livinglogic.de> <2mhe4fm72n.fsf@starship.python.net> Message-ID: <20030818133158.GQ8069@epoch.metaslash.com> On Mon, Aug 18, 2003 at 02:25:04PM +0100, Michael Hudson wrote: > > I did notice that there are more than just leaks here, I'm afraid: > > >>> class BadSeq(tuple): > ... def __getitem__(self, i): > ... raise IndexError > ... > [25508 refs] > >>> filter(None, BadSeq((1,))) > Segmentation fault I found this seg fault too. The attached patch fixes the problem. I think it's correct. Neal -- Index: Python/bltinmodule.c =================================================================== RCS file: /cvsroot/python/python/dist/src/Python/bltinmodule.c,v retrieving revision 2.294 diff -w -u -r2.294 bltinmodule.c --- Python/bltinmodule.c 14 Aug 2003 20:37:34 -0000 2.294 +++ Python/bltinmodule.c 18 Aug 2003 13:31:23 -0000 @@ -2174,6 +2174,8 @@ if (tuple->ob_type->tp_as_sequence && tuple->ob_type->tp_as_sequence->sq_item) { item = tuple->ob_type->tp_as_sequence->sq_item(tuple, i); + if (item == NULL) + goto Fail_1; } else { PyErr_SetString(PyExc_TypeError, "filter(): unsubscriptable tuple"); goto Fail_1; From theller at python.net Mon Aug 18 16:40:06 2003 From: theller at python.net (Thomas Heller) Date: Mon Aug 18 09:40:41 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <2mekzjm6u1.fsf@starship.python.net> (Michael Hudson's message of "Mon, 18 Aug 2003 14:30:14 +0100") References: <007601c364fe$3a49ed20$e841fea9@oemcomputer> <2mekzjm6u1.fsf@starship.python.net> Message-ID: <1xvjrsnd.fsf@python.net> Michael Hudson writes: > "Raymond Hettinger" writes: > >> The third week in September would be a good target date if you >> could do it then. > > So long as you mean "the end of the third week" (I'm away 6-14 Sept.), > sounds good. Same for me. Thomas From aahz at pythoncraft.com Mon Aug 18 11:02:41 2003 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 18 10:02:45 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: <16192.53827.329638.239507@montanaro.dyndns.org> References: <16192.18051.288852.969201@montanaro.dyndns.org> <20030818033814.GA24174@panix.com> <16192.53827.329638.239507@montanaro.dyndns.org> Message-ID: <20030818140241.GA1282@panix.com> On Mon, Aug 18, 2003, Skip Montanaro wrote: > > >> I don't know how (or if) this should work with universal newline > >> support. We expect files to be opened in binary mode, so I don't > >> know if universal newline support applies. > > aahz> Why do you expect binary rather than text mode? I've always > aahz> thought of CSV files as text and would expect the usual line > aahz> ending issues that are supposed to be solved with universal > aahz> newlines. > > I think the csv module was started before universal newline mode > was available. We allow the user to specify the line terminator, > and I suppose it could be anything, hence binary mode. We'll have > to consider whether universal newline mode - along with an implicit > restriction in possible EOL sequences - is the way to go. How about EOL is None means universal newlines and is the default? -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From guido at python.org Mon Aug 18 08:03:43 2003 From: guido at python.org (Guido van Rossum) Date: Mon Aug 18 10:03:44 2003 Subject: [Python-Dev] PyMapping_Check Bug? In-Reply-To: Your message of "Mon, 18 Aug 2003 12:28:21 +0200." <6qlltr2ray.fsf@salmakis.intevation.de> References: <16181.13094.559794.648236@montanaro.dyndns.org> <200308091752.h79HqL313056@12-236-84-31.client.attbi.com> <6qlltr2ray.fsf@salmakis.intevation.de> Message-ID: <200308181403.h7IE3hd11813@12-236-84-31.client.attbi.com> > Are these OK, though: > > >>> operator.isMappingType(()) > True > >>> operator.isMappingType([]) > True > >>> operator.isMappingType("") > True These are because those types have all grown a mapping-style getitem implementation in order to support extended slicing. > I just found out that this change is the cause for a Bug in my shapelib > bindings which at one point try to determine whether a method argument > is a dictionary like object by testing whether PyMapping_Check returns > true and that assume it's a sequence otherwise. > As a work-around it seems I can use isSequenceType/PySequence_Check to > effectively turn the test around and determine whether the object is a > sequence and assume it's a mapping otherwise. Still, my suggestion is not to try to guess whether an object is a sequence or a mapping. The two APIs are too similar, and becoming more so in each release. Instead, you should have an explicit indicator passed by the caller (maybe a keyword argument). --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at pobox.com Mon Aug 18 10:26:10 2003 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 18 10:26:24 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: <20030818140241.GA1282@panix.com> References: <16192.18051.288852.969201@montanaro.dyndns.org> <20030818033814.GA24174@panix.com> <16192.53827.329638.239507@montanaro.dyndns.org> <20030818140241.GA1282@panix.com> Message-ID: <16192.57858.301752.30684@montanaro.dyndns.org> >> We'll have to consider whether universal newline mode - along with an >> implicit restriction in possible EOL sequences - is the way to go. aahz> How about EOL is None means universal newlines and is the default? My worry is that if you have (for example) a \n embedded in a field when the EOL character is a \r that the parsing would get screwed up. That \n is a character in a field, not the end of a record. (Such fields may always have to be quoted, but I'm not sure.) At this level it's really an issue for Andrew MacNamara or Dave Cole. I'm not too familiar with the underlying C code. I suspect in the end that the csv module may have to do all it's own I/O. That would be a PITA, but might be required. Skip From aahz at pythoncraft.com Mon Aug 18 11:34:55 2003 From: aahz at pythoncraft.com (Aahz) Date: Mon Aug 18 10:34:59 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: <16192.57858.301752.30684@montanaro.dyndns.org> References: <16192.18051.288852.969201@montanaro.dyndns.org> <20030818033814.GA24174@panix.com> <16192.53827.329638.239507@montanaro.dyndns.org> <20030818140241.GA1282@panix.com> <16192.57858.301752.30684@montanaro.dyndns.org> Message-ID: <20030818143455.GA28440@panix.com> On Mon, Aug 18, 2003, Skip Montanaro wrote: > > >> We'll have to consider whether universal newline mode - along with an > >> implicit restriction in possible EOL sequences - is the way to go. > > aahz> How about EOL is None means universal newlines and is the default? > > My worry is that if you have (for example) a \n embedded in a field when the > EOL character is a \r that the parsing would get screwed up. That \n is a > character in a field, not the end of a record. (Such fields may always have > to be quoted, but I'm not sure.) At this level it's really an issue for > Andrew MacNamara or Dave Cole. I'm not too familiar with the underlying C > code. Then that person needs to specify '\r' as EOL (and open the file as binary). There's no one-size fits all and yes, you normally do need to quote string fields with embedded newlines with CSV. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From skip at pobox.com Mon Aug 18 11:11:14 2003 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 18 11:11:29 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: <20030818143455.GA28440@panix.com> References: <16192.18051.288852.969201@montanaro.dyndns.org> <20030818033814.GA24174@panix.com> <16192.53827.329638.239507@montanaro.dyndns.org> <20030818140241.GA1282@panix.com> <16192.57858.301752.30684@montanaro.dyndns.org> <20030818143455.GA28440@panix.com> Message-ID: <16192.60562.820665.478564@montanaro.dyndns.org> aahz> Then that person needs to specify '\r' as EOL (and open the file aahz> as binary). There's no one-size fits all and yes, you normally do aahz> need to quote string fields with embedded newlines with CSV. Then it's better than we never use universal newline (or plain text) mode, and do our own line parsing. Without it, I think the opportunities for confusion are too great. Skip From tim at zope.com Mon Aug 18 12:57:14 2003 From: tim at zope.com (Tim Peters) Date: Mon Aug 18 11:57:59 2003 Subject: [Python-Dev] Berkeley breakage Message-ID: Over in the spambayes project, we get reports of database corruption from people using Sleepycat bsddb. The most recent comment on a bug report is revealing: http://sf.net/tracker/?func=detail&atid=498103&aid=788051&group_id=61702 """ Date: 2003-08-18 04:09 Sender: fufsource i did some experimenting with various bsddb3 versions: - with db-3.3.11 the python segfaults and core is dumped - with db-3.2.9 the database is corrupted - with db-4.1.25 everything works as it should (no db corruption) """ spambayes makes elementary use of a Berkeley DB, just accessing via the dict interface -- inserts, deletes and lookups, but no cursors, no transactions, nothing "fancy". I don't have time to dig into this, but assuming the report is correct, how can we "encourage" Unix weenies to use db-4.1.25? (On Windows, db-4.1.25 is shipped with the installer.) If the problems with older versions are so severe, maybe the Python wrapper should do a version check and refuse to run if it finds an old version? From skip at pobox.com Mon Aug 18 12:19:15 2003 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 18 12:19:26 2003 Subject: [Python-Dev] Berkeley breakage In-Reply-To: References: Message-ID: <16192.64643.224595.664073@montanaro.dyndns.org> Tim> I don't have time to dig into this, but assuming the report is Tim> correct, how can we "encourage" Unix weenies to use db-4.1.25? (On Tim> Windows, db-4.1.25 is shipped with the installer.) If the problems Tim> with older versions are so severe, maybe the Python wrapper should Tim> do a version check and refuse to run if it finds an old version? I'm using it just fine with 3.3.11 on Mac OS X. I'm not using pop3proxy though, just hammiefilter (for both scoring and incremental training) or hammie (for training from scratch). According to the pybsddb project's README.txt file (updated five weeks ago): This wrapper should be compatible with BerkeleyDB releases going back to 3.1.17 up to and including DB 4.1.25. I'm not saying the poster is wrong, just that his problem is not trivially confirmed. Skip From theller at python.net Mon Aug 18 19:35:36 2003 From: theller at python.net (Thomas Heller) Date: Mon Aug 18 12:36:06 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <67C75048-CF6F-11D7-A73A-000A27B19B96@cwi.nl> (Jack Jansen's message of "Sat, 16 Aug 2003 00:25:42 +0200") References: <67C75048-CF6F-11D7-A73A-000A27B19B96@cwi.nl> Message-ID: Jack Jansen writes: > On vrijdag, aug 15, 2003, at 18:37 Europe/Amsterdam, Tom Emerson wrote: > >> Currently the module search path (and the related directory names set >> as a side effect) is determined by looking at the environment that the >> executable calling Py_Initialize() is running in. Hence if I've >> embedded Python 2.3 and also have Python 2.3 installed in (say >> /usr/local) it is going to use the Python paths in /usr/local/ over >> those in my customized embedded version. >> >> As far as I can tell, the only way I can control this behavior is to >> rewrite Py_GetPath and friends in my custom build. >> >> In my case the user of my application has a configuration file which >> specifies the pathnames for platform (in-)dependent files, both Python >> and other. But I cannot pass this information on to Py_Initialize() >> and on into Py_GetPath. >> >> Is it worth providing an alternative initialization API that allows >> these values to be specified explicitly instead of having them >> computed? Or is there a reason not to do this? > > +1. > Wouldn't a Py_SetPath function do the trick, which would initially set module_search_path (if it's not already set)? Thomas From tim at zope.com Mon Aug 18 13:36:44 2003 From: tim at zope.com (Tim Peters) Date: Mon Aug 18 12:39:26 2003 Subject: [Python-Dev] Berkeley breakage In-Reply-To: <16192.64643.224595.664073@montanaro.dyndns.org> Message-ID: [Skip Montanaro] > I'm using it just fine with 3.3.11 on Mac OS X. I'm not using > pop3proxy though, just hammiefilter (for both scoring and incremental > training) or hammie (for training from scratch). > > According to the pybsddb project's README.txt file (updated five > weeks ago): > > This wrapper should be compatible with BerkeleyDB releases going > back to > 3.1.17 up to and including DB 4.1.25. > > I'm not saying the poster is wrong, just that his problem is not > trivially confirmed. Berkeley version numbers seem partially insane, though: if you have 3.3.11, that doesn't tell us whether you've installed either, neither, or both of the available Sleepycat patches *to* 3.3.11. I don't know whether those patches fix potential corruption problems in 3.3.11, but IIRC Sleepycat patches don't bump the version number regardless. From tree at basistech.com Mon Aug 18 13:40:25 2003 From: tree at basistech.com (Tom Emerson) Date: Mon Aug 18 12:46:33 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: References: <67C75048-CF6F-11D7-A73A-000A27B19B96@cwi.nl> Message-ID: <16193.377.351938.408977@magrathea.basistech.com> Thomas Heller writes: [...] > >> Is it worth providing an alternative initialization API that allows > >> these values to be specified explicitly instead of having them > >> computed? Or is there a reason not to do this? > > Wouldn't a Py_SetPath function do the trick, which would initially set > module_search_path (if it's not already set)? Yes, that is essentially what I propose adding: void Py_SetPaths(modulepath, prefix, execprefix, fullpath); It needs to do more than just set module_search_path though, since as a side effect of determining module_search_path, calculate_path() also sets the values for prefix, exec_prefix, and prog_path which are returned by Py_GetPrefix, Py_GetExecPrefix, and Py_GetProgramFullPath respectively, which are used by for sys.prefix and sys.exec_prefix, and sys.executable. Py_GetExecPrefix is also used by the _PyPopen function in posixmodule.c. In any event, the idea is that an embedding application may know the values for each of these paths and can force them by calling Py_SetPaths prior to Py_Initialize, with the appropriate caveat that the caller better know what they are doing or strange things might happen. -tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From skip at pobox.com Mon Aug 18 13:05:27 2003 From: skip at pobox.com (Skip Montanaro) Date: Mon Aug 18 13:05:47 2003 Subject: [Python-Dev] Berkeley breakage In-Reply-To: References: <16192.64643.224595.664073@montanaro.dyndns.org> Message-ID: <16193.1879.493170.151143@montanaro.dyndns.org> Tim> Berkeley version numbers seem partially insane, though: if you have Tim> 3.3.11, that doesn't tell us whether you've installed either, Tim> neither, or both of the available Sleepycat patches *to* 3.3.11. I Tim> don't know whether those patches fix potential corruption problems Tim> in 3.3.11, but IIRC Sleepycat patches don't bump the version number Tim> regardless. That's a good question. There is a patch file in the /sw/fink directory which patches several files in the distribution: db-3.3.11/build_vxworks/db.h Thu Dec 20 15:03:34 2001 db-3.3.11/db185/db185.c Thu Dec 20 15:03:30 2001 db-3.3.11/db185/db185_int.in Thu Dec 20 15:03:30 2001 db-3.3.11/dist/configure Thu Dec 20 15:04:25 2001 db-3.3.11/include_auto/db185_ext.in Thu Dec 20 15:03:30 2001 db-3.3.11/include_auto/db185_uext.in Thu Dec 20 15:03:30 2001 db-3.3.11/include/mutex.h Fri Oct 25 21:57:35 2002 I think the dates are patch dates. Still, there's nothing explicit - like a README file - which says, "we installed the foo and bar patches. Comparing the above files with http://www.sleepycat.com/update/3.3.11/patch.3.3.11.html suggests that I have both applied. Neither looks like it fixes any runtime bugs which we'd encounter. One's to fix a compile problem. The other's specific to the vxworks platform. Skip From mwh at python.net Mon Aug 18 19:16:48 2003 From: mwh at python.net (Michael Hudson) Date: Mon Aug 18 13:16:52 2003 Subject: [Python-Dev] refleak hunting fun! In-Reply-To: <20030818133158.GQ8069@epoch.metaslash.com> (Neal Norwitz's message of "Mon, 18 Aug 2003 09:31:59 -0400") References: <2mk79gp53x.fsf@starship.python.net> <3F3BD826.5060600@livinglogic.de> <2m65kzp56d.fsf@starship.python.net> <3F3CD43C.3090206@livinglogic.de> <2mk79fnjw6.fsf@starship.python.net> <3F4019E2.1080500@livinglogic.de> <2mhe4fm72n.fsf@starship.python.net> <20030818133158.GQ8069@epoch.metaslash.com> Message-ID: <2mbrumnawv.fsf@starship.python.net> Neal Norwitz writes: > On Mon, Aug 18, 2003 at 02:25:04PM +0100, Michael Hudson wrote: >> >> I did notice that there are more than just leaks here, I'm afraid: >> >> >>> class BadSeq(tuple): >> ... def __getitem__(self, i): >> ... raise IndexError >> ... >> [25508 refs] >> >>> filter(None, BadSeq((1,))) >> Segmentation fault > > I found this seg fault too. > > The attached patch fixes the problem. I think it's correct. > Me too. Cheers, mwh -- A.D. 1517: Martin Luther nails his 95 Theses to the church door and is promptly moderated down to (-1, Flamebait). -- http://slashdot.org/comments.pl?sid=01/02/09/1815221&cid=52 (although I've seen it before) From harri.pasanen at trema.com Mon Aug 18 20:39:05 2003 From: harri.pasanen at trema.com (Harri Pasanen) Date: Mon Aug 18 13:39:40 2003 Subject: [Python-Dev] PEP 311 Simplified Global Interpreter Lock Acquisition for Extensions Message-ID: <200308181939.05396.harri.pasanen@trema.com> Hi, Just prior to my vacation I ran into a bit of trouble with my Python embedding, as much to my dismay my C++ program that had been working fine under Linux deadlocked under Windows. Apparently the underlying C threading API's were behaving in subtly different ways. After boiling down from my original embedded Python app which features omniORB, omniORBpy, and bunch of other goodies, I got down to the following testcase: #include class PythonLocker { public: PythonLocker () { _tstate = PyGILState_Ensure (); } ~PythonLocker() { PyGILState_Release(_tstate); } } private: PyGILState_STATE _tstate; }; int main (int argc, const char **argv) { Py_Initialize(); PyEval_InitThreads(); PyEval_ReleaseLock (); // This is the problem { PythonLocker pl; PyObject* usermod = PyImport_ImportModule ("echo"); // The previous line deadlocks on win32 } return 0; } The above works fine on Linux, but deadlocks on Win32. Tracing the execution on Win32 showed that it hanged in thread_nt.h: DWORD EnterNonRecursiveMutex(PNRMUTEX mutex, BOOL wait) { ... ret = InterlockedIncrement(&mutex->owned) ? /* Some thread owns the mutex, let's wait... */ WaitForSingleObject(mutex->hevent, INFINITE) : WAIT_OBJECT_0 ; The gotcha being that when entering above, mutex->owned was == -2, so it incorrectly entered the wait. I had a private discussion with Mark about this, and he quickly pointed out that I should not call PyEval_ReleaseLock(). Although this is not exactly clear from the PEP, which says "Apart from the existing, standard Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS macros, it is assumed that no additional thread state API functions will be used by the extension." I did not think PyEval_ReleaseLock() would qualify for the above, but somehow it does, under win32 at least. Now if I modify thread_nt.h, simple mindedly by changing the above to: ret = InterlockedIncrement(&mutex->owned) > 0 ? /* Some thread owns the mutex, let's wait... */ WaitForSingleObject(mutex->hevent, INFINITE) : WAIT_OBJECT_0 ; everything works for me on Win32 as well, including python regression tests. But I haven't wrestled with the code in question in detail to know what gremlins that change might unleash elsewhere. Without modifying Python, I can make my testcase working with the following change to my test program: { Py_Initialize(); PyEval_InitThreads(); { PythonLocker init_pl }; // release the lock // PyEval_ReleaseLock (); { // this block could potentially be called from another thread PythonLocker pl; PyObject* usermod = PyImport_ImportModule ("echo"); // The previous line deadlocks on win32 } In the C API world this is a bit more convoluted looking, as I need to call PyGILState_Ensure(), and PyGILState_Restore, even if after PyEval_InitThreads() I already know I have the lock. *** Sorry for rambling, but this is in the hope that someone else can avoid banging their heads against this as I have. And it is a bit unclear to me also if this kind of a platform difference between Linux and Win32 is a bug, or a feature. Ideally the Python C API should be bug-compatible. So possibly wrong usage should cause all platforms to fail. Btw., the new PyGILState_* functions do not appear to be part of the Python/C API documentation, there is only the PEP. Anyway, thanks to Mark for this definite improvement on the state of things. Harri From barry at python.org Mon Aug 18 18:50:27 2003 From: barry at python.org (Barry Warsaw) Date: Mon Aug 18 13:50:27 2003 Subject: [Python-Dev] Berkeley breakage In-Reply-To: References: Message-ID: <1061228995.1958.12.camel@yyz> On Mon, 2003-08-18 at 12:36, Tim Peters wrote: > Berkeley version numbers seem partially insane, though: if you have 3.3.11, > that doesn't tell us whether you've installed either, neither, or both of > the available Sleepycat patches *to* 3.3.11. Correct. That ain't the way Sleepcat does things. :) > I don't know whether those > patches fix potential corruption problems in 3.3.11, but IIRC Sleepycat > patches don't bump the version number regardless. A quick scan of the 3.3.11 patches doesn't indicate anything that would fix the reported problems. In the dim recesses of my memory, I recall similar unexplainable corruptions and core dumps when using 3.3.11. -Barry From jjl at pobox.com Mon Aug 18 19:49:56 2003 From: jjl at pobox.com (John J Lee) Date: Mon Aug 18 13:50:43 2003 Subject: [Python-Dev] Re: hook for standalone executable In-Reply-To: <3F3C98C4.19981.4C64CC0A@localhost> Message-ID: On Fri, 15 Aug 2003, Gordon McMillan wrote: [...] > It may be a surprise to the developer community, but there are > a lot of Unix users who are not developers and do not have > root priveleges on their machines. These people want to install > to ~/bin and don't particularly care what the install looks like. > > In this country it's usually analysts, statisticians and scientists. > In other countries where MS is not so popular, it's anybody. [...] Missed this discussion... A concrete example: the Pybliographer project (bibliographic database manager). Pyblio has had real problems with buggy shared libraries and Python & library versioning, and would certainly benefit from this kind of thing. The obvious problem with rpm / deb packages is that, in reality, a Redhat rpm isn't always portable to a SuSe system, etc. (there are a whole lot of GNU/Linux distributions out there, especially when you consider the rapid release schedule of many distributions!). Single-file executables eliminate that problem (though of course having a single executable doesn't preclude packaging in an rpm / deb -- makes it trivial to make portable packages, in fact). 'Proper' binary packages certainly have their place, but so do one-file executables, IMHO. Gordon's point about people outside North America, Western / Central Europe and Oz / NZ is very important -- that group of users certainly isn't going to shrink! John From tim at zope.com Mon Aug 18 17:46:55 2003 From: tim at zope.com (Tim Peters) Date: Mon Aug 18 16:47:46 2003 Subject: [Python-Dev] Insane change to md5sum.py in 2.3 Message-ID: The 2.3 version of the Tools script md5sum.py opens files in text mode by default. This is plain crazy -- even having an option to open a file in text mode is nuts for an md5 program. 2.2's version was much simpler and didn't have this deadly (well, it's deadly on Windows ...) flaw. I would like to remove the -t and -b switches altogether, except than anyone else who learned the hard way that they *have* to pass -b to the 2.3 version on Windows to get the right answer would get screwed. So I'd like to switch the default to binary mode instead. Objections? From jepler at unpythonic.net Mon Aug 18 17:50:11 2003 From: jepler at unpythonic.net (Jeff Epler) Date: Mon Aug 18 17:50:18 2003 Subject: [Python-Dev] Insane change to md5sum.py in 2.3 In-Reply-To: References: Message-ID: <20030818215010.GA4760@unpythonic.net> I'd advoate to treat -t as an error, ignore -b (or give a warning), and remove -b as soon as possible according to the regular deprecation rules. I agree, -t/-b are insane. Jeff From andymac at bullseye.apana.org.au Tue Aug 19 10:07:13 2003 From: andymac at bullseye.apana.org.au (Andrew MacIntyre) Date: Mon Aug 18 21:16:18 2003 Subject: [Python-Dev] Berkeley breakage In-Reply-To: <1061228995.1958.12.camel@yyz> References: <1061228995.1958.12.camel@yyz> Message-ID: <20030819090046.U78297@bullseye.apana.org.au> On Tue, 18 Aug 2003, Barry Warsaw wrote: > A quick scan of the 3.3.11 patches doesn't indicate anything that would > fix the reported problems. In the dim recesses of my memory, I recall > similar unexplainable corruptions and core dumps when using 3.3.11. Before the 2.3 release, I was testing all of 3.3.11, 4.0.14 and 4.1.25 builds of bsddb3 on FreeBSD. The 3.3.11 build always coredumps in the same place in regression test, while both 4.0.14 & 4.1.25 complete the test. I didn't have time to pursue the problem to see whether it was the FreeBSD build of DB 3.3.11 was the problem. I also forgot to file a bug report about the fact that the bsddb3 regression test doesn't clean out the DB environment it uses before getting down to business - a crash leaving environment info around (which is useful for debugging) screws a subsequent test run. Regards, Andrew. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia From barry at python.org Tue Aug 19 04:13:40 2003 From: barry at python.org (Barry Warsaw) Date: Mon Aug 18 23:13:41 2003 Subject: [Python-Dev] Found a bug! Message-ID: <1061262784.30737.80.camel@anthem> Well, I was debugging a Mailman user's problem when I hit what I think is a bug in mimelib/email pkg. I don't yet know if this is related to the MM bug because I can't get as far through the code, but you don't care about that. :) I believe mimelib is not handling RFC 2231 headers such as the following correctly: Content-Disposition: inline; filename*0="This%20is%20even%20more%20"; filename*1="%2A%2A%2Afun%2A%2A%2A%20"; filename*2="is it not.pdf" Try creating a message with a header like this and doing a msg.get_filename() on it -- you'll get a TypeError. The bug is that get_filename() isn't prepared for a missing language parameter. The fix is probably simple, and I'll try to resist doing too much refactoring . I'll check in a test case and a patch and backport it to the Python 2.3.1 and 2.2.4 branches. -Barry From barry at python.org Tue Aug 19 04:16:37 2003 From: barry at python.org (Barry Warsaw) Date: Mon Aug 18 23:16:38 2003 Subject: [Python-Dev] Found a bug! (in the email package) In-Reply-To: <1061262784.30737.80.camel@anthem> References: <1061262784.30737.80.camel@anthem> Message-ID: <1061262961.30737.85.camel@anthem> On Mon, 2003-08-18 at 23:13, Barry Warsaw wrote: > Well, I was debugging a Mailman user's problem when I hit what I think > is a bug in mimelib/email pkg. Oh, and sorry for the lame-ass subject. I was originally going to send the message just to mimelib-devel, where it would have been obvious what I was talking about. :) -Barry From greg at cosc.canterbury.ac.nz Tue Aug 19 16:30:05 2003 From: greg at cosc.canterbury.ac.nz (Greg Ewing) Date: Mon Aug 18 23:30:19 2003 Subject: [Python-Dev] pirate 0.01 alpha! In-Reply-To: Message-ID: <200308190330.h7J3U5G20632@oma.cosc.canterbury.ac.nz> Michal Wallace : > As a special bonus, it currently runs dog slow! The > microthreads example runs about 8-9 times slower on > pirate than on regular python. But that's not parrot's > fault: the generated code is completely unoptimized. It's fair, though -- the regular Python's bytecode is completely unoptimised, too. :-) Greg Ewing, Computer Science Dept, +--------------------------------------+ University of Canterbury, | A citizen of NewZealandCorp, a | Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. | greg@cosc.canterbury.ac.nz +--------------------------------------+ From shane.holloway at ieee.org Mon Aug 18 22:44:09 2003 From: shane.holloway at ieee.org (Shane Holloway (IEEE)) Date: Mon Aug 18 23:44:26 2003 Subject: [Python-Dev] Graph exploration with generators In-Reply-To: <20030818052822.GA89739@polya.axista.com> References: <20030818052822.GA89739@polya.axista.com> Message-ID: <3F419D09.5070709@ieee.org> Well, I've been holding this generators idea for after the Python 2.3 release since it would be a Python 2. or later feature. That and I didn't want to delay the release by even a the time to read an email. ;) Over the last few months, I've been using generators more and more to do lazy graph exploration. As a concrete example, take the following binary tree type structure: class Tree(object): def __init__(self, left=(), right=()): self.left = left self.right = right def __iter__(self): for each in self.left: yield each yield self for each in self.right: yield each This works great for iterating over ad-hoc graphs, and since "left" and "right" simply have to be iterable, you can chain trees to graphs, lists, or anything iterable. However, for each "chained" generator quite a bit of speed is lost. I assume this loss is due to restoring each context in the chain. (I would need help verifying this assumption.) What I would like to propose is a "yield *" type statement, where "yield *collection" would have the equivalent functionality of "for item in collection: yield item". The above example would then appear as follows: class TreeWithNewSyntax(object): def __init__(self, left=(), right=()): self.left = left self.right = right def __iter__(self): yield *self.left yield self yield *self.right The syntax in particular might be an abuse of the call(*args) metaphor, but it avoids a new language keyword and has the same general idea. By allowing this syntax, the need for restoring the context of (count-1) chains would circumvent for a chained generator solution. I admit that this is a pretty specific optimization area, but generators are a very natural way to express this type of problem in code. An implementation would speed these type of deeply nested generator solutions. I look forward to all your thoughts! Thanks, -Shane Holloway From andrewm at object-craft.com.au Tue Aug 19 14:45:57 2003 From: andrewm at object-craft.com.au (Andrew McNamara) Date: Mon Aug 18 23:46:01 2003 Subject: [Python-Dev] file read-ahead with Mac end-of-line In-Reply-To: Message from Aahz of "Mon, 18 Aug 2003 10:34:55 -0400." <20030818143455.GA28440@panix.com> References: <16192.18051.288852.969201@montanaro.dyndns.org> <20030818033814.GA24174@panix.com> <16192.53827.329638.239507@montanaro.dyndns.org> <20030818140241.GA1282@panix.com> <16192.57858.301752.30684@montanaro.dyndns.org> <20030818143455.GA28440@panix.com> Message-ID: <20030819034557.323B33CA49@coffee.object-craft.com.au> >Then that person needs to specify '\r' as EOL (and open the file as >binary). There's no one-size fits all and yes, you normally do need to >quote string fields with embedded newlines with CSV. The csv module, as defined in 2.3, takes an iterable as the source of input lines. It doesn't make sense to then attempt additional EOL processing within the csv module - the current EOL processing in the csv module is a hang-over from it's past life as a stand-alone module that worked with 1.5.2, etc. I would propose that we remove most of the EOL processing from the csv module, making it simply ignore any terminal whitespace characters. If someone has EOL conventions that aren't catered to by the Python universal newline code, they can supply an alternate iterable source of lines. This should make the csv module behave as other parts of python behave (less nasty surprises). -- Andrew McNamara, Senior Developer, Object Craft http://www.object-craft.com.au/ From eppstein at ics.uci.edu Mon Aug 18 22:00:26 2003 From: eppstein at ics.uci.edu (David Eppstein) Date: Tue Aug 19 00:00:30 2003 Subject: [Python-Dev] Re: Graph exploration with generators References: <20030818052822.GA89739@polya.axista.com> <3F419D09.5070709@ieee.org> Message-ID: In article <3F419D09.5070709@ieee.org>, "Shane Holloway (IEEE)" wrote: > I assume this loss is due to restoring each context in the chain. (I > would need help verifying this assumption.) > What I would like to propose is a "yield *" type statement, where "yield > *collection" would have the equivalent > functionality of "for item in collection: yield item". The above example > would then appear as follows: It's been proposed before, with somewhat different syntax ("yield every" instead of "yield *"): http://tinyurl.com/kfvc The syntax alone doesn't help speed things up, but iirc Greg Ewing had some good ideas for doing this efficiently. I think the concensus was, though, that generators are so new that we haven't had enough experience to know what the best ways of using them are, so it would be premature to settle on chaining them this way. -- David Eppstein http://www.ics.uci.edu/~eppstein/ Univ. of California, Irvine, School of Information & Computer Science From Jack.Jansen at cwi.nl Tue Aug 19 13:18:53 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Tue Aug 19 06:18:59 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <16193.377.351938.408977@magrathea.basistech.com> Message-ID: <88B3041E-D22E-11D7-9FC0-000A27B19B96@cwi.nl> On maandag, aug 18, 2003, at 18:40 Europe/Amsterdam, Tom Emerson wrote: > Thomas Heller writes: > [...] >>>> Is it worth providing an alternative initialization API that allows >>>> these values to be specified explicitly instead of having them >>>> computed? Or is there a reason not to do this? >> >> Wouldn't a Py_SetPath function do the trick, which would initially set >> module_search_path (if it's not already set)? > > Yes, that is essentially what I propose adding: > > void Py_SetPaths(modulepath, prefix, execprefix, fullpath); Note that if we're going to tackle this I think we should also have a way to disable the other code that looks at the environment to set the various flags. -- - Jack Jansen http://www.cwi.nl/~jack - - If I can't dance I don't want to be part of your revolution -- Emma Goldman - From michal at sabren.com Tue Aug 19 11:17:48 2003 From: michal at sabren.com (Michal Wallace) Date: Tue Aug 19 10:17:54 2003 Subject: [Python-Dev] pirate 0.01 alpha! In-Reply-To: <200308190330.h7J3U5G20632@oma.cosc.canterbury.ac.nz> Message-ID: On Tue, 19 Aug 2003, Greg Ewing wrote: > Michal Wallace : > > > As a special bonus, it currently runs dog slow! The > > microthreads example runs about 8-9 times slower on > > pirate than on regular python. But that's not parrot's > > fault: the generated code is completely unoptimized. > > It's fair, though -- the regular Python's bytecode is > completely unoptimised, too. :-) Yeah, but as I told someone else who made this point, it's really not a fair fight. Running something as simple as "x=x+x" through pirate currently allocates at least 4 new objects (possibly up to 8, depending what x is) and does three separate searches through the frame stack for the value of x. And that's just off the top of my head... :) Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------- From mfb at lotusland.dyndns.org Tue Aug 19 10:10:39 2003 From: mfb at lotusland.dyndns.org (Matthew F. Barnes) Date: Tue Aug 19 10:20:50 2003 Subject: [Python-Dev] Unification of logging in Python's Standard Library Message-ID: <56286.130.76.32.20.1061302239.squirrel@server.lotusland.dyndns.org> I posted this message to comp.lang.python last night. Alex Martelli suggested that I post it directly to python-dev. ---------- Matthew Barnes wrote: > With the inclusion of the new 'logging' module in Python 2.3, I'm > wondering whether there are plans to update the rest of the Standard > Library to use this module wherever there is logging to be done. I > can think of a few Standard Library modules off the top of my head > that currently "roll their own" logging system: > > asyncore.py > BaseHTTPServer.py > cgi.py > doctest.py > imaplib.py > unittest.py > > A single, unified logging system across the entire Python Standard > Library would be a worthy goal for some future release (Python 3000, > or maybe even 2.4/2.5?). This seems a good idea, and quite feasible for 2.4 (we do have over a year before 2.4 is going to be feature-frozen, after all). I would suggest you post this proposal directly to Python-dev, though -- here on c.l.py it's too likely to get accidentally overlooked! Alex From james at daa.com.au Tue Aug 19 23:04:49 2003 From: james at daa.com.au (James Henstridge) Date: Tue Aug 19 10:25:26 2003 Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST] In-Reply-To: <20030814022128.GN3095@async.com.br> References: <20030814022128.GN3095@async.com.br> Message-ID: <3F422E81.5000803@daa.com.au> On 14/08/2003 10:21 AM, Christian Reis wrote: >So, in an attempt to garner comments (now that we have 2.3 off the >chopping block) I'm reposting my PEP proposal (with minor updates). >Comments would be appreciated, of course (nudges Barry slightly after >him getting me to write this on my only free Sunday in months ;) > In my opinion, I think it is worth investigating this. One of the things I love about Python is the way how it can be used to glue different bits of code together to form useful programs. Python's handling of LC_NUMERIC seems to work against this goal. According to the POSIX standard, standard library functions like strtod() and printf()/sprintf() work with locale representations of floating point numbers once the setlocale() function has been called. For locale aware libraries, it seems quite sensible for them to use standard library functions to format dates for display, and reading dates entered by the user (which may use a comma for the decimal point under some locales). Now since Python uses the C standard library functions to parse floating point numbers too, and doesn't want these string <-> float conversions to be locale sensitive. The solution currently in place is to declare that LC_NUMERIC must be C for Python programs (or applications embedding Python) or things will go wrong. If a bit of Python code wishes to format a float according to the locale it can call the locale.format() and to read a float from the user, can use locale.atof(). These use locale data read while the locale was set to the correct value temporarily. Unfortunately, this does not help external libraries that know nothing of Python's requirements about how locales are set up, so often they will always represent and read floating point numbers under the C locale (using a full stop as the decimal point). This is the problem Christian ran into with the GtkSpinButton widget in GTK, and I would not be surprised if other people have run into the problem as well. There are two solutions to this problem that I can see: 1. modify Python so that it doesn't use locale sensitive conversion functions when it wants to convert floats in a locale independent manner. 2. modify every external library that makes use of the standard library strtod()/sprintf(), expecting locale sensitive float conversions, to use some other API. Clearly (1) is the easier option, as there is a finite (and quite small) amount of code to change. For (2), there is potentially an unlimited amount of code to change. As Christian said, there is code in glib (not to be confused with glibc: the GNU C library) that could act as a basis for locale independent float conversion functions in Python. The code was written by Alex Larsson (who works at Red Hat, so I suppose they own the copyright), who is willing to license it under Python's terms. You can see the history of the two functions (g_ascii_strtod() and g_ascii_formatd()) here: http://cvs.gnome.org/bonsai/cvsblame.cgi?file=glib/glib/gstrfuncs.c#328 There are very minor alterations by other people (they look minor enough that the FSF wouldn't require a copyright assignment), but you could always use the versions from the initial checkin (rev 1.77) if that is a problem. One of the alternatives that some programs use to do locale independent conversions using code a bit like this: char *oldlocale = setlocale(LC_NUMERIC, "C"); num = strtod(string, NULL); setlocale(LC_NUMERIC, oldlocale); This particular code snippet has some problems that I think make it unsuitable for Python: * setlocale() affects the whole process, so this sort of operation could affect the results of strtod, printf, etc for some other thread in the program. * setlocale() is not reentrant. Oldlocale in the above snippet is a pointer to a static string. If you surround the snippet with another pair of setlocale() calls, you can get unexpected results. This means that adding setlocale() calls to random Python API calls has the potential to break existing code. Alex's code from glib does not suffer from these problems. To sum it up, the current status-quo in Python w.r.t. locales is causing problems for some problems people want to use Python for. It would be nice to fix this problem. James. -- Email: james@daa.com.au WWW: http://www.daa.com.au/~james/ From tree at basistech.com Tue Aug 19 12:00:11 2003 From: tree at basistech.com (Tom Emerson) Date: Tue Aug 19 11:05:58 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <88B3041E-D22E-11D7-9FC0-000A27B19B96@cwi.nl> References: <16193.377.351938.408977@magrathea.basistech.com> <88B3041E-D22E-11D7-9FC0-000A27B19B96@cwi.nl> Message-ID: <16194.15227.214357.303033@magrathea.basistech.com> Jack Jansen writes: > > void Py_SetPaths(modulepath, prefix, execprefix, fullpath); > > Note that if we're going to tackle this I think we should also have > a way to disable the other code that looks at the environment to set > the various flags. I think you get this for free because calculate_path is never called when module_search_path is set. tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From fog at initd.org Tue Aug 19 18:51:52 2003 From: fog at initd.org (Federico Di Gregorio) Date: Tue Aug 19 12:21:56 2003 Subject: [Python-Dev] PyMapping_Check Bug? (fwd) In-Reply-To: <16186.36599.22737.157720@montanaro.dyndns.org> References: <16186.36599.22737.157720@montanaro.dyndns.org> Message-ID: <1061308312.1524.7.camel@localhost> Il mer, 2003-08-13 alle 21:18, Skip Montanaro ha scritto: > (muffed the first send...) > > >>>>> "Guido" == Guido van Rossum writes: > > >> Passing along a fragment from the psycopg list where Federico Di Gregorio > >> (fog@initd.org) wrote: > >> > >> .... because on python 2.3 PyMapping_Check apperently return True for > >> every new-type instance.... > >> > >> Is this true? If so, is it a bug? > > Guido> I don't see this. operator.isMappingType() maps directly to > Guido> PyMapping_Check(); and I tried this: I tryed this (and this is where the problem lies wrt psycopg): Python 2.3+ (#2, Aug 10 2003, 11:33:47) [GCC 3.3.1 (Debian)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import operator >>> operator.isMappingType(()) True >>> operator.isMappingType([]) True apparently a tuple, a list or any subtype *is* a mapping type. please, cc me because i am not on python-devel. thank you very much, federico -- Federico Di Gregorio Debian GNU/Linux Developer fog@debian.org INIT.D Developer fog@initd.org Io la coscienza l'ho cagata da piccolo. -- Natale Titotto -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: Questa parte del messaggio =?ISO-8859-1?Q?=E8?= firmata Url : http://mail.python.org/pipermail/python-dev/attachments/20030819/709b488c/attachment.bin From barry at python.org Tue Aug 19 17:31:37 2003 From: barry at python.org (Barry Warsaw) Date: Tue Aug 19 12:31:38 2003 Subject: [Python-Dev] Unification of logging in Python's Standard Library In-Reply-To: <56286.130.76.32.20.1061302239.squirrel@server.lotusland.dyndns.org> References: <56286.130.76.32.20.1061302239.squirrel@server.lotusland.dyndns.org> Message-ID: <1061310695.1131.2.camel@geddy> Re: making logging consistent in the standard library... On Tue, 2003-08-19 at 10:10, Matthew F. Barnes wrote: > This seems a good idea, and quite feasible for 2.4 (we do have over > a year before 2.4 is going to be feature-frozen, after all). I would > suggest you post this proposal directly to Python-dev, though -- > here on c.l.py it's too likely to get accidentally overlooked! +1 -Barry From administrator at vt.edu Tue Aug 19 13:48:45 2003 From: administrator at vt.edu (administrator@vt.edu) Date: Tue Aug 19 12:48:48 2003 Subject: [Python-Dev] Virus Warning Message-ID: <200308191648.BNJ30186@vivi.cc.vt.edu> The message you emailed to hunnings@vt.edu, dated 08/19/03 12:48:45, contains the W32/Sobig-F virus in the application.pif attachment. The action taken was: deleted the attachment. From administrator at vt.edu Tue Aug 19 15:31:29 2003 From: administrator at vt.edu (administrator@vt.edu) Date: Tue Aug 19 14:31:37 2003 Subject: [Python-Dev] Virus Warning Message-ID: <200308191831.BUG70316@zidane.cc.vt.edu> The message you emailed to jbaney@vt.edu, dated 08/19/03 14:31:28, contains the W32/Sobig-F virus in the wicked_scr.scr attachment. The action taken was: deleted the attachment. From guido at python.org Tue Aug 19 10:41:50 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 19 15:10:31 2003 Subject: [Python-Dev] Unification of logging in Python's Standard Library In-Reply-To: Your message of "Tue, 19 Aug 2003 09:10:39 CDT." <56286.130.76.32.20.1061302239.squirrel@server.lotusland.dyndns.org> References: <56286.130.76.32.20.1061302239.squirrel@server.lotusland.dyndns.org> Message-ID: <200308191641.h7JGfop14260@12-236-84-31.client.attbi.com> > > With the inclusion of the new 'logging' module in Python 2.3, I'm > > wondering whether there are plans to update the rest of the Standard > > Library to use this module wherever there is logging to be done. I > > can think of a few Standard Library modules off the top of my head > > that currently "roll their own" logging system: > > > > asyncore.py > > BaseHTTPServer.py > > cgi.py > > doctest.py > > imaplib.py > > unittest.py > > > > A single, unified logging system across the entire Python Standard > > Library would be a worthy goal for some future release (Python 3000, > > or maybe even 2.4/2.5?). > > This seems a good idea, and quite feasible for 2.4 (we do have over > a year before 2.4 is going to be feature-frozen, after all). I would > suggest you post this proposal directly to Python-dev, though -- > here on c.l.py it's too likely to get accidentally overlooked! +1 --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at python.net Tue Aug 19 22:26:47 2003 From: theller at python.net (Thomas Heller) Date: Tue Aug 19 15:27:48 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <16194.15227.214357.303033@magrathea.basistech.com> (Tom Emerson's message of "Tue, 19 Aug 2003 11:00:11 -0400") References: <16193.377.351938.408977@magrathea.basistech.com> <88B3041E-D22E-11D7-9FC0-000A27B19B96@cwi.nl> <16194.15227.214357.303033@magrathea.basistech.com> Message-ID: <8yppla88.fsf@python.net> Tom Emerson writes: > Jack Jansen writes: >> > void Py_SetPaths(modulepath, prefix, execprefix, fullpath); >> >> Note that if we're going to tackle this I think we should also have >> a way to disable the other code that looks at the environment to set >> the various flags. > > I think you get this for free because calculate_path is never called > when module_search_path is set. IMO Jack meant variables like Py_OptimizeFlag and Py_DebugFlag which are set at the beginning of Py_Initialize from some env vars. Thomas From kavkeeper at ns1.epcwc.com Wed Aug 20 17:15:55 2003 From: kavkeeper at ns1.epcwc.com (kavkeeper@ns1.epcwc.com) Date: Fri Aug 22 08:27:20 2003 Subject: [Python-Dev] Virus found in message from you! Message-ID: Kaspersky Anti-Virus 4.0.0 reports a problem: you sent a message with a virus ! In the following message: ---------------------- From:python-dev@python.org To:tluu@epcusa.com ---------------------- Please check your computer with Kaspersky Antivirus Scanner! From dave at boost-consulting.com Thu Aug 21 19:35:21 2003 From: dave at boost-consulting.com (David Abrahams) Date: Fri Aug 22 08:50:22 2003 Subject: [Python-Dev] Re: Graph exploration with generators References: <20030818052822.GA89739@polya.axista.com> <3F419D09.5070709@ieee.org> Message-ID: "Shane Holloway (IEEE)" writes: > Well, I've been holding this generators idea for after the Python 2.3 > release since it would be a Python 2. or later feature. That and I > didn't want to delay the release by even a the time to read an email. > ;) > > Over the last few months, I've been using generators more and more to > do lazy graph exploration. As a concrete example, take the following > binary tree type structure: > > class Tree(object): > def __init__(self, left=(), right=()): > self.left = left > self.right = right > > def __iter__(self): > for each in self.left: > yield each > yield self > for each in self.right: > yield each > > This works great for iterating over ad-hoc graphs, and since "left" > and "right" simply have to be iterable, you can chain trees to graphs, > lists, or anything iterable. However, for each "chained" generator > quite a bit of speed is lost. I assume this loss is due to restoring > each context in the chain. (I would need help verifying this > assumption.) > What I would like to propose is a "yield *" type statement, where > "yield *collection" would have the equivalent functionality of "for > item in collection: yield item". The above example would then appear > as follows: I wonder if this is improves the speed and is expressive enough: class YieldSeq(object): __slots__ = [ 'outer', 'inner' ] def __init__(self, *seq): self.outer = iter(seq) self.inner = iter(()) # start with something empty def __iter__(self): return self def next(self): while 1: try: return self.inner.next() except StopIteration: self.inner = iter(self.outer.next()) class Tree(object): def __init__(self, name, left=(), right=()): self.name = name self.left = left self.right = right def __iter__(self): return YieldSeq(self.left, (self,), self.right) def __repr__(self): return self.name t = Tree('a', Tree('b', Tree('c'), Tree('d')), Tree('e', Tree('f'), Tree('g') ) ) print [ c for c in t ] Of course I realize it's not as beautiful as using generators... -- Dave Abrahams Boost Consulting www.boost-consulting.com From eppstein at ics.uci.edu Wed Aug 20 09:13:24 2003 From: eppstein at ics.uci.edu (David Eppstein) Date: Fri Aug 22 09:27:52 2003 Subject: [Python-Dev] Re: Graph exploration with generators In-Reply-To: References: Message-ID: <18857488.1061367204@[10.0.1.2]> On 8/20/03 10:53 AM -0400 Kevin Jacobs wrote: > 99% of the "problem" can be taken care of by adding a relatively simple > optimization to the interpreter. It involves adding the necessary > book-keeping to propogate values yielded by a generator directly to the > next expression it is needed in, and conversely to return to the generator > directly when the next value is requested. Say we have a depth-N tree, > then for each leaf node, the interpreter must pass a value up to N-1 > levels of generators. If the values yielded are direct generator calls, > then all N-1 of those yields can be elided. This allows "yield *" to be > written efficiently as a simple C-function, that is equivalent to this > Python generator: > > def yield_star(iterable): > for i in iterable: > yield i > > Interestingly enough, this same optimization can also be applied to return > expressions/values, and is related to tail-call optimizations done by > traditional compilers and functional-language interpreters. It is also > very much related to continuation-passing-style (CPS) transformations, > which for Python generators, have very tractable semantics. If you are implying (when you say "all N-1 of those yields can be elided" that recursive chains of "yield *" can be handled in such a way that each generated item is passed through the whole chain in constant time, then you are mistaken. I can prove a nonconstant lower bound via a simulation of union-find, and the best amortized upper bound I know is logarithmic. Ewing's technique, that I referred to in a previous message, should do better in practice but not in the worst case. The problem is that, if a generator X calls "yield *" on a generator Y, we can not safely assume that all of Y's output goes to X. X can be suspended and in the meantime someone else can request values from Y. -- David Eppstein http://www.ics.uci.edu/~eppstein/ Univ. of California, Irvine, School of Information & Computer Science From alexl at redhat.com Fri Aug 22 03:21:59 2003 From: alexl at redhat.com (Alexander Larsson) Date: Fri Aug 22 09:38:57 2003 Subject: [Python-Dev] Re: g_ascii_strtod/snprintf and Python In-Reply-To: <20030814184908.GA1249@async.com.br> References: <20030814021408.GL3095@async.com.br> <1060842501.6956.45.camel@localhost.localdomain> <20030814184908.GA1249@async.com.br> Message-ID: <1061369455.21871.114.camel@localhost.localdomain> On Thu, 2003-08-14 at 20:49, Christian Reis wrote: > Actually, you're correct. Guido replied to python-dev today (but > unfortunately omitted you from the CC: list) the following: > > From: Guido van Rossum > Date: Thu, 14 Aug 2003 07:54:19 -0700 > Cc: python-dev@python.org > X-Spambayes-Classification: ham; 0.00 > > > > Which form do I need to fill out? "Proposed Contributor > > > Agreement" seems a bit much. I don't need a cvs account. Maybe > > > "Proposed Contibution Licensing Agreement"? > > > > Well, the situation is a bit confused at the moment. To my > > knowledge *noone* has signed one of these agreements yet -- I > > certainly haven't. > > That's because we're still discussing them with lawyers (a year too > long :-( ). > > Eventually we'll want one from you too. In this specific case of > code that is known to be distributed as part of glibc as well, I > think it's important to have an explicit paper trail from the > original author (who better be the author of *all* lines he's > contributing to us :-). > > I recommend that he sign the Proposed Contribution Licensing > Agreement. > > --Guido van Rossum (home page: http://www.python.org/~guido/) > > So, being Guido the BDFL, I'd take his word on it. I think the process > would be filling out the form, mailing it and dropping a note to > python-dev@python.org (or replying to the original mail) letting people > know that the form has been sent. I'll try and take care of the rest > from there (and I'll let you know if something has come up). I mailed this today. =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Alexander Larsson Red Hat, Inc alexl@redhat.com alla@lysator.liu.se He's a Nobel prize-winning amnesiac firefighter with a mysterious suitcase handcuffed to his arm. She's a virginal out-of-work politician looking for love in all the wrong places. They fight crime! From jacobs at penguin.theopalgroup.com Wed Aug 20 11:53:47 2003 From: jacobs at penguin.theopalgroup.com (Kevin Jacobs) Date: Fri Aug 22 09:47:38 2003 Subject: [Python-Dev] Re: Graph exploration with generators In-Reply-To: Message-ID: On Mon, 18 Aug 2003, David Eppstein wrote: > In article <3F419D09.5070709@ieee.org>, > "Shane Holloway (IEEE)" wrote: > > > I assume this loss is due to restoring each context in the chain. (I > > would need help verifying this assumption.) > > What I would like to propose is a "yield *" type statement, where "yield > > *collection" would have the equivalent > > functionality of "for item in collection: yield item". The above example > > would then appear as follows: > > It's been proposed before, with somewhat different syntax ("yield every" > instead of "yield *"): http://tinyurl.com/kfvc > > The syntax alone doesn't help speed things up, but iirc Greg Ewing had > some good ideas for doing this efficiently. I think the concensus was, > though, that generators are so new that we haven't had enough experience > to know what the best ways of using them are, so it would be premature > to settle on chaining them this way. 99% of the "problem" can be taken care of by adding a relatively simple optimization to the interpreter. It involves adding the necessary book-keeping to propogate values yielded by a generator directly to the next expression it is needed in, and conversely to return to the generator directly when the next value is requested. Say we have a depth-N tree, then for each leaf node, the interpreter must pass a value up to N-1 levels of generators. If the values yielded are direct generator calls, then all N-1 of those yields can be elided. This allows "yield *" to be written efficiently as a simple C-function, that is equivalent to this Python generator: def yield_star(iterable): for i in iterable: yield i Interestingly enough, this same optimization can also be applied to return expressions/values, and is related to tail-call optimizations done by traditional compilers and functional-language interpreters. It is also very much related to continuation-passing-style (CPS) transformations, which for Python generators, have very tractable semantics. Regards, -Kevin -- -- Kevin Jacobs The OPAL Group - Enterprise Systems Architect Voice: (440) 871-6725 x 19 E-mail: jacobs@theopalgroup.com Fax: (440) 871-6722 WWW: http://www.theopalgroup.com/ From theller at python.net Wed Aug 20 19:57:30 2003 From: theller at python.net (Thomas Heller) Date: Fri Aug 22 10:20:03 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <1xvjrsnd.fsf@python.net> (Thomas Heller's message of "Mon, 18 Aug 2003 15:40:06 +0200") References: <007601c364fe$3a49ed20$e841fea9@oemcomputer> <2mekzjm6u1.fsf@starship.python.net> <1xvjrsnd.fsf@python.net> Message-ID: I'm just, after downloading, building and installing a lot of stuff, trying to make sure that I really can build the Windows installer for 2.3.1. I'm practicing on the trunk - I don't think anyone wants to review the changes to the windows installer, so I'll check them in shortly, it seems that everything works so far. The only question remaining is how to build the html docs. I don't think it works on Windows (I remember having built the pdf docs with TeTeX, but that was it). Do I have to fire up linux in a vmware box to build them, or are there other possibilities? Thomas From idm at idmcomp.com Wed Aug 20 19:57:57 2003 From: idm at idmcomp.com (Ian D. Mead/IDM Computer Solutuions) Date: Fri Aug 22 10:37:36 2003 Subject: [Python-Dev] Your last email to this domain was deleted In-Reply-To: <200308202257.h7KMvpg19585@idm01.> References: <200308202257.h7KMvpg19585@idm01.> Message-ID: <200308202257.h7KMvvH19637@idmcomp.com> I just received a message from you. Unfortunately it contained an attachment that is not permitted to be sent to this server. The message was DELETED! Please zip the attachment and resend the message with the zipped attachment. If you did not send a message, it was probably a virus that reported you as the sender. There is no need to reply to this message. Thank you. Ian From viruscheck at rz.uni-osnabrueck.de Thu Aug 21 17:47:53 2003 From: viruscheck at rz.uni-osnabrueck.de (viruscheck@rz.uni-osnabrueck.de) Date: Fri Aug 22 10:43:54 2003 Subject: [Python-Dev] Virus in Mail entdeckt / Virus found in the message Message-ID: <3F44DB99.0002CD.17996@sanode5eth1.rz.uni-osnabrueck.de> >>> Absender <<< Diese Nachricht wurde automatisch generiert von: Mail Scanner, Universitaet Osnabrueck, Rechenzentrum E-Mail: VirusCheck@rz.uni-osnabrueck.de >>> Informationen <<< Der Mail Scanner hat eine infizierte Mail entdeckt, die Ihre Absender-Adresse traegt. Diese Mail ist *nicht* an die Empfaenger weitergeleitet worden. Absender: Empfaenger: >>> Virus-Beschreibung (englisch) <<< Email data: MessageID: <200308211447.h7LEliTA000632@mail-in-1.serv.uni-osnabrueck.de> From: To: Cc: Subject: Re: Approved Scanning part [] Scanning part [thank_you.pif] Attachment validity check: passed. Virus identity found: W32/Sobig-F >>> Massnahmen <<< Moeglicherweise ist Ihre Absender-Adresse ohne Ihr Wissen verwendet worden. Sollte sich Ihr Rechner als viren-frei herausstellen, betrachten Sie diese Mail nur als Benachrichtigung, dass Viren unter missbraeuchlicher Verwendung Ihrer Absender-Adresse verschickt worden sind. In diesem Fall sind keine weiteren Massnahmen notwendig. Andernfalls pruefen Sie bitte den Datei-Anhang (Attachment) auf den gemeldeten Virus ("Virus identity found"), bevor Sie die Mail erneut versenden. Hinweise zur Desinfektion finden Sie auf der Heimatseite von Sophos unter http://www.sophos.com. Mit freundlichen Gruessen Mail Scanner, Universitaet Osnabrueck, Rechenzentrum E-Mail: VirusCheck@rz.uni-osnabrueck.de From SAVMSE at afflink.com Wed Aug 20 10:23:10 2003 From: SAVMSE at afflink.com (SAVMSE@afflink.com) Date: Fri Aug 22 11:07:37 2003 Subject: [Python-Dev] Symantec AVF detected that you sent a message with a prohibited subject line Message-ID: <3bb001c36726$954e4a20$2ffeffd0@afflink.local> Subject of the message: Re: Your application Recipient of the message: Zhuozhou Liu Subject of the message: Re: Your application Recipient of the message: Zhuozhou Liu From theller at python.net Wed Aug 20 20:16:23 2003 From: theller at python.net (Thomas Heller) Date: Fri Aug 22 12:02:44 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: (Thomas Heller's message of "Wed, 20 Aug 2003 18:57:30 +0200") References: <007601c364fe$3a49ed20$e841fea9@oemcomputer> <2mekzjm6u1.fsf@starship.python.net> <1xvjrsnd.fsf@python.net> Message-ID: Thomas Heller writes: > I'm just, after downloading, building and installing a lot of stuff, > trying to make sure that I really can build the Windows installer for > 2.3.1. > > I'm practicing on the trunk - I don't think anyone wants to review the > changes to the windows installer, so I'll check them in shortly, it > seems that everything works so far. > > The only question remaining is how to build the html docs. > I don't think it works on Windows (I remember having built the pdf docs > with TeTeX, but that was it). > > Do I have to fire up linux in a vmware box to build them, or are there > other possibilities? Seems I should have read pep 101 before posting. Sorry for the noise. Thomas From MAILER-DAEMON at sover.net Wed Aug 20 12:44:29 2003 From: MAILER-DAEMON at sover.net (Mail Delivery Subsystem) Date: Fri Aug 22 12:04:27 2003 Subject: [Python-Dev] Warning: could not send message for past 4 hours Message-ID: <200308201544.h7K6XZo23539@maillist0.sover.net> ********************************************** ** THIS IS A WARNING MESSAGE ONLY ** ** YOU DO NOT NEED TO RESEND YOUR MESSAGE ** ********************************************** The original message was received at Wed, 20 Aug 2003 02:31:34 -0400 (EDT) from pcp01709343pcs.south01.md.comcast.net [68.50.248.199] ----- The following addresses had transient non-fatal errors ----- ----- Transcript of session follows ----- ... while talking to gro.dd.org.: >>> RCPT To: <<< 450 : User unknown in local recipient table ... Deferred: 450 : User unknown in local recipient table Warning: message still undelivered after 4 hours Will keep trying until message is 5 days old -------------- next part -------------- Skipped content of type message/delivery-status-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/rfc822-headers Size: 646 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20030820/d193b96a/attachment-0001.bin From Jack.Jansen at cwi.nl Wed Aug 20 13:26:48 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Fri Aug 22 15:51:35 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <16194.15227.214357.303033@magrathea.basistech.com> Message-ID: On Tuesday, August 19, 2003, at 05:00 PM, Tom Emerson wrote: > Jack Jansen writes: >>> void Py_SetPaths(modulepath, prefix, execprefix, fullpath); >> >> Note that if we're going to tackle this I think we should also have >> a way to disable the other code that looks at the environment to set >> the various flags. > > I think you get this for free because calculate_path is never called > when module_search_path is set. Nope... Look at the top of Py_Initialize() in pythonrun.c. It's looking at all the environment variables itself. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From MAILER-DAEMON at cisco.com Wed Aug 20 03:43:24 2003 From: MAILER-DAEMON at cisco.com (Mail Delivery Subsystem) Date: Fri Aug 22 15:52:11 2003 Subject: [Python-Dev] Returned mail: see transcript for details Message-ID: <200308200943.h7K9hO1g005450@sj-inbound-1.cisco.com> The original message was received at Wed, 20 Aug 2003 02:43:05 -0700 (PDT) from [207.164.58.87] ----- The following addresses had permanent fatal errors ----- (reason: 552 5.0.0 SOBIG.F Virus outbreak - temp fix - change your subject) (expanded from: ) ----- Transcript of session follows ----- ... while talking to sj-core-1.cisco.com.: >>> DATA <<< 552 5.0.0 SOBIG.F Virus outbreak - temp fix - change your subject 554 5.0.0 Service unavailable -------------- next part -------------- Skipped content of type message/delivery-status-------------- next part -------------- An embedded message was scrubbed... From: Subject: Re: Wicked screensaver Date: Wed, 20 Aug 2003 5:42:42 --0400 Size: 893 Url: http://mail.python.org/pipermail/python-dev/attachments/20030820/be889eb9/attachment-0001.eml From viewcvs-bounces at lyra.org Wed Aug 20 10:23:07 2003 From: viewcvs-bounces at lyra.org (viewcvs-bounces@lyra.org) Date: Fri Aug 22 16:33:44 2003 Subject: [Python-Dev] Your message to viewcvs awaits moderator approval Message-ID: Your mail to 'viewcvs' with the subject Re: Details Is being held until the list moderator can review it for approval. The reason it is being held: Post by non-member to a members-only list Either the message will get posted to the list, or you will receive notification of the moderator's decision. If you would like to cancel this posting, please visit the following URL: http://mailman.lyra.org/mailman/confirm/viewcvs/b80d5ade1a6e76bc5863eedbfc4f2a5472a0373c From gstein at lyra.org Wed Aug 20 00:00:04 2003 From: gstein at lyra.org (gstein@lyra.org) Date: Fri Aug 22 16:39:08 2003 Subject: [Python-Dev] received your email Message-ID: <200308200600.h7K604LA017111@nebula.lyra.org> Hi, [ Re: Re: Thank you! ] I have received your email, but it may take a while to respond. I'm really sorry to have to hook up this auto-responder, as it is so impersonal. However, I get a lot of email every day and find it very difficult to keep up with it. Please be patient while I try to get to your message. Please feel free to resend your message if you think I've missed it. I'll always respond to personal email first. If your email is regarding some of the software that I work on (if you have questions, comments, suggestions, etc), then please resend it to the appropriate mailing list: mod_dav WebDAV ViewCVS Subversion edna Thank you! Cheers, -g -- Greg Stein, http://www.lyra.org/ From postmast at ucsbuxc.ucsb.edu Tue Aug 19 11:10:45 2003 From: postmast at ucsbuxc.ucsb.edu (postmast@ucsbuxc.ucsb.edu) Date: Fri Aug 22 17:01:23 2003 Subject: [Python-Dev] Virus Alert Message-ID: <200308191710.h7JHAjS24849@ucsbuxc.ucsb.edu> A mail message that you apparently sent to joan.magruder contains the WORM_SOBIG.F virus in the thank_you.pif file you included. The virus has been quarantined on ucsbuxc.ucsb.edu and the rest of the message was delivered to the recipient(s). If you have any questions please contact your local email support staff, or postmaster@ucsbuxc.ucsb.edu. From MAILER-DAEMON at sover.net Wed Aug 20 12:39:24 2003 From: MAILER-DAEMON at sover.net (Mail Delivery Subsystem) Date: Fri Aug 22 17:04:26 2003 Subject: [Python-Dev] Warning: could not send message for past 4 hours Message-ID: <200308201539.h7K6XZF23539@maillist0.sover.net> ********************************************** ** THIS IS A WARNING MESSAGE ONLY ** ** YOU DO NOT NEED TO RESEND YOUR MESSAGE ** ********************************************** The original message was received at Wed, 20 Aug 2003 02:04:13 -0400 (EDT) from pcp01709343pcs.south01.md.comcast.net [68.50.248.199] ----- The following addresses had transient non-fatal errors ----- ----- Transcript of session follows ----- ... while talking to gro.dd.org.: >>> RCPT To: <<< 450 : User unknown in local recipient table ... Deferred: 450 : User unknown in local recipient table Warning: message still undelivered after 4 hours Will keep trying until message is 5 days old -------------- next part -------------- Skipped content of type message/delivery-status-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/rfc822-headers Size: 640 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20030820/5492962a/attachment-0001.bin From tree at basistech.com Wed Aug 20 12:11:48 2003 From: tree at basistech.com (Tom Emerson) Date: Fri Aug 22 17:57:05 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: References: <16194.15227.214357.303033@magrathea.basistech.com> Message-ID: <16195.36788.437058.459054@magrathea.basistech.com> As I look at Py_Initialize() further, I see some other 'features' that could be problematic when embedding: particularly the calls to Py_FatalError. An embedding application may be able to continue even if the Python interpreter cannot be initialized... certainly it should be up to the embedding application on how to handle the error, instead of having abort() called for it. It would also be nice if there were no calls fprintf and friends within the initialization path when doing embedded initialization. tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From tree at basistech.com Wed Aug 20 12:07:58 2003 From: tree at basistech.com (Tom Emerson) Date: Fri Aug 22 17:57:50 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <8yppla88.fsf@python.net> References: <16193.377.351938.408977@magrathea.basistech.com> <88B3041E-D22E-11D7-9FC0-000A27B19B96@cwi.nl> <16194.15227.214357.303033@magrathea.basistech.com> <8yppla88.fsf@python.net> Message-ID: <16195.36558.653251.612076@magrathea.basistech.com> Thomas Heller writes: > IMO Jack meant variables like Py_OptimizeFlag and Py_DebugFlag which > are set at the beginning of Py_Initialize from some env vars. Indeed, and I agree. To maintain compatibility perhaps a new initialization function should be created that takes the argument, and Py_Initialize() calls this. -tree -- Tom Emerson Basis Technology Corp. Software Architect http://www.basistech.com "Beware the lollipop of mediocrity: lick it once and you suck forever" From MAILER-DAEMON at rcn.com Wed Aug 20 12:06:22 2003 From: MAILER-DAEMON at rcn.com (Mail Delivery Subsystem) Date: Fri Aug 22 18:58:24 2003 Subject: [Python-Dev] Returned mail: Over quota Message-ID: <200308201506.AWZ77492@ms08.mrf.mail.rcn.net> The original message was received at Wed, 20 Aug 2003 11:06:21 -0400 (EDT) from localhost ----- The following addresses had permanent delivery errors ----- drib@rcn.com -------------- next part -------------- Skipped content of type message/delivery-status-------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: text/rfc822-headers Size: 1002 bytes Desc: not available Url : http://mail.python.org/pipermail/python-dev/attachments/20030820/a0e03b9a/attachment.bin From postmaster at utc.fr Fri Aug 22 20:47:18 2003 From: postmaster at utc.fr (postmaster@utc.fr) Date: Fri Aug 22 20:08:20 2003 Subject: [Python-Dev] Virus Alert Message-ID: <20030822174717.DAE2F9329@gamma.utc.fr> Have detected a virus (WORM_SOBIG.F.DAM) in a mail from on 08/22/2003 19:47:07 with an action quarantined. From SAVMSE at afflink.com Fri Aug 22 16:17:10 2003 From: SAVMSE at afflink.com (SAVMSE@afflink.com) Date: Fri Aug 22 20:42:49 2003 Subject: [Python-Dev] Symantec AVF detected that you sent a message with a prohibited subject line Message-ID: <01aa01c368ea$5de2b380$2ffeffd0@afflink.local> Subject of the message: Thank you! Recipient of the message: Zhuozhou Liu Subject of the message: Thank you! Recipient of the message: Zhuozhou Liu From mlzhylcsx at china.com Fri Aug 22 17:09:36 2003 From: mlzhylcsx at china.com (Devon Stephenson) Date: Sat Aug 23 03:27:56 2003 Subject: [Python-Dev] You can order Anti-depressants, weightlosmeds, and pain relief medsonline with no prescriiption za g nnhcqisa Message-ID: <5--9s$j-$$w$q2j7j35j@for43pcaj> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20030822/77607f5c/attachment.htm From postmaster at s001.interlize.net Fri Aug 22 15:50:54 2003 From: postmaster at s001.interlize.net (MailScanner) Date: Sat Aug 23 04:56:23 2003 Subject: [Python-Dev] Warning: E-mail viruses detected Message-ID: <200308221250.h7MCosTX004408@s001.interlize.net> Our virus detector has just been triggered by a message you sent:- To: arjan@isgelukkig.nl Subject: Your details Date: Fri Aug 22 14:50:53 2003 One or more of the attachments (application.pif) are on the list of unacceptable attachments for this site and will not have been delivered. Consider renaming the files or putting them into a "zip" file to avoid this constraint. The virus detector said this about the message: Report: Shortcuts to MS-Dos programs are very dangerous in email (application.pif) No programs allowed (application.pif) -- Moerstaal MailScanner From MAILER-DAEMON at utc.fr Fri Aug 22 20:47:02 2003 From: MAILER-DAEMON at utc.fr (Mail Delivery System) Date: Sat Aug 23 06:20:20 2003 Subject: [Python-Dev] Undelivered Mail Returned to Sender Message-ID: <20030822174702.9B434984B@kappa.utc.fr> ------------------ Virus Warning Message ------- Found virus WORM_SOBIG.F.DAM in file document_all.pif The uncleanable file document_all.pif is moved to /var/iscan/virus/virzvbjnq. --------------------------------------------------------- -------------- next part -------------- This is the Postfix program at host kappa.utc.fr. I'm sorry to have to inform you that the message returned below could not be delivered to one or more destinations. For further assistance, please send mail to If you do so, please include this problem report. You can delete your own text from the message returned below. The Postfix program : unknown user: "sebastien.barre" -------------- next part -------------- Skipped content of type message/delivery-status-------------- next part -------------- An embedded message was scrubbed... From: Subject: **PROBABLEMENT*DU*SPAM** Re: Wicked screensaver Date: Fri, 22 Aug 2003 13:46:39 --0400 Size: 3540 Url: http://mail.python.org/pipermail/python-dev/attachments/20030822/7741b9bd/attachment.eml From NMXEXSPP1_MANTE61 at dow.com Fri Aug 22 19:56:17 2003 From: NMXEXSPP1_MANTE61 at dow.com (Nemx Power Tools for MS Exchange Server_MANTE61_1) Date: Sat Aug 23 06:26:06 2003 Subject: [Python-Dev] Virus Notification: A virus has been detected in a message origin ating from yourself Message-ID: From: python-dev@python.org [SMTP:python-dev@python.org] To: Date: Fri, Aug 22 2003, 12:45:49 PM Subject: Re: Details The message contained 1 virus(es): your_details.pif infected with the Sobig.F@mm virus - - - This message is generated by anti-virus safeguards established at The Dow Chemical Company. Do not reply to this message. You should contact your computer support personnel to take suitable measures to remove the virus from your documents or machine. From sympa at inria.fr Fri Aug 22 22:12:39 2003 From: sympa at inria.fr (SYMPA) Date: Sun Aug 24 05:14:58 2003 Subject: [Python-Dev] Moderating your message Message-ID: <200308221912.h7MJCdg19049@bellacoola.inria.fr> Your message for list scilab1 has been forwarded to editor(s) From tim.one at comcast.net Fri Aug 22 23:10:15 2003 From: tim.one at comcast.net (Tim Peters) Date: Sun Aug 24 12:02:42 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: Message-ID: [Thomas Heller] > I'm just, after downloading, building and installing a lot of stuff, > trying to make sure that I really can build the Windows installer for > 2.3.1. > > I'm practicing on the trunk - I don't think anyone wants to review the > changes to the windows installer, so I'll check them in shortly, it > seems that everything works so far. Cool! I hope you tried it on a non-privileged account under XP Algerian with omelets in the install path . > The only question remaining is how to build the html docs. > I don't think it works on Windows (I remember having built the pdf > docs with TeTeX, but that was it). > > Do I have to fire up linux in a vmware box to build them, or are there > other possibilities? I've never built them. As http://www.python.org/peps/pep-0101.html says, Fred builds release docs, and then Fred tells Tim Peters where the documentation file is. by which it means a .bz2 archive of the generated HTML, stored on python.org. I download it, unpack it into a child directory of PCbuild, then rename the directory to "html". That's where the installer expects to find it. If you want to practice, try the most recent docs: http://www.python.org/ftp/python/doc/2.3/html-2.3.tar.bz2 Another project we ran out of time for was building the Windows HTML docs into a .chm file; there's a quite capable script for doing that in Doc/tools/prechm.py but it hasn't been updated since the last time it worked. From guido at python.org Sat Aug 23 08:34:10 2003 From: guido at python.org (Guido van Rossum) Date: Sun Aug 24 12:58:25 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: Your message of "Wed, 20 Aug 2003 11:11:48 EDT." <16195.36788.437058.459054@magrathea.basistech.com> References: <16194.15227.214357.303033@magrathea.basistech.com> <16195.36788.437058.459054@magrathea.basistech.com> Message-ID: <200308231434.h7NEYAa20786@12-236-84-31.client.attbi.com> > As I look at Py_Initialize() further, I see some other 'features' that > could be problematic when embedding: particularly the calls to > Py_FatalError. An embedding application may be able to continue even > if the Python interpreter cannot be initialized... certainly it should > be up to the embedding application on how to handle the error, instead > of having abort() called for it. Unclear. You cannot completely avoid the possibility of Py_FatalError() being called in Python. The Py_FatalError() calls in Py_Initialize() are no different than the ones elsewhere in Python: they are only expected when you run out of memory in this stage. What might be useful would be a way for an embedding app to "hook" Py_FatalError() though, so that the embedding app can direct the error message to its own logging stream. > It would also be nice if there were no calls fprintf and friends > within the initialization path when doing embedded initialization. Right. Apart from a few inside debug #ifdefs, I see one or two that might realistically come up, and that should be fixed. In particular the line fprintf(stderr, "python: Can't reopen .pyc file\n"); should probably be replaced by a call to PySys_WriteStderr(). I'm not sure what to do about the line fprintf(stderr, "lost sys.stderr\n"); because if sys.stderr can't be found, there's no other place to send an error. Maybe in addition to a hookable Py_FatalError() we should also make PySys_WriteStderr() hookable. Of course it should always first try sys.stderr, but if that fails it should fall back to the hook rather than to stdio's stderr. --Guido van Rossum (home page: http://www.python.org/~guido/) From skip at mojam.com Sun Aug 24 08:00:39 2003 From: skip at mojam.com (Skip Montanaro) Date: Sun Aug 24 15:01:30 2003 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200308241200.h7OC0dT10033@manatee.mojam.com> Bug/Patch Summary ----------------- 469 open / 4048 total bugs (+20) 183 open / 2331 total patches (+10) New Bugs -------- a bug in IDLE on Python 2.3 i think (2003-08-17) http://python.org/sf/790162 curses.ascii.unctrl broken (2003-08-17) http://python.org/sf/790356 Section 11.20: xmlrpclib Details (2003-08-19) http://python.org/sf/791067 Tutorial: mutable objects may be keys (2003-08-19) http://python.org/sf/791397 Make logging consistent in the standard library (2003-08-19) http://python.org/sf/791410 Re: Bugs item #765456 (2003-08-19) http://python.org/sf/791445 test_threading (2003-08-19) http://python.org/sf/791542 Arguments tooltip wrong if def contains tuple (2003-08-20) http://python.org/sf/791968 urllib.urlopen for https doesn't always provide readlines (2003-08-20) http://python.org/sf/792101 csv.DictReader parms inconsistent with docs (2003-08-21) http://python.org/sf/792558 SimpleXMLRPCServer cannot handle large requests (2003-08-21) http://python.org/sf/792570 RESET_ERROR is not defined(logging module) (2003-08-21) http://python.org/sf/792649 Misleading [i:j:k] slicing description (2003-08-21) http://python.org/sf/792656 Easier-to-create alternative Python installer for Windows (2003-08-22) http://python.org/sf/793078 runnig thread multiple times (2003-08-23) http://python.org/sf/793687 Section 13.1 HTMLParser documentation error (2003-08-23) http://python.org/sf/793702 sgmllib parser problem (2003-08-23) http://python.org/sf/793753 pyconfig.h defines _POSIX_C_SOURCE, conflicting with feature (2003-08-23) http://python.org/sf/793764 gc.get_referrers() is inherently dangerous (2003-08-23) http://python.org/sf/793822 using itertools.izip to mutate tuples (2003-08-23) http://python.org/sf/793826 New Patches ----------- add SafeConfigParser to __all__ (2003-08-18) http://python.org/sf/790443 breakpoint command lists in pdb (2003-08-18) http://python.org/sf/790710 inconsistency with implementation(logging) (2003-08-19) http://python.org/sf/791153 POP3 over SSL support for poplib (2003-08-19) http://python.org/sf/791706 factor out SMTPHandler.emit date handling (2003-08-20) http://python.org/sf/791776 timetuple() returns a struct_time (2003-08-20) http://python.org/sf/792338 Tidying error messages in compile.c (2003-08-21) http://python.org/sf/792869 implement htmllib.HTMLParser.reset (2003-08-22) http://python.org/sf/793021 Add --remove-source option to setup.py (2003-08-22) http://python.org/sf/793070 urllib SSL authentication docs are wrong (2003-08-22) http://python.org/sf/793553 clear SGMLParser.__starttag_text on reset() (2003-08-22) http://python.org/sf/793559 Closed Bugs ----------- Byte-order bug in socket-module getaddrinfo.c (2003-08-06) http://python.org/sf/784031 "SetFrontProcess failed,-606" error for Tkinter under OS X (2003-08-16) http://python.org/sf/789926 Closed Patches -------------- redirect fails in urllib2 (2003-05-01) http://python.org/sf/731153 From csmail at support.prognet.com Sun Aug 24 01:22:31 2003 From: csmail at support.prognet.com (csmail@support.prognet.com) Date: Sun Aug 24 15:13:46 2003 Subject: [Python-Dev] We will not receive your email Message-ID: <200308240722.h7O7MVQH002863@support.real.com> Greetings, You have received this message as a result of replying via email to a RealNetworks Order Receipt or Customer Service message. We will -not- receive this email, and have no means of responding. If you received this mail as a result of replying to your Order Receipt from our e-commerce system at www.real.com or www.realstore.com, you will need to submit your service request at the following URL: http://service.real.com/faq/contcs.html If you received this message as a result of replying to a customer service response, there are instructions on how to reply to the customer service agent that responded to your inquiry. This reply method is handled with a web form, and you may find the URL in the Customer Service response message. Thanks again for seeking assistance, and we hope that your problem is resolved shortly. Best regards, Customer Service RealNetworks, Inc. From mzarate at uoguelph.ca Sun Aug 24 15:37:12 2003 From: mzarate at uoguelph.ca (Martin Zarate) Date: Sun Aug 24 18:20:33 2003 Subject: [Python-Dev] Embedded python module search path References: <16194.15227.214357.303033@magrathea.basistech.com> <16195.36788.437058.459054@magrathea.basistech.com> <200308231434.h7NEYAa20786@12-236-84-31.client.attbi.com> Message-ID: <005c01c36a6e$bc65f090$8001a8c0@PXTL> Hi, my name is Martin Zarate, and i'm working on a 3d game engine for educational and urban visualization purposes. Our engine handles scripting with an embedded Python interpreter (we designed our own customized class structure, threading system, etc). As of yet, we've never had to actually modify the Python interpreter itself, so I'm loathe to start. Our chief concern is this: our engine is designed with extensibility in mind - it detects plugins of new objects and new code entering the system. This code may or may not be trusted, and rexec is dead. That's a problem. I realize rexec will not be coming back. I don't need full rexec, I have a much simpler requirement - I don't want the python interpreter to have access to the system. The embedding app (Daedalus) handles feeding in of modules and content through Py_CompileString and PyImport_ExecCodeModule, as well as building local namespaces in which the code is run. Any access to the embedding system is through custom data types and extension modules. My point is that none of the system builtins or major modules are used - and those builtins and modules are what allow the user to access and corrupt the system. While much of the builtins are still needed (basic data types, etc) most of the built-in functions such as filesystem and systemcalls are liabilities. They could play with the file system, manipulate the system, and do other things. So, my question is this: is there any way to compile Python as a true standalone? That is, the only access to the system is through extension modules? I can't find any documentation on how to control what builtin modules and functions are compiled in with Python. Is there any interest in such a project? Or, if I develop this myself (although I have no idea how secure it could be - I don't know the builtins very well) would be any interest in makign a patch/PEP of it? This sort of thing would be a boon to anyone embedding python. I believe many embedded apps could use this sort of feature (at the very least to keep the bloat down). Sincerely, Martin From pxtl at rogers.com Sun Aug 24 15:52:11 2003 From: pxtl at rogers.com (Pxtl) Date: Sun Aug 24 18:21:02 2003 Subject: [Python-Dev] embedding python Message-ID: <00a401c36a70$d40059a0$8001a8c0@PXTL> (Sent this already once from another account - don't think it worked) Hi, my name is Martin Zarate, and i'm working on a 3d game engine for educational and urban visualization purposes. Our engine handles scripting with an embedded Python interpreter (we designed our own customized class structure, threading system, etc). As of yet, we've never had to actually modify the Python interpreter itself, so I'm loathe to start. Our chief concern is this: our engine is designed with extensibility in mind - it detects plugins of new objects and new code entering the system. This code may or may not be trusted, and rexec is dead. That's a problem. I realize rexec will not be coming back. I don't need full rexec, I have a much simpler requirement - I don't want the python interpreter to have access to the system. The embedding app (Daedalus) handles feeding in of modules and content through Py_CompileString and PyImport_ExecCodeModule, as well as building local namespaces in which the code is run. Any access to the embedding system is through custom data types and extension modules. My point is that none of the system builtins or major modules are used - and those builtins and modules are what allow the user to access and corrupt the system. While much of the builtins are still needed (basic data types, etc) most of the built-in functions such as filesystem and systemcalls are liabilities. They could play with the file system, manipulate the system, and do other things. So, my question is this: is there any way to compile Python as a true standalone? That is, the only access to the system is through extension modules? I can't find any documentation on how to control what builtin modules and functions are compiled in with Python. Is there any interest in such a project? Or, if I develop this myself (although I have no idea how secure it could be - I don't know the builtins very well) would be any interest in makign a patch/PEP of it? This sort of thing would be a boon to anyone embedding python. I believe many embedded apps could use this sort of feature (at the very least to keep the bloat down). Sincerely, Martin From frpythoneers-admin at lists.community.tummy.com Sun Aug 24 16:53:02 2003 From: frpythoneers-admin at lists.community.tummy.com (frpythoneers-admin@lists.community.tummy.com) Date: Sun Aug 24 18:53:35 2003 Subject: [Python-Dev] Your message to FRPythoneers awaits moderator approval Message-ID: <20030824215302.7899.42181.Mailman@community.tummy.com> Your mail to 'FRPythoneers' with the subject Re: Details Is being held until the list moderator can review it for approval. The reason it is being held: SpamAssassin thinks there may be spam in this message. Either the message will get posted to the list, or you will receive notification of the moderator's decision. From guido at python.org Sun Aug 24 22:27:40 2003 From: guido at python.org (Guido van Rossum) Date: Mon Aug 25 00:28:22 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: Your message of "Sun, 24 Aug 2003 14:37:12 EDT." <005c01c36a6e$bc65f090$8001a8c0@PXTL> References: <16194.15227.214357.303033@magrathea.basistech.com> <16195.36788.437058.459054@magrathea.basistech.com> <200308231434.h7NEYAa20786@12-236-84-31.client.attbi.com> <005c01c36a6e$bc65f090$8001a8c0@PXTL> Message-ID: <200308250427.h7P4Re602992@12-236-84-31.client.attbi.com> From guido at python.org Sun Aug 24 22:36:54 2003 From: guido at python.org (Guido van Rossum) Date: Mon Aug 25 00:37:28 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: Your message of "Sun, 24 Aug 2003 14:37:12 EDT." <005c01c36a6e$bc65f090$8001a8c0@PXTL> References: <16194.15227.214357.303033@magrathea.basistech.com> <16195.36788.437058.459054@magrathea.basistech.com> <200308231434.h7NEYAa20786@12-236-84-31.client.attbi.com> <005c01c36a6e$bc65f090$8001a8c0@PXTL> Message-ID: <200308250436.h7P4asN03013@12-236-84-31.client.attbi.com> > I realize rexec will not be coming back. I don't need full rexec, I have a > much simpler requirement - I don't want the python interpreter to have > access to the system. The embedding app (Daedalus) handles feeding in of > modules and content through Py_CompileString and PyImport_ExecCodeModule, > as well as building local namespaces in which the code is run. Any access > to the embedding system is through custom data types and extension modules. > > My point is that none of the system builtins or major modules are used - and > those builtins and modules are what allow the user to access and corrupt the > system. While much of the builtins are still needed (basic data types, etc) > most of the built-in functions such as filesystem and systemcalls are > liabilities. They could play with the file system, manipulate the system, > and do other things. So, my question is this: is there any way to compile > Python as a true standalone? That is, the only access to the system is > through extension modules? I can't find any documentation on how to control > what builtin modules and functions are compiled in with Python. > > Is there any interest in such a project? Or, if I develop this myself > (although I have no idea how secure it could be - I don't know the builtins > very well) would be any interest in makign a patch/PEP of it? This sort of > thing would be a boon to anyone embedding python. I believe many embedded > apps could use this sort of feature (at the very least to keep the bloat > down). Well, in standard Python, the only access to the system is *also* through extension modules -- if you count __builtin__ as an extension module. The other extension module you want to avoid is the posix module (under Windows, the nt module). It should be a simple matter to remove this from your module search path. If you are right that you don't need access to the few builtins that can do system calls at all (I think it's just open and file, but you may want to check), you can simply delete them from the __builtin__ module at the start. I would delete remove as well, since remove(__builtin__) brings deleted builtins back to life. And you'd have to provide an __import__ replacement that restricts what you can import; but again you can do that at the start, before running any untrusted code. Is this clear, or do you need more explanation? (PS: sorry for the empty email I sent you before. My fingers slipped.) --Guido van Rossum (home page: http://www.python.org/~guido/) From mailer-daemon at mailgate1.aa.com Mon Aug 25 01:41:09 2003 From: mailer-daemon at mailgate1.aa.com (mailer-daemon@mailgate1.aa.com) Date: Mon Aug 25 01:39:01 2003 Subject: [Python-Dev] Virus Alert Message-ID: <200308250541.h7P5f9QS025965@mailhub1.aa.com> The mail message (file: thank_you.pif) you sent to webmaster@aa.com contains a virus. (on the network) From Jack.Jansen at cwi.nl Mon Aug 25 12:50:45 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Mon Aug 25 05:51:31 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <200308250436.h7P4asN03013@12-236-84-31.client.attbi.com> Message-ID: <98F4B03C-D6E1-11D7-A09E-0030655234CE@cwi.nl> On Monday, August 25, 2003, at 06:36 AM, Guido van Rossum wrote: > Well, in standard Python, the only access to the system is *also* > through extension modules -- if you count __builtin__ as an extension > module. The other extension module you want to avoid is the posix > module (under Windows, the nt module). It should be a simple matter > to remove this from your module search path. No, it isn't: simply doing "open = type(sys.stdout)" will revive open for you. So you'd really have to make sure no file objects are accessible either. And there's lots more loopholes like this. With the current type system I think the only real solution would be to block this at a very low level, i.e. removing file objects from your build, or at least completely disabling their side-effects. -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From veer45q2 at verizon.com Mon Aug 25 14:00:53 2003 From: veer45q2 at verizon.com (Arimeth) Date: Mon Aug 25 09:01:25 2003 Subject: [Python-Dev] bwarsaw,"In my 3 years of doing the business, Message-ID: <200308250815443.SM01476@smtp0492.mail.yahoo.com> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20030825/4cec3a8f/attachment-0001.htm From shane at zope.com Mon Aug 25 13:22:00 2003 From: shane at zope.com (Shane Hathaway) Date: Mon Aug 25 12:24:16 2003 Subject: [Python-Dev] Embedded python module search path In-Reply-To: <98F4B03C-D6E1-11D7-A09E-0030655234CE@cwi.nl> References: <200308250436.h7P4asN03013@12-236-84-31.client.attbi.com> <98F4B03C-D6E1-11D7-A09E-0030655234CE@cwi.nl> Message-ID: <3F4A37A8.7090603@zope.com> Jack Jansen wrote: > On Monday, August 25, 2003, at 06:36 AM, Guido van Rossum wrote: > >> Well, in standard Python, the only access to the system is *also* >> through extension modules -- if you count __builtin__ as an extension >> module. The other extension module you want to avoid is the posix >> module (under Windows, the nt module). It should be a simple matter >> to remove this from your module search path. > > > No, it isn't: simply doing "open = type(sys.stdout)" will revive open > for you. So you'd really have to make sure no file objects are accessible > either. And there's lots more loopholes like this. > > With the current type system I think the only real solution would be > to block this at a very low level, i.e. removing file objects from your > build, or at least completely disabling their side-effects. FWIW, Zope takes an approach to restricted Python code that's worth considering. We once thought rexec and Bastion would eventually supercede Zope's "RestrictedPython" package, so not a lot of effort went into non-Zope-specific documentation. However, RestrictedPython has outlived both rexec and Bastion, so maybe detailed documentation would now be valuable. Here is a general overview of the approach RestrictedPython takes: - All builtins and modules are guilty until proven innocent. Restricted modules have a special __builtins__ and an __import__ hook. - We use a modified compiler, based on the now-standard compiler module, to prevent exec statements and hook print statements. The compiler also adds hooks for getattr, setattr, delattr, getitem, setitem, and delitem operations. Augmented assignment is disallowed (too complicated to support.) - The type() builtin is considered unsafe. It opens a big unknown. However, a same_type() builtin is provided, which is close enough for most purposes. There are safe equivalents for other builtins as well. - Here's the hard one for some people to swallow: the compiler prevents restricted scripts from using names that start with an underscore. Being able to define a name like "__import__" could get around the hooks. This might be considered draconian, but no one has spotted any holes yet in the safety net, and the benefit of being able to script in Python outweighs the losses. It doesn't implement resource limitations, like preventing scripts from eating up all available RAM or simply never terminating. True resource limitations would require running scripts in a separate process. RestrictedPython is also a boring name. However, RestrictedPython is safer than anything else we know of in the Python world. Shane From theller at python.net Mon Aug 25 19:30:15 2003 From: theller at python.net (Thomas Heller) Date: Mon Aug 25 12:30:49 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: (Tim Peters's message of "Fri, 22 Aug 2003 22:10:15 -0400") References: Message-ID: "Tim Peters" writes: > Another project we ran out of time for was building the Windows HTML docs > into a .chm file; there's a quite capable script for doing that in > > Doc/tools/prechm.py > > but it hasn't been updated since the last time it worked. This script works with the current docs as well. But including it in the windows installer instead of the 1250 html pages will raise the size by ~200 kB (probably the full text index), but worse, pydoc expects the normal html pages. hh.exe can decompile the chm file, but is it a good idea to do this during (or after) installation? Thomas From postmaster at dvag.com Mon Aug 25 19:58:26 2003 From: postmaster at dvag.com (postmaster@dvag.com) Date: Mon Aug 25 12:59:02 2003 Subject: [Python-Dev] Virus Alert Message-ID: <200308251658.h7PGwQ314304@PTVS01.dvag.com> Die Mail Message (Datei: document_9446.pif) welche Sie an stefan.palaschevsky@dvag.com versendet haben, enhielt ein Virus .(in the network) From tim.one at comcast.net Mon Aug 25 18:14:38 2003 From: tim.one at comcast.net (Tim Peters) Date: Mon Aug 25 17:15:18 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: Message-ID: [Tim] >> Another project we ran out of time for was building the Windows HTML >> docs into a .chm file; there's a quite capable script for doing that >> in >> >> Doc/tools/prechm.py >> >> but it hasn't been updated since the last time it worked. [Thomas Heller] > This script works with the current docs as well. But including it in > the windows installer instead of the 1250 html pages will raise the > size by ~200 kB (probably the full text index), The size of what? The installer .exe? I'm not sure anyone would notice, given how large it is already. We certainly lose more than 200KB after installation to the file-padding and directory-entry burdens of spreading the docs over 1000+ separate files. > but worse, pydoc expects the normal html pages. A fully indexed .chm file is much better for searching than pydoc. Sounds like a case of lose-a-little, gain-a-lot. > hh.exe can decompile the chm file, but is it a good idea to do this > during (or after) installation? If there's a nice searchable .chm file, I wouldn't want a thousand separate HTML files in addition. From pedronis at bluewin.ch Tue Aug 26 03:38:20 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Mon Aug 25 20:37:22 2003 Subject: [Python-Dev] Fwd: PEP 310(with-syntax): close synonym of __exit__ Message-ID: <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> In PEP 310 Reliable Acquisition/Release Pairs: http://www.python.org/peps/pep-0310.html """ Open Issues Should existing classes (for example, file-like objects and locks) gain appropriate __enter__ and __exit__ methods? The obvious reason in favour is convenience (no adapter needed). The argument against is that if built-in files have this but (say) StringIO does not, then code that uses "with" on a file object can't be reused with a StringIO object. So __exit__ = close becomes a part of the "file-like object" protocol, which user-defined classes may need to support. """ maybe this is too much DWIMish, but it occurred to me that a further option would be for the semantics to be: var = expr if hasattr(var, "__enter__"): var.__enter__() try: suite finally: if hasattr(var, "__exit__"): var.__exit__() elif hasattr(var, "close"): # consider also close as a synonym of __exit__ var.close() I'm not proposing to do without __exit__, I recall the next/__next__ debate, but I'm wondering if this is well documented, how many times considering close a synomym of __exit__ would do the wrong thing, such that the user would have to hide close somehow to use 'with' & some object, and this wrt the times someone would need a wrapper to have __exit__ trigger a close. I don't think it is worth to have a debate right now, still this should maybe be added to the PEP as an option to consider. regards. From theller at python.net Tue Aug 26 12:54:34 2003 From: theller at python.net (Thomas Heller) Date: Tue Aug 26 05:55:09 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: (Tim Peters's message of "Mon, 25 Aug 2003 17:14:38 -0400") References: Message-ID: "Tim Peters" writes: > [Tim] >>> Another project we ran out of time for was building the Windows HTML >>> docs into a .chm file; there's a quite capable script for doing that >>> in >>> >>> Doc/tools/prechm.py >>> >>> but it hasn't been updated since the last time it worked. > > [Thomas Heller] >> This script works with the current docs as well. But including it in >> the windows installer instead of the 1250 html pages will raise the >> size by ~200 kB (probably the full text index), > > The size of what? The installer .exe? I'm not sure anyone would notice, > given how large it is already. Sure. I just wanted to mention that switching to .chm doesn't decrease the download size for the installer exe. > > A fully indexed .chm file is much better for searching than pydoc. Sounds > like a case of lose-a-little, gain-a-lot. > Ok, should this already go into 2.3.1? Thomas From mwh at python.net Tue Aug 26 13:42:49 2003 From: mwh at python.net (Michael Hudson) Date: Tue Aug 26 07:42:52 2003 Subject: [Python-Dev] Fwd: PEP 310(with-syntax): close synonym of __exit__ In-Reply-To: <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> (Samuele Pedroni's message of "Tue, 26 Aug 2003 02:38:20 +0200") References: <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> Message-ID: <2m7k50k5l2.fsf@starship.python.net> Samuele Pedroni writes: [...] > maybe this is too much DWIMish, but it occurred to me that a further > option would be for the semantics to be: > > var = expr > > if hasattr(var, "__enter__"): > var.__enter__() > > try: > suite > finally: > if hasattr(var, "__exit__"): > var.__exit__() > elif hasattr(var, "close"): # consider also close as a synonym of __exit__ > var.close() > > I'm not proposing to do without __exit__, I recall the next/__next__ > debate, but I'm wondering if this is well documented, how many times > considering close a synomym of __exit__ would do the wrong thing, such > that the user would have to hide close somehow to use 'with' & some > object, and this wrt the times someone would need a wrapper to have > __exit__ trigger a close. > > I don't think it is worth to have a debate right now, still this > should maybe be added to the PEP as an option to consider. FWIW, I think this has come up before. For *me*, I think having less magic is worth the pain of adding "__exit__ = close" to a few class statements. Do you want to add "release" as a synonym too? I don't. Cheers, mwh -- This makes it possible to pass complex object hierarchies to a C coder who thinks computer science has made no worthwhile advancements since the invention of the pointer. -- Gordon McMillan, 30 Jul 1998 From mcherm at mcherm.com Tue Aug 26 05:50:23 2003 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 26 07:50:25 2003 Subject: [Python-Dev] Py2.3.1 Message-ID: <1061898623.3f4b497fc31b6@mcherm.com> Tim Peters writes: > If there's a nice searchable .chm file, I wouldn't want a thousand separate > HTML files in addition. I'm a Windows user, and I tend to prefer using html docs. Am I unusual in this regard? -- Michael Chermside From mhammond at skippinet.com.au Tue Aug 26 22:56:09 2003 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue Aug 26 07:56:10 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <1061898623.3f4b497fc31b6@mcherm.com> Message-ID: <044e01c36bc9$0a581890$f502a8c0@eden> > I'm a Windows user, and I tend to prefer using html docs. Am > I unusual in this > regard? The issue is more the general functionality of the documentation; specifically the index and full text search. These are the killer features in this case. A bigger issue may be that we can not assume the viewer is installed in every PC. Mark. From theller at python.net Tue Aug 26 15:09:41 2003 From: theller at python.net (Thomas Heller) Date: Tue Aug 26 08:10:16 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <044e01c36bc9$0a581890$f502a8c0@eden> (Mark Hammond's message of "Tue, 26 Aug 2003 21:56:09 +1000") References: <044e01c36bc9$0a581890$f502a8c0@eden> Message-ID: "Mark Hammond" writes: > The issue is more the general functionality of the documentation; > specifically the index and full text search. These are the killer features > in this case. > > A bigger issue may be that we can not assume the viewer is installed in > every PC. Since the win32all docs are also in htmlhelp format, do you care about this issue in the installer? And a quick search didn't find anything about what windows versions do not include the htmlhelp feature. A barenbones Win95, without IE, probably? Thomas From theller at python.net Tue Aug 26 15:15:54 2003 From: theller at python.net (Thomas Heller) Date: Tue Aug 26 08:16:28 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <1061898623.3f4b497fc31b6@mcherm.com> (Michael Chermside's message of "Tue, 26 Aug 2003 04:50:23 -0700") References: <1061898623.3f4b497fc31b6@mcherm.com> Message-ID: Michael Chermside writes: > Tim Peters writes: >> If there's a nice searchable .chm file, I wouldn't want a thousand separate >> HTML files in addition. > > I'm a Windows user, and I tend to prefer using html docs. Am I unusual in this > regard? Yes, you are. IMO. Maybe. I have setup things so that I can press F2 in Xemacs to search context sensitive in Python's chm file, very nice. OTOH, 'hh -decompile ' will decompile the chm file into separate html files again. But there's also the html archive available for download. Thomas From pedronis at bluewin.ch Tue Aug 26 15:52:41 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue Aug 26 08:51:26 2003 Subject: [Python-Dev] Fwd: PEP 310(with-syntax): close synonym of __exit__ In-Reply-To: <2m7k50k5l2.fsf@starship.python.net> References: <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030826142905.026d67e8@pop.bluewin.ch> At 12:42 26.08.2003 +0100, Michael Hudson wrote: >Samuele Pedroni writes: > >[...] > > maybe this is too much DWIMish, but it occurred to me that a further > > option would be for the semantics to be: > > > > var = expr > > > > if hasattr(var, "__enter__"): > > var.__enter__() > > > > try: > > suite > > finally: > > if hasattr(var, "__exit__"): > > var.__exit__() > > elif hasattr(var, "close"): # consider also close as a synonym of > __exit__ > > var.close() > > > > I'm not proposing to do without __exit__, I recall the next/__next__ > > debate, but I'm wondering if this is well documented, how many times > > considering close a synomym of __exit__ would do the wrong thing, such > > that the user would have to hide close somehow to use 'with' & some > > object, and this wrt the times someone would need a wrapper to have > > __exit__ trigger a close. > > > > I don't think it is worth to have a debate right now, still this > > should maybe be added to the PEP as an option to consider. > >FWIW, I think this has come up before. ok, then the PEP should mention it, or it will come out again. > For *me*, I think having less >magic is worth if it's documented, I'm not sure its magic, both the pair __enter__,__exit__ and close alone seem reasonable protocols for 'with', given the common usage pattern of close. >the pain of adding "__exit__ = close" to a few class >statements. all file-like objects, HTTP connections, DBI objects (cursors and connections have close). close is naturally a __exit__ They are not all in the std lib and under our control btw. > Do you want to add "release" as a synonym too? I don't. No. See above. Anyway the PEP should mention this, that's what PEPs are for. Maybe I'm just catering to the lazy, and considering the out-of-the-box experience of the new feature . regards. From pedronis at bluewin.ch Tue Aug 26 16:36:46 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue Aug 26 09:35:27 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators Message-ID: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> this is something we discussed with Guido, and also Moshe Zadka at Europython. Guido thought it seems reasonable enough, if the details can be nailed. I have written it down so the idea doesn't get lost, for the moment is more a matter of whether it can get a number, and then it can go dormant for a while. - * - PEP: XXX Title: Resource-Release Support for Generators Version: $Revision$ Last-Modified: $Date$ Author: Samuele Pedroni Status: Draft Type: Standards Track Content-Type: text/plain Created: 25-Aug-2003 Python-Version: 2.4 Post-History: Abstract Generators allow for natural coding and abstraction of traversal over data. Currently if external resources needing proper timely release are involved, generators are unfortunately not adequate. The typical idiom for timely release is not supported, a yield statement is not allowed in the try clause of a try-finally statement inside a generator. The finally clause execution cannot be either guaranteed or enforced. This PEP proposes that generators support a close method and destruction semantics, such that the restriction can be lifted, expanding the applicability of generators. Rationale Python generators allow for natural coding of many data traversal scenarios Their instantiation produces iterators, i.e. first-class objects abstracting traversal (with all the advantages of first- classness). In this respect they match in power and offer some advantages over the approach using iterator methods taking a (smalltalkish) block. On the other hand, given current limitations (no yield allowed in a try clause of a try-finally inside a generator) the latter approach seems better suited at encapsulating not only traversal but also exception handling and proper resource acquisition and release. Let's consider an example (for simplicity, files in read-mode are used): def all_lines(index_path): for path in file(index_path,"r"): for line in file(path.strip(),"r"): yield line this is short and to the point, but the try-finally for timely closing of the files cannot be added. (While instead of a path, a file, whose closing then would be responsibility of the caller, could be passed in as argument, the same is not applicable for the files opened depending on the contents of the index). If we want timely release, we have to sacrifice the simplicity and directness of the generator-only approach: (e.g.) class AllLines: def __init__(self,index_path): self.index_path = index_path self.index = None self.document = None def __iter__(self): self.index = file(self.index_path,"r") for path in self.index: self.document = file(path.strip(),"r") for line in self.document: yield line self.document.close() self.document = None def close(self): if self.index: self.index.close() if self.document: self.document.close() to be used as: all_lines = AllLines("index.txt") try: for line in all_lines: ... finally: all_lines.close() The more convoluted solution implementing timely release, seems to offer a precious hint. What we have done is encapsulating our traversal in an object (iterator) with a close method. This PEP proposes that generators should grow such a close method with such semantics that the example could be rewritten as: def all_lines(index_path): index = file(index_path,"r") try: for path in file(index_path,"r"): document = file(path.strip(),"r") try: for line in document: yield line finally: document.close() finally: index.close() all = all_lines("index.txt") try: for line in all: ... finally: all.close() PEP 255 [1] disallows yield inside a try clause of a try-finally statement, because the execution of the finally clause cannot be guaranteed as required by try-finally semantics. The semantics of the proposed close method should be such, that while the finally clause execution still cannot be guaranteed, it can be enforced when required. The semantics of generator destruction on the other hand should be extended in order to implement a best-effort policy for the general case. This strikes as a reasonable compromise, the resulting global behavior being similar to that of files and closing. Possible Semantics A close() method should be implemented for generator objects. 1) If a generator is already terminated, close should be a no-op. Otherwise: (two alternative solutions) (Return Semantics) The generator should be resumed, generator execution should continue as if the instruction at re-entry point is a return. Consequently finally clauses surrounding the re-entry point would be executed, in the case of a then allowed try-yield-finally pattern. Issues: is it important to be able to distinguish forced termination by close, normal termination, exception propagation from generator or generator-called code? In the normal case it seems not, finally clauses should be there to work the same in all these cases, still this semantics could make such a distinction hard. Except-clauses, like by a normal return, are not executed, such clauses in legacy generators expect to be executed for exceptions raised by the generator or by code called from it. Not executing them in the close case seems correct. OR (Exception Semantics) The generator should be resumed and execution should continue as if a special-purpose exception (e.g. CloseGenerator) has been raised at re-entry point. Close implementation should consume and not propagate further this exception. Issues: should StopIteration be reused for this purpose? Probably not. We would like close to be a harmless operation for legacy generators, which could contain code catching StopIteration to deal with other generators/iterators. In general, with exception semantics, it is unclear what to do if the generator does not terminate or we do not receive the special exception propagated back. Other different exceptions should probably be propagated, but consider this possible legacy generator code: try: ... yield ... ... except: # or except Exception:, etc raise Exception("boom") If close is invoked with the generator suspended after the yield, the except clause would catch our special purpose exception, so we would get a different exception propagated back, which in this case ought to be reasonably consumed and ignored but in general should be propagated, but separating these scenarios seem hard. The exception approach has the advantage to let the generator distinguish between termination cases and have more control. On the other hand clear-cut semantics seem harder to define. 2) Generator destruction should invoke close method behavior. Remarks If this proposal is accepted, it should become common practice to document whether a generator acquires resources, so that its close method ought to be called. If a generator is no longer used, calling close should be harmless. On the other hand, in the typical scenario the code that instantiated the generator should call close if required by it, generic code dealing with iterators/generators instantiated elsewhere should typically not be littered with close calls. The rare case of code that has acquired ownership of and need to properly deal with all of iterators, generators and generators acquiring resources that need timely release, is easily solved: if hasattr(iterator,'close'): iterator.close() Open Issues Definitive semantics ought to be chosen, implementation issues should be explored. Alternative Ideas The idea that the yield placement limitation should be removed and that generator destruction should trigger execution of finally clauses has been proposed more than once. Alone it cannot guarantee that timely release of resources acquired by a generator can be enforced. PEP 288 [2] proposes a more general solution, allowing custom exception passing to generators. References [1] PEP 255 Simple Generators http://www.python.org/peps/pep-0255.html [2] PEP 288 Generators Attributes and Exceptions http://www.python.org/peps/pep-0288.html Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 End: From aahz at pythoncraft.com Tue Aug 26 10:35:45 2003 From: aahz at pythoncraft.com (Aahz) Date: Tue Aug 26 09:35:51 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <1061898623.3f4b497fc31b6@mcherm.com> References: <1061898623.3f4b497fc31b6@mcherm.com> Message-ID: <20030826133545.GA23529@panix.com> On Tue, Aug 26, 2003, Michael Chermside wrote: > Tim Peters writes: >> >> If there's a nice searchable .chm file, I wouldn't want a thousand separate >> HTML files in addition. > > I'm a Windows user, and I tend to prefer using html docs. Am I unusual > in this regard? You may be unusual, but you do have company. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From mhammond at skippinet.com.au Wed Aug 27 00:33:07 2003 From: mhammond at skippinet.com.au (Mark Hammond) Date: Tue Aug 26 09:36:40 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: Message-ID: <01e101c36bd7$0508f680$f502a8c0@eden> > Since the win32all docs are also in htmlhelp format, do you care about > this issue in the installer? Nope - and no one has complained either :) > And a quick search didn't find anything about what windows versions do > not include the htmlhelp feature. A barenbones Win95, without IE, > probably? Yeah, I expect you are correct, and we already have dropped support for Windows 95. But for the sake of completeness I thought it worth mentioning :) I think it would be a great move. Mark. From nas-python at python.ca Tue Aug 26 07:49:01 2003 From: nas-python at python.ca (Neil Schemenauer) Date: Tue Aug 26 09:42:34 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators In-Reply-To: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> Message-ID: <20030826134901.GA30762@glacier.arctrix.com> Samuele Pedroni wrote: > This PEP proposes that generators should grow such a close method > with such semantics that the example could be rewritten as: > > def all_lines(index_path): > index = file(index_path,"r") > try: > for path in file(index_path,"r"): ^^^^^^^^^^^^^^^^^^^^ > document = file(path.strip(),"r") > try: > for line in document: > yield line > finally: > document.close() > finally: > index.close() I think that should be 'index'. Neil From pedronis at bluewin.ch Tue Aug 26 16:58:15 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue Aug 26 09:57:00 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators In-Reply-To: <20030826134901.GA30762@glacier.arctrix.com> References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030826155723.02636d08@pop.bluewin.ch> At 06:49 26.08.2003 -0700, Neil Schemenauer wrote: >Samuele Pedroni wrote: > > This PEP proposes that generators should grow such a close method > > with such semantics that the example could be rewritten as: > > > > def all_lines(index_path): > > index = file(index_path,"r") > > try: > > for path in file(index_path,"r"): > ^^^^^^^^^^^^^^^^^^^^ > > document = file(path.strip(),"r") > > try: > > for line in document: > > yield line > > finally: > > document.close() > > finally: > > index.close() > >I think that should be 'index'. yup, thanks. From guido at python.org Tue Aug 26 08:02:26 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 26 10:02:47 2003 Subject: [Python-Dev] Fwd: PEP 310(with-syntax): close synonym of __exit__ In-Reply-To: Your message of "Tue, 26 Aug 2003 12:42:49 BST." <2m7k50k5l2.fsf@starship.python.net> References: <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> <2m7k50k5l2.fsf@starship.python.net> Message-ID: <200308261402.h7QE2Rn04188@12-236-84-31.client.attbi.com> > FWIW, I think this has come up before. For *me*, I think having less > magic is worth the pain of adding "__exit__ = close" to a few class > statements. Do you want to add "release" as a synonym too? I don't. Agreed. "Only one way." --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 26 08:08:18 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 26 10:08:36 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: Your message of "Tue, 26 Aug 2003 09:35:45 EDT." <20030826133545.GA23529@panix.com> References: <1061898623.3f4b497fc31b6@mcherm.com> <20030826133545.GA23529@panix.com> Message-ID: <200308261408.h7QE8IL04218@12-236-84-31.client.attbi.com> > >> If there's a nice searchable .chm file, I wouldn't want a > >> thousand separate HTML files in addition. > > > > I'm a Windows user, and I tend to prefer using html docs. Am I unusual > > in this regard? > > You may be unusual, but you do have company. I find it hard to judge this, since I don't have a lot of experience with the .chm format. I think we can do an experiment: put the .chm files in the 2.3.1 release, and if everyone hates it (or if the BDFL hates it :-) we can roll it back in 2.3.2. (If there were an alpha/beta cycle for 2.3.1 tht would be the place to experiment, but as it seems there won't be, I'm fine with 2.3.1 itself being the beta cycle.) --Guido van Rossum (home page: http://www.python.org/~guido/) From niemeyer at conectiva.com Tue Aug 26 14:54:35 2003 From: niemeyer at conectiva.com (Gustavo Niemeyer) Date: Tue Aug 26 12:55:13 2003 Subject: [Python-Dev] Fwd: PEP 310(with-syntax): close synonym of __exit__ In-Reply-To: <5.2.1.1.0.20030826142905.026d67e8@pop.bluewin.ch> References: <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> <5.2.1.1.0.20030826142905.026d67e8@pop.bluewin.ch> Message-ID: <20030826165435.GA12790@ibook.distro.conectiva> [...] > if it's documented, I'm not sure its magic, Have you ever read perlvar documentation? :-) -- Gustavo Niemeyer http://niemeyer.net From tim.one at comcast.net Tue Aug 26 14:32:00 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 26 13:34:17 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: Message-ID: [Thomas Heller] > Sure. I just wanted to mention that switching to .chm doesn't > decrease the download size for the installer exe. So it's pretty mucn neutral by that measure. After installation, it saves a lot of space (the 1000+ HTML files expand to > 12MB now). >> A fully indexed .chm file is much better for searching than pydoc. >> Sounds like a case of lose-a-little, gain-a-lot. > Ok, should this already go into 2.3.1? I think so. Note that IDLE may need a bit of fiddling, to keep its Help -> Python Docs menu entry working. From tim.one at comcast.net Tue Aug 26 14:37:29 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 26 13:40:19 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <1061898623.3f4b497fc31b6@mcherm.com> Message-ID: [Michael Chermside] > I'm a Windows user, and I tend to prefer using html docs. Am I > unusual in this regard? I don't know. Have you used a .chm file for Python docs (one has been separately downloadable for years, but not from python.org)? The display is identical to what you'd see if you used IE to browse the HTML docs directly today, so it's unclear what it is about "html docs" you might prefer. The prime usability gain is that the .chm file supports full-text search across the whole set of Python docs, including Boolean (AND, OR, NOT) and proximity (NEAR) searches. From tim.one at comcast.net Tue Aug 26 14:42:01 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 26 13:45:31 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <20030826133545.GA23529@panix.com> Message-ID: [Michael Chermside] >> I'm a Windows user, and I tend to prefer using html docs. Am I >> unusual in this regard? [Aahz] > You may be unusual, but you do have company. I don't know you used Windows, Aahz. Then congratulations on upgrading to a real OS . From fdrake at acm.org Tue Aug 26 14:43:45 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue Aug 26 13:53:50 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: References: Message-ID: <16203.40017.707481.805254@grendel.zope.com> Tim Peters writes: > I think so. Note that IDLE may need a bit of fiddling, to keep its Help -> > Python Docs menu entry working. IDLE currently looks for the index.html file in a few places (which depend on platform); if it can't find it, it uses the documentation on python.org. It should be too hard to change it to load the HTML Help viewer if it finds the .chm file on Windows, and to still fall back to the HTML or the online documentation if the .chm can't be found. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From mcherm at mcherm.com Tue Aug 26 11:59:39 2003 From: mcherm at mcherm.com (Michael Chermside) Date: Tue Aug 26 14:01:13 2003 Subject: [Python-Dev] Py2.3.1 Message-ID: <1061920779.3f4ba00b589f4@mcherm.com> [Michael Chermside] > I'm a Windows user, and I tend to prefer using html docs. Am I > unusual in this regard? [Tim Peters] > I don't know. Have you used a .chm file for Python docs (one has been > separately downloadable for years, but not from python.org)? Nope. [Tim continues] > The display is > identical to what you'd see if you used IE to browse the HTML docs directly > today, so it's unclear what it is about "html docs" you might prefer. Mostly just that it's "more standard". I can view in a browser of my choice, launch a browser from within my editor/IDE, set bookmarks (but I don't usually do that), share on a drive and view from unix (but I don't do that), etc. [Tim again] > The > prime usability gain is that the .chm file supports full-text search across > the whole set of Python docs, including Boolean (AND, OR, NOT) and proximity > (NEAR) searches. Well, that's a significant feature! I don't want to knock a technology that I haven't even really tried... just wanted to point out that there are some advantages to HTML. I guess I think Guido's "beta test" idea sounds good -- I can't think of any other way we'd actually get feedback from users. Maybe a mention in "What's New In" under the title "where are the html docs?" could give the gripers a place to complain to so they can be counted. Nobody-uses-html-any-more-that-was-so-1990's lly yours, -- Michael Chermside From fdrake at acm.org Tue Aug 26 15:03:22 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Tue Aug 26 14:04:54 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <16203.40017.707481.805254@grendel.zope.com> References: <16203.40017.707481.805254@grendel.zope.com> Message-ID: <16203.41194.367779.676289@grendel.zope.com> Fred L. Drake, Jr. writes: > It should be too hard to change it to load the HTML Help viewer if it > finds the .chm file on Windows, and to still fall back to the HTML or > the online documentation if the .chm can't be found. Tim suggested that I meant that it *shouldn't* be too hard to change it. He was wrong; I said what I meant. ;-) It *should* be hard; if changing Python code were hard, Python programmers would have a better shot at job security. Since it isn't, changing IDLE won't be too hard, and no one will ever be able to keep a job programming in Python. Now, if that isn't what we love about Python, I don't know what is! ;-) -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From pedronis at bluewin.ch Tue Aug 26 21:38:59 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue Aug 26 14:37:36 2003 Subject: [Python-Dev] Fwd: PEP 310(with-syntax): close synonym of __exit__ In-Reply-To: <200308261402.h7QE2Rn04188@12-236-84-31.client.attbi.com> References: <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> <2m7k50k5l2.fsf@starship.python.net> Message-ID: <5.2.1.1.0.20030826203300.02694d80@pop.bluewin.ch> At 07:02 26.08.2003 -0700, Guido van Rossum wrote: > > FWIW, I think this has come up before. For *me*, I think having less > > magic is worth the pain of adding "__exit__ = close" to a few class > > statements. Do you want to add "release" as a synonym too? I don't. > >Agreed. "Only one way." your call :). Btw, if __exit__ is the only spelling then: var = expr if hasattr(var, "__enter__"): var.__enter__() try: suite finally: var.__exit__() is maybe a better semantics. If someone uses some of his legacy file-like objects etc, but forgets to define at least __exit__, he gets a fail-fast behavior, otherwise he may have to track down data corruption or something like that... regards. From guido at python.org Tue Aug 26 14:06:54 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 26 16:07:07 2003 Subject: [Python-Dev] Fwd: PEP 310(with-syntax): close synonym of __exit__ In-Reply-To: Your message of "Tue, 26 Aug 2003 20:38:59 +0200." <5.2.1.1.0.20030826203300.02694d80@pop.bluewin.ch> References: <5.2.1.1.0.20030826023524.0268c1d8@pop.bluewin.ch> <2m7k50k5l2.fsf@starship.python.net> <5.2.1.1.0.20030826203300.02694d80@pop.bluewin.ch> Message-ID: <200308262006.h7QK6sp04730@12-236-84-31.client.attbi.com> > Btw, if __exit__ is the only spelling then: > > var = expr > > if hasattr(var, "__enter__"): > var.__enter__() > > try: > suite > > finally: > var.__exit__() > > is maybe a better semantics. If someone uses some of his legacy > file-like objects etc, but forgets to define at least __exit__, he > gets a fail-fast behavior, otherwise he may have to track down data > corruption or something like that... Good idea! --Guido van Rossum (home page: http://www.python.org/~guido/) From python at rcn.com Tue Aug 26 17:17:41 2003 From: python at rcn.com (Raymond Hettinger) Date: Tue Aug 26 16:24:37 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> Message-ID: <00a401c36c0f$76952160$c304a044@oemcomputer> > This PEP proposes that generators should grow such a close method > with such semantics that the example could be rewritten as: > > def all_lines(index_path): > index = file(index_path,"r") > try: > for path in file(index_path,"r"): > document = file(path.strip(),"r") > try: > for line in document: > yield line > finally: > document.close() > finally: > index.close() > > > all = all_lines("index.txt") > try: > for line in all: > ... > finally: > all.close() [snip] > PEP 288 [2] proposes a more general solution, allowing custom > exception passing to generators. Is there any reason to prefer gen.close() over the more general solution gen.throw(Close) which results in nearly identical code and yet allows other exception types to be handled as well? Note, the general purpose solution is a natural extension of the existing syntax and is easily implemented without messing with 'try/finally'. Pretty much all that was holding up the general solution was that I had not convinced Guido that the clean-up problem exists in practice. It looks like you've surmounted that obstacle for me. Raymond Hettinger From aahz at pythoncraft.com Tue Aug 26 17:32:10 2003 From: aahz at pythoncraft.com (Aahz) Date: Tue Aug 26 16:32:15 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: References: <20030826133545.GA23529@panix.com> Message-ID: <20030826203210.GB15377@panix.com> On Tue, Aug 26, 2003, Tim Peters wrote: > > I don't know you used Windows, Aahz. Then congratulations on > upgrading to a real OS . Oh, I avoid Windows as much as possible -- and when I *do* use Windows, I try to make it as much like a Unix machine as possible. <0.2 wink> -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From Jack.Jansen at cwi.nl Tue Aug 26 23:49:43 2003 From: Jack.Jansen at cwi.nl (Jack Jansen) Date: Tue Aug 26 16:49:48 2003 Subject: [Python-Dev] Installer for OSX 10.3, where to put it? Message-ID: Folks, I'm starting work on the MacPython 2.3 installer for MacOSX 10.3, which will install only the bits that Apple doesn't include (one extension module plus the IDE and other applets). But, I'm a bit at loss for where to put this. I was going to put it on the release23-maint branch, but it isn't technically a bug fix. Still, it seems the most logical place. Any objections? -- Jack Jansen, , http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman From guido at python.org Tue Aug 26 14:50:52 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 26 16:51:14 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators In-Reply-To: Your message of "Tue, 26 Aug 2003 16:17:41 EDT." <00a401c36c0f$76952160$c304a044@oemcomputer> References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> <00a401c36c0f$76952160$c304a044@oemcomputer> Message-ID: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com> > > PEP 288 [2] proposes a more general solution, allowing custom > > exception passing to generators. > > Is there any reason to prefer gen.close() over the more general solution > gen.throw(Close) which results in nearly identical code and yet allows > other exception types to be handled as well? After re-skimming PEP 288 I'm still not convinced that a more general problem exists for which .close() isn't sufficient. The one motivating example there (writing a log file) seems forced and can be done in other ways. > Note, the general purpose solution is a natural extension of the existing > syntax and is easily implemented without messing with 'try/finally'. I don't understand this remark. AFAICT PEP 288 doesn't propose new syntax, only a new method and its semantics. And I don't see Samuele's solution as "messing with try/finally". > Pretty much all that was holding up the general solution was that > I had not convinced Guido that the clean-up problem exists in > practice. It looks like you've surmounted that obstacle for me. But you still haven't convinced me of the need for the more generalized PEP 288 mechanism. I do think that the possibility of implementing PEP 288 in the future suggests that Samuele's .close() should be implemented in terms of a special exception, not in terms of a 'return'. The spec needs to define clearly what should happen if the generator catches and ignores the exception, e.g.: def forever(): while True: try: yield None except: pass f = forever() f.next() f.close() Clearly at this point the generator reaches the yield again. What should happen then? Should it suspend so that a subsequent call to f.next() can receive another value? Or should reaching yield after the generator is closed raise another exception? I'm leaning towards the latter, despite the fact that it will cause an infinite loop in this case -- that's no different when you have a print statement instead of a yield statement. (Mentally substituting a print for a yield is often a useful way to think about a generator. The reverse can also be useful to consider converting a non-generator to a generator: if it prints a sequence of values, it can also yield the same sequence.) Another comment on Samuele's PEP: It is sort of sad that the *user* of a generator has to know that the generator's close() must be called. Normally, the beauty of using a try/finally for cleanup is that your callers don't need to know about it. But I see no way around this. And this is still an argument that pleads against the whole thing, either PEP 288 or Samuele's smaller variant: the usual near-guarantee that code in a finally clause will be executed no matter what (barring fatal errors, os._exit() or os.execv()) does not apply. And this was the original argument against allowing yield inside try/finally. But the need for cleanup is also clear, so I like Samuele's KISS compromise. --Guido van Rossum (home page: http://www.python.org/~guido/) From pedronis at bluewin.ch Wed Aug 27 00:09:50 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue Aug 26 17:08:33 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators In-Reply-To: <00a401c36c0f$76952160$c304a044@oemcomputer> References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> Message-ID: <5.2.1.1.0.20030826223131.02716ba0@pop.bluewin.ch> this was written before reading Guido's last comments. At 16:17 26.08.2003 -0400, Raymond Hettinger wrote: > > This PEP proposes that generators should grow such a close method > > with such semantics that the example could be rewritten as: > > > > def all_lines(index_path): > > index = file(index_path,"r") > > try: > > for path in file(index_path,"r"): > > document = file(path.strip(),"r") > > try: > > for line in document: > > yield line > > finally: > > document.close() > > finally: > > index.close() > > > > > > all = all_lines("index.txt") > > try: > > for line in all: > > ... > > finally: > > all.close() >[snip] > > PEP 288 [2] proposes a more general solution, allowing custom > > exception passing to generators. > >Is there any reason to prefer gen.close() over the more general solution >gen.throw(Close) which results in nearly identical code and yet allows >other exception types to be handled as well? > >Note, the general purpose solution is a natural extension of the existing >syntax and is easily implemented without messing with 'try/finally'. 1) I think we want to enable try-finally, because is the typical way to spell resource release: f = file(...,"r") try: ... except GeneratorClose: f.close() return else: f.close() or f = file(...,"r") fastexit = 0 try: ... except GeneratorClose: fastexit = 0 f.close() if fastexit: return 2) even if we had gen.throw(...), I think it would be better to have explicitly gen.close(), it expresses intention better IMO and feels like file.close() etc... 3) for the purpose of close, it seems that forced-return vs. throwing an exception on generator side, can have more clearly definable semantics, although it has some limitations too. so the question is whether there are use cases for the more general gen.throw(...) different from gen.close() purpose, and if we have it whether we can layer a gen.close() with the proper semantics on top of it, i.e. we should then clearly think about the issues for exceptions-semantics for gen.close(). gen.throw is a bigger gun but they don't kill one another regards. From pedronis at bluewin.ch Wed Aug 27 00:20:43 2003 From: pedronis at bluewin.ch (Samuele Pedroni) Date: Tue Aug 26 17:19:16 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators In-Reply-To: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com> References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> <00a401c36c0f$76952160$c304a044@oemcomputer> Message-ID: <5.2.1.1.0.20030826231011.026a2ec0@pop.bluewin.ch> At 13:50 26.08.2003 -0700, Guido van Rossum wrote: >The spec needs to define clearly what should happen if the generator >catches and ignores the exception, e.g.: > > def forever(): > while True: > try: > yield None > except: > pass > > f = forever() > f.next() > f.close() > >Clearly at this point the generator reaches the yield again. What >should happen then? Should it suspend so that a subsequent call to >f.next() can receive another value? Or should reaching yield after >the generator is closed raise another exception? I'm leaning towards >the latter, despite the fact that it will cause an infinite loop in >this case -- that's no different when you have a print statement >instead of a yield statement. > also try: ... yield ... except: # or except Exception raise a new different exception is not a clear-cut situation. close should probably propagate exceptions different from CloseGenerator but here the right thing to do is fuzzy. It also another case of the problems of too broad-catching except clauses, in some sense the exception generated by close is like an async exception like KeyboardInterrupt. >Another comment on Samuele's PEP: It is sort of sad that the *user* of >a generator has to know that the generator's close() must be called. >Normally, the beauty of using a try/finally for cleanup is that your >callers don't need to know about it. But I see no way around this. >And this is still an argument that pleads against the whole thing, >either PEP 288 or Samuele's smaller variant: the usual near-guarantee >that code in a finally clause will be executed no matter what (barring >fatal errors, os._exit() or os.execv()) does not apply. And this was >the original argument against allowing yield inside try/finally. But >the need for cleanup is also clear, so I like Samuele's KISS compromise. yes, we cannot totally win. From oussoren at cistron.nl Wed Aug 27 00:36:02 2003 From: oussoren at cistron.nl (Ronald Oussoren) Date: Tue Aug 26 17:36:10 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators In-Reply-To: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com> References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> <00a401c36c0f$76952160$c304a044@oemcomputer> <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com> Message-ID: <4A78E0D2-D80D-11D7-94C4-0003931CFE24@cistron.nl> On Tuesday, 26 August 2003, at 22:50, Guido van Rossum wrote: > Another comment on Samuele's PEP: It is sort of sad that the *user* of > a generator has to know that the generator's close() must be called. > Normally, the beauty of using a try/finally for cleanup is that your > callers don't need to know about it. But I see no way around this. I have no idea what the issues are w.r.t. cleanup for generators, but wouldn't it be more natural to make sure that the finally clause would be executed when the generator is garbage collected? Back to lurking, Ronald From logistix at cathoderaymission.net Tue Aug 26 19:13:08 2003 From: logistix at cathoderaymission.net (logistix) Date: Tue Aug 26 18:07:09 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <1061920779.3f4ba00b589f4@mcherm.com> Message-ID: <000001c36c1f$3d0946f0$20bba8c0@XP> > > [Tim continues] > > The display is > > identical to what you'd see if you used IE to browse the HTML docs > > directly today, so it's unclear what it is about "html > docs" you might > > prefer. > > [Michael] > Mostly just that it's "more standard". I can view in a > browser of my choice, launch a browser from within my > editor/IDE, set bookmarks (but I don't usually do that), > share on a drive and view from unix (but I don't do that), etc. > The standard build procedure for *nix systems doesn't install ANY documentation (maybe a manpage?). Users who want the HTML version download it and install it where they want. Windows users can presumably do the same. .chm files are the way windows most users expect to see help, right or wrong. On a more pragmatic level, there is a slight "usability bug" with the current setup. By default, if you have an IE window open and choose "Python Manuals" from the menu, the help will spawn in the existing window instead of spawning an new one. This is a Microsoft "feature", PythonWin isn't doing anything wrong. It can be minorly annoying if you didn't have the previous page bookmarked and start to do some serious browsing. It can be a major annoyance if you're in the middle of using some sort of full featured web-app and lose your work. From guido at python.org Tue Aug 26 18:00:43 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 26 20:00:54 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime Message-ID: <200308270000.h7R00ir04986@12-236-84-31.client.attbi.com> I was watching file modification times on my Windows box (strange hobby, I know :-), and I noticed that after a fresh install of Python, the .pyc files seem to be written when the first code that imports the corresponding module runs, rather than all of the .pyc files being compiled at once by the installer. Wasn't there code in the installer that precompiles all modules? I know the Unix install does this, and I vaguely remember that the Windows installer did this too -- or was it only the Win32all installer??? If there's code to do that in the Windows installer now, it seems it's not working. If there isn't such code, perhaps there should be? --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one at comcast.net Tue Aug 26 21:43:13 2003 From: tim.one at comcast.net (Tim Peters) Date: Tue Aug 26 20:44:09 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: <200308270000.h7R00ir04986@12-236-84-31.client.attbi.com> Message-ID: [Guido] > I was watching file modification times on my Windows box (strange > hobby, I know :-), and I noticed that after a fresh install of Python, > the .pyc files seem to be written when the first code that imports the > corresponding module runs, rather than all of the .pyc files being > compiled at once by the installer. Wasn't there code in the installer > that precompiles all modules? Not in the PLabs Windows installer, no. > I know the Unix install does this, and I vaguely remember that the > Windows installer did this too -- or was it only the Win32all > installer??? Probably the latter, then. > If there's code to do that in the Windows installer now, it seems it's > not working. There isn't, so it's working fine . > If there isn't such code, perhaps there should be? Why? Not that increasing installation time and disk consumption aren't worthy goals ... From guido at python.org Tue Aug 26 20:31:47 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 26 22:31:59 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: Your message of "Tue, 26 Aug 2003 20:43:13 EDT." References: Message-ID: <200308270231.h7R2Vlq05116@12-236-84-31.client.attbi.com> > [Guido] > > I was watching file modification times on my Windows box (strange > > hobby, I know :-), and I noticed that after a fresh install of Python, > > the .pyc files seem to be written when the first code that imports the > > corresponding module runs, rather than all of the .pyc files being > > compiled at once by the installer. Wasn't there code in the installer > > that precompiles all modules? [Tim] > Not in the PLabs Windows installer, no. > > > I know the Unix install does this, and I vaguely remember that the > > Windows installer did this too -- or was it only the Win32all > > installer??? > > Probably the latter, then. > > > If there's code to do that in the Windows installer now, it seems it's > > not working. > > There isn't, so it's working fine . OK, so maybe I was hallucinating. > > If there isn't such code, perhaps there should be? > > Why? Not that increasing installation time and disk consumption aren't > worthy goals ... If this isn't done, a problem might be (and this is why this is always done on Unix) that if the user who installs Python has more privileges than the user who uses Python, the user who uses Python may not be able to write the directory containing .pyc files, so they end up recompiling all modules each time they are loaded. I expect this will be more of a problem as typical Windows users and installations (e.g. XP) become more security aware, software is installed by Administrator, and users don't have Administrator privileges. I guess the way to implement it (and I believe Mark Hammond did indeed do this for win32all) is to run Python near the end of the installer with the compileall.py script as an argument. Feeling-quite-the-Windows-XP-expert-lately, --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Tue Aug 26 21:57:59 2003 From: guido at python.org (Guido van Rossum) Date: Tue Aug 26 23:58:39 2003 Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators In-Reply-To: Your message of "Tue, 26 Aug 2003 23:36:02 +0200." <4A78E0D2-D80D-11D7-94C4-0003931CFE24@cistron.nl> References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch> <00a401c36c0f$76952160$c304a044@oemcomputer> <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com> <4A78E0D2-D80D-11D7-94C4-0003931CFE24@cistron.nl> Message-ID: <200308270358.h7R3vxm05236@12-236-84-31.client.attbi.com> > I have no idea what the issues are w.r.t. cleanup for generators, but > wouldn't it be more natural to make sure that the finally clause would > be executed when the generator is garbage collected? The problem is that the language definition doesn't define *when* GC happens, if at all. E.g. in Jython this depends on the Java GC. And even in CPython, it's easily conceivable that an accidental cycle in the user data structures prevents collection. So there is a need for explicit control in some cases. > Back to lurking, Good idea. :-) --Guido van Rossum (home page: http://www.python.org/~guido/) From tim.one at comcast.net Wed Aug 27 01:00:05 2003 From: tim.one at comcast.net (Tim Peters) Date: Wed Aug 27 00:08:05 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: <200308270231.h7R2Vlq05116@12-236-84-31.client.attbi.com> Message-ID: [Guido] > If this isn't done, a problem might be (and this is why this is always > done on Unix) that if the user who installs Python has more privileges > than the user who uses Python, the user who uses Python may not be > able to write the directory containing .pyc files, so they end up > recompiling all modules each time they are loaded. > > I expect this will be more of a problem as typical Windows users and > installations (e.g. XP) become more security aware, software is > installed by Administrator, and users don't have Administrator > privileges. > > I guess the way to implement it (and I believe Mark Hammond did indeed > do this for win32all) is to run Python near the end of the installer > with the compileall.py script as an argument. > > Feeling-quite-the-Windows-XP-expert-lately, Cool. If an organization has enough money to afford an administrator who installs stuff for unprivileged masses, they have enough money to pay Thomas to make this change <0.5 wink>. Mark once told me he compiles stuff at the end of the win32all install because generating Windows type libraries can take a long time, and users griped about feeling the pain of that on first use otherwise. That's a reason I can understand . If you can too, then it's more important to precompile everything needed for IDLE to start up the first time. From mhammond at skippinet.com.au Wed Aug 27 15:20:42 2003 From: mhammond at skippinet.com.au (Mark Hammond) Date: Wed Aug 27 00:21:06 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: <200308270231.h7R2Vlq05116@12-236-84-31.client.attbi.com> Message-ID: <18c201c36c52$9450fec0$f502a8c0@eden> [Guido] > I expect this will be more of a problem as typical Windows users and > installations (e.g. XP) become more security aware, software is I expect you are correct. My reason was a little more practical - certain h2py.py generated scripts would takes *ages* to compile (way back when sub GHz machines were the norm!) and this made the first pywin32 program they used (often Pythonwin) very very slow to start. I didn't want to give the wrong impression about how slow it really was . > I guess the way to implement it (and I believe Mark Hammond did indeed > do this for win32all) is to run Python near the end of the installer > with the compileall.py script as an argument. Quite frankly, I would find a DOS window executing "compileall" a deadly sin - so I got a little carried away and wrote a WISE extension, so that I got a cute progress bar during the compilation :) However, I haven't advocated or implemented this for the main installer I would prefer to see that effort spent moving from WISE to Inno. The next Inno version will include the previously optional "scripting extensions", and this should make it practical to use with Python. Obviously, a WISE extension wouldn't be heading in this direction. FYI, the new version of inno is at http://www.jrsoftware.org/is4.php, and its license (http://www.jrsoftware.org/files/is/license.txt) would seem a better fit for Python than WISE. Could make a cute little PSA sponsored project . Pity it uses pascal . Mark From chrism at zope.com Wed Aug 27 05:40:18 2003 From: chrism at zope.com (Chris McDonough) Date: Wed Aug 27 00:40:20 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: <18c201c36c52$9450fec0$f502a8c0@eden> References: <18c201c36c52$9450fec0$f502a8c0@eden> Message-ID: <1061959346.10603.74.camel@athlon.dc.dc.cox.net> The newest Zope installs schlep along a whole Python installation (including win32all), and precompile all the pyc/pyos at the end of the install process. This includes all py files shipped as part of the Python stdlib, the win32all extension py files, and all Zope py files. It does take a bit, but it's not intolerable, even on the creaky 500MHz Celeron on which I build the stuff. I think this was probably just an overgeneralization carried over from the install process for RPMs. Not compiling .py files after installation on UNIX typically results in needing to recompile them every time Pytyhon is invoked because the directories they're typically installed in are generally not normal-user-writable. This probably isn't the case for the majority of Windows installs, but we ignore that fact and do it anyway. nothing-quite-like-building-python-using-make-bash-and-pascal-ly y'rs, - C On Wed, 2003-08-27 at 00:20, Mark Hammond wrote: > [Guido] > > I expect this will be more of a problem as typical Windows users and > > installations (e.g. XP) become more security aware, software is > > I expect you are correct. My reason was a little more practical - certain > h2py.py generated scripts would takes *ages* to compile (way back when sub > GHz machines were the norm!) and this made the first pywin32 program they > used (often Pythonwin) very very slow to start. I didn't want to give the > wrong impression about how slow it really was . > > > I guess the way to implement it (and I believe Mark Hammond did indeed > > do this for win32all) is to run Python near the end of the installer > > with the compileall.py script as an argument. > > Quite frankly, I would find a DOS window executing "compileall" a deadly > sin - so I got a little carried away and wrote a WISE extension, so that I > got a cute progress bar during the compilation :) > > However, I haven't advocated or implemented this for the main installer I > would prefer to see that effort spent moving from WISE to Inno. The next > Inno version will include the previously optional "scripting extensions", > and this should make it practical to use with Python. Obviously, a WISE > extension wouldn't be heading in this direction. > > FYI, the new version of inno is at http://www.jrsoftware.org/is4.php, and > its license (http://www.jrsoftware.org/files/is/license.txt) would seem a > better fit for Python than WISE. Could make a cute little PSA sponsored > project . Pity it uses pascal . > > Mark > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > http://mail.python.org/mailman/listinfo/python-dev From theller at python.net Wed Aug 27 12:15:17 2003 From: theller at python.net (Thomas Heller) Date: Wed Aug 27 05:16:07 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <16203.40017.707481.805254@grendel.zope.com> (Fred L. Drake, Jr.'s message of "Tue, 26 Aug 2003 13:43:45 -0400") References: <16203.40017.707481.805254@grendel.zope.com> Message-ID: "Fred L. Drake, Jr." writes: > Tim Peters writes: > > I think so. Note that IDLE may need a bit of fiddling, to keep its Help -> > > Python Docs menu entry working. > > IDLE currently looks for the index.html file in a few places (which > depend on platform); if it can't find it, it uses the documentation on > python.org. > > It should be too hard to change it to load the HTML Help viewer if it > finds the .chm file on Windows, and to still fall back to the HTML or > the online documentation if the .chm can't be found. Changing it is trivial, EditorWindow.help_url must point to Python23.chm (if it exists). I can do this. Even nicer would be context-sensitive keyword help, but it seems IDLE doesn't support it, right? Thomas From fdrake at acm.org Wed Aug 27 10:49:08 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed Aug 27 09:59:49 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: References: <16203.40017.707481.805254@grendel.zope.com> Message-ID: <16204.46804.944055.691796@grendel.zope.com> Thomas Heller writes: > Even nicer would be context-sensitive keyword help, but it seems IDLE > doesn't support it, right? Not currently, but part of the reason is that the Python docs don't really expose the necessary information to support that. I've started working on that in a branch of the Doc/ tree, but haven't had a lot of time to work on it lately. -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From theller at python.net Wed Aug 27 17:11:21 2003 From: theller at python.net (Thomas Heller) Date: Wed Aug 27 10:12:07 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <16204.46804.944055.691796@grendel.zope.com> (Fred L. Drake, Jr.'s message of "Wed, 27 Aug 2003 09:49:08 -0400") References: <16203.40017.707481.805254@grendel.zope.com> <16204.46804.944055.691796@grendel.zope.com> Message-ID: "Fred L. Drake, Jr." writes: > Thomas Heller writes: > > Even nicer would be context-sensitive keyword help, but it seems IDLE > > doesn't support it, right? > > Not currently, but part of the reason is that the Python docs don't > really expose the necessary information to support that. I've started > working on that in a branch of the Doc/ tree, but haven't had a lot of > time to work on it lately. But doesn't something like swish (I've used that in the past) allow to index and search the doc tree? OTOH: pyhelp.cgi does keyword search using the same technique as the one used to the the htmlhelp keywords, and is cross platform as well ;-) Thomas From fdrake at acm.org Wed Aug 27 11:15:25 2003 From: fdrake at acm.org (Fred L. Drake, Jr.) Date: Wed Aug 27 10:16:20 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: References: <16203.40017.707481.805254@grendel.zope.com> <16204.46804.944055.691796@grendel.zope.com> Message-ID: <16204.48381.500804.702110@grendel.zope.com> Thomas Heller writes: > But doesn't something like swish (I've used that in the past) allow to > index and search the doc tree? Should be able to, if someone does the work to integrate it. I'm not sure if there's currently a Python binding; I only looked at their website briefly a month or so ago. > OTOH: pyhelp.cgi does keyword search using the same technique as the one > used to the the htmlhelp keywords, and is cross platform as well ;-) Then by all means, feel free to hook that up in IDLE! -Fred -- Fred L. Drake, Jr. PythonLabs at Zope Corporation From logistix at cathoderaymission.net Wed Aug 27 12:01:06 2003 From: logistix at cathoderaymission.net (logistix) Date: Wed Aug 27 11:53:15 2003 Subject: [Python-Dev] Py2.3.1 In-Reply-To: <16204.46804.944055.691796@grendel.zope.com> Message-ID: On Wed, 27 Aug 2003, Fred L. Drake, Jr. wrote: > > Thomas Heller writes: > > Even nicer would be context-sensitive keyword help, but it seems IDLE > > doesn't support it, right? > > Not currently, but part of the reason is that the Python docs don't > really expose the necessary information to support that. I've started > working on that in a branch of the Doc/ tree, but haven't had a lot of > time to work on it lately. > > > -Fred > I don't know if this will help, I wrote a utility class for PyXR that manages to harvest links for about 80-90% of builtin functions and standard modules from the HTML docs. I originally tried to look at the latex sources directly, but that proved to be too complex. It takes a callback fucntion that receives a scope-tuple and appropriate url. Like: ('map',), "http://dsdfsdfs" ('socket', 'socket'), "http://dsfsdf" ... Its in the htmlCrawler.py file in the PyXR dist at: http://tinyurl.com/ld34 OR you can view the PyXR'ed source (with a description of my assumptions) at: http://tinyurl.com/ld2z -Grant From theller at python.net Wed Aug 27 22:02:28 2003 From: theller at python.net (Thomas Heller) Date: Wed Aug 27 15:03:07 2003 Subject: Conext sensitive help for IDLE (was: [Python-Dev] Py2.3.1) In-Reply-To: <16204.48381.500804.702110@grendel.zope.com> (Fred L. Drake, Jr.'s message of "Wed, 27 Aug 2003 10:15:25 -0400") References: <16203.40017.707481.805254@grendel.zope.com> <16204.46804.944055.691796@grendel.zope.com> <16204.48381.500804.702110@grendel.zope.com> Message-ID: "Fred L. Drake, Jr." writes: > Thomas Heller writes: > > But doesn't something like swish (I've used that in the past) allow to > > index and search the doc tree? > > Should be able to, if someone does the work to integrate it. I'm not > sure if there's currently a Python binding; I only looked at their > website briefly a month or so ago. > > > OTOH: pyhelp.cgi does keyword search using the same technique as the one > > used to the the htmlhelp keywords, and is cross platform as well ;-) > > Then by all means, feel free to hook that up in IDLE! (I fear this is getting off-topic for python-dev) The only thing I do not know is how to display a list of urls found, and let the user select one. Is there any functionality in IDLE for this? Thomas From raymond.hettinger at verizon.net Wed Aug 27 16:22:38 2003 From: raymond.hettinger at verizon.net (Raymond Hettinger) Date: Wed Aug 27 15:28:21 2003 Subject: [Python-Dev] Fixing Patches and Bugs for Py2.3.1 Message-ID: <004701c36cd0$b2056040$f92ba044@oemcomputer> With Martin and Brett away from the keyboard, the pending bug/patch pool is growing faster than it is shrinking. Even if you only have a little time, please scan through the list to set priorities, endorse good ideas, trump the bad, assign to the appropriate party, etc. If you bless an idea and suggest an approach, sometimes I can take it from there and get it implemented, tested, and documented. Even if no immediate action is to be taken, a comment of some sort will let the contributor know that their efforts haven't fallen on deaf ears. I've solicited participation from newsgroup followers and would like to their efforts to be rewarded with responsiveness. Raymond Hettinger From pje at telecommunity.com Wed Aug 27 17:40:10 2003 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 27 16:44:39 2003 Subject: [Python-Dev] readline.clear_history() Message-ID: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> I've been working on a set of Python tools that use readline, but want to keep history separate between different interaction modes. Unfortunately, this really needs to be able to access readline's clear_history(), as read_history_file() leaves existing history intact. I'd be happy to whip up a patch to add this (as readline.clear_history()), but I was wondering if perhaps the reason it's not currently exported by the readline module is a compatibility issue for older readline implementations that are officially supported. Thanks. From tjreedy at udel.edu Wed Aug 27 17:59:31 2003 From: tjreedy at udel.edu (Terry Reedy) Date: Wed Aug 27 16:59:39 2003 Subject: [Python-Dev] Archive flooding Message-ID: In the past 24 hours, gmane.comp.python.devel was flooded with about 20000 py-dev posts from the last 3 years. (Is this the whole archive?) Fortunately, they had the original posting dates so current stuff could be sorted out and the rest 'catchup'ed away. Just curious -- did these come from or go to the mailing list (and people mailboxes) also? (If the latter, a plus for the news gateway.) Is this an effect of the current virus storm or just a strange coincidence? TJR From barry at python.org Wed Aug 27 22:05:35 2003 From: barry at python.org (Barry Warsaw) Date: Wed Aug 27 17:05:57 2003 Subject: [Python-Dev] Archive flooding In-Reply-To: References: Message-ID: <1062018297.19312.2.camel@yyz> On Wed, 2003-08-27 at 16:59, Terry Reedy wrote: > In the past 24 hours, gmane.comp.python.devel was flooded with about > 20000 py-dev posts from the last 3 years. (Is this the whole > archive?) Fortunately, they had the original posting dates so > current stuff could be sorted out and the rest 'catchup'ed away. > > Just curious -- did these come from or go to the mailing list (and > people mailboxes) also? (If the latter, a plus for the news gateway.) > Is this an effect of the current virus storm or just a strange > coincidence? Over in list-owners there was a request to Gmane-ify all our public lists that weren't currently archived over there. There was general consensus that this was a good thing, so I guess the process has started. I'm sure these came from the lists' .mbox archive files. -Barry From guido at python.org Wed Aug 27 15:45:44 2003 From: guido at python.org (Guido van Rossum) Date: Wed Aug 27 17:46:00 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: Your message of "Wed, 27 Aug 2003 16:40:10 EDT." <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> References: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> Message-ID: <200308272145.h7RLji106564@12-236-84-31.client.attbi.com> > I've been working on a set of Python tools that use readline, but want to > keep history separate between different interaction modes. Unfortunately, > this really needs to be able to access readline's clear_history(), as > read_history_file() leaves existing history intact. Sounds like a good feature. > I'd be happy to whip up a patch to add this (as readline.clear_history()), > but I was wondering if perhaps the reason it's not currently exported by > the readline module is a compatibility issue for older readline > implementations that are officially supported. Thanks. It's probably laziness. When I created the initial readline module, I didn't know anything about the C API of GNU readline. All I wanted was basic readline functionality, and I applied the YAGNI principle vigorously. Gradually, various people have added APIs for various useful GNU readline APIs. But there might be a version issue as well. I have various versions of the readline sources online just for this purpose (GNU readline is notorious for not providing an easy way to check at compile-time which version you are using), and I note that clear_history() is present in 2.2, but not in 2.0. I seem to recall that 2.2 is the oldest widely used version, so I think you're safe. --Guido van Rossum (home page: http://www.python.org/~guido/) From pje at telecommunity.com Wed Aug 27 18:56:18 2003 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 27 17:57:42 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: <200308272145.h7RLji106564@12-236-84-31.client.attbi.com> References: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> Message-ID: <5.1.1.6.0.20030827174942.02095030@telecommunity.com> At 02:45 PM 8/27/03 -0700, Guido van Rossum wrote: > > I've been working on a set of Python tools that use readline, but want to > > keep history separate between different interaction modes. Unfortunately, > > this really needs to be able to access readline's clear_history(), as > > read_history_file() leaves existing history intact. > >Sounds like a good feature. > > > I'd be happy to whip up a patch to add this (as readline.clear_history()), > > but I was wondering if perhaps the reason it's not currently exported by > > the readline module is a compatibility issue for older readline > > implementations that are officially supported. Thanks. > >It's probably laziness. When I created the initial readline module, I >didn't know anything about the C API of GNU readline. All I wanted >was basic readline functionality, and I applied the YAGNI principle >vigorously. Gradually, various people have added APIs for various >useful GNU readline APIs. > >But there might be a version issue as well. I have various versions >of the readline sources online just for this purpose (GNU readline is >notorious for not providing an easy way to check at compile-time which >version you are using), and I note that clear_history() is present in >2.2, but not in 2.0. I seem to recall that 2.2 is the oldest widely >used version, so I think you're safe. Hmmm... the CVS log for Modules/readline.c makes mention of changes made to assure backward compatibility with readline 2.1 and 2.0, and #ifdefs some items as a result. I suppose I could wrap a clear_history in the same. However, that leads to a couple of questions: * Should clear_history() only appear in the readline module if the facility exists? * If it should always appear, should it be a no-op if the facility isn't available, or raise an error? "Errors should never pass silently" suggests that if it does appear, it should raise an error if the facility doesn't exist. So, I guess the question is, should you get an error trying to access clear_history(), or an error calling it? (And in the latter case, is NotImplementedError the right thing to raise?) Last question (I hope): as a feature, I presume this has to wait for 2.4 to get in, yes? From guido at python.org Wed Aug 27 16:13:57 2003 From: guido at python.org (Guido van Rossum) Date: Wed Aug 27 18:14:14 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: Your message of "Wed, 27 Aug 2003 17:56:18 EDT." <5.1.1.6.0.20030827174942.02095030@telecommunity.com> References: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> <5.1.1.6.0.20030827174942.02095030@telecommunity.com> Message-ID: <200308272213.h7RMDvh06652@12-236-84-31.client.attbi.com> > >But there might be a version issue as well. I have various versions > >of the readline sources online just for this purpose (GNU readline is > >notorious for not providing an easy way to check at compile-time which > >version you are using), and I note that clear_history() is present in > >2.2, but not in 2.0. I seem to recall that 2.2 is the oldest widely > >used version, so I think you're safe. > > Hmmm... the CVS log for Modules/readline.c makes mention of changes made > to assure backward compatibility with readline 2.1 and 2.0, and #ifdefs > some items as a result. I suppose I could wrap a clear_history in the > same. That's always better, of course. How old were those log entries? > However, that leads to a couple of questions: > > * Should clear_history() only appear in the readline module if the facility > exists? Yes. > * If it should always appear, should it be a no-op if the facility isn't > available, or raise an error? N/A. > "Errors should never pass silently" suggests that if it does appear, it > should raise an error if the facility doesn't exist. So, I guess the > question is, should you get an error trying to access clear_history(), or > an error calling it? (And in the latter case, is NotImplementedError the > right thing to raise?) I like it if the function is not available at all when it's not implemented. When you naively try to use it, the behavior is pretty much the same: you get an exception. But a program that wants to do soemthing different if the feature exists can test for presence of the attribute, and that's always better than having to call it and see if it raises NotImplementedError. NotImplementedError is mostly for abstract base classes. An extra advantage is that by testing for presence of the feature, you can cope with older Python versions is the same way as you cope with ancient readline versions. > Last question (I hope): as a feature, I presume this has to wait for 2.4 to > get in, yes? Probably. I'm personally not so stringent for minor stuff like this (hey, I added bool() to 2.2.1 :-), but it seems others are increasingly resisting even minor feature additions in micro releases (except in the email package :-). --Guido van Rossum (home page: http://www.python.org/~guido/) From barry at python.org Wed Aug 27 23:18:50 2003 From: barry at python.org (Barry Warsaw) Date: Wed Aug 27 18:18:51 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: <5.1.1.6.0.20030827174942.02095030@telecommunity.com> References: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> <5.1.1.6.0.20030827174942.02095030@telecommunity.com> Message-ID: <1062022698.19312.5.camel@yyz> On Wed, 2003-08-27 at 17:56, Phillip J. Eby wrote: > * Should clear_history() only appear in the readline module if the facility > exists? Yes. > * If it should always appear, should it be a no-op if the facility isn't > available, or raise an error? > > "Errors should never pass silently" suggests that if it does appear, it > should raise an error if the facility doesn't exist. So, I guess the > question is, should you get an error trying to access clear_history(), or > an error calling it? (And in the latter case, is NotImplementedError the > right thing to raise?) Error when accessing it. > Last question (I hope): as a feature, I presume this has to wait for 2.4 to > get in, yes? Hmm, it won't break backward compatibility, so maybe 2.3.1 would be okay. OTOH, it ain't a bug fix. OTOOH, 2.4 is a loonnngg way away. -Barry From barry at python.org Wed Aug 27 23:23:45 2003 From: barry at python.org (Barry Warsaw) Date: Wed Aug 27 18:24:30 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: <200308272213.h7RMDvh06652@12-236-84-31.client.attbi.com> References: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> <5.1.1.6.0.20030827174942.02095030@telecommunity.com> <200308272213.h7RMDvh06652@12-236-84-31.client.attbi.com> Message-ID: <1062022992.19312.7.camel@yyz> On Wed, 2003-08-27 at 18:13, Guido van Rossum wrote: > Probably. I'm personally not so stringent for minor stuff like this > (hey, I added bool() to 2.2.1 :-), but it seems others are > increasingly resisting even minor feature additions in micro releases > (except in the email package :-). Touche! :) -Barry From pje at telecommunity.com Wed Aug 27 19:23:13 2003 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 27 18:24:41 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: <1062022698.19312.5.camel@yyz> References: <5.1.1.6.0.20030827174942.02095030@telecommunity.com> <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> <5.1.1.6.0.20030827174942.02095030@telecommunity.com> Message-ID: <5.1.1.6.0.20030827181951.02083340@telecommunity.com> At 06:18 PM 8/27/03 -0400, Barry Warsaw wrote: > > Last question (I hope): as a feature, I presume this has to wait for > 2.4 to > > get in, yes? > >Hmm, it won't break backward compatibility, so maybe 2.3.1 would be >okay. OTOH, it ain't a bug fix. OTOOH, 2.4 is a loonnngg way away. As a practical matter, I'll initially be writing this as a patch against 2.2, which is where I actually need it. :) But I'll port it to be against 2.3 before I submit it. Hmmm... that makes me think of tests... I guess I won't really be able to supply a test for the feature, since that would just be testing whether the feature is available. From pje at telecommunity.com Wed Aug 27 19:31:53 2003 From: pje at telecommunity.com (Phillip J. Eby) Date: Wed Aug 27 18:32:54 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: <200308272213.h7RMDvh06652@12-236-84-31.client.attbi.com> References: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> <5.1.1.6.0.20030827174942.02095030@telecommunity.com> Message-ID: <5.1.1.6.0.20030827182332.020830d0@telecommunity.com> At 03:13 PM 8/27/03 -0700, Guido van Rossum wrote: > > >But there might be a version issue as well. I have various versions > > >of the readline sources online just for this purpose (GNU readline is > > >notorious for not providing an easy way to check at compile-time which > > >version you are using), and I note that clear_history() is present in > > >2.2, but not in 2.0. I seem to recall that 2.2 is the oldest widely > > >used version, so I think you're safe. > > > > Hmmm... the CVS log for Modules/readline.c makes mention of changes made > > to assure backward compatibility with readline 2.1 and 2.0, and #ifdefs > > some items as a result. I suppose I could wrap a clear_history in the > > same. > >That's always better, of course. How old were those log entries? Last December there's a checkin by you of a patch from Magnus Lie Hetland to support 2.0/2.1, on the Python 2.3 branch. More recently, this support was backported to the Python 2.2 maintenance branch, and released in 2.2.3. In each case, the change revolves around an #ifdef for HAVE_RL_COMPLETION_APPEND_CHARACTER, so presumably I can use the same #ifdef to protect clear_history(). (Unless it's preferred to have a distinct flag for this, but I have zero autoconf knowledge/experience, so I might have to ask someone for help on that part.) >An extra advantage is that by testing for >presence of the feature, you can cope with older Python versions is >the same way as you cope with ancient readline versions. Excellent point! I'll do it that way, then. From guido at python.org Wed Aug 27 17:28:46 2003 From: guido at python.org (Guido van Rossum) Date: Wed Aug 27 19:28:57 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: Your message of "Wed, 27 Aug 2003 18:23:13 EDT." <5.1.1.6.0.20030827181951.02083340@telecommunity.com> References: <5.1.1.6.0.20030827174942.02095030@telecommunity.com> <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> <5.1.1.6.0.20030827174942.02095030@telecommunity.com> <5.1.1.6.0.20030827181951.02083340@telecommunity.com> Message-ID: <200308272328.h7RNSkA06766@12-236-84-31.client.attbi.com> > Hmmm... that makes me think of tests... I guess I won't really be able to > supply a test for the feature, since that would just be testing whether the > feature is available. For interactive stuff like this, absence of unit tests is acceptable. --Guido van Rossum (home page: http://www.python.org/~guido/) From guido at python.org Wed Aug 27 17:29:55 2003 From: guido at python.org (Guido van Rossum) Date: Wed Aug 27 19:30:09 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: Your message of "Wed, 27 Aug 2003 18:31:53 EDT." <5.1.1.6.0.20030827182332.020830d0@telecommunity.com> References: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> <5.1.1.6.0.20030827174942.02095030@telecommunity.com> <5.1.1.6.0.20030827182332.020830d0@telecommunity.com> Message-ID: <200308272329.h7RNTt106782@12-236-84-31.client.attbi.com> > > > Hmmm... the CVS log for Modules/readline.c makes mention of changes made > > > to assure backward compatibility with readline 2.1 and 2.0, and #ifdefs > > > some items as a result. I suppose I could wrap a clear_history in the > > > same. > > > >That's always better, of course. How old were those log entries? > > Last December there's a checkin by you of a patch from Magnus Lie Hetland > to support 2.0/2.1, on the Python 2.3 branch. More recently, this support > was backported to the Python 2.2 maintenance branch, and released in > 2.2.3. In each case, the change revolves around an #ifdef for > HAVE_RL_COMPLETION_APPEND_CHARACTER, so presumably I can use the same > #ifdef to protect clear_history(). (Unless it's preferred to have a > distinct flag for this, but I have zero autoconf knowledge/experience, so I > might have to ask someone for help on that part.) Nah, it looks like those featuers were introduced in the same GNU readline release, so it's okay to test for the same symbol. --Guido van Rossum (home page: http://www.python.org/~guido/) From bac at OCF.Berkeley.EDU Wed Aug 27 18:08:44 2003 From: bac at OCF.Berkeley.EDU (Brett C.) Date: Wed Aug 27 20:08:56 2003 Subject: [Python-Dev] Moving Tuesday; offline for 2 weeks In-Reply-To: <3F4064DC.9040504@ocf.berkeley.edu> References: <3F4064DC.9040504@ocf.berkeley.edu> Message-ID: <3F4D480C.6030203@ocf.berkeley.edu> Brett C. wrote: > I am moving Tuesday morning down to San Luis Obispo. The problem with > this is that I am going to lose Net for two weeks thanks to SBC saying > it will take 10 to 12 working days for me to get my DSL turned on; I > ordered it this past Friday. > > So I have disabled delivery on all lists (pydotorg-related lists, > PyCon-related lists) but python-dev, but that is just so that I have a > copy for when I do the next Summary. There is a chance I will be able > to check between now and and when I get DSL, but if I do it will only be > python-dev and I have no clue if that will even happen. > OK, I am back after getting rid of over 7,000 spams and virus warnings. Give me a week to get back into the groove. If you sent me an email and I don't respond relatively soon then send it again since I probably then deleted the email by accident. > Hopefully I will be productive during my time "off". > Uh, no. But I know Java better than I did before (that was a painful experience). -Brett From barry at python.org Thu Aug 28 02:43:14 2003 From: barry at python.org (Barry Warsaw) Date: Wed Aug 27 21:43:17 2003 Subject: [Python-Dev] New SIG: email-sig Message-ID: <1062034958.998.19.camel@anthem> After unanimous consent on the meta-sig and mimelib-devel lists, I've gone ahead and created the new email-sig. See http://www.python.org/sigs/email-sig for details. This SIG's mission is to design and implement version 3.0 of the email package for Python 2.4. Please use this SIG instead of the mimelib-devel list from now on. -Barry From mfb at lotusland.dyndns.org Wed Aug 27 19:09:35 2003 From: mfb at lotusland.dyndns.org (Matthew F. Barnes) Date: Wed Aug 27 23:12:33 2003 Subject: [Python-Dev] Extending hex() and oct() to accept strings Message-ID: <3706.130.76.32.145.1062025775.squirrel@server.lotusland.dyndns.org> Forgive me if this idea has been already proposed and shot down. Earlier today I was trying to print some binary data I received over a socket in hexadecimal format so that I could decipher the contents. The first thing I instinctively tried was hex(datastring) and was somewhat suprised to find that it didn't work. I wandered around the documentation a bit looking for a simple and "obvious" way to do this and came up empty (I would appreciate it if someone could enlighten me if there already is a way). For the rest of the day it kinda bothered me that my first attempt didn't work. After spending probably not enough time thinking about this, I came up with what might be a reasonable extension for hex() and oct() that would allow those functions to accept a string and produce a quoted hexadecimal or octal representation of the input string using the escape notation \xhh or \ooo. To illustrate: >>> hex('ABC') "'\\x41\\x42\\x43'" >>> print hex('ABC') # This is what I was trying to do today '\x41\x42\x43' >>> eval(hex('ABC')) 'ABC' >>> oct('ABC') "'\\101\\102\\103'" >>> print oct('ABC') '\101\102\103' >>> eval(oct('ABC')) 'ABC' Are there any subtle issues with supporting this that I'm not seeing? Matthew Barnes From guido at python.org Wed Aug 27 21:52:05 2003 From: guido at python.org (Guido van Rossum) Date: Wed Aug 27 23:52:16 2003 Subject: [Python-Dev] Extending hex() and oct() to accept strings In-Reply-To: Your message of "Wed, 27 Aug 2003 18:09:35 CDT." <3706.130.76.32.145.1062025775.squirrel@server.lotusland.dyndns.org> References: <3706.130.76.32.145.1062025775.squirrel@server.lotusland.dyndns.org> Message-ID: <200308280352.h7S3q5P07084@12-236-84-31.client.attbi.com> > To illustrate: > > >>> hex('ABC') > "'\\x41\\x42\\x43'" > >>> print hex('ABC') # This is what I was trying to do today > '\x41\x42\x43' > >>> eval(hex('ABC')) > 'ABC' > > >>> oct('ABC') > "'\\101\\102\\103'" > >>> print oct('ABC') > '\101\102\103' > >>> eval(oct('ABC')) > 'ABC' > > Are there any subtle issues with supporting this that I'm not seeing? This is indeed elegant because it preserves the property that hex() and oct() have for numbers: eval(hex(x)) == x == eval(oct(x)). However, I'm not sure if it should be added. It adds yet another feature to document and support (think of Jython, Pychecker, etc.), and I think that the number of people who care about hexadecimal strings is becoming a vanishingly small fraction of the total number of programmers. (Nobody has cared about octal for a long time; the only use for it at this point is to show unix file permission bits.) And I think that what you really wanted is probably closer to this: >>> import binascii >>> binascii.hexlify('ABC') '414243' >>> --Guido van Rossum (home page: http://www.python.org/~guido/) From halley at play-bow.org Thu Aug 28 07:30:09 2003 From: halley at play-bow.org (Bob Halley) Date: Thu Aug 28 02:30:11 2003 Subject: [Python-Dev] Extending hex() and oct() to accept strings In-Reply-To: <200308280352.h7S3q5P07084@12-236-84-31.client.attbi.com> References: <3706.130.76.32.145.1062025775.squirrel@server.lotusland.dyndns.org> <200308280352.h7S3q5P07084@12-236-84-31.client.attbi.com> Message-ID: Guido van Rossum writes: > And I think that what you really wanted is probably closer to this: > > >>> import binascii > >>> binascii.hexlify('ABC') > '414243' > >>> Or >>> 'ABC'.encode('hex_codec') '414243' >>> '414243'.decode('hex_codec') 'ABC' /Bob From aleaxit at yahoo.com Thu Aug 28 10:30:26 2003 From: aleaxit at yahoo.com (Alex Martelli) Date: Thu Aug 28 03:31:04 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: <200308272328.h7RNSkA06766@12-236-84-31.client.attbi.com> References: <5.1.1.6.0.20030827174942.02095030@telecommunity.com> <5.1.1.6.0.20030827181951.02083340@telecommunity.com> <200308272328.h7RNSkA06766@12-236-84-31.client.attbi.com> Message-ID: <200308280930.26854.aleaxit@yahoo.com> On Thursday 28 August 2003 01:28 am, Guido van Rossum wrote: > > Hmmm... that makes me think of tests... I guess I won't really be able > > to supply a test for the feature, since that would just be testing > > whether the feature is available. > > For interactive stuff like this, absence of unit tests is acceptable. pexpect (or rebuilding some of its functionality on top of raw pty) should allow unit tests of textmode interactive stuff, though -- in theory, at least. Alex From theller at python.net Thu Aug 28 11:09:24 2003 From: theller at python.net (Thomas Heller) Date: Thu Aug 28 04:10:13 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: (Tim Peters's message of "Wed, 27 Aug 2003 00:00:05 -0400") References: Message-ID: <7k4yfbkb.fsf@python.net> "Tim Peters" writes: > [Guido] >> If this isn't done, a problem might be (and this is why this is always >> done on Unix) that if the user who installs Python has more privileges >> than the user who uses Python, the user who uses Python may not be >> able to write the directory containing .pyc files, so they end up >> recompiling all modules each time they are loaded. >> >> I expect this will be more of a problem as typical Windows users and >> installations (e.g. XP) become more security aware, software is >> installed by Administrator, and users don't have Administrator >> privileges. >> >> I guess the way to implement it (and I believe Mark Hammond did indeed >> do this for win32all) is to run Python near the end of the installer >> with the compileall.py script as an argument. >> >> Feeling-quite-the-Windows-XP-expert-lately, > > Cool. If an organization has enough money to afford an administrator who > installs stuff for unprivileged masses, they have enough money to pay Thomas > to make this change <0.5 wink>. While I'm still waiting for the money to come in, I have changed the wise script (but this is still uncommitted) to fire up compileall.py on the standard library, exluding the Lib/test subdir, in both normal and optimized mode. This can be disabled in the advanced install options dialog. It fires up this ugly dos box, but I don't have time nor motivation to write a wise extension only for a progress bar instead - the only other possibility would be to run the compilation with pythonw.exe, and so totally hide it from the user, and run it in the background without waiting for it. If I find time before my vacation, I can try to build an installer beta and upload it to starship (hopefully this works again), if anyone wants to try it out. > Mark once told me he compiles stuff at the end of the win32all install > because generating Windows type libraries can take a long time, and users > griped about feeling the pain of that on first use otherwise. > > That's a reason I can understand . If you can too, then it's more > important to precompile everything needed for IDLE to start up the first > time. Thomas From mwh at python.net Thu Aug 28 13:18:39 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 28 07:18:25 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: <200308280930.26854.aleaxit@yahoo.com> (Alex Martelli's message of "Thu, 28 Aug 2003 09:30:26 +0200") References: <5.1.1.6.0.20030827174942.02095030@telecommunity.com> <5.1.1.6.0.20030827181951.02083340@telecommunity.com> <200308272328.h7RNSkA06766@12-236-84-31.client.attbi.com> <200308280930.26854.aleaxit@yahoo.com> Message-ID: <2mr836xc6o.fsf@starship.python.net> Alex Martelli writes: > On Thursday 28 August 2003 01:28 am, Guido van Rossum wrote: >> > Hmmm... that makes me think of tests... I guess I won't really be able >> > to supply a test for the feature, since that would just be testing >> > whether the feature is available. >> >> For interactive stuff like this, absence of unit tests is acceptable. > > pexpect (or rebuilding some of its functionality on top of raw pty) > should allow unit tests of textmode interactive stuff, though -- in > theory, at least. I suspect this approach would find more out about bugs in the pty implementations of old versions of Solaris than anything to do with Python. Cheers, mwh -- ... but I guess there are some things that are so gross you just have to forget, or it'll destroy something within you. perl is the first such thing I have known. -- Erik Naggum, comp.lang.lisp From mwh at python.net Thu Aug 28 13:21:16 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 28 07:21:01 2003 Subject: [Python-Dev] readline.clear_history() In-Reply-To: <200308272213.h7RMDvh06652@12-236-84-31.client.attbi.com> (Guido van Rossum's message of "Wed, 27 Aug 2003 15:13:57 -0700") References: <5.1.1.6.0.20030827155638.01f05c20@mail.rapidsite.net> <5.1.1.6.0.20030827174942.02095030@telecommunity.com> <200308272213.h7RMDvh06652@12-236-84-31.client.attbi.com> Message-ID: <2mn0duxc2b.fsf@starship.python.net> Guido van Rossum writes: > Probably. I'm personally not so stringent for minor stuff like this > (hey, I added bool() to 2.2.1 :-), but it seems others are > increasingly resisting even minor feature additions in micro releases > (except in the email package :-). The problem with adding things like this is that it makes it harder to port code from *2.3.1 to 2.3* and at least in principle it would be nice if code that works on 2.3.1 works on 2.3 unless it tickles a specific bug that has been fixed. And, come Panther, there are going to be rather a lot of Python 2.3 installs out there. Cheers, mwh -- : Giant screaming pieces of excrement, they are. I have a feeling that some of the people in here have a MUCH more exciting time relieving themselves than I do. -- Mike Sphar & Dave Brown, asr From mwh at python.net Thu Aug 28 13:22:23 2003 From: mwh at python.net (Michael Hudson) Date: Thu Aug 28 07:22:07 2003 Subject: [Python-Dev] Archive flooding In-Reply-To: <1062018297.19312.2.camel@yyz> (Barry Warsaw's message of "27 Aug 2003 17:05:03 -0400") References: <1062018297.19312.2.camel@yyz> Message-ID: <2misoixc0g.fsf@starship.python.net> Barry Warsaw writes: > On Wed, 2003-08-27 at 16:59, Terry Reedy wrote: >> In the past 24 hours, gmane.comp.python.devel was flooded with about >> 20000 py-dev posts from the last 3 years. (Is this the whole >> archive?) Fortunately, they had the original posting dates so >> current stuff could be sorted out and the rest 'catchup'ed away. >> >> Just curious -- did these come from or go to the mailing list (and >> people mailboxes) also? (If the latter, a plus for the news gateway.) >> Is this an effect of the current virus storm or just a strange >> coincidence? > > Over in list-owners there was a request to Gmane-ify all our public > lists that weren't currently archived over there. There was general > consensus that this was a good thing, so I guess the process has > started. I'm sure these came from the lists' .mbox archive files. Further to this, http://gmane.org/import.php might be interesting reading. Cheers, mwh -- I think perhaps we should have electoral collages and construct our representatives entirely of little bits of cloth and papier mache. -- Owen Dunn, ucam.chat, from his review of the year From John.McLaughlin at reachin.se Thu Aug 28 16:10:54 2003 From: John.McLaughlin at reachin.se (John McLaughlin) Date: Thu Aug 28 08:58:11 2003 Subject: [Python-Dev] Enhancement suggestion for module new.py Message-ID: <87DAC3BFA900D41199FF00E018A0042F652332@HOBBES> Hi all, I hope this is the right place to make this type of suggestion. I put together the following two functions and have found them very useful. I think they probably belong in the new.py module along with instancemethod(), code() and function() that they rely on. Any thoughts? Cheers, -John import new from types import ClassType def installmethod( function, object, name = None ): """ This function adds either a bound method to an instance or an unbound method to a class. If name is ommited it defaults to the name of the given function. Example: a = A() def f( self, x, y ): self.z = x + y installmethod( f, A, "add" ) a.add( 2, 4 ) print a.z installmethod( lambda self, i: self.l[i], a, "listIndex" ) print a.listIndex( 5 ) """ if name == None: name = function.func_name else: function = renamefunction( function, name ) if type(object) == ClassType: setattr( object, name, new.instancemethod( function, None, object ) ) else: setattr( object, name, new.instancemethod( function, object, object.__class__ ) ) def renamefunction( function, name ): """ This function returns a function identical to the given one, but with the given name. """ c = function.func_code if c.co_name != name: # rename the code object. c = new.code( c.co_argcount, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code, c.co_consts, c.co_names, c.co_varnames, c.co_filename, name, c.co_firstlineno, c.co_lnotab ) if function.func_defaults != None: return new.function( c, function.func_globals, name, function.func_defaults ) return new.function( c, function.func_globals, name ) if __name__ == '__main__': # example class A: def installButton( self, name ): def f( self, name=name ): print name, 'pressed' installmethod( f, self, name ) a = A() a.installButton( 'foo' ) a.foo() # A().foo() raises AttributeError def f( self, x, y ): self.z = x + y installmethod( f, A, "add" ) a.add( 2, 4 ) print a.z installmethod( lambda self, i: self.l[i], A, "listIndex" ) a.l = list( "abc" ) print a.listIndex( 1 ) From mfb at lotusland.dyndns.org Thu Aug 28 09:20:06 2003 From: mfb at lotusland.dyndns.org (Matthew F. Barnes) Date: Thu Aug 28 09:31:59 2003 Subject: [Python-Dev] Extending hex() and oct() to accept strings In-Reply-To: References: <3706.130.76.32.145.1062025775.squirrel@server.lotusland.dyndns.org><2 00308280352.h7S3q5P07084@12-236-84-31.client.attbi.com> Message-ID: <1528.192.168.1.1.1062076806.squirrel@server.lotusland.dyndns.org> >> >>> import binascii >> >>> binascii.hexlify('ABC') >> '414243' >> >>> > > Or > > >>> 'ABC'.encode('hex_codec') > '414243' > > >>> '414243'.decode('hex_codec') > 'ABC' The only defense I can think to offer for the hex() and oct() idea is that I felt like there was something significant about the fact that it was the first thing I thought to try. It seemed like the "obvious" way to do it. While both of these solutions are quite readable now that I see them, I'm not sure that I'd call either of them obvious. However, I can certainly appreciate the cost of adding bells and whistles to the language, especially for something that would probably not see much use (as Guido pointed out). Thank you both for the tips. Matthew Barnes From guido at python.org Thu Aug 28 09:02:40 2003 From: guido at python.org (Guido van Rossum) Date: Thu Aug 28 11:03:01 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: Your message of "Thu, 28 Aug 2003 10:09:24 +0200." <7k4yfbkb.fsf@python.net> References: <7k4yfbkb.fsf@python.net> Message-ID: <200308281502.h7SF2ef08015@12-236-84-31.client.attbi.com> > While I'm still waiting for the money to come in, I have changed the > wise script (but this is still uncommitted) to fire up compileall.py > on the standard library, exluding the Lib/test subdir, in both normal > and optimized mode. This can be disabled in the advanced install > options dialog. > It fires up this ugly dos box, but I don't have time nor motivation to > write a wise extension only for a progress bar instead - the only other > possibility would be to run the compilation with pythonw.exe, and so > totally hide it from the user, and run it in the background without > waiting for it. That would be bad I think. Personally, I can live with the DOS box, although I wonder if it would scare Tim's sisters. Maybe you can make this another compilation option, one that is off by default? > If I find time before my vacation, I can try to build an installer beta > and upload it to starship (hopefully this works again), if anyone wants > to try it out. I'd love to, so please do. --Guido van Rossum (home page: http://www.python.org/~guido/) From theller at python.net Thu Aug 28 22:13:58 2003 From: theller at python.net (Thomas Heller) Date: Thu Aug 28 15:14:37 2003 Subject: HTMLHelp for Py2.3.1 (Was: Re: [Python-Dev] Py2.3.1) In-Reply-To: (Tim Peters's message of "Tue, 26 Aug 2003 13:32:00 -0400") References: Message-ID: "Tim Peters" writes: > [Thomas Heller] >> Sure. I just wanted to mention that switching to .chm doesn't >> decrease the download size for the installer exe. > > So it's pretty mucn neutral by that measure. After installation, it saves a > lot of space (the 1000+ HTML files expand to > 12MB now). > >>> A fully indexed .chm file is much better for searching than pydoc. >>> Sounds like a case of lose-a-little, gain-a-lot. > >> Ok, should this already go into 2.3.1? > > I think so. Note that IDLE may need a bit of fiddling, to keep its Help -> > Python Docs menu entry working. What is the purpose of this registry entry HKLM\Software\Python\PythonCore\2.3\Help\Main Python Documentation ? Thomas From tim.one at comcast.net Thu Aug 28 16:23:35 2003 From: tim.one at comcast.net (Tim Peters) Date: Thu Aug 28 15:24:42 2003 Subject: HTMLHelp for Py2.3.1 (Was: Re: [Python-Dev] Py2.3.1) In-Reply-To: Message-ID: [Thomas Heller] > What is the purpose of this registry entry > > HKLM\Software\Python\PythonCore\2.3\Help\Main Python Documentation ? I don't know. Core Python doesn't use the registry for anything in normal operation, so I'm copying Mark Hammond (who probably set this key way back when for his own inscrutable reasons). From theller at python.net Thu Aug 28 22:43:12 2003 From: theller at python.net (Thomas Heller) Date: Thu Aug 28 15:44:23 2003 Subject: [Python-Dev] Re: HTMLHelp for Py2.3.1 In-Reply-To: (Tim Peters's message of "Thu, 28 Aug 2003 15:23:35 -0400") References: Message-ID: "Tim Peters" writes: > [Thomas Heller] >> What is the purpose of this registry entry >> >> HKLM\Software\Python\PythonCore\2.3\Help\Main Python Documentation ? > > I don't know. Core Python doesn't use the registry for anything in normal > operation, so I'm copying Mark Hammond (who probably set this key way back > when for his own inscrutable reasons). It seems PythonWin uses this - it has a menu entry for 'Python Manuals'. And it works correctly with Python23.chm. Thomas From theller at python.net Thu Aug 28 23:21:38 2003 From: theller at python.net (Thomas Heller) Date: Thu Aug 28 16:22:17 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: <200308281502.h7SF2ef08015@12-236-84-31.client.attbi.com> (Guido van Rossum's message of "Thu, 28 Aug 2003 08:02:40 -0700") References: <7k4yfbkb.fsf@python.net> <200308281502.h7SF2ef08015@12-236-84-31.client.attbi.com> Message-ID: Guido van Rossum writes: >> If I find time before my vacation, I can try to build an installer beta >> and upload it to starship (hopefully this works again), if anyone wants >> to try it out. > > I'd love to, so please do. > Temporary download url: http://starship.python.net/crew/theller/python-2.3.1-beta1.exe This is built from CVS 23maint branch, some hours ago. The uncommited changes I made are here, together with additional comments: http://www.python.org/sf/796919 Only slightly tested so far. I'm nearly off for vacation, until Sept, 15. Thomas From mhammond at skippinet.com.au Fri Aug 29 14:56:12 2003 From: mhammond at skippinet.com.au (Mark Hammond) Date: Thu Aug 28 23:56:25 2003 Subject: HTMLHelp for Py2.3.1 (Was: Re: [Python-Dev] Py2.3.1) In-Reply-To: Message-ID: <03a201c36de1$7cc4b040$f502a8c0@eden> > [Thomas Heller] > > What is the purpose of this registry entry > > > > HKLM\Software\Python\PythonCore\2.3\Help\Main Python Documentation ? > > I don't know. Core Python doesn't use the registry for > anything in normal > operation, so I'm copying Mark Hammond (who probably set this > key way back > when for his own inscrutable reasons). Yep - it is just there so tools can locate all installed Python documentation. Pythonwin grows its help list based on this, and lists its .chm there. I have idea if IDLE uses it. Mark. From aahz at pythoncraft.com Fri Aug 29 01:13:01 2003 From: aahz at pythoncraft.com (Aahz) Date: Fri Aug 29 00:13:05 2003 Subject: [Python-Dev] Enhancement suggestion for module new.py In-Reply-To: <87DAC3BFA900D41199FF00E018A0042F652332@HOBBES> References: <87DAC3BFA900D41199FF00E018A0042F652332@HOBBES> Message-ID: <20030829041301.GA1103@panix.com> On Thu, Aug 28, 2003, John McLaughlin wrote: > > I hope this is the right place to make this type of suggestion. I > put together the following two functions and have found them very > useful. I think they probably belong in the new.py module along > with instancemethod(), code() and function() that they rely on. Any > thoughts? It's the right place for discussion, but if you want your suggestion remembered, you'll need to submit a SourceForge patch. -- Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/ This is Python. We don't care much about theory, except where it intersects with useful practice. --Aahz From spamController at cashette.com Fri Aug 29 05:40:12 2003 From: spamController at cashette.com (Spam Controller) Date: Fri Aug 29 04:40:45 2003 Subject: [Python-Dev] Failed to deliver mail to jin@cashette.com Message-ID: <22010742.1062146412775.JavaMail.root@ns4> Delivery error: jin@cashette.com is not a valid Cashette account. __________________________ Tired of spam? Get your own free, spam-controlled email account at http://www.cashette.com. You get paid if spammed! From tzot at sil-tec.gr Fri Aug 29 14:22:30 2003 From: tzot at sil-tec.gr (Christos Georgiou) Date: Fri Aug 29 06:23:05 2003 Subject: [Python-Dev] Unassigned SF bugs that could be closed Message-ID: <20030829102230.E067A4635D@artemis.sil-tec.gr> 794658 os.chmod docs don't say that S_IRUSR, etc, are in stat modul - A patch has been offered 793687 runnig thread multiple times - Seems not a bug, just lack of documentation study 788457 Cannot decode/encode anything! - Incomplete grasp of unicode concepts by submitter 782752 ImportError: cannot import name __all__ - make failed to build sre, so it's happened only once These for now. I believe this is my first message to python-dev; I generally have abstained from posting (although I lurk for quite some time), because my free time is very little (and not on a regular schedule) and my contributions come in sporadic bursts, so I can't help as much as I would like to. This message is related to Raymond's call for help (otherwise there would be only comments in SF). Cheers everybody :) From administrator at vt.edu Fri Aug 29 11:13:44 2003 From: administrator at vt.edu (administrator@vt.edu) Date: Fri Aug 29 11:04:55 2003 Subject: [Python-Dev] Virus Warning Message-ID: <200308291413.BOZ13364@vivi.cc.vt.edu> The message you emailed to rwgrubbs@vt.edu, dated 08/29/03 10:13:43, contains the W32/Sobig-F virus in the application.pif attachment. The action taken was: deleted the attachment. From tjreedy at udel.edu Fri Aug 29 17:42:56 2003 From: tjreedy at udel.edu (Terry Reedy) Date: Fri Aug 29 16:43:10 2003 Subject: [Python-Dev] Re: Unassigned SF bugs that could be closed References: <20030829102230.E067A4635D@artemis.sil-tec.gr> Message-ID: "Christos Georgiou" wrote in message news:20030829102230.E067A4635D@artemis.sil-tec.gr... > 794658 > os.chmod docs don't say that S_IRUSR, etc, are in stat modul > - A patch has been offered etc > I believe this is my first message to python-dev; I generally > have abstained from posting (although I lurk for quite some > time), because my free time is very little (and not on a > regular schedule) and my contributions come in sporadic > bursts, so I can't help as much as I would like to. > > This message is related to Raymond's call for help (otherwise > there would be only comments in SF). Bugs can be closed by the either priviliged (and busy) developers or by the original submitter. Any comments added are automatically sent to the submitter. When I am sure a bug should be closed by the submitter (without further action), I have occasionally added a diplomatic note explaining why and with a request something like "Unless you disagree, could you close this? I can't." When you feel bold enough, you could try this also. Terry J. Reedy From michael at millenniumshakespeare.com Fri Aug 29 20:43:02 2003 From: michael at millenniumshakespeare.com (Michael) Date: Fri Aug 29 22:43:46 2003 Subject: [Python-Dev] Millennium Shakespeare for Children Message-ID: <3a7$dqzvy$rb68xh4bvuwlzz9jnkr$> An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20030829/e5d844fa/attachment.htm From michael at millenniumshakespeare.com Fri Aug 29 21:57:15 2003 From: michael at millenniumshakespeare.com (Michael) Date: Fri Aug 29 23:57:02 2003 Subject: [Python-Dev] Millennium Shakespeare for Children Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20030829/a85843c7/attachment.htm From michael at millenniumshakespeare.com Sat Aug 30 10:40:50 2003 From: michael at millenniumshakespeare.com (Michael) Date: Sat Aug 30 12:41:04 2003 Subject: [Python-Dev] Millennium Shakespeare for Children Message-ID: An HTML attachment was scrubbed... URL: http://mail.python.org/pipermail/python-dev/attachments/20030830/00d1465b/attachment.htm From martin at v.loewis.de Sun Aug 31 01:18:33 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sat Aug 30 20:18:35 2003 Subject: [Python-Dev] hook for standalone executable In-Reply-To: References: <000701c35e37$e6145410$f502a8c0@eden> <16181.5523.50443.755048@montanaro.dyndns.org> <3F351C02.4090903@v.loewis.de> Message-ID: Thomas Heller writes: > Isn't it possible to parse the executable file format, looking for the > location where the Py_EmbeddedZipfile variable is located, and change > it's value? That's exactly what I meant, yes. > That's what I understand when you say 'patch', and I > wouldn't call this 'linking'. Well, a linker does nothing more... parse the executable file format (potentially given as object files instead of shared libraries or executables), find locations of variables, and create a new output file where the variables have been "patched"... I wasn't really asking for a full-featured linker. Regards, Martin From martin at v.loewis.de Sun Aug 31 01:23:43 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sat Aug 30 20:23:44 2003 Subject: [Python-Dev] python 2.3 bug patch In-Reply-To: <16183.43899.846057.884164@montanaro.dyndns.org> References: <20030811135829.GA8249@turtle.science-computing.de> <16183.43899.846057.884164@montanaro.dyndns.org> Message-ID: Skip Montanaro writes: > I'd prefer to clutter up the configure script instead of the C code. > (Martin v. L?wis may have a comment on that though.) I'm completely with you, here. If configure can find out things, it should. Regards, Martin From martin at v.loewis.de Sun Aug 31 01:27:26 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sat Aug 30 20:27:27 2003 Subject: [Python-Dev] pyconfig.h installation issue In-Reply-To: <16185.14955.742957.657483@magrathea.basistech.com> References: <16185.14955.742957.657483@magrathea.basistech.com> Message-ID: Tom Emerson writes: > This begs another install target, which just installs the items that > end up in the exec-prefix directories. Then on each platform I can > just install the platform-specific code. > > Thoughts? It would be best if you could submit a complete patch to sourceforge. I thought I could follow your explanations, but towards the end, you started confusing me - actual code would have been easier to follow. Regards, Martin From martin at v.loewis.de Sun Aug 31 01:39:38 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sat Aug 30 20:39:39 2003 Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST] In-Reply-To: <3F422E81.5000803@daa.com.au> References: <20030814022128.GN3095@async.com.br> <3F422E81.5000803@daa.com.au> Message-ID: James Henstridge writes: > As Christian said, there is code in glib (not to be confused with > glibc: the GNU C library) that could act as a basis for locale > independent float conversion functions in Python. I very much doubt that this statement is true. Are you sure this code supports all the platforms where Python runs? E.g. what about the three (!) different floating point formats on a VAX? > One of the alternatives that some programs use to do locale > independent conversions using code a bit like this: > char *oldlocale = setlocale(LC_NUMERIC, "C"); > num = strtod(string, NULL); > setlocale(LC_NUMERIC, oldlocale); Unfortunately, this is not thread-safe, so it is clearly out of question. > To sum it up, the current status-quo in Python w.r.t. locales is > causing problems for some problems people want to use Python for. It > would be nice to fix this problem. Certainly. However, incorporating glib code is not a solution. *Calling* glib code (where available) might be a solution. Also, the standard C++ library supports multiple concurrent locale objects, so calling *that* (where available) might be an option. Furthermore, the C++ library is often implemented on top of some C-only library, so calling that library would be better, as it would keep the C++ runtime library out of the Python prerequisites. Regards, Martin From martin at v.loewis.de Sun Aug 31 01:41:53 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sat Aug 30 20:41:54 2003 Subject: [spambayes-dev] RE: [Python-Dev] RE: [Spambayes] Question (orpossibly a bug report) In-Reply-To: <20030814023515.GO3095@async.com.br> References: <020901c35236$e5576f10$f502a8c0@eden> <20030814023515.GO3095@async.com.br> Message-ID: Christian Reis writes: > I don't understand this bit. You'd rather use an undocumented API > function than an open source, well-tested, properly licensed set of > functions? Precisely. I don't want to maintain any more floating-point code. Regards, Martin From martin at v.loewis.de Sun Aug 31 02:07:54 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sat Aug 30 21:07:56 2003 Subject: [Python-Dev] Berkeley breakage In-Reply-To: References: Message-ID: "Tim Peters" writes: > I don't have time to dig into this, but assuming the report is correct, how > can we "encourage" Unix weenies to use db-4.1.25? (On Windows, db-4.1.25 is > shipped with the installer.) If the problems with older versions are so > severe, maybe the Python wrapper should do a version check and refuse to run > if it finds an old version? It's not clear to me that the problems are severe. One would have to analyse the problems (in particular, the crashes). Perhaps _bsddb is using the underlying API in a not-really-supported fashion... For example, strange things used to happen if you close the database while still having cursors around. It might be that the observed crashes have similar causes, and that more recent versions of Sleepycat got more robust against misuse. Then, we could fix the problems by making _bsddb more robust in itself. Regards, Martin From tim.one at comcast.net Sat Aug 30 22:25:41 2003 From: tim.one at comcast.net (Tim Peters) Date: Sat Aug 30 21:26:20 2003 Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST] In-Reply-To: Message-ID: [James Henstridge] >> As Christian said, there is code in glib (not to be confused with >> glibc: the GNU C library) that could act as a basis for locale >> independent float conversion functions in Python. [Martin v. L?wis] > I very much doubt that this statement is true. Are you sure this code > supports all the platforms where Python runs? E.g. what about the > three (!) different floating point formats on a VAX? Well, you should look at the patch: it doesn't know anything about internal fp formats -- all conversions are performed in the end by calling the platform C's strtod() or snprintf(). What it does do is: 1. For string to double, preprocess the input string to change it to use current-locale spelling before calling the platform C strtod(). 2. For double to string, postprocess the result of the platform C snprintf() to replace current-locale spelling with a "standard" spelling. So this is much more string-munging code than it is floating-point code. Indeed, there doesn't appear to be a single floating-point operation in the entire patch (apart from calls to platform string<->float functions). OTOH, despite the claims, it doesn't look threadsafe to me: there's no guarantee, e.g., that the idea of current locale g_ascii_strtod() obtains from localeconv() at its start is still in effect by the time g_ascii_strtod() gets around to calling strtod(). So at best it solves part of one relevant problem here (other relevant problems include that platform C libraries disagree about how to spell infinities, NaNs and signed zeroes; about how many digits to use for an exponent; and about how to round results (for example, >>> "%.1f" % 2.25 '2.3' >>> on Windows, but most (not all!) flavors of Unix produce the IEEE-754 to-nearest/even rounded '2.2' instead)). It's easy to write portable, perfectly-rounding string<->double conversion routines without calling any platform functions. The rub is that "fast" goes out the window then, unless you give up at least one of {portable, accurate}. From tim.one at comcast.net Sat Aug 30 23:29:35 2003 From: tim.one at comcast.net (Tim Peters) Date: Sat Aug 30 22:30:09 2003 Subject: [Python-Dev] Windows installer and .pyc files mtime In-Reply-To: Message-ID: [Thomas Heller] > Temporary download url: > > http://starship.python.net/crew/theller/python-2.3.1-beta1.exe > > This is built from CVS 23maint branch, some hours ago. The uncommited > changes I made are here, together with additional comments: > > http://www.python.org/sf/796919 > > Only slightly tested so far. Slightly more now, on Win98SE. Cool! I thought something was wrong because the install on Win98SE went faster than I was used to. Turns out that's just because we're installing 1,200+ fewer files than we used to (thanks to shipping all the HTML docs in a single .chm file). Everything I tried worked fine. I think the *default* should be not to compile .pyc files. Regardless of the default, popping up a DOS box during compilation doesn't bother me at all. From martin at v.loewis.de Sun Aug 31 04:03:44 2003 From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) Date: Sat Aug 30 23:03:46 2003 Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST] In-Reply-To: References: Message-ID: "Tim Peters" writes: > 1. For string to double, preprocess the input string to change it to > use current-locale spelling before calling the platform C strtod(). I see (I was confused by the presence of a table of bytes). This is much worse, then: How can it possibly know what formats the C library expects in the current locale? What if the C library insists that a thousands-separator is used when the locale has one? etc. Regards, Martin From skip at mojam.com Sun Aug 31 08:00:50 2003 From: skip at mojam.com (Skip Montanaro) Date: Sun Aug 31 08:00:58 2003 Subject: [Python-Dev] Weekly Python Bug/Patch Summary Message-ID: <200308311200.h7VC0oH05583@manatee.mojam.com> Bug/Patch Summary ----------------- 470 open / 4073 total bugs (+3) 187 open / 2342 total patches (+4) New Bugs -------- Cannot retrieve name of super object (2003-04-28) http://python.org/sf/729103 cygwin builds do not embed (2003-08-24) http://python.org/sf/794140 double symlinking corrupts sys.path[0] (2003-08-24) http://python.org/sf/794291 email bug with message/rfc822 (2003-08-24) http://python.org/sf/794458 email module param parsing bug (2003-08-24) http://python.org/sf/794466 Named groups limitation in sre (2003-08-25) http://python.org/sf/794819 email.Message param parsing problem II (2003-08-25) http://python.org/sf/795081 Documentation (2003-08-26) http://python.org/sf/795649 sdist ignores scripts argument in setup (2003-08-27) http://python.org/sf/796042 Parser wart (2003-08-27) http://python.org/sf/796116 ntpath.expanduser() is still wrong (2003-08-27) http://python.org/sf/796219 locale.setlocale(locale.LC_ALL, "de") raises exception (2003-08-29) http://python.org/sf/797447 csv.DictWriter -- inconsistent doc & code (2003-08-30) http://python.org/sf/797844 Small problems with the csv module's documentation (2003-08-30) http://python.org/sf/797853 select module doesn't allow any iterable. (2003-08-31) http://python.org/sf/798046 IDLE / PyOS_InputHook (2003-08-31) http://python.org/sf/798058 New Patches ----------- startup file compiler flags (2003-08-24) http://python.org/sf/794400 __file__ attribute missing from dynamicly loaded module (2003-08-25) http://python.org/sf/794826 license inconsistency (2003-08-26) http://python.org/sf/795531 CGIHTTPServer fix (2003-08-28) http://python.org/sf/796772 decode message attachments in email.Message (2003-08-28) http://python.org/sf/796908 Windows installer changes for 2.3.1 (2003-08-28) http://python.org/sf/796919 Closed Bugs ----------- An extended definition of "non-overlapping" would save time. (2003-05-04) http://python.org/sf/732120 urllib: open_https generates wrong Authorize header (2003-07-23) http://python.org/sf/776542 Bug in "Build and C API Changes" section of what's new doc? (2003-07-30) http://python.org/sf/780231 ImportError: cannot import name __all__ (2003-08-04) http://python.org/sf/782752 Fatal Python error: unknown scope (2003-08-06) http://python.org/sf/784075 zlib monotonic test -- false assumptions or bug in zlib.lib (2003-08-08) http://python.org/sf/785222 compiler.compileFile fails on csv.py (2003-08-13) http://python.org/sf/788011 Cannot decode/encode anything! (2003-08-13) http://python.org/sf/788457 curses.ascii.unctrl broken (2003-08-17) http://python.org/sf/790356 Tutorial: mutable objects may be keys (2003-08-19) http://python.org/sf/791397 Re: Bugs item #765456 (2003-08-19) http://python.org/sf/791445 Misleading [i:j:k] slicing description (2003-08-21) http://python.org/sf/792656 runnig thread multiple times (2003-08-23) http://python.org/sf/793687 Section 13.1 HTMLParser documentation error (2003-08-23) http://python.org/sf/793702 using itertools.izip to mutate tuples (2003-08-23) http://python.org/sf/793826 Closed Patches -------------- Port tests to unittest (Part 2) (2003-05-13) http://python.org/sf/736962 test_largefile cleanup patch (2003-08-11) http://python.org/sf/786591 From tim.one at comcast.net Sun Aug 31 23:47:34 2003 From: tim.one at comcast.net (Tim Peters) Date: Sun Aug 31 22:48:12 2003 Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST] In-Reply-To: Message-ID: [Tim, about what the patch does] >> ... >> 1. For string to double, preprocess the input string to change it to >> use current-locale spelling before calling the platform C >> strtod(). [Martin] > I see (I was confused by the presence of a table of bytes). Right, the table appears to be there just to support locale-independent character classification. > This is much worse, then: How can it possibly know what formats the C > library expects in the current locale? What if the C library insists that > a thousands-separator is used when the locale has one? etc. I'm not sure that's a realistic objection. The patch appears to be trying to replace only the decimal point (if any), with localeconv()->decimal_point, and I've certainly not seen a locale that refuses to accept, e.g., 1234 5678 meaning the same as 1234.5678 in "C" locale. The draft C99 standard I have handy here says (in its strtod() section): In other than the "C" locale, additional locale-specific subject sequence forms may be accepted. and "additional" implies to me that every locale must accept at least the basic floating-point spellings described before that quoted sentence.