writes:
Jack> On 7-sep-03, at 21:15, Martin v. L?wis wrote:
>>> Maintenance branches are for bug fixes
>>> only;
>>
>> This question is heavily debated. Alex Martelli, for example, favours
>> a policy where new (in a strict sense) features are acceptable if they
>> don't break anything, and are "minor", in some sense.
Jack> ... but with Python 2.3 we have the situation that the majority of
Jack> Python installations (I think it's safe to take the guess that
Jack> MacOSX 10.3 installations will soon outnumber all other Pythons
Jack> together) will stay at 2.3 until the next release of MacOSX.
I wouldn't be so sure of that. Aside from the many Linux distributions
which include some version of Python, it's clear that HP/Compaq is including
some version of Python with their new Windows computers (based on all the
newbie questions about this fielded at webmaster@python.org). Still, the
number of vendors delivering Python with their computers and the long string
of problems caused by RedHat shipping Python 1.5.2 w/ 7.x long after 2.0 was
history suggests that micro releases be reserved entirely for bug fixes.
Skip
From barry at python.org Mon Sep 8 14:18:30 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 8 09:18:32 2003
Subject: [Python-Dev] Documenting branch policy
In-Reply-To: <20030907161900.GA3195@cthulhu.gerg.ca>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
Message-ID: <1063027104.16344.9.camel@anthem>
On Sun, 2003-09-07 at 12:19, Greg Ward wrote:
> Release branches
>
> Every release up to and including the final release for a new major
> Python version is accompanied by a release branches.
There's even some debate about these (I've caught up with the thread, so
I know you've withdrawn this change). I've been backing off the use of
release branches because they create more complexity in a world where
none of us have much time to deal with it. For 2.3, python-dev was
really good about heeding calls for checkin freezes, and the time
between wanting to cut the release and actually doing it was pretty
short, so I don't see much need for release branches. I updated PEP 101
to reflect this.
-Barry
From harri.pasanen at trema.com Mon Sep 8 16:39:56 2003
From: harri.pasanen at trema.com (Harri Pasanen)
Date: Mon Sep 8 09:40:07 2003
Subject: [Python-Dev] New functionality in micro releases (was:
=?iso-8859-1?q?Documenting branch?= policy)
In-Reply-To: <16219.47489.553300.299720@montanaro.dyndns.org>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16219.47489.553300.299720@montanaro.dyndns.org>
Message-ID: <200309081539.56700.harri.pasanen@trema.com>
On Monday 08 September 2003 01:04, Skip Montanaro wrote:
> >>>>> "Jack" == Jack Jansen writes:
>
> Jack> ... but with Python 2.3 we have the situation that the
> majority of Jack> Python installations (I think it's safe to take
> the guess that Jack> MacOSX 10.3 installations will soon outnumber
> all other Pythons Jack> together) will stay at 2.3 until the next
> release of MacOSX.
>
> I wouldn't be so sure of that. Aside from the many Linux
> distributions which include some version of Python, it's clear that
> HP/Compaq is including some version of Python with their new
> Windows computers (based on all the newbie questions about this
> fielded at webmaster@python.org). Still, the number of vendors
> delivering Python with their computers and the long string of
> problems caused by RedHat shipping Python 1.5.2 w/ 7.x long after
> 2.0 was history suggests that micro releases be reserved entirely
> for bug fixes.
>
Seconded, from where I'm sitting MaxOSX is not even on the radar, but
Linux is all over the place, on various platforms, and all those we
use have Python out of the box. RedHat on Itanium still seems to
have 1.5.2 for instance. Mandrake 9.1 is on 2.2.2.
Mandrake 9.2 will have Python 2.3.
Personally I would have guessed there to be more Mandrake Linux
installations than MacOSX installations - but in reality I have no
idea.
Just a datapoint...
Harri
From guido at python.org Mon Sep 8 07:59:02 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep 8 09:59:21 2003
Subject: [Python-Dev] New functionality in micro releases (was:
Documenting branch policy)
In-Reply-To: Your message of "Sun, 07 Sep 2003 18:04:33 CDT."
<16219.47489.553300.299720@montanaro.dyndns.org>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16219.47489.553300.299720@montanaro.dyndns.org>
Message-ID: <200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>
> Jack> ... but with Python 2.3 we have the situation that the majority of
> Jack> Python installations (I think it's safe to take the guess that
> Jack> MacOSX 10.3 installations will soon outnumber all other Pythons
> Jack> together) will stay at 2.3 until the next release of MacOSX.
[Skip]
> I wouldn't be so sure of that. Aside from the many Linux distributions
> which include some version of Python, it's clear that HP/Compaq is including
> some version of Python with their new Windows computers (based on all the
> newbie questions about this fielded at webmaster@python.org). Still, the
> number of vendors delivering Python with their computers and the long string
> of problems caused by RedHat shipping Python 1.5.2 w/ 7.x long after 2.0 was
> history suggests that micro releases be reserved entirely for bug fixes.
I agree with Jack & Just that MacOSX is a strong argument for keeping
new features, no matter how harmless, out of micro releases. But I'm
not sure how Red Hat shipping 1.5.2 proves this point; there were no
micro releases of 1.5 after that.
--Guido van Rossum (home page: http://www.python.org/~guido/)
From skip at pobox.com Mon Sep 8 10:09:55 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep 8 10:10:20 2003
Subject: [Python-Dev] New functionality in micro releases (was:
Documenting branch policy)
In-Reply-To: <200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16219.47489.553300.299720@montanaro.dyndns.org>
<200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>
Message-ID: <16220.36275.971661.113208@montanaro.dyndns.org>
Guido> But I'm not sure how Red Hat shipping 1.5.2 proves this point;
Guido> there were no micro releases of 1.5 after that.
True, but it demonstrates how much agony changes to the language -
regardless of the versioning - can cause.
Skip
From guido at python.org Mon Sep 8 08:19:48 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep 8 10:20:16 2003
Subject: [Python-Dev] New functionality in micro releases (was:
Documenting branch policy)
In-Reply-To: Your message of "Mon, 08 Sep 2003 09:09:55 CDT."
<16220.36275.971661.113208@montanaro.dyndns.org>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16219.47489.553300.299720@montanaro.dyndns.org>
<200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>
<16220.36275.971661.113208@montanaro.dyndns.org>
Message-ID: <200309081419.h88EJmA11895@12-236-84-31.client.attbi.com>
> Guido> But I'm not sure how Red Hat shipping 1.5.2 proves this point;
> Guido> there were no micro releases of 1.5 after that.
>
> True, but it demonstrates how much agony changes to the language -
> regardless of the versioning - can cause.
>
> Skip
Then that's an argument against changes to the language in general.
We've had that argument, and the resulting policy is specified in PEPs
4 and 5 (amongst others).
--Guido van Rossum (home page: http://www.python.org/~guido/)
From skip at pobox.com Mon Sep 8 11:08:30 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep 8 11:08:35 2003
Subject: [Python-Dev] New functionality in micro releases (was:
Documenting branch policy)
In-Reply-To: <200309081419.h88EJmA11895@12-236-84-31.client.attbi.com>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16219.47489.553300.299720@montanaro.dyndns.org>
<200309081359.h88Dx2G11829@12-236-84-31.client.attbi.com>
<16220.36275.971661.113208@montanaro.dyndns.org>
<200309081419.h88EJmA11895@12-236-84-31.client.attbi.com>
Message-ID: <16220.39790.575212.507229@montanaro.dyndns.org>
>> True, but it demonstrates how much agony changes to the language -
>> regardless of the versioning - can cause.
Guido> Then that's an argument against changes to the language in
Guido> general. We've had that argument, and the resulting policy is
Guido> specified in PEPs 4 and 5 (amongst others).
I wasn't trying to suggest the language shouldn't evolve, just that ...
Oh, never mind. I'm sure you actually know what I meant.
S
From fdrake at acm.org Mon Sep 8 14:27:52 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Sep 8 13:28:02 2003
Subject: [Python-Dev] New functionality in micro releases (was:
Documenting branch policy)
In-Reply-To:
References: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
Message-ID: <16220.48152.685701.574897@grendel.zope.com>
Just van Rossum writes:
> It's been a huge pain with 2.2 vs. 2.2.1 even: all stuff written for
> 2.2.1 that uses booleans does _not_ work on plain 2.2, which is what's
No code written for 2.2.1 should be using bool(), True, or False;
those were not documented in 2.2.1 *at all* as far as I can tell. The
description of bool() was added to the docs in 2.2.3, and I'm not sure
why, though it is noted that it was added in 2.2.1. I'd be happy to
rip it out of the documentation, or add a stronger compatibility
warning, for 2.2.4.
> installed on OSX 10.2. Which sortof kills the advantage of having a
> Python installed with the OS to begin with. In retrospect, I would have
> been strongly against adding booleans to 2.2.1 for this reason. Apart
> from real bugs, I think every program that works on 2.x.y should work on
> 2.x.z, regardless of whether y > z or y < z.
Depending on just what you mean "every program that works", this could
completely prevent these bugfix releases. We might not be able to
remove a core dump since it would allow code to run that was not run
before, thereby changing the behavior of the code.
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From python at rcn.com Mon Sep 8 19:29:10 2003
From: python at rcn.com (Raymond Hettinger)
Date: Mon Sep 8 18:33:50 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
branch policy)
References: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16220.48152.685701.574897@grendel.zope.com>
Message-ID: <007201c37658$a077a0e0$a511c797@oemcomputer>
[Just van Rossum]
> > Apart
> > from real bugs, I think every program that works on 2.x.y should work on
> > 2.x.z, regardless of whether y > z or y < z.
[Fred Drake]
> Depending on just what you mean "every program that works", this could
> completely prevent these bugfix releases. We might not be able to
> remove a core dump since it would allow code to run that was not run
> before, thereby changing the behavior of the code.
JvR did qualify it with: "apart from real bugs". The only question is how
"real" does it have to be. I think all of the following should be fair game:
* not matching documented behavior (for example: '%F" % x)
* usability fixes
* performance issues (strptime caching for example)
* something that can only be crashed by an obscure case
(Armin Rigo's itertools.izip hack)
* leaks (the massive leak in the array module)
* security fixes
* enhancing the docs (chm files, examples, new docs, etc.)
* adding capabilities to standalone Tools/scripts or IDLE
* updates to separately maintained packages (like "email")
so that between major releases, Python won't get terribly
far behind the stable releases of the package.
I think of micro releases as being the same as service patches which are
supposed to be painless upgrades that any admin or user can load without
fear of breaking existing code. That should likely be the standard rather
than requiring that all programs work irrespective of the micro version.
If an application is affected by a bug in an earlier micro version, it
isn't terribly unreasonable to ask the users to download the service patch
which should be relatively effortless as compared with installing a new
major release.
Raymond Hettinger
From guido at python.org Mon Sep 8 17:13:33 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Sep 8 19:13:52 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
branch policy)
In-Reply-To: Your message of "Mon, 08 Sep 2003 18:29:10 EDT."
<007201c37658$a077a0e0$a511c797@oemcomputer>
References: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16220.48152.685701.574897@grendel.zope.com>
<007201c37658$a077a0e0$a511c797@oemcomputer>
Message-ID: <200309082313.h88NDX712738@12-236-84-31.client.attbi.com>
> * not matching documented behavior (for example: '%F" % x)
Depends on how you fix these. Fixing the code may break more code
than fixing the docs (assuming the mismatch existed for a long time)!
> * usability fixes
What are these? Seems like a loophole the size of a large truck.
> * performance issues (strptime caching for example)
> * something that can only be crashed by an obscure case
> (Armin Rigo's itertools.izip hack)
> * leaks (the massive leak in the array module)
> * security fixes
But every attempt should be made to fix these without changing APIs.
> * enhancing the docs (chm files, examples, new docs, etc.)
> * adding capabilities to standalone Tools/scripts or IDLE
Questionable, if these are installed. Esp. now that IDLE is part of
the standard library.
> * updates to separately maintained packages (like "email")
> so that between major releases, Python won't get terribly
> far behind the stable releases of the package.
Again, this should be done with the utmost backwards compatibility in
mind, and even then, new features are new features, and thus may cause
the problem Jack and Just warn about.
> I think of micro releases as being the same as service patches which are
> supposed to be painless upgrades that any admin or user can load without
> fear of breaking existing code. That should likely be the standard rather
> than requiring that all programs work irrespective of the micro version.
>
> If an application is affected by a bug in an earlier micro version, it
> isn't terribly unreasonable to ask the users to download the service patch
> which should be relatively effortless as compared with installing a new
> major release.
Right, but keep in mind that few people except hardcore Python users
are going to upgrade their Python, no matter how painless, just to run
some small piece of software they downloaded. They'll just toss the
download as one of so many broken things, proof that free software
sucks.
--Guido van Rossum (home page: http://www.python.org/~guido/)
From pje at telecommunity.com Mon Sep 8 21:02:13 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Sep 8 20:02:24 2003
Subject: [Python-Dev] New functionality in micro releases
(was:Documenting branch policy)
In-Reply-To: <200309082313.h88NDX712738@12-236-84-31.client.attbi.com>
References:
<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16220.48152.685701.574897@grendel.zope.com>
<007201c37658$a077a0e0$a511c797@oemcomputer>
Message-ID: <5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
At 04:13 PM 9/8/03 -0700, Guido van Rossum wrote:
>Right, but keep in mind that few people except hardcore Python users
>are going to upgrade their Python, no matter how painless, just to run
>some small piece of software they downloaded. They'll just toss the
>download as one of so many broken things, proof that free software
>sucks.
I've even seen *developers* do this... Somebody who works in my department
here griped about PEAK not working on their machine and when I went to
troubleshoot it, I found they had a lesser version (2.2.1) of Python than
the docs explicitly required (2.2.2, which fixed a couple of relevant
new-style class bugs). (Makes me think I should put some code in to check
the Python version and exit with a helpful error message.)
It does suggest that being able to specify the version of Python required
by a script or module, would be a helpful idiom. sys.requireversion(),
perhaps? Guess I should make a quick check to be sure you haven't already
used the time machine and put this in... :)
Of course, the sad bit is that even if there were a sys.requireversion(),
it wouldn't be in the versions where it was actually needed: the older ones!
From sholden at holdenweb.com Mon Sep 8 22:06:39 2003
From: sholden at holdenweb.com (Steve Holden)
Date: Tue Sep 9 00:52:25 2003
Subject: [Python-Dev] New functionality in micro releases
(was:Documentingbranch policy)
In-Reply-To: <200309082313.h88NDX712738@12-236-84-31.client.attbi.com>
Message-ID:
[Raymond Hettinger]
> > * not matching documented behavior (for example: '%F" % x)
>
[Guido]
> Depends on how you fix these. Fixing the code may break more code
> than fixing the docs (assuming the mismatch existed for a long time)!
>
Such as the socket mismatch that was fixed in (IIRC) 1.6, to loud
complaints from half the network programming world? This still bites
occasionally when old cord has to be brought forward to more recent
versions (though, of course, it isn't hard to make the necessary
changes).
[...]
> > * updates to separately maintained packages (like "email")
> > so that between major releases, Python won't get terribly
> > far behind the stable releases of the package.
>
> Again, this should be done with the utmost backwards compatibility in
> mind, and even then, new features are new features, and thus may cause
> the problem Jack and Just warn about.
>
Yup. Any potential for code breakage is going to cause somebody
somewhere a certain amount of pain. It's these edge cases that cause
most of the problems, since overall the version-to-version compatibility
picture is actually remarkably good. But people will cheerfully ignore
the 99.99% of their code that *doesn't* break, and bitch about the
little bits that do :-)
> > I think of micro releases as being the same as service
> patches which are
> > supposed to be painless upgrades that any admin or user can
> load without
> > fear of breaking existing code. That should likely be the
> standard rather
> > than requiring that all programs work irrespective of the
> micro version.
> >
> > If an application is affected by a bug in an earlier micro
> version, it
> > isn't terribly unreasonable to ask the users to download
> the service patch
> > which should be relatively effortless as compared with
> installing a new
> > major release.
>
> Right, but keep in mind that few people except hardcore Python users
> are going to upgrade their Python, no matter how painless, just to run
> some small piece of software they downloaded. They'll just toss the
> download as one of so many broken things, proof that free software
> sucks.
>
If you want something to be true sufficiently hard, anything convenient
will be seen as supporting evidence.
Raymond did, however, pinpoint the reason why I haven't yet started
using True and False.
what-*me*-conservative-ly y'rs - steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/pwp/
Interview with GvR August 14, 2003 http://www.onlamp.com/python/
From eppstein at ics.uci.edu Mon Sep 8 18:48:50 2003
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue Sep 9 01:07:48 2003
Subject: [Python-Dev] Re: New functionality in micro releases
(was:Documenting branch policy)
References: <007201c37658$a077a0e0$a511c797@oemcomputer>
<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16220.48152.685701.574897@grendel.zope.com>
<007201c37658$a077a0e0$a511c797@oemcomputer>
<200309082313.h88NDX712738@12-236-84-31.client.attbi.com>
<5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
Message-ID:
In article <5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>,
"Phillip J. Eby" wrote:
> It does suggest that being able to specify the version of Python required
> by a script or module, would be a helpful idiom. sys.requireversion(),
> perhaps? Guess I should make a quick check to be sure you haven't already
> used the time machine and put this in... :)
>
> Of course, the sad bit is that even if there were a sys.requireversion(),
> it wouldn't be in the versions where it was actually needed: the older ones!
Ok, so what's needed isn't a new sys.feature, it's some short
boilerplate code you could include in all your python apps, to make sure
anyone who tries to run it on an old system is informed of the problem.
Something like:
import sys
class PythonTooOld(Exception): pass
if sys.version < '2.2':
raise PythonTooOld, 'This code needs Python 2.2 or greater.'
--
David Eppstein http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science
From aleaxit at yahoo.com Tue Sep 9 08:20:37 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Tue Sep 9 01:20:43 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
branch policy)
In-Reply-To: <5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
References:
<007201c37658$a077a0e0$a511c797@oemcomputer>
<5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
Message-ID: <200309090720.37252.aleaxit@yahoo.com>
On Tuesday 09 September 2003 02:02 am, Phillip J. Eby wrote:
...
> Of course, the sad bit is that even if there were a sys.requireversion(),
> it wouldn't be in the versions where it was actually needed: the older
> ones!
Actually this would "fail-RIGHT": on older versions the program would
die right there with an AttributeException and a traceback about a
line such as sys.requireversion('2.3.2'), which is already a clear
enough indication of exactly what is wrong -- on newer ones with an
even clearer exception, of course. In any case, one could dress up
the sys.requireversion call in a try/except to make its absence no problem.
Alex
From fdrake at acm.org Mon Sep 8 23:37:59 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep 9 01:49:37 2003
Subject: [Python-Dev] New functionality in micro releases
(was:Documenting branch policy)
In-Reply-To: <5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
References:
<5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16220.48152.685701.574897@grendel.zope.com>
<007201c37658$a077a0e0$a511c797@oemcomputer>
<5.1.1.6.0.20030908195222.01f1a860@telecommunity.com>
Message-ID: <16221.15623.111594.33933@grendel.zope.com>
Phillip J. Eby writes:
> It does suggest that being able to specify the version of Python required
> by a script or module, would be a helpful idiom. sys.requireversion(),
> perhaps? Guess I should make a quick check to be sure you haven't already
> used the time machine and put this in... :)
I'm not aware of any such thing.
> Of course, the sad bit is that even if there were a sys.requireversion(),
> it wouldn't be in the versions where it was actually needed: the older ones!
Comparison with sys.version_info would be trivial, and seems
sufficient:
if sys.version_info < (2,2,2):
print "Get a newer version of Python!"
else:
main()
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From mfb at lotusland.dyndns.org Tue Sep 9 02:25:11 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Tue Sep 9 02:38:11 2003
Subject: [Python-Dev] New functionality in micro
releases(was:Documenting branch policy)
In-Reply-To: <16221.15623.111594.33933@grendel.zope.com>
References: <5FE933B2-E171-11D7-A
B7D-000A27B19B96@cwi.nl><16220.48152.685701.574897@grendel.zope.com><007201c376
58$a077a0e0$a511c797@oemcomputer><5.1.1.6.0.20030908195222.01f1a860@te
lecommunity.com> <16221.15623.111594.33933@grendel.zope.com>
Message-ID: <4269.192.168.1.1.1063088711.squirrel@server.lotusland.dyndns.org>
Fred L. Drake, Jr. said:
> Comparison with sys.version_info would be trivial, and seems
> sufficient:
>
> if sys.version_info < (2,2,2):
> print "Get a newer version of Python!"
> else:
> main()
FWIW - I often use something close to this near the top of my own modules
(realizing that it gets skipped when the -O option is provided):
assert sys.version_info >= (2,2,2), 'requires Python 2.2.2 or better'
Matthew Barnes
From Jack.Jansen at cwi.nl Tue Sep 9 12:26:54 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Tue Sep 9 05:27:01 2003
Subject: [Python-Dev] Documenting branch policy
In-Reply-To: <1063027104.16344.9.camel@anthem>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<1063027104.16344.9.camel@anthem>
Message-ID:
On 8-sep-03, at 15:18, Barry Warsaw wrote:
>> Every release up to and including the final release for a new major
>> Python version is accompanied by a release branches.
>
> There's even some debate about these (I've caught up with the thread,
> so
> I know you've withdrawn this change). I've been backing off the use of
> release branches because they create more complexity in a world where
> none of us have much time to deal with it.
They served me well for the MacPython-OS9 releases. But as 2.3 is going
to
be the last of those anyway I could live without them, I guess.
Although: they'd still be useful in case of unforeseen problems with a
distribution that are not code-related. Think of things like the Windows
installer turning out to be broken on certain machine, so you want to
build
the installer again (but not Python itself).
--
Jack Jansen, , http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma
Goldman
From barry at python.org Tue Sep 9 13:09:36 2003
From: barry at python.org (Barry Warsaw)
Date: Tue Sep 9 08:09:37 2003
Subject: [Python-Dev] New functionality in micro releases
(was:Documenting branch policy)
In-Reply-To: <007201c37658$a077a0e0$a511c797@oemcomputer>
References: <5FE933B2-E171-11D7-AB7D-000A27B19B96@cwi.nl>
<16220.48152.685701.574897@grendel.zope.com>
<007201c37658$a077a0e0$a511c797@oemcomputer>
Message-ID: <1063109371.16781.21.camel@anthem>
On Mon, 2003-09-08 at 18:29, Raymond Hettinger wrote:
> * updates to separately maintained packages (like "email")
> so that between major releases, Python won't get terribly
> far behind the stable releases of the package.
You have to take things on a case-by-case basis. I actually agonized
quite a bit about the email package in 2.2, but after discussing things
with Guido and the mimelib community, there was general consensus that
not fixing/upgrading the package was worse than doing so. But I doubt
we'll do the same thing for Python 2.3, because the package is more
stable now, has been integrated longer, and the planned changes are
probably more radical.
This does point out a problem with the batteries-included philosophy
though: it's hard to upgrade packages that come with Python. Say we
release email 3.0 and you want to use it in Python 2.3, what do you do?
You can't just do a distutils install, because site-packages is searched
after the standard library. I think we need more flexibility in
installing distutils packages, so we can install in various locations
that override standard packages (e.g. user-centric, system-centric,
etc.).
In general though, it's a good thing to default to a policy where micro
releases are bug fixes only. Given the reality of the resources
available, time between major releases, and the fact that Python
development is a wholly volunteer effort, some amount of ugliness is
bound to invade the process. That's where the BDFL uses his judgement
to make a decision and we live with it.
Take the True/False thing. OOH, it was a new feature added to a micro
release. OTOH it was backward compatible, helped future compatibility,
and was easily coded around to avoid micro-version skew. Taken as a
whole, and given when the decision was made, it was probably a good one
although it caused a little bit of pain.
-Barry
From barry at python.org Tue Sep 9 13:14:11 2003
From: barry at python.org (Barry Warsaw)
Date: Tue Sep 9 08:14:12 2003
Subject: [Python-Dev] Documenting branch policy
In-Reply-To:
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<1063027104.16344.9.camel@anthem>
Message-ID: <1063109647.16781.24.camel@anthem>
On Tue, 2003-09-09 at 05:26, Jack Jansen wrote:
> On 8-sep-03, at 15:18, Barry Warsaw wrote:
>
> >> Every release up to and including the final release for a new major
> >> Python version is accompanied by a release branches.
> >
> > There's even some debate about these (I've caught up with the thread,
> > so
> > I know you've withdrawn this change). I've been backing off the use of
> > release branches because they create more complexity in a world where
> > none of us have much time to deal with it.
>
> They served me well for the MacPython-OS9 releases. But as 2.3 is going
> to
> be the last of those anyway I could live without them, I guess.
>
> Although: they'd still be useful in case of unforeseen problems with a
> distribution that are not code-related. Think of things like the Windows
> installer turning out to be broken on certain machine, so you want to
> build
> the installer again (but not Python itself).
That's a good point. Release branches are probably overkill for
alpha/beta/rc releases, but probably make sense for final releases. We
actually did create one for 2.3, but I screwed up when I named it and
that caused a tiny bit of pain in moving to the maintenance branch.
OTOH, for a final release, maybe release branch == maintenance branch.
-Barry
From fdrake at acm.org Tue Sep 9 09:16:47 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep 9 08:16:53 2003
Subject: [Pydotorg] Re: [Python-Dev] Documenting branch policy
In-Reply-To: <1063109647.16781.24.camel@anthem>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<1063027104.16344.9.camel@anthem>
<1063109647.16781.24.camel@anthem>
Message-ID: <16221.50351.725375.52522@grendel.zope.com>
Barry Warsaw writes:
> That's a good point. Release branches are probably overkill for
> alpha/beta/rc releases, but probably make sense for final releases. We
> actually did create one for 2.3, but I screwed up when I named it and
> that caused a tiny bit of pain in moving to the maintenance branch.
> OTOH, for a final release, maybe release branch == maintenance branch.
I think that's sufficient. There's no need for them to be separate.
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From gward at python.net Tue Sep 9 09:24:47 2003
From: gward at python.net (Greg Ward)
Date: Tue Sep 9 08:24:53 2003
Subject: [Python-Dev] Documenting branch policy
In-Reply-To: <1063109647.16781.24.camel@anthem>
References: <20030907161900.GA3195@cthulhu.gerg.ca>
<1063027104.16344.9.camel@anthem>
<1063109647.16781.24.camel@anthem>
Message-ID: <20030909122447.GB15042@cthulhu.gerg.ca>
On 09 September 2003, Barry Warsaw said:
> That's a good point. Release branches are probably overkill for
> alpha/beta/rc releases, but probably make sense for final releases. We
> actually did create one for 2.3, but I screwed up when I named it and
> that caused a tiny bit of pain in moving to the maintenance branch.
> OTOH, for a final release, maybe release branch == maintenance branch.
Well, if nothing else this thread has demonstrated that I'm *not* the only
one who's a little confused by the release/maintenance branching policy.
*phew*!
Greg
--
Greg Ward http://www.gerg.ca/
Just because you're paranoid doesn't mean they *aren't* out to get you.
From anthony at interlink.com.au Wed Sep 10 02:18:30 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Sep 9 11:18:52 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
branch policy)
In-Reply-To: <1063109371.16781.21.camel@anthem>
Message-ID: <200309091518.h89FIURh024097@localhost.localdomain>
>>> Barry Warsaw wrote
> This does point out a problem with the batteries-included philosophy
> though: it's hard to upgrade packages that come with Python. Say we
> release email 3.0 and you want to use it in Python 2.3, what do you do?
> You can't just do a distutils install, because site-packages is searched
> after the standard library. I think we need more flexibility in
> installing distutils packages, so we can install in various locations
> that override standard packages (e.g. user-centric, system-centric,
> etc.).
The approach taken by the XML add-on package doesn't seem to be the
right approach, either. At the moment your choices are to manually install
the new code over the old, or else have a separate install area that's
manually added to the start of the PYTHONPATH (this is what mailman
does, right?) Neither of these approaches seem ideal. The latter is what
I do for various systems that need newer versions of packages that are
bundled with python, it'd be nice if there was a better way - but I can't
see one.
Anthony
From tim.one at comcast.net Tue Sep 9 12:25:50 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep 9 11:25:48 2003
Subject: [Python-Dev] New functionality in micro releases
(was:Documentingbranch policy)
In-Reply-To: <200309091518.h89FIURh024097@localhost.localdomain>
Message-ID:
[Barry Warsaw]
> ...
> This does point out a problem with the batteries-included philosophy
I was still basically asleep when I saw this msg, and my bleary eyes first
read that as "barriers-included".
> though: it's hard to upgrade packages that come with Python.
Indeed. Batteries, barriers, same thing .
From barry at python.org Tue Sep 9 16:50:15 2003
From: barry at python.org (Barry Warsaw)
Date: Tue Sep 9 11:50:16 2003
Subject: [Python-Dev] New functionality in micro releases
(was:Documenting branch policy)
In-Reply-To: <200309091518.h89FIURh024097@localhost.localdomain>
References: <200309091518.h89FIURh024097@localhost.localdomain>
Message-ID: <1063122609.16781.48.camel@anthem>
On Tue, 2003-09-09 at 11:18, Anthony Baxter wrote:
> The approach taken by the XML add-on package doesn't seem to be the
> right approach, either. At the moment your choices are to manually install
> the new code over the old, or else have a separate install area that's
> manually added to the start of the PYTHONPATH (this is what mailman
> does, right?)
Right, and it sucks because there are multiple top-level entry points
into the system (e.g. command line and cron scripts) and each has to
mangle the path very early on. Then there's the pain of not being able
to run for the source directory because the configure/install procedure
sets up the path hacking module with the proper paths. Blech.
-Barry
From guido at python.org Tue Sep 9 10:17:15 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Sep 9 12:17:37 2003
Subject: [Python-Dev] New functionality in micro releases
(was:Documentingbranch policy)
In-Reply-To: Your message of "Mon, 08 Sep 2003 21:06:39 EDT."
References:
Message-ID: <200309091617.h89GHFb13948@12-236-84-31.client.attbi.com>
> [Guido]
> > Depends on how you fix these. Fixing the code may break more code
> > than fixing the docs (assuming the mismatch existed for a long time)!
[Steve]
> Such as the socket mismatch that was fixed in (IIRC) 1.6, to loud
> complaints from half the network programming world? This still bites
> occasionally when old cord has to be brought forward to more recent
> versions (though, of course, it isn't hard to make the necessary
> changes).
That was not a micro release. I'm glad we fixed that one then; now it
would be a much bigger disaster. (And it was not a matter of
adjusting to the docs, but adjusting to the fact that not all socets
deal with host/port pairs.)
> [...]
> Yup. Any potential for code breakage is going to cause somebody
> somewhere a certain amount of pain. It's these edge cases that cause
> most of the problems, since overall the version-to-version compatibility
> picture is actually remarkably good. But people will cheerfully ignore
> the 99.99% of their code that *doesn't* break, and bitch about the
> little bits that do :-)
For people with a large code base and real customers, the only working
approach is long, careful testing with each new Python release. Hence
the call for Python-in-a-tie. (Though I haven't heard much about that
recently -- I've even heard a call to upgrade the tie to 2.3.)
--Guido van Rossum (home page: http://www.python.org/~guido/)
From pinki_mallick at mentorg.com Tue Sep 9 12:25:18 2003
From: pinki_mallick at mentorg.com (Mallick, Pinki)
Date: Tue Sep 9 14:25:24 2003
Subject: [Python-Dev] memory related issues in Python - bug??
Message-ID:
Hi,
I have a very big XML file that I am trying to parse using xml.dom and xml.dom.minidom. This file has around 100 nodes. Each of these nodes(say 'child_X') have a number of child nodes and each of these child nodes in turn has a text node as its child. So when I try to retrieve all the data from each 'child_X' node, I use this code,
grandChildren = child_X.childNodes()
for grandChildNode in grandChildren:
equation = equation + str(grandChildNode.firstChild.data)
This works properly for all files. But when I am trying to use it on a very large XML file, one of the node's data is incomplete. eg. if grandChildNode.toxml() returns "y18" for this node, then "str(grandChildNode.firstChild.data)" returns only "y" instead of "y18".
This looks like some memory problem. Is there any workaround for this problem.
Thanks for any suggestion or help,
Pinki Mallick
From fdrake at acm.org Tue Sep 9 15:31:41 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Sep 9 14:31:50 2003
Subject: [Python-Dev] memory related issues in Python - bug??
In-Reply-To:
References:
Message-ID: <16222.7309.203774.926825@grendel.zope.com>
Mallick, Pinki writes:
> I have a very big XML file that I am trying to parse using xml.dom
> and xml.dom.minidom. This file has around 100 nodes. Each of these
...
This message was also posted to the XML-SIG (where it belongs) and the
webmaster address for python.org (where it doesn't). Please don't
spam several lists with a question like this.
I've responded to the question in the appropriate forum.
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From aahz at pythoncraft.com Tue Sep 9 16:16:31 2003
From: aahz at pythoncraft.com (Aahz)
Date: Tue Sep 9 15:16:35 2003
Subject: [Python-Dev] memory related issues in Python - bug??
In-Reply-To:
References:
Message-ID: <20030909191631.GA1022@panix.com>
On Tue, Sep 09, 2003, Mallick, Pinki wrote:
>
> I have a very big XML file that I am trying to parse using xml.dom
> and xml.dom.minidom.
You've sent this question to two inappropriate places:
webmaster@python.org and python-dev@python.org
Please use the Usenet group comp.lang.python (or subscribe to
python-list) and post your question there.
--
Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/
"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan
From martin at v.loewis.de Tue Sep 9 21:30:51 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 9 16:30:52 2003
Subject: [Python-Dev] New functionality in micro releases (was:Documenting
branch policy)
In-Reply-To: <200309091518.h89FIURh024097@localhost.localdomain>
References: <200309091518.h89FIURh024097@localhost.localdomain>
Message-ID:
Anthony Baxter writes:
> The approach taken by the XML add-on package doesn't seem to be the
> right approach, either. At the moment your choices are to manually install
> the new code over the old, or else have a separate install area that's
> manually added to the start of the PYTHONPATH (this is what mailman
> does, right?)
Are you referring to PyXML here? It allows to extend/replace the
existing library with neither copying the new code over the old nor
with adding anything to PYTHONPATH. Instead, the core library expects
to be replaced, and is actively looking for a replacement.
Regards,
Martin
From guido at python.org Tue Sep 9 22:32:51 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Sep 10 00:33:13 2003
Subject: [Python-Dev] Quick: want to give a Python talk at Linuxworld NY?
Message-ID: <200309100432.h8A4Wps15627@12-236-84-31.client.attbi.com>
I was just approached by Chris DiBona (chris at damagestudios dot
com) about talking at the next Linuxworld in New York, in January.
There are plenty of PHP and Perl talks but no Python talk yet. Time
is of the essence, they are finalizing the program *now*. Reward is a
free pass to everything. Spread the word!
--Guido van Rossum (home page: http://www.python.org/~guido/)
From guido at python.org Wed Sep 10 00:15:37 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Sep 10 02:16:00 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: Your message of "Sun, 07 Sep 2003 22:13:02 CDT."
<4101.192.168.1.1.1062990782.squirrel@server.lotusland.dyndns.org>
References: <4101.192.168.1.1.1062990782.squirrel@server.lotusland.dyndns.org>
Message-ID: <200309100615.h8A6FbM15765@12-236-84-31.client.attbi.com>
> The method for doing so is modelled after the socket module's new timeout
> functionality [3], where each module that performs logging defines a pair
> of module-level functions named getdefaultlogger() and setdefaultlogger()
> that return and accept (respectively) a logging.Logger instance. The
> initial default logger for each of these modules is named after the module
> itself (i.e. logging.getLogger(__name__)) and its behavior mimics that of
> the Python 2.3 version of the module in terms of format and destination of
> the logging messages.
Hm... Isn't the logging module's ability to set separate *handlers*
sufficient to get the desired flexibility? Rather than the
application specifying a different *logger* object, the application
can specify a different *handler* object. I think this was the
originally intended design. It places less of a burden on the modules
that use logging (no need to define getdefaultlogger() and
setdefaultlogger()) and makes it possible for the different handlers
to be specified by the logging module's configuration file, without
the application having to support this.
Then what remains is a convention for the name of the logger used by
standard library modules; perhaps the module name should suffice?
> So an application that wants to override the logging behavior of a
> standard library module can do so simply by passing a customized logger
> object to the module's setdefaultlogger() function. This works out even
> better for modules where all the logging is encapsulated in a class.
>
> The PEP calls for each class that performs logging to define an attribute
> called self.logger, and upon instantiation initialize self.logger to the
> module's *current* default logger (again, modelled after how the timeout
> value for a socket object is initialized). This allows each instance of a
> class to have its own unique logging behavior.
What behavior exactly are you thinking of? A logger object has almost
no semantics apart from passing log events on to handlers; by design,
all the work is done in the handlers.
> The first item for discussion is whether this seems like a reasonable
> approach. The second is identifying which standard library modules are
> candidates for modification using this approach. I've identified the
> following by simply searching for the word "logging" in the source code,
> but I'm not sure whether this list is complete:
>
> asyncore
> BaseHTTPServer
> cgi
> doctest
> imaplib
> unittest
> the distutils package
>
> I have patches ready for asyncore and BaseHTTPServer and doctest, which
> fit the model I described above quite nicely.
I think it would be useful to let these use the logging module even
without the proposed setdefaultlogger() etc. architecture.
> I determined that no changes are necessary for the cgi module, since
> it doesn't even use the logging functions it defines (initlog,
> dolog, etc.).
They're intended for use by CGI applications. They should probably be
rewritten to direct the calls to the logging module.
> I'm currently working on a way to replicate imaplib's command
> history using LogRecords, which may call for a new type of
> logging.BufferingHandler subclass that maintains a circular queue of
> LogRecords.
That might be a useful feature on its own for the logging package.
Note though, that at a grander scale, the RotatingFileLogger class
already supports a way to keep only the last N messages (for much
larger N, though :-).
> And the distutils package seems prepped for using the logging
> package already.
>
> The oddball is unittest. Unittest is what motivated me to bring up the
> logging issue in the first place, but I can't (or more precisely - am not
> in a position to) decide what direction that module should take w.r.t. if
> and how it should use the logging package. Should the TextTestRunner
> class be modified to use a logging.Logger instead of a _WritelnDecorator,
> or should a separate TestRunner subclass (a LoggingTestRunner perhaps) be
> defined instead? I'm hoping that Steve Purcell or others may have some
> thoughts on this.
To reach Steve, you should probably post to the PyUnit list (sorry,
you'll have to Google for details, I only vaguely recall that there is
such a list).
Personally, I think that unittest is such a special case that it might
be best to leave it alone...
> I would also appreciate hearing any alternatives to the approach I've
> suggested, as well as comments on the strengths and weaknesses of each.
--Guido van Rossum (home page: http://www.python.org/~guido/)
From vilmanis at internode.on.net Wed Sep 10 23:09:29 2003
From: vilmanis at internode.on.net (Yuri Vilmanis)
Date: Wed Sep 10 08:40:37 2003
Subject: [Python-Dev] Base-n integer formatting
Message-ID: <200309102209.29603.vilmanis@internode.on.net>
I have three feature requests for you: no. 1 is quite fundamental, no. 2 would
be a 'nice' useful feature and no. 3 is a bit more esoteric, but very useful
in certain applications
1) print formatting for integers is currently only available for octal, hex,
and decimal. This seems rather odd, as string to integer conversion is
available for bases 2 through 36. End result: when I want to do things in
base 6 (rare, but it has happened twice) or much more commonly in binary, I
can read the numbers in, but cannot print the results - ie quite useless.
2) Input and output formatting for unary (which is commonly referred to as
base 1, yes, I know it's not 'really' base 1 by the general deinition of
base, but "base 1" is the convention) would also be good, as would roman
numeral support, especially in word-processing applications or IO from any
formatted document with roman page/section numbers
3) negative bases: I have needed this more times than I think some people
would care to imagine. These representations have several very useful
properties such as implementation-independent signdness (hence arbirtrary
word length), and some more interesting properties I won't expand on here.
Context: I find python very handy not only as a programming/scripting language
but also as a scratchpad for various number theory investigations.
Unfortunately the output formatting leaves all but the most basic
investigations out in the cold, and lack of negative base input/output makes
it harder to use python to look for certain patterns in result sets, not to
mention getting the data in...
Thanks,
Yuri
From mchermside at ingdirect.com Wed Sep 10 10:30:08 2003
From: mchermside at ingdirect.com (Chermside, Michael)
Date: Wed Sep 10 09:30:19 2003
Subject: [Python-Dev] Base-n integer formatting
Message-ID: <7F171EB5E155544CAC4035F0182093F042126F@INGDEXCHSANC1.ingdirect.com>
Yuri:
I think you have some interesting ideas, but this is really not the
best place to present them. The newsgroup comp.lang.python, also a
mailing list at python-list@python.org, is really the best place to
discuss these ideas. Python-dev is intended for discussion about
the development of Python. New ideas or language features are best
introduced first on c.l.py, then based on the feedback there, written
up as a PEP, *THEN* folks will start discussing it on python-dev as
they figure out HOW (and perhaps whether) to implement it. There's
an exception -- regular Python developers sometimes introduce ideas
on this list, but that's a very short list of people, and it's a
bit different, because if they bring it up it's because they're
already volunteering to implement it (and probably have a prototype
up and running already).
If you'll contact me _separately_, I'll be glad to give you my
thoughts on your suggestions.
-- Michael Chermside
-----Original Message-----
From: Yuri Vilmanis [mailto:vilmanis@internode.on.net]
Sent: Wednesday, September 10, 2003 8:39 AM
To: python-dev@python.org
Subject: [Python-Dev] Base-n integer formatting
I have three feature requests for you: no. 1 is quite fundamental, no. 2 would
be a 'nice' useful feature and no. 3 is a bit more esoteric, but very useful
in certain applications
1) print formatting for integers is currently only available for octal, hex,
and decimal. This seems rather odd, as string to integer conversion is
available for bases 2 through 36. End result: when I want to do things in base 6 (rare, but it has happened twice) or much more commonly in binary, I
can read the numbers in, but cannot print the results - ie quite useless.
2) Input and output formatting for unary (which is commonly referred to as base 1, yes, I know it's not 'really' base 1 by the general deinition of base, but "base 1" is the convention) would also be good, as would roman numeral support, especially in word-processing applications or IO from any formatted document with roman page/section numbers
3) negative bases: I have needed this more times than I think some people would care to imagine. These representations have several very useful
properties such as implementation-independent signdness (hence arbirtrary word length), and some more interesting properties I won't expand on here.
Context: I find python very handy not only as a programming/scripting language
but also as a scratchpad for various number theory investigations.
Unfortunately the output formatting leaves all but the most basic
investigations out in the cold, and lack of negative base input/output makes
it harder to use python to look for certain patterns in result sets, not to mention getting the data in...
Thanks,
Yuri
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
This email may contain confidential or privileged information. If you believe you have received the message in error, please notify the sender and delete the message without copying or disclosing it.
From mfb at lotusland.dyndns.org Wed Sep 10 09:29:46 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Wed Sep 10 09:43:11 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <200309100615.h8A6FbM15765@12-236-84-31.client.attbi.com>
References: <4101.192.168.1.1.1062990782.squirrel@server.lotusland.dyndns.org>
<200309100615.h8A6FbM15765@12-236-84-31.client.attbi.com>
Message-ID: <3412.192.168.1.1.1063200586.squirrel@server.lotusland.dyndns.org>
Guido van Rossum said:
> Hm... Isn't the logging module's ability to set separate *handlers*
> sufficient to get the desired flexibility? Rather than the
> application specifying a different *logger* object, the application
> can specify a different *handler* object. I think this was the
> originally intended design. It places less of a burden on the modules
> that use logging (no need to define getdefaultlogger() and
> setdefaultlogger()) and makes it possible for the different handlers
> to be specified by the logging module's configuration file, without
> the application having to support this.
I was thinking of the logging hierarchy when I came up with this idea. If
a module references a specific logger object explicitly when a logging
message needs to be issued (e.g. logging.getLogger(__name__).info(msg)),
then the module's logging output is forever handled at a fixed place in
the hierarchy.
An application may want to organize things differently. For example, it
may want to redirect asyncore's log messages to a logger named
"myapp.network.asyncore" so that its messages are propagated to the
"myapp.network" and "myapp" loggers defined by the application. Those
loggers could apply additional handlers or filters to asyncore's log
messages. In such a case the application could set this up as follows:
asyncore.setdefaultlogger(logging.getLogger('myapp.network.asyncore'))
> What behavior exactly are you thinking of? A logger object has almost
> no semantics apart from passing log events on to handlers; by design,
> all the work is done in the handlers.
I used the word "behavior" to refer to the series of handlers and filters
that are applied to a log message. Different instances of the same class
could log to different logger objects, and thereby have different "logging
behaviors". Unfortunately none of the modules I've identified in the
stdlib seem to provide a motivating example of why you'd ever want to do
this. I can try to elaborate further if the benefit is not clear.
>> I determined that no changes are necessary for the cgi module, since
>> it doesn't even use the logging functions it defines (initlog,
>> dolog, etc.).
>
> They're intended for use by CGI applications. They should probably be
> rewritten to direct the calls to the logging module.
I don't have much experience with this module so correct me if I'm wrong,
but to my knowledge the cgi module does not call its own logging
functions. This implies to me that it's essentially up to the CGI
application to perform its own logging. If that's true then I wonder if
CGI applications would be better off just be using the logging package
directly, rather than routing log messages through the cgi module.
The existing log functions in cgi should probably remain for backwards
compatibility, but I just have to wonder if it's really worth updating
them.
>> I'm currently working on a way to replicate imaplib's command
>> history using LogRecords, which may call for a new type of
>> logging.BufferingHandler subclass that maintains a circular queue of
>> LogRecords.
>
> That might be a useful feature on its own for the logging package.
> Note though, that at a grander scale, the RotatingFileLogger class
> already supports a way to keep only the last N messages (for much
> larger N, though :-).
Thanks, I'll look into that.
> To reach Steve, you should probably post to the PyUnit list (sorry,
> you'll have to Google for details, I only vaguely recall that there is
> such a list).
If Pyunit-interest is what you mean, the last time I checked it didn't
seem to be very active anymore and was filled with mostly spam. I'll try
to reach Steve directly.
> Personally, I think that unittest is such a special case that it might
> be best to leave it alone...
I think I agree with you. Unless Steve has any ideas, I'm going to drop
it from the forthcoming PEP.
From guido at python.org Wed Sep 10 08:18:00 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Sep 10 10:18:26 2003
Subject: [Python-Dev] Base-n integer formatting
In-Reply-To: Your message of "Wed, 10 Sep 2003 22:09:29 +0930."
<200309102209.29603.vilmanis@internode.on.net>
References: <200309102209.29603.vilmanis@internode.on.net>
Message-ID: <200309101418.h8AEI0f16363@12-236-84-31.client.attbi.com>
Yuri,
Those conversions are so easily coded in Python itself that I see no
reason to build them into the language or standard library.
--Guido van Rossum (home page: http://www.python.org/~guido/)
From Paul.Moore at atosorigin.com Wed Sep 10 17:49:21 2003
From: Paul.Moore at atosorigin.com (Moore, Paul)
Date: Wed Sep 10 11:50:09 2003
Subject: [Python-Dev] RE: [python-win32] pb with string conversion to float
and GetObject call
Message-ID: <16E1010E4581B049ABC51D4975CEDB8803060C76@UKDCX001.uk.int.atosorigin.com>
From: popov [mailto:outlook@evpopov.com]
> Here's a strange one:
>
> print float('8.4')
>
> Works as expected.
> But:
>
> import win32com.client
> obj = win32com.client.GetObject ("winmgmts:")
> print float('8.4')
>
> Throws a 'ValueError: invalid literal for float(): 8.4' exception on the
> print line !
> If you use '8,4' instead of '8.4', then it works (!). So it seems that some
> locals have been modified here by the call to GetObject (without this call -
> leaving only the import statement - the problem does not show up).
> Note that using 'LDAP:' instead of 'winmgmts:', everything is ok. So maybe
> the culprit is the winmgmts provider...
> If someone could test this and let me know the results. Thx.
There's a discussion going on on python-dev at the moment (the messages
mention LC_NUMERIC in the subject) on basically this issue. Python's
float conversion routines rely on the underlying C locale remaining
unchanged. It's known that 3rd party extensions can violate this, and
also that Microsoft MAPI mucks up the locale behind the scenes. From
what you are reporting, it looks like the WMI stuff also does this.
I don't know of any fix for sure - you might be able to use the functions
from the locale module to set the locale value back to "C". Assuming that
an acceptable fix is found, something should be in Python 2.4 (I don't know
if the fix will be deemed appropriate for a maintenance release of 2.3).
Hope this helps,
Paul.
PS I'm copying this to python-dev, as evidence of another area where this
issue comes up - I hope that's acceptable.
From mcherm at mcherm.com Wed Sep 10 09:58:25 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Sep 10 11:58:26 2003
Subject: [Python-Dev] Consistent logging in the standard library
Message-ID: <1063209505.3f5f4a21b564e@mcherm.com>
Quoth Matthew F. Barnes:
> An application may want to organize things differently. For example, it
> may want to redirect asyncore's log messages to a logger named
> "myapp.network.asyncore"
Seems like an uncommon request, but one which shouldn't be made
impossible for those who want it. But couldn't it be achieved by
writing a handler which simply re-logged the message at the new
spot in the hierarchy?
From fdrake at acm.org Wed Sep 10 13:04:07 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Wed Sep 10 12:04:18 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <1063209505.3f5f4a21b564e@mcherm.com>
References: <1063209505.3f5f4a21b564e@mcherm.com>
Message-ID: <16223.19319.350549.596704@grendel.zope.com>
Michael Chermside writes:
> Seems like an uncommon request, but one which shouldn't be made
> impossible for those who want it. But couldn't it be achieved by
> writing a handler which simply re-logged the message at the new
> spot in the hierarchy?
That would solve that use case handily; if that's the right approach,
such a handler should be provided as part of the standard set of
handlers.
The use case for a .setlogger() method on the various objects that
perform logging is different, so that is still needed.
Generally, I'm not convinced that a getdefaultlogger() is needed or
helpful (it seems an implementation default for the module), though
I'm not at all opposed to the setdefaultlogger() for the affected
modules.
-Fred
--
Fred L. Drake, Jr.
PythonLabs at Zope Corporation
From kiko at async.com.br Wed Sep 10 14:08:59 2003
From: kiko at async.com.br (Christian Reis)
Date: Wed Sep 10 12:10:25 2003
Subject: [Python-Dev] Re: Be Honest about LC_NUMERIC [REPOST]
In-Reply-To:
References:
Message-ID: <20030910160858.GE2405@async.com.br>
On Mon, Sep 01, 2003 at 02:30:23PM -0400, Tim Peters wrote:
> There's no way of using C's locale gimmicks that's threadsafe, short of all
> callers agreeing to follow a beyond-standard-C exclusion protocol -- which
> is the same as saying "no way" in reality. So that's part of one problem no
> patch of this ilk *can* solve. It's not that the patch doesn't try hard
> enough, it's that this approach is inherently inadequate to solve all of
> this particular problem.
>
> > It is just that the patch does not "feel" right, given that there must
> > be "native" locale-inaware parsing of floating point constants
> > somewhere on each platform (atleast on those that support C++98).
>
> I haven't found one on Windows (doesn't mean it doesn't exist, does mean
> it's apparently well hidden if it does exist).
Just to follow up, today I found a thread on opengroup.org that
discusses locale-safe APIs in the C library. They don't suggest anything
very positive in the way of standardization :-/
http://www.opengroup.org/austin/mailarchives/austin-group-l/msg00763.html
Take care,
--
Christian Reis, Senior Engineer, Async Open Source, Brazil.
http://async.com.br/~kiko/ | [+55 16] 261 2331 | NMFL
From Mariselaschwebach at pcvo.com Wed Sep 10 22:32:24 2003
From: Mariselaschwebach at pcvo.com (Women Helpline)
Date: Wed Sep 10 15:37:26 2003
Subject: [Python-Dev] Luscious, Seductive L|ps can be yours
Message-ID: <551x9i9qj5mae9$7u-6@rh1.c.xp.d6>
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20030910/b1cc80f2/attachment-0001.htm
From mfb at lotusland.dyndns.org Wed Sep 10 17:21:47 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Wed Sep 10 17:35:06 2003
Subject: [Python-Dev] Consistent logging in the standard library
In-Reply-To: <16223.19319.350549.596704@grendel.zope.com>
References: <1063209505.3f5f4a21b564e@mcherm.com>
<16223.19319.350549.596704@grendel.zope.com>
Message-ID: <30174.130.76.32.145.1063228907.squirrel@server.lotusland.dyndns.org>
Fred L. Drake, Jr. said:
>
> Michael Chermside writes:
> > Seems like an uncommon request, but one which shouldn't be made
> > impossible for those who want it. But couldn't it be achieved by
> > writing a handler which simply re-logged the message at the new
> > spot in the hierarchy?
>
> That would solve that use case handily; if that's the right approach,
> such a handler should be provided as part of the standard set of
> handlers.
I agree, this seems much cleaner than what I've proposed.
In fact, the "ForwardingHandler" idea could be submitted to SourceForge as
a separate patch for logging, and then the issue drops out of this
"consistent logging" PEP altogether.
So that changes my proposal as follows:
- The getdefaultlogger() and setdefaultlogger() functions go away.
- The self.logger attribute of class instances can be initialized to
logging.getLogger(__name__).
- Some kind of "ForwardingHandler" can be added to the module's logger to
forward messages elsewhere in the logging hierarchy. But this is not
likely to be a common use case.
Matthew Barnes
From fuerte at sci.fi Thu Sep 11 20:23:05 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 12:23:02 2003
Subject: [Python-Dev] Making python C-api thread safe
Message-ID: <3F60A169.3070409@sci.fi>
Background: I'm new to Python. I just embedded Python into a
multi-threaded application, and extended Python by callbacks as well.
During this process I realised that Python is not internally as
beautiful as externally. It is not thread safe. This surprised me,
because Python is a quite young programming language. I started thinking
how to fix this, how to make Python thread safe. Currently Python does
not support multiprocessor machines or hyperthreading processors. It is
only a matter of time when this has to be fixed, better soon than later.
How to make C-api thread safe: It's quite simple, in fact. Thread should
first call
From aahz at pythoncraft.com Thu Sep 11 13:44:22 2003
From: aahz at pythoncraft.com (Aahz)
Date: Thu Sep 11 12:44:27 2003
Subject: [Python-Dev] Making python C-api thread safe
In-Reply-To: <3F60A169.3070409@sci.fi>
References: <3F60A169.3070409@sci.fi>
Message-ID: <20030911164421.GA897@panix.com>
On Thu, Sep 11, 2003, Harri Pesonen wrote:
>
> Background: I'm new to Python.
This issue gets discussed regularly on comp.lang.python; python-dev is
for discussion of specific changes to the Python language and libraries.
Until you are familiar with the way threading works in Python and have
some concrete proposal to make, please keep the discussion on c.l.py.
--
Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/
"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan
From fuerte at sci.fi Thu Sep 11 20:53:10 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 12:53:11 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
Message-ID: <3F60A876.7020204@sci.fi>
Sorry about the previous post! :-)
Background: I'm new to Python. I just embedded Python into a
multi-threaded application, and extended Python by callbacks as well.
During this process I realised that Python is not internally as
beautiful as externally. It is not thread safe. This surprised me,
because Python is a quite young programming language. I started thinking
how to fix this, how to make Python thread safe. Currently Python does
not support multiprocessor machines or hyperthreading processors. It is
only a matter of time when this has to be fixed, better soon than later.
How to make C-API thread safe: It's quite simple, in fact. Thread should
first call
PyThreadState* Py_NewInterpreter()
and use this thread state pointer in every following API call. This
thread state pointer should point to a memory structure, that holds
everything that Python interpreter needs. There should be no global
variables, not even None, except
const char* Py_GetVersion()
and such. So every Python function should have this thread state as the
first argument:
int PyRun_SimpleString(PyThreadState*tstate, char *command)
and so on. Also callbacks should have it, about every API function.
Advantages: The following functions would not be needed any more, it
would make the API much simpler:
void PyEval_InitThreads( )
void PyEval_AcquireLock( )
void PyEval_ReleaseLock( )
void PyEval_AcquireThread( PyThreadState *tstate)
void PyEval_ReleaseThread( PyThreadState *tstate)
PyThreadState* PyEval_SaveThread( )
void PyEval_RestoreThread( PyThreadState *tstate)
Py_BEGIN_ALLOW_THREADS
Py_END_ALLOW_THREADS
Py_BLOCK_THREADS
Py_UNBLOCK_THREADS
PyInterpreterState* PyInterpreterState_New( )
void PyInterpreterState_Clear( PyInterpreterState *interp)
void PyInterpreterState_Delete( PyInterpreterState *interp)
PyThreadState* PyThreadState_New( PyInterpreterState *interp)
void PyThreadState_Clear( PyThreadState *tstate)
void PyThreadState_Delete( PyThreadState *tstate)
PyThreadState* PyThreadState_Get( )
PyThreadState* PyThreadState_Swap( PyThreadState *tstate)
The current documentation says: "In order to support multi-threaded
Python programs, the interpreter regularly releases and reacquires the
lock -- by default, every 100 bytecode instructions". This would not be
needed anymore. It would make the interpreter faster.
Making this change would take a couple of weeks, if the job would be
divided between modules to different persons. The change is trivial, but
it would have to be made everywhere.
Disadvantages: Everything old would break. Thread safe Python should
have a version number significantly higher, like Python 3. Old
applications would work with old Python versions, with old Python DLLs,
but they would need to be rewritten for the new API. It would be a
trivial task, but still.
Disclaimer: I already got mail saying: "
This issue gets discussed regularly on comp.lang.python; python-dev is
for discussion of specific changes to the Python language and libraries.
Until you are familiar with the way threading works in Python and have
some concrete proposal to make, please keep the discussion on c.l.py."
The current situation is very unfortunate, having a single threaded implementation in the modern computer age. I have a concrete proposal, which would break everything :-) but I think that is the only solution.
Thanks for reading,
Harri
From pje at telecommunity.com Thu Sep 11 14:10:06 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Sep 11 13:10:24 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60A876.7020204@sci.fi>
Message-ID: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
At 07:53 PM 9/11/03 +0300, Harri Pesonen wrote:
>Sorry about the previous post! :-)
>
>Background: I'm new to Python. I just embedded Python into a
>multi-threaded application, and extended Python by callbacks as well.
>During this process I realised that Python is not internally as beautiful
>as externally. It is not thread safe. This surprised me, because Python is
>a quite young programming language. I started thinking how to fix this,
>how to make Python thread safe. Currently Python does not support
>multiprocessor machines or hyperthreading processors. It is only a matter
>of time when this has to be fixed, better soon than later.
>
>How to make C-API thread safe: It's quite simple, in fact. Thread should
>first call
>
>PyThreadState* Py_NewInterpreter()
>
>and use this thread state pointer in every following API call. This thread
>state pointer should point to a memory structure, that holds everything
>that Python interpreter needs. There should be no global variables, not
>even None, except
This only makes individual Python interpreters thread-safe. That might
certainly be useful for some embedded applications, but it does nothing for
thread-safety *within* an interpreter.
In other words, Python would *still* need a global interpreter lock, in
order for threaded programs written in Python to behave properly.
So, your solution would help only creators of multithreaded embedded
applications, wherein each thread desired an isolated Python
interpreter. Meanwhile, it would impose both a one-time and ongoing cost
penalty on *all other* Python users, without giving them any benefit in return.
From fuerte at sci.fi Thu Sep 11 21:47:49 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 13:47:46 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
Message-ID: <3F60B545.6030100@sci.fi>
Phillip J. Eby wrote:
> At 07:53 PM 9/11/03 +0300, Harri Pesonen wrote:
>
>> Sorry about the previous post! :-)
>>
>> Background: I'm new to Python. I just embedded Python into a
>> multi-threaded application, and extended Python by callbacks as well.
>> During this process I realised that Python is not internally as
>> beautiful as externally. It is not thread safe. This surprised me,
>> because Python is a quite young programming language. I started
>> thinking how to fix this, how to make Python thread safe. Currently
>> Python does not support multiprocessor machines or hyperthreading
>> processors. It is only a matter of time when this has to be fixed,
>> better soon than later.
>>
>> How to make C-API thread safe: It's quite simple, in fact. Thread
>> should first call
>>
>> PyThreadState* Py_NewInterpreter()
>>
>> and use this thread state pointer in every following API call. This
>> thread state pointer should point to a memory structure, that holds
>> everything that Python interpreter needs. There should be no global
>> variables, not even None, except
>
>
> This only makes individual Python interpreters thread-safe. That
> might certainly be useful for some embedded applications, but it does
> nothing for thread-safety *within* an interpreter.
>
> In other words, Python would *still* need a global interpreter lock,
> in order for threaded programs written in Python to behave properly.
>
> So, your solution would help only creators of multithreaded embedded
> applications, wherein each thread desired an isolated Python
> interpreter. Meanwhile, it would impose both a one-time and ongoing
> cost penalty on *all other* Python users, without giving them any
> benefit in return.
You are right, I thought only about making interpreters thread safe,
thanks for pointing this out. But this is still needed, it is
unavoidable in the long run.
What do you mean about the penalty? If you mean having the state pointer
in stack in every call, I don't think so.
If we think about making Python threads thread-safe, within the
interpreter, then I think that it should be left for the application
deverloper, in principle. He should create "critical sections" and other
locks when needed (I don't know how it is done currently). I can imagine
that None and other values will cause problems, when two threads
increment or decrement the reference count at the same time. Of course
there is a simple solution for that, InterlockedIncrement and
InterlockedDecrement in Win32 at least.
I can imagine that there are other problems with garbage collector, and
so on. I don't want to go into details about this, I got my message
through, I was almost losing sleep when I realised the current situation.
But my basic message is this: Python needs to be made thread safe.
Making the individual interpreters thread safe is trivial, and benefits
many people, and is a necessary first step; making threads within
interpreter thread safe is possible as well, at least if you leave
something for the developer, as you should, as you do in every other
programming language as well.
Harri
From brian at sweetapp.com Thu Sep 11 12:06:43 2003
From: brian at sweetapp.com (Brian Quinlan)
Date: Thu Sep 11 14:05:57 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60B545.6030100@sci.fi>
Message-ID: <001001c3788f$75798320$21795418@dell1700>
> Making the individual interpreters thread safe is trivial, and
benefits
> many people,
I'm not sure that I believe this statement. Isn't this of benefit only
to people who want to have multiple interpreter instances running
concurrently in an embedding situation? I doubt that there is an army of
people who want to do that.
> and is a necessary first step; making threads within
> interpreter thread safe is possible as well, at least if you leave
> something for the developer, as you should, as you do in every other
> programming language as well.
Of course it is possible (and would likely be of benefit to more than a
handful of Python users). The problem is that it requires effort, would
likely require massive code breakage and the locking required would
likely significantly decrease Python's single-thread performance.
Cheers,
Brian
From aahz at pythoncraft.com Thu Sep 11 15:21:36 2003
From: aahz at pythoncraft.com (Aahz)
Date: Thu Sep 11 14:21:41 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <001001c3788f$75798320$21795418@dell1700>
References: <3F60B545.6030100@sci.fi> <001001c3788f$75798320$21795418@dell1700>
Message-ID: <20030911182136.GA12857@panix.com>
Please, people, let's take this off python-dev and move it to
comp.lang.python.
--
Aahz (aahz@pythoncraft.com) <*> http://www.pythoncraft.com/
"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan
From dave at boost-consulting.com Thu Sep 11 16:11:54 2003
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Sep 11 15:13:05 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
Message-ID:
I'd post a SF bug but it keeps insisting I log in, even after I have
already.
There's a bug, either in the code or docs for
PyUnicode_FromEncodedObject. The docs read:
"Unicode objects are passed back as-is with incremented
refcount. Note: These cannot be decoded; passing a non-NULL value
for encoding will result in a TypeError."
'tain't so; the following shows that the error is unconditional.
if (obj == NULL) {
PyErr_BadInternalCall();
return NULL;
}
#if 0
/* For b/w compatibility we also accept Unicode objects provided
that no encodings is given and then redirect to
PyObject_Unicode() which then applies the additional logic for
Unicode subclasses.
NOTE: This API should really only be used for object which
represent *encoded* Unicode !
*/
if (PyUnicode_Check(obj)) {
if (encoding) {
PyErr_SetString(PyExc_TypeError,
"decoding Unicode is not supported");
return NULL;
}
return PyObject_Unicode(obj);
}
#else
if (PyUnicode_Check(obj)) {
PyErr_SetString(PyExc_TypeError,
"decoding Unicode is not supported");
return NULL;
}
#endif
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
From fuerte at sci.fi Thu Sep 11 23:16:03 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 15:16:06 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
Message-ID: <3F60C9F3.4060107@sci.fi>
Phillip J. Eby wrote:
> At 08:47 PM 9/11/03 +0300, Harri Pesonen wrote:
>
>> But my basic message is this: Python needs to be made thread safe.
>> Making the individual interpreters thread safe is trivial, and
>> benefits many people, and is a necessary first step;
>
>
> It's far from trivial - you're talking about invalidating every piece
> of C code written for Python over a multi-year people by dozens upon
> dozens of extension authors.
The change is trivial in Python C API. I already said that it would
break everything outside the Python distribution, but the change in
other applications is also trivial.
> It doesn't benefit many people: only those using isolated interpreters
> embedded in a multithreaded C program.
I don't know how many people are writing threads in Python, either. I
guess that not so many. In my case I only need a thread safe
interpreter, I don't create threads in Python code. So just having what
I described would be enough for me: no need for global interpreter lock,
and Python would be really multithreading. It would benefit many people,
I'm sure.
>
>> making threads within interpreter thread safe is possible as well, at
>> least if you leave something for the developer, as you should, as you
>> do in every other programming language as well.
>
>
> You misunderstand. Those "critical sections" are for the most part in
> Python's C code, not in the Python script.
Yes, I'm aware of the None problem at least (only one instance of it).
Please enlighten me about the other critical sections? Object
allocation/freeing?
> I'm guessing you haven't done much writing of C extensions for Python
> (or Python core C), or else you'd realize why trying to make
> INCREF/DECREF threadsafe would absolutely decimate performance.
> Reference count updates happen *way* too often in normal code flow.
>
I also knew that already. But how else can you do it?
Of course, changing Python to not have a single None would help a lot.
Or, perhaps it could have a single None, but in case of None, the
reference count would have no meaning, it would never be deallocated,
because it would be checked in code. Maybe it does it already, I don't know.
I'm also wondering why this problem has not been addressed before? If I
had the power to change Python, this would be the first thing I did.
Harri
From jeremy at alum.mit.edu Thu Sep 11 16:32:45 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu Sep 11 15:32:55 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60C9F3.4060107@sci.fi>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
<3F60C9F3.4060107@sci.fi>
Message-ID: <1063308764.2069.235.camel@localhost.localdomain>
On Thu, 2003-09-11 at 15:16, Harri Pesonen wrote:
> I'm also wondering why this problem has not been addressed before? If I
> had the power to change Python, this would be the first thing I did.
Try coming up with a patch. I expect it would be considered provided
that it was maximally backwards compatible with the existing C API and
did not reduce performance on benchmarks like pystone.
That is to say, sure it would be nice, but at what cost?
Jeremy
From fuerte at sci.fi Fri Sep 12 00:31:25 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Thu Sep 11 16:31:22 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <1063308764.2069.235.camel@localhost.localdomain>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
<3F60C9F3.4060107@sci.fi>
<1063308764.2069.235.camel@localhost.localdomain>
Message-ID: <3F60DB9D.8060205@sci.fi>
Jeremy Hylton wrote:
>On Thu, 2003-09-11 at 15:16, Harri Pesonen wrote:
>
>
>>I'm also wondering why this problem has not been addressed before? If I
>>had the power to change Python, this would be the first thing I did.
>>
>>
>
>Try coming up with a patch. I expect it would be considered provided
>that it was maximally backwards compatible with the existing C API and
>did not reduce performance on benchmarks like pystone.
>
It would be totally incompatible with the existing C API. The
performance would be better.
>That is to say, sure it would be nice, but at what cost?
>
>
At cost of breaking up with the past, but eventually it has to be done.
If I had time, I would create this multithreading Python myself, call it
a different language perhaps. I don't know if the Python licence allows
this.
Harri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20030911/a31103f3/attachment.htm
From martin at v.loewis.de Thu Sep 11 21:46:35 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 11 16:46:35 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
In-Reply-To:
References:
Message-ID:
David Abrahams writes:
> I'd post a SF bug but it keeps insisting I log in, even after I have
> already.
It does not insist, it just explains that you have logged
on. Unfortunately, there is no way to make this message disappear once
you have logged on, so all of us see it all the time.
>
> There's a bug, either in the code or docs for
> PyUnicode_FromEncodedObject. The docs read:
>
> "Unicode objects are passed back as-is with incremented
> refcount. Note: These cannot be decoded; passing a non-NULL value
> for encoding will result in a TypeError."
Please do submit a bug report for that.
Regards,
Martin
From martin at v.loewis.de Thu Sep 11 21:54:11 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Sep 11 16:54:13 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60C9F3.4060107@sci.fi>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
<3F60C9F3.4060107@sci.fi>
Message-ID:
Harri Pesonen writes:
> The change is trivial in Python C API. I already said that it would
> break everything outside the Python distribution, but the change in
> other applications is also trivial.
If it is trivial, would you mind posting a patch somewhere?
> > You misunderstand. Those "critical sections" are for the most part
> > in Python's C code, not in the Python script.
>
> Yes, I'm aware of the None problem at least (only one instance of
> it). Please enlighten me about the other critical sections? Object
> allocation/freeing?
Yes, that, plus:
- allocation of/access to small numbers
- access to global variables in extension modules
(e.g. cursesmodule.c:PyCursesError)
- type objects
etc.
> Of course, changing Python to not have a single None would help a
> lot. Or, perhaps it could have a single None, but in case of None, the
> reference count would have no meaning, it would never be deallocated,
> because it would be checked in code. Maybe it does it already, I don't
> know.
So how can you know a patch would be trivial?
> I'm also wondering why this problem has not been addressed before? If
> I had the power to change Python, this would be the first thing I did.
Please go ahead and post a patch. You might find that the patch is
difficult to write, and, once written, will have many errors. Once
those are fixed, you might find that Python becomes painfully slow,
and lose a lot of portability.
Regards,
Martin
From martin at v.loewis.de Fri Sep 12 00:07:56 2003
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu Sep 11 17:08:40 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60DB9D.8060205@sci.fi>
References: <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com> <5.1.1.6.0.20030911130607.02426ec0@telecommunity.com> <5.1.1.6.0.20030911142317.02b88640@telecommunity.com> <3F60C9F3.4060107@sci.fi> <1063308764.2069.235.camel@localhost.localdomain>
<3F60DB9D.8060205@sci.fi>
Message-ID: <3F60E42C.1040501@v.loewis.de>
Harri Pesonen wrote:
> If I had time, I would create this multithreading Python myself, call it
> a different language perhaps. I don't know if the Python licence allows
> this.
The license is only on the implementation (source code), not on the
language itself.
You are free to provide alternative implementations of the language,
without having to ask anybody for permission.
But please don't call such a thing trivial when you know it is
inherently complex.
Regards,
Martin
From greg at cosc.canterbury.ac.nz Fri Sep 12 16:11:28 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Sep 11 23:11:46 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <3F60C9F3.4060107@sci.fi>
Message-ID: <200309120311.h8C3BSe01781@oma.cosc.canterbury.ac.nz>
> Yes, I'm aware of the None problem at least (only one instance of it).
> Please enlighten me about the other critical sections? Object
> allocation/freeing?
None is only the molecule at the tip of the iceberg. If you're
talking about threads within a Python interpreter, doing just about
anything at all to any Python object is a critical section. The amount
of locking required to deal with that at a fine-grained level would
totally kill performance, not to mention being hugely tedious and
error-prone to implement.
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury, | A citizen of NewZealandCorp, a |
Christchurch, New Zealand | wholly-owned subsidiary of USA Inc. |
greg@cosc.canterbury.ac.nz +--------------------------------------+
From dave at boost-consulting.com Fri Sep 12 00:35:40 2003
From: dave at boost-consulting.com (David Abrahams)
Date: Thu Sep 11 23:36:48 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
In-Reply-To: (Martin v.
=?iso-8859-1?q?L=F6wis's?= message of "11 Sep 2003 22:46:13 +0200")
References:
Message-ID:
martin@v.loewis.de (Martin v. L?wis) writes:
> David Abrahams writes:
>
>> I'd post a SF bug but it keeps insisting I log in, even after I have
>> already.
>
> It does not insist, it just explains that you have logged
> on. Unfortunately, there is no way to make this message disappear once
> you have logged on, so all of us see it all the time.
No, seriously. There is no "submit new" button/link for me.
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
From fuerte at sci.fi Fri Sep 12 08:56:55 2003
From: fuerte at sci.fi (Harri Pesonen)
Date: Fri Sep 12 00:56:56 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <5.1.1.6.0.20030911162016.02027750@telecommunity.com>
References: <5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911130607.02426ec0@telecommunity.com>
<5.1.1.6.0.20030911142317.02b88640@telecommunity.com>
<5.1.1.6.0.20030911162016.02027750@telecommunity.com>
Message-ID: <3F615217.8060106@sci.fi>
Phillip J. Eby wrote:
> Please do not CC: my mail to Python-Dev again; I intentionally did not
> include python-dev on my CC: because it was asked that we move this
> thread elsewhere.
>
> At 10:16 PM 9/11/03 +0300, Harri Pesonen wrote:
>
>> Phillip J. Eby wrote:
>>
>>> At 08:47 PM 9/11/03 +0300, Harri Pesonen wrote:
>>>
>>>> But my basic message is this: Python needs to be made thread safe.
>>>> Making the individual interpreters thread safe is trivial, and
>>>> benefits many people, and is a necessary first step;
>>>
>>>
>>>
>>> It's far from trivial - you're talking about invalidating every
>>> piece of C code written for Python over a multi-year people by
>>> dozens upon dozens of extension authors.
>>
>>
>> The change is trivial in Python C API. I already said that it would
>> break everything outside the Python distribution, but the change in
>> other applications is also trivial.
>
>
> How do you propose that C code called *from* Python *receive* the
> threadstate pointer?
Exactly like that. Is there a problem? I'm suggesting that every
function call gets that pointer, unless the function can get it from
some other argument, that contains a pointer to it.
>
>>> It doesn't benefit many people: only those using isolated
>>> interpreters embedded in a multithreaded C program.
>>
>>
>> I don't know how many people are writing threads in Python, either. I
>> guess that not so many. In my case I only need a thread safe
>> interpreter, I don't create threads in Python code. So just having
>> what I described would be enough for me: no need for global
>> interpreter lock, and Python would be really multithreading. It would
>> benefit many people, I'm sure.
>
>
> Obviously, it's enough for you, or you wouldn't be proposing it. What
> does it do for me? Nothing whatsoever, except add needless overhead
> and make me rewrite every C extension I've ever written for Python.
> So, by and large, you're not going to get much support for your change
> from Python developers, especially those who write C extensions, or
> depend on extensions written by others.
Probably. That's why I'm thinking now that the language should be called
something else, like MPython for "multi-threading Python". It would be
99% compatible with the existing Python syntax, but have different
internals.
>
>> Yes, I'm aware of the None problem at least (only one instance of
>> it). Please enlighten me about the other critical sections? Object
>> allocation/freeing?
>
>
> Data structure manipulations, e.g. all use of dictionaries. Python
> spends most of its time doing dictionary lookups or modifications, all
> of which need to be protected.
After sleeping over night, I think that I got it. :-) The simple
solution is, that each thread created in Python gets its own independent
interpreter state as well. And there could be a separate thread-global
interpreter state for shared memory access. Access to this global state
would always be synchronized. There could even be multiple named global
states, so that the thread interlocking could be minimized. The python
syntax for creating objects in this global state should be invented:
synchronize a = "abcd"
Also when creating the new thread, it's arguments would be copied from
the creating state to the new state.
What does it sound? Of course it would be incompatible with the current
threading system in Python, but it would be totally multithreading, no
global interpreter lock needed. It would be faster than current Python,
there would be no need to free or acquire the lock when calling OS
functions, and no need to check how many byte codes have been processed,
etc.
>
>>> I'm guessing you haven't done much writing of C extensions for
>>> Python (or Python core C), or else you'd realize why trying to make
>>> INCREF/DECREF threadsafe would absolutely decimate performance.
>>> Reference count updates happen *way* too often in normal code flow.
>>
>> I also knew that already. But how else can you do it?
>
>
> The way it's done now! :)
I understand why the current Python works like it does. But I think that
it's time for the next generation. If you don't do it, and I have no
time now to do it, I'm still sure that this is done at some point,
rather sooner than later.
>
>> Of course, changing Python to not have a single None would help a
>> lot. Or, perhaps it could have a single None, but in case of None,
>> the reference count would have no meaning, it would never be
>> deallocated, because it would be checked in code. Maybe it does it
>> already, I don't know.
>
>
> I really don't mean to be rude (another reason I'm writing this to you
> privately), but this paragraph shows you are *really* new to Python
> both at the level of coding in Python and coding with Python's C API.
> I wish I could explain in detail why, but there's really far too much
> that you don't understand and it would take me too long. I will
> attempt to summarize a very few points, however: first, identity
> (pointer comparison) is a part of the Python language, so you can't
> have multiple None instances any more than you can have more than one
> value be NULL in C. Second, at the C level, all Python objects
> (including None) have an absolutely uniform API, so having refcount
> behavior be different for different kinds of objects is not at all
> practical. Third, if you had more than one Py_None at the C level,
> you'd either have to make Py_None a macro, or rewrite all the C. If
> you don't think that's a problem, you have absolutely no idea how much
> C code out there is written to the Python API.
Yes, Py_None would be a macro. All access to interpreter state would go
through the interpreter state pointer that is always in stack, the first
argument each C API function gets. That pointer should be named so that
the macros will always work ("tState", for example, so that Py_None
macro would expand to tState->mPy_None, for example).
>
>> I'm also wondering why this problem has not been addressed before?
>
>
> It has; the cure is worse than the disease. A few years ago, somebody
> wrote a "free-threading" version of Python, which locked individual
> data objects rather than use the global interpreter lock. The
> performance for single-threaded programs was abominable, and the
> performance gain even on multiprocessor machines was not thought worth
> the cost. So the project was scrapped.
There would be no locking in my proposal, except when accessing the
shared memory global thread state.
I don't know, I got mail about writing a PEP. It is clear that it would
not be accepted, because it would break the existing API. The change is
so big that I think that it has to be called a different language.
This is the last message I will make about this matter (before actually
starting to code it), so I'm posting this to python-list as well,
because this is too important to be ignored. Python *needs* to be
free-threading...
Harri
From martin at v.loewis.de Fri Sep 12 06:56:01 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Fri Sep 12 01:56:03 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
In-Reply-To:
References:
Message-ID:
David Abrahams writes:
> > It does not insist, it just explains that you have logged
> > on. Unfortunately, there is no way to make this message disappear once
> > you have logged on, so all of us see it all the time.
>
> No, seriously. There is no "submit new" button/link for me.
What happens if you navigate to
http://sourceforge.net/tracker/?group_id=5470&atid=105470
? Do you have cookies enabled in your browser?
Regards,
Martin
From dave at boost-consulting.com Fri Sep 12 08:38:49 2003
From: dave at boost-consulting.com (David Abrahams)
Date: Fri Sep 12 07:39:57 2003
Subject: [Python-Dev] PyUnicode_FromEncodedObject
In-Reply-To: (Martin v.
=?iso-8859-1?q?L=F6wis's?= message of "12 Sep 2003 07:55:47 +0200")
References:
Message-ID:
martin@v.loewis.de (Martin v. L?wis) writes:
> David Abrahams writes:
>
>> > It does not insist, it just explains that you have logged
>> > on. Unfortunately, there is no way to make this message disappear once
>> > you have logged on, so all of us see it all the time.
>>
>> No, seriously. There is no "submit new" button/link for me.
>
> What happens if you navigate to
>
> http://sourceforge.net/tracker/?group_id=5470&atid=105470
>
> ? Do you have cookies enabled in your browser?
Yep. Submitted now.
--
Dave Abrahams
Boost Consulting
www.boost-consulting.com
From python at rcn.com Fri Sep 12 14:17:18 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Sep 12 14:48:13 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
Message-ID: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
[Raymond]
> > I would like to backport this patch to Py2.3.1.
> > The effort to provide a full mapping interface to all
> > mapping like objects was attempted in Py2.3 and several
> > modules for the bsddb package were updated, but this
> > one was missed and the package was left half converted.
> >
> > IIRC, dbhash and bsddb don't affect the Apple MacIntosh
> > users. Also, since this effort was started for bsddb and
> > only half completed, I view it to be a bit of a bugfix as
> > well as being featurelike. It certainly affects the usability
> > of the module (the looping example and related text in
> > the docs were both wrong -- that would not have happened
> > if the normal looping expectations were supported).
[GvR]
> Can you discuss this on python-dev?
Guys, are you okay with backporting this?
Raymond Hettinger
Index: __init__.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/bsddb/__init__.py,v
retrieving revision 1.5
retrieving revision 1.6
diff -C2 -d -r1.5 -r1.6
*** __init__.py 24 Apr 2003 16:02:44 -0000 1.5
--- __init__.py 12 Sep 2003 06:33:37 -0000 1.6
***************
*** 53,58 ****
#----------------------------------------------------------------------
! class _DBWithCursor:
"""
A simple wrapper around DB that makes it look like the bsddbobject in
--- 53,59 ----
#----------------------------------------------------------------------
+ import UserDict
! class _DBWithCursor(UserDict.DictMixin):
"""
A simple wrapper around DB that makes it look like the bsddbobject in
***************
*** 145,148 ****
--- 146,157 ----
return self.db.sync()
+ def __iter__(self):
+ try:
+ yield self.first()[0]
+ next = self.next
+ while 1:
+ yield next()[0]
+ except _bsddb.DBNotFoundError:
+ return
#----------------------------------------------------------------------
From skip at pobox.com Fri Sep 12 15:02:13 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Sep 12 15:02:27 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
Message-ID: <16226.6197.573407.585746@montanaro.dyndns.org>
>> > IIRC, dbhash and bsddb don't affect the Apple MacIntosh users.
I don't know what you mean here. Do you mean Mac OS 9 or Mac OS X? I
use bsddb all the time on Mac OS X.
Raymond> [GvR]
>> Can you discuss this on python-dev?
Raymond> Guys, are you okay with backporting this?
What are we discussing? Just adding an iterator to objects opened with the
old bsddb API?
Skip
From python at rcn.com Fri Sep 12 15:19:01 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Sep 12 15:23:55 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
<16226.6197.573407.585746@montanaro.dyndns.org>
Message-ID: <005a01c37962$b94a0e20$30bc958d@oemcomputer>
[Raymond]
> >> > IIRC, dbhash and bsddb don't affect the Apple MacIntosh users.
[Skip]
> I don't know what you mean here. Do you mean Mac OS 9 or Mac OS X? I
> use bsddb all the time on Mac OS X.
Hmm, I'll have to fix the docs too. They currently list bsddb as being
only for Unix and Windows.
[Skip]
> What are we discussing? Just adding an iterator to objects opened with the
> old bsddb API?
Yes. Iterators and the rest of the mapping API (__iter__, pop,
popitem, iteritems, setdefault, etc).
Like dumbdbm and shelve, this is done by including UserDict.DictMixin
as a base class.
Raymond Hettinger
#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################
From skip at pobox.com Fri Sep 12 15:55:44 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Sep 12 16:10:09 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <005a01c37962$b94a0e20$30bc958d@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
<16226.6197.573407.585746@montanaro.dyndns.org>
<005a01c37962$b94a0e20$30bc958d@oemcomputer>
Message-ID: <16226.9408.548163.145368@montanaro.dyndns.org>
>> I don't know what you mean here. Do you mean Mac OS 9 or Mac OS X?
>> I use bsddb all the time on Mac OS X.
Raymond> Hmm, I'll have to fix the docs too. They currently list bsddb
Raymond> as being only for Unix and Windows.
For non-GUI stuff Mac OS X is more Unix than Macintosh. ;-)
Raymond> Yes. Iterators and the rest of the mapping API (__iter__, pop,
Raymond> popitem, iteritems, setdefault, etc).
Raymond> Like dumbdbm and shelve, this is done by including
Raymond> UserDict.DictMixin as a base class.
I say go for it unless there's a serious backward compatibility problem. I
suspect the vast majority of the people who use bsddb do so through the
anydbm module, so if usage that way doesn't break you should be okay.
Skip
From jeremy at alum.mit.edu Fri Sep 12 16:44:15 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri Sep 12 16:44:29 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
Message-ID: <1063399454.2069.299.camel@localhost.localdomain>
On Fri, 2003-09-12 at 14:17, Raymond Hettinger wrote:
> [Raymond]
> > > I would like to backport this patch to Py2.3.1.
> > > The effort to provide a full mapping interface to all
> > > mapping like objects was attempted in Py2.3 and several
> > > modules for the bsddb package were updated, but this
> > > one was missed and the package was left half converted.
> > >
> > > IIRC, dbhash and bsddb don't affect the Apple MacIntosh
> > > users. Also, since this effort was started for bsddb and
> > > only half completed, I view it to be a bit of a bugfix as
> > > well as being featurelike. It certainly affects the usability
> > > of the module (the looping example and related text in
> > > the docs were both wrong -- that would not have happened
> > > if the normal looping expectations were supported).
>
> [GvR]
> > Can you discuss this on python-dev?
>
> Guys, are you okay with backporting this?
Isn't this just the sort of little feature that was the subject of
recent discussion on the dangers of backporting. It would mean that
someone writing an app against Python 2.3 could never be sure whether
the feature existed. In practice, that would mean developers using
2.3.x would only find out about problems after deployment when they bump
into a user with the original 2.3.
I'm not sure how convincing I find this argument, but it's got some
merit.
Jeremy
From python at rcn.com Fri Sep 12 19:50:40 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Sep 12 20:11:51 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
<1063399454.2069.299.camel@localhost.localdomain>
Message-ID: <008a01c3798a$f7198b40$b439c797@oemcomputer>
[Raymond]
> > > > I would like to backport this patch to Py2.3.1.
> > > > The effort to provide a full mapping interface to all
> > > > mapping like objects was attempted in Py2.3 and several
> > > > modules for the bsddb package were updated, but this
> > > > one was missed and the package was left half converted.
> > > >
> > > > IIRC, dbhash and bsddb don't affect the Apple MacIntosh
> > > > users. Also, since this effort was started for bsddb and
> > > > only half completed, I view it to be a bit of a bugfix as
> > > > well as being featurelike. It certainly affects the usability
> > > > of the module (the looping example and related text in
> > > > the docs were both wrong -- that would not have happened
> > > > if the normal looping expectations were supported).
[GvR]
> > > Can you discuss this on python-dev?
[Raymond]
> > Guys, are you okay with backporting this?
[Jeremy]
> Isn't this just the sort of little feature that was the subject of
> recent discussion on the dangers of backporting. It would mean that
> someone writing an app against Python 2.3 could never be sure whether
> the feature existed. In practice, that would mean developers using
> 2.3.x would only find out about problems after deployment when they bump
> into a user with the original 2.3.
>
> I'm not sure how convincing I find this argument, but it's got some
> merit.
It has some. That's why GvR had me bring it to python-dev
so you guys could help decide on the proper balance.
But for that one thought, the decision to apply is clear cut.
Thoughts in favor of applying:
* the existing interface is a pain and is misdocumented in Py2.3.0
* the patch was already half complete for 2.3.0 (applied
to dbobj and dbshelve) __init__.py was just missed).
* the patch was already applied to dumbdbm and shelve for Py2.3.0
* if needed, it's not hard to write code that automatically
adjusts for Py2.3.0:
if not f.__class__.__bases__:
f.__class__.__bases = (UserDict.DictMixin,)
* waiting another 18 months to put this in just results in that
many more installed Pythons out there without the patch
Raymond
From martin at v.loewis.de Sat Sep 13 00:52:37 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Sep 13 00:52:50 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <008a01c3798a$f7198b40$b439c797@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
<1063399454.2069.299.camel@localhost.localdomain>
<008a01c3798a$f7198b40$b439c797@oemcomputer>
Message-ID:
"Raymond Hettinger" writes:
> It has some. That's why GvR had me bring it to python-dev
> so you guys could help decide on the proper balance.
> But for that one thought, the decision to apply is clear cut.
There is no proper balance. If you believe in the policy "no new
features", this policy is absolute - it would be pointless to accept
some new features, but reject others on grounds of the policy.
This policy is mutually exclusive with "new features are fine as long
as they don't break anything". You can get opinions, but you cannot
get consensus.
Regards,
Martin
From python at rcn.com Sat Sep 13 02:14:12 2003
From: python at rcn.com (Raymond Hettinger)
Date: Sat Sep 13 02:19:06 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer><1063399454.2069.299.camel@localhost.localdomain><008a01c3798a$f7198b40$b439c797@oemcomputer>
Message-ID: <007c01c379be$40efa460$b439c797@oemcomputer>
[Raymond]
> > It has some. That's why GvR had me bring it to python-dev
> > so you guys could help decide on the proper balance.
> > But for that one thought, the decision to apply is clear cut.
[Martin]
> There is no proper balance. If you believe in the policy "no new
> features", this policy is absolute - it would be pointless to accept
> some new features, but reject others on grounds of the policy.
>
> This policy is mutually exclusive with "new features are fine as long
> as they don't break anything". You can get opinions, but you cannot
> get consensus.
Well put!
Even bugfixes are features in that something will work under Py2.3.1
that won't work under Py2.3.0. So, even an absolutist policy on our
end shouldn't create a false sense of security or relieve a developer
from testing his or her application on the lowest numbered Python
for which it is claimed to run (including micro-releases).
That has always been true. If you developed something under Py2.2.3
and expected it to run under Py2.2.0, the only way to be sure would be
to test it with Py2.2.0.
In this particular case, we are (IMHO) much better off adding an
iterable interface than leaving current situation with "db.first()"
and "while 1: db.next()" wrapped in a try/except for an
undocumented exception class -- yuck. Worse still, the docs
for Py2.3.0 cannot be changed and they are wrong on several
counts (the methods return items instead of keys; EOF is signaled
by an exception instead of None; and the example doesn't run).
Raymond
#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################
From arigo at tunes.org Sat Sep 13 09:46:53 2003
From: arigo at tunes.org (Armin Rigo)
Date: Sat Sep 13 09:48:38 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators
In-Reply-To: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
References: <5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
<00a401c36c0f$76952160$c304a044@oemcomputer>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
Message-ID: <20030913134653.GA12752@vicky.ecs.soton.ac.uk>
Hello,
(My reply didn't seem to have reached python-dev... second try.)
On Tue, Aug 26, 2003 at 01:50:52PM -0700, Guido van Rossum wrote:
> Another comment on Samuele's PEP: It is sort of sad that the *user* of
> a generator has to know that the generator's close() must be called.
> Normally, the beauty of using a try/finally for cleanup is that your
> callers don't need to know about it. But I see no way around this.
What about letting the 'for' handle this? It's the most common way generators
are used. When a 'for' loop on a generator-iterator finishes it would call the
close() method of the iterator, which on generators would (say) simulate a
return from the latest 'yield'.
Well, close() might not be such a good name because it would probably break
exsiting code (e.g. closing files unexpectedly), but __exit__() might do. In
other words we could import some of the proposed functionality of the 'with'
keyword (PEP 310) into 'for'. I think it makes sense because 'for' is already
defined in term of implicit calls to next(), the only method of iterators; so
if a second method is added, 'for' can be taught about it too.
Armin
From pedronis at bluewin.ch Sat Sep 13 10:18:51 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sat Sep 13 10:17:11 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for
Generators
In-Reply-To: <20030913134653.GA12752@vicky.ecs.soton.ac.uk>
References: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
<00a401c36c0f$76952160$c304a044@oemcomputer>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
Message-ID: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
At 14:46 13.09.2003 +0100, Armin Rigo wrote:
>Hello,
>
>(My reply didn't seem to have reached python-dev... second try.)
>
>On Tue, Aug 26, 2003 at 01:50:52PM -0700, Guido van Rossum wrote:
> > Another comment on Samuele's PEP: It is sort of sad that the *user* of
> > a generator has to know that the generator's close() must be called.
> > Normally, the beauty of using a try/finally for cleanup is that your
> > callers don't need to know about it. But I see no way around this.
>
>What about letting the 'for' handle this? It's the most common way generators
>are used. When a 'for' loop on a generator-iterator finishes it would call the
>close() method of the iterator, which on generators would (say) simulate a
>return from the latest 'yield'.
>
>Well, close() might not be such a good name because it would probably break
>exsiting code (e.g. closing files unexpectedly), but __exit__() might do. In
>other words we could import some of the proposed functionality of the 'with'
>keyword (PEP 310) into 'for'. I think it makes sense because 'for' is already
>defined in term of implicit calls to next(), the only method of iterators; so
>if a second method is added, 'for' can be taught about it too.
I expect generators to grow also a __exit__ if they grow a close and PEP
310 is accepted,
this idiom will then be common
with g = gen()
for v in g:
...
but it will be common for files and file-like too. It can be reasonable to
discuss
whether we want a special syntax then that conflates 'with' and 'for' behavior.
But we can experiment a while before seeing whether writing the idiom is
unbearably
tedious ;).
OTOH conflating 'with' and 'for' just for generators seems a rather ad-hoc
breaking of
orthoganility of the two, you could not write anymore code like this:
g = gen()
for v in g:
... do something up to a point ...
...
for v in g:
...
now this is rare but still breaking orthoganility of primitives is
something I would think twice about.
regards.
From pedronis at bluewin.ch Sat Sep 13 10:23:20 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sat Sep 13 10:21:40 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for
Generators
In-Reply-To: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
References: <20030913134653.GA12752@vicky.ecs.soton.ac.uk>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
<00a401c36c0f$76952160$c304a044@oemcomputer>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
Message-ID: <5.2.1.1.0.20030913162229.026d3248@pop.bluewin.ch>
>
>
>OTOH conflating 'with' and 'for' just for generators seems a rather ad-hoc
>breaking of
>orthoganility of the two, you could not write anymore code like this:
sorry, hmm, orthogonality that I know what is is.
From worktogether at runbox.com Sat Sep 13 18:42:54 2003
From: worktogether at runbox.com (Dr. Newell)
Date: Sat Sep 13 18:42:52 2003
Subject: [Python-Dev] To You as a Nicer Person
Message-ID: <417-220039613224254410@runbox.com>
Dear python-dev:
You may ask yourself
> How to make people like you and respect you
> How to win friends
> How to let your conduct help your health, work, job, career, success, relationships, spirit, mind, well-being, ...
> How to make your life smoother and happier
> How to do whatever you like without being unpleasant to other people
> How to develop good conduct in your children or students
> How to make the world peaceful and better
You can find all the answers to these questions, and much more, in this great handbook:
" Complete Conduct Principles for the 21st Century " by Dr. John Newton
It is the best educational GIFT idea for children, friends, relatives, classmates, students, parents, teachers, other educators, ..., particularly at this special time.
BENEFITS to each individual reader: Many! -- such as for health, work, job, career, success, self-improvement, education, relationships, spirit, mind, well-being, and much more -- almost all the areas that are important to you in the 21st century. People around you will benefit, too. (Please see the preface of the book for details.)
EVERYONE may find this handbook useful and helpful, regardless of age (from children to oldsters), occupation, rank, status, gender, religious belief, race, nationality, country, or region.
If you are a parent or a teacher, you can learn how to develop good conduct in your children or students from this handbook. Please advise your children or students to read the book. It will result in great benefits for both you and them.
This book is a must for EVERYONE to be better prepared for personal conduct for the rest of the 21st century.
The book's content is obvious from its title. The complete useful conduct principles cover not only what we should do, but also what we should not do -- especially those faults people make often and easily.
This timely, unique, and very important handbook is designed to suit most people, and is self-contained and user-friendly.
The book was also praised as "a compendium of concisely expressed, practical, informative, pertinent, workable advice" by Michael J. Carson, a professional book reviewer.
This book is significantly different and better than competitive works.
"Unlike most books of this subject, it is NOT a religious book, nor is a collection of old conduct rules."
Some of its innovative contents may help solve problems that the Western culture cannot.
The book's merit and importance have been recognized and praised by many experts, elected public officials, and world leaders.
As a result of popular demand, the book has been a top "Featured Item" in some top on-line bookstores in some subjects, such as "Personal Practical Guides" in "Nonfiction" and "Reference", "Conduct" in "Health, Mind & Body", "Life skills guides", "Ethics & Moral Philosophy", ...
"The book will also be effective for violence prevention for the whole society." said some experts.
How to make the world peaceful and better ---
You can find the solution in the book.
Let's work together to make the world peaceful and better!
The author, John Newton, holds a Ph.D. from MIT, and does researches at Harvard. His long-term research on "The personal conduct in the human society of the 21st century" resulted in this book.
Before the human beings went into the 21st century, the compassionate, merciful, courageous and farsighted Dr. Newton had issued a number of warning predictions, some of which have already been proved in the new century.
The book is published by NCWO, headquartered beside Harvard University and MIT, two leading institutes of new knowledge and literature. The publisher has a deep commitment to publishing only the best.
NCWO is an educational, non-profit, non-partisan, and honorary organization; it endeavors to make the 21st century nicer than ever before. To accomplish its mission, NCWO is proud to introduce this book.
The Web site of NCWO has been chosen as one of "Top Non Profit Sites Chosen by Type Non Profit Editors".
The book is available in two types of binding: Hardcover (ISBN 0967370574; case bound, Smyth sewn; with dust jacket) and Paperback (ISBN 0967370582; perfect bound). Both editions are unabridged, and are printed on 60 lb, natural, acid-free, excellent and healthful paper. You can get the book from many fine on-line bookstores and traditional bookstores.
For your convenience, (if you wish for an Internet link; otherwise you may skip this section) I herewith provide you with a link directly to the book page of each edition in Half.com by eBay, a popular on-line discount mall:
for paperback:
http://half.ebay.com/cat/buy/prod.cgi?cpid=2425993
for hardcover:
http://half.ebay.com/cat/buy/prod.cgi?cpid=2425992
Please forward this e-mail to people you know -- children, friends, relatives, classmates, students, parents, teachers, other educators, ..., because they can benefit from it, too. This can be a wonderful kindness you provide to them!
Sincerely yours,
(python-dev, best wishes to you!)
Christopher Newell, Ph.D.
Cambridge, Massachusetts, USA
P.S.
Some educational units, ranging from the level of nation or state to individual school or university, have ordered the book as textbook, reference book, gift to students, or as an active action to prevent school violence, to improve education and to benefit students, teachers & parents.
To have more people benefit from the book, please consider suggesting to the schools -- your children attend, or you yourself attend, have attended before, teach at, or serve -- that the book be used for fundraising for the schools. The book is an ideal fundraising tool. For example, it may used as a premium or a re-sale product for the fundraising. The successful fundraising will significantly help school education. Better yet, each supporter and his/her family will benefit from the book. Suggesting to the parent-teacher associations (organizations) (PTA/PTO) of the schools is also a good idea.
From raymond.hettinger at verizon.net Sat Sep 13 21:05:18 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat Sep 13 21:27:13 2003
Subject: [Python-Dev] SF #805304: bug or feature?
Message-ID: <000201c37a5e$aa6b9e80$ed24c797@oemcomputer>
Fred reported that super objects don't respond well to language constructs
that bypass attribute lookup. For example, supobj.__setitem__(name,value)
works if __setitem__() is defined in the target, but supobj[name]=value
will raise a TypeError. www.python.org/sf/805304
Once the fix is approved, there is a question of whether it should
be backported. The case against it is that some programs written
under Py2.3.1 or Py2.2.4 won't run on Py2.3.0 or Py2.2.3 and the
patch can be viewed as an API expansion. The other point of view
is that super objects should have always behaved this way and that
the patch just fixes buggy behavior.
Raymond
From skip at mojam.com Sun Sep 14 08:00:34 2003
From: skip at mojam.com (Skip Montanaro)
Date: Sun Sep 14 08:00:43 2003
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200309141200.h8EC0Yi25498@manatee.mojam.com>
Bug/Patch Summary
-----------------
475 open / 4113 total bugs (+10)
193 open / 2371 total patches (+8)
New Bugs
--------
cgi.CGIHTTPRequestHandler remembers QUERY_STRING (2003-09-07)
http://python.org/sf/801992
Mode argument of dumbdbm does not work (2003-09-07)
http://python.org/sf/802128
tkFont may reuse font names (2003-09-08)
http://python.org/sf/802310
Bogus use of Tkinter.PhotoImage can give sig11 (2003-09-08)
http://python.org/sf/802417
uu.decode prints to stderr (2003-09-09)
http://python.org/sf/803413
sgmllib doesn't support hex or Unicode character references (2003-09-09)
http://python.org/sf/803422
plat-mac/applesingle.py needs cosmetic changes (2003-09-09)
http://python.org/sf/803498
urllib/urllib2(?) timeouts (2003-09-10)
http://python.org/sf/803634
Crypto terminology for crypto hash function (2003-09-10)
http://python.org/sf/804113
invalid use of setlocale (2003-09-11)
http://python.org/sf/804543
PyUnicode_FromEncodedObject (2003-09-12)
http://python.org/sf/805015
Inappropriate error received using socket timeout (2003-09-12)
http://python.org/sf/805194
_tkinter compilation fails (2003-09-12)
http://python.org/sf/805200
super instances don't support item assignment (2003-09-12)
http://python.org/sf/805304
IDLE can't save UTF-8 files on Windows (2003-09-13)
http://python.org/sf/805728
Minor Python Intro update (2003-09-13)
http://python.org/sf/805788
New Patches
-----------
Suggested change in how ntpath.expanduser works (2003-09-07)
http://python.org/sf/802159
better parser error message for non-EOL following line cont. (2003-09-07)
http://python.org/sf/802188
[_ssl.c] SSL_write() called with -1 as size (2003-09-10)
http://python.org/sf/803998
A ForwardingHandler for logging (2003-09-10)
http://python.org/sf/804180
unittest's main don't handle TestSuite in command line (2003-09-10)
http://python.org/sf/804212
Lib/email/Encoders.py iso-2022 is 7bit (2003-09-11)
http://python.org/sf/804885
fix --without-threads (2003-09-13)
http://python.org/sf/805604
PTH related fixes (2003-09-13)
http://python.org/sf/805613
make test_fcntl 64bit clean (2003-09-13)
http://python.org/sf/805626
Fix linking with shared libpython (2003-09-13)
http://python.org/sf/805678
Highlight builtins (2003-09-13)
http://python.org/sf/805830
Closed Bugs
-----------
Add docs for 'basestring' (2002-05-24)
http://python.org/sf/560286
htmllib.HTMLParser.anchorlist problem (2003-03-28)
http://python.org/sf/711632
2 problems with IDLE (2003-07-21)
http://python.org/sf/775061
RESET_ERROR is not defined(logging module) (2003-08-21)
http://python.org/sf/792649
select module doesn't allow any iterable. (2003-08-31)
http://python.org/sf/798046
test_dumbdbm failing (2003-09-04)
http://python.org/sf/800824
Making "|" directive from REs a bit clearer (2003-09-04)
http://python.org/sf/800899
Bad RE in scanf example (2003-09-05)
http://python.org/sf/801306
Closed Patches
--------------
Allow os.access to handle Unicode file name (2003-08-17)
http://python.org/sf/790000
implement htmllib.HTMLParser.reset (2003-08-22)
http://python.org/sf/793021
zipimport 64-bit fix (2003-09-06)
http://python.org/sf/801821
From martin at v.loewis.de Sun Sep 14 18:08:57 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Sep 14 18:09:10 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <007c01c379be$40efa460$b439c797@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
<1063399454.2069.299.camel@localhost.localdomain>
<008a01c3798a$f7198b40$b439c797@oemcomputer>
<007c01c379be$40efa460$b439c797@oemcomputer>
Message-ID:
"Raymond Hettinger" writes:
> In this particular case, we are (IMHO) much better off adding an
> iterable interface than leaving current situation with "db.first()"
> and "while 1: db.next()" wrapped in a try/except for an
> undocumented exception class -- yuck.
As nobody has protested against this specific change (yet?), I guess
you can go ahead and apply it. In doing so, you have to accept the
burden of taking blame in case this turns out to be a bad idea.
> Worse still, the docs for Py2.3.0 cannot be changed and they are
> wrong on several counts (the methods return items instead of keys;
> EOF is signaled by an exception instead of None; and the example
> doesn't run).
That would be a documentation bug. However, documentation bugs are
generally best fixed by correcting the documentation, not by adjusting
the code to match the documentation (atleast, for maintenance
releases).
Regards,
Martin
From martin at v.loewis.de Sun Sep 14 18:12:42 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Sep 14 18:12:42 2003
Subject: [Python-Dev] SF #805304: bug or feature?
In-Reply-To: <000201c37a5e$aa6b9e80$ed24c797@oemcomputer>
References: <000201c37a5e$aa6b9e80$ed24c797@oemcomputer>
Message-ID:
"Raymond Hettinger" writes:
> Once the fix is approved, there is a question of whether it should
> be backported. The case against it is that some programs written
> under Py2.3.1 or Py2.2.4 won't run on Py2.3.0 or Py2.2.3 and the
> patch can be viewed as an API expansion. The other point of view
> is that super objects should have always behaved this way and that
> the patch just fixes buggy behavior.
Whether it is a bug or not can be best clarified by looking at the
documentation: If it is documented that attributes of super objects
always follow MRO, then there is a bug. That might be a doc bug, but
Special cases aren't special enough to break the rules.
makes clear it wouldn't be a doc bug, but a bug in the code.
OTOH, if this is not documented at all, you clearly have a doc bug.
Regards,
Martin
From jeremy at alum.mit.edu Sun Sep 14 18:21:15 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Sun Sep 14 18:21:33 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To:
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
<1063399454.2069.299.camel@localhost.localdomain>
<008a01c3798a$f7198b40$b439c797@oemcomputer>
<007c01c379be$40efa460$b439c797@oemcomputer>
Message-ID: <1063578075.2093.67.camel@localhost.localdomain>
On Sun, 2003-09-14 at 18:08, Martin v. L?wis wrote:
> "Raymond Hettinger" writes:
>
> > In this particular case, we are (IMHO) much better off adding an
> > iterable interface than leaving current situation with "db.first()"
> > and "while 1: db.next()" wrapped in a try/except for an
> > undocumented exception class -- yuck.
>
> As nobody has protested against this specific change (yet?), I guess
> you can go ahead and apply it. In doing so, you have to accept the
> burden of taking blame in case this turns out to be a bad idea.
Perhaps we could wait to hear from Jack or Just or someone who was
concerned about feature creep in maintenance releases. I think we
should decide now whether to follow the policy they propose or we should
drop it. Right now, it feels like it's in limbo.
Jeremy
From tim.one at comcast.net Sun Sep 14 18:36:09 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 14 18:37:57 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063578075.2093.67.camel@localhost.localdomain>
Message-ID:
[Jeremy]
> Perhaps we could wait to hear from Jack or Just or someone who was
> concerned about feature creep in maintenance releases. I think we
> should decide now whether to follow the policy they propose or we
> should drop it. Right now, it feels like it's in limbo.
Guido needs to set the direction here. Slamming new features into micro
releases became irresistible after the 1.5.2 experience, where a very long
time passed without a new Python release (not even a bugfix release). That
naturally caused people to want to get every improvement in ASAP.
We were able to counteract that successfully in the early days of
PythonLabs, because then we pumped out two minor releases per year, and it
was much easier to sell "leave the micro release a pure bugfix release --
the next minor release is only a few months away".
Then 2.3 stretched out over 18 months, and there's no plan for 2.4 in
evidence yet. Given this most recent history, and the unlikelihood that
PythonLabs's (or a workalike equivalent's) early days will repeat itself
soon, if I had a minor feature I really wanted to get in, I'd try to get it
into a 2.3 micro release.
OTOH, if Guido decides to go back to a schedule-driven release for 2.4, and
it's not terribly far in the future, then it will again be much easier to
sell 2.3 micro releases as bugfix-only (and stick to that).
We're all pretty creative about what we'll call "a bug", when it suits a
feature we believe in <0.7 wink>.
From niemeyer at conectiva.com Sun Sep 14 20:06:37 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Sun Sep 14 20:07:14 2003
Subject: [Python-Dev] datetime issues
Message-ID: <20030915000637.GC27698@ibook>
Greetings!
Is there any reason for not having a single keyword parameter for
every function accepting a tzinfo instance? I've noticed that some
require 'tz', and others 'tzinfo'. Is it too late to fix that?
Also, is there any further work going on to improve the datetime module?
I'm not suggesting we should mirror mx.DateTime functionalities, but two
features I miss from mx.DateTime is the DateFrom(), which does its best
to parse a given string, and the RelativeDateTime() functionality.
This has probably been raised in the past, so if that's the case, I'm
sorry. Marc-Andre, what's your opinion about reusing code from
mx.DateTime into datetime?
Anyway, reusing code or not, I'll probably put sometime on it in the
future, if this looks interesting to everyone (after I finish my current
python pendencies, like SRE's recursivity removal).
--
Gustavo Niemeyer
http://niemeyer.net
From tim.one at comcast.net Sun Sep 14 20:35:35 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Sep 14 20:37:26 2003
Subject: [Python-Dev] datetime issues
In-Reply-To: <20030915000637.GC27698@ibook>
Message-ID:
[Gustavo Niemeyer]
> Is there any reason for not having a single keyword parameter for
> every function accepting a tzinfo instance?
Just the lack of anyone pointing it out in the module's 18-month review
process .
> I've noticed that some require 'tz', and others 'tzinfo'. Is it too
> late to fix that?
Of course, although it's not too late to add synonyms. It appears that the
functions "with a lot of arguments" use tzinfo, and those with only one or
two arguments tz now. I doubt anyone would bother to use the tz keyword-arg
name for a few-argument function (now(), fromtimestamp(), astimezone()), so
I'd rather that those few grow a tzinfo synonym. tz can be deprecated too,
if you're bold.
> Also, is there any further work going on to improve the datetime
> module?
Not that I'm aware of. I'm not working on it anymore (other than keeping my
out for bug reports, of which there have been blessedly few).
> I'm not suggesting we should mirror mx.DateTime functionalities, but
> two features I miss from mx.DateTime is the DateFrom(), which does its
> best to parse a given string, and the RelativeDateTime() functionality.
>
> This has probably been raised in the past, so if that's the case, I'm
> sorry. Marc-Andre, what's your opinion about reusing code from
> mx.DateTime into datetime?
>
> Anyway, reusing code or not, I'll probably put sometime on it in the
> future, if this looks interesting to everyone (after I finish my
> current python pendencies, like SRE's recursivity removal).
If Guido wanted to stay out of the relatively clear time zone
business, I can imagine his interest in trying to guess how to parse date
and time strings. Apart from Brett's strptime(), I believe the email module
has some useful string->datetime code. It may be good to fold such stuff in
this area as Python already supports into datetime somehow.
From arigo at tunes.org Mon Sep 15 06:48:36 2003
From: arigo at tunes.org (Armin Rigo)
Date: Mon Sep 15 06:50:28 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators
In-Reply-To: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
References: <200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
<00a401c36c0f$76952160$c304a044@oemcomputer>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
<5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
Message-ID: <20030915104836.GA1143@vicky.ecs.soton.ac.uk>
Hello Samuele,
On Sat, Sep 13, 2003 at 04:18:51PM +0200, Samuele Pedroni wrote:
> OTOH conflating 'with' and 'for' just for generators seems a rather ad-hoc
> breaking of
> orthoganility of the two, you could not write anymore code like this:
>
> g = gen()
> for v in g:
> ... do something up to a point ...
> ...
> for v in g:
> ...
I had thought about this. This occurs when you 'break' out of the first loop.
I would say that NOT calling the __exit__() method in this specific case seems
quite intuitive, the 'break' meaning 'just exit from the loop now without any
further processing, skipping the 'else' part if present'.
Armin
From sholden at holdenweb.com Mon Sep 15 07:59:36 2003
From: sholden at holdenweb.com (Steve Holden)
Date: Mon Sep 15 08:03:00 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To:
Message-ID:
> -----Original Message-----
> From: python-dev-bounces@python.org
> [mailto:python-dev-bounces@python.org]On Behalf Of Tim Peters
> Sent: Sunday, September 14, 2003 6:36 PM
> To: jeremy@alum.mit.edu; Martin v. L?wis
> Cc: Raymond Hettinger; python-dev@python.org
> Subject: RE: [Python-Dev] python/dist/src/Lib/bsddb
> __init__.py,1.5,1.6
>
>
> [Jeremy]
> > Perhaps we could wait to hear from Jack or Just or someone who was
> > concerned about feature creep in maintenance releases. I think we
> > should decide now whether to follow the policy they propose or we
> > should drop it. Right now, it feels like it's in limbo.
>
> Guido needs to set the direction here. Slamming new features
> into micro
> releases became irresistible after the 1.5.2 experience,
> where a very long
> time passed without a new Python release (not even a bugfix
> release). That
> naturally caused people to want to get every improvement in ASAP.
>
> We were able to counteract that successfully in the early days of
> PythonLabs, because then we pumped out two minor releases per
> year, and it
> was much easier to sell "leave the micro release a pure
> bugfix release --
> the next minor release is only a few months away".
>
> Then 2.3 stretched out over 18 months, and there's no plan for 2.4 in
> evidence yet. Given this most recent history, and the
> unlikelihood that
> PythonLabs's (or a workalike equivalent's) early days will
> repeat itself
> soon, if I had a minor feature I really wanted to get in, I'd
> try to get it
> into a 2.3 micro release.
>
> OTOH, if Guido decides to go back to a schedule-driven
> release for 2.4, and
> it's not terribly far in the future, then it will again be
> much easier to
> sell 2.3 micro releases as bugfix-only (and stick to that).
>
> We're all pretty creative about what we'll call "a bug", when
> it suits a
> feature we believe in <0.7 wink>.
>
So, what we need is a plan for a 2.4 release, then. How soon would it
need to be, though? And who can commit time to making it happen. Seems
to me that some formerly active core developers are currently busy in
other realms. Is it feasible to have a 2.4 plan, or are we ready to
contemplate (gasp) 3.0? What help can such as I, not a practiced C coder
and fairly heavily committed and thinly spread, be?
regards
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/pwp/
Interview with GvR August 14, 2003 http://www.onlamp.com/python/
From mcherm at mcherm.com Mon Sep 15 08:04:51 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Sep 15 08:04:56 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
Message-ID: <1063627491.3f65aae37a3b7@mcherm.com>
[Jeremy]
> Isn't this just the sort of little feature that was the subject of
> recent discussion on the dangers of backporting. It would mean that
> someone writing an app against Python 2.3 could never be sure whether
> the feature existed. In practice, that would mean developers using
> 2.3.x would only find out about problems after deployment when they bump
> into a user with the original 2.3.
>
> I'm not sure how convincing I find this argument, but it's got some
> merit.
[Martin]
> There is no proper balance. If you believe in the policy "no new
> features", this policy is absolute - it would be pointless to accept
> some new features, but reject others on grounds of the policy.
I don't really understand this absolutist position. Suppose (hypothetical
example made extreme for retorical purposes) we discover that there was
a HUGE bug in release 2.3. Using a variable named "Q" as the argument of
a for loop causes the interpreter to seg-fault. This is CLEARLY a bad
thing... capital one-letter variable names are uncommon, but not THAT
uncommon! So I suppose that 2.3.1 would need to include a fix for this
bug.
But THEN anyone writing code for 2.3.x could never be SURE whether the
bug existed. If they used a capital Q in a for loop, their code would
eventually be run by somebody with 2.3.0, causing a crash. The only way
to be safe would be to test their code on 2.3.0. How is this "bug fix"
any different from a "new feature"?
I understand the argument "avoid new features in maintanence releases
because there we value stability FAR more than we value features". I
understand the argument "avoid new features in maintanence releases
because features should be added on the main trunk to encourage the
(eventual) use of new versions of Python". But I don't understand
the argument "PROHIBIT new features in maintanence releases because
then programs written under the maintanence release might not run
under the original release", because to me it seems to be logically
equivalent to "prohibit maintanence releases".
[Martin]
> This policy is mutually exclusive with "new features are fine as long
> as they don't break anything". You can get opinions, but you cannot
> get consensus.
If "consensus" is taken to mean "universal agreement", then I'd agree,
but if we mean "general agreement or accord" (ie, everyone gets heard
then NEARLY everyone agrees and the naysayers decide they can live
with it), then we might have a shot. I'd certainly say (and I think
most here would agree) that even ONE seriously intended firmly held
argument against a particular new feature (in a maint. release) would
make me think VERY carefully before allowing that feature.
-- Michael Chermside
From amk at amk.ca Mon Sep 15 08:42:58 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Sep 15 08:43:06 2003
Subject: [Python-Dev] Re: datetime issues
References: <20030915000637.GC27698@ibook>
Message-ID:
On Sun, 14 Sep 2003 20:35:35 -0400, Tim Peters wrote:
> If Guido wanted to stay out of the relatively clear time zone
> business, I can imagine his interest in trying to guess how to parse date
> and time strings. Apart from Brett's strptime(), I believe the email
> module
> has some useful string->datetime code.
The PyXML package includes a parser (xml.utils.iso8601) for the date/time
format most commonly used in XML applications. That and a parser for
RFC822 dates would meet the needs of many technical applications (though
not ones where users can enter dates -- but that problem may be too
complicated for the stdlib). I'll happily convert the iso8601 module for
addition to the standard library if date parsing is deemed a topic of
interest.
--amk
From amk at amk.ca Mon Sep 15 08:51:02 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Sep 15 08:51:19 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References:
Message-ID:
On Mon, 15 Sep 2003 07:59:36 -0400, Steve Holden
wrote:
> other realms. Is it feasible to have a 2.4 plan, or are we ready to
> contemplate (gasp) 3.0?
I'd be in favor of setting an arbitrary date for a 2.4 release, and when
that date rolls around stabilizing and releasing the CVS tree, no matter
what's been implemented or not implemented. If we've made major language
revisions by that time, fine, but it's worthwhile even if 2.4 is just 2.3 +
(various bugfixes deemed too risky for 2.3.1) + (a few new modules).
PEP 320 estimates February 2005, a 19-month gap matching that between 2.2
and 2.3. That seems a bit too long, especially if there are no large
changes that require testing; aiming for autumn 2004, beginning to freeze
in August, seems reasonable.
--amk
From barry at python.org Mon Sep 15 09:10:22 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 15 09:10:33 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To:
References:
Message-ID: <1063631421.10716.2.camel@anthem>
On Mon, 2003-09-15 at 08:51, A.M. Kuchling wrote:
> I'd be in favor of setting an arbitrary date for a 2.4 release, and when
> that date rolls around stabilizing and releasing the CVS tree, no matter
> what's been implemented or not implemented. If we've made major language
> revisions by that time, fine, but it's worthwhile even if 2.4 is just 2.3 +
> (various bugfixes deemed too risky for 2.3.1) + (a few new modules).
>
> PEP 320 estimates February 2005, a 19-month gap matching that between 2.2
> and 2.3. That seems a bit too long, especially if there are no large
> changes that require testing; aiming for autumn 2004, beginning to freeze
> in August, seems reasonable.
+1
-Barry
From barry at python.org Mon Sep 15 09:14:17 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 15 09:14:22 2003
Subject: [Python-Dev] Re: datetime issues
In-Reply-To:
References: <20030915000637.GC27698@ibook>
Message-ID: <1063631656.10716.7.camel@anthem>
On Mon, 2003-09-15 at 08:42, A.M. Kuchling wrote:
> I believe the email
> > module
> > has some useful string->datetime code.
>
> The PyXML package includes a parser (xml.utils.iso8601) for the date/time
> format most commonly used in XML applications. That and a parser for
> RFC822 dates would meet the needs of many technical applications (though
> not ones where users can enter dates -- but that problem may be too
> complicated for the stdlib). I'll happily convert the iso8601 module for
> addition to the standard library if date parsing is deemed a topic of
> interest.
Yep, I think email._parseaddr.parsedate{,_tz}() is an RFC 2822 compliant
date parser. Note that there's also Unix-from dates which are slightly
different.
I'd also like to see factored out various date formating functions, so
that the module is more or less symmetric.
-Barry
From nas-python at python.ca Mon Sep 15 10:47:12 2003
From: nas-python at python.ca (Neil Schemenauer)
Date: Mon Sep 15 10:39:57 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To:
References:
Message-ID: <20030915144712.GA7448@mems-exchange.org>
On Mon, Sep 15, 2003 at 08:51:02AM -0400, A.M. Kuchling wrote:
> I'd be in favor of setting an arbitrary date for a 2.4 release, and when
> that date rolls around stabilizing and releasing the CVS tree, no matter
> what's been implemented or not implemented.
+1
From tim.one at comcast.net Mon Sep 15 10:44:07 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Sep 15 10:44:18 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To:
Message-ID:
[Andrew Kuchling]
> I'd be in favor of setting an arbitrary date for a 2.4 release, and
> when that date rolls around stabilizing and releasing the CVS tree,
> no matter what's been implemented or not implemented. If we've made
> major language revisions by that time, fine, but it's worthwhile even
> if 2.4 is just 2.3 + (various bugfixes deemed too risky for 2.3.1) +
> (a few new modules).
>
> PEP 320 estimates February 2005, a 19-month gap matching that between
> 2.2 and 2.3. That seems a bit too long, especially if there are no
> large changes that require testing; aiming for autumn 2004, beginning
> to freeze in August, seems reasonable.
I'd like to see 2.4 released no later than 29-Jul-2004. That's a year after
2.3 got released. I don't have nearly as many years left to me as most of
you do .
From barry at python.org Mon Sep 15 10:55:31 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 15 10:55:37 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To:
References:
Message-ID: <1063637731.10716.25.camel@anthem>
On Mon, 2003-09-15 at 10:44, Tim Peters wrote:
> I'd like to see 2.4 released no later than 29-Jul-2004.
I like the tradition this sets. I can't think of a better birthday
present to one of the best Canadian bassists in the world.
-Barry
From jeremy at alum.mit.edu Mon Sep 15 10:57:02 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon Sep 15 10:56:56 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To:
References:
Message-ID: <1063637821.2093.132.camel@localhost.localdomain>
On Mon, 2003-09-15 at 08:51, A.M. Kuchling wrote:
> I'd be in favor of setting an arbitrary date for a 2.4 release, and when
> that date rolls around stabilizing and releasing the CVS tree, no matter
> what's been implemented or not implemented. If we've made major language
> revisions by that time, fine, but it's worthwhile even if 2.4 is just 2.3 +
> (various bugfixes deemed too risky for 2.3.1) + (a few new modules).
>
> PEP 320 estimates February 2005, a 19-month gap matching that between 2.2
> and 2.3. That seems a bit too long, especially if there are no large
> changes that require testing; aiming for autumn 2004, beginning to freeze
> in August, seems reasonable.
In the absence of any plans for major new features, I think this is a
good plan. A rough schedule might be:
2.4a1 April
2.4a2 late May/early June
2.4b1 July 9
2.4b2 Aug. 6
2.4 f Sep. 3
We should also make a plan for a 2.3 maintenance release. There have
been a lot of checkins, but I haven't gotten a sense for how significant
the bugs were.
Jeremy
From neal at metaslash.com Mon Sep 15 11:03:13 2003
From: neal at metaslash.com (Neal Norwitz)
Date: Mon Sep 15 11:03:21 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063637821.2093.132.camel@localhost.localdomain>
References:
<1063637821.2093.132.camel@localhost.localdomain>
Message-ID: <20030915150313.GI1240@epoch.metaslash.com>
On Mon, Sep 15, 2003 at 10:57:02AM -0400, Jeremy Hylton wrote:
>
> We should also make a plan for a 2.3 maintenance release. There have
> been a lot of checkins, but I haven't gotten a sense for how significant
> the bugs were.
I think 2.3 has been great! There were a few memory leaks plugged.
I don't even recall any crashes specific to 2.3. I'm very happy with
the apparent robustness of 2.3. Maybe nobody is using it? :-)
As far as the release schedule for 2.4, I haven't heard any squawking
recently about Python changing too fast. But then again, I have no
time to read c.l.p anymore. Maybe there's a correlation? :-)
Neal
From barry at python.org Mon Sep 15 11:08:37 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Sep 15 11:08:43 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <20030915150313.GI1240@epoch.metaslash.com>
References:
<1063637821.2093.132.camel@localhost.localdomain>
<20030915150313.GI1240@epoch.metaslash.com>
Message-ID: <1063638517.10716.32.camel@anthem>
On Mon, 2003-09-15 at 11:03, Neal Norwitz wrote:
> I think 2.3 has been great! There were a few memory leaks plugged.
> I don't even recall any crashes specific to 2.3. I'm very happy with
> the apparent robustness of 2.3. Maybe nobody is using it? :-)
I'm using it for everything, and I agree! It's a very solid release. I
still think we should plan for a 2.3.1 soonish, but the hold up may just
be finding a release manager to own it. I'd rather spread the love this
time. :)
> As far as the release schedule for 2.4, I haven't heard any squawking
> recently about Python changing too fast.
We can't let that stand, now can we? :) Maybe Guido can post about his
big plans for 2.4. There's still time to get PEPpy too.
-Barry
From anthony at interlink.com.au Mon Sep 15 11:11:35 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Sep 15 11:13:01 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063638517.10716.32.camel@anthem>
Message-ID: <200309151511.h8FFBZ0j029922@localhost.localdomain>
>>> Barry Warsaw wrote
> I'm using it for everything, and I agree! It's a very solid release. I
> still think we should plan for a 2.3.1 soonish, but the hold up may just
> be finding a release manager to own it. I'd rather spread the love this
> time. :)
I've already put my hand up for it, and Raymond's offered to help. I'm thinking
about maybe starting this process late next week?
(I've already done at least one minor release before, so I've a fair idea
of what's involved. Hey, hang on, didn't I write PEP102 for this? ;)
Anthony
--
Anthony Baxter
It's never too late to have a happy childhood.
From pje at telecommunity.com Mon Sep 15 11:26:27 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Sep 15 11:26:41 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for
Generators
In-Reply-To: <20030915104836.GA1143@vicky.ecs.soton.ac.uk>
References: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
<00a401c36c0f$76952160$c304a044@oemcomputer>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
<5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
Message-ID: <5.1.1.6.0.20030915112135.03013ec0@telecommunity.com>
At 11:48 AM 9/15/03 +0100, Armin Rigo wrote:
>Hello Samuele,
>
>On Sat, Sep 13, 2003 at 04:18:51PM +0200, Samuele Pedroni wrote:
> > OTOH conflating 'with' and 'for' just for generators seems a rather ad-hoc
> > breaking of
> > orthoganility of the two, you could not write anymore code like this:
> >
> > g = gen()
> > for v in g:
> > ... do something up to a point ...
> > ...
> > for v in g:
> > ...
>
>I had thought about this. This occurs when you 'break' out of the first loop.
>I would say that NOT calling the __exit__() method in this specific case seems
>quite intuitive, the 'break' meaning 'just exit from the loop now without any
>further processing, skipping the 'else' part if present'.
Hmmm... You realize this is also going to break the similarity between:
i = iter(something)
while 1:
try:
j=i.next()
except StopIteration:
break
and
for j in iter(something):
pass
The former is already complex enough as a mental model. I think it gets
altogether too complex if one also has to consider the enter/leave
issue. This strikes me as an area like try/except vs. try/finally: it
really should be a separate block, just for the sake of explicitness. As
much as it would be cool to have the automatic release, I think I'd rather use:
with i = iter(something):
for j in i:
...
And make the resource management very visible.
From skip at pobox.com Mon Sep 15 11:32:37 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Sep 15 11:32:49 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063638517.10716.32.camel@anthem>
References:
<1063637821.2093.132.camel@localhost.localdomain>
<20030915150313.GI1240@epoch.metaslash.com>
<1063638517.10716.32.camel@anthem>
Message-ID: <16229.56213.235834.239788@montanaro.dyndns.org>
Barry> On Mon, 2003-09-15 at 11:03, Neal Norwitz wrote:
>> I think 2.3 has been great! There were a few memory leaks plugged.
>> I don't even recall any crashes specific to 2.3. I'm very happy with
>> the apparent robustness of 2.3. Maybe nobody is using it? :-)
Barry> I'm using it for everything, and I agree!
Ditto on my development machine, except I just cvs up periodically, so I'm
generally running from CVS as of a few days ago, not even 2.3. I do intend
to update to 2.3 on the Mojam webserver in a short while. That should
provide a nice speed boost over the 2.2.3 I'm currently running there.
>> As far as the release schedule for 2.4, I haven't heard any squawking
>> recently about Python changing too fast.
Barry> We can't let that stand, now can we? :)
I suspect that's simply because the focus was on library and performance
improvements instead of new language features.
Barry> There's still time to get PEPpy too.
I would like to get back to PEP 304 (Control of pyc file location). I got
stuck when someone raised some good objections about how it would play on
Windows with it's multiple rooted file system, but if those problems can be
ironed out I think it would be a good addition to the system.
Skip
From niemeyer at conectiva.com Mon Sep 15 12:26:51 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Sep 15 12:27:32 2003
Subject: [Python-Dev] datetime issues
In-Reply-To:
References: <20030915000637.GC27698@ibook>
Message-ID: <20030915162651.GA30600@ibook.distro.conectiva>
> [Gustavo Niemeyer]
> > Is there any reason for not having a single keyword parameter for
> > every function accepting a tzinfo instance?
>
> Just the lack of anyone pointing it out in the module's 18-month review
> process .
Would you belive that I was.. erm.. sleeping!? :-)
> > I've noticed that some require 'tz', and others 'tzinfo'. Is it too
> > late to fix that?
>
> Of course, although it's not too late to add synonyms. It appears
> that the functions "with a lot of arguments" use tzinfo, and those
> with only one or two arguments tz now. I doubt anyone would bother to
> use the tz keyword-arg name for a few-argument function (now(),
> fromtimestamp(), astimezone()), so I'd rather that those few grow a
> tzinfo synonym. tz can be deprecated too, if you're bold.
It'd be nice to have these synonyms. I'll put it in my ever growing
todo list.
> > Also, is there any further work going on to improve the datetime
> > module?
>
> Not that I'm aware of. I'm not working on it anymore (other than
> keeping my out for bug reports, of which there have been blessedly
> few).
>
> > I'm not suggesting we should mirror mx.DateTime functionalities, but
> > two features I miss from mx.DateTime is the DateFrom(), which does
> > its best to parse a given string, and the RelativeDateTime()
> > functionality.
> >
> > This has probably been raised in the past, so if that's the case, I'm
> > sorry. Marc-Andre, what's your opinion about reusing code from
> > mx.DateTime into datetime?
> >
> > Anyway, reusing code or not, I'll probably put sometime on it in the
> > future, if this looks interesting to everyone (after I finish my
> > current python pendencies, like SRE's recursivity removal).
>
> If Guido wanted to stay out of the relatively clear time zone
> business, I can imagine his interest in trying to guess how to parse
> date and time strings. Apart from Brett's strptime(), I believe the
> email module has some useful string->datetime code. It may be good to
> fold such stuff in this area as Python already supports into datetime
> somehow.
Cool. I'll give it a try as well. I have a working relativedelta
implementation now. I'll polish it a little further and publish
it somewhere for tests.
Thanks!
--
Gustavo Niemeyer
http://niemeyer.net
From amk at amk.ca Mon Sep 15 12:29:11 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Sep 15 12:29:18 2003
Subject: [Python-Dev] Re: datetime issues
References: <20030915000637.GC27698@ibook>
<1063631656.10716.7.camel@anthem>
Message-ID:
On 15 Sep 2003 09:14:17 -0400, Barry Warsaw wrote:
> I'd also like to see factored out various date formating functions, so
> that the module is more or less symmetric.
Probably this is PEP territory. Unless someone else wants to do it, I'll
volunteer to produce a draft PEP for adding parsing and formatting
functions.
--amk
From amk at amk.ca Mon Sep 15 12:44:29 2003
From: amk at amk.ca (A.M. Kuchling)
Date: Mon Sep 15 12:44:37 2003
Subject: [Python-Dev] Re: Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
References:
<1063637821.2093.132.camel@localhost.localdomain>
Message-ID:
On Mon, 15 Sep 2003 10:57:02 -0400, Jeremy Hylton
wrote:
> We should also make a plan for a 2.3 maintenance release. There have
> been a lot of checkins, but I haven't gotten a sense for how significant
> the bugs were.
I watch the checkins for the sake of the 2.4 document, and haven't seen any
bugfixes that seemed really critical. No one in c.l.py seems to be running
into any of the memory leaks that were fixed, and none of the fixed bugs
seem likely to silently lose or corrupt data.
Is it worth holding 2.3.1 until after Panther ships, on the assumption that
its release will turn up more users and more bugs? If not, 2.3.1's timing
can probably be left up to the release manager, who can make the release
whenever he feels like it.
--amk
From niemeyer at conectiva.com Mon Sep 15 12:50:26 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Sep 15 12:51:04 2003
Subject: [Python-Dev] Re: datetime issues
In-Reply-To:
References: <20030915000637.GC27698@ibook>
Message-ID: <20030915165026.GB30600@ibook.distro.conectiva>
> >If Guido wanted to stay out of the relatively clear time zone
> >business, I can imagine his interest in trying to guess how to parse date
> >and time strings. Apart from Brett's strptime(), I believe the email
> >module
> >has some useful string->datetime code.
>
> The PyXML package includes a parser (xml.utils.iso8601) for the date/time
> format most commonly used in XML applications. That and a parser for
Cool! Thanks for mentioning it!
> RFC822 dates would meet the needs of many technical applications (though
> not ones where users can enter dates -- but that problem may be too
> complicated for the stdlib).
I do not agree. I've done a small research and there seems to be many
nice generic implementations of these parsers, which would benefit
technical applications *and* hand made dates (some of them below[1]),
and it doesn't look that complex. I'll be working to provide trial
implementation collecting the best features described in these papers in
the near future.
Btw, the relativedelta is mostly ready. That's the first step to provide
relative dates in the parser.
> I'll happily convert the iso8601 module for addition to the standard
> library if date parsing is deemed a topic of interest.
I'm not yet sure if date parsing and other improvements in the date/time
area is a topic of general interest, but at least I am interested. :-)
[1]
http://www.egenix.com/files/python/mxDateTime.html
http://ringmaster.arc.nasa.gov/tools/time_formats.html
http://www.thinkage.ca/english/gcos/expl/b/lib/0tosec.html
http://search.cpan.org/author/MUIR/Time-modules-2003.0211/lib/Time/ParseDate.pm
--
Gustavo Niemeyer
http://niemeyer.net
From niemeyer at conectiva.com Mon Sep 15 12:55:14 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Mon Sep 15 12:55:51 2003
Subject: [Python-Dev] Re: datetime issues
In-Reply-To:
References: <20030915000637.GC27698@ibook>
<1063631656.10716.7.camel@anthem>
Message-ID: <20030915165514.GC30600@ibook.distro.conectiva>
> >I'd also like to see factored out various date formating functions, so
> >that the module is more or less symmetric.
>
> Probably this is PEP territory. Unless someone else wants to do it,
> I'll volunteer to produce a draft PEP for adding parsing and
> formatting functions.
Please, don't discard the generic parser in the PEP.
--
Gustavo Niemeyer
http://niemeyer.net
From mwh at python.net Mon Sep 15 14:01:37 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Sep 15 14:03:04 2003
Subject: [Python-Dev] refleak status
Message-ID:
After a week or so of frantically searching for these things, I sort of
forgot about the reference leak hunt I started in Python's tests. A
week ago I ran my hacked regrtest.py again, made some notes, tried &
failed to send them to python-dev, went on holiday, came back, digged a
little more and am trying to send again...
real leaks i think i've fixed (am syncing up to CVS now, will test
overnight).
test_class leaked [66, 66, 66, 66] references
test_hotshot leaked [111, 111, 111, 111] references
real leaks i cant fix:
test_compile leaked [45, 45, 45, 45] references
lambda a,a:0 worst offender. confused.
red herrings:
test_codeccallbacks leaked [2, 2, 2, 2] references
callback registry
test_mutants leaked [-1489, 1520, -755, -183] references
randomized test
suspected bugs in _Py_RefTotal tracking:
test_descr leaked [2, 2, 2, 2] references
something to do with resurrection
test_gc leaked [18, 18, 18, 18] references
test_trashcan ... not a real leak, it seems
suspected red herrings:
test_cpickle leaked [6, 3, 2, 1] references
test_logging leaked [82, 82, 82, 82] references
(bet this is registry stuff of one sort or another)
test_threadedtempfile leaked [-74, 3, 3, 161] references
test_unicode leaked [7, 7, 7, 7] references
registries again
apparent refleaks that still need digging:
test_minidom leaked [1, 1, 1, 1] references
test_new leaked [2, 2, 2, 2] references
test_pkg leaked [8, 8, 9, 9] references
test_pkgimport leaked [5, 6, 6, 6] references
test_sax leaked [1762, 1762, 1762, 1762] references
Wouldn't like to claim anything about the following tests:
6 tests failed:
test_dircache test_doctest test_getopt test_multifile
test_warnings test_zipimport
27 tests skipped:
test_al test_bsddb test_bsddb3 test_cd test_cl test_curses test_dl
test_email_codecs test_gdbm test_gl test_imgfile test_largefile
test_linuxaudiodev test_locale test_mpz test_nis
test_normalization test_ossaudiodev test_pep277 test_poll
test_socket_ssl test_socketserver test_sunaudiodev test_timeout
test_urllibnet test_winreg test_winsound
Those skips are all expected on darwin.
(test_threaded_import deadlocks, so add that to the list)
Should investigate why the failing tests fail when run repeatedly and
check that they're not hiding ref leaks.
Cheers,
mwh
From bac at OCF.Berkeley.EDU Mon Sep 15 14:20:21 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Mon Sep 15 14:20:36 2003
Subject: [Python-Dev] Re: python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063637821.2093.132.camel@localhost.localdomain>
References:
<1063637821.2093.132.camel@localhost.localdomain>
Message-ID: <3F6602E5.1040300@ocf.berkeley.edu>
Jeremy Hylton wrote:
> On Mon, 2003-09-15 at 08:51, A.M. Kuchling wrote:
>
>>I'd be in favor of setting an arbitrary date for a 2.4 release, and when
>>that date rolls around stabilizing and releasing the CVS tree, no matter
>>what's been implemented or not implemented. If we've made major language
>>revisions by that time, fine, but it's worthwhile even if 2.4 is just 2.3 +
>>(various bugfixes deemed too risky for 2.3.1) + (a few new modules).
>>
>>PEP 320 estimates February 2005, a 19-month gap matching that between 2.2
>>and 2.3. That seems a bit too long, especially if there are no large
>>changes that require testing; aiming for autumn 2004, beginning to freeze
>>in August, seems reasonable.
>
>
> In the absence of any plans for major new features, I think this is a
> good plan. A rough schedule might be:
>
> 2.4a1 April
> 2.4a2 late May/early June
> 2.4b1 July 9
> 2.4b2 Aug. 6
> 2.4 f Sep. 3
>
+1
We could also try to schedule micro releases two months after the
initial release. That seems to have been a decent amount of time to get
in bug reports and fix them. Then again, standardizing micro releases
might cause people to hold off on using the newest release and thus we
might get less testing from users.
-Brett
From recht at netbsd.org Mon Sep 15 17:07:59 2003
From: recht at netbsd.org (Marc Recht)
Date: Mon Sep 15 17:08:15 2003
Subject: [Python-Dev] Python and POSIX
Message-ID: <35230000.1063660079@leeloo.intern.geht.de>
Hi!
Currently Python configure process sets quite a few POSIX/XOPEN macros (eg.
_POSIX_C_SOURCE/_XOPEN_SOURCE) to archieve portability. The is a good thing
- for the interpreter. The problem with this is that these macros are
exported to Python extensions via pyconfig.h. That way every extension
which uses pyconfig.h is limited to POSIX/XOPEN functionality and not all
library features of the OS (eg. RPC) are available for the extension. For
the systems that supported it the Python configure process also sets macros
which activate all library functions (_NETBSD_SOURCE, __EXTENSIONS__) and
thus avoiding POSIX as a work-around. On all systems I'm aware of this is
the same feature-set you get if you don't set any macro at all, but
unfortunately not all systems support such a macro.
So, why not split pyconfig.h up into an external and internal header? That
way the the interpreter could be POSIX confirming (which is indeed a good
thing) without enforcing POSIX to the extensions.
Regards,
Marc
From martin at v.loewis.de Mon Sep 15 17:58:14 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep 15 18:02:57 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <1063627491.3f65aae37a3b7@mcherm.com>
References: <1063627491.3f65aae37a3b7@mcherm.com>
Message-ID:
Michael Chermside writes:
> I don't really understand this absolutist position.
I think the rationale is that developers don't actively *use* bug
fixes, whereas they do use new features.
So when you tell them "we have booleans now", they will drop
everything else and start putting booleans all over the place. In the
middle of that, they think "wait, what about 2.2", and they will test
2.2.1, and find that it works just fine. They run into problems only
after they release the software.
With bug fixes, it is different: People won't actively change their
working code to deliberately formulate it in a way that breaks on
older Python versions. Instead, upon reading that the bug has been
fixed, they typically think "does not concern me", and proceed with
whatever they were doing, anyway. In some cases, they may happen to
write new code that relies on a recent bug fix. When they discover
that older versions don't work with their code, they just bump the
prerequisites, or add a work-around.
In addition, there is always the concern that new code may break
existing code, so the amount of new code should be kept to the
smallest possible minimum. Adding new features is beyond that minimum,
so it should not be done (regardless of whether that new code is
likely to break existing code, upon visual inspection).
Regards,
Martin
From martin at v.loewis.de Mon Sep 15 18:05:12 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Sep 15 18:05:29 2003
Subject: [Python-Dev] Python and POSIX
In-Reply-To: <35230000.1063660079@leeloo.intern.geht.de>
References: <35230000.1063660079@leeloo.intern.geht.de>
Message-ID:
Marc Recht writes:
> On all systems I'm aware of this is the same feature-set you get if
> you don't set any macro at all, but unfortunately not all systems
> support such a macro.
You should take a look at Solaris, then. Depending on whether you
define XPG macros (I forgot which ones specifically), the socket
functions mutate from BSD (e.g. connect(3BSD)) to XPG functions
(connect(3XPG)). This is implemented by exposing
#pragma redefine_extname connect __xpg_connect
or some such in socket.h, if the XPG macro is defined.
Likewise, on HP-UX, you select one of the three (!) curses libraries
on the system through these macros.
> So, why not split pyconfig.h up into an external and internal header?
That would be possible if one would
a) provide a patch, and
b) could demonstrate that, on all relevant systems, lack of such macros
cannot possible have an effect on proper operation of extension modules.
In particular, don't try to make LFS support "internal": Extensions
modules are known to break if they use a different "struct stat" from
the one that the core uses.
Regards,
Martin
From python at rcn.com Tue Sep 16 00:46:04 2003
From: python at rcn.com (Raymond Hettinger)
Date: Tue Sep 16 00:50:54 2003
Subject: [Python-Dev] Re: Re: python/dist/src/Lib/bsddb __init__.py, 1.5,
1.6
References: <1063637821.2093.132.camel@localhost.localdomain>
Message-ID: <004101c37c0d$701be040$0c06a044@oemcomputer>
> Is it worth holding 2.3.1 until after Panther ships, on the assumption that
> its release will turn up more users and more bugs? If not, 2.3.1's timing
> can probably be left up to the release manager, who can make the release
> whenever he feels like it.
Last month, we worked out a tentantive plan to start the 2.3.1 release
process in the third week of September. I think we should stick with
that. The initial round of bugfixes is an important step for 2.3 being
viewed as stable and to show everyone that development/support
has not vanished.
If critical bugs surface later, that can be a consideration for 2.3.2.
There is no reason to be stingy with the initial bugfix release.
Raymond
#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################
From bac at OCF.Berkeley.EDU Tue Sep 16 01:09:07 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Sep 16 01:09:20 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
Message-ID: <3F669AF3.8050605@ocf.berkeley.edu>
Some of you may remember that I was working on closing a bug for Red Hat
6.1 where time.tzset was being detected by 'configure' when it shouldn't
have (it's busted on that platform) for 2.3.0 . Unfortunately a working
patch never surfaced in time to be thoroughly tested.
Well, Kurt Kaiser wrote up a patch ( http://www.python.org/sf/762934 )
that seems sound. I have gotten zero responses from anyone to test it
on Red Hat 6.1 and a remember that some people here were helping to test
last time (I also remember this led to the creation of a wiki page
listing platforms various people could test things on but I can't find
the page).
I would like to get this in for 2.3.1 (assuming this is considered
reasonable; not sure after the discussion that has gone on over what
should go into a micro release) if possible. So if anyone is running
Red Hat 6.1 or has easy access to one I would appreciate it if you took
a look at the patch and gave it a go (don't forget the various steps
needed to make the patch work; read the follow-ups to see what needs to
be done).
-Brett
From mal at lemburg.com Tue Sep 16 03:40:59 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue Sep 16 03:40:54 2003
Subject: [Python-Dev] datetime issues
In-Reply-To: <20030915000637.GC27698@ibook>
References: <20030915000637.GC27698@ibook>
Message-ID: <3F66BE8B.7060307@lemburg.com>
Gustavo Niemeyer wrote:
> Also, is there any further work going on to improve the datetime module?
> I'm not suggesting we should mirror mx.DateTime functionalities, but two
> features I miss from mx.DateTime is the DateFrom(), which does its best
> to parse a given string, and the RelativeDateTime() functionality.
>
> This has probably been raised in the past, so if that's the case, I'm
> sorry. Marc-Andre, what's your opinion about reusing code from
> mx.DateTime into datetime?
The mxDateTime code comes under the eGenix Public License, so
reuse is permittted under the terms of the license. That said,
I doubt that you want to maintain a date/time parser in the
Python core -- I certainly don't :-)
> Anyway, reusing code or not, I'll probably put sometime on it in the
> future, if this looks interesting to everyone (after I finish my current
> python pendencies, like SRE's recursivity removal).
--
Marc-Andre Lemburg
eGenix.com
Professional Python Software directly from the Source (#1, Sep 16 2003)
>>> Python/Zope Products & Consulting ... http://www.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From mwh at python.net Tue Sep 16 04:59:28 2003
From: mwh at python.net (Michael Hudson)
Date: Tue Sep 16 04:58:58 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F669AF3.8050605@ocf.berkeley.edu> (Brett C.'s message of
"Mon, 15 Sep 2003 22:09:07 -0700")
References: <3F669AF3.8050605@ocf.berkeley.edu>
Message-ID: <2mznh5az2n.fsf@starship.python.net>
"Brett C." writes:
> I would like to get this in for 2.3.1 (assuming this is considered
> reasonable; not sure after the discussion that has gone on over what
> should go into a micro release) if possible. So if anyone is running
> Red Hat 6.1 or has easy access to one I would appreciate it if you
> took a look at the patch and gave it a go
I no longer have access to RH 6.1 (the starship is now Debian testing).
Cheers,
mwh
--
Of course, it obviously is beta hardware so such things are to be
expected, but that doesn't mean that you can't point your fingers
and generate a nelson style HAHA at a multi billion dollar
corporation's expense. -- CmdrTaco on slashdot.org
From arigo at tunes.org Tue Sep 16 06:50:04 2003
From: arigo at tunes.org (Armin Rigo)
Date: Tue Sep 16 06:52:03 2003
Subject: [Python-Dev] pre-PEP: Resource-Release Support for Generators
In-Reply-To: <5.1.1.6.0.20030915112135.03013ec0@telecommunity.com>
References: <5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
<5.2.1.1.0.20030826151628.023f8d68@pop.bluewin.ch>
<00a401c36c0f$76952160$c304a044@oemcomputer>
<200308262050.h7QKoqD04780@12-236-84-31.client.attbi.com>
<5.2.1.1.0.20030913160918.02773bc0@pop.bluewin.ch>
<5.1.1.6.0.20030915112135.03013ec0@telecommunity.com>
Message-ID: <20030916105004.GA11247@vicky.ecs.soton.ac.uk>
Hello Phillip,
On Mon, Sep 15, 2003 at 11:26:27AM -0400, Phillip J. Eby wrote:
> i = iter(something)
> while 1:
> try:
> j=i.next()
> except StopIteration:
> break
>
> (...) is already complex enough as a mental model.
Makes sense. Then let's make sure, if both 'with' and 'yield within
try:finally' are accepted, that they are compatible, e.g. by having and
__exit__() method on generators (and not just a close()).
Armin
From recht at netbsd.org Tue Sep 16 06:59:01 2003
From: recht at netbsd.org (Marc Recht)
Date: Tue Sep 16 06:59:06 2003
Subject: [Python-Dev] Python and POSIX
In-Reply-To:
References: <35230000.1063660079@leeloo.intern.geht.de>
Message-ID: <46570000.1063709941@leeloo.intern.geht.de>
> You should take a look at Solaris, then. Depending on whether you
> define XPG macros (I forgot which ones specifically), the socket
> functions mutate from BSD (e.g. connect(3BSD)) to XPG functions
> (connect(3XPG)). This is implemented by exposing
>
># pragma redefine_extname connect __xpg_connect
>
> or some such in socket.h, if the XPG macro is defined.
I had a quick glance at the Solaris 8 headers and it seems that's
influenced by __EXTENSIONS__, too. So it _should_ be the same as without
the macros.
> Likewise, on HP-UX, you select one of the three (!) curses libraries
> on the system through these macros.
Oh. Didn't know about HP-UX. That's weird.
>> So, why not split pyconfig.h up into an external and internal header?
>
> That would be possible if one would
> a) provide a patch, and
OK. I justed wanted to verify if there's enough interest (and no general
rejection) in such a patch and if anyone else is already working on it.
> b) could demonstrate that, on all relevant systems, lack of such macros
> cannot possible have an effect on proper operation of extension
> modules.
That's a bit harder since I don't have access to all "relevant" systems. I
could test on the open-source x86 platforms, though. And IIRC CompileFarm
has Solaris, too.
And I _believe_ that - since these changes came up with Python 2.3 (with
the exception of _POSIX_1_SOURCE and _POSIX_SOURCE) - that it will break
not much or anything at all if at least _XOPEN_SOURCE are moved
_XOPEN_SOURCE_EXTENDED to an internal header (or an internal part of
pyconfig.h). (As a first step.)
But, you're right there shouldn't be a rash decision and this needs much
testing.
I think I'll just go on and provide a patch as basis for further discussion.
Regards,
Marc
From neal at metaslash.com Tue Sep 16 08:40:17 2003
From: neal at metaslash.com (Neal Norwitz)
Date: Tue Sep 16 08:40:21 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F669AF3.8050605@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
Message-ID: <20030916124017.GB13879@epoch.metaslash.com>
On Mon, Sep 15, 2003 at 10:09:07PM -0700, Brett C. wrote:
>
> Well, Kurt Kaiser wrote up a patch ( http://www.python.org/sf/762934 )
> that seems sound. I have gotten zero responses from anyone to test it
> on Red Hat 6.1 and a remember that some people here were helping to test
> last time (I also remember this led to the creation of a wiki page
> listing platforms various people could test things on but I can't find
> the page).
http://www.python.org/cgi-bin/moinmoin/PythonTesters
From niemeyer at conectiva.com Tue Sep 16 09:54:36 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue Sep 16 09:55:27 2003
Subject: [Python-Dev] datetime issues
In-Reply-To: <3F66BE8B.7060307@lemburg.com>
References: <20030915000637.GC27698@ibook> <3F66BE8B.7060307@lemburg.com>
Message-ID: <20030916135436.GA2509@ibook.distro.conectiva>
> The mxDateTime code comes under the eGenix Public License, so
> reuse is permittted under the terms of the license. That said,
> I doubt that you want to maintain a date/time parser in the
> Python core -- I certainly don't :-)
Thanks. I've got the relativedelta type ready. It's based on your
specification (thanks for it), but uses no code from your module (you'll
notice that clearly when looking at the code). Indeed, it has even a
different algorithm, which solves some of the current "problems" with
your implementation (like getting 03/03 when adding one month to 31/01).
You're free to adopt it, if you feel like so. I'll be releasing that
code under the PSF license.
I'll probably work on a generic parser, once I get some more time.
Depending on the results, I'll submit it for review and possible
inclusion as well.
Here is the documentation of relativedelta, FWIW:
"""
The relativedelta type is based on the specification of the excelent
work done by M.-A. Lemburg in his mx.DateTime extension. However,
notice that this type does *NOT* implement the same algorithm as
his work. Do *NOT* expect it to behave like mx.DateTime's counterpart.
There's two different ways to build a relativedelta instance. The
first one is passing it two date/datetime classes:
relativedelta(datetime1, datetime2)
And the other way is to use the following keyword arguments:
year, month, day, hour, minute, seconds, microseconds:
Absolute information.
years, months, weeks, days, hours, minutes, seconds, microseconds:
Relative information, may be negative.
weekday:
Tuple with (wday, nth), specifying the nth relative weekday.
Here is the behavior of operations with relativedelta:
1) Calculate the absolute year, using the 'year' argument, or the
original datetime year, if the argument is not present.
2) Add the relative 'years' argument to the absolute year.
3) Do steps 1 and 2 for month/months.
4) Calculate the absolute day, using the 'day' argument, or the
original datetime day, if the argument is not present. Then,
subtract from the day until it fits in the year and month
found after their operations.
5) Add the relative 'days' argument to the absolute day. Notice
that the 'weeks' argument is multiplied by 7 and added to
'days'.
6) Do steps 1 and 2 for hour/hours, minute/minutes, second/seconds,
microsecond/microseconds.
7) If the 'weekday' argument is present, calculate the weekday,
with the given (wday, nth) tuple. wday is the index of the
weekday (0-6, 0=Mon), and nth is the number of weeks to add
forward or backward, depending on its signal. Notice that if
the calculated date is already Monday, for example, using
(0, 1) or (0, -1) won't change the day.
"""
And here is a little demo:
>>> from dtutil import *; from datetime import *
>>> relativedelta(datetime.now(),datetime(1983,9,10,10,00))
relativedelta(years=+20, days=+6, minutes=+25, seconds=+4,
microseconds=+231402)
>>> datetime(1983,9,10,10,00)+relativedelta(years=+20, days=+6,
minutes=+25, seconds=+4, microseconds=+231402)
datetime.datetime(2003, 9, 16, 10, 25, 4, 231402)
>>> relativedelta(datetime(2000,2,29),datetime(2001,3,29))
relativedelta(years=-1, months=-1)
>>> datetime(2001,3,29)+relativedelta(years=-1, months=-1)
datetime.datetime(2000, 2, 29, 0, 0)
>>> relativedelta(datetime(2000,2,29),datetime(2001,3,30))
relativedelta(years=-1, months=-1)
>>> datetime(2001,3,30)+relativedelta(years=-1, months=-1)
datetime.datetime(2000, 2, 29, 0, 0)
>>> relativedelta(datetime(2000,2,29),datetime(2001,2,28))
relativedelta(months=-11, days=-28)
>>> relativedelta(datetime(2000,2,28),datetime(2001,2,28))
relativedelta(years=-1)
>>> relativedelta(datetime(2000,2,27),datetime(2001,2,28))
relativedelta(years=-1, days=-1)
>>> relativedelta(datetime(2001,2,28),datetime(2000,2,29))
relativedelta(years=+1)
>>> relativedelta(datetime(2001,2,28),datetime(2000,2,28))
relativedelta(years=+1)
>>> relativedelta(datetime(2001,2,28),datetime(2000,2,27))
relativedelta(years=+1, days=+1)
>>> import calendar
>>> date.today()+relativedelta(weekday=(calendar.MONDAY, +2))
datetime.date(2003, 9, 29)
>>> date.today()+relativedelta(weekday=(calendar.MONDAY, -1))
datetime.date(2003, 9, 15)
--
Gustavo Niemeyer
http://niemeyer.net
From skip at pobox.com Tue Sep 16 10:17:54 2003
From: skip at pobox.com (Skip Montanaro)
Date: Tue Sep 16 10:18:06 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F669AF3.8050605@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
Message-ID: <16231.7058.987290.575924@montanaro.dyndns.org>
Brett> Well, Kurt Kaiser wrote up a patch (
Brett> http://www.python.org/sf/762934 ) that seems sound. I have
Brett> gotten zero responses from anyone to test it on Red Hat 6.1 and a
Brett> remember that some people here were helping to test last time (I
Brett> also remember this led to the creation of a wiki page listing
Brett> platforms various people could test things on but I can't find
Brett> the page).
That would be:
http://www.python.org/cgi-bin/moinmoin/PythonTesters
I see Kurt Kaiser as the only person listed who can test for RH < 6.2.
Perhaps you should broadcast a call for testers to c.l.py, referring to the
above page and asking in particular for RH 6.1 folks to chime in.
Skip
From tim at zope.com Tue Sep 16 10:34:23 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 10:34:49 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063684862.2093.193.camel@localhost.localdomain>
Message-ID:
[Tim]
>> $9 = {_ob_next = 0x0, _ob_prev = 0x0, ob_refcnt = 0,
>> ob_type = 0x8130160, length = 1, str = 0x41903130,
>> hash = -1, defenc = 0x0}
>>
>> that certainly means the object has been released already.
[Jeremy]
> This is code that is trying to revive an object from the freelist, so
> it isn't a big surprise that the object looks like it has been freed.
Doh -- yup, of course.
> Now here's a very strange thing. If I apply the following patch, the
> problem goes away. The patch doesn't make a lot of sense to me, so
> I'm guessing that it's just a lucky patch -- avoiding the problem
> without fixing the cause.
We should move this to python-dev. OK, I just did . Another clue
about the cause coming up.
> The thing that is curious is that we change the test against
> unicode->str[0], which is probably an unsigned int,
Why? Are you doing a UCS4 build? The expansion of Py_UNICODE (unicode->str
is an array of Py_UNICODE thingies) can't be guessed from where I sit (it
expands to "unsigned short" on Windows, but that's because that's hardcoded
in PC\pyconfig.h; I'm not sure where you get its expansion from).
> so that we also test that it is > 0. In the specific case that
> crashes, the debugger says the value is -875836469.
That's probably a great clus! In hex, that's 0xcbcbcbcb, which is the
special byte pattern the debug pymalloc sprays into freshly-allocated
memory. IOW, unicode->str contains (in a release build) uninitialized heap
trash at this point, and so there's nothing it can be tested against that
makes any sense. This suggests a deeper bug in the Unicode support.
> Does that make any sense? Do we actually have a signed value here?
You have to figure out what Py_UNICODE expands to in your build to answer
that one.
> Or does it get converted to a signed value when the comparison occurs?
If gdb showed you -875836469 in response to "unicode->str[0]" in isolation,
I have to guess Py_UNICODE is expanding to a signed 4-byte type in your
build.
> Jeremy
>
> Index: unicodeobject.c
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Objects/unicodeobject.c,v
> retrieving revision 2.196
> diff -c -r2.196 unicodeobject.c
> *** unicodeobject.c 16 Sep 2003 03:41:45 -0000 2.196
> --- unicodeobject.c 16 Sep 2003 03:55:53 -0000
> ***************
> *** 130,138 ****
> /* Resizing shared object (unicode_empty or single character
> objects) in-place is not allowed. Use PyUnicode_Resize()
> instead ! */
> if (unicode == unicode_empty ||
> (unicode->length == 1 &&
> ! unicode->str[0] < 256 &&
> unicode_latin1[unicode->str[0]] == unicode)) {
> PyErr_SetString(PyExc_SystemError,
> "can't resize shared unicode objects");
> --- 130,142 ----
> /* Resizing shared object (unicode_empty or single character
> objects) in-place is not allowed. Use PyUnicode_Resize()
> instead ! */
> if (unicode == unicode_empty ||
> (unicode->length == 1 &&
> ! unicode->str[0] < 256 && unicode->str[0] > 0 &&
> unicode_latin1[unicode->str[0]] == unicode)) {
> PyErr_SetString(PyExc_SystemError,
> "can't resize shared unicode objects");
Belaboring the obvious, if unicode->str[0] is really heap trash at this
point, there's no safe test.
From jeremy at zope.com Tue Sep 16 11:05:19 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Tue Sep 16 11:05:50 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To:
References:
Message-ID: <1063724718.4868.2.camel@localhost.localdomain>
I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
fixed the problem. The comment describe the code is really unnerving:
/* Limit for the Unicode object free list stay alive optimization.
...
Note: This is an experimental feature ! If you get core dumps when
using Unicode objects, turn this feature off.
*/
Why was this experimental feature released with Python if it causes core
dumps?
Jeremy
From tim at zope.com Tue Sep 16 12:12:26 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 12:12:48 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063724718.4868.2.camel@localhost.localdomain>
Message-ID:
[Jeremy Hylton]
> I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
> fixed the problem.
We don't know why, though, nor do we have a small test case, right?
> The comment describe the code is really unnerving:
>
> /* Limit for the Unicode object free list stay alive optimization.
>
> ...
>
> Note: This is an experimental feature ! If you get core dumps when
> using Unicode objects, turn this feature off.
>
> */
>
> Why was this experimental feature released with Python if it causes
> core dumps?
Good question .
Going back to an earlier msg:
> Index: unicodeobject.c
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Objects/unicodeobject.c,v
> retrieving revision 2.196
> diff -c -r2.196 unicodeobject.c
> *** unicodeobject.c 16 Sep 2003 03:41:45 -0000 2.196
> --- unicodeobject.c 16 Sep 2003 03:55:53 -0000
> ***************
> *** 130,138 ****
> /* Resizing shared object (unicode_empty or single character
> objects) in-place is not allowed. Use PyUnicode_Resize()
> instead ! */
> if (unicode == unicode_empty ||
> (unicode->length == 1 &&
> ! unicode->str[0] < 256 &&
> unicode_latin1[unicode->str[0]] == unicode)) {
> PyErr_SetString(PyExc_SystemError,
> "can't resize shared unicode objects");
> --- 130,142 ----
> /* Resizing shared object (unicode_empty or single character
> objects) in-place is not allowed. Use PyUnicode_Resize()
> instead ! */
> if (unicode == unicode_empty ||
> (unicode->length == 1 &&
> ! unicode->str[0] < 256 && unicode->str[0] > 0 &&
> unicode_latin1[unicode->str[0]] == unicode)) {
> PyErr_SetString(PyExc_SystemError,
> "can't resize shared unicode objects");
> Belaboring the obvious, if unicode->str[0] is really heap trash at
> this point, there's no safe test.
On second thought, the test could be correct even if it does read up heap
trash: the
unicode_latin1[unicode->str[0]] == unicode
part is a very strong condition. If str[0] does contain heap trash, it
can't pass, but it could cause Purify (etc) to whine about reading
uninitialized memory. If we don't care about that,
unicode->length == 1 &&
(unsigned int)unicode->str[0] < 256U &&
unicode_latin1[unicode->str[0]] == unicode
would be a safe and correct guard.
Still, it doesn't look to me like the code *expected* that str could contain
uninitialized heap trash at this point, so I'd like someone who thinks they
understand this code to think about how that could happen (it apparently is
happening, so "beats me" isn't good enough ).
From patmiller at llnl.gov Tue Sep 16 12:24:45 2003
From: patmiller at llnl.gov (Pat Miller)
Date: Tue Sep 16 12:24:50 2003
Subject: [Python-Dev] Making python C-API thread safe (try 2)
In-Reply-To: <200309120311.h8C3BSe01781@oma.cosc.canterbury.ac.nz>
References: <200309120311.h8C3BSe01781@oma.cosc.canterbury.ac.nz>
Message-ID: <3F67394D.305@llnl.gov>
I used to work on the SISAL compiler which was a fine-grained parallel
functional language. It reference counted its allocated data.
Updating the reference count is a critical section.
We were spending 40% of our time blocked for reference counting on only 4 threads.
We eventually taught the compiler to do reference count optimizations
(e.g. put all readers of an object before the writers), but these
optimizations are not suitable to the Python interpreter.
With this same reasoning, you can infer why C++ smart pointers are
also a bad idea for threaded code.
Pat
--
Patrick Miller | (925) 423-0309 | http://www.llnl.gov/CASC/people/pmiller
All you need in this life is ignorance and confidence, and then success is
sure. -- Mark Twain
From jeremy at zope.com Tue Sep 16 12:31:20 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Tue Sep 16 12:31:45 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To:
References:
Message-ID: <1063729880.4868.8.camel@localhost.localdomain>
On Tue, 2003-09-16 at 12:12, Tim Peters wrote:
> [Jeremy Hylton]
> > I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
> > fixed the problem.
>
> We don't know why, though, nor do we have a small test case, right?
Not a small test case, no, but I have boiled it down a bit. If you run
the tests from a Zope checkout on the Zope-2_7-branch, you can provoke
the crash with "test.py Template."
> Going back to an earlier msg:
>
> > Index: unicodeobject.c
> > ===================================================================
> > RCS file: /cvsroot/python/python/dist/src/Objects/unicodeobject.c,v
> > retrieving revision 2.196
> > diff -c -r2.196 unicodeobject.c
> > *** unicodeobject.c 16 Sep 2003 03:41:45 -0000 2.196
> > --- unicodeobject.c 16 Sep 2003 03:55:53 -0000
> > ***************
> > *** 130,138 ****
> > /* Resizing shared object (unicode_empty or single character
> > objects) in-place is not allowed. Use PyUnicode_Resize()
> > instead ! */
> > if (unicode == unicode_empty ||
> > (unicode->length == 1 &&
> > ! unicode->str[0] < 256 &&
> > unicode_latin1[unicode->str[0]] == unicode)) {
> > PyErr_SetString(PyExc_SystemError,
> > "can't resize shared unicode objects");
> > --- 130,142 ----
> > /* Resizing shared object (unicode_empty or single character
> > objects) in-place is not allowed. Use PyUnicode_Resize()
> > instead ! */
> > if (unicode == unicode_empty ||
> > (unicode->length == 1 &&
> > ! unicode->str[0] < 256 && unicode->str[0] > 0 &&
> > unicode_latin1[unicode->str[0]] == unicode)) {
> > PyErr_SetString(PyExc_SystemError,
> > "can't resize shared unicode objects");
>
> > Belaboring the obvious, if unicode->str[0] is really heap trash at
> > this point, there's no safe test.
>
> On second thought, the test could be correct even if it does read up heap
> trash: the
>
> unicode_latin1[unicode->str[0]] == unicode
>
> part is a very strong condition. If str[0] does contain heap trash, it
> can't pass, but it could cause Purify (etc) to whine about reading
> uninitialized memory. If we don't care about that,
It could also cause a segmentation violation depending on the value of
the stack trash it reads. At least, that's what appears to be going
wrong.
>
> unicode->length == 1 &&
> (unsigned int)unicode->str[0] < 256U &&
> unicode_latin1[unicode->str[0]] == unicode
>
> would be a safe and correct guard.
That sounds better.
> Still, it doesn't look to me like the code *expected* that str could contain
> uninitialized heap trash at this point, so I'd like someone who thinks they
> understand this code to think about how that could happen (it apparently is
> happening, so "beats me" isn't good enough ).
I certainly don't understand the code yet.
Jeremy
From bac at OCF.Berkeley.EDU Tue Sep 16 12:53:41 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Sep 16 12:54:02 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <16231.7058.987290.575924@montanaro.dyndns.org>
References: <3F669AF3.8050605@ocf.berkeley.edu>
<16231.7058.987290.575924@montanaro.dyndns.org>
Message-ID: <3F674015.8090801@ocf.berkeley.edu>
Skip Montanaro wrote:
> Brett> Well, Kurt Kaiser wrote up a patch (
> Brett> http://www.python.org/sf/762934 ) that seems sound. I have
> Brett> gotten zero responses from anyone to test it on Red Hat 6.1 and a
> Brett> remember that some people here were helping to test last time (I
> Brett> also remember this led to the creation of a wiki page listing
> Brett> platforms various people could test things on but I can't find
> Brett> the page).
>
> That would be:
>
> http://www.python.org/cgi-bin/moinmoin/PythonTesters
>
> I see Kurt Kaiser as the only person listed who can test for RH < 6.2.
> Perhaps you should broadcast a call for testers to c.l.py, referring to the
> above page and asking in particular for RH 6.1 folks to chime in.
>
Jordan Krushen pointed out that the patch is actually opened against RH
6.2, not 6.1 like I originally said. But I don't think it is that big
of an issue since there is a bug report for this against SuSE. I think
the key thing here is an older version of glibc since that would be what
defines the tzset function.
But I will consider sending a note out to c.l.py about this. Thanks for
the suggestion, Skip.
-Brett
From tim at zope.com Tue Sep 16 12:58:21 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 12:58:45 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063729880.4868.8.camel@localhost.localdomain>
Message-ID:
[Jeremy Hylton]
>>> I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
>>> fixed the problem.
[Tim
>> We don't know why, though, nor do we have a small test case, right?
[Jeremy]
> Not a small test case, no, but I have boiled it down a bit. If you
> run the tests from a Zope checkout on the Zope-2_7-branch, you can
> provoke the crash with "test.py Template."
Sorry, I cannot. I tried a fresh Zope-2_7-branch, on Windows, with
released Python 2.3, release build
current CVS Python, release build
current CVS Python, debug build
"test.py Template" gave the same result in all cases: 77 tests run, 1
failure (AssertionError: HTML Output Changed) in checkStringExpressions.
But note that Py_UNICODE is unsigned short on this box, and reading a trash
one of those as an index isn't nearly as likely to reach into unmapped
memory as on a box where you get 4 bytes of trash.
>> ...
>> On second thought, the test could be correct even if it does read up
>> heap trash: the
>>
>> unicode_latin1[unicode->str[0]] == unicode
>>
>> part is a very strong condition. If str[0] does contain heap trash,
>> it can't pass, but it could cause Purify (etc) to whine about reading
>> uninitialized memory. If we don't care about that,
> It could also cause a segmentation violation depending on the value of
> the stack trash it reads. At least, that's what appears to be going
> wrong.
Right, I meant the condition is very strong *if* the unicode->str[0] part is
known to lie in range(256). unicode_latin1 is statically declared to
contain 256 elements, so if the trash index is in range(256) then the index
resolves to legitimate memory.
>> unicode->length == 1 &&
>> (unsigned int)unicode->str[0] < 256U &&
>> unicode_latin1[unicode->str[0]] == unicode
>> would be a safe and correct guard.
> That sounds better.
To worm around something that shouldn't happen at all, sure .
>> Still, it doesn't look to me like the code *expected* that str could
>> contain uninitialized heap trash at this point, so I'd like someone
>> who thinks they understand this code to think about how that could
>> happen (it apparently is happening, so "beats me" isn't good enough
>> ).
> I certainly don't understand the code yet.
Me neither, but offhand didn't see anything glaringly wrong. If some
internal Unicode operation decided to allocate a short Unicode string, but
freed it before filling in any of the string bytes, I suppose the keepalive
optimization would retain a Unicode object with uninitialized str space on
the unicode free list. A subsequent _PyUnicode_New could grab that and try
to boost its ->str size. That could explain it.
From jeremy at alum.mit.edu Tue Sep 16 12:58:43 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue Sep 16 12:58:47 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063729880.4868.8.camel@localhost.localdomain>
References:
<1063729880.4868.8.camel@localhost.localdomain>
Message-ID: <1063731523.4868.11.camel@localhost.localdomain>
On Tue, 2003-09-16 at 12:31, Jeremy Hylton wrote:
> On Tue, 2003-09-16 at 12:12, Tim Peters wrote:
> > [Jeremy Hylton]
> > > I disabled the KEEPALIVE_SIZE_LIMIT, as you suggested, and that also
> > > fixed the problem.
> >
> > We don't know why, though, nor do we have a small test case, right?
>
> Not a small test case, no, but I have boiled it down a bit. If you run
> the tests from a Zope checkout on the Zope-2_7-branch, you can provoke
> the crash with "test.py Template."
Okay. I have a smaller test case, though I don't know what I couldn't
find it earlier. If you run "test.py DocumentTemplate UnicodeStr" by
itself, you get the crash.
The DocumentTemplate implementation has C and Python implementations of
some utilities. It uses the C by default and falls back to Python if
the C extension can't be imported. If I change the code so that it
always uses the Python implementation, the crash goes away.
Jeremy
From tim at zope.com Tue Sep 16 13:11:13 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 13:12:00 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To:
Message-ID:
[Jeremy]
>> Not a small test case, no, but I have boiled it down a bit. If you
>> run the tests from a Zope checkout on the Zope-2_7-branch, you can
>> provoke the crash with "test.py Template."
[Tim]
> Sorry, I cannot. I tried a fresh Zope-2_7-branch, on Windows, with
>
> released Python 2.3, release build
> current CVS Python, release build
> current CVS Python, debug build
>
> "test.py Template" gave the same result in all cases: 77 tests run, 1
> failure (AssertionError: HTML Output Changed) in
> checkStringExpressions.
>
> But note that Py_UNICODE is unsigned short on this box, and reading a
> trash one of those as an index isn't nearly as likely to reach into
> unmapped memory as on a box where you get 4 bytes of trash.
What I *could* do, though, is add this to unicode_resize():
if (unicode != unicode_empty && unicode->length == 1) {
assert(unicode->str[0] != 0xcbcbU);
}
That checks for the debug pymalloc's "uninitialized memory" byte pattern
appropriate for this box, and that assert does indeed trigger when running
test.py Template
It fires while running checkUnicodeInserts, but checkUnicodeInserts does not
fail in isolation. So checkUnicodeInserts is the victim of something an
earlier test left on the unicode freelist.
From tim at zope.com Tue Sep 16 13:15:53 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 13:16:58 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063731523.4868.11.camel@localhost.localdomain>
Message-ID:
[Jeremy Hylton]
> Okay. I have a smaller test case, though I don't know what I couldn't
> find it earlier. If you run "test.py DocumentTemplate UnicodeStr" by
> itself, you get the crash.
As before, nothing crashes here, but running that test does trigger the
debug-build-specific assert I added here (see last msg). Good sleuthing!
> The DocumentTemplate implementation has C and Python implementations
> of some utilities. It uses the C by default and falls back to Python
> if the C extension can't be imported. If I change the code so that it
> always uses the Python implementation, the crash goes away.
I'm guessing
lib/python/DocumentTemplate/cDocumentTemplate.c
From theller at python.net Tue Sep 16 13:45:40 2003
From: theller at python.net (Thomas Heller)
Date: Tue Sep 16 13:45:22 2003
Subject: [Python-Dev] Re: HTMLHelp for Py2.3.1
In-Reply-To: (Boriz
Izaguirre's message of "Sun, 7 Sep 2003 15:52:36 +0200")
References:
Message-ID: <8yook4or.fsf@python.net>
"Boriz Izaguirre" writes:
> [Thomas]
>>[Fred L. Drake]
>>> IDLE currently looks for the index.html file in a few places (which
>>> depend on platform); if it can't find it, it uses the documentation on
>>> python.org.
>>>
>>> It should be too hard to change it to load the HTML Help viewer if it
>>> finds the .chm file on Windows, and to still fall back to the HTML or
>>> the online documentation if the .chm can't be found.
>>
>>Changing it is trivial, EditorWindow.help_url must point to Python23.chm
>>(if it exists). I can do this.
>
> Beware that for .html you want webbrowser.open(url) and for .chm you
> want os.startfile(url)
> There used to be a patch in Idle-dev for this.
> http://tinyurl.com/mj1s
>
It seems this patch is already applied in 2.3.
>>Even nicer would be context-sensitive keyword help, but it seems IDLE
>>doesn't support it, right?
>
> Standard .chm format already includes useful index data to do
> context-sensitive search. A couple of years ago I made an Idle extension
> to manage this. It wasn't difficult then, and I think it's even easier
> now. The problem I found is that you pass the selected text (the one you
> are looking for) to HTMLHelp system by calling a Win32 API.
I know this, and how to do the windows side. What I meant was that I
don't know enough about IDLE (or Tkinter) to implement the functionality
to find the word under the cursor.
> You'll need win32all installed. That's the way PythonWin works, by the way.
> I guess this makes it a no-no for standard Python, right?
>
I don't think win32all wraps the HTMLHelp api, does it?
OTOH, I'm waiting for a reason to include ctypes with core python ;-)
> Regards,
> -Hernan
>
Thomas
From theller at python.net Tue Sep 16 13:47:06 2003
From: theller at python.net (Thomas Heller)
Date: Tue Sep 16 13:46:46 2003
Subject: [Python-Dev] Windows installer and .pyc files mtime
In-Reply-To: (Tim
Peters's message of "Sat, 30 Aug 2003 22:29:35 -0400")
References:
Message-ID: <4qzck4md.fsf@python.net>
"Tim Peters" writes:
> [Thomas Heller]
>> Temporary download url:
>>
>> http://starship.python.net/crew/theller/python-2.3.1-beta1.exe
>>
>> This is built from CVS 23maint branch, some hours ago. The uncommited
>> changes I made are here, together with additional comments:
>>
>> http://www.python.org/sf/796919
>>
>> Only slightly tested so far.
>
> Slightly more now, on Win98SE. Cool! I thought something was wrong because
> the install on Win98SE went faster than I was used to. Turns out that's
> just because we're installing 1,200+ fewer files than we used to (thanks to
> shipping all the HTML docs in a single .chm file).
>
> Everything I tried worked fine. I think the *default* should be not to
> compile .pyc files. Regardless of the default, popping up a DOS box during
> compilation doesn't bother me at all.
Great, I'll change it. But when there's an option to compile .pyc,
shouldn't there also be an option for .pyo (and should it use -O or -OO)?
Thomas
From tim at zope.com Tue Sep 16 13:58:30 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 13:58:44 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To:
Message-ID:
About Jeremy's segfault, we've debugged this in an IM session, and it's a
bug in Python's (2.1, 2.2, 2.3, doesn't matter) unicodeobject.c. We'll fix
it. You're most likely to see a segfault under a UCS4 build. Just another
reason to upgrade to Windows .
From tim at zope.com Tue Sep 16 14:13:30 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 14:13:44 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To:
Message-ID:
[Tim]
> ...
> If some internal Unicode operation decided to allocate a short Unicode
> string, but freed it before filling in any of the string bytes, I
> suppose the keepalive optimization would retain a Unicode object with
> uninitialized str space on the unicode free list. A subsequent
> _PyUnicode_New could grab that and try to boost its ->str size. That
> could explain it.
And that turned out to be the case. One example (there are more) is in
PyUnicode_DecodeASCII(): the local
PyUnicodeObject *v;
gets initialized:
v = _PyUnicode_New(size);
Suppose size is 1. Suppose the string coming in is "\xc8". The first
iteration of the loop sets a "ordinal not in range(128)" error, and jumps to
onError:
onError:
Py_XDECREF(v);
unicode_dealloc() then stuffs v on unicode_freelist, and because the length
"is small" (size == 1), v->str[] is retained, still holding uninitialized
heap trash.
A later _PyUnicode_New() grabs this off unicode_freelist, decides to boost
the str space via unicode_resize(), and the latter blows up in the
unicode_latin1[unicode->str[0]] == unicode)) {
check because unicode->str[0] happens to be gigantically negative on
Jeremy's box, and the preceding
unicode->str[0] < 256 &&
check is too weak. Or is there an implicit assumption that Py_UNICODE is
always an unsigned type (in which case, why isn't the literal 256U?; and in
which case, it doesn't seem to be true on Jeremy's box).
From tim at zope.com Tue Sep 16 14:47:00 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 14:48:12 2003
Subject: [Python-Dev] Windows installer and .pyc files mtime
In-Reply-To: <4qzck4md.fsf@python.net>
Message-ID:
[Tim]
>> Everything I tried worked fine. I think the *default* should be not
>> to compile .pyc files. Regardless of the default, popping up a DOS
>> box during compilation doesn't bother me at all.
[Thomas Heller]
> Great, I'll change it. But when there's an option to compile .pyc,
> shouldn't there also be an option for .pyo (and should it use -O or
> -OO)?
Sounds like YAGNI. In the years I did the Windows installer, nobody asked
for compilation. I haven't seen anyone here ask for it either, just a few
people saying they think other people might want it. Since I wasn't one of
them, you're going to have to ask someone who thinks they can channel these
oddly unassertive users .
From martin at v.loewis.de Tue Sep 16 15:52:26 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 16 15:52:46 2003
Subject: [Python-Dev] Python and POSIX
In-Reply-To: <46570000.1063709941@leeloo.intern.geht.de>
References: <35230000.1063660079@leeloo.intern.geht.de>
<46570000.1063709941@leeloo.intern.geht.de>
Message-ID:
Marc Recht writes:
> I had a quick glance at the Solaris 8 headers and it seems that's
> influenced by __EXTENSIONS__, too. So it _should_ be the same as
> without the macros.
While that is true, I would hope that you come to the conclusion that
these macros can potentially do much more than simply hiding library
functions. Instead, their *primary* purpose is to request Single Unix
behaviour in case a platform implements both the standard behaviour,
and the traditional one (regardless of which tradition the system
follows).
So I would be *very* careful to enable these macros only for a few
translation units; I'm pretty sure that the "law" would be on the
vendor's side if inconsistent usage of these defines would be found to
cause problems, in Irix 27, or ValueBSD 3.
> That's a bit harder since I don't have access to all "relevant"
> systems. I could test on the open-source x86 platforms, though. And
> IIRC CompileFarm has Solaris, too.
If the change is only for 2.4, that would be fine - that code
hopefully will get testing on several other platforms before 2.4 is
released.
> And I _believe_ that - since these changes came up with Python 2.3
> (with the exception of _POSIX_1_SOURCE and _POSIX_SOURCE) - that it
> will break not much or anything at all if at least _XOPEN_SOURCE are
> moved _XOPEN_SOURCE_EXTENDED to an internal header (or an internal
> part of pyconfig.h).
Believes should not be trusted too much when it comes to
portability. If you can think of an obscure interpretation of some
standard, it is almost certain that there is some vendor out there who
has chosen that interpretation.
Regards,
Martin
From martin at v.loewis.de Tue Sep 16 15:55:46 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 16 15:56:21 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To:
References:
Message-ID:
"Tim Peters" writes:
> Still, it doesn't look to me like the code *expected* that str could contain
> uninitialized heap trash at this point, so I'd like someone who thinks they
> understand this code to think about how that could happen (it apparently is
> happening, so "beats me" isn't good enough ).
I would like to see a stack trace.
Regards,
Martin
From martin at v.loewis.de Tue Sep 16 16:00:27 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 16 16:00:45 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To: <1063729880.4868.8.camel@localhost.localdomain>
References:
<1063729880.4868.8.camel@localhost.localdomain>
Message-ID:
Jeremy Hylton writes:
> > Still, it doesn't look to me like the code *expected* that str could contain
> > uninitialized heap trash at this point, so I'd like someone who thinks they
> > understand this code to think about how that could happen (it apparently is
> > happening, so "beats me" isn't good enough ).
>
> I certainly don't understand the code yet.
The code you quote is about the "singleton" Unicode character
(i.e. those with code points < 256). If you create a Unicode object S
with len(S)==1, and ord(S) < 256, you don't necessarily get a new
object, but you may get an existing (cached) one. If you then apply
PyUnicde_Resize to that object, you get an error because resizing that
object would require changing all shared copies, which is bad because
Unicode object are immutable.
I agree with Tim that, in no case, resize should be called for a
garbage string - only valid strings (i.e. with truly allocated memory)
should ever be resized.
Regards,
Martin
From martin at v.loewis.de Tue Sep 16 16:05:18 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Sep 16 16:05:50 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To:
References:
Message-ID:
"Tim Peters" writes:
> Or is there an implicit assumption that Py_UNICODE is always an
> unsigned type (in which case, why isn't the literal 256U?
Likely, the author of the code was subconsciously assuming
so. However, that assumption is invalid, and sharing should only
happen for ordinals in range(256).
Regards,
Martin
From tim.one at comcast.net Tue Sep 16 16:46:31 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep 16 16:46:42 2003
Subject: [Python-Dev] RE: [Zope-Coders] core dump in Zope 2.7 test suite
In-Reply-To:
Message-ID:
[Martin v. L?wis]
> ...
> I agree with Tim that, in no case, resize should be called for a
> garbage string - only valid strings (i.e. with truly allocated memory)
> should ever be resized.
unicode->str pointed to properly allocated memory in the failing cases, but
to uninitialized allocated memory. So str[0] was just some random
collection of bits from the heap, and checking
random bits < 256
wasn't enough to weed out the cases where the random bits happened to look
like a negative integer. For Jeremy, str[0] was 0xcbcbcbcb (for me it was
0xcbcb), and using that as an index blew up.
...
[Tim]
>> Or is there an implicit assumption that Py_UNICODE is always an
>> unsigned type (in which case, why isn't the literal 256U?
[Martin]
> Likely, the author of the code was subconsciously assuming
> so. However, that assumption is invalid, and sharing should only
> happen for ordinals in range(256).
I've since patched Jeremy's patch to try to live with that Py_UNICODE may be
a signed type too.
From mal at lemburg.com Tue Sep 16 17:22:26 2003
From: mal at lemburg.com (M.-A. Lemburg)
Date: Tue Sep 16 17:22:28 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
unicodeobject.c, 2.197, 2.198
In-Reply-To:
References:
Message-ID: <3F677F12.80409@lemburg.com>
tim_one@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory sc8-pr-cvs1:/tmp/cvs-serv17421/Objects
>
> Modified Files:
> unicodeobject.c
> Log Message:
> On c.l.py, Martin v. L?wis said that Py_UNICODE could be of a signed type,
> so fiddle Jeremy's fix to live with that. Also added more comments.
Note that the implementation will bomb in several places if
Py_UNICODE is a signed type.
Py_UNICODE was never intended to be a signed type, so the proper
fix would be to add logic so that Py_UNICODE gets forced to be an
unsigned type.
The only case where Py_UNICODE could become signed is via
a compiler that defines wchar_t to be signed -- rather unlikely.
--
Marc-Andre Lemburg
eGenix.com
Professional Python Software directly from the Source (#1, Sep 16 2003)
>>> Python/Zope Products & Consulting ... http://www.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::
From tim.one at comcast.net Tue Sep 16 17:39:58 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Sep 16 17:40:04 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
unicodeobject.c, 2.197, 2.198
In-Reply-To: <3F677F12.80409@lemburg.com>
Message-ID:
[M.-A. Lemburg]
> Note that the implementation will bomb in several places if
> Py_UNICODE is a signed type.
>
> Py_UNICODE was never intended to be a signed type, so the proper
> fix would be to add logic so that Py_UNICODE gets forced to be an
> unsigned type.
Jeremy believed Py_UNICODE was already an unsigned type on his box, and that
was the box with the segfaults. I don't know. Comparison with a signed int
confused the issues to the point where we gave up and just fixed it .
> The only case where Py_UNICODE could become signed is via
> a compiler that defines wchar_t to be signed -- rather unlikely.
The C standard requires wchar_t to be an integer type, but doesn't constrain
it further to an unsigned integer type. I don't like relying on
non-standard assumptions in cases where there's little-to-no cost in not
relying on them. For example, the cast I put in with this patch is probably
a nop on most boxes, just forcing an unsigned comparison (which must have
been the original intent, if Py_UNICODE was assumed to be an unsigned type).
From gjc at inescporto.pt Tue Sep 16 18:04:42 2003
From: gjc at inescporto.pt (Gustavo J A M Carneiro)
Date: Tue Sep 16 18:06:34 2003
Subject: Solutions for LC_NUMERIC, was Re: [Python-Dev] Re: Be Honest
about LC_NUMERIC [REPOST]
In-Reply-To:
References:
<20030903134155.GB649@async.com.br>
Message-ID: <1063749875.20959.15.camel@emperor.homelinux.net>
A Qua, 2003-09-03 ?s 22:32, Martin v. L?wis escreveu:
> Christian Reis writes:
>
> > "On glibc-based systems, Python is safe from runtime changes in the
> > LC_NUMERIC locale category. On other platforms, changing LC_NUMERIC
> > at runtime may cause float conversion routines to fail. This may
> > impact the Python interpreter internally."
>
> I could accept such a limitation. However, I maintain that the aspects
> must be fully understood, and I agree with Guido that they must be
> documented in the documentation proper (i.e. as comments in the source
> code, or the user documentation).
>
> So I suggest that you proceed with the PEP and the implementation
> as-is. The PEP would need to be enhanced to summarize this discussion
> (perhaps listing the alternatives that you just mentioned); the patch
> would need to be enhanced to be more simple, readable, and in the
> style of the rest of the Python source code, and would need to include
> documentation changes and test cases (although some test cases
> probably already exist).
I have updated the patch[1], with James Henstridge's cleaned up code
(and some more cleanup). I am aware it still needs more comments and
test cases, but interested parties might want to start looking at it.
Please note that, after some discussion, we have concluded it is
better not to use glibc locale extensions to implement thread-safe float
conversion. The main reasons are: (1) simplicity (no ifdefs); (2)
consistency between glibc and non-glibc platforms. We should accept
the fact python programs should not be allowed change locale after
creating threads, period. Perhaps python itself should raise an
exception if this is ever attempted. This is subject to discussion, of
course! ;)
Regards.
[1]
http://sourceforge.net/tracker/download.php?group_id=5470&atid=305470&file_id=61688&aid=774665
--
Gustavo J. A. M. Carneiro
From lists at webcrunchers.com Tue Sep 16 16:50:09 2003
From: lists at webcrunchers.com (John D.)
Date: Tue Sep 16 18:44:29 2003
Subject: [Python-Dev] python/dist/src/Lib/bsddb __init__.py,1.5,1.6
In-Reply-To: <005a01c37962$b94a0e20$30bc958d@oemcomputer>
References: <001d01c3795d$c2ba1400$30bc958d@oemcomputer>
<16226.6197.573407.585746@montanaro.dyndns.org>
Message-ID:
Raymond writes:
>[Raymond]
>> >> > IIRC, dbhash and bsddb don't affect the Apple MacIntosh users.
>[Skip]
>> I don't know what you mean here. Do you mean Mac OS 9 or Mac OS X? I
>> use bsddb all the time on Mac OS X.
>
>Hmm, I'll have to fix the docs too. They currently list bsddb as being
>only for Unix and Windows.
While you are at it, please add a few more examples of how to use "bsddb3"
in particular the "dbtables" section. Like how do you select all from a table.
JD
From tim at zope.com Tue Sep 16 21:34:50 2003
From: tim at zope.com (Tim Peters)
Date: Tue Sep 16 21:35:40 2003
Subject: [Python-Dev] RE: [Zope-CVS] CVS: Products/Ape/lib/apelib/zodb3 -
serializers.py:1.3storage.py:1.8
In-Reply-To: <200309162100.h8GL08H01454@cvs.baymountain.com>
Message-ID:
[Shane Hathaway]
> Modified Files:
> serializers.py storage.py
> Log Message:
> Fixed Python 2.3 compatibility.
>
> - If you append to a list while it is being pickled, Python 2.3 will
> pick up the new items. Previous versions of Python did not.
How do you grow a list while it's being pickled? Must be another thread
doing that, and if so that's mighty dubious. If you're pickling to a
genuine file object, then cPickle releases the GIL around its fwrite()
calls, so that's one way. But in the context of the patch:
+ p.persistent_id = lambda ob: None # Stop recording references
p.dump(unmanaged)
s = outfile.getvalue()
event.addUnmanagedPersistentObjects(unmanaged)
the use of .getvalue() suggests the pickle target isn't a real file. Then
cPickle never releases the GIL (cStringIO doesn't either), and no other
thread should dare muck with a Python object (like the Python list getting
pickled) without holding the GIL.
However this ends up happening, if you instead shrink a list while it's
being pickled, Pythons earlier than 2.3 can end up trying to pickle random
recycled memory. Their cPickles captured the list length once at the start
of pickling the list, and that coding optimization is appropriate only if
the GIL is never released. 2.3 uses the iteration protocol instead, so
"sees" dynamic changes in the list size.
Copying Python-Dev because this may reveal a new mountain of ways to crash
the interpreter: mutate objects in devious ways while they're getting
cPickled.
From bac at OCF.Berkeley.EDU Tue Sep 16 23:09:44 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Tue Sep 16 23:09:49 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F674015.8090801@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
<16231.7058.987290.575924@montanaro.dyndns.org>
<3F674015.8090801@ocf.berkeley.edu>
Message-ID: <3F67D078.1000106@ocf.berkeley.edu>
Brett C. wrote:
> Skip Montanaro wrote:
>
>> Brett> Well, Kurt Kaiser wrote up a patch (
>> Brett> http://www.python.org/sf/762934 ) that seems sound. I have
>> Brett> gotten zero responses from anyone to test it on Red Hat 6.1
>> and a
>> Brett> remember that some people here were helping to test last
>> time (I
>> Brett> also remember this led to the creation of a wiki page listing
>> Brett> platforms various people could test things on but I can't find
>> Brett> the page).
>>
>> That would be:
>>
>> http://www.python.org/cgi-bin/moinmoin/PythonTesters
>>
>> I see Kurt Kaiser as the only person listed who can test for RH < 6.2.
>> Perhaps you should broadcast a call for testers to c.l.py, referring
>> to the
>> above page and asking in particular for RH 6.1 folks to chime in.
>>
>
> Jordan Krushen pointed out that the patch is actually opened against RH
> 6.2, not 6.1 like I originally said. But I don't think it is that big
> of an issue since there is a bug report for this against SuSE. I think
> the key thing here is an older version of glibc since that would be what
> defines the tzset function.
>
Well, Jordan emailed me personally with the result of running RH 6.1 and
RH 6.2 under VMWare; both passed (thanks, Jordan)! So, does anyone have
any objections to me checking this in, or do people feel this is not
enough testing to show the patch does what it needs to?
And a question on patching configure.pre.in: is that all I have to patch
or do I need to run reconfigure and then also patch configure.in?
-Brett
From martin at v.loewis.de Wed Sep 17 00:26:57 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Sep 17 00:27:20 2003
Subject: [Python-Dev] Trying to fix time.tzset detection for 2.3.1
In-Reply-To: <3F67D078.1000106@ocf.berkeley.edu>
References: <3F669AF3.8050605@ocf.berkeley.edu>
<16231.7058.987290.575924@montanaro.dyndns.org>
<3F674015.8090801@ocf.berkeley.edu>
<3F67D078.1000106@ocf.berkeley.edu>
Message-ID:
"Brett C." writes:
> And a question on patching configure.pre.in: is that all I have to
> patch or do I need to run reconfigure and then also patch configure.in?
There is no configure.pre.in in Python.
Martin
From hernan at orgmf.com.ar Wed Sep 17 02:53:18 2003
From: hernan at orgmf.com.ar (=?us-ascii?Q?Hernan_Martinez_Foffani?=)
Date: Wed Sep 17 02:55:24 2003
Subject: [Python-Dev] RE: HTMLHelp for Py2.3.1
In-Reply-To: <8yook4or.fsf@python.net>
Message-ID: