From g.brandl at gmx.net  Sat Nov  1 18:43:08 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 01 Nov 2008 18:43:08 +0100
Subject: [Python-Dev] My patches
In-Reply-To: <94bdd2610810310118y34b72d9ej6f40cbfc9216568f@mail.gmail.com>
References: <200810301108.35591.victor.stinner@haypocalc.com>	<20081030110442.1b6ce78e@resist.wooz.org>	<20081030150427.GA9631@amk-desktop.matrixgroup.net>	<79990c6b0810300855r791fad95w940d5d2bc7d0abd6@mail.gmail.com>	<94bdd2610810300909hf7c0337va779682982198b1d@mail.gmail.com>	<490AA9C2.403@gmail.com>
	<94bdd2610810310118y34b72d9ej6f40cbfc9216568f@mail.gmail.com>
Message-ID: <gei4ff$5r5$1@ger.gmane.org>

Tarek Ziad? schrieb:
> On Fri, Oct 31, 2008 at 7:46 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>>
>>> What about having two level of devs ?
>>>
>>> + core developers + standard library developers
>>>
>>> [cut]
>>
>>
>> So I'd suggest thinking about developer responsibilities more in terms
>> of areas of expertise rather than "levels" of developers. Those of us
>> that happen to understand the guts of the compiler or the VM aren't more
>> competent globally or more trusted than those maintaining the various
>> modules in the standard library - just interested in different things.
> 
> Right,
> 
> I would like to share my experience about this, if it can be helpful.
> 
> I have focused so far in distutils, which, I believe
> was not in the top priority of core developers during the last year.
> (If this is not true
> please forgive me).

I think it's safe to say that anything not directly involved in Python
3000 changes was not top priority for most core developers.

> Anyway, so I am starting to become quite specialized in this part of
> Python, and I pushed patches for it in the tracker.
> 
> The patches I wrote that made it so far took between 4 to 8 months to
> be applied.
> 
> I have really simple patches for distutils that are just adding tests,
> that are waiting for
> review. I guess these patches will be reviewed in a few months, I am
> failry confident
> about that. I know core developers are drowned into more important topics.
> And this area of Python is being highly discussed to be refactored, maybe
> outside the stdlib at some point, but there are a *lot* of simple
> things to do today in there.
> 
> So basically, if I want to be efficient in distutils I need to become
> a core developer,
> and this means (Guido said) I need to start providing patches for the
> rest of the Python
> code and eventually (after a few years I guess) maybe become a core developer.

I don't think this is what Guido meant. Or, put differently, it depends on
what being a "core developer" means. If it isn't the same as "committer",
maybe. However, there is nothing wrong with being a committer and an expert
on part of the code.

> Then I'll be able to work in distutils because at that point in the
> future I'll be trusted.
> 
> I can't do that ! I don't have the time to become a Python core code expert.
> But in my everyday work I became a packaging / deploying specialist.
> 
> So my point is : if I am "trusted" at some point in the work I am doing in
> distutils, will I have a commit access there ?

If I'm not mistaken, nobody will object to that. In the beginning you should
find another developer who reviews your (pending) checkins, so that code style
and other conventions are maintained.

The most important thing for distutils is of course backwards compatibility,
so that nobody is forced to rewrite their setup.py scripts.
Especially for distutils, there is the SIG mailing list which is meant as the
authoritative group for distutils decisions. (I'm not subscribed to it, so
perhaps you've already discussed your intended changes there.)

cheers,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From fijall at gmail.com  Sat Nov  1 21:12:56 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 1 Nov 2008 21:12:56 +0100
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
Message-ID: <693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>

> ...

>
> We know it is the plan for PyPy to work in this way, and also that
> Jython and Ironpython works like that (using the host vm's GC), so it
> seems to be somehow agreeable with the python semantics (perhaps not
> really with __del__ but they are not really nice anyway).
>

PyPy has a semi-advanced generational moving gc these days. It's not
as well tuned as JVMs one, but it works quite well. Regarding semantic
changes, there is a couple which as far as I remember are details
which you should not rely on anyway (At least some of the below
applies also for Jython/IronPython):

* __del__ methods are not called immediately when object is not in a cycle

* all objects are collected, which means objects in cycles are broken
in arbitrary order (gc.garbage is always empty), but normal ordering
is preserved.

* id's don't always resemble address of object in memory

* resurrected objects have __del__ called only once in total.

I think for example last one is so obscure detail that if someone
relies on it it's his problem :)

Cheers,
fijal

From greg at krypto.org  Sat Nov  1 22:32:05 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 1 Nov 2008 14:32:05 -0700
Subject: [Python-Dev] [ANN] VPython 0.1
In-Reply-To: <87abcu2zvb.fsf@svirfneblin.org>
References: <873aindc0e.fsf@svirfneblin.org> <49004BA4.5050601@egenix.com>
	<87abcu2zvb.fsf@svirfneblin.org>
Message-ID: <52dc1c820811011432q4606ab3co91e204d62c80d2eb@mail.gmail.com>

On Fri, Oct 24, 2008 at 12:53 AM, J. Sievers <cadr4u at gmail.com> wrote:
> "M.-A. Lemburg" <mal at egenix.com> writes:
>
> [snip]
>> BTW: I hope you did not use pybench to get profiles of the opcodes.
>> That would most certainly result in good results for pybench, but
>> less good ones for general applications such as Django or Zope/Plone.
>
> Algorithm used for superinstruction selection:
>
> 1) idea: LOAD_CONST/LOAD_FAST + some suffix
> 2) potential suffixes:
>   $ grep '..*(..*--..*)$' ceval.vmg | grep 'a1 a2 --' > INSTRUCTIONS
> 3) delete any instruction that I felt wouldn't be particularly frequent
>   from INSTRUCTIONS (e.g. raise_varargs)
> 4) use awk to generate superinstruction definitions
>
> Not only is this relatively unbiased but also very low effort.
>
> -jakob
>

Thanks Jakob.  I'm glad to see that you've done this.  I had been
hoping to see super instructions tried on the python VM for years but
never found the time to do it myself.

-gps

From ncoghlan at gmail.com  Sun Nov  2 01:21:26 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 02 Nov 2008 10:21:26 +1000
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>
Message-ID: <490CF286.6090704@gmail.com>

Maciej Fijalkowski wrote:
>> ...
> 
>> We know it is the plan for PyPy to work in this way, and also that
>> Jython and Ironpython works like that (using the host vm's GC), so it
>> seems to be somehow agreeable with the python semantics (perhaps not
>> really with __del__ but they are not really nice anyway).
>>
> 
> PyPy has a semi-advanced generational moving gc these days. It's not
> as well tuned as JVMs one, but it works quite well. Regarding semantic
> changes, there is a couple which as far as I remember are details
> which you should not rely on anyway (At least some of the below
> applies also for Jython/IronPython):
> 
> * __del__ methods are not called immediately when object is not in a cycle
> 
> * all objects are collected, which means objects in cycles are broken
> in arbitrary order (gc.garbage is always empty), but normal ordering
> is preserved.
> 
> * id's don't always resemble address of object in memory
> 
> * resurrected objects have __del__ called only once in total.

Yep, I'm pretty those are all even explicitly documented as
implementation details of CPython (PEP 343's with statement was largely
added as a cross-implementation way of guaranteeing prompt cleanup of
resources like file handles without relying on CPython's __del__
semantics or writing your own try/finally statements everywhere).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From barry at python.org  Sun Nov  2 13:33:15 2008
From: barry at python.org (Barry Warsaw)
Date: Sun, 2 Nov 2008 07:33:15 -0500
Subject: [Python-Dev] My patches
In-Reply-To: <4909C30F.1070100@trueblade.com>
References: <200810301108.35591.victor.stinner@haypocalc.com>
	<20081030110442.1b6ce78e@resist.wooz.org>
	<200810301302.02796.victor.stinner@haypocalc.com>
	<200810301319.15760.eckhardt@satorlaser.com>
	<4909C30F.1070100@trueblade.com>
Message-ID: <C9D0DF1B-1857-4956-AB2E-CF3E4E51B679@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 30, 2008, at 10:22 AM, Eric Smith wrote:

> Ulrich Eckhardt wrote:
>> On Thursday 30 October 2008, Victor Stinner wrote:
>>>> One of the reasons why I'm very keen on us moving to a distributed
>>>> version control system is to help break the logjam on core  
>>>> developers.
>>> Yeah, exactly :-) Does anyone already maintain a distributed tree?
>>> Mercurial, GIT, anything else?
>> Bazaar. Take a look at the developers' pages on python.org, they  
>> mention that a BZR checkout is available. I know that it works  
>> (though the initial checkout is glacially slow) but I don't know  
>> what "official" support it has or what is planned with it.
>
> I'd like to try it out, but the instructions on http://www.python.org/dev/bazaar/ 
>  say to get wget http://code.python.org/snapshots/python-bzr-snapshot.tar.bz2 
> , which is a 404. Anyone know the actual URL?

This is fixed now.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ2eDnEjvBPtnXfVAQJf+gP7BvsB7NtENGqbQR7oGMGjie75MrEn362Y
hXcgALWn9UkaiSdcrL+b9DFLY8MYppYUvKEJw92gU9jMzIzgcdH57DGBkudx/PHS
hkvPhtGewdSX1QY88CkOYjJ7UYnhrhZWUzN67Yea16QKBwPKQFljsC26toqX+08k
J+vZDkAsmwE=
=Gy0x
-----END PGP SIGNATURE-----

From barry at python.org  Sun Nov  2 13:42:20 2008
From: barry at python.org (Barry Warsaw)
Date: Sun, 2 Nov 2008 07:42:20 -0500
Subject: [Python-Dev] My patches
In-Reply-To: <20081030150427.GA9631@amk-desktop.matrixgroup.net>
References: <200810301108.35591.victor.stinner@haypocalc.com>
	<20081030110442.1b6ce78e@resist.wooz.org>
	<20081030150427.GA9631@amk-desktop.matrixgroup.net>
Message-ID: <0256230C-B00D-4BBB-AC8A-95FF4ED24AFC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 30, 2008, at 11:04 AM, A.M. Kuchling wrote:

> On Thu, Oct 30, 2008 at 11:04:42AM +0000, Barry Warsaw wrote:
>> One of the reasons why I'm very keen on us moving to a distributed  
>> version
>> control system is to help break the logjam on core developers.   
>> True, your
>> code will still not be able to land in the "official" branch  
>> without core
>> developer intervention, but you will be able to share your code,  
>> fixes,
>> branches with everyone in a much more live way than patches in a  
>> tracker.
>
> I don't see how a DVCS will fix anything.  The bottleneck is in
> assessing patches for inclusion in the master tree; not enough people
> are doing that.  We'd just end up with lots of proposed branches
> waiting to be merged, instead of patches to be applied.

I disagree.  Dealing with patches is much more painful than dealing  
with branches.  A patch is a dead thing sitting in a tracker.  They  
easily get out of date, are difficult to apply and update, etc.  Live  
branches are much easier to review, update, track changes in the  
trunk, and share.

Yes, it still means we'll have a lot of branches to review, but the  
overhead of reviewing them and shepherding them to landing will go down.

> (What a DVCS might enable is making it easier to do larger
> experiments, like the recent Vmgen work, and publish them in a form
> that people can download.  We could create SVN branches now, but that
> means people would then have commit access to all of the Python
> source.)

A dvcs means that people can publish their branches in a wide variety  
of ways.  Trusted developers can push their branches to  
code.python.org.  Non-core developers can use one of the free public  
dvcs branch hosting service.  Industrious users can self-publish their  
branches.  Anyone with email can bundle a branch and send it via a  
smart email package (much different than just-a-patch).

This means both large and small experiments are very easy to perform.   
So are small branches that fix a particular bug or add a small  
feature.  In a good dvcs, branches are very cheap to make and share,  
regardless of whether you're a core developer or a casual hacker.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ2gLXEjvBPtnXfVAQJlaQQAq8uOlzqCjZbukBOj3Na5CqzUu1xaS4RU
356FpQO2TykQDguNp0dOq40JGJGvypsvDCh89YTT+c32M0KfxftQ/h0cHbC7Vjs5
0pXCdXA+WIAyiCVbql7iBxxvmPB2FfEyhN5TPGdMe2R6c+Cp5zCt4zxL6RAeLZ+F
ITJsJt4Suvw=
=wK17
-----END PGP SIGNATURE-----

From rasky at develer.com  Sun Nov  2 14:26:47 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Sun, 2 Nov 2008 13:26:47 +0000 (UTC)
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>
	<490CF286.6090704@gmail.com>
Message-ID: <gek9qn$1qi$1@ger.gmane.org>

On Sun, 02 Nov 2008 10:21:26 +1000, Nick Coghlan wrote:

> Maciej Fijalkowski wrote:
>>> ...
>> 
>>> We know it is the plan for PyPy to work in this way, and also that
>>> Jython and Ironpython works like that (using the host vm's GC), so it
>>> seems to be somehow agreeable with the python semantics (perhaps not
>>> really with __del__ but they are not really nice anyway).
>>>
>>>
>> PyPy has a semi-advanced generational moving gc these days. It's not as
>> well tuned as JVMs one, but it works quite well. Regarding semantic
>> changes, there is a couple which as far as I remember are details which
>> you should not rely on anyway (At least some of the below applies also
>> for Jython/IronPython):
>> 
>> * __del__ methods are not called immediately when object is not in a
>> cycle
>> 
>> * all objects are collected, which means objects in cycles are broken
>> in arbitrary order (gc.garbage is always empty), but normal ordering is
>> preserved.
>> 
>> * id's don't always resemble address of object in memory
>> 
>> * resurrected objects have __del__ called only once in total.
> 
> Yep, I'm pretty those are all even explicitly documented as
> implementation details of CPython (PEP 343's with statement was largely
> added as a cross-implementation way of guaranteeing prompt cleanup of
> resources like file handles without relying on CPython's __del__
> semantics or writing your own try/finally statements everywhere).

Though there is a fair difference from "explicitly documented as 
implementation details" and the real-world code where programmers have 
learnt to save code lines by relying on the reference-counting semantics.

[[ my 0.2: it would be a great loss if we lose reference-counting 
semantic (eg: objects deallocated as soon as they exit the scope). I 
would bargain that for a noticable speed increase of course, but my own 
experience with standard GCs from other languages has been less than 
stellar. ]]
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com


From meher4u1 at gmail.com  Sun Nov  2 15:40:55 2008
From: meher4u1 at gmail.com (Aditi Meher)
Date: Sun, 2 Nov 2008 20:10:55 +0530
Subject: [Python-Dev] buffer function
Message-ID: <608380240811020640v21905a7bwf7bc1bb0442eaedc@mail.gmail.com>

i am using it postgresql as back-end and HTML as front-end,i want to
display 10-10 records at a time which is there in the database using
python.so what is function for buffer that we can use it in python?i
am able to connect my databse in python,but dont know how to create
buffer in python and how to take records from tables which are there
in the database?can please provide me any kind of function or code for
buffer in python?

i have one more doubt that how to call python script in HTML?

PLEASE REPLY

From steve at holdenweb.com  Sun Nov  2 16:38:04 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 02 Nov 2008 10:38:04 -0500
Subject: [Python-Dev] buffer function
In-Reply-To: <608380240811020640v21905a7bwf7bc1bb0442eaedc@mail.gmail.com>
References: <608380240811020640v21905a7bwf7bc1bb0442eaedc@mail.gmail.com>
Message-ID: <490DC95C.4050707@holdenweb.com>

Aditi Meher wrote:
> i am using it postgresql as back-end and HTML as front-end,i want to
> display 10-10 records at a time which is there in the database using
> python.so what is function for buffer that we can use it in python?i
> am able to connect my databse in python,but dont know how to create
> buffer in python and how to take records from tables which are there
> in the database?can please provide me any kind of function or code for
> buffer in python?
> 
> i have one more doubt that how to call python script in HTML?
> 
> PLEASE REPLY

Please submit this question to the comp.lang.python group (python-list
at python dot org). Python-dev is for discussion or the development *of*
Python, not development *with* Python.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From solipsis at pitrou.net  Sun Nov  2 19:13:18 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 2 Nov 2008 18:13:18 +0000 (UTC)
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gedmtg$eop$1@ger.gmane.org> <490AA4B9.5060307@gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
Message-ID: <loom.20081102T180853-494@post.gmane.org>


Hi,

Jesse Noller <jnoller <at> gmail.com> writes:
> If python were to have free threading, courtesy of a lack
> of the GIL, it would help those people quite a bit. Sometimes you just
> need shared state. Myself? I used multiprocess *and* threads all the
> time for various reasons.

I think it is important to remind that the GIL doesn't prevent (almost) true
multithreading. The only thing it prevents is full use of multi-CPU resources in
a single process. But if you are using threads to reduce latencies rather than
increase throughput, the GIL isn't really an obstacle (it probably makes things
a bit worse than a free-threading scenario, but not that much; and you can tweak
sys.setcheckinterval()).

Regards

Antoine.



From stefan_ml at behnel.de  Sun Nov  2 21:02:15 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 02 Nov 2008 21:02:15 +0100
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <4909FECF.7000703@voidspace.org.uk>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk>
Message-ID: <490E0747.8050401@behnel.de>

Michael Foord wrote:
> Moving more C extensions to an implementation based on ctypes would be
> enormously useful for PyPy, IronPython and Jython, but ctypes is not yet
> as portable as Python itself which could be an issue (although one worth
> resolving).

In the same line, moving more extensions to a high-level language like Cython,
instead of writing them in straight C, would make a later switch to a
different environment a lot easier, as the extensions could be regenerated
with a modified Cython compiler (obviously minus some fixing of premature
optimisations and the like).

Stefan


From stefan_ml at behnel.de  Sun Nov  2 21:02:15 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 02 Nov 2008 21:02:15 +0100
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <4909FECF.7000703@voidspace.org.uk>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk>
Message-ID: <490E0747.8050401@behnel.de>

Michael Foord wrote:
> Moving more C extensions to an implementation based on ctypes would be
> enormously useful for PyPy, IronPython and Jython, but ctypes is not yet
> as portable as Python itself which could be an issue (although one worth
> resolving).

In the same line, moving more extensions to a high-level language like Cython,
instead of writing them in straight C, would make a later switch to a
different environment a lot easier, as the extensions could be regenerated
with a modified Cython compiler (obviously minus some fixing of premature
optimisations and the like).

Stefan


From eric at trueblade.com  Sun Nov  2 21:34:14 2008
From: eric at trueblade.com (Eric Smith)
Date: Sun, 02 Nov 2008 15:34:14 -0500
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <gek9qn$1qi$1@ger.gmane.org>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>	<490CF286.6090704@gmail.com>
	<gek9qn$1qi$1@ger.gmane.org>
Message-ID: <490E0EC6.1070404@trueblade.com>

Giovanni Bajo wrote:
> [[ my 0.2: it would be a great loss if we lose reference-counting 
> semantic (eg: objects deallocated as soon as they exit the scope). I 
> would bargain that for a noticable speed increase of course, but my own 
> experience with standard GCs from other languages has been less than 
> stellar. ]]

And my $0.02:

I'd gladly trade deterministic destruction (due to reference counting or 
any other mechanism) for improved performance. I've often thought of 
creating a mode where destruction didn't happen right away with 
reference counting, just so I could find places where I'm relying on it. 
  I consider it a bug to rely on reference counting to close files, for 
example. Maybe I should just run under Jython or IronPython everyone 
once in a while.

From rhamph at gmail.com  Sun Nov  2 22:07:45 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 2 Nov 2008 15:07:45 -0600
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <490E0EC6.1070404@trueblade.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>
	<490CF286.6090704@gmail.com> <gek9qn$1qi$1@ger.gmane.org>
	<490E0EC6.1070404@trueblade.com>
Message-ID: <aac2c7cb0811021307j11aafb84i66eacfa55c23eaad@mail.gmail.com>

On Sun, Nov 2, 2008 at 2:34 PM, Eric Smith <eric at trueblade.com> wrote:
> Giovanni Bajo wrote:
>>
>> [[ my 0.2: it would be a great loss if we lose reference-counting semantic
>> (eg: objects deallocated as soon as they exit the scope). I would bargain
>> that for a noticable speed increase of course, but my own experience with
>> standard GCs from other languages has been less than stellar. ]]
>
> And my $0.02:
>
> I'd gladly trade deterministic destruction (due to reference counting or any
> other mechanism) for improved performance. I've often thought of creating a
> mode where destruction didn't happen right away with reference counting,
> just so I could find places where I'm relying on it.  I consider it a bug to
> rely on reference counting to close files, for example. Maybe I should just
> run under Jython or IronPython everyone once in a while.

I've considered making files issue a warning if they've got buffered
writes and they're not explicitly closed.  Although my feeling is a
read-only file should produce the same warning, there's situations
where "proper" error handling is very difficult or impossible.


-- 
Adam Olsen, aka Rhamphoryncus

From greg.ewing at canterbury.ac.nz  Sun Nov  2 23:33:11 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 03 Nov 2008 11:33:11 +1300
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <490E0EC6.1070404@trueblade.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>
	<490CF286.6090704@gmail.com> <gek9qn$1qi$1@ger.gmane.org>
	<490E0EC6.1070404@trueblade.com>
Message-ID: <490E2AA7.4070206@canterbury.ac.nz>

Eric Smith wrote:

> I'd gladly trade deterministic destruction (due to reference counting or 
> any other mechanism) for improved performance.

Another thing to consider is that refcounting spreads out the
time spent doing GC evenly over the execution of the program,
so that you don't get pauses occurring at random times.

Sometimes in games I've found that I had to disable cyclic
GC in order to get smooth frame rates. With no refcounting
I wouldn't have the option of doing that. I'd be disappointed
if that meant I could no longer use Python for these kinds of
games.

-- 
Greg

From skip at pobox.com  Mon Nov  3 00:51:50 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 2 Nov 2008 17:51:50 -0600
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <loom.20081102T180853-494@post.gmane.org>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gedmtg$eop$1@ger.gmane.org> <490AA4B9.5060307@gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
	<loom.20081102T180853-494@post.gmane.org>
Message-ID: <18702.15638.223872.706216@montanaro-dyndns-org.local>


    Antoine> I think it is important to remind that the GIL doesn't prevent
    Antoine> (almost) true multithreading. The only thing it prevents is
    Antoine> full use of multi-CPU resources in a single process. 

I believe everyone here knows that.  I believe what most people are
clamoring for is to make "full use of their multi-CPU resources in a single
process".

Skip

From skip at pobox.com  Mon Nov  3 00:55:03 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 2 Nov 2008 17:55:03 -0600
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <490E0EC6.1070404@trueblade.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>
	<490CF286.6090704@gmail.com> <gek9qn$1qi$1@ger.gmane.org>
	<490E0EC6.1070404@trueblade.com>
Message-ID: <18702.15831.164083.97731@montanaro-dyndns-org.local>


    Eric> I consider it a bug to rely on reference counting to close files,

We can mostly have our cake and eat it too using the "with" statement.  In
most cases it should be sufficient I would think.

Skip


From brett at python.org  Mon Nov  3 01:05:56 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 2 Nov 2008 16:05:56 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
Message-ID: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>

I have started the DVCS PEP which can be seen at
http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
beyond the rationale, usage scenarios I plan to use, and what other
sections I plan to write.

At this point I am looking for any suggestions for fundamental usage
scenarios that I am missing from the PEP. If you think the few already
listed are missing some core part of a VCS, please let me know.

And just to stave off some emails, I have two comments to make. One,
please do not start sending me info on how to fill in the usage
scenarios. I want to first solidify what the scenarios are. Plus I
want to figure them out on my own to get a feel of the documentation
for the tools.

Second, as of right now Git is not in the running. Ignoring the fact I
dislike the tool (my issues with it are documented in the python-dev
archives), there is also the fact that it would be nicer to have
Python have as its VCS something written in Python instead of C/Perl.

-Brett

From eric at trueblade.com  Mon Nov  3 01:45:15 2008
From: eric at trueblade.com (Eric Smith)
Date: Sun, 02 Nov 2008 19:45:15 -0500
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <18702.15831.164083.97731@montanaro-dyndns-org.local>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>
	<490CF286.6090704@gmail.com> <gek9qn$1qi$1@ger.gmane.org>
	<490E0EC6.1070404@trueblade.com>
	<18702.15831.164083.97731@montanaro-dyndns-org.local>
Message-ID: <490E499B.9000703@trueblade.com>

skip at pobox.com wrote:
>     Eric> I consider it a bug to rely on reference counting to close files,
> 
> We can mostly have our cake and eat it too using the "with" statement.  In
> most cases it should be sufficient I would think.

True, and I meant to mention that. But unfortunately, my work projects 
are stuck on 2.4 for the time being, so that doesn't help me much.

Eric.

From gustavo at niemeyer.net  Mon Nov  3 02:08:11 2008
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Sun, 2 Nov 2008 23:08:11 -0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
Message-ID: <643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>

Hi Brett,

> At this point I am looking for any suggestions for fundamental usage
> scenarios that I am missing from the PEP. If you think the few already
> listed are missing some core part of a VCS, please let me know.

As an initial disclaimer, I use bzr in my daily routine.  That said,
I'm sending below a few mostly unbiased high-level ideas, just based
on useful ways we explore the DVCS-aspect on our normal daily
workflow.

== Coordinated development with dependent features ==

A variation on coordinated development, which is actually one of the
main motivators for DVCS.  Branch A is evolving out of the mainline,
and it depends on a feature that is on branch B which is also not yet
integrated.  Parallel development of both should be nicely supported.
I'm sure all DVCS will do that in a decent form, but figuring how this
works may be instructive and worth mentioning.

== Centralization of feature-specific development ==

That's a curious one when we're talking about distributed development,
isn't it? :-)   Think about the following scenario: 5 people working
in parallel on a tricky feature which will take a while to get to the
mainline.  Each developer works on a specific aspect of the feature
for a few hours/days in their own branch, and then merges to and from
what's considered as the feature-mainline.  In essence, the problem is
that it's messy to just go towards the "everyone merges from everyone"
route when working in a common problem.  We've found that being able
to use an svn-like approach for the *feature* mainline (a branch that
is shared by all defining the status quo) is a handy way to handle
that.  I'm sure there are other approaches to solve the same problem,
but it's interesting to consider what they are for each tool since in
our experience the problem will show up eventually, and supporting it
nicely makes a big difference on the outcome.

== Attaching of history-carrying diffs ==

This one may feel weird as well, and I'm guessing some people might
react as "just send a URL to a branch".  In practice, it is handy many
times to just attach something to the bug tracker, for instance.  This
means that the "branch" is kept in the history of the bug.  Also, with
something like this, someone external to the development process may
very easily provide his changes in the bug or in a mail without having
to set up any infrastructure/accounts/whatever at all to provide his
branch.


That's it for now.  If I can think of any other use cases from our
routine that might serve as things to explore in such a comparison,
I'll let you know.

-- 
Gustavo Niemeyer
http://niemeyer.net

From aleaxit at gmail.com  Mon Nov  3 03:10:45 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Sun, 2 Nov 2008 19:10:45 -0700
Subject: [Python-Dev] My patches
In-Reply-To: <0256230C-B00D-4BBB-AC8A-95FF4ED24AFC@python.org>
References: <200810301108.35591.victor.stinner@haypocalc.com>
	<20081030110442.1b6ce78e@resist.wooz.org>
	<20081030150427.GA9631@amk-desktop.matrixgroup.net>
	<0256230C-B00D-4BBB-AC8A-95FF4ED24AFC@python.org>
Message-ID: <e8a0972d0811021810v5ebe0dbbl7ea7db5c4b097610@mail.gmail.com>

On Sun, Nov 2, 2008 at 5:42 AM, Barry Warsaw <barry at python.org> wrote:
   ...
> A dvcs means that people can publish their branches in a wide variety of
> ways.  Trusted developers can push their branches to code.python.org.
>  Non-core developers can use one of the free public dvcs branch hosting
> service.  Industrious users can self-publish their branches.  Anyone with

BTW, if we go with hg, I recommend bitbucket.org as a "free public
(hg) hosting service" of choice -- Jesper (the owner and developer of
the site) is friendly and solicitous in his maintenance, the whole
site is quite OS-friendly in general (free "professional-level"
accounts and support for open-source projects which choose to host
there) and Python-friendly in particular (I gather the site's coded in
Python), and my experience there has been nothing short of excellent.
All it's missing is a simple code review tool like code.google.com's
or Rietveld, but Jesper's promised me he would integrate something to
that effect...


Alex

From musiccomposition at gmail.com  Mon Nov  3 04:03:52 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sun, 2 Nov 2008 21:03:52 -0600
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
Message-ID: <1afaf6160811021903j3614225qadfbd15a58d54326@mail.gmail.com>

On Sun, Nov 2, 2008 at 6:05 PM, Brett Cannon <brett at python.org> wrote:
> I have started the DVCS PEP which can be seen at
> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
> beyond the rationale, usage scenarios I plan to use, and what other
> sections I plan to write.
>
> At this point I am looking for any suggestions for fundamental usage
> scenarios that I am missing from the PEP. If you think the few already
> listed are missing some core part of a VCS, please let me know.

I think one very important requirement of Python's VCS is advanced
merging support. The trunk and py3k are very special branches because
a commit in one should not necessarily find its way into the other. To
that end, I think it's critical to have svnmerge.py-like blocking
(preferably with unblocking, too) and fine cherry-picking of
revisions.


-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From sonia_16 at live.com  Mon Nov  3 04:59:02 2008
From: sonia_16 at live.com (Sonia)
Date: Mon, 3 Nov 2008 08:59:02 +0500
Subject: [Python-Dev] =?windows-1256?q?__How_to_select_text_of_text_field_?=
 =?windows-1256?q?in_python=FECard?=
In-Reply-To: <mailman.21986.1225678248.3486.python-dev@python.org>
References: <mailman.21986.1225678248.3486.python-dev@python.org>
Message-ID: <BLU114-W454EF34F15E7095875A9E811D0@phx.gbl>


Hi, I need to select a text of text field! Which function is used for selection of text of text field? Sonia,
 
> From: python-dev-request at python.org> Subject: Python-Dev Digest, Vol 64, Issue 4> To: python-dev at python.org> Date: Mon, 3 Nov 2008 03:10:48 +0100> > Send Python-Dev mailing list submissions to> python-dev at python.org> > To subscribe or unsubscribe via the World Wide Web, visit> http://mail.python.org/mailman/listinfo/python-dev> or, via email, send a message with subject or body 'help' to> python-dev-request at python.org> > You can reach the person managing the list at> python-dev-owner at python.org> > When replying, please edit your Subject line so it is more specific> than "Re: Contents of Python-Dev digest..."> > > Today's Topics:> > 1. Re: Fwd: Removal of GIL through refcounting removal. (Eric Smith)> 2. Re: Fwd: Removal of GIL through refcounting removal. (Adam Olsen)> 3. Re: Fwd: Removal of GIL through refcounting removal. (Greg Ewing)> 4. Re: Fwd: Removal of GIL through refcounting removal.> (skip at pobox.com)> 5. Re: Fwd: Removal of GIL through refcounting removal.> (skip at pobox.com)> 6. Looking for VCS usage scenarios (Brett Cannon)> 7. Re: Fwd: Removal of GIL through refcounting removal. (Eric Smith)> 8. Re: Looking for VCS usage scenarios (Gustavo Niemeyer)> 9. Re: My patches (Alex Martelli)> > > ----------------------------------------------------------------------> > Message: 1> Date: Sun, 02 Nov 2008 15:34:14 -0500> From: Eric Smith <eric at trueblade.com>> Subject: Re: [Python-Dev] Fwd: Removal of GIL through refcounting> removal.> To: python-dev at python.org> Message-ID: <490E0EC6.1070404 at trueblade.com>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed> > Giovanni Bajo wrote:> > [[ my 0.2: it would be a great loss if we lose reference-counting > > semantic (eg: objects deallocated as soon as they exit the scope). I > > would bargain that for a noticable speed increase of course, but my own > > experience with standard GCs from other languages has been less than > > stellar. ]]> > And my $0.02:> > I'd gladly trade deterministic destruction (due to reference counting or > any other mechanism) for improved performance. I've often thought of > creating a mode where destruction didn't happen right away with > reference counting, just so I could find places where I'm relying on it. > I consider it a bug to rely on reference counting to close files, for > example. Maybe I should just run under Jython or IronPython everyone > once in a while.> > > ------------------------------> > Message: 2> Date: Sun, 2 Nov 2008 15:07:45 -0600> From: "Adam Olsen" <rhamph at gmail.com>> Subject: Re: [Python-Dev] Fwd: Removal of GIL through refcounting> removal.> To: "Eric Smith" <eric at trueblade.com>> Cc: python-dev at python.org> Message-ID:> <aac2c7cb0811021307j11aafb84i66eacfa55c23eaad at mail.gmail.com>> Content-Type: text/plain; charset=ISO-8859-1> > On Sun, Nov 2, 2008 at 2:34 PM, Eric Smith <eric at trueblade.com> wrote:> > Giovanni Bajo wrote:> >>> >> [[ my 0.2: it would be a great loss if we lose reference-counting semantic> >> (eg: objects deallocated as soon as they exit the scope). I would bargain> >> that for a noticable speed increase of course, but my own experience with> >> standard GCs from other languages has been less than stellar. ]]> >> > And my $0.02:> >> > I'd gladly trade deterministic destruction (due to reference counting or any> > other mechanism) for improved performance. I've often thought of creating a> > mode where destruction didn't happen right away with reference counting,> > just so I could find places where I'm relying on it. I consider it a bug to> > rely on reference counting to close files, for example. Maybe I should just> > run under Jython or IronPython everyone once in a while.> > I've considered making files issue a warning if they've got buffered> writes and they're not explicitly closed. Although my feeling is a> read-only file should produce the same warning, there's situations> where "proper" error handling is very difficult or impossible.> > > -- > Adam Olsen, aka Rhamphoryncus> > > ------------------------------> > Message: 3> Date: Mon, 03 Nov 2008 11:33:11 +1300> From: Greg Ewing <greg.ewing at canterbury.ac.nz>> Subject: Re: [Python-Dev] Fwd: Removal of GIL through refcounting> removal.> To: python-dev at python.org> Message-ID: <490E2AA7.4070206 at canterbury.ac.nz>> Content-Type: text/plain; charset=UTF-8; format=flowed> > Eric Smith wrote:> > > I'd gladly trade deterministic destruction (due to reference counting or > > any other mechanism) for improved performance.> > Another thing to consider is that refcounting spreads out the> time spent doing GC evenly over the execution of the program,> so that you don't get pauses occurring at random times.> > Sometimes in games I've found that I had to disable cyclic> GC in order to get smooth frame rates. With no refcounting> I wouldn't have the option of doing that. I'd be disappointed> if that meant I could no longer use Python for these kinds of> games.> > -- > Greg> > > ------------------------------> > Message: 4> Date: Sun, 2 Nov 2008 17:51:50 -0600> From: skip at pobox.com> Subject: Re: [Python-Dev] Fwd: Removal of GIL through refcounting> removal.> To: Antoine Pitrou <solipsis at pitrou.net>> Cc: python-dev at python.org> Message-ID: <18702.15638.223872.706216 at montanaro-dyndns-org.local>> Content-Type: text/plain; charset=us-ascii> > > Antoine> I think it is important to remind that the GIL doesn't prevent> Antoine> (almost) true multithreading. The only thing it prevents is> Antoine> full use of multi-CPU resources in a single process. > > I believe everyone here knows that. I believe what most people are> clamoring for is to make "full use of their multi-CPU resources in a single> process".> > Skip> > > ------------------------------> > Message: 5> Date: Sun, 2 Nov 2008 17:55:03 -0600> From: skip at pobox.com> Subject: Re: [Python-Dev] Fwd: Removal of GIL through refcounting> removal.> To: Eric Smith <eric at trueblade.com>> Cc: python-dev at python.org> Message-ID: <18702.15831.164083.97731 at montanaro-dyndns-org.local>> Content-Type: text/plain; charset=us-ascii> > > Eric> I consider it a bug to rely on reference counting to close files,> > We can mostly have our cake and eat it too using the "with" statement. In> most cases it should be sufficient I would think.> > Skip> > > > ------------------------------> > Message: 6> Date: Sun, 2 Nov 2008 16:05:56 -0800> From: "Brett Cannon" <brett at python.org>> Subject: [Python-Dev] Looking for VCS usage scenarios> To: "Python dev" <python-dev at python.org>> Message-ID:> <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2 at mail.gmail.com>> Content-Type: text/plain; charset=UTF-8> > I have started the DVCS PEP which can be seen at> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there> beyond the rationale, usage scenarios I plan to use, and what other> sections I plan to write.> > At this point I am looking for any suggestions for fundamental usage> scenarios that I am missing from the PEP. If you think the few already> listed are missing some core part of a VCS, please let me know.> > And just to stave off some emails, I have two comments to make. One,> please do not start sending me info on how to fill in the usage> scenarios. I want to first solidify what the scenarios are. Plus I> want to figure them out on my own to get a feel of the documentation> for the tools.> > Second, as of right now Git is not in the running. Ignoring the fact I> dislike the tool (my issues with it are documented in the python-dev> archives), there is also the fact that it would be nicer to have> Python have as its VCS something written in Python instead of C/Perl.> > -Brett> > > ------------------------------> > Message: 7> Date: Sun, 02 Nov 2008 19:45:15 -0500> From: Eric Smith <eric at trueblade.com>> Subject: Re: [Python-Dev] Fwd: Removal of GIL through refcounting> removal.> To: skip at pobox.com> Cc: python-dev at python.org> Message-ID: <490E499B.9000703 at trueblade.com>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed> > skip at pobox.com wrote:> > Eric> I consider it a bug to rely on reference counting to close files,> > > > We can mostly have our cake and eat it too using the "with" statement. In> > most cases it should be sufficient I would think.> > True, and I meant to mention that. But unfortunately, my work projects > are stuck on 2.4 for the time being, so that doesn't help me much.> > Eric.> > > ------------------------------> > Message: 8> Date: Sun, 2 Nov 2008 23:08:11 -0200> From: "Gustavo Niemeyer" <gustavo at niemeyer.net>> Subject: Re: [Python-Dev] Looking for VCS usage scenarios> To: "Brett Cannon" <brett at python.org>> Cc: Python dev <python-dev at python.org>> Message-ID:> <643d90130811021708r5d39cc4bvf65fedea82395cc at mail.gmail.com>> Content-Type: text/plain; charset=ISO-8859-1> > Hi Brett,> > > At this point I am looking for any suggestions for fundamental usage> > scenarios that I am missing from the PEP. If you think the few already> > listed are missing some core part of a VCS, please let me know.> > As an initial disclaimer, I use bzr in my daily routine. That said,> I'm sending below a few mostly unbiased high-level ideas, just based> on useful ways we explore the DVCS-aspect on our normal daily> workflow.> > == Coordinated development with dependent features ==> > A variation on coordinated development, which is actually one of the> main motivators for DVCS. Branch A is evolving out of the mainline,> and it depends on a feature that is on branch B which is also not yet> integrated. Parallel development of both should be nicely supported.> I'm sure all DVCS will do that in a decent form, but figuring how this> works may be instructive and worth mentioning.> > == Centralization of feature-specific development ==> > That's a curious one when we're talking about distributed development,> isn't it? :-) Think about the following scenario: 5 people working> in parallel on a tricky feature which will take a while to get to the> mainline. Each developer works on a specific aspect of the feature> for a few hours/days in their own branch, and then merges to and from> what's considered as the feature-mainline. In essence, the problem is> that it's messy to just go towards the "everyone merges from everyone"> route when working in a common problem. We've found that being able> to use an svn-like approach for the *feature* mainline (a branch that> is shared by all defining the status quo) is a handy way to handle> that. I'm sure there are other approaches to solve the same problem,> but it's interesting to consider what they are for each tool since in> our experience the problem will show up eventually, and supporting it> nicely makes a big difference on the outcome.> > == Attaching of history-carrying diffs ==> > This one may feel weird as well, and I'm guessing some people might> react as "just send a URL to a branch". In practice, it is handy many> times to just attach something to the bug tracker, for instance. This> means that the "branch" is kept in the history of the bug. Also, with> something like this, someone external to the development process may> very easily provide his changes in the bug or in a mail without having> to set up any infrastructure/accounts/whatever at all to provide his> branch.> > > That's it for now. If I can think of any other use cases from our> routine that might serve as things to explore in such a comparison,> I'll let you know.> > -- > Gustavo Niemeyer> http://niemeyer.net> > > ------------------------------> > Message: 9> Date: Sun, 2 Nov 2008 19:10:45 -0700> From: "Alex Martelli" <aleaxit at gmail.com>> Subject: Re: [Python-Dev] My patches> To: "Barry Warsaw" <barry at python.org>> Cc: amk at amk.ca, python-dev at python.org> Message-ID:> <e8a0972d0811021810v5ebe0dbbl7ea7db5c4b097610 at mail.gmail.com>> Content-Type: text/plain; charset=ISO-8859-1> > On Sun, Nov 2, 2008 at 5:42 AM, Barry Warsaw <barry at python.org> wrote:> ...> > A dvcs means that people can publish their branches in a wide variety of> > ways. Trusted developers can push their branches to code.python.org.> > Non-core developers can use one of the free public dvcs branch hosting> > service. Industrious users can self-publish their branches. Anyone with> > BTW, if we go with hg, I recommend bitbucket.org as a "free public> (hg) hosting service" of choice -- Jesper (the owner and developer of> the site) is friendly and solicitous in his maintenance, the whole> site is quite OS-friendly in general (free "professional-level"> accounts and support for open-source projects which choose to host> there) and Python-friendly in particular (I gather the site's coded in> Python), and my experience there has been nothing short of excellent.> All it's missing is a simple code review tool like code.google.com's> or Rietveld, but Jesper's promised me he would integrate something to> that effect...> > > Alex> > > ------------------------------> > _______________________________________________> Python-Dev mailing list> Python-Dev at python.org> http://mail.python.org/mailman/listinfo/python-dev> > > End of Python-Dev Digest, Vol 64, Issue 4> *****************************************
_________________________________________________________________
Connect to the next generation of MSN Messenger?
http://imagine-msn.com/messenger/launch80/default.aspx?locale=en-us&source=wlmailtagline
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/f4162b42/attachment-0001.htm>

From musiccomposition at gmail.com  Mon Nov  3 05:01:01 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sun, 2 Nov 2008 22:01:01 -0600
Subject: [Python-Dev]
	=?utf-8?q?How_to_select_text_of_text_field_in_python?=
	=?utf-8?b?4oCPQ2FyZA==?=
In-Reply-To: <BLU114-W454EF34F15E7095875A9E811D0@phx.gbl>
References: <mailman.21986.1225678248.3486.python-dev@python.org>
	<BLU114-W454EF34F15E7095875A9E811D0@phx.gbl>
Message-ID: <1afaf6160811022001k483b927do411aeb6d8f960d14@mail.gmail.com>

2008/11/2 Sonia <sonia_16 at live.com>:
>
> Hi,
>
> I need to select a text of text field! Which function is used for selection
> of text of text field?

Please ask your question on comp.lang.python or the Python card
mailing list. This mailing list is for the development of core Python,
not developing with Python.



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From adriangolding at gmail.com  Mon Nov  3 08:04:46 2008
From: adriangolding at gmail.com (adrian golding)
Date: Mon, 3 Nov 2008 15:04:46 +0800
Subject: [Python-Dev] file open in python interpreter
Message-ID: <d3dcda8c0811022304oecee3bs84227aa7308c381e@mail.gmail.com>

hi all, i am trying to find out where is the part of the code in the python
interpreter that opens up the .py file and parses it. in particular, i am
trying to find the file open command in that file.  I greped and i just want
to make sure this is it: /Python-2.6/Parser/pgenmain.c
i am intending to take a hash measurement of the .py file just before i open
it to run the script.  is the above file the right place to call for the
measurement before the file open function?

thank you - adrian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/82aaacc2/attachment.htm>

From schmir at gmail.com  Mon Nov  3 08:46:27 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Mon, 3 Nov 2008 08:46:27 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
Message-ID: <932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>

On Mon, Nov 3, 2008 at 1:05 AM, Brett Cannon <brett at python.org> wrote:
> I have started the DVCS PEP which can be seen at
> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
> beyond the rationale, usage scenarios I plan to use, and what other
> sections I plan to write.
>

I think you really should not exclude any dvcs based on it's
implementation language.
I.e. requiring it being written in python for the sake of "eating your
own dogfood" is just a very weak argument. git is certainly missing
from your list.

Regards,
- Ralf

From rhamph at gmail.com  Mon Nov  3 09:18:13 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 3 Nov 2008 02:18:13 -0600
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <490E2AA7.4070206@canterbury.ac.nz>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<693bc9ab0811011312i6331c790j387ac23da670cdb2@mail.gmail.com>
	<490CF286.6090704@gmail.com> <gek9qn$1qi$1@ger.gmane.org>
	<490E0EC6.1070404@trueblade.com> <490E2AA7.4070206@canterbury.ac.nz>
Message-ID: <aac2c7cb0811030018i19d3d774k3878c2bdaa499a63@mail.gmail.com>

On Sun, Nov 2, 2008 at 4:33 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Eric Smith wrote:
>
>> I'd gladly trade deterministic destruction (due to reference counting or
>> any other mechanism) for improved performance.
>
> Another thing to consider is that refcounting spreads out the
> time spent doing GC evenly over the execution of the program,
> so that you don't get pauses occurring at random times.
>
> Sometimes in games I've found that I had to disable cyclic
> GC in order to get smooth frame rates. With no refcounting
> I wouldn't have the option of doing that. I'd be disappointed
> if that meant I could no longer use Python for these kinds of
> games.

I consider realtime games to be an essential use case and will be
pursuing incremental GC with my experiments.


-- 
Adam Olsen, aka Rhamphoryncus

From lgautier at gmail.com  Mon Nov  3 11:22:01 2008
From: lgautier at gmail.com (laurent)
Date: Mon, 03 Nov 2008 11:22:01 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
Message-ID: <1225707721.12592.35.camel@hot-spring>



On Mon, 2008-11-03 at 08:46 +0100, Ralf Schmitt wrote:
> On Mon, Nov 3, 2008 at 1:05 AM, Brett Cannon <brett at python.org> wrote:
> > I have started the DVCS PEP which can be seen at
> > http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
> > beyond the rationale, usage scenarios I plan to use, and what other
> > sections I plan to write.
> >
> 
> I think you really should not exclude any dvcs based on it's
> implementation language.
> I.e. requiring it being written in python for the sake of "eating your
> own dogfood" is just a very weak argument.
>  git is certainly missing from your list.

It does certainly miss from the list, but the argument might be more
"favor your users"; call it nepotism if you like.

If I understand it clearly, it is not a requirement, but more a good
point.

If it does "taste like dog food" in places, then it might be easier to:
- make informed statements about what might be the reason (on an
implementation standpoint - regarding features, that's an other story)
- have the cook more receptive to comments



L.



> Regards,
> - Ralf
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/lgautier%40gmail.com


From barry at python.org  Mon Nov  3 12:20:56 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 06:20:56 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
Message-ID: <0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 2:46 AM, Ralf Schmitt wrote:

> On Mon, Nov 3, 2008 at 1:05 AM, Brett Cannon <brett at python.org> wrote:
>> I have started the DVCS PEP which can be seen at
>> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
>> beyond the rationale, usage scenarios I plan to use, and what other
>> sections I plan to write.
>>
>
> I think you really should not exclude any dvcs based on it's
> implementation language.
> I.e. requiring it being written in python for the sake of "eating your
> own dogfood" is just a very weak argument. git is certainly missing
> from your list.

Sticking with a dvcs implemented in Python makes the best sense,  
especially when you consider the plugin architecture.  When we  
selected a new tracker, we didn't make implementation in Python a  
requirement, but instead a high hurdle.  Meaning, if a tracker wasn't  
written in Python it had to be way better than those written in Python.

As for dvcs, I think git would have to show overwhelming advantage  
over bzr or hg to be considered.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ7emXEjvBPtnXfVAQLbogP/RKxjGVs1STW8ghF/99JeRv8fVhwrHQxw
d2WyeuOC4wfU1iicEsbjCgTIKQOKmlnzZ1EO/D9DYc565Vv/+pVGJjSMMCmxS0/V
w2MXhGUJp9RKjCZEbUKc6aVszvUIAmQNp1GGWTeEZqzaRm/srSJH3XMsyn4/xMjO
qdsl1sQgAj0=
=ejCl
-----END PGP SIGNATURE-----

From gh at ghaering.de  Mon Nov  3 12:29:36 2008
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Mon, 03 Nov 2008 12:29:36 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <490E0747.8050401@behnel.de>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>
	<490E0747.8050401@behnel.de>
Message-ID: <490EE0A0.2060700@ghaering.de>

Stefan Behnel wrote:
> Michael Foord wrote:
>> Moving more C extensions to an implementation based on ctypes would be
>> enormously useful for PyPy, IronPython and Jython, but ctypes is not yet
>> as portable as Python itself which could be an issue (although one worth
>> resolving).
> 
> In the same line, moving more extensions to a high-level language like Cython,
> instead of writing them in straight C, would make a later switch to a
> different environment a lot easier, as the extensions could be regenerated
> with a modified Cython compiler (obviously minus some fixing of premature
> optimisations and the like).

Is using Cython for anything in Modules/ really an option? In my limited 
experiments with it, I did like it.

But using it for Python standard library stuff doesn't look quite right 
to me:

- Option 1: distribute Cython with Python and integrate into build process
-- Ouch!

- Option 2: only distribute generated source files
-- developers still need to have Cython installed
-- you have to trust Cython; who will really review the generated code?

-- Gerhard

From musiccomposition at gmail.com  Mon Nov  3 14:25:20 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 3 Nov 2008 07:25:20 -0600
Subject: [Python-Dev] file open in python interpreter
In-Reply-To: <d3dcda8c0811022304oecee3bs84227aa7308c381e@mail.gmail.com>
References: <d3dcda8c0811022304oecee3bs84227aa7308c381e@mail.gmail.com>
Message-ID: <1afaf6160811030525t3cca1e7aha7def92b400bdfe0@mail.gmail.com>

On Mon, Nov 3, 2008 at 1:04 AM, adrian golding <adriangolding at gmail.com> wrote:
> hi all, i am trying to find out where is the part of the code in the python
> interpreter that opens up the .py file and parses it. in particular, i am
> trying to find the file open command in that file.  I greped and i just want
> to make sure this is it: /Python-2.6/Parser/pgenmain.c
> i am intending to take a hash measurement of the .py file just before i open
> it to run the script.  is the above file the right place to call for the
> measurement before the file open function?

You want Parser/tokenizer.c.



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From musiccomposition at gmail.com  Mon Nov  3 14:29:34 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 3 Nov 2008 07:29:34 -0600
Subject: [Python-Dev] file open in python interpreter
In-Reply-To: <1afaf6160811030525t3cca1e7aha7def92b400bdfe0@mail.gmail.com>
References: <d3dcda8c0811022304oecee3bs84227aa7308c381e@mail.gmail.com>
	<1afaf6160811030525t3cca1e7aha7def92b400bdfe0@mail.gmail.com>
Message-ID: <1afaf6160811030529i76cc9893l1a5ebcde264a63f9@mail.gmail.com>

On Mon, Nov 3, 2008 at 7:25 AM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Mon, Nov 3, 2008 at 1:04 AM, adrian golding <adriangolding at gmail.com> wrote:
>> hi all, i am trying to find out where is the part of the code in the python
>> interpreter that opens up the .py file and parses it. in particular, i am
>> trying to find the file open command in that file.  I greped and i just want
>> to make sure this is it: /Python-2.6/Parser/pgenmain.c
>> i am intending to take a hash measurement of the .py file just before i open
>> it to run the script.  is the above file the right place to call for the
>> measurement before the file open function?
>
> You want Parser/tokenizer.c.

Sorry, that's not correct. opening of modules happens in
Python/import.c. There's also a case in Modules/main.c.

>
>
>
> --
> Cheers,
> Benjamin Peterson
> "There's nothing quite as beautiful as an oboe... except a chicken
> stuck in a vacuum cleaner."
>



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From ironfroggy at gmail.com  Mon Nov  3 14:46:35 2008
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Mon, 3 Nov 2008 08:46:35 -0500
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <490EE0A0.2060700@ghaering.de>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gecpks$cgc$1@ger.gmane.org> <gecqd4$g9g$1@ger.gmane.org>
	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>
	<4909EF67.5010104@trueblade.com>
	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk> <490E0747.8050401@behnel.de>
	<490EE0A0.2060700@ghaering.de>
Message-ID: <76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>

On Mon, Nov 3, 2008 at 6:29 AM, Gerhard H?ring <gh at ghaering.de> wrote:
> Stefan Behnel wrote:
>>
>> Michael Foord wrote:
>>>
>>> Moving more C extensions to an implementation based on ctypes would be
>>> enormously useful for PyPy, IronPython and Jython, but ctypes is not yet
>>> as portable as Python itself which could be an issue (although one worth
>>> resolving).
>>
>> In the same line, moving more extensions to a high-level language like
>> Cython,
>> instead of writing them in straight C, would make a later switch to a
>> different environment a lot easier, as the extensions could be regenerated
>> with a modified Cython compiler (obviously minus some fixing of premature
>> optimisations and the like).
>
> Is using Cython for anything in Modules/ really an option? In my limited
> experiments with it, I did like it.
>
> But using it for Python standard library stuff doesn't look quite right to
> me:
>
> - Option 1: distribute Cython with Python and integrate into build process
> -- Ouch!

Can you be a bit more descriptive?

> - Option 2: only distribute generated source files
> -- developers still need to have Cython installed
> -- you have to trust Cython; who will really review the generated code?

Who reviews the machine code from gcc?

I would love to see the option to write the lower levels in something
other than C, but obviously any choice would have to be a good one.
Otherwise, we end up stuck or with lots of different languages all
being used and making understanding the full codebase harder. For
example, I've wondered if RPython would ever reach the point it could
be considered in the same way, but I don't think it would be wise to
consider both. So, the question I see isn't if Cython should be
allowed for standard library modules, but if the landscape of such
solutions is at a point that any of them is ready to be committed to.

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

From ncoghlan at gmail.com  Mon Nov  3 14:57:07 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 03 Nov 2008 23:57:07 +1000
Subject: [Python-Dev] file open in python interpreter
In-Reply-To: <1afaf6160811030529i76cc9893l1a5ebcde264a63f9@mail.gmail.com>
References: <d3dcda8c0811022304oecee3bs84227aa7308c381e@mail.gmail.com>	<1afaf6160811030525t3cca1e7aha7def92b400bdfe0@mail.gmail.com>
	<1afaf6160811030529i76cc9893l1a5ebcde264a63f9@mail.gmail.com>
Message-ID: <490F0333.5010803@gmail.com>

Benjamin Peterson wrote:
> On Mon, Nov 3, 2008 at 7:25 AM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Mon, Nov 3, 2008 at 1:04 AM, adrian golding <adriangolding at gmail.com> wrote:
>>> hi all, i am trying to find out where is the part of the code in the python
>>> interpreter that opens up the .py file and parses it. in particular, i am
>>> trying to find the file open command in that file.  I greped and i just want
>>> to make sure this is it: /Python-2.6/Parser/pgenmain.c
>>> i am intending to take a hash measurement of the .py file just before i open
>>> it to run the script.  is the above file the right place to call for the
>>> measurement before the file open function?
>> You want Parser/tokenizer.c.
> 
> Sorry, that's not correct. opening of modules happens in
> Python/import.c. There's also a case in Modules/main.c.

And some indirect ones from runpy.py (via pkgutils) if you use the -m
switch, or are executing a zipfile or directory.

But for the specific case of an exact filename being provided on the
command line, then main.c is the one the original poster will want to
look at (line 567 to be exact).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From gh at ghaering.de  Mon Nov  3 15:15:32 2008
From: gh at ghaering.de (=?UTF-8?B?R2VyaGFyZCBIw6RyaW5n?=)
Date: Mon, 03 Nov 2008 15:15:32 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>
	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>
	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
Message-ID: <490F0784.4050408@ghaering.de>

Calvin Spealman wrote:
> On Mon, Nov 3, 2008 at 6:29 AM, Gerhard H?ring <gh at ghaering.de> wrote:
>> Stefan Behnel wrote:
>>> Michael Foord wrote:
>>>> Moving more C extensions to an implementation based on ctypes would be
>>>> enormously useful for PyPy, IronPython and Jython, but ctypes is not yet
>>>> as portable as Python itself which could be an issue (although one worth
>>>> resolving).
>>> In the same line, moving more extensions to a high-level language like
>>> Cython,
>>> instead of writing them in straight C, would make a later switch to a
>>> different environment a lot easier, as the extensions could be regenerated
>>> with a modified Cython compiler (obviously minus some fixing of premature
>>> optimisations and the like).
>> Is using Cython for anything in Modules/ really an option? In my limited
>> experiments with it, I did like it.
>>
>> But using it for Python standard library stuff doesn't look quite right to
>> me:
>>
>> - Option 1: distribute Cython with Python and integrate into build process
>> -- Ouch!
> 
> Can you be a bit more descriptive?

Cython is still being worked on (intensively, it seems). Bundling it 
with Python means deciding on a particular version probably for an 
entire major release lifecycle (use Cython x.y.{newest} for Python 2.7, 
for example).

>> - Option 2: only distribute generated source files
>> -- developers still need to have Cython installed
>> -- you have to trust Cython; who will really review the generated code?
> 
> Who reviews the machine code from gcc?

That's comparing apples and eggs :-P But it may be that I'm a little 
paranoid here.

> I would love to see the option to write the lower levels in something
> other than C, 

Absolutely. That's why I tried to reimplement pysqlite in something 
easier to maintain than handwritten Python C API. There's a ctypes-based 
version in its Mercurial repository that's good enough to be used from 
PyPy now. And a started Cython-based one.

> but obviously any choice would have to be a good one.
> [...]  So, the question I see isn't if Cython should be
> allowed for standard library modules, but if the landscape of such
> solutions is at a point that any of them is ready to be committed to.

ACK.

-- Gerhard

From skip at pobox.com  Mon Nov  3 15:34:07 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 3 Nov 2008 08:34:07 -0600
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <490F0784.4050408@ghaering.de>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gecpks$cgc$1@ger.gmane.org> <gecqd4$g9g$1@ger.gmane.org>
	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>
	<4909EF67.5010104@trueblade.com>
	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk> <490E0747.8050401@behnel.de>
	<490EE0A0.2060700@ghaering.de>
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
	<490F0784.4050408@ghaering.de>
Message-ID: <18703.3039.835260.398411@montanaro-dyndns-org.local>

    >>> - Option 2: only distribute generated source files
    >>> -- developers still need to have Cython installed
    >>> -- you have to trust Cython; who will really review the generated code?
    >> 
    >> Who reviews the machine code from gcc?

    Gerhard> That's comparing apples and eggs :-P But it may be that I'm a
    Gerhard> little paranoid here.

Agreed.  When Cython is as widely used as gcc and has as comprehensive a set
of test cases (who knows? it may well already be comprehensive enough) and
supports all the compilers which Python supports then we can probably skip
the output code review step.

Skip

From bill.janssen at gmail.com  Mon Nov  3 17:21:20 2008
From: bill.janssen at gmail.com (Bill Janssen)
Date: Mon, 3 Nov 2008 08:21:20 -0800
Subject: [Python-Dev] Packaging the PyPI version of the SSL module for Debian
Message-ID: <4b3e516a0811030821nae22517ja74c1ee75a37e349@mail.gmail.com>

As some of you know, I've provided a PyPI version of the 2.6/3.x "ssl"
module, for use with older versions of Python.  I've received this request
to tweak it for Debian, and I thought I'd ask those of you who may have
already done it for advice on the various issues Cyril raises here.  I'm not
Debian-knowledgable, myself.

Bill

---------- Forwarded message ----------
From: Cyril Brulebois <kibi at debian.org>
Date: Mon, Jul 14, 2008 at 11:37 AM
Subject: Packaging (python-)ssl for Debian
To: python.ssl.maintainer at gmail.com


Hi,

I've noticed the following Request For Packaging [1], and it turned out
that it was exactly what I was looking for (a real ssl module, not just
read/write on a socket). So I started packaging it for Debian, and here
are a few comments, and some points that I'd like you to address, so
that I can upload it into the archive.

 1. http://bugs.debian.org/453101


The main concern currently is the license, since "Python (MIT-like)" is
quite vague. One has no clue which Python version is concerned, since as
far as I can tell, there are at least 2.4.2 and 2.5.2 [2,3] out with
(slightly but still) different licenses. So, pointing to the precise
version would really help people. At least the URL to the license would
be needed, but it would be even better to include it into the source
package, so that there's no possible doubt, and so that one isn't
dependent from being online to actually read it.

 2. http://www.python.org/download/releases/2.4.2/license/
 3. http://www.python.org/download/releases/2.5.2/license/

I'd suggest including it in a LICENSE file, and then point to it using
something like "LICENSE: Python x.y.z (MIT-like), see LICENSE file".

It's then up to you to choose whether to state clearly in each file
"This file is licensed under the terms of the Python x.y.z license, see
LICENSE file." (which I encourage you to do), or to state at the very
beginning of this LICENSE file that "the whole (python-)ssl package is
licensed under the following terms (Python x.y.z license):" and then
quote your favourite license.


Also, you're listing the contributors in various places, PKG-INFO and
setup.py. That's nice, but it might be a better idea to point to a
single AUTHORS file, where you would keep a single list of the
contributors. It's not a problem per se, but you may want to consider
this.

Related to authorship is copyright assignment, and that one is a blocker
for the inclusion into Debian. Every file should include copyright
information so that one can know who is holding the copyrights on this
or that file. Having a very quick look, the following files would need
it: *.py, *.h, *.c. PKG-INFO could eventually list all copyright lines,
but I'm not sure it's a widely-used practice in the Python world, just
tell me ;-); the Makefile might deserve (c) info as well; MANIFEST and
*.pem files don't need it.

Just for the records, one specifies copyrights in the following manner:
 Copyright 2008 Some Author
 Copyright 2004, 2007 Another One
 Copyright 2004-2008 Yet Another One

"Copr." or "(c)" can be used instead of "Copyright". "(C)" can be added
(it's harmless) but has no legal meaning, so can't be used alone. No
need to list everyone having ever contributed a single patch, it's
sufficient to list people having done the real work and worthy
modifications to it. Adding people to copyright holders is usally a
matter of project policy.

If you don't feel like listing the copyright holders in PKG-INFO, you
could decide for using something like:
,---[ AUTHORS ]---
|
| Main developers:
|   Copyright 2004 John Doe
|   Copyright 2004-2008 Alice Wonderland
|
| Contributors:
|   Peter Pan
|   Foo Bar
|
`---

This way, you just have to do the following to keep your software
uptodate WRT legalese things:
 - Add/update a Copyright line at the top of .c/.h/? file when a
  significant modification is included.
 - Add/update the Copyright line in the AUTHORS file accordingly.
 - Or add/update the Contributors list if that doesn't warrant a
  Copyright line.

So that there's no need to update every file (like PKG-INFO and
setup.py), you only have to make them point to AUTHORS for authorship
information.


Once that is done, I should be able to upload it to Debian, where the
first upload will be double-check by ftpmasters, whose role is to make
sure that no obvious packaging error is made, but mainly to make sure
that the archive is kept OK from a legal point of view. That's why I'm
starting with this quite boring mail, and I'm sorry for that. :)


Mraw,
KiBi.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)

iEYEARECAAYFAkh7quoACgkQeGfVPHR5Nd1LQwCaA????????
2Y0AnitpaRf3zybMhNxstnDrKaJc8K3/
=eriZ
-----END PGP SIGNATURE-----
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/9533fe1f/attachment.htm>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: signature.asc
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/9533fe1f/attachment.asc>

From thomas at python.org  Mon Nov  3 17:54:34 2008
From: thomas at python.org (Thomas Wouters)
Date: Mon, 3 Nov 2008 17:54:34 +0100
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <20081031102524.75dfd13f@resist.wooz.org>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<20081031102524.75dfd13f@resist.wooz.org>
Message-ID: <9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>

On Fri, Oct 31, 2008 at 11:25, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Oct 31, 2008, at 03:50 AM, Eric Smith wrote:
>
> >I posted this yesterday about using bzr:
> >
> > >I'd like to try it out, but the instructions on
> > > http://www.python.org/dev/bazaar/ say to get wget
> > > http://code.python.org/snapshots/python-bzr-snapshot.tar.bz2, which is
> > > a 404. Anyone know the actual URL?
>
> Looks like this is fixed now too.
>
> Aside: Now that Python 2.6 isn't on the trunk, we need to add a bzr mirror
> for
> python26-maint.


FWIW, I put one up this weekend, and it seems to be intact and OK.
(bzr+ssh://pythonbzr at code.python.org/python/2.6/ or
http://code.python.org/python/2.6/ )

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/0cd766fb/attachment.htm>

From martin at v.loewis.de  Mon Nov  3 18:47:28 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Mon, 03 Nov 2008 18:47:28 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>
	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>
	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
Message-ID: <490F3930.6050904@v.loewis.de>

>> - Option 1: distribute Cython with Python and integrate into build process
>> -- Ouch!
> 
> Can you be a bit more descriptive?

Gerhard's elaboration (of us creating a fork of Cython then) is
convincing; there is also the issue of changes to the API to consider.
When we change the API now, we have to find the 20..50 places that
need to get adjusted. If we change the API then, we need to change
the generator - which complicates the evolution. For a Cython outside
of Python, we might not be able to change the API at all.

In addition, there is also the bootstrapping problem. Cython imports
a number of builtin modules - if they are written in Cython, we have
the bootstrapping problem.

Regards,
Martin

From ctb at msu.edu  Mon Nov  3 18:49:01 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Mon, 3 Nov 2008 09:49:01 -0800
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
References: <392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gecpks$cgc$1@ger.gmane.org> <gecqd4$g9g$1@ger.gmane.org>
	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>
	<4909EF67.5010104@trueblade.com>
	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk>
	<490E0747.8050401@behnel.de> <490EE0A0.2060700@ghaering.de>
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
Message-ID: <20081103174901.GI32006@idyll.org>

-> I would love to see the option to write the lower levels in something
-> other than C, but obviously any choice would have to be a good one.
-> Otherwise, we end up stuck or with lots of different languages all
-> being used and making understanding the full codebase harder. For
-> example, I've wondered if RPython would ever reach the point it could
-> be considered in the same way, but I don't think it would be wise to
-> consider both. So, the question I see isn't if Cython should be
-> allowed for standard library modules, but if the landscape of such
-> solutions is at a point that any of them is ready to be committed to.

What is the situation twixt Pyrex and Cython?  As I understand it,
Cython is a non-backwards-compatible fork of Pyrex, forked for the usual
reasons [0].  Have many or most people switched to Cython, or is there
still a respectable community using Pyrex, or ...?

I'm involved in a project that depends on Pyrex and there was no clear
reason for us to switch to Cython.  I've also seen criticisms of
Cython's maturity level (which presumably also apply to Pyrex).  I'd be
interested in hearing about that...

...or is switching to Cython/Pyrex/foo a non-starter?

cheers,
--titus

[0] Which is to say: a variety of reasons, many of which are obviously
arguable, otherwise the Pyrex maintainer would have quit maintaining
Pyrex :).  But let's not go into them!
-- 
C. Titus Brown, ctb at msu.edu

From brett at python.org  Mon Nov  3 18:57:26 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 09:57:26 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
Message-ID: <bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>

On Sun, Nov 2, 2008 at 17:08, Gustavo Niemeyer <gustavo at niemeyer.net> wrote:
> Hi Brett,
>
>> At this point I am looking for any suggestions for fundamental usage
>> scenarios that I am missing from the PEP. If you think the few already
>> listed are missing some core part of a VCS, please let me know.
>
> As an initial disclaimer, I use bzr in my daily routine.  That said,
> I'm sending below a few mostly unbiased high-level ideas, just based
> on useful ways we explore the DVCS-aspect on our normal daily
> workflow.
>
> == Coordinated development with dependent features ==
>
> A variation on coordinated development, which is actually one of the
> main motivators for DVCS.  Branch A is evolving out of the mainline,
> and it depends on a feature that is on branch B which is also not yet
> integrated.  Parallel development of both should be nicely supported.
> I'm sure all DVCS will do that in a decent form, but figuring how this
> works may be instructive and worth mentioning.
>

Since I have never seen that come up during Python's development I am
going to leave it out. But I do see the benefit and how it might help
with future work.

> == Centralization of feature-specific development ==
>
> That's a curious one when we're talking about distributed development,
> isn't it? :-)   Think about the following scenario: 5 people working
> in parallel on a tricky feature which will take a while to get to the
> mainline.  Each developer works on a specific aspect of the feature
> for a few hours/days in their own branch, and then merges to and from
> what's considered as the feature-mainline.  In essence, the problem is
> that it's messy to just go towards the "everyone merges from everyone"
> route when working in a common problem.  We've found that being able
> to use an svn-like approach for the *feature* mainline (a branch that
> is shared by all defining the status quo) is a handy way to handle
> that.  I'm sure there are other approaches to solve the same problem,
> but it's interesting to consider what they are for each tool since in
> our experience the problem will show up eventually, and supporting it
> nicely makes a big difference on the outcome.
>

This one is covered already.

> == Attaching of history-carrying diffs ==
>
> This one may feel weird as well, and I'm guessing some people might
> react as "just send a URL to a branch".  In practice, it is handy many
> times to just attach something to the bug tracker, for instance.  This
> means that the "branch" is kept in the history of the bug.  Also, with
> something like this, someone external to the development process may
> very easily provide his changes in the bug or in a mail without having
> to set up any infrastructure/accounts/whatever at all to provide his
> branch.

As I mentioned in the doc, the first example allows for whatever the
DVCS does for diffs, whether that is a full-on branch or not.

>
>
> That's it for now.  If I can think of any other use cases from our
> routine that might serve as things to explore in such a comparison,
> I'll let you know.

Thanks, Gustavo!

-Brett

From ctb at msu.edu  Mon Nov  3 18:58:03 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Mon, 3 Nov 2008 09:58:03 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
Message-ID: <20081103175803.GL32006@idyll.org>

-> Sticking with a dvcs implemented in Python makes the best sense,  
-> especially when you consider the plugin architecture.  When we  
-> selected a new tracker, we didn't make implementation in Python a  
-> requirement, but instead a high hurdle.  Meaning, if a tracker wasn't  
-> written in Python it had to be way better than those written in Python.

I worry about the idea of hacking in any way, shape or form, on the
version control system used to maintain the Python source code.  I place
VCSes at the compiler- or OS-level of the toolchain, because you have
the option of fundamentally screwing up the entire project by playing
with them.

So from that perspective it's better to keep it *out* of Python to
remove the temptation to hack :)

-> As for dvcs, I think git would have to show overwhelming advantage  
-> over bzr or hg to be considered.

I personally have found git very, very powerful and good, albeit
difficult to learn.  It is guaranteed to scale (unless Python gets to be
significantly bigger and more active than Linux, at any rate) and it has
a large, very technically capable, and supported user community already.

I think there are reasons why git should be at least strongly
considered.

--titus
-- 
C. Titus Brown, ctb at msu.edu

From solipsis at pitrou.net  Mon Nov  3 18:57:10 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 3 Nov 2008 17:57:10 +0000 (UTC)
Subject: [Python-Dev] Looking for VCS usage scenarios
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
Message-ID: <loom.20081103T175509-746@post.gmane.org>


Hi Brett,

Brett Cannon <brett <at> python.org> writes:
> 
> I have started the DVCS PEP which can be seen at
> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
> beyond the rationale, usage scenarios I plan to use, and what other
> sections I plan to write.

I'm not sure that's the kind of feedback you want, but it isn't clear what you
mean with "checkout". Is it just a working copy (then it would be better to use
this term)? Is it some kind of shallow clone containing only part of the
history? Or anything else?

Regards

Antoine.



From brett at python.org  Mon Nov  3 18:59:40 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 09:59:40 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <1afaf6160811021903j3614225qadfbd15a58d54326@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<1afaf6160811021903j3614225qadfbd15a58d54326@mail.gmail.com>
Message-ID: <bbaeab100811030959m3797f05enec5f3810fe52b08a@mail.gmail.com>

On Sun, Nov 2, 2008 at 19:03, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Sun, Nov 2, 2008 at 6:05 PM, Brett Cannon <brett at python.org> wrote:
>> I have started the DVCS PEP which can be seen at
>> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
>> beyond the rationale, usage scenarios I plan to use, and what other
>> sections I plan to write.
>>
>> At this point I am looking for any suggestions for fundamental usage
>> scenarios that I am missing from the PEP. If you think the few already
>> listed are missing some core part of a VCS, please let me know.
>
> I think one very important requirement of Python's VCS is advanced
> merging support. The trunk and py3k are very special branches because
> a commit in one should not necessarily find its way into the other. To
> that end, I think it's critical to have svnmerge.py-like blocking
> (preferably with unblocking, too) and fine cherry-picking of
> revisions.
>

Add the blocking to the backport scenario.

-Brett

From brett at python.org  Mon Nov  3 19:05:15 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 10:05:15 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <20081103175803.GL32006@idyll.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
Message-ID: <bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>

On Mon, Nov 3, 2008 at 09:58, C. Titus Brown <ctb at msu.edu> wrote:
> -> Sticking with a dvcs implemented in Python makes the best sense,
> -> especially when you consider the plugin architecture.  When we
> -> selected a new tracker, we didn't make implementation in Python a
> -> requirement, but instead a high hurdle.  Meaning, if a tracker wasn't
> -> written in Python it had to be way better than those written in Python.
>
> I worry about the idea of hacking in any way, shape or form, on the
> version control system used to maintain the Python source code.  I place
> VCSes at the compiler- or OS-level of the toolchain, because you have
> the option of fundamentally screwing up the entire project by playing
> with them.
>
> So from that perspective it's better to keep it *out* of Python to
> remove the temptation to hack :)
>

I don't expect us to hack on the VCS itself. I am thinking more like
plug-ins commit hooks, etc.; the infrastructure around the VCS.

> -> As for dvcs, I think git would have to show overwhelming advantage
> -> over bzr or hg to be considered.
>
> I personally have found git very, very powerful and good, albeit
> difficult to learn

You can say that again. And that is a worry to me. Python gets patches
from people of all skill levels where ease of use for the VCS needs to
be considered. The Linux kernel probably doesn't get as many patches
from newbies as the barrier of entry is higher to contribute.

I have yet to have met anyone who thinks git is great while having
used another DVCS as extensively (and I mean I have never found
someone who has used two DVCSs extensively).

>.  It is guaranteed to scale (unless Python gets to be
> significantly bigger and more active than Linux, at any rate) and it has
> a large, very technically capable, and supported user community already.
>

I think any of the DVCSs will scale. But I will be taking some
performance numbers so scalability will be taken into consideration.

> I think there are reasons why git should be at least strongly
> considered.

Well, we will see, but as of right now my use of git has left a nasty
taste in my mouth that will take a lot of proverbial mouthwash to get
rid of and allow it to be considered in this PEP.

-Brett

From brett at python.org  Mon Nov  3 19:06:46 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 10:06:46 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <loom.20081103T175509-746@post.gmane.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<loom.20081103T175509-746@post.gmane.org>
Message-ID: <bbaeab100811031006k768743c5t4252d184e93b0579@mail.gmail.com>

On Mon, Nov 3, 2008 at 09:57, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> Hi Brett,
>
> Brett Cannon <brett <at> python.org> writes:
>>
>> I have started the DVCS PEP which can be seen at
>> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
>> beyond the rationale, usage scenarios I plan to use, and what other
>> sections I plan to write.
>
> I'm not sure that's the kind of feedback you want, but it isn't clear what you
> mean with "checkout". Is it just a working copy (then it would be better to use
> this term)? Is it some kind of shallow clone containing only part of the
> history? Or anything else?
>

A working copy. That's why I mentioned Subversion as the example in
the definition. I have updated it.

-Brett

From ctb at msu.edu  Mon Nov  3 19:08:17 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Mon, 3 Nov 2008 10:08:17 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
Message-ID: <20081103180816.GN32006@idyll.org>

On Mon, Nov 03, 2008 at 10:05:15AM -0800, Brett Cannon wrote:
-> I have yet to have met anyone who thinks git is great while having
-> used another DVCS as extensively (and I mean I have never found
-> someone who has used two DVCSs extensively).

git is great!  I'm switching to it from darcs for all my future
projects.

(darcs, of course, is kind of a low bar: it has some scalability issues,
and it is feature-poor relative to hg and bzr in patch cherry-picking,
esp.)

:)

--titus
-- 
C. Titus Brown, ctb at msu.edu

From schmir at gmail.com  Mon Nov  3 19:13:19 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Mon, 3 Nov 2008 19:13:19 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
Message-ID: <932f8baf0811031013l47299269r38b55d2dbc17e8a9@mail.gmail.com>

On Mon, Nov 3, 2008 at 7:05 PM, Brett Cannon <brett at python.org> wrote:
> I have yet to have met anyone who thinks git is great while having
> used another DVCS as extensively (and I mean I have never found
> someone who has used two DVCSs extensively).

I have used mercurial extensively (before having used git) and I think
git is great.
It gives you much more freedom to work with your source code than mercurial.

- Ralf

From solipsis at pitrou.net  Mon Nov  3 19:19:08 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 3 Nov 2008 18:19:08 +0000 (UTC)
Subject: [Python-Dev] Looking for VCS usage scenarios
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
Message-ID: <loom.20081103T181817-535@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> 
> At this point I am looking for any suggestions for fundamental usage
> scenarios that I am missing from the PEP. If you think the few already
> listed are missing some core part of a VCS, please let me know.

You might want to refine the "patch review" scenario with a variant using
Rietveld and its upload script.




From barry at python.org  Mon Nov  3 19:34:43 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 13:34:43 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <20081103175803.GL32006@idyll.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
Message-ID: <4BFBAC56-BEDE-4B0A-A0FD-0BB4681F3A11@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 12:58 PM, C. Titus Brown wrote:

> -> Sticking with a dvcs implemented in Python makes the best sense,
> -> especially when you consider the plugin architecture.  When we
> -> selected a new tracker, we didn't make implementation in Python a
> -> requirement, but instead a high hurdle.  Meaning, if a tracker  
> wasn't
> -> written in Python it had to be way better than those written in  
> Python.
>
> I worry about the idea of hacking in any way, shape or form, on the
> version control system used to maintain the Python source code.  I  
> place
> VCSes at the compiler- or OS-level of the toolchain, because you have
> the option of fundamentally screwing up the entire project by playing
> with them.
>
> So from that perspective it's better to keep it *out* of Python to
> remove the temptation to hack :)

:)  But actually more interesting is whether we want to add plugins  
that assist Python dev workflow.  For example, let's say we wanted to  
have a 'fixes' command that automatically updated the Roundup tracker  
with the branch information.  I'd personally much rather write 10  
lines of Python to add such a plugin than any amount of C or Perl, or  
whatever else. :)

> -> As for dvcs, I think git would have to show overwhelming advantage
> -> over bzr or hg to be considered.
>
> I personally have found git very, very powerful and good, albeit
> difficult to learn.  It is guaranteed to scale (unless Python gets  
> to be
> significantly bigger and more active than Linux, at any rate) and it  
> has
> a large, very technically capable, and supported user community  
> already.
>
> I think there are reasons why git should be at least strongly
> considered.

Powerful, scalable, active development and user community would  
certainly apply to the Python-based dvcses.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ9ERHEjvBPtnXfVAQLGEgP/VTjZLo4FJQ3oUGZp5eGHJdvkhOkmJPX+
bKhw09eoR6yuKbcRcPkqjqU9z8T4+gCdrOsiyNE98/Cr14xtAr2tYq2zEj0iFb/L
snjVaZuelrlzHV3V6avAs3n8ua+X/rb4tG3r7vc8djH4qeAFw3aMYqYDkodL3BoO
/x8NVlfPj7o=
=z3TI
-----END PGP SIGNATURE-----

From thomas at python.org  Mon Nov  3 19:35:06 2008
From: thomas at python.org (Thomas Wouters)
Date: Mon, 3 Nov 2008 19:35:06 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
Message-ID: <9e804ac0811031035v3b7a816bwe8b886da59207a6b@mail.gmail.com>

On Mon, Nov 3, 2008 at 18:57, Brett Cannon <brett at python.org> wrote:

> On Sun, Nov 2, 2008 at 17:08, Gustavo Niemeyer <gustavo at niemeyer.net>
> wrote:
> > Hi Brett,
> >
> >> At this point I am looking for any suggestions for fundamental usage
> >> scenarios that I am missing from the PEP. If you think the few already
> >> listed are missing some core part of a VCS, please let me know.
> >
> > As an initial disclaimer, I use bzr in my daily routine.  That said,
> > I'm sending below a few mostly unbiased high-level ideas, just based
> > on useful ways we explore the DVCS-aspect on our normal daily
> > workflow.
> >
> > == Coordinated development with dependent features ==
> >
> > A variation on coordinated development, which is actually one of the
> > main motivators for DVCS.  Branch A is evolving out of the mainline,
> > and it depends on a feature that is on branch B which is also not yet
> > integrated.  Parallel development of both should be nicely supported.
> > I'm sure all DVCS will do that in a decent form, but figuring how this
> > works may be instructive and worth mentioning.
> >
>
> Since I have never seen that come up during Python's development I am
> going to leave it out. But I do see the benefit and how it might help
> with future work.


Euhm, wut? It hasn't come up during Python's development because Python is
being developed in a VCS with very limited branches :) I'm partial to VCS's
with proper branching (as you know) and I've been using that mode of
development for many years. I've done development and maintenance of
multi-developer projects using both the CVS/SVN nonbranching approach and
the typical DVCS branch-often approach using BitKeeper, Bazaar and Mercurial
(as well as the sort-of one-off-branch Perforce approach where you can
easily 'wrap' a single change but can't really do dependant changes) -- and
whenever possible I use the branch-often approach with dependant branches
all over the place, especially when working on large, complicated changes.
Building them up out of separate 'components' requires a little more
administration (you have to remember which branch to submit to) but it makes
debugging, piecemeal discussion and batchwise integration a *lot* easier.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/ac9109a3/attachment.htm>

From barry at python.org  Mon Nov  3 19:36:12 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 13:36:12 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <932f8baf0811031013l47299269r38b55d2dbc17e8a9@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<932f8baf0811031013l47299269r38b55d2dbc17e8a9@mail.gmail.com>
Message-ID: <160F5200-0E9F-4144-9CEE-29ADED66D60B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 1:13 PM, Ralf Schmitt wrote:

> I have used mercurial extensively (before having used git) and I think
> git is great.
> It gives you much more freedom to work with your source code than  
> mercurial.

Ralf, can you describe what you mean in more detail?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ9EnHEjvBPtnXfVAQLAQgP9HFGmvq3dq60oUryeiVXI2sgWQvUXbYzk
6Nhg796sPyGwzuO8PoLq6CvxNzqvah25KMznjOxx0MpMzhSKEgJPJwxkBLpIYJUy
Enz3JrLt+r3do0pNLvgkAz9gENh90QLWt1amVvvt2c8ahua2hGpxPN4Y0YUFbeIf
bbxP7bcvD+U=
=aPaJ
-----END PGP SIGNATURE-----

From eduardo.padoan at gmail.com  Mon Nov  3 19:41:56 2008
From: eduardo.padoan at gmail.com (Eduardo O. Padoan)
Date: Mon, 3 Nov 2008 16:41:56 -0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4BFBAC56-BEDE-4B0A-A0FD-0BB4681F3A11@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<4BFBAC56-BEDE-4B0A-A0FD-0BB4681F3A11@python.org>
Message-ID: <dea92f560811031041t440af08fv113b5346d36de857@mail.gmail.com>

On Mon, Nov 3, 2008 at 4:34 PM, Barry Warsaw <barry at python.org> wrote:
> :)  But actually more interesting is whether we want to add plugins that
> assist Python dev workflow.  For example, let's say we wanted to have a
> 'fixes' command that automatically updated the Roundup tracker with the
> branch information.  I'd personally much rather write 10 lines of Python to
> add such a plugin than any amount of C or Perl, or whatever else. :)

(Which, btw, is already supported by the "--fixes" option on bzr's commit :)

just-giving-my-R$0.02-ly yours,

-- 
    Eduardo de Oliveira Padoan
http://djangopeople.net/edcrypt/
"Distrust those in whom the desire to punish is strong." -- Goethe,
Nietzsche, Dostoevsky

From musiccomposition at gmail.com  Mon Nov  3 19:43:39 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 3 Nov 2008 12:43:39 -0600
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <dea92f560811031041t440af08fv113b5346d36de857@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<4BFBAC56-BEDE-4B0A-A0FD-0BB4681F3A11@python.org>
	<dea92f560811031041t440af08fv113b5346d36de857@mail.gmail.com>
Message-ID: <1afaf6160811031043l49d04b31jbf7582ba15ad515e@mail.gmail.com>

On Mon, Nov 3, 2008 at 12:41 PM, Eduardo O. Padoan
<eduardo.padoan at gmail.com> wrote:
> On Mon, Nov 3, 2008 at 4:34 PM, Barry Warsaw <barry at python.org> wrote:
>> :)  But actually more interesting is whether we want to add plugins that
>> assist Python dev workflow.  For example, let's say we wanted to have a
>> 'fixes' command that automatically updated the Roundup tracker with the
>> branch information.  I'd personally much rather write 10 lines of Python to
>> add such a plugin than any amount of C or Perl, or whatever else. :)
>
> (Which, btw, is already supported by the "--fixes" option on bzr's commit :)

The --fixes option allows you to note where a bug is fixed in your
repo. I think what Barry is envisioning is a tool that would
automatically close the issue on Roundup.



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From jnoller at gmail.com  Mon Nov  3 19:45:43 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Mon, 3 Nov 2008 13:45:43 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
Message-ID: <4222a8490811031045t6eacb4a6qfd425b7224a033e4@mail.gmail.com>

On Mon, Nov 3, 2008 at 1:05 PM, Brett Cannon <brett at python.org> wrote:
> On Mon, Nov 3, 2008 at 09:58, C. Titus Brown <ctb at msu.edu> wrote:
>> -> Sticking with a dvcs implemented in Python makes the best sense,
>> -> especially when you consider the plugin architecture.  When we
>> -> selected a new tracker, we didn't make implementation in Python a
>> -> requirement, but instead a high hurdle.  Meaning, if a tracker wasn't
>> -> written in Python it had to be way better than those written in Python.
>>
>> I worry about the idea of hacking in any way, shape or form, on the
>> version control system used to maintain the Python source code.  I place
>> VCSes at the compiler- or OS-level of the toolchain, because you have
>> the option of fundamentally screwing up the entire project by playing
>> with them.
>>
>> So from that perspective it's better to keep it *out* of Python to
>> remove the temptation to hack :)
>>
>
> I don't expect us to hack on the VCS itself. I am thinking more like
> plug-ins commit hooks, etc.; the infrastructure around the VCS.
>
>> -> As for dvcs, I think git would have to show overwhelming advantage
>> -> over bzr or hg to be considered.
>>
>> I personally have found git very, very powerful and good, albeit
>> difficult to learn
>
> You can say that again. And that is a worry to me. Python gets patches
> from people of all skill levels where ease of use for the VCS needs to
> be considered. The Linux kernel probably doesn't get as many patches
> from newbies as the barrier of entry is higher to contribute.
>
> I have yet to have met anyone who thinks git is great while having
> used another DVCS as extensively (and I mean I have never found
> someone who has used two DVCSs extensively).
>
>>.  It is guaranteed to scale (unless Python gets to be
>> significantly bigger and more active than Linux, at any rate) and it has
>> a large, very technically capable, and supported user community already.
>>
>
> I think any of the DVCSs will scale. But I will be taking some
> performance numbers so scalability will be taken into consideration.
>
>> I think there are reasons why git should be at least strongly
>> considered.
>
> Well, we will see, but as of right now my use of git has left a nasty
> taste in my mouth that will take a lot of proverbial mouthwash to get
> rid of and allow it to be considered in this PEP.
>

I don't see how git can be considered given poor windows support -
compilation on OS/X can be a bear too.

And I echo the need/want to be able to customize the workflow and
integration with the tracker/etc with something a bit more flexible.
The bzr plugin system is nice.

Also the ability to completely nuke the local-work-copies commit
history ("cleaning it up") worries me, but I'm also paranoid.

-jesse

From cstejerean at gmail.com  Mon Nov  3 19:46:25 2008
From: cstejerean at gmail.com (Cosmin Stejerean)
Date: Mon, 3 Nov 2008 12:46:25 -0600
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4BFBAC56-BEDE-4B0A-A0FD-0BB4681F3A11@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<4BFBAC56-BEDE-4B0A-A0FD-0BB4681F3A11@python.org>
Message-ID: <276266d0811031046r45a2799cx3e228a75b0fb08fa@mail.gmail.com>

On Mon, Nov 3, 2008 at 12:34 PM, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 3, 2008, at 12:58 PM, C. Titus Brown wrote:
>
>  -> Sticking with a dvcs implemented in Python makes the best sense,
>> -> especially when you consider the plugin architecture.  When we
>> -> selected a new tracker, we didn't make implementation in Python a
>> -> requirement, but instead a high hurdle.  Meaning, if a tracker wasn't
>> -> written in Python it had to be way better than those written in Python.
>>
>> I worry about the idea of hacking in any way, shape or form, on the
>> version control system used to maintain the Python source code.  I place
>> VCSes at the compiler- or OS-level of the toolchain, because you have
>> the option of fundamentally screwing up the entire project by playing
>> with them.
>>
>> So from that perspective it's better to keep it *out* of Python to
>> remove the temptation to hack :)
>>
>
> :)  But actually more interesting is whether we want to add plugins that
> assist Python dev workflow.  For example, let's say we wanted to have a
> 'fixes' command that automatically updated the Roundup tracker with the
> branch information.  I'd personally much rather write 10 lines of Python to
> add such a plugin than any amount of C or Perl, or whatever else. :)
>


There is no reason you can't develop such a tool on top of git using Python.
In the true spirit of Unix command line utilities you can write tools that
combine or extend the functionality of existing tools in whatever language
you feel comfortable. For example take a look at the repo tool used by
Android to interact with the git repository.
http://source.android.com/download

I've never felt the need to add plugins to my version control system and
have to learn it's API. I've been satisfied with creating a collection of
python scripts or shell scripts to add whatever functionality I needed.

-- 
Cosmin Stejerean
http://www.offbytwo.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/e71f5b76/attachment.htm>

From ismail at namtrac.org  Mon Nov  3 19:48:43 2008
From: ismail at namtrac.org (=?UTF-8?Q?=C4=B0smail_D=C3=B6nmez?=)
Date: Mon, 3 Nov 2008 20:48:43 +0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4222a8490811031045t6eacb4a6qfd425b7224a033e4@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<4222a8490811031045t6eacb4a6qfd425b7224a033e4@mail.gmail.com>
Message-ID: <19e566510811031048v4c37f973jf344502f0ecb0a6c@mail.gmail.com>

Hi,

On Mon, Nov 3, 2008 at 20:45, Jesse Noller <jnoller at gmail.com> wrote:
[...]
> I don't see how git can be considered given poor windows support -
> compilation on OS/X can be a bear too.

I use git on Linux/Mac/Windows day to day, see

http://code.google.com/p/git-osx-installer/
http://code.google.com/p/msysgit/

Regards,
ismail

From josiah.carlson at gmail.com  Mon Nov  3 19:50:21 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Mon, 3 Nov 2008 10:50:21 -0800
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <18702.15638.223872.706216@montanaro-dyndns-org.local>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gedmtg$eop$1@ger.gmane.org> <490AA4B9.5060307@gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
	<loom.20081102T180853-494@post.gmane.org>
	<18702.15638.223872.706216@montanaro-dyndns-org.local>
Message-ID: <e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>

On Sun, Nov 2, 2008 at 3:51 PM, <skip at pobox.com> wrote:

>
>    Antoine> I think it is important to remind that the GIL doesn't prevent
>    Antoine> (almost) true multithreading. The only thing it prevents is
>    Antoine> full use of multi-CPU resources in a single process.
>
> I believe everyone here knows that.  I believe what most people are
> clamoring for is to make "full use of their multi-CPU resources in a single
> process".
>

Which is, arguably, silly.  As we've seen in the last 2 months with Chrome,
multiple processes for a single "program" is actually a pretty good idea.
 With the multiprocessing module in the standard library offering a
threading-like interface, people no longer have any excuses for not fully
exploiting their multiple cores in Python.

 - Josiah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/d4da8c93/attachment.htm>

From eduardo.padoan at gmail.com  Mon Nov  3 19:59:22 2008
From: eduardo.padoan at gmail.com (Eduardo O. Padoan)
Date: Mon, 3 Nov 2008 16:59:22 -0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <1afaf6160811031043l49d04b31jbf7582ba15ad515e@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<4BFBAC56-BEDE-4B0A-A0FD-0BB4681F3A11@python.org>
	<dea92f560811031041t440af08fv113b5346d36de857@mail.gmail.com>
	<1afaf6160811031043l49d04b31jbf7582ba15ad515e@mail.gmail.com>
Message-ID: <dea92f560811031059l2e5cfd9dydeaf8b62c3ef003b@mail.gmail.com>

On Mon, Nov 3, 2008 at 4:43 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Mon, Nov 3, 2008 at 12:41 PM, Eduardo O. Padoan
> <eduardo.padoan at gmail.com> wrote:
>> On Mon, Nov 3, 2008 at 4:34 PM, Barry Warsaw <barry at python.org> wrote:
>>> :)  But actually more interesting is whether we want to add plugins that
>>> assist Python dev workflow.  For example, let's say we wanted to have a
>>> 'fixes' command that automatically updated the Roundup tracker with the
>>> branch information.  I'd personally much rather write 10 lines of Python to
>>> add such a plugin than any amount of C or Perl, or whatever else. :)
>>
>> (Which, btw, is already supported by the "--fixes" option on bzr's commit :)
>
> The --fixes option allows you to note where a bug is fixed in your
> repo. I think what Barry is envisioning is a tool that would
> automatically close the issue on Roundup.

Ouch, sure. I agree, it should be easy to do with a plugin.
I couldn't dedicate enough yet to write a plugin, but as far as I have
seen, the bzr API is quite pythonic.


> --
> Cheers,
> Benjamin Peterson
> "There's nothing quite as beautiful as an oboe... except a chicken
> stuck in a vacuum cleaner."
>



-- 
    Eduardo de Oliveira Padoan
http://djangopeople.net/edcrypt/
"Distrust those in whom the desire to punish is strong." -- Goethe,
Nietzsche, Dostoevsky

From curt at hagenlocher.org  Mon Nov  3 19:59:23 2008
From: curt at hagenlocher.org (Curt Hagenlocher)
Date: Mon, 3 Nov 2008 11:59:23 -0700
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gedmtg$eop$1@ger.gmane.org> <490AA4B9.5060307@gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
	<loom.20081102T180853-494@post.gmane.org>
	<18702.15638.223872.706216@montanaro-dyndns-org.local>
	<e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>
Message-ID: <d2155e360811031059x26245c8ei7c0fe96e53deffaa@mail.gmail.com>

On Mon, Nov 3, 2008 at 11:50 AM, Josiah Carlson <josiah.carlson at gmail.com>wrote:

> On Sun, Nov 2, 2008 at 3:51 PM, <skip at pobox.com> wrote:
>
>>
>>    Antoine> I think it is important to remind that the GIL doesn't prevent
>>    Antoine> (almost) true multithreading. The only thing it prevents is
>>    Antoine> full use of multi-CPU resources in a single process.
>>
>> I believe everyone here knows that.  I believe what most people are
>> clamoring for is to make "full use of their multi-CPU resources in a
>> single
>> process".
>>
>
> Which is, arguably, silly.  As we've seen in the last 2 months with Chrome,
> multiple processes for a single "program" is actually a pretty good idea.
>  With the multiprocessing module in the standard library offering a
> threading-like interface, people no longer have any excuses for not fully
> exploiting their multiple cores in Python.
>

There is no shortage of algorithms (such as matrix multiplication) that are
parallelizable but not particularly good candidates for an IPC-based
multiprocessing paradigm.

--
Curt Hagenlocher
curt at hagenlocher.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/5f5c0751/attachment.htm>

From paul.w.miller at wmich.edu  Mon Nov  3 18:56:37 2008
From: paul.w.miller at wmich.edu (Paul Miller)
Date: Mon, 03 Nov 2008 12:56:37 -0500
Subject: [Python-Dev] Optionally using GMP to implement long if available
Message-ID: <1225734997.14412.5.camel@paul-laptop.wmich.edu>

I've read some of the past discussion about including GMP into the
python core and understand the reasons for not doing so.  Rather than
that, what about patching Python's long implementation to use GMP if
it's available, and the default implementation if not?  Are there any
philosophical or technical objections to this?  If not, I would consider
drafting a patch.

Thanks,

Paul


From musiccomposition at gmail.com  Mon Nov  3 20:19:54 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 3 Nov 2008 13:19:54 -0600
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
Message-ID: <1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>

On Mon, Nov 3, 2008 at 11:56 AM, Paul Miller <paul.w.miller at wmich.edu> wrote:
> I've read some of the past discussion about including GMP into the
> python core and understand the reasons for not doing so.  Rather than
> that, what about patching Python's long implementation to use GMP if
> it's available, and the default implementation if not?  Are there any
> philosophical or technical objections to this?  If not, I would consider
> drafting a patch.

The main objection is that GMP is licensed under LGPL which I believe
conflicts with Python's very open license. Also, there would
incompatibilities between versions of Python that had GMP enabled and
those that didn't with regards to floating points.

>
> Thanks,
>
> Paul




-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From p.f.moore at gmail.com  Mon Nov  3 20:47:34 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 3 Nov 2008 19:47:34 +0000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <19e566510811031048v4c37f973jf344502f0ecb0a6c@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<4222a8490811031045t6eacb4a6qfd425b7224a033e4@mail.gmail.com>
	<19e566510811031048v4c37f973jf344502f0ecb0a6c@mail.gmail.com>
Message-ID: <79990c6b0811031147i70bcfdd9j3ad8483a0074637b@mail.gmail.com>

2008/11/3 ?smail D?nmez <ismail at namtrac.org>:
> On Mon, Nov 3, 2008 at 20:45, Jesse Noller <jnoller at gmail.com> wrote:
> [...]
>> I don't see how git can be considered given poor windows support -
>> compilation on OS/X can be a bear too.

I would say that strong support of all of Python's key platforms would
be a requirement (sorry, I haven't had time to read the PEP yet to see
if it already says this). That means Unix (specifically, Linux and
OS/X, and probably a few other Unix variants) and Windows.

>
> I use git on Linux/Mac/Windows day to day, see
>
> http://code.google.com/p/git-osx-installer/
> http://code.google.com/p/msysgit/

I find the fact that msysgit is a separate project, rather than a part
of core git, is a significant point. Personally, I have dabbled with
git on Windows and it certainly doesn't feel as well-supported as
Bazaar or Mercurial. It's certainly getting closer, but I don't think
even the Git project themselves would say it's there yet (the Git
homepage points to msysgit for binaries, and that page explicitly says
that Windows is only "officially" supported under cygwin). (I could go
on, but I won't - suffice it to say that git doesn't yet feel "native"
on Windows).

Paul.

From gustavo at niemeyer.net  Mon Nov  3 20:57:53 2008
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Mon, 3 Nov 2008 17:57:53 -0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
Message-ID: <643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>

> Since I have never seen that come up during Python's development I am
> going to leave it out. But I do see the benefit and how it might help
> with future work.

Of course, that's entirely up to you.  But it strikes me as an odd
approach to the selection of scenarios for a tool whose intention is
precisely to support new usage scenarios.

> As I mentioned in the doc, the first example allows for whatever the
> DVCS does for diffs, whether that is a full-on branch or not.

Are you looking for new scenarios or for the validation of your document? :-)

I'm talking about history-carrying diffs which can be attached and
sent via mail, not about viewing plain diffs, which every VCS tool in
the world can do since version 0.0.

Anyway, I guess you already have what you want.  Good luck picking the
right choice!

-- 
Gustavo Niemeyer
http://niemeyer.net

From victor.stinner at haypocalc.com  Mon Nov  3 21:41:33 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 3 Nov 2008 21:41:33 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
Message-ID: <200811032141.34063.victor.stinner@haypocalc.com>

Hi,

Le Monday 03 November 2008 18:56:37 Paul Miller, vous avez ?crit?:
> I've read some of the past discussion about including GMP into the
> python core and understand the reasons for not doing so.

Please, check this issue: http://bugs.python.org/issue1814

I patched Python3 to use GMP because I thaugh that GMP is much faster than 
Python for integer computation. The "problem" is that GMP target are integers 
much bigger than 10**100. As I remember, GMP has a special hack for very 
small integer: integer which can be stored in one CPU word, but it's not 
enough to speed up Python.

With my last patch, with GMP, Python is 2% *slower*.

My patch can be improved, but I expected +20% to +100% to no -2% :-/

And there is also the license issue...

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From skip at pobox.com  Mon Nov  3 21:45:45 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 3 Nov 2008 14:45:45 -0600
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gedmtg$eop$1@ger.gmane.org> <490AA4B9.5060307@gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
	<loom.20081102T180853-494@post.gmane.org>
	<18702.15638.223872.706216@montanaro-dyndns-org.local>
	<e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>
Message-ID: <18703.25337.338108.460274@montanaro-dyndns-org.local>


    >> I believe everyone here knows that.  I believe what most people are
    >> clamoring for is to make "full use of their multi-CPU resources in a
    >> single process".

    Josiah> Which is, arguably, silly.  As we've seen in the last 2 months
    Josiah> with Chrome, multiple processes for a single "program" is
    Josiah> actually a pretty good idea.

I have no idea what Chrome is.  Is it a CPU-intensive algorithm which can be
parallelized?

    Josiah> With the multiprocessing module in the standard library offering
    Josiah> a threading-like interface, people no longer have any excuses
    Josiah> for not fully exploiting their multiple cores in Python.

Except for communication overhead caused by replacing shared memory with
I/O?

Skip

From matthieu.brucher at gmail.com  Mon Nov  3 21:48:08 2008
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Mon, 3 Nov 2008 21:48:08 +0100
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <18703.25337.338108.460274@montanaro-dyndns-org.local>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<gedmtg$eop$1@ger.gmane.org> <490AA4B9.5060307@gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
	<loom.20081102T180853-494@post.gmane.org>
	<18702.15638.223872.706216@montanaro-dyndns-org.local>
	<e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>
	<18703.25337.338108.460274@montanaro-dyndns-org.local>
Message-ID: <e76aa17f0811031248s2c718863l5585a89ffa8b33fc@mail.gmail.com>

2008/11/3  <skip at pobox.com>:
>
>    >> I believe everyone here knows that.  I believe what most people are
>    >> clamoring for is to make "full use of their multi-CPU resources in a
>    >> single process".
>
>    Josiah> Which is, arguably, silly.  As we've seen in the last 2 months
>    Josiah> with Chrome, multiple processes for a single "program" is
>    Josiah> actually a pretty good idea.
>
> I have no idea what Chrome is.  Is it a CPU-intensive algorithm which can be
> parallelized?

It's Google webbrowser ;)

Matthieu Brucher

>    Josiah> With the multiprocessing module in the standard library offering
>    Josiah> a threading-like interface, people no longer have any excuses
>    Josiah> for not fully exploiting their multiple cores in Python.
>
> Except for communication overhead caused by replacing shared memory with
> I/O?
>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/matthieu.brucher%40gmail.com
>



-- 
Information System Engineer, Ph.D.
Website: http://matthieu-brucher.developpez.com/
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn: http://www.linkedin.com/in/matthieubrucher

From skip at pobox.com  Mon Nov  3 21:48:58 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 3 Nov 2008 14:48:58 -0600
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
Message-ID: <18703.25530.532307.815707@montanaro-dyndns-org.local>


    Benjamin> The main objection is that GMP is licensed under LGPL which I
    Benjamin> believe conflicts with Python's very open license. 

If GMP itself isn't included with Python how can there be a licensing issue?

Skip

From barry at python.org  Mon Nov  3 22:09:20 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 16:09:20 -0500
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <18698.56752.992826.599106@montanaro-dyndns-org.local>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<490AC5B4.6080507@v.loewis.de>
	<18698.56752.992826.599106@montanaro-dyndns-org.local>
Message-ID: <7E4E9932-8D38-42E2-A414-C41B8044E486@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 31, 2008, at 6:28 AM, skip at pobox.com wrote:

>
>    Martin> I have now restored the original URL structure, and moved  
> the
>    Martin> loggerhead installation to
>
>    Martin> http://code.python.org/loggerhead/
>
> A couple nits.  Leaving off the trailing / yields a 404.

This should be fixed now via a RedirectPermanent.

>  (No big deal
> though).  More importantly, there seem to be no images, e.g.:
>
>    http://code.python.org/static/images/ico_folder.gif
>
> Looks like it should be
>
>    http://code.python.org/loggerhead/static/images/ico_folder.gif

I think Martin fixed these; at least I don't see this problem any more.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ9ogHEjvBPtnXfVAQKSiAP/Y30HaPDs54GaeB1P+e2CIpQANUlsEm5d
ePdfPvSKKXbKGjlYI6jsHdT75oT2q0BVo5DDGN6YcU3oISIIZF6X9AP86LXCqLeI
FPU1JEsR+Deds3iALxvd2r306c9gzTFz3j30mjvNUF8HinTheE8lG3j0qCnYSVQI
aqClcXudv5s=
=Adh8
-----END PGP SIGNATURE-----

From barry at python.org  Mon Nov  3 22:15:06 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 16:15:06 -0500
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<20081031102524.75dfd13f@resist.wooz.org>
	<9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
Message-ID: <5E6F59CC-B29A-4D00-9D89-B4D97ABB8034@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 11:54 AM, Thomas Wouters wrote:

> FWIW, I put one up this weekend, and it seems to be intact and OK.  
> (bzr+ssh://pythonbzr at code.python.org/python/2.6/ or http://code.python.org/python/2.6/ 
>  )

Excellent, thanks!  This is getting mirrored with updates, right?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ9p2nEjvBPtnXfVAQJoVgP+JLJHK9FRK0xsLtiMSW2BrrnYuCjwhidX
yRdNK6oE/hiUKGCjO8G+IptqsDJl5MxFKYZ7RAhK9HjCsjoLW/aQc3zAJuwb+dyX
fowpwrBV1pPRTB2IP840ImEATfufb8Mwzo5H8G9k+dx1BTHjY4o5JLQXFREBUNuu
jUKSPMOr9NU=
=wur0
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Mon Nov  3 22:39:15 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 03 Nov 2008 22:39:15 +0100
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <7E4E9932-8D38-42E2-A414-C41B8044E486@python.org>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<490AC5B4.6080507@v.loewis.de>
	<18698.56752.992826.599106@montanaro-dyndns-org.local>
	<7E4E9932-8D38-42E2-A414-C41B8044E486@python.org>
Message-ID: <490F6F83.4010009@v.loewis.de>

>>  (No big deal
>> though).  More importantly, there seem to be no images, e.g.:
> 
>>    http://code.python.org/static/images/ico_folder.gif
> 
>> Looks like it should be
> 
>>    http://code.python.org/loggerhead/static/images/ico_folder.gif
> 
> I think Martin fixed these; at least I don't see this problem any more.

Right - but only so through a --prefix option to serve-branches. If
you make it a reboot-safe service, be sure to copy this option.

Regards,
Martin

From barry at python.org  Mon Nov  3 22:45:24 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 16:45:24 -0500
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <490F6F83.4010009@v.loewis.de>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<490AC5B4.6080507@v.loewis.de>
	<18698.56752.992826.599106@montanaro-dyndns-org.local>
	<7E4E9932-8D38-42E2-A414-C41B8044E486@python.org>
	<490F6F83.4010009@v.loewis.de>
Message-ID: <A48766C3-7256-45B2-A29B-51165A4311EF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 4:39 PM, Martin v. L?wis wrote:

>>> (No big deal
>>> though).  More importantly, there seem to be no images, e.g.:
>>
>>>   http://code.python.org/static/images/ico_folder.gif
>>
>>> Looks like it should be
>>
>>>   http://code.python.org/loggerhead/static/images/ico_folder.gif
>>
>> I think Martin fixed these; at least I don't see this problem any  
>> more.
>
> Right - but only so through a --prefix option to serve-branches. If
> you make it a reboot-safe service, be sure to copy this option.

Will do, thanks.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ9w9HEjvBPtnXfVAQKaFwP/ZnHhz39R52yUFGc9vgJTISFXgTkXk9AS
CPeG+FGWuJVdAHHNdIZ7zv8s1Nbl4KoDnHzJdvL2AQjCspJjjibnh/ss39BlGxZe
Qq4FFsbKX59NWL0lbBAyKrHRBT2eX1EIbGgrBvGgNUSHRecoJC30UpIxz/DC/cur
xdSg7kK/jm8=
=cwwj
-----END PGP SIGNATURE-----

From brett at python.org  Mon Nov  3 22:50:09 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 13:50:09 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <loom.20081103T181817-535@post.gmane.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<loom.20081103T181817-535@post.gmane.org>
Message-ID: <bbaeab100811031350x1ceda2f6laabaec03fbf7c846@mail.gmail.com>

On Mon, Nov 3, 2008 at 10:19, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Brett Cannon <brett <at> python.org> writes:
>>
>> At this point I am looking for any suggestions for fundamental usage
>> scenarios that I am missing from the PEP. If you think the few already
>> listed are missing some core part of a VCS, please let me know.
>
> You might want to refine the "patch review" scenario with a variant using
> Rietveld and its upload script.

Well, I don't want to propose new scenarios that are not widely used
yet. But I will cover support by patch review tools in the
Platform/Tools Support section. Thanks for the suggestion, Antoine.

-Brett

From martin at v.loewis.de  Mon Nov  3 22:56:21 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 03 Nov 2008 22:56:21 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <18703.25530.532307.815707@montanaro-dyndns-org.local>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
Message-ID: <490F7385.5090302@v.loewis.de>

skip at pobox.com wrote:
>     Benjamin> The main objection is that GMP is licensed under LGPL which I
>     Benjamin> believe conflicts with Python's very open license. 
> 
> If GMP itself isn't included with Python how can there be a licensing issue?

On Windows, the GMP binaries would be incorporated into pythonxy.dll.
This would force anybody providing a copy of pythonxy.dll to also
provide the sources of GMP.

Regards,
Martin

From brett at python.org  Mon Nov  3 22:56:33 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 13:56:33 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <9e804ac0811031035v3b7a816bwe8b886da59207a6b@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<9e804ac0811031035v3b7a816bwe8b886da59207a6b@mail.gmail.com>
Message-ID: <bbaeab100811031356rd97c65q8b42f8e79151623a@mail.gmail.com>

On Mon, Nov 3, 2008 at 10:35, Thomas Wouters <thomas at python.org> wrote:
>
>
> On Mon, Nov 3, 2008 at 18:57, Brett Cannon <brett at python.org> wrote:
>>
>> On Sun, Nov 2, 2008 at 17:08, Gustavo Niemeyer <gustavo at niemeyer.net>
>> wrote:
>> > Hi Brett,
>> >
>> >> At this point I am looking for any suggestions for fundamental usage
>> >> scenarios that I am missing from the PEP. If you think the few already
>> >> listed are missing some core part of a VCS, please let me know.
>> >
>> > As an initial disclaimer, I use bzr in my daily routine.  That said,
>> > I'm sending below a few mostly unbiased high-level ideas, just based
>> > on useful ways we explore the DVCS-aspect on our normal daily
>> > workflow.
>> >
>> > == Coordinated development with dependent features ==
>> >
>> > A variation on coordinated development, which is actually one of the
>> > main motivators for DVCS.  Branch A is evolving out of the mainline,
>> > and it depends on a feature that is on branch B which is also not yet
>> > integrated.  Parallel development of both should be nicely supported.
>> > I'm sure all DVCS will do that in a decent form, but figuring how this
>> > works may be instructive and worth mentioning.
>> >
>>
>> Since I have never seen that come up during Python's development I am
>> going to leave it out. But I do see the benefit and how it might help
>> with future work.
>
> Euhm, wut? It hasn't come up during Python's development because Python is
> being developed in a VCS with very limited branches :) I'm partial to VCS's
> with proper branching (as you know) and I've been using that mode of
> development for many years. I've done development and maintenance of
> multi-developer projects using both the CVS/SVN nonbranching approach and
> the typical DVCS branch-often approach using BitKeeper, Bazaar and Mercurial
> (as well as the sort-of one-off-branch Perforce approach where you can
> easily 'wrap' a single change but can't really do dependant changes) -- and
> whenever possible I use the branch-often approach with dependant branches
> all over the place, especially when working on large, complicated changes.
> Building them up out of separate 'components' requires a little more
> administration (you have to remember which branch to submit to) but it makes
> debugging, piecemeal discussion and batchwise integration a *lot* easier.
>

I know it isn't used because of what VCSs we have used, which is why I
have no experience with it and feel uncomfortable using it as a
necessary usage scenario to evaluate whether a DVCS is better than
svn, and if so which one.

But then again, having one scenario that shows svn's weakness directly
wouldn't hurt. I could see a scenario where I start to fix something
in branch A, realize that a deeper issue needs to be fixed, leading to
branch B, and then have branch A depend on branch B. Is that possible?
Or is injecting branch dependencies like that not workable? If it
doesn't work, then a branch A/B that a branch C is dependent on would
also work as a scenario (e.g. reworking the testing framework where
you are doing a bunch of different things at once that are culminating
in a single new testing branch that collects everything).

Those sound like what you are getting after?

-Brett

From brett at python.org  Mon Nov  3 23:03:27 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 14:03:27 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>
Message-ID: <bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>

On Mon, Nov 3, 2008 at 11:57, Gustavo Niemeyer <gustavo at niemeyer.net> wrote:
>> Since I have never seen that come up during Python's development I am
>> going to leave it out. But I do see the benefit and how it might help
>> with future work.
>
> Of course, that's entirely up to you.  But it strikes me as an odd
> approach to the selection of scenarios for a tool whose intention is
> precisely to support new usage scenarios.
>
>> As I mentioned in the doc, the first example allows for whatever the
>> DVCS does for diffs, whether that is a full-on branch or not.
>
> Are you looking for new scenarios or for the validation of your document? :-)
>

Both. First and foremost I am looking for any scenarios people are
using now for svn that I didn't cover. After that I can probably add
some DVCS-specific things. But the problem with that is my DVCS
experience is limited and thus I don't want to add a scenario that
seems whizbang cool but in real life is never used; premature
optimization and all that.

> I'm talking about history-carrying diffs which can be attached and
> sent via mail, not about viewing plain diffs, which every VCS tool in
> the world can do since version 0.0.
>

As I said, one of the scenarios already says patches can be whatever
the DVCS supports the best; plain diffs, branches, etc. And the
comments for that scenario will point out any perks from that feature.

As for the "via mail" feature, I use Gmail so it doesn't mean anything to me. =)

> Anyway, I guess you already have what you want.  Good luck picking the
> right choice!

Oh, there won't be a right choice (except in my mind), there will just
be the best choice that could be made based on the circumstances.
Working on the issue tracker taught me that lesson.

-Brett

From brett at python.org  Mon Nov  3 23:13:30 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 14:13:30 -0800
Subject: [Python-Dev] New "stage" field in the tracker
Message-ID: <bbaeab100811031413v5d759923l40075cae562ae91b@mail.gmail.com>

Martin added a new "stage" field in the tracker so that issues can now
be more clearly identified in terms of what is needed to move them
forward. For now it is probably best to continue to use both keywords
and stage values (i.e. the "patch" keyword is implied when the stage
of an issue goes beyond "needs patch" and the need for a "patch
review" is covered by the "commit review" stage). Once the use of the
stage gets far enough along we can discussed ditching the keywords.

Anyway, cheers to our first step in improving our workflow!

-Brett

From ncoghlan at gmail.com  Mon Nov  3 23:14:32 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 04 Nov 2008 08:14:32 +1000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031356rd97c65q8b42f8e79151623a@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>	<9e804ac0811031035v3b7a816bwe8b886da59207a6b@mail.gmail.com>
	<bbaeab100811031356rd97c65q8b42f8e79151623a@mail.gmail.com>
Message-ID: <490F77C8.7020108@gmail.com>

Brett Cannon wrote:
> But then again, having one scenario that shows svn's weakness directly
> wouldn't hurt. I could see a scenario where I start to fix something
> in branch A, realize that a deeper issue needs to be fixed, leading to
> branch B, and then have branch A depend on branch B. Is that possible?
> Or is injecting branch dependencies like that not workable? If it
> doesn't work, then a branch A/B that a branch C is dependent on would
> also work as a scenario (e.g. reworking the testing framework where
> you are doing a bunch of different things at once that are culminating
> in a single new testing branch that collects everything).
> 
> Those sound like what you are getting after?

Hmm, perhaps your concrete example of this could be a case where someone
was working on an object capabilities security model and then realised
they needed a more flexible implementation of the import system before
their security mechanism was going to work ;)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From thomas at python.org  Mon Nov  3 23:22:35 2008
From: thomas at python.org (Thomas Wouters)
Date: Mon, 3 Nov 2008 23:22:35 +0100
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <5E6F59CC-B29A-4D00-9D89-B4D97ABB8034@python.org>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<20081031102524.75dfd13f@resist.wooz.org>
	<9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
	<5E6F59CC-B29A-4D00-9D89-B4D97ABB8034@python.org>
Message-ID: <9e804ac0811031422v4a205a21ld48e09e9baf8f1c5@mail.gmail.com>

On Mon, Nov 3, 2008 at 22:15, Barry Warsaw <barry at python.org> wrote:

> On Nov 3, 2008, at 11:54 AM, Thomas Wouters wrote:
>
>  FWIW, I put one up this weekend, and it seems to be intact and OK.
>> (bzr+ssh://pythonbzr at code.python.org/python/2.6/ or
>> http://code.python.org/python/2.6/ )
>>
>
> Excellent, thanks!  This is getting mirrored with updates, right?
>

Exactly the same way 2.5, trunk and 3.0 are, yes.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/ca1e179c/attachment.htm>

From greg at krypto.org  Mon Nov  3 23:26:21 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 3 Nov 2008 14:26:21 -0800
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <200811032141.34063.victor.stinner@haypocalc.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
Message-ID: <52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>

On Mon, Nov 3, 2008 at 12:41 PM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> Le Monday 03 November 2008 18:56:37 Paul Miller, vous avez ?crit :
>> I've read some of the past discussion about including GMP into the
>> python core and understand the reasons for not doing so.
>
> Please, check this issue: http://bugs.python.org/issue1814
>
> I patched Python3 to use GMP because I thaugh that GMP is much faster than
> Python for integer computation. The "problem" is that GMP target are integers
> much bigger than 10**100. As I remember, GMP has a special hack for very
> small integer: integer which can be stored in one CPU word, but it's not
> enough to speed up Python.
>
> With my last patch, with GMP, Python is 2% *slower*.
>
> My patch can be improved, but I expected +20% to +100% to no -2% :-/

One optimization that could be done to the existing Python longobject
code is to allow it to use larger digits.  Currently it is hardcoded
to use 15bit digits.

The most common desktop+server CPUs in the world (x86) all support
efficient 32bit*32bit -> 64bit multiply so there is no good reason to
limit python itself to 15bit digits when on either x86 or x86_64.  30
or 31bit digits would be more efficient as more numbers would then be
a single digit shortcutting their computations and the number of
digits for others would be half.

The ability to use 15bit digits is good for other cpus but is wasteful
when the hardware supports more (32bit x86 and all 64bit archs).

>
> And there is also the license issue...
>
> --
> Victor Stinner aka haypo
> http://www.haypocalc.com/blog/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>

From barry at python.org  Mon Nov  3 23:33:10 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 17:33:10 -0500
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <9e804ac0811031422v4a205a21ld48e09e9baf8f1c5@mail.gmail.com>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<20081031102524.75dfd13f@resist.wooz.org>
	<9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
	<5E6F59CC-B29A-4D00-9D89-B4D97ABB8034@python.org>
	<9e804ac0811031422v4a205a21ld48e09e9baf8f1c5@mail.gmail.com>
Message-ID: <3A6A8368-D66C-46D9-980C-65AC0F6B7F45@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 5:22 PM, Thomas Wouters wrote:

> Exactly the same way 2.5, trunk and 3.0 are, yes.

Beauty, thanks.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ98J3EjvBPtnXfVAQLD0AQAhvDoJ85HtO0o/KxeU//kRjid7j0SVIJA
OaKnMA/om71Ft5733ZIJYE1jQ1+WFr8TZggiD5NG6d3Af8f8Q0Ep2ujBpOYjKhcQ
85DEmeY2WkoeYqMZXFhEa76BrrPHQtOzEzagRTGIvLnw3W5DbKzTawg8D4z4qmG3
vtQkzEg7Lfw=
=cGIL
-----END PGP SIGNATURE-----

From thomas at python.org  Mon Nov  3 23:39:53 2008
From: thomas at python.org (Thomas Wouters)
Date: Mon, 3 Nov 2008 23:39:53 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031356rd97c65q8b42f8e79151623a@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<9e804ac0811031035v3b7a816bwe8b886da59207a6b@mail.gmail.com>
	<bbaeab100811031356rd97c65q8b42f8e79151623a@mail.gmail.com>
Message-ID: <9e804ac0811031439p43649da2p8cd912395205124c@mail.gmail.com>

On Mon, Nov 3, 2008 at 22:56, Brett Cannon <brett at python.org> wrote:

> But then again, having one scenario that shows svn's weakness directly
> wouldn't hurt. I could see a scenario where I start to fix something
> in branch A, realize that a deeper issue needs to be fixed, leading to
> branch B, and then have branch A depend on branch B. Is that possible?
> Or is injecting branch dependencies like that not workable? If it
> doesn't work, then a branch A/B that a branch C is dependent on would
> also work as a scenario (e.g. reworking the testing framework where
> you are doing a bunch of different things at once that are culminating
> in a single new testing branch that collects everything).


Here's a real-life Python example: http://bugs.python.org/issue2292. I
actually developed that in two separate branches, one depending on the
other: one branch for *just* the changes to functioncalls, to generalize *-
and **-unpacking in that context, and one branch to add *- and **-unpacking
in other contexts, which was a much more contentious proposal. The division
means that the first part can be committed without the second part *without
any extra effort from anyone*. I wouldn't have to undo changes, I wouldn't
have to reapply diffs or do a painful manual merge were the first branch to
be merged into the trunk. It would be completely straightforward and
effortless.

Of course, that's just a small-scale example. I've done much larger of these
in the past, but not on Python. You can easily dream up more examples,
though: a branch for the gforth generated ceval loop, with separate branches
off of that branch with different approaches to the superinstructions. The
right one could be picked at a later time, or branches could be merged, or
they could all be omitted. The generated ceval loop could be backported to
2.7 without including the superinstructions that were merged into 3.0.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/caf07f58/attachment.htm>

From gustavo at niemeyer.net  Mon Nov  3 23:38:45 2008
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Mon, 3 Nov 2008 20:38:45 -0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>
	<bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>
Message-ID: <643d90130811031438o7c0609faya8cd60b61e23593f@mail.gmail.com>

> Both. First and foremost I am looking for any scenarios people are
> using now for svn that I didn't cover. After that I can probably add
> some DVCS-specific things. But the problem with that is my DVCS
> experience is limited and thus I don't want to add a scenario that

So try to listen to people that actually use these tools and are
trying to help you.

> seems whizbang cool but in real life is never used; premature
> optimization and all that.

As I mentioned early in my mail, all the scenarios I described were
specifically meant to expose common situations that we go through in
our workflow.  I'm not picking weird things to try favoring any
specific tool, even because I believe all tools should support these
very basic desires.

> As I said, one of the scenarios already says patches can be whatever
> the DVCS supports the best; plain diffs, branches, etc. And the
> comments for that scenario will point out any perks from that feature.

"""
This scenario to meant to represent the steps required for a non-core
developer who has a one-off patch they want to create from a read-only
checkout.
"""

Ok, so would you mind to expand that to describe that the tool should
send the history of the "one-off patch", and what it means to not send
it?

Otherwise you're really comparing oranges and apples.  If I send you
"svn diff" or "bzr diff", you get a plain diff.  Merging this will not
carry the history of who made the chances, what were the incremental
steps, and will handle conflicts without context, which may be way
more boring to perform than when you actually have full history
information.  That's not the same as using "bzr send".

Again, that's a real world situation, not something I'm making up to
favor bzr.  I'm pretty sure hg should have something similar too (and
git does for sure).

> As for the "via mail" feature, I use Gmail so it doesn't mean anything to me. =)

Please ask Tim to teach you how the "mail" word got in "Gmail".  ;-)

-- 
Gustavo Niemeyer
http://niemeyer.net

From barry at python.org  Mon Nov  3 23:54:43 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 17:54:43 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031356rd97c65q8b42f8e79151623a@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<9e804ac0811031035v3b7a816bwe8b886da59207a6b@mail.gmail.com>
	<bbaeab100811031356rd97c65q8b42f8e79151623a@mail.gmail.com>
Message-ID: <7C5EAAA3-A7FF-4C07-A573-0418A81AAF5A@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 4:56 PM, Brett Cannon wrote:

> But then again, having one scenario that shows svn's weakness directly
> wouldn't hurt. I could see a scenario where I start to fix something
> in branch A, realize that a deeper issue needs to be fixed, leading to
> branch B, and then have branch A depend on branch B. Is that possible?

This is something I do all the time with bzr.  A related workflow is  
one where you're developing several inter-related dependent branches.   
In those scenarios, I personally find Bazaar looms indispensable.  I'm  
told that bzr looms are similar to hg quilt.

Until I started using looms I had no idea of their power and  
transformative effect on my code development.  This is one of the  
problems of coming up with scenarios: you don't know what you're  
missing.

> Or is injecting branch dependencies like that not workable? If it
> doesn't work, then a branch A/B that a branch C is dependent on would
> also work as a scenario (e.g. reworking the testing framework where
> you are doing a bunch of different things at once that are culminating
> in a single new testing branch that collects everything).
>
> Those sound like what you are getting after?

Let me try to describe a scenario and see if it fits what you're  
looking for.

I'm fixing a bug in urllib2.py when I realize there's a lower-level  
bug in base64.py.  I'd like to cleanly suspend my work on urllib2.py,  
then "push my stack" to start working on the fix for base64.py.  Half  
way through that I realize there's also a bug in binascii that I need  
to fix.  I'd now like to "push my stack" once more to fix the binascii  
problem.

Now that binascii is fixed, I'd like to pop my stack to complete the  
base64 fix, but without losing all the changes I made to binascii.   
Similarly, when I've fixed base64, I want to retain those fixes and  
pop my stack back up to urllib2.

I've now got three related fixes.  Do I get each reviewed  
independently or together?  Do I land them independently or together?   
Can my dvcs help me manage all these inter-dependent branches?

For bzr, the answer is "yes" and my own workflow to handle this uses  
looms, which you can think of as a kind of "stacked branches"  
approach.  Each layer in the stack is called a "thread" and I can  
easily create new threads in the stack.  So I might approach the above  
scenario like so:

bzr branch trunk issue1234
cd issue1234
bzr loomify --nick trunk
bzr create-thread urllib2-fix
# hack hack hack, oops!  gotta fix the bug in base64
bzr commit -m'Fixes to urllib2 so far'
bzr down-thread
bzr create-thread base64-fix
# hack hack hack, oops!  gotta fix binascii
bzr commit -m'Fixes to base64 so far'
bzr down-thread
bzr create-thread binascii-fix
# hack hack hack.  ah, all is good
bzr commit -m'Fixes to binascii'
bzr up-thread
# At this point, all my binascii-fix changes are merged in
# resolve any conflicts that arose and commit, then finish
# hacking on base64
bzr commit -m'Fixes to base64'
bzr up-thread
# At this point all my binascii-fix and base64-fix changes
# are merged in.  resolve any conflicts that arose and commit
# then finish hacking on urllib2
bzr commit -m'Fixes to urllib2'

Here's what's even cooler.  Let say, while I was working on all this,  
someone landed a change to trunk that I want in my branch.  I can 'bzr  
down-thread' to the lowest thread (i.e. trunk), pull in those changes,  
and then using 'bzr up-thread' merge them into all the threads above.   
Then I go back to hacking.

There are all kinds of scenarios based on this one, and I hope the  
above makes sense.  It's things like this (and 'bzr shelve') which  
make using a dvcs like Bazaar so nice.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ+BNHEjvBPtnXfVAQLLrwP/aJuetZxYI0S0SvT5QXJG6wARI2K9vdc5
zpUMHofCkxAq4tv/2Ni7g46jUNTayH4A94sAUqjE0QPM3SFl22/TNjga4ZgaHdjt
7Ma9iX7+s5OJenvV0kd3ReN6KvcKQhcQZ+P4DkWBJu0KDxGptUk/WGer3aDdQNzV
jn88QaHpEBk=
=fPRR
-----END PGP SIGNATURE-----

From barry at python.org  Mon Nov  3 23:58:33 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 17:58:33 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>
	<bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>
Message-ID: <C1083904-EE78-42F5-8AFD-CC8392FD37E4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 5:03 PM, Brett Cannon wrote:

> As I said, one of the scenarios already says patches can be whatever
> the DVCS supports the best; plain diffs, branches, etc. And the
> comments for that scenario will point out any perks from that feature.

It's actually a profound improvement not to have to deal with  
traditional patch files as much.  The feel... dead somehow, compared  
to a live branch which can track changes to the trunk, etc.

> As for the "via mail" feature, I use Gmail so it doesn't mean  
> anything to me. =)

It should though.  In Bazaar, a bundle is essentially the revisions in  
your branch that are packaged to transport over RFC 2821 and 2822. :)   
That opens up some really cool abilities, such as sending a bundle to  
a buildbot or patch queue manager for automated testing.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ+CGXEjvBPtnXfVAQKOnAQAs/cr+g091KyjteQaY/quBBIJ53PUoilC
MLK9fNLz4jq0eDtl+Rw49rmAFmhKJQaJqx91dZszPEc4FyP9lZByqTbNoCLFhPJP
asmxWS263hjx8tXL+RfX+C0Atk9OWzy7j45oa3/J9aBui4ihyI5Yh8/9l8KiCude
6TajzeRcIWs=
=6euF
-----END PGP SIGNATURE-----

From barry at python.org  Tue Nov  4 00:00:43 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 18:00:43 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <9e804ac0811031439p43649da2p8cd912395205124c@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<9e804ac0811031035v3b7a816bwe8b886da59207a6b@mail.gmail.com>
	<bbaeab100811031356rd97c65q8b42f8e79151623a@mail.gmail.com>
	<9e804ac0811031439p43649da2p8cd912395205124c@mail.gmail.com>
Message-ID: <5DB60908-2451-4F47-AFA1-EEE11476EF4D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 5:39 PM, Thomas Wouters wrote:

> Here's a real-life Python example: http://bugs.python.org/issue2292. I
> actually developed that in two separate branches, one depending on the
> other: one branch for *just* the changes to functioncalls, to  
> generalize *-
> and **-unpacking in that context, and one branch to add *- and **- 
> unpacking
> in other contexts, which was a much more contentious proposal. The  
> division
> means that the first part can be committed without the second part  
> *without
> any extra effort from anyone*. I wouldn't have to undo changes, I  
> wouldn't
> have to reapply diffs or do a painful manual merge were the first  
> branch to
> be merged into the trunk. It would be completely straightforward and
> effortless.

Great example.  The looms scenario I posted earlier would be perfect  
for this.  I do this all the time with larger feature development.  I  
can have 4 or 5 threads in a typical loom, though I know of people who  
use many more.  It makes developing depending features actually  
manageable.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ+Cm3EjvBPtnXfVAQJkMwQAg3r/JFAOE3RlxbrtPl48QOyjS3PhN7xg
+wOoVz+FgLTkx2187xbSqBjy8K69uC3R3Y/1Foc7OmhUEtTGMfmStwoWdjB3f2zi
BOgYDPcEaDHC5opfDWtbWpYOJzVcfRJJxhAbevHlKeUSX0/bFd39UY58DXgwR9aI
BAjTj4Gt66U=
=nbTw
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Tue Nov  4 00:20:36 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 04 Nov 2008 00:20:36 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
Message-ID: <490F8744.8010302@v.loewis.de>

> One optimization that could be done to the existing Python longobject
> code is to allow it to use larger digits.  Currently it is hardcoded
> to use 15bit digits.
> 
> The most common desktop+server CPUs in the world (x86) all support
> efficient 32bit*32bit -> 64bit multiply so there is no good reason to
> limit python itself to 15bit digits when on either x86 or x86_64.

Perhaps Tim Peters should also comment here - but if you can come up
with a patch that does that in a portable way, I would be in favor.
The challenge, ISTM, is to make it still compile on all systems
(regardless of how inefficient it might be on minor platforms).

Regards,
Martin

From musiccomposition at gmail.com  Tue Nov  4 00:39:26 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 3 Nov 2008 17:39:26 -0600
Subject: [Python-Dev] XXX do we need a new policy?
Message-ID: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>

Grepping through Python's sources tells me that we have over 2,000
"XXX" comments. The thing that irks me about them is that the have a
very slow rate of being resolved, since they usually act more as
"notes to self" rather than easily attainable tasks.

So, I propose that we adopt a policy similar to Twisted's: All "XXX"
comments must have an issue in the tracker and an accompanying link
with the source code. That way we'll have a forum for discussing the
changes and deciding whether they are reasonable enough to
implemented.

-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From guido at python.org  Tue Nov  4 00:44:44 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 3 Nov 2008 15:44:44 -0800
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
Message-ID: <ca471dc20811031544o551d4962q5bda42b742ce9ed1@mail.gmail.com>

On Mon, Nov 3, 2008 at 3:39 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> Grepping through Python's sources tells me that we have over 2,000
> "XXX" comments. The thing that irks me about them is that the have a
> very slow rate of being resolved, since they usually act more as
> "notes to self" rather than easily attainable tasks.
>
> So, I propose that we adopt a policy similar to Twisted's: All "XXX"
> comments must have an issue in the tracker and an accompanying link
> with the source code. That way we'll have a forum for discussing the
> changes and deciding whether they are reasonable enough to
> implemented.

That seems excessively draconian. Even at Google we don't have a rule
like that, and we're sure big on process and interlinked tools. If you
want to, you could add a different magic keyword, but personally, I
don't see XXX comments necessarily as "to be resolved" -- merely as
flags for someone perusing the code looking to change it or digging
for the cause of some problem to pay special attention.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Tue Nov  4 00:49:04 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 3 Nov 2008 18:49:04 -0500
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
Message-ID: <EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 3, 2008, at 6:39 PM, Benjamin Peterson wrote:

> Grepping through Python's sources tells me that we have over 2,000
> "XXX" comments. The thing that irks me about them is that the have a
> very slow rate of being resolved, since they usually act more as
> "notes to self" rather than easily attainable tasks.
>
> So, I propose that we adopt a policy similar to Twisted's: All "XXX"
> comments must have an issue in the tracker and an accompanying link
> with the source code. That way we'll have a forum for discussing the
> changes and deciding whether they are reasonable enough to
> implemented.

We have a very similar policy here at work.  Sadly I don't think it  
does much on actually resolving the issues.  XXX bugs tend to be  
pretty minor in the scheme of things and they're almost always lower  
on the priority list than everything else.

That being said, we recognize that XXX bugs incur "technical debt" and  
reducing that debt is something we're devoting Real Time to doing.

I see that Guido is not keen on the idea, and I'm not sure my  
observations help sway things one way or the other.  OTOH, it would be  
nice if at least we always add our own identifier (initials, nick,  
email address) and a date to the XXX so we at least know who was  
talking about what.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSQ+N8HEjvBPtnXfVAQLSPAP/ZKHOTaPxNPySfxPwGmbLSTXjSst+z4/f
1SkhkKjOq8/pjjftZW8n6AWb4WSIWXcyCPJ3FGOAXaP141px2yMfRQnXXbdyH6vQ
6sW12e4ZC2J8UPUyzOxiFJb5F8/HhmIMVifzV1vkEf60QYo9lCD614rBdiW30q9v
pWudrLQUMEM=
=1USW
-----END PGP SIGNATURE-----

From guido at python.org  Tue Nov  4 01:04:05 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 3 Nov 2008 16:04:05 -0800
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
	<EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>
Message-ID: <ca471dc20811031604j5c8b940bndd81b8e253ebc1be@mail.gmail.com>

On Mon, Nov 3, 2008 at 3:49 PM, Barry Warsaw <barry at python.org> wrote:
> On Nov 3, 2008, at 6:39 PM, Benjamin Peterson wrote:
>
>> Grepping through Python's sources tells me that we have over 2,000
>> "XXX" comments. The thing that irks me about them is that the have a
>> very slow rate of being resolved, since they usually act more as
>> "notes to self" rather than easily attainable tasks.
>>
>> So, I propose that we adopt a policy similar to Twisted's: All "XXX"
>> comments must have an issue in the tracker and an accompanying link
>> with the source code. That way we'll have a forum for discussing the
>> changes and deciding whether they are reasonable enough to
>> implemented.
>
> We have a very similar policy here at work.  Sadly I don't think it does
> much on actually resolving the issues.  XXX bugs tend to be pretty minor in
> the scheme of things and they're almost always lower on the priority list
> than everything else.
>
> That being said, we recognize that XXX bugs incur "technical debt" and
> reducing that debt is something we're devoting Real Time to doing.
>
> I see that Guido is not keen on the idea, and I'm not sure my observations
> help sway things one way or the other.  OTOH, it would be nice if at least
> we always add our own identifier (initials, nick, email address) and a date
> to the XXX so we at least know who was talking about what.

At Google we use TODO(<username>): Blah, blah. I don't think ading a
date would help much but I wouldn't be opposed to this.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Tue Nov  4 01:37:29 2008
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 3 Nov 2008 18:37:29 -0600
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <490F7385.5090302@v.loewis.de>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
Message-ID: <18703.39241.442461.116001@montanaro-dyndns-org.local>


    Benjamin> The main objection is that GMP is licensed under LGPL which I
    Benjamin> believe conflicts with Python's very open license.

    >> If GMP itself isn't included with Python how can there be a licensing
    >> issue?

    Martin> On Windows, the GMP binaries would be incorporated into
    Martin> pythonxy.dll.  This would force anybody providing a copy of
    Martin> pythonxy.dll to also provide the sources of GMP.

As I understand it the proposal was to allow people to substitute GMP for
Python's long implementation.  Just deliver binaries with the Python long
version if you don't want to distribute the GMP source.  OTOH, it should be
no big deal to drop a zip archive of the GMP sources which correspond to the
code bound into the DLL.  OTOOH, doesn't Windows support dynamic linking?
Can't pythonxy.dll dynamically link to a gmpMN.dll?

Skip

From thomas at python.org  Tue Nov  4 02:03:15 2008
From: thomas at python.org (Thomas Wouters)
Date: Tue, 4 Nov 2008 02:03:15 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <18703.39241.442461.116001@montanaro-dyndns-org.local>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
Message-ID: <9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>

On Tue, Nov 4, 2008 at 01:37, <skip at pobox.com> wrote:

>
>    Benjamin> The main objection is that GMP is licensed under LGPL which I
>    Benjamin> believe conflicts with Python's very open license.
>
>    >> If GMP itself isn't included with Python how can there be a licensing
>    >> issue?
>
>     Martin> On Windows, the GMP binaries would be incorporated into
>    Martin> pythonxy.dll.  This would force anybody providing a copy of
>    Martin> pythonxy.dll to also provide the sources of GMP.
>
> As I understand it the proposal was to allow people to substitute GMP for
> Python's long implementation.  Just deliver binaries with the Python long
> version if you don't want to distribute the GMP source.  OTOH, it should be
> no big deal to drop a zip archive of the GMP sources which correspond to
> the
> code bound into the DLL.  OTOOH, doesn't Windows support dynamic linking?
> Can't pythonxy.dll dynamically link to a gmpMN.dll?
>

Neither of those (shipping sources or dynamically linking to GMP) would
solve the LGPL issue. People who distribute that build of Python would still
be held by the LGPL -- such as shipping any sources that they embed that
Python into.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081104/f4101a5a/attachment.htm>

From andrewm at object-craft.com.au  Tue Nov  4 02:32:28 2008
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Tue, 04 Nov 2008 12:32:28 +1100
Subject: [Python-Dev] Python2.5 _sre deepcopy regression?
In-Reply-To: <20081023022432.B262B59C019@longblack.object-craft.com.au> 
References: <20081023022432.B262B59C019@longblack.object-craft.com.au>
Message-ID: <20081104013228.846126007E1@longblack.object-craft.com.au>

<prod> I posted this week ago, but haven't seen any comments. Issue
416670 is probably the most relevent ticket.

The buggy changeset I mention, 38430 on the release24-maint branch is
one that had been forward and back-ported for a while. I haven't found
the motivation for that change, but it hasn't been applied to any version
of Python later than 2.4.

>In version of Python prior to 2.5, it would appear that deepcopying
>compiled regular expressions worked by accident:
>
>2.4:
>
>    >>> copy.deepcopy(re.compile(''))
>    <_sre.SRE_Pattern object at 0xb7d53ef0>
>
>2.5:
>
>    >>> copy.deepcopy(re.compile(''))
>    Traceback (most recent call last):
>      File "<stdin>", line 1, in <module>
>      File "/usr/lib/python2.5/copy.py", line 173, in deepcopy
>        y = copier(memo)
>    TypeError: cannot deepcopy this pattern object
>
>I say "by accident", since the SRE_Pattern object in 2.4 has
>a __deepcopy__ method which raises the "cannot deepcopy this pattern
>object" TypeError, however this method isn't found by copy.deepcopy()
>in 2.4, and copy.deepcopy() falls back to using the pickle logic.
>
>The _sre source has #ifdef-out support for __deepcopy__, issue 416670
>has the gory details:
>
>    http://bugs.python.org/issue416670
>
>Changeset 38430 on the release24-maint branch introduced the changes
>that stopped SRE_Pattern.__deepcopy__ being found. r38430 was a patch
>forward ported from 2.3, but never ported to the trunk (probably a good
>thing, too).
>
>Thoughts?
-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From stephen at xemacs.org  Tue Nov  4 03:07:53 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 04 Nov 2008 11:07:53 +0900
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
	<9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>
Message-ID: <871vxsutty.fsf@uwakimon.sk.tsukuba.ac.jp>

Thomas Wouters writes:

 > Neither of those (shipping sources or dynamically linking to GMP) would
 > solve the LGPL issue. People who distribute that build of Python would still
 > be held by the LGPL -- such as shipping any sources that they embed that
 > Python into.

No, that's exactly what the "L" in "LGPL" means.  You only have to
ship the sources to any object module (library or program) actually
containing GMP code, including any modifications you make to them.  So
if GMP is kept entirely in a separate library and you don't borrow any
code from it, the rest of your code is not affected by the LGPL.

This includes any marshalling code that does nothing but load the GMP
library and connect it up to Python's internal API; that need not be
LGPL unless you add it to the GMP library.

The "problem" with the LGPL is the administrative burden of tracking
which sources need to be distributed under what conditions, not that
it insinuates itself into your own code.


From josiah.carlson at gmail.com  Tue Nov  4 03:11:43 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Mon, 3 Nov 2008 18:11:43 -0800
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <d2155e360811031059x26245c8ei7c0fe96e53deffaa@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<gedmtg$eop$1@ger.gmane.org> <490AA4B9.5060307@gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
	<loom.20081102T180853-494@post.gmane.org>
	<18702.15638.223872.706216@montanaro-dyndns-org.local>
	<e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>
	<d2155e360811031059x26245c8ei7c0fe96e53deffaa@mail.gmail.com>
Message-ID: <e6511dbf0811031811i7d18fa2w7782a9d99666e3b4@mail.gmail.com>

On Mon, Nov 3, 2008 at 10:59 AM, Curt Hagenlocher <curt at hagenlocher.org>wrote:

> On Mon, Nov 3, 2008 at 11:50 AM, Josiah Carlson <josiah.carlson at gmail.com>wrote:
>
>> On Sun, Nov 2, 2008 at 3:51 PM, <skip at pobox.com> wrote:
>>
>>>
>>>    Antoine> I think it is important to remind that the GIL doesn't
>>> prevent
>>>    Antoine> (almost) true multithreading. The only thing it prevents is
>>>    Antoine> full use of multi-CPU resources in a single process.
>>>
>>> I believe everyone here knows that.  I believe what most people are
>>> clamoring for is to make "full use of their multi-CPU resources in a
>>> single
>>> process".
>>>
>>
>> Which is, arguably, silly.  As we've seen in the last 2 months with
>> Chrome, multiple processes for a single "program" is actually a pretty good
>> idea.  With the multiprocessing module in the standard library offering a
>> threading-like interface, people no longer have any excuses for not fully
>> exploiting their multiple cores in Python.
>>
>
> There is no shortage of algorithms (such as matrix multiplication) that are
> parallelizable but not particularly good candidates for an IPC-based
> multiprocessing paradigm.
>

Ahh, but those algorithms aren't going to be written in Python; they are
going to be written in C, and are going to manipulate memory directly.  With
such things, you can use standard Python threads, call into your C runtime,
and release the GIL.  Alternatively, you can use the mmap module to store
your data, shared across multiple processes, using the same direct-memory
access as you would with multiple threads and GIL release.

Also, most local-only communications primitives (named pipes, anonymous
pipes, unix domain sockets, ...) use zero/one copy implementations, so as
long as your RPC isn't slow, you can do pretty well even on the Python side
(especially if you pre-allocate your receive buffer, and fill in the data as
you receive it; this is pretty much what mutablebytes was created for, now
we just need a proper memoryview).

So again, I claim that not using multiple processes for your multi-core
machine in order to use your multiple cores to their full extent is silly.
 As an aside, Python array.array() instances have a char* and length, so if
you are careful, you can create an array.array object from an mmap pointer,
and get fairly decent performance even in Python with shared memory.
 Someone should probably look into allowing array.array() to take a
read/readwrite memoryview as an argument to support such things, as well as
allowing mmaps to be passed via multiprocessing (if they aren't already
supported).

 - Josiah
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081103/ce22a0d8/attachment.htm>

From stephen at xemacs.org  Tue Nov  4 02:59:00 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 04 Nov 2008 10:59:00 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4222a8490811031045t6eacb4a6qfd425b7224a033e4@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<4222a8490811031045t6eacb4a6qfd425b7224a033e4@mail.gmail.com>
Message-ID: <873ai8uu8r.fsf@uwakimon.sk.tsukuba.ac.jp>

Jesse Noller writes:

 > I don't see how git can be considered given poor windows support -
 > compilation on OS/X can be a bear too.

I can't speak to the "poor Windows support", but I've been compiling
both in MacPorts (pretty much every MacPorts release, which is like
weekly) and from the kernel.org repo (intermittently, 1-3 months
apart) on Mac OS X since git 0.99 and the only trouble is with the
documentation (and the never-ending stream of bugs in port and/or the
Portfile, of course).  I doubt this is a problem on Mac OS X any more,
if it ever was.

 > Also the ability to completely nuke the local-work-copies commit
 > history ("cleaning it up") worries me, but I'm also paranoid.

bzr and hg both offer a similar feature.  They just make it less
convenient and possibly more dangerous to use (because the operation
that cleans up history also destroys it, where in git cleaning up
history creates garbage).  People who dislike merge turds will use it
and there is nothing you can really do about that.

git is at least as safe in this respect as any other dVCS, because it
allows you to get the unwanted history out of your face, but it then
becomes garbage remains in the repository until you actually do a
git-gc --prune.  It's easy to collect dangling heads and check whether
you need them.  Again, some people will throw them away, but there's
nothing you can do about that.  You don't have to throw away your own
history, though.

From brett at python.org  Tue Nov  4 03:33:51 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 18:33:51 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <C1083904-EE78-42F5-8AFD-CC8392FD37E4@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>
	<bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>
	<C1083904-EE78-42F5-8AFD-CC8392FD37E4@python.org>
Message-ID: <bbaeab100811031833l3dc29672lf182ded076fe9182@mail.gmail.com>

On Mon, Nov 3, 2008 at 14:58, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 3, 2008, at 5:03 PM, Brett Cannon wrote:
>
>> As I said, one of the scenarios already says patches can be whatever
>> the DVCS supports the best; plain diffs, branches, etc. And the
>> comments for that scenario will point out any perks from that feature.
>
> It's actually a profound improvement not to have to deal with traditional
> patch files as much.  The feel... dead somehow, compared to a live branch
> which can track changes to the trunk, etc.
>

Which I am sure will be pointed out in the scenario. I just want some
file I can attach to an issue, period. After that the perks of what
that file contains in terms of history, etc., can be enumerated.

>> As for the "via mail" feature, I use Gmail so it doesn't mean anything to
>> me. =)
>
> It should though.  In Bazaar, a bundle is essentially the revisions in your
> branch that are packaged to transport over RFC 2821 and 2822. :)  That opens
> up some really cool abilities, such as sending a bundle to a buildbot or
> patch queue manager for automated testing.

But this just smells of premature optimization to me. For instance, as
of this exact moment Martin is basically the only person staying on
top of the tracker. He also is typically the buildbot person as well.
And this is not from a lack of trying to get help with the tracker
multiple times. It's just people just don't stick around, as you know.

Trying to expand our buildbot infrastructure to accept patches to test
out or some patch queue manager might be nice, but I want to be
realistic with what we have now. That's why I am not worrying about
this email feature; until I know that we will actually use it and have
the manpower to implement it I consider it a perk that might never be
used. It's all great to dream about the future and what could be, but
that is far beyond just trying to get us to switch to a DVCS and is
just icing on the cake.

-Brett

From musiccomposition at gmail.com  Tue Nov  4 03:35:12 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Mon, 3 Nov 2008 20:35:12 -0600
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <ca471dc20811031604j5c8b940bndd81b8e253ebc1be@mail.gmail.com>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
	<EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>
	<ca471dc20811031604j5c8b940bndd81b8e253ebc1be@mail.gmail.com>
Message-ID: <1afaf6160811031835r5868842bj82a70aa26cec2c47@mail.gmail.com>

On Mon, Nov 3, 2008 at 6:04 PM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Nov 3, 2008 at 3:49 PM, Barry Warsaw <barry at python.org> wrote:
>> On Nov 3, 2008, at 6:39 PM, Benjamin Peterson wrote:
>>
>>> Grepping through Python's sources tells me that we have over 2,000
>>> "XXX" comments. The thing that irks me about them is that the have a
>>> very slow rate of being resolved, since they usually act more as
>>> "notes to self" rather than easily attainable tasks.
>>>
>>> So, I propose that we adopt a policy similar to Twisted's: All "XXX"
>>> comments must have an issue in the tracker and an accompanying link
>>> with the source code. That way we'll have a forum for discussing the
>>> changes and deciding whether they are reasonable enough to
>>> implemented.
>>
>> We have a very similar policy here at work.  Sadly I don't think it does
>> much on actually resolving the issues.  XXX bugs tend to be pretty minor in
>> the scheme of things and they're almost always lower on the priority list
>> than everything else.
>>
>> That being said, we recognize that XXX bugs incur "technical debt" and
>> reducing that debt is something we're devoting Real Time to doing.
>>
>> I see that Guido is not keen on the idea, and I'm not sure my observations
>> help sway things one way or the other.  OTOH, it would be nice if at least
>> we always add our own identifier (initials, nick, email address) and a date
>> to the XXX so we at least know who was talking about what.
>
> At Google we use TODO(<username>): Blah, blah. I don't think ading a
> date would help much but I wouldn't be opposed to this.

That reminds me of the other problem with XXX. They get out of date
without being removed. I've even seen an XXX comment followed by
another asking, "Is this still valid?".

You're probably right that adding a date would help much unless we
agreed to remove old comments with little value after a year or so.


-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From brett at python.org  Tue Nov  4 03:35:56 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 18:35:56 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <873ai8uu8r.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<4222a8490811031045t6eacb4a6qfd425b7224a033e4@mail.gmail.com>
	<873ai8uu8r.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <bbaeab100811031835q156334d3q7c70f0755cb41167@mail.gmail.com>

On Mon, Nov 3, 2008 at 17:59, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Jesse Noller writes:
>
>  > I don't see how git can be considered given poor windows support -
>  > compilation on OS/X can be a bear too.
>
> I can't speak to the "poor Windows support", but I've been compiling
> both in MacPorts (pretty much every MacPorts release, which is like
> weekly) and from the kernel.org repo (intermittently, 1-3 months
> apart) on Mac OS X since git 0.99 and the only trouble is with the
> documentation (and the never-ending stream of bugs in port and/or the
> Portfile, of course).  I doubt this is a problem on Mac OS X any more,
> if it ever was.
>

Well, when I decided to compile git from scratch on OS X when Neil
posted the git mirror it took bloody forever thanks to the docs having
an insane build requirement.

-Brett

From tim.peters at gmail.com  Tue Nov  4 03:37:28 2008
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 3 Nov 2008 21:37:28 -0500
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <490F8744.8010302@v.loewis.de>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
Message-ID: <1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>

[Gregory P. Smith]
>> One optimization that could be done to the existing Python longobject
>> code is to allow it to use larger digits.  Currently it is hardcoded
>> to use 15bit digits.
>>
>> The most common desktop+server CPUs in the world (x86) all support
>> efficient 32bit*32bit -> 64bit multiply so there is no good reason to
>> limit python itself to 15bit digits when on either x86 or x86_64.

[Martin v. L?wis]
> Perhaps Tim Peters should also comment here - but if you can come up
> with a patch that does that in a portable way, I would be in favor.
> The challenge, ISTM, is to make it still compile on all systems
> (regardless of how inefficient it might be on minor platforms).

Eh -- the strong points of Python's long implementation have always
been reliability and portability.  Both are greatly aided by sticking
to operations spellable in standard C89, and by avoiding
platform-specific trickery & #ifdef'ery.  So, no, I'm not keen on
this.  Note that while 32x32->64 multiply is supported by x86 HW, that
doesn't mean there's a uniform way to get at this HW capability across
C compilers.  In contrast, (at least) efficient HW 15x15->30 multiply
is universally spelled in C via "i*j" :-)

A similar gripe applies to schemes to replace the long implementation
by GMP (optionally or not):  it adds complexity to the code.  I like
GMP myself, but am happy using one of the Python GMP wrappers when I
/want/ GMP (as timings in other messages show, using GMP is a speed
loser before ints grow "really big").

Indeed, if I had it to do over again, I would balk even at adding
Karatsuba multiplication to Python (it added extra complexity with no
real payback, given that those who care about the speed of very long
int multiplication are far better off using a GMP wrapper anyway).

grouchily y'rs  - tim

From brett at python.org  Tue Nov  4 03:40:52 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 18:40:52 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <643d90130811031438o7c0609faya8cd60b61e23593f@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>
	<bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>
	<643d90130811031438o7c0609faya8cd60b61e23593f@mail.gmail.com>
Message-ID: <bbaeab100811031840ud569667p10b84ed11da88c9d@mail.gmail.com>

On Mon, Nov 3, 2008 at 14:38, Gustavo Niemeyer <gustavo at niemeyer.net> wrote:
>> Both. First and foremost I am looking for any scenarios people are
>> using now for svn that I didn't cover. After that I can probably add
>> some DVCS-specific things. But the problem with that is my DVCS
>> experience is limited and thus I don't want to add a scenario that
>
> So try to listen to people that actually use these tools and are
> trying to help you.
>

I am and why I will add either Thomas' or Barry's scenario.

>> seems whizbang cool but in real life is never used; premature
>> optimization and all that.
>
> As I mentioned early in my mail, all the scenarios I described were
> specifically meant to expose common situations that we go through in
> our workflow.  I'm not picking weird things to try favoring any
> specific tool, even because I believe all tools should support these
> very basic desires.
>

I didn't think you were picking scenarios that were odd, but I am sure
you also have an IT staff to make sure your workflow can support all
of that (see my reply to Barry).

>> As I said, one of the scenarios already says patches can be whatever
>> the DVCS supports the best; plain diffs, branches, etc. And the
>> comments for that scenario will point out any perks from that feature.
>
> """
> This scenario to meant to represent the steps required for a non-core
> developer who has a one-off patch they want to create from a read-only
> checkout.
> """
>
> Ok, so would you mind to expand that to describe that the tool should
> send the history of the "one-off patch", and what it means to not send
> it?
>

"Send" as in automatically submit?

> Otherwise you're really comparing oranges and apples.  If I send you
> "svn diff" or "bzr diff", you get a plain diff.  Merging this will not
> carry the history of who made the chances, what were the incremental
> steps, and will handle conflicts without context, which may be way
> more boring to perform than when you actually have full history
> information.  That's not the same as using "bzr send".
>

Hey, if what ``bzr send`` generates can be uploaded to
bugs.python.org, then that is what will be listed in the scenario.

> Again, that's a real world situation, not something I'm making up to
> favor bzr.  I'm pretty sure hg should have something similar too (and
> git does for sure).
>
>> As for the "via mail" feature, I use Gmail so it doesn't mean anything to me. =)
>
> Please ask Tim to teach you how the "mail" word got in "Gmail".  ;-)

I have learned to ignore my uncle for years so it won't do much good. =)

-Brett

From brett at python.org  Tue Nov  4 03:40:52 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 3 Nov 2008 18:40:52 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <643d90130811031438o7c0609faya8cd60b61e23593f@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>
	<bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>
	<643d90130811031438o7c0609faya8cd60b61e23593f@mail.gmail.com>
Message-ID: <bbaeab100811031840ud569667p10b84ed11da88c9d@mail.gmail.com>

On Mon, Nov 3, 2008 at 14:38, Gustavo Niemeyer <gustavo at niemeyer.net> wrote:
>> Both. First and foremost I am looking for any scenarios people are
>> using now for svn that I didn't cover. After that I can probably add
>> some DVCS-specific things. But the problem with that is my DVCS
>> experience is limited and thus I don't want to add a scenario that
>
> So try to listen to people that actually use these tools and are
> trying to help you.
>

I am and why I will add either Thomas' or Barry's scenario.

>> seems whizbang cool but in real life is never used; premature
>> optimization and all that.
>
> As I mentioned early in my mail, all the scenarios I described were
> specifically meant to expose common situations that we go through in
> our workflow.  I'm not picking weird things to try favoring any
> specific tool, even because I believe all tools should support these
> very basic desires.
>

I didn't think you were picking scenarios that were odd, but I am sure
you also have an IT staff to make sure your workflow can support all
of that (see my reply to Barry).

>> As I said, one of the scenarios already says patches can be whatever
>> the DVCS supports the best; plain diffs, branches, etc. And the
>> comments for that scenario will point out any perks from that feature.
>
> """
> This scenario to meant to represent the steps required for a non-core
> developer who has a one-off patch they want to create from a read-only
> checkout.
> """
>
> Ok, so would you mind to expand that to describe that the tool should
> send the history of the "one-off patch", and what it means to not send
> it?
>

"Send" as in automatically submit?

> Otherwise you're really comparing oranges and apples.  If I send you
> "svn diff" or "bzr diff", you get a plain diff.  Merging this will not
> carry the history of who made the chances, what were the incremental
> steps, and will handle conflicts without context, which may be way
> more boring to perform than when you actually have full history
> information.  That's not the same as using "bzr send".
>

Hey, if what ``bzr send`` generates can be uploaded to
bugs.python.org, then that is what will be listed in the scenario.

> Again, that's a real world situation, not something I'm making up to
> favor bzr.  I'm pretty sure hg should have something similar too (and
> git does for sure).
>
>> As for the "via mail" feature, I use Gmail so it doesn't mean anything to me. =)
>
> Please ask Tim to teach you how the "mail" word got in "Gmail".  ;-)

I have learned to ignore my uncle for years so it won't do much good. =)

-Brett

From guido at python.org  Tue Nov  4 03:47:07 2008
From: guido at python.org (Guido van Rossum)
Date: Mon, 3 Nov 2008 19:47:07 -0700
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <1afaf6160811031835r5868842bj82a70aa26cec2c47@mail.gmail.com>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
	<EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>
	<ca471dc20811031604j5c8b940bndd81b8e253ebc1be@mail.gmail.com>
	<1afaf6160811031835r5868842bj82a70aa26cec2c47@mail.gmail.com>
Message-ID: <ca471dc20811031847n706aeba9r5c3378ae384cc12b@mail.gmail.com>

On Mon, Nov 3, 2008 at 7:35 PM, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Mon, Nov 3, 2008 at 6:04 PM, Guido van Rossum <guido at python.org> wrote:
>> On Mon, Nov 3, 2008 at 3:49 PM, Barry Warsaw <barry at python.org> wrote:
>>> On Nov 3, 2008, at 6:39 PM, Benjamin Peterson wrote:
>>>
>>>> Grepping through Python's sources tells me that we have over 2,000
>>>> "XXX" comments. The thing that irks me about them is that the have a
>>>> very slow rate of being resolved, since they usually act more as
>>>> "notes to self" rather than easily attainable tasks.
>>>>
>>>> So, I propose that we adopt a policy similar to Twisted's: All "XXX"
>>>> comments must have an issue in the tracker and an accompanying link
>>>> with the source code. That way we'll have a forum for discussing the
>>>> changes and deciding whether they are reasonable enough to
>>>> implemented.
>>>
>>> We have a very similar policy here at work.  Sadly I don't think it does
>>> much on actually resolving the issues.  XXX bugs tend to be pretty minor in
>>> the scheme of things and they're almost always lower on the priority list
>>> than everything else.
>>>
>>> That being said, we recognize that XXX bugs incur "technical debt" and
>>> reducing that debt is something we're devoting Real Time to doing.
>>>
>>> I see that Guido is not keen on the idea, and I'm not sure my observations
>>> help sway things one way or the other.  OTOH, it would be nice if at least
>>> we always add our own identifier (initials, nick, email address) and a date
>>> to the XXX so we at least know who was talking about what.
>>
>> At Google we use TODO(<username>): Blah, blah. I don't think ading a
>> date would help much but I wouldn't be opposed to this.
>
> That reminds me of the other problem with XXX. They get out of date
> without being removed.

Not that often.

> I've even seen an XXX comment followed by
> another asking, "Is this still valid?".

That's a clue though that the code being commented is extremely hairy
-- if the second commenter couldn't deduce whether the first comment
was still valid, a double red flag is well worth having!

> You're probably right that adding a date would help much unless we
> agreed to remove old comments with little value after a year or so.

I disagree. They should be removed when the issue they refer to is
removed. No sooner, no later. Simply removing every XXX comment older
than a year would not be helpful. The code base is so large that over
2000 XXX doesn't faze me particular. There are over 4000 files in the
Python 2.6 source code tarball!

The right thing to do with XXX comments is to read them when you're in
their vicinity, and to act when the urge becomes too strong to deal
with any one in particular. Dealing with them en masse is just asking
for a migraine.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Tue Nov  4 04:06:46 2008
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 3 Nov 2008 19:06:46 -0800
Subject: [Python-Dev] XXX do we need a new policy?
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com><EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org><ca471dc20811031604j5c8b940bndd81b8e253ebc1be@mail.gmail.com><1afaf6160811031835r5868842bj82a70aa26cec2c47@mail.gmail.com>
	<ca471dc20811031847n706aeba9r5c3378ae384cc12b@mail.gmail.com>
Message-ID: <8B7649D05482432CA836BBB9501637D4@RaymondLaptop1>

From: "Guido van Rossum" <guido at python.org>
> The right thing to do with XXX comments is to read them when you're in
> their vicinity, and to act when the urge becomes too strong to deal
> with any one in particular. Dealing with them en masse is just asking
> for a migraine.

I concur.


Raymond

From stephen at xemacs.org  Tue Nov  4 06:21:08 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 04 Nov 2008 14:21:08 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
Message-ID: <87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>

Brett Cannon writes:

 > I have yet to have met anyone who thinks git is great while having
 > used another DVCS as extensively (and I mean I have never found
 > someone who has used two DVCSs extensively).

When XEmacs was considering changing from CVS, I used Darcs as my
primary VCS for about 4 months, including a mammoth (>1MB patch)
merge.  Since Dec 2007, Mercurial has been the official XEmacs VCS.
Nowadays I'm more management than developer but I love git, and will
not use either Darcs or Mercurial for any project where git is an
option.  (Somebody else did the work of moving the CVS history, so
they got to choose Mercurial -- in hindsight, it would have been worth
doing the work....)  I don't know if that counts as "extensive".

I like git because
(1) I like the model of exposing the commit DAG directly as a graph of
    objects.
(2) It's very fast.
(3) It does not promote a particular style of development.  Both
    merging parallel branch tips and rebasing to serialize branches on
    the trunk are well-supported.  (Mercurial and especially Bazaar do
    support the merging style better than git does.)
(4) Branching is cheap and fast.  I typically have a subbranch for
    almost every typo/minor fix I do in a working branch, which I then
    cherrypick into the mainline.  (This workflow avoids merge
    conflicts due to cherrypicking typo fixes directly from the
    subbranch.  Mercurial makes such cherrypicking relatively
    inconvenient, and I often make mistakes and commit too much in the
    wrong branch.  In Darcs this can be very painful because of
    dependencies the cherrypick drags in.)  Switching branches is a
    sub-second operation until the diff gets to be about 15-20 files.
(5) All branches are explicit.  You commit to the current branch.
(6) Files to commit must be named in the commit command, marked with
    an add command, or included via the --all option.
(7) A fairly natural, if ugly, syntax for specifying revisions,
    ranges, and various operations on ranges in log and diff
    commands.  No "revision numbers" that vary randomly according to
    workspace.

I dislike Darcs because
(1) The DAG is implicit.
(2) It's slow.
(3) I never know what I'll get when I ask to pull a single patch;
    Darcs's criteria for dependency are opaque, at least to me.
(4) It's hard to script and really likes to be used interactively.

I dislike Mercurial because
(1) It strongly encourages a commit-then-merge style which results in
    a large number of "merge turds" in the history.  Since most
    "merges" succeed because the changes are in different files, these
    are very annoying to me.
(2) The default revision numbering typically results in rather bizarre
    diffs near merges, but there is no easy way to specify a
    particular parent (except the first) without looking up the log.
(3) Commits everything in the workspace by default.
(4) Commit is silent by default, so you don't realize how much you
    have committed until you push ... and have succeeded so you can no
    longer roll it back safely.
(5) Creates new branches without being asked, which then need to be
    merged, thus strongly encouraging the commit-then-merge style.
(6) I don't trust its compute-ancestors-separately-per-file merge
    algorithm.  If this really works, there's nothing wrong in
    principle with CVS!
(7) A lot of features require plugins, and the result is command
    proliferation, though unlike git only "porcelain" is exposed.

I haven't used Bazaar beyond "bzr pull" of Mailman once a week or so,
so I don't dislike it.  Things I have observed or have seen discussed
on the bazaar mailing list that you might want to consider:
(1) The UI is as baroque as git's, once you consider all the plugins
    and GUIs that are available.  Lots of different workspace styles
    (ordinary branching, stacked branching, looms -- similar to
    quilts?, lightweight checkouts, ...) are supported with a
    corresponding increase in subcommand count and/or options.
(2) New repo formats are added frequently, and taking advantage of new
    features often requires upgrading your repo format.  So-called
    lightweight checkouts can be especially annoying as they involve
    leaving the history on the server, making distributed work
    problematic.
(3) Bazaar is very good at supporting the kind of refactoring that
    involves lots of file/directory renames and/or splitting/combining.
(4) Bazaar is claimed to have especially good merging support.
(5) Bazaar has an idiosyncratic log format that displays branches and
    merges "nicely" by choosing a principal branch, and indenting
    subsidiary branches.  This view changes depending on the repo,
    AIUI.  Some people prefer to leave that to a separate command
    (a graphical DAG viewer or something like "git-show-branches").
(6) In some common use patterns (eg, "bzr log | less"), Bazaar
    currently does not scale.

 > >.  It is guaranteed to scale (unless Python gets to be
 > > significantly bigger and more active than Linux, at any rate) and it has
 > > a large, very technically capable, and supported user community already.
 > 
 > I think any of the DVCSs will scale. But I will be taking some
 > performance numbers so scalability will be taken into consideration.

On the contrary.  Bazaar is currently known *not* to scale, and the
bazaar developers have a number of hypotheses about why, and are
working hard on fixing the acknowledged problem.  Emacs made the
decision to use bzr "because it's a fellow GNU project" early this
year, but they're still using CVS because of ongoing pushback against
the performance problems of bzr.  Let's put it this way: on my iBook
G4, for the same Emacs repository (ie, containing the same subset of
versions), "gitk" puts up the whole DAG in living color in about 10
seconds, while "bzr log" takes almost 5 minutes to return the *first*
revision.  There are workarounds, of course, but the default form of
that command (and several others) is very slow in that repo.

My understanding is that to deal fully with these problems, the Bazaar
developers plan to change the repo file format.  Some progress has
been made, (small) quantitative improvements have been made, but AFAIK
bzr still has bad worst-case performance for some common operations on
moderately large repos (way smaller than the Linux kernel).

 > Well, we will see, but as of right now my use of git has left a nasty
 > taste in my mouth that will take a lot of proverbial mouthwash to get
 > rid of and allow it to be considered in this PEP.

It's your PEP, but if you don't take git seriously, I expect a lot of
people will refuse to take your PEP seriously.

N.B.  It is *not* obvious that you or the PSF should cater to those
people.  It is relatively simple, though of course somewhat annoying
and inconvenient, to set up a local bidirectional gateway between the
"official" dVCS and one's preferred one.  I think you probably do want
a compromise that everybody will use, but you should keep the "keep
your own repo in any format you want" alternative in mind as a way to
gauge how much claimed pain you should acknowledge.<wink>


From martin at v.loewis.de  Tue Nov  4 06:45:44 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 04 Nov 2008 06:45:44 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <18703.39241.442461.116001@montanaro-dyndns-org.local>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
Message-ID: <490FE188.5000401@v.loewis.de>

>     Martin> On Windows, the GMP binaries would be incorporated into
>     Martin> pythonxy.dll.  This would force anybody providing a copy of
>     Martin> pythonxy.dll to also provide the sources of GMP.
> 
> As I understand it the proposal was to allow people to substitute GMP for
> Python's long implementation.  Just deliver binaries with the Python long
> version if you don't want to distribute the GMP source.

Ah, as an option, it could work.

> OTOH, it should be
> no big deal to drop a zip archive of the GMP sources which correspond to the
> code bound into the DLL.

How would end users then extract the sources from the DLL? How would
they learn that they even have them in the first place?

Also, people would complain about the increase in size - it's 3.2MB
as a zip file (1,7M as .tar.bz2).

> OTOOH, doesn't Windows support dynamic linking?
> Can't pythonxy.dll dynamically link to a gmpMN.dll?

How would that help in not having to distribute the source? Dynamic
linking still requires that the code is available at run-time.

Regards,
Martin

From martin at v.loewis.de  Tue Nov  4 06:48:38 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 04 Nov 2008 06:48:38 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <871vxsutty.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>	<18703.25530.532307.815707@montanaro-dyndns-org.local>	<490F7385.5090302@v.loewis.de>	<18703.39241.442461.116001@montanaro-dyndns-org.local>	<9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>
	<871vxsutty.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <490FE236.5010009@v.loewis.de>

Stephen J. Turnbull wrote:
> Thomas Wouters writes:
> 
>  > Neither of those (shipping sources or dynamically linking to GMP) would
>  > solve the LGPL issue. People who distribute that build of Python would still
>  > be held by the LGPL -- such as shipping any sources that they embed that
>  > Python into.
> 
> No, that's exactly what the "L" in "LGPL" means.  You only have to
> ship the sources to any object module (library or program) actually
> containing GMP code, including any modifications you make to them.  So
> if GMP is kept entirely in a separate library and you don't borrow any
> code from it, the rest of your code is not affected by the LGPL.

Sure. However, when you ship pythonxy.dll to your users, and it links
with gmp.dll, you *will* have to ship gmp.dll to your users, as well -
they just won't accept having to download gmp.dll from elsewhere (they
may not even know what Python is). So then you have to include the
source (of GMP, not Python). The L is really only about "include source
just from the library (gmp), not the application (Python)".

Regards,
Martin

From martin at v.loewis.de  Tue Nov  4 07:04:26 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 04 Nov 2008 07:04:26 +0100
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
	<EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>
Message-ID: <490FE5EA.2010009@v.loewis.de>

> I see that Guido is not keen on the idea, and I'm not sure my
> observations help sway things one way or the other.  OTOH, it would be
> nice if at least we always add our own identifier (initials, nick, email
> address) and a date to the XXX so we at least know who was talking about
> what.

I find it fairly easy to use "svn annotate" to learn about the source
of an XXX comment. In many cases, the XXX comment is fairly obvious,
anyway, so knowing who added it, and when, doesn't provide much useful
information.

Regards,
Martin

From stephen at xemacs.org  Tue Nov  4 07:32:50 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 04 Nov 2008 15:32:50 +0900
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <490FE236.5010009@v.loewis.de>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
	<9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>
	<871vxsutty.fsf@uwakimon.sk.tsukuba.ac.jp>
	<490FE236.5010009@v.loewis.de>
Message-ID: <87od0wt2zx.fsf@uwakimon.sk.tsukuba.ac.jp>

"Martin v. L?wis" writes:
 > Stephen J. Turnbull wrote:
 > > Thomas Wouters writes:
 > > 
 > >  > Neither of those (shipping sources or dynamically linking to
 > >  > GMP) would solve the LGPL issue. People who distribute that
 > >  > build of Python would still be held by the LGPL -- such as
 > >  > shipping any sources that they embed that Python into.
 > > 
 > > No, that's exactly what the "L" in "LGPL" means.  You only have to
 > > ship the sources to any object module (library or program) actually
 > > containing GMP code, including any modifications you make to them.  So
 > > if GMP is kept entirely in a separate library and you don't borrow any
 > > code from it, the rest of your code is not affected by the LGPL.
 > 
 > Sure. However, when you ship pythonxy.dll to your users, and it links
 > with gmp.dll, you *will* have to ship gmp.dll to your users [and]
 > then you have to include the source (of GMP, not Python).

Right.  There is definitely an administrative cost here, one that
Python itself doesn't impose.  This may not be a big deal for some
downstream projects, but that is for each project to decide.  I agree,
it is quite reasonable for Python to prefer to avoid the issue by
allowing downstream to add GMP or not as it likes, and simplify its
own maintenance burden by only providing the simple portable good-
enough-for-government-work implementation.

Thomas's wording strongly suggested to me that he thought that the
presence of gmp.dll would force a project to include not only GMP
source, but Python source *and* even my Killer-App's source.  That's
not true, and that's the only issue I was trying to address.

From adriangolding at gmail.com  Tue Nov  4 09:16:38 2008
From: adriangolding at gmail.com (adrian golding)
Date: Tue, 4 Nov 2008 16:16:38 +0800
Subject: [Python-Dev] file open in python interpreter
In-Reply-To: <490F0333.5010803@gmail.com>
References: <d3dcda8c0811022304oecee3bs84227aa7308c381e@mail.gmail.com>
	<1afaf6160811030525t3cca1e7aha7def92b400bdfe0@mail.gmail.com>
	<1afaf6160811030529i76cc9893l1a5ebcde264a63f9@mail.gmail.com>
	<490F0333.5010803@gmail.com>
Message-ID: <d3dcda8c0811040016v78ede8f2o120b2fcf44067470@mail.gmail.com>

hi all,

(benjamin and nick thank you!) i have another question to ask something
about permissions for the python interpreter.  in my earlier post, i was
saying i want to measure the python script before it is parsed.  what
happens is that i will write the measurement of that script file to another
file (call is 'measurereq') i have, it gives a Permission denied error.

i have modified other programs to do the same measurement (eg, modified Bash
to measure .sh files) and it works fine.  So i am suspecting it has
something to do with the python files. ie, i'm thinking the Permission
denied error is not because of i'm unable to write on my 'measurereq' file,
but its because i'm unable to measure (read) the python file.

any clues? how can i go around this?
thanks - adrian

On Mon, Nov 3, 2008 at 9:57 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Benjamin Peterson wrote:
> > On Mon, Nov 3, 2008 at 7:25 AM, Benjamin Peterson
> > <musiccomposition at gmail.com> wrote:
> >> On Mon, Nov 3, 2008 at 1:04 AM, adrian golding <adriangolding at gmail.com>
> wrote:
> >>> hi all, i am trying to find out where is the part of the code in the
> python
> >>> interpreter that opens up the .py file and parses it. in particular, i
> am
> >>> trying to find the file open command in that file.  I greped and i just
> want
> >>> to make sure this is it: /Python-2.6/Parser/pgenmain.c
> >>> i am intending to take a hash measurement of the .py file just before i
> open
> >>> it to run the script.  is the above file the right place to call for
> the
> >>> measurement before the file open function?
> >> You want Parser/tokenizer.c.
> >
> > Sorry, that's not correct. opening of modules happens in
> > Python/import.c. There's also a case in Modules/main.c.
>
> And some indirect ones from runpy.py (via pkgutils) if you use the -m
> switch, or are executing a zipfile or directory.
>
> But for the specific case of an exact filename being provided on the
> command line, then main.c is the one the original poster will want to
> look at (line 567 to be exact).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081104/1d742944/attachment.htm>

From greg.ewing at canterbury.ac.nz  Tue Nov  4 09:24:05 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Nov 2008 21:24:05 +1300
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <20081103174901.GI32006@idyll.org>
References: <392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>
	<gecpks$cgc$1@ger.gmane.org> <gecqd4$g9g$1@ger.gmane.org>
	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>
	<4909EF67.5010104@trueblade.com>
	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk> <490E0747.8050401@behnel.de>
	<490EE0A0.2060700@ghaering.de>
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
	<20081103174901.GI32006@idyll.org>
Message-ID: <491006A5.3040206@canterbury.ac.nz>

C. Titus Brown wrote:

> Cython is a non-backwards-compatible fork of Pyrex, forked for the usual
> reasons [0].

As I see it, there are two main reasons for the fork:

(1) I prefer to develop slowly and carefully, whereas the
Cython people like to rush ahead and try out wild ideas.

(2) There's a difference in philosophy between Pyrex and
Cython. The goal of Pyrex is to provide an easy way to
write efficient extension modules, and exact compatibility
with Python is secondary. The Cython developers want
to create a true superset of Python, so compatibility is
more important to them.

Considering that Pyrex favours efficiency over compatibility
with Python where there's a conflict, it would perhaps be
more appropriate for standard library use, if such a thing
were to be considered at all -- which doesn't seem very
likely in the near future.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Nov  4 09:30:02 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Nov 2008 21:30:02 +1300
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
Message-ID: <4910080A.2070004@canterbury.ac.nz>

Brett Cannon wrote:

> I have yet to have met anyone who thinks git is great while having
> used another DVCS as extensively (and I mean I have never found
> someone who has used two DVCSs extensively).

It's entirely possible that there's only room for one
VCS at a time in the average human brain. I have enough
trouble keeping the intricacies of hg in my head from
one session to the next. I wouldn't relish the idea
of having to use another similar-but-not-quite-the-same
tool at the same time.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Tue Nov  4 09:47:54 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Nov 2008 21:47:54 +1300
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <490F7385.5090302@v.loewis.de>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
Message-ID: <49100C3A.1010001@canterbury.ac.nz>

Martin v. L?wis wrote:

> On Windows, the GMP binaries would be incorporated into pythonxy.dll.
> This would force anybody providing a copy of pythonxy.dll to also
> provide the sources of GMP.

I thought the suggestion was to provide a way of optionally
compiling Python to use GMP. The standard Python distribution
for Windows wouldn't be compiled that way, so it wouldn't
have any GMP-related code in its dll.

-- 
Greg

From thomas at python.org  Tue Nov  4 09:55:20 2008
From: thomas at python.org (Thomas Wouters)
Date: Tue, 4 Nov 2008 09:55:20 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <87od0wt2zx.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
	<9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>
	<871vxsutty.fsf@uwakimon.sk.tsukuba.ac.jp>
	<490FE236.5010009@v.loewis.de>
	<87od0wt2zx.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <9e804ac0811040055q72afb693mbce35aea7e6b3f41@mail.gmail.com>

On Tue, Nov 4, 2008 at 07:32, Stephen J. Turnbull <stephen at xemacs.org>wrote:

> "Martin v. L?wis" writes:
>  > Stephen J. Turnbull wrote:
>  > > Thomas Wouters writes:
>  > >
>  > >  > Neither of those (shipping sources or dynamically linking to
>  > >  > GMP) would solve the LGPL issue. People who distribute that
>  > >  > build of Python would still be held by the LGPL -- such as
>  > >  > shipping any sources that they embed that Python into.
>  > >
>  > > No, that's exactly what the "L" in "LGPL" means.  You only have to
>  > > ship the sources to any object module (library or program) actually
>  > > containing GMP code, including any modifications you make to them.  So
>  > > if GMP is kept entirely in a separate library and you don't borrow any
>  > > code from it, the rest of your code is not affected by the LGPL.
>  >
>  > Sure. However, when you ship pythonxy.dll to your users, and it links
>  > with gmp.dll, you *will* have to ship gmp.dll to your users [and]
>  > then you have to include the source (of GMP, not Python).
>
> Right.  There is definitely an administrative cost here, one that
> Python itself doesn't impose.  This may not be a big deal for some
> downstream projects, but that is for each project to decide.  I agree,
> it is quite reasonable for Python to prefer to avoid the issue by
> allowing downstream to add GMP or not as it likes, and simplify its
> own maintenance burden by only providing the simple portable good-
> enough-for-government-work implementation.
>
> Thomas's wording strongly suggested to me that he thought that the
> presence of gmp.dll would force a project to include not only GMP
> source, but Python source *and* even my Killer-App's source.  That's
> not true, and that's the only issue I was trying to address.
>

Ah, but not true according to who? If you read the LGPL, you'll note that it
specifically says that 'linking in' makes the LGPL library part of the work
(or rather, makes the whole a derived work.) Considering that, on Windows,
the linking is done at compiletime and only the loading is done at runtime,
it's unclear to me that this is not the case here. I don't know if RMS and
the FSF think of it this way, but I stopped trying to guess their thoughts
and actions a long time ago. It is certainly the case that such a
combination is enough to scare off corporate lawyers who aren't sure either
(most of them, I bet) and would advise against using that build of Python
because of the LGPL components. It's not without cost to Python, even
discounting the administrativia.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081104/1c65ebc1/attachment.htm>

From ncoghlan at gmail.com  Tue Nov  4 10:25:24 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 04 Nov 2008 19:25:24 +1000
Subject: [Python-Dev] file open in python interpreter
In-Reply-To: <d3dcda8c0811040016v78ede8f2o120b2fcf44067470@mail.gmail.com>
References: <d3dcda8c0811022304oecee3bs84227aa7308c381e@mail.gmail.com>	<1afaf6160811030525t3cca1e7aha7def92b400bdfe0@mail.gmail.com>	<1afaf6160811030529i76cc9893l1a5ebcde264a63f9@mail.gmail.com>	<490F0333.5010803@gmail.com>
	<d3dcda8c0811040016v78ede8f2o120b2fcf44067470@mail.gmail.com>
Message-ID: <49101504.9080507@gmail.com>

adrian golding wrote:
> hi all,
> 
> (benjamin and nick thank you!) i have another question to ask something
> about permissions for the python interpreter.  in my earlier post, i was
> saying i want to measure the python script before it is parsed.  what
> happens is that i will write the measurement of that script file to
> another file (call is 'measurereq') i have, it gives a Permission denied
> error.
> 
> i have modified other programs to do the same measurement (eg, modified
> Bash to measure .sh files) and it works fine.  So i am suspecting it has
> something to do with the python files. ie, i'm thinking the Permission
> denied error is not because of i'm unable to write on my 'measurereq'
> file, but its because i'm unable to measure (read) the python file.
> 
> any clues? how can i go around this?

You're getting out of python-dev territory now - I'm happy to help with
some tips on interpreter internals, but I'm not going to attempt to
debug-from-afar a modified version of the interpreter (particularly when
the problem relates to file permission issues which are highly OS and
file system dependent).

That said, the usual debugging tactic of divide and conquer (e.g. adding
trace calls to figure out exactly which step is failing, or just
calculating the hash without attempting to write it to disk) will
probably be your friend.

Failing that, you might want to try asking for further assistance on
comp.lang.python (aka python-list at python.org).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Nov  4 10:36:05 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 04 Nov 2008 19:36:05 +1000
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <ca471dc20811031544o551d4962q5bda42b742ce9ed1@mail.gmail.com>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
	<ca471dc20811031544o551d4962q5bda42b742ce9ed1@mail.gmail.com>
Message-ID: <49101785.5010709@gmail.com>

Guido van Rossum wrote:
> On Mon, Nov 3, 2008 at 3:39 PM, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> Grepping through Python's sources tells me that we have over 2,000
>> "XXX" comments. The thing that irks me about them is that the have a
>> very slow rate of being resolved, since they usually act more as
>> "notes to self" rather than easily attainable tasks.
>>
>> So, I propose that we adopt a policy similar to Twisted's: All "XXX"
>> comments must have an issue in the tracker and an accompanying link
>> with the source code. That way we'll have a forum for discussing the
>> changes and deciding whether they are reasonable enough to
>> implemented.
> 
> That seems excessively draconian. Even at Google we don't have a rule
> like that, and we're sure big on process and interlinked tools. If you
> want to, you could add a different magic keyword, but personally, I
> don't see XXX comments necessarily as "to be resolved" -- merely as
> flags for someone perusing the code looking to change it or digging
> for the cause of some problem to pay special attention.
> 

That's the way I generally use them (e.g. there's one in runpy.py
pointing out why _run_module_as_main isn't part of the module's public API).

Ben's right that any XXX comments that actually turn out to be real
problems should be placed in the issue tracker, but those that are just
"here be dragons!" warnings or "could we do this better?" musings aren't
really a problem. Even the "is this portable?" questions are OK until we
find a platform where the relevant code doesn't work.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From thomas at python.org  Tue Nov  4 10:39:27 2008
From: thomas at python.org (Thomas Wouters)
Date: Tue, 4 Nov 2008 10:39:27 +0100
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <490FE5EA.2010009@v.loewis.de>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
	<EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>
	<490FE5EA.2010009@v.loewis.de>
Message-ID: <9e804ac0811040139x3e328618n6c07399c604a31b0@mail.gmail.com>

On Tue, Nov 4, 2008 at 07:04, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> > I see that Guido is not keen on the idea, and I'm not sure my
> > observations help sway things one way or the other.  OTOH, it would be
> > nice if at least we always add our own identifier (initials, nick, email
> > address) and a date to the XXX so we at least know who was talking about
> > what.
>
> I find it fairly easy to use "svn annotate" to learn about the source
> of an XXX comment. In many cases, the XXX comment is fairly obvious,
> anyway, so knowing who added it, and when, doesn't provide much useful
> information.
>

Ah, but that's not true in Subversion: you just see who committed the XXX
comment, not who wrote it :-) I've been using the XXX(twouters) approach for
a while and it's pretty convenient because who wrote the comment changes how
you interpret the comment. See for instance r42313 and r42717 of
Python/ceval.c: if I'd known the original comment was added by Jeremy, I
would have interpreted the question correctly the first time around. (As it
was, Jeremy came up to me at PyCon, I think :)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081104/7ff0be6e/attachment.htm>

From brenocon at gmail.com  Tue Nov  4 10:52:53 2008
From: brenocon at gmail.com (Brendan O'Connor)
Date: Tue, 4 Nov 2008 01:52:53 -0800
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <490F0784.4050408@ghaering.de>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<gecqd4$g9g$1@ger.gmane.org>
	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>
	<4909EF67.5010104@trueblade.com>
	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk> <490E0747.8050401@behnel.de>
	<490EE0A0.2060700@ghaering.de>
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
	<490F0784.4050408@ghaering.de>
Message-ID: <fba9be1d0811040152t65641622sefb00abf6b00b075@mail.gmail.com>

>>> Is using Cython for anything in Modules/ really an option? In my limited
>>> experiments with it, I did like it.
>>>
>>> But using it for Python standard library stuff doesn't look quite right to
>>> me:
>>>
>>> - Option 1: distribute Cython with Python and integrate into build process
>>> -- Ouch!
>>
>> Can you be a bit more descriptive?
>
> Cython is still being worked on (intensively, it seems). Bundling it
> with Python means deciding on a particular version probably for
> an entire major release lifecycle (use Cython x.y.{newest} for
> Python 2.7, for example).

If we're talking about Cython for core libraries...

For the subset of python that also works in Cython, I've seen a 10-30%
performance speedups on (admittedly trivial) test code.  Conceivably,
*everything* could be Cython-compiled if you cared about performance
more than transparency.

Brendan
anyall.org

From glyph at divmod.com  Tue Nov  4 11:43:13 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Tue, 04 Nov 2008 10:43:13 -0000
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <ca471dc20811031544o551d4962q5bda42b742ce9ed1@mail.gmail.com>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
	<ca471dc20811031544o551d4962q5bda42b742ce9ed1@mail.gmail.com>
Message-ID: <20081104104313.6400.233138530.divmod.xquotient.871@weber.divmod.com>


On 3 Nov, 11:44 pm, guido at python.org wrote:
>On Mon, Nov 3, 2008 at 3:39 PM, Benjamin Peterson
><musiccomposition at gmail.com> wrote:
>>Grepping through Python's sources tells me that we have over 2,000
>>"XXX" comments.

>>So, I propose that we adopt a policy similar to Twisted's: All "XXX"
>>comments must have an issue in the tracker and an accompanying link
>>with the source code.

>That seems excessively draconian. (...) personally, I
>don't see XXX comments necessarily as "to be resolved" -- merely as
>flags for someone perusing the code looking to change it or digging
>for the cause of some problem to pay special attention.

For what it's worth, in my experience this is one of the least draconian 
areas of Twisted policy ;-).  I'll describe our reasoning for the policy 
and the effect that it's had.

If a developer working on a feature is inclined to type

    # somebody, do something to something else
    somebody.doSomething(somethingElse)

then the intent is typically to say "this is a confusing snippet of 
code, it bears explanation beyond what is clearly readable in the code 
itself".  Documenting the "why" and "what" happens in the docstring, so 
we only have comments in areas of code where the "how" or the "who" 
(i.e. "only JP and Glyph understand this") is difficult to understand. 
In Twisted, this is almost always some wicked busted operating system 
API, like how you get undocumented errnos out of send(), recv() or 
select() on some platform.

However, if they're inclined to type

    # TODO: somebody should really do more stuff here
    somebody.doSomething(somethingElse)

most commonly, what this means is "before I merge this feature, I really 
want to fix this potential issue so we don't need to deal with it in the 
future".  The policy applies to "XXX" because in my personal experience 
it's always used as a synonym for "TODO".

My subjective impression is that in 75% of the cases where a comment 
like this is caught in a code review, the original author of the branch 
had actually forgotten they had added a comment like this and will 
eagerly go and fix it, to avoid dealing with the issues that it might 
raise in a release in the future.  I can't recall a case where an author 
(myself included) was not happy to be alerted to a comment like this.

In a significant minority of cases, although the author has still 
usually forgotten (if they had remembered they would have filed a ticket 
preemptively, after all), the task in question is actually too big or 
too esoteric to bother with in the same branch, so it gets deferred. 
Filing a ticket is the natural extension of this, so that we don't lose 
track of the task in question.

The consequence of this policy has been higher quality bugfixes (because 
contributors don't forget what they're doing), and increased visibility 
into problems in our codebase(s) (since we use this on Divmod projects 
too) which directly translates into less duplicate grumbling about the 
same issue over and over again in pointless comments.

Also, I hear a lot of kvetching about Twisted policy, especially review 
latency (I hear you guys are familiar with that problem as well), but as 
I recall nobody has ever complained that this was too much work.  When 
we instituted the policy, as with most of our policies, there is a 
grandfather clause: we didn't go back and trawl through all of the 
existing XXX and TODO comments, we just made it so you couldn't add 
_new_ XXX and TODO comments without filing and linking to a ticket.

I haven't seen "XXX" used as a way to say "pay special attention" 
because our use of comments is extremely spare.  If there's a comment at 
all that means the code is sufficiently wacky that it warrants special 
attention.  Also, we don't write much code in C, so the preponderance of 
"sufficiently wacky" code is small ;).

Looking at the Python codebase, though, (specifically the 2.6 release), 
there are definitely "XXX" comments which do not appear to be actionable 
tasks by their authors.  Guido's certainly right about that: not all XXX 
comments in Python are "to be resolved".  But, a subjective assessment 
of the first couple of pages of such comments indicates that a majority 
of them are.  A lot of them are cryptic, too.  One of the benefits of 
review scrutiny of comments is a chance for the reviewer to pipe up and 
say "This comment is incomprehensible.  Can you explain it (on a ticket) 
in a way that someone else might fix it, rather than a note to 
yourself?" (with the implication being "a note to yourself that nobody, 
including you six months later, is ever going to understand").

I think that a Twisted reviewer encountering a comment like /* XXX From 
here until type is allocated, "return NULL" leaks bases! */ would 
probably say something like "XXX should probably be something more 
informative, like 'WARNING:' or 'CAREFUL:'".  Clearly if XXX is being 
used in this way there's no point in filing a ticket, and the comment 
here is perfectly clear; the policy is not intended to cover this case.

In fact it might be good to have a whole special comment convention for 
notes about the constantly horrific tightrope-walk over lava that is 
memory-management and object lifecycles in C.  If anything, "XXX" is a 
bit too easy to glaze over in a codebase with >2000 "XXX" comments.  But 
I don't write this type of code too often, so maybe to a more fluent 
speaker of C "/* XXX memory junk */" is an idiomatic thing to see.

Personally I'd recommend that Python adopt such a policy, but mainly for 
your own sanity.  It's not something that I think would make a huge 
difference to anyone outside the core development group, except perhaps 
to generate smaller, easier issues for new contributors to work on for 
practice.  Certainly not something I'm going to push for - my intent 
here is just to share our experience.

From glyph at divmod.com  Tue Nov  4 11:54:31 2008
From: glyph at divmod.com (glyph at divmod.com)
Date: Tue, 04 Nov 2008 10:54:31 -0000
Subject: [Python-Dev] XXX do we need a new policy?
In-Reply-To: <ca471dc20811031847n706aeba9r5c3378ae384cc12b@mail.gmail.com>
References: <1afaf6160811031539n6bd4825dl6f51e801433fd97f@mail.gmail.com>
	<EBD121A7-76E6-4F9D-B49F-2F3F9BB68360@python.org>
	<ca471dc20811031604j5c8b940bndd81b8e253ebc1be@mail.gmail.com>
	<1afaf6160811031835r5868842bj82a70aa26cec2c47@mail.gmail.com>
	<ca471dc20811031847n706aeba9r5c3378ae384cc12b@mail.gmail.com>
Message-ID: <20081104105431.6400.382514002.divmod.xquotient.890@weber.divmod.com>

On 02:47 am, guido at python.org wrote:
>I disagree. They should be removed when the issue they refer to is
>removed. No sooner, no later. Simply removing every XXX comment older
>than a year would not be helpful. The code base is so large that over
>2000 XXX doesn't faze me particular. There are over 4000 files in the
>Python 2.6 source code tarball!
>
>The right thing to do with XXX comments is to read them when you're in
>their vicinity, and to act when the urge becomes too strong to deal
>with any one in particular. Dealing with them en masse is just asking
>for a migraine.

I _strongly_ agree with what Guido is saying here.  I already mentioned 
this indirectly in my other post to this thread but I feel like it's 
worth emphasizing, especially given that I think that the Twisted core 
team (myself included) is widely perceived, for good or ill, as being 
process-crazed maniacs.

We have a sort of meta-policy of "never try to boil the ocean".  We 
started Twisted with a freewheeling, anything-goes commit policy, where 
if you had an account on the CVS server (my home desktop computer) you 
could do whatever you wanted: no code review, no compatibility, no 
testing.  There's still a lot of code left over from those days, and a 
lot of it works fine.

Every policy we have applies to _changes_ to the code; nobody has ever 
proposed a policy modification where we first fix all 1000 instances of 
X and then institute a policy on future X.  Quite the opposite: the 
whole point of most of our development process is to avoid needing to do 
broad, sweeping changes.

So, while I don't really care much one way or the other about Python's 
treatment of future XXX, I definitely think it would be a terrible idea 
to try to deal with all XXXes at once, or even start an ongoing process 
of gradually farming through the old XXXes; that would almost certainly 
be a waste of everyone's time.

From dickinsm at gmail.com  Tue Nov  4 12:00:11 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 4 Nov 2008 11:00:11 +0000
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
Message-ID: <5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>

[Martin v. L?wis]
> Perhaps Tim Peters should also comment here - but if you can come up
> with a patch that does that in a portable way, I would be in favor.
> The challenge, ISTM, is to make it still compile on all systems
> (regardless of how inefficient it might be on minor platforms).

I've just uploaded a patch to the issue tracker that allows Python
to use 30-bit digits on platforms having a 64-bit integer type:

http://bugs.python.org/issue4258

[Tim Peters]
> Note that while 32x32->64 multiply is supported by x86 HW, that
> doesn't mean there's a uniform way to get at this HW capability across
> C compilers.  In contrast, (at least) efficient HW 15x15->30 multiply
> is universally spelled in C via "i*j" :-)

If we're talking about standards and portability, doesn't "i*j" fail
on those (nonexistent?) platforms where the 'int' type is only 16-bits?
Shouldn't this be something like "(long)i * j" instead?

And for 32x32 -> 64, can't this simply be replaced by "(uint64_t)i * j",
where uint64_t is as in C99?  I'd hope that most compilers would
manage to turn this into the appropriate 32x32-bit hardware multiply.

I agree that very-long-integer optimizations probably don't really belong in
Python, but this patch should also provide significant benefits for short
and medium-sized integers.  I guess I need to go away and do some
benchmarking...

Mark

From greg.ewing at canterbury.ac.nz  Tue Nov  4 10:25:42 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 04 Nov 2008 22:25:42 +1300
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <490FE236.5010009@v.loewis.de>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
	<9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>
	<871vxsutty.fsf@uwakimon.sk.tsukuba.ac.jp>
	<490FE236.5010009@v.loewis.de>
Message-ID: <49101516.50909@canterbury.ac.nz>

Martin v. L?wis wrote:
> you *will* have to ship gmp.dll to your users, as well ... So then
 > you have to include the source (of GMP

Are you sure? I thought the source-provision requirements
of the *GPL licences only apply when you distribute a
*modified* version of something. Here you're just shipping
an unmodified GMP library.

-- 
Greg

From matthieu.brucher at gmail.com  Tue Nov  4 12:20:11 2008
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Tue, 4 Nov 2008 12:20:11 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <49101516.50909@canterbury.ac.nz>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
	<9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>
	<871vxsutty.fsf@uwakimon.sk.tsukuba.ac.jp>
	<490FE236.5010009@v.loewis.de> <49101516.50909@canterbury.ac.nz>
Message-ID: <e76aa17f0811040320w5f9bfc8fk6ae4f8fe7b44bee5@mail.gmail.com>

2008/11/4 Greg Ewing <greg.ewing at canterbury.ac.nz>:
> Martin v. L?wis wrote:
>>
>> you *will* have to ship gmp.dll to your users, as well ... So then
>
>> you have to include the source (of GMP
>
> Are you sure? I thought the source-provision requirements
> of the *GPL licences only apply when you distribute a
> *modified* version of something. Here you're just shipping
> an unmodified GMP library.

In fact you only have to provide a way to get the sources of the
library. You don't have to ship the library with the source.

As for the LGPL on Windows, wxWidgets does this for a long time now
and a program does not have to be open source or LGPL or whatever.

Matthieu
-- 
Information System Engineer, Ph.D.
Website: http://matthieu-brucher.developpez.com/
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn: http://www.linkedin.com/in/matthieubrucher

From stephen at xemacs.org  Tue Nov  4 12:31:19 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 04 Nov 2008 20:31:19 +0900
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <9e804ac0811040055q72afb693mbce35aea7e6b3f41@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
	<9e804ac0811031703l2dbadff8u7df50573a2413734@mail.gmail.com>
	<871vxsutty.fsf@uwakimon.sk.tsukuba.ac.jp>
	<490FE236.5010009@v.loewis.de>
	<87od0wt2zx.fsf@uwakimon.sk.tsukuba.ac.jp>
	<9e804ac0811040055q72afb693mbce35aea7e6b3f41@mail.gmail.com>
Message-ID: <87iqr3u3qw.fsf@uwakimon.sk.tsukuba.ac.jp>

Thomas Wouters writes:

 > Ah, but not true according to who? [...]

 > It is certainly the case that such a combination is enough to scare
 > off corporate lawyers who aren't sure either (most of them, I bet)
 > and would advise against using that build of Python because of the
 > LGPL components.

I don't think this is a matter of "according to whom", except where
you have a timid client and a FUD-ful lawyer, which is a plausible
case, I admit.  Copyright law is very literal about this kind of
thing, and does not distinguish between bytes that represent a
symbolic name for a function, and bytes that represent the address of
the function.  If the LGPL licenses symbolic references as long as the
function object code is not copied into your executable, then
numerical addresses are also OK as long as the function object code is
not copied into the executable you distribute.

I agree the language used in the LGPL is unfortunately imprecise.  But
the FSF's intent and understanding is quite clear about the issues you
raise.  See their License FAQ and David Turner's article about Java
vs. the LGPL:

http://www.fsf.org/licensing/licenses/gpl-faq.html#Prelinking
http://www.fsf.org/licensing/licenses/gpl-faq.html#LGPLv3ContributorVersion
http://www.fsf.org/licensing/licenses/lgpl-java.html

I see nothing anywhere to indicate that the FSF thinks that the LGPL
should work differently in a practical sense on Windows than it does
on GNU/Linux, and while Turner's article refers to language
environments rather than OSes, it's pretty clear that the assertion
that "the LGPL just works as intended and expected" applies to OS
environments too.

From skip at pobox.com  Tue Nov  4 13:00:03 2008
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 4 Nov 2008 06:00:03 -0600
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <490FE188.5000401@v.loewis.de>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
	<490FE188.5000401@v.loewis.de>
Message-ID: <18704.14659.600239.209777@montanaro-dyndns-org.local>


    >> OTOH, it should be no big deal to drop a zip archive of the GMP
    >> sources which correspond to the code bound into the DLL.

    Martin> How would end users then extract the sources from the DLL? How
    Martin> would they learn that they even have them in the first place?

I think you missed an implied comma in my statement.  I meant:

    1. distribute pythonxy.dll which binds to GMP.
    2. distribute a gmpmn.zip which contains the source for the version used
       in #1.

Skip

From Dennis.Benzinger at gmx.net  Tue Nov  4 14:30:14 2008
From: Dennis.Benzinger at gmx.net (Dennis Benzinger)
Date: Tue, 04 Nov 2008 14:30:14 +0100
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>	<geed1t$uo3$1@ger.gmane.org>
	<490AB8E2.9050004@trueblade.com>	<20081031102524.75dfd13f@resist.wooz.org>
	<9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
Message-ID: <49104E66.5020000@gmx.net>

Hi Thomas!

Am 03.11.2008 17:54, Thomas Wouters schrieb:
> [...]
> FWIW, I put one up this weekend, and it seems to be intact and OK.
> (bzr+ssh://pythonbzr at code.python.org/python/2.6/ or
> http://code.python.org/python/2.6/ )
> [...]

Can you update http://www.python.org/dev/bazaar/ ?
For example "Branch details" needs to be updated because the trunk
branch is no longer Python 2.6.

Thanks,
Dennis Benzinger

From santagada at gmail.com  Tue Nov  4 14:31:12 2008
From: santagada at gmail.com (Leonardo Santagada)
Date: Tue, 4 Nov 2008 11:31:12 -0200
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <e6511dbf0811031811i7d18fa2w7782a9d99666e3b4@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<gedmtg$eop$1@ger.gmane.org> <490AA4B9.5060307@gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
	<loom.20081102T180853-494@post.gmane.org>
	<18702.15638.223872.706216@montanaro-dyndns-org.local>
	<e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>
	<d2155e360811031059x26245c8ei7c0fe96e53deffaa@mail.gmail.com>
	<e6511dbf0811031811i7d18fa2w7782a9d99666e3b4@mail.gmail.com>
Message-ID: <EF6A1152-B162-444F-BA35-B38279EC4463@gmail.com>


On Nov 4, 2008, at 12:11 AM, Josiah Carlson wrote:

> There is no shortage of algorithms (such as matrix multiplication)  
> that are parallelizable but not particularly good candidates for an  
> IPC-based multiprocessing paradigm.
>
> Ahh, but those algorithms aren't going to be written in Python; they  
> are going to be written in C, and are going to manipulate memory  
> directly.  With such things, you can use standard Python threads,  
> call into your C runtime, and release the GIL.  Alternatively, you  
> can use the mmap module to store your data, shared across multiple  
> processes, using the same direct-memory access as you would with  
> multiple threads and GIL release.


That's the thing, right now you can't even use python to sketch one of  
those algorithms and for example compare performance with a non  
parallel version.

Also with four cores on common intel processor you might be happy with  
your algo being 4x faster while being quite easy to develop (not  
needing to use c).

I personally think this will be too much work on

--
Leonardo Santagada
santagada at gmail.com




From gustavo at niemeyer.net  Tue Nov  4 14:59:11 2008
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Tue, 4 Nov 2008 11:59:11 -0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>

Hello Stephen,

> I haven't used Bazaar beyond "bzr pull" of Mailman once a week or so,
> so I don't dislike it.  Things I have observed or have seen discussed
> on the bazaar mailing list that you might want to consider:
> (1) The UI is as baroque as git's, once you consider all the plugins
>    and GUIs that are available.  Lots of different workspace styles
>    (ordinary branching, stacked branching, looms -- similar to
>    quilts?, lightweight checkouts, ...) are supported with a
>    corresponding increase in subcommand count and/or options.
> (2) New repo formats are added frequently, and taking advantage of new
>    features often requires upgrading your repo format.  So-called
>    lightweight checkouts can be especially annoying as they involve
>    leaving the history on the server, making distributed work
>    problematic.

Both arguments strike me as odd.  Having the *option* to leave your
history on the server shouldn't be a problem, right?  The same goes
for (1): having more ways to use the tool isn't exactly a problem.

> My understanding is that to deal fully with these problems, the Bazaar
> developers plan to change the repo file format.  Some progress has
> been made, (small) quantitative improvements have been made, but AFAIK
> bzr still has bad worst-case performance for some common operations on
> moderately large repos (way smaller than the Linux kernel).

How large?  Which repositories?  Which operations?  Which version of Bazaar?

I'm sorry for jumping up on this, but you've mentioned one specific
issue, which is being looked at, and then have generalized here.
Without naming the issues, it's really hard to discuss anything.  It
is true that Bazaar performance was much worse in the past, and it is
true that it's being improved, and will continue to improve.  In case
Bazaar is a good deal for Python nowadays, it should only get better
over time.

-- 
Gustavo Niemeyer
http://niemeyer.net

From thomas at python.org  Tue Nov  4 15:01:24 2008
From: thomas at python.org (Thomas Wouters)
Date: Tue, 4 Nov 2008 15:01:24 +0100
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <49104E66.5020000@gmx.net>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<20081031102524.75dfd13f@resist.wooz.org>
	<9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
	<49104E66.5020000@gmx.net>
Message-ID: <9e804ac0811040601t402e8133x3907e9d758e597e8@mail.gmail.com>

On Tue, Nov 4, 2008 at 14:30, Dennis Benzinger <Dennis.Benzinger at gmx.net>wrote:

> Hi Thomas!
>
> Am 03.11.2008 17:54, Thomas Wouters schrieb:
> > [...]
> > FWIW, I put one up this weekend, and it seems to be intact and OK.
> > (bzr+ssh://pythonbzr at code.python.org/python/2.6/ or
> > http://code.python.org/python/2.6/ )
> > [...]
>
> Can you update http://www.python.org/dev/bazaar/ ?
> For example "Branch details" needs to be updated because the trunk
> branch is no longer Python 2.6.
>

Nope, I have no idea how to edit those pages properly, sorry. I'm sure
somebarryone does.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081104/d46b8406/attachment.htm>

From barry at python.org  Tue Nov  4 15:08:22 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 4 Nov 2008 09:08:22 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <0968BCF9-C996-4069-A0CC-A165F76844FA@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 4, 2008, at 12:21 AM, Stephen J. Turnbull wrote:

> (2) New repo formats are added frequently, and taking advantage of new
>    features often requires upgrading your repo format.  So-called
>    lightweight checkouts can be especially annoying as they involve
>    leaving the history on the server, making distributed work
>    problematic.

It should be noted that this is a client-side choice.  I can choose to  
have a heavyweight (i.e. full or partial) local revision history and  
work in a distributed way, while you choose to do a lightweight  
checkout and use the repository much as you currently use Subversion.   
The server side or master repo doesn't care.

I like that Bazaar has this option, though I personally don't use it,  
nor would I teach/introduce it early on, unless I was specifically  
trying to guide existing Subversion users into the world of dvcs.   
OTOH, I know people who like this style of working very much.

> (3) Bazaar is very good at supporting the kind of refactoring that
>    involves lots of file/directory renames and/or splitting/combining.

My understanding is that Bazaar tracks changes to directories whereas  
Mercurial (and git?) do not.  According to the Mercurial FAQs I've  
read, they claim this only affects empty directories.

> (4) Bazaar is claimed to have especially good merging support.

And multiple ones at that.  The default almost always DTRT, but what's  
nice is that if you do see an occasional text conflict, you can  
usually re-merge with a different algorithm to clear things up.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRBXVnEjvBPtnXfVAQJh3AP+IMnvHQSvuTwQwYCx1zyposZcbF7JBIFc
fkbnLBZ1Yn4JDiwPCDthzP2C0h8TAnshjeEPmr+On3sy+DizSxAT73jjhoCY3KoN
lqZEaJQ4pGpUg6PNeRjHYxmYQ8vjNhM3gVBXRYjMlEuFG+S5vxhxqcVKvNEjKdl3
vVErzP69D1o=
=zcux
-----END PGP SIGNATURE-----

From gustavo at niemeyer.net  Tue Nov  4 15:09:08 2008
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Tue, 4 Nov 2008 12:09:08 -0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811031833l3dc29672lf182ded076fe9182@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<643d90130811021708r5d39cc4bvf65fedea82395cc@mail.gmail.com>
	<bbaeab100811030957r567a13ddyfd1289052e73fe39@mail.gmail.com>
	<643d90130811031157l72a48f2avdec1eaf4a7fdda21@mail.gmail.com>
	<bbaeab100811031403p2fd3f9f0r46ce44d3e7bb8a76@mail.gmail.com>
	<C1083904-EE78-42F5-8AFD-CC8392FD37E4@python.org>
	<bbaeab100811031833l3dc29672lf182ded076fe9182@mail.gmail.com>
Message-ID: <643d90130811040609s79619559s25472b3473570a91@mail.gmail.com>

> Trying to expand our buildbot infrastructure to accept patches to test
> out or some patch queue manager might be nice, but I want to be
> realistic with what we have now. That's why I am not worrying about
> this email feature; until I know that we will actually use it and have
> the manpower to implement it I consider it a perk that might never be
> used. It's all great to dream about the future and what could be, but
> that is far beyond just trying to get us to switch to a DVCS and is
> just icing on the cake.

It's not an email feature.  It's about bundling revisions in a way
that may be cheaply sent or attached in any way wanted.  It is useful
in practice.  Attachments to bugs, mails, and so on are not novelty
nor optimization.  Check out how many patches you have attached to the
bug tracker.  Now consider how better would it be if these patches
carried history information which allowed them to be merged
considering context, and carried logs, authorship information, and
incremental history.

-- 
Gustavo Niemeyer
http://niemeyer.net

From barry at python.org  Tue Nov  4 15:19:21 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 4 Nov 2008 09:19:21 -0500
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <9e804ac0811040601t402e8133x3907e9d758e597e8@mail.gmail.com>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>
	<geed1t$uo3$1@ger.gmane.org> <490AB8E2.9050004@trueblade.com>
	<20081031102524.75dfd13f@resist.wooz.org>
	<9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
	<49104E66.5020000@gmx.net>
	<9e804ac0811040601t402e8133x3907e9d758e597e8@mail.gmail.com>
Message-ID: <5D4125E7-0C66-45FA-B911-74B16C2D965F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 4, 2008, at 9:01 AM, Thomas Wouters wrote:

> Nope, I have no idea how to edit those pages properly, sorry. I'm  
> sure somebarryone does.

I do.  I've been meaning to update those pages to mention loggerhead  
too.  I'll try to do that today.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRBZ6XEjvBPtnXfVAQKMHQQApaUnvSUFeYu0hI9wFbLm8peQ1miwlrkI
KFcZq26nFtQb/rSualxYR1ln82LIlGSLpR9PkYHzpbN+/FYGGXk+IrZ2OsUHalvd
1gsklmeHBc/CdJijFsfUw9XEQUZWgzobeGJz6RH5m4+JY8bmi1AdaNZ2nUnPd56j
50Gq6R0yqUs=
=NwuV
-----END PGP SIGNATURE-----

From p.f.moore at gmail.com  Tue Nov  4 17:14:24 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 4 Nov 2008 16:14:24 +0000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
Message-ID: <79990c6b0811040814r37024471ye481180c3987410e@mail.gmail.com>

2008/11/4 Gustavo Niemeyer <gustavo at niemeyer.net>:
> How large?  Which repositories?  Which operations?  Which version of Bazaar?

As large as the Python repository. The Python repository (:-)). Local
clone of the repo, when not using a shared repository (I know, "don't
do that" - but it is nevertheless a specific performance complaint
which I have, relating to the way I want to work with a DVCS). About
1.6. I can't recall precisely which version I tried, but it was fairly
recent.

> I'm sorry for jumping up on this, but you've mentioned one specific
> issue, which is being looked at, and then have generalized here.
> Without naming the issues, it's really hard to discuss anything.  It
> is true that Bazaar performance was much worse in the past, and it is
> true that it's being improved, and will continue to improve.  In case
> Bazaar is a good deal for Python nowadays, it should only get better
> over time.

There is a serious danger that this discussion will degenerate into
DVCS advocacy flamewars. Can I suggest that for now, we concentrate on
describing what operations are needed, in abstract terms. I applaud
Brett's initial stance that we need to look at what we need to do
based on workflow right now - it avoids focussing too much on "how my
DVCS has neat feature X which is self-evidently wonderful" arguments
(note - I am not saying that Gustavo is doing that, just that things
could go that way).

Once we have the core workflow covered, *then* would be a sensible
time to look very cautiously at features of a generic DVCS which
*might* be useful. But let's be conservative here - after all, there
is effort in simply replicating the python-checkins email hooks, and
expecting people to have the time to add new customisations as well,
could be a significant issue.

Hmm, having said all this, one huge benefit of any DVCS is the ability
for individuals to work on changes offline with full version control
(the well-known downside being that people don't publish changes as
soon as they should). Two thoughts come from that:

1. Availability of an easy-to-use location for random users to publish
branches would be an important factor.
2. The option to easily set up a bidirectional gateway to other DVCSs
might be beneficial to some users.

The nice thing about both of these is that they *don't* have to be
provided by python.org, as long as they are available.

Paul.

PS I still haven't found time to read Brett's PEP. Sorry. I will
before giving into the temptation to add further to this thread. I
promise!

From victor.stinner at haypocalc.com  Tue Nov  4 18:38:12 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 4 Nov 2008 18:38:12 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
Message-ID: <200811041838.12867.victor.stinner@haypocalc.com>

Le Monday 03 November 2008 18:56:37 Paul Miller, vous avez ?crit?:
> Rather than that, what about patching Python's long implementation 
> to use GMP if it's available, and the default implementation if not?

Yes, I like this suggestion of two flavors. Python with GMP and Python without 
GMP (builtin integer library). It's "easy" to change the long type 
implementation since only few modules use PyLong internals:
 - mashal: import/export (.pyc)
   => core function might be moved to longobject.c
 - math: use some special features of the PyLong type
   => macro can be written for that
or simplify a #ifdef.

I first wrote a patch in January 2008 to test Python3 + GMP. After your email 
I updated my patch and fix it to pass most regression tests. Depending of the 
patch version, the overhead (of using GMP instead a builtin library) is 
between 14% and 3% (Python is slower with GMP). I used pystone which is not a 
math application.

More details at:
http://bugs.python.org/issue1814

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From g.brandl at gmx.net  Tue Nov  4 18:45:04 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 04 Nov 2008 18:45:04 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
Message-ID: <geq1n0$vng$1@ger.gmane.org>

Ralf Schmitt schrieb:
> On Mon, Nov 3, 2008 at 1:05 AM, Brett Cannon <brett at python.org> wrote:
>> I have started the DVCS PEP which can be seen at
>> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
>> beyond the rationale, usage scenarios I plan to use, and what other
>> sections I plan to write.
>>
> 
> I think you really should not exclude any dvcs based on it's
> implementation language.
> I.e. requiring it being written in python for the sake of "eating your
> own dogfood" is just a very weak argument. git is certainly missing
> from your list.

And by then, why not include darcs, GNU arch and monotone?

Seriously, it's going to be hard enough to choose between hg and bzr
(at least I hope so, because it would show that both are excellent tools).
And since actually the implementation language makes sense, see the
other replies, we don't need to make it even harder.

cheers,
Georg


From schmir at gmail.com  Tue Nov  4 18:57:12 2008
From: schmir at gmail.com (Ralf Schmitt)
Date: Tue, 4 Nov 2008 18:57:12 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <geq1n0$vng$1@ger.gmane.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<geq1n0$vng$1@ger.gmane.org>
Message-ID: <932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>

On Tue, Nov 4, 2008 at 6:45 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Ralf Schmitt schrieb:
>> I think you really should not exclude any dvcs based on it's
>> implementation language.
>> I.e. requiring it being written in python for the sake of "eating your
>> own dogfood" is just a very weak argument. git is certainly missing
>> from your list.
>
> And by then, why not include darcs, GNU arch and monotone?
>

please include all of them and choose the best one. that was my point.

Regards,
- Ralf

From turnbull at sk.tsukuba.ac.jp  Tue Nov  4 19:00:47 2008
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Wed, 05 Nov 2008 03:00:47 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
Message-ID: <87fxm7tlps.fsf@uwakimon.sk.tsukuba.ac.jp>

Gustavo Niemeyer writes:

 > Both arguments strike me as odd.

I'm an odd fellow, what can I say?

 > Having the *option* to leave your history on the server shouldn't
 > be a problem, right?

Only if you later try to use it.<wink>

 > The same goes for (1): having more ways to use the tool isn't
 > exactly a problem.

It isn't exactly a good thing, either: TOOWDTI.

However, my intent was simply to respond to Brett, who mentioned that
one consideration for a VCS for Python is simplicity/ease of use for
occasional contributors, and proliferation of commands and styles is
not conducive to that.  Bazaar suffers from this problem although in a
different way and lesser degree compared to git.

 > How large?  Which repositories?  Which operations?  Which version
 > of Bazaar?

I'm not going to go there; this has been discussed exhaustively on the
bazaar at canonical.com list, as well as on lists of projects considering
adoption of bazaar.  You should look in their archives for the
results, where you will also find the discussion of the importance of
the results and plans to address problems by the developers.

Here again I'm simply responding to Brett who is under the impression
that all dVCSes can scale.  At present, bzr is one that cannot; the
user must accept far worse performance, across all operations, than is
available with the competing dVCSes, and the perception gets worse
with scale.

 > In case Bazaar is a good deal for Python nowadays, it should only
 > get better over time.

This is true.  Performance is not everything to everyone.  Most Bazaar
users don't care at all; they say things like "who cares about a few
seconds in bzr log when it gets the merge right almost every time".
But we don't yet know that most Python core developers will feel the
same way, and it simply is not appropriate at this point in time to
dismiss performance concerns with "oh, I'm sure they all can scale."

From barry at python.org  Tue Nov  4 19:13:17 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 4 Nov 2008 13:13:17 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87fxm7tlps.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
	<87fxm7tlps.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <48942028-4048-429B-9714-FE8C49EA4241@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 4, 2008, at 1:00 PM, Stephen J. Turnbull wrote:

> This is true.  Performance is not everything to everyone.  Most Bazaar
> users don't care at all; they say things like "who cares about a few
> seconds in bzr log when it gets the merge right almost every time".
> But we don't yet know that most Python core developers will feel the
> same way, and it simply is not appropriate at this point in time to
> dismiss performance concerns with "oh, I'm sure they all can scale."

We're Python programmers.  We're used to people telling us our tool is  
too slow.  We just say it does the job superbly and it's usually fast  
enough. :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRCQv3EjvBPtnXfVAQK43AQAhexjo3SVeWV9/3PYlR+PQPvZH72XY6XG
FuuWN6cfm0XGCAoyRTOv2DfnwYhHw1btYqCeP5yjxOuU1WogOfY1AvpSzWPYDi9R
Y1HQArXl9detWSAi5nzL2R8Z57jDnVpLxHscBMGhNxkzyqjQV+Dx2x2Zj3fqw8ac
NbOanHZewCk=
=4ES6
-----END PGP SIGNATURE-----

From cstejerean at gmail.com  Tue Nov  4 19:21:05 2008
From: cstejerean at gmail.com (Cosmin Stejerean)
Date: Tue, 4 Nov 2008 12:21:05 -0600
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <48942028-4048-429B-9714-FE8C49EA4241@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
	<87fxm7tlps.fsf@uwakimon.sk.tsukuba.ac.jp>
	<48942028-4048-429B-9714-FE8C49EA4241@python.org>
Message-ID: <276266d0811041021x254a7ab9y3b120d47ffae2af2@mail.gmail.com>

On Tue, Nov 4, 2008 at 12:13 PM, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 4, 2008, at 1:00 PM, Stephen J. Turnbull wrote:
>
>  This is true.  Performance is not everything to everyone.  Most Bazaar
>> users don't care at all; they say things like "who cares about a few
>> seconds in bzr log when it gets the merge right almost every time".
>> But we don't yet know that most Python core developers will feel the
>> same way, and it simply is not appropriate at this point in time to
>> dismiss performance concerns with "oh, I'm sure they all can scale."
>>
>
> We're Python programmers.  We're used to people telling us our tool is too
> slow.  We just say it does the job superbly and it's usually fast enough. :)
>
> - -Barry
>

I don't agree with completely dismissing performance just because it's
Python. Yes, Python is fast enough most of the time, but when it's not we
put a lot of effort into making it faster. That's why we have a good
collection of modules with C extensions to speed up computationally
intensive applications and half a dozen ways of writing said C extensions. I
see no excuse to let the fact that it's Python make it acceptable to have an
application with otherwise unacceptable performance.

-- 
Cosmin Stejerean
http://www.offbytwo.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081104/c22a85c2/attachment.htm>

From barry at python.org  Tue Nov  4 19:32:23 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 4 Nov 2008 13:32:23 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <276266d0811041021x254a7ab9y3b120d47ffae2af2@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
	<87fxm7tlps.fsf@uwakimon.sk.tsukuba.ac.jp>
	<48942028-4048-429B-9714-FE8C49EA4241@python.org>
	<276266d0811041021x254a7ab9y3b120d47ffae2af2@mail.gmail.com>
Message-ID: <35C62199-3592-48EE-8FAB-CF8A2E5BBD60@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 4, 2008, at 1:21 PM, Cosmin Stejerean wrote:

> I don't agree with completely dismissing performance just because it's
> Python. Yes, Python is fast enough most of the time, but when it's  
> not we
> put a lot of effort into making it faster. That's why we have a good
> collection of modules with C extensions to speed up computationally
> intensive applications and half a dozen ways of writing said C  
> extensions. I
> see no excuse to let the fact that it's Python make it acceptable to  
> have an
> application with otherwise unacceptable performance.

I guess the humor of that got lost in translation.  You had to be there.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRCVN3EjvBPtnXfVAQIdOAP/VxEc9ptQdrrJuthZM+wEeYy2eHMTno59
eFGurHysKDl5AUcYPt0dCeP8GAiuvZmU2Vp8/07HmMkgJPPtWbmxHZYTLZ2tEm/A
HmXBcsz1icc0dteu0SyeBUwDOz4puv9I5oF6NM879QEt4ykQz0hzM3dVyW6JsfN7
qIlGI8wGEgI=
=yUNI
-----END PGP SIGNATURE-----

From tim.peters at gmail.com  Tue Nov  4 19:33:00 2008
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 4 Nov 2008 13:33:00 -0500
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
Message-ID: <1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>

Hey, Mark -- let's establish some background here first.  It's a fact
that the product of two N-bit integers can require 2N bits, and also a
fact that lots of HW supports NxN->2N bit integer multiplication
directly.

However, it's unfortunately also a fact that standard C has no
corresponding concept:  "*" in C always means NxN->N multiplication
(single-width product, with possibility of overflow).  I don't know
whether C99 improved this situation -- I know it was proposed to add
some "double-width integer product" /functions/, but I don't know
whether that was adopted.  I do know that "*" remained "single-width".

[Tim Peters]
>> Note that while 32x32->64 multiply is supported by x86 HW, that
>> doesn't mean there's a uniform way to get at this HW capability across
>> C compilers.  In contrast, (at least) efficient HW 15x15->30 multiply
>> is universally spelled in C via "i*j" :-)

['Mark Dickinson]
> If we're talking about standards and portability, doesn't "i*j" fail
> on those (nonexistent?) platforms where the 'int' type is only 16-bits?
> Shouldn't this be something like "(long)i * j" instead?

Sorry, I should have made type declarations explicit.  In Python, the
basic long building block is "digit", which is typedef'ed to C
unsigned short.  C89 guarantees this holds at least 16 bits.  Whenever
two digits are multiplied, the code intends to cast (at least) one of
them to "twodigits" first (if you ever see a spot where this doesn't
happen, it's a bug).  "twodigits" is typedef'ed to C long.  C89
guarantees that a long holds at least 32 bits.

So C guarantees that we're doing (at least) 32x32->32 multiplication
whenever you see code like

    digit i, j;
    twodigits k;

    k = (twodigits)i * j;

In particular, the (at least) 32x32->32 C89 guarantees for that is at
least 15x15->30, and the latter is all that longobject.c intends to
rely on.  Along with the cast to twodigits, this is achieved across
all conforming C compilers simply by using infix "*".  The code from
1990 still works fine, on everything from cell phones to archaic Cray
boxes.


> And for 32x32 -> 64, can't this simply be replaced by "(uint64_t)i * j",
> where uint64_t is as in C99?  I'd hope that most compilers would
> manage to turn this into the appropriate 32x32-bit hardware multiply.

1. That's C99, not C89, and therefore less portable.

2. On platforms that support it, this is at least 64x64->64 multiplication,
   potentially much more expensive than the 32x32->64 (or 31x31->62?)
   flavor you /intend/ to move to.

3. There's no way to know exactly what compilers will do with this short of
   staring at generated code.

FYI, in a previous life working in speech recognition, under
Microsoft's Visual Studio 6 the only way we found to get at the
Pentium's 32x32->64 HW ability efficiently was via using inline
assembler.  For example, using various MSVC spellings of "64-bit int"
instead for the inputs usually generated external calls to a
long-winded C library "piece by piece" multiplication routine (which,
at the time, emulated 64x64->128 multiplication, then threw away the
high 64 bits).

Again, the fundamental problem here is the disconnect between what
some HW is capable of and what C allows to express (at least through
C89).  That's why it's impossible here to write portable code in C
that's also efficient.  Even what Python does now is vulnerable on the
efficiency count:  on some weird platforms, "long" is 64 bits, and so
multiplying a pair of twodigits incurs the expense of  (usually
non-native) 64x64->64 multiplication.


> I agree that very-long-integer optimizations probably don't really belong in
> Python,

Depends in part on whether Python can attract as many obsessed
maintainers and porters for such gonzo algorithms as GMP attracts ;-)


> but this patch should also provide significant benefits for short
> and medium-sized integers.  I guess I need to go away and do some
> benchmarking...

If you can /get at/ HW 32x32->64 int multiply, of course that would be faster.

From barry at python.org  Tue Nov  4 19:50:57 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 4 Nov 2008 13:50:57 -0500
Subject: [Python-Dev] hg branch gone?
In-Reply-To: <49104E66.5020000@gmx.net>
References: <bbaeab100810302055s654f7ee1k213b5e9acf04e877@mail.gmail.com>	<geed1t$uo3$1@ger.gmane.org>
	<490AB8E2.9050004@trueblade.com>	<20081031102524.75dfd13f@resist.wooz.org>
	<9e804ac0811030854g4264133q539e67970b2fd4be@mail.gmail.com>
	<49104E66.5020000@gmx.net>
Message-ID: <CF6B7D02-69E7-4CD0-B020-2E35B1DEAC9B@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 4, 2008, at 8:30 AM, Dennis Benzinger wrote:

> Am 03.11.2008 17:54, Thomas Wouters schrieb:
>> [...]
>> FWIW, I put one up this weekend, and it seems to be intact and OK.
>> (bzr+ssh://pythonbzr at code.python.org/python/2.6/ or
>> http://code.python.org/python/2.6/ )
>> [...]
>
> Can you update http://www.python.org/dev/bazaar/ ?
> For example "Branch details" needs to be updated because the trunk
> branch is no longer Python 2.6.

Done.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRCZkXEjvBPtnXfVAQLyswQAnGDLokaK9sHtNGcL92TJfTokGecUKPiH
ozgvS86T2Ydo6R9Qb71uJf+q/0iTnSyZ6LsGemKe93KrEwjmsksz9nSFrMkSwEH9
5m69nN2gaOLx7yJq3SHMBup2iJepFa++sK9IBph5eivrRsA3We+olKuOxNkDokKe
O1PDssFbmlo=
=hWPQ
-----END PGP SIGNATURE-----

From mal at egenix.com  Tue Nov  4 19:58:21 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 04 Nov 2008 19:58:21 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <200811041838.12867.victor.stinner@haypocalc.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811041838.12867.victor.stinner@haypocalc.com>
Message-ID: <49109B4D.1060904@egenix.com>

On 2008-11-04 18:38, Victor Stinner wrote:
> Le Monday 03 November 2008 18:56:37 Paul Miller, vous avez ?crit :
>> Rather than that, what about patching Python's long implementation 
>> to use GMP if it's available, and the default implementation if not?
> 
> Yes, I like this suggestion of two flavors. Python with GMP and Python without 
> GMP (builtin integer library). It's "easy" to change the long type 
> implementation since only few modules use PyLong internals:
>  - mashal: import/export (.pyc)
>    => core function might be moved to longobject.c
>  - math: use some special features of the PyLong type
>    => macro can be written for that
> or simplify a #ifdef.

Ahem, what about the gazillion extension modules out there ?

You cannot simply change the long implementation and expect
everything to just work. Besides, like Tim already said: adding
lots and lots of #ifdefs is just going to make it harder
to maintain the implementation without gaining anything much.

I don't really see much of a need for this anyway. If people
want to use GMP they can download gmpy or PyGMP or our
experimental mxNumber extension:

http://www.egenix.com/products/python/mxExperimental/mxNumber/

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 04 2008)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611

From victor.stinner at haypocalc.com  Tue Nov  4 20:01:15 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 4 Nov 2008 20:01:15 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
	<1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
Message-ID: <200811042001.15457.victor.stinner@haypocalc.com>

About 31, 32, 63 or 64 bits: I guess that you want to avoid integer overflow. 
Intel has an "overflow" flag, changed for all instructions. For other CPUs, 
you can use emulate this flag. Example with the type int that I used in my 
GMP patch:

Add:
  int a, b, sum;
  sum = a + b;
  exact = ((a < 0) ^ (b < 0)) || ((a < 0) == (sum < 0));

Substract:
  int a, b, diff;
  diff = a + b;
  exact = ((a < 0) == (b < 0)) || ((a < 0) == (diff < 0));

Multiply:
  int a, b, product;
  if (a == 0 || b == 0) {
     product = 0;  /* exact */
  } else if (a != INT_MIN || (b == 1)) {
     product = a * b;
     exact = (product / b) == a);
  } else {
     /* INT_MIN * -1 = -INT_MIN: overflow */
  }

Divide:
  int a, b, q;
  if (a != INT_MIN || b != -1) {
     q = a / b;   /* exact */
  } else {
     /* INT_MIN / -1 = -INT_MIN: overflow */
  }

Checking overflow may costs more than using a smaller base. Only a benchmark 
can answer ;-)

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From stefan_ml at behnel.de  Tue Nov  4 20:08:06 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 04 Nov 2008 20:08:06 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <490F0784.4050408@ghaering.de>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
	<490F0784.4050408@ghaering.de>
Message-ID: <geq6im$k8m$1@ger.gmane.org>

Gerhard H?ring wrote:
> Cython is still being worked on (intensively, it seems). Bundling it
> with Python means deciding on a particular version probably for an
> entire major release lifecycle (use Cython x.y.{newest} for Python 2.7,
> for example).

Yes, that is definitely an issue. Cython is heavily being worked on, a lot of
features were implemented this summer. This means that even slightly different
Cython versions can generate heavily different code.

The project has made inclusion into Python's stdlib a goal right from the
beginning. However, it is not currently clear when we will reach a suitable
level of Python language compliance to make a language/feature freeze. Note
that this only means a state where we decide to keep new features out. What's
in there now will not go away, so existing Cython code is pretty much future
proof (it even runs on Py3 already :).


>> I would love to see the option to write the lower levels in something
>> other than C, 

Well, Cython is definitely an option for that.


> Absolutely. That's why I tried to reimplement pysqlite in something
> easier to maintain than handwritten Python C API. There's a ctypes-based
> version in its Mercurial repository that's good enough to be used from
> PyPy now. And a started Cython-based one.

ctypes makes sense for projects that do not require a high-speed interface,
i.e. if you do major things behind the interface and only call into it from
time to time, choosing ctypes will keep your code more portable without
requiring a C compiler. However, if speed matters then it's hard to beat
Cython even with hand-written C code.


>> but obviously any choice would have to be a good one.
>> [...]  So, the question I see isn't if Cython should be
>> allowed for standard library modules, but if the landscape of such
>> solutions is at a point that any of them is ready to be committed to.

Well, we had reports that existing, non-trivial C modules could be rewritten
in Cython within a few days. But even if you are only talking about new
modules that get added to the stdlib, all you really need to become
independent of a specific Cython version is a good test suite for your code to
make sure it keeps working as expected when you update your compiler.

Stefan


From ctb at msu.edu  Tue Nov  4 20:11:22 2008
From: ctb at msu.edu (C. Titus Brown)
Date: Tue, 4 Nov 2008 11:11:22 -0800
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <geq6im$k8m$1@ger.gmane.org>
References: <gecqd4$g9g$1@ger.gmane.org>
	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>
	<4909EF67.5010104@trueblade.com>
	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk>
	<490E0747.8050401@behnel.de> <490EE0A0.2060700@ghaering.de>
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
	<490F0784.4050408@ghaering.de> <geq6im$k8m$1@ger.gmane.org>
Message-ID: <20081104191122.GD26084@idyll.org>

-> >> but obviously any choice would have to be a good one.
-> >> [...]  So, the question I see isn't if Cython should be
-> >> allowed for standard library modules, but if the landscape of such
-> >> solutions is at a point that any of them is ready to be committed to.
-> 
-> Well, we had reports that existing, non-trivial C modules could be rewritten
-> in Cython within a few days. But even if you are only talking about new
-> modules that get added to the stdlib, all you really need to become
-> independent of a specific Cython version is a good test suite for your code to
-> make sure it keeps working as expected when you update your compiler.

Hey, wait, isn't that also a requirement for Py3k? 

:)

--titus
-- 
C. Titus Brown, ctb at msu.edu

From stefan_ml at behnel.de  Tue Nov  4 20:17:41 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 04 Nov 2008 20:17:41 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <18703.3039.835260.398411@montanaro-dyndns-org.local>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>
	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>
	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>	<490F0784.4050408@ghaering.de>
	<18703.3039.835260.398411@montanaro-dyndns-org.local>
Message-ID: <geq74l$mi4$1@ger.gmane.org>

skip at pobox.com wrote:
>     >>> - Option 2: only distribute generated source files
>     >>> -- developers still need to have Cython installed
>     >>> -- you have to trust Cython; who will really review the generated code?
>     >> 
>     >> Who reviews the machine code from gcc?
> 
>     Gerhard> That's comparing apples and eggs :-P But it may be that I'm a
>     Gerhard> little paranoid here.
> 
> Agreed.  When Cython is as widely used as gcc and has as comprehensive a set
> of test cases (who knows? it may well already be comprehensive enough)

The test suite is pretty huge already, works on Py2.3 through Py3.0 (without
regenerating the C source code, BTW), and it tests C compilation as well as
C++ compilation of the generated source code.


> and supports all the compilers which Python supports

I didn't hear any complains so far. Portable C code is definitely a project goal.


> then we can probably skip the output code review step.

I think the main advantage for stdlib modules is actually the maintenance
cost. Having a single, easy-to-read code base for extension modules that
compiles without modification in Py2.6/7 and Py3.0/3.1 (and 2.3, 2.4 and 2.5),
makes life a lot easier for both maintainers and backporters.

Stefan


From martin at v.loewis.de  Tue Nov  4 20:53:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 04 Nov 2008 20:53:40 +0100
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <18704.14659.600239.209777@montanaro-dyndns-org.local>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<1afaf6160811031119k4d954477n2431e3d4adb572df@mail.gmail.com>
	<18703.25530.532307.815707@montanaro-dyndns-org.local>
	<490F7385.5090302@v.loewis.de>
	<18703.39241.442461.116001@montanaro-dyndns-org.local>
	<490FE188.5000401@v.loewis.de>
	<18704.14659.600239.209777@montanaro-dyndns-org.local>
Message-ID: <4910A844.2060409@v.loewis.de>

skip at pobox.com wrote:
>     >> OTOH, it should be no big deal to drop a zip archive of the GMP
>     >> sources which correspond to the code bound into the DLL.
> 
>     Martin> How would end users then extract the sources from the DLL? How
>     Martin> would they learn that they even have them in the first place?
> 
> I think you missed an implied comma in my statement.  I meant:
> 
>     1. distribute pythonxy.dll which binds to GMP.
>     2. distribute a gmpmn.zip which contains the source for the version used
>        in #1.

Ah, I see. That *is* a big deal to many Python users, which want to
distribute pythonxy.dll as part of their application, in a single file
(e.g. py2exe), and then don't want to worry about shipping another zip
file along with their single-file executable (plus including a third
file explaining why you need to do this).

Regards,
Martin

From martin at v.loewis.de  Tue Nov  4 20:59:52 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 04 Nov 2008 20:59:52 +0100
Subject: [Python-Dev] file open in python interpreter
In-Reply-To: <d3dcda8c0811040016v78ede8f2o120b2fcf44067470@mail.gmail.com>
References: <d3dcda8c0811022304oecee3bs84227aa7308c381e@mail.gmail.com>	<1afaf6160811030525t3cca1e7aha7def92b400bdfe0@mail.gmail.com>	<1afaf6160811030529i76cc9893l1a5ebcde264a63f9@mail.gmail.com>	<490F0333.5010803@gmail.com>
	<d3dcda8c0811040016v78ede8f2o120b2fcf44067470@mail.gmail.com>
Message-ID: <4910A9B8.3090109@v.loewis.de>

> any clues? how can i go around this?

What Nick said. Run it in a debugger, set a breakpoint on the failing
system call, and triple check that the arguments you are passing are
really the ones that you want to pass. Are you sure the current
directory is what you think it is? Are you sure that the relative path
doesn't, by accident, refer to the root directory?

Regards,
Martin

From martin at v.loewis.de  Tue Nov  4 21:06:10 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 04 Nov 2008 21:06:10 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <geq6im$k8m$1@ger.gmane.org>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>	<490F0784.4050408@ghaering.de>
	<geq6im$k8m$1@ger.gmane.org>
Message-ID: <4910AB32.7020309@v.loewis.de>

> The project has made inclusion into Python's stdlib a goal right from the
> beginning.

Ah, that changes my view of it significantly. If the authors want to
contribute it to Python some day, I'm looking forward to that (assuming
that they then close their official branch, and make the version inside
Python the maintained one).

That is also independent of whether standard library modules get written
in Cython. I would expect that some may (in particular, if they focus on
wrapping an external library), whereas others might stay what they are
(in particular, when they are in the real core of the interpreter).

> ctypes makes sense for projects that do not require a high-speed interface,
> i.e. if you do major things behind the interface and only call into it from
> time to time, choosing ctypes will keep your code more portable without
> requiring a C compiler. However, if speed matters then it's hard to beat
> Cython even with hand-written C code.

I would personally prefer a Cython integration over a ctypes one, for
the standard library (and supported inclusion of ctypes into Python
regardless).

Regards,
Martin

From dickinsm at gmail.com  Tue Nov  4 21:59:11 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 4 Nov 2008 20:59:11 +0000
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
	<1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
Message-ID: <5c6f2a5d0811041259k404a8554j6df126bba20872fa@mail.gmail.com>

On Tue, Nov 4, 2008 at 6:33 PM, Tim Peters <tim.peters at gmail.com> wrote:
> Whenever
> two digits are multiplied, the code intends to cast (at least) one of
> them to "twodigits" first (if you ever see a spot where this doesn't
> happen, it's a bug).  "twodigits" is typedef'ed to C long.  C89
> guarantees that a long holds at least 32 bits.

There are indeed one or two spots that seem to be missing a cast,
for example the line "rem -= hi*n" in inplace_divrem1.  I've fixed
all those I found, in the issue 4258 patch.

>> And for 32x32 -> 64, can't this simply be replaced by "(uint64_t)i * j",
> 1. That's C99, not C89, and therefore less portable.

Understood;  my thought was to use uint32_t and uint64_t for
digit and twodigits when available (with longs being stored in
base 2**30), falling back to the current ushort/ulong/base 2**15
otherwise.  On Unix, autoconf makes this easy by providing
macros like 'AC_TYPE_INT32_T', which makes sure that
int32_t is defined to be an integer type of exact width 32,
when available.

> 2. On platforms that support it, this is at least 64x64->64 multiplication,
>   potentially much more expensive than the 32x32->64 (or 31x31->62?)
>   flavor you /intend/ to move to.
>
> 3. There's no way to know exactly what compilers will do with this short of
>   staring at generated code.

Yes---I've done the staring for gcc, so I now have precisely *one*
data point, which is that various flavours of gcc on x86/x86_64
*are* clever enough to turn

  (uint64_t)my_uint32 * my_other_uint32

into the appropriate HW instruction.  Unfortunately I don't have
easy access to other compilers or platforms right now. :-(
Am still working on the benchmarking, but I'm definitely seeing
speedup on gcc/x86---between 10% and 100% depending
on the operations.

> FYI, in a previous life working in speech recognition, under
> Microsoft's Visual Studio 6 the only way we found to get at the
> Pentium's 32x32->64 HW ability efficiently was via using inline
> assembler.

Urk.  We definitely don't want to go there.  Though I guess this
is how packages like gmp and GP/Pari manage.

> C89).  That's why it's impossible here to write portable code in C
> that's also efficient.  Even what Python does now is vulnerable on the

But maybe it's possible to write portable code (by providing fallbacks)
that turns out to be efficient on the majority of mainstream systems?
The extent of the ifdef'ery in the patch is really rather small:  one
(compound) #ifdef in longintrepr.h for defining digit, twodigits, stwodigits
etc, and a couple more for the places where digits are read and written
in marshal.c.

>> I agree that very-long-integer optimizations probably don't really belong in
>> Python,
>
> Depends in part on whether Python can attract as many obsessed
> maintainers and porters for such gonzo algorithms as GMP attracts ;-)

Well, you can count me in. :)

Mark

From ncoghlan at gmail.com  Tue Nov  4 22:28:31 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 05 Nov 2008 07:28:31 +1000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	<geq1n0$vng$1@ger.gmane.org>
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
Message-ID: <4910BE7F.9040900@gmail.com>

Ralf Schmitt wrote:
> On Tue, Nov 4, 2008 at 6:45 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>> Ralf Schmitt schrieb:
>>> I think you really should not exclude any dvcs based on it's
>>> implementation language.
>>> I.e. requiring it being written in python for the sake of "eating your
>>> own dogfood" is just a very weak argument. git is certainly missing
>>> from your list.
>> And by then, why not include darcs, GNU arch and monotone?
>>
> 
> please include all of them and choose the best one. that was my point.

The PEP can at least mention darcs, arch, monotone, and then drop them
on the basis of not having a compelling reason to prefer them to the 3
more popular choices. I don't see any need to go into the "other
choices" further than that - our needs aren't wildly divergent from that
of other open source projects, and while popularity is a poor final
criteria, it's a decent filter to narrow the scope of the initial
investigation.

For git, without a killer feature that is "oh my god, wow!" when
compared to the features offered by Mercurial and Bazaar, the
implementation language is enough to push it to the back of the list for
me. Sure, that's a pretty lousy criteria in the absolute sense, but
given that it is obvious that there are two perfectly acceptable options
written in the language we ourselves are implementing, then it should
take something pretty impressive to get us to look at a third choice.

That would leave the PEP to focus on the Mercurial/Bazaar comparison (as
Brett originally intended) and I'm sure even that limited comparison is
going to give the infrastructure committee (and any other willing
assistants from python-dev) plenty of work to do).

The main thing I would suggest is that some kind of rationale like the
one above be included into the PEP itself to try to forestall the folks
going "but what about darcs/arch/monotone/git/whatever".

Also, something that should be done for ANY candidate VCS: translate the
current Python developer FAQ to give the appropriate answers for the
candidate VCS.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From tjreedy at udel.edu  Tue Nov  4 22:36:58 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 04 Nov 2008 16:36:58 -0500
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <geq74l$mi4$1@ger.gmane.org>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>	<490F0784.4050408@ghaering.de>	<18703.3039.835260.398411@montanaro-dyndns-org.local>
	<geq74l$mi4$1@ger.gmane.org>
Message-ID: <geqf9k$l09$1@ger.gmane.org>

Stefan Behnel wrote:

> I think the main advantage for stdlib modules is actually the maintenance
> cost. Having a single, easy-to-read code base for extension modules that
> compiles without modification in Py2.6/7 and Py3.0/3.1 (and 2.3, 2.4 and 2.5),
> makes life a lot easier for both maintainers and backporters.

Do you have any plans to support/use 3.0 type annotations so that one 
could develop function-oriented code in 3.0 and then compile efficient C 
(for whatever CPython version) without adding Python-incompatible cdefs?


From josiah.carlson at gmail.com  Tue Nov  4 23:18:06 2008
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Tue, 4 Nov 2008 14:18:06 -0800
Subject: [Python-Dev] Fwd: Removal of GIL through refcounting removal.
In-Reply-To: <EF6A1152-B162-444F-BA35-B38279EC4463@gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<76fd5acf0810310459v625e4b48u1e9a64a7057309e5@mail.gmail.com>
	<490B0584.7080006@voidspace.org.uk>
	<4222a8490810310703i6058fbf1j1b4f9dea511d3b77@mail.gmail.com>
	<loom.20081102T180853-494@post.gmane.org>
	<18702.15638.223872.706216@montanaro-dyndns-org.local>
	<e6511dbf0811031050w3383d56dgc110377a801f333d@mail.gmail.com>
	<d2155e360811031059x26245c8ei7c0fe96e53deffaa@mail.gmail.com>
	<e6511dbf0811031811i7d18fa2w7782a9d99666e3b4@mail.gmail.com>
	<EF6A1152-B162-444F-BA35-B38279EC4463@gmail.com>
Message-ID: <e6511dbf0811041418t7fa2b755x844bed0f6763d1ec@mail.gmail.com>

On Tue, Nov 4, 2008 at 5:31 AM, Leonardo Santagada <santagada at gmail.com> wrote:
>
> On Nov 4, 2008, at 12:11 AM, Josiah Carlson wrote:
>
>> There is no shortage of algorithms (such as matrix multiplication) that
>> are parallelizable but not particularly good candidates for an IPC-based
>> multiprocessing paradigm.
>>
>> Ahh, but those algorithms aren't going to be written in Python; they are
>> going to be written in C, and are going to manipulate memory directly.  With
>> such things, you can use standard Python threads, call into your C runtime,
>> and release the GIL.  Alternatively, you can use the mmap module to store
>> your data, shared across multiple processes, using the same direct-memory
>> access as you would with multiple threads and GIL release.
>
>
> That's the thing, right now you can't even use python to sketch one of those
> algorithms and for example compare performance with a non parallel version.

Why not?  The data needs to come from *somewhere*, it doesn't appear
from the aether into your RAM.  And even if it *does* appear from
nowhere, you can always use synthetic data to compare single process
vs. multiprocess - in pure Python.

> Also with four cores on common intel processor you might be happy with your
> algo being 4x faster while being quite easy to develop (not needing to use
> c).

Indeed, but it's not uncommon that using psyco you can get 2-4x (if
not occasionally 10-100x), with similar (though generally superior)
performance gains from using Pyrex/Cython.  And really, after
implementing your base algorithm in Python, translating it to C/C++ is
generally pretty straightforward for anything not involving complex
data structures or string parsing.

> I personally think this will be too much work on

And that's up to every programmer to decide for themselves.  I've done
a few of the things I've mentioned (shared mmaps, C code controlled by
Python, etc.), and have been very pleased with the speedups I
attained.  With better tools (Python, multiprocessing, Pyrex/Cython,
...) now available, I can't help but say, "back in my day, we just
wrote it in C and were *happy* that we didn't have to use Fortran". ;)

 - Josiah

From martin at v.loewis.de  Tue Nov  4 23:19:57 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 04 Nov 2008 23:19:57 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4910BE7F.9040900@gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	<geq1n0$vng$1@ger.gmane.org>	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
	<4910BE7F.9040900@gmail.com>
Message-ID: <4910CA8D.9010303@v.loewis.de>

> Also, something that should be done for ANY candidate VCS: translate the
> current Python developer FAQ to give the appropriate answers for the
> candidate VCS.

What I would like to see for at least the favored system: provide a demo
installation that is complete in the sense that immediate switchover
would be possible. I know this is probably far down the road, but I like
to bring it up at this point so that proponents are aware that doing
that might be required.

Of course, proponents of competing systems are encouraged to do the same
thing.

FWIW, I don't consider the current bazaar installation sufficient here.
It does give a useful insight for those of us unfamiliar with that
kind of system, and certainly allows those who want to develop with bzr
already to do so - in the dvcs manner. However, for a complete "demo"
installation, I'd like to see a repository that contains all revisions,
not just selected ones, has post-commit email notifications (or
explains why it can't provide these), etc.

Regards,
Martin

P.S. After we move to the new machine, there should be plenty of disk
space for this kind of experiment.

From brett at python.org  Wed Nov  5 01:13:47 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 4 Nov 2008 16:13:47 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4910BE7F.9040900@gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<geq1n0$vng$1@ger.gmane.org>
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
	<4910BE7F.9040900@gmail.com>
Message-ID: <bbaeab100811041613o2097416brf4291d5f1e7ccc9a@mail.gmail.com>

On Tue, Nov 4, 2008 at 13:28, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Ralf Schmitt wrote:
>> On Tue, Nov 4, 2008 at 6:45 PM, Georg Brandl <g.brandl at gmx.net> wrote:
>>> Ralf Schmitt schrieb:
>>>> I think you really should not exclude any dvcs based on it's
>>>> implementation language.
>>>> I.e. requiring it being written in python for the sake of "eating your
>>>> own dogfood" is just a very weak argument. git is certainly missing
>>>> from your list.
>>> And by then, why not include darcs, GNU arch and monotone?
>>>
>>
>> please include all of them and choose the best one. that was my point.
>
> The PEP can at least mention darcs, arch, monotone, and then drop them
> on the basis of not having a compelling reason to prefer them to the 3
> more popular choices. I don't see any need to go into the "other
> choices" further than that - our needs aren't wildly divergent from that
> of other open source projects, and while popularity is a poor final
> criteria, it's a decent filter to narrow the scope of the initial
> investigation.
>

Tossed in two sentences.

> For git, without a killer feature that is "oh my god, wow!" when
> compared to the features offered by Mercurial and Bazaar, the
> implementation language is enough to push it to the back of the list for
> me. Sure, that's a pretty lousy criteria in the absolute sense, but
> given that it is obvious that there are two perfectly acceptable options
> written in the language we ourselves are implementing, then it should
> take something pretty impressive to get us to look at a third choice.
>

Which is exactly how I feel.

> That would leave the PEP to focus on the Mercurial/Bazaar comparison (as
> Brett originally intended) and I'm sure even that limited comparison is
> going to give the infrastructure committee (and any other willing
> assistants from python-dev) plenty of work to do).
>

I must be insane.

-Brett

From brett at python.org  Wed Nov  5 01:34:00 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 4 Nov 2008 16:34:00 -0800
Subject: [Python-Dev] DVCS PEP update
Message-ID: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>

OK, I give: git is in the running. But do realize it will take a lot
for it to beat out bzr or hg.

I have emailed some people who have shown allegiance to a specific
DVCS to seeif they are willing to fill in the usage scenarios for me
along with converting the dev FAQ so that we all have a clear way to
learn how to use the things. If any of the people I emailed turn down
being the leader fo ra specific DVCS I will email here looking for
volunteers.

-Brett

From greg.ewing at canterbury.ac.nz  Wed Nov  5 02:11:59 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 05 Nov 2008 14:11:59 +1300
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <276266d0811041021x254a7ab9y3b120d47ffae2af2@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
	<87fxm7tlps.fsf@uwakimon.sk.tsukuba.ac.jp>
	<48942028-4048-429B-9714-FE8C49EA4241@python.org>
	<276266d0811041021x254a7ab9y3b120d47ffae2af2@mail.gmail.com>
Message-ID: <4910F2DF.3050005@canterbury.ac.nz>

Cosmin Stejerean wrote:
> 
> Yes, Python is fast enough most of the time, but when it's not 
> we put a lot of effort into making it faster. That's why we have a good 
> collection of modules with C extensions to speed up computationally 
> intensive applications

So the Pythonic solution is, of course, to make one of
the existing VCSes written in C available as an extension
module. :-)

-- 
Greg

From stephen at xemacs.org  Wed Nov  5 03:36:25 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 05 Nov 2008 11:36:25 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <276266d0811041021x254a7ab9y3b120d47ffae2af2@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
	<87fxm7tlps.fsf@uwakimon.sk.tsukuba.ac.jp>
	<48942028-4048-429B-9714-FE8C49EA4241@python.org>
	<276266d0811041021x254a7ab9y3b120d47ffae2af2@mail.gmail.com>
Message-ID: <87abceuceu.fsf@uwakimon.sk.tsukuba.ac.jp>

Cosmin Stejerean writes:
 > On Tue, Nov 4, 2008 at 12:13 PM, Barry Warsaw <barry at python.org> wrote:

 > > We're Python programmers.  We're used to people telling us our
 > > tool is too slow.  We just say it does the job superbly and it's
 > > usually fast enough. :)

 > I don't agree with completely dismissing performance just because it's
 > Python.

Oh, it's just an in-joke.  He's doing a "my snake is longer than your
snake" because I sniped at Gustavo with a "TOOWTDI".

 > I see no excuse to let the fact that it's Python make it acceptable
 > to have an application with otherwise unacceptable performance.

Barry and many others obviously find the performance of non-git VCSes
acceptable.  On the other hand, Brett has conceded that git's
performance has enough fans that it has to be on the table (even
though he admits to a strong bias against it, that's the major
concession).

Performance is *always* a consideration.  IMO, now that it has been
acknowledged that performance varies considerably among VCSes, it's
more important at this point to figure out what Python development's
*other* requirements are.


From ondrej at certik.cz  Wed Nov  5 10:04:36 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 5 Nov 2008 10:04:36 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <4910AB32.7020309@v.loewis.de>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>
	<4909EF67.5010104@trueblade.com>
	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>
	<4909FECF.7000703@voidspace.org.uk> <490E0747.8050401@behnel.de>
	<490EE0A0.2060700@ghaering.de>
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>
	<490F0784.4050408@ghaering.de> <geq6im$k8m$1@ger.gmane.org>
	<4910AB32.7020309@v.loewis.de>
Message-ID: <85b5c3130811050104v54ec1dbfy9f800f7a94e33d38@mail.gmail.com>

On Tue, Nov 4, 2008 at 9:06 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> The project has made inclusion into Python's stdlib a goal right from the
>> beginning.
>
> Ah, that changes my view of it significantly. If the authors want to
> contribute it to Python some day, I'm looking forward to that (assuming
> that they then close their official branch, and make the version inside
> Python the maintained one).
>
> That is also independent of whether standard library modules get written
> in Cython. I would expect that some may (in particular, if they focus on
> wrapping an external library), whereas others might stay what they are
> (in particular, when they are in the real core of the interpreter).

I think it is also a good idea to write things using pure Python
syntax in Cython, so that all other Python implementations, like
Jython, Pypy, IronPython can just take it and run it in pure Python
mode. Pure Python syntax means that the code runs in Python
unmodified, but can also be compiled with Cython. Pure Python syntax
was only recently added to Cython, so I guess it should be well tested
first.

What do you think?

Ondrej

From p.f.moore at gmail.com  Wed Nov  5 11:17:22 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Nov 2008 10:17:22 +0000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87abceuceu.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<0F7C718C-A1A0-4B30-BD67-AE9F2A68A50B@python.org>
	<20081103175803.GL32006@idyll.org>
	<bbaeab100811031005g79d9162btb31458999ff432cf@mail.gmail.com>
	<87r65st6bf.fsf@uwakimon.sk.tsukuba.ac.jp>
	<643d90130811040559p10e27fadi3c1e196b4fe133ce@mail.gmail.com>
	<87fxm7tlps.fsf@uwakimon.sk.tsukuba.ac.jp>
	<48942028-4048-429B-9714-FE8C49EA4241@python.org>
	<276266d0811041021x254a7ab9y3b120d47ffae2af2@mail.gmail.com>
	<87abceuceu.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <79990c6b0811050217idad27b2r88d222b57caf4271@mail.gmail.com>

2008/11/5 Stephen J. Turnbull <stephen at xemacs.org>:
>  > I see no excuse to let the fact that it's Python make it acceptable
>  > to have an application with otherwise unacceptable performance.
>
> Barry and many others obviously find the performance of non-git VCSes
> acceptable.  On the other hand, Brett has conceded that git's
> performance has enough fans that it has to be on the table (even
> though he admits to a strong bias against it, that's the major
> concession).

Oy! "Non-git VCSes" indeed! :-)

Barry and others find the performance of *Bazaar* acceptable. I (and
equally many others, I believe) don't. Mercurial is significantly
faster than Bazaar (I don't know how Mercurial speed compares to git
but both are orders of magnitude faster than Bazaar in at least some
cases).

I believe that the performance of Bazaar is still its major
disadvantage, regardless of the Bazaar developers' ongoing work on the
issue and the often-repeated "well, yes, but if you do things this
way, that's much faster" workarounds.

> Performance is *always* a consideration.  IMO, now that it has been
> acknowledged that performance varies considerably among VCSes, it's
> more important at this point to figure out what Python development's
> *other* requirements are.

That is a fair point. My instinct is that python-dev's requirements on
a DVCS are likely to be fairly basic (at least initially) and most
DVCSes will be able to satisfy them. At that point, the arguments get
more difficult ("nice to have" extra features which may not actuially
get used in practice, we don't know without experience, versus
immediate performance benefits which could easily change over time).

Paul.

From p.f.moore at gmail.com  Wed Nov  5 12:09:24 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Nov 2008 11:09:24 +0000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
Message-ID: <79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>

2008/11/3 Brett Cannon <brett at python.org>:
> At this point I am looking for any suggestions for fundamental usage
> scenarios that I am missing from the PEP. If you think the few already
> listed are missing some core part of a VCS, please let me know.

My apologies, I can't check if this is already there (firewall rules
from this account :-() but can I suggest the following use case:

An average user (ie, not a core developer) finds an issue, and has an
idea how to fix it. He raises a tracker item, checks out the Python
sources, makes a fix, and wants to upload it to the tracker. Key
points here are the initial work needed to grab a development
checkout, and the ability to bundle up a fix for upload to the
tracker. (I'm specifically thinking of a casual user, not a developer
who already has a Python checkout to work on).

I'll freely admit a (not very) hidden bias here - the slowness of an
initial clone (or going through the "download a shared repo, unpack
it, create a branch and update" rigmarole) makes this a nasty test for
Bazaar. But I do nevertheless think it's an important use case, as
it's all about encouraging casual users to contribute.

Paul.

From skip at pobox.com  Wed Nov  5 12:20:45 2008
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 5 Nov 2008 05:20:45 -0600
Subject: [Python-Dev] test - please ignore
Message-ID: <18705.33165.627331.486918@montanaro-dyndns-org.local>

(working on spam system - just hit 'd')

From stefan_ml at behnel.de  Wed Nov  5 14:10:46 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 5 Nov 2008 13:10:46 +0000 (UTC)
Subject: [Python-Dev] Using Cython for standard library?
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>	<490F0784.4050408@ghaering.de>	<18703.3039.835260.398411@montanaro-dyndns-org.local>
	<geq74l$mi4$1@ger.gmane.org> <geqf9k$l09$1@ger.gmane.org>
Message-ID: <loom.20081105T125602-719@post.gmane.org>

Terry Reedy wrote:
> Stefan Behnel wrote:
>> I think the main advantage for stdlib modules is actually the maintenance
>> cost. Having a single, easy-to-read code base for extension modules that
>> compiles without modification in Py2.6/7 and Py3.0/3.1 (and 2.3, 2.4 and
>> 2.5), makes life a lot easier for both maintainers and backporters.
> 
> Do you have any plans to support/use 3.0 type annotations so that one 
> could develop function-oriented code in 3.0 and then compile efficient C 
> (for whatever CPython version) without adding Python-incompatible cdefs?

That is still an official TODO, but there isn't currently any support for it,
nor any effort to implement it (patches obviously welcome).

The problem is that it only applies to function parameters, not to local
variables or anything else, so the gain is actually very small. On the other
hand, the "pure Python mode", as it is currently implemented, is much more
powerful and allows you to annotate most things in Cython without loosing
the ability to run the code unchanged in plain Python (including Jython and
friends, although I never tested that).

http://wiki.cython.org/pure

One really neat feature is that you can put a .pxd file next to your .py file
and let it override the function signatures and classes. So you do not even
need Py3 annotations, which have the obvious disadvantage of requiring Py3.

Stefan



From dripton at ripton.net  Wed Nov  5 15:04:28 2008
From: dripton at ripton.net (David Ripton)
Date: Wed, 5 Nov 2008 06:04:28 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
Message-ID: <20081105140428.GA20949@vidar.dreamhost.com>

On 2008.11.05 11:09:24 +0000, Paul Moore wrote:
> An average user (ie, not a core developer) finds an issue, and has an
> idea how to fix it. He raises a tracker item, checks out the Python
> sources, makes a fix, and wants to upload it to the tracker. Key
> points here are the initial work needed to grab a development
> checkout, and the ability to bundle up a fix for upload to the
> tracker. (I'm specifically thinking of a casual user, not a developer
> who already has a Python checkout to work on).
> 
> I'll freely admit a (not very) hidden bias here - the slowness of an
> initial clone (or going through the "download a shared repo, unpack
> it, create a branch and update" rigmarole) makes this a nasty test for
> Bazaar. But I do nevertheless think it's an important use case, as
> it's all about encouraging casual users to contribute.

All timings very approximate:

Time for average user to check out Python sources with bzr: 10 minutes

Time for average user to check out Python sources with git or hg: 1 minute

Time for average user's trivial patch to be reviewed and committed: 1 year

I love DVCS as much as the next guy, but checkout time is so not the
bottleneck for this use case.

-- 
David Ripton    dripton at ripton.net

From barry at python.org  Wed Nov  5 15:13:44 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 5 Nov 2008 09:13:44 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
Message-ID: <D5B12534-4DB3-4FA5-90E6-88CDDB3E8C03@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 5, 2008, at 6:09 AM, Paul Moore wrote:

> I'll freely admit a (not very) hidden bias here - the slowness of an
> initial clone (or going through the "download a shared repo, unpack
> it, create a branch and update" rigmarole) makes this a nasty test for
> Bazaar. But I do nevertheless think it's an important use case, as
> it's all about encouraging casual users to contribute.

One thing that we're not taking advantage of on code.python.org for  
Bazaar is "stacked branches".  This is basically an indirection  
technique that doesn't require a full revision history download to get  
started.  Stacked branches greatly improve performance for the initial  
pull and push, and reduces disk usage on the server.

When the new machine is in place, I'll make sure we provide stacked  
branches for the MvL Experiment.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRGqGHEjvBPtnXfVAQLuSAQAjStZk2nzSsvQN+bDwHryJUVKuutfz9uw
ffCjaG9SRiXGaYSfNWs4Ao94fnYiRZVmqYiSHg8NTydR4g54sAF5FKbA1Bgswp9F
l52ObdFWKhjwkpfuq1i+IAG2HEqZyiV0naZi4xQmMBKhFd2SFw+qUS9qzGExdCXw
PFDYAiS9xKg=
=bUlo
-----END PGP SIGNATURE-----

From thomas at python.org  Wed Nov  5 15:52:55 2008
From: thomas at python.org (Thomas Wouters)
Date: Wed, 5 Nov 2008 15:52:55 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4910CA8D.9010303@v.loewis.de>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<geq1n0$vng$1@ger.gmane.org>
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
Message-ID: <9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>

On Tue, Nov 4, 2008 at 23:19, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> FWIW, I don't consider the current bazaar installation sufficient here.
> It does give a useful insight for those of us unfamiliar with that
> kind of system, and certainly allows those who want to develop with bzr
> already to do so - in the dvcs manner. However, for a complete "demo"
> installation, I'd like to see a repository that contains all revisions,
> not just selected ones, has post-commit email notifications (or
> explains why it can't provide these), etc.
>

Without a doubt the bazaar branches need a little more loving attention to
make them a full working demo, but it's mostly details. The branches *do*
contain the whole history, and not just 'select revisions':

3.0% bzr log
------------------------------------------------------------
revno: 37403
committer: martin.v.loewis
timestamp: Tue 2008-11-04 20:45:29 +0000
message:
  Merged revisions 67098 via svnmerge from
  svn+ssh://pythondev at svn.python.org/python/trunk

  ........
    r67098 | martin.v.loewis | 2008-11-04 21:40:09 +0100 (Di, 04 Nov 2008) |
2 l
ines

    Issue #4204: Fixed module build errors on FreeBSD 4.
  ........
[...]
------------------------------------------------------------
revno: 2
committer: guido
timestamp: Thu 1990-08-09 14:25:15 +0000
message:
  Initial revision
------------------------------------------------------------
revno: 1
committer:
timestamp: Thu 1990-08-09 14:25:15 +0000
message:
  New repository initialized by cvs2svn.


-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081105/b1214821/attachment.htm>

From lists at cheimes.de  Wed Nov  5 15:55:09 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 05 Nov 2008 15:55:09 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <4910AB32.7020309@v.loewis.de>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>	<490F0784.4050408@ghaering.de>	<geq6im$k8m$1@ger.gmane.org>
	<4910AB32.7020309@v.loewis.de>
Message-ID: <4911B3CD.6070606@cheimes.de>

Martin v. L?wis wrote:
> Ah, that changes my view of it significantly. If the authors want to
> contribute it to Python some day, I'm looking forward to that (assuming
> that they then close their official branch, and make the version inside
> Python the maintained one).
> 
> That is also independent of whether standard library modules get written
> in Cython. I would expect that some may (in particular, if they focus on
> wrapping an external library), whereas others might stay what they are
> (in particular, when they are in the real core of the interpreter).
>
> I would personally prefer a Cython integration over a ctypes one, for
> the standard library (and supported inclusion of ctypes into Python
> regardless).

I agree from the bottom of my heart. In the past I've used Pyrex Cython 
for wrapping libraries and small snippets of C speedup with great 
success. The learning curve is smooth and experiments show quick success.

Once Cython's development has slowed down and stabilized I like to see 
its usage for non-critical extension modules. Cython should make it 
easier to write wrappers for libraries or the IO performance improvement 
module, Amaury is working on. However I wouldn't want to use it for 
anything critical to the core of Python. I like to keep that C code of 
the core as readable as possible.

Christian

From p.f.moore at gmail.com  Wed Nov  5 16:31:40 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Nov 2008 15:31:40 +0000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <D5B12534-4DB3-4FA5-90E6-88CDDB3E8C03@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<D5B12534-4DB3-4FA5-90E6-88CDDB3E8C03@python.org>
Message-ID: <79990c6b0811050731n26c57861u7ffabe2505cf1782@mail.gmail.com>

2008/11/5 Barry Warsaw <barry at python.org>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 5, 2008, at 6:09 AM, Paul Moore wrote:
>
>> I'll freely admit a (not very) hidden bias here - the slowness of an
>> initial clone (or going through the "download a shared repo, unpack
>> it, create a branch and update" rigmarole) makes this a nasty test for
>> Bazaar. But I do nevertheless think it's an important use case, as
>> it's all about encouraging casual users to contribute.
>
> One thing that we're not taking advantage of on code.python.org for Bazaar
> is "stacked branches".  This is basically an indirection technique that
> doesn't require a full revision history download to get started.  Stacked
> branches greatly improve performance for the initial pull and push, and
> reduces disk usage on the server.

While that would be good, my understanding is that stacked branches in
Bazaar only work (for history operations) while you're online. So they
make Bazaar work a little like a centralised VCS, I guess. Not sure
how that's a step forward :-)

> When the new machine is in place, I'll make sure we provide stacked branches
> for the MvL Experiment.

It would certainly be nice to be able to try such things out.

Paul.

From p.f.moore at gmail.com  Wed Nov  5 16:35:54 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Nov 2008 15:35:54 +0000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <20081105140428.GA20949@vidar.dreamhost.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<20081105140428.GA20949@vidar.dreamhost.com>
Message-ID: <79990c6b0811050735g240816cfm8ddc552343eea7e4@mail.gmail.com>

2008/11/5 David Ripton <dripton at ripton.net>:
> All timings very approximate:
>
> Time for average user to check out Python sources with bzr: 10 minutes
>
> Time for average user to check out Python sources with git or hg: 1 minute
>
> Time for average user's trivial patch to be reviewed and committed: 1 year
>
> I love DVCS as much as the next guy, but checkout time is so not the
> bottleneck for this use case.

:-) That's a fair point. But it's not the point I was trying to make,
which is that I'd want whatever DVCS is chosen to make the initial
experience of a casual user / newcomer as easy as possible. Why
discourage them in the first 10 minutes (which, BTW, is much faster
than my experience with bzr last time I tried the Python repo) when we
can make them suffer for a whole year? :-) :-)

Paul.

From jimjjewett at gmail.com  Wed Nov  5 17:22:00 2008
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 5 Nov 2008 12:22:00 -0400
Subject: [Python-Dev] Looking for VCS usage scenarios
Message-ID: <fb6fbf560811050822x3d6a55bdo39947218d55cf2c9@mail.gmail.com>

David Ripton wrote:

> Time for average user to check out Python sources with bzr: 10 minutes

> Time for average user to check out Python sources with git or hg: 1 minute

> Time for average user's trivial patch to be reviewed and committed: 1 year

> I love DVCS as much as the next guy, but checkout time is so not the
> bottleneck for this use case.

I think Paul's point is that he wants to support people who have not
previously planned to contribute to python.  Writing the patch may be
a matter of minutes, once they implement the fix for themselves.

Downloading a new VCS is a major commitment of time and disk space.
(And is there setup, and dealing with proxies?)  It doesn't take as
long (calendar) as waiting for the review, but it takes long enough
(clock) that people may not bother to do it.  And if they don't, what
was the point of switching to a DCVS?

-jJ

From tjreedy at udel.edu  Wed Nov  5 18:54:49 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 05 Nov 2008 12:54:49 -0500
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <loom.20081105T125602-719@post.gmane.org>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>	<490F0784.4050408@ghaering.de>	<18703.3039.835260.398411@montanaro-dyndns-org.local>	<geq74l$mi4$1@ger.gmane.org>
	<geqf9k$l09$1@ger.gmane.org>
	<loom.20081105T125602-719@post.gmane.org>
Message-ID: <gesml3$99c$1@ger.gmane.org>

Stefan Behnel wrote:
> Terry Reedy wrote:

>> Do you have any plans to support/use 3.0 type annotations so that one 
>> could develop function-oriented code in 3.0 and then compile efficient C 
>> (for whatever CPython version) without adding Python-incompatible cdefs?
> 
> That is still an official TODO, but there isn't currently any support for it,
> nor any effort to implement it (patches obviously welcome).
> 
> The problem is that it only applies to function parameters, not to local

I was presuming some type inferencing for locals ;-)

> variables or anything else, so the gain is actually very small.

Why I said 'function-oriented'.

 > On the other
> hand, the "pure Python mode", as it is currently implemented, is much more
> powerful and allows you to annotate most things in Cython without loosing
> the ability to run the code unchanged in plain Python (including Jython and
> friends, although I never tested that).
> 
> http://wiki.cython.org/pure

Ah.  Missed it.  Very interesting and even better.
How is one supposed to get there from the main page?
http://wiki.cython.org/Cython
Perhaps under "*Using early binding techniques to improve speed"
you could add "* Early binding in pure Python code." or some such.

> One really neat feature is that you can put a .pxd file next to your .py file
> and let it override the function signatures and classes. So you do not even
> need Py3 annotations, which have the obvious disadvantage of requiring Py3.

That is not much of a disadvantage for code intended to be compiled. 
But the annotation is limited.  Support for nested defs and generators 
is much more critical.

Typo? Should 'typedef(cython.p_int)' be 'cython.typedef(cython.p_int)'?
Also, the 'enhancement' page has separate misspelled 'seperate' twice.

Terry Jan Reedy


From skip at pobox.com  Wed Nov  5 19:16:51 2008
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 5 Nov 2008 12:16:51 -0600
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
Message-ID: <18705.58131.933909.115314@montanaro-dyndns-org.local>


I apologize that I haven't read whatever Brett's written so far, but I just
haven't had time, and don't know if there's a PEP yet (and if so, what its
number is).  I did want to get my questions/confusion on the record though.

What DVCS fits my poor brain best?  I feel I'm like a dinosaur not being
able to figure out how I'm supposed to contribute changes to the system.

Do I:

    * commit my changes to some central branch?

    * commit my changes locally then create diffs I then submit to the
      tracker?

    * commit locally then push them somewhere?

    * commit locally then ask someone to pull?

    * Not commit anything anywhere but just submit patches to the tracker?

In addition:

    * Will there be a central repository?

    * How will I know which of possibly many repos is "authoritative"?

    * How will I discover other repos?  For example, if the safethread stuff
      is sitting somewhere in a repository can I find it on my own somehow?

    * Will a DVCS allow simpler operation as if we are still using a
      centralized system like CVS or Subversion?

Thx,

Skip


From somelauw at yahoo.com  Wed Nov  5 19:26:35 2008
From: somelauw at yahoo.com (L V)
Date: Wed, 5 Nov 2008 10:26:35 -0800 (PST)
Subject: [Python-Dev] Why don't range and xrange threat floats as floats?
Message-ID: <502974.52539.qm@web45703.mail.sp1.yahoo.com>

Why don't range and xrange threat floats as floats?
Is there any good reason range and xrange don't threat floats as floats but as integers?
When I enter float arguments in a range, the floats are treated as integers. (+ some warning)

This is how I think it should work:
>>>range(0, 1, 0.1)
[0.0, 0.10000000000000001, 0.20000000000000001, 0.29999999999999999, 0.40000000000000002, 0.5, 0.59999999999999998, 0.69999999999999996, 0.80000000000000004, 0.90000000000000002]

I got these results by:
>>>[x/10. for x in xrange(10)]

I'm not looking for alternatives, just for the reason that it works this way.



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081105/77193f89/attachment.htm>

From tjreedy at udel.edu  Wed Nov  5 19:32:41 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 05 Nov 2008 13:32:41 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <79990c6b0811050735g240816cfm8ddc552343eea7e4@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>	<20081105140428.GA20949@vidar.dreamhost.com>
	<79990c6b0811050735g240816cfm8ddc552343eea7e4@mail.gmail.com>
Message-ID: <gesos3$hjm$1@ger.gmane.org>

Paul Moore wrote:
> 2008/11/5 David Ripton <dripton at ripton.net>:
>> All timings very approximate:
>>
>> Time for average user to check out Python sources with bzr: 10 minutes
>>
>> Time for average user to check out Python sources with git or hg: 1 minute
>>
>> Time for average user's trivial patch to be reviewed and committed: 1 year
>>
>> I love DVCS as much as the next guy, but checkout time is so not the
>> bottleneck for this use case.
> 
> :-) That's a fair point. But it's not the point I was trying to make,
> which is that I'd want whatever DVCS is chosen to make the initial
> experience of a casual user / newcomer as easy as possible. Why
> discourage them in the first 10 minutes (which, BTW, is much faster
> than my experience with bzr last time I tried the Python repo) when we
> can make them suffer for a whole year? :-) :-)

It does get to the point that the current bottleneck is code review, to 
the point that people may not submit patches because it seems nearly 
useless*.  And often, when a patch does get reviewed, the diff is 
obsolete and needs to be redone versus the changed 'current' trunk.  I 
presume that patches as branches would alleviate this last part.

So I think easier review should be a prime consideration for 
infrastructure improvement.  If I go to the tracker now and click on a 
'patch', I get a sometime easy, usually difficult, and sometimes 
impossible to read diff.  With a wide-screen monitor, I would like a 
side-by-side display with differences marked, as with Guido's code 
review tool and other such displays I have seen here and there.

*The current quick review and implementation of doc suggestions and 
patches, on the other hand, has encouraged more submissions from me and, 
I believe, others.


From leif.walsh at gmail.com  Wed Nov  5 19:38:45 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Wed, 5 Nov 2008 13:38:45 -0500
Subject: [Python-Dev] Why don't range and xrange threat floats as floats?
In-Reply-To: <502974.52539.qm@web45703.mail.sp1.yahoo.com>
References: <502974.52539.qm@web45703.mail.sp1.yahoo.com>
Message-ID: <cc7430500811051038wecd9febmc2dd26377424faa5@mail.gmail.com>

On Wed, Nov 5, 2008 at 1:26 PM, L V <somelauw at yahoo.com> wrote:
> Why don't range and xrange threat floats as floats?
> Is there any good reason range and xrange don't threat floats as floats but
> as integers?
> When I enter float arguments in a range, the floats are treated as integers.
> (+ some warning)
>
> This is how I think it should work:
>>>>range(0, 1, 0.1)
> [0.0, 0.10000000000000001, 0.20000000000000001, 0.29999999999999999,
> 0.40000000000000002, 0.5, 0.59999999999999998, 0.69999999999999996,
> 0.80000000000000004, 0.90000000000000002]
>
> I got these results by:
>>>>[x/10. for x in xrange(10)]
>
> I'm not looking for alternatives, just for the reason that it works this
> way.

This seems like a question to be asked on python-list, not python-dev.

In any case, my best guess is that it's because of what you just
showed, that the functionality you want is there already if you use a
generator expression.  There isn't much reason to make (x)range more
complicated by letting it accept floats, when you can just scale the
range you actually want up to some integral skip value, call range,
and then scale it back down.

-- 
Cheers,
Leif

From guido at python.org  Wed Nov  5 19:56:23 2008
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Nov 2008 10:56:23 -0800
Subject: [Python-Dev] Why don't range and xrange threat floats as floats?
In-Reply-To: <502974.52539.qm@web45703.mail.sp1.yahoo.com>
References: <502974.52539.qm@web45703.mail.sp1.yahoo.com>
Message-ID: <ca471dc20811051056l139e0e03i4967109276d8ba91@mail.gmail.com>

On Wed, Nov 5, 2008 at 10:26 AM, L V <somelauw at yahoo.com> wrote:
> Why don't range and xrange threat floats as floats?
> Is there any good reason range and xrange don't threat floats as floats but
> as integers?
> When I enter float arguments in a range, the floats are treated as integers.
> (+ some warning)
>
> This is how I think it should work:
>>>>range(0, 1, 0.1)
> [0.0, 0.10000000000000001, 0.20000000000000001, 0.29999999999999999,
> 0.40000000000000002, 0.5, 0.59999999999999998, 0.69999999999999996,
> 0.80000000000000004, 0.90000000000000002]
>
> I got these results by:
>>>>[x/10. for x in xrange(10)]
>
> I'm not looking for alternatives, just for the reason that it works this
> way.

The reason is that due to the rounding issues inherent to a binary
representation of floating point, it is easy to give examples where
it's ambiguous whether the end point was meant to be included or not.
This could happen e.g. when the binary representation of the end point
was rounded up a bit, while the step would be rounded down.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From somelauw at yahoo.com  Wed Nov  5 20:09:35 2008
From: somelauw at yahoo.com (L V)
Date: Wed, 5 Nov 2008 11:09:35 -0800 (PST)
Subject: [Python-Dev] Why don't range and xrange threat floats as floats?
References: <502974.52539.qm@web45703.mail.sp1.yahoo.com>
	<ca471dc20811051056l139e0e03i4967109276d8ba91@mail.gmail.com>
Message-ID: <943809.75899.qm@web45712.mail.sp1.yahoo.com>

Thank you very much for your answer. I have wondered about this for a long time.




________________________________
From: Guido van Rossum <guido at python.org>
To: L V <somelauw at yahoo.com>
Cc: python-dev at python.org
Sent: Wednesday, November 5, 2008 7:56:23 PM
Subject: Re: [Python-Dev] Why don't range and xrange threat floats as floats?

On Wed, Nov 5, 2008 at 10:26 AM, L V <somelauw at yahoo.com> wrote:
> Why don't range and xrange threat floats as floats?
> Is there any good reason range and xrange don't threat floats as floats but
> as integers?
> When I enter float arguments in a range, the floats are treated as integers.
> (+ some warning)
>
> This is how I think it should work:
>>>>range(0, 1, 0.1)
> [0.0, 0.10000000000000001, 0.20000000000000001, 0.29999999999999999,
> 0.40000000000000002, 0.5, 0.59999999999999998, 0.69999999999999996,
> 0.80000000000000004, 0.90000000000000002]
>
> I got these results by:
>>>>[x/10. for x in xrange(10)]
>
> I'm not looking for alternatives, just for the reason that it works this
> way.

The reason is that due to the rounding issues inherent to a binary
representation of floating point, it is easy to give examples where
it's ambiguous whether the end point was meant to be included or not.
This could happen e.g. when the binary representation of the end point
was rounded up a bit, while the step would be rounded down.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)



      
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081105/848cd70c/attachment.htm>

From martin at v.loewis.de  Wed Nov  5 20:10:17 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 05 Nov 2008 20:10:17 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <85b5c3130811050104v54ec1dbfy9f800f7a94e33d38@mail.gmail.com>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	
	<4909EF67.5010104@trueblade.com>	
	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	
	<4909FECF.7000703@voidspace.org.uk>
	<490E0747.8050401@behnel.de>	 <490EE0A0.2060700@ghaering.de>	
	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>	
	<490F0784.4050408@ghaering.de> <geq6im$k8m$1@ger.gmane.org>	
	<4910AB32.7020309@v.loewis.de>
	<85b5c3130811050104v54ec1dbfy9f800f7a94e33d38@mail.gmail.com>
Message-ID: <4911EF99.9060609@v.loewis.de>

> I think it is also a good idea to write things using pure Python
> syntax in Cython, so that all other Python implementations, like
> Jython, Pypy, IronPython can just take it and run it in pure Python
> mode. Pure Python syntax means that the code runs in Python
> unmodified, but can also be compiled with Cython. Pure Python syntax
> was only recently added to Cython, so I guess it should be well tested
> first.
> 
> What do you think?

I wait. I have never dealt with Cython, and will leave it the time to
mature that its authors find necessary. When it is ready, we will see
what it can do.

Regards,
Martin


From martin at v.loewis.de  Wed Nov  5 20:15:47 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 05 Nov 2008 20:15:47 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	
	<geq1n0$vng$1@ger.gmane.org>	
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
Message-ID: <4911F0E3.9020607@v.loewis.de>

> Without a doubt the bazaar branches need a little more loving attention
> to make them a full working demo, but it's mostly details. The branches
> *do* contain the whole history, and not just 'select revisions':

But there are dozens of branches which aren't represented, plus all the
tags (IIUC). Those would need to get converted also, don't you agree?

Regards,
Martin

From barry at python.org  Wed Nov  5 20:38:39 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 5 Nov 2008 14:38:39 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4911F0E3.9020607@v.loewis.de>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	
	<geq1n0$vng$1@ger.gmane.org>	
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
	<4911F0E3.9020607@v.loewis.de>
Message-ID: <45D2E849-A598-4B48-9850-EED801DC51CC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 5, 2008, at 2:15 PM, Martin v. L?wis wrote:

>> Without a doubt the bazaar branches need a little more loving  
>> attention
>> to make them a full working demo, but it's mostly details. The  
>> branches
>> *do* contain the whole history, and not just 'select revisions':
>
> But there are dozens of branches which aren't represented, plus all  
> the
> tags (IIUC). Those would need to get converted also, don't you agree?

Yes.  My understanding, though I haven't tried it yet, is that newer  
versions of the bzr-svn plugin do a good job at a full conversion.   
Basically, every svn branch becomes a bzr branch and all svn tags are  
converted to bzr tags, which are not separate branches, but actual  
symbolic names for revisions in other branches.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRH2P3EjvBPtnXfVAQK/UAP+IuE4hOuixfS/YMv/7/8LYIABIkSxX7BT
eO/9295aSeKnY57h32IGyBqPnd67fKYaJ58Be8fDdC3P15ks00wCFIsS4Q2v0LKI
pDAxAn5GDRKQFoPZUnpMQXvBwEE9S08jEM7a24cyX111JHX0kVmcFy6nzDATQAv/
g8n03TgS1/g=
=AC1n
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Wed Nov  5 20:44:50 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 05 Nov 2008 20:44:50 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <45D2E849-A598-4B48-9850-EED801DC51CC@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	
	<geq1n0$vng$1@ger.gmane.org>	
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
	<4911F0E3.9020607@v.loewis.de>
	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>
Message-ID: <4911F7B2.6020403@v.loewis.de>

> Yes.  My understanding, though I haven't tried it yet, is that newer
> versions of the bzr-svn plugin do a good job at a full conversion. 
> Basically, every svn branch becomes a bzr branch and all svn tags are
> converted to bzr tags, which are not separate branches, but actual
> symbolic names for revisions in other branches.

Unless somebody had committed to the tag - right?

Regards,
Martin

From novalis at openplans.org  Wed Nov  5 20:48:19 2008
From: novalis at openplans.org (David Turner)
Date: Wed, 05 Nov 2008 14:48:19 -0500
Subject: [Python-Dev] AST-level type inference optimizations
Message-ID: <1225914499.32372.18.camel@gentle>

I wrote a patch to Tom Lee's AST optimization branch, which I have
submitted at http://bugs.python.org/issue4264.  Here's a brief
explanation of what the patch does, followed by a little general
discussion of the future.

Python bytecode includes at least one operation which is not directly
accessible from Python code: LIST_APPEND, which pops a value and a list
off the stack and appends the value to the list. This is used in
generating code for list comprehensions:

[x for x in somelist]

generates the following code for the body of the loop:

...
LOAD_FAST 1#a local is generated to hold the new list
LOAD_FAST 2 #the iteration variable, x
LIST_APPEND
...

Whereas if you were to try to write the comprehension directly, you
would get:

LOAD_FAST 1
LOAD_ATTR 0 #an index into the constant table: ?append?
LOAD_FAST 2
CALL_FUNCTION 1 #remove x and the append function from the top of the
stack and push the result of the call
POP_TOP

This is much slower. In part, it?s the cost of doing the attribute
lookup each time, which is why it?s common to see code like
a = []
ap = a.append
for x in ...: 
   ap(x + ...) 
return a 

But the function call is naturally slower than the simpler LIST_APPEND
operation. My patch tries to determine statically if a local is all
circumstances holds a list, and if so, generates LIST_APPEND whenever
user code would call local.append.   It also works with the manual
optimization mentioned above -- it tracks whether variables hold a
method call to append on a list.  It?s not perfect ? in particular, I
could track local types on a more fine-grained level than per-function.
Anyway, the patch is a win on microbenchmarks. 

There's a lot more we could do with function-local type inference.  We
could eliminate temp variables, and unnecessary stores to variables that
are never used again.  We could potentially track the types of what goes
into lists, and generate special-case code for numbers (using array), or
strings (using stringio or something).  And there's probably more than
I'm not thinking of right now.

What do you think?


From barry at python.org  Wed Nov  5 20:53:07 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 5 Nov 2008 14:53:07 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4911F7B2.6020403@v.loewis.de>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	
	<geq1n0$vng$1@ger.gmane.org>	
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
	<4911F0E3.9020607@v.loewis.de>
	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>
	<4911F7B2.6020403@v.loewis.de>
Message-ID: <6610D2F2-9E99-4E43-9049-29C2143AA9D2@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 5, 2008, at 2:44 PM, Martin v. L?wis wrote:

>> Yes.  My understanding, though I haven't tried it yet, is that newer
>> versions of the bzr-svn plugin do a good job at a full conversion.
>> Basically, every svn branch becomes a bzr branch and all svn tags are
>> converted to bzr tags, which are not separate branches, but actual
>> symbolic names for revisions in other branches.
>
> Unless somebody had committed to the tag - right?

That would be insane, right? :)

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRH5o3EjvBPtnXfVAQLsYwQAoFSPuMOblPNegZNH63Oz8SrDEG9QMLqO
ATvH/HHJAwuQ7Qz4WFiPLJbxf/JWLZZBKM2Idfk5z6icRXr1ynWbPn2SkcTtcAZ6
0G4KCYH/luOPJWTuzuWU+DBLp372Sp3psj2nJXnmmNgaxZyBZospe16pJS+mhpds
cIUc6VEGGCI=
=cMwI
-----END PGP SIGNATURE-----

From thomas at python.org  Wed Nov  5 20:56:08 2008
From: thomas at python.org (Thomas Wouters)
Date: Wed, 5 Nov 2008 20:56:08 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4911F0E3.9020607@v.loewis.de>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<geq1n0$vng$1@ger.gmane.org>
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
	<4911F0E3.9020607@v.loewis.de>
Message-ID: <9e804ac0811051156t7b6f5255sa286addf0cb0ac0@mail.gmail.com>

On Wed, Nov 5, 2008 at 20:15, "Martin v. L?wis" <martin at v.loewis.de> wrote:

> > Without a doubt the bazaar branches need a little more loving attention
> > to make them a full working demo, but it's mostly details. The branches
> > *do* contain the whole history, and not just 'select revisions':
>
> But there are dozens of branches which aren't represented, plus all the
> tags (IIUC). Those would need to get converted also, don't you agree?
>

Before we can make the switch to Bazaar, sure -- if we do. But there is only
a little history in those branches, compared to the history in the branches
we have in bazaar now (trunk, 3.0, 2.6, 2.5), and the "demo" of how it would
work is not affected by those other branches at all.

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081105/c01bd64c/attachment.htm>

From stefan_ml at behnel.de  Wed Nov  5 20:56:10 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 05 Nov 2008 20:56:10 +0100
Subject: [Python-Dev] Using Cython for standard library?
In-Reply-To: <gesml3$99c$1@ger.gmane.org>
References: <392528d30810300855g33b8130flc3098f81700bab08@mail.gmail.com>	<392528d30810300913w723271d2i855b4d78fc2cd9b5@mail.gmail.com>	<gecpks$cgc$1@ger.gmane.org>	<gecqd4$g9g$1@ger.gmane.org>	<ca471dc20810301025p47843333yff585b8507d9d521@mail.gmail.com>	<4909EF67.5010104@trueblade.com>	<ca471dc20810301034x7d6adb88wb875100e648d5c28@mail.gmail.com>	<4909FECF.7000703@voidspace.org.uk>	<490E0747.8050401@behnel.de>	<490EE0A0.2060700@ghaering.de>	<76fd5acf0811030546y1984fd8fx467661351de22975@mail.gmail.com>	<490F0784.4050408@ghaering.de>	<18703.3039.835260.398411@montanaro-dyndns-org.local>	<geq74l$mi4$1@ger.gmane.org>	<geqf9k$l09$1@ger.gmane.org>	<loom.20081105T125602-719@post.gmane.org>
	<gesml3$99c$1@ger.gmane.org>
Message-ID: <gestoq$43r$1@ger.gmane.org>

Terry Reedy wrote:
> Stefan Behnel wrote:
>> One really neat feature is that you can put a .pxd file next to your .py
>> file and let it override the function signatures and classes. So you do
>> not even need Py3 annotations, which have the obvious disadvantage of
>> requiring Py3.
> 
> That is not much of a disadvantage for code intended to be compiled.

... unless you have a reason for caring about a pure Python mode, which
usually is because you want it to run in pure Python. :)

When Cython becomes part of the stdlib, say, in Py3.1 and/or 2.7, this means
that those future Python versions that support annotations will (mostly) also
support Cython compilation. Earlier CPython platforms and non-CPython
interpreters are the problem here.


> Support for nested defs and generators is much more critical.

Agreed. Given the current status of the implementation, that's only a question
of time, not a technical issue.


> Typo? Should 'typedef(cython.p_int)' be 'cython.typedef(cython.p_int)'?
> Also, the 'enhancement' page has separate misspelled 'seperate' twice.

Thanks for catching those. I also added a link to the Wiki.

Stefan


From cstejerean at gmail.com  Wed Nov  5 21:00:25 2008
From: cstejerean at gmail.com (Cosmin Stejerean)
Date: Wed, 5 Nov 2008 14:00:25 -0600
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <18705.58131.933909.115314@montanaro-dyndns-org.local>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
Message-ID: <4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>


On Nov 5, 2008, at 12:16 PM, skip at pobox.com wrote:

>
> I apologize that I haven't read whatever Brett's written so far, but  
> I just
> haven't had time, and don't know if there's a PEP yet (and if so,  
> what its
> number is).  I did want to get my questions/confusion on the record  
> though.
>
> What DVCS fits my poor brain best?  I feel I'm like a dinosaur not  
> being
> able to figure out how I'm supposed to contribute changes to the  
> system.
>
> Do I:
>
>    * commit my changes to some central branch?

Not exactly. If you had commit access to the central repository you  
could commit then push, which would be the DVCS equivalent of  
committing to a central branch.

>    * commit my changes locally then create diffs I then submit to the
>      tracker?

Possible.

>    * commit locally then push them somewhere?

If you have commit access this would be the way to get your changes in.

>    * commit locally then ask someone to pull?
>

Often preferred way to submit patches, as you can continue to maintain  
the patch locally against newer versions of trunk so that the patch is  
not obsolete by the time people finally get around to it.

>    * Not commit anything anywhere but just submit patches to the  
> tracker?
>

Likely possible, but it's good to have the patch committed locally so  
you can modify it or continue to build upon it until it gets accepted.

> In addition:
>
>    * Will there be a central repository?
>

Generally there should be a central authoritative repository where  
people can turn to for the official version.

>    * How will I know which of possibly many repos is "authoritative"?
>

The authoritative repo should generally be linked to from the website  
so that people can easily find it.

>    * How will I discover other repos?  For example, if the  
> safethread stuff
>      is sitting somewhere in a repository can I find it on my own  
> somehow?
>

I'm not aware of any decentralized system for discovering  
repositories. Something like github or bitbucket could be used which  
help you discover repositories, but a wiki page with a list of  
alternative repositories and their purpose should suffice.

>    * Will a DVCS allow simpler operation as if we are still using a
>      centralized system like CVS or Subversion?
>

I think commit locally then push (if you have commit access) or email  
a patch (or upload to tracker, etc) can work almost exactly as the  
subversion workflow (the only differences being possible slight  
variations on the command line interface). DVCS should enable more  
interesting workflows (like working against multiple remote  
repositories), but it shouldn't prevent the simple uses cases that  
people are used to from a centralized VCS.

- Cosmin


From martin at v.loewis.de  Wed Nov  5 21:45:00 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 05 Nov 2008 21:45:00 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <9e804ac0811051156t7b6f5255sa286addf0cb0ac0@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	
	<geq1n0$vng$1@ger.gmane.org>	
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>	
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>	
	<4911F0E3.9020607@v.loewis.de>
	<9e804ac0811051156t7b6f5255sa286addf0cb0ac0@mail.gmail.com>
Message-ID: <491205CC.30905@v.loewis.de>

> Before we can make the switch to Bazaar, sure -- if we do.

That is my whole point. Before we switch to whatever DVCS, this
system should have a complete installation, with all pieces in
place.

I was just pointing out that the bazaar installation is not
complete in this respect - I was not asking that it be completed,
at least not at this point in time.

Regards,
Martin

From brett at python.org  Wed Nov  5 21:48:49 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Nov 2008 12:48:49 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
Message-ID: <bbaeab100811051248g1c7760a2ue669581e823b5321@mail.gmail.com>

On Wed, Nov 5, 2008 at 03:09, Paul Moore <p.f.moore at gmail.com> wrote:
> 2008/11/3 Brett Cannon <brett at python.org>:
>> At this point I am looking for any suggestions for fundamental usage
>> scenarios that I am missing from the PEP. If you think the few already
>> listed are missing some core part of a VCS, please let me know.
>
> My apologies, I can't check if this is already there (firewall rules
> from this account :-() but can I suggest the following use case:
>
> An average user (ie, not a core developer) finds an issue, and has an
> idea how to fix it. He raises a tracker item, checks out the Python
> sources, makes a fix, and wants to upload it to the tracker. Key
> points here are the initial work needed to grab a development
> checkout, and the ability to bundle up a fix for upload to the
> tracker. (I'm specifically thinking of a casual user, not a developer
> who already has a Python checkout to work on).
>

It's the first scenario, so yes, it's covered.

-Brett

From brett at python.org  Wed Nov  5 21:52:11 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Nov 2008 12:52:11 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <79990c6b0811050735g240816cfm8ddc552343eea7e4@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<20081105140428.GA20949@vidar.dreamhost.com>
	<79990c6b0811050735g240816cfm8ddc552343eea7e4@mail.gmail.com>
Message-ID: <bbaeab100811051252se32d4ecief61ca0b541dbade@mail.gmail.com>

On Wed, Nov 5, 2008 at 07:35, Paul Moore <p.f.moore at gmail.com> wrote:
> 2008/11/5 David Ripton <dripton at ripton.net>:
>> All timings very approximate:
>>
>> Time for average user to check out Python sources with bzr: 10 minutes
>>
>> Time for average user to check out Python sources with git or hg: 1 minute
>>
>> Time for average user's trivial patch to be reviewed and committed: 1 year
>>
>> I love DVCS as much as the next guy, but checkout time is so not the
>> bottleneck for this use case.
>
> :-) That's a fair point. But it's not the point I was trying to make,
> which is that I'd want whatever DVCS is chosen to make the initial
> experience of a casual user / newcomer as easy as possible. Why
> discourage them in the first 10 minutes (which, BTW, is much faster
> than my experience with bzr last time I tried the Python repo) when we
> can make them suffer for a whole year? :-) :-)

And improving our tool set and issue workflow is to help with that
time lag from patch submission to commit. We have to fix the means to
the end in order to improve the end itself.

-Brett

From brett at python.org  Wed Nov  5 21:57:48 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Nov 2008 12:57:48 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <18705.58131.933909.115314@montanaro-dyndns-org.local>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
Message-ID: <bbaeab100811051257h357b0bcak273968981c00cbdf@mail.gmail.com>

On Wed, Nov 5, 2008 at 10:16,  <skip at pobox.com> wrote:
>
> I apologize that I haven't read whatever Brett's written so far, but I just
> haven't had time, and don't know if there's a PEP yet (and if so, what its
> number is).  I did want to get my questions/confusion on the record though.
>

It's currently just a draft on Google Docs:
http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 .

> What DVCS fits my poor brain best?  I feel I'm like a dinosaur not being
> able to figure out how I'm supposed to contribute changes to the system.

I will add an Typical Workflow section to the PEP so people get an
idea of what it would most likely be like.

-Brett

From brett at python.org  Wed Nov  5 21:55:24 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Nov 2008 12:55:24 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <491205CC.30905@v.loewis.de>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<geq1n0$vng$1@ger.gmane.org>
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
	<4911F0E3.9020607@v.loewis.de>
	<9e804ac0811051156t7b6f5255sa286addf0cb0ac0@mail.gmail.com>
	<491205CC.30905@v.loewis.de>
Message-ID: <bbaeab100811051255h8e5ad8do616c74012699dd50@mail.gmail.com>

On Wed, Nov 5, 2008 at 12:45, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Before we can make the switch to Bazaar, sure -- if we do.
>
> That is my whole point. Before we switch to whatever DVCS, this
> system should have a complete installation, with all pieces in
> place.
>
> I was just pointing out that the bazaar installation is not
> complete in this respect - I was not asking that it be completed,
> at least not at this point in time.
>

And I agree with Martin we will need a full install, but might as well
see if we can eliminate a DVCS from the list first before going to the
trouble of getting a full installation going.

-Brett

From ncoghlan at gmail.com  Wed Nov  5 22:31:18 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 06 Nov 2008 07:31:18 +1000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <79990c6b0811050731n26c57861u7ffabe2505cf1782@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>	<D5B12534-4DB3-4FA5-90E6-88CDDB3E8C03@python.org>
	<79990c6b0811050731n26c57861u7ffabe2505cf1782@mail.gmail.com>
Message-ID: <491210A6.1080305@gmail.com>

Paul Moore wrote:
> 2008/11/5 Barry Warsaw <barry at python.org>:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> On Nov 5, 2008, at 6:09 AM, Paul Moore wrote:
>>
>>> I'll freely admit a (not very) hidden bias here - the slowness of an
>>> initial clone (or going through the "download a shared repo, unpack
>>> it, create a branch and update" rigmarole) makes this a nasty test for
>>> Bazaar. But I do nevertheless think it's an important use case, as
>>> it's all about encouraging casual users to contribute.
>> One thing that we're not taking advantage of on code.python.org for Bazaar
>> is "stacked branches".  This is basically an indirection technique that
>> doesn't require a full revision history download to get started.  Stacked
>> branches greatly improve performance for the initial pull and push, and
>> reduces disk usage on the server.
> 
> While that would be good, my understanding is that stacked branches in
> Bazaar only work (for history operations) while you're online. So they
> make Bazaar work a little like a centralised VCS, I guess. Not sure
> how that's a step forward :-)

You only need full history in order to ask "how did this code get to
this state" type questions. That's something a core dev will
occasionally want to know (which is why we would take to the time to
setup a full checkout no matter how long it takes), but utterly
irrelevant to a casual committer such as the one in your "write and
submit a quick patch" scenario.

Also, the DVCSs shouldn't be compared just against each other - we need
the benchmark numbers against the current SVN repository as well.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From p.f.moore at gmail.com  Thu Nov  6 00:24:15 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 5 Nov 2008 23:24:15 +0000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <491210A6.1080305@gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<D5B12534-4DB3-4FA5-90E6-88CDDB3E8C03@python.org>
	<79990c6b0811050731n26c57861u7ffabe2505cf1782@mail.gmail.com>
	<491210A6.1080305@gmail.com>
Message-ID: <79990c6b0811051524p17a19ebv67cba63a4c5361a@mail.gmail.com>

2008/11/5 Nick Coghlan <ncoghlan at gmail.com>:
> Paul Moore wrote:
>> While that would be good, my understanding is that stacked branches in
>> Bazaar only work (for history operations) while you're online. So they
>> make Bazaar work a little like a centralised VCS, I guess. Not sure
>> how that's a step forward :-)
>
> You only need full history in order to ask "how did this code get to
> this state" type questions. That's something a core dev will
> occasionally want to know (which is why we would take to the time to
> setup a full checkout no matter how long it takes), but utterly
> irrelevant to a casual committer such as the one in your "write and
> submit a quick patch" scenario.

Agreed. But I'm very hazy on how Bazaar stacked branches work in
practice. If I create a nice fast stacked branch on my PC, then go
offline, precisely what operations can I no longer do? And will the
ones that can't work fully, work "for the first bit" (by which I mean
something like bzr log | less, exiting before I page through the lot -
that's my normal history search process, because my poor brain is too
slow to remember the flags to limit the history :-))?

Anyway, this is probably getting too much into details to be useful.
Paul.

From tseaver at palladion.com  Thu Nov  6 01:53:32 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 05 Nov 2008 19:53:32 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4911F7B2.6020403@v.loewis.de>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>		<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>		<geq1n0$vng$1@ger.gmane.org>		<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>		<4910BE7F.9040900@gmail.com>
	<4910CA8D.9010303@v.loewis.de>	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>	<4911F0E3.9020607@v.loewis.de>	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>
	<4911F7B2.6020403@v.loewis.de>
Message-ID: <4912400C.4060204@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin v. L?wis wrote:
>> Yes.  My understanding, though I haven't tried it yet, is that newer
>> versions of the bzr-svn plugin do a good job at a full conversion. 
>> Basically, every svn branch becomes a bzr branch and all svn tags are
>> converted to bzr tags, which are not separate branches, but actual
>> symbolic names for revisions in other branches.
> 
> Unless somebody had committed to the tag - right?

svn doesn't have any true tags, AFAIK:  everything is a branch.
Committing to a tag for "housekeeping" is a not-uncommon pattern in some
projects (e.g., to avoid making spurious housekeeping commits on the
source branch which have to be undone after making the tag).  The
canonical example would be changing the 'version' argument passed to
distutils / setuptools setup().

Under this culture / pattern, one would tag the release, then modify the
version on the quasi-tag branch, rather than committing the released
version of setup.py to the source branch, only to follow it immediately
with another commit reverting the release-specific change.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJEkAM+gerLs4ltQ4RAoztAKDKP4kACUon4vWNg78eZHg3ZL45CgCgkfn0
yXz5q1bhwWs77u3j/6isgOQ=
=oqoH
-----END PGP SIGNATURE-----


From stephen at xemacs.org  Thu Nov  6 02:09:14 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Nov 2008 10:09:14 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <6610D2F2-9E99-4E43-9049-29C2143AA9D2@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<geq1n0$vng$1@ger.gmane.org>
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
	<4911F0E3.9020607@v.loewis.de>
	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>
	<4911F7B2.6020403@v.loewis.de>
	<6610D2F2-9E99-4E43-9049-29C2143AA9D2@python.org>
Message-ID: <87y6zxsls5.fsf@uwakimon.sk.tsukuba.ac.jp>

Barry Warsaw writes:
 > -----BEGIN PGP SIGNED MESSAGE-----
 > Hash: SHA1
 > 
 > On Nov 5, 2008, at 2:44 PM, Martin v. L?wis wrote:

 > > Unless somebody had committed to the tag - right?
 > 
 > That would be insane, right? :)

AFAIK it's not insane, just impossible.

Of course in any system you can use "vc tag --force TAG-TO-OVERWRITE".

From stephen at xemacs.org  Thu Nov  6 02:36:28 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Nov 2008 10:36:28 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
Message-ID: <87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>

In what follows, caveat IANB (I am not Brett, and neither is
Cosmin<wink>), but there is some experience with these systems, and my
recommendations are based on that.

Cosmin Stejerean writes:
 > On Nov 5, 2008, at 12:16 PM, skip at pobox.com wrote:

 > > What DVCS fits my poor brain best?  I feel I'm like a dinosaur
 > > not being able to figure out how I'm supposed to contribute
 > > changes to the system.

You need not feel that way.  It's not you---the flexibility of dVCS
means that until the Powers That Be promulgate a Workflow, this will
be ambiguous.

This is part of the purpose of the PEP.  We[1] will be presenting the
5-finger exercises required to accomplish typical (and perhaps some
not-so-typical) tasks, as well as benchmarks for the various systems.

 > > Do I:
 > >
 > >    * commit my changes to some central branch?

Call this the "record && commit to authoritative" workflow.

 > Not exactly. If you had commit access to the central repository you  
 > could commit then push, which would be the DVCS equivalent of  
 > committing to a central branch.

The workflow where general contributors commit directly to the trunk
surely won't be used in Python, because of the instability it would
cause.  It would be possible to have a staging branch for this
purpose, but IMO that's not a very effective use of a dVCS.[2]

It is useful to avoid the term "commit" here because its semantics
vary across systems.  As Cosmin points out, in a dVCS, what is
accomplished by "vc commit" in CVS is done as "vc commit; vc push".  I
use the terminology "record" for the action of adding a workspace-
based patch or snapshot to a repository.  "push" (and "pull") move
content between repositories.  Unfortunately "commit" is the name of
the record command in most dVCSes, so this terminology probably won't
catch on.

Also, when talking about "where to commit" in terms of communication
among developers, you should probably refer to storage locations as
"repositories".  "Branch" is another term that has varying semantics
in different VCSes.  In some systems (git) it is reasonable to think
of repositories containing more than one branch, and branches as
existing in more than one repository (but this isn't quite robust in
git because branch names are just names, not first-class objects).  In
others (Darcs is the extreme) repository == branch == workspace.

(I'm trying to get permission to publish a 3rd party's draft document
that goes into these issues in detail; here I just want to raise
awareness that the intuitions that go with CVS/Subversion usage of
various terms is *not* always going to carry over to dVCSes.)

 > >    * commit my changes locally then create diffs I then submit to the
 > >      tracker?

"Record && patch" workflow.

 > Possible.

But again not very effective.  Under a dVCS I believe these patches
will languish in the tracker as they do today, unless tools are
written to automatically pull them into a repo somewhere.

 > >    * commit locally then push them somewhere?

"Record && push to candidate" workflow.

If we go with Bazaar, this is very likely to occur, especially if
Canonical's launchpad is the host.  This is what Linux kernel does on
git.kernel.org as well, if I understand their workflow correctly, and
what github helps to support.  I imagine Mercurial has an equivalent
but I'm not familiar with it.

 > >    * commit locally then ask someone to pull?

"Record && request pull" workflow.

 > Often preferred way to submit patches, as you can continue to maintain  
 > the patch locally against newer versions of trunk so that the patch is  
 > not obsolete by the time people finally get around to it.

I disagree.  This doesn't scale to Python size.  For distributed VC to
work, somebody has to maintain a repo 24x7.  Python has to do this for
the trunk; the additional burden for contributed patches is not great.
There is no real advantage to having contributors do so, too.[3]
Integrators and interested third parties also must keep track of
contributor's repo URLs.  (Cf. Skip's question about discovering repos.)
Not happy stuff.

The "record && push" workflow scales much better for numbers of
contributors, as each contributor needs only to maintain one "push"
URL, and integrators only one "pull" base URL.

 > >    * Not commit anything anywhere but just submit patches to the  
 > > tracker?

"Patch from workspace" workflow.

 > Likely possible, but it's good to have the patch committed locally so  
 > you can modify it or continue to build upon it until it gets accepted.

The same considerations as "record && patch" also apply here.

 > > In addition:
 > >
 > >    * Will there be a central repository?
 >
 > Generally there should be a central authoritative repository where  
 > people can turn to for the official version.

Ie, "yes".  There's no point in a PEP unless there's going to be a
central repo and a defined workflow for getting contributions into it.

Note that you can always maintain your own local repo with dVCS.

 > >    * How will I know which of possibly many repos is "authoritative"?
 > 
 > The authoritative repo should generally be linked to from the website  
 > so that people can easily find it.

That depends.  The notion of "authoritative" gets weakened in a
distributed system, and probably more important is "which repo will be
used to make the next official release".

However, although I can't say what the mechanism will be, be sure you
will not have a problem learning which is authoritative for the trunk
or where to find RCs and releases.  (If you do, it's a doc problem and
it will be fixed quickly.)

You may have more trouble with third-party patches gotten from
third-party repos.  GNU Arch has a system for handling this (patch
names contain the originating repo).  That was one of the first things
the Bazaar people discarded from Arch, though.  Darcs has something
similar, but again Darcs is not a candidate here.  I think for such
"maverick" contributions there will never really be a substitute for
watching the ML and tracker like a hawk.

 > >    * How will I discover other repos?  For example, if the
 > >      safethread stuff is sitting somewhere in a repository can I
 > >      find it on my own somehow?
 > 
 > I'm not aware of any decentralized system for discovering  
 > repositories. Something like github or bitbucket could be used which  
 > help you discover repositories, but a wiki page with a list of  
 > alternative repositories and their purpose should suffice.

Most likely the repos you care about will be hosted in a central
location, and will be browsable from a single base URL.  See
http://git.kernel.org/ for the git version of a browser.  Mercurial
and Bazaar support similar facilities either in the VCS itself or in
an easily available add-on.  Fancier support is available via systems
like github and Launchpad.

If you care about minor, locally hosted, patches, you'll have to
follow the tracker and mailing list closely and grep out the URLs to
them.  However, systems like Launchpad and github make it feasible to
have branches for a single patch.

GNU Arch also had some systems for third-party repo discovery and
maintenance of a database of them, but they ended up creating a
"Supermirror" which gave a similar workflow to Launchpad/github et
al.  So I think you can probably discount the existence or development
of such a discovery system.

 > >    * Will a DVCS allow simpler operation as if we are still using a
 > >      centralized system like CVS or Subversion?

Yes and no.  There is nothing to prevent a formal workflow like that
in CVS/Subversion.  However, the separation of "commit" into "record
&& push to authoritative" leaves open the possibility of annoying
glitches until you get used to it, and even then it's easy to forget
to push or to forget that you've committed not-for-pushing stuff, etc,
etc.  In practice it is probably simpler to use a dVCS-specialized
workflow like "record && push to candidate".

Footnotes: 
[1]  I have time constraints which may not be acceptable to Brett, but
he's offered to delegate the presentation of git to me, and those of
hg and bzr to others.

[2]  This is how XEmacs and Scheme48 do it.

[3]  The insurance value is small.  Integrators and other interested
parties will surely be keeping local copies of almost all branches; if
the central repo were to be wiped out, most branches will be
recoverable from those widely distributed copies.

From stephen at xemacs.org  Thu Nov  6 03:56:17 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Nov 2008 11:56:17 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4912400C.4060204@palladion.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>
	<geq1n0$vng$1@ger.gmane.org>
	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>
	<4910BE7F.9040900@gmail.com> <4910CA8D.9010303@v.loewis.de>
	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>
	<4911F0E3.9020607@v.loewis.de>
	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>
	<4911F7B2.6020403@v.loewis.de> <4912400C.4060204@palladion.com>
Message-ID: <87prl9sgtq.fsf@uwakimon.sk.tsukuba.ac.jp>

Tres Seaver writes:

 > svn doesn't have any true tags, AFAIK:  everything is a branch.

Yow!  I couldn't have imagined that would be true.  And didn't....


From brett at python.org  Thu Nov  6 05:50:38 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Nov 2008 20:50:38 -0800
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
Message-ID: <bbaeab100811052050x719b6fcdx9f9e24937e14ea1@mail.gmail.com>

On Tue, Nov 4, 2008 at 16:34, Brett Cannon <brett at python.org> wrote:
> OK, I give: git is in the running. But do realize it will take a lot
> for it to beat out bzr or hg.
>
> I have emailed some people who have shown allegiance to a specific
> DVCS to seeif they are willing to fill in the usage scenarios for me
> along with converting the dev FAQ so that we all have a clear way to
> learn how to use the things. If any of the people I emailed turn down
> being the leader fo ra specific DVCS I will email here looking for
> volunteers.

So my first choice for Mercurial fell through. If you would like to
represent Mercurial, let me know.

-Brett

From brett at python.org  Thu Nov  6 06:18:36 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 5 Nov 2008 21:18:36 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <bbaeab100811052118tae0d142r88edae7db1e55da1@mail.gmail.com>

On Wed, Nov 5, 2008 at 17:36, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> In what follows, caveat IANB (I am not Brett, and neither is
> Cosmin<wink>), but there is some experience with these systems, and my
> recommendations are based on that.
>

Wow, I'm part of an acronym! That's a first.

> Cosmin Stejerean writes:
>  > On Nov 5, 2008, at 12:16 PM, skip at pobox.com wrote:
>
>  > > What DVCS fits my poor brain best?  I feel I'm like a dinosaur
>  > > not being able to figure out how I'm supposed to contribute
>  > > changes to the system.
>
> You need not feel that way.  It's not you---the flexibility of dVCS
> means that until the Powers That Be promulgate a Workflow, this will
> be ambiguous.
>

It also took me quite a while to finally grasp exactly how the typical
workflow could go with a DVCS.

> This is part of the purpose of the PEP.  We[1] will be presenting the
> 5-finger exercises required to accomplish typical (and perhaps some
> not-so-typical) tasks, as well as benchmarks for the various systems.
>
>  > > Do I:
>  > >
>  > >    * commit my changes to some central branch?
>
> Call this the "record && commit to authoritative" workflow.
>
>  > Not exactly. If you had commit access to the central repository you
>  > could commit then push, which would be the DVCS equivalent of
>  > committing to a central branch.
>
> The workflow where general contributors commit directly to the trunk
> surely won't be used in Python, because of the instability it would
> cause.  It would be possible to have a staging branch for this
> purpose, but IMO that's not a very effective use of a dVCS.[2]
>

I assume by "general contributor" you mean "anybody" and not "core developer".

> It is useful to avoid the term "commit" here because its semantics
> vary across systems.  As Cosmin points out, in a dVCS, what is
> accomplished by "vc commit" in CVS is done as "vc commit; vc push".  I
> use the terminology "record" for the action of adding a workspace-
> based patch or snapshot to a repository.  "push" (and "pull") move
> content between repositories.  Unfortunately "commit" is the name of
> the record command in most dVCSes, so this terminology probably won't
> catch on.
>

This is why there is a Terminology section to the PEP; people have not
fully agreed on terms yet.

> Also, when talking about "where to commit" in terms of communication
> among developers, you should probably refer to storage locations as
> "repositories".  "Branch" is another term that has varying semantics
> in different VCSes.  In some systems (git) it is reasonable to think
> of repositories containing more than one branch, and branches as
> existing in more than one repository (but this isn't quite robust in
> git because branch names are just names, not first-class objects).  In
> others (Darcs is the extreme) repository == branch == workspace.
>
> (I'm trying to get permission to publish a 3rd party's draft document
> that goes into these issues in detail; here I just want to raise
> awareness that the intuitions that go with CVS/Subversion usage of
> various terms is *not* always going to carry over to dVCSes.)
>
>  > >    * commit my changes locally then create diffs I then submit to the
>  > >      tracker?
>
> "Record && patch" workflow.
>
>  > Possible.
>
> But again not very effective.  Under a dVCS I believe these patches
> will languish in the tracker as they do today, unless tools are
> written to automatically pull them into a repo somewhere.
>
>  > >    * commit locally then push them somewhere?
>
> "Record && push to candidate" workflow.
>
> If we go with Bazaar, this is very likely to occur, especially if
> Canonical's launchpad is the host.  This is what Linux kernel does on
> git.kernel.org as well, if I understand their workflow correctly, and
> what github helps to support.  I imagine Mercurial has an equivalent
> but I'm not familiar with it.
>

http://www.bitbucket.org has free Mercurial hosting for open source projects.

As for how git.kernel.org works, I believe that won't work for Python
without a cultural shift in how Python development happens. Linus has
subsystem maintainers who are in charge of certain subsections of the
Linux kernel. They are the ones that accept the various patches from
people contributing. I believe Linus then pulls from these people into
his tree which is basically the authoritative tree for the kernel.

For Python development we don't really have subsystem maintainers. We
have unofficial ones (e.g. Georg with the docs), but I honestly think
that this has not worked for us over the years as people come and go.
I think part of the reason the Linux kernel can get away with the
structure it has is because people get paid to help maintain it while
we don't. Plus I don't think Guido wants to act as BDFL on every
potential patch into Python with his 50% time.

>  > >    * commit locally then ask someone to pull?
>
> "Record && request pull" workflow.
>
>  > Often preferred way to submit patches, as you can continue to maintain
>  > the patch locally against newer versions of trunk so that the patch is
>  > not obsolete by the time people finally get around to it.
>
> I disagree.  This doesn't scale to Python size.  For distributed VC to
> work, somebody has to maintain a repo 24x7.  Python has to do this for
> the trunk; the additional burden for contributed patches is not great.
> There is no real advantage to having contributors do so, too.[3]
> Integrators and interested third parties also must keep track of
> contributor's repo URLs.  (Cf. Skip's question about discovering repos.)
> Not happy stuff.
>
> The "record && push" workflow scales much better for numbers of
> contributors, as each contributor needs only to maintain one "push"
> URL, and integrators only one "pull" base URL.
>
>  > >    * Not commit anything anywhere but just submit patches to the
>  > > tracker?
>
> "Patch from workspace" workflow.
>
>  > Likely possible, but it's good to have the patch committed locally so
>  > you can modify it or continue to build upon it until it gets accepted.
>
> The same considerations as "record && patch" also apply here.
>
>  > > In addition:
>  > >
>  > >    * Will there be a central repository?
>  >
>  > Generally there should be a central authoritative repository where
>  > people can turn to for the official version.
>
> Ie, "yes".  There's no point in a PEP unless there's going to be a
> central repo and a defined workflow for getting contributions into it.
>
> Note that you can always maintain your own local repo with dVCS.
>
>  > >    * How will I know which of possibly many repos is "authoritative"?
>  >
>  > The authoritative repo should generally be linked to from the website
>  > so that people can easily find it.
>
> That depends.  The notion of "authoritative" gets weakened in a
> distributed system, and probably more important is "which repo will be
> used to make the next official release".
>
> However, although I can't say what the mechanism will be, be sure you
> will not have a problem learning which is authoritative for the trunk
> or where to find RCs and releases.  (If you do, it's a doc problem and
> it will be fixed quickly.)
>

Right. The concept of the "trunk" will continue to exist, as will what
the official maintenance repos and those locations will be written
down.

> You may have more trouble with third-party patches gotten from
> third-party repos.  GNU Arch has a system for handling this (patch
> names contain the originating repo).  That was one of the first things
> the Bazaar people discarded from Arch, though.  Darcs has something
> similar, but again Darcs is not a candidate here.  I think for such
> "maverick" contributions there will never really be a substitute for
> watching the ML and tracker like a hawk.
>
>  > >    * How will I discover other repos?  For example, if the
>  > >      safethread stuff is sitting somewhere in a repository can I
>  > >      find it on my own somehow?
>  >
>  > I'm not aware of any decentralized system for discovering
>  > repositories. Something like github or bitbucket could be used which
>  > help you discover repositories, but a wiki page with a list of
>  > alternative repositories and their purpose should suffice.
>
> Most likely the repos you care about will be hosted in a central
> location, and will be browsable from a single base URL.  See
> http://git.kernel.org/ for the git version of a browser.  Mercurial
> and Bazaar support similar facilities either in the VCS itself or in
> an easily available add-on.  Fancier support is available via systems
> like github and Launchpad.
>

We have http://code.python.org/ for this. And yes, you kind of just
have to know since any random branches that might be out there will
not be in the branches/ directory like in svn. But honestly how often
does anyone just browse the branches/ directory anyway?

-Brett

From stephen at xemacs.org  Thu Nov  6 07:16:00 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 06 Nov 2008 15:16:00 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811052118tae0d142r88edae7db1e55da1@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
	<bbaeab100811052118tae0d142r88edae7db1e55da1@mail.gmail.com>
Message-ID: <87od0ts7kv.fsf@uwakimon.sk.tsukuba.ac.jp>

Brett Cannon writes:

 > > You need not feel that way.  It's not you---the flexibility of dVCS
 > > means that until the Powers That Be promulgate a Workflow, this will
 > > be ambiguous.
 > 
 > It also took me quite a while to finally grasp exactly how the typical
 > workflow could go with a DVCS.

I don't think there is a "typical" workflow for DVCS yet.  For
example, unreconstructed Archers generally do use a "record && request
pull" (from their local repos) workflow.  On the other hand, XEmacs
treats its Mercurial repository as "CVS that can do merges sanely"---
patches are submitted to a mailing list for review, but I don't think
any of the core devs' patches ever get applied any more; rather they
go into the trunk quickly and devs and testers alike pull them from
there.  The Bazaar guys have a crazy quilt of "Bundle Buggies" and
"Patch Queue Managers" semi-integrated into Launchpad.  The Darcs
people have a surprisingly traditional "record && send patch" workflow.

My opinion is that Python's current workflow is pretty good, and
there's not reason to start anywhere but there.

As has been acerbically mentioned, like most mature projects with a
fairly close-knit group of core devs there are problems with casual
contributors' patches getting neglected.  I think that a DVCS can
contribute fairly directly to more efficient reviewing, and it will be
easier to make workflow improvements needed for efficient reviewing
with a DVCS.

Hm ... those require ideas would require the contributor to use the
DVCS.  That may be an argument for a Python-based VCS that could be
bundled with the sources, an hg-lite or bzr-lite.  ("But git is still
faster!"<wink>)

 > > The workflow where general contributors commit directly to the trunk
 > > surely won't be used in Python, because of the instability it would
 > > cause.
 > 
 > I assume by "general contributor" you mean "anybody" and not "core
 > developer".

That's right.  The current workflow where the senior committers "fire
at will", except when the svn repo is frozen for a release, could be
simulated exactly with any of the DVCSes.

It might make sense to have an alternative workflow that restricts the
"next-to-be-released" repo to the release manager, but that is a
discussion for later, not something that should be implemented
immediately (if at all).

 > This is why there is a Terminology section to the PEP; people have not
 > fully agreed on terms yet.

I forgot to mention that although your terminology is somewhat
different from what I recommend, so far your definitions seem coherent
and your usage accurate.  ("Seem" is not a reflection on you, but on
the inherent ambiguity in trying to adapt terms developed for CVS to
DVCSes.)

 > As for how git.kernel.org works, I believe that won't work for Python
 > without a cultural shift in how Python development happens. Linus has
 > subsystem maintainers who are in charge of certain subsections of the
 > Linux kernel.

In bringing up git.kernel.org, I'm not referring to the project
workflow, just to the ability to freely create branches and/or
subsidiary repos, and easily browse them on the host.  At least for
core devs, and in the case of Launchpad, I believe anyone with a
Launchpad user can create repos.

OTOH, any workflow changes can and IMO should be discussed and
implemented gradually.

 > We have http://code.python.org/ for this. And yes, you kind of just
 > have to know since any random branches that might be out there will
 > not be in the branches/ directory like in svn. But honestly how often
 > does anyone just browse the branches/ directory anyway?

I don't know about svn; I find it pretty painful to use for anything
but sequential tasking (update ... hack ... commit ... repeat).  But
if you look at http://git.kernel.org/, you'll see a thicket of repos,
each of which probably contains a few to a few hundred[1] branches
(mostly merged to repo mainline or abandoned, of course).  I hear
kernel devs *do* browse those to find out what each other are up to,
as do lkml wannabes.  I know that even locally, I often abandon tasks
on XEmacs where I've bitten off more than I want to chew at the
moment.  When I do free up some time, browsing my stale branches is a
common way to find a project.

Again, this possibility is not an imperative to "be like Linux
kernel", but it is an opportunity to improve the technology for
communication among developers.


Footnotes: 
[1]  Seriously.  git had to develop a packed format for its list of
branches and tags because people use so many of them it was impacting
performance!

From martin at v.loewis.de  Thu Nov  6 07:58:30 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 06 Nov 2008 07:58:30 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87y6zxsls5.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	<geq1n0$vng$1@ger.gmane.org>	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	<4910BE7F.9040900@gmail.com>	<4910CA8D.9010303@v.loewis.de>	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>	<4911F0E3.9020607@v.loewis.de>	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>	<4911F7B2.6020403@v.loewis.de>	<6610D2F2-9E99-4E43-9049-29C2143AA9D2@python.org>
	<87y6zxsls5.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <49129596.7030403@v.loewis.de>

>  > > Unless somebody had committed to the tag - right?
>  > 
>  > That would be insane, right? :)
> 
> AFAIK it's not insane, just impossible.

IIRC, I did that for the 2.5.2 tag (or some such), correcting
the spelling of "2st" to "2nd" for the release date.

Regards,
Martin

From g.brandl at gmx.net  Thu Nov  6 09:40:50 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 06 Nov 2008 09:40:50 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87prl9sgtq.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	<geq1n0$vng$1@ger.gmane.org>	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	<4910BE7F.9040900@gmail.com>
	<4910CA8D.9010303@v.loewis.de>	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>	<4911F0E3.9020607@v.loewis.de>	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>	<4911F7B2.6020403@v.loewis.de>
	<4912400C.4060204@palladion.com>
	<87prl9sgtq.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <geuaii$qrd$1@ger.gmane.org>

Stephen J. Turnbull schrieb:
> Tres Seaver writes:
> 
>  > svn doesn't have any true tags, AFAIK:  everything is a branch.
> 
> Yow!  I couldn't have imagined that would be true.  And didn't....

It's not exactly true either. svn doesn't have either tags or branches,
it has only a single (directory) tree per repository, of which individual
subdirectories can be checked out individually. This is the root of the
usual first step in creating a SVN repo: make "branches", "trunk" and
"tags" subdirs.

In this context it is not surprising that there is no separate branch or
tag concept. I think the reason is that CVS *had* branches and tags as
separate concepts and it must have been relatively painful to use them.

cheers,
Georg


From ncoghlan at gmail.com  Thu Nov  6 12:35:32 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 06 Nov 2008 21:35:32 +1000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87od0ts7kv.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>	<18705.58131.933909.115314@montanaro-dyndns-org.local>	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>	<bbaeab100811052118tae0d142r88edae7db1e55da1@mail.gmail.com>
	<87od0ts7kv.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4912D684.9060605@gmail.com>

Stephen J. Turnbull wrote:
> I don't know about svn; I find it pretty painful to use for anything
> but sequential tasking (update ... hack ... commit ... repeat).  But
> if you look at http://git.kernel.org/, you'll see a thicket of repos,
> each of which probably contains a few to a few hundred[1] branches
> (mostly merged to repo mainline or abandoned, of course).

If it isn't already there, suspending work on something to work on
something else would make a very nice scenario to cover, as it is
something even the core devs sometimes have to deal with.

I'd expect any DVCS to be able to handily beat what I currently do with SVN:

- # hack on first task
- svn diff > _patches/first_task.diff
- svn revert -R .
- svn update
- # hack on second task
- svn commit OR svn diff > _patches/second_task.diff (depending on
whether or not I want it reviewed by someone else before committing it)
- svn revert -R .
- patch -p0 < _patches/first_task.diff
- # back to hacking on first task

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From cournape at gmail.com  Thu Nov  6 12:45:55 2008
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 6 Nov 2008 20:45:55 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4912D684.9060605@gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
	<bbaeab100811052118tae0d142r88edae7db1e55da1@mail.gmail.com>
	<87od0ts7kv.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4912D684.9060605@gmail.com>
Message-ID: <5b8d13220811060345r6b971ee1j140983e547ac219@mail.gmail.com>

On Thu, Nov 6, 2008 at 8:35 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

>
> I'd expect any DVCS to be able to handily beat what I currently do with SVN:

Yes, it does. I have extensive experience in bzr, less in git (but
vastly prefer it since I have been using it), and both are relatively
good for this kind of scenario.

In git, you can for example create a new branch (often called feature
branch, also technically, it is just a branch):

git co -b newbranch1 # Create new branch1 and go into it
# hack, commit
git cb -n newbranch2 # create new branch2
# More hacking
git co newbranch1 # go back to branch 1

It can be more elaborate than that, using stash to set uncommitted
changes, rebase to update your changes on the top of the last revision
of the main branch you are tracking (trunk in svn parlance). Bzr is
similar, except that it is more commands to switch from branch to
branch; bzr has the shelve command to do as git stash. You can also
use loom, but I never used them extensively (loom are a plugin
extension of bzr, and not well documented; I could not understand it,
at least, although I did not try too hard).

cheers,

David

From gustavo at niemeyer.net  Thu Nov  6 13:04:55 2008
From: gustavo at niemeyer.net (Gustavo Niemeyer)
Date: Thu, 6 Nov 2008 10:04:55 -0200
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <643d90130811060404m7c9cd783o609c375b85ec48a8@mail.gmail.com>

>  > >    * Will a DVCS allow simpler operation as if we are still using a
>  > >      centralized system like CVS or Subversion?
>
> Yes and no.  There is nothing to prevent a formal workflow like that
> in CVS/Subversion.  However, the separation of "commit" into "record
> && push to authoritative" leaves open the possibility of annoying
> glitches until you get used to it, and even then it's easy to forget
> to push or to forget that you've committed not-for-pushing stuff, etc,
> etc.  In practice it is probably simpler to use a dVCS-specialized
> workflow like "record && push to candidate".

Bazaar offers the same semantic if you want to use it.  In practice,
when you use "bzr checkout" to get a branch, it becomes bound to the
remote branch.  When you commit to a local bound branch, the change is
automatically pushed to the server, including the needed locking
semantics, so the workflow is pretty much the same as if it were a
Subversion checkout (hence the command name).  Branches can be freely
bound and unbound without effort, if desired.

In the team I work on we've been using this for a while for the
"authoritative" branches.  So it's a distributed workflow otherwise,
but at the end of the life cycle of a branch, we merge and commit to a
bound branch pretty much like committing a change to Subversion.

-- 
Gustavo Niemeyer
http://niemeyer.net

From tseaver at palladion.com  Thu Nov  6 13:51:19 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Thu, 06 Nov 2008 07:51:19 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <geuaii$qrd$1@ger.gmane.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	<geq1n0$vng$1@ger.gmane.org>	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	<4910BE7F.9040900@gmail.com>	<4910CA8D.9010303@v.loewis.de>	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>	<4911F0E3.9020607@v.loewis.de>	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>	<4911F7B2.6020403@v.loewis.de>	<4912400C.4060204@palladion.com>	<87prl9sgtq.fsf@uwakimon.sk.tsukuba.ac.jp>
	<geuaii$qrd$1@ger.gmane.org>
Message-ID: <4912E847.6070404@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Georg Brandl wrote:
> Stephen J. Turnbull schrieb:
>> Tres Seaver writes:
>>
>>  > svn doesn't have any true tags, AFAIK:  everything is a branch.
>>
>> Yow!  I couldn't have imagined that would be true.  And didn't....
> 
> It's not exactly true either. svn doesn't have either tags or branches,
> it has only a single (directory) tree per repository, of which individual
> subdirectories can be checked out individually. This is the root of the
> usual first step in creating a SVN repo: make "branches", "trunk" and
> "tags" subdirs.

The SVN docs use the term "branch" for this concept:

  http://svnbook.red-bean.com/en/1.0/ch04.html#svn-ch-4-sect-1

And they document that tags are really branches:

  http://svnbook.red-bean.com/en/1.0/ch04s06.html

If you have some other definition for "branch", that's fine, but you
will need to outline how that differs from SVN's use of the term, given
that SVN is the reference model for Python development at ths moment.

> In this context it is not surprising that there is no separate branch or
> tag concept. I think the reason is that CVS *had* branches and tags as
> separate concepts and it must have been relatively painful to use them.

At the conceptual level, a tag would be the equivalent of a "symlink",
rather than a copy:  if they existed in SVN, tags would be pointers to
another branch, frozen in at a given revision. The SVN team denies the
utility of tags, which doesn't change the fact that other folks like and
use them effectively in other systems.

Any issues related to the implementation of tags and branches in CVS are
irrelevant to the usefulness of tags:  I have found them very useful in
many other VCSes (ClearCase, Perforce, etc.), and find their deliberate
omission from SVN to be a misfeature.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJEuhH+gerLs4ltQ4RAuFWAKCRg7LnBeUGJVYXUwQ9eRsq36bz8QCgx7fv
04UtndVdgiGUQOp7UAUTE9I=
=SvBQ
-----END PGP SIGNATURE-----


From dirkjan at ochtman.nl  Thu Nov  6 14:04:01 2008
From: dirkjan at ochtman.nl (Dirkjan Ochtman)
Date: Thu, 6 Nov 2008 13:04:01 +0000 (UTC)
Subject: [Python-Dev] DVCS PEP update
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<bbaeab100811052050x719b6fcdx9f9e24937e14ea1@mail.gmail.com>
Message-ID: <loom.20081106T130302-870@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> So my first choice for Mercurial fell through. If you would like to
> represent Mercurial, let me know.

I can represent Mercurial, though it may be (more?) helpful if some of the
Mercurial-using python-dev regulars can occasionally step in.

Cheers,

Dirkjan


From leif.walsh at gmail.com  Thu Nov  6 14:22:30 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Thu, 6 Nov 2008 08:22:30 -0500
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <loom.20081106T130302-870@post.gmane.org>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<bbaeab100811052050x719b6fcdx9f9e24937e14ea1@mail.gmail.com>
	<loom.20081106T130302-870@post.gmane.org>
Message-ID: <cc7430500811060522r48e8e7e6t88d9a38f3342391b@mail.gmail.com>

On Thu, Nov 6, 2008 at 8:04 AM, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
> I can represent Mercurial, though it may be (more?) helpful if some of the
> Mercurial-using python-dev regulars can occasionally step in.

I like mercurial, so I'll do this if I see something, but I don't use
it for anything that heavy-duty, so I don't expect to be able to say
anything very convincing, or, I suppose, intelligent.

-- 
Cheers,
Leif

From p.f.moore at gmail.com  Thu Nov  6 15:03:34 2008
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 6 Nov 2008 14:03:34 +0000
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <loom.20081106T130302-870@post.gmane.org>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<bbaeab100811052050x719b6fcdx9f9e24937e14ea1@mail.gmail.com>
	<loom.20081106T130302-870@post.gmane.org>
Message-ID: <79990c6b0811060603m271b6535v806933ec66d913f9@mail.gmail.com>

2008/11/6 Dirkjan Ochtman <dirkjan at ochtman.nl>:
> Brett Cannon <brett <at> python.org> writes:
>> So my first choice for Mercurial fell through. If you would like to
>> represent Mercurial, let me know.
>
> I can represent Mercurial, though it may be (more?) helpful if some of the
> Mercurial-using python-dev regulars can occasionally step in.

I'll help out where I can. But my Mercurial usage is pretty basic
(linear history on personal projects, mainly) so I'll avoid the
complex questions :-)

Paul

From lists at cheimes.de  Thu Nov  6 15:23:31 2008
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 06 Nov 2008 15:23:31 +0100
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
Message-ID: <geuul3$r3$1@ger.gmane.org>

Brett Cannon wrote:
> I have emailed some people who have shown allegiance to a specific
> DVCS to seeif they are willing to fill in the usage scenarios for me
> along with converting the dev FAQ so that we all have a clear way to
> learn how to use the things. If any of the people I emailed turn down
> being the leader fo ra specific DVCS I will email here looking for
> volunteers.

Hey Brett!

I'm missing a few points in your use case list:

* Does the DVCS handle merges over file reneames, directory renames
   and file moves to another directory?

* Are there any graphical interfaces to the DVCS? Tortoise svn is
   a killer application for Windows users.

* How hard is it to install the DVCS for a user few or no CLI
   experience? Are there installers for the most common systems
   (Mac, Windows, Debian and RPM based distros)? Do the installers
   and packages require uncommon extensions, 3rd party tools or
   admin privileges?

Christian


From barry at python.org  Thu Nov  6 16:30:20 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Nov 2008 10:30:20 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4C06C2E1-0226-4301-84BD-EEE03525BDED@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 5, 2008, at 8:36 PM, Stephen J. Turnbull wrote:

> I disagree.  This doesn't scale to Python size.  For distributed VC to
> work, somebody has to maintain a repo 24x7.  Python has to do this for
> the trunk; the additional burden for contributed patches is not great.
> There is no real advantage to having contributors do so, too.[3]
> Integrators and interested third parties also must keep track of
> contributor's repo URLs.  (Cf. Skip's question about discovering  
> repos.)
> Not happy stuff.

There's an interesting automation available when hosting branches on  
Launchpad and using a patch queue manager (PQM in particular).  I  
believe this is not quite deployed so I might be speaking out of  
class, and it's unlikely that Python would do anything but self-host  
its branches (though this doesn't necessarily prevent the following).   
Still I will describe this so you see the possibilities.

Launchpad has something call "merge proposals".  When a branch is  
ready, the original developer creates a merge proposal which is  
basically a request for review.  The dev can ask for a specific  
reviewer (e.g. barry for the email package), or he can ask anyone on  
the team for a review.  LP doesn't impose a specific workflow, so some  
projects require 2 or more reviews for any piece of code, others maybe  
1 review, etc.

Let's say I review Stephen's branch to fix a bug in the email  
package.  I request some changes, Stephen makes them, and now the  
branch looks great.  I can approve the branch and now neither Stephen  
nor I need to do anything in order to get that branch pushed to the  
mainline.

This works under the covers as follows: when approved, the branch  
details get added to a queue managed by Launchpad.  Python.org would  
run a PQM that watches this queue.  Python's PQM just picks branches  
off the top, one by one, and it tries to land them.  If any conflicts  
arise or tests fail, the branch doesn't merge.  Otherwise, it lands  
and all the appropriate metadata about the branch (and its associated  
bug) gets updated automatically.

This is a nice model for several reasons.  It's the bot's  
responsibility to do the grunt work of merging, landing, and pushing  
new branches to the mainline.  Humans only care about developing the  
code and reviewing the results.  Also, this aligns with something I've  
been very interested in from a release manager's POV: I'd like the  
mainline branches to always be in a releasable state.  We can now  
define that releasable state to be "whatever adheres to the bot's  
landing rules".

I'm skeptical Python would ever adopt this workflow, if only because  
there is technical work that would have to be done (i.e. we'd have to  
run the bot).  Still, I wanted to explain this so that you could get a  
sense of what's possible.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRMNjnEjvBPtnXfVAQJHXwP/T6WbEIvWC8OJh8i4UnmHvgj8hbT5tbED
2sQRBHNxrBavwhvENUTAM0n4a8j+jSwof8PszflRbiNLVANYKJ1wWHfevkkPcwL7
mzcgtZqpkijWdJoGOeif04LcjuiYopSSEtPLRC9QdDygwlZmhhvbPFVQ2xvtc6C/
4DO12JDPICA=
=Zn9N
-----END PGP SIGNATURE-----

From barry at python.org  Thu Nov  6 16:46:55 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Nov 2008 10:46:55 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <2F0728DC-8333-4168-B72F-6D758BA60781@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 5, 2008, at 8:36 PM, Stephen J. Turnbull wrote:

> You need not feel that way.  It's not you---the flexibility of dVCS
> means that until the Powers That Be promulgate a Workflow, this will
> be ambiguous.

You're absolutely right.  Adopting a dvcs opens up a much larger world  
of possible workflows and best practices, both for an individual and  
for a project.  It's a case of too much choice, so I feel strongly  
that Python should adopt and explain exactly one preferred workflow  
that will work for most people and use cases.  People can still  
experiment and find alternatives if they want.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRMRb3EjvBPtnXfVAQJU0AQAugEbLciCd2EIzqPrPOmWykoK7+u1fCAh
yLnc3aFeserk50WwYkfovZDZESNs8F8ADZ2CIVGK+5AWW+MC17YAx2P4M0gUEeLY
ulTQmrpLuRZHVpD6vqoM9hnI/vvsLwTqcYgqB1lfSLdSZ8RnJTnUPQ7UY7Eq7kfS
gmxDhUqYKVg=
=T9CJ
-----END PGP SIGNATURE-----

From barry at python.org  Thu Nov  6 17:09:26 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Nov 2008 11:09:26 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4912D684.9060605@gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>	<18705.58131.933909.115314@montanaro-dyndns-org.local>	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>	<bbaeab100811052118tae0d142r88edae7db1e55da1@mail.gmail.com>
	<87od0ts7kv.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4912D684.9060605@gmail.com>
Message-ID: <78064AE9-9095-4771-B797-7B498F0AEBA0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 6, 2008, at 6:35 AM, Nick Coghlan wrote:

> If it isn't already there, suspending work on something to work on
> something else would make a very nice scenario to cover, as it is
> something even the core devs sometimes have to deal with.

Indeed. as a release manager, I have to manage several branches in  
parallel.  Several are for patches I'm looking at for release  
blockers, one perhaps for the PEP 101 administrivia, perhaps another  
pristine one as a baseline, not to mention several I'd have for my own  
bugs and features.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRMWtnEjvBPtnXfVAQK1iwP+NShDlAWqvgTh9vO5XbCOSCsC4DXy2KGm
eMpG6dBAdbSy4skV0fR9FW1x5b/HrQU2M8y/9LTq4G67DQdkCTzKbJHgW8QaNtG1
BeAhUOaf2URzLlSezbBhrkFxWL45r3bx175eTsKvMhwpxzjSTxCLseT/eaROYlAW
P8btKM8Q9xI=
=Mrl4
-----END PGP SIGNATURE-----

From g.brandl at gmx.net  Thu Nov  6 19:10:46 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 06 Nov 2008 19:10:46 +0100
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4912E847.6070404@palladion.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<932f8baf0811022346j45f6af0dtba217f681f230213@mail.gmail.com>	<geq1n0$vng$1@ger.gmane.org>	<932f8baf0811040957o7002b48eu948f79d485d62c9e@mail.gmail.com>	<4910BE7F.9040900@gmail.com>	<4910CA8D.9010303@v.loewis.de>	<9e804ac0811050652k2c3722b9i3875cc638a5059b6@mail.gmail.com>	<4911F0E3.9020607@v.loewis.de>	<45D2E849-A598-4B48-9850-EED801DC51CC@python.org>	<4911F7B2.6020403@v.loewis.de>	<4912400C.4060204@palladion.com>	<87prl9sgtq.fsf@uwakimon.sk.tsukuba.ac.jp>	<geuaii$qrd$1@ger.gmane.org>
	<4912E847.6070404@palladion.com>
Message-ID: <gevbv6$k3u$1@ger.gmane.org>

Tres Seaver schrieb:
> Georg Brandl wrote:
>> Stephen J. Turnbull schrieb:
>>> Tres Seaver writes:
>>>
>>>  > svn doesn't have any true tags, AFAIK:  everything is a branch.
>>>
>>> Yow!  I couldn't have imagined that would be true.  And didn't....
> 
>> It's not exactly true either. svn doesn't have either tags or branches,
>> it has only a single (directory) tree per repository, of which individual
>> subdirectories can be checked out individually. This is the root of the
>> usual first step in creating a SVN repo: make "branches", "trunk" and
>> "tags" subdirs.
> 
> The SVN docs use the term "branch" for this concept:
> 
>   http://svnbook.red-bean.com/en/1.0/ch04.html#svn-ch-4-sect-1
> 
> And they document that tags are really branches:
> 
>   http://svnbook.red-bean.com/en/1.0/ch04s06.html

Quoting from that page:

"""
In Subversion, there's no difference between a tag and a branch. Both are just
ordinary directories that are created by copying.
"""

But you're right, it's a question of definitions.

Georg


From brett at python.org  Thu Nov  6 23:43:32 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 6 Nov 2008 14:43:32 -0800
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <geuul3$r3$1@ger.gmane.org>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<geuul3$r3$1@ger.gmane.org>
Message-ID: <bbaeab100811061443h4b249f75m2bd7638766977293@mail.gmail.com>

On Thu, Nov 6, 2008 at 06:23, Christian Heimes <lists at cheimes.de> wrote:
> Brett Cannon wrote:
>>
>> I have emailed some people who have shown allegiance to a specific
>> DVCS to seeif they are willing to fill in the usage scenarios for me
>> along with converting the dev FAQ so that we all have a clear way to
>> learn how to use the things. If any of the people I emailed turn down
>> being the leader fo ra specific DVCS I will email here looking for
>> volunteers.
>
> Hey Brett!
>
> I'm missing a few points in your use case list:
>
> * Does the DVCS handle merges over file reneames, directory renames
>  and file moves to another directory?
>

I consider that a perk feature.

> * Are there any graphical interfaces to the DVCS? Tortoise svn is
>  a killer application for Windows users.
>

Covered in the OS/Tools section.

> * How hard is it to install the DVCS for a user few or no CLI
>  experience? Are there installers for the most common systems
>  (Mac, Windows, Debian and RPM based distros)? Do the installers
>  and packages require uncommon extensions, 3rd party tools or
>  admin privileges?

Also in the OS/Tools section.

-Brett

From brett at python.org  Thu Nov  6 23:46:48 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 6 Nov 2008 14:46:48 -0800
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <loom.20081106T130302-870@post.gmane.org>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<bbaeab100811052050x719b6fcdx9f9e24937e14ea1@mail.gmail.com>
	<loom.20081106T130302-870@post.gmane.org>
Message-ID: <bbaeab100811061446y19633a74kda5bf6b677129cfa@mail.gmail.com>

On Thu, Nov 6, 2008 at 05:04, Dirkjan Ochtman <dirkjan at ochtman.nl> wrote:
> Brett Cannon <brett <at> python.org> writes:
>> So my first choice for Mercurial fell through. If you would like to
>> represent Mercurial, let me know.
>
> I can represent Mercurial, though it may be (more?) helpful if some of the
> Mercurial-using python-dev regulars can occasionally step in.
>

I just gave the position, although thanks for offering!

If people want to help, here are the people covering the DVCSs, so contact them:

Bazaar: Barry Warsaw
git: Stephen Turnbull
Mercurial: Alexandre Vasalotti

These people are going to fill in the scenarios for their respective
DVCS and also rewrite the dev FAQ for their DVCS so we have the basic
commands documented in a common format.

-Brett

From barry at python.org  Thu Nov  6 23:52:43 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Nov 2008 17:52:43 -0500
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <bbaeab100811061443h4b249f75m2bd7638766977293@mail.gmail.com>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<geuul3$r3$1@ger.gmane.org>
	<bbaeab100811061443h4b249f75m2bd7638766977293@mail.gmail.com>
Message-ID: <786BC151-0C3C-4438-BFEF-8BA2522EC7E0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 6, 2008, at 5:43 PM, Brett Cannon wrote:

> On Thu, Nov 6, 2008 at 06:23, Christian Heimes <lists at cheimes.de>  
> wrote:
>> Brett Cannon wrote:
>>>
>>> I have emailed some people who have shown allegiance to a specific
>>> DVCS to seeif they are willing to fill in the usage scenarios for me
>>> along with converting the dev FAQ so that we all have a clear way to
>>> learn how to use the things. If any of the people I emailed turn  
>>> down
>>> being the leader fo ra specific DVCS I will email here looking for
>>> volunteers.
>>
>> Hey Brett!
>>
>> I'm missing a few points in your use case list:
>>
>> * Does the DVCS handle merges over file reneames, directory renames
>> and file moves to another directory?
>>
>
> I consider that a perk feature.

OTOH, the lack of this is one of the things that piss me of most about  
subversion.  It's really nice when you're in charge of your tool  
rather than the other way 'round!

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRN1PHEjvBPtnXfVAQKULQP9GDRrT76LCZpVRFNCOP6K6CvVGIvtd9I7
ryPk1RFYTqIduEC3FLvhQh0xjxT2HGo/99m0EZZbM4ZxMbhwvM4Sw8ppkcCFwtig
aOcHviDr0SNK6DU3L36/zMPN/Wnap3ltF09nqWYcjcbBdR94oae1K7SF7nkPCSn/
oIb2a9ShBAE=
=WnB/
-----END PGP SIGNATURE-----

From brett at python.org  Fri Nov  7 00:00:57 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 6 Nov 2008 15:00:57 -0800
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <786BC151-0C3C-4438-BFEF-8BA2522EC7E0@python.org>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<geuul3$r3$1@ger.gmane.org>
	<bbaeab100811061443h4b249f75m2bd7638766977293@mail.gmail.com>
	<786BC151-0C3C-4438-BFEF-8BA2522EC7E0@python.org>
Message-ID: <bbaeab100811061500r66f16bd6ic2e41a02f4bb5cbd@mail.gmail.com>

On Thu, Nov 6, 2008 at 14:52, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 6, 2008, at 5:43 PM, Brett Cannon wrote:
>
>> On Thu, Nov 6, 2008 at 06:23, Christian Heimes <lists at cheimes.de> wrote:
>>>
>>> Brett Cannon wrote:
>>>>
>>>> I have emailed some people who have shown allegiance to a specific
>>>> DVCS to seeif they are willing to fill in the usage scenarios for me
>>>> along with converting the dev FAQ so that we all have a clear way to
>>>> learn how to use the things. If any of the people I emailed turn down
>>>> being the leader fo ra specific DVCS I will email here looking for
>>>> volunteers.
>>>
>>> Hey Brett!
>>>
>>> I'm missing a few points in your use case list:
>>>
>>> * Does the DVCS handle merges over file reneames, directory renames
>>> and file moves to another directory?
>>>
>>
>> I consider that a perk feature.
>
> OTOH, the lack of this is one of the things that piss me of most about
> subversion.  It's really nice when you're in charge of your tool rather than
> the other way 'round!

I'm not saying it wouldn't be very nice to have, just I am as of this
moment not going to toss a tool out the door because it doesn't do it
better than svn.

-Brett

From ncoghlan at gmail.com  Fri Nov  7 01:47:41 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 07 Nov 2008 10:47:41 +1000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <2F0728DC-8333-4168-B72F-6D758BA60781@python.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>	<18705.58131.933909.115314@montanaro-dyndns-org.local>	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
	<2F0728DC-8333-4168-B72F-6D758BA60781@python.org>
Message-ID: <4913902D.9040108@gmail.com>

Barry Warsaw wrote:
> On Nov 5, 2008, at 8:36 PM, Stephen J. Turnbull wrote:
> 
>> You need not feel that way.  It's not you---the flexibility of dVCS
>> means that until the Powers That Be promulgate a Workflow, this will
>> be ambiguous.
> 
> You're absolutely right.  Adopting a dvcs opens up a much larger world
> of possible workflows and best practices, both for an individual and for
> a project.  It's a case of too much choice, so I feel strongly that
> Python should adopt and explain exactly one preferred workflow that will
> work for most people and use cases.  People can still experiment and
> find alternatives if they want.

This is an area where I think the initial DVCS PEP shouldn't be too
ambitious, and focus mainly on the improvements to things we currently
do with Subversion and Roundup:
- individuals suspending work on one task (e.g. a new feature) to switch
to something else (e.g. fixing a bug) (current workflow is multiple
checkouts or dumping your first task in a patch file, reverting, working
on the second task, dumping or committing it, then patching back to the
first task)
- backporting and forward porting patches between 3.x/3.x-1/2.y/2.y-1
(current workflow based on svnmerge)
- developing and maintaining patches
- reviewing patches
- collaboration amongst multiple developers on complex patches (current
workflow is either forking the standard library and creating a project
in the sandbox or somewhere else like Google Code, or else posting and
downloading a series of patches on the tracker, or, if all developers
involved have SVN access, creating a SVN branch and maintaining it with
SVN merge)

There's a lot of room for improvement just in those areas, even before
we start getting into the potential alternate workflows that a DVCS may
enable.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From brett at python.org  Fri Nov  7 02:04:19 2008
From: brett at python.org (Brett Cannon)
Date: Thu, 6 Nov 2008 17:04:19 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4913902D.9040108@gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
	<2F0728DC-8333-4168-B72F-6D758BA60781@python.org>
	<4913902D.9040108@gmail.com>
Message-ID: <bbaeab100811061704o26348857k8302a7501ce958ea@mail.gmail.com>

On Thu, Nov 6, 2008 at 16:47, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Barry Warsaw wrote:
>> On Nov 5, 2008, at 8:36 PM, Stephen J. Turnbull wrote:
>>
>>> You need not feel that way.  It's not you---the flexibility of dVCS
>>> means that until the Powers That Be promulgate a Workflow, this will
>>> be ambiguous.
>>
>> You're absolutely right.  Adopting a dvcs opens up a much larger world
>> of possible workflows and best practices, both for an individual and for
>> a project.  It's a case of too much choice, so I feel strongly that
>> Python should adopt and explain exactly one preferred workflow that will
>> work for most people and use cases.  People can still experiment and
>> find alternatives if they want.
>
> This is an area where I think the initial DVCS PEP shouldn't be too
> ambitious, and focus mainly on the improvements to things we currently
> do with Subversion and Roundup:

Exactly. This is why I asked for only known scenarios we all run into
and every one of them has an example svn way of doing it.

> - individuals suspending work on one task (e.g. a new feature) to switch
> to something else (e.g. fixing a bug) (current workflow is multiple
> checkouts or dumping your first task in a patch file, reverting, working
> on the second task, dumping or committing it, then patching back to the
> first task)

Sort of covered.

> - backporting and forward porting patches between 3.x/3.x-1/2.y/2.y-1
> (current workflow based on svnmerge)

Covered.

> - developing and maintaining patches

Covered.

> - reviewing patches

Covered.

> - collaboration amongst multiple developers on complex patches (current
> workflow is either forking the standard library and creating a project
> in the sandbox or somewhere else like Google Code, or else posting and
> downloading a series of patches on the tracker, or, if all developers
> involved have SVN access, creating a SVN branch and maintaining it with
> SVN merge)
>

Covered.

-Brett

From stephen at xemacs.org  Fri Nov  7 03:15:06 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 07 Nov 2008 11:15:06 +0900
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <4912D684.9060605@gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>
	<18705.58131.933909.115314@montanaro-dyndns-org.local>
	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>
	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>
	<bbaeab100811052118tae0d142r88edae7db1e55da1@mail.gmail.com>
	<87od0ts7kv.fsf@uwakimon.sk.tsukuba.ac.jp>
	<4912D684.9060605@gmail.com>
Message-ID: <87ej1os2mt.fsf@uwakimon.sk.tsukuba.ac.jp>

Nick Coghlan writes:

 > If it isn't already there, suspending work on something to work on
 > something else would make a very nice scenario to cover, as it is
 > something even the core devs sometimes have to deal with.
 > 
 > I'd expect any DVCS to be able to handily beat what I currently do
 > with SVN:

Sure, but that's just you.<wink>  AIUI the "quilt" tool that kernel
people use simply automates exactly that process for you in svn or any
other VCS.  (I don't use quilts myself.)  The various DVCSes also have
specialized tools for managing patches.  Darcs is patch-oriented from
the get-go, ditto Arch.  Bazaar has "looms", Mercurial has "queues".
The advantage of looms and queues is that the patches are managed with
the VCS; I don't recall whether there's a way to do that with quilts,
but I presume so.

However, I suspect if you're expecting to do this more efficiently in
a DVCS, you'll be disappointed.  I'll make this scenario concrete in
the PEP (for git), but I'd say that the "operation count" would be
perhaps higher for me, using git as I like to, than in your scenario.
Of course they would be VCS operations, not using patch and diff.
What doing it in a DVCS provides is (1) a VCS record of what you did,
(2) which can easily be published, and (3) the option of using the VCS
to merge the two tasks later.

In other words, *D*VCS is about keeping and (optionally) communicating
*local* history, not about reducing the number of VC-related operations
you do.

From stephen at xemacs.org  Fri Nov  7 03:23:37 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 07 Nov 2008 11:23:37 +0900
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <bbaeab100811061443h4b249f75m2bd7638766977293@mail.gmail.com>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<geuul3$r3$1@ger.gmane.org>
	<bbaeab100811061443h4b249f75m2bd7638766977293@mail.gmail.com>
Message-ID: <87d4h8s28m.fsf@uwakimon.sk.tsukuba.ac.jp>

Brett Cannon writes:

 > I consider that a perk feature.

Let's not judge which features are perks or essential quite yet.  (If
the number of features/scenarios/whatnot becomes a burden on you, then
you must triage them, of course.)

But a lot of people will not know what the features are unless they go
in the PEP.  Eg, I've never actually encountered the merge-across-rename
issue in the wild, for example.  A scenario where I could actually try
it for myself would be educational, as many people I respect swear by it.

From ncoghlan at gmail.com  Fri Nov  7 03:27:23 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 07 Nov 2008 12:27:23 +1000
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <87ej1os2mt.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>	<79990c6b0811050309p59065508wd12bbecfe889568@mail.gmail.com>	<18705.58131.933909.115314@montanaro-dyndns-org.local>	<4C97F206-2D1B-433D-B537-B3171AD3C983@gmail.com>	<87wsfhskir.fsf@uwakimon.sk.tsukuba.ac.jp>	<bbaeab100811052118tae0d142r88edae7db1e55da1@mail.gmail.com>	<87od0ts7kv.fsf@uwakimon.sk.tsukuba.ac.jp>	<4912D684.9060605@gmail.com>
	<87ej1os2mt.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <4913A78B.5060309@gmail.com>

Stephen J. Turnbull wrote:
> In other words, *D*VCS is about keeping and (optionally) communicating
> *local* history, not about reducing the number of VC-related operations
> you do.

The bit I left out where it can get painful is when the patch call to go
back to the first task fails due to changes picked up in the "svn
update" before working on the second task.

Even if the basic flow isn't any simpler (suspend original task, start
new task, finish new task, go back to original task), a bit of
assistance with that last step (when necessary) wouldn't go astray at all :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steve at holdenweb.com  Fri Nov  7 03:31:50 2008
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 06 Nov 2008 21:31:50 -0500
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
Message-ID: <gf09ao$pne$1@ger.gmane.org>

Brett Cannon wrote:
> I have started the DVCS PEP which can be seen at
> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
> beyond the rationale, usage scenarios I plan to use, and what other
> sections I plan to write.
> 
Brett:

Would you care to get a PEP number allocated to this draft document? I
think it's likely that it will eventually be published, and I'd like a
number to use in reference to it if possible, please.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From barry at python.org  Fri Nov  7 04:53:35 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 6 Nov 2008 22:53:35 -0500
Subject: [Python-Dev] RELEASED Python 3.0rc2
Message-ID: <60B04061-65DA-4F44-8396-02F4FF0B4B47@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team and the Python community, I  
am happy to announce the second release candidate for Python 3.0.

This is a release candidate, so while it is not suitable for  
production environments, we strongly encourage you to download and  
test this release on your software.  We expect only critical bugs to  
be fixed between now and the final release, currently planned for 03- 
Dec-2008.

If you find things broken or incorrect, please submit bug reports at

     http://bugs.python.org

For more information and downloadable distributions, see the Python
3.0 website:

     http://www.python.org/download/releases/3.0/

See PEP 361 for release schedule details:

     http://www.python.org/dev/peps/pep-0361/

Enjoy,
- -Barry

Barry Warsaw
barry at python.org
Python 2.6/3.0 Release Manager
(on behalf of the entire python-dev team)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRO7wHEjvBPtnXfVAQIYrQP+Lynpa/p7VMY/YxJyjxiBI0bvOATPIKbE
jqu9ZFwXlO19+G4bFiAoYviY5UFYPm3TpbMoso2qNoJsJFLSt4d+AycWWcaXKd08
vpifsxVoWvdLCLZtT7ioMBJh/juu+Pchf2o2l+PHm5aWlLvq/24uu8YKbpSKKbr9
K4gB4ecYd3A=
=3UPl
-----END PGP SIGNATURE-----

From victor.stinner at haypocalc.com  Fri Nov  7 10:53:38 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 7 Nov 2008 10:53:38 +0100
Subject: [Python-Dev] RELEASED Python 3.0rc2
In-Reply-To: <60B04061-65DA-4F44-8396-02F4FF0B4B47@python.org>
References: <60B04061-65DA-4F44-8396-02F4FF0B4B47@python.org>
Message-ID: <200811071053.38908.victor.stinner@haypocalc.com>

Hi,

Great job Barry and all contributors who fixed the "last" bugs ;-)

Le Friday 07 November 2008 04:53:35 Barry Warsaw, vous avez ?crit?:
> We expect only critical bugs to be fixed between now and the 
> final release, currently planned for 03-Dec-2008.

The document "What's new in Python 3.0" in should be updated:
   http://docs.python.org/dev/3.0/whatsnew/3.0.html

"PEP 352: Exceptions must derive from BaseException. This is the root of the 
exception hierarchy."
   I prefer to derive from Exception to be able to use 
   "exept Exception as: ..." which doesn't catch SystemExit 
   nor KeyboardInterrupt.

"PEP 3134: Exception chaining. (The __context__ feature from the PEP hasn?t 
been implemented yet in 3.0a2.)"
   The feature is now implemented!

"PEP 237: long renamed to int. (...) sys.maxint was also removed since the int 
type has no maximum value anymore."
   What about the new sys.maxsize constant? Oh, it's written at the bottom, 
   "Removed sys.maxint. Use sys.maxsize." Both paragraphs should be merged.

"Optimizations (...) 33% slower (...) we expect to be optimizing string and 
integer operations significantly before the final 3.0 release!"
   I don't expect "significant" changes before the final release. I worked on
   some patches about the int type (use base 2^30 instead of 2^15, GMP, etc.),
   but all patches optimize large integer (more than 1 digit, or more than 20
   digits) whereas most integers in Python are very small.

   About str=>unicode, I also don't expect changes. On character in now 
   4 bytes (or 2 bytes), whereas Python2 used 1 byte. This introduce an
   overhead. Most functions at C level use an conversion from byte
   string (ASCII) to Unicode (eg. PyErr_SetString). We should directly use
   wide string (eg. PyErr_SetWideString?).

"Porting to Python 3.0"
   This section is very small and gives few informations. There is nothing
   about 2to3 (just two references in the document). I also read somewhere
   that someone wrote a document explaining how to port a C extension to
   Python3.

What about a link to the "What's new in Python 2.6" document? Most people are 
still using Python 2.4 or 2.5. And Python3 is Python 2.5 + <changes in Python 
2.6> + <changes in Python 3.0)>.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From barry at python.org  Fri Nov  7 14:31:59 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 7 Nov 2008 08:31:59 -0500
Subject: [Python-Dev] [Python-3000]  RELEASED Python 3.0rc2
In-Reply-To: <200811071053.38908.victor.stinner@haypocalc.com>
References: <60B04061-65DA-4F44-8396-02F4FF0B4B47@python.org>
	<200811071053.38908.victor.stinner@haypocalc.com>
Message-ID: <0A787651-013A-4635-BC8A-EC8C52B53654@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 7, 2008, at 4:53 AM, Victor Stinner wrote:

> Hi,
>
> Great job Barry and all contributors who fixed the "last" bugs ;-)

Thanks!

> The document "What's new in Python 3.0" in should be updated:
>   http://docs.python.org/dev/3.0/whatsnew/3.0.html

Issue 2306, assigned to Guido.
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRRDT3EjvBPtnXfVAQJUsAP/S9rFA0V4HDp80oxycy0coWDaW2HAnHA4
ombn/+HjWtS2zTIbCkqdFfFsZ05DRDQN7LNKZVkV1sRsmzJ9fASITQP6mpUiFy/f
Aq6I0Z73jHlZtINuDRI5ZaCQCrxHPM/lTdjEP3h0fGxtW0wEibr1/rep3gAJMomL
2V6y2mCOc8I=
=H0+/
-----END PGP SIGNATURE-----

From jim at zope.com  Fri Nov  7 15:24:56 2008
From: jim at zope.com (Jim Fulton)
Date: Fri, 7 Nov 2008 09:24:56 -0500
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
Message-ID: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>


Are there plans for a 2.4 release that includes:

http://bugs.python.org/issue2620, and

http://svn.python.org/view?rev=65335&view=rev

?

These (and other fixes) address:
http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2315

Jim

--
Jim Fulton
Zope Corporation



From status at bugs.python.org  Fri Nov  7 18:06:36 2008
From: status at bugs.python.org (Python tracker)
Date: Fri,  7 Nov 2008 18:06:36 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20081107170636.6ECA6780CA@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/31/08 - 11/07/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2161 open (+24) / 13941 closed (+10) / 16102 total (+34)

Open issues with patches:   709

Average duration of open issues: 714 days.
Median duration of open issues: 1971 days.

Open Issues Breakdown
   open  2145 (+24)
pending    16 ( +0)

Issues Created Or Reopened (35)
_______________________________

xml.etree.ElementTree does not read xml-text over page bonderies 11/02/08
       http://bugs.python.org/issue4100    reopened ocean-city                
       patch                                                                   

execution model - clear and complete example in documentation    11/01/08
       http://bugs.python.org/issue4246    created  robwolfe                  
                                                                               

Docs: Provide some examples of "pass" use in the tutorial.       11/02/08
CLOSED http://bugs.python.org/issue4247    created  jafo                      
       patch, patch, easy, needs review                                        

"class in None" in html help keyword                             11/02/08
CLOSED http://bugs.python.org/issue4248    created  ocean-city                
       patch                                                                   

HTTPResponse instance has no attribute 'code'                    11/02/08
       http://bugs.python.org/issue4249    created  jbperez                   
                                                                               

urllib2.py: HTTPResponse instance has no attribute 'code'        11/02/08
       http://bugs.python.org/issue4250    created  jbperez                   
                                                                               

library.pdf documentation: "See also: Module <xxx>" does not lin 11/02/08
       http://bugs.python.org/issue4251    created  wplappert                 
                                                                               

inactive item not shown in html help index pane                  11/03/08
       http://bugs.python.org/issue4252    created  ocean-city                
       patch                                                                   

Maildir dumpmessage on                                           11/03/08
       http://bugs.python.org/issue4253    created  marcus at internetnowasp.net 
                                                                               

_cursesmodule.c callable update_lines_cols()                     11/03/08
       http://bugs.python.org/issue4254    created  nemesis                   
       patch                                                                   

timing module refers to non-existent documentation               11/03/08
CLOSED http://bugs.python.org/issue4255    created  stevenjd                  
                                                                               

optparse: provide a simple way to get a programmatically useful  11/03/08
       http://bugs.python.org/issue4256    created  andybuckley               
                                                                               

Documentation for socket.gethostname() needs tweaking            11/03/08
CLOSED http://bugs.python.org/issue4257    created  roysmith                  
                                                                               

Use 30-bit digits instead of 15-bit digits for Python integers.  11/04/08
       http://bugs.python.org/issue4258    created  marketdickinson           
       patch                                                                   

Update pydoc URLs                                                11/04/08
       http://bugs.python.org/issue4259    created  akuchling                 
                                                                               

ctypes.xFUNCTYPE are decorators.                                 11/05/08
       http://bugs.python.org/issue4260    created  LambertDW                 
                                                                               

The pwd module doesn't distinguish between errors and no user    11/05/08
       http://bugs.python.org/issue4261    created  astrand                   
                                                                               

import and compile() do not treat trailing whitespace the same   11/05/08
       http://bugs.python.org/issue4262    created  mseaborn                  
                                                                               

BufferedWriter non-blocking overage                              11/05/08
       http://bugs.python.org/issue4263    created  severb                    
       patch                                                                   

Patch: optimize code to use LIST_APPEND instead of calling list. 11/05/08
       http://bugs.python.org/issue4264    created  novalis_dt                
       patch                                                                   

shutil.copyfile() leaks file descriptors when disk fills         11/05/08
       http://bugs.python.org/issue4265    created  ammon_riley               
                                                                               

Python 3.0 docs are broken.                                      11/07/08
CLOSED http://bugs.python.org/issue4266    reopened barry                     
                                                                               

sqlite3 documentation                                            11/06/08
CLOSED http://bugs.python.org/issue4267    created  gagenellina               
       patch                                                                   

functions in email package listed under wrong module             11/06/08
CLOSED http://bugs.python.org/issue4268    created  gagenellina               
                                                                               

Spaces not showing correctly in "split()" documentation on http: 11/06/08
CLOSED http://bugs.python.org/issue4269    created  SkyLined                  
                                                                               

struct module: pack/unpack and byte order on x86_64              11/06/08
       http://bugs.python.org/issue4270    created  mmm77                     
                                                                               

conversion tool does not fix "from Tkinter import N,E"           11/06/08
       http://bugs.python.org/issue4271    created  wplappert                 
                                                                               

set timestamp in gzip stream                                     11/06/08
       http://bugs.python.org/issue4272    created  jfrechet                  
       patch                                                                   

cycle created by profile.run                                     11/06/08
       http://bugs.python.org/issue4273    created  darrenr                   
                                                                               

Finish what's new in 3.0                                         11/07/08
CLOSED http://bugs.python.org/issue4274    created  christian.heimes          
                                                                               

socketserver example code not correctly ported to py3k           11/07/08
       http://bugs.python.org/issue4275    created  macd                      
                                                                               

IDLE in 2.6 fails to launch                                      11/07/08
CLOSED http://bugs.python.org/issue4276    created  JanKarel                  
                                                                               

asynchat's handle_error inconsistency                            11/07/08
       http://bugs.python.org/issue4277    created  giampaolo.rodola          
       patch                                                                   

optparse quirks                                                  11/07/08
       http://bugs.python.org/issue4278    created  kbriggs                   
                                                                               

Module 'parser' fails to build                                   11/07/08
       http://bugs.python.org/issue4279    created  kirkshorts                
       patch                                                                   



Issues Now Closed (28)
______________________

imaplib does not run under Python 3                               406 days
       http://bugs.python.org/issue1210    christian.heimes          
       patch                                                                   

manpages for sphinx-build & sphinx-quickstart                     208 days
       http://bugs.python.org/issue2608    georg.brandl              
                                                                               

sphinx - table of contents doesn't render correctly (html)        132 days
       http://bugs.python.org/issue3203    georg.brandl              
                                                                               

nntplib module broken by str to unicode conversion                 69 days
       http://bugs.python.org/issue3714    christian.heimes          
       patch                                                                   

poplib module broken by str to unicode conversion                  68 days
       http://bugs.python.org/issue3727    christian.heimes          
       patch                                                                   

tkinter Menu.delete bug                                            53 days
       http://bugs.python.org/issue3774    ocean-city                
       patch, easy                                                             

Update RELNOTES file                                               63 days
       http://bugs.python.org/issue3775    barry                     
                                                                               

parsermodule won't validate relative imports                       29 days
       http://bugs.python.org/issue4048    benjamin.peterson         
       patch                                                                   

extra "\fi" in sphinx.sty, line 62                                 14 days
       http://bugs.python.org/issue4166    georg.brandl              
                                                                               

Inline Markup :const: shows up in Documentation                    14 days
       http://bugs.python.org/issue4167    georg.brandl              
                                                                               

segfault with pickle if 4th or 5th item of tuple returned by __r    9 days
       http://bugs.python.org/issue4176    alexandre.vassalotti      
       patch                                                                   

Tilde compression isn't applied in TOC                             12 days
       http://bugs.python.org/issue4189    georg.brandl              
                                                                               

adapt sphinx-quickstart for windows                                10 days
       http://bugs.python.org/issue4203    georg.brandl              
                                                                               

Cannot build _multiprocessing, math, mmap and readline of Python   10 days
       http://bugs.python.org/issue4204    loewis                    
       patch                                                                   

frozen packages set an improper __path__ value                     10 days
       http://bugs.python.org/issue4211    benjamin.peterson         
       patch, needs review                                                     

unicode_literals doesn't work in exec                               3 days
       http://bugs.python.org/issue4225    benjamin.peterson         
       patch, needs review                                                     

has_key doc could be clearer                                        2 days
       http://bugs.python.org/issue4243    loewis                    
                                                                               

threading documentation: reorder sections                           6 days
       http://bugs.python.org/issue4245    georg.brandl              
                                                                               

Docs: Provide some examples of "pass" use in the tutorial.          5 days
       http://bugs.python.org/issue4247    georg.brandl              
       patch, patch, easy, needs review                                        

"class in None" in html help keyword                                1 days
       http://bugs.python.org/issue4248    georg.brandl              
       patch                                                                   

timing module refers to non-existent documentation                  0 days
       http://bugs.python.org/issue4255    benjamin.peterson         
                                                                               

Documentation for socket.gethostname() needs tweaking               0 days
       http://bugs.python.org/issue4257    benjamin.peterson         
                                                                               

Python 3.0 docs are broken.                                         0 days
       http://bugs.python.org/issue4266    georg.brandl              
                                                                               

sqlite3 documentation                                               0 days
       http://bugs.python.org/issue4267    georg.brandl              
       patch                                                                   

functions in email package listed under wrong module                0 days
       http://bugs.python.org/issue4268    georg.brandl              
                                                                               

Spaces not showing correctly in "split()" documentation on http:    0 days
       http://bugs.python.org/issue4269    georg.brandl              
                                                                               

Finish what's new in 3.0                                            0 days
       http://bugs.python.org/issue4274    benjamin.peterson         
                                                                               

IDLE in 2.6 fails to launch                                         0 days
       http://bugs.python.org/issue4276    benjamin.peterson         
                                                                               



Top Issues Most Discussed (10)
______________________________

 17 Use 30-bit digits instead of 15-bit digits for Python integers.    3 days
open    http://bugs.python.org/issue4258   

  8 Python 3.0 docs are broken.                                        0 days
closed  http://bugs.python.org/issue4266   

  8 has_key doc could be clearer                                       2 days
closed  http://bugs.python.org/issue4243   

  7 open(0, closefd=False) prints 3 warnings                           9 days
open    http://bugs.python.org/issue4233   

  6 Patch: optimize code to use LIST_APPEND instead of calling list    2 days
open    http://bugs.python.org/issue4264   

  5 Crash when importing builtin module during interpreter shutdown    9 days
open    http://bugs.python.org/issue4236   

  5 Victor Stinner's GMP patch for longs                             300 days
open    http://bugs.python.org/issue1814   

  4 BufferedWriter non-blocking overage                                2 days
open    http://bugs.python.org/issue4263   

  4 _cursesmodule.c callable update_lines_cols()                       4 days
open    http://bugs.python.org/issue4254   

  4 Docs: Provide some examples of "pass" use in the tutorial.         5 days
closed  http://bugs.python.org/issue4247   




From musiccomposition at gmail.com  Fri Nov  7 18:39:48 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 7 Nov 2008 11:39:48 -0600
Subject: [Python-Dev] [Python-3000]  RELEASED Python 3.0rc2
In-Reply-To: <200811071053.38908.victor.stinner@haypocalc.com>
References: <60B04061-65DA-4F44-8396-02F4FF0B4B47@python.org>
	<200811071053.38908.victor.stinner@haypocalc.com>
Message-ID: <1afaf6160811070939m3170bcd5j8613a26d71ed190a@mail.gmail.com>

On Fri, Nov 7, 2008 at 3:53 AM, Victor Stinner
<victor.stinner at haypocalc.com> wrote:
> Hi,
>
> Great job Barry and all contributors who fixed the "last" bugs ;-)

Which reminds me that this release's star developer award goes to
Victor for his hard work on fixing up the networking libraries for
Py3k!



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From steve at holdenweb.com  Fri Nov  7 18:42:11 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 07 Nov 2008 12:42:11 -0500
Subject: [Python-Dev] [Python-3000]  RELEASED Python 3.0rc2
In-Reply-To: <1afaf6160811070939m3170bcd5j8613a26d71ed190a@mail.gmail.com>
References: <60B04061-65DA-4F44-8396-02F4FF0B4B47@python.org>	<200811071053.38908.victor.stinner@haypocalc.com>
	<1afaf6160811070939m3170bcd5j8613a26d71ed190a@mail.gmail.com>
Message-ID: <49147DF3.8010709@holdenweb.com>

Benjamin Peterson wrote:
> On Fri, Nov 7, 2008 at 3:53 AM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Great job Barry and all contributors who fixed the "last" bugs ;-)
> 
> Which reminds me that this release's star developer award goes to
> Victor for his hard work on fixing up the networking libraries for
> Py3k!
> 
Yay, Victor!!!!

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Fri Nov  7 18:42:11 2008
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 07 Nov 2008 12:42:11 -0500
Subject: [Python-Dev] [Python-3000]  RELEASED Python 3.0rc2
In-Reply-To: <1afaf6160811070939m3170bcd5j8613a26d71ed190a@mail.gmail.com>
References: <60B04061-65DA-4F44-8396-02F4FF0B4B47@python.org>	<200811071053.38908.victor.stinner@haypocalc.com>
	<1afaf6160811070939m3170bcd5j8613a26d71ed190a@mail.gmail.com>
Message-ID: <49147DF3.8010709@holdenweb.com>

Benjamin Peterson wrote:
> On Fri, Nov 7, 2008 at 3:53 AM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Great job Barry and all contributors who fixed the "last" bugs ;-)
> 
> Which reminds me that this release's star developer award goes to
> Victor for his hard work on fixing up the networking libraries for
> Py3k!
> 
Yay, Victor!!!!

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From brett at python.org  Fri Nov  7 18:55:45 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 7 Nov 2008 09:55:45 -0800
Subject: [Python-Dev] DVCS PEP update
In-Reply-To: <87d4h8s28m.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <bbaeab100811041634u65aa7f92u298bf484e6f63c6@mail.gmail.com>
	<geuul3$r3$1@ger.gmane.org>
	<bbaeab100811061443h4b249f75m2bd7638766977293@mail.gmail.com>
	<87d4h8s28m.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <bbaeab100811070955ica74576ufc0bd19a0ef9c46c@mail.gmail.com>

On Thu, Nov 6, 2008 at 18:23, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Brett Cannon writes:
>
>  > I consider that a perk feature.
>
> Let's not judge which features are perks or essential quite yet.  (If
> the number of features/scenarios/whatnot becomes a burden on you, then
> you must triage them, of course.)
>
> But a lot of people will not know what the features are unless they go
> in the PEP.  Eg, I've never actually encountered the merge-across-rename
> issue in the wild, for example.  A scenario where I could actually try
> it for myself would be educational, as many people I respect swear by it.
>

I never said perk features would not be mentioned in the PEP. The
Special Features section is for the various DVCSs to puff up their
chests and show off.

-Brett

From brett at python.org  Fri Nov  7 18:58:24 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 7 Nov 2008 09:58:24 -0800
Subject: [Python-Dev] Looking for VCS usage scenarios
In-Reply-To: <gf09ao$pne$1@ger.gmane.org>
References: <bbaeab100811021605o16bb60fapfbf30e9549e1a7c2@mail.gmail.com>
	<gf09ao$pne$1@ger.gmane.org>
Message-ID: <bbaeab100811070958wae9f90evfab15fcc7ee2c44d@mail.gmail.com>

On Thu, Nov 6, 2008 at 18:31, Steve Holden <steve at holdenweb.com> wrote:
> Brett Cannon wrote:
>> I have started the DVCS PEP which can be seen at
>> http://docs.google.com/Doc?id=dg7fctr4_40dvjkdg64 . Not much is there
>> beyond the rationale, usage scenarios I plan to use, and what other
>> sections I plan to write.
>>
> Brett:
>
> Would you care to get a PEP number allocated to this draft document? I
> think it's likely that it will eventually be published, and I'd like a
> number to use in reference to it if possible, please.
>

Sure, although I will initially just have the PEP in svn point to the
Google doc since it is easier to work with everyone helping me out
that way.

-Brett

From barry at python.org  Fri Nov  7 21:24:42 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 7 Nov 2008 15:24:42 -0500
Subject: [Python-Dev] [Python-3000]  RELEASED Python 3.0rc2
In-Reply-To: <1afaf6160811070939m3170bcd5j8613a26d71ed190a@mail.gmail.com>
References: <60B04061-65DA-4F44-8396-02F4FF0B4B47@python.org>
	<200811071053.38908.victor.stinner@haypocalc.com>
	<1afaf6160811070939m3170bcd5j8613a26d71ed190a@mail.gmail.com>
Message-ID: <2655FBFB-B7BD-4567-8FF2-573626E7BD55@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 7, 2008, at 12:39 PM, Benjamin Peterson wrote:

> On Fri, Nov 7, 2008 at 3:53 AM, Victor Stinner
> <victor.stinner at haypocalc.com> wrote:
>> Hi,
>>
>> Great job Barry and all contributors who fixed the "last" bugs ;-)
>
> Which reminds me that this release's star developer award goes to
> Victor for his hard work on fixing up the networking libraries for
> Py3k!

Indeed, great work Victor!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRSkCnEjvBPtnXfVAQK7MAP/ayV9U241Xi1+s02LNphTwSNhJFDW9mbP
UhN1PVINDkS6kqs8GYQ6iO5KsTi20eQxjuTsITLZzFsNuGUKXRcIzkzibWZBMR7m
3WjHC5heGwaxYmaPYmcHUFipdW8T0vYGwiNmk/TWinHx11KSCEhHHeP/Mcr/xd+9
j+VCcl+45pI=
=S+vx
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Sat Nov  8 00:07:33 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 08 Nov 2008 00:07:33 +0100
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>
Message-ID: <4914CA35.8020409@v.loewis.de>

> Are there plans for a 2.4 release that includes:

Yes, my plan is to release 2.5.3 and 2.4.6 simultaneously.
2.4.6 will essentially contain everything that is on the
branch today, unless somebody comes up with an urgent patch
that also needs consideration. FWIW, there won't be another
2.3.x release.

Regards,
Martin

From fperez.net at gmail.com  Sat Nov  8 05:01:21 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 07 Nov 2008 20:01:21 -0800
Subject: [Python-Dev] Feedback from numerical/math community on PEP 225
Message-ID: <gf32uj$7nu$1@ger.gmane.org>

Hi all,

a while back there was a discussion about new operators for the language, which
ended in people mentioning that the status of PEP 225 was still undecided and
that it was the most likely route to consider in this discussion.  I offered
to collect some feedback from the numerical and math/scientific computing
communities and report back here.

Here are the results, with some background and links (I'll also paste the
original reST file at the end for reference and archival):

https://cirl.berkeley.edu/fperez/static/numpy-pep225/numpy-pep225.html

I'll mention this thread on the numpy and sage lists in case anyone from the
original discussions would like to pitch in.  

I hope this feedback is useful for you guys to make a decision.  I should note
that I'm just acting as a scribe here, relaying feedback (though I do like the
PEP and think it would be very useful for numpy).  I would most certainly NOT
be available for implementing the PEP in Python itself...

Cheers,

f


### Original reST source for feedback doc kept at
### https://cirl.berkeley.edu/fperez/static/numpy-pep225/numpy-pep225.html

=================================================================
 Discussion regarding possible new operators in Python (PEP 225)
=================================================================

.. Author: Fernando Perez
.. Contact: Fernando.Perez at berkeley.edu
.. Time-stamp: "2008-10-28 16:47:52 fperez"
.. Copyright: this document has been placed in the public domain.

.. contents::
..
    1  Introduction
    2  Background: matrix multiplication and Numpy
    3  Summary from the NumPy community
    4  Arguments neutral towards or against the PEP
    5  Other considerations and suggestions
      5.1  Operator form for logical_X functions
      5.2  Alternate comparisons
      5.3  Why just go one step?
      5.4  Alternate syntax
      5.5  Unicode operators
      5.6  D-language inspired syntax


Introduction
============

In the Python-dev mailing lists, there were recently two threads regarding the
possibility of adding to the language a new set of operators.  This would
provide easy syntactic support for behavior such as element-wise and
matrix-like multiplication for numpy arrays.  The original python-dev threads
are here:

* http://mail.python.org/pipermail/python-dev/2008-July/081508.html
* http://mail.python.org/pipermail/python-dev/2008-July/081551.html

And the actual PEP that discusses this at length is PEP 225:

* http://www.python.org/dev/peps/pep-0225/

In order to provide feedback from the scientific/numerical community, a
discussion was held on the numpy mailing list.  This document is a brief
summary of this discussion, in addition to some issues that were brought up
during a BOF session that was held during the SciPy'08 conference.  The point
of this document is to provide the Python development team (and ultimately
Guido) with feedback from a community that would be likely an important target
of the features discussed in PEP 225, hoping that a final decision can be made
on the PEP, either as-is or with modifications arising from this feedback.

This document contains a summary of an original discussion whose thread in the
numpy list can be found here:

* http://projects.scipy.org/pipermail/numpy-discussion/2008-August/036769.html
* http://projects.scipy.org/pipermail/numpy-discussion/2008-August/036858.html

and it has been further updated with a final round of vetting.  The final
thread which this document summarizes is here:

* http://projects.scipy.org/pipermail/numpy-discussion/2008-October/038234.html


Background: matrix multiplication and Numpy
===========================================

It is probably useful, for the sake of the Python dev team, to provide a bit of
background regarding the array/matrix situation in `Numpy
<http://numpy.scipy.org>`_, which is at the heart of much of this.  The numpy
library provides, in addition to the basic arrays that support element-wise
operations::

    In [1]: from numpy import array, matrix

    In [2]: a = array([[1,2],[3,4]])

    In [3]: b = array([[10,20],[30,40]])

    In [4]: a*b
    Out[4]:
    array([[ 10,  40],
           [ 90, 160]])

also an object called ``matrix`` that implements classic matrix multiplication
semantics for the ``*`` operator::
           
    In [5]: aM = matrix(a)

    In [6]: bM = matrix(b)

    In [7]: aM*bM
    Out[7]:
    matrix([[ 70, 100],
            [150, 220]])

The existence of two almost-but-not-quite identical objects at the core of
numpy has been the source of a large amount of discussion and effort to provide
smoother integration.  Yet, despite much work it is the opinion of many that
the situation will always remain unsatisfactory, as many pitfalls continue to
exist.  It is very easy to pass a matrix to some routine that does numerical
manipulations on its inputs and forgets to verify that a matrix was received,
and to end up with an array instead afterwards.  While much work has gone into
making the core numpy and scipy libraries 'matrix-proof', arbitrary third party
code may or may not be matrix aware, and thus this problem is likely to remain
an open issue.

Many more details on this can be read here:

* http://www.scipy.org/MatrixIndexing
* http://scipy.org/NewMatrixSpec

Multiplication is perhaps the main need for matrices, and the proposed PEP
could provide a viable solution for this problem.  Exponentiation is also
useful and covered by the PEP, and the ``~/`` division could perhaps be an
acceptable way of implementing matlab's `slash operator`_.

.. _slash operator:
http://www.mathworks.com/access/helpdesk/help/techdoc/index.html?/access/helpdesk/help/techdoc/ref/mrdivide.html


Summary from the NumPy community
================================

Overall the response to PEP 225 ranges from positive to very positive.  There
was particular interest in the benefits of having both `PEP 225`_ and `PEP
335`_, since together these could address the key language-based issues
encountered by numpy.

.. _PEP 225: http://www.python.org/dev/peps/pep-0225
.. _PEP 335: http://www.python.org/dev/peps/pep-0335

Participants expressed that it would solve real-world, practical problems for
them and enable them to use Python more effectively in teaching and research
involving numerical work, in particular (but not limited to) topics involving
linear algebra.

As Python's importance in scientific computing grows, both as an educational
and a research tool, it becomes more critical that it can be used as a
'drop-in' replacement to existing systems.  PEP 225 would lower the notational
barrier to using Python for tasks where languages like Matlab(TM) and IDL(TM)
are typically the default choice.  Expressing common algorithms, especially in
the domain of linear algebra, would be easier and more readable if numpy could
provide operator-level support for matrix operations without introducing new
top-level objects.

As an example of the relevance of this, we note a new NSF-funded project for
developing scientific computing curricular materials called Secant_.  This is
just one example amongst many of how Python is growing as the open tool of
choice for new scientific computing efforts.

.. _Secant: http://secant.cs.purdue.edu


Arguments neutral towards or against the PEP
============================================

While the response was overall positive, there were a few differing opinions,
listed here.

* Some expressed a neutral position: they felt there was really no overwhelming
  need for this and that people could get by with the tools offered today by
  Python.

* Some contributors feel that the real problem for numpy is *one* operator,
  multiplication.  Thus, some feel that only one should be added (reducing the
  reach of PEP 225), others that this simply isn't enough to justify the PEP's
  acceptance.
  
* Some felt that perhaps the time is not right for this, and that one should
  wait instead a few years for Unicode support to become more pervasive, and
  implement this idea via Unicode instead (see below).

* [ *Note by Robert Kern* ]. I really only care about having just *one*
  extra operator, one that I can (ab)use for matrix multiplication. It's the
  only operation that is common enough and with one obvious implementation (I'm
  looking at you, Matlab's "``\``") to warrant it, IMO. Doubling the number of
  operators and special methods is not a price that I'm willing to pay to get
  it, though.

* [ *Note by David Warde-Farley* ]. Even though Robert and others (myself
  included) really only care about an operator to facilitate matrix
  multiplication, I think most of us would support the ``~*=`` (mentioned in
  the PEP) as a natural companion to ``~*``.

  Am I right about this? I realize it can't buy you the same in-place semantics
  as ``+-=`` and friends currently enjoy with numpy arrays, but I think most
  folks would just *expect* it to work, i.e. if ``foo`` is 3x4 and ``bar`` is
  4x1 (or maybe a length 4 rank 1 array) then::

       foo ~*= bar

  results in foo now pointing to an array that (however you choose to handle
  rank) contains exactly 3 elements.
  
  
Other considerations and suggestions
====================================

Here we list some additional considerations that were presented.


Operator form for logical_X functions
-------------------------------------

Since the meaning of the logical, short-circuiting operators ``and`` and ``or``
in Python can't be redefined (they are keywords of the language), Numpy
implements its own versions as functions: ``logical_and`` and ``logical_or``
(as well as ``not`` and ``xor`` versions).  It would be good to extend the PEP
to cover type-defined implementations of logical operators.  Today, numpy
users often abuses the ``&`` and ``|`` operators for logical operations, but
the behavior of these isn't exactly that of Python logical operators::

    In [35]: a
    Out[35]: array([0, 1, 2, 3, 4])

    In [36]: b
    Out[36]: array([False,  True, False,  True, False], dtype=bool)

    In [37]: a & b
    Out[37]: array([0, 1, 0, 1, 0])

    In [38]: [x and y for x,y in zip(a,b)]
    Out[38]: [0, True, False, True, False]

    In [39]: a | b
    Out[39]: array([0, 1, 2, 3, 4])

    In [40]: [x or y for x,y in zip(a,b)]
    Out[40]: [False, 1, 2, 3, 4]

Note however, that this problem could perhaps be better dealt with if `PEP
335`_ were accepted.  That PEP is currently in draft mode, but it would address
the issues raised above for numpy, allowing `PEP 225`_ to keep its current
scope.

.. _PEP 225: http://www.python.org/dev/peps/pep-0225
.. _PEP 335: http://www.python.org/dev/peps/pep-0335


Alternate comparisons
---------------------

It could be useful to extend the original PEP not only to cover the basic
arithmetic operators, but also comparisons.  Having for a given object
alternate comparison mechanisms would allow one to declare, for example,
special orderings in fields for objects which can be ordered in different ways
depending on the structure in which they are considered to live.

[XXX - more justification for the field ordering example? This was suggested at
scipy, but I don't recall all the details]


Why just go one step?
---------------------

Instead, it would be nicer to have user-definable multiplication operators such
that for example::

  a ~1* b
  a ~2* b

etc... are all valid.  The user could thus define as many ~N* operators as
desired, for arbitrary integer values of N.  The proponents of this idea argue
that this provides a general solution to the problem of user-definable
operators for complex objects while maintaining the spirit of the PEP.  There's
the concern that if one new family of operators is allowed, we'll want more in
the future and it is thus worthwhile solving the general problem once and for
all.  The R language is cited as a precedent with a similar syntax, where
``%op%`` allows for arbitrary operators to be declared.



Alternate syntax
----------------

The only other syntax to receive much support was to use ``!`` instead of ``~``
as the special character.


Unicode operators
-----------------

A number of proposals were made to consider Unicode operators instead of ascii
2-character combinations.  While this was favored by one group, it was also
strongly opposed by another who feared difficulty of editing with standard
tools, font problems and readability problems.  Several people had problems
displaying the provided examples in their browsers.


D-language inspired syntax
--------------------------

One contributor suggested a syntax taken from the D language::

    # create array of doubles
    a = ...
    # add 42 to all elements
    a[] += 42.
    # apply a function
    b = sin(a[])
    # or this?
    b = sin[](a)

    c = a[] + b[]




From victor.stinner at haypocalc.com  Sat Nov  8 12:41:27 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 8 Nov 2008 12:41:27 +0100
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <4914CA35.8020409@v.loewis.de>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>
	<4914CA35.8020409@v.loewis.de>
Message-ID: <200811081241.27894.victor.stinner@haypocalc.com>

Le Saturday 08 November 2008 00:07:33 Martin v. L?wis, vous avez ?crit?:
> Yes, my plan is to release 2.5.3 and 2.4.6 simultaneously.
> 2.4.6 will essentially contain everything that is on the
> branch today, unless somebody comes up with an urgent patch
> that also needs consideration.
 
Skip Montanaro and others hackers started to port the multiprocessing module 
to Python 2.4 and 2.5.
   http://code.google.com/p/python-multiprocessing/

Is it planned to include it in 2.4.6 and/or 2.5.3? I don't know the status of 
the project, but it looks like it's still in alpha or beta phase (so far to 
be stable and robust). Or can we at least include the needed patches? 
Especially th threading + fork issue:
   http://bugs.python.org/issue874900

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From martin at v.loewis.de  Sat Nov  8 13:24:30 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 08 Nov 2008 13:24:30 +0100
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <200811081241.27894.victor.stinner@haypocalc.com>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>	<4914CA35.8020409@v.loewis.de>
	<200811081241.27894.victor.stinner@haypocalc.com>
Message-ID: <491584FE.3020405@v.loewis.de>

> Skip Montanaro and others hackers started to port the multiprocessing module 
> to Python 2.4 and 2.5.
>    http://code.google.com/p/python-multiprocessing/
> 
> Is it planned to include it in 2.4.6 and/or 2.5.3?

Neither. For 2.4.x, we only accept security fixes; for 2.5.3, only bug
fixes (and after that, only security fixes). New features are rejected.

> Especially th threading + fork issue:
>    http://bugs.python.org/issue874900

That one is already backported to 2.5, right?

Regards,
Martin


From barry at python.org  Sat Nov  8 13:44:16 2008
From: barry at python.org (Barry Warsaw)
Date: Sat, 8 Nov 2008 07:44:16 -0500
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <491584FE.3020405@v.loewis.de>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>	<4914CA35.8020409@v.loewis.de>
	<200811081241.27894.victor.stinner@haypocalc.com>
	<491584FE.3020405@v.loewis.de>
Message-ID: <655C07D4-6FDD-4AE3-AD60-2F4CD64EE8E4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 8, 2008, at 7:24 AM, Martin v. L?wis wrote:

>> Skip Montanaro and others hackers started to port the  
>> multiprocessing module
>> to Python 2.4 and 2.5.
>>   http://code.google.com/p/python-multiprocessing/
>>
>> Is it planned to include it in 2.4.6 and/or 2.5.3?
>
> Neither. For 2.4.x, we only accept security fixes; for 2.5.3, only bug
> fixes (and after that, only security fixes). New features are  
> rejected.

I completely agree.  If only there was an independent repository of  
add-on packages for Python available... <wink>.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRWJoHEjvBPtnXfVAQIa7wP9FF0+VbZcZWX33RPh8CTVsP/DjVhIvkLr
qLIBwyqByyVhx3kNSxHqzPmtSclOjjtfps04rcD7kPi3f9EaOb7vlvlPtiNjU95n
ti3nBszrJKLeDzBz3CagpCLVBZbXDGWAD8k3dtMGHuwR6TJZNbrHkQ1PvAaBL/wJ
Krw0ph/Qs/8=
=Fxa0
-----END PGP SIGNATURE-----

From jnoller at gmail.com  Sat Nov  8 14:38:49 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Sat, 8 Nov 2008 08:38:49 -0500
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <200811081241.27894.victor.stinner@haypocalc.com>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>
	<4914CA35.8020409@v.loewis.de>
	<200811081241.27894.victor.stinner@haypocalc.com>
Message-ID: <B408A684-6957-4BF7-B851-311CF221D1FD@gmail.com>

No - multiprocessing should not go to the official 2.4/2.5 branches.  
The PEP specified 2.6 and beyond.

The fix for 874900 is not a bad idea though.

On Nov 8, 2008, at 6:41 AM, Victor Stinner  
<victor.stinner at haypocalc.com> wrote:

> Le Saturday 08 November 2008 00:07:33 Martin v. L?wis, vous avez ?cr 
> it :
>> Yes, my plan is to release 2.5.3 and 2.4.6 simultaneously.
>> 2.4.6 will essentially contain everything that is on the
>> branch today, unless somebody comes up with an urgent patch
>> that also needs consideration.
>
> Skip Montanaro and others hackers started to port the  
> multiprocessing module
> to Python 2.4 and 2.5.
>   http://code.google.com/p/python-multiprocessing/
>
> Is it planned to include it in 2.4.6 and/or 2.5.3? I don't know the  
> status of
> the project, but it looks like it's still in alpha or beta phase (so  
> far to
> be stable and robust). Or can we at least include the needed patches?
> Especially th threading + fork issue:
>   http://bugs.python.org/issue874900
>
> -- 
> Victor Stinner aka haypo
> http://www.haypocalc.com/blog/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jnoller%40gmail.com

From jnoller at gmail.com  Sat Nov  8 14:41:13 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Sat, 8 Nov 2008 08:41:13 -0500
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <655C07D4-6FDD-4AE3-AD60-2F4CD64EE8E4@python.org>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>
	<4914CA35.8020409@v.loewis.de>
	<200811081241.27894.victor.stinner@haypocalc.com>
	<491584FE.3020405@v.loewis.de>
	<655C07D4-6FDD-4AE3-AD60-2F4CD64EE8E4@python.org>
Message-ID: <F35C58AD-C648-43BD-9958-61F1B7310DC3@gmail.com>



On Nov 8, 2008, at 7:44 AM, Barry Warsaw <barry at python.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On Nov 8, 2008, at 7:24 AM, Martin v. L?wis wrote:
>
>>> Skip Montanaro and others hackers started to port the  
>>> multiprocessing module
>>> to Python 2.4 and 2.5.
>>>  http://code.google.com/p/python-multiprocessing/
>>>
>>> Is it planned to include it in 2.4.6 and/or 2.5.3?
>>
>> Neither. For 2.4.x, we only accept security fixes; for 2.5.3, only  
>> bug
>> fixes (and after that, only security fixes). New features are  
>> rejected.
>
> I completely agree.  If only there was an independent repository of  
> add-on packages for Python available... <wink>.
>
> - -Barry
>

The backported version of MP is on pypi. Fancy that.

From martin at v.loewis.de  Sat Nov  8 17:50:43 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 08 Nov 2008 17:50:43 +0100
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <B408A684-6957-4BF7-B851-311CF221D1FD@gmail.com>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>	<4914CA35.8020409@v.loewis.de>	<200811081241.27894.victor.stinner@haypocalc.com>
	<B408A684-6957-4BF7-B851-311CF221D1FD@gmail.com>
Message-ID: <4915C363.8050204@v.loewis.de>

> The fix for 874900 is not a bad idea though.

Doing what specifically (that has not been done yet) is not a bad idea?

Regards,
Martin

From jnoller at gmail.com  Sat Nov  8 18:16:34 2008
From: jnoller at gmail.com (Jesse Noller)
Date: Sat, 8 Nov 2008 12:16:34 -0500
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <4915C363.8050204@v.loewis.de>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>
	<4914CA35.8020409@v.loewis.de>
	<200811081241.27894.victor.stinner@haypocalc.com>
	<B408A684-6957-4BF7-B851-311CF221D1FD@gmail.com>
	<4915C363.8050204@v.loewis.de>
Message-ID: <75AB5C68-184A-445E-B194-EE8EBAF9F08C@gmail.com>



On Nov 8, 2008, at 11:50 AM, "Martin v. L?wis" <martin at v.loewis.de>  
wrote:

>> The fix for 874900 is not a bad idea though.
>
> Doing what specifically (that has not been done yet) is not a bad  
> idea?
>
> Regards,
> Martin

Nothing, I think you're correct and the fix is in 2.5

From barry at python.org  Sat Nov  8 19:16:26 2008
From: barry at python.org (Barry Warsaw)
Date: Sat, 8 Nov 2008 13:16:26 -0500
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <F35C58AD-C648-43BD-9958-61F1B7310DC3@gmail.com>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>
	<4914CA35.8020409@v.loewis.de>
	<200811081241.27894.victor.stinner@haypocalc.com>
	<491584FE.3020405@v.loewis.de>
	<655C07D4-6FDD-4AE3-AD60-2F4CD64EE8E4@python.org>
	<F35C58AD-C648-43BD-9958-61F1B7310DC3@gmail.com>
Message-ID: <94C77BA1-F7EE-48D7-AAE7-5BDCC150FD51@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 8, 2008, at 8:41 AM, Jesse Noller wrote:

> The backported version of MP is on pypi. Fancy that.

SHHH!  Don't tell Guido you've made dups of his time machine keys.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRXXenEjvBPtnXfVAQIMIgP+MK8WNirYeRiJELSAx7CO++IbqEYjIp7Z
+HLvjrKW9X8+Pzl7YuXizeRuUI1fnb5DLYmbB2AfnJs6wPPnTmVv6CJNHFF7g9nU
nBCRboHPPihJK4r3tpIRyFqPhitN/Whx7s7brG5YO3tl+JPceMF9oF9X4jMUtr+i
hspEMOG5/RU=
=rgCy
-----END PGP SIGNATURE-----

From tjreedy at udel.edu  Sat Nov  8 20:57:07 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 08 Nov 2008 14:57:07 -0500
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <491584FE.3020405@v.loewis.de>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>	<4914CA35.8020409@v.loewis.de>	<200811081241.27894.victor.stinner@haypocalc.com>
	<491584FE.3020405@v.loewis.de>
Message-ID: <gf4qub$th1$1@ger.gmane.org>

Martin v. L?wis wrote:
>> Skip Montanaro and others hackers started to port the multiprocessing module 
>> to Python 2.4 and 2.5.
>>    http://code.google.com/p/python-multiprocessing/
>>
>> Is it planned to include it in 2.4.6 and/or 2.5.3?
> 
> Neither. For 2.4.x, we only accept security fixes; for 2.5.3, only bug
> fixes (and after that, only security fixes). New features are rejected.

If there are release notes, they could mention mp's availability on pypi.


From skip at pobox.com  Sun Nov  9 03:06:09 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 8 Nov 2008 20:06:09 -0600
Subject: [Python-Dev] Released fixes for CVE-2008-2315 for Python 2.4?
In-Reply-To: <gf4qub$th1$1@ger.gmane.org>
References: <A645EC11-2437-42A4-B716-7AEE37B6777B@zope.com>
	<4914CA35.8020409@v.loewis.de>
	<200811081241.27894.victor.stinner@haypocalc.com>
	<491584FE.3020405@v.loewis.de> <gf4qub$th1$1@ger.gmane.org>
Message-ID: <18710.17809.843542.381031@montanaro-dyndns-org.local>


    Terry> If there are release notes, they could mention mp's availability
    Terry> on pypi.

I think the best place to reference it would be in the 2.6/3.0 module
documentation.

Skip


From brett at python.org  Sun Nov  9 04:04:30 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 8 Nov 2008 19:04:30 -0800
Subject: [Python-Dev] Getting Set Up dev doc
Message-ID: <bbaeab100811081904l2d54f4d3nc7e13147d28d91cc@mail.gmail.com>

Thanks to the flu that I am getting over I finally got around to start
writing the dev docs I have been planning to author for years. The
first one is how to get set up, with the draft at
http://docs.google.com/Doc?id=dg7fctr4_5457424jfh . Obviously any
corrections are appreciated.

About the only thing I am unsure about with the doc is what the proper
separation between this doc and the dev FAQ should be. At this point
the FAQ has organically grown to the svn and ssh cheatsheet, which
makes me think that I should move all tool-specific info in this doc
to the dev FAQ and just reference the specific sections from this doc.
But I am somewhat hesitant about doing that as I don't want the FAQ to
get overloaded and just become The Dev Doc For Everything.

And so everyone knows where I am going with this, I plan to have docs
on the life of an issue (with a good chunk dedicated to writing
patches), various ways to get involved, and how to get stuff into
Python. That should cover all of the basics along with the dev FAQ.
Those of you who attended my tutorial on sprinting on the core at
PyCon 2008 should recognize how I am breaking the information down in
my head.

-Brett

From michele.simionato at gmail.com  Sun Nov  9 05:21:02 2008
From: michele.simionato at gmail.com (Michele Simionato)
Date: Sun, 9 Nov 2008 05:21:02 +0100
Subject: [Python-Dev] Feedback from numerical/math community on PEP 225
In-Reply-To: <4edc17eb0811072033y5b14e244k317170b4b3a59751@mail.gmail.com>
References: <gf32uj$7nu$1@ger.gmane.org>
	<4edc17eb0811072033y5b14e244k317170b4b3a59751@mail.gmail.com>
Message-ID: <4edc17eb0811082021s4fced347j4bb4ba71709989a@mail.gmail.com>

On Sat, Nov 8, 2008 at 5:01 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
> a while back there was a discussion about new operators for the language, which
> ended in people mentioning that the status of PEP 225 was still undecided and
> that it was the most likely route to consider in this discussion.  I offered
> to collect some feedback from the numerical and math/scientific computing
> communities and report back here.

While not an user of numpy, I feel legitimate to give my feedback as a
person with a scientific background.
I personally have always felt the lack of a binary operator expressing
non-commutative multiplication.
It could be used for matrices, but also for functions composition and
other more abstract things. I think a single new operator is all is
needed.

                Michele Simionato

From tseaver at palladion.com  Sun Nov  9 14:30:33 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 09 Nov 2008 08:30:33 -0500
Subject: [Python-Dev] Getting Set Up dev doc
In-Reply-To: <bbaeab100811081904l2d54f4d3nc7e13147d28d91cc@mail.gmail.com>
References: <bbaeab100811081904l2d54f4d3nc7e13147d28d91cc@mail.gmail.com>
Message-ID: <4916E5F9.3020408@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brett Cannon wrote:
> Thanks to the flu that I am getting over I finally got around to start
> writing the dev docs I have been planning to author for years. The
> first one is how to get set up, with the draft at
> http://docs.google.com/Doc?id=dg7fctr4_5457424jfh . Obviously any
> corrections are appreciated.
> 
> About the only thing I am unsure about with the doc is what the proper
> separation between this doc and the dev FAQ should be. At this point
> the FAQ has organically grown to the svn and ssh cheatsheet, which
> makes me think that I should move all tool-specific info in this doc
> to the dev FAQ and just reference the specific sections from this doc.
> But I am somewhat hesitant about doing that as I don't want the FAQ to
> get overloaded and just become The Dev Doc For Everything.

Given the choice, I would move the ssh / svn docs *out* of the FAQ into
your document, leaving pointers to the new sections behind.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJFuX5+gerLs4ltQ4RAk5hAJ9ESEGElL/T05wHF3stMoA8I0z0eACfaAWz
cBsv4yXSGTpNWVCRV8JGbY4=
=167p
-----END PGP SIGNATURE-----


From leif.walsh at gmail.com  Mon Nov 10 02:21:53 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Sun, 9 Nov 2008 20:21:53 -0500
Subject: [Python-Dev] Feedback from numerical/math community on PEP 225
In-Reply-To: <gf32uj$7nu$1@ger.gmane.org>
References: <gf32uj$7nu$1@ger.gmane.org>
Message-ID: <cc7430500811091721rb02b5ecr3562f70f676f9447@mail.gmail.com>

On Fri, Nov 7, 2008 at 11:01 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> a while back there was a discussion about new operators for the language, which
> ended in people mentioning that the status of PEP 225 was still undecided and
> that it was the most likely route to consider in this discussion.  I offered
> to collect some feedback from the numerical and math/scientific computing
> communities and report back here.

I've only used a bit of numpy, but I have to say, the ~*/etc. syntax
looks extremely ugly.  There doesn't really seem to be a solution to
this that isn't ugly in some way, but the op(a, b) version looks best
to me (I also think a.op(b) is out of the question).

-- 
Cheers,
Leif

From tim.peters at gmail.com  Mon Nov 10 05:11:23 2008
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 9 Nov 2008 23:11:23 -0500
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <5c6f2a5d0811041259k404a8554j6df126bba20872fa@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
	<1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
	<5c6f2a5d0811041259k404a8554j6df126bba20872fa@mail.gmail.com>
Message-ID: <1f7befae0811092011i1b7163d4xf4de2477d6593632@mail.gmail.com>

[Tim Peters]
>> ..
>> Whenever two digits are multiplied, the code intends to
>> cast (at least) one of them to "twodigits" first (if you
>> ever see a spot where this doesn't happen, it's a bug).

[Mark Dickinson]
> There are indeed one or two spots that seem to be missing a
> cast, for example the line "rem -= hi*n" in inplace_divrem1.

Definitely a bug!  Alas, it's not surprising -- there are no platforms
on which this bug has a visible consequence (because `digit` is
currently type `unsigned short`, C coerces `hi` and `n` to `int`
before multiplying, and on all platforms to date a C int is at least
32 bits, so that the multiply is at least 32x32->32 despite the lack
of a `twodigts` cast).

>> ...
>> 3. There's no way to know exactly what compilers will do with
>>    this short of staring at generated code.

> Yes---I've done the staring for gcc, so I now have precisely *one*
> data point, which is that various flavours of gcc on x86/x86_64
> *are* clever enough to turn
>
>  (uint64_t)my_uint32 * my_other_uint32
>
> into the appropriate HW instruction.

Nice!  Is this a documented feature?  "The problem" is that it
probably depends on a combination of gcc version and compilation
flags, and because /knowing/ whether it works requires staring at
generated code, there's probably no sane way to automate detection of
when, if, and under what conditions it breaks.  "Serious" packages use
assembler to avoid all this uncertainty.

> Unfortunately I don't have easy access to other compilers or
> platforms right now. :-(

Sorry, neither do I.  If you can dream up a way to automate testing
for generation of the desired HW instructions, you could post the test
program here and I bet a lot of people would run it.  Maybe even if
you just described what to look for "by eyeball" in the generated
code.


> Am still working on the benchmarking, but I'm definitely seeing
> speedup on gcc/x86---between 10% and 100% depending
> on the operations.

Sure -- it's not surprising that HW crunching more bits at a time is
significantly faster.

>> FYI, in a previous life working in speech recognition, under
>> Microsoft's Visual Studio 6 the only way we found to get at the
>> Pentium's 32x32->64 HW ability efficiently was via using inline
>> assembler.

> Urk.  We definitely don't want to go there.  Though I guess this
> is how packages like gmp and GP/Pari manage.

1. I have no idea what versions of Microsoft's compiler
   after MSVC 6 do here; perhaps it's no longer an issue
   (the Windows Python distro no longer uses MSVC 6).

2. If this is thought to be worth having, then on very
   widely used platforms I think a good case /can/ be
   made for incorporating some assembler.

3. GMP is "speed at any cost" -- they use assembler
   even when it's a stupid idea ;-)

> ..
> But maybe it's possible to write portable code (by providing fallbacks)
> that turns out to be efficient on the majority of mainstream systems?

If "it works" under the gcc and Windows compilers du jour on x86
systems, that probably covers over 90% of Python installations.  Good
enough -- stop before it gets pointlessly hard ;-)


> The extent of the ifdef'ery in the patch is really rather small:  one
> (compound) #ifdef in longintrepr.h for defining digit, twodigits, stwodigits
> etc, and a couple more for the places where digits are read and written
> in marshal.c.

But so far it only works with an unknown subset of gcc versions,
right?  These things don't get simpler, alas :-(


>>> I agree that very-long-integer optimizations probably don't really belong in
>>> Python,

>> Depends in part on whether Python can attract as many obsessed
>> maintainers and porters for such gonzo algorithms as GMP attracts ;-)

> Well, you can count me in. :)

Last I looked (which was at least 3 years ago), GMP's source code was
bigger than all of Python's combined.  For a start, I'll have the PSF
draw up a contract obligating you to lifetime servitude :-)

From greg.ewing at canterbury.ac.nz  Mon Nov 10 05:52:03 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 10 Nov 2008 17:52:03 +1300
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1f7befae0811092011i1b7163d4xf4de2477d6593632@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
	<1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
	<5c6f2a5d0811041259k404a8554j6df126bba20872fa@mail.gmail.com>
	<1f7befae0811092011i1b7163d4xf4de2477d6593632@mail.gmail.com>
Message-ID: <4917BDF3.3080802@canterbury.ac.nz>

Tim Peters wrote:
> because /knowing/ whether it works requires staring at
> generated code, there's probably no sane way to automate detection of
> when, if, and under what conditions it breaks.

Maybe it could be tested by compiling two small test programs,
one using (uint64_t)my_uint32 * my_other_uint32 and the other
using my_uint64 * my_other_uint64, and seeing whether one is
faster than the other?

-- 
Greg

From nick at craig-wood.com  Mon Nov 10 17:26:30 2008
From: nick at craig-wood.com (Nick Craig-Wood)
Date: Mon, 10 Nov 2008 16:26:30 +0000
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
	<1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
Message-ID: <20081110162630.2D51814C741@irishsea.home.craig-wood.com>

Tim Peters <tim.peters at gmail.com> wrote:
> > And for 32x32 -> 64, can't this simply be replaced by "(uint64_t)i * j",
> > where uint64_t is as in C99?  I'd hope that most compilers would
> > manage to turn this into the appropriate 32x32-bit hardware multiply.
> 
>  1. That's C99, not C89, and therefore less portable.
> 
>  2. On platforms that support it, this is at least 64x64->64 multiplication,
>     potentially much more expensive than the 32x32->64 (or 31x31->62?)
>     flavor you /intend/ to move to.
> 
>  3. There's no way to know exactly what compilers will do with this short of
>     staring at generated code.

I've relied in the past for the compiler generating a 32*32->64 bit
multiply for this code

#include <stdint.h>

uint64_t mul(uint32_t a, uint32_t b)
{
    return a*b;
}

Looking at the assembler it produces (x86)

mul:
        pushl   %ebp
        xorl    %edx, %edx
        movl    %esp, %ebp
        movl    12(%ebp), %eax
        imull   8(%ebp), %eax
        popl    %ebp
        ret

Which I'm pretty sure is a 32x32->64 bit mul (though my x86 assembler
foo is weak).

I think a compiler would have to be pretty stupid not to take this
optimisation... But then there are some pretty stupid compilers out
there!

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick

From dickinsm at gmail.com  Mon Nov 10 18:42:05 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Nov 2008 17:42:05 +0000
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <20081110162630.2D51814C741@irishsea.home.craig-wood.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
	<1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
	<20081110162630.2D51814C741@irishsea.home.craig-wood.com>
Message-ID: <5c6f2a5d0811100942q45fdf02awfbbfb8cbebcd692f@mail.gmail.com>

On Mon, Nov 10, 2008 at 4:26 PM, Nick Craig-Wood <nick at craig-wood.com> wrote:
>
> Looking at the assembler it produces (x86)
>
> mul:
>        pushl   %ebp
>        xorl    %edx, %edx
>        movl    %esp, %ebp
>        movl    12(%ebp), %eax
>        imull   8(%ebp), %eax
>        popl    %ebp
>        ret
>
> Which I'm pretty sure is a 32x32->64 bit mul (though my x86 assembler
> foo is weak).

My x86 assembler is also weak (or perhaps I should say nonexistent),
but I think this does exactly what the C standard says it should: that is,
it returns just the low 32-bits of the product.

Looking at the assembler, I think the imull does a 32-bit by
32-bit multiply and puts its (truncated) result back into the 32-bit
register eax.  I'd guess that the 64-bit result is being returned to
the calling routine in registers edx (high 32 bits) and eax (low 32 bits);
this explains why edx has to be zeroed with the 'xorl' instruction.

And if we were really expecting a 64-bit result then there should
be an unsigned multiply (mull) there instead of a signed multiply
(imull);  of course they're the same modulo 2**32, so for a 32-bit
result it doesn't matter which is used.

Mark

From dickinsm at gmail.com  Mon Nov 10 18:57:05 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Mon, 10 Nov 2008 17:57:05 +0000
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <20081110162630.2D51814C741@irishsea.home.craig-wood.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
	<1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
	<20081110162630.2D51814C741@irishsea.home.craig-wood.com>
Message-ID: <5c6f2a5d0811100957y6af637betf0c9e2e0e145542@mail.gmail.com>

Here's the test code I was using, modeled on the basic operation
that longobject needs:  multiply two digits together and extract
high and low digits from the result.


typedef uint32_t digit;
typedef uint64_t twodigits;
#define SHIFT 30
#define MASK (((digit)1 << SHIFT) - 1)

/* multiply a and b, and split into high digit (returned)
   and low digit (put into *low) */

extern digit
digit_mul(digit *low, digit a, digit b) {
    twodigits prod;
    prod = (twodigits)a * b;
    *low = (digit)(prod & MASK);
    return (digit)(prod >> SHIFT);
}

Using gcc 4.0.1 on 32-bit x86 and compiling with "gcc -O1 -S test.c"
gives, for me, a file test.s that looks like:

	.text
.globl _digit_mul
_digit_mul:
	pushl	%ebp
	movl	%esp, %ebp
	pushl	%esi
	subl	$4, %esp
	movl	16(%ebp), %eax
	mull	12(%ebp)
	movl	%eax, %esi
	andl	$1073741823, %esi
	movl	8(%ebp), %ecx
	movl	%esi, (%ecx)
	shrdl	$30, %edx, %eax
	addl	$4, %esp
	popl	%esi
	leave
	ret
	.subsections_via_symbols

There's only a single mull instruction there, showing that gcc
is doing the right thing, at least when optimization is turned on.
Without optimization, gcc produces three separate
multiplications, two of which are multiplications
by zero.

But if I compile with the -m64 flag to force 64-bit then the
multiply becomes:

        imulq   %rsi, %rdx

which looks a lot like a 64 x 64 -> 64 multiply to me.  This
seems inefficient, when a 32 x 32 -> 64 bit multiply ought
to be good enough.  But maybe there isn't a significant
performance difference on x86_64?

Mark

From jukka.aho at iki.fi  Tue Nov 11 13:00:36 2008
From: jukka.aho at iki.fi (Jukka Aho)
Date: Tue, 11 Nov 2008 14:00:36 +0200
Subject: [Python-Dev] Python 3.0rc2: problem with exec()ing files
Message-ID: <gfbs53$b31$1@ger.gmane.org>

The current Py3k documentation [1] states that the built-in exec() 
function should be able to execute code from "open file objects":

--- 8< ---

exec(object[, globals[, locals]])

This function supports dynamic execution of Python code. object must be 
either a string, an open file object, or a code object. [...] If it is 
an open file, the file is parsed until EOF and executed.

--- 8< ---

However, at least on Python 3.0rc2 [2], this does not seem to work. 
Please consider the following example:

1) First, let's prepare a source file which will contain a single line 
of Python code (it is assumed here that it is safe to save the file in 
the current working path):

 >>> f = open("exectest.py", mode="wt", encoding="utf-8")
 >>> f.write("print('Hello world!')\n")
 22
 >>> f.close()

2) Then, try executing that newly created file with the exec() function. 
What I get is this:

 >>> f = open("exectest.py", mode="rb")
 >>> exec(f)
 Traceback (most recent call last):
   File "<pyshell#30>", line 1, in <module>
     exec(f)
 TypeError: exec() arg 1 must be a string, bytes or code object
 >>> f
 <io.BufferedReader object at 0x013832D0>

* * *

So, am I seeing this behavior because...

a) I've misinterpreted the available documentation

b) The documentation does not agree with the current development goals

c) It is a bug or a yet-to-be-implemented feature of the built-in exec() 
function?

_____

[1] <http://docs.python.org/dev/3.0/library/functions.html#exec>

[2] Python 3.0rc2 (r30rc2:67141, Nov  7 2008, 11:43:46) [MSC v.1500 32 
bit (Intel)] on win32, to be exact.

-- 
znark



From victor.stinner at haypocalc.com  Tue Nov 11 14:25:18 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 11 Nov 2008 14:25:18 +0100
Subject: [Python-Dev] Optimize Python long integers
Message-ID: <200811111425.18392.victor.stinner@haypocalc.com>

Hi,

Patches
=======

There are some very interesting propositions (with patches!) to optimize 
Python int and long types (especially the long integers).
--------------------------------
haypo: Macros for PyLong: sign, number of digits, fits in an int
http://bugs.python.org/issue4294

marketdickinson: Use 30-bit digits instead of 15-bit digits for integers
http://bugs.python.org/issue4258

pernici: faster long multiplication
http://bugs.python.org/issue3944

haypo: Victor Stinner's GMP patch for longs
http://bugs.python.org/issue1814

fredrikj: Asymptotically faster divmod and str(long)
http://bugs.python.org/issue3451
--------------------------------

See also:
--------------------------------
fredrikj: create a numbits() method for int and long types
http://bugs.python.org/issue3439
--------------------------------


Benchmark
=========

I tried to do benchmark on all these patches using pystone or pybench, but the 
results are inaccurate. Pystone results change with +/- 20% with the same 
code on different runs. I tried more loops (pystone 250000), but it doesn't 
change anything. Pybench is better it is also inaccurate to benchmark 
operations on integers.

That's I started to write a *basic* benchmark tool to compare the different 
patches: see file bench_int.py of the issue #4294.

Benchmark on a 64 bits CPU @2.5 GHz :
-------------------------------
original     : 896.5 ms
 + macros    : 891.0 ms (+0.6%) | 
 ++ optimize : 884.3 ms (+1.4%) |-- issue #4294
 +++ shift   : 880.8 ms (+1.7%) |
fast long    : 746.8 ms (+16%)  -- issue #3944 
GMP          : 700.9 ms (+22%)  -- issue #1814
30 bits      : 659.9 ms (+26%)  -- issue #4258
-------------------------------

Benchmark on a 32 bits CPU @3.0 GHz :
---------------------------
original  : 1564.3 ms
fast long : 1591.7 ms (-2%)
GMP       : 1564.4 ms (=)
30 bits   : 1497.3 ms (+4%)
---------------------------

Don't trust the benchmark results since my tool is young and may also be 
inaccurate, but it's a good start to compare the patches.

Notes:
 - my macro patch doesn't want to optimize anything, it's just
   a preparation for new patches; but I also attached "optimization"
   patches which are useless (only +1.7% with all patches).
 - 30 bits is always faster
 - GMP is faster on 64 bits or at same speed on 32 bits
 - fast long is slower on a 32 bits CPU


Python integers
===============

I tried to understand why the "30 bits" patch is not as fast as I expected. I 
introduced statistics in the long type: see long_stat.patch of the issue 
#4258. Summary (on a 32 bits CPU):

- PyLong_FromLong(), long_add() and long_compare() are the 3 most common
  operations on integers. 
- Most integers are in range [-255; 255], large integers are rare. With 
  make and pystone programs, largest integer has 1321 bits, but there is
  just one such integer. Another is 33 bits, but all other integers fits 
  in 32 bits (without the sign, so 33 bits with the sign). I saw that the 
  symbol table use memory address in a dictionary key (so 32 bits on a
  32 bits CPU and 64 bits on a 32 bits CPU).
  => we have to focus on small integers
- pystone is inaccurate to benchmark integers: it only uses a big integer
  (more than 1 digit in base 2^30) on 1.000.000 integers
  => don't use pystone!


I don't have a solution working on any CPU with all numbers, this email is 
just a summary of the interesting integer patches.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From solipsis at pitrou.net  Tue Nov 11 14:35:08 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 11 Nov 2008 13:35:08 +0000 (UTC)
Subject: [Python-Dev] Optimize Python long integers
References: <200811111425.18392.victor.stinner@haypocalc.com>
Message-ID: <loom.20081111T133230-675@post.gmane.org>

Victor Stinner <victor.stinner <at> haypocalc.com> writes:
> 
> I tried to do benchmark on all these patches using pystone or pybench, but the 
> results are inaccurate. Pystone results change with +/- 20% with the same 
> code on different runs. I tried more loops (pystone 250000), but it doesn't 
> change anything. Pybench is better it is also inaccurate to benchmark 
> operations on integers.
> 
> That's I started to write a *basic* benchmark tool to compare the different 
> patches: see file bench_int.py of the issue #4294.

If you want to benchmark arithmetic on large integers, you may try out the
pidigits test from the Computer Language Shootout :
http://shootout.alioth.debian.org/u64/benchmark.php?test=pidigits&lang=python&id=1

cheers

Antoine.



From aahz at pythoncraft.com  Tue Nov 11 16:24:20 2008
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 11 Nov 2008 07:24:20 -0800
Subject: [Python-Dev] Python 3.0rc2: problem with exec()ing files
In-Reply-To: <gfbs53$b31$1@ger.gmane.org>
References: <gfbs53$b31$1@ger.gmane.org>
Message-ID: <20081111152420.GA1853@panix.com>

On Tue, Nov 11, 2008, Jukka Aho wrote:
>
> The current Py3k documentation [1] states that the built-in exec()  
> function should be able to execute code from "open file objects":

Second of all, this discussion should probably go on the python-3000
list.  But first of all, please file a bug report -- that's the only way
we have of tracking things.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From doko at ubuntu.com  Tue Nov 11 16:22:46 2008
From: doko at ubuntu.com (Matthias Klose)
Date: Tue, 11 Nov 2008 16:22:46 +0100
Subject: [Python-Dev] Python 2.5.3: call for patches
In-Reply-To: <48EB0F5C.602@v.loewis.de>
References: <48EB0F5C.602@v.loewis.de>
Message-ID: <4919A346.60507@ubuntu.com>

Martin v. L?wis schrieb:
> Within a few weeks, we will release Python 2.5.3. This will be the last
> bug fix release of Python 2.5, afterwards, future releases of 2.5 will
> only include security fixes, and no binaries (for Windows or OSX) will
> be provided anymore (from python.org).
> 
> In principle, the release will include all changes that are already on
> the release25-maint branch in subversion [1]. If you think that specific
> changes should be considered, please create an issue in the bug tracker
> [2], and label it with the 2.5.3 version. Backports of changes that
> are already released in Python 2.6 but may apply to 2.5 are of
> particular interest.

I would like to apply fixes for some CVE's which are addressed in 2.5 but not
yet in 2.4. this would include

CVE-2007-4965
CVE-2008-1679
CVE-2008-1721
CVE-2008-2315
CVE-2008-3144
CVE-2008-1887
CVE-2008-4864

Matthias


From tseaver at palladion.com  Tue Nov 11 18:00:04 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Tue, 11 Nov 2008 12:00:04 -0500
Subject: [Python-Dev] Python 2.5.3: call for patches
In-Reply-To: <4919A346.60507@ubuntu.com>
References: <48EB0F5C.602@v.loewis.de> <4919A346.60507@ubuntu.com>
Message-ID: <4919BA14.9020008@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Matthias Klose wrote:
> Martin v. L?wis schrieb:
>> Within a few weeks, we will release Python 2.5.3. This will be the last
>> bug fix release of Python 2.5, afterwards, future releases of 2.5 will
>> only include security fixes, and no binaries (for Windows or OSX) will
>> be provided anymore (from python.org).
>>
>> In principle, the release will include all changes that are already on
>> the release25-maint branch in subversion [1]. If you think that specific
>> changes should be considered, please create an issue in the bug tracker
>> [2], and label it with the 2.5.3 version. Backports of changes that
>> are already released in Python 2.6 but may apply to 2.5 are of
>> particular interest.
> 
> I would like to apply fixes for some CVE's which are addressed in 2.5 but not
> yet in 2.4. this would include
> 
> CVE-2007-4965
> CVE-2008-1679
> CVE-2008-1721
> CVE-2008-2315
> CVE-2008-3144
> CVE-2008-1887
> CVE-2008-4864

+1 from a non-core developer who still has to keep sites using 2.4 up
and running.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJGboU+gerLs4ltQ4RAmdLAJ9VDv8GpnhnNYI9tK0Sn9ClAbzg/wCbBxDD
aKXYjnK50Cbeb8fp/QL/kaE=
=cUE7
-----END PGP SIGNATURE-----


From guido at python.org  Tue Nov 11 18:09:10 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 11 Nov 2008 09:09:10 -0800
Subject: [Python-Dev] Python 3.0rc2: problem with exec()ing files
In-Reply-To: <20081111152420.GA1853@panix.com>
References: <gfbs53$b31$1@ger.gmane.org> <20081111152420.GA1853@panix.com>
Message-ID: <ca471dc20811110909k4f48d1e7kad9790c277ea0733@mail.gmail.com>

2008/11/11 Aahz <aahz at pythoncraft.com>:
> On Tue, Nov 11, 2008, Jukka Aho wrote:
>>
>> The current Py3k documentation [1] states that the built-in exec()
>> function should be able to execute code from "open file objects":
>
> Second of all, this discussion should probably go on the python-3000
> list.  But first of all, please file a bug report -- that's the only way
> we have of tracking things.

Third of all, make it a doc bug -- the feature is gone and isn't coming back.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Tue Nov 11 20:37:57 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 11 Nov 2008 20:37:57 +0100
Subject: [Python-Dev] Python 2.5.3: call for patches
In-Reply-To: <4919A346.60507@ubuntu.com>
References: <48EB0F5C.602@v.loewis.de> <4919A346.60507@ubuntu.com>
Message-ID: <4919DF15.2090406@v.loewis.de>

> I would like to apply fixes for some CVE's which are addressed in 2.5 but not
> yet in 2.4. this would include
> 
> CVE-2007-4965
> CVE-2008-1679
> CVE-2008-1721
> CVE-2008-2315
> CVE-2008-3144
> CVE-2008-1887
> CVE-2008-4864

Can you identify the revisions that would need backporting?

I could only find (trunk revisions)
  CVE-2007-4965: r65880
  CVE-2008-1721: r62235, issue2586
  CVE-2008-3144: issue2588, issue2589, r63734, r63728.
  CVE-2008-1887: issue2587, r62261, r62271
  CVE-2008-4864: r66689

So what about

  CVE-2008-1679: claimed to be issue1179 in the CVE, but
                 that says it fixes CVE-2007-4965 only?
  CVE-2008-2315

In principle, this is fine with me, so go ahead.

Regards,
Martin

From musiccomposition at gmail.com  Tue Nov 11 22:44:06 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 11 Nov 2008 15:44:06 -0600
Subject: [Python-Dev] Python 3.0rc2: problem with exec()ing files
In-Reply-To: <ca471dc20811110909k4f48d1e7kad9790c277ea0733@mail.gmail.com>
References: <gfbs53$b31$1@ger.gmane.org> <20081111152420.GA1853@panix.com>
	<ca471dc20811110909k4f48d1e7kad9790c277ea0733@mail.gmail.com>
Message-ID: <1afaf6160811111344m82bba25sd1a1a9bd29929d16@mail.gmail.com>

On Tue, Nov 11, 2008 at 11:09 AM, Guido van Rossum <guido at python.org> wrote:
> 2008/11/11 Aahz <aahz at pythoncraft.com>:
>> On Tue, Nov 11, 2008, Jukka Aho wrote:
>>>
>>> The current Py3k documentation [1] states that the built-in exec()
>>> function should be able to execute code from "open file objects":
>>
>> Second of all, this discussion should probably go on the python-3000
>> list.  But first of all, please file a bug report -- that's the only way
>> we have of tracking things.
>
> Third of all, make it a doc bug -- the feature is gone and isn't coming back.

No need now! Fixed in r67188.


-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From thomas at python.org  Tue Nov 11 23:40:05 2008
From: thomas at python.org (Thomas Wouters)
Date: Tue, 11 Nov 2008 23:40:05 +0100
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <200811111425.18392.victor.stinner@haypocalc.com>
References: <200811111425.18392.victor.stinner@haypocalc.com>
Message-ID: <9e804ac0811111440w31d36enb9c890382f4a4951@mail.gmail.com>

On Tue, Nov 11, 2008 at 14:25, Victor Stinner
<victor.stinner at haypocalc.com>wrote:

> There are some very interesting propositions (with patches!) to optimize
> Python int and long types (especially the long integers).


Here's another one:
http://code.python.org/loggerhead/users/twouters/intopt-- integer
inlining through pointer tagging trickery. In Python 2.6 it costs
2-4% overall performance but increases integer arithmetic (in the range
[-0x40000000, 0x40000000) only) by 10-20% according to my rough measurements
(I haven't tried your benchmark yet.) I haven't ported it to 3.0 but it
should provide a bigger win there. It also breaks API compatibility in a few
ways: Py_TYPE(o) and Py_REFCNT(o) are no longer valid lvalues, and they and
Py_INCREF(o) and Py_DECREF(o) may all evaluate 'o' twice. And, worst of all,
it exposes the tagged pointers to third-party extensions, so anything not
doing typechecks with Py_TYPE(o) will likely cause buserrors.

In retrospect, perhaps this is too controversial to be added to the list ;-)
I don't really expect this to be something CPython would want to use as-is,
although there may be use for tagged pointers in more controlled
environments (like function locals.)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081111/71f64805/attachment.htm>

From martin at v.loewis.de  Wed Nov 12 00:14:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 12 Nov 2008 00:14:40 +0100
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <200811111425.18392.victor.stinner@haypocalc.com>
References: <200811111425.18392.victor.stinner@haypocalc.com>
Message-ID: <491A11E0.7010301@v.loewis.de>

> There are some very interesting propositions (with patches!) to optimize 
> Python int and long types (especially the long integers).

Just trying to clarify the focus: would you like to see any of these
applied to 2.6? To me, it's clear that they are out of scope now, as
they don't fix bugs.

Regards,
Martin

From barry at python.org  Wed Nov 12 00:52:53 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Nov 2008 18:52:53 -0500
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <491A11E0.7010301@v.loewis.de>
References: <200811111425.18392.victor.stinner@haypocalc.com>
	<491A11E0.7010301@v.loewis.de>
Message-ID: <402AB712-82B7-4C07-BE3B-55CB0478C759@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 11, 2008, at 6:14 PM, Martin v. L?wis wrote:

>> There are some very interesting propositions (with patches!) to  
>> optimize
>> Python int and long types (especially the long integers).
>
> Just trying to clarify the focus: would you like to see any of these
> applied to 2.6?

No!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRoa1nEjvBPtnXfVAQKwwgP/YvkcFbFc+RDV3VSJ6EHWuBVOdG9YFEGq
Riq2GAst7kBMrteMfMHSv0Vb3elngLPRCKxTndUIV9B/ksfVQEHNbz9l1z7HRxmZ
0jVeYCkXCj923bsZ48Gq1MmcZ1d07TERfSVCDXnKooQgj8GlNqT70ru/0+eMFk8d
wKLL6cpdak0=
=cNoh
-----END PGP SIGNATURE-----

From victor.stinner at haypocalc.com  Wed Nov 12 00:56:41 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 12 Nov 2008 00:56:41 +0100
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <491A11E0.7010301@v.loewis.de>
References: <200811111425.18392.victor.stinner@haypocalc.com>
	<491A11E0.7010301@v.loewis.de>
Message-ID: <200811120056.42113.victor.stinner@haypocalc.com>

Le Wednesday 12 November 2008 00:14:40, vous avez ?crit?:
> > There are some very interesting propositions (with patches!) to optimize
> > Python int and long types (especially the long integers).
>
> Just trying to clarify the focus: would you like to see any of these
> applied to 2.6?

All optimisations patches are long and may introduce new regressions. It's 
better to wait for 2.7/3.1.

But it would be nice to get numbits() method (or property?) in Python 3.0(.1) 
and/or Python 2.6.1.


-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From barry at python.org  Wed Nov 12 01:02:44 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 11 Nov 2008 19:02:44 -0500
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <200811120056.42113.victor.stinner@haypocalc.com>
References: <200811111425.18392.victor.stinner@haypocalc.com>
	<491A11E0.7010301@v.loewis.de>
	<200811120056.42113.victor.stinner@haypocalc.com>
Message-ID: <60339C3D-6381-48A7-B4EF-20CDA85D30EC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 11, 2008, at 6:56 PM, Victor Stinner wrote:

> Le Wednesday 12 November 2008 00:14:40, vous avez ?crit :
>>> There are some very interesting propositions (with patches!) to  
>>> optimize
>>> Python int and long types (especially the long integers).
>>
>> Just trying to clarify the focus: would you like to see any of these
>> applied to 2.6?
>
> All optimisations patches are long and may introduce new  
> regressions. It's
> better to wait for 2.7/3.1.
>
> But it would be nice to get numbits() method (or property?) in  
> Python 3.0(.1)
> and/or Python 2.6.1.

Anyone remember the bool debacle?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRodJHEjvBPtnXfVAQK0VwP/f2vyfahYxhRh/ug+ekMp63rhVvy2iMTn
VXndnaKqJtJovjuM3YAGQk9/8l6tD4w0DklAi4e175aBvwzRkWb4RwMHGMO2/jn1
mNjloHqku6qIg6+p7jS5ytwsH6sGndgAjARY7jFE8OgYoYPrxtTabgXpr9HM631K
a2j8FUmCVQ8=
=yirj
-----END PGP SIGNATURE-----

From dickinsm at gmail.com  Wed Nov 12 01:03:48 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 12 Nov 2008 00:03:48 +0000
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <491A11E0.7010301@v.loewis.de>
References: <200811111425.18392.victor.stinner@haypocalc.com>
	<491A11E0.7010301@v.loewis.de>
Message-ID: <5c6f2a5d0811111603o4d4d8d66y40fbb0e0dd375a3a@mail.gmail.com>

On Tue, Nov 11, 2008 at 11:14 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Just trying to clarify the focus: would you like to see any of these
> applied to 2.6? To me, it's clear that they are out of scope now, as
> they don't fix bugs.

There are some minor bugs in longobject.c that I think should be applied
to 2.6.1 and 3.0.1 (not worth it for 3.0).  I'll try to put together a
separate patch containing these.  They're mostly either missing casts
or places where int should have been changed to Py_ssize_t.

numbits shouldn't go into 2.6 or 3.0.

Mark

From dickinsm at gmail.com  Wed Nov 12 01:16:30 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 12 Nov 2008 00:16:30 +0000
Subject: [Python-Dev] n.numbits: method or property?
Message-ID: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>

As Victor Stinner mentioned in a recent thread, there's a patch in
the works to add a numbits ... um ... gadget to integers, returning
the number of bits needed to represent the integer (or more
precisely, the ceiling of the log to base 2 of the integer).  See

http://bugs.python.org/issue3439

Question for python-dev: if numbits were to be added, should it
be a property or a method? (Or something else entirely?)

More generally, what are the guidelines for determining
when it's appropriate to make something a property rather
than a method?

In favour of numbits being a (read-only, of course) property:

  - it looks nicer that way
  - it's a fast O(1) method, so is comparable to an attribute access
    in terms of efficiency
  - it can reasonably be thought of as a fundamental property of
    a binary integer
  - there are precedents, as Raymond points out in the issue
    discussion:  e.g. z.real, z.imag

Thoughts?

Mark

From victor.stinner at haypocalc.com  Wed Nov 12 01:37:17 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Wed, 12 Nov 2008 01:37:17 +0100
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
Message-ID: <200811120137.17615.victor.stinner@haypocalc.com>

> n.numbits: method or property?

Fredrik Johansson and me agree to use a property.

My last patch (numbits-4.patch) implement numbits as a property. Examples:
>>> (1023).numbits
10
>>> x=1023L; x.numbits
10L
>>> x=2**(2**10); x.numbits
1025L
>>> x=2**(2**10); x.numbits.numbits   # combo!
11L

Victor

From dickinsm at gmail.com  Wed Nov 12 01:41:20 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 12 Nov 2008 00:41:20 +0000
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
Message-ID: <5c6f2a5d0811111641p41bb11a9k9335a31a4c8e4610@mail.gmail.com>

On Wed, Nov 12, 2008 at 12:16 AM, Mark Dickinson <dickinsm at gmail.com> wrote:
> precisely, the ceiling of the log to base 2 of the integer).  See

D'oh.   s/ceiling/1+floor/

Mark

From mike.klaas at gmail.com  Wed Nov 12 02:59:12 2008
From: mike.klaas at gmail.com (Mike Klaas)
Date: Tue, 11 Nov 2008 17:59:12 -0800
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
Message-ID: <0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>

On 11-Nov-08, at 4:16 PM, Mark Dickinson wrote:
>
> More generally, what are the guidelines for determining
> when it's appropriate to make something a property rather
> than a method?

Both are awkward on numeric types in python, necessitating brackets or  
a space before the dot:

(1).__doc__
1 .__doc__

I'd suggest a third alternative, which is a standalone function in math:

from math import numbits:
numbits(1)

-Mike

From steve at holdenweb.com  Wed Nov 12 03:07:41 2008
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 11 Nov 2008 21:07:41 -0500
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <200811120056.42113.victor.stinner@haypocalc.com>
References: <200811111425.18392.victor.stinner@haypocalc.com>	<491A11E0.7010301@v.loewis.de>
	<200811120056.42113.victor.stinner@haypocalc.com>
Message-ID: <491A3A6D.8070101@holdenweb.com>

Victor Stinner wrote:
> Le Wednesday 12 November 2008 00:14:40, vous avez ?crit :
>>> There are some very interesting propositions (with patches!) to optimize
>>> Python int and long types (especially the long integers).
>> Just trying to clarify the focus: would you like to see any of these
>> applied to 2.6?
> 
> All optimisations patches are long and may introduce new regressions. It's 
> better to wait for 2.7/3.1.
> 
> But it would be nice to get numbits() method (or property?) in Python 3.0(.1) 
> and/or Python 2.6.1.
> 
> 
But that would be new functionality in a micro-release, which is verboten.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Wed Nov 12 03:07:41 2008
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 11 Nov 2008 21:07:41 -0500
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <200811120056.42113.victor.stinner@haypocalc.com>
References: <200811111425.18392.victor.stinner@haypocalc.com>	<491A11E0.7010301@v.loewis.de>
	<200811120056.42113.victor.stinner@haypocalc.com>
Message-ID: <491A3A6D.8070101@holdenweb.com>

Victor Stinner wrote:
> Le Wednesday 12 November 2008 00:14:40, vous avez ?crit :
>>> There are some very interesting propositions (with patches!) to optimize
>>> Python int and long types (especially the long integers).
>> Just trying to clarify the focus: would you like to see any of these
>> applied to 2.6?
> 
> All optimisations patches are long and may introduce new regressions. It's 
> better to wait for 2.7/3.1.
> 
> But it would be nice to get numbits() method (or property?) in Python 3.0(.1) 
> and/or Python 2.6.1.
> 
> 
But that would be new functionality in a micro-release, which is verboten.

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From facundobatista at gmail.com  Wed Nov 12 03:08:40 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 12 Nov 2008 00:08:40 -0200
Subject: [Python-Dev] A statistic for Python tickets
Message-ID: <e04bdf310811111808r43ea815ct553f7a7721bbbb7d@mail.gmail.com>

Hi all! I generated this info for other list, thought that it could be
valuable to share it here...

I've been collecting some statistics since January, as I'm concerned
about the issue of tickets quantity just getting higher.

I've been saving twice a day (everytime I generate this [0] info), the
following numbers for *still open* tickets:

 - How many tickets are open since: less than one week, between one
week and two months, between two months and a year, more than a year.

- How many tickets don't have any comment in the same periods.

- How many tickets per keyword.

- How many tickets per version.

I just graphed the first number.... how many tickets are open since
the different periods. As you can see here [1], the tickets that are
open since less than a week remain more or less constant. Those opened
between a week and two months even seem to go down a little in the
last weeks.

But those opened more than two months ago just increase.

Looking at this information, and applying some experience with the
tickets, my conclusion is that when a ticket is fresh, is easier to
handle it, but when a ticket starts to getting older, it just stalls.

I can think now two reasons for this behaviour: the first is that in
some tickets when a discussion arises, sometimes it gets to a dead
point, from where it's very difficult to get out. The second is that
in a lot of cases more info is needed from the OP, and the older the
ticket, the less responsive is the OP.

In general, I think that a lot of older tickets drain a lot of effort
from developers, but I don't know which is the best way to handle
this, :(

Note: the "jump" that appears in the numbers at the beginning was an
import of old SourceForge tickets to the new system, IIRC.

Regards,

[0] http://www.taniquetil.com.ar/cgi-bin/pytickets.py
[1] http://www.taniquetil.com.ar/facundo/stathist.png

--
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From tjreedy at udel.edu  Wed Nov 12 04:28:07 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 11 Nov 2008 22:28:07 -0500
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
Message-ID: <gfdig0$ar4$1@ger.gmane.org>

Mike Klaas wrote:
> On 11-Nov-08, at 4:16 PM, Mark Dickinson wrote:
>>
>> More generally, what are the guidelines for determining
>> when it's appropriate to make something a property rather
>> than a method?
> 
> Both are awkward on numeric types in python, necessitating brackets or a 
> space before the dot:
> 
> (1).__doc__
> 1 .__doc__
> 
> I'd suggest a third alternative, which is a standalone function in math:
> 
> from math import numbits:
> numbits(1)

Math is pretty much float, not int functions.


From tjreedy at udel.edu  Wed Nov 12 04:30:31 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 11 Nov 2008 22:30:31 -0500
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <200811120056.42113.victor.stinner@haypocalc.com>
References: <200811111425.18392.victor.stinner@haypocalc.com>	<491A11E0.7010301@v.loewis.de>
	<200811120056.42113.victor.stinner@haypocalc.com>
Message-ID: <gfdikh$ar4$2@ger.gmane.org>

Victor Stinner wrote:
> Le Wednesday 12 November 2008 00:14:40, vous avez ?crit :
>>> There are some very interesting propositions (with patches!) to optimize
>>> Python int and long types (especially the long integers).
>> Just trying to clarify the focus: would you like to see any of these
>> applied to 2.6?
> 
> All optimisations patches are long and may introduce new regressions. It's 
> better to wait for 2.7/3.1.
> 
> But it would be nice to get numbits() method (or property?) in Python 3.0(.1) 
> and/or Python 2.6.1.

Try something on Pypi if you want to distribute to non-svn users sooner 
than 2.7/3.1.


From rhamph at gmail.com  Wed Nov 12 06:02:18 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 11 Nov 2008 22:02:18 -0700
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
Message-ID: <aac2c7cb0811112102m52222352ud39894ef4d4dd561@mail.gmail.com>

On Tue, Nov 11, 2008 at 6:59 PM, Mike Klaas <mike.klaas at gmail.com> wrote:
> On 11-Nov-08, at 4:16 PM, Mark Dickinson wrote:
>>
>> More generally, what are the guidelines for determining
>> when it's appropriate to make something a property rather
>> than a method?
>
> Both are awkward on numeric types in python, necessitating brackets or a
> space before the dot:
>
> (1).__doc__
> 1 .__doc__

It's only awkward on constants, which should be pretty rare for a
numbits property.  The expected use case is on a variable.


-- 
Adam Olsen, aka Rhamphoryncus

From scott+python-dev at scottdial.com  Wed Nov 12 06:40:27 2008
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 12 Nov 2008 00:40:27 -0500
Subject: [Python-Dev] A statistic for Python tickets
In-Reply-To: <e04bdf310811111808r43ea815ct553f7a7721bbbb7d@mail.gmail.com>
References: <e04bdf310811111808r43ea815ct553f7a7721bbbb7d@mail.gmail.com>
Message-ID: <491A6C4B.8010807@scottdial.com>

Facundo Batista wrote:
> I just graphed the first number.... how many tickets are open since
> the different periods. As you can see here [1], the tickets that are
> open since less than a week remain more or less constant. Those opened
> between a week and two months even seem to go down a little in the
> last weeks.
> 
> But those opened more than two months ago just increase.

I think you are finding the answer you want in the data. If I am not
mistaken, by definition those tickets that are over a week old (and
still open) get moved from the "week old" category (decrease) and into
the "two month old" category (increase). The same being true of "two
month old" -> "year" and "year" -> "older". So you really cannot say
with any sort of certainty why "week", "two_month", or "year" decrease
at all.

I think you really have just roughly taken a moving average of the
number of tickets opened with different window sizes. What you really
want to know is how many tickets that are up to X days old are newly
resolved (the ticket-resolution rate). Then, I think you could make the
statements you just made.

BTW, I'm not necessarily disagreeing with your insights into ticket
pathology.

-Scott

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From greg.ewing at canterbury.ac.nz  Wed Nov 12 11:59:54 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 12 Nov 2008 23:59:54 +1300
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <gfdig0$ar4$1@ger.gmane.org>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
	<gfdig0$ar4$1@ger.gmane.org>
Message-ID: <491AB72A.2030905@canterbury.ac.nz>

Terry Reedy wrote:

> Math is pretty much float, not int functions.

Also, it's supposed to be confining itself to
wrapping the C math library.

-- 
Greg


From solipsis at pitrou.net  Wed Nov 12 16:15:45 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 12 Nov 2008 15:15:45 +0000 (UTC)
Subject: [Python-Dev] n.numbits: method or property?
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
	<gfdig0$ar4$1@ger.gmane.org> <491AB72A.2030905@canterbury.ac.nz>
Message-ID: <loom.20081112T151327-586@post.gmane.org>

Greg Ewing <greg.ewing <at> canterbury.ac.nz> writes:
> 
> Terry Reedy wrote:
> 
> > Math is pretty much float, not int functions.
> 
> Also, it's supposed to be confining itself to
> wrapping the C math library.

Too late, because we now have a math.factorial() function.

As for numbits, I think it should be a method, because while it's very fast for
the current implementation of longs, it could be longer to calculate for a long
implementation using something other than a power-of-2 base.

Having it as a method may also be more consistent with a hypothetical popcount()
method.

Regards

Antoine.



From doko at ubuntu.com  Wed Nov 12 08:31:03 2008
From: doko at ubuntu.com (Matthias Klose)
Date: Wed, 12 Nov 2008 08:31:03 +0100
Subject: [Python-Dev] Python 2.5.3: call for patches
In-Reply-To: <4919DF15.2090406@v.loewis.de>
References: <48EB0F5C.602@v.loewis.de> <4919A346.60507@ubuntu.com>
	<4919DF15.2090406@v.loewis.de>
Message-ID: <491A8637.8000602@ubuntu.com>

Martin v. L?wis schrieb:
>> I would like to apply fixes for some CVE's which are addressed in 2.5 but not
>> yet in 2.4. this would include
>>
>> CVE-2007-4965
>> CVE-2008-1679
>> CVE-2008-1721
>> CVE-2008-2315
>> CVE-2008-3144
>> CVE-2008-1887
>> CVE-2008-4864
> 
> Can you identify the revisions that would need backporting?
> 
> I could only find (trunk revisions)
>   CVE-2007-4965: r65880
>   CVE-2008-1721: r62235, issue2586
>   CVE-2008-3144: issue2588, issue2589, r63734, r63728.
>   CVE-2008-1887: issue2587, r62261, r62271
>   CVE-2008-4864: r66689
> 
> So what about
> 
>   CVE-2008-1679: claimed to be issue1179 in the CVE, but
>                  that says it fixes CVE-2007-4965 only?

the original fix for CVE-2007-4965 did miss two chunks, which are included in
r65878 on the 2.5 branch.

>   CVE-2008-2315

this is r65334 on the 2.5 branch and r65335 on the trunk:
Security patches from Apple:  prevent int overflow when allocating memory
this was already checked in, with an added NEWS item in 2.4.5. Moved this
to 2.4.6.

> In principle, this is fine with me, so go ahead.

Done.

From dickinsm at gmail.com  Wed Nov 12 17:07:52 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 12 Nov 2008 16:07:52 +0000
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <loom.20081112T151327-586@post.gmane.org>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
	<gfdig0$ar4$1@ger.gmane.org> <491AB72A.2030905@canterbury.ac.nz>
	<loom.20081112T151327-586@post.gmane.org>
Message-ID: <5c6f2a5d0811120807v1acc3d87r5d9c638866579ece@mail.gmail.com>

On Wed, Nov 12, 2008 at 3:15 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> As for numbits, I think it should be a method, because while it's very fast for
> the current implementation of longs, it could be longer to calculate for a long
> implementation using something other than a power-of-2 base.

Right:  numbits is only a natural property of a *binary* integer.

On the other hand, I can't realistically see Python ever adopting a
non power-of-two based implementation.

Mark

From aahz at pythoncraft.com  Wed Nov 12 17:10:46 2008
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 12 Nov 2008 08:10:46 -0800
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <5c6f2a5d0811120807v1acc3d87r5d9c638866579ece@mail.gmail.com>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
	<gfdig0$ar4$1@ger.gmane.org> <491AB72A.2030905@canterbury.ac.nz>
	<loom.20081112T151327-586@post.gmane.org>
	<5c6f2a5d0811120807v1acc3d87r5d9c638866579ece@mail.gmail.com>
Message-ID: <20081112161046.GA25795@panix.com>

On Wed, Nov 12, 2008, Mark Dickinson wrote:
> On Wed, Nov 12, 2008 at 3:15 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> 
>> As for numbits, I think it should be a method, because while it's
>> very fast for the current implementation of longs, it could be longer
>> to calculate for a long implementation using something other than a
>> power-of-2 base.
>
> Right: numbits is only a natural property of a *binary* integer.
>
> On the other hand, I can't realistically see Python ever adopting a
> non power-of-two based implementation.

What do you call Decimal?  ;-)
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From solipsis at pitrou.net  Wed Nov 12 17:11:36 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 12 Nov 2008 16:11:36 +0000 (UTC)
Subject: [Python-Dev] n.numbits: method or property?
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
	<gfdig0$ar4$1@ger.gmane.org> <491AB72A.2030905@canterbury.ac.nz>
	<loom.20081112T151327-586@post.gmane.org>
	<5c6f2a5d0811120807v1acc3d87r5d9c638866579ece@mail.gmail.com>
Message-ID: <loom.20081112T161044-593@post.gmane.org>

Mark Dickinson <dickinsm <at> gmail.com> writes:
> 
> Right:  numbits is only a natural property of a *binary* integer.
> 
> On the other hand, I can't realistically see Python ever adopting a
> non power-of-two based implementation.

No, but Python is duck-typed and alternate integer classes could adopt such an
implementation while trying to respect the Python-level long API.

cheers

Antoine.



From dickinsm at gmail.com  Wed Nov 12 19:19:57 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Wed, 12 Nov 2008 18:19:57 +0000
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <loom.20081112T161044-593@post.gmane.org>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
	<gfdig0$ar4$1@ger.gmane.org> <491AB72A.2030905@canterbury.ac.nz>
	<loom.20081112T151327-586@post.gmane.org>
	<5c6f2a5d0811120807v1acc3d87r5d9c638866579ece@mail.gmail.com>
	<loom.20081112T161044-593@post.gmane.org>
Message-ID: <5c6f2a5d0811121019j273d063eq5f39274fcd9bf503@mail.gmail.com>

[Antoine Pitrou]
> No, but Python is duck-typed and alternate integer classes could adopt such an
> implementation while trying to respect the Python-level long API.

Good point.  That makes me feel a little uneasy about adding numbits
at all.  Though I guess there are other parts of Python's integer API that are
inherently linked to a binary representation, namely, the bit operations and
shifts.  And there'd be no *requirement* for alternative integer types to
implement numbits (it wouldn't be part of the Integral ABC, for example).

[Aahz]
> What do you call Decimal?  ;-)

Um.  A stdlib module?  I guess I should have said that I can't see Python
adopting a nonbinary implementation for the 'int/long' types.

It *is* fun to dream about, though: there are some definite advantages
to an internal decimal representation---notably, faster str <-> int
conversion.  And it turns out that overall performance doesn't suffer:
I've coded up a Python extension module that implements decimal
integers (stored internally in base 10**9) whose performance handily
beats that of the current binary int/long.

Mark

From greg.ewing at canterbury.ac.nz  Wed Nov 12 21:54:08 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 13 Nov 2008 09:54:08 +1300
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <loom.20081112T151327-586@post.gmane.org>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
	<gfdig0$ar4$1@ger.gmane.org> <491AB72A.2030905@canterbury.ac.nz>
	<loom.20081112T151327-586@post.gmane.org>
Message-ID: <491B4270.3080208@canterbury.ac.nz>

Antoine Pitrou wrote:

> As for numbits, I think it should be a method

It feels more method-like to me too, because it's something
derived from the int's value rather than an independent
piece of information.

-- 
Greg

From greg.ewing at canterbury.ac.nz  Wed Nov 12 22:07:53 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 13 Nov 2008 10:07:53 +1300
Subject: [Python-Dev] n.numbits: method or property?
In-Reply-To: <20081112161046.GA25795@panix.com>
References: <5c6f2a5d0811111616w77937b6y12d75d91dfdaa405@mail.gmail.com>
	<0D66A390-2C67-4B43-93E8-ED6011714761@gmail.com>
	<gfdig0$ar4$1@ger.gmane.org> <491AB72A.2030905@canterbury.ac.nz>
	<loom.20081112T151327-586@post.gmane.org>
	<5c6f2a5d0811120807v1acc3d87r5d9c638866579ece@mail.gmail.com>
	<20081112161046.GA25795@panix.com>
Message-ID: <491B45A9.1090200@canterbury.ac.nz>

Aahz wrote:

> What do you call Decimal?  ;-)

If you're working with decimal numbers, you're more
likely to want a numdigits() method.

-- 
Greg

From martin at v.loewis.de  Wed Nov 12 22:50:51 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 12 Nov 2008 22:50:51 +0100
Subject: [Python-Dev] Python 2.5.3: call for patches
In-Reply-To: <491A8637.8000602@ubuntu.com>
References: <48EB0F5C.602@v.loewis.de>
	<4919A346.60507@ubuntu.com>	<4919DF15.2090406@v.loewis.de>
	<491A8637.8000602@ubuntu.com>
Message-ID: <491B4FBB.4080306@v.loewis.de>

>> In principle, this is fine with me, so go ahead.
> 
> Done.

Thanks for looking into these!

Martin

From dpeterson at enthought.com  Thu Nov 13 01:12:37 2008
From: dpeterson at enthought.com (Dave Peterson)
Date: Wed, 12 Nov 2008 18:12:37 -0600
Subject: [Python-Dev] How does one build Python25.chm on Windows?
Message-ID: <491B70F5.1060809@enthought.com>

I'm trying to build a Python25.chm file using the source in the Python 
2.5.2 tarball.  

I'm not really a Windows guru so I'm at a loss of even where to begin as 
the Doc\README file doesn't mention how to build .chm files at all.   
I've done a number of web searches with things like "how to build 
python25.chm" but am not finding any useful hits.   (Most hits tell 
people how to use the Python25.chm for various purposes but none on 
actually building it that I've seen.)


Any pointers would be appreciated!


-- Dave


From martin at v.loewis.de  Thu Nov 13 01:30:06 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 13 Nov 2008 01:30:06 +0100
Subject: [Python-Dev] How does one build Python25.chm on Windows?
In-Reply-To: <491B70F5.1060809@enthought.com>
References: <491B70F5.1060809@enthought.com>
Message-ID: <491B750E.4040100@v.loewis.de>

> I'm not really a Windows guru so I'm at a loss of even where to begin as
> the Doc\README file doesn't mention how to build .chm files at all.  

You need to build the html files, then run prechm.py, then invoke the
HTML Help Workshop.

HTH,
Martin

From techtonik at gmail.com  Thu Nov 13 13:49:06 2008
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 13 Nov 2008 14:49:06 +0200
Subject: [Python-Dev] Upgrade SVN server to 1.5.4
Message-ID: <d34314100811130449x3f544910jee365f271ebb13bf@mail.gmail.com>

I use new sparse checkouts feature added in SVN 1.5 a lot to save some
traffic when working with Python code.  However, when server is older
than 1.5, partial checkouts filter directory tree on client side
fetching whole SVN tree again and again. Are there any objections
against upgrading server to 1.5?

Perhaps Python merging mechanics can be improved with upgrade too.

-- 
--anatoly t.

From techtonik at gmail.com  Thu Nov 13 15:01:16 2008
From: techtonik at gmail.com (anatoly techtonik)
Date: Thu, 13 Nov 2008 16:01:16 +0200
Subject: [Python-Dev] My patches
In-Reply-To: <4909C30F.1070100@trueblade.com>
References: <200810301108.35591.victor.stinner@haypocalc.com>
	<20081030110442.1b6ce78e@resist.wooz.org>
	<200810301302.02796.victor.stinner@haypocalc.com>
	<200810301319.15760.eckhardt@satorlaser.com>
	<4909C30F.1070100@trueblade.com>
Message-ID: <d34314100811130601y51a4dfbbree320cf9add2d3b3@mail.gmail.com>

On Thu, Oct 30, 2008 at 4:22 PM, Eric Smith <eric at trueblade.com> wrote:

>> Bazaar. Take a look at the developers' pages on python.org, they mention
>> that a BZR checkout is available. I know that it works (though the initial
>> checkout is glacially slow) but I don't know what "official" support it has
>> or what is planned with it.
>
> I'd like to try it out, but the instructions on
> http://www.python.org/dev/bazaar/ say to get wget
> http://code.python.org/snapshots/python-bzr-snapshot.tar.bz2, which is a
> 404. Anyone know the actual URL?

Sorry if it is too late. Launchpad has Bazaar mirror of Python code.
https://code.launchpad.net/~python-dev/python/trunk
perhaps instructions on http://www.python.org/dev/bazaar/ could be updated
to reflect that fact.

-- 
--anatoly t.

From ondrej at certik.cz  Thu Nov 13 15:53:05 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Thu, 13 Nov 2008 15:53:05 +0100
Subject: [Python-Dev] Optimize Python long integers
In-Reply-To: <9e804ac0811111440w31d36enb9c890382f4a4951@mail.gmail.com>
References: <200811111425.18392.victor.stinner@haypocalc.com>
	<9e804ac0811111440w31d36enb9c890382f4a4951@mail.gmail.com>
Message-ID: <85b5c3130811130653y173f726dv3c9c48ffce7bf3dc@mail.gmail.com>

On Tue, Nov 11, 2008 at 11:40 PM, Thomas Wouters <thomas at python.org> wrote:
>
>
> On Tue, Nov 11, 2008 at 14:25, Victor Stinner <victor.stinner at haypocalc.com>
> wrote:
>>
>> There are some very interesting propositions (with patches!) to optimize
>> Python int and long types (especially the long integers).
>
> Here's another one: http://code.python.org/loggerhead/users/twouters/intopt
> -- integer inlining through pointer tagging trickery. In Python 2.6 it costs
> 2-4% overall performance but increases integer arithmetic (in the range
> [-0x40000000, 0x40000000) only) by 10-20% according to my rough measurements
> (I haven't tried your benchmark yet.) I haven't ported it to 3.0 but it
> should provide a bigger win there. It also breaks API compatibility in a few
> ways: Py_TYPE(o) and Py_REFCNT(o) are no longer valid lvalues, and they and
> Py_INCREF(o) and Py_DECREF(o) may all evaluate 'o' twice. And, worst of all,
> it exposes the tagged pointers to third-party extensions, so anything not
> doing typechecks with Py_TYPE(o) will likely cause buserrors.
>
> In retrospect, perhaps this is too controversial to be added to the list ;-)
> I don't really expect this to be something CPython would want to use as-is,
> although there may be use for tagged pointers in more controlled
> environments (like function locals.)

You might also try sympy (http://code.google.com/p/sympy/). Calculates
10**5 decima digits of pi:

pure Python integers:

$ MPMATH_NOGMPY=yes ipython
In [1]: from sympy import pi

In [2]: %time a=pi.evalf(10**5)
CPU times: user 8.56 s, sys: 0.00 s, total: 8.56 s
Wall time: 8.56 s

gmpy integers:

$ ipython
In [1]: from sympy import pi

In [2]: %time a=pi.evalf(10**5)
CPU times: user 0.28 s, sys: 0.00 s, total: 0.28 s
Wall time: 0.28 s


So with gmpy, it is 30x faster.

Ondrej

From barry at python.org  Thu Nov 13 16:18:21 2008
From: barry at python.org (Barry Warsaw)
Date: Thu, 13 Nov 2008 10:18:21 -0500
Subject: [Python-Dev] My patches
In-Reply-To: <d34314100811130601y51a4dfbbree320cf9add2d3b3@mail.gmail.com>
References: <200810301108.35591.victor.stinner@haypocalc.com>
	<20081030110442.1b6ce78e@resist.wooz.org>
	<200810301302.02796.victor.stinner@haypocalc.com>
	<200810301319.15760.eckhardt@satorlaser.com>
	<4909C30F.1070100@trueblade.com>
	<d34314100811130601y51a4dfbbree320cf9add2d3b3@mail.gmail.com>
Message-ID: <B644B645-CC3B-4001-A546-FA78028F065D@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 13, 2008, at 9:01 AM, anatoly techtonik wrote:

> On Thu, Oct 30, 2008 at 4:22 PM, Eric Smith <eric at trueblade.com>  
> wrote:
>
>>> Bazaar. Take a look at the developers' pages on python.org, they  
>>> mention
>>> that a BZR checkout is available. I know that it works (though the  
>>> initial
>>> checkout is glacially slow) but I don't know what "official"  
>>> support it has
>>> or what is planned with it.
>>
>> I'd like to try it out, but the instructions on
>> http://www.python.org/dev/bazaar/ say to get wget
>> http://code.python.org/snapshots/python-bzr-snapshot.tar.bz2, which  
>> is a
>> 404. Anyone know the actual URL?
>
> Sorry if it is too late. Launchpad has Bazaar mirror of Python code.
> https://code.launchpad.net/~python-dev/python/trunk
> perhaps instructions on http://www.python.org/dev/bazaar/ could be  
> updated
> to reflect that fact.

I've updated that page to point to the launchpad mirrors.  Thanks.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSRxFPnEjvBPtnXfVAQJtWwP/SONN5FuLamUjYtU7s0KIAZ0I3v9YYm0E
4JGLARI+Jugh+IdMlUmJUquA5CZj4FoPgYNl1oK0FPho1IbDzZVQzGu5mjwOPesY
nRnzEpO1+lHQlsqTX2NPDYFfgE5DHWvVVWlvbLchazCZNeGD8jIv9708wavEvtZ5
gyuNTLab78M=
=chHn
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Thu Nov 13 20:22:44 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 13 Nov 2008 20:22:44 +0100
Subject: [Python-Dev] Upgrade SVN server to 1.5.4
In-Reply-To: <d34314100811130449x3f544910jee365f271ebb13bf@mail.gmail.com>
References: <d34314100811130449x3f544910jee365f271ebb13bf@mail.gmail.com>
Message-ID: <491C7E84.6090405@v.loewis.de>

> I use new sparse checkouts feature added in SVN 1.5 a lot to save some
> traffic when working with Python code.  However, when server is older
> than 1.5, partial checkouts filter directory tree on client side
> fetching whole SVN tree again and again. Are there any objections
> against upgrading server to 1.5?

Yes - Debian stable does not include it. When lenny gets released, we'll
upgrade.

Regards,
Martin

From skip at pobox.com  Fri Nov 14 05:50:44 2008
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 13 Nov 2008 22:50:44 -0600
Subject: [Python-Dev] test message - spam work...
Message-ID: <18717.932.44532.170046@montanaro-dyndns-org.local>

Working on the spam filters...  hit 'd' now...

S

From daniel.furrer at gmail.com  Fri Nov 14 15:27:54 2008
From: daniel.furrer at gmail.com (Daniel Furrer)
Date: Fri, 14 Nov 2008 15:27:54 +0100
Subject: [Python-Dev] compiler optimizations: collecting ideas
Message-ID: <bc17039c0811140627p38c3a1faw6171c7a74102037b@mail.gmail.com>

Hello everybody,

As part of an advanced compiler design course at our university (ETH
Zurich), we have to implement an optimization (or various thereof).
I've spoken with a couple of people who are, like me, very fascinated by
Python.
So I would just like to ask if somebody has any ideas on what we could do?

Our constraints:
- We are 4 persons, each willing to potentially spend around 4-6 full days
of work.
- We've been working on generating Control Flow Graphs, generating Static
Single Assignment Form (inserting phi-nodes) and removing it again. We've
also implemented techniques like constant folding, copy propagation, Common
Subexpression Elimination etc.

We're looking for:
- Hints as to which python compiler would be best to work on. (The official
Python compiler package seems very nice to work with actually...)
- If there are any requests in the area of optimization that we could have a
look at

Any feedback would be highly appreciated as well as pointers to specific
bugs in the tracker and other relevant discussions.

Best regards,
Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081114/445013e4/attachment.htm>

From steve at pearwood.info  Fri Nov 14 16:06:04 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 15 Nov 2008 02:06:04 +1100
Subject: [Python-Dev] compiler optimizations: collecting ideas
In-Reply-To: <bc17039c0811140627p38c3a1faw6171c7a74102037b@mail.gmail.com>
References: <bc17039c0811140627p38c3a1faw6171c7a74102037b@mail.gmail.com>
Message-ID: <200811150206.04781.steve@pearwood.info>

On Sat, 15 Nov 2008 01:27:54 am Daniel Furrer wrote:
[snip]
> We're looking for:
> - Hints as to which python compiler would be best to work on. (The
> official Python compiler package seems very nice to work with
> actually...) - If there are any requests in the area of optimization
> that we could have a look at

Even though the PEP has been rejected, I do like the idea of optimising 
chains of if...elif:

http://www.python.org/dev/peps/pep-0275/


-- 
Steven

From ncoghlan at gmail.com  Fri Nov 14 17:42:47 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 15 Nov 2008 02:42:47 +1000
Subject: [Python-Dev] compiler optimizations: collecting ideas
In-Reply-To: <bc17039c0811140627p38c3a1faw6171c7a74102037b@mail.gmail.com>
References: <bc17039c0811140627p38c3a1faw6171c7a74102037b@mail.gmail.com>
Message-ID: <491DAA87.6060703@gmail.com>

Daniel Furrer wrote:
> Hello everybody,
> 
> As part of an advanced compiler design course at our university (ETH
> Zurich), we have to implement an optimization (or various thereof).
> I've spoken with a couple of people who are, like me, very fascinated by
> Python.
> So I would just like to ask if somebody has any ideas on what we could do?
> 
> Our constraints:
> - We are 4 persons, each willing to potentially spend around 4-6 full
> days of work.
> - We've been working on generating Control Flow Graphs, generating
> Static Single Assignment Form (inserting phi-nodes) and removing it
> again. We've also implemented techniques like constant folding, copy
> propagation, Common Subexpression Elimination etc.
> 
> We're looking for:
> - Hints as to which python compiler would be best to work on. (The
> official Python compiler package seems very nice to work with actually...)
> - If there are any requests in the area of optimization that we could
> have a look at
> 
> Any feedback would be highly appreciated as well as pointers to specific
> bugs in the tracker and other relevant discussions.

Some random (hopefully helpful) thoughts:
- the Python compiler package is a bit of a dead end (we dropped it for
Py3k because converting/maintaining two full compilers would have been a
lot of work and seemed somewhat redundant now that the AST can be
manipulated from Python code)
- 3.0's syntax is cleaner than that of 2.x (since some of the old
backward compatibility issues are gone), but 2.x is in more widespread
use (unsurprisingly, since 3.0 isn't even expected to be released until
next month)
- there is a branch in SVN where Thomas Lee (IIRC) is working on moving
some of the current optimisations from the bytecode generation phase
back to the AST generation phase.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From status at bugs.python.org  Fri Nov 14 18:07:01 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 14 Nov 2008 18:07:01 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20081114170701.1062178291@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (11/07/08 - 11/14/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2179 open (+31) / 13969 closed (+15) / 16148 total (+46)

Open issues with patches:   716

Average duration of open issues: 713 days.
Median duration of open issues: 2016 days.

Open Issues Breakdown
   open  2162 (+31)
pending    17 ( +0)

Issues Created Or Reopened (49)
_______________________________

re module treats raw strings as normal strings                   11/11/08
       http://bugs.python.org/issue4185    reopened akuchling                 
                                                                               

cycle created by profile.run                                     11/07/08
       http://bugs.python.org/issue4273    reopened benjamin.peterson         
                                                                               

Version Checker                                                  11/07/08
       http://bugs.python.org/issue4280    created  cjw                       
                                                                               

Idle - incorrectly displaying a character (Latin capital letter  11/08/08
CLOSED http://bugs.python.org/issue4281    reopened amaury.forgeotdarc        
                                                                               

exec(unicode): invalid charset when #coding:xxx spec is used     11/08/08
       http://bugs.python.org/issue4282    created  shidot                    
       patch                                                                   

"python3.0 setup.py install --user" raises AttributeError        11/08/08
CLOSED http://bugs.python.org/issue4283    created  hagen                     
       patch                                                                   

Python 2.6 64-bit + Vista 64 = Bad MSVCRT                        11/08/08
CLOSED http://bugs.python.org/issue4284    created  DaGoodBoy                 
                                                                               

Use a named tuple for sys.version_info                           11/09/08
       http://bugs.python.org/issue4285    created  brett.cannon              
       easy                                                                    

Discrepancy in format string documentation                       11/09/08
CLOSED http://bugs.python.org/issue4286    created  dlfjessup                 
                                                                               

Broken URL in documentation style guide                          11/09/08
CLOSED http://bugs.python.org/issue4287    created  facorreia                 
                                                                               

parsermodule and grammar variable                                11/09/08
       http://bugs.python.org/issue4288    created  rpetrov                   
       patch                                                                   

Python 2.6 installer crashes when selecting 'advanced' and cance 11/09/08
       http://bugs.python.org/issue4289    created  fabioz                    
                                                                               

2to3 fails with sympy                                            11/09/08
CLOSED http://bugs.python.org/issue4290    created  certik                    
                                                                               

Allow Division of datetime.timedelta Objects                     11/09/08
       http://bugs.python.org/issue4291    created  Jeremy Banks              
                                                                               

2to3 fails to convert "from something import (a, b, c)"          11/10/08
CLOSED http://bugs.python.org/issue4292    created  certik                    
       patch                                                                   

Thread Safe Py_AddPendingCall                                    11/10/08
       http://bugs.python.org/issue4293    created  krisvale                  
       patch, patch                                                            

Macros for PyLong: sign, number of digits, fits in an int        11/10/08
       http://bugs.python.org/issue4294    created  haypo                     
       patch                                                                   

closing stdout in a child process on cygwin means that process d 11/10/08
       http://bugs.python.org/issue4295    created  zooko                     
                                                                               

Python assumes identity implies equivalence; contradicts NaN     11/11/08
CLOSED http://bugs.python.org/issue4296    created  mikecurtis                
       patch                                                                   

Add error_log attribute to optparse.OptionParser                 11/11/08
       http://bugs.python.org/issue4297    created  odd_bloke                 
       patch                                                                   

pickle segfault or MemoryError on invalid input                  11/11/08
CLOSED http://bugs.python.org/issue4298    created  hagen                     
       patch                                                                   

3.0rc2.msi Install Fails (Error Code 2755)                       11/11/08
       http://bugs.python.org/issue4299    created  sharksuit                 
                                                                               

error in multiprocessing logging docs                            11/11/08
CLOSED http://bugs.python.org/issue4300    created  dursobr                   
                                                                               

incorrect and inconsistent logging in multiprocessing            11/11/08
       http://bugs.python.org/issue4301    created  dursobr                   
                                                                               

smtplib.py initialisation defect                                 11/12/08
       http://bugs.python.org/issue4302    created  piers                     
       patch                                                                   

[2.5 regression] ctypes fails to build on arm-linux-gnu          11/12/08
       http://bugs.python.org/issue4303    created  doko                      
                                                                               

build mode which fails for build failures in extensions          11/12/08
       http://bugs.python.org/issue4304    created  doko                      
                                                                               

ctypes fails to build on mipsel-linux-gnu (detects mips instead  11/12/08
       http://bugs.python.org/issue4305    created  doko                      
                                                                               

email package with unicode subject/body                          11/12/08
       http://bugs.python.org/issue4306    created  haypo                     
       patch                                                                   

inspect.FullArgSpec does not match the docs                      11/12/08
CLOSED http://bugs.python.org/issue4307    created  hagen                     
       patch                                                                   

repr of httplib.IncompleteRead is stupid                         11/12/08
       http://bugs.python.org/issue4308    created  cjw296                    
       patch                                                                   

ctypes documentation                                             11/12/08
       http://bugs.python.org/issue4309    created  LambertDW                 
                                                                               

Comparison of two ints returns wrong result                      11/12/08
CLOSED http://bugs.python.org/issue4310    created  kbrandt                   
                                                                               

Race condition on Multiprocessing module documentation           11/12/08
CLOSED http://bugs.python.org/issue4311    created  alejolp                   
                                                                               

Unicode in distutils meta-data?                                  11/13/08
       http://bugs.python.org/issue4312    created  hagen                     
                                                                               

IDLE segfault at exit                                            11/13/08
       http://bugs.python.org/issue4313    created  haypo                     
                                                                               

isalpha bug                                                      11/13/08
CLOSED http://bugs.python.org/issue4314    created  ZooKeeper                 
                                                                               

On some Python builds, exec in a function can't create shadows o 11/13/08
       http://bugs.python.org/issue4315    created  ssb22                     
                                                                               

Improper use of [] in configure.in leads to useless regexp in co 11/13/08
       http://bugs.python.org/issue4316    created  snj                       
       patch                                                                   

Buffer overflow in imageop module                                11/13/08
       http://bugs.python.org/issue4317    created  amaury.forgeotdarc        
       patch, needs review                                                     

optparse: formatting of help text/descriptions                   11/14/08
       http://bugs.python.org/issue4318    created  akuchling                 
                                                                               

optparse and non-ascii help strings                              11/14/08
       http://bugs.python.org/issue4319    created  akuchling                 
                                                                               

optparse: "1 2 3" should be seen as one string                   11/14/08
       http://bugs.python.org/issue4320    created  akuchling                 
                                                                               

unintended syntax error with decorators, parenthesis, and dots?  11/14/08
       http://bugs.python.org/issue4321    created  erickt                    
                                                                               

function with modified __name__ uses original name when there's  11/14/08
       http://bugs.python.org/issue4322    created  erickt                    
                                                                               

Wrong encoding in files saved from IDLE (3.0rc2 on Windows)      11/14/08
       http://bugs.python.org/issue4323    created  andre                     
       patch                                                                   

locale documentation is inconsistent                             11/14/08
       http://bugs.python.org/issue4324    created  kaiw                      
                                                                               

Installer crash on cancel installation.                          11/14/08
CLOSED http://bugs.python.org/issue4325    created  dreamlusion               
                                                                               

Documentation missing for OptionGroup class in optparse          11/14/08
       http://bugs.python.org/issue1665333 reopened akuchling                 
                                                                               



Issues Now Closed (34)
______________________

String format operator '%i' fails for large floats                318 days
       http://bugs.python.org/issue1716    marketdickinson           
                                                                               

test_ntpath.py converted to unittest                              290 days
       http://bugs.python.org/issue1949    amaury.forgeotdarc        
       patch, easy                                                             

Wrong doc for `calendar.Calendar.iterweekdays`                    182 days
       http://bugs.python.org/issue2869    daonb                     
       patch                                                                   

test_zipfile64 fails                                              170 days
       http://bugs.python.org/issue2971    amaury.forgeotdarc        
                                                                               

weakref subclass segfault                                         152 days
       http://bugs.python.org/issue3100    amaury.forgeotdarc        
       patch                                                                   

py3k fails under Windows if "-c" or "-m" is given a non-ascii va   64 days
       http://bugs.python.org/issue3705    amaury.forgeotdarc        
       patch                                                                   

Built-in compile() and ast module doc issues                       26 days
       http://bugs.python.org/issue4118    benjamin.peterson         
                                                                               

Tilde compression isn't applied in TOC                             17 days
       http://bugs.python.org/issue4189    georg.brandl              
                                                                               

Python core crashes with associated files.                         14 days
       http://bugs.python.org/issue4226    benjamin.peterson         
                                                                               

HTTPResponse instance has no attribute 'code'                       5 days
       http://bugs.python.org/issue4249    tjreedy                   
                                                                               

urllib2.py: HTTPResponse instance has no attribute 'code'           6 days
       http://bugs.python.org/issue4250    benjamin.peterson         
                                                                               

library.pdf documentation: "See also: Module <xxx>" does not lin    6 days
       http://bugs.python.org/issue4251    georg.brandl              
                                                                               

conversion tool does not fix "from Tkinter import N,E"              2 days
       http://bugs.python.org/issue4271    benjamin.peterson         
                                                                               

socketserver example code not correctly ported to py3k              2 days
       http://bugs.python.org/issue4275    benjamin.peterson         
       patch                                                                   

Idle - incorrectly displaying a character (Latin capital letter     1 days
       http://bugs.python.org/issue4281    loewis                    
                                                                               

"python3.0 setup.py install --user" raises AttributeError           0 days
       http://bugs.python.org/issue4283    georg.brandl              
       patch                                                                   

Python 2.6 64-bit + Vista 64 = Bad MSVCRT                           0 days
       http://bugs.python.org/issue4284    loewis                    
                                                                               

Discrepancy in format string documentation                          0 days
       http://bugs.python.org/issue4286    benjamin.peterson         
                                                                               

Broken URL in documentation style guide                             0 days
       http://bugs.python.org/issue4287    benjamin.peterson         
                                                                               

2to3 fails with sympy                                               1 days
       http://bugs.python.org/issue4290    certik                    
                                                                               

2to3 fails to convert "from something import (a, b, c)"             1 days
       http://bugs.python.org/issue4292    certik                    
       patch                                                                   

Python assumes identity implies equivalence; contradicts NaN        2 days
       http://bugs.python.org/issue4296    marketdickinson           
       patch                                                                   

pickle segfault or MemoryError on invalid input                     0 days
       http://bugs.python.org/issue4298    amaury.forgeotdarc        
       patch                                                                   

error in multiprocessing logging docs                               0 days
       http://bugs.python.org/issue4300    benjamin.peterson         
                                                                               

inspect.FullArgSpec does not match the docs                         0 days
       http://bugs.python.org/issue4307    benjamin.peterson         
       patch                                                                   

Comparison of two ints returns wrong result                         0 days
       http://bugs.python.org/issue4310    marketdickinson           
                                                                               

Race condition on Multiprocessing module documentation              0 days
       http://bugs.python.org/issue4311    benjamin.peterson         
                                                                               

isalpha bug                                                         0 days
       http://bugs.python.org/issue4314    haypo                     
                                                                               

Installer crash on cancel installation.                             0 days
       http://bugs.python.org/issue4325    amaury.forgeotdarc        
                                                                               

pclose raises spurious exception on win32                        1121 days
       http://bugs.python.org/issue1328851 amaury.forgeotdarc        
       patch                                                                   

pdb: fix for  1326406 (import  __main__ pdb failure)             1001 days
       http://bugs.python.org/issue1429539 belopolsky                
       patch                                                                   

Visual Studio 2005 CRT error handler                              941 days
       http://bugs.python.org/issue1471243 amaury.forgeotdarc        
                                                                               

New-style classes fail to cleanup attributes                      810 days
       http://bugs.python.org/issue1545463 belopolsky                
       patch                                                                   

Drop-Handler for Python files                                     636 days
       http://bugs.python.org/issue1656675 loewis                    
                                                                               



Top Issues Most Discussed (10)
______________________________

 22 Python assumes identity implies equivalence; contradicts NaN       2 days
closed  http://bugs.python.org/issue4296   

 14 Macros for PyLong: sign, number of digits, fits in an int          4 days
open    http://bugs.python.org/issue4294   

  8 Add DTrace probes                                                 33 days
open    http://bugs.python.org/issue4111   

  6 isalpha bug                                                        0 days
closed  http://bugs.python.org/issue4314   

  6 Thread Safe Py_AddPendingCall                                      4 days
open    http://bugs.python.org/issue4293   

  6 2to3 fails with sympy                                              1 days
closed  http://bugs.python.org/issue4290   

  6 parsermodule and grammar variable                                  5 days
open    http://bugs.python.org/issue4288   

  6 Idle - incorrectly displaying a character (Latin capital letter    1 days
closed  http://bugs.python.org/issue4281   

  6 Module 'parser' fails to build                                     7 days
open    http://bugs.python.org/issue4279   

  6 socketserver example code not correctly ported to py3k             2 days
closed  http://bugs.python.org/issue4275   




From stefan_ml at behnel.de  Fri Nov 14 19:24:46 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 14 Nov 2008 19:24:46 +0100
Subject: [Python-Dev] compiler optimizations: collecting ideas
In-Reply-To: <200811150206.04781.steve@pearwood.info>
References: <bc17039c0811140627p38c3a1faw6171c7a74102037b@mail.gmail.com>
	<200811150206.04781.steve@pearwood.info>
Message-ID: <gfkfpe$4eh$1@ger.gmane.org>

Steven D'Aprano wrote:
> On Sat, 15 Nov 2008 01:27:54 am Daniel Furrer wrote:
> [snip]
>> We're looking for:
>> - Hints as to which python compiler would be best to work on. (The
>> official Python compiler package seems very nice to work with
>> actually...) - If there are any requests in the area of optimization
>> that we could have a look at
> 
> Even though the PEP has been rejected, I do like the idea of optimising 
> chains of if...elif:
> 
> http://www.python.org/dev/peps/pep-0275/

As a side note: Cython does this optimisation and translates a series of
if-else statements (as well as "or" expressions) into C switch statements. The
speedup is pretty impressive at times.

Stefan


From stefan_ml at behnel.de  Fri Nov 14 19:59:24 2008
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 14 Nov 2008 19:59:24 +0100
Subject: [Python-Dev] compiler optimizations: collecting ideas
In-Reply-To: <bc17039c0811140627p38c3a1faw6171c7a74102037b@mail.gmail.com>
References: <bc17039c0811140627p38c3a1faw6171c7a74102037b@mail.gmail.com>
Message-ID: <gfkhqc$coa$1@ger.gmane.org>

Daniel Furrer wrote:
> As part of an advanced compiler design course at our university (ETH
> Zurich), we have to implement an optimization (or various thereof).
> I've spoken with a couple of people who are, like me, very fascinated by
> Python.
> So I would just like to ask if somebody has any ideas on what we could do?

Plenty! :)


> Our constraints:
> - We are 4 persons, each willing to potentially spend around 4-6 full days
> of work.
> - We've been working on generating Control Flow Graphs, generating Static
> Single Assignment Form (inserting phi-nodes) and removing it again. We've
> also implemented techniques like constant folding, copy propagation, Common
> Subexpression Elimination etc.
> 
> We're looking for:
> - Hints as to which python compiler would be best to work on. (The official
> Python compiler package seems very nice to work with actually...)

Cython is a Python-to-C compiler that is itself written in Python. It's good
enough to compile major parts of itself already, and the generated C code is
fast enough to challenge even experienced C programmers.

http://cython.org/

A really nice thing about Cython is that it targets efficient C code, not
optimised assembly. This allows Cython to focus its optimisations on things
that the C compiler cannot do, i.e. it tries to generate C code that makes it
clear to the C compiler what the intentions of the Python code were, and can
then leave the platform specific and low-level optimisations to the C compiler.

The optimisation infrastructure works in tree transformations through the
Visitor pattern, so it's actually quite simple to add new optimisations. A
couple of examples:

http://hg.cython.org/cython-devel/file/tip/Cython/Compiler/Optimize.py
http://hg.cython.org/cython-devel/file/tip/Cython/Compiler/AutoDocTransforms.py


> - If there are any requests in the area of optimization that we could have a
> look at

Our Wiki has a some (more or less up-to-date) enhancement proposals (CEPs):

http://wiki.cython.org/enhancements

But a major thing that Cython currently lacks (and that would definitely allow
many new optimisations) is a real Control Flow Graph. It has some initial flow
control tracking, but that mainly targeted variable initialisation at the
time. Many other optimisations, especially type inference attempts, largely
depend on better flow control tracking.

There are some initial thoughts here:

http://wiki.cython.org/enhancements/FlowGraph

It's not very complete yet, but this is definitely something that is worth
some more discussion on our mailing list.

Stefan


From dreamingforward at gmail.com  Fri Nov 14 23:07:04 2008
From: dreamingforward at gmail.com (average)
Date: Fri, 14 Nov 2008 15:07:04 -0700
Subject: [Python-Dev] compiler optimizations: collecting ideas
Message-ID: <913f9f570811141407j6a8b5bdat67be0ac0aa14c20a@mail.gmail.com>

"Daniel Furrer" <daniel.furrer at gmail.com> wrote:
> As part of an advanced compiler design course at our university (ETH
> Zurich), we have to implement an optimization (or various thereof).
> I've spoken with a couple of people who are, like me, very fascinated by
> Python.
> So I would just like to ask if somebody has any ideas on what we could do?
>
> - We've been working on generating Control Flow Graphs, generating Static
> Single Assignment Form (inserting phi-nodes) and removing it again. We've
> also implemented techniques like constant folding, copy propagation, Common
> Subexpression Elimination etc.

How about a graphical program-visualization meta-tool that would allow
making optimizations (as well as debugging) easier.  Take your control
flow graph data put it into fractal or recursive/foldable graphical
"cloud" and visualize the running program therein.  Find interesting
audio and visual constructs (color, intensity, etc) to isomorphically
map interesting programming data into non-arbitrary visual iconography
and design.  Each entrance into a code-block is like a photon entering
a molecular form, and as more photons enter a given block in a given
time frame, heat is generated in that code block, causing it to glow
for an amount of time.  Code that runs for a long time, glows as it is
run.  If it returns a value all that warmth in the code block gets
turned into a flash of light and the code-block goes dark again.  The
description is pretty sloppy, but I know there's a killer app there
somewhere.  I've been wanting such a tool for awhile.  (Imagine using
such a tool with a powerful abstract language like LISP.  It would
remove all the obtuse parenthetical list expressions and put it into a
graphical visual language where nodes consisting of subgraphs could be
moved around as needed.  Language syntax would become more-or-less
obsolete!)

Go even further and imagine the internet in such a cloud, where users
can navigate by looking at "heat" activity therein.  Each time a user
visits and "looks" at a given site eventually condenses into "water"
and you have pools of interest....

Please tell me this is a cool idea or not so I don't wonder why no one
else is thinking about such things.  And on your way, save the
world...  Oh, and, if it is a cool idea, is there any company
interested in hiring?  haha

marcos

From victor.stinner at haypocalc.com  Fri Nov 14 23:59:03 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 14 Nov 2008 23:59:03 +0100
Subject: [Python-Dev] datetime and timedelta enhancement
Message-ID: <200811142359.03340.victor.stinner@haypocalc.com>

Hi,

There are some interresting tickets about the datetime module:
#1673409: datetime module missing some important methods
#1083: Confusing error message when dividing timedelta using /
#2706: datetime: define division timedelta/timedelta
#4291: Allow Division of datetime.timedelta Objects

Wanted features:
1- convert a datetime object to an epoch value (numbers of seconds since 
   the 1st january 1970), eg. with a new totimestamp() method
2- convert a timedelta to a specific unit (eg. seconds, days, weeks, etc.)
3- compute the ratio of two timedelta, eg. for a progress bar

--

About (1), no patch is available yet. Some points are not fixed yet:
 a- use an integer to avoid floating point problems?
 b- use a float to keep the microseconds?
 c- limit the datetime to the time_t type (32 bits signed integer)?
    Which mean the following range:

>>> from datetime import datetime, timedelta
>>> t0 = datetime(1970, 1, 1, 0, 0, 0)
>>> print t0 + timedelta(seconds=-2**31), "-", t0 + timedelta(seconds=2**31-1)
1901-12-13 20:45:52 - 2038-01-19 03:14:07

The following point may answer to questions (a) and (b)...

--

For the points (2) and (3): webograph and belopolsky proposed to use the 
division of two timedelta objects. Examples (using the patch attached to 
#2706):

>>> from __future__ import division
>>> from datetime import datetime, timedelta
>>> one_second = timedelta(seconds=1)
>>> print timedelta(hours=1, minutes=1, microseconds=3) / one_second
3660.000003
>>> print timedelta(hours=1, minutes=1, microseconds=3) // one_second
3660
>>> print divmod(timedelta(hours=1, minutes=1, microseconds=3), one_second)
(3660L, datetime.timedelta(0, 0, 3))

We have the float representation, the integer part of the division, and the 
integer part + the remainder as a timedelta object. You have the choice to 
use the best format for your needs :)

Use another denominator to get other units, eg.
   weeks = dt / timedelta(days=7)

Then come the question of the accepted types:
   timedelta / float?
   divmod(timedelta, int)?
   etc.

--

About the conversion of a datetime object to time_t, the division can be 
reused like that:

>>> epoch = datetime.fromtimestamp(0)
>>> print epoch
1970-01-01 01:00:00
>>> d = datetime.fromtimestamp(34930.5)
>>> (d - epoch) / timedelta(seconds=1)   # integer
34930.5
>>> (d - epoch) // timedelta(seconds=1)  # float
34930L

datetime.totimestamp() can be implemented to produce a float in range [-2**31; 
2**31-1]. Why a float? To be symetric with fromtimestamp()! If you need 
older/newer timestamp or you want integers, use the examples with the 
divison.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Sat Nov 15 01:41:00 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 15 Nov 2008 01:41:00 +0100
Subject: [Python-Dev] datetime and timedelta enhancement
In-Reply-To: <200811142359.03340.victor.stinner@haypocalc.com>
References: <200811142359.03340.victor.stinner@haypocalc.com>
Message-ID: <200811150141.00652.victor.stinner@haypocalc.com>

> datetime.totimestamp() can be implemented to produce a float in range
> [-2**31; 2**31-1]

An implementation of this method is proposed as a patch in issue #2736.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Sat Nov 15 02:01:42 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 15 Nov 2008 02:01:42 +0100
Subject: [Python-Dev] datetime and timedelta enhancement
In-Reply-To: <200811142359.03340.victor.stinner@haypocalc.com>
References: <200811142359.03340.victor.stinner@haypocalc.com>
Message-ID: <200811150201.42271.victor.stinner@haypocalc.com>

> 1- convert a datetime object to an epoch value (numbers of seconds since
>    the 1st january 1970), eg. with a new totimestamp() method
> 2- convert a timedelta to a specific unit (eg. seconds, days, weeks, etc.)

Another solution is proposed by Christian Heimes: "implement __int__
and __float__ on timedelta objects: int(timedelta) -> seconds,
float(timedelta) -> seconds.micros".

This solution only answer to the need "conversion to seconds" (which is the 
most common case). To get week, you would have to write:
   int(dt) / (3600*24*7)
which feels me less natural (pythonic?) than:
   dt // timedelta(days=7)

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From victor.stinner at haypocalc.com  Sat Nov 15 02:10:32 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Sat, 15 Nov 2008 02:10:32 +0100
Subject: [Python-Dev] datetime and timedelta enhancement
In-Reply-To: <200811150201.42271.victor.stinner@haypocalc.com>
References: <200811142359.03340.victor.stinner@haypocalc.com>
	<200811150201.42271.victor.stinner@haypocalc.com>
Message-ID: <200811150210.32656.victor.stinner@haypocalc.com>

Le Saturday 15 November 2008 02:01:42 Victor Stinner, vous avez ?crit?:
> > 1- convert a datetime object to an epoch value (numbers of seconds since
> >    the 1st january 1970), eg. with a new totimestamp() method
> > 2- convert a timedelta to a specific unit (eg. seconds, days, weeks,
> > etc.)
>
> Another solution is proposed by Christian Heimes: "implement __int__
> and __float__ on timedelta objects (...)

The solution was implemented as patch in:
   http://bugs.python.org/issue1665292

The issue is closed and there was a problem with hash():
   timedelta(0, 1) == 1   
but 
   hash(timedelta(0, 1)) != hash(1)

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From brett at python.org  Sat Nov 15 08:35:43 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 14 Nov 2008 23:35:43 -0800
Subject: [Python-Dev] Getting Set Up doc now online
Message-ID: <bbaeab100811142335s44bc610dn6abc5be8b51ca2db@mail.gmail.com>

The Setup doc I sent to the list a little while back has now been put
online at http://www.python.org/dev/setup/ . I also fleshed out the
dev FAQ so that technical details are all in there while the Setup doc
just outlines basically key points.

-Brett

From mal at egenix.com  Sat Nov 15 14:24:57 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 15 Nov 2008 14:24:57 +0100
Subject: [Python-Dev] datetime and timedelta enhancement
In-Reply-To: <200811142359.03340.victor.stinner@haypocalc.com>
References: <200811142359.03340.victor.stinner@haypocalc.com>
Message-ID: <491ECDA9.1050705@egenix.com>

On 2008-11-14 23:59, Victor Stinner wrote:
> Hi,
> 
> There are some interresting tickets about the datetime module:
> #1673409: datetime module missing some important methods
> #1083: Confusing error message when dividing timedelta using /
> #2706: datetime: define division timedelta/timedelta
> #4291: Allow Division of datetime.timedelta Objects
> 
> Wanted features:
> 1- convert a datetime object to an epoch value (numbers of seconds since 
>    the 1st january 1970), eg. with a new totimestamp() method
> 2- convert a timedelta to a specific unit (eg. seconds, days, weeks, etc.)
> 3- compute the ratio of two timedelta, eg. for a progress bar

Since the datetime module turned out to be mostly a reimplementation
of mxDateTime, why not continue down that road ?

    http://www.egenix.com/products/python/mxBase/mxDateTime/

Let's see:

>>> from mx.DateTime import *

>>> DateTime(2008,11,15).ticks()
1226703600.0

>>> TimeDelta(seconds=100)
<mx.DateTime.DateTimeDelta object for '00:01:40.00' at 837030>

>>> TimeDelta(seconds=100) / TimeDelta(seconds=50)
2.0

>>> TimeDelta(seconds=100).seconds
100.0

>>> TimeDelta(seconds=100).days
0.0011574074074074073

>>> TimeDelta(seconds=100).weeks
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: weeks

Guess I could add a .weeks attribute to mxDateTime, but no one ever
asked for that so far.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 15 2008)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-11-12: Released mxODBC.Connect 0.9.3      http://python.egenix.com/

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611

From greg at krypto.org  Sun Nov 16 00:17:45 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 15 Nov 2008 15:17:45 -0800
Subject: [Python-Dev] Optionally using GMP to implement long if available
In-Reply-To: <1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
References: <1225734997.14412.5.camel@paul-laptop.wmich.edu>
	<200811032141.34063.victor.stinner@haypocalc.com>
	<52dc1c820811031426i55ecf53dt4a1078e7e490fe26@mail.gmail.com>
	<490F8744.8010302@v.loewis.de>
	<1f7befae0811031837g1eb5fd45y4027b40d221cc53e@mail.gmail.com>
	<5c6f2a5d0811040300j4cd7b659gadddc0f113ee27ac@mail.gmail.com>
	<1f7befae0811041033w8520a7n51ad693f23376871@mail.gmail.com>
Message-ID: <52dc1c820811151517y68728179if4d64db5cd1a1935@mail.gmail.com>

On Tue, Nov 4, 2008 at 10:33 AM, Tim Peters <tim.peters at gmail.com> wrote:
>
> 2. On platforms that support it, this is at least 64x64->64 multiplication,
>   potentially much more expensive than the 32x32->64 (or 31x31->62?)
>   flavor you /intend/ to move to.

Thats a good point, thanks!

I am not averse to including a tiny bit of platform (i386) specific
inline asm (in its own header file as a macro to make it easy to
maintain and easy to turn off and easy to add versions for someone
elses favorite 32bit platform) to get that when compiled to use 30bit
digits since the C language has no way to express it directly.

-gps

From ncoghlan at gmail.com  Sun Nov 16 02:14:12 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 16 Nov 2008 11:14:12 +1000
Subject: [Python-Dev] datetime and timedelta enhancement
In-Reply-To: <491ECDA9.1050705@egenix.com>
References: <200811142359.03340.victor.stinner@haypocalc.com>
	<491ECDA9.1050705@egenix.com>
Message-ID: <491F73E4.7090100@gmail.com>

M.-A. Lemburg wrote:
> Guess I could add a .weeks attribute to mxDateTime, but no one ever
> asked for that so far.

Given that there are at least 3 different ways to define the number of
"weeks" between two dates, it may be something best left to applications
to worry about.

OOo implements 2 of them [1] for its WEEKS() function, and there's then
a fairly obvious 3rd variant based on a Sunday to Saturday week.

Cheers,
Nick.

[1]
http://wiki.services.openoffice.org/wiki/Documentation/How_Tos/Calc:_WEEKS_function



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ironfroggy at gmail.com  Sun Nov 16 07:39:11 2008
From: ironfroggy at gmail.com (Calvin Spealman)
Date: Sun, 16 Nov 2008 01:39:11 -0500
Subject: [Python-Dev] Summaries for Number 2008, First Half
Message-ID: <76fd5acf0811152239w508903a6y5472b5507a329b8@mail.gmail.com>

Wow, how long has this been? OK, so these aren't really the summaries.
It is half-done summaries, which I've posted to the wiki for community
collaboration. Anyone with a better understanding a conversation or
simply the knowledge that I don't know what I'm talking about half the
time, the summary can be edited easily now. I'd like to try a schedule
of posting the draft to the wiki, leaving it up for edits for two
weeks, and publishing the final summaries when the next summary draft
is prepared and put up for review. Does that sound like a decent plan
that might actually see the return of the summaries be stuck with?

The draft is at
http://wiki.python.org/moin/PyDevSummaries/2008/November/FirstHalf

Of course, if you don't have time to edit this but do want to make any
comments, suggestions, or corrections, or you want to suggest that any
thread should be promoted from the bottom list or demoted down to it,
just let me know. And the current draft is found below.

=============
Announcements
=============

This is the draft of the python-dev mailing list summaries for the
first half of November 2008. Please make any corrections,
improvements, or comments. The contents will be finalized at the end
of the month, when the summaries' draft for the second half of
November 2008 are posted. I hope to keep this schedule regularly.

=========
Summaries
=========

-------------------------------
Looking for VCS usage scenarios
-------------------------------

Brett Cannon solicits any use cases and scenarios that would give an
understanding for the motivations in proposing Python move to or
mirror officially via some distributed version control. Much
discussion ensued that I didn't have the time to follow. This thread
was over 100 posts, so anyone who followed it well, please give as
concise an overview as you can.

Contributing thread:

- `Looking for VCS usage scenarios
<http://mail.python.org/pipermail/python-dev/2008-November/083272.html>`__

---------------------------------------------------
Optionally using GMP to implement long if available
---------------------------------------------------

The suggestion of using the GMP library to implement Python's long
numbers was brought in the context of an compile time option, falling
back to the current in-house implementation when GMP is not available.
However, the discussion points out that most distribution of Python is
binary and so the LGPL issues that keep us from using GMP still apply.

Some of the conversation also suggestion various improvements to our
own implementation, mostly focused on increasing size of the base long
values are stored in, particularly on 64bit systems.

Contributing thread:

- `Optionally using GMP to implement long if available
<http://mail.python.org/pipermail/python-dev/2008-November/083315.html>`__

------------------------------------------------
Fwd: Removal of GIL through refcounting removal.
------------------------------------------------

Yet Another Attempt At Removing The GIL was proposed, and had a more
positive reception than most of the times it has been brought up (in
my memory). While the removal of the GIL has been both proposed and
successfully attempted in the past, there have always been outstanding
issues that keep the lock around. One of the primary issues is that of
the C API and extension modules, which all expect reference counting.
A technique used in IronClad, a project that provides use of CPython
extensions in the IronPython implementation of the language, was
suggested to provide reference counting semantics to legacy
extensions, even if the internal objects no longer require them.

Contributing thread:

- `Fwd: Removal of GIL through refcounting removal.
<http://mail.python.org/pipermail/python-dev/2008-November/083256.html>`__

----------------------------------
Using Cython for standard library?
----------------------------------

Without much expectation that any action will actually be taken any
time soon, the possibility of using Cython (and, by extension, similar
solutions) was given consideration. While the benefits of these
solutions are obvious, there is no particular best of breed nor yet a
point of stability that a mature project might stand on. It would be
exciting to one day see some variation of this happen, but that day is
not today.

Contributing thread:

- `Using Cython for standard library?
<http://mail.python.org/pipermail/python-dev/2008-November/083284.html>`__

------------------------------
n.numbits: method or property?
------------------------------

The basic question of a proposed method or property, numbits, drew
into a small discussion about the inherent assumption of binary
representation of our numbers. The numbits method or property would
provide the smallest number of bits required to represent a given
integer.

Contributing thread:

- `n.numbits: method or property?
<http://mail.python.org/pipermail/python-dev/2008-November/083528.html>`__

----------------------------
XXX do we need a new policy?
----------------------------

Some concerns were raised for the number of XXX, TODO, and other types
of comment markers in the Python source. While the OP appears to worry
about the number of open issues this represents, the general concensus
seems to conclude that any real issues have associated tickets, there
is little benefit in dealing with all of these at once, and the 2000
or so of them represents nothing significant, in the first place.

Contributing thread:

- `XXX do we need a new policy?
<http://mail.python.org/pipermail/python-dev/2008-November/083342.html>`__

---------------
DVCS PEP update
---------------

It was announced that git is being included as one of the distributed
version control solutions in consideration for any proposal to move
from Subversion. Some back and forth on the various questions against
each option and who is representing each was also given.

Contributing thread:

- `DVCS PEP update
<http://mail.python.org/pipermail/python-dev/2008-November/083412.html>`__

------------------------------------------------
Released fixes for CVE-2008-2315 for Python 2.4?
------------------------------------------------

It was confirmed that an additional bugfix release of 2.4 will be
made, 2.4.6 in a simultaneous release with 2.5.3. However, this
accepts only security fixes so the multiprocessing module backport is
not eligable.

Contributing thread:

- `Released fixes for CVE-2008-2315 for Python 2.4?
<http://mail.python.org/pipermail/python-dev/2008-November/083484.html>`__

---------------
hg branch gone?
---------------

Contributing thread:

- `hg branch gone?
<http://mail.python.org/pipermail/python-dev/2008-November/083292.html>`__

-----------------------------
Optimize Python long integers
-----------------------------

Several outstanding patches to optimize int and especially long
objects were benchmarked and applied to the 2.6 branch for comparison.
Although they are not eligable for the bugfix releases of 2.6 or 3.0,
they may be likely improvements we'll see in 2.7 and 3.1 releases in
the future. However, some related and minor bug fixes to long are
likely to be applied much sooner.

Contributing thread:

- `Optimize Python long integers
<http://mail.python.org/pipermail/python-dev/2008-November/083514.html>`__

-------------------------------
file open in python interpreter
-------------------------------

Contributing thread:

- `file open in python interpreter
<http://mail.python.org/pipermail/python-dev/2008-November/083279.html>`__

----------
My patches
----------

Contributing thread:

- `My patches <http://mail.python.org/pipermail/python-dev/2008-November/083255.html>`__

----------------------------------------
compiler optimizations: collecting ideas
----------------------------------------

Contributing thread:

- `compiler optimizations: collecting ideas
<http://mail.python.org/pipermail/python-dev/2008-November/083557.html>`__

------------------------------
Python 2.5.3: call for patches
------------------------------

Contributing thread:

- `Python 2.5.3: call for patches
<http://mail.python.org/pipermail/python-dev/2008-November/083517.html>`__

----------------------------------
datetime and timedelta enhancement
----------------------------------

Contributing thread:

- `datetime and timedelta enhancement
<http://mail.python.org/pipermail/python-dev/2008-November/083564.html>`__



================
Deferred Threads
================

==================
Previous Summaries
==================

===============
Skipped Threads
===============

- `Why don't range and xrange threat floats as floats?
<http://mail.python.org/pipermail/python-dev/2008-November/083429.html>`__
- `[Python-3000] RELEASED Python 3.0rc2
<http://mail.python.org/pipermail/python-dev/2008-November/083483.html>`__
- `Python 3.0rc2: problem with exec()ing files
<http://mail.python.org/pipermail/python-dev/2008-November/083513.html>`__
- `Feedback from numerical/math community on PEP 225
<http://mail.python.org/pipermail/python-dev/2008-November/083493.html>`__
- `buffer function
<http://mail.python.org/pipermail/python-dev/2008-November/083262.html>`__
- `How to select text of text field in python&#8207;Card
<http://mail.python.org/pipermail/python-dev/2008-November/083277.html>`__
- `RELEASED Python 3.0rc2
<http://mail.python.org/pipermail/python-dev/2008-November/083481.html>`__
- `Summary of Python tracker Issues
<http://mail.python.org/pipermail/python-dev/2008-November/083485.html>`__
- `Getting Set Up dev doc
<http://mail.python.org/pipermail/python-dev/2008-November/083504.html>`__
- `A statistic for Python tickets
<http://mail.python.org/pipermail/python-dev/2008-November/083534.html>`__
- `How does one build Python25.chm on Windows?
<http://mail.python.org/pipermail/python-dev/2008-November/083549.html>`__
- `Upgrade SVN server to 1.5.4
<http://mail.python.org/pipermail/python-dev/2008-November/083551.html>`__
- `[ANN] VPython 0.1
<http://mail.python.org/pipermail/python-dev/2008-November/083257.html>`__
- `Packaging the PyPI version of the SSL module for Debian
<http://mail.python.org/pipermail/python-dev/2008-November/083291.html>`__
- `New "stage" field in the tracker
<http://mail.python.org/pipermail/python-dev/2008-November/083331.html>`__
- `Python2.5 _sre deepcopy regression?
<http://mail.python.org/pipermail/python-dev/2008-November/083348.html>`__
- `test - please ignore
<http://mail.python.org/pipermail/python-dev/2008-November/083418.html>`__
- `AST-level type inference optimizations
<http://mail.python.org/pipermail/python-dev/2008-November/083438.html>`__
- `test message - spam work...
<http://mail.python.org/pipermail/python-dev/2008-November/083556.html>`__
- `Getting Set Up doc now online
<http://mail.python.org/pipermail/python-dev/2008-November/083568.html>`__

-- 
Read my blog! I depend on your acceptance of my opinion! I am interesting!
http://techblog.ironfroggy.com/
Follow me if you're into that sort of thing: http://www.twitter.com/ironfroggy

From taleinat at gmail.com  Sun Nov 16 14:00:58 2008
From: taleinat at gmail.com (Tal Einat)
Date: Sun, 16 Nov 2008 15:00:58 +0200
Subject: [Python-Dev] datetime.date.today() raises "AttributeError: time"
Message-ID: <7afdee2f0811160500g44421c26o64765d2acf91a712@mail.gmail.com>

For an unknown reason, datetime.date.today() began throwing a cryptic
"AttributeError: time" exception. It took me a while to figure out
that this was caused by an accidental overriding of the built-in
'time' module.

Here's an example interactive session which shows the problem:

[tal ~]$ touch time.py
[tal ~]$ python
Python 2.5.2 (r252:60911, Jul 17 2008, 10:47:50)
[GCC 4.0.1 (Apple Inc. build 5484)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import time
>>> time.__file__
'time.py'
>>> import datetime
>>> datetime.date.today()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: time
>>>

Here I used version 2.5.2, but I checked and this also happens on 2.6.


It this desired behavior?

At the very least the exception should be more detailed, perhaps to
the point of suggesting the probable cause of the error (i.e.
overriding the time module).

- Tal

From steve at holdenweb.com  Sun Nov 16 14:21:18 2008
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 16 Nov 2008 08:21:18 -0500
Subject: [Python-Dev] datetime.date.today() raises "AttributeError: time"
In-Reply-To: <7afdee2f0811160500g44421c26o64765d2acf91a712@mail.gmail.com>
References: <7afdee2f0811160500g44421c26o64765d2acf91a712@mail.gmail.com>
Message-ID: <gfp6od$il5$1@ger.gmane.org>

Tal Einat wrote:
> For an unknown reason, datetime.date.today() began throwing a cryptic
> "AttributeError: time" exception. It took me a while to figure out
> that this was caused by an accidental overriding of the built-in
> 'time' module.
> 
> Here's an example interactive session which shows the problem:
> 
> [tal ~]$ touch time.py
> [tal ~]$ python
> Python 2.5.2 (r252:60911, Jul 17 2008, 10:47:50)
> [GCC 4.0.1 (Apple Inc. build 5484)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import time
>>>> time.__file__
> 'time.py'
>>>> import datetime
>>>> datetime.date.today()
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> AttributeError: time
> 
> Here I used version 2.5.2, but I checked and this also happens on 2.6.
> 
> 
> It this desired behavior?
> 
> At the very least the exception should be more detailed, perhaps to
> the point of suggesting the probable cause of the error (i.e.
> overriding the time module).
> 
How is this different from any other case where you import a module with
a standard library name conflict, thereby confusing modules loaded later
standard library. Should we do the same for any error induced in such a way?

regards
 Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From taleinat at gmail.com  Sun Nov 16 14:55:25 2008
From: taleinat at gmail.com (Tal Einat)
Date: Sun, 16 Nov 2008 15:55:25 +0200
Subject: [Python-Dev] datetime.date.today() raises "AttributeError: time"
In-Reply-To: <7afdee2f0811160500g44421c26o64765d2acf91a712@mail.gmail.com>
References: <7afdee2f0811160500g44421c26o64765d2acf91a712@mail.gmail.com>
Message-ID: <7afdee2f0811160555y3cb71afp460e267c29a96827@mail.gmail.com>

Steve Holden wrote:
> Tal Einat wrote:
>> It this desired behavior?
>>
>> At the very least the exception should be more detailed, perhaps to
>> the point of suggesting the probable cause of the error (i.e.
>> overriding the time module).
>>
> How is this different from any other case where you import a module with
> a standard library name conflict, thereby confusing modules loaded later
> standard library. Should we do the same for any error induced in such a way?

The difference is that here the exception is generated directly in the
C code so you don't get an intelligible traceback. The C code for
datetime imports the time module via the Python C API.

In other words, here a function from a module in the stdlib, datetime,
barfs unexpectedly because I happen to have a file name time.py
hanging around in some directory. There is no traceback and no
intelligible exception message, just "AttributeError: time". I had to
dig through datetime's C code to figure out which module was being
imported via the Python C API, which turned out to be time.

 This is rare enough that I've never had something like this happen to
me in seven years of heavy Python programming.

- Tal

From aahz at pythoncraft.com  Sun Nov 16 16:19:49 2008
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 16 Nov 2008 07:19:49 -0800
Subject: [Python-Dev] Summaries for Number 2008, First Half
In-Reply-To: <76fd5acf0811152239w508903a6y5472b5507a329b8@mail.gmail.com>
References: <76fd5acf0811152239w508903a6y5472b5507a329b8@mail.gmail.com>
Message-ID: <20081116151949.GA27887@panix.com>

Thanks for getting back to this!

On Sun, Nov 16, 2008, Calvin Spealman wrote:
>
> ---------------------------------------------------
> Optionally using GMP to implement long if available
> ---------------------------------------------------

I'd combine the long optimization thread here -- I don't see any reason
for separate thread summaries.  Or at least put them next to each other.

> ---------------
> hg branch gone?
> ---------------

I'd skip this thread

> ----------
> My patches
> ----------

The summary should use clear titles, especially if they don't have an
actual summary for a thread.  I'd title this "How do I get my patches
accepted?" or "Can someone please look at my patches?"
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

From ggpolo at gmail.com  Sun Nov 16 17:43:54 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Sun, 16 Nov 2008 14:43:54 -0200
Subject: [Python-Dev] datetime.date.today() raises "AttributeError: time"
In-Reply-To: <7afdee2f0811160555y3cb71afp460e267c29a96827@mail.gmail.com>
References: <7afdee2f0811160500g44421c26o64765d2acf91a712@mail.gmail.com>
	<7afdee2f0811160555y3cb71afp460e267c29a96827@mail.gmail.com>
Message-ID: <ac2200130811160843y2292fad4he6cf78bbb696d4d@mail.gmail.com>

On Sun, Nov 16, 2008 at 11:55 AM, Tal Einat <taleinat at gmail.com> wrote:
> Steve Holden wrote:
>> Tal Einat wrote:
>>> It this desired behavior?
>>>
>>> At the very least the exception should be more detailed, perhaps to
>>> the point of suggesting the probable cause of the error (i.e.
>>> overriding the time module).
>>>
>> How is this different from any other case where you import a module with
>> a standard library name conflict, thereby confusing modules loaded later
>> standard library. Should we do the same for any error induced in such a way?
>
> The difference is that here the exception is generated directly in the
> C code so you don't get an intelligible traceback. The C code for
> datetime imports the time module via the Python C API.
>
> In other words, here a function from a module in the stdlib, datetime,
> barfs unexpectedly because I happen to have a file name time.py
> hanging around in some directory. There is no traceback and no
> intelligible exception message, just "AttributeError: time". I had to
> dig through datetime's C code to figure out which module was being
> imported via the Python C API, which turned out to be time.

Just like Steve told you, this isn't different from other cases. But,
at least you get a message a bit more verbose in most cases, like:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'module' object has no attribute 'time'

Then I went to look why this wasn't happening with datetime too, and I
found out that PyObject_CallMethod in abstract.c re sets the exception
message that would have been set by PyObject_GetAttr by now. Maybe
someone can tell me why it is doing that, for now a patch is attached
here (I didn't resist to not remove two trailing whitespaces).

>
>  This is rare enough that I've never had something like this happen to
> me in seven years of heavy Python programming.
>
> - Tal
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves
-------------- next part --------------
A non-text attachment was scrubbed...
Name: abstract_dont_re_set_except_msg.diff
Type: text/x-patch
Size: 981 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081116/dfd3abd4/attachment.bin>

From ginovation at gmail.com  Mon Nov 17 08:17:11 2008
From: ginovation at gmail.com (ganesh gajre)
Date: Mon, 17 Nov 2008 12:47:11 +0530
Subject: [Python-Dev] How to read fonts in python
Message-ID: <387483930811162317w54a667c2ub377eb5bfdf17f6@mail.gmail.com>

Hello to all,
I am writing a program in python to convert Indic true type fonts in
Unicode. I like to know is there any way to read the fonts and on the basis
of that i can use map file to convert the font in unicode.

Please help me.

regards,
Ginovation
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081117/31dc090f/attachment-0001.htm>

From amauryfa at gmail.com  Mon Nov 17 10:18:31 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 17 Nov 2008 10:18:31 +0100
Subject: [Python-Dev] How to read fonts in python
In-Reply-To: <387483930811162317w54a667c2ub377eb5bfdf17f6@mail.gmail.com>
References: <387483930811162317w54a667c2ub377eb5bfdf17f6@mail.gmail.com>
Message-ID: <e27efe130811170118y1d5db0ach3c2a1ffb0938bdac@mail.gmail.com>

Hello,

On Mon, Nov 17, 2008 at 08:17, ganesh gajre <ginovation at gmail.com> wrote:
> Hello to all,
> I am writing a program in python to convert Indic true type fonts in
> Unicode. I like to know is there any way to read the fonts and on the basis
> of that i can use map file to convert the font in unicode.
>
> Please help me.

The python-dev mailing list is for the development *of* python, not
for development *with* python.

Please direct your question to the comp.lang.python newsgroup (or the
python-list at python dot org mailing list)
I am sure you will quickly get a response there.

-- 
Amaury Forgeot d'Arc

From ncoghlan at gmail.com  Mon Nov 17 10:19:52 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 17 Nov 2008 19:19:52 +1000
Subject: [Python-Dev] How to read fonts in python
In-Reply-To: <387483930811162317w54a667c2ub377eb5bfdf17f6@mail.gmail.com>
References: <387483930811162317w54a667c2ub377eb5bfdf17f6@mail.gmail.com>
Message-ID: <49213738.9060008@gmail.com>

ganesh gajre wrote:
> Hello to all,
> I am writing a program in python to convert Indic true type fonts in
> Unicode. I like to know is there any way to read the fonts and on the
> basis of that i can use map file to convert the font in unicode.

The python-dev list is for the development *of* Python, not development
*with* Python.

This question would be better directed to the comp.lang.python newsgroup
(also available as python-list at python.org)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ginovation at gmail.com  Mon Nov 17 11:01:27 2008
From: ginovation at gmail.com (ganesh gajre)
Date: Mon, 17 Nov 2008 15:31:27 +0530
Subject: [Python-Dev] How to read fonts in python
In-Reply-To: <49213738.9060008@gmail.com>
References: <387483930811162317w54a667c2ub377eb5bfdf17f6@mail.gmail.com>
	<49213738.9060008@gmail.com>
Message-ID: <387483930811170201y3d49dd65ie28e1cd1b55ed9d4@mail.gmail.com>

Hello,
Thanks for guiding me.

Ginovation

On Mon, Nov 17, 2008 at 2:49 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> ganesh gajre wrote:
> > Hello to all,
> > I am writing a program in python to convert Indic true type fonts in
> > Unicode. I like to know is there any way to read the fonts and on the
> > basis of that i can use map file to convert the font in unicode.
>
> The python-dev list is for the development *of* Python, not development
> *with* Python.
>
> This question would be better directed to the comp.lang.python newsgroup
> (also available as python-list at python.org)
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081117/3179ccdd/attachment.htm>

From mal at egenix.com  Mon Nov 17 12:37:51 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Mon, 17 Nov 2008 12:37:51 +0100
Subject: [Python-Dev] datetime and timedelta enhancement
In-Reply-To: <491F73E4.7090100@gmail.com>
References: <200811142359.03340.victor.stinner@haypocalc.com>	<491ECDA9.1050705@egenix.com>
	<491F73E4.7090100@gmail.com>
Message-ID: <4921578F.9030707@egenix.com>

On 2008-11-16 02:14, Nick Coghlan wrote:
> M.-A. Lemburg wrote:
>> Guess I could add a .weeks attribute to mxDateTime, but no one ever
>> asked for that so far.
> 
> Given that there are at least 3 different ways to define the number of
> "weeks" between two dates, it may be something best left to applications
> to worry about.

I'd just use the term "weeks" as meaning 7 full days and then
return a float value for fractions. That's the same convention used
for .days. Anything more complicated would need to use DateTime
values (see below).

> OOo implements 2 of them [1] for its WEEKS() function, and there's then
> a fairly obvious 3rd variant based on a Sunday to Saturday week.
> 
> Cheers,
> Nick.
> 
> [1]
> http://wiki.services.openoffice.org/wiki/Documentation/How_Tos/Calc:_WEEKS_function

If you need ISO week counting or any other date based counting
mechanism, you need to know the two DateTime values you're dealing
with and possibly the calendar you're using.

mxDateTime has an .iso_week attribute to help with this, e.g.

>>> Date(2008,11,17).iso_week
(2008, 47, 1)

http://en.wikipedia.org/wiki/ISO_week

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 17 2008)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-11-12: Released mxODBC.Connect 0.9.3      http://python.egenix.com/

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611

From barry at python.org  Tue Nov 18 02:56:11 2008
From: barry at python.org (Barry Warsaw)
Date: Mon, 17 Nov 2008 20:56:11 -0500
Subject: [Python-Dev] 2.6.1 and 3.0
Message-ID: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Martin suggests, and I agree, that we should release Python 3.0 final  
and 2.6.1 at the same time.  Makes sense to me.  That would mean that  
Python 2.6.1 should be ready on 03-Dec (well, if Python 3.0 is ready  
then!).

I'm still planning the last Python 3.0 release candidate for this  
Wednesday.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSIgu3EjvBPtnXfVAQKzGgP/XH2szIZdG9nvZTI2M9iWXuz/tBwH6ncd
Kv70ATpttQEg/bmuRp5nSmg1p7hxSmTqu9waq4qdc07IPa+ofTngbunUKkTrbZoo
E/r72dGw29pou7B6NVh/g5Db8Tl0yNJBd6vmpEUbCvUDBpljqgxCdj0uw/RiDluj
5Ek2biim7ww=
=twyi
-----END PGP SIGNATURE-----

From facundobatista at gmail.com  Tue Nov 18 11:03:02 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 18 Nov 2008 08:03:02 -0200
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
Message-ID: <e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>

2008/11/17 Barry Warsaw <barry at python.org>:

> Martin suggests, and I agree, that we should release Python 3.0 final and
> 2.6.1 at the same time.  Makes sense to me.  That would mean that Python
> 2.6.1 should be ready on 03-Dec (well, if Python 3.0 is ready then!).

2.6.1 only two months after 2.6? Why so quickly?

Anyway, I don't see any added value in the synchronization, so taking
in consideration all the effort you're putting in these releases, I
would just want to minimize your workload... which is easier to you?
doing both at the same time or not?

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From ncoghlan at gmail.com  Tue Nov 18 11:55:50 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 18 Nov 2008 20:55:50 +1000
Subject: [Python-Dev] Issue 4195: Can't execute packages with -m in Python
	2.6/3.0
Message-ID: <49229F36.5070206@gmail.com>

I need a release manager call on whether or not the proposed resolution
of this issue goes beyond what is acceptable for a bug fix in 2.6.1.

Short version:
- Python 2.5 allowed packages to be executed with -m, but in a broken way
- I disabled the broken way for 2.6, but didn't provide a replacement
- The patch attached to 4195 once again allows execution of packages
with -m, but in a clean way similar to the way directories and zipfiles
can now be executed
- I would like to commit that patch for 3.0/2.6.1, but I'm concerned
that it crosses the "no new features" line

Long version:

There was a bug in python 2.5 that allowed a package name to be passed
to the -m switch or runpy.run_module() and it would mostly work.

However, the 'mostly' was due to the fact that doing this put the
internal import machinery into a slightly inconsistent state: the
interpreter was running code from inside a package, but that package
wasn't actually present in sys.modules. This could lead to assorted hard
to trace import-related weirdness, similar to what you can get when
executing a file from inside a package by name. You would often get away
with it, but good luck figuring out what is happening if things go wrong
(and you aren't already an expert on Python import mechanics).

Since the ability to execute packages wasn't intentional, I added the
missing check to block it explicitly in 2.6 (but left it alone for 2.5).
It seemed like a really niche feature, so I didn't figure out a clean
replacement for it at the time.

Benjamin noticed this change earlier in the 2.6 release cycle, I pointed
out it was a deliberate change, and that's the way it stayed until after
2.6 was released.

After the release, Andi Vajda (from the JCC project [1]) contacted me,
and together we worked out a better implementation of package support
for the -m switch (and runpy.run_module) [2] by extending the
__main__.py approach used to support direct execution of zipfiles and
directories [3].

That's what I would like to add, since it nicely complements the ability
to execute zipfiles and directories, while also restoring the ability to
pass a package name to the -m switch (but in a way that keeps the import
machinery in a consistent state).

Cheers,
Nick.

[1] http://pypi.python.org/pypi/JCC

[2] http://bugs.python.org/issue4195 (package execution with -m)
[3] http://bugs.python.org/issue1739468 (zipfile execution)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From ncoghlan at gmail.com  Tue Nov 18 11:57:18 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 18 Nov 2008 20:57:18 +1000
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
Message-ID: <49229F8E.7080800@gmail.com>

Facundo Batista wrote:
> 2008/11/17 Barry Warsaw <barry at python.org>:
> 
>> Martin suggests, and I agree, that we should release Python 3.0 final and
>> 2.6.1 at the same time.  Makes sense to me.  That would mean that Python
>> 2.6.1 should be ready on 03-Dec (well, if Python 3.0 is ready then!).
> 
> 2.6.1 only two months after 2.6? Why so quickly?
> 
> Anyway, I don't see any added value in the synchronization, so taking
> in consideration all the effort you're putting in these releases, I
> would just want to minimize your workload... which is easier to you?
> doing both at the same time or not?

There have been several corrections made to the 2to3 conversion tool -
it would be good to get those in developer's hands at the same time that
3.0 final becomes available.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From victor.stinner at haypocalc.com  Tue Nov 18 13:14:37 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Tue, 18 Nov 2008 13:14:37 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
Message-ID: <200811181314.37443.victor.stinner@haypocalc.com>

Le Tuesday 18 November 2008 11:03:02 Facundo Batista, vous avez ?crit?:
> 2008/11/17 Barry Warsaw <barry at python.org>:
> > Martin suggests, and I agree, that we should release Python 3.0 final and
> > 2.6.1 at the same time.  Makes sense to me.  That would mean that Python
> > 2.6.1 should be ready on 03-Dec (well, if Python 3.0 is ready then!).
>
> 2.6.1 only two months after 2.6? Why so quickly?

Release Early, Release Often?

I love release :-) I don't like waiting months to see the bugfixes applied 
everywhere.

Victor

From sidnei at enfoldsystems.com  Tue Nov 18 14:23:09 2008
From: sidnei at enfoldsystems.com (Sidnei da Silva)
Date: Tue, 18 Nov 2008 11:23:09 -0200
Subject: [Python-Dev] PyPI Submission Failing
Message-ID: <a7a2b76b0811180523w6e9e2c00hb030aaa665767ab9@mail.gmail.com>

I haven't seen anyone mention it on this list, and there was one
message on distutils-sig but it seems to have gone ignored [1].

Uploading packages to PyPI seems to be failing since at least
Saturday. I saw a mention somewhere [2] that it would be under
maintenance around this date, so whatever was done, it seems to have
broken package uploads.

[1] http://mail.python.org/pipermail/distutils-sig/2008-November/010491.html
[2] http://mail.python.org/pipermail/catalog-sig/2008-November/001926.html

-- 
Sidnei da Silva
Enfold Systems
http://enfoldsystems.com
Fax +1 832 201 8856
Office +1 713 942 2377 Ext 214
Skype zopedc

From barry at python.org  Tue Nov 18 15:34:44 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 18 Nov 2008 09:34:44 -0500
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
Message-ID: <4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 18, 2008, at 5:03 AM, Facundo Batista wrote:

> 2008/11/17 Barry Warsaw <barry at python.org>:
>
>> Martin suggests, and I agree, that we should release Python 3.0  
>> final and
>> 2.6.1 at the same time.  Makes sense to me.  That would mean that  
>> Python
>> 2.6.1 should be ready on 03-Dec (well, if Python 3.0 is ready then!).
>
> 2.6.1 only two months after 2.6? Why so quickly?

Actually, I've wanted to do timed releases, though I think monthly is  
unrealistic.  Maybe every two months is about the right time frame.   
Timed releases are nice because everybody then knows when a patch is  
due, from developers to downstream consumers.

> Anyway, I don't see any added value in the synchronization, so taking
> in consideration all the effort you're putting in these releases, I
> would just want to minimize your workload... which is easier to you?
> doing both at the same time or not?

We're getting releases down to a science now! :)  Actually the most  
painful part is updating the web site, so I plan adding some  
automation around that process too.  OTOH, this is the first point  
release I'll be doing with the new script, so it'll be interesting to  
debug that process.

As for synchronization, I think it's a good habit to get into, if my  
plan to do timed releases works out.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSLShXEjvBPtnXfVAQL90QP/UfWRXr0esTim+DtQJs9Fd/+Lj9PpuBV6
UCB7pAwl8uf7qIAwjDkCsdg3VD/wxzmzuwDAB8T19PF5dNxsrKWdBEzhymVpLU8T
cch0Vlaevm6Co/kDp8VhyoKlPs7LDhGkC2G04qDSOETo8Ci84rBOlWd7n1KvUrYZ
01Pn6eZHdqA=
=k9FS
-----END PGP SIGNATURE-----

From barry at python.org  Tue Nov 18 15:35:51 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 18 Nov 2008 09:35:51 -0500
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <gfuen7$nkd$1@ger.gmane.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<gfuen7$nkd$1@ger.gmane.org>
Message-ID: <175832DD-76C5-4216-AA20-ED7BB62AF53F@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 18, 2008, at 8:07 AM, Christian Heimes wrote:

> Barry Warsaw wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>> Martin suggests, and I agree, that we should release Python 3.0  
>> final and 2.6.1 at the same time.  Makes sense to me.  That would  
>> mean that Python 2.6.1 should be ready on 03-Dec (well, if Python  
>> 3.0 is ready then!).
>
> Should we release 2.6.1rc1, too?

Do we need rc's for point releases?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSLSyHEjvBPtnXfVAQJuMAP/cv59kjeFz5DxMk1hMrwXdNQvSs5Ge0lZ
ICC4DeKmz0gXZ0+PoZc4Yi9HBAQ8g7ZfKptzIPnEUrg65wV8KS6OzcK5KX5aptvF
Mqi+cmD3TPImsOEGoPnJUtlUZ7ZETrY2LSzdIIFqIE5yO1HBt3ohBcdM95+V2zQl
zt0uV+F4fnw=
=7N3R
-----END PGP SIGNATURE-----

From lists at cheimes.de  Tue Nov 18 15:52:31 2008
From: lists at cheimes.de (Christian Heimes)
Date: Tue, 18 Nov 2008 15:52:31 +0100
Subject: [Python-Dev] 2.6.1 and 3.0
In-Reply-To: <4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
Message-ID: <4922D6AF.6080400@cheimes.de>

Barry Warsaw wrote:
> Actually, I've wanted to do timed releases, though I think monthly is 
> unrealistic.  Maybe every two months is about the right time frame.  
> Timed releases are nice because everybody then knows when a patch is 
> due, from developers to downstream consumers.

 From my point of view bi-monthly release are too much. For a ?.?.1
release two months are fine because several issues are found by 3rd
party authors. But after that a release every quarter is sufficient.

* .1 release two months after the .0 release
* .2, .3, .4 and .5 release every quarter
* about here the next minor release gets out
* .6 and further releases after 6 months when necessary

Christian


From guido at python.org  Tue Nov 18 16:51:40 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 18 Nov 2008 07:51:40 -0800
Subject: [Python-Dev] Issue 4195: Can't execute packages with -m in
	Python 2.6/3.0
In-Reply-To: <49229F36.5070206@gmail.com>
References: <49229F36.5070206@gmail.com>
Message-ID: <ca471dc20811180751k317e9b6ey5a7dbffe2379b6ac@mail.gmail.com>

I think it crosses the line.

On Tue, Nov 18, 2008 at 2:55 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I need a release manager call on whether or not the proposed resolution
> of this issue goes beyond what is acceptable for a bug fix in 2.6.1.
>
> Short version:
> - Python 2.5 allowed packages to be executed with -m, but in a broken way
> - I disabled the broken way for 2.6, but didn't provide a replacement
> - The patch attached to 4195 once again allows execution of packages
> with -m, but in a clean way similar to the way directories and zipfiles
> can now be executed
> - I would like to commit that patch for 3.0/2.6.1, but I'm concerned
> that it crosses the "no new features" line
>
> Long version:
>
> There was a bug in python 2.5 that allowed a package name to be passed
> to the -m switch or runpy.run_module() and it would mostly work.
>
> However, the 'mostly' was due to the fact that doing this put the
> internal import machinery into a slightly inconsistent state: the
> interpreter was running code from inside a package, but that package
> wasn't actually present in sys.modules. This could lead to assorted hard
> to trace import-related weirdness, similar to what you can get when
> executing a file from inside a package by name. You would often get away
> with it, but good luck figuring out what is happening if things go wrong
> (and you aren't already an expert on Python import mechanics).
>
> Since the ability to execute packages wasn't intentional, I added the
> missing check to block it explicitly in 2.6 (but left it alone for 2.5).
> It seemed like a really niche feature, so I didn't figure out a clean
> replacement for it at the time.
>
> Benjamin noticed this change earlier in the 2.6 release cycle, I pointed
> out it was a deliberate change, and that's the way it stayed until after
> 2.6 was released.
>
> After the release, Andi Vajda (from the JCC project [1]) contacted me,
> and together we worked out a better implementation of package support
> for the -m switch (and runpy.run_module) [2] by extending the
> __main__.py approach used to support direct execution of zipfiles and
> directories [3].
>
> That's what I would like to add, since it nicely complements the ability
> to execute zipfiles and directories, while also restoring the ability to
> pass a package name to the -m switch (but in a way that keeps the import
> machinery in a consistent state).
>
> Cheers,
> Nick.
>
> [1] http://pypi.python.org/pypi/JCC
>
> [2] http://bugs.python.org/issue4195 (package execution with -m)
> [3] http://bugs.python.org/issue1739468 (zipfile execution)
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Tue Nov 18 17:20:12 2008
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 18 Nov 2008 10:20:12 -0600
Subject: [Python-Dev] PyPI Submission Failing
In-Reply-To: <a7a2b76b0811180523w6e9e2c00hb030aaa665767ab9@mail.gmail.com>
References: <a7a2b76b0811180523w6e9e2c00hb030aaa665767ab9@mail.gmail.com>
Message-ID: <18722.60220.502197.190991@montanaro-dyndns-org.local>


    Sidnei> I haven't seen anyone mention it on this list, and there was one
    Sidnei> message on distutils-sig but it seems to have gone ignored [1].

I've passed your note along to the website maintainers.

Thanks,

-- 
Skip Montanaro - skip at pobox.com - http://smontanaro.dyndns.org/

From g.brandl at gmx.net  Tue Nov 18 18:46:56 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 18 Nov 2008 18:46:56 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <175832DD-76C5-4216-AA20-ED7BB62AF53F@python.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<gfuen7$nkd$1@ger.gmane.org>
	<175832DD-76C5-4216-AA20-ED7BB62AF53F@python.org>
Message-ID: <gfuv2g$kg9$1@ger.gmane.org>

Barry Warsaw schrieb:
> On Nov 18, 2008, at 8:07 AM, Christian Heimes wrote:
> 
>> Barry Warsaw wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>> Martin suggests, and I agree, that we should release Python 3.0  
>>> final and 2.6.1 at the same time.  Makes sense to me.  That would  
>>> mean that Python 2.6.1 should be ready on 03-Dec (well, if Python  
>>> 3.0 is ready then!).
> 
>> Should we release 2.6.1rc1, too?
> 
> Do we need rc's for point releases?

I think we did them in the past. There probably never was a significant
change between the rc and the final, but Murphy dictates that once you
stop doing the rc, the final will be embarrassingly broken :)

cheers,
Georg


From dinov at microsoft.com  Tue Nov 18 22:01:44 2008
From: dinov at microsoft.com (Dino Viehland)
Date: Tue, 18 Nov 2008 13:01:44 -0800
Subject: [Python-Dev] bytearray methods returning self...
Message-ID: <350E7D38B6D819428718949920EC23555644813C89@NA-EXMSG-C102.redmond.corp.microsoft.com>

I've been implementing bytearray for IronPython and I noticed a couple of spots where the non-mutating methods of bytearray return self.

In 2.6 but not in 3.0 RC2:

x = bytearray(b'abc')
y = x.replace(b'abc', b'bar', 0)
id(x) == id(y)

In 2.6 and in 3.0 RC2:

t = bytearray()
for i in range(256): t.append(i)

x = bytearray(b'')
y = x.translate(t)
id(x) == id(y)

Given that bytearray always returns a new instance on all of the other methods shouldn't it return a new instance here as well?  I'm just thinking someone could be planning on mutating the result here and not expecting their original bytearray to be mutated.

From guido at python.org  Tue Nov 18 22:14:13 2008
From: guido at python.org (Guido van Rossum)
Date: Tue, 18 Nov 2008 13:14:13 -0800
Subject: [Python-Dev] bytearray methods returning self...
In-Reply-To: <350E7D38B6D819428718949920EC23555644813C89@NA-EXMSG-C102.redmond.corp.microsoft.com>
References: <350E7D38B6D819428718949920EC23555644813C89@NA-EXMSG-C102.redmond.corp.microsoft.com>
Message-ID: <ca471dc20811181314t9ed7bb0y9db4e33c2f40cb63@mail.gmail.com>

Good catch, this is a bug IMO and we should fix it in 2.6.1 and 3.0rc3.

Mind filing a bug so we can keep track of it?

--Guido

On Tue, Nov 18, 2008 at 1:01 PM, Dino Viehland <dinov at microsoft.com> wrote:
> I've been implementing bytearray for IronPython and I noticed a couple of spots where the non-mutating methods of bytearray return self.
>
> In 2.6 but not in 3.0 RC2:
>
> x = bytearray(b'abc')
> y = x.replace(b'abc', b'bar', 0)
> id(x) == id(y)
>
> In 2.6 and in 3.0 RC2:
>
> t = bytearray()
> for i in range(256): t.append(i)
>
> x = bytearray(b'')
> y = x.translate(t)
> id(x) == id(y)
>
> Given that bytearray always returns a new instance on all of the other methods shouldn't it return a new instance here as well?  I'm just thinking someone could be planning on mutating the result here and not expecting their original bytearray to be mutated.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Tue Nov 18 22:17:22 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 18 Nov 2008 13:17:22 -0800
Subject: [Python-Dev] SSH key for work
Message-ID: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>

Can someone please add the attached SSH 2 DSA key for me? I want to be
able to help out with the rc tomorrow while I am at work.

-Brett
-------------- next part --------------
A non-text attachment was scrubbed...
Name: id_dsa.pub
Type: application/octet-stream
Size: 623 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081118/fc058637/attachment.obj>

From musiccomposition at gmail.com  Tue Nov 18 22:23:31 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 18 Nov 2008 15:23:31 -0600
Subject: [Python-Dev] SSH key for work
In-Reply-To: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>
References: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>
Message-ID: <1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>

On Tue, Nov 18, 2008 at 3:17 PM, Brett Cannon <brett at python.org> wrote:
> Can someone please add the attached SSH 2 DSA key for me? I want to be
> able to help out with the rc tomorrow while I am at work.

It's probably best to send this to pydotorg at python.org.
>



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From dinov at microsoft.com  Tue Nov 18 22:30:12 2008
From: dinov at microsoft.com (Dino Viehland)
Date: Tue, 18 Nov 2008 13:30:12 -0800
Subject: [Python-Dev] bytearray methods returning self...
In-Reply-To: <ca471dc20811181314t9ed7bb0y9db4e33c2f40cb63@mail.gmail.com>
References: <350E7D38B6D819428718949920EC23555644813C89@NA-EXMSG-C102.redmond.corp.microsoft.com>
	<ca471dc20811181314t9ed7bb0y9db4e33c2f40cb63@mail.gmail.com>
Message-ID: <350E7D38B6D819428718949920EC23555644813CC5@NA-EXMSG-C102.redmond.corp.microsoft.com>

Sure, it's now bug 4348 - http://bugs.python.org/issue4348

-----Original Message-----
From: gvanrossum at gmail.com [mailto:gvanrossum at gmail.com] On Behalf Of Guido van Rossum
Sent: Tuesday, November 18, 2008 1:14 PM
To: Dino Viehland
Cc: python-dev at python.org dev
Subject: Re: [Python-Dev] bytearray methods returning self...

Good catch, this is a bug IMO and we should fix it in 2.6.1 and 3.0rc3.

Mind filing a bug so we can keep track of it?

--Guido

On Tue, Nov 18, 2008 at 1:01 PM, Dino Viehland <dinov at microsoft.com> wrote:
> I've been implementing bytearray for IronPython and I noticed a couple of spots where the non-mutating methods of bytearray return self.
>
> In 2.6 but not in 3.0 RC2:
>
> x = bytearray(b'abc')
> y = x.replace(b'abc', b'bar', 0)
> id(x) == id(y)
>
> In 2.6 and in 3.0 RC2:
>
> t = bytearray()
> for i in range(256): t.append(i)
>
> x = bytearray(b'')
> y = x.translate(t)
> id(x) == id(y)
>
> Given that bytearray always returns a new instance on all of the other methods shouldn't it return a new instance here as well?  I'm just thinking someone could be planning on mutating the result here and not expecting their original bytearray to be mutated.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)


From barry at python.org  Tue Nov 18 22:36:47 2008
From: barry at python.org (Barry Warsaw)
Date: Tue, 18 Nov 2008 16:36:47 -0500
Subject: [Python-Dev] bytearray methods returning self...
In-Reply-To: <ca471dc20811181314t9ed7bb0y9db4e33c2f40cb63@mail.gmail.com>
References: <350E7D38B6D819428718949920EC23555644813C89@NA-EXMSG-C102.redmond.corp.microsoft.com>
	<ca471dc20811181314t9ed7bb0y9db4e33c2f40cb63@mail.gmail.com>
Message-ID: <19D4131C-0BC2-4822-A716-62F465DF3EB7@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 18, 2008, at 4:14 PM, Guido van Rossum wrote:

> Good catch, this is a bug IMO and we should fix it in 2.6.1 and  
> 3.0rc3.

Ah, a /real/ test of the time machine!  Though you can avoid the risk  
of a rip in the time-space continuum by trying to get the fix into  
3.0rc4, planned for release tomorrow.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSM1b3EjvBPtnXfVAQLrtQP+OEDfWftaUJjLqdKlbH9uSfXKueKUrbJT
EZ3HPwSO1Ka5p1zKqNeSiON8MQLTMIzKW2at7YXk4nj0I7YOuho7QxatOVodZ+Er
0bA2w+pnhbJAAcRmhA0nskhXP4F7B8nOJiI6sYpiXxthBYgUu+jsGhad6kky6kwz
Kf7PjLIEi3k=
=NGqq
-----END PGP SIGNATURE-----

From brett at python.org  Tue Nov 18 22:40:48 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 18 Nov 2008 13:40:48 -0800
Subject: [Python-Dev] SSH key for work
In-Reply-To: <1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>
References: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>
	<1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>
Message-ID: <bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>

On Tue, Nov 18, 2008 at 13:23, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Tue, Nov 18, 2008 at 3:17 PM, Brett Cannon <brett at python.org> wrote:
>> Can someone please add the attached SSH 2 DSA key for me? I want to be
>> able to help out with the rc tomorrow while I am at work.
>
> It's probably best to send this to pydotorg at python.org.

Well, chances are that Martin will add the key and he reads both, so I
didn't worry about it. =)

But I guess we should decide exactly which mailing list to send to and
put that in the dev FAQ.

-Brett

From eric at trueblade.com  Tue Nov 18 22:46:01 2008
From: eric at trueblade.com (Eric Smith)
Date: Tue, 18 Nov 2008 16:46:01 -0500
Subject: [Python-Dev] SSH key for work
In-Reply-To: <bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>
References: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>	<1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>
	<bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>
Message-ID: <49233799.4060406@trueblade.com>

Brett Cannon wrote:
> On Tue, Nov 18, 2008 at 13:23, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Tue, Nov 18, 2008 at 3:17 PM, Brett Cannon <brett at python.org> wrote:
>>> Can someone please add the attached SSH 2 DSA key for me? I want to be
>>> able to help out with the rc tomorrow while I am at work.
>> It's probably best to send this to pydotorg at python.org.
> 
> Well, chances are that Martin will add the key and he reads both, so I
> didn't worry about it. =)
> 
> But I guess we should decide exactly which mailing list to send to and
> put that in the dev FAQ.

The python-committers list?

From musiccomposition at gmail.com  Tue Nov 18 22:47:58 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Tue, 18 Nov 2008 15:47:58 -0600
Subject: [Python-Dev] SSH key for work
In-Reply-To: <bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>
References: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>
	<1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>
	<bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>
Message-ID: <1afaf6160811181347w20ebfb2ah95e37e2b98f54797@mail.gmail.com>

On Tue, Nov 18, 2008 at 3:40 PM, Brett Cannon <brett at python.org> wrote:
> On Tue, Nov 18, 2008 at 13:23, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Tue, Nov 18, 2008 at 3:17 PM, Brett Cannon <brett at python.org> wrote:
>>> Can someone please add the attached SSH 2 DSA key for me? I want to be
>>> able to help out with the rc tomorrow while I am at work.
>>
>> It's probably best to send this to pydotorg at python.org.
>
> Well, chances are that Martin will add the key and he reads both, so I
> didn't worry about it. =)
>
> But I guess we should decide exactly which mailing list to send to and
> put that in the dev FAQ.

Section 3.3 of that FAQ:

Can I make check-ins from machines other than the one I generated the keys on?

Yes, all you need is to make sure that the machine you want to check
in code from has both the public and private keys in the standard
place that ssh will look for them (i.e. ~/.ssh on Unix machines).
Please note that although the key file ending in .pub contains your
user name and machine name in it, that information is not used by the
verification process, therefore these key files can be moved to a
different computer and used for verification. Please guard your keys
and never share your private key with anyone. If you lose the media on
which your keys are stored or the machine on which your keys are
stored, be sure to report this to pydotorg at python.org at the same time
that you change your keys.

>
> -Brett
>



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From brett at python.org  Tue Nov 18 23:00:17 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 18 Nov 2008 14:00:17 -0800
Subject: [Python-Dev] SSH key for work
In-Reply-To: <1afaf6160811181347w20ebfb2ah95e37e2b98f54797@mail.gmail.com>
References: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>
	<1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>
	<bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>
	<1afaf6160811181347w20ebfb2ah95e37e2b98f54797@mail.gmail.com>
Message-ID: <bbaeab100811181400h4b0d2b15vac60fef082002d03@mail.gmail.com>

On Tue, Nov 18, 2008 at 13:47, Benjamin Peterson
<musiccomposition at gmail.com> wrote:
> On Tue, Nov 18, 2008 at 3:40 PM, Brett Cannon <brett at python.org> wrote:
>> On Tue, Nov 18, 2008 at 13:23, Benjamin Peterson
>> <musiccomposition at gmail.com> wrote:
>>> On Tue, Nov 18, 2008 at 3:17 PM, Brett Cannon <brett at python.org> wrote:
>>>> Can someone please add the attached SSH 2 DSA key for me? I want to be
>>>> able to help out with the rc tomorrow while I am at work.
>>>
>>> It's probably best to send this to pydotorg at python.org.
>>
>> Well, chances are that Martin will add the key and he reads both, so I
>> didn't worry about it. =)
>>
>> But I guess we should decide exactly which mailing list to send to and
>> put that in the dev FAQ.
>
> Section 3.3 of that FAQ:
>
> Can I make check-ins from machines other than the one I generated the keys on?
>

I know I can simply copy over my private SSH key and it is protected
by password, but I would rather not expose my personal SSH key more
than I need to and since I don't control my work machine and it's
exposure level to others I would rather use another key.

-Brett

From martin at v.loewis.de  Tue Nov 18 23:14:35 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 18 Nov 2008 23:14:35 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <175832DD-76C5-4216-AA20-ED7BB62AF53F@python.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<gfuen7$nkd$1@ger.gmane.org>
	<175832DD-76C5-4216-AA20-ED7BB62AF53F@python.org>
Message-ID: <49233E4B.5070008@v.loewis.de>

>> Should we release 2.6.1rc1, too?
> 
> Do we need rc's for point releases?

We have been doing them in the past, a week before the release.

In this case, I could accept a waiver, given that the previous
release acts very well as a release candidate for this release.

Regards,
Martin

From ncoghlan at gmail.com  Tue Nov 18 23:16:39 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 19 Nov 2008 08:16:39 +1000
Subject: [Python-Dev] SSH key for work
In-Reply-To: <bbaeab100811181400h4b0d2b15vac60fef082002d03@mail.gmail.com>
References: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>	<1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>	<bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>	<1afaf6160811181347w20ebfb2ah95e37e2b98f54797@mail.gmail.com>
	<bbaeab100811181400h4b0d2b15vac60fef082002d03@mail.gmail.com>
Message-ID: <49233EC7.5010006@gmail.com>

Brett Cannon wrote:
> On Tue, Nov 18, 2008 at 13:47, Benjamin Peterson
> <musiccomposition at gmail.com> wrote:
>> On Tue, Nov 18, 2008 at 3:40 PM, Brett Cannon <brett at python.org> wrote:
>>> On Tue, Nov 18, 2008 at 13:23, Benjamin Peterson
>>> <musiccomposition at gmail.com> wrote:
>>>> On Tue, Nov 18, 2008 at 3:17 PM, Brett Cannon <brett at python.org> wrote:
>>>>> Can someone please add the attached SSH 2 DSA key for me? I want to be
>>>>> able to help out with the rc tomorrow while I am at work.
>>>> It's probably best to send this to pydotorg at python.org.
>>> Well, chances are that Martin will add the key and he reads both, so I
>>> didn't worry about it. =)
>>>
>>> But I guess we should decide exactly which mailing list to send to and
>>> put that in the dev FAQ.
>> Section 3.3 of that FAQ:
>>
>> Can I make check-ins from machines other than the one I generated the keys on?
>>
> 
> I know I can simply copy over my private SSH key and it is protected
> by password, but I would rather not expose my personal SSH key more
> than I need to and since I don't control my work machine and it's
> exposure level to others I would rather use another key.

I think Benjamin was referring more to the last sentence:
"If you lose the media on which your keys are stored or the machine on
which your keys are stored, be sure to report this to
pydotorg at python.org at the same time that you change your keys."

Adding a second key is similar to changing the keys. That said, I don't
think python-committers existed when that FAQ entry was written. It's
really up to the folks that can add new SSH keys as to which list is
most convenient for them though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From martin at v.loewis.de  Tue Nov 18 23:17:23 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 18 Nov 2008 23:17:23 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <4922D6AF.6080400@cheimes.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de>
Message-ID: <49233EF3.9040303@v.loewis.de>

> From my point of view bi-monthly release are too much. For a ?.?.1
> release two months are fine because several issues are found by 3rd
> party authors. But after that a release every quarter is sufficient.
> 
> * .1 release two months after the .0 release
> * .2, .3, .4 and .5 release every quarter
> * about here the next minor release gets out
> * .6 and further releases after 6 months when necessary

In the past, we had been striving for releases every 6 month.
This was already very difficult to achieve.

While I'm happy that Barry has automated his part to a high degree,
my part is, unfortunately, much less automated. I could personally
automate the build process a bit more, but part of it is also testing
of the installers, which is manual.

Regards,
Martin

From martin at v.loewis.de  Wed Nov 19 00:35:44 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 19 Nov 2008 00:35:44 +0100
Subject: [Python-Dev] PyPI Submission Failing
In-Reply-To: <a7a2b76b0811180523w6e9e2c00hb030aaa665767ab9@mail.gmail.com>
References: <a7a2b76b0811180523w6e9e2c00hb030aaa665767ab9@mail.gmail.com>
Message-ID: <49235150.9020609@v.loewis.de>

Sidnei da Silva wrote:
> I haven't seen anyone mention it on this list, and there was one
> message on distutils-sig but it seems to have gone ignored [1].

Please report PyPI bugs to the PyPI bug tracker in the future,
linked from each PyPI page. I personally don't read distutils-sig
(but catalog-sig, which is the list for PyPI discussion).

Regards,
Martin

From martin at v.loewis.de  Wed Nov 19 00:40:08 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 19 Nov 2008 00:40:08 +0100
Subject: [Python-Dev] SSH key for work
In-Reply-To: <49233EC7.5010006@gmail.com>
References: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>	<1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>	<bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>	<1afaf6160811181347w20ebfb2ah95e37e2b98f54797@mail.gmail.com>	<bbaeab100811181400h4b0d2b15vac60fef082002d03@mail.gmail.com>
	<49233EC7.5010006@gmail.com>
Message-ID: <49235258.7080506@v.loewis.de>

> Adding a second key is similar to changing the keys. That said, I don't
> think python-committers existed when that FAQ entry was written. It's
> really up to the folks that can add new SSH keys as to which list is
> most convenient for them though.

I had been trying to set up an email alias for the keychain ever since
I set up that mechanism, but I failed doing so. In practice, sending
it to pydotorg, python-dev, or me personally makes no difference most
of the time (although Neal will sometimes also update python-dev keys,
and Andrew will update pydotorg/www keys quite often).

Regards,
Martin

From brett at python.org  Wed Nov 19 02:13:06 2008
From: brett at python.org (Brett Cannon)
Date: Tue, 18 Nov 2008 17:13:06 -0800
Subject: [Python-Dev] SSH key for work
In-Reply-To: <49235258.7080506@v.loewis.de>
References: <bbaeab100811181317l28d04541sdfd5ae7b4797345b@mail.gmail.com>
	<1afaf6160811181323t4a88266bvcb6f3decde2b27ac@mail.gmail.com>
	<bbaeab100811181340x796ae4d6kde390121ec2367da@mail.gmail.com>
	<1afaf6160811181347w20ebfb2ah95e37e2b98f54797@mail.gmail.com>
	<bbaeab100811181400h4b0d2b15vac60fef082002d03@mail.gmail.com>
	<49233EC7.5010006@gmail.com> <49235258.7080506@v.loewis.de>
Message-ID: <bbaeab100811181713l2e7c079fr1eb6ac83934fb601@mail.gmail.com>

On Tue, Nov 18, 2008 at 15:40, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Adding a second key is similar to changing the keys. That said, I don't
>> think python-committers existed when that FAQ entry was written. It's
>> really up to the folks that can add new SSH keys as to which list is
>> most convenient for them though.
>
> I had been trying to set up an email alias for the keychain ever since
> I set up that mechanism, but I failed doing so. In practice, sending
> it to pydotorg, python-dev, or me personally makes no difference most
> of the time (although Neal will sometimes also update python-dev keys,
> and Andrew will update pydotorg/www keys quite often).
>

I will just say pydotorg in the FAQ for now then.

-Brett

From stephen at xemacs.org  Wed Nov 19 02:55:32 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 19 Nov 2008 10:55:32 +0900
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <49233EF3.9040303@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
Message-ID: <87wsf0mqcr.fsf@xemacs.org>

"Martin v. L?wis" writes:

 > While I'm happy that Barry has automated his part to a high degree,
 > my part is, unfortunately, much less automated. I could personally
 > automate the build process a bit more, but part of it is also testing
 > of the installers, which is manual.

Maybe you could delegate a lot of the testing to competent volunteers?
That would be probably 2 times as much work the first couple of times,
(you'd need to formalize your "script" for testing[1] and then
check that the volunteers are understanding it correctly, etc) but if
they are reliable you could turn that around a lot faster in the future.


Footnotes: 
[1]  Doesn't Windows have a way to send synthetic GUI events to a
program?  There ought to be a way to really script that, as the Python
installer process presumbly doesn't change much from release to release.



From sidnei at enfoldsystems.com  Wed Nov 19 03:27:22 2008
From: sidnei at enfoldsystems.com (Sidnei da Silva)
Date: Wed, 19 Nov 2008 00:27:22 -0200
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <87wsf0mqcr.fsf@xemacs.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org>
Message-ID: <a7a2b76b0811181827y30b547d1n3fcf7e4c10c0c60d@mail.gmail.com>

On Tue, Nov 18, 2008 at 11:55 PM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Footnotes:
> [1]  Doesn't Windows have a way to send synthetic GUI events to a
> program?  There ought to be a way to really script that, as the Python
> installer process presumbly doesn't change much from release to release.

There's at least PyWinAuto[1], Watsup[2] and winGuiAuto[3].

[1] http://pywinauto.seleniumhq.org/
[2] http://www.tizmoi.net/watsup/intro.html
[3] http://www.brunningonline.net/simon/blog/archives/winGuiAuto.py.html

-- 
Sidnei da Silva
Enfold Systems
http://enfoldsystems.com
Fax +1 832 201 8856
Office +1 713 942 2377 Ext 214
Skype zopedc

From martin at v.loewis.de  Wed Nov 19 07:02:23 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 19 Nov 2008 07:02:23 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <87wsf0mqcr.fsf@xemacs.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org>
Message-ID: <4923ABEF.50900@v.loewis.de>

>  > While I'm happy that Barry has automated his part to a high degree,
>  > my part is, unfortunately, much less automated. I could personally
>  > automate the build process a bit more, but part of it is also testing
>  > of the installers, which is manual.
> 
> Maybe you could delegate a lot of the testing to competent volunteers?

That's not the issue - I don't mind spending that time. However, it
means that several hours pass between starting the release process, and
making the binaries available - during this time, users always complain
why the Windows binaries are not released yet.

With additional volunteers, availability of the binaries would lag even
more behind the release announcement.

> [1]  Doesn't Windows have a way to send synthetic GUI events to a
> program?  There ought to be a way to really script that, as the Python
> installer process presumbly doesn't change much from release to release.

I also need to involve different machines, e.g. XP machines and Vista
machines, and machines that have Visual Studio installed and machines
that don't. Plus, I need to log into each machine in different ways:
as admin user and non-admin user. The automated GUI testing only really
works for a logged-in user.

Regards,
Martin

From stephen at xemacs.org  Wed Nov 19 08:32:18 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 19 Nov 2008 16:32:18 +0900
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <4923ABEF.50900@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org> <4923ABEF.50900@v.loewis.de>
Message-ID: <87iqqkmarh.fsf@xemacs.org>

"Martin v. L?wis" writes:

 > That's not the issue - I don't mind spending that time. However, it
 > means that several hours pass between starting the release process,
 > and making the binaries available - during this time, users always
 > complain why the Windows binaries are not released yet.

For "several hours" delay?  Shame on the complainers!  Ubuntu and
MacPorts users have to wait days or weeks for installers.  Debian
stable users, years!<wink>

My understanding was that the biggest problem with keeping to a
6-month cycle has always been that it's still a long enough time frame
that people will rush to get an 80%-done project into the release just
before deadline, causing extra reviewing effort for the senior
committers and effort and delays for everyone for bug fixing.  One
month is probably short enough that people will be willing to submit
things at a more appropriate stage in development.  Still, it's the
review and polishing-up effort that is the bottleneck, it seems to me.
Not the installers.

From facundobatista at gmail.com  Wed Nov 19 10:53:55 2008
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 19 Nov 2008 07:53:55 -0200
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <49233EF3.9040303@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
Message-ID: <e04bdf310811190153h664a4e6qe932ba10be26da78@mail.gmail.com>

2008/11/18 "Martin v. L?wis" <martin at v.loewis.de>:

> While I'm happy that Barry has automated his part to a high degree,
> my part is, unfortunately, much less automated. I could personally
> automate the build process a bit more, but part of it is also testing
> of the installers, which is manual.

Martin, maybe we can help you with the installers testing.

While I don't have a clue about compiling complex software in Windows
(and also want to stay away from that), I have a virtualbox with a win
xp in my workstation, so I could try an installer.

Maybe you could put a wiki page somewhere with a small recipe about
what to look when testing an installer, and then produce all the
versions, upload to it, and alert us here. So we go, download one of
them, try it, and then mark it as tested with our name (maybe we can
look after two or three testers per installer).

I don't know if this will be quicker, but surely will lower your
workload regarding this, which is a good thing.

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From fuzzyman at voidspace.org.uk  Wed Nov 19 12:16:56 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 19 Nov 2008 11:16:56 +0000
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <4923ABEF.50900@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>
	<4923ABEF.50900@v.loewis.de>
Message-ID: <4923F5A8.5010505@voidspace.org.uk>

Martin v. L?wis wrote:
>>  > While I'm happy that Barry has automated his part to a high degree,
>>  > my part is, unfortunately, much less automated. I could personally
>>  > automate the build process a bit more, but part of it is also testing
>>  > of the installers, which is manual.
>>
>> Maybe you could delegate a lot of the testing to competent volunteers?
>>     
>
> That's not the issue - I don't mind spending that time. However, it
> means that several hours pass between starting the release process, and
> making the binaries available - during this time, users always complain
> why the Windows binaries are not released yet.
>
> With additional volunteers, availability of the binaries would lag even
> more behind the release announcement.
>
>   

Installer tests can definitely be automated, and there is also a Python 
API to the virtualbox VM. I wonder if it would be possible to automate 
testing all the installers in various scenarios - each running 
simultaneously in a VM.

Michael

>> [1]  Doesn't Windows have a way to send synthetic GUI events to a
>> program?  There ought to be a way to really script that, as the Python
>> installer process presumbly doesn't change much from release to release.
>>     
>
> I also need to involve different machines, e.g. XP machines and Vista
> machines, and machines that have Visual Studio installed and machines
> that don't. Plus, I need to log into each machine in different ways:
> as admin user and non-admin user. The automated GUI testing only really
> works for a logged-in user.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/


From stephen at xemacs.org  Wed Nov 19 15:43:44 2008
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 19 Nov 2008 23:43:44 +0900
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <4923F5A8.5010505@voidspace.org.uk>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org> <4923ABEF.50900@v.loewis.de>
	<4923F5A8.5010505@voidspace.org.uk>
Message-ID: <87bpwbn5cv.fsf@xemacs.org>

Michael Foord writes:

 > Installer tests can definitely be automated, and there is also a Python 
 > API to the virtualbox VM. I wonder if it would be possible to automate 
 > testing all the installers in various scenarios - each running 
 > simultaneously in a VM.

Now that would be an impressive tour de force!


From barry at python.org  Wed Nov 19 16:11:32 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 19 Nov 2008 10:11:32 -0500
Subject: [Python-Dev] 2.6.1 and 3.0
In-Reply-To: <4922D6AF.6080400@cheimes.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de>
Message-ID: <352DBA22-D814-43C5-84F8-BD9AEE756DCD@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 18, 2008, at 9:52 AM, Christian Heimes wrote:

> Barry Warsaw wrote:
>> Actually, I've wanted to do timed releases, though I think monthly  
>> is unrealistic.  Maybe every two months is about the right time  
>> frame.  Timed releases are nice because everybody then knows when a  
>> patch is due, from developers to downstream consumers.
>
> From my point of view bi-monthly release are too much. For a ?.?.1
> release two months are fine because several issues are found by 3rd
> party authors. But after that a release every quarter is sufficient.
>
> * .1 release two months after the .0 release
> * .2, .3, .4 and .5 release every quarter
> * about here the next minor release gets out
> * .6 and further releases after 6 months when necessary

Timed releases have a lot of advantages, and I would like to see if we  
can adopt them and realize these benefits.  What I like most about  
them is that everyone knows what's happening when and can coordinate  
efforts.  Developers will know automatically (no reminders or alarms)  
when the next release is happening, so they can schedule what they  
want to do more easily.  Release experts can block out the appropriate  
time on their schedules and plan more efficiently.  Downstream  
consumers have a better idea of when updates are available and can  
lobby for certain critical bugs to be fixed in a timely and  
predictable manner.

I think 6 months is too long between releases -- it might as well not  
be timed.  It sounds like the Windows side is a bit of a pain, and  
since we're all busy, one month is probably too soon.  That's why I  
proposed bi-monthly.

I really want our releases to be predictable.  I don't think we have  
to worry about nothing getting committed to the trees in 2 months time.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSQspHEjvBPtnXfVAQIfwwP8DzaIge8b1rL9/zACiwZ5nOn9S5d+ng+p
zjSSvDKgfxX5kEMfUQQuJgI6GIOPvUm0wsmdZnH5f5AD86/1Qz1ugsBkHXO6BWWl
LEI2jNjsIU9m1icQkQSnENxJoI5BFFA9upewT1zwo9md4cErzQLiK+WQrblu1hXE
GKaxW0Xrva4=
=ZI9e
-----END PGP SIGNATURE-----

From barry at python.org  Wed Nov 19 16:17:44 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 19 Nov 2008 10:17:44 -0500
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <gfuv2g$kg9$1@ger.gmane.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<gfuen7$nkd$1@ger.gmane.org>
	<175832DD-76C5-4216-AA20-ED7BB62AF53F@python.org>
	<gfuv2g$kg9$1@ger.gmane.org>
Message-ID: <7D574FB8-8806-4670-813B-69C9F6CB810E@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 18, 2008, at 12:46 PM, Georg Brandl wrote:

> Barry Warsaw schrieb:
>> On Nov 18, 2008, at 8:07 AM, Christian Heimes wrote:
>>
>>> Barry Warsaw wrote:
>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>> Hash: SHA1
>>>> Martin suggests, and I agree, that we should release Python 3.0
>>>> final and 2.6.1 at the same time.  Makes sense to me.  That would
>>>> mean that Python 2.6.1 should be ready on 03-Dec (well, if Python
>>>> 3.0 is ready then!).
>>
>>> Should we release 2.6.1rc1, too?
>>
>> Do we need rc's for point releases?
>
> I think we did them in the past. There probably never was a  
> significant
> change between the rc and the final, but Murphy dictates that once you
> stop doing the rc, the final will be embarrassingly broken :)

True.  If the rc's are actually tested and help avoid embarrassment  
I'm all for them.  If it's just extra work that few will test, then  
let's skip them and just do brown bag releases if necessary.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSQuGHEjvBPtnXfVAQJCfwQAky+ORhR0LaoZ0nevGBkEkl5LZbP0+A4a
p0puGCnxuY6DVqx38dJUPLqt+wle+Zw9QX4PhhaalbTWyOQScKQk0p0CxagLnTeG
GvlyTQLUM9RxFzolnzcY8mU8ewGnCJp16d7TR40AmMZ/geV/xMDzxL+tPKwiq/5p
C4j+VmFHnMU=
=EGqf
-----END PGP SIGNATURE-----

From barry at python.org  Wed Nov 19 16:27:42 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 19 Nov 2008 10:27:42 -0500
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <49233EF3.9040303@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
Message-ID: <3C7340EB-C07C-4E0A-BA44-CB536012FEBF@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 18, 2008, at 5:17 PM, Martin v. L?wis wrote:

>> From my point of view bi-monthly release are too much. For a ?.?.1
>> release two months are fine because several issues are found by 3rd
>> party authors. But after that a release every quarter is sufficient.
>>
>> * .1 release two months after the .0 release
>> * .2, .3, .4 and .5 release every quarter
>> * about here the next minor release gets out
>> * .6 and further releases after 6 months when necessary
>
> In the past, we had been striving for releases every 6 month.
> This was already very difficult to achieve.
>
> While I'm happy that Barry has automated his part to a high degree,
> my part is, unfortunately, much less automated. I could personally
> automate the build process a bit more, but part of it is also testing
> of the installers, which is manual.

Martin, I'm keen on figuring out a way to reduce your workload, and  
also to coordinate releases better between us.  I /think/ with timed  
releases I can tag a little early and give you something to work on so  
that the actual release is a matter of fiddling web pages and sending  
an announcement.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSQwbnEjvBPtnXfVAQIOuAP/fxzFpp886TordGNdd4tusqasL/VK2lpr
wbhcfwh5TQbVhkhi9CVUFa7BNXCpgxG1nqWT9+ynSdNKIYKnK8kkjRE7FhEYantP
TYkuRGI+2DznKjRBtVNXJq+JNktARWKhQwFkc0AmqooCYvhxqt9T5AkEgN4jRn4s
YBLaex4g3rA=
=Oi0b
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Wed Nov 19 19:52:08 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 19 Nov 2008 19:52:08 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <e04bdf310811190153h664a4e6qe932ba10be26da78@mail.gmail.com>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<e04bdf310811190153h664a4e6qe932ba10be26da78@mail.gmail.com>
Message-ID: <49246058.9090009@v.loewis.de>

> Martin, maybe we can help you with the installers testing.

Thanks for the offer. See my other message, though - this is not the
point. If everything goes well, offloading testing just means that
I have to wait some time for the testers to come back, and do other
stuff meanwhile.

For the majority of alpha and beta releases, something went wrong
each time. A file was forgotten to be included in the installer
generator, causing it to be missing on the target system. I forgot
to perform a manual build step, causing the installer to fail, and
so on. Then I have to debug the problem, and restart the production
process from scratch. Offloading to testers in this case would just mean
that I wait much longer until I can release, and it might not be
possible to complete the build within a single day.

> I don't know if this will be quicker, but surely will lower your
> workload regarding this, which is a good thing.

Thanks again - but I do typically find the time to do the release
(if not, it gets delayed by another day).

Regards,
Martin

From martin at v.loewis.de  Wed Nov 19 20:02:04 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 19 Nov 2008 20:02:04 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <4923F5A8.5010505@voidspace.org.uk>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>
	<4923ABEF.50900@v.loewis.de> <4923F5A8.5010505@voidspace.org.uk>
Message-ID: <492462AC.6090601@v.loewis.de>

> Installer tests can definitely be automated, and there is also a Python
> API to the virtualbox VM. I wonder if it would be possible to automate
> testing all the installers in various scenarios - each running
> simultaneously in a VM.

I do use VMs, yes. However, they don't run on my workstation - which
is 32-bit XP. It might be possible to automate it, but IMO, the
effort of setting this up would be higher than the actual time spend
in doing it manually, assuming we have no more than a dozen releases per
year.

Regards,
Martin

From martin at v.loewis.de  Wed Nov 19 20:18:31 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 19 Nov 2008 20:18:31 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <3C7340EB-C07C-4E0A-BA44-CB536012FEBF@python.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<3C7340EB-C07C-4E0A-BA44-CB536012FEBF@python.org>
Message-ID: <49246687.2040204@v.loewis.de>

> Martin, I'm keen on figuring out a way to reduce your workload, and also
> to coordinate releases better between us.  I /think/ with timed releases
> I can tag a little early and give you something to work on so that the
> actual release is a matter of fiddling web pages and sending an
> announcement.

Again - the work load is not so much an issue at the moment, and I
expect it to be reduced even after 3.0 is finally released and 2.5
retired.

I would indeed appreciate tighter coordination. Anthony's process
differed from yours primarily in him waiting for the release
announcements until the binaries are actually available. That might
mean that a day or two might pass, but it did help to remove the
feeling of working under tight deadlines.

Regards,
Martin

From barry at python.org  Wed Nov 19 20:37:36 2008
From: barry at python.org (Barry Warsaw)
Date: Wed, 19 Nov 2008 14:37:36 -0500
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <49246687.2040204@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<3C7340EB-C07C-4E0A-BA44-CB536012FEBF@python.org>
	<49246687.2040204@v.loewis.de>
Message-ID: <6B34AD98-187C-4876-A410-E0A843EEF2C4@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 19, 2008, at 2:18 PM, Martin v. L?wis wrote:

>> Martin, I'm keen on figuring out a way to reduce your workload, and  
>> also
>> to coordinate releases better between us.  I /think/ with timed  
>> releases
>> I can tag a little early and give you something to work on so that  
>> the
>> actual release is a matter of fiddling web pages and sending an
>> announcement.
>
> Again - the work load is not so much an issue at the moment, and I
> expect it to be reduced even after 3.0 is finally released and 2.5
> retired.
>
> I would indeed appreciate tighter coordination. Anthony's process
> differed from yours primarily in him waiting for the release
> announcements until the binaries are actually available. That might
> mean that a day or two might pass, but it did help to remove the
> feeling of working under tight deadlines.

Let's try this for 3.0rc4 then.  I think all it means is that I won't  
push the new pages or make the announcement until you verify that the  
Windows builds are ready and available.  We can still use python- 
committers to coordinate when that will happen, and I'll still do all  
the release mechanics from my end as normal.  It's okay if the  
announcement happens Friday or over the weekend.

I will also try to get up early to do the release before my work day  
starts, to better coordinate with Euro time.  So expect me on #python- 
dev tomorrow (my morning).

Will that work for you?

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSRrAnEjvBPtnXfVAQJiDwP/ZcbHnwkvWligaP2a3OXEmZ30GZoG1NQn
+Lj/j4YNANkhxZ4Vgg9gkMH3mQ+eTwWdqr1/VM3LTW+fFXhdtAaAG1NsvHAlkAE0
N+DgEOEv4aMuO6MZplv/1kh4WeFC7SsnEX7bLext0QZITdBaL65dUN8Kt8G/ZeTG
w3lQ01nBFqY=
=InnO
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Wed Nov 19 20:40:38 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 19 Nov 2008 20:40:38 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <gg1jne$so8$1@ger.gmane.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>
	<gg1jne$so8$1@ger.gmane.org>
Message-ID: <49246BB6.7000607@v.loewis.de>

> In which case why not just hold the release until all installers are
> available? 

That is how Anthony Baxter handled things, indeed, and I would
appreciate if we would return to that procedure.

> Or are the complainers Python developers who know what goes on behind
> the scenes?

No - typically outsiders, who report that the links are broken (if the
links get updated and the files are missing), or that the links are old
(if the links are not updated). I think these people also try to be
helpful (in addition to being frustrated that the release announcement
is meaningless to them, and that they have to poll the release page).

>> With additional volunteers, availability of the binaries would lag even
>> more behind the release announcement.
>>
> I really appreciate the dedicated work you put in to the Windows
> installers (as I am sure many others do also), but I wouldn't want to
> saddle you with it indefinitely. How well is the procedure documented?

IIRC, Christian Heimes did one of the alpha or beta releases, with what
little documentation is available, so it's definitely doable.

The tricky part really is when it breaks (which it does more often than
not), in which case you need to understand msi.py, for which you need to
understand MSI. IMO, the Microsoft is excellent (in being fairly
precise), but the learning curve is high. The mechanical part of it can
is completely automated - we produce daily MSI files in a buildbot
slave (which may or may not work - I haven't checked in a while)

> I
> ask this in hopes that you aren't a potential single point of failure in
> the release process.

I think several of the "Windows people" could jump in, not just
Christian. That would be best done in a beta release or release
candidate, since one does get things wrong the first time.

Regards,
Martin

From martin at v.loewis.de  Wed Nov 19 20:50:41 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 19 Nov 2008 20:50:41 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <6B34AD98-187C-4876-A410-E0A843EEF2C4@python.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<3C7340EB-C07C-4E0A-BA44-CB536012FEBF@python.org>
	<49246687.2040204@v.loewis.de>
	<6B34AD98-187C-4876-A410-E0A843EEF2C4@python.org>
Message-ID: <49246E11.6030206@v.loewis.de>

> I will also try to get up early to do the release before my work day
> starts, to better coordinate with Euro time.  So expect me on
> #python-dev tomorrow (my morning).
> 
> Will that work for you?

If you delay the announcement until the binaries are ready, you should
feel free to work on it whenever it suits you best, as far as I'm
concerned (of course, to coordinate with Georg, you might still prefer
to work during the European daylight).

I'll be busy with lectures tomorrow most of the day, and can't start
working on the installer before 14:00 UTC (which I think is 9:00
your time). Around what time would you expect to have the tag set?

Regards,
Martin

From tjreedy at udel.edu  Wed Nov 19 21:19:21 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 19 Nov 2008 15:19:21 -0500
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <6B34AD98-187C-4876-A410-E0A843EEF2C4@python.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>
	<49233EF3.9040303@v.loewis.de>	<3C7340EB-C07C-4E0A-BA44-CB536012FEBF@python.org>	<49246687.2040204@v.loewis.de>
	<6B34AD98-187C-4876-A410-E0A843EEF2C4@python.org>
Message-ID: <gg1sc8$tus$1@ger.gmane.org>

Barry Warsaw wrote:
> -----BEGIN PGP SIGNED MESSAGE-----

> 
> Let's try this for 3.0rc4 then.

The current release is rc2.  Skipping rc3 would confuse people'-)


From mrts at mrts.pri.ee  Thu Nov 20 10:37:31 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Thu, 20 Nov 2008 11:37:31 +0200
Subject: [Python-Dev] CVE tracking
Message-ID: <49252FDB.7090904@mrts.pri.ee>

Hello!

Does someone systematically track the CVE vulnerability list?

Ideally, Python security officers would have close collaboration with 
whoever
manages CVE (like distribution security officers do), so that

 * every CVE issue would have a corresponding ticket on Python bug tracker
   (perhaps the process can be automated to some degree?)

 * that ticket would be referred to in CVE vulnerability page "References"
   section (see e.g.
   http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-2315 ,
   that does not have a corresponding Python bug tracker link)

 * all CVE issues would be listed in 
http://www.python.org/news/security/ with
   corresponding information about when the fix has been or will be commited
   and which upcoming or past release incorporates it.

Some relevant links:
http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=python
http://secunia.com/advisories/product/14172/?task=advisories

From kristjan at ccpgames.com  Thu Nov 20 16:59:38 2008
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 20 Nov 2008 15:59:38 +0000
Subject: [Python-Dev] issue 4336
Message-ID: <4E9372E6B2234D4F859320D896059A95101A4F42F9@exchis.ccp.ad.local>

Hello there.
I recently created this issue:  http://bugs.python.org/issue4336
I've had no response so far.  This patch solved a serious performance problem with simple xmlrpc queries for us.
Any interest or thoughts?

I am only slightly uncomfortable with changing the buffersize.   I notice that socket._fileobject.read()
Disallows small recv() calls for performance reasons and forces buffer to at least 8k.  But
readline() does no such things and becomes very inefficient for the unbuffered case.  Maybe it would
be better just to fix socket._fileobject to have _rbufsize be always at lest 8k

K
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081120/37aa1d6f/attachment.htm>

From gisle at activestate.com  Thu Nov 20 18:37:37 2008
From: gisle at activestate.com (Gisle Aas)
Date: Thu, 20 Nov 2008 18:37:37 +0100
Subject: [Python-Dev] CVE tracking
In-Reply-To: <49252FDB.7090904@mrts.pri.ee>
References: <49252FDB.7090904@mrts.pri.ee>
Message-ID: <54C291A4-8120-4DF6-800C-4FACAAE8274A@activestate.com>

Perl had a few CVE because of its rmtree implementation.  Removing  
trees is risky business if root runs the function while other users  
have access to manipulate the tree.  Python's shutils.rmtree seems to  
have many of the same issues.
For instance http://bugs.debian.org/286922 shows how to get root to  
remove /etc/passwd.  The attack should work with shutils.rmtree as  
well.  The referenced bug is a followup to CVE-2005-0448.
This just to show that there are relevant CVEs that don't have the  
keyword "python" attached to them.
--Gisle


From tjreedy at udel.edu  Thu Nov 20 18:59:56 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 20 Nov 2008 12:59:56 -0500
Subject: [Python-Dev] issue 4336
In-Reply-To: <4E9372E6B2234D4F859320D896059A95101A4F42F9@exchis.ccp.ad.local>
References: <4E9372E6B2234D4F859320D896059A95101A4F42F9@exchis.ccp.ad.local>
Message-ID: <gg48is$b69$1@ger.gmane.org>

Kristj?n Valur J?nsson wrote:
> Hello there.
> 
> I recently created this issue:  http://bugs.python.org/issue4336

3 days ago.

> I?ve had no response so far. 

Possible reasons: no automatic assignee; core developers focused on 
getting 2.5.3, 2.6.1, and 3.0.0 out in the next few weeks; some people 
(me, at least) use Friday new issues list for issue review.

 > This patch solved a serious performance
> problem with simple xmlrpc queries for us.
> 
> Any interest or thoughts?
> 
> I am only slightly uncomfortable with changing the buffersize.   I 
> notice that socket._fileobject.read()  Disallows small recv() calls for performance
 >  reasons and forces buffer to at least 8k.  But
> readline() does no such things and becomes very inefficient for the 
> unbuffered case.  Maybe it would be better just to fix socket._fileobject
 >  to have _rbufsize be always at lest 8k

You might want to add these thoughts to the issue too.


From barry at python.org  Fri Nov 21 16:06:44 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 21 Nov 2008 10:06:44 -0500
Subject: [Python-Dev] RELEASED Python 3.0rc3
Message-ID: <CCD64757-114A-4B3B-9427-4E8BA6F6FCD3@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team and the Python community, I  
am happy to announce the third and last planned release candidate for  
Python 3.0.

This is a release candidate, so while it is not quite suitable for  
production environments, we strongly encourage you to download and  
test this release on your software.  We expect only critical bugs to  
be fixed between now and the final release, currently planned for 03- 
Dec-2008.

If you find things broken or incorrect, please submit bug reports at

    http://bugs.python.org

Please read the RELNOTES file in the distribution for important  
details about this release.  For more information and downloadable  
distributions, see the Python 3.0 website:

    http://www.python.org/download/releases/3.0/

See PEP 361 for release schedule details:

    http://www.python.org/dev/peps/pep-0361/

Enjoy,
- -Barry

Barry Warsaw
barry at python.org
Python 2.6/3.0 Release Manager
(on behalf of the entire python-dev team)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSSbOhHEjvBPtnXfVAQLzBwP/dS2j4XhZMNdb28TG3ZblkSmlPS4IU20U
Vvq85inUkJ6idwKZBqa6brrD1hbqrl4UjKZh4/ppzhIwsJtFMlMiqnkHVrvIYFBG
Yg+pQdO5HQzrw9K04aTdtNiKTiiJNIkqWdQQUd573XBFODRAIaq0qwk9C24kXeZM
e3xNgNRxfmY=
=TvxY
-----END PGP SIGNATURE-----

From status at bugs.python.org  Fri Nov 21 18:06:45 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 21 Nov 2008 18:06:45 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20081121170645.3954E78534@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (11/14/08 - 11/21/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2194 open (+35) / 14007 closed (+18) / 16201 total (+53)

Open issues with patches:   728

Average duration of open issues: 712 days.
Median duration of open issues: 2043 days.

Open Issues Breakdown
   open  2177 (+35)
pending    17 ( +0)

Issues Created Or Reopened (56)
_______________________________

Tkinter binding involving Control-spacebar raises unicode error  11/18/08
       http://bugs.python.org/issue1028    reopened ocean-city                
       patch                                                                   

Byte/string inconsistencies between different dbm modules        11/21/08
       http://bugs.python.org/issue3799    reopened brett.cannon              
       patch, needs review                                                     

type of UserList instance returns class instead of instance      11/14/08
CLOSED http://bugs.python.org/issue4326    created  chafporte                 
                                                                               

Patch: simplify complex constant assignment statements           11/14/08
       http://bugs.python.org/issue4327    created  novalis_dt                
       patch                                                                   

"??" in u"foo" raises a misleading error                         11/15/08
       http://bugs.python.org/issue4328    created  ezio.melotti              
                                                                               

base64 does not properly handle unicode strings                  11/15/08
       http://bugs.python.org/issue4329    created  mbecker                   
                                                                               

wsgiref.validate doesn't accept arguments to readline            11/16/08
CLOSED http://bugs.python.org/issue4330    created  ianb                      
                                                                               

Can't use _functools.partial() created function as method        11/16/08
       http://bugs.python.org/issue4331    created  ssadler                   
                                                                               

asyncore.file_dispatcher does not use dup()'ed fd                11/17/08
CLOSED http://bugs.python.org/issue4332    created  christianbecke            
       patch                                                                   

Reworked Dialog.py                                               11/17/08
       http://bugs.python.org/issue4333    created  gpolo                     
       patch                                                                   

Mac Build Script is broken for 2.6 release                       11/17/08
CLOSED http://bugs.python.org/issue4334    created  ngift                     
                                                                               

inspect.getsourcelines ignores last line in module               11/17/08
       http://bugs.python.org/issue4335    created  RafeSacks                 
                                                                               

Fix performance issues in xmlrpclib                              11/17/08
       http://bugs.python.org/issue4336    created  krisvale                  
       patch, patch, easy                                                      

Iteration over a map object with list()                          11/17/08
CLOSED http://bugs.python.org/issue4337    created  jmfauth                   
                                                                               

TypeError (bytes/str) in distutils command "upload"              11/17/08
CLOSED http://bugs.python.org/issue4338    created  hagen                     
       patch                                                                   

Fix set-like dictview doc                                        11/17/08
CLOSED http://bugs.python.org/issue4339    created  tjreedy                   
                                                                               

xmlrpc.client - default 'SlowParser' not defined                 11/17/08
       http://bugs.python.org/issue4340    created  mwatkins                  
                                                                               

Update __hash__ doc                                              11/17/08
CLOSED http://bugs.python.org/issue4341    created  tjreedy                   
                                                                               

(Tkinter) Please backport these                                  11/18/08
       http://bugs.python.org/issue4342    created  gpolo                     
                                                                               

New function in Tkinter.py: setup_master                         11/18/08
       http://bugs.python.org/issue4343    created  gpolo                     
       patch                                                                   

crash upon launch                                                11/18/08
CLOSED http://bugs.python.org/issue4344    created  source.mod                
                                                                               

Implement nb_nonzero for PyTclObject                             11/18/08
       http://bugs.python.org/issue4345    created  gpolo                     
       patch                                                                   

PyObject_CallMethod changes the exception message already set by 11/18/08
       http://bugs.python.org/issue4346    created  gpolo                     
       patch                                                                   

Dependencies of graminit.h are not rebuilt when the file is rege 11/18/08
       http://bugs.python.org/issue4347    created  thomas.lee                
       patch                                                                   

bytearray methods returning self                                 11/18/08
CLOSED http://bugs.python.org/issue4348    created  DinoV                     
       patch, needs review                                                     

sys.path includes extraneous junk                                11/18/08
CLOSED http://bugs.python.org/issue4349    created  exarkun                   
       patch, needs review                                                     

Remove dead code from Tkinter.py                                 11/19/08
       http://bugs.python.org/issue4350    created  gpolo                     
       patch                                                                   

[PATCH] Better stacklevel for GzipFile.filename DeprecationWarni 11/19/08
       http://bugs.python.org/issue4351    created  pjenvey                   
       patch                                                                   

imp.find_module() fails with a UnicodeDecodeError when called wi 11/19/08
       http://bugs.python.org/issue4352    created  Jukka Aho                 
                                                                               

Move description what a trace should should return to settrace f 11/19/08
CLOSED http://bugs.python.org/issue4353    created  rocky                     
                                                                               

distutils.command.register is broken                             11/19/08
CLOSED http://bugs.python.org/issue4354    created  hagen                     
       patch                                                                   

Error in docs of urllib.request and urllib.parse                 11/19/08
       http://bugs.python.org/issue4355    created  hagen                     
       patch                                                                   

Add "key" argument to "bisect" module functions                  11/19/08
CLOSED http://bugs.python.org/issue4356    created  tebeka                    
                                                                               

frozen?set operations create incorrectly initialized instances o 11/19/08
CLOSED http://bugs.python.org/issue4357    created  alexhsamuel               
                                                                               

Segfault in stringobject.c                                       11/19/08
       http://bugs.python.org/issue4358    created  farshad                   
                                                                               

at runtime, distutils uses buildtime files                       11/19/08
       http://bugs.python.org/issue4359    created  a.badger                  
                                                                               

SystemError when method has both super() & closure               11/20/08
CLOSED http://bugs.python.org/issue4360    created  kaizhu                    
       patch                                                                   

Docstring for "Lib/string.py" is outdated                        11/20/08
       http://bugs.python.org/issue4361    created  thp                       
       patch                                                                   

FileIO object in io module                                       11/20/08
CLOSED http://bugs.python.org/issue4362    reopened barry                     
       patch, needs review                                                     

Make uuid module functions usable without ctypes                 11/20/08
       http://bugs.python.org/issue4363    created  hdima                     
       patch                                                                   

error in multiprocessing docs - rawvalue                         11/20/08
       http://bugs.python.org/issue4364    created  dursobr                   
                                                                               

Add CRT version info in msvcrt module                            11/20/08
       http://bugs.python.org/issue4365    created  cdavid                    
       patch                                                                   

cannot find -lpython2.5 when buinding Python 2.5.2 on FreeBSD 4. 11/20/08
       http://bugs.python.org/issue4366    created  akitada                   
                                                                               

segmentation fault in ast_for_atom                               11/20/08
       http://bugs.python.org/issue4367    created  ot                        
       patch, needs review                                                     

A bug in ncurses.h still exists in FreeBSD 4.9 - 4.11            11/20/08
       http://bugs.python.org/issue4368    created  akitada                   
       patch                                                                   

Error building to a nonstandard prefix (with patch)              11/20/08
       http://bugs.python.org/issue4369    created  ot                        
       patch                                                                   

warning: unknown conversion type character `z' in format         11/20/08
       http://bugs.python.org/issue4370    created  akitada                   
                                                                               

coerce gone---but not from docs                                  11/20/08
       http://bugs.python.org/issue4371    created  LambertDW                 
                                                                               

__cmp__ removal not in What's New                                11/20/08
CLOSED http://bugs.python.org/issue4372    created  tjreedy                   
                                                                               

Reference leaks in Python 3.0rc3                                 11/21/08
       http://bugs.python.org/issue4373    created  christian.heimes          
       patch                                                                   

Pickle tests fail w/o _pickle extension                          11/21/08
       http://bugs.python.org/issue4374    created  christian.heimes          
                                                                               

inspect.getsource doesn't work with PYTHONPATH and source compil 11/21/08
       http://bugs.python.org/issue4375    created  erickt                    
                                                                               

Nested ctypes 'BigEndianStructure' fails                         11/21/08
       http://bugs.python.org/issue4376    created  castironpi                
                                                                               

tokenize.detect_encoding() and Mac newline                       11/21/08
       http://bugs.python.org/issue4377    created  haypo                     
                                                                               

howto doc update                                                 11/21/08
       http://bugs.python.org/issue4378    created  LambertDW                 
                                                                               

Fix exception pickling: Move initial args assignment to BaseExce 11/18/08
       http://bugs.python.org/issue1692335 reopened gvanrossum                
       patch                                                                   



Issues Now Closed (40)
______________________

Problem with doctest and decorated functions                      441 days
       http://bugs.python.org/issue1108    skip.montanaro            
                                                                               

conditional jump to a POP_TOP optimization                        253 days
       http://bugs.python.org/issue2260    rhettinger                
       patch                                                                   

NULL member in modules_by_index                                   131 days
       http://bugs.python.org/issue3327    loewis                    
       patch, easy                                                             

test_cpickle crash on AMD64 Windows build                          88 days
       http://bugs.python.org/issue3640    loewis                    
       patch, 64bit                                                            

pressing HOME key in IDLE editor ends IDLE                         81 days
       http://bugs.python.org/issue3679    tjreedy                   
                                                                               

configure --with-threads on cygwin => crash on thread related te   58 days
       http://bugs.python.org/issue3947    ocean-city                
       patch                                                                   

ignored exceptions in generators (regression?)                     43 days
       http://bugs.python.org/issue4040    benjamin.peterson         
                                                                               

Document the effects of NotImplemented on == and !=                43 days
       http://bugs.python.org/issue4087    marketdickinson           
                                                                               

Documenting set comparisons and operations                         39 days
       http://bugs.python.org/issue4090    rhettinger                
       patch                                                                   

name conflict in ScrolledCanvas.__init__() in Lib/turtle.py        36 days
       http://bugs.python.org/issue4116    loewis                    
       patch                                                                   

"__getattr__" can't be a descriptor                                19 days
       http://bugs.python.org/issue4230    benjamin.peterson         
       patch                                                                   

Python 2.6 installer crashes when selecting 'advanced' and cance   10 days
       http://bugs.python.org/issue4289    loewis                    
                                                                               

email package with unicode subject/body                             8 days
       http://bugs.python.org/issue4306    barry                     
       patch                                                                   

Unicode in distutils meta-data?                                     4 days
       http://bugs.python.org/issue4312    loewis                    
                                                                               

IDLE segfault at exit                                               5 days
       http://bugs.python.org/issue4313    ocean-city                
                                                                               

Improper use of [] in configure.in leads to useless regexp in co    2 days
       http://bugs.python.org/issue4316    georg.brandl              
       patch                                                                   

Buffer overflow in imageop module                                   5 days
       http://bugs.python.org/issue4317    amaury.forgeotdarc        
       patch                                                                   

unintended syntax error with decorators, parenthesis, and dots?     1 days
       http://bugs.python.org/issue4321    georg.brandl              
                                                                               

locale documentation is inconsistent                                1 days
       http://bugs.python.org/issue4324    georg.brandl              
                                                                               

type of UserList instance returns class instead of instance         3 days
       http://bugs.python.org/issue4326    georg.brandl              
                                                                               

wsgiref.validate doesn't accept arguments to readline               1 days
       http://bugs.python.org/issue4330    ianb                      
                                                                               

asyncore.file_dispatcher does not use dup()'ed fd                   3 days
       http://bugs.python.org/issue4332    josiahcarlson             
       patch                                                                   

Mac Build Script is broken for 2.6 release                          0 days
       http://bugs.python.org/issue4334    benjamin.peterson         
                                                                               

Iteration over a map object with list()                             1 days
       http://bugs.python.org/issue4337    jmfauth                   
                                                                               

TypeError (bytes/str) in distutils command "upload"                 3 days
       http://bugs.python.org/issue4338    amaury.forgeotdarc        
       patch                                                                   

Fix set-like dictview doc                                           0 days
       http://bugs.python.org/issue4339    benjamin.peterson         
                                                                               

Update __hash__ doc                                                 0 days
       http://bugs.python.org/issue4341    benjamin.peterson         
                                                                               

crash upon launch                                                   0 days
       http://bugs.python.org/issue4344    loewis                    
                                                                               

bytearray methods returning self                                    1 days
       http://bugs.python.org/issue4348    benjamin.peterson         
       patch, needs review                                                     

sys.path includes extraneous junk                                   0 days
       http://bugs.python.org/issue4349    benjamin.peterson         
       patch, needs review                                                     

Move description what a trace should should return to settrace f    1 days
       http://bugs.python.org/issue4353    benjamin.peterson         
                                                                               

distutils.command.register is broken                                1 days
       http://bugs.python.org/issue4354    loewis                    
       patch                                                                   

Add "key" argument to "bisect" module functions                     1 days
       http://bugs.python.org/issue4356    marketdickinson           
                                                                               

frozen?set operations create incorrectly initialized instances o    0 days
       http://bugs.python.org/issue4357    amaury.forgeotdarc        
                                                                               

SystemError when method has both super() & closure                  0 days
       http://bugs.python.org/issue4360    barry                     
       patch                                                                   

FileIO object in io module                                          0 days
       http://bugs.python.org/issue4362    benjamin.peterson         
       patch, needs review                                                     

__cmp__ removal not in What's New                                   1 days
       http://bugs.python.org/issue4372    haypo                     
                                                                               

Using deiconify() hangs on exit                                  2646 days
       http://bugs.python.org/issue453489  amaury.forgeotdarc        
                                                                               

sys.stdout problems with pythonw.exe                             1616 days
       http://bugs.python.org/issue973507  amaury.forgeotdarc        
                                                                               

A subclass of set doesn't always have __init__ called.            547 days
       http://bugs.python.org/issue1721812 rhettinger                
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 15 datetime: define division timedelta/timedelta                    208 days
open    http://bugs.python.org/issue2706   

 12 Tkinter binding involving Control-spacebar raises unicode error    4 days
open    http://bugs.python.org/issue1028   

 10 TypeError (bytes/str) in distutils command "upload"                3 days
closed  http://bugs.python.org/issue4338   

 10 type of UserList instance returns class instead of instance        3 days
closed  http://bugs.python.org/issue4326   

 10 missing update() in _Screen.setup() of Lib/turtle.py              39 days
open    http://bugs.python.org/issue4117   

 10 Byte/string inconsistencies between different dbm modules          1 days
open    http://bugs.python.org/issue3799   

  8 FileIO object in io module                                         0 days
closed  http://bugs.python.org/issue4362   

  8 bytearray methods returning self                                   1 days
closed  http://bugs.python.org/issue4348   

  7 Reference leaks in Python 3.0rc3                                   1 days
open    http://bugs.python.org/issue4373   

  7 SystemError when method has both super() & closure                 0 days
closed  http://bugs.python.org/issue4360   




From barry at python.org  Fri Nov 21 21:26:01 2008
From: barry at python.org (Barry Warsaw)
Date: Fri, 21 Nov 2008 15:26:01 -0500
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <gg1sc8$tus$1@ger.gmane.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>
	<49233EF3.9040303@v.loewis.de>	<3C7340EB-C07C-4E0A-BA44-CB536012FEBF@python.org>	<49246687.2040204@v.loewis.de>
	<6B34AD98-187C-4876-A410-E0A843EEF2C4@python.org>
	<gg1sc8$tus$1@ger.gmane.org>
Message-ID: <697974D1-698C-4196-9B8A-132207E3D8B0@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 19, 2008, at 3:19 PM, Terry Reedy wrote:

>> Let's try this for 3.0rc4 then.
>
> The current release is rc2.  Skipping rc3 would confuse people'-)

Yeah, my calendar was wrong, but the PEP (and more importantly...  
code!) was right :).

There is nooooo rc4!
- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (Darwin)

iQCVAwUBSScZWXEjvBPtnXfVAQKpQAQAi9Q8rfgcCVXmQ2tIqaiAVKOQHDPQdfhF
lyDWHg+6i2EGrbs0Jju5GB9YML1yNga3X85zfQSedu6mgpA4dV6NvW988N3Wp4oG
ztDGT7yLxwYe4Wy606FF6lxSlXSvXQRLc/Nf1qgn8dDGskQKO2LZ+fUHW0BIWDBN
RFAuZqzdWQY=
=9Z8w
-----END PGP SIGNATURE-----

From ncoghlan at gmail.com  Sat Nov 22 00:52:42 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 22 Nov 2008 09:52:42 +1000
Subject: [Python-Dev] Adding zipfile and directory execution to What's New
	for Python 2.6.1?
Message-ID: <492749CA.3030507@gmail.com>

A trick we added to 2.6 that users seem to not notice is that it is now
possible to execute zipfiles and directories from the command line by
including a __main__.py module (that module will then be executed, but
more importantly, the zipfile or directory name will be added to the
beginning of sys.path so related modules can be found easily).

I went looking for this in the 2.6 What's New and couldn't find anything
about it (certainly not in the most obvious section - the one that
describes the other changes to the interpreter command line options and
environment variables).

Did I just miss something? Or should more on this be added to the What's
New for 2.6.1?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From musiccomposition at gmail.com  Sat Nov 22 01:28:59 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Fri, 21 Nov 2008 18:28:59 -0600
Subject: [Python-Dev] Adding zipfile and directory execution to What's
	New for Python 2.6.1?
In-Reply-To: <492749CA.3030507@gmail.com>
References: <492749CA.3030507@gmail.com>
Message-ID: <1afaf6160811211628n4b0fcd7dh42af00f63b82b291@mail.gmail.com>

On Fri, Nov 21, 2008 at 5:52 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> A trick we added to 2.6 that users seem to not notice is that it is now
> possible to execute zipfiles and directories from the command line by
> including a __main__.py module (that module will then be executed, but
> more importantly, the zipfile or directory name will be added to the
> beginning of sys.path so related modules can be found easily).
>
> I went looking for this in the 2.6 What's New and couldn't find anything
> about it (certainly not in the most obvious section - the one that
> describes the other changes to the interpreter command line options and
> environment variables).
>
> Did I just miss something? Or should more on this be added to the What's
> New for 2.6.1?

Feel free to add it! There were so many new things in 2.6, that we
were bound to miss a few things.

>
> Cheers,
> Nick.



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From dinov at microsoft.com  Sat Nov 22 01:51:50 2008
From: dinov at microsoft.com (Dino Viehland)
Date: Fri, 21 Nov 2008 16:51:50 -0800
Subject: [Python-Dev] format specification mini-language docs...
Message-ID: <350E7D38B6D819428718949920EC235556486D00A7@NA-EXMSG-C102.redmond.corp.microsoft.com>

Ok, now I'm implementing __format__ support for IronPython.  The format spec mini-language docs say that a presentation type of None is the same as 'g' for floating point / decimal values.  But these two formats seem to differ based upon how they handle whole numbers:

>>> 2.0.__format__('')
'2.0'
>>> 2.0.__format__('g')
'2'

The docs also say that 'g' prints it as fixed point unless the number is too large.  But the fixed point format differs from what 'f' would print.  I guess it didn't say they'd both print it as fixed point w/ a precision of 6 or anything but it seems a little unclear.

>>> 2.0.__format__('g')
'2'
>>> 2.0.__format__('f')
'2.000000'

Finally providing any sign character seems to cause +1.0#INF and friends to be returned instead of inf as is documented:

>>> 10e667.__format__('+')
'+1.0#INF'
>>> 10e667.__format__('')
'inf'


Are these just doc bugs?  The inf issue is the only one that seems particularly weird to me.

From prologic at shortcircuit.net.au  Sat Nov 22 02:17:26 2008
From: prologic at shortcircuit.net.au (James Mills)
Date: Sat, 22 Nov 2008 11:17:26 +1000
Subject: [Python-Dev] RELEASED Python 3.0rc3
In-Reply-To: <CCD64757-114A-4B3B-9427-4E8BA6F6FCD3@python.org>
References: <CCD64757-114A-4B3B-9427-4E8BA6F6FCD3@python.org>
Message-ID: <e1a84d570811211717w3235b9dbl2e42abb8512bc0ac@mail.gmail.com>

On Sat, Nov 22, 2008 at 1:06 AM, Barry Warsaw <barry at python.org> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On behalf of the Python development team and the Python community, I am
> happy to announce the third and last planned release candidate for Python
> 3.0.

Whoohoo! :) Great works guys!

--JamesMills

-- 
--
-- "Problems are solved by method"

From eric at trueblade.com  Sat Nov 22 03:03:43 2008
From: eric at trueblade.com (Eric Smith)
Date: Fri, 21 Nov 2008 21:03:43 -0500
Subject: [Python-Dev] format specification mini-language docs...
In-Reply-To: <350E7D38B6D819428718949920EC235556486D00A7@NA-EXMSG-C102.redmond.corp.microsoft.com>
References: <350E7D38B6D819428718949920EC235556486D00A7@NA-EXMSG-C102.redmond.corp.microsoft.com>
Message-ID: <4927687F.5030901@trueblade.com>

Dino Viehland wrote:
> Ok, now I'm implementing __format__ support for IronPython.  The format spec mini-language docs say that a presentation type of None is the same as 'g' for floating point / decimal values.  

Awesome! Thanks for doing this.

 >But these two formats seem to differ based upon how they handle whole 
numbers:
>>>> 2.0.__format__('')
> '2.0'
>>>> 2.0.__format__('g')
> '2'

I think the docs are wrong. For floats, the PEP 
(http://www.python.org/dev/peps/pep-3101/) says:

'' (None) - similar to 'g', except that it prints at least one digit 
after the decimal point.

> The docs also say that 'g' prints it as fixed point unless the number is too large.  But the fixed point format differs from what 'f' would print.  I guess it didn't say they'd both print it as fixed point w/ a precision of 6 or anything but it seems a little unclear.
> 
>>>> 2.0.__format__('g')
> '2'
>>>> 2.0.__format__('f')
> '2.000000'

This is to be compatible with %-formatting:
$ ./python
Python 2.7a0 (trunk:67325, Nov 21 2008, 20:35:33)
[GCC 4.1.2 20070626 (Red Hat 4.1.2-13)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> '%f %g' % (2.0, 2.0)
'2.000000 2'

> Finally providing any sign character seems to cause +1.0#INF and friends to be returned instead of inf as is documented:
> 
>>>> 10e667.__format__('+')
> '+1.0#INF'
>>>> 10e667.__format__('')
> 'inf'

Yes, that does seem odd. Let me look at it a bit and I'll comment on it, 
hopefully this weekend. I have a pending fix for 2.7/3.1 to make inf 
handling more consistent across platforms, it might take care of this 
case, too.

> Are these just doc bugs?  The inf issue is the only one that seems particularly weird to me.

Agreed.

Eric.

From dalcinl at gmail.com  Sat Nov 22 04:04:19 2008
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sat, 22 Nov 2008 00:04:19 -0300
Subject: [Python-Dev] Issue 4195: Can't execute packages with -m in
	Python 2.6/3.0
In-Reply-To: <ca471dc20811180751k317e9b6ey5a7dbffe2379b6ac@mail.gmail.com>
References: <49229F36.5070206@gmail.com>
	<ca471dc20811180751k317e9b6ey5a7dbffe2379b6ac@mail.gmail.com>
Message-ID: <e7ba66e40811211904q743e8f71r895e78883333e56a@mail.gmail.com>

Unless this is considered a regression...

On Tue, Nov 18, 2008 at 12:51 PM, Guido van Rossum <guido at python.org> wrote:
> I think it crosses the line.
>
> On Tue, Nov 18, 2008 at 2:55 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I need a release manager call on whether or not the proposed resolution
>> of this issue goes beyond what is acceptable for a bug fix in 2.6.1.
>>
>> Short version:
>> - Python 2.5 allowed packages to be executed with -m, but in a broken way
>> - I disabled the broken way for 2.6, but didn't provide a replacement
>> - The patch attached to 4195 once again allows execution of packages
>> with -m, but in a clean way similar to the way directories and zipfiles
>> can now be executed
>> - I would like to commit that patch for 3.0/2.6.1, but I'm concerned
>> that it crosses the "no new features" line
>>
>> Long version:
>>
>> There was a bug in python 2.5 that allowed a package name to be passed
>> to the -m switch or runpy.run_module() and it would mostly work.
>>
>> However, the 'mostly' was due to the fact that doing this put the
>> internal import machinery into a slightly inconsistent state: the
>> interpreter was running code from inside a package, but that package
>> wasn't actually present in sys.modules. This could lead to assorted hard
>> to trace import-related weirdness, similar to what you can get when
>> executing a file from inside a package by name. You would often get away
>> with it, but good luck figuring out what is happening if things go wrong
>> (and you aren't already an expert on Python import mechanics).
>>
>> Since the ability to execute packages wasn't intentional, I added the
>> missing check to block it explicitly in 2.6 (but left it alone for 2.5).
>> It seemed like a really niche feature, so I didn't figure out a clean
>> replacement for it at the time.
>>
>> Benjamin noticed this change earlier in the 2.6 release cycle, I pointed
>> out it was a deliberate change, and that's the way it stayed until after
>> 2.6 was released.
>>
>> After the release, Andi Vajda (from the JCC project [1]) contacted me,
>> and together we worked out a better implementation of package support
>> for the -m switch (and runpy.run_module) [2] by extending the
>> __main__.py approach used to support direct execution of zipfiles and
>> directories [3].
>>
>> That's what I would like to add, since it nicely complements the ability
>> to execute zipfiles and directories, while also restoring the ability to
>> pass a package name to the -m switch (but in a way that keeps the import
>> machinery in a consistent state).
>>
>> Cheers,
>> Nick.
>>
>> [1] http://pypi.python.org/pypi/JCC
>>
>> [2] http://bugs.python.org/issue4195 (package execution with -m)
>> [3] http://bugs.python.org/issue1739468 (zipfile execution)
>>
>> --
>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>> ---------------------------------------------------------------
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From guido at python.org  Sat Nov 22 04:16:01 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 21 Nov 2008 19:16:01 -0800
Subject: [Python-Dev] Issue 4195: Can't execute packages with -m in
	Python 2.6/3.0
In-Reply-To: <e7ba66e40811211904q743e8f71r895e78883333e56a@mail.gmail.com>
References: <49229F36.5070206@gmail.com>
	<ca471dc20811180751k317e9b6ey5a7dbffe2379b6ac@mail.gmail.com>
	<e7ba66e40811211904q743e8f71r895e78883333e56a@mail.gmail.com>
Message-ID: <ca471dc20811211916t499335f1j6d0856fba804901e@mail.gmail.com>

I don't think that removing an unintentional and subtly broken feature
is a regression.

On Fri, Nov 21, 2008 at 7:04 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> Unless this is considered a regression...
>
> On Tue, Nov 18, 2008 at 12:51 PM, Guido van Rossum <guido at python.org> wrote:
>> I think it crosses the line.
>>
>> On Tue, Nov 18, 2008 at 2:55 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> I need a release manager call on whether or not the proposed resolution
>>> of this issue goes beyond what is acceptable for a bug fix in 2.6.1.
>>>
>>> Short version:
>>> - Python 2.5 allowed packages to be executed with -m, but in a broken way
>>> - I disabled the broken way for 2.6, but didn't provide a replacement
>>> - The patch attached to 4195 once again allows execution of packages
>>> with -m, but in a clean way similar to the way directories and zipfiles
>>> can now be executed
>>> - I would like to commit that patch for 3.0/2.6.1, but I'm concerned
>>> that it crosses the "no new features" line
>>>
>>> Long version:
>>>
>>> There was a bug in python 2.5 that allowed a package name to be passed
>>> to the -m switch or runpy.run_module() and it would mostly work.
>>>
>>> However, the 'mostly' was due to the fact that doing this put the
>>> internal import machinery into a slightly inconsistent state: the
>>> interpreter was running code from inside a package, but that package
>>> wasn't actually present in sys.modules. This could lead to assorted hard
>>> to trace import-related weirdness, similar to what you can get when
>>> executing a file from inside a package by name. You would often get away
>>> with it, but good luck figuring out what is happening if things go wrong
>>> (and you aren't already an expert on Python import mechanics).
>>>
>>> Since the ability to execute packages wasn't intentional, I added the
>>> missing check to block it explicitly in 2.6 (but left it alone for 2.5).
>>> It seemed like a really niche feature, so I didn't figure out a clean
>>> replacement for it at the time.
>>>
>>> Benjamin noticed this change earlier in the 2.6 release cycle, I pointed
>>> out it was a deliberate change, and that's the way it stayed until after
>>> 2.6 was released.
>>>
>>> After the release, Andi Vajda (from the JCC project [1]) contacted me,
>>> and together we worked out a better implementation of package support
>>> for the -m switch (and runpy.run_module) [2] by extending the
>>> __main__.py approach used to support direct execution of zipfiles and
>>> directories [3].
>>>
>>> That's what I would like to add, since it nicely complements the ability
>>> to execute zipfiles and directories, while also restoring the ability to
>>> pass a package name to the -m switch (but in a way that keeps the import
>>> machinery in a consistent state).
>>>
>>> Cheers,
>>> Nick.
>>>
>>> [1] http://pypi.python.org/pypi/JCC
>>>
>>> [2] http://bugs.python.org/issue4195 (package execution with -m)
>>> [3] http://bugs.python.org/issue1739468 (zipfile execution)
>>>
>>> --
>>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>>> ---------------------------------------------------------------
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
>>
>
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Sat Nov 22 05:40:14 2008
From: python at rcn.com (Raymond Hettinger)
Date: Fri, 21 Nov 2008 20:40:14 -0800
Subject: [Python-Dev] Issue 4195: Can't execute packages with -m
	inPython 2.6/3.0
References: <49229F36.5070206@gmail.com><ca471dc20811180751k317e9b6ey5a7dbffe2379b6ac@mail.gmail.com><e7ba66e40811211904q743e8f71r895e78883333e56a@mail.gmail.com>
	<ca471dc20811211916t499335f1j6d0856fba804901e@mail.gmail.com>
Message-ID: <00792C7938CC44F99CF5A2EB9333916D@RaymondLaptop1>

In concur that it is not a regression (esp for Py2.6).
OTOH, it would be nice to have -m run as expected.
It seems reasonable to me to get this working for 3.0.

Raymond


----- Original Message ----- 
From: "Guido van Rossum" <guido at python.org>
To: "Lisandro Dalcin" <dalcinl at gmail.com>
Cc: "Nick Coghlan" <ncoghlan at gmail.com>; "Python Dev" <python-dev at python.org>
Sent: Friday, November 21, 2008 7:16 PM
Subject: Re: [Python-Dev] Issue 4195: Can't execute packages with -m inPython 2.6/3.0


I don't think that removing an unintentional and subtly broken feature
is a regression.

On Fri, Nov 21, 2008 at 7:04 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> Unless this is considered a regression...
>
> On Tue, Nov 18, 2008 at 12:51 PM, Guido van Rossum <guido at python.org> wrote:
>> I think it crosses the line.
>>
>> On Tue, Nov 18, 2008 at 2:55 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> I need a release manager call on whether or not the proposed resolution
>>> of this issue goes beyond what is acceptable for a bug fix in 2.6.1.
>>>
>>> Short version:
>>> - Python 2.5 allowed packages to be executed with -m, but in a broken way
>>> - I disabled the broken way for 2.6, but didn't provide a replacement
>>> - The patch attached to 4195 once again allows execution of packages
>>> with -m, but in a clean way similar to the way directories and zipfiles
>>> can now be executed
>>> - I would like to commit that patch for 3.0/2.6.1, but I'm concerned
>>> that it crosses the "no new features" line
>>>
>>> Long version:
>>>
>>> There was a bug in python 2.5 that allowed a package name to be passed
>>> to the -m switch or runpy.run_module() and it would mostly work.
>>>
>>> However, the 'mostly' was due to the fact that doing this put the
>>> internal import machinery into a slightly inconsistent state: the
>>> interpreter was running code from inside a package, but that package
>>> wasn't actually present in sys.modules. This could lead to assorted hard
>>> to trace import-related weirdness, similar to what you can get when
>>> executing a file from inside a package by name. You would often get away
>>> with it, but good luck figuring out what is happening if things go wrong
>>> (and you aren't already an expert on Python import mechanics).
>>>
>>> Since the ability to execute packages wasn't intentional, I added the
>>> missing check to block it explicitly in 2.6 (but left it alone for 2.5).
>>> It seemed like a really niche feature, so I didn't figure out a clean
>>> replacement for it at the time.
>>>
>>> Benjamin noticed this change earlier in the 2.6 release cycle, I pointed
>>> out it was a deliberate change, and that's the way it stayed until after
>>> 2.6 was released.
>>>
>>> After the release, Andi Vajda (from the JCC project [1]) contacted me,
>>> and together we worked out a better implementation of package support
>>> for the -m switch (and runpy.run_module) [2] by extending the
>>> __main__.py approach used to support direct execution of zipfiles and
>>> directories [3].
>>>
>>> That's what I would like to add, since it nicely complements the ability
>>> to execute zipfiles and directories, while also restoring the ability to
>>> pass a package name to the -m switch (but in a way that keeps the import
>>> machinery in a consistent state).
>>>
>>> Cheers,
>>> Nick.
>>>
>>> [1] http://pypi.python.org/pypi/JCC
>>>
>>> [2] http://bugs.python.org/issue4195 (package execution with -m)
>>> [3] http://bugs.python.org/issue1739468 (zipfile execution)
>>>
>>> --
>>> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
>>> ---------------------------------------------------------------
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dalcinl%40gmail.com
>>
>
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/python%40rcn.com 


From vinay_sajip at yahoo.co.uk  Sat Nov 22 12:12:22 2008
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 22 Nov 2008 11:12:22 +0000 (UTC)
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
Message-ID: <loom.20081122T111103-661@post.gmane.org>

Brett Cannon has suggested [1] that the logging package should provide an
implementation of warnings.showwarning which redirects to logging. Here are my
first thoughts about how this might work:

A new function, showwarning( message, category, filename, lineno[, file]) will
be added to the logging package. To redirect all warnings to the logging system,
it will only be necessary to do the following:

warnings.showwarning = logging.showwarning

The implementation of logging.showwarning will call formatwarning(message,
category, filename, lineno) and will log the resulting string to a warnings
logger named "stdlib.warnings" (or perhaps "std.warnings") with level
logging.WARNING. The optional file argument to logging.showwarning is only for
signature compatibility with warnings.showwarning and will be ignored; in order
to configure logging of warnings to any particular destination, the logging
configuration will need to add appropriate handlers to the warnings logger. The
precise format of the logging message will be determined by the logging
configuration in effect, i.e. any formatters configured for the handlers
attached to the logger.

Does this sound like a reasonable approach? Can someone please suggest any
improvements, or let me know if I've missed anything?

Regards,


Vinay Sajip

[1] http://bugs.python.org/issue4384



From brett at python.org  Sat Nov 22 20:50:46 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 22 Nov 2008 11:50:46 -0800
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
In-Reply-To: <loom.20081122T111103-661@post.gmane.org>
References: <loom.20081122T111103-661@post.gmane.org>
Message-ID: <bbaeab100811221150p453d4f5dt3b836f835b8822d2@mail.gmail.com>

On Sat, Nov 22, 2008 at 03:12, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon has suggested [1] that the logging package should provide an
> implementation of warnings.showwarning which redirects to logging. Here are my
> first thoughts about how this might work:
>
> A new function, showwarning( message, category, filename, lineno[, file]) will
> be added to the logging package. To redirect all warnings to the logging system,
> it will only be necessary to do the following:
>
> warnings.showwarning = logging.showwarning
>
> The implementation of logging.showwarning will call formatwarning(message,
> category, filename, lineno) and will log the resulting string to a warnings
> logger named "stdlib.warnings" (or perhaps "std.warnings") with level
> logging.WARNING. The optional file argument to logging.showwarning is only for
> signature compatibility with warnings.showwarning and will be ignored; in order
> to configure logging of warnings to any particular destination, the logging
> configuration will need to add appropriate handlers to the warnings logger. The
> precise format of the logging message will be determined by the logging
> configuration in effect, i.e. any formatters configured for the handlers
> attached to the logger.
>
> Does this sound like a reasonable approach? Can someone please suggest any
> improvements, or let me know if I've missed anything?
>

Basically what I had in mind, so it sounds reasonable to me.

-Brett

From sidnei at enfoldsystems.com  Sat Nov 22 21:12:44 2008
From: sidnei at enfoldsystems.com (Sidnei da Silva)
Date: Sat, 22 Nov 2008 18:12:44 -0200
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
In-Reply-To: <loom.20081122T111103-661@post.gmane.org>
References: <loom.20081122T111103-661@post.gmane.org>
Message-ID: <a7a2b76b0811221212s4efae09ctdbcdab18376b990f@mail.gmail.com>

On Sat, Nov 22, 2008 at 9:12 AM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Brett Cannon has suggested [1] that the logging package should provide an
> implementation of warnings.showwarning which redirects to logging. Here are my
> first thoughts about how this might work:
>
> A new function, showwarning( message, category, filename, lineno[, file]) will
> be added to the logging package. To redirect all warnings to the logging system,
> it will only be necessary to do the following:
>
> warnings.showwarning = logging.showwarning
<snip>
>
> Does this sound like a reasonable approach? Can someone please suggest any
> improvements, or let me know if I've missed anything?

I do have one suggestion for improvement: instead of requiring the
person to do this monkey patching, add a new 'log' action to the
warnings filter as described by PEP 230 [1] (see: 'The Warnings
Filter' section). This way, changing the behavior of how warnings are
displayed (or not) is kept consistent and documented.

[1] http://www.python.org/dev/peps/pep-0230/

-- 
Sidnei da Silva
Enfold Systems
http://enfoldsystems.com
Fax +1 832 201 8856
Office +1 713 942 2377 Ext 214
Skype zopedc

From Scott.Daniels at Acm.Org  Sat Nov 22 23:19:41 2008
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sat, 22 Nov 2008 14:19:41 -0800
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
In-Reply-To: <loom.20081122T111103-661@post.gmane.org>
References: <loom.20081122T111103-661@post.gmane.org>
Message-ID: <4928857D.4080204@Acm.Org>

Vinay Sajip wrote:
> Brett Cannon has suggested [1] that the logging package should provide an
> implementation of warnings.showwarning which redirects to logging. Here are my
> first thoughts about how this might work:
> 
> A new function, showwarning( message, category, filename, lineno[, file]) will
> be added to the logging package. To redirect all warnings to the logging system,
> it will only be necessary to do the following:
> 
> warnings.showwarning = logging.showwarning
> 
> The implementation of logging.showwarning will call formatwarning(message,
> category, filename, lineno) and will log the resulting string to a warnings
> logger named "stdlib.warnings" (or perhaps "std.warnings") with level
> logging.WARNING. The optional file argument to logging.showwarning is only for
> signature compatibility with warnings.showwarning and will be ignored; in order
> to configure logging of warnings to any particular destination, the logging
> configuration will need to add appropriate handlers to the warnings logger. The
> precise format of the logging message will be determined by the logging
> configuration in effect, i.e. any formatters configured for the handlers
> attached to the logger.

Match the new warning protocol exactly:
     def showwarning(message, category, filename, lineno,
                     file=None, line=None):
         ...
     If the line is not None, use it (which will happen if you pass it
     along to showwarning).
     If the line is None and file is not None, use file to get to your
     line, rather than opening the filename which may not work.

and:
     def formatwarning(message, category, filename, lineno, line=None):
         ...


Having chased problems around in idle code for a bit, I can assure you
that more code than you suspect calls warnings.formatwarning and
warnings.showwarning.  If you get it wrong, a deprecation warning can
prevent a module like md5 from loading (for example).


--Scott David Daniels
Scott.Daniels at Acm.Org


From Scott.Daniels at Acm.Org  Sat Nov 22 23:44:32 2008
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sat, 22 Nov 2008 14:44:32 -0800
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
In-Reply-To: <4928857D.4080204@Acm.Org>
References: <loom.20081122T111103-661@post.gmane.org>
	<4928857D.4080204@Acm.Org>
Message-ID: <gga1qu$6kq$1@ger.gmane.org>

Scott David Daniels wrote:
 >
 > Match the new warning protocol exactly:
 >     def showwarning(message, category, filename, lineno,
 >                     file=None, line=None):
 >         ...
 >     If the line is not None, use it (which will happen if you pass it
 >     along to showwarning).
 >     If the line is None and file is not None, use file to get to your
 >     line, rather than opening the filename which may not work.
 >

Sorry, this had no sooner left my terminal than I realized that the file
argument is the output target, and nothing to do with fetching source.
I'd not put the data into the log if file is provided, because it may
be code trying to extract output formatting.

I'd do something like this:
     if file is not None:
         old_showwarning(message, category, filename, lineno, file, line)
         return
at the top of the code.


--Scott David Daniels
Scott.Daniels at Acm.Org


From vinay_sajip at yahoo.co.uk  Sun Nov 23 02:53:35 2008
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 23 Nov 2008 01:53:35 +0000 (UTC)
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
References: <loom.20081122T111103-661@post.gmane.org>
	<a7a2b76b0811221212s4efae09ctdbcdab18376b990f@mail.gmail.com>
Message-ID: <loom.20081123T014845-321@post.gmane.org>

Sidnei da Silva <sidnei <at> enfoldsystems.com> writes:

> I do have one suggestion for improvement: instead of requiring the
> person to do this monkey patching, add a new 'log' action to the
> warnings filter as described by PEP 230 [1] (see: 'The Warnings
> Filter' section). This way, changing the behavior of how warnings are
> displayed (or not) is kept consistent and documented.
> 
> [1] http://www.python.org/dev/peps/pep-0230/
> 

I see from PEP-230 the following under "Rejected Concerns" stated by GvR:

"Paul Prescod, Barry Warsaw and Fred Drake have brought up several additional
concerns that I feel aren't critical.  I address them here (the concerns are
paraphrased, not exactly their words):"

    [other concerns omitted]
    - Barry: I'd like to add my own warning action.  Maybe if `action'
      could be a callable as well as a string.  Then in my IDE, I
      could set that to "mygui.popupWarningsDialog".

      Response: For that purpose you would override warnings.showwarning().
    [other concerns omitted]

So, monkey-patching appears to be the suggested approach. Indeed, ISTM that the
showwarning function is there specifically for the purpose of being
monkey-patched out.

Regards,


Vinay Sajip


From brett at python.org  Sun Nov 23 03:58:59 2008
From: brett at python.org (Brett Cannon)
Date: Sat, 22 Nov 2008 18:58:59 -0800
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
In-Reply-To: <loom.20081123T014845-321@post.gmane.org>
References: <loom.20081122T111103-661@post.gmane.org>
	<a7a2b76b0811221212s4efae09ctdbcdab18376b990f@mail.gmail.com>
	<loom.20081123T014845-321@post.gmane.org>
Message-ID: <bbaeab100811221858n41038296j5ae0c71b911efa51@mail.gmail.com>

On Sat, Nov 22, 2008 at 17:53, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Sidnei da Silva <sidnei <at> enfoldsystems.com> writes:
>
>> I do have one suggestion for improvement: instead of requiring the
>> person to do this monkey patching, add a new 'log' action to the
>> warnings filter as described by PEP 230 [1] (see: 'The Warnings
>> Filter' section). This way, changing the behavior of how warnings are
>> displayed (or not) is kept consistent and documented.
>>
>> [1] http://www.python.org/dev/peps/pep-0230/
>>
>
> I see from PEP-230 the following under "Rejected Concerns" stated by GvR:
>
> "Paul Prescod, Barry Warsaw and Fred Drake have brought up several additional
> concerns that I feel aren't critical.  I address them here (the concerns are
> paraphrased, not exactly their words):"
>
>    [other concerns omitted]
>    - Barry: I'd like to add my own warning action.  Maybe if `action'
>      could be a callable as well as a string.  Then in my IDE, I
>      could set that to "mygui.popupWarningsDialog".
>
>      Response: For that purpose you would override warnings.showwarning().
>    [other concerns omitted]
>
> So, monkey-patching appears to be the suggested approach. Indeed, ISTM that the
> showwarning function is there specifically for the purpose of being
> monkey-patched out.
>

It is specifically there to be overridden (and as an aside, it was a
pain to support in the C port of warnings), so it really isn't
monkey-patching. =)

-Brett

From solipsis at pitrou.net  Sun Nov 23 12:27:50 2008
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 23 Nov 2008 11:27:50 +0000 (UTC)
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
References: <loom.20081122T111103-661@post.gmane.org>
	<a7a2b76b0811221212s4efae09ctdbcdab18376b990f@mail.gmail.com>
	<loom.20081123T014845-321@post.gmane.org>
	<bbaeab100811221858n41038296j5ae0c71b911efa51@mail.gmail.com>
Message-ID: <loom.20081123T112716-480@post.gmane.org>

Brett Cannon <brett <at> python.org> writes:
> 
> It is specifically there to be overridden (and as an aside, it was a
> pain to support in the C port of warnings), so it really isn't
> monkey-patching. =)

Should we coin a new word for this? Pretty-patching?

cheers

Antoine.



From ncoghlan at gmail.com  Sun Nov 23 12:38:32 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 23 Nov 2008 21:38:32 +1000
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
In-Reply-To: <loom.20081123T112716-480@post.gmane.org>
References: <loom.20081122T111103-661@post.gmane.org>	<a7a2b76b0811221212s4efae09ctdbcdab18376b990f@mail.gmail.com>	<loom.20081123T014845-321@post.gmane.org>	<bbaeab100811221858n41038296j5ae0c71b911efa51@mail.gmail.com>
	<loom.20081123T112716-480@post.gmane.org>
Message-ID: <492940B8.1060905@gmail.com>

Antoine Pitrou wrote:
> Brett Cannon <brett <at> python.org> writes:
>> It is specifically there to be overridden (and as an aside, it was a
>> pain to support in the C port of warnings), so it really isn't
>> monkey-patching. =)
> 
> Should we coin a new word for this? Pretty-patching?

When the module docs tell you it's OK, it's just overriding the
function. If the docs don't tell you to do it, it's monkey patching :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From tseaver at palladion.com  Sun Nov 23 14:15:57 2008
From: tseaver at palladion.com (Tres Seaver)
Date: Sun, 23 Nov 2008 08:15:57 -0500
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
In-Reply-To: <492940B8.1060905@gmail.com>
References: <loom.20081122T111103-661@post.gmane.org>	<a7a2b76b0811221212s4efae09ctdbcdab18376b990f@mail.gmail.com>	<loom.20081123T014845-321@post.gmane.org>	<bbaeab100811221858n41038296j5ae0c71b911efa51@mail.gmail.com>	<loom.20081123T112716-480@post.gmane.org>
	<492940B8.1060905@gmail.com>
Message-ID: <4929578D.2010000@palladion.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Nick Coghlan wrote:
> Antoine Pitrou wrote:
>> Brett Cannon <brett <at> python.org> writes:
>>> It is specifically there to be overridden (and as an aside, it was a
>>> pain to support in the C port of warnings), so it really isn't
>>> monkey-patching. =)
>> Should we coin a new word for this? Pretty-patching?
> 
> When the module docs tell you it's OK, it's just overriding the
> function. If the docs don't tell you to do it, it's monkey patching :)

Monkey-patching is never inheritance-based:  it means replacing a method
/ attribute of a module or class from a different module, after import time.


Tres.
- --
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJKVeN+gerLs4ltQ4RAvWKAJ47URWBPaqgRoA6LUZ7d1pOuAk9nQCgqBk7
1gxVqwbnbfSzFDUE0wwfyjg=
=Yv7d
-----END PGP SIGNATURE-----


From sargentd at die-welt.net  Mon Nov 24 01:44:43 2008
From: sargentd at die-welt.net (Evgeni Golov)
Date: Mon, 24 Nov 2008 01:44:43 +0100
Subject: [Python-Dev] mlockall() in Python?
Message-ID: <ggctdr$hnb$2@ger.gmane.org>

Hi,

I'd like to write a small daemon in Python, which should never be
swapped out (on Linux, this daemon will be Linux specific, so no need
in a platform-independent solution).

In C I'd do:
#include <sys/mman.h>
mlockall(MCL_FUTURE);
//do stuff here
munlockall();

Is there anything similar in Python?

TIA & kind regards
Evgeni Golov


From ggpolo at gmail.com  Mon Nov 24 02:31:15 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Sun, 23 Nov 2008 23:31:15 -0200
Subject: [Python-Dev] mlockall() in Python?
In-Reply-To: <ggctdr$hnb$2@ger.gmane.org>
References: <ggctdr$hnb$2@ger.gmane.org>
Message-ID: <ac2200130811231731l617e770en483c1ff95b4d9209@mail.gmail.com>

On Sun, Nov 23, 2008 at 10:44 PM, Evgeni Golov <sargentd at die-welt.net> wrote:
> Hi,
>
> I'd like to write a small daemon in Python, which should never be
> swapped out (on Linux, this daemon will be Linux specific, so no need
> in a platform-independent solution).
>
> In C I'd do:
> #include <sys/mman.h>
> mlockall(MCL_FUTURE);
> //do stuff here
> munlockall();
>
> Is there anything similar in Python?
>

Python doesn't wrap those function itself, but maybe some extension
modules does (or you could wrap it a second:
http://pastebin.com/m3286272c http://pastebin.com/m33ca908d)

Also, this list is for development *of* Python, not development *with*
Python (one of the most used messages this month).

> TIA & kind regards
> Evgeni Golov
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves

From victor.stinner at haypocalc.com  Mon Nov 24 02:37:59 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Mon, 24 Nov 2008 02:37:59 +0100
Subject: [Python-Dev] subprocess.Popen: change default buffer size?
Message-ID: <200811240237.59288.victor.stinner@haypocalc.com>

subprocess.Popen is much SLOWER than os.popen() on Mac and Solaris.

The goal is to read the output of a command (through a pipe) as fast as 
possible. The problem is the pipe buffering (the reader file in the Python 
process).

Today, subprocess.Popen() uses bufsize=0 by default. It should be 
bufsize=(-1): use the system default buffer size.

==> http://bugs.python.org/issue4194

Note: On Linux the performances between subprocess (unbuffered) and popen() 
(buffered) are the same, which is quite strange.

Victor

From mrts at mrts.pri.ee  Mon Nov 24 13:05:11 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Mon, 24 Nov 2008 14:05:11 +0200
Subject: [Python-Dev] CVE tracking
Message-ID: <492A9877.3070009@mrts.pri.ee>

I created a script that parses the
http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=python
Python-related CVE list and classifies the CVEs as follows:

 * "ok" -- CVE has references to bugs.python.org

 * "warnings" -- CVE has references to Python SVN revisions
   or an issue in bugs.python.org refers to it (i.e. the probelm is
   probably fixed, but the CVE should really be updated to link
   to the issue that is probably listed in bugs.python.org)

 * "errors" -- CVE does have no references to Python issues or SVN
   nor does any issue in bugs.python.org have references to the CVE id

The script is at http://dpaste.com/hold/92930/
The results are at http://dpaste.com/hold/92929/

There were 35 errors, 8 warnings, 5 CVEs were OK.

In an ideal world, the references would be symmetric, i.e. every
Python-related CVE would have references to one or more issues in
bugs.python.org and these issues would also refer back to the CVE id.

###

As for the rmtree problem that Gisle Aas raised, this seems to apply
as of Python 2.6:

---

# emulate removing /etc
$ sudo cp -a /etc /root/etc/
$ sudo python2.6
 >>> for i in xrange(0, 50000):
...      with open("/root/etc/" + str(i), "w") as f:
...             f.write("0")
...
$ ls /root/etc > orig_list.txt

$ mkdir /tmp/attack
$ cp -a /root/etc/* /tmp/attack

$ sudo python2.6
 >>> from shutil import rmtree
 >>> rmtree('/tmp/attack')
 >>> # press ctrl-z to suspend execution
^Z
[1]+  Stopped                 sudo python2.6

$ mv /tmp/attack /tmp/dummy; ln -s /root/etc /tmp/attack
$ fg
sudo python2.6
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.6/shutil.py", line 225, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/usr/local/lib/python2.6/shutil.py", line 223, in rmtree
    os.rmdir(path)
OSError: [Errno 20] Not a directory: '/tmp/attack'

$ ls /root/etc > new_list.txt
$ diff -q orig_list.txt new_list.txt
Files orig_list.txt and new_list.txt differ

---

If the attack weren't possible, the lists would not differ.

From tjreedy at udel.edu  Mon Nov 24 16:44:04 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 24 Nov 2008 10:44:04 -0500
Subject: [Python-Dev] CVE tracking
In-Reply-To: <492A9877.3070009@mrts.pri.ee>
References: <492A9877.3070009@mrts.pri.ee>
Message-ID: <ggei41$jao$1@ger.gmane.org>

Mart Somermaa wrote:
> I created a script that parses the
> http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=python
> Python-related CVE list and classifies the CVEs as follows:
> 
> * "ok" -- CVE has references to bugs.python.org
> 
> * "warnings" -- CVE has references to Python SVN revisions
>   or an issue in bugs.python.org refers to it (i.e. the probelm is
>   probably fixed, but the CVE should really be updated to link
>   to the issue that is probably listed in bugs.python.org)
> 
> * "errors" -- CVE does have no references to Python issues or SVN
>   nor does any issue in bugs.python.org have references to the CVE id
> 
> The script is at http://dpaste.com/hold/92930/
> The results are at http://dpaste.com/hold/92929/
> 
> There were 35 errors, 8 warnings, 5 CVEs were OK.
> 
> In an ideal world, the references would be symmetric, i.e. every
> Python-related CVE would have references to one or more issues in
> bugs.python.org and these issues would also refer back to the CVE id.

When I looked through that list a week or so ago, I noticed that some 
issues were obviously related to the Python distribution itself, but 
others were appeared to be Python application problems.  It is not an 
'error' for the latter to have no reference to or from bugs.python.org. 
I suspect human perusal is need to make the determination.


From jeremy at alum.mit.edu  Mon Nov 24 17:58:12 2008
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 24 Nov 2008 11:58:12 -0500
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <20081121170645.3954E78534@psf.upfronthosting.co.za>
References: <20081121170645.3954E78534@psf.upfronthosting.co.za>
Message-ID: <e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>

I wanted to ask a policy question on the bug tracker.  What are we
doing with bugs filed against Python 2.4?
This bug http://bugs.python.org/issue1208304 reports a fd leak in
Python 2.4, which doesn't exist in the head.  Since Python 2.4 is in
security-fix only mode, is it fair to close this with a won't fix?

A meta-question:  Is there a document that has any tips on how to
handle bug reports?  It might be helpful to have a list of hints or
policy decisions to help people decide what to do with bugs.

Jeremy

On Fri, Nov 21, 2008 at 12:06 PM, Python tracker <status at bugs.python.org> wrote:
>
> ACTIVITY SUMMARY (11/14/08 - 11/21/08)
> Python tracker at http://bugs.python.org/
>
> To view or respond to any of the issues listed below, click on the issue
> number.  Do NOT respond to this message.
>
>
>  2194 open (+35) / 14007 closed (+18) / 16201 total (+53)
>
> Open issues with patches:   728
>
> Average duration of open issues: 712 days.
> Median duration of open issues: 2043 days.
>
> Open Issues Breakdown
>   open  2177 (+35)
> pending    17 ( +0)
>
> Issues Created Or Reopened (56)
> _______________________________
>
> Tkinter binding involving Control-spacebar raises unicode error  11/18/08
>       http://bugs.python.org/issue1028    reopened ocean-city
>       patch
>
> Byte/string inconsistencies between different dbm modules        11/21/08
>       http://bugs.python.org/issue3799    reopened brett.cannon
>       patch, needs review
>
> type of UserList instance returns class instead of instance      11/14/08
> CLOSED http://bugs.python.org/issue4326    created  chafporte
>
>
> Patch: simplify complex constant assignment statements           11/14/08
>       http://bugs.python.org/issue4327    created  novalis_dt
>       patch
>
> "?" in u"foo" raises a misleading error                         11/15/08
>       http://bugs.python.org/issue4328    created  ezio.melotti
>
>
> base64 does not properly handle unicode strings                  11/15/08
>       http://bugs.python.org/issue4329    created  mbecker
>
>
> wsgiref.validate doesn't accept arguments to readline            11/16/08
> CLOSED http://bugs.python.org/issue4330    created  ianb
>
>
> Can't use _functools.partial() created function as method        11/16/08
>       http://bugs.python.org/issue4331    created  ssadler
>
>
> asyncore.file_dispatcher does not use dup()'ed fd                11/17/08
> CLOSED http://bugs.python.org/issue4332    created  christianbecke
>       patch
>
> Reworked Dialog.py                                               11/17/08
>       http://bugs.python.org/issue4333    created  gpolo
>       patch
>
> Mac Build Script is broken for 2.6 release                       11/17/08
> CLOSED http://bugs.python.org/issue4334    created  ngift
>
>
> inspect.getsourcelines ignores last line in module               11/17/08
>       http://bugs.python.org/issue4335    created  RafeSacks
>
>
> Fix performance issues in xmlrpclib                              11/17/08
>       http://bugs.python.org/issue4336    created  krisvale
>       patch, patch, easy
>
> Iteration over a map object with list()                          11/17/08
> CLOSED http://bugs.python.org/issue4337    created  jmfauth
>
>
> TypeError (bytes/str) in distutils command "upload"              11/17/08
> CLOSED http://bugs.python.org/issue4338    created  hagen
>       patch
>
> Fix set-like dictview doc                                        11/17/08
> CLOSED http://bugs.python.org/issue4339    created  tjreedy
>
>
> xmlrpc.client - default 'SlowParser' not defined                 11/17/08
>       http://bugs.python.org/issue4340    created  mwatkins
>
>
> Update __hash__ doc                                              11/17/08
> CLOSED http://bugs.python.org/issue4341    created  tjreedy
>
>
> (Tkinter) Please backport these                                  11/18/08
>       http://bugs.python.org/issue4342    created  gpolo
>
>
> New function in Tkinter.py: setup_master                         11/18/08
>       http://bugs.python.org/issue4343    created  gpolo
>       patch
>
> crash upon launch                                                11/18/08
> CLOSED http://bugs.python.org/issue4344    created  source.mod
>
>
> Implement nb_nonzero for PyTclObject                             11/18/08
>       http://bugs.python.org/issue4345    created  gpolo
>       patch
>
> PyObject_CallMethod changes the exception message already set by 11/18/08
>       http://bugs.python.org/issue4346    created  gpolo
>       patch
>
> Dependencies of graminit.h are not rebuilt when the file is rege 11/18/08
>       http://bugs.python.org/issue4347    created  thomas.lee
>       patch
>
> bytearray methods returning self                                 11/18/08
> CLOSED http://bugs.python.org/issue4348    created  DinoV
>       patch, needs review
>
> sys.path includes extraneous junk                                11/18/08
> CLOSED http://bugs.python.org/issue4349    created  exarkun
>       patch, needs review
>
> Remove dead code from Tkinter.py                                 11/19/08
>       http://bugs.python.org/issue4350    created  gpolo
>       patch
>
> [PATCH] Better stacklevel for GzipFile.filename DeprecationWarni 11/19/08
>       http://bugs.python.org/issue4351    created  pjenvey
>       patch
>
> imp.find_module() fails with a UnicodeDecodeError when called wi 11/19/08
>       http://bugs.python.org/issue4352    created  Jukka Aho
>
>
> Move description what a trace should should return to settrace f 11/19/08
> CLOSED http://bugs.python.org/issue4353    created  rocky
>
>
> distutils.command.register is broken                             11/19/08
> CLOSED http://bugs.python.org/issue4354    created  hagen
>       patch
>
> Error in docs of urllib.request and urllib.parse                 11/19/08
>       http://bugs.python.org/issue4355    created  hagen
>       patch
>
> Add "key" argument to "bisect" module functions                  11/19/08
> CLOSED http://bugs.python.org/issue4356    created  tebeka
>
>
> frozen?set operations create incorrectly initialized instances o 11/19/08
> CLOSED http://bugs.python.org/issue4357    created  alexhsamuel
>
>
> Segfault in stringobject.c                                       11/19/08
>       http://bugs.python.org/issue4358    created  farshad
>
>
> at runtime, distutils uses buildtime files                       11/19/08
>       http://bugs.python.org/issue4359    created  a.badger
>
>
> SystemError when method has both super() & closure               11/20/08
> CLOSED http://bugs.python.org/issue4360    created  kaizhu
>       patch
>
> Docstring for "Lib/string.py" is outdated                        11/20/08
>       http://bugs.python.org/issue4361    created  thp
>       patch
>
> FileIO object in io module                                       11/20/08
> CLOSED http://bugs.python.org/issue4362    reopened barry
>       patch, needs review
>
> Make uuid module functions usable without ctypes                 11/20/08
>       http://bugs.python.org/issue4363    created  hdima
>       patch
>
> error in multiprocessing docs - rawvalue                         11/20/08
>       http://bugs.python.org/issue4364    created  dursobr
>
>
> Add CRT version info in msvcrt module                            11/20/08
>       http://bugs.python.org/issue4365    created  cdavid
>       patch
>
> cannot find -lpython2.5 when buinding Python 2.5.2 on FreeBSD 4. 11/20/08
>       http://bugs.python.org/issue4366    created  akitada
>
>
> segmentation fault in ast_for_atom                               11/20/08
>       http://bugs.python.org/issue4367    created  ot
>       patch, needs review
>
> A bug in ncurses.h still exists in FreeBSD 4.9 - 4.11            11/20/08
>       http://bugs.python.org/issue4368    created  akitada
>       patch
>
> Error building to a nonstandard prefix (with patch)              11/20/08
>       http://bugs.python.org/issue4369    created  ot
>       patch
>
> warning: unknown conversion type character `z' in format         11/20/08
>       http://bugs.python.org/issue4370    created  akitada
>
>
> coerce gone---but not from docs                                  11/20/08
>       http://bugs.python.org/issue4371    created  LambertDW
>
>
> __cmp__ removal not in What's New                                11/20/08
> CLOSED http://bugs.python.org/issue4372    created  tjreedy
>
>
> Reference leaks in Python 3.0rc3                                 11/21/08
>       http://bugs.python.org/issue4373    created  christian.heimes
>       patch
>
> Pickle tests fail w/o _pickle extension                          11/21/08
>       http://bugs.python.org/issue4374    created  christian.heimes
>
>
> inspect.getsource doesn't work with PYTHONPATH and source compil 11/21/08
>       http://bugs.python.org/issue4375    created  erickt
>
>
> Nested ctypes 'BigEndianStructure' fails                         11/21/08
>       http://bugs.python.org/issue4376    created  castironpi
>
>
> tokenize.detect_encoding() and Mac newline                       11/21/08
>       http://bugs.python.org/issue4377    created  haypo
>
>
> howto doc update                                                 11/21/08
>       http://bugs.python.org/issue4378    created  LambertDW
>
>
> Fix exception pickling: Move initial args assignment to BaseExce 11/18/08
>       http://bugs.python.org/issue1692335 reopened gvanrossum
>       patch
>
>
>
> Issues Now Closed (40)
> ______________________
>
> Problem with doctest and decorated functions                      441 days
>       http://bugs.python.org/issue1108    skip.montanaro
>
>
> conditional jump to a POP_TOP optimization                        253 days
>       http://bugs.python.org/issue2260    rhettinger
>       patch
>
> NULL member in modules_by_index                                   131 days
>       http://bugs.python.org/issue3327    loewis
>       patch, easy
>
> test_cpickle crash on AMD64 Windows build                          88 days
>       http://bugs.python.org/issue3640    loewis
>       patch, 64bit
>
> pressing HOME key in IDLE editor ends IDLE                         81 days
>       http://bugs.python.org/issue3679    tjreedy
>
>
> configure --with-threads on cygwin => crash on thread related te   58 days
>       http://bugs.python.org/issue3947    ocean-city
>       patch
>
> ignored exceptions in generators (regression?)                     43 days
>       http://bugs.python.org/issue4040    benjamin.peterson
>
>
> Document the effects of NotImplemented on == and !=                43 days
>       http://bugs.python.org/issue4087    marketdickinson
>
>
> Documenting set comparisons and operations                         39 days
>       http://bugs.python.org/issue4090    rhettinger
>       patch
>
> name conflict in ScrolledCanvas.__init__() in Lib/turtle.py        36 days
>       http://bugs.python.org/issue4116    loewis
>       patch
>
> "__getattr__" can't be a descriptor                                19 days
>       http://bugs.python.org/issue4230    benjamin.peterson
>       patch
>
> Python 2.6 installer crashes when selecting 'advanced' and cance   10 days
>       http://bugs.python.org/issue4289    loewis
>
>
> email package with unicode subject/body                             8 days
>       http://bugs.python.org/issue4306    barry
>       patch
>
> Unicode in distutils meta-data?                                     4 days
>       http://bugs.python.org/issue4312    loewis
>
>
> IDLE segfault at exit                                               5 days
>       http://bugs.python.org/issue4313    ocean-city
>
>
> Improper use of [] in configure.in leads to useless regexp in co    2 days
>       http://bugs.python.org/issue4316    georg.brandl
>       patch
>
> Buffer overflow in imageop module                                   5 days
>       http://bugs.python.org/issue4317    amaury.forgeotdarc
>       patch
>
> unintended syntax error with decorators, parenthesis, and dots?     1 days
>       http://bugs.python.org/issue4321    georg.brandl
>
>
> locale documentation is inconsistent                                1 days
>       http://bugs.python.org/issue4324    georg.brandl
>
>
> type of UserList instance returns class instead of instance         3 days
>       http://bugs.python.org/issue4326    georg.brandl
>
>
> wsgiref.validate doesn't accept arguments to readline               1 days
>       http://bugs.python.org/issue4330    ianb
>
>
> asyncore.file_dispatcher does not use dup()'ed fd                   3 days
>       http://bugs.python.org/issue4332    josiahcarlson
>       patch
>
> Mac Build Script is broken for 2.6 release                          0 days
>       http://bugs.python.org/issue4334    benjamin.peterson
>
>
> Iteration over a map object with list()                             1 days
>       http://bugs.python.org/issue4337    jmfauth
>
>
> TypeError (bytes/str) in distutils command "upload"                 3 days
>       http://bugs.python.org/issue4338    amaury.forgeotdarc
>       patch
>
> Fix set-like dictview doc                                           0 days
>       http://bugs.python.org/issue4339    benjamin.peterson
>
>
> Update __hash__ doc                                                 0 days
>       http://bugs.python.org/issue4341    benjamin.peterson
>
>
> crash upon launch                                                   0 days
>       http://bugs.python.org/issue4344    loewis
>
>
> bytearray methods returning self                                    1 days
>       http://bugs.python.org/issue4348    benjamin.peterson
>       patch, needs review
>
> sys.path includes extraneous junk                                   0 days
>       http://bugs.python.org/issue4349    benjamin.peterson
>       patch, needs review
>
> Move description what a trace should should return to settrace f    1 days
>       http://bugs.python.org/issue4353    benjamin.peterson
>
>
> distutils.command.register is broken                                1 days
>       http://bugs.python.org/issue4354    loewis
>       patch
>
> Add "key" argument to "bisect" module functions                     1 days
>       http://bugs.python.org/issue4356    marketdickinson
>
>
> frozen?set operations create incorrectly initialized instances o    0 days
>       http://bugs.python.org/issue4357    amaury.forgeotdarc
>
>
> SystemError when method has both super() & closure                  0 days
>       http://bugs.python.org/issue4360    barry
>       patch
>
> FileIO object in io module                                          0 days
>       http://bugs.python.org/issue4362    benjamin.peterson
>       patch, needs review
>
> __cmp__ removal not in What's New                                   1 days
>       http://bugs.python.org/issue4372    haypo
>
>
> Using deiconify() hangs on exit                                  2646 days
>       http://bugs.python.org/issue453489  amaury.forgeotdarc
>
>
> sys.stdout problems with pythonw.exe                             1616 days
>       http://bugs.python.org/issue973507  amaury.forgeotdarc
>
>
> A subclass of set doesn't always have __init__ called.            547 days
>       http://bugs.python.org/issue1721812 rhettinger
>       patch
>
>
>
> Top Issues Most Discussed (10)
> ______________________________
>
>  15 datetime: define division timedelta/timedelta                    208 days
> open    http://bugs.python.org/issue2706
>
>  12 Tkinter binding involving Control-spacebar raises unicode error    4 days
> open    http://bugs.python.org/issue1028
>
>  10 TypeError (bytes/str) in distutils command "upload"                3 days
> closed  http://bugs.python.org/issue4338
>
>  10 type of UserList instance returns class instead of instance        3 days
> closed  http://bugs.python.org/issue4326
>
>  10 missing update() in _Screen.setup() of Lib/turtle.py              39 days
> open    http://bugs.python.org/issue4117
>
>  10 Byte/string inconsistencies between different dbm modules          1 days
> open    http://bugs.python.org/issue3799
>
>  8 FileIO object in io module                                         0 days
> closed  http://bugs.python.org/issue4362
>
>  8 bytearray methods returning self                                   1 days
> closed  http://bugs.python.org/issue4348
>
>  7 Reference leaks in Python 3.0rc3                                   1 days
> open    http://bugs.python.org/issue4373
>
>  7 SystemError when method has both super() & closure                 0 days
> closed  http://bugs.python.org/issue4360
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>
>

From brett at python.org  Mon Nov 24 18:54:53 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 24 Nov 2008 09:54:53 -0800
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>
References: <20081121170645.3954E78534@psf.upfronthosting.co.za>
	<e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>
Message-ID: <bbaeab100811240954w548d4f09ib8dfc188f5a869b8@mail.gmail.com>

On Mon, Nov 24, 2008 at 08:58, Jeremy Hylton <jeremy at alum.mit.edu> wrote:
> I wanted to ask a policy question on the bug tracker.  What are we
> doing with bugs filed against Python 2.4?
> This bug http://bugs.python.org/issue1208304 reports a fd leak in
> Python 2.4, which doesn't exist in the head.  Since Python 2.4 is in
> security-fix only mode, is it fair to close this with a won't fix?
>

I think so.

> A meta-question:  Is there a document that has any tips on how to
> handle bug reports?  It might be helpful to have a list of hints or
> policy decisions to help people decide what to do with bugs.

It's coming. I just have not gotten around to it yet. I plan to
outline the basic steps in the life of an issue and that includes
closing an issue and the various reasons this can happen.

-Brett


>
> Jeremy
>
> On Fri, Nov 21, 2008 at 12:06 PM, Python tracker <status at bugs.python.org> wrote:
>>
>> ACTIVITY SUMMARY (11/14/08 - 11/21/08)
>> Python tracker at http://bugs.python.org/
>>
>> To view or respond to any of the issues listed below, click on the issue
>> number.  Do NOT respond to this message.
>>
>>
>>  2194 open (+35) / 14007 closed (+18) / 16201 total (+53)
>>
>> Open issues with patches:   728
>>
>> Average duration of open issues: 712 days.
>> Median duration of open issues: 2043 days.
>>
>> Open Issues Breakdown
>>   open  2177 (+35)
>> pending    17 ( +0)
>>
>> Issues Created Or Reopened (56)
>> _______________________________
>>
>> Tkinter binding involving Control-spacebar raises unicode error  11/18/08
>>       http://bugs.python.org/issue1028    reopened ocean-city
>>       patch
>>
>> Byte/string inconsistencies between different dbm modules        11/21/08
>>       http://bugs.python.org/issue3799    reopened brett.cannon
>>       patch, needs review
>>
>> type of UserList instance returns class instead of instance      11/14/08
>> CLOSED http://bugs.python.org/issue4326    created  chafporte
>>
>>
>> Patch: simplify complex constant assignment statements           11/14/08
>>       http://bugs.python.org/issue4327    created  novalis_dt
>>       patch
>>
>> "?" in u"foo" raises a misleading error                         11/15/08
>>       http://bugs.python.org/issue4328    created  ezio.melotti
>>
>>
>> base64 does not properly handle unicode strings                  11/15/08
>>       http://bugs.python.org/issue4329    created  mbecker
>>
>>
>> wsgiref.validate doesn't accept arguments to readline            11/16/08
>> CLOSED http://bugs.python.org/issue4330    created  ianb
>>
>>
>> Can't use _functools.partial() created function as method        11/16/08
>>       http://bugs.python.org/issue4331    created  ssadler
>>
>>
>> asyncore.file_dispatcher does not use dup()'ed fd                11/17/08
>> CLOSED http://bugs.python.org/issue4332    created  christianbecke
>>       patch
>>
>> Reworked Dialog.py                                               11/17/08
>>       http://bugs.python.org/issue4333    created  gpolo
>>       patch
>>
>> Mac Build Script is broken for 2.6 release                       11/17/08
>> CLOSED http://bugs.python.org/issue4334    created  ngift
>>
>>
>> inspect.getsourcelines ignores last line in module               11/17/08
>>       http://bugs.python.org/issue4335    created  RafeSacks
>>
>>
>> Fix performance issues in xmlrpclib                              11/17/08
>>       http://bugs.python.org/issue4336    created  krisvale
>>       patch, patch, easy
>>
>> Iteration over a map object with list()                          11/17/08
>> CLOSED http://bugs.python.org/issue4337    created  jmfauth
>>
>>
>> TypeError (bytes/str) in distutils command "upload"              11/17/08
>> CLOSED http://bugs.python.org/issue4338    created  hagen
>>       patch
>>
>> Fix set-like dictview doc                                        11/17/08
>> CLOSED http://bugs.python.org/issue4339    created  tjreedy
>>
>>
>> xmlrpc.client - default 'SlowParser' not defined                 11/17/08
>>       http://bugs.python.org/issue4340    created  mwatkins
>>
>>
>> Update __hash__ doc                                              11/17/08
>> CLOSED http://bugs.python.org/issue4341    created  tjreedy
>>
>>
>> (Tkinter) Please backport these                                  11/18/08
>>       http://bugs.python.org/issue4342    created  gpolo
>>
>>
>> New function in Tkinter.py: setup_master                         11/18/08
>>       http://bugs.python.org/issue4343    created  gpolo
>>       patch
>>
>> crash upon launch                                                11/18/08
>> CLOSED http://bugs.python.org/issue4344    created  source.mod
>>
>>
>> Implement nb_nonzero for PyTclObject                             11/18/08
>>       http://bugs.python.org/issue4345    created  gpolo
>>       patch
>>
>> PyObject_CallMethod changes the exception message already set by 11/18/08
>>       http://bugs.python.org/issue4346    created  gpolo
>>       patch
>>
>> Dependencies of graminit.h are not rebuilt when the file is rege 11/18/08
>>       http://bugs.python.org/issue4347    created  thomas.lee
>>       patch
>>
>> bytearray methods returning self                                 11/18/08
>> CLOSED http://bugs.python.org/issue4348    created  DinoV
>>       patch, needs review
>>
>> sys.path includes extraneous junk                                11/18/08
>> CLOSED http://bugs.python.org/issue4349    created  exarkun
>>       patch, needs review
>>
>> Remove dead code from Tkinter.py                                 11/19/08
>>       http://bugs.python.org/issue4350    created  gpolo
>>       patch
>>
>> [PATCH] Better stacklevel for GzipFile.filename DeprecationWarni 11/19/08
>>       http://bugs.python.org/issue4351    created  pjenvey
>>       patch
>>
>> imp.find_module() fails with a UnicodeDecodeError when called wi 11/19/08
>>       http://bugs.python.org/issue4352    created  Jukka Aho
>>
>>
>> Move description what a trace should should return to settrace f 11/19/08
>> CLOSED http://bugs.python.org/issue4353    created  rocky
>>
>>
>> distutils.command.register is broken                             11/19/08
>> CLOSED http://bugs.python.org/issue4354    created  hagen
>>       patch
>>
>> Error in docs of urllib.request and urllib.parse                 11/19/08
>>       http://bugs.python.org/issue4355    created  hagen
>>       patch
>>
>> Add "key" argument to "bisect" module functions                  11/19/08
>> CLOSED http://bugs.python.org/issue4356    created  tebeka
>>
>>
>> frozen?set operations create incorrectly initialized instances o 11/19/08
>> CLOSED http://bugs.python.org/issue4357    created  alexhsamuel
>>
>>
>> Segfault in stringobject.c                                       11/19/08
>>       http://bugs.python.org/issue4358    created  farshad
>>
>>
>> at runtime, distutils uses buildtime files                       11/19/08
>>       http://bugs.python.org/issue4359    created  a.badger
>>
>>
>> SystemError when method has both super() & closure               11/20/08
>> CLOSED http://bugs.python.org/issue4360    created  kaizhu
>>       patch
>>
>> Docstring for "Lib/string.py" is outdated                        11/20/08
>>       http://bugs.python.org/issue4361    created  thp
>>       patch
>>
>> FileIO object in io module                                       11/20/08
>> CLOSED http://bugs.python.org/issue4362    reopened barry
>>       patch, needs review
>>
>> Make uuid module functions usable without ctypes                 11/20/08
>>       http://bugs.python.org/issue4363    created  hdima
>>       patch
>>
>> error in multiprocessing docs - rawvalue                         11/20/08
>>       http://bugs.python.org/issue4364    created  dursobr
>>
>>
>> Add CRT version info in msvcrt module                            11/20/08
>>       http://bugs.python.org/issue4365    created  cdavid
>>       patch
>>
>> cannot find -lpython2.5 when buinding Python 2.5.2 on FreeBSD 4. 11/20/08
>>       http://bugs.python.org/issue4366    created  akitada
>>
>>
>> segmentation fault in ast_for_atom                               11/20/08
>>       http://bugs.python.org/issue4367    created  ot
>>       patch, needs review
>>
>> A bug in ncurses.h still exists in FreeBSD 4.9 - 4.11            11/20/08
>>       http://bugs.python.org/issue4368    created  akitada
>>       patch
>>
>> Error building to a nonstandard prefix (with patch)              11/20/08
>>       http://bugs.python.org/issue4369    created  ot
>>       patch
>>
>> warning: unknown conversion type character `z' in format         11/20/08
>>       http://bugs.python.org/issue4370    created  akitada
>>
>>
>> coerce gone---but not from docs                                  11/20/08
>>       http://bugs.python.org/issue4371    created  LambertDW
>>
>>
>> __cmp__ removal not in What's New                                11/20/08
>> CLOSED http://bugs.python.org/issue4372    created  tjreedy
>>
>>
>> Reference leaks in Python 3.0rc3                                 11/21/08
>>       http://bugs.python.org/issue4373    created  christian.heimes
>>       patch
>>
>> Pickle tests fail w/o _pickle extension                          11/21/08
>>       http://bugs.python.org/issue4374    created  christian.heimes
>>
>>
>> inspect.getsource doesn't work with PYTHONPATH and source compil 11/21/08
>>       http://bugs.python.org/issue4375    created  erickt
>>
>>
>> Nested ctypes 'BigEndianStructure' fails                         11/21/08
>>       http://bugs.python.org/issue4376    created  castironpi
>>
>>
>> tokenize.detect_encoding() and Mac newline                       11/21/08
>>       http://bugs.python.org/issue4377    created  haypo
>>
>>
>> howto doc update                                                 11/21/08
>>       http://bugs.python.org/issue4378    created  LambertDW
>>
>>
>> Fix exception pickling: Move initial args assignment to BaseExce 11/18/08
>>       http://bugs.python.org/issue1692335 reopened gvanrossum
>>       patch
>>
>>
>>
>> Issues Now Closed (40)
>> ______________________
>>
>> Problem with doctest and decorated functions                      441 days
>>       http://bugs.python.org/issue1108    skip.montanaro
>>
>>
>> conditional jump to a POP_TOP optimization                        253 days
>>       http://bugs.python.org/issue2260    rhettinger
>>       patch
>>
>> NULL member in modules_by_index                                   131 days
>>       http://bugs.python.org/issue3327    loewis
>>       patch, easy
>>
>> test_cpickle crash on AMD64 Windows build                          88 days
>>       http://bugs.python.org/issue3640    loewis
>>       patch, 64bit
>>
>> pressing HOME key in IDLE editor ends IDLE                         81 days
>>       http://bugs.python.org/issue3679    tjreedy
>>
>>
>> configure --with-threads on cygwin => crash on thread related te   58 days
>>       http://bugs.python.org/issue3947    ocean-city
>>       patch
>>
>> ignored exceptions in generators (regression?)                     43 days
>>       http://bugs.python.org/issue4040    benjamin.peterson
>>
>>
>> Document the effects of NotImplemented on == and !=                43 days
>>       http://bugs.python.org/issue4087    marketdickinson
>>
>>
>> Documenting set comparisons and operations                         39 days
>>       http://bugs.python.org/issue4090    rhettinger
>>       patch
>>
>> name conflict in ScrolledCanvas.__init__() in Lib/turtle.py        36 days
>>       http://bugs.python.org/issue4116    loewis
>>       patch
>>
>> "__getattr__" can't be a descriptor                                19 days
>>       http://bugs.python.org/issue4230    benjamin.peterson
>>       patch
>>
>> Python 2.6 installer crashes when selecting 'advanced' and cance   10 days
>>       http://bugs.python.org/issue4289    loewis
>>
>>
>> email package with unicode subject/body                             8 days
>>       http://bugs.python.org/issue4306    barry
>>       patch
>>
>> Unicode in distutils meta-data?                                     4 days
>>       http://bugs.python.org/issue4312    loewis
>>
>>
>> IDLE segfault at exit                                               5 days
>>       http://bugs.python.org/issue4313    ocean-city
>>
>>
>> Improper use of [] in configure.in leads to useless regexp in co    2 days
>>       http://bugs.python.org/issue4316    georg.brandl
>>       patch
>>
>> Buffer overflow in imageop module                                   5 days
>>       http://bugs.python.org/issue4317    amaury.forgeotdarc
>>       patch
>>
>> unintended syntax error with decorators, parenthesis, and dots?     1 days
>>       http://bugs.python.org/issue4321    georg.brandl
>>
>>
>> locale documentation is inconsistent                                1 days
>>       http://bugs.python.org/issue4324    georg.brandl
>>
>>
>> type of UserList instance returns class instead of instance         3 days
>>       http://bugs.python.org/issue4326    georg.brandl
>>
>>
>> wsgiref.validate doesn't accept arguments to readline               1 days
>>       http://bugs.python.org/issue4330    ianb
>>
>>
>> asyncore.file_dispatcher does not use dup()'ed fd                   3 days
>>       http://bugs.python.org/issue4332    josiahcarlson
>>       patch
>>
>> Mac Build Script is broken for 2.6 release                          0 days
>>       http://bugs.python.org/issue4334    benjamin.peterson
>>
>>
>> Iteration over a map object with list()                             1 days
>>       http://bugs.python.org/issue4337    jmfauth
>>
>>
>> TypeError (bytes/str) in distutils command "upload"                 3 days
>>       http://bugs.python.org/issue4338    amaury.forgeotdarc
>>       patch
>>
>> Fix set-like dictview doc                                           0 days
>>       http://bugs.python.org/issue4339    benjamin.peterson
>>
>>
>> Update __hash__ doc                                                 0 days
>>       http://bugs.python.org/issue4341    benjamin.peterson
>>
>>
>> crash upon launch                                                   0 days
>>       http://bugs.python.org/issue4344    loewis
>>
>>
>> bytearray methods returning self                                    1 days
>>       http://bugs.python.org/issue4348    benjamin.peterson
>>       patch, needs review
>>
>> sys.path includes extraneous junk                                   0 days
>>       http://bugs.python.org/issue4349    benjamin.peterson
>>       patch, needs review
>>
>> Move description what a trace should should return to settrace f    1 days
>>       http://bugs.python.org/issue4353    benjamin.peterson
>>
>>
>> distutils.command.register is broken                                1 days
>>       http://bugs.python.org/issue4354    loewis
>>       patch
>>
>> Add "key" argument to "bisect" module functions                     1 days
>>       http://bugs.python.org/issue4356    marketdickinson
>>
>>
>> frozen?set operations create incorrectly initialized instances o    0 days
>>       http://bugs.python.org/issue4357    amaury.forgeotdarc
>>
>>
>> SystemError when method has both super() & closure                  0 days
>>       http://bugs.python.org/issue4360    barry
>>       patch
>>
>> FileIO object in io module                                          0 days
>>       http://bugs.python.org/issue4362    benjamin.peterson
>>       patch, needs review
>>
>> __cmp__ removal not in What's New                                   1 days
>>       http://bugs.python.org/issue4372    haypo
>>
>>
>> Using deiconify() hangs on exit                                  2646 days
>>       http://bugs.python.org/issue453489  amaury.forgeotdarc
>>
>>
>> sys.stdout problems with pythonw.exe                             1616 days
>>       http://bugs.python.org/issue973507  amaury.forgeotdarc
>>
>>
>> A subclass of set doesn't always have __init__ called.            547 days
>>       http://bugs.python.org/issue1721812 rhettinger
>>       patch
>>
>>
>>
>> Top Issues Most Discussed (10)
>> ______________________________
>>
>>  15 datetime: define division timedelta/timedelta                    208 days
>> open    http://bugs.python.org/issue2706
>>
>>  12 Tkinter binding involving Control-spacebar raises unicode error    4 days
>> open    http://bugs.python.org/issue1028
>>
>>  10 TypeError (bytes/str) in distutils command "upload"                3 days
>> closed  http://bugs.python.org/issue4338
>>
>>  10 type of UserList instance returns class instead of instance        3 days
>> closed  http://bugs.python.org/issue4326
>>
>>  10 missing update() in _Screen.setup() of Lib/turtle.py              39 days
>> open    http://bugs.python.org/issue4117
>>
>>  10 Byte/string inconsistencies between different dbm modules          1 days
>> open    http://bugs.python.org/issue3799
>>
>>  8 FileIO object in io module                                         0 days
>> closed  http://bugs.python.org/issue4362
>>
>>  8 bytearray methods returning self                                   1 days
>> closed  http://bugs.python.org/issue4348
>>
>>  7 Reference leaks in Python 3.0rc3                                   1 days
>> open    http://bugs.python.org/issue4373
>>
>>  7 SystemError when method has both super() & closure                 0 days
>> closed  http://bugs.python.org/issue4360
>>
>>
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>>
>>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From jeremy at alum.mit.edu  Mon Nov 24 19:02:13 2008
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 24 Nov 2008 13:02:13 -0500
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <bbaeab100811240954w548d4f09ib8dfc188f5a869b8@mail.gmail.com>
References: <20081121170645.3954E78534@psf.upfronthosting.co.za>
	<e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>
	<bbaeab100811240954w548d4f09ib8dfc188f5a869b8@mail.gmail.com>
Message-ID: <e8bf7a530811241002q13a9e9ear132b69f25bd14a14@mail.gmail.com>

On Mon, Nov 24, 2008 at 12:54 PM, Brett Cannon <brett at python.org> wrote:
> On Mon, Nov 24, 2008 at 08:58, Jeremy Hylton <jeremy at alum.mit.edu> wrote:
>> I wanted to ask a policy question on the bug tracker.  What are we
>> doing with bugs filed against Python 2.4?
>> This bug http://bugs.python.org/issue1208304 reports a fd leak in
>> Python 2.4, which doesn't exist in the head.  Since Python 2.4 is in
>> security-fix only mode, is it fair to close this with a won't fix?
>>
>
> I think so.
>
>> A meta-question:  Is there a document that has any tips on how to
>> handle bug reports?  It might be helpful to have a list of hints or
>> policy decisions to help people decide what to do with bugs.
>
> It's coming. I just have not gotten around to it yet. I plan to
> outline the basic steps in the life of an issue and that includes
> closing an issue and the various reasons this can happen.

If you put up skeleton doc, I can add a note to this effect.

Jeremy

>
> -Brett
>
>
>>
>> Jeremy
>>
>> On Fri, Nov 21, 2008 at 12:06 PM, Python tracker <status at bugs.python.org> wrote:
>>>
>>> ACTIVITY SUMMARY (11/14/08 - 11/21/08)
>>> Python tracker at http://bugs.python.org/
>>>
>>> To view or respond to any of the issues listed below, click on the issue
>>> number.  Do NOT respond to this message.
>>>
>>>
>>>  2194 open (+35) / 14007 closed (+18) / 16201 total (+53)
>>>
>>> Open issues with patches:   728
>>>
>>> Average duration of open issues: 712 days.
>>> Median duration of open issues: 2043 days.
>>>
>>> Open Issues Breakdown
>>>   open  2177 (+35)
>>> pending    17 ( +0)
>>>
>>> Issues Created Or Reopened (56)
>>> _______________________________
>>>
>>> Tkinter binding involving Control-spacebar raises unicode error  11/18/08
>>>       http://bugs.python.org/issue1028    reopened ocean-city
>>>       patch
>>>
>>> Byte/string inconsistencies between different dbm modules        11/21/08
>>>       http://bugs.python.org/issue3799    reopened brett.cannon
>>>       patch, needs review
>>>
>>> type of UserList instance returns class instead of instance      11/14/08
>>> CLOSED http://bugs.python.org/issue4326    created  chafporte
>>>
>>>
>>> Patch: simplify complex constant assignment statements           11/14/08
>>>       http://bugs.python.org/issue4327    created  novalis_dt
>>>       patch
>>>
>>> "?" in u"foo" raises a misleading error                         11/15/08
>>>       http://bugs.python.org/issue4328    created  ezio.melotti
>>>
>>>
>>> base64 does not properly handle unicode strings                  11/15/08
>>>       http://bugs.python.org/issue4329    created  mbecker
>>>
>>>
>>> wsgiref.validate doesn't accept arguments to readline            11/16/08
>>> CLOSED http://bugs.python.org/issue4330    created  ianb
>>>
>>>
>>> Can't use _functools.partial() created function as method        11/16/08
>>>       http://bugs.python.org/issue4331    created  ssadler
>>>
>>>
>>> asyncore.file_dispatcher does not use dup()'ed fd                11/17/08
>>> CLOSED http://bugs.python.org/issue4332    created  christianbecke
>>>       patch
>>>
>>> Reworked Dialog.py                                               11/17/08
>>>       http://bugs.python.org/issue4333    created  gpolo
>>>       patch
>>>
>>> Mac Build Script is broken for 2.6 release                       11/17/08
>>> CLOSED http://bugs.python.org/issue4334    created  ngift
>>>
>>>
>>> inspect.getsourcelines ignores last line in module               11/17/08
>>>       http://bugs.python.org/issue4335    created  RafeSacks
>>>
>>>
>>> Fix performance issues in xmlrpclib                              11/17/08
>>>       http://bugs.python.org/issue4336    created  krisvale
>>>       patch, patch, easy
>>>
>>> Iteration over a map object with list()                          11/17/08
>>> CLOSED http://bugs.python.org/issue4337    created  jmfauth
>>>
>>>
>>> TypeError (bytes/str) in distutils command "upload"              11/17/08
>>> CLOSED http://bugs.python.org/issue4338    created  hagen
>>>       patch
>>>
>>> Fix set-like dictview doc                                        11/17/08
>>> CLOSED http://bugs.python.org/issue4339    created  tjreedy
>>>
>>>
>>> xmlrpc.client - default 'SlowParser' not defined                 11/17/08
>>>       http://bugs.python.org/issue4340    created  mwatkins
>>>
>>>
>>> Update __hash__ doc                                              11/17/08
>>> CLOSED http://bugs.python.org/issue4341    created  tjreedy
>>>
>>>
>>> (Tkinter) Please backport these                                  11/18/08
>>>       http://bugs.python.org/issue4342    created  gpolo
>>>
>>>
>>> New function in Tkinter.py: setup_master                         11/18/08
>>>       http://bugs.python.org/issue4343    created  gpolo
>>>       patch
>>>
>>> crash upon launch                                                11/18/08
>>> CLOSED http://bugs.python.org/issue4344    created  source.mod
>>>
>>>
>>> Implement nb_nonzero for PyTclObject                             11/18/08
>>>       http://bugs.python.org/issue4345    created  gpolo
>>>       patch
>>>
>>> PyObject_CallMethod changes the exception message already set by 11/18/08
>>>       http://bugs.python.org/issue4346    created  gpolo
>>>       patch
>>>
>>> Dependencies of graminit.h are not rebuilt when the file is rege 11/18/08
>>>       http://bugs.python.org/issue4347    created  thomas.lee
>>>       patch
>>>
>>> bytearray methods returning self                                 11/18/08
>>> CLOSED http://bugs.python.org/issue4348    created  DinoV
>>>       patch, needs review
>>>
>>> sys.path includes extraneous junk                                11/18/08
>>> CLOSED http://bugs.python.org/issue4349    created  exarkun
>>>       patch, needs review
>>>
>>> Remove dead code from Tkinter.py                                 11/19/08
>>>       http://bugs.python.org/issue4350    created  gpolo
>>>       patch
>>>
>>> [PATCH] Better stacklevel for GzipFile.filename DeprecationWarni 11/19/08
>>>       http://bugs.python.org/issue4351    created  pjenvey
>>>       patch
>>>
>>> imp.find_module() fails with a UnicodeDecodeError when called wi 11/19/08
>>>       http://bugs.python.org/issue4352    created  Jukka Aho
>>>
>>>
>>> Move description what a trace should should return to settrace f 11/19/08
>>> CLOSED http://bugs.python.org/issue4353    created  rocky
>>>
>>>
>>> distutils.command.register is broken                             11/19/08
>>> CLOSED http://bugs.python.org/issue4354    created  hagen
>>>       patch
>>>
>>> Error in docs of urllib.request and urllib.parse                 11/19/08
>>>       http://bugs.python.org/issue4355    created  hagen
>>>       patch
>>>
>>> Add "key" argument to "bisect" module functions                  11/19/08
>>> CLOSED http://bugs.python.org/issue4356    created  tebeka
>>>
>>>
>>> frozen?set operations create incorrectly initialized instances o 11/19/08
>>> CLOSED http://bugs.python.org/issue4357    created  alexhsamuel
>>>
>>>
>>> Segfault in stringobject.c                                       11/19/08
>>>       http://bugs.python.org/issue4358    created  farshad
>>>
>>>
>>> at runtime, distutils uses buildtime files                       11/19/08
>>>       http://bugs.python.org/issue4359    created  a.badger
>>>
>>>
>>> SystemError when method has both super() & closure               11/20/08
>>> CLOSED http://bugs.python.org/issue4360    created  kaizhu
>>>       patch
>>>
>>> Docstring for "Lib/string.py" is outdated                        11/20/08
>>>       http://bugs.python.org/issue4361    created  thp
>>>       patch
>>>
>>> FileIO object in io module                                       11/20/08
>>> CLOSED http://bugs.python.org/issue4362    reopened barry
>>>       patch, needs review
>>>
>>> Make uuid module functions usable without ctypes                 11/20/08
>>>       http://bugs.python.org/issue4363    created  hdima
>>>       patch
>>>
>>> error in multiprocessing docs - rawvalue                         11/20/08
>>>       http://bugs.python.org/issue4364    created  dursobr
>>>
>>>
>>> Add CRT version info in msvcrt module                            11/20/08
>>>       http://bugs.python.org/issue4365    created  cdavid
>>>       patch
>>>
>>> cannot find -lpython2.5 when buinding Python 2.5.2 on FreeBSD 4. 11/20/08
>>>       http://bugs.python.org/issue4366    created  akitada
>>>
>>>
>>> segmentation fault in ast_for_atom                               11/20/08
>>>       http://bugs.python.org/issue4367    created  ot
>>>       patch, needs review
>>>
>>> A bug in ncurses.h still exists in FreeBSD 4.9 - 4.11            11/20/08
>>>       http://bugs.python.org/issue4368    created  akitada
>>>       patch
>>>
>>> Error building to a nonstandard prefix (with patch)              11/20/08
>>>       http://bugs.python.org/issue4369    created  ot
>>>       patch
>>>
>>> warning: unknown conversion type character `z' in format         11/20/08
>>>       http://bugs.python.org/issue4370    created  akitada
>>>
>>>
>>> coerce gone---but not from docs                                  11/20/08
>>>       http://bugs.python.org/issue4371    created  LambertDW
>>>
>>>
>>> __cmp__ removal not in What's New                                11/20/08
>>> CLOSED http://bugs.python.org/issue4372    created  tjreedy
>>>
>>>
>>> Reference leaks in Python 3.0rc3                                 11/21/08
>>>       http://bugs.python.org/issue4373    created  christian.heimes
>>>       patch
>>>
>>> Pickle tests fail w/o _pickle extension                          11/21/08
>>>       http://bugs.python.org/issue4374    created  christian.heimes
>>>
>>>
>>> inspect.getsource doesn't work with PYTHONPATH and source compil 11/21/08
>>>       http://bugs.python.org/issue4375    created  erickt
>>>
>>>
>>> Nested ctypes 'BigEndianStructure' fails                         11/21/08
>>>       http://bugs.python.org/issue4376    created  castironpi
>>>
>>>
>>> tokenize.detect_encoding() and Mac newline                       11/21/08
>>>       http://bugs.python.org/issue4377    created  haypo
>>>
>>>
>>> howto doc update                                                 11/21/08
>>>       http://bugs.python.org/issue4378    created  LambertDW
>>>
>>>
>>> Fix exception pickling: Move initial args assignment to BaseExce 11/18/08
>>>       http://bugs.python.org/issue1692335 reopened gvanrossum
>>>       patch
>>>
>>>
>>>
>>> Issues Now Closed (40)
>>> ______________________
>>>
>>> Problem with doctest and decorated functions                      441 days
>>>       http://bugs.python.org/issue1108    skip.montanaro
>>>
>>>
>>> conditional jump to a POP_TOP optimization                        253 days
>>>       http://bugs.python.org/issue2260    rhettinger
>>>       patch
>>>
>>> NULL member in modules_by_index                                   131 days
>>>       http://bugs.python.org/issue3327    loewis
>>>       patch, easy
>>>
>>> test_cpickle crash on AMD64 Windows build                          88 days
>>>       http://bugs.python.org/issue3640    loewis
>>>       patch, 64bit
>>>
>>> pressing HOME key in IDLE editor ends IDLE                         81 days
>>>       http://bugs.python.org/issue3679    tjreedy
>>>
>>>
>>> configure --with-threads on cygwin => crash on thread related te   58 days
>>>       http://bugs.python.org/issue3947    ocean-city
>>>       patch
>>>
>>> ignored exceptions in generators (regression?)                     43 days
>>>       http://bugs.python.org/issue4040    benjamin.peterson
>>>
>>>
>>> Document the effects of NotImplemented on == and !=                43 days
>>>       http://bugs.python.org/issue4087    marketdickinson
>>>
>>>
>>> Documenting set comparisons and operations                         39 days
>>>       http://bugs.python.org/issue4090    rhettinger
>>>       patch
>>>
>>> name conflict in ScrolledCanvas.__init__() in Lib/turtle.py        36 days
>>>       http://bugs.python.org/issue4116    loewis
>>>       patch
>>>
>>> "__getattr__" can't be a descriptor                                19 days
>>>       http://bugs.python.org/issue4230    benjamin.peterson
>>>       patch
>>>
>>> Python 2.6 installer crashes when selecting 'advanced' and cance   10 days
>>>       http://bugs.python.org/issue4289    loewis
>>>
>>>
>>> email package with unicode subject/body                             8 days
>>>       http://bugs.python.org/issue4306    barry
>>>       patch
>>>
>>> Unicode in distutils meta-data?                                     4 days
>>>       http://bugs.python.org/issue4312    loewis
>>>
>>>
>>> IDLE segfault at exit                                               5 days
>>>       http://bugs.python.org/issue4313    ocean-city
>>>
>>>
>>> Improper use of [] in configure.in leads to useless regexp in co    2 days
>>>       http://bugs.python.org/issue4316    georg.brandl
>>>       patch
>>>
>>> Buffer overflow in imageop module                                   5 days
>>>       http://bugs.python.org/issue4317    amaury.forgeotdarc
>>>       patch
>>>
>>> unintended syntax error with decorators, parenthesis, and dots?     1 days
>>>       http://bugs.python.org/issue4321    georg.brandl
>>>
>>>
>>> locale documentation is inconsistent                                1 days
>>>       http://bugs.python.org/issue4324    georg.brandl
>>>
>>>
>>> type of UserList instance returns class instead of instance         3 days
>>>       http://bugs.python.org/issue4326    georg.brandl
>>>
>>>
>>> wsgiref.validate doesn't accept arguments to readline               1 days
>>>       http://bugs.python.org/issue4330    ianb
>>>
>>>
>>> asyncore.file_dispatcher does not use dup()'ed fd                   3 days
>>>       http://bugs.python.org/issue4332    josiahcarlson
>>>       patch
>>>
>>> Mac Build Script is broken for 2.6 release                          0 days
>>>       http://bugs.python.org/issue4334    benjamin.peterson
>>>
>>>
>>> Iteration over a map object with list()                             1 days
>>>       http://bugs.python.org/issue4337    jmfauth
>>>
>>>
>>> TypeError (bytes/str) in distutils command "upload"                 3 days
>>>       http://bugs.python.org/issue4338    amaury.forgeotdarc
>>>       patch
>>>
>>> Fix set-like dictview doc                                           0 days
>>>       http://bugs.python.org/issue4339    benjamin.peterson
>>>
>>>
>>> Update __hash__ doc                                                 0 days
>>>       http://bugs.python.org/issue4341    benjamin.peterson
>>>
>>>
>>> crash upon launch                                                   0 days
>>>       http://bugs.python.org/issue4344    loewis
>>>
>>>
>>> bytearray methods returning self                                    1 days
>>>       http://bugs.python.org/issue4348    benjamin.peterson
>>>       patch, needs review
>>>
>>> sys.path includes extraneous junk                                   0 days
>>>       http://bugs.python.org/issue4349    benjamin.peterson
>>>       patch, needs review
>>>
>>> Move description what a trace should should return to settrace f    1 days
>>>       http://bugs.python.org/issue4353    benjamin.peterson
>>>
>>>
>>> distutils.command.register is broken                                1 days
>>>       http://bugs.python.org/issue4354    loewis
>>>       patch
>>>
>>> Add "key" argument to "bisect" module functions                     1 days
>>>       http://bugs.python.org/issue4356    marketdickinson
>>>
>>>
>>> frozen?set operations create incorrectly initialized instances o    0 days
>>>       http://bugs.python.org/issue4357    amaury.forgeotdarc
>>>
>>>
>>> SystemError when method has both super() & closure                  0 days
>>>       http://bugs.python.org/issue4360    barry
>>>       patch
>>>
>>> FileIO object in io module                                          0 days
>>>       http://bugs.python.org/issue4362    benjamin.peterson
>>>       patch, needs review
>>>
>>> __cmp__ removal not in What's New                                   1 days
>>>       http://bugs.python.org/issue4372    haypo
>>>
>>>
>>> Using deiconify() hangs on exit                                  2646 days
>>>       http://bugs.python.org/issue453489  amaury.forgeotdarc
>>>
>>>
>>> sys.stdout problems with pythonw.exe                             1616 days
>>>       http://bugs.python.org/issue973507  amaury.forgeotdarc
>>>
>>>
>>> A subclass of set doesn't always have __init__ called.            547 days
>>>       http://bugs.python.org/issue1721812 rhettinger
>>>       patch
>>>
>>>
>>>
>>> Top Issues Most Discussed (10)
>>> ______________________________
>>>
>>>  15 datetime: define division timedelta/timedelta                    208 days
>>> open    http://bugs.python.org/issue2706
>>>
>>>  12 Tkinter binding involving Control-spacebar raises unicode error    4 days
>>> open    http://bugs.python.org/issue1028
>>>
>>>  10 TypeError (bytes/str) in distutils command "upload"                3 days
>>> closed  http://bugs.python.org/issue4338
>>>
>>>  10 type of UserList instance returns class instead of instance        3 days
>>> closed  http://bugs.python.org/issue4326
>>>
>>>  10 missing update() in _Screen.setup() of Lib/turtle.py              39 days
>>> open    http://bugs.python.org/issue4117
>>>
>>>  10 Byte/string inconsistencies between different dbm modules          1 days
>>> open    http://bugs.python.org/issue3799
>>>
>>>  8 FileIO object in io module                                         0 days
>>> closed  http://bugs.python.org/issue4362
>>>
>>>  8 bytearray methods returning self                                   1 days
>>> closed  http://bugs.python.org/issue4348
>>>
>>>  7 Reference leaks in Python 3.0rc3                                   1 days
>>> open    http://bugs.python.org/issue4373
>>>
>>>  7 SystemError when method has both super() & closure                 0 days
>>> closed  http://bugs.python.org/issue4360
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>>>
>>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>>
>

From brett at python.org  Mon Nov 24 19:24:23 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 24 Nov 2008 10:24:23 -0800
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <e8bf7a530811241002q13a9e9ear132b69f25bd14a14@mail.gmail.com>
References: <20081121170645.3954E78534@psf.upfronthosting.co.za>
	<e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>
	<bbaeab100811240954w548d4f09ib8dfc188f5a869b8@mail.gmail.com>
	<e8bf7a530811241002q13a9e9ear132b69f25bd14a14@mail.gmail.com>
Message-ID: <bbaeab100811241024q4d62f92ag6790e67ef5d2db92@mail.gmail.com>

On Mon, Nov 24, 2008 at 10:02, Jeremy Hylton <jeremy at alum.mit.edu> wrote:
> On Mon, Nov 24, 2008 at 12:54 PM, Brett Cannon <brett at python.org> wrote:
>> On Mon, Nov 24, 2008 at 08:58, Jeremy Hylton <jeremy at alum.mit.edu> wrote:
>>> I wanted to ask a policy question on the bug tracker.  What are we
>>> doing with bugs filed against Python 2.4?
>>> This bug http://bugs.python.org/issue1208304 reports a fd leak in
>>> Python 2.4, which doesn't exist in the head.  Since Python 2.4 is in
>>> security-fix only mode, is it fair to close this with a won't fix?
>>>
>>
>> I think so.
>>
>>> A meta-question:  Is there a document that has any tips on how to
>>> handle bug reports?  It might be helpful to have a list of hints or
>>> policy decisions to help people decide what to do with bugs.
>>
>> It's coming. I just have not gotten around to it yet. I plan to
>> outline the basic steps in the life of an issue and that includes
>> closing an issue and the various reasons this can happen.
>
> If you put up skeleton doc, I can add a note to this effect.
>

The doc I have for it is made up like five bullet points, so nothing
to really put up yet. But I did add another bullet point for this.

-Brett


> Jeremy
>
>>
>> -Brett
>>
>>
>>>
>>> Jeremy
>>>
>>> On Fri, Nov 21, 2008 at 12:06 PM, Python tracker <status at bugs.python.org> wrote:
>>>>
>>>> ACTIVITY SUMMARY (11/14/08 - 11/21/08)
>>>> Python tracker at http://bugs.python.org/
>>>>
>>>> To view or respond to any of the issues listed below, click on the issue
>>>> number.  Do NOT respond to this message.
>>>>
>>>>
>>>>  2194 open (+35) / 14007 closed (+18) / 16201 total (+53)
>>>>
>>>> Open issues with patches:   728
>>>>
>>>> Average duration of open issues: 712 days.
>>>> Median duration of open issues: 2043 days.
>>>>
>>>> Open Issues Breakdown
>>>>   open  2177 (+35)
>>>> pending    17 ( +0)
>>>>
>>>> Issues Created Or Reopened (56)
>>>> _______________________________
>>>>
>>>> Tkinter binding involving Control-spacebar raises unicode error  11/18/08
>>>>       http://bugs.python.org/issue1028    reopened ocean-city
>>>>       patch
>>>>
>>>> Byte/string inconsistencies between different dbm modules        11/21/08
>>>>       http://bugs.python.org/issue3799    reopened brett.cannon
>>>>       patch, needs review
>>>>
>>>> type of UserList instance returns class instead of instance      11/14/08
>>>> CLOSED http://bugs.python.org/issue4326    created  chafporte
>>>>
>>>>
>>>> Patch: simplify complex constant assignment statements           11/14/08
>>>>       http://bugs.python.org/issue4327    created  novalis_dt
>>>>       patch
>>>>
>>>> "?" in u"foo" raises a misleading error                         11/15/08
>>>>       http://bugs.python.org/issue4328    created  ezio.melotti
>>>>
>>>>
>>>> base64 does not properly handle unicode strings                  11/15/08
>>>>       http://bugs.python.org/issue4329    created  mbecker
>>>>
>>>>
>>>> wsgiref.validate doesn't accept arguments to readline            11/16/08
>>>> CLOSED http://bugs.python.org/issue4330    created  ianb
>>>>
>>>>
>>>> Can't use _functools.partial() created function as method        11/16/08
>>>>       http://bugs.python.org/issue4331    created  ssadler
>>>>
>>>>
>>>> asyncore.file_dispatcher does not use dup()'ed fd                11/17/08
>>>> CLOSED http://bugs.python.org/issue4332    created  christianbecke
>>>>       patch
>>>>
>>>> Reworked Dialog.py                                               11/17/08
>>>>       http://bugs.python.org/issue4333    created  gpolo
>>>>       patch
>>>>
>>>> Mac Build Script is broken for 2.6 release                       11/17/08
>>>> CLOSED http://bugs.python.org/issue4334    created  ngift
>>>>
>>>>
>>>> inspect.getsourcelines ignores last line in module               11/17/08
>>>>       http://bugs.python.org/issue4335    created  RafeSacks
>>>>
>>>>
>>>> Fix performance issues in xmlrpclib                              11/17/08
>>>>       http://bugs.python.org/issue4336    created  krisvale
>>>>       patch, patch, easy
>>>>
>>>> Iteration over a map object with list()                          11/17/08
>>>> CLOSED http://bugs.python.org/issue4337    created  jmfauth
>>>>
>>>>
>>>> TypeError (bytes/str) in distutils command "upload"              11/17/08
>>>> CLOSED http://bugs.python.org/issue4338    created  hagen
>>>>       patch
>>>>
>>>> Fix set-like dictview doc                                        11/17/08
>>>> CLOSED http://bugs.python.org/issue4339    created  tjreedy
>>>>
>>>>
>>>> xmlrpc.client - default 'SlowParser' not defined                 11/17/08
>>>>       http://bugs.python.org/issue4340    created  mwatkins
>>>>
>>>>
>>>> Update __hash__ doc                                              11/17/08
>>>> CLOSED http://bugs.python.org/issue4341    created  tjreedy
>>>>
>>>>
>>>> (Tkinter) Please backport these                                  11/18/08
>>>>       http://bugs.python.org/issue4342    created  gpolo
>>>>
>>>>
>>>> New function in Tkinter.py: setup_master                         11/18/08
>>>>       http://bugs.python.org/issue4343    created  gpolo
>>>>       patch
>>>>
>>>> crash upon launch                                                11/18/08
>>>> CLOSED http://bugs.python.org/issue4344    created  source.mod
>>>>
>>>>
>>>> Implement nb_nonzero for PyTclObject                             11/18/08
>>>>       http://bugs.python.org/issue4345    created  gpolo
>>>>       patch
>>>>
>>>> PyObject_CallMethod changes the exception message already set by 11/18/08
>>>>       http://bugs.python.org/issue4346    created  gpolo
>>>>       patch
>>>>
>>>> Dependencies of graminit.h are not rebuilt when the file is rege 11/18/08
>>>>       http://bugs.python.org/issue4347    created  thomas.lee
>>>>       patch
>>>>
>>>> bytearray methods returning self                                 11/18/08
>>>> CLOSED http://bugs.python.org/issue4348    created  DinoV
>>>>       patch, needs review
>>>>
>>>> sys.path includes extraneous junk                                11/18/08
>>>> CLOSED http://bugs.python.org/issue4349    created  exarkun
>>>>       patch, needs review
>>>>
>>>> Remove dead code from Tkinter.py                                 11/19/08
>>>>       http://bugs.python.org/issue4350    created  gpolo
>>>>       patch
>>>>
>>>> [PATCH] Better stacklevel for GzipFile.filename DeprecationWarni 11/19/08
>>>>       http://bugs.python.org/issue4351    created  pjenvey
>>>>       patch
>>>>
>>>> imp.find_module() fails with a UnicodeDecodeError when called wi 11/19/08
>>>>       http://bugs.python.org/issue4352    created  Jukka Aho
>>>>
>>>>
>>>> Move description what a trace should should return to settrace f 11/19/08
>>>> CLOSED http://bugs.python.org/issue4353    created  rocky
>>>>
>>>>
>>>> distutils.command.register is broken                             11/19/08
>>>> CLOSED http://bugs.python.org/issue4354    created  hagen
>>>>       patch
>>>>
>>>> Error in docs of urllib.request and urllib.parse                 11/19/08
>>>>       http://bugs.python.org/issue4355    created  hagen
>>>>       patch
>>>>
>>>> Add "key" argument to "bisect" module functions                  11/19/08
>>>> CLOSED http://bugs.python.org/issue4356    created  tebeka
>>>>
>>>>
>>>> frozen?set operations create incorrectly initialized instances o 11/19/08
>>>> CLOSED http://bugs.python.org/issue4357    created  alexhsamuel
>>>>
>>>>
>>>> Segfault in stringobject.c                                       11/19/08
>>>>       http://bugs.python.org/issue4358    created  farshad
>>>>
>>>>
>>>> at runtime, distutils uses buildtime files                       11/19/08
>>>>       http://bugs.python.org/issue4359    created  a.badger
>>>>
>>>>
>>>> SystemError when method has both super() & closure               11/20/08
>>>> CLOSED http://bugs.python.org/issue4360    created  kaizhu
>>>>       patch
>>>>
>>>> Docstring for "Lib/string.py" is outdated                        11/20/08
>>>>       http://bugs.python.org/issue4361    created  thp
>>>>       patch
>>>>
>>>> FileIO object in io module                                       11/20/08
>>>> CLOSED http://bugs.python.org/issue4362    reopened barry
>>>>       patch, needs review
>>>>
>>>> Make uuid module functions usable without ctypes                 11/20/08
>>>>       http://bugs.python.org/issue4363    created  hdima
>>>>       patch
>>>>
>>>> error in multiprocessing docs - rawvalue                         11/20/08
>>>>       http://bugs.python.org/issue4364    created  dursobr
>>>>
>>>>
>>>> Add CRT version info in msvcrt module                            11/20/08
>>>>       http://bugs.python.org/issue4365    created  cdavid
>>>>       patch
>>>>
>>>> cannot find -lpython2.5 when buinding Python 2.5.2 on FreeBSD 4. 11/20/08
>>>>       http://bugs.python.org/issue4366    created  akitada
>>>>
>>>>
>>>> segmentation fault in ast_for_atom                               11/20/08
>>>>       http://bugs.python.org/issue4367    created  ot
>>>>       patch, needs review
>>>>
>>>> A bug in ncurses.h still exists in FreeBSD 4.9 - 4.11            11/20/08
>>>>       http://bugs.python.org/issue4368    created  akitada
>>>>       patch
>>>>
>>>> Error building to a nonstandard prefix (with patch)              11/20/08
>>>>       http://bugs.python.org/issue4369    created  ot
>>>>       patch
>>>>
>>>> warning: unknown conversion type character `z' in format         11/20/08
>>>>       http://bugs.python.org/issue4370    created  akitada
>>>>
>>>>
>>>> coerce gone---but not from docs                                  11/20/08
>>>>       http://bugs.python.org/issue4371    created  LambertDW
>>>>
>>>>
>>>> __cmp__ removal not in What's New                                11/20/08
>>>> CLOSED http://bugs.python.org/issue4372    created  tjreedy
>>>>
>>>>
>>>> Reference leaks in Python 3.0rc3                                 11/21/08
>>>>       http://bugs.python.org/issue4373    created  christian.heimes
>>>>       patch
>>>>
>>>> Pickle tests fail w/o _pickle extension                          11/21/08
>>>>       http://bugs.python.org/issue4374    created  christian.heimes
>>>>
>>>>
>>>> inspect.getsource doesn't work with PYTHONPATH and source compil 11/21/08
>>>>       http://bugs.python.org/issue4375    created  erickt
>>>>
>>>>
>>>> Nested ctypes 'BigEndianStructure' fails                         11/21/08
>>>>       http://bugs.python.org/issue4376    created  castironpi
>>>>
>>>>
>>>> tokenize.detect_encoding() and Mac newline                       11/21/08
>>>>       http://bugs.python.org/issue4377    created  haypo
>>>>
>>>>
>>>> howto doc update                                                 11/21/08
>>>>       http://bugs.python.org/issue4378    created  LambertDW
>>>>
>>>>
>>>> Fix exception pickling: Move initial args assignment to BaseExce 11/18/08
>>>>       http://bugs.python.org/issue1692335 reopened gvanrossum
>>>>       patch
>>>>
>>>>
>>>>
>>>> Issues Now Closed (40)
>>>> ______________________
>>>>
>>>> Problem with doctest and decorated functions                      441 days
>>>>       http://bugs.python.org/issue1108    skip.montanaro
>>>>
>>>>
>>>> conditional jump to a POP_TOP optimization                        253 days
>>>>       http://bugs.python.org/issue2260    rhettinger
>>>>       patch
>>>>
>>>> NULL member in modules_by_index                                   131 days
>>>>       http://bugs.python.org/issue3327    loewis
>>>>       patch, easy
>>>>
>>>> test_cpickle crash on AMD64 Windows build                          88 days
>>>>       http://bugs.python.org/issue3640    loewis
>>>>       patch, 64bit
>>>>
>>>> pressing HOME key in IDLE editor ends IDLE                         81 days
>>>>       http://bugs.python.org/issue3679    tjreedy
>>>>
>>>>
>>>> configure --with-threads on cygwin => crash on thread related te   58 days
>>>>       http://bugs.python.org/issue3947    ocean-city
>>>>       patch
>>>>
>>>> ignored exceptions in generators (regression?)                     43 days
>>>>       http://bugs.python.org/issue4040    benjamin.peterson
>>>>
>>>>
>>>> Document the effects of NotImplemented on == and !=                43 days
>>>>       http://bugs.python.org/issue4087    marketdickinson
>>>>
>>>>
>>>> Documenting set comparisons and operations                         39 days
>>>>       http://bugs.python.org/issue4090    rhettinger
>>>>       patch
>>>>
>>>> name conflict in ScrolledCanvas.__init__() in Lib/turtle.py        36 days
>>>>       http://bugs.python.org/issue4116    loewis
>>>>       patch
>>>>
>>>> "__getattr__" can't be a descriptor                                19 days
>>>>       http://bugs.python.org/issue4230    benjamin.peterson
>>>>       patch
>>>>
>>>> Python 2.6 installer crashes when selecting 'advanced' and cance   10 days
>>>>       http://bugs.python.org/issue4289    loewis
>>>>
>>>>
>>>> email package with unicode subject/body                             8 days
>>>>       http://bugs.python.org/issue4306    barry
>>>>       patch
>>>>
>>>> Unicode in distutils meta-data?                                     4 days
>>>>       http://bugs.python.org/issue4312    loewis
>>>>
>>>>
>>>> IDLE segfault at exit                                               5 days
>>>>       http://bugs.python.org/issue4313    ocean-city
>>>>
>>>>
>>>> Improper use of [] in configure.in leads to useless regexp in co    2 days
>>>>       http://bugs.python.org/issue4316    georg.brandl
>>>>       patch
>>>>
>>>> Buffer overflow in imageop module                                   5 days
>>>>       http://bugs.python.org/issue4317    amaury.forgeotdarc
>>>>       patch
>>>>
>>>> unintended syntax error with decorators, parenthesis, and dots?     1 days
>>>>       http://bugs.python.org/issue4321    georg.brandl
>>>>
>>>>
>>>> locale documentation is inconsistent                                1 days
>>>>       http://bugs.python.org/issue4324    georg.brandl
>>>>
>>>>
>>>> type of UserList instance returns class instead of instance         3 days
>>>>       http://bugs.python.org/issue4326    georg.brandl
>>>>
>>>>
>>>> wsgiref.validate doesn't accept arguments to readline               1 days
>>>>       http://bugs.python.org/issue4330    ianb
>>>>
>>>>
>>>> asyncore.file_dispatcher does not use dup()'ed fd                   3 days
>>>>       http://bugs.python.org/issue4332    josiahcarlson
>>>>       patch
>>>>
>>>> Mac Build Script is broken for 2.6 release                          0 days
>>>>       http://bugs.python.org/issue4334    benjamin.peterson
>>>>
>>>>
>>>> Iteration over a map object with list()                             1 days
>>>>       http://bugs.python.org/issue4337    jmfauth
>>>>
>>>>
>>>> TypeError (bytes/str) in distutils command "upload"                 3 days
>>>>       http://bugs.python.org/issue4338    amaury.forgeotdarc
>>>>       patch
>>>>
>>>> Fix set-like dictview doc                                           0 days
>>>>       http://bugs.python.org/issue4339    benjamin.peterson
>>>>
>>>>
>>>> Update __hash__ doc                                                 0 days
>>>>       http://bugs.python.org/issue4341    benjamin.peterson
>>>>
>>>>
>>>> crash upon launch                                                   0 days
>>>>       http://bugs.python.org/issue4344    loewis
>>>>
>>>>
>>>> bytearray methods returning self                                    1 days
>>>>       http://bugs.python.org/issue4348    benjamin.peterson
>>>>       patch, needs review
>>>>
>>>> sys.path includes extraneous junk                                   0 days
>>>>       http://bugs.python.org/issue4349    benjamin.peterson
>>>>       patch, needs review
>>>>
>>>> Move description what a trace should should return to settrace f    1 days
>>>>       http://bugs.python.org/issue4353    benjamin.peterson
>>>>
>>>>
>>>> distutils.command.register is broken                                1 days
>>>>       http://bugs.python.org/issue4354    loewis
>>>>       patch
>>>>
>>>> Add "key" argument to "bisect" module functions                     1 days
>>>>       http://bugs.python.org/issue4356    marketdickinson
>>>>
>>>>
>>>> frozen?set operations create incorrectly initialized instances o    0 days
>>>>       http://bugs.python.org/issue4357    amaury.forgeotdarc
>>>>
>>>>
>>>> SystemError when method has both super() & closure                  0 days
>>>>       http://bugs.python.org/issue4360    barry
>>>>       patch
>>>>
>>>> FileIO object in io module                                          0 days
>>>>       http://bugs.python.org/issue4362    benjamin.peterson
>>>>       patch, needs review
>>>>
>>>> __cmp__ removal not in What's New                                   1 days
>>>>       http://bugs.python.org/issue4372    haypo
>>>>
>>>>
>>>> Using deiconify() hangs on exit                                  2646 days
>>>>       http://bugs.python.org/issue453489  amaury.forgeotdarc
>>>>
>>>>
>>>> sys.stdout problems with pythonw.exe                             1616 days
>>>>       http://bugs.python.org/issue973507  amaury.forgeotdarc
>>>>
>>>>
>>>> A subclass of set doesn't always have __init__ called.            547 days
>>>>       http://bugs.python.org/issue1721812 rhettinger
>>>>       patch
>>>>
>>>>
>>>>
>>>> Top Issues Most Discussed (10)
>>>> ______________________________
>>>>
>>>>  15 datetime: define division timedelta/timedelta                    208 days
>>>> open    http://bugs.python.org/issue2706
>>>>
>>>>  12 Tkinter binding involving Control-spacebar raises unicode error    4 days
>>>> open    http://bugs.python.org/issue1028
>>>>
>>>>  10 TypeError (bytes/str) in distutils command "upload"                3 days
>>>> closed  http://bugs.python.org/issue4338
>>>>
>>>>  10 type of UserList instance returns class instead of instance        3 days
>>>> closed  http://bugs.python.org/issue4326
>>>>
>>>>  10 missing update() in _Screen.setup() of Lib/turtle.py              39 days
>>>> open    http://bugs.python.org/issue4117
>>>>
>>>>  10 Byte/string inconsistencies between different dbm modules          1 days
>>>> open    http://bugs.python.org/issue3799
>>>>
>>>>  8 FileIO object in io module                                         0 days
>>>> closed  http://bugs.python.org/issue4362
>>>>
>>>>  8 bytearray methods returning self                                   1 days
>>>> closed  http://bugs.python.org/issue4348
>>>>
>>>>  7 Reference leaks in Python 3.0rc3                                   1 days
>>>> open    http://bugs.python.org/issue4373
>>>>
>>>>  7 SystemError when method has both super() & closure                 0 days
>>>> closed  http://bugs.python.org/issue4360
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Python-Dev mailing list
>>>> Python-Dev at python.org
>>>> http://mail.python.org/mailman/listinfo/python-dev
>>>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>>>>
>>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> http://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>>>
>>
>

From mrts at mrts.pri.ee  Mon Nov 24 19:43:17 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Mon, 24 Nov 2008 20:43:17 +0200
Subject: [Python-Dev] CVE tracking
Message-ID: <492AF5C5.60101@mrts.pri.ee>

> When I looked through that list a week or so ago, I noticed that some 
> issues were obviously related to the Python distribution itself, but 
> others were appeared to be Python application problems.

I looked through the list now and weeded out irrelevant CVEs (by putting them into
the ignore list in the script).
Also, now the output has descriptions of the CVEs as well, so it's more readable.

Improved output: http://dpaste.com/hold/93386/
Improved script (with a proper IGNORED_LIST): http://dpaste.com/hold/93388/

The results are much better:
5 OK's, 8 WARNings, 7 ERRORs.

Most of the errors are from 2007 or before, the only error from 2008 is an
obscure Tools/faqwiz/move-faqwiz.sh-related one.


From brett at python.org  Mon Nov 24 20:28:12 2008
From: brett at python.org (Brett Cannon)
Date: Mon, 24 Nov 2008 11:28:12 -0800
Subject: [Python-Dev] CVE tracking
In-Reply-To: <492AF5C5.60101@mrts.pri.ee>
References: <492AF5C5.60101@mrts.pri.ee>
Message-ID: <bbaeab100811241128k502c9e02n79cd3c71c9341c6@mail.gmail.com>

On Mon, Nov 24, 2008 at 10:43, Mart Somermaa <mrts at mrts.pri.ee> wrote:
>> When I looked through that list a week or so ago, I noticed that some
>> issues were obviously related to the Python distribution itself, but others
>> were appeared to be Python application problems.
>
> I looked through the list now and weeded out irrelevant CVEs (by putting
> them into
> the ignore list in the script).
> Also, now the output has descriptions of the CVEs as well, so it's more
> readable.
>
> Improved output: http://dpaste.com/hold/93386/
> Improved script (with a proper IGNORED_LIST): http://dpaste.com/hold/93388/
>
> The results are much better:
> 5 OK's, 8 WARNings, 7 ERRORs.
>
> Most of the errors are from 2007 or before, the only error from 2008 is an
> obscure Tools/faqwiz/move-faqwiz.sh-related one.
>

Thanks for doing this, Mart! But I know that at least for me I won't
be able to look at the list until some time after 3.0 is released. And
I suspect I am not the only member of the PSRT that this will be true
for.

If anyone wants to toss this list up on the wiki and go through to
help figure out what is needed for each (and either update the CVE as
needed or file an issue on the bug tracker mentioning the CVE; bonus
if you fix it as well) that would be helpful.

-Brett

From martin at v.loewis.de  Mon Nov 24 22:52:58 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 24 Nov 2008 22:52:58 +0100
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>
References: <20081121170645.3954E78534@psf.upfronthosting.co.za>
	<e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>
Message-ID: <492B223A.5050105@v.loewis.de>

Jeremy Hylton wrote:
> I wanted to ask a policy question on the bug tracker.  What are we
> doing with bugs filed against Python 2.4?
> This bug http://bugs.python.org/issue1208304 reports a fd leak in
> Python 2.4, which doesn't exist in the head.  Since Python 2.4 is in
> security-fix only mode, is it fair to close this with a won't fix?

Correct.

Regards,
Martin


From tjreedy at udel.edu  Tue Nov 25 01:03:32 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 24 Nov 2008 19:03:32 -0500
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <492B223A.5050105@v.loewis.de>
References: <20081121170645.3954E78534@psf.upfronthosting.co.za>	<e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>
	<492B223A.5050105@v.loewis.de>
Message-ID: <ggffcl$u9m$1@ger.gmane.org>

Martin v. L?wis wrote:
> Jeremy Hylton wrote:
>> I wanted to ask a policy question on the bug tracker.  What are we
>> doing with bugs filed against Python 2.4?
>> This bug http://bugs.python.org/issue1208304 reports a fd leak in
>> Python 2.4, which doesn't exist in the head.  Since Python 2.4 is in
>> security-fix only mode, is it fair to close this with a won't fix?
> 
> Correct.

If a bug exists and is fixed in a later release -- I would call it 'out 
of date' instead.


From martin at v.loewis.de  Tue Nov 25 01:36:02 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 25 Nov 2008 01:36:02 +0100
Subject: [Python-Dev] Summary of Python tracker Issues
In-Reply-To: <ggffcl$u9m$1@ger.gmane.org>
References: <20081121170645.3954E78534@psf.upfronthosting.co.za>	<e8bf7a530811240858v252f27eqd01cf6e8d30df3bc@mail.gmail.com>	<492B223A.5050105@v.loewis.de>
	<ggffcl$u9m$1@ger.gmane.org>
Message-ID: <492B4872.3050003@v.loewis.de>

>> Correct.
> 
> If a bug exists and is fixed in a later release -- I would call it 'out
> of date' instead.

Hmm. You have a point, but it should be "Fixed" then, and we have indeed
closed bug reports as "fixed" if later releases fixed the bug.

It probably doesn't really matter - if the complaint was specifically
that the bug is in 2.4.x, with a request to fix it in 2.4.x+1, then
it's "won't fix".

Regards,
Martin

From vinay_sajip at yahoo.co.uk  Tue Nov 25 10:00:43 2008
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 25 Nov 2008 09:00:43 +0000 (UTC)
Subject: [Python-Dev] Redirecting warnings.showwarning to logging
References: <loom.20081122T111103-661@post.gmane.org>
	<4928857D.4080204@Acm.Org> <gga1qu$6kq$1@ger.gmane.org>
Message-ID: <loom.20081125T085748-402@post.gmane.org>

Based on feedback here, I have a slightly revised proposal. Scott makes the
point that if the file parameter passed in to showwarning is not None, the
caller may be trying to extract output formatting. So, it's not enough just to
assign warnings.showwarning = logging.showwarning, since access to the original
warnings.showwarning may be needed. So, the public API in logging would be

def captureWarnings(capture):
    """
    If capture is true, redirect all warnings to the logging package.
    If capture is False, ensure that warnings are not redirected to logging
    but to their original destinations.
    """

Another new function, _showwarning( message, category, filename, lineno[, file
[, line]]) will be added to the logging package. This implementation of
showwarning will first check to see if the file parameter is None. If a file is
specified, it will delegate to the original warnings implementation of
showwarning. Otherwise, it will call warnings.formatwarning(message, category,
filename, lineno[, line]) and will log the resulting string to a warnings logger
named "py.warnings" [1] with level logging.WARNING. In order to configure
logging of warnings to any particular destination, the logging configuration
code will need to add appropriate handlers to the warnings logger. The precise
format of the logging message will be determined by the logging configuration in
effect, i.e. any formatters configured for the handlers attached to the logger.

I hope this covers everything, but please tell me if that's not the case.

Regards,


Vinay Sajip

[1] http://mail.python.org/pipermail/python-dev/2004-October/049282.html



From eric at trueblade.com  Tue Nov 25 13:38:28 2008
From: eric at trueblade.com (Eric Smith)
Date: Tue, 25 Nov 2008 07:38:28 -0500
Subject: [Python-Dev] format specification mini-language docs...
In-Reply-To: <350E7D38B6D819428718949920EC235556486D00A7@NA-EXMSG-C102.redmond.corp.microsoft.com>
References: <350E7D38B6D819428718949920EC235556486D00A7@NA-EXMSG-C102.redmond.corp.microsoft.com>
Message-ID: <492BF1C4.4050807@trueblade.com>

Dino Viehland wrote:

<previously discussed cases deleted>
> Finally providing any sign character seems to cause +1.0#INF and friends to be returned instead of inf as is documented:
> 
>>>> 10e667.__format__('+')
> '+1.0#INF'
>>>> 10e667.__format__('')
> 'inf'
> 
> 
> Are these just doc bugs?  The inf issue is the only one that seems particularly weird to me.

I think the inf one is a bug. Would you mind opening a bug and assigning 
it to me? Thanks.

Eric.


From davidbranniganz at gmail.com  Wed Nov 26 09:35:59 2008
From: davidbranniganz at gmail.com (kai)
Date: Wed, 26 Nov 2008 00:35:59 -0800
Subject: [Python-Dev] run 2to3 generated scripts under python2.6 + unicode +
	extension support
Message-ID: <d51964030811260035w3280f742w3eb8c0fcc65ac563@mail.gmail.com>

there's an alpha-release interpreter based on cpython2.6 that has
support for 2.6 extensions, & can automatically generate & import
files using 2to3 (& pure python3.0 scripts as well).  it consists of 3
files & is intended to coexist w/ an existing python2.6 installation

http://pypi.python.org/pypi/py3to2

enclosed is its patch summary against Python-2.6 (most of it deals w/
backporting 3.0 opcodes).

here's a test run of it (version 2008.11.23) importing most of
Python2.6's standard library regenerated by 2to3:







2TO3 COMPATIBILITY TEST
  $ python setup.py dev --py2to3test
  ...
  tested 200 2to3 generated scripts from 2.6.py3to2 (r26:66714, Nov 25
2008, 22:10:03)
  [GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] standard library

  0 skipped:


  23 couldn't import required modules:
  BaseHTTPServer CGIHTTPServer cgi cookielib copy dummy_threading
HTMLParser httplib macurl2path mimetypes os pdb pydoc re robotparser
sgmllib SimpleHTTPServer SimpleXMLRPCServer _strptime tempfile
threading urllib2 urllib

  6 were non-utf8 compliant scripts:
  base64 getopt heapq shlex smtpd tarfile

  14 failed import due to other reasons:
  anydbm dbhash doctest fractions _LWPCookieJar _MozillaCookieJar
pickle sets socket tabnanny UserList UserString whichdb xmlrpclib

  158 passed import:
  _abcoll abc aifc ast asynchat asyncore atexit audiodev Bastion bdb
binhex bisect calendar cgitb chunk cmd codecs codeop code collections
colorsys commands compileall ConfigParser contextlib Cookie copy_reg
cProfile csv decimal difflib dircache dis DocXMLRPCServer dumbdbm
dummy_thread filecmp fileinput fnmatch formatter fpformat ftplib
functools __future__ genericpath getpass gettext glob gzip hashlib
hmac htmlentitydefs htmllib ihooks imaplib imghdr imputil inspect io
keyword linecache locale macpath mailbox mailcap markupbase md5 mhlib
mimetools MimeWriter mimify modulefinder multifile mutex netrc new
nntplib ntpath nturl2path numbers opcode optparse os2emxpath
__phello__.foo pickletools pipes pkgutil platform plistlib popen2
poplib posixfile posixpath pprint profile pstats pty pyclbr py_compile
pydoc_topics Queue quopri random repr rexec rfc822 rlcompleter runpy
sched sha shelve shutil site smtplib sndhdr SocketServer sre_compile
sre_constants sre_parse sre ssl stat statvfs StringIO stringold
stringprep string struct subprocess sunaudio sunau symbol symtable
telnetlib textwrap this _threading_local timeit toaiff tokenize token
traceback trace tty types unittest urlparse UserDict user uuid uu
warnings wave weakref webbrowser xdrlib xmllib zipfile







PYTHON2.6 COMPATIBILITY TEST
  $ python setup.py dev --maketest
  ...
  324 tests OK.
  36 tests skipped:
      test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
      test_cd test_cl test_codecmaps_cn test_codecmaps_hk
      test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
      test_dl test_gdbm test_gl test_imageop test_imgfile test_kqueue
      test_linuxaudiodev test_macos test_macostools test_normalization
      test_ossaudiodev test_pep277 test_py3kwarn test_scriptpackages
      test_socketserver test_startfile test_sunaudiodev test_timeout
      test_urllib2net test_urllibnet test_winreg test_winsound
      test_zipfile64
  1 skip unexpected on linux2:
      test_gdbm
  completed 2.6 (r26:66714, Nov 17 2008, 13:35:14)
  [GCC 3.4.6 20060404 (Red Hat 3.4.6-10)] regression tests
-------------- next part --------------
A non-text attachment was scrubbed...
Name: py3to2.diff
Type: application/octet-stream
Size: 76483 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081126/1c876b4a/attachment-0001.obj>

From akitada at gmail.com  Wed Nov 26 18:28:47 2008
From: akitada at gmail.com (Akira Kitada)
Date: Thu, 27 Nov 2008 02:28:47 +0900
Subject: [Python-Dev] distutils doesn't use some compiler options when
	building
Message-ID: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>

Hi,

I encountered a weird problem using distutils.
Generally, distutils try to use the same compiler options used for
building Python interpreter,
but it looks like some of them are omitted sometimes.

- CPPFLAGS are not retrieved from the config and only ones in env are used.
- OPT is retrieved from the config, but it's only used when env has CFLAGS.

See: http://svn.python.org/view/python/trunk/Lib/distutils/sysconfig.py

"""
def customize_compiler(compiler):
...
    if compiler.compiler_type == "unix":
        (cc, cxx, opt, cflags, ccshared, ldshared, so_ext) = \
            get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
                            'CCSHARED', 'LDSHARED', 'SO')

        if 'CC' in os.environ:
            cc = os.environ['CC']
        if 'CXX' in os.environ:
            cxx = os.environ['CXX']
        if 'LDSHARED' in os.environ:
            ldshared = os.environ['LDSHARED']
        if 'CPP' in os.environ:
            cpp = os.environ['CPP']
        else:
            cpp = cc + " -E"           # not always
        if 'LDFLAGS' in os.environ:
            ldshared = ldshared + ' ' + os.environ['LDFLAGS']
        if 'CFLAGS' in os.environ:
            cflags = opt + ' ' + os.environ['CFLAGS']
            ldshared = ldshared + ' ' + os.environ['CFLAGS']
        if 'CPPFLAGS' in os.environ:
            cpp = cpp + ' ' + os.environ['CPPFLAGS']
            cflags = cflags + ' ' + os.environ['CPPFLAGS']
            ldshared = ldshared + ' ' + os.environ['CPPFLAGS']

        cc_cmd = cc + ' ' + cflags
        compiler.set_executables(
            preprocessor=cpp,
            compiler=cc_cmd,
            compiler_so=cc_cmd + ' ' + ccshared,
            compiler_cxx=cxx,
            linker_so=ldshared,
            linker_exe=cc)

        compiler.shared_lib_extension = so_ext
"""

Are these logics are intentional or just a bug?
If this is intentional behavior, why is that being this way?

Thanks,

From gerald.koenig at hp.com  Wed Nov 26 20:06:44 2008
From: gerald.koenig at hp.com (Koenig, Gerald)
Date: Wed, 26 Nov 2008 19:06:44 +0000
Subject: [Python-Dev] Python for windows.
In-Reply-To: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
Message-ID: <238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>

Hi all,

I am working For Hewlett-Packard designing PC Consumer Desktop

We have been including Python since over 10 years now on our systems.

It is a wonderful language and very powerful.

Now I am having a small issue and I was wondering how I can solve it.

So I would like to know who should I contact to be able to work on that issue together ?

The issue is about some requirement I need to have on our systems and even tools need to pass those requirement.

Regards

Gerald Koenig
Software Builds Architect
Hewlett-Packard
Work Phone:     +(1) 408 873 6202


From leif.walsh at gmail.com  Wed Nov 26 20:21:27 2008
From: leif.walsh at gmail.com (Leif Walsh)
Date: Wed, 26 Nov 2008 14:21:27 -0500
Subject: [Python-Dev] Python for windows.
In-Reply-To: <238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
Message-ID: <cc7430500811261121u69a72c5s52900ab34c9b3f51@mail.gmail.com>

This list is for the development of python.  Questions about
programming with python go to c.l.python or python-list at python dot
org.

If your question is about the development of python, you can probably
just ask here.

On Wed, Nov 26, 2008 at 2:06 PM, Koenig, Gerald <gerald.koenig at hp.com> wrote:
> Hi all,
>
> I am working For Hewlett-Packard designing PC Consumer Desktop
>
> We have been including Python since over 10 years now on our systems.
>
> It is a wonderful language and very powerful.
>
> Now I am having a small issue and I was wondering how I can solve it.
>
> So I would like to know who should I contact to be able to work on that issue together ?
>
> The issue is about some requirement I need to have on our systems and even tools need to pass those requirement.
>
> Regards
>
> Gerald Koenig
> Software Builds Architect
> Hewlett-Packard
> Work Phone:     +(1) 408 873 6202
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/leif.walsh%40gmail.com
>



-- 
Cheers,
Leif

From gerald.koenig at hp.com  Wed Nov 26 20:24:27 2008
From: gerald.koenig at hp.com (Koenig, Gerald)
Date: Wed, 26 Nov 2008 19:24:27 +0000
Subject: [Python-Dev] Python for windows.
In-Reply-To: <cc7430500811261121u69a72c5s52900ab34c9b3f51@mail.gmail.com>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<cc7430500811261121u69a72c5s52900ab34c9b3f51@mail.gmail.com>
Message-ID: <238A96A773B3934685A7269CC8A8D0423F7FBF7201@GVW0436EXB.americas.hpqcorp.net>

Hi,

This is a question about development how can python meet "OEM Ready programs".
Right now most of python is passing that programs but not all of it.

Right now some of the executable are failing some of the tests. Only few test are failings.

That why I need a contact I don't want to swamp that mailing list with a lot of things.

Gerald



-----Original Message-----
From: Leif Walsh [mailto:leif.walsh at gmail.com]
Sent: Wednesday, November 26, 2008 11:21 AM
To: Koenig, Gerald
Cc: python-dev at python.org
Subject: Re: [Python-Dev] Python for windows.

This list is for the development of python.  Questions about
programming with python go to c.l.python or python-list at python dot
org.

If your question is about the development of python, you can probably
just ask here.

On Wed, Nov 26, 2008 at 2:06 PM, Koenig, Gerald <gerald.koenig at hp.com> wrote:
> Hi all,
>
> I am working For Hewlett-Packard designing PC Consumer Desktop
>
> We have been including Python since over 10 years now on our systems.
>
> It is a wonderful language and very powerful.
>
> Now I am having a small issue and I was wondering how I can solve it.
>
> So I would like to know who should I contact to be able to work on that issue together ?
>
> The issue is about some requirement I need to have on our systems and even tools need to pass those requirement.
>
> Regards
>
> Gerald Koenig
> Software Builds Architect
> Hewlett-Packard
> Work Phone:     +(1) 408 873 6202
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/leif.walsh%40gmail.com
>



--
Cheers,
Leif

From brett at python.org  Wed Nov 26 20:48:45 2008
From: brett at python.org (Brett Cannon)
Date: Wed, 26 Nov 2008 11:48:45 -0800
Subject: [Python-Dev] Python for windows.
In-Reply-To: <238A96A773B3934685A7269CC8A8D0423F7FBF7201@GVW0436EXB.americas.hpqcorp.net>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<cc7430500811261121u69a72c5s52900ab34c9b3f51@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF7201@GVW0436EXB.americas.hpqcorp.net>
Message-ID: <bbaeab100811261148q268bbb31ta7533edfbf50e1a7@mail.gmail.com>

On Wed, Nov 26, 2008 at 11:24, Koenig, Gerald <gerald.koenig at hp.com> wrote:
> Hi,
>
> This is a question about development how can python meet "OEM Ready programs".
> Right now most of python is passing that programs but not all of it.
>
> Right now some of the executable are failing some of the tests. Only few test are failings.
>
> That why I need a contact I don't want to swamp that mailing list with a lot of things.
>

If the failings are because of a bug in Python, then bugs.python.org
or here is the right place. Otherwise comp.lang.python/python-list is
the proper place. Don't worry about swamping the list with questions;
it's there to answer questions and already gets lots of traffic.

-Brett

From martin at v.loewis.de  Wed Nov 26 21:03:59 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 26 Nov 2008 21:03:59 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <ggk94d$pr1$1@ger.gmane.org>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>
	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>
	<4923ABEF.50900@v.loewis.de>	<gg1jne$so8$1@ger.gmane.org>
	<49246BB6.7000607@v.loewis.de> <ggk94d$pr1$1@ger.gmane.org>
Message-ID: <492DABAF.5020808@v.loewis.de>

> I always wondered why it was necessary to write msi.py in the first 
> place. Maintaining it is surely a big effort and requires understanding
> of a dark library which a few people have (IMO it's a much higher effort 
> than setting up automated tests in a bunch of VM, which you said is "not 
> worth it").
> 
> There are plenty of MSI installer generator programs

Originally it was written because none of the MSI generator programs
were capable of packaging Python. In particular, none was capable of
creating 64-bit packages (which were first needed to create the
Itanium packages).

> and Python's needs 
> do not seem so weird to require a custom MSI generator.

Python's needs are fairly weird, so I'm very skeptical that any other
generator is capable of doing what msi.py does (or, if it was capable
of doing that, that it was then any simpler than msi.py).

The critical part is that you need a powerful way to specify what files
to package (having to select them in a UI is unacceptable, as the set
of files constantly changes - the current generator can cope with many
types of file additions without needing any change).

> I'm sure the 
> Python Software Foundation would easily get a free license of one of the 
> good commercial MSI installer generators.

Can you recommend a specific one?

In addition, I'm also skeptical wrt. commercial setup tools. We had been
using Wise for a while, and it was a management problem because the
license was only available on a single machine - so it was difficult
for anybody else to jump in and do a release.

> In short: if msi.py and the fact it breaks is part of the issue here, 
> it's very easy to solve in my opinion.

I'm very skeptical that this statement is actually true.

Regards,
Martin

From tjreedy at udel.edu  Wed Nov 26 21:17:41 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 26 Nov 2008 15:17:41 -0500
Subject: [Python-Dev] Python for windows.
In-Reply-To: <238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
Message-ID: <ggkat4$2q5$1@ger.gmane.org>

Koenig, Gerald wrote:

> I am working For Hewlett-Packard designing PC Consumer Desktop
> We have been including Python since over 10 years now on our systems.

I am writing this on a Pavilion that came with Python2.2.  I hope you 
are able to continue including Python.

> Now I am having a small issue and I was wondering how I can solve it.
> So I would like to know who should I contact to be able to work on that issue together ?
> The issue is about some requirement I need to have on our systems and even tools need to pass those requirement.

If you are looking for one person to give you professional support 
(which would not be me for your problem) with professional discretion 
for professional pay, you should say so.  As far as I know, the Python 
Software Foundation does not have an on-call corporate-support person.

If you are will to take your chances with volunteer support from whoever 
responds to each item, then you can choose between the general list for 
issues relating to past and current releases or the bug tracker or this 
list for issue relating to future releases (including Pythonx.y.z 
bug-fix releases).  Without more information (Which version is not 
passing "OEM Ready"? Have previous versions passed such?), it is hard to 
say more.

Terry Jan Reedy



From fuzzyman at voidspace.org.uk  Wed Nov 26 21:21:46 2008
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 26 Nov 2008 20:21:46 +0000
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <492DABAF.5020808@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>	<gg1jne$so8$1@ger.gmane.org>	<49246BB6.7000607@v.loewis.de>
	<ggk94d$pr1$1@ger.gmane.org> <492DABAF.5020808@v.loewis.de>
Message-ID: <492DAFDA.4080909@voidspace.org.uk>

Martin v. L?wis wrote:
>> I always wondered why it was necessary to write msi.py in the first 
>> place. Maintaining it is surely a big effort and requires understanding
>> of a dark library which a few people have (IMO it's a much higher effort 
>> than setting up automated tests in a bunch of VM, which you said is "not 
>> worth it").
>>
>> There are plenty of MSI installer generator programs
>>     
>
> Originally it was written because none of the MSI generator programs
> were capable of packaging Python. In particular, none was capable of
> creating 64-bit packages (which were first needed to create the
> Itanium packages).
>
>   
>> and Python's needs 
>> do not seem so weird to require a custom MSI generator.
>>     
>
> Python's needs are fairly weird, so I'm very skeptical that any other
> generator is capable of doing what msi.py does (or, if it was capable
> of doing that, that it was then any simpler than msi.py).
>
> The critical part is that you need a powerful way to specify what files
> to package (having to select them in a UI is unacceptable, as the set
> of files constantly changes - the current generator can cope with many
> types of file additions without needing any change).
>
>   

Wix is an msi creator (open source) that takes XML files as the input. 
It is also capable of creating 64bit installers. At Resolver Systems we 
use CPython scripts to generate the XML as input for Wix.

It would still need *some* code therefore, but maybe simpler if someone 
wanted to do the work. :-)

It would work well with the files being generated from an XML templating 
language like Mako which is what we will be switching to at Resolver 
Systems.

http://wix.sourceforge.net/

Michael Foord

>> I'm sure the 
>> Python Software Foundation would easily get a free license of one of the 
>> good commercial MSI installer generators.
>>     
>
> Can you recommend a specific one?
>
> In addition, I'm also skeptical wrt. commercial setup tools. We had been
> using Wise for a while, and it was a management problem because the
> license was only available on a single machine - so it was difficult
> for anybody else to jump in and do a release.
>
>   
>> In short: if msi.py and the fact it breaks is part of the issue here, 
>> it's very easy to solve in my opinion.
>>     
>
> I'm very skeptical that this statement is actually true.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>   


-- 
http://www.ironpythoninaction.com/
http://www.voidspace.org.uk/blog



From josepharmbruster at gmail.com  Wed Nov 26 21:22:55 2008
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Wed, 26 Nov 2008 15:22:55 -0500
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <492DABAF.5020808@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org> <4923ABEF.50900@v.loewis.de>
	<gg1jne$so8$1@ger.gmane.org> <49246BB6.7000607@v.loewis.de>
	<ggk94d$pr1$1@ger.gmane.org> <492DABAF.5020808@v.loewis.de>
Message-ID: <938f42d70811261222g6f027478re1776ca3014ab003@mail.gmail.com>

Martin,

What is the rationale behind using an MSI ?  Has anyone attempted to create
a Python installer using something a bit simpler, like NSIS [
http://nsis.sourceforge.net/Main_Page]?  If not, what are the reasons?

Joe

On Wed, Nov 26, 2008 at 3:03 PM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> > I always wondered why it was necessary to write msi.py in the first
> > place. Maintaining it is surely a big effort and requires understanding
> > of a dark library which a few people have (IMO it's a much higher effort
> > than setting up automated tests in a bunch of VM, which you said is "not
> > worth it").
> >
> > There are plenty of MSI installer generator programs
>
> Originally it was written because none of the MSI generator programs
> were capable of packaging Python. In particular, none was capable of
> creating 64-bit packages (which were first needed to create the
> Itanium packages).
>
> > and Python's needs
> > do not seem so weird to require a custom MSI generator.
>
> Python's needs are fairly weird, so I'm very skeptical that any other
> generator is capable of doing what msi.py does (or, if it was capable
> of doing that, that it was then any simpler than msi.py).
>
> The critical part is that you need a powerful way to specify what files
> to package (having to select them in a UI is unacceptable, as the set
> of files constantly changes - the current generator can cope with many
> types of file additions without needing any change).
>
> > I'm sure the
> > Python Software Foundation would easily get a free license of one of the
> > good commercial MSI installer generators.
>
> Can you recommend a specific one?
>
> In addition, I'm also skeptical wrt. commercial setup tools. We had been
> using Wise for a while, and it was a management problem because the
> license was only available on a single machine - so it was difficult
> for anybody else to jump in and do a release.
>
> > In short: if msi.py and the fact it breaks is part of the issue here,
> > it's very easy to solve in my opinion.
>
> I'm very skeptical that this statement is actually true.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/josepharmbruster%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081126/afdd7482/attachment.htm>

From martin at v.loewis.de  Wed Nov 26 21:29:28 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 26 Nov 2008 21:29:28 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
Message-ID: <492DB1A8.60909@v.loewis.de>

> Now I am having a small issue and I was wondering how I can solve it.
> 
> So I would like to know who should I contact to be able to work on that issue together ?

Please understand how open source development works: lots of volunteers,
few formal commitments.

If you feel it's a "political issue", and best resolved by signing
contracts and whatnot, contact the PSF.

If it's a technical issue that you consider a bug, or missing feature,
you can discuss it here - unless you can already formulate a specific
bug report, which should go to bugs.python.org.

If it's technical, but requires signing an NDA to learn what the issue
is, it will be very difficult. Everything we do in Python is public,
archived, indexed, and visible in any way you can imagine.

Regards,
Martin

From martin at v.loewis.de  Wed Nov 26 21:34:18 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 26 Nov 2008 21:34:18 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <492DAFDA.4080909@voidspace.org.uk>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>	<gg1jne$so8$1@ger.gmane.org>	<49246BB6.7000607@v.loewis.de>
	<ggk94d$pr1$1@ger.gmane.org> <492DABAF.5020808@v.loewis.de>
	<492DAFDA.4080909@voidspace.org.uk>
Message-ID: <492DB2CA.9020803@v.loewis.de>

> Wix is an msi creator (open source) that takes XML files as the input.
> It is also capable of creating 64bit installers. At Resolver Systems we
> use CPython scripts to generate the XML as input for Wix.
> 
> It would still need *some* code therefore, but maybe simpler if someone
> wanted to do the work. :-)

I had looked at WiX before, and found that it can't do out of the box
what I want to do - I still would need to generate the input files,
e.g. with a script (and I'm happy to hear that you can confirm that
analysis).

I also had quite some problems understanding it, and can understand
msi.py much better (surprise, surprise). For a newcomer, my feeling is
that learning WiX and learning msi.py is about the same effort - you
really need to "get" MSI files.

Regards,
Martin

From elliot at canonical.com  Wed Nov 26 21:20:05 2008
From: elliot at canonical.com (Elliot Murphy)
Date: Wed, 26 Nov 2008 15:20:05 -0500
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <492DABAF.5020808@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>	<gg1jne$so8$1@ger.gmane.org>	<49246BB6.7000607@v.loewis.de>
	<ggk94d$pr1$1@ger.gmane.org> <492DABAF.5020808@v.loewis.de>
Message-ID: <492DAF75.5050408@canonical.com>

Martin v. L?wis wrote:
>> I'm sure the 
>> Python Software Foundation would easily get a free license of one of the 
>> good commercial MSI installer generators.
> 
> Can you recommend a specific one?
> 
> In addition, I'm also skeptical wrt. commercial setup tools. We had been
> using Wise for a while, and it was a management problem because the
> license was only available on a single machine - so it was difficult
> for anybody else to jump in and do a release.
> 

I've also had terrible times with installshield and other things in the
past, but I've been very very pleased with WiX:

http://wix.sourceforge.net/

Free, open source, and it gave me absolute control over how the MSI and
MSM modules were built, using text files so I could store them in
version control, spit out installers from our automated build, etc. The
source format is XML (shrug), so on that project we even wrote a WiX
parser to calculate SCons dependencies all the way through the MSM to
the MSI, so that we could tell when a source file was changed what
installers needed to be resigned and shipped to customers.

It's really nice to be able to code review installer changes, and to
have automated builds spit out .msi files alongside the .debs and .rpms
and more.

-- 
Elliot Murphy | https://launchpad.net/~statik/

From martin at v.loewis.de  Wed Nov 26 21:39:27 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 26 Nov 2008 21:39:27 +0100
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <938f42d70811261222g6f027478re1776ca3014ab003@mail.gmail.com>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>	
	<87wsf0mqcr.fsf@xemacs.org> <4923ABEF.50900@v.loewis.de>	
	<gg1jne$so8$1@ger.gmane.org> <49246BB6.7000607@v.loewis.de>	
	<ggk94d$pr1$1@ger.gmane.org> <492DABAF.5020808@v.loewis.de>
	<938f42d70811261222g6f027478re1776ca3014ab003@mail.gmail.com>
Message-ID: <492DB3FF.8070801@v.loewis.de>

> What is the rationale behind using an MSI ?  Has anyone attempted to
> create a Python installer using something a bit simpler, like NSIS
> [http://nsis.sourceforge.net/Main_Page]?  If not, what are the reasons?

It's a lot of effort to look at any such tool (and I really mean a *lot*
of effort - like a full week). That's why nobody did it.

When I looked at MSI, I did it because it has a few unique features:
- it supports 64-bit installers, which now is an absolute requirement
  (people really do want to use the AMD64 binaries)
  a shallow look at the feature list of NSIS suggest that NSIS would
  fail this requirement.
- it supports installation through Windows Domain policy. I would be
  willing to drop this requirement, but I believe some users would not
  be happy. Nothing but MSI has this capability (by design of Windows
  Active Directory).

Regards,
Martin

From gerald.koenig at hp.com  Wed Nov 26 21:35:03 2008
From: gerald.koenig at hp.com (Koenig, Gerald)
Date: Wed, 26 Nov 2008 20:35:03 +0000
Subject: [Python-Dev] Python for windows.
In-Reply-To: <ggkat4$2q5$1@ger.gmane.org>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org>
Message-ID: <238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>

Hi all,

Right now we are including 2.5.2
I am planning on rolling to 2.6 very soon.

I completely understand that this is a volunteer organization.
That why I am already proposing that HP will submit for your guys after we figure out how to fix the issues if it is possible to fix them of course.

The OEM Ready program is: http://msdn.microsoft.com/en-us/windowsvista/cc315067.aspx
Personally I am ready to do the job of submitting and testing if you pass all the tests.

You can find some info about all those test here also:
        http://www.codeproject.com/KB/vista/Certified_for_Vista.aspx?display=PrintAll&fid=406261&df=90&mpp=25&noise=3&sort=Position&view=Quick&fr=26


The list of issues are pretty simple:

        - Some of the executable deliver in the python package It does not have manifest that is compliant with UAC guidelines..
                c:\program files\python\lib\distutils\command\\wininst-6.0.exe
                c:\program files\python\lib\distutils\command\\wininst-7.1.exe
                c:\program files\python\lib\distutils\command\\wininst-8.0.exe

                We could add a manifest manually outside the executable if there is no manifest at all in those.

        - This c:\windows\installer\{110eb5c4-e995-4cfb-ab80-a5f315bea9e8}\python_icon.exe do not answers to the Restart Manager Aware
                Not sure what can be the solution for that one.

        - Last one is some of your executable do not contain any version numbers inside.

Gerald







-----Original Message-----
From: python-dev-bounces+gerald.koenig=hp.com at python.org [mailto:python-dev-bounces+gerald.koenig=hp.com at python.org] On Behalf Of Terry Reedy
Sent: Wednesday, November 26, 2008 12:18 PM
To: python-dev at python.org
Subject: Re: [Python-Dev] Python for windows.

Koenig, Gerald wrote:

> I am working For Hewlett-Packard designing PC Consumer Desktop
> We have been including Python since over 10 years now on our systems.

I am writing this on a Pavilion that came with Python2.2.  I hope you
are able to continue including Python.

> Now I am having a small issue and I was wondering how I can solve it.
> So I would like to know who should I contact to be able to work on that issue together ?
> The issue is about some requirement I need to have on our systems and even tools need to pass those requirement.

If you are looking for one person to give you professional support
(which would not be me for your problem) with professional discretion
for professional pay, you should say so.  As far as I know, the Python
Software Foundation does not have an on-call corporate-support person.

If you are will to take your chances with volunteer support from whoever
responds to each item, then you can choose between the general list for
issues relating to past and current releases or the bug tracker or this
list for issue relating to future releases (including Pythonx.y.z
bug-fix releases).  Without more information (Which version is not
passing "OEM Ready"? Have previous versions passed such?), it is hard to
say more.

Terry Jan Reedy


_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/gerald.koenig%40hp.com

From gerald.koenig at hp.com  Wed Nov 26 21:53:17 2008
From: gerald.koenig at hp.com (Koenig, Gerald)
Date: Wed, 26 Nov 2008 20:53:17 +0000
Subject: [Python-Dev] Python for windows.
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org> 
Message-ID: <238A96A773B3934685A7269CC8A8D0423F7FBF72A1@GVW0436EXB.americas.hpqcorp.net>

Hi Terry,

And like you I really want to keep python on our systems.

That why I am contacting python Foundation and try to fix those issues together.

We are already running all those tests in house here.

Gerald




-----Original Message-----
From: Koenig, Gerald
Sent: Wednesday, November 26, 2008 12:35 PM
To: python-dev at python.org
Subject: RE: [Python-Dev] Python for windows.

Hi all,

Right now we are including 2.5.2
I am planning on rolling to 2.6 very soon.

I completely understand that this is a volunteer organization.
That why I am already proposing that HP will submit for your guys after we figure out how to fix the issues if it is possible to fix them of course.

The OEM Ready program is: http://msdn.microsoft.com/en-us/windowsvista/cc315067.aspx
Personally I am ready to do the job of submitting and testing if you pass all the tests.

You can find some info about all those test here also:
        http://www.codeproject.com/KB/vista/Certified_for_Vista.aspx?display=PrintAll&fid=406261&df=90&mpp=25&noise=3&sort=Position&view=Quick&fr=26


The list of issues are pretty simple:

        - Some of the executable deliver in the python package It does not have manifest that is compliant with UAC guidelines..
                c:\program files\python\lib\distutils\command\\wininst-6.0.exe
                c:\program files\python\lib\distutils\command\\wininst-7.1.exe
                c:\program files\python\lib\distutils\command\\wininst-8.0.exe

                We could add a manifest manually outside the executable if there is no manifest at all in those.

        - This c:\windows\installer\{110eb5c4-e995-4cfb-ab80-a5f315bea9e8}\python_icon.exe do not answers to the Restart Manager Aware
                Not sure what can be the solution for that one.

        - Last one is some of your executable do not contain any version numbers inside.

Gerald







-----Original Message-----
From: python-dev-bounces+gerald.koenig=hp.com at python.org [mailto:python-dev-bounces+gerald.koenig=hp.com at python.org] On Behalf Of Terry Reedy
Sent: Wednesday, November 26, 2008 12:18 PM
To: python-dev at python.org
Subject: Re: [Python-Dev] Python for windows.

Koenig, Gerald wrote:

> I am working For Hewlett-Packard designing PC Consumer Desktop
> We have been including Python since over 10 years now on our systems.

I am writing this on a Pavilion that came with Python2.2.  I hope you
are able to continue including Python.

> Now I am having a small issue and I was wondering how I can solve it.
> So I would like to know who should I contact to be able to work on that issue together ?
> The issue is about some requirement I need to have on our systems and even tools need to pass those requirement.

If you are looking for one person to give you professional support
(which would not be me for your problem) with professional discretion
for professional pay, you should say so.  As far as I know, the Python
Software Foundation does not have an on-call corporate-support person.

If you are will to take your chances with volunteer support from whoever
responds to each item, then you can choose between the general list for
issues relating to past and current releases or the bug tracker or this
list for issue relating to future releases (including Pythonx.y.z
bug-fix releases).  Without more information (Which version is not
passing "OEM Ready"? Have previous versions passed such?), it is hard to
say more.

Terry Jan Reedy


_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/gerald.koenig%40hp.com

From martin at v.loewis.de  Wed Nov 26 22:54:05 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 26 Nov 2008 22:54:05 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <1227736209.6739.9.camel@ozzu>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>	
	<87wsf0mqcr.fsf@xemacs.org> <4923ABEF.50900@v.loewis.de>	
	<gg1jne$so8$1@ger.gmane.org> <49246BB6.7000607@v.loewis.de>	
	<ggk94d$pr1$1@ger.gmane.org> <492DABAF.5020808@v.loewis.de>
	<1227736209.6739.9.camel@ozzu>
Message-ID: <492DC57D.1010101@v.loewis.de>

> I've had good results with Advanced Installer:
> http://www.advancedinstaller.com/feats-list.html

So how much effort would it be to create a Python installer?
Could you kindly provide one?

Regards,
Martin

From rasky at develer.com  Wed Nov 26 22:50:09 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Wed, 26 Nov 2008 22:50:09 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <492DABAF.5020808@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org> <4923ABEF.50900@v.loewis.de>
	<gg1jne$so8$1@ger.gmane.org> <49246BB6.7000607@v.loewis.de>
	<ggk94d$pr1$1@ger.gmane.org>  <492DABAF.5020808@v.loewis.de>
Message-ID: <1227736209.6739.9.camel@ozzu>

On mer, 2008-11-26 at 21:03 +0100, "Martin v. L?wis" wrote:

> > I'm sure the 
> > Python Software Foundation would easily get a free license of one of the 
> > good commercial MSI installer generators.
> 
> Can you recommend a specific one?

I've had good results with Advanced Installer:
http://www.advancedinstaller.com/feats-list.html

It does support 64-bit packages, and it uses a XML file as input. It
supports Vista and UAC, per-user and per-machine install, registry
modification, environment variables, upgrades/downgrades/side installs,
online installs. And it's free as in beer. The commercial version has
many more features, but I don't think Python needs them.

But the basic idea is that this tool totally abstracts the MSI details.
I know *nothing* of MSI but I'm fully able to use this tool and produce
installers with more features than those I notice within Python's
installer.
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com



From mhammond at skippinet.com.au  Wed Nov 26 22:59:15 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 27 Nov 2008 08:59:15 +1100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
Message-ID: <042401c95012$3bff99d0$b3fecd70$@com.au>

> I completely understand that this is a volunteer organization.
> That why I am already proposing that HP will submit for your guys after
> we figure out how to fix the issues if it is possible to fix them of
> course.

I don't fully understand why it is in HPs interests to install a normal
python.dev installer, and have their other bundled software rely on it
working forever.  IMO it is likely people will not understand the
association, and may upgrade or remove the package.

Wouldn't it make more sense for HP to treat Python as a "private" tool and
bundle it hidden away inside their applications?  This is the exact approach
many applications written in Python take - I can't think of any commercial
applications that rely on the standard installer - think the Python
implemented IDEs, the various Zope windows binaries, etc.

Or obviously I'm missing some key point :)

> The OEM Ready program is: http://msdn.microsoft.com/en-
> us/windowsvista/cc315067.aspx
> Personally I am ready to do the job of submitting and testing if you
> pass all the tests.
> 
> You can find some info about all those test here also:
> 
> http://www.codeproject.com/KB/vista/Certified_for_Vista.aspx?display=Pr
> intAll&fid=406261&df=90&mpp=25&noise=3&sort=Position&view=Quick&fr=26

But these are written with applications in mind - Python isn't an
application - its used to *write* applications.  I don't see a good reason
to support these guidelines.  I do see a reason to help support people
ensure their Python implemented applications can meet the guidelines, but
I'd never advise such people to install the python.org installer and have
their application use Python from that directory. 

In other words: Python implemented apps should meet the guidelines, but as
such apps shouldn't use the standard installer, there is no need for Python
itself to meet them.

>         - Some of the executable deliver in the python package It does
> not have manifest that is compliant with UAC guidelines..
>                 c:\program files\python\lib\distutils\command\\wininst-
> 6.0.exe
>                 c:\program files\python\lib\distutils\command\\wininst-
> 7.1.exe
>                 c:\program files\python\lib\distutils\command\\wininst-
> 8.0.exe

These are stubs for installers.  It is the installers created from these
stubs that need the manifest, as the manifest requirements will differ for
each use of the stub.

Ditto for python.exe etc - its impossible to add a reasonable manifest, as
the manifest requirements would depend entirely on what the python program
itself does.  There is no way python.exe can know what it will be asked to
do.

> 
>                 We could add a manifest manually outside the executable
> if there is no manifest at all in those.

See above - we don't know ahead of time what an appropriate manifest is.  I
agree manifests need thought and work for Python, but I'm fairly sure the
answer isn't to try and come up with them ahead of time and assume they
apply universally.

>         - This c:\windows\installer\{110eb5c4-e995-4cfb-ab80-
> a5f315bea9e8}\python_icon.exe do not answers to the Restart Manager
> Aware

That sounds like something nice to have, but assuming we recommend against
using the standard Python installer for 3rd party applications, this is a
problem for the application installer, not ours.  In other words, I'm sure
we'd welcome a fix to this, but I'd still recommend against using our
installer anyway!

>         - Last one is some of your executable do not contain any
> version numbers inside.

If the binaries don't currently include the python version number, I'd agree
that would be a nice addition.

Maybe your and others lives would be made easier if the Python MSI installer
was split up to support "merge modules"?  That way you could roll your own
installer that met any guidelines or requirements that mattered to you,
while still ensuring you got all the files that the official installer would
have installed.  I'm hand-waving a little here, but it sounds like we can do
more to help *others* meet guidelines for their Python based apps instead of
trying and meet it for our installer...

Cheers,

Mark


From gerald.koenig at hp.com  Wed Nov 26 23:02:39 2008
From: gerald.koenig at hp.com (Koenig, Gerald)
Date: Wed, 26 Nov 2008 22:02:39 +0000
Subject: [Python-Dev] Python for windows.
In-Reply-To: <042401c95012$3bff99d0$b3fecd70$@com.au>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
Message-ID: <238A96A773B3934685A7269CC8A8D0423F7FBF72F6@GVW0436EXB.americas.hpqcorp.net>

Hi Mark,

Well we are having a lot of our teams using python are the script languages :)

It is not only one team using it

That why we use the normal installer :)

Gerald

-----Original Message-----
From: Mark Hammond [mailto:skippy.hammond at gmail.com] On Behalf Of Mark Hammond
Sent: Wednesday, November 26, 2008 1:59 PM
To: Koenig, Gerald; python-dev at python.org
Subject: RE: [Python-Dev] Python for windows.

> I completely understand that this is a volunteer organization.
> That why I am already proposing that HP will submit for your guys after
> we figure out how to fix the issues if it is possible to fix them of
> course.

I don't fully understand why it is in HPs interests to install a normal
python.dev installer, and have their other bundled software rely on it
working forever.  IMO it is likely people will not understand the
association, and may upgrade or remove the package.

Wouldn't it make more sense for HP to treat Python as a "private" tool and
bundle it hidden away inside their applications?  This is the exact approach
many applications written in Python take - I can't think of any commercial
applications that rely on the standard installer - think the Python
implemented IDEs, the various Zope windows binaries, etc.

Or obviously I'm missing some key point :)

> The OEM Ready program is: http://msdn.microsoft.com/en-
> us/windowsvista/cc315067.aspx
> Personally I am ready to do the job of submitting and testing if you
> pass all the tests.
>
> You can find some info about all those test here also:
>
> http://www.codeproject.com/KB/vista/Certified_for_Vista.aspx?display=Pr
> intAll&fid=406261&df=90&mpp=25&noise=3&sort=Position&view=Quick&fr=26

But these are written with applications in mind - Python isn't an
application - its used to *write* applications.  I don't see a good reason
to support these guidelines.  I do see a reason to help support people
ensure their Python implemented applications can meet the guidelines, but
I'd never advise such people to install the python.org installer and have
their application use Python from that directory.

In other words: Python implemented apps should meet the guidelines, but as
such apps shouldn't use the standard installer, there is no need for Python
itself to meet them.

>         - Some of the executable deliver in the python package It does
> not have manifest that is compliant with UAC guidelines..
>                 c:\program files\python\lib\distutils\command\\wininst-
> 6.0.exe
>                 c:\program files\python\lib\distutils\command\\wininst-
> 7.1.exe
>                 c:\program files\python\lib\distutils\command\\wininst-
> 8.0.exe

These are stubs for installers.  It is the installers created from these
stubs that need the manifest, as the manifest requirements will differ for
each use of the stub.

Ditto for python.exe etc - its impossible to add a reasonable manifest, as
the manifest requirements would depend entirely on what the python program
itself does.  There is no way python.exe can know what it will be asked to
do.

>
>                 We could add a manifest manually outside the executable
> if there is no manifest at all in those.

See above - we don't know ahead of time what an appropriate manifest is.  I
agree manifests need thought and work for Python, but I'm fairly sure the
answer isn't to try and come up with them ahead of time and assume they
apply universally.

>         - This c:\windows\installer\{110eb5c4-e995-4cfb-ab80-
> a5f315bea9e8}\python_icon.exe do not answers to the Restart Manager
> Aware

That sounds like something nice to have, but assuming we recommend against
using the standard Python installer for 3rd party applications, this is a
problem for the application installer, not ours.  In other words, I'm sure
we'd welcome a fix to this, but I'd still recommend against using our
installer anyway!

>         - Last one is some of your executable do not contain any
> version numbers inside.

If the binaries don't currently include the python version number, I'd agree
that would be a nice addition.

Maybe your and others lives would be made easier if the Python MSI installer
was split up to support "merge modules"?  That way you could roll your own
installer that met any guidelines or requirements that mattered to you,
while still ensuring you got all the files that the official installer would
have installed.  I'm hand-waving a little here, but it sounds like we can do
more to help *others* meet guidelines for their Python based apps instead of
trying and meet it for our installer...

Cheers,

Mark


From gerald.koenig at hp.com  Wed Nov 26 23:11:35 2008
From: gerald.koenig at hp.com (Koenig, Gerald)
Date: Wed, 26 Nov 2008 22:11:35 +0000
Subject: [Python-Dev] Python for windows.
In-Reply-To: <238A96A773B3934685A7269CC8A8D0423F7FBF72F6@GVW0436EXB.americas.hpqcorp.net>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
	<238A96A773B3934685A7269CC8A8D0423F7FBF72F6@GVW0436EXB.americas.hpqcorp.net>
Message-ID: <238A96A773B3934685A7269CC8A8D0423F7FBF7303@GVW0436EXB.americas.hpqcorp.net>

Hi Mark,

See below my answers,

We install your installer, for people to use python are the main language script on the system.

My team recommend others team to use this as the main scripting languages for different software's, Due to that we use the standard install and people developed their script on the systems.

We also leave it on the system for our customer later  if they want to use special function of our consumer desktop like application drivers recovery for example.

As it is on the system it would need to be passing those requirement for OEM ready because we use python like an application in our case to developed script / programs using your normal installer.
This is the main reason we have it on all our system since over 10 years now :)

Gerald



-----Original Message-----
From: python-dev-bounces+gerald.koenig=hp.com at python.org [mailto:python-dev-bounces+gerald.koenig=hp.com at python.org] On Behalf Of Koenig, Gerald
Sent: Wednesday, November 26, 2008 2:03 PM
To: mhammond at skippinet.com.au; python-dev at python.org
Subject: Re: [Python-Dev] Python for windows.

Hi Mark,

Well we are having a lot of our teams using python are the script languages :)

It is not only one team using it

That why we use the normal installer :)

Gerald

-----Original Message-----
From: Mark Hammond [mailto:skippy.hammond at gmail.com] On Behalf Of Mark Hammond
Sent: Wednesday, November 26, 2008 1:59 PM
To: Koenig, Gerald; python-dev at python.org
Subject: RE: [Python-Dev] Python for windows.

> I completely understand that this is a volunteer organization.
> That why I am already proposing that HP will submit for your guys after
> we figure out how to fix the issues if it is possible to fix them of
> course.

I don't fully understand why it is in HPs interests to install a normal
python.dev installer, and have their other bundled software rely on it
working forever.  IMO it is likely people will not understand the
association, and may upgrade or remove the package.

Wouldn't it make more sense for HP to treat Python as a "private" tool and
bundle it hidden away inside their applications?  This is the exact approach
many applications written in Python take - I can't think of any commercial
applications that rely on the standard installer - think the Python
implemented IDEs, the various Zope windows binaries, etc.

Or obviously I'm missing some key point :)

> The OEM Ready program is: http://msdn.microsoft.com/en-
> us/windowsvista/cc315067.aspx
> Personally I am ready to do the job of submitting and testing if you
> pass all the tests.
>
> You can find some info about all those test here also:
>
> http://www.codeproject.com/KB/vista/Certified_for_Vista.aspx?display=Pr
> intAll&fid=406261&df=90&mpp=25&noise=3&sort=Position&view=Quick&fr=26

But these are written with applications in mind - Python isn't an
application - its used to *write* applications.  I don't see a good reason
to support these guidelines.  I do see a reason to help support people
ensure their Python implemented applications can meet the guidelines, but
I'd never advise such people to install the python.org installer and have
their application use Python from that directory.

In other words: Python implemented apps should meet the guidelines, but as
such apps shouldn't use the standard installer, there is no need for Python
itself to meet them.

>         - Some of the executable deliver in the python package It does
> not have manifest that is compliant with UAC guidelines..
>                 c:\program files\python\lib\distutils\command\\wininst-
> 6.0.exe
>                 c:\program files\python\lib\distutils\command\\wininst-
> 7.1.exe
>                 c:\program files\python\lib\distutils\command\\wininst-
> 8.0.exe

These are stubs for installers.  It is the installers created from these
stubs that need the manifest, as the manifest requirements will differ for
each use of the stub.

Ditto for python.exe etc - its impossible to add a reasonable manifest, as
the manifest requirements would depend entirely on what the python program
itself does.  There is no way python.exe can know what it will be asked to
do.

>
>                 We could add a manifest manually outside the executable
> if there is no manifest at all in those.

See above - we don't know ahead of time what an appropriate manifest is.  I
agree manifests need thought and work for Python, but I'm fairly sure the
answer isn't to try and come up with them ahead of time and assume they
apply universally.

>         - This c:\windows\installer\{110eb5c4-e995-4cfb-ab80-
> a5f315bea9e8}\python_icon.exe do not answers to the Restart Manager
> Aware

That sounds like something nice to have, but assuming we recommend against
using the standard Python installer for 3rd party applications, this is a
problem for the application installer, not ours.  In other words, I'm sure
we'd welcome a fix to this, but I'd still recommend against using our
installer anyway!

>         - Last one is some of your executable do not contain any
> version numbers inside.

If the binaries don't currently include the python version number, I'd agree
that would be a nice addition.

Maybe your and others lives would be made easier if the Python MSI installer
was split up to support "merge modules"?  That way you could roll your own
installer that met any guidelines or requirements that mattered to you,
while still ensuring you got all the files that the official installer would
have installed.  I'm hand-waving a little here, but it sounds like we can do
more to help *others* meet guidelines for their Python based apps instead of
trying and meet it for our installer...

Cheers,

Mark

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/gerald.koenig%40hp.com

From lists at cheimes.de  Wed Nov 26 23:15:43 2008
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 26 Nov 2008 23:15:43 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <1227736209.6739.9.camel@ozzu>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>
	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>
	<4923ABEF.50900@v.loewis.de>	<gg1jne$so8$1@ger.gmane.org>
	<49246BB6.7000607@v.loewis.de>	<ggk94d$pr1$1@ger.gmane.org>
	<492DABAF.5020808@v.loewis.de> <1227736209.6739.9.camel@ozzu>
Message-ID: <492DCA8F.4050105@cheimes.de>

Giovanni Bajo wrote:
> On mer, 2008-11-26 at 21:03 +0100, "Martin v. L?wis" wrote:
> 
>>> I'm sure the 
>>> Python Software Foundation would easily get a free license of one of the 
>>> good commercial MSI installer generators.
>> Can you recommend a specific one?
> 
> I've had good results with Advanced Installer:
> http://www.advancedinstaller.com/feats-list.html
> 
> It does support 64-bit packages, and it uses a XML file as input. It
> supports Vista and UAC, per-user and per-machine install, registry
> modification, environment variables, upgrades/downgrades/side installs,
> online installs. And it's free as in beer. The commercial version has
> many more features, but I don't think Python needs them.

The free edition is missing at least one important feature:

Merge Modules into your installation
Create self-contained MSI packages, by including and configuring the 
required merge modules.

Christian

From martin at v.loewis.de  Wed Nov 26 23:17:36 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 26 Nov 2008 23:17:36 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
Message-ID: <492DCB00.4010506@v.loewis.de>

>         - Some of the executable deliver in the python package It does not have manifest that is compliant with UAC guidelines..
>                 c:\program files\python\lib\distutils\command\\wininst-6.0.exe
>                 c:\program files\python\lib\distutils\command\\wininst-7.1.exe
>                 c:\program files\python\lib\distutils\command\\wininst-8.0.exe

Hmm. These binaries are not meant to be run as executables themselves.
Instead, they are meant to be integrated into setup programs as-is.
wininst-6.0.exe, in particular, is created by MSVC 6.0, which isn't
even capable of dealing with manifests.

Would it help compliance if we renamed them into .dat, so that the
conformance test doesn't recognize them as .exe files? If that doesn't
trick it, we could also binary-invert the contents (or the first 4 bytes).

>         - This c:\windows\installer\{110eb5c4-e995-4cfb-ab80-a5f315bea9e8}\python_icon.exe do not answers to the Restart Manager Aware
>                 Not sure what can be the solution for that one.

Can you please elaborate? How does one detect that something is Restart
Manager Aware? The file, again, is not ever meant to be run. Instead,
it only serves as a container for icons. If it helps to use a different
icon container (e.g. a DLL), that might solve the problem.
Alternatively, if you can suggest the most trivial "hello world" style
application that is Restart Manager Aware, that would also probably
solve the problem.

See PC/empty.c for the source of python_icons.exe.

>         - Last one is some of your executable do not contain any version numbers inside.

I was tempted to say "this is not true", but it probably is - only
pythonxy.dll has the version resource.

Patches are welcome. IIUC, PC/python_nt.rc is where the version resource
for pythonxy.dll is defined, and PC/python_exe.rc defines the, well,
.exe resources. For compliance, it's probably ok if python_exe.rc
includes python_nt.rc, but it would probably be better if the version
of python.exe didn't say "Python DLL" for "InternalName". So if you can
come up with a series of defines and #includes that solves it all
without any code duplication - that would be appreciated.

For the other executables (other than python.exe and pythonw.exe):
they aren't really executables. So if renaming them to .dll or
even to something else solves the problem, we can happily do that.
If they really need to get a version resource, so be it - they
should all be version 1.0, and InternalName "we just included
a version resource so that Microsoft's branding is happy".

Regards,
Martin

From mhammond at skippinet.com.au  Wed Nov 26 23:31:29 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 27 Nov 2008 09:31:29 +1100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492DCB00.4010506@v.loewis.de>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<492DCB00.4010506@v.loewis.de>
Message-ID: <043101c95016$bcadf730$3609e590$@com.au>

Martin writes:

> >                 c:\program
> files\python\lib\distutils\command\\wininst-8.0.exe
> 
> Hmm. These binaries are not meant to be run as executables themselves.
> Instead, they are meant to be integrated into setup programs as-is.
> wininst-6.0.exe, in particular, is created by MSVC 6.0, which isn't
> even capable of dealing with manifests.
> 
> Would it help compliance if we renamed them into .dat, so that the
> conformance test doesn't recognize them as .exe files? 

That is probably a good idea regardless of the outcome of this discussion.

Cheers,

Mark



From martin at v.loewis.de  Wed Nov 26 23:31:58 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 26 Nov 2008 23:31:58 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <042401c95012$3bff99d0$b3fecd70$@com.au>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
Message-ID: <492DCE5E.5080602@v.loewis.de>

> But these are written with applications in mind - Python isn't an
> application - its used to *write* applications.  I don't see a good reason
> to support these guidelines.  I do see a reason to help support people
> ensure their Python implemented applications can meet the guidelines, but
> I'd never advise such people to install the python.org installer and have
> their application use Python from that directory. 

I disagree. While it might not be really necessary for HP to have Python
comply to a formal test suite, I'm sure other users do need to pass
these test suites for whatever reason, and they are unhappy if they
don't pass out of the box.

Cleary, "we" (python-dev) would never run any of these test suites; it's
to much effort to buy them, run them, and study the results. So I'm
really glad HP spends the time on that stuff - not because HP needs it,
but because "smaller" users might run into it, and then frustrate
because they can't resolve the issues (rather than contacting us).

> These are stubs for installers.  It is the installers created from these
> stubs that need the manifest, as the manifest requirements will differ for
> each use of the stub.

See my other message. This "obviously" comes from a formal test suite.
So it's not necessary to convince the operator of the test suite that
the test is flawed, but to convince the test suite to look elsewhere.
I think we can do that.

> Ditto for python.exe etc - its impossible to add a reasonable manifest, as
> the manifest requirements would depend entirely on what the python program
> itself does.  There is no way python.exe can know what it will be asked to
> do.

FWIW, python.exe *does* have a manifest. The manifest is primarily about
static linking. I don't know what other uses a manifest may have, but if
it makes the test suite happy that python.exe has a manifest, then I'm
happy myself also.

> See above - we don't know ahead of time what an appropriate manifest is.  I
> agree manifests need thought and work for Python, but I'm fairly sure the
> answer isn't to try and come up with them ahead of time and assume they
> apply universally.

Can you please elaborate? What specific elements of the manifest are you
thinking of that are not universally correct for python.exe?

>>         - This c:\windows\installer\{110eb5c4-e995-4cfb-ab80-
>> a5f315bea9e8}\python_icon.exe do not answers to the Restart Manager
>> Aware
> 
> That sounds like something nice to have, but assuming we recommend against
> using the standard Python installer for 3rd party applications, this is a
> problem for the application installer, not ours.  In other words, I'm sure
> we'd welcome a fix to this, but I'd still recommend against using our
> installer anyway!

Perhaps. I wouldn't go so far, though. It's surely puzzling if a system
comes with a pre-installed Python, but if that Python actually works,
I don't think that does much damage. Of course, anybody embedding Python
*should* integrate it into his own application and installation
procedure. If we really want this guideline to be followed, we should
document explicitly how one does that (and no, pointing people to py2exe
is not sufficient).

> Maybe your and others lives would be made easier if the Python MSI installer
> was split up to support "merge modules"?  That way you could roll your own
> installer that met any guidelines or requirements that mattered to you,
> while still ensuring you got all the files that the official installer would
> have installed.  I'm hand-waving a little here, but it sounds like we can do
> more to help *others* meet guidelines for their Python based apps instead of
> trying and meet it for our installer...

If this list of complaints was complete, I don't find it too difficult
to comply. If a merge module would contain the entire library (including
distutils), then the wininst.exe issue would still exist for the merge
module (for example).

OTOH, if somebody wanted to contribute a procedure that builds merge
modules (in whatever granularity) along with the msi file - that would
also be appreciated.

Regards,
Martin

From martin at v.loewis.de  Wed Nov 26 23:38:54 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Wed, 26 Nov 2008 23:38:54 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <492DCA8F.4050105@cheimes.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>	<gg1jne$so8$1@ger.gmane.org>	<49246BB6.7000607@v.loewis.de>	<ggk94d$pr1$1@ger.gmane.org>	<492DABAF.5020808@v.loewis.de>
	<1227736209.6739.9.camel@ozzu> <492DCA8F.4050105@cheimes.de>
Message-ID: <492DCFFE.2060903@v.loewis.de>

> Merge Modules into your installation
> Create self-contained MSI packages, by including and configuring the
> required merge modules.

Right. Still, if people want to go this route (I personally don't),
I think it would be useful to build an installer from the free edition.
You can then run Tools/msi/merge.py, which adds the CRT merge module
into the MSI file (mostly as-is, except for discarding the ALLUSERS
property from that merge module). Alternatively, for testing, you can
just assume that the CRT is already installed.

When we then have a script that generates a mostly-complete installer,
I'm sure Giovanni would be happy to add support for the CRT merge
module to see how the tool fares (my expectation is that it breaks,
as I assume it just doesn't deal with the embedded ALLUSERS property
correctly - merge.py really uses a bad hack here).

Regards,
Martin

From rasky at develer.com  Wed Nov 26 23:46:56 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Wed, 26 Nov 2008 23:46:56 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <492DCFFE.2060903@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>
	<gg1jne$so8$1@ger.gmane.org>	<49246BB6.7000607@v.loewis.de>
	<ggk94d$pr1$1@ger.gmane.org>	<492DABAF.5020808@v.loewis.de>
	<1227736209.6739.9.camel@ozzu> <492DCA8F.4050105@cheimes.de>
	<492DCFFE.2060903@v.loewis.de>
Message-ID: <1227739616.6739.13.camel@ozzu>

On mer, 2008-11-26 at 23:38 +0100, "Martin v. L?wis" wrote:
> > Merge Modules into your installation
> > Create self-contained MSI packages, by including and configuring the
> > required merge modules.
> 
> Right. Still, if people want to go this route (I personally don't),
> I think it would be useful to build an installer from the free edition.
> You can then run Tools/msi/merge.py, which adds the CRT merge module
> into the MSI file (mostly as-is, except for discarding the ALLUSERS
> property from that merge module). Alternatively, for testing, you can
> just assume that the CRT is already installed.

So, deducing from your reply, this "merge module" is a thing that allows
to install the CRT (and other shared components)? I quickly googled but
I'm not really into the msi slang, so I'm not sure I understood.

> When we then have a script that generates a mostly-complete installer,
> I'm sure Giovanni would be happy to add support for the CRT merge
> module to see how the tool fares (my expectation is that it breaks,
> as I assume it just doesn't deal with the embedded ALLUSERS property
> correctly - merge.py really uses a bad hack here).

Another option is to contact the Advanced Installer vendor and ask for a
free license for the Python Software Foundation. This would mean that
everybody in the world would still be able to build an installer without
CRT, and only PSF would build the official one with CRT bundled. I
personally don't see this as a show-stopper (does anyone ever build
the .msi besides Martin?).
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com



From rasky at develer.com  Wed Nov 26 23:49:52 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Wed, 26 Nov 2008 23:49:52 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <492DC57D.1010101@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org> <4923ABEF.50900@v.loewis.de>
	<gg1jne$so8$1@ger.gmane.org> <49246BB6.7000607@v.loewis.de>
	<ggk94d$pr1$1@ger.gmane.org>  <492DABAF.5020808@v.loewis.de>
	<1227736209.6739.9.camel@ozzu>  <492DC57D.1010101@v.loewis.de>
Message-ID: <1227739792.6739.15.camel@ozzu>

On mer, 2008-11-26 at 22:54 +0100, "Martin v. L?wis" wrote:
> > I've had good results with Advanced Installer:
> > http://www.advancedinstaller.com/feats-list.html
> 
> So how much effort would it be to create a Python installer?
> Could you kindly provide one?

In my case, the biggest effort would be finding out what needs to be put
within the installer. If you can give me a pointer to where the current
build process reads the complete file list to put within the .msi (and
their relative destination path), I can try and build a simple test
installer, on which we can start doing some evaluations.
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com



From larry.bugbee at boeing.com  Wed Nov 26 23:37:36 2008
From: larry.bugbee at boeing.com (Bugbee, Larry)
Date: Wed, 26 Nov 2008 14:37:36 -0800
Subject: [Python-Dev] Python for windows.
In-Reply-To: <mailman.25385.1227729845.3486.python-dev@python.org>
References: <mailman.25385.1227729845.3486.python-dev@python.org>
Message-ID: <9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>

Gerald, if there is an expectation some of your HP windows users would
like to use OpenSSL with Python, there is this little matter of
including applink.c in the build.  

All, and not to start flames, but I still do not understand why
applink.c isn't included in python's main (conditionally) instead of
expecting users, many of them novices, to do the build.  ???

Larry 



[snip]

Message: 2
Date: Wed, 26 Nov 2008 19:06:44 +0000
From: "Koenig, Gerald" <gerald.koenig at hp.com>
Subject: [Python-Dev] Python for windows.
To: "python-dev at python.org" <python-dev at python.org>
Message-ID:
	
<238A96A773B3934685A7269CC8A8D0423F7FBF71E9 at GVW0436EXB.americas.hpqcorp.
net>
	
Content-Type: text/plain; charset="us-ascii"

Hi all,

I am working For Hewlett-Packard designing PC Consumer Desktop

We have been including Python since over 10 years now on our systems.

It is a wonderful language and very powerful.

Now I am having a small issue and I was wondering how I can solve it.

So I would like to know who should I contact to be able to work on that
issue together ?

The issue is about some requirement I need to have on our systems and
even tools need to pass those requirement.

Regards

Gerald Koenig
Software Builds Architect
Hewlett-Packard
Work Phone:     +(1) 408 873 6202

------------------------------

[snip]

From cdcasey at gmail.com  Wed Nov 26 23:42:30 2008
From: cdcasey at gmail.com (chris)
Date: Wed, 26 Nov 2008 16:42:30 -0600
Subject: [Python-Dev] building Tcl/Tk to deploy on other platforms?
Message-ID: <ead057790811261442w2dce570cg52733854934ecd93@mail.gmail.com>

I built Python 2.5.2 on RedHat3 and wrapped it up with some other
stuff that was deployed on RedHat4. When trying to fire up Idle on
RedHat4, there is an error that states a usable init.tcl cannot be
found.

Python is built on RedHat3 against Tcl/Tk 8.3, so even installing
Tcl/Tk on RedHat4 would not work, as it is 8.4.

I've noticed when Python is installed on Windows, all the necessary
stuff is also installed in the Python25 directory. Is there a similar
way to do this on linux?

Thanks,
-Chris

From mhammond at skippinet.com.au  Wed Nov 26 23:51:58 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 27 Nov 2008 09:51:58 +1100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492DCE5E.5080602@v.loewis.de>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
	<492DCE5E.5080602@v.loewis.de>
Message-ID: <043e01c95019$9955a0a0$cc00e1e0$@com.au>

> > But these are written with applications in mind - Python isn't an
> > application - its used to *write* applications.  I don't see a good
> reason
> > to support these guidelines.  I do see a reason to help support
> people
> > ensure their Python implemented applications can meet the guidelines,
> but
> > I'd never advise such people to install the python.org installer and
> have
> > their application use Python from that directory.
> 
> I disagree. While it might not be really necessary for HP to have
> Python
> comply to a formal test suite, I'm sure other users do need to pass
> these test suites for whatever reason, and they are unhappy if they
> don't pass out of the box.

IIUC, the test suite is about having the Python installer certified as "OEM
Ready", which means a few special things - including, IIUC, the "right" to
be installed in a new PC.  My broader point is that I would advise against
any application vendor reusing the standard Python installer for their
application, as it exposes the ability for the user to remove it, thereby
breaking the application.  I would recommend that the application vendor
take full control of the Python they rely on and install a "private" copy
(either in their own directory or in a directory named other than "Python
X.X".)

In other words, I'm all for helping application vendors in every way to have
their installers meet such guidelines, but I see having the Python installer
itself meet such guidelines as effectively pointless as no one would
actually need it.

Obviously I'm wrong here though - HP obviously *is* wanting to install it as
is, and doesn't mind that the user could accidently break their scripts.
I'm very surprised by that, but I have to accept it :)  On the other hand,
I'd really like to ensure they understand the risks and for me to understand
why those risks are acceptable...

> > Ditto for python.exe etc - its impossible to add a reasonable
> manifest, as
> > the manifest requirements would depend entirely on what the python
> program
> > itself does.  There is no way python.exe can know what it will be
> asked to
> > do.
> 
> FWIW, python.exe *does* have a manifest. The manifest is primarily
> about static linking. I don't know what other uses a manifest may have,

The issue is regarding the Vista "user access control" flags, but yes, I see
that python.exe does include a UAC entry in its manifest.  As I mentioned,
it will obviously not be approriate for all Python apps, but it is there and
that is what matters for this discussion.

> Can you please elaborate? What specific elements of the manifest are
> you thinking of that are not universally correct for python.exe?

A specific Python script may need a different UAC flag to work correctly in
all cases - specifically one that requires elevation.  If the manifest was
appropriate for that script Windows would automatically elevate, but as
things stand, the user would need to manually elevate before executing that
script.

> Perhaps. I wouldn't go so far, though. It's surely puzzling if a system
> comes with a pre-installed Python, but if that Python actually works,
> I don't think that does much damage.

Agreed.  I'm just surprised app authors are willing to take the risk this
implies.  I still occasionally get email from people (presumably via my
email addy which used to be in Pythonwin) saying "I've got this Python thing
installed on my PC - is it a virus?  Can I remove it?".  I tend to say "it?s
a programming language - if you don't know you need it, you probably don't,
so sure, uninstall it". 

> Of course, anybody embedding
> Python
> *should* integrate it into his own application and installation
> procedure. If we really want this guideline to be followed, we should
> document explicitly how one does that (and no, pointing people to
> py2exe is not sufficient).

Actually, I don't think its that hard.  Just plonk what you need (or
*everything* except what you know you don't need, eg docs) in the same
directory structure, test, rinse and repeat.  Ideally you would also tweak
the resource in pythonxx.dll which tells it the registry key to use too.  If
you had special UAC requirements you might even make a copy of python.exe
which is identical except for the manifest, but now we are starting to get
into app specific requirements...

Cheers,

Mark


From gerald.koenig at hp.com  Wed Nov 26 23:48:57 2008
From: gerald.koenig at hp.com (Koenig, Gerald)
Date: Wed, 26 Nov 2008 22:48:57 +0000
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492DCE5E.5080602@v.loewis.de>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au> <492DCE5E.5080602@v.loewis.de>
Message-ID: <238A96A773B3934685A7269CC8A8D0423F7FBF732B@GVW0436EXB.americas.hpqcorp.net>

Hi all,


Yes we spend the time and resources to do those tests.

You do not have to worry about that :) and We will submit to Microsoft instead of you like that you do not have to worry about it. Microsoft is totally fine with it. I already asked them if it was fine.
Unless you do not want us to do it.

We do it for 2 reasons:

        We are trying to have all our systems OEM ready Compliant.

        Second all our customers will not have to worry about it, as it is already passing.

I can promise you Python on our system Python work perfectly. We use it very often as our main scripting/programming tool like some other would use jscript or C#.
We also follow your license agreement to never change anything from the packages.

That why I went to this development board to see if some change can be done to make it OEM ready :)
I didn't want to change anything by myself.

So out of the three issue:

        - The three executable could be change to .dat ?

        - So if I understand well python_icon.exe is only stocking all the icon for python ?

        - I just checked for the version number it is not required expect for the uninstalled key which is already clean.




GErald

-----Original Message-----
From: "Martin v. L?wis" [mailto:martin at v.loewis.de]
Sent: Wednesday, November 26, 2008 2:32 PM
To: mhammond at skippinet.com.au
Cc: Koenig, Gerald; python-dev at python.org
Subject: Re: [Python-Dev] Python for windows.

> But these are written with applications in mind - Python isn't an
> application - its used to *write* applications.  I don't see a good reason
> to support these guidelines.  I do see a reason to help support people
> ensure their Python implemented applications can meet the guidelines, but
> I'd never advise such people to install the python.org installer and have
> their application use Python from that directory.

I disagree. While it might not be really necessary for HP to have Python
comply to a formal test suite, I'm sure other users do need to pass
these test suites for whatever reason, and they are unhappy if they
don't pass out of the box.

Cleary, "we" (python-dev) would never run any of these test suites; it's
to much effort to buy them, run them, and study the results. So I'm
really glad HP spends the time on that stuff - not because HP needs it,
but because "smaller" users might run into it, and then frustrate
because they can't resolve the issues (rather than contacting us).

> These are stubs for installers.  It is the installers created from these
> stubs that need the manifest, as the manifest requirements will differ for
> each use of the stub.

See my other message. This "obviously" comes from a formal test suite.
So it's not necessary to convince the operator of the test suite that
the test is flawed, but to convince the test suite to look elsewhere.
I think we can do that.

> Ditto for python.exe etc - its impossible to add a reasonable manifest, as
> the manifest requirements would depend entirely on what the python program
> itself does.  There is no way python.exe can know what it will be asked to
> do.

FWIW, python.exe *does* have a manifest. The manifest is primarily about
static linking. I don't know what other uses a manifest may have, but if
it makes the test suite happy that python.exe has a manifest, then I'm
happy myself also.

> See above - we don't know ahead of time what an appropriate manifest is.  I
> agree manifests need thought and work for Python, but I'm fairly sure the
> answer isn't to try and come up with them ahead of time and assume they
> apply universally.

Can you please elaborate? What specific elements of the manifest are you
thinking of that are not universally correct for python.exe?

>>         - This c:\windows\installer\{110eb5c4-e995-4cfb-ab80-
>> a5f315bea9e8}\python_icon.exe do not answers to the Restart Manager
>> Aware
>
> That sounds like something nice to have, but assuming we recommend against
> using the standard Python installer for 3rd party applications, this is a
> problem for the application installer, not ours.  In other words, I'm sure
> we'd welcome a fix to this, but I'd still recommend against using our
> installer anyway!

Perhaps. I wouldn't go so far, though. It's surely puzzling if a system
comes with a pre-installed Python, but if that Python actually works,
I don't think that does much damage. Of course, anybody embedding Python
*should* integrate it into his own application and installation
procedure. If we really want this guideline to be followed, we should
document explicitly how one does that (and no, pointing people to py2exe
is not sufficient).

> Maybe your and others lives would be made easier if the Python MSI installer
> was split up to support "merge modules"?  That way you could roll your own
> installer that met any guidelines or requirements that mattered to you,
> while still ensuring you got all the files that the official installer would
> have installed.  I'm hand-waving a little here, but it sounds like we can do
> more to help *others* meet guidelines for their Python based apps instead of
> trying and meet it for our installer...

If this list of complaints was complete, I don't find it too difficult
to comply. If a merge module would contain the entire library (including
distutils), then the wininst.exe issue would still exist for the merge
module (for example).

OTOH, if somebody wanted to contribute a procedure that builds merge
modules (in whatever granularity) along with the msi file - that would
also be appreciated.

Regards,
Martin

From doko at ubuntu.com  Wed Nov 26 23:55:07 2008
From: doko at ubuntu.com (Matthias Klose)
Date: Wed, 26 Nov 2008 23:55:07 +0100
Subject: [Python-Dev] Module/Setup.dist maintainance
Message-ID: <492DD3CB.8050702@ubuntu.com>

I was experimenting with profiled builds and building more extensions as
builtins instead of loadable modules, noticing that Module/Setup.dist is not
kept updated. Currently only the section above the *shared* line is really used
for the build, but everything else doesn't seem to be updated (extensions are
missing or out of date). Is there another way to build extensions as builtins,
or should this file kept be up to date?

  Matthias

From ggpolo at gmail.com  Thu Nov 27 00:09:18 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Wed, 26 Nov 2008 21:09:18 -0200
Subject: [Python-Dev] building Tcl/Tk to deploy on other platforms?
In-Reply-To: <ead057790811261442w2dce570cg52733854934ecd93@mail.gmail.com>
References: <ead057790811261442w2dce570cg52733854934ecd93@mail.gmail.com>
Message-ID: <ac2200130811261509k71515260x82125638c3b22e71@mail.gmail.com>

On Wed, Nov 26, 2008 at 8:42 PM, chris <cdcasey at gmail.com> wrote:
> I built Python 2.5.2 on RedHat3 and wrapped it up with some other
> stuff that was deployed on RedHat4. When trying to fire up Idle on
> RedHat4, there is an error that states a usable init.tcl cannot be
> found.
>

Then you have to set the TCL_LIBRARY environment variable, it should
point to the directory that contains the correct init.tcl file.
If it then complains about not finding tk.tcl, then you have to set
the TK_LIBRARY env var too.

> Python is built on RedHat3 against Tcl/Tk 8.3, so even installing
> Tcl/Tk on RedHat4 would not work, as it is 8.4.
>
> I've noticed when Python is installed on Windows, all the necessary
> stuff is also installed in the Python25 directory. Is there a similar
> way to do this on linux?
>

Adjust the TCL_LIBRARY and TK_LIBRARY as needed.

> Thanks,
> -Chris
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves

From mhammond at skippinet.com.au  Thu Nov 27 00:11:12 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 27 Nov 2008 10:11:12 +1100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <238A96A773B3934685A7269CC8A8D0423F7FBF732B@GVW0436EXB.americas.hpqcorp.net>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
	<492DCE5E.5080602@v.loewis.de>
	<238A96A773B3934685A7269CC8A8D0423F7FBF732B@GVW0436EXB.americas.hpqcorp.net>
Message-ID: <043f01c9501c$4b1fa040$e15ee0c0$@com.au>

> I can promise you Python on our system Python work perfectly.

I'm sure it does :)  I'm more concerned about *your* apps not working when
the user, or a "helpful" friend, uninstalls this Python thing that they
don't use.  I'm very interested to know why you don't see this as a
significant problem with a relatively easy solution?

> very often as our main scripting/programming tool like some other would
> use jscript or C#.

Yes, but apps written in jscript and C# tend not to rely on something
external, out of their control, that the user may remove.  Even for apps
written in C which rely on the C runtime library, the advice is still to
ship a private copy of that CRT.  OTOH, I guess some .NET apps *do* fall
into this category...

> So out of the three issue:
> 
>         - The three executable could be change to .dat ?

I think that should be quite easy to do - I'll volunteer for this one (but
not for a few days)

> 
>         - So if I understand well python_icon.exe is only stocking all
> the icon for python ?

I'm not familiar with that.


>         - I just checked for the version number it is not required
> expect for the uninstalled key which is already clean.

So to be clear, you don't desire any changes here?

BTW - isn't there also a "\Program Files" requirement...?

Cheers,

Mark


From martin at v.loewis.de  Thu Nov 27 00:29:42 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 27 Nov 2008 00:29:42 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <1227739616.6739.13.camel@ozzu>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	
	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	
	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>	
	<gg1jne$so8$1@ger.gmane.org>	<49246BB6.7000607@v.loewis.de>	
	<ggk94d$pr1$1@ger.gmane.org>	<492DABAF.5020808@v.loewis.de>	
	<1227736209.6739.9.camel@ozzu> <492DCA8F.4050105@cheimes.de>	
	<492DCFFE.2060903@v.loewis.de> <1227739616.6739.13.camel@ozzu>
Message-ID: <492DDBE6.60205@v.loewis.de>

> So, deducing from your reply, this "merge module" is a thing that allows
> to install the CRT (and other shared components)? 

Correct. More generally, a merge module is a something like an MSI
library (.a). It includes a set of files and snippets of an installation
procedure for them.

> Another option is to contact the Advanced Installer vendor and ask for a
> free license for the Python Software Foundation. This would mean that
> everybody in the world would still be able to build an installer without
> CRT, and only PSF would build the official one with CRT bundled. I
> personally don't see this as a show-stopper (does anyone ever build
> the .msi besides Martin?).

I personally don't have any interest to spend any time on an alternative
technology. The current technology works fine for me, and I understand
it fully. Everybody in the world is able to build an installer today,
also. However, I won't stop anybody else from working a switch to a
different technology, either.

Regards,
Martin



From akitada at gmail.com  Thu Nov 27 00:39:07 2008
From: akitada at gmail.com (Akira Kitada)
Date: Thu, 27 Nov 2008 08:39:07 +0900
Subject: [Python-Dev] Fwd: distutils doesn't use some compiler options when
	building
In-Reply-To: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
Message-ID: <90bb445a0811261539m61665df0n14eb60da2d44303d@mail.gmail.com>

I didn't know distutils has its own list.
Forwarding.


---------- Forwarded message ----------
From: Akira Kitada <akitada at gmail.com>
Date: Thu, Nov 27, 2008 at 2:28 AM
Subject: distutils doesn't use some compiler options when building
To: python-dev at python.org


Hi,

I encountered a weird problem using distutils.
Generally, distutils try to use the same compiler options used for
building Python interpreter,
but it looks like some of them are omitted sometimes.

- CPPFLAGS are not retrieved from the config and only ones in env are used.
- OPT is retrieved from the config, but it's only used when env has CFLAGS.

See: http://svn.python.org/view/python/trunk/Lib/distutils/sysconfig.py

"""
def customize_compiler(compiler):
...
   if compiler.compiler_type == "unix":
       (cc, cxx, opt, cflags, ccshared, ldshared, so_ext) = \
           get_config_vars('CC', 'CXX', 'OPT', 'CFLAGS',
                           'CCSHARED', 'LDSHARED', 'SO')

       if 'CC' in os.environ:
           cc = os.environ['CC']
       if 'CXX' in os.environ:
           cxx = os.environ['CXX']
       if 'LDSHARED' in os.environ:
           ldshared = os.environ['LDSHARED']
       if 'CPP' in os.environ:
           cpp = os.environ['CPP']
       else:
           cpp = cc + " -E"           # not always
       if 'LDFLAGS' in os.environ:
           ldshared = ldshared + ' ' + os.environ['LDFLAGS']
       if 'CFLAGS' in os.environ:
           cflags = opt + ' ' + os.environ['CFLAGS']
           ldshared = ldshared + ' ' + os.environ['CFLAGS']
       if 'CPPFLAGS' in os.environ:
           cpp = cpp + ' ' + os.environ['CPPFLAGS']
           cflags = cflags + ' ' + os.environ['CPPFLAGS']
           ldshared = ldshared + ' ' + os.environ['CPPFLAGS']

       cc_cmd = cc + ' ' + cflags
       compiler.set_executables(
           preprocessor=cpp,
           compiler=cc_cmd,
           compiler_so=cc_cmd + ' ' + ccshared,
           compiler_cxx=cxx,
           linker_so=ldshared,
           linker_exe=cc)

       compiler.shared_lib_extension = so_ext
"""

Are these logics are intentional or just a bug?
If this is intentional behavior, why is that being this way?

Thanks,

From martin at v.loewis.de  Thu Nov 27 00:39:44 2008
From: martin at v.loewis.de (=?windows-1252?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 27 Nov 2008 00:39:44 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <043e01c95019$9955a0a0$cc00e1e0$@com.au>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
	<492DCE5E.5080602@v.loewis.de>
	<043e01c95019$9955a0a0$cc00e1e0$@com.au>
Message-ID: <492DDE40.2040206@v.loewis.de>

> IIUC, the test suite is about having the Python installer certified as "OEM
> Ready", which means a few special things - including, IIUC, the "right" to
> be installed in a new PC.  My broader point is that I would advise against
> any application vendor reusing the standard Python installer for their
> application, as it exposes the ability for the user to remove it, thereby
> breaking the application.

Right, and I agree with it. However, that is HP's choice, and while
there is a theoretical possibility that users break their systems, in
practice, most users are too scared to actually attempt such breakage.

However, "OEM ready" sounds like a good goal to achieve. Python has been
shipping as part of the operating system on Linux for many years now,
and ships with Solaris and OSX for several years now. I see nothing
wrong with Python being preinstalled on PCs. If this approach would
become standard practice, py2exe and friends would become much simpler
(and I do know that I'm dreaming now).

> Obviously I'm wrong here though - HP obviously *is* wanting to install it as
> is, and doesn't mind that the user could accidently break their scripts.
> I'm very surprised by that, but I have to accept it :)  On the other hand,
> I'd really like to ensure they understand the risks and for me to understand
> why those risks are acceptable...

See above. I think the risk is theoretical. In any case, applications
*can* make sure pythonxy.dll survives uninstallation (by increasing its
refcount), although this probably doesn't help much if the standard
library is gone.

> Agreed.  I'm just surprised app authors are willing to take the risk this
> implies.  I still occasionally get email from people (presumably via my
> email addy which used to be in Pythonwin) saying "I've got this Python thing
> installed on my PC - is it a virus?  Can I remove it?".  I tend to say "it?s
> a programming language - if you don't know you need it, you probably don't,
> so sure, uninstall it". 

We get these messages on python-help also. We even have

http://www.python.org/doc/faq/installed/

so it *is* a frequently-asked question. I think that is a good thing.

Regards,
Martin

From martin at v.loewis.de  Thu Nov 27 00:44:31 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 27 Nov 2008 00:44:31 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <1227739792.6739.15.camel@ozzu>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	
	<4922D6AF.6080400@cheimes.de> <49233EF3.9040303@v.loewis.de>	
	<87wsf0mqcr.fsf@xemacs.org> <4923ABEF.50900@v.loewis.de>	
	<gg1jne$so8$1@ger.gmane.org> <49246BB6.7000607@v.loewis.de>	
	<ggk94d$pr1$1@ger.gmane.org> <492DABAF.5020808@v.loewis.de>	
	<1227736209.6739.9.camel@ozzu> <492DC57D.1010101@v.loewis.de>
	<1227739792.6739.15.camel@ozzu>
Message-ID: <492DDF5F.7090603@v.loewis.de>

> In my case, the biggest effort would be finding out what needs to be put
> within the installer. If you can give me a pointer to where the current
> build process reads the complete file list to put within the .msi (and
> their relative destination path), I can try and build a simple test
> installer, on which we can start doing some evaluations.

The simplest approach might be to look at what it actually installs.
If you want to read the specifcation: it's in
Tools/msi/msi.py:add_files. directory.add_file takes a file, and
optionally a source file (which defaults to the respective source
directory). You also need to consider the features structure; there
is a "current" feature at any point in time, and all components being
added get added to the current feature.

HTH,
Martin

From martin at v.loewis.de  Thu Nov 27 00:48:45 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 27 Nov 2008 00:48:45 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>
References: <mailman.25385.1227729845.3486.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>
Message-ID: <492DE05D.4020805@v.loewis.de>

> All, and not to start flames, but I still do not understand why
> applink.c isn't included in python's main (conditionally) instead of
> expecting users, many of them novices, to do the build.  ???

One reason is that I don't know what applink is, and why I should
care about it. (I may have known in the past, but then I have forgotten
since).

Regards,
Martin

From martin at v.loewis.de  Thu Nov 27 00:51:08 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 27 Nov 2008 00:51:08 +0100
Subject: [Python-Dev] building Tcl/Tk to deploy on other platforms?
In-Reply-To: <ac2200130811261509k71515260x82125638c3b22e71@mail.gmail.com>
References: <ead057790811261442w2dce570cg52733854934ecd93@mail.gmail.com>
	<ac2200130811261509k71515260x82125638c3b22e71@mail.gmail.com>
Message-ID: <492DE0EC.7020601@v.loewis.de>

> Adjust the TCL_LIBRARY and TK_LIBRARY as needed.

We should also point out that the FixTk module already does
that, but is imported only on Windows. So if you arrange to
always import FixTk, then it should Just Work.

Regards,
Martin

From martin at v.loewis.de  Thu Nov 27 00:52:03 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 27 Nov 2008 00:52:03 +0100
Subject: [Python-Dev] Module/Setup.dist maintainance
In-Reply-To: <492DD3CB.8050702@ubuntu.com>
References: <492DD3CB.8050702@ubuntu.com>
Message-ID: <492DE123.3090907@v.loewis.de>

Matthias Klose wrote:
> Is there another way to build extensions as builtins,
> or should this file kept be up to date?

It should be kept up-to-date, IMO. It's just that people forget to do
that (or don't even know what it is).

Regards,
Martin

From cdcasey at gmail.com  Thu Nov 27 01:07:15 2008
From: cdcasey at gmail.com (chris)
Date: Wed, 26 Nov 2008 18:07:15 -0600
Subject: [Python-Dev] building Tcl/Tk to deploy on other platforms?
In-Reply-To: <492DE0EC.7020601@v.loewis.de>
References: <ead057790811261442w2dce570cg52733854934ecd93@mail.gmail.com>
	<ac2200130811261509k71515260x82125638c3b22e71@mail.gmail.com>
	<492DE0EC.7020601@v.loewis.de>
Message-ID: <ead057790811261607v3fe321e1nef2568ec50cb7ab0@mail.gmail.com>

This may work on the deployed side, although I'm skeptical if I've
built against a different version of Tcl/Tk than exists. I don't see
how this helps on the build side, althought I'll look into it.

On Wed, Nov 26, 2008 at 5:51 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Adjust the TCL_LIBRARY and TK_LIBRARY as needed.
>
> We should also point out that the FixTk module already does
> that, but is imported only on Windows. So if you arrange to
> always import FixTk, then it should Just Work.
>
> Regards,
> Martin
>

From ggpolo at gmail.com  Thu Nov 27 02:02:23 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Wed, 26 Nov 2008 23:02:23 -0200
Subject: [Python-Dev] building Tcl/Tk to deploy on other platforms?
In-Reply-To: <492DE0EC.7020601@v.loewis.de>
References: <ead057790811261442w2dce570cg52733854934ecd93@mail.gmail.com>
	<ac2200130811261509k71515260x82125638c3b22e71@mail.gmail.com>
	<492DE0EC.7020601@v.loewis.de>
Message-ID: <ac2200130811261702u460089efj544a4e62b800eb36@mail.gmail.com>

On Wed, Nov 26, 2008 at 9:51 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Adjust the TCL_LIBRARY and TK_LIBRARY as needed.
>
> We should also point out that the FixTk module already does
> that, but is imported only on Windows. So if you arrange to
> always import FixTk, then it should Just Work.
>

It would Just Work if you had python and tcl/tk installed with the
same paths as the ones used by the windows installer.

> Regards,
> Martin
>


-- 
-- Guilherme H. Polo Goncalves

From larry.bugbee at boeing.com  Thu Nov 27 02:09:56 2008
From: larry.bugbee at boeing.com (Bugbee, Larry)
Date: Wed, 26 Nov 2008 17:09:56 -0800
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492DE05D.4020805@v.loewis.de>
References: <mailman.25385.1227729845.3486.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>
	<492DE05D.4020805@v.loewis.de>
Message-ID: <9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>

> > All, and not to start flames, but I still do not understand why 
> > applink.c isn't included in python's main (conditionally) instead 
> > of expecting users, many of them novices, to do the build.  ???
> 
> One reason is that I don't know what applink is, and why I should 
> care about it. (I may have known in the past, but then I have 
> forgotten since).

Yeah, it's been a while for me too.

As I recall, OpenSSL, a long while ago stopped, supporting some idiosyncrasies associated with Windows I/O and opted for a "cleaner" approach, that of requiring developers to link a small file, applink.c, into the app's main.  applink.c is provided by the OpenSSL folks.

>From http://www.openssl.org/docs/crypto/OPENSSL_Applink.html:
    OPENSSL_Applink is application-side interface which 
    provides a glue between OpenSSL BIO layer and Win32 
    compiler run-time environment. Even though it appears 
    at application side, it's essentially OpenSSL private 
    interface. For this reason application developers are 
    not expected to implement it, but to compile provided 
    module with compiler of their choice and link it into 
    the target application. The referred module is available 
    as <openssl>/ms/applink.c.

For most custom apps this is a simple process of adding "#include applink.c" to the app's main().  The problem for Python developers is that their Python program is not main(), and if python.exe does not have the OPENSSL_Applink interface, they cannot import M2Crypto, pyOpenSSL, or use ctypes to wrap OpenSSL, and write a PEM file without throwing an error.  (That said, Daniel Clark says he is not experiencing problems with NCrypt.  I have not verified.)

One solution is for those users to build their own Python from scratch.  Many/most cannot do it, especially if they don't have the right version of Visual Studio.  I know I don't.  I simply stopped developing on Windows.

Arguably better is to have the include statement in the python source file, python.c, appropriately conditioned.  ...and to provide a copy of applink.c.   That way, there would be no more need to try to find or do a custom build.  Everybody building for a Windows platform will have applink included and access to OpenSSL becomes a non-issue.  

Larry

From janssen at parc.com  Thu Nov 27 02:49:59 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 26 Nov 2008 17:49:59 PST
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492DCE5E.5080602@v.loewis.de>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
	<492DCE5E.5080602@v.loewis.de>
Message-ID: <43751.1227750599@parc.com>

Martin v. L?wis <martin at v.loewis.de> wrote:

> Perhaps. I wouldn't go so far, though. It's surely puzzling if a system
> comes with a pre-installed Python, but if that Python actually works,
> I don't think that does much damage.

OS X does come with pre-installed Python, so this is a debate we have
from time to time on the mac-python mailing list.  Personally, I always
come down on the side of using the "system python", instead of
installing a newer or different version.  If, of course, it's sufficient
for your application, and my experience is that it usually is.

Bill

From janssen at parc.com  Thu Nov 27 02:54:14 2008
From: janssen at parc.com (Bill Janssen)
Date: Wed, 26 Nov 2008 17:54:14 PST
Subject: [Python-Dev] Python for windows.
In-Reply-To: <9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>
References: <mailman.25385.1227729845.3486.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>
	<492DE05D.4020805@v.loewis.de>
	<9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>
Message-ID: <43855.1227750854@parc.com>

Bugbee, Larry <larry.bugbee at boeing.com> wrote:

> For most custom apps this is a simple process of adding "#include applink.c" to the app's main().  The problem for Python developers is that their Python program is not main(), and if python.exe does not have the OPENSSL_Applink interface, they cannot import M2Crypto, pyOpenSSL, or use ctypes to wrap OpenSSL, and write a PEM file without throwing an error.  (That said, Daniel Clark says he is not experiencing problems with NCrypt.  I have not verified.)

I'm probably missing something here.  Python 2.6/3.x uses OpenSSL for
the hashlib and ssl modules.  Doesn't this mean that this applink.c file
is part of the standard build now?

Bill

From mhammond at skippinet.com.au  Thu Nov 27 03:47:54 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 27 Nov 2008 13:47:54 +1100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492DDE40.2040206@v.loewis.de>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
	<492DCE5E.5080602@v.loewis.de>
	<043e01c95019$9955a0a0$cc00e1e0$@com.au>
	<492DDE40.2040206@v.loewis.de>
Message-ID: <045801c9503a$8e85d2f0$ab9178d0$@com.au>

> Right, and I agree with it. However, that is HP's choice, and while
> there is a theoretical possibility that users break their systems, in
> practice, most users are too scared to actually attempt such breakage.
> 
> However, "OEM ready" sounds like a good goal to achieve.

Agreed too - I don't think we have ever agreed so much Martin :)  Most of
the issues discussed so far don't bother us at all, but in the back of my
mind has always been what would happen if an "OEM Ready" guideline conflicts
with what we would otherwise choose to do.

FYI, I'm looking at google's HTML version of the MS doc via googling '"oem
ready program" filetype:docx' and hitting the "view as HTML" link.
 
The only conflict I see here is the requirement to install into "\Program
Files" and I'm surprised that hasn't been raised in this thread.  An
interesting question we might need to face is exactly how much being "OEM
Ready" is worth to Python itself if it requires us to make compromises we
wouldn't otherwise make.  But given Gerald hasn't mentioned this
requirement, I must acknowledge it is still only a theoretical concern.
However, should such a situation arise, my position would probably be that
unless it was MS suggesting it be preloaded on *all* PCs, we should
sacrifice that part of being "OEM Ready" to best look after the interests of
people who seek it out for download.

Cheers,

Mark


From larry.bugbee at boeing.com  Thu Nov 27 04:12:08 2008
From: larry.bugbee at boeing.com (Bugbee, Larry)
Date: Wed, 26 Nov 2008 19:12:08 -0800
Subject: [Python-Dev] Python for windows.
In-Reply-To: <43855.1227750854@parc.com>
References: <mailman.25385.1227729845.3486.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>
	<492DE05D.4020805@v.loewis.de>
	<9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>
	<43855.1227750854@parc.com>
Message-ID: <9418DB6C0B9D434190E54A78E931C3D1086DE353@XCH-NW-7V1.nw.nos.boeing.com>

Not necessarily.  I have no problems with hashlib or Python/M2Crypto/OpenSSL as long as I'm doing memory-oriented operations.  It is only when the function does I/O that errors occur.  ...like reading/writing a PEM file, randpool.dat, etc.  



-----Original Message-----
From: Bill Janssen [mailto:janssen at parc.com] 
Sent: Wednesday, November 26, 2008 5:54 PM
To: Bugbee, Larry
Cc: "Martin v. L?wis"; Koenig, Gerald; python-dev at python.org
Subject: Re: [Python-Dev] Python for windows.

Bugbee, Larry <larry.bugbee at boeing.com> wrote:

> For most custom apps this is a simple process of adding "#include 
> applink.c" to the app's main().  The problem for Python developers is 
> that their Python program is not main(), and if python.exe does not 
> have the OPENSSL_Applink interface, they cannot import M2Crypto, 
> pyOpenSSL, or use ctypes to wrap OpenSSL, and write a PEM file without 
> throwing an error.  (That said, Daniel Clark says he is not 
> experiencing problems with NCrypt.  I have not verified.)

I'm probably missing something here.  Python 2.6/3.x uses OpenSSL for the hashlib and ssl modules.  Doesn't this mean that this applink.c file is part of the standard build now?

Bill

From tjreedy at udel.edu  Thu Nov 27 05:29:27 2008
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 26 Nov 2008 23:29:27 -0500
Subject: [Python-Dev] Python for windows.
In-Reply-To: <043f01c9501c$4b1fa040$e15ee0c0$@com.au>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>	<042401c95012$3bff99d0$b3fecd70$@com.au>	<492DCE5E.5080602@v.loewis.de>	<238A96A773B3934685A7269CC8A8D0423F7FBF732B@GVW0436EXB.americas.hpqcorp.net>
	<043f01c9501c$4b1fa040$e15ee0c0$@com.au>
Message-ID: <ggl7n6$dq4$1@ger.gmane.org>

Mark Hammond wrote:
>
> BTW - isn't there also a "\Program Files" requirement...?

The requirement as I read it is that /Program Files be the over-rideable 
*default*, as is normal for Windows programs.  HP installed 2.2 on my 
machine in /Python2.2.  Since HP does the installation, I presume they 
can override the default if they wish.

The current Python installers are unusual for Windows in having the 
default be anything else.  I personally am now putting Python in 
/Programs along with other development tools whose directory tree I need 
to access, so it matters not to me.

If HP is happy with stock installs after 10 years, they must not have a 
severe problem with uninstalls.  The one suggestion I have for HP, now 
that we have a reps attention, is that they add a README.txt file to the 
Python directory that explains what is it, why it should not be 
uninstalled, and where to get more info about how the user can use this 
bonus feature of having bought an HP machine ;-).

Terry


From martin at v.loewis.de  Thu Nov 27 07:40:20 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 27 Nov 2008 07:40:20 +0100
Subject: [Python-Dev] building Tcl/Tk to deploy on other platforms?
In-Reply-To: <ac2200130811261702u460089efj544a4e62b800eb36@mail.gmail.com>
References: <ead057790811261442w2dce570cg52733854934ecd93@mail.gmail.com>	<ac2200130811261509k71515260x82125638c3b22e71@mail.gmail.com>	<492DE0EC.7020601@v.loewis.de>
	<ac2200130811261702u460089efj544a4e62b800eb36@mail.gmail.com>
Message-ID: <492E40D4.7050605@v.loewis.de>

> It would Just Work if you had python and tcl/tk installed with the
> same paths as the ones used by the windows installer.

Right - the OP will probably need to supply a different version of
FixTk also.

Regards,
Martin

From martin at v.loewis.de  Thu Nov 27 07:44:39 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 27 Nov 2008 07:44:39 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>
References: <mailman.25385.1227729845.3486.python-dev@python.org>	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>	<492DE05D.4020805@v.loewis.de>
	<9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>
Message-ID: <492E41D7.5050603@v.loewis.de>

> As I recall, OpenSSL, a long while ago stopped, supporting some
> idiosyncrasies associated with Windows I/O and opted for a "cleaner"
> approach, that of requiring developers to link a small file,
> applink.c, into the app's main.  applink.c is provided by the OpenSSL
> folks.

[...]

Ok, this explains what it is.

> Arguably better is to have the include statement in the python source
> file, python.c, appropriately conditioned.  ...and to provide a copy
> of applink.c.   That way, there would be no more need to try to find
> or do a custom build.  Everybody building for a Windows platform will
> have applink included and access to OpenSSL becomes a non-issue.

However, this doesn't explain why I should care about it. What is the
problem that is being solved for Python? Why is access to OpenSSL an
issue currently?

Regards,
Martin

From martin at v.loewis.de  Thu Nov 27 07:48:40 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 27 Nov 2008 07:48:40 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <045801c9503a$8e85d2f0$ab9178d0$@com.au>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>	<042401c95012$3bff99d0$b3fecd70$@com.au>	<492DCE5E.5080602@v.loewis.de>	<043e01c95019$9955a0a0$cc00e1e0$@com.au>	<492DDE40.2040206@v.loewis.de>
	<045801c9503a$8e85d2f0$ab9178d0$@com.au>
Message-ID: <492E42C8.7060305@v.loewis.de>

> The only conflict I see here is the requirement to install into "\Program
> Files" and I'm surprised that hasn't been raised in this thread. 

The question is whether the "OEM ready" is a property of the installer,
or a property of the installed. The OEM can chose to install Python into
program files, and while there is occasional breakage, we do want to
support this case - just not make it the default.

> However, should such a situation arise, my position would probably be that
> unless it was MS suggesting it be preloaded on *all* PCs, we should
> sacrifice that part of being "OEM Ready" to best look after the interests of
> people who seek it out for download.

So much agreement :-)

Regards,
Martin

From mal at egenix.com  Thu Nov 27 10:06:15 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 27 Nov 2008 10:06:15 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <9418DB6C0B9D434190E54A78E931C3D1086DE353@XCH-NW-7V1.nw.nos.boeing.com>
References: <mailman.25385.1227729845.3486.python-dev@python.org>	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>	<492DE05D.4020805@v.loewis.de>	<9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>	<43855.1227750854@parc.com>
	<9418DB6C0B9D434190E54A78E931C3D1086DE353@XCH-NW-7V1.nw.nos.boeing.com>
Message-ID: <492E6307.5080904@egenix.com>

On 2008-11-27 04:12, Bugbee, Larry wrote:
> Not necessarily.  I have no problems with hashlib or Python/M2Crypto/OpenSSL as long as I'm doing memory-oriented operations.  It is only when the function does I/O that errors occur.  ...like reading/writing a PEM file, randpool.dat, etc.  

FWIW: pyOpenSSL works just fine with Python 2.3 - 2.6... including
the APIs that read or write certificates.

> -----Original Message-----
> From: Bill Janssen [mailto:janssen at parc.com] 
> Sent: Wednesday, November 26, 2008 5:54 PM
> To: Bugbee, Larry
> Cc: "Martin v. L?wis"; Koenig, Gerald; python-dev at python.org
> Subject: Re: [Python-Dev] Python for windows.
> 
> Bugbee, Larry <larry.bugbee at boeing.com> wrote:
> 
>> For most custom apps this is a simple process of adding "#include 
>> applink.c" to the app's main().  The problem for Python developers is 
>> that their Python program is not main(), and if python.exe does not 
>> have the OPENSSL_Applink interface, they cannot import M2Crypto, 
>> pyOpenSSL, or use ctypes to wrap OpenSSL, and write a PEM file without 
>> throwing an error.  (That said, Daniel Clark says he is not 
>> experiencing problems with NCrypt.  I have not verified.)
> 
> I'm probably missing something here.  Python 2.6/3.x uses OpenSSL for the hashlib and ssl modules.  Doesn't this mean that this applink.c file is part of the standard build now?
> 
> Bill
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/mal%40egenix.com

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 27 2008)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-11-12: Released mxODBC.Connect 0.9.3      http://python.egenix.com/

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611

From rasky at develer.com  Thu Nov 27 10:29:32 2008
From: rasky at develer.com (Giovanni Bajo)
Date: Thu, 27 Nov 2008 10:29:32 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <492DDBE6.60205@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>
	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>
	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>
	<gg1jne$so8$1@ger.gmane.org>	<49246BB6.7000607@v.loewis.de>
	<ggk94d$pr1$1@ger.gmane.org>	<492DABAF.5020808@v.loewis.de>
	<1227736209.6739.9.camel@ozzu> <492DCA8F.4050105@cheimes.de>
	<492DCFFE.2060903@v.loewis.de> <1227739616.6739.13.camel@ozzu>
	<492DDBE6.60205@v.loewis.de>
Message-ID: <1227778172.6944.8.camel@ozzu>

On gio, 2008-11-27 at 00:29 +0100, "Martin v. L?wis" wrote:
> > So, deducing from your reply, this "merge module" is a thing that allows
> > to install the CRT (and other shared components)? 
> 
> Correct. More generally, a merge module is a something like an MSI
> library (.a). It includes a set of files and snippets of an installation
> procedure for them.

OK. One question: why CRT doesn't get installed as regular files near to
the python executable? That's how I usually ship it, but maybe Python
has some special need.

> > Another option is to contact the Advanced Installer vendor and ask for a
> > free license for the Python Software Foundation. This would mean that
> > everybody in the world would still be able to build an installer without
> > CRT, and only PSF would build the official one with CRT bundled. I
> > personally don't see this as a show-stopper (does anyone ever build
> > the .msi besides Martin?).
> 
> I personally don't have any interest to spend any time on an alternative
> technology. The current technology works fine for me, and I understand
> it fully. Everybody in the world is able to build an installer today,
> also. However, I won't stop anybody else from working a switch to a
> different technology, either.

I proposed an alternatives because I read you saying: "The tricky part
really is when it breaks (which it does more often than
not), in which case you need to understand msi.py, for which you need to
understand MSI". Which means that maybe everybody *has tools* to build
an installer today, but only a few people have the required knowledge to
really do releases on Windows.

So I believe that switching to an alternative that doesn't require full
understanding of MSI and msi.py would probably low the barrier and allow
more people to help you out.
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com



From kristjan at ccpgames.com  Thu Nov 27 15:50:07 2008
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Thu, 27 Nov 2008 14:50:07 +0000
Subject: [Python-Dev] socket.c, _rbufsize
Message-ID: <4E9372E6B2234D4F859320D896059A9510C633B8B7@exchis.ccp.ad.local>

I came across this in socket.c:
        # _rbufsize is the suggested recv buffer size.  It is *strictly*
        # obeyed within readline() for recv calls.  If it is larger than
        # default_bufsize it will be used for recv calls within read().

What I worry about is the readline() case.  Is there a reason why we want to strictly obey it for that function?  Note that in the documentation for _fileobject.read() it says:
        # Use max, disallow tiny reads in a loop as they are very inefficient.

The same argument surely applies for readline().

The reason I am fretting about this is that httplib.py (and therefore xmlrpclib.py) specify bufsize=0 when createing their socket fileobjects, presumably to make sure that write() operations are not buffered but flushed immediately.  But this has the side effect of setting the _rbufsize to 1, and so readline() calls become very slow.

I suggest that readline() be made to use at least defaultbufsize, like read().  Any thoughts?

Cheers,

Kristj?n



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081127/98ba228d/attachment.htm>

From christian at cheimes.de  Thu Nov 27 15:58:43 2008
From: christian at cheimes.de (Christian Heimes)
Date: Thu, 27 Nov 2008 15:58:43 +0100
Subject: [Python-Dev] Python 2.6/3.0,
	IEEE 754 floating point semantics and S60
Message-ID: <492EB5A3.1000608@cheimes.de>

A while ago I contacted Jukka Laurila from the Nokia developer board 
about IEEE 754 support on S60 phones. The information from Jukka may be 
useful for future reference.

Christian

-------- Original Message --------
Subject: Re: Python 2.6/3.0, IEEE 754 floating point semantics and S60

We've been battling with some float problems recently as well in the 
course of our 2.5 core porting. The principal problem has been imprecise 
floating point formatting/parsing routines which lead to the unfortunate 
case of not being able to roundtrip a Python float to string and back 
bit-exactly. I've been trying to get to the bottom of that problem and 
so far it seems like the root cause is a poorly written C library.

Though on the other hand, no one from Symbian is yet to confirm to me 
that the software-emulated floating point routines correspond to 
anything like IEEE 754. Most S60 devices lack floating point hardware.

The best guess is that the floating point support is IEEE 754'ish, with 
imprecise formatting and parsing routines in the standard library.

-Jukka

On 2.2.2008 3.54, "ext Crys @ Developer Discussion Boards"
> This is a message from Crys at Developer Discussion Boards (
> http://discussion.forum.nokia.com/forum/index.php ).  The Developer Discussion
> Boards owners cannot accept any responsibility for the contents of the email.
> 
> To email Crys, you can use this online form:
> http://discussion.forum.nokia.com/forum/sendmessage.php?do=mailmember&u=187359
> 
> This is the message:
> Hallo jplauril!
> 
> I'm a Python core developer. Mikko Ohtamaa said that I should contact you
> about this matter.
> 
> Mark Dickinson and I are working on several math and floating point related
> enhancements for Python 2.6 and 3.0. The enhancements include better support
> for IEEE754 style NaN and INF, C99 math functions and consistent errors and
> return values across platforms.
> 
> Can you give use some detailed information about how Nokia's S60 series
> processes and stores double precision floats? Information about Qtopia would
> be useful as well.
> 
> See http://permalink.gmane.org/gmane.comp.python.devel/91591
> 
> Your's truly
> Christian Heimes
> 



From martin at v.loewis.de  Thu Nov 27 16:24:38 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Thu, 27 Nov 2008 16:24:38 +0100
Subject: [Python-Dev] [Python-3000]   2.6.1 and 3.0
In-Reply-To: <1227778172.6944.8.camel@ozzu>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>	
	<e04bdf310811180203h47330c43w4e4b2090065516c7@mail.gmail.com>	
	<4F0B9197-E929-41B4-8441-D6E9E7762955@python.org>	
	<4922D6AF.6080400@cheimes.de>	<49233EF3.9040303@v.loewis.de>	
	<87wsf0mqcr.fsf@xemacs.org>	<4923ABEF.50900@v.loewis.de>	
	<gg1jne$so8$1@ger.gmane.org>	<49246BB6.7000607@v.loewis.de>	
	<ggk94d$pr1$1@ger.gmane.org>	<492DABAF.5020808@v.loewis.de>	
	<1227736209.6739.9.camel@ozzu> <492DCA8F.4050105@cheimes.de>	
	<492DCFFE.2060903@v.loewis.de> <1227739616.6739.13.camel@ozzu>	
	<492DDBE6.60205@v.loewis.de> <1227778172.6944.8.camel@ozzu>
Message-ID: <492EBBB6.1060203@v.loewis.de>

Giovanni Bajo wrote:
> On gio, 2008-11-27 at 00:29 +0100, "Martin v. L?wis" wrote:
>>> So, deducing from your reply, this "merge module" is a thing that allows
>>> to install the CRT (and other shared components)? 
>> Correct. More generally, a merge module is a something like an MSI
>> library (.a). It includes a set of files and snippets of an installation
>> procedure for them.
> 
> OK. One question: why CRT doesn't get installed as regular files near to
> the python executable? That's how I usually ship it, but maybe Python
> has some special need.

When installing "for all users", pythonxy.dll goes into system32. This,
in turn, requires the CRT to be installed globally (which meant into
system32 for VS6 and VS7.1, but means using SxS for VS 2008). It's
necessary to install it into system32 so that PythonCOM can find it
(alternatively, we could now also making it an SxS assembly).

VS2008 adds another twist: assembly manifests. As a consequence of this
technology, if Python 2.6 is installed "just for me" on Windows Vista
(i.e. the CRT is next to the executable), it just won't work, because
the extension modules (.pyd) can't find the CRT.

> I proposed an alternatives because I read you saying: "The tricky part
> really is when it breaks (which it does more often than
> not), in which case you need to understand msi.py, for which you need to
> understand MSI". Which means that maybe everybody *has tools* to build
> an installer today, but only a few people have the required knowledge to
> really do releases on Windows.
> 
> So I believe that switching to an alternative that doesn't require full
> understanding of MSI and msi.py would probably low the barrier and allow
> more people to help you out.

I remain skeptical. You replace the need to learn MSI with the need to
learn this tool, and not only to work around the limitations of MSI, but
also around the limitations of the tool you have chosen.

Regards,
Martin

From mrts at mrts.pri.ee  Thu Nov 27 15:40:39 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Thu, 27 Nov 2008 16:40:39 +0200
Subject: [Python-Dev] __import__ problems
Message-ID: <492EB167.4010107@mrts.pri.ee>

Python programmers need to dynamically load submodules instead of
top-level modules -- given a string with module hierarchy, e.g.
'foo.bar.baz', access to the tail module 'baz' is required instead
of 'foo'.

Currently, the common hack for that is to use

>>> modname = 'foo.bar.baz' mod = __import__(modname, {}, {}, [''])

This, however, is indeed an undocumented hack and, what's worse,
causes 'baz' to be imported twice, as 'baz' and 'baz.' (with tail
dot). The problem is reported in [1] and the idiom pops up in about
2000 (sic!) lines in Google Code Search [2].

There at least two workarounds:
  * the getattr approach documented in [3]
  * use __import__(modname, {}, {}, [modname.rsplit(".", 1)[-1]])

As both of them are clumsy and inefficient, I created a simple patch
for __import__ [4] that adds yet another argument, 'submodule'
(maybe 'tail_module' would have been more appropriate) that caters
for that particular use case:

>>> __import__('foo.bar.baz') # submodule=False by default
<module 'foo' from 'foo/__init__.py'>

>>> __import__('foo.bar.baz', submodule=True)
<module 'foo.bar.baz' from 'foo/bar/baz.py'>

>>> __import__('foo.bar.baz', fromlist=['baz'])
<module 'foo.bar.baz' from 'foo/bar/baz.py'>

---

While I was doing that, I noticed that the import_module_level()
function that does the gruntwork behind __import__ does not entirely
match the documentation [3].

Namely, [3] states "the statement from spam.ham import eggs results in
__import__('spam.ham', globals(), locals(), ['eggs'], -1)."

This is incorrect:

>>> __import__('foo.bar', globals(), locals(), ['baz'], -1)
<module 'foo.bar' from 'foo/bar/__init__.py'>

i.e. 'bar' is imported, not 'baz' (or 'ham' and not 'eggs').

As a matter of fact, anything can be in 'fromlist' (the reason for
the API abuse seen in [2]):

>>> __import__('foo.bar.baz', globals(), locals(),
... ['this_is_a_bug'], -1)
<module 'foo.bar.baz' from 'foo/bar/baz/__init__.py'>

So, effectively, 'fromlist' is already functioning as a boolean that
indicates whether the tail or toplevel module is imported.

Proposal:

  * either fix __import__ to behave as documented:

     # from foo.bar import baz
     >>> __import__('foo.bar', fromlist=['baz'])
     <module 'foo.bar.baz' from 'foo/bar/baz/__init__.py'>

     # from foo.bar import baz, baq
     >>> __import__('foo.bar', fromlist=['baz', 'baq'])
     (<module 'foo.bar.baz' from 'foo/bar/baz/__init__.py'>,
     <module 'foo.bar.baq' from 'foo/bar/baq/__init__.py'>)

    and add the 'submodule' argument to support the common
    __import__ use case [4],

  * or if that is not feasible, retain the current boolean behaviour
    and make that explicit by renaming 'fromlist' to 'submodule' (and
    require the latter to be a boolean, not a list).

Too bad I couldn't come up with this before, 3.0 would have been a
perfect opportunity to get things right (one way or the other).

---

References:
[1] http://bugs.python.org/issue2090
[2] http://google.com/codesearch?hl=en&lr=&q=__import__.*%5C%5B%27%27%5C%5D
[3] http://docs.python.org/library/functions.html#__import__
[4] http://bugs.python.org/issue4438

From hrvoje.niksic at avl.com  Thu Nov 27 17:14:57 2008
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Thu, 27 Nov 2008 17:14:57 +0100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492EB167.4010107@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee>
Message-ID: <492EC781.9020900@avl.com>

Mart Somermaa wrote:
> There at least two workarounds:
>   * the getattr approach documented in [3]

I can't comment on the rest, but the getattr seems overly complicated. 
If you need just the module, what's wrong with:

__import__(modname)
modobj = sys.modules[modname]

From torne-pythondev at wolfpuppy.org.uk  Thu Nov 27 19:31:39 2008
From: torne-pythondev at wolfpuppy.org.uk (Torne Wuff)
Date: Thu, 27 Nov 2008 18:31:39 +0000
Subject: [Python-Dev] Python 2.6/3.0,
	IEEE 754 floating point semantics	and S60
In-Reply-To: <492EB5A3.1000608@cheimes.de>
References: <492EB5A3.1000608@cheimes.de>
Message-ID: <20081127183139.GA2027@wolfpuppy.org.uk>

On Thu, Nov 27 08 at  3:58:43PM +0100, Christian Heimes wrote:
> A while ago I contacted Jukka Laurila from the Nokia developer board  
> about IEEE 754 support on S60 phones. The information from Jukka may be  
> useful for future reference.

OK, I want to weigh in here, but first I need a lil disclaimer: I'm a
Symbian core developer and theoretically in a position to fix this kind
of thing, but I am not speaking in an official capacity and nothing here
should be taken to represent Symbian or Nokia etc etc blah blah :)

Also, I'm not quite a floating point expert, but have had to deal with
various issues in the past, and wrote some of our hardware floating
point support code, so the below is my best guesses.

> We've been battling with some float problems recently as well in the  
> course of our 2.5 core porting. The principal problem has been imprecise  
> floating point formatting/parsing routines which lead to the unfortunate  
> case of not being able to roundtrip a Python float to string and back  
> bit-exactly. I've been trying to get to the bottom of that problem and  
> so far it seems like the root cause is a poorly written C library.

The formatting/parsing routines for floats are not imprecise, they just
don't print enough digits to make roundtripping work. We only print 15
digits, because that's the most digits we can guarantee to calculate
accurately. Working out the 16th and 17th digits requires that you have
a higher-than-double-precision intermediate representation, which is not
guaranteed, see below. So, I'd like to suggest that "poorly written" is
not true. :)

> Though on the other hand, no one from Symbian is yet to confirm to me  
> that the software-emulated floating point routines correspond to  
> anything like IEEE 754. Most S60 devices lack floating point hardware.

I think I might've actually been the one to receive these queries.. I
did try to explain. We should be IEEE 754 compliant. We have a range of
tests which attempt to prove this. If someone has a case demonstrating
otherwise please let me know and I'll raise an issue. :)

Floating point hardware makes it worse, though, not better: ARM's VFP
floating point hardware has only single and double precision. Our
software implementation uses 64 bit mantissas for intermediates, which
is indeed enough to print 17 digits correctly, but on platforms where
the VFP is enabled globally, there is no such higher intermediate
precision and were we to try and print 17 digits some of them would be
wrong.

So, I'm not sure that it's actually possible for us to do better; maybe
it is, maybe not.

> The best guess is that the floating point support is IEEE 754'ish, with  
> imprecise formatting and parsing routines in the standard library.

IEEE 754 doesn't include string formatting or parsing as far as I know,
so I think this is irrelevant to whether we are compliant :)

Does *any* standard actually promise that roundtripping will work, or is
it just assumed?

-- 
Torne Wuff
torne at wolfpuppy.org.uk

From dickinsm at gmail.com  Thu Nov 27 20:27:03 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 27 Nov 2008 19:27:03 +0000
Subject: [Python-Dev] Python 2.6/3.0,
	IEEE 754 floating point semantics and S60
In-Reply-To: <20081127183139.GA2027@wolfpuppy.org.uk>
References: <492EB5A3.1000608@cheimes.de>
	<20081127183139.GA2027@wolfpuppy.org.uk>
Message-ID: <5c6f2a5d0811271127y285b234bw4420a8ef9c94c0e4@mail.gmail.com>

Torne,

Many thanks for your input!

> IEEE 754 doesn't include string formatting or parsing as far as I know,
> so I think this is irrelevant to whether we are compliant :)

I think IEEE 754 is relevant. :-)

There's a section 5.6 in the original 1985 standard that's called
something like:  "Binary <-> Decimal conversion" that covers
this.  If I'm reading it right (and I'm far from sure that I am), it
seems to say that for doubles one has to provide decimal
output up to 17 significant digits, but that digits beyond the
17th don't have to be accurate.  It also requires that the output is
correctly rounded for some subrange of the range of all doubles,
and there are weaker accuracy requirements for doubles outside
this subrange.

I think the updated 2008 version of the standard is somewhat
stricter, requiring correct rounding for conversions to and from
decimal in all cases.

> Does *any* standard actually promise that roundtripping will work, or is
> it just assumed?

I believe that roundtripping is a consequence of the 'correct
rounding' requirements in IEEE 754-2008, rather than a
direct requirement.  It might also be a consequence of the
weaker IEEE 754-1985 requirements, but I'm not sure.

Mark

From dickinsm at gmail.com  Thu Nov 27 20:50:47 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Thu, 27 Nov 2008 19:50:47 +0000
Subject: [Python-Dev] Python 2.6/3.0,
	IEEE 754 floating point semantics and S60
In-Reply-To: <5c6f2a5d0811271127y285b234bw4420a8ef9c94c0e4@mail.gmail.com>
References: <492EB5A3.1000608@cheimes.de>
	<20081127183139.GA2027@wolfpuppy.org.uk>
	<5c6f2a5d0811271127y285b234bw4420a8ef9c94c0e4@mail.gmail.com>
Message-ID: <5c6f2a5d0811271150u79b57525m9adb6762158814b2@mail.gmail.com>

Update: it looks like IEEE 754-1985 does require that
roundtripping works, at least when the rounding mode
is round-to-nearest.  From section 5.6 again:

"""
When rounding to nearest, conversion from binary to decimal and back to
binary shall be the identity as long as the decimal string is carried to the
maximum precision specified in Table 2, namely, 9 digits for single and 17
digits for double.
"""

Mark

From nicole at cats-muvva.net  Thu Nov 27 20:52:43 2008
From: nicole at cats-muvva.net (Nicole King)
Date: Thu, 27 Nov 2008 19:52:43 +0000
Subject: [Python-Dev] Taint Mode in Python 3.0 RC3
Message-ID: <200811271952.43316.nicole@cats-muvva.net>

Dear All,

I found I needed support for taint mode in python and have done some work to 
realise this. It's by no means complete at this time, but I'm floating this 
idea on this group to see how much interest there is.

The implementation is pretty simple:

- an extra field in PyObject to maintain the taint status
- a couple of extra functions __gettaint__() that returns the taint status and 
__settaint__(value) that sets the taint value, returning the previous status
- an additional command-line flag -a and environment variable 
PYTHONIGNORETAINT that suppress taint checking
- a few macros defined in Objects/object.h to support taint management
- a new built-in exception, PyExc_TaintError, for reporting operations on 
tainted objects

You can pick up the diff at 
http://www.cats-muvva.net/software/Python-taint-diff-3.0rc3.tar.bz2

This diff is ONLY for 3.0RC3. Given that I have only a limited understanding 
of the internals of Python (but it's very clearly written), I'm sure there 
are some behaviours I've overlooked.

I have done very little testing at this point, but, at least, everything 
compiles!

Regards

Nicole King (aka CatsMuvva)

From greg at krypto.org  Thu Nov 27 21:40:29 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 27 Nov 2008 12:40:29 -0800
Subject: [Python-Dev] [Python-3000] 2.6.1 and 3.0
In-Reply-To: <492EBBB6.1060203@v.loewis.de>
References: <3E43E049-6F3F-4FAA-9746-3FDE38F34A39@python.org>
	<ggk94d$pr1$1@ger.gmane.org> <492DABAF.5020808@v.loewis.de>
	<1227736209.6739.9.camel@ozzu> <492DCA8F.4050105@cheimes.de>
	<492DCFFE.2060903@v.loewis.de> <1227739616.6739.13.camel@ozzu>
	<492DDBE6.60205@v.loewis.de> <1227778172.6944.8.camel@ozzu>
	<492EBBB6.1060203@v.loewis.de>
Message-ID: <52dc1c820811271240p7552c55dt16e2e88203703ec4@mail.gmail.com>

I am not at all a windows person but I have used
http://www.dennisbareis.com/makemsi.htm in the past to automate editing and
tweaking some MSI files for testing.  It can also be used to generate new
ones.  It looks like it would still require something to generate its own
input description.  Regardless, just wanted to offer the link so people are
aware that it exists.  I have no opinion on what actually gets used so long
as its automated.

-gps

On Thu, Nov 27, 2008 at 7:24 AM, "Martin v. L?wis" <martin at v.loewis.de>wrote:

> Giovanni Bajo wrote:
> > On gio, 2008-11-27 at 00:29 +0100, "Martin v. L?wis" wrote:
> >>> So, deducing from your reply, this "merge module" is a thing that
> allows
> >>> to install the CRT (and other shared components)?
> >> Correct. More generally, a merge module is a something like an MSI
> >> library (.a). It includes a set of files and snippets of an installation
> >> procedure for them.
> >
> > OK. One question: why CRT doesn't get installed as regular files near to
> > the python executable? That's how I usually ship it, but maybe Python
> > has some special need.
>
> When installing "for all users", pythonxy.dll goes into system32. This,
> in turn, requires the CRT to be installed globally (which meant into
> system32 for VS6 and VS7.1, but means using SxS for VS 2008). It's
> necessary to install it into system32 so that PythonCOM can find it
> (alternatively, we could now also making it an SxS assembly).
>
> VS2008 adds another twist: assembly manifests. As a consequence of this
> technology, if Python 2.6 is installed "just for me" on Windows Vista
> (i.e. the CRT is next to the executable), it just won't work, because
> the extension modules (.pyd) can't find the CRT.
>
> > I proposed an alternatives because I read you saying: "The tricky part
> > really is when it breaks (which it does more often than
> > not), in which case you need to understand msi.py, for which you need to
> > understand MSI". Which means that maybe everybody *has tools* to build
> > an installer today, but only a few people have the required knowledge to
> > really do releases on Windows.
> >
> > So I believe that switching to an alternative that doesn't require full
> > understanding of MSI and msi.py would probably low the barrier and allow
> > more people to help you out.
>
> I remain skeptical. You replace the need to learn MSI with the need to
> learn this tool, and not only to work around the limitations of MSI, but
> also around the limitations of the tool you have chosen.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081127/3ccc6809/attachment-0001.htm>

From greg at krypto.org  Thu Nov 27 21:58:31 2008
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 27 Nov 2008 12:58:31 -0800
Subject: [Python-Dev] socket.c, _rbufsize
In-Reply-To: <4E9372E6B2234D4F859320D896059A9510C633B8B7@exchis.ccp.ad.local>
References: <4E9372E6B2234D4F859320D896059A9510C633B8B7@exchis.ccp.ad.local>
Message-ID: <52dc1c820811271258p5671b94ase247b3fe0b34a47b@mail.gmail.com>

I've created http://bugs.python.org/issue4448 to track this issue.

On Thu, Nov 27, 2008 at 6:50 AM, Kristj?n Valur J?nsson <
kristjan at ccpgames.com> wrote:

>  I came across this in socket.c:
>
>         # _rbufsize is the suggested recv buffer size.  It is *strictly*
>
>         # obeyed within readline() for recv calls.  If it is larger than
>
>         # default_bufsize it will be used for recv calls within read().
>
>
>
> What I worry about is the readline() case.  Is there a reason why we want
> to strictly obey it for that function?  Note that in the documentation for
> _fileobject.read() it says:
>
>         # Use max, disallow tiny reads in a loop as they are very
> inefficient.
>
>
>
> The same argument surely applies for readline().
>
>
>
> The reason I am fretting about this is that httplib.py (and therefore
> xmlrpclib.py) specify bufsize=0 when createing their socket fileobjects,
> presumably to make sure that write() operations are not buffered but flushed
> immediately.  But this has the side effect of setting the _rbufsize to 1,
> and so readline() calls become very slow.
>
>
>
> I suggest that readline() be made to use at least defaultbufsize, like
> read().  Any thoughts?
>
>
>
> Cheers,
>
>
>
> Kristj?n
>
>
>
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081127/0549991c/attachment.htm>

From ncoghlan at gmail.com  Thu Nov 27 22:22:42 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Nov 2008 07:22:42 +1000
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492EB167.4010107@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee>
Message-ID: <492F0FA2.10905@gmail.com>

Mart Somermaa wrote:
>>>> __import__('foo.bar', globals(), locals(), ['baz'], -1)
> <module 'foo.bar' from 'foo/bar/__init__.py'>
> 
> i.e. 'bar' is imported, not 'baz' (or 'ham' and not 'eggs').

You're reading it wrong. 'baz' *is* imported, but 'bar' is returned from
the function call. You will find that the import statement generates
some additional opcodes to then do the assignment statements off the VM
stack.

This is stated explicitly in the documentation of __import__:

"""Note that even though locals() and ['eggs'] are passed in as
arguments, the __import__() function does not set the local variable
named eggs; this is done by subsequent code that is generated for the
import statement."""

And as described later:

"""when using from spam.ham import eggs, the spam.ham subpackage must be
used to find the eggs variable."""

i.e.

"from foo.bar import baz" ---->

<stack top> = __import__('foo.bar', globals(), locals(), ['baz'], -1)
baz = <stack top>.baz

When there are multiple names being imported or an 'as' clause is
involved, I hope the reasons for doing it this way become more obvious:

"from foo.bar import baz, bob" ---->

<stack top> = __import__('foo.bar', globals(), locals(), ['baz', 'bob'], -1)
baz = <stack top>.baz
bob = <stack top>.bob

"from foo.bar import baz as bob" ---->

<stack top> = __import__('foo.bar', globals(), locals(), ['baz', 'bob'], -1)
bob = <stack top>.baz

There's a perfectly correct approach documented as part of the
__import__ docs that you referenced in your original email. If
developers decide not to use that approach and use an undocumented hack
instead, they have no right to complain when their code doesn't work
properly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From larry.bugbee at boeing.com  Thu Nov 27 22:20:02 2008
From: larry.bugbee at boeing.com (Bugbee, Larry)
Date: Thu, 27 Nov 2008 13:20:02 -0800
Subject: [Python-Dev] Python for windows.
References: <mailman.25385.1227729845.3486.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>
	<492DE05D.4020805@v.loewis.de> 
Message-ID: <9418DB6C0B9D434190E54A78E931C3D1086DE3BB@XCH-NW-7V1.nw.nos.boeing.com>

> > > All, and not to start flames, but I still do not understand why 
> > > applink.c isn't included in python's main (conditionally) instead 
> > > of expecting users, many of them novices, to do the build.  ???
> > 
> > One reason is that I don't know what applink is, and why I should care 
> > about it. (I may have known in the past, but then I have forgotten 
> > since).
> 
> Yeah, it's been a while for me too.
> 
> As I recall, OpenSSL, a long while ago stopped, supporting some 
> idiosyncrasies associated with Windows I/O and opted for a "cleaner" 
> approach, that of requiring developers to link a small file, applink.c, 
> into the app's main.  applink.c is provided by the OpenSSL folks.

This is a years old issue and while I periodically revisited and confirmed the problem remained (again as late as a month or two ago), all of this is from memory and I cannot tell you version numbers and the like.  Before pursuing further, I should do virgin installs and test again.  I'll report back my findings in a couple of days, either way.  

My thanks,

Larry

From mrts at mrts.pri.ee  Thu Nov 27 23:39:14 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Fri, 28 Nov 2008 00:39:14 +0200
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492F0FA2.10905@gmail.com>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
Message-ID: <492F2192.2050102@mrts.pri.ee>

Nick Coghlan wrote:
> i.e.
>
> "from foo.bar import baz" ---->
>
> <stack top> = __import__('foo.bar', globals(), locals(), ['baz'], -1)
> baz = <stack top>.baz
>
> When there are multiple names being imported or an 'as' clause is
> involved, I hope the reasons for doing it this way become more obvious:
>
> "from foo.bar import baz, bob" ---->
>
> <stack top> = __import__('foo.bar', globals(), locals(), ['baz', 'bob'], -1)
> baz = <stack top>.baz
> bob = <stack top>.bob
>
> "from foo.bar import baz as bob" ---->
>
> <stack top> = __import__('foo.bar', globals(), locals(), ['baz', 'bob'], -1)
> bob = <stack top>.baz
>   

Thanks for clarifying this! I'd say the current wording of the docs is 
non-obvious
in that regard and your examples would be a nice addition to them.

> There's a perfectly correct approach documented as part of the
> __import__ docs that you referenced in your original email. If
> developers decide not to use that approach and use an undocumented hack
> instead, they have no right to complain when their code doesn't work
> properly.
>   

There's a good reason for the hack -- the documented loop over components
+ getattr is both a bit ugly and inefficient (as is
fromlist=[modname.rsplit(".", 1)[-1]]). The variant proposed by Hrvoje 
Niksic:

 >>> __import__(modname)
 >>> mod = sys.modules[modname]

looks more appealing, but comes with the drawback that sys has to be 
imported for
that purpose only.

As none of the variants is really in the spirit of Python's zen and the 
use case
"given a string of dotted module names, import the tail module" is 
really common,
I'd still say we should go with the flow and provide people what they 
need explicitly
(i.e. something in the lines of  http://bugs.python.org/issue4438 ) -- 
and disable
the hack on the same occasion (by the way, is there a reason why both
__import__('foo', fromlist=['invalid']) and __import__('foo', 
fromlist=['']) don't raise
an error as of now?). This way the hacks will be eventually fixed in all 
those 2000
lines listed in Google Code.

Perhaps 3.1 and 2.7 would be an appropriate chance for that?

From greg.ewing at canterbury.ac.nz  Fri Nov 28 00:00:20 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 28 Nov 2008 12:00:20 +1300
Subject: [Python-Dev] Python for windows.
In-Reply-To: <9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>
References: <mailman.25385.1227729845.3486.python-dev@python.org>
	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>
	<492DE05D.4020805@v.loewis.de>
	<9418DB6C0B9D434190E54A78E931C3D1086DE340@XCH-NW-7V1.nw.nos.boeing.com>
Message-ID: <492F2684.4060700@canterbury.ac.nz>

Bugbee, Larry wrote:

> As I recall, OpenSSL, a long while ago stopped, supporting some idiosyncrasies
 > associated with Windows I/O and opted for a "cleaner" approach, that of 
requiring
> developers to link a small file, applink.c, into the app's main.

Could it not be linked into the openssl extension module
instead? It's already assumed that Python extension modules
are linked against the correct version of the runtime for
the python.exe being used.

-- 
Greg

From nicole at cats-muvva.net  Thu Nov 27 23:57:10 2008
From: nicole at cats-muvva.net (Nicole King)
Date: Thu, 27 Nov 2008 22:57:10 +0000
Subject: [Python-Dev] Taint Mode in Python 3.0 RC3
Message-ID: <200811272257.10984.nicole@cats-muvva.net>

Dear All,

Apologies that web site was not working earlier. I believe that I've now fixed 
it. The patch can still be found at 
http://www.cats-muvva.net/software/Python-taint-diff-3.0rc3.tar.bz2.

CatsMuvva

From greg.ewing at canterbury.ac.nz  Fri Nov 28 00:04:40 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 28 Nov 2008 12:04:40 +1300
Subject: [Python-Dev] Python for windows.
In-Reply-To: <045801c9503a$8e85d2f0$ab9178d0$@com.au>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au> <492DCE5E.5080602@v.loewis.de>
	<043e01c95019$9955a0a0$cc00e1e0$@com.au> <492DDE40.2040206@v.loewis.de>
	<045801c9503a$8e85d2f0$ab9178d0$@com.au>
Message-ID: <492F2788.7040300@canterbury.ac.nz>

Mark Hammond wrote:

> The only conflict I see here is the requirement to install into "\Program
> Files"

Doesn't that just mean that if an OEM decides to preinstall it,
they need to put it in Program Files? They're at liberty to
do that.

-- 
Greg

From lists at cheimes.de  Fri Nov 28 00:15:19 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 28 Nov 2008 00:15:19 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492DE05D.4020805@v.loewis.de>
References: <mailman.25385.1227729845.3486.python-dev@python.org>	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>
	<492DE05D.4020805@v.loewis.de>
Message-ID: <492F2A07.7030000@cheimes.de>

Martin v. L?wis wrote:
>> All, and not to start flames, but I still do not understand why
>> applink.c isn't included in python's main (conditionally) instead of
>> expecting users, many of them novices, to do the build.  ???
> 
> One reason is that I don't know what applink is, and why I should
> care about it. (I may have known in the past, but then I have forgotten
> since).

Applink is roughly explained at 
http://www.openssl.org/support/faq.html#PROG2. The matter was discussed 
about half a year ago but no decision was made. See 
http://mail.python.org/pipermail/python-dev/2008-March/077424.html

applink.c is just a table of integer constants to function pointers. It 
makes mixing of different CRTs secure. You'll get the idea after reading 
the file, Martin. A similar approach could be useful for Python, too.

Christian

From mhammond at skippinet.com.au  Fri Nov 28 04:55:42 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Fri, 28 Nov 2008 14:55:42 +1100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492F2788.7040300@canterbury.ac.nz>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>
	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>
	<ggkat4$2q5$1@ger.gmane.org>
	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>
	<042401c95012$3bff99d0$b3fecd70$@com.au>
	<492DCE5E.5080602@v.loewis.de>
	<043e01c95019$9955a0a0$cc00e1e0$@com.au>
	<492DDE40.2040206@v.loewis.de>
	<045801c9503a$8e85d2f0$ab9178d0$@com.au>
	<492F2788.7040300@canterbury.ac.nz>
Message-ID: <04ce01c9510d$3132d750$939885f0$@com.au>

Greg writes:
> Mark Hammond wrote:
> 
> > The only conflict I see here is the requirement to install into
> "\Program Files"
> 
> Doesn't that just mean that if an OEM decides to preinstall it,
> they need to put it in Program Files? They're at liberty to
> do that.

I'm not very familiar with the OEM process, but I believe the software is
automatically installed after the first boot from MSI files supplied by the
OEM partner (so technically, its not "pre-installed" at all).  I don't know
enough about how that works to know how easy it would be to override that
option during installation - hopefully Gerald will fill us in with more
details...

[Hrm - looking closer at that HTML link I sent before, it says specifically
"Per-machine installs must install to Program Files by default in order to
pass this test case" - that seems pretty clear...]

Cheers,

Mark


From martin at v.loewis.de  Fri Nov 28 07:42:43 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 28 Nov 2008 07:42:43 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492F2A07.7030000@cheimes.de>
References: <mailman.25385.1227729845.3486.python-dev@python.org>	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>	<492DE05D.4020805@v.loewis.de>
	<492F2A07.7030000@cheimes.de>
Message-ID: <492F92E3.80300@v.loewis.de>

> Applink is roughly explained at
> http://www.openssl.org/support/faq.html#PROG2. The matter was discussed
> about half a year ago but no decision was made. See
> http://mail.python.org/pipermail/python-dev/2008-March/077424.html
> 
> applink.c is just a table of integer constants to function pointers. It
> makes mixing of different CRTs secure. You'll get the idea after reading
> the file, Martin. A similar approach could be useful for Python, too.

I (now) understand all that. I still don't understand why Python should
use it, *for OpenSSL*. (that Python might use something like that for
itself is probably not the point Larry is trying to make).

Now that I read the archives, I see that I didn't understand in
February what specific problem would be solved. I still don't see today
how it would solve a problem.

Regards,
Martin

From hrvoje.niksic at avl.com  Fri Nov 28 09:33:08 2008
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 28 Nov 2008 09:33:08 +0100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492F2192.2050102@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
	<492F2192.2050102@mrts.pri.ee>
Message-ID: <492FACC4.3020907@avl.com>

Mart Somermaa wrote:
> The variant proposed by Hrvoje Niksic:
> 
>  >>> __import__(modname)
>  >>> mod = sys.modules[modname]
> 
> looks more appealing, but comes with the drawback that sys has to be 
> imported for that purpose only.

That is not a real drawback, as "sys" will certainly be present in the 
system, so the "importing" boils down to a dict lookup and a variable 
assignment.

Having said that, I'd add that I found the behavior of __import__ 
counter-intuitive, but assumed there's a good reason for it.  If I 
hadn't known about sys.modules beforehand, I would have probably gone 
the chained-getattr route as well.

From ncoghlan at gmail.com  Fri Nov 28 10:34:36 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Nov 2008 19:34:36 +1000
Subject: [Python-Dev] Python for windows.
In-Reply-To: <04ce01c9510d$3132d750$939885f0$@com.au>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>	<042401c95012$3bff99d0$b3fecd70$@com.au>	<492DCE5E.5080602@v.loewis.de>	<043e01c95019$9955a0a0$cc00e1e0$@com.au>	<492DDE40.2040206@v.loewis.de>	<045801c9503a$8e85d2f0$ab9178d0$@com.au>	<492F2788.7040300@canterbury.ac.nz>
	<04ce01c9510d$3132d750$939885f0$@com.au>
Message-ID: <492FBB2C.5000309@gmail.com>

Mark Hammond wrote:
> Greg writes:
>> Mark Hammond wrote:
>>
>>> The only conflict I see here is the requirement to install into
>> "\Program Files"
>>
>> Doesn't that just mean that if an OEM decides to preinstall it,
>> they need to put it in Program Files? They're at liberty to
>> do that.
> 
> I'm not very familiar with the OEM process, but I believe the software is
> automatically installed after the first boot from MSI files supplied by the
> OEM partner (so technically, its not "pre-installed" at all).  I don't know
> enough about how that works to know how easy it would be to override that
> option during installation - hopefully Gerald will fill us in with more
> details...
> 
> [Hrm - looking closer at that HTML link I sent before, it says specifically
> "Per-machine installs must install to Program Files by default in order to
> pass this test case" - that seems pretty clear...]

Given that the links in Gerald's examples were under Program Files, I
had assumed that HP were tweaking the installer at least that much...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From mrts at mrts.pri.ee  Fri Nov 28 11:02:21 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Fri, 28 Nov 2008 12:02:21 +0200
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492F0FA2.10905@gmail.com>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
Message-ID: <492FC1AD.70106@mrts.pri.ee>

>> The variant proposed by Hrvoje Niksic:
>> 
>>  >>> __import__(modname)
>>  >>> mod = sys.modules[modname]
>> 
>> looks more appealing, but comes with the drawback that sys has to be 
>> imported for that purpose only.
>
> That is not a real drawback, as "sys" will certainly be present in the 
> system, so the "importing" boils down to a dict lookup and a variable 
> assignment.

I meant that you have to

>>> import sys

only to access sys.modules (i.e. importing sys may not be necessary otherwise).

Compare

>>> mod = __import__(modname, submodule=True)

with

>>> import sys
>>> __import__(modname)
>>> mod = sys.modules[modname]



From mal at egenix.com  Fri Nov 28 11:33:25 2008
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 28 Nov 2008 11:33:25 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492F2A07.7030000@cheimes.de>
References: <mailman.25385.1227729845.3486.python-dev@python.org>	<9418DB6C0B9D434190E54A78E931C3D1086DE279@XCH-NW-7V1.nw.nos.boeing.com>	<492DE05D.4020805@v.loewis.de>
	<492F2A07.7030000@cheimes.de>
Message-ID: <492FC8F5.2040908@egenix.com>

On 2008-11-28 00:15, Christian Heimes wrote:
> Martin v. L?wis wrote:
>>> All, and not to start flames, but I still do not understand why
>>> applink.c isn't included in python's main (conditionally) instead of
>>> expecting users, many of them novices, to do the build.  ???
>>
>> One reason is that I don't know what applink is, and why I should
>> care about it. (I may have known in the past, but then I have forgotten
>> since).
> 
> Applink is roughly explained at
> http://www.openssl.org/support/faq.html#PROG2. The matter was discussed
> about half a year ago but no decision was made. See
> http://mail.python.org/pipermail/python-dev/2008-March/077424.html
> 
> applink.c is just a table of integer constants to function pointers. It
> makes mixing of different CRTs secure. You'll get the idea after reading
> the file, Martin. A similar approach could be useful for Python, too.

So that's why we don't see a problem with pyOpenSSL. From the first
link:

"""
Your application must link against the same version of the Win32 C-Runtime
against which your openssl libraries were linked. The default version for
OpenSSL is /MD - "Multithreaded DLL".
"""

and later on:

"""
As per 0.9.8 the above limitation is eliminated for .DLLs. ...
Instead of re-compiling OpenSSL toolkit, ...[you have to add]
<install-root>/include/openssl/applink.c ... to your application project
or simply #include-d in one [and only one] of your application source files.
...
[Note that] it is as important to add CRYPTO_malloc_init prior first call
to OpenSSL.
"""

In our eGenix pyOpenSSL distribution we ship the Windows DLLs for
OpenSSL together with the compiled PYDs for pyOpenSSL - all compiled
using the same compiler settings.

Python for Windows does the same, so there should be no issue either.

>From the comment it appears that you only see problems, if you try to use
those extensions from a Python executable that was compiled using
different settings, e.g. an embedded Python interpreter.

Note that neither Python nor pyOpenSSL call the required CRYPTO_malloc_init()
prior to using the other SSL APIs, so even including applink.c would
not help - you have to add this call to the used extensions as well.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Nov 28 2008)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2008-11-12: Released mxODBC.Connect 0.9.3      http://python.egenix.com/

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611

From hrvoje.niksic at avl.com  Fri Nov 28 11:36:44 2008
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 28 Nov 2008 11:36:44 +0100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492FC1AD.70106@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
	<492FC1AD.70106@mrts.pri.ee>
Message-ID: <492FC9BC.9020506@avl.com>

Mart Somermaa wrote:
> I meant that you have to
> 
>>>> import sys
> 
> only to access sys.modules (i.e. importing sys may not be necessary otherwise).

I understand that, but I'm arguing that that's a non-problem.  Importing 
sys is a regular thing in Python, not an exception.  You need sys to get 
to sys.argv, sys.exit, sys.stdout, etc. -- it's not like sys is an 
infrequently used module.  Since sys is always present, importing it is 
not an efficiency problem, either.

>>>> mod = __import__(modname, submodule=True)
> 
> with
> 
>>>> import sys
>>>> __import__(modname)
>>>> mod = sys.modules[modname]

"import sys" is normally located near the beginning of the file (and 
needed by other things), so the actual code snippet would really contain 
only those two lines, which don't strike me as bad.  Ideally, __import__ 
would simply return the "tail" imported module in the first place, but I 
don't think introducing a boolean keyword argument really improves the 
design.

From fijall at gmail.com  Fri Nov 28 11:49:23 2008
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 28 Nov 2008 11:49:23 +0100
Subject: [Python-Dev] Taint Mode in Python 3.0 RC3
In-Reply-To: <200811272257.10984.nicole@cats-muvva.net>
References: <200811272257.10984.nicole@cats-muvva.net>
Message-ID: <693bc9ab0811280249v2bc9f63br7ee61477ddc88d35@mail.gmail.com>

I get 404 object not found.

On Thu, Nov 27, 2008 at 11:57 PM, Nicole King <nicole at cats-muvva.net> wrote:
> Dear All,
>
> Apologies that web site was not working earlier. I believe that I've now fixed
> it. The patch can still be found at
> http://www.cats-muvva.net/software/Python-taint-diff-3.0rc3.tar.bz2.
>
> CatsMuvva

From ncoghlan at gmail.com  Fri Nov 28 12:48:51 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 28 Nov 2008 21:48:51 +1000
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492FC1AD.70106@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
	<492FC1AD.70106@mrts.pri.ee>
Message-ID: <492FDAA3.8030706@gmail.com>

Mart Somermaa wrote:
>>> The variant proposed by Hrvoje Niksic:
>>>
>>>  >>> __import__(modname)
>>>  >>> mod = sys.modules[modname]
>>>
>>> looks more appealing, but comes with the drawback that sys has to be
>>> imported for that purpose only.
>>
>> That is not a real drawback, as "sys" will certainly be present in the
>> system, so the "importing" boils down to a dict lookup and a variable
>> assignment.
> 
> I meant that you have to
> 
>>>> import sys
> 
> only to access sys.modules (i.e. importing sys may not be necessary
> otherwise).

As Hrvoje has pointed out, 'sys' is part of the internal interpreter
machinery - it's there as soon as the interpreter starts. The import
call for it just grabs it out of the module cache and creates a
reference to it in the current namespace.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From hrvoje.niksic at avl.com  Fri Nov 28 13:38:14 2008
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 28 Nov 2008 13:38:14 +0100
Subject: [Python-Dev] Python under valgrind
Message-ID: <492FE636.50905@avl.com>

A friend pointed out that running python under valgrind (simply 
"valgrind python") produces a lot of "invalid read" errors.  Reading up 
on Misc/README.valgrind only seems to describe why "uninitialized reads" 
should occur, not invalid ones.  For example:

$ valgrind python
[... lots of output ...]
==31428== Invalid read of size 4
==31428==    at 0x808EBDF: PyObject_Free (in /usr/bin/python2.5)
==31428==    by 0x810DD0A: (within /usr/bin/python2.5)
==31428==    by 0x810DD34: PyNode_Free (in /usr/bin/python2.5)
==31428==    by 0x80EDAD9: PyRun_InteractiveOneFlags (in /usr/bin/python2.5)
==31428==    by 0x80EDDB7: PyRun_InteractiveLoopFlags (in 
/usr/bin/python2.5)
==31428==    by 0x80EE515: PyRun_AnyFileExFlags (in /usr/bin/python2.5)
==31428==    by 0x80595E6: Py_Main (in /usr/bin/python2.5)
==31428==    by 0x8058961: main (in /usr/bin/python2.5)
==31428==  Address 0x43bf010 is 3,112 bytes inside a block of size 6,016 
free'd
==31428==    at 0x4024B4A: free (vg_replace_malloc.c:323)
==31428==    by 0x8059C07: (within /usr/bin/python2.5)
==31428==    by 0x80EDAA5: PyRun_InteractiveOneFlags (in /usr/bin/python2.5)
...

valgrind claims that Python reads 4 bytes inside a block on which free() 
has already been called.  Is valgrind wrong, or is Python really doing 
that?  Googling revealed previous reports of this, normally answered by 
a reference to README.valgrind.  But README.valgrind justifies reading 
from ununitialized memory, which doesn't help me understand how reading 
from the middle of a block of freed memory (more precisely, memory on 
which the libc free() has already been called) would be okay.

I suppose valgrind could be confused by PyFree's pool address validation 
that intentionally reads the memory just before the allocated block, and 
incorrectly attributes it to a previously allocated (and hence freed) 
block, but I can't prove that.  Has anyone investigated this kind of 
valgrind report?

From gjcarneiro at gmail.com  Fri Nov 28 13:54:38 2008
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Fri, 28 Nov 2008 12:54:38 +0000
Subject: [Python-Dev] Python under valgrind
In-Reply-To: <492FE636.50905@avl.com>
References: <492FE636.50905@avl.com>
Message-ID: <a467ca4f0811280454k7aaf5de4ha95dd15109664228@mail.gmail.com>

2008/11/28 Hrvoje Niksic <hrvoje.niksic at avl.com>

> A friend pointed out that running python under valgrind (simply "valgrind
> python") produces a lot of "invalid read" errors.  Reading up on
> Misc/README.valgrind only seems to describe why "uninitialized reads" should
> occur, not invalid ones.  For example:
>
> $ valgrind python
> [... lots of output ...]
> ==31428== Invalid read of size 4
> ==31428==    at 0x808EBDF: PyObject_Free (in /usr/bin/python2.5)
> ==31428==    by 0x810DD0A: (within /usr/bin/python2.5)
> ==31428==    by 0x810DD34: PyNode_Free (in /usr/bin/python2.5)
> ==31428==    by 0x80EDAD9: PyRun_InteractiveOneFlags (in
> /usr/bin/python2.5)
> ==31428==    by 0x80EDDB7: PyRun_InteractiveLoopFlags (in
> /usr/bin/python2.5)
> ==31428==    by 0x80EE515: PyRun_AnyFileExFlags (in /usr/bin/python2.5)
> ==31428==    by 0x80595E6: Py_Main (in /usr/bin/python2.5)
> ==31428==    by 0x8058961: main (in /usr/bin/python2.5)
> ==31428==  Address 0x43bf010 is 3,112 bytes inside a block of size 6,016
> free'd
> ==31428==    at 0x4024B4A: free (vg_replace_malloc.c:323)
> ==31428==    by 0x8059C07: (within /usr/bin/python2.5)
> ==31428==    by 0x80EDAA5: PyRun_InteractiveOneFlags (in
> /usr/bin/python2.5)
> ...
>
> valgrind claims that Python reads 4 bytes inside a block on which free()
> has already been called.  Is valgrind wrong, or is Python really doing that?
>  Googling revealed previous reports of this, normally answered by a
> reference to README.valgrind.  But README.valgrind justifies reading from
> ununitialized memory, which doesn't help me understand how reading from the
> middle of a block of freed memory (more precisely, memory on which the libc
> free() has already been called) would be okay.
>
> I suppose valgrind could be confused by PyFree's pool address validation
> that intentionally reads the memory just before the allocated block, and
> incorrectly attributes it to a previously allocated (and hence freed) block,
> but I can't prove that.  Has anyone investigated this kind of valgrind
> report?


I can't answer your question directly, but I can tell you that whenever I
have to debug memory problems with python extensions is usually use my own
python compiled with --with-pydebug --without-pymalloc.  It really helps
with valgrind.

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20081128/13408172/attachment.htm>

From amauryfa at gmail.com  Fri Nov 28 13:56:34 2008
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Fri, 28 Nov 2008 13:56:34 +0100
Subject: [Python-Dev]  Python under valgrind
In-Reply-To: <e27efe130811280455p2c037a77g367b81d099164f47@mail.gmail.com>
References: <492FE636.50905@avl.com>
	<e27efe130811280455p2c037a77g367b81d099164f47@mail.gmail.com>
Message-ID: <e27efe130811280456y6316f962o6ed67fe810b2b1c2@mail.gmail.com>

Hrvoje Niksic wrote:
> A friend pointed out that running python under valgrind (simply "valgrind
> python") produces a lot of "invalid read" errors.  Reading up on
> Misc/README.valgrind only seems to describe why "uninitialized reads" should
> occur, not invalid ones.  For example:
 [...]
> I suppose valgrind could be confused by PyFree's pool address validation
> that intentionally reads the memory just before the allocated block, and
> incorrectly attributes it to a previously allocated (and hence freed) block,
> but I can't prove that.  Has anyone investigated this kind of valgrind
> report?

Did you use the suppressions file as suggested in Misc/README.valgrind?

   --suppressions=Misc/valgrind-python.supp

--
Amaury Forgeot d'Arc

From victor.stinner at haypocalc.com  Fri Nov 28 14:09:12 2008
From: victor.stinner at haypocalc.com (Victor Stinner)
Date: Fri, 28 Nov 2008 14:09:12 +0100
Subject: [Python-Dev] Python under valgrind
In-Reply-To: <e27efe130811280456y6316f962o6ed67fe810b2b1c2@mail.gmail.com>
References: <492FE636.50905@avl.com>
	<e27efe130811280455p2c037a77g367b81d099164f47@mail.gmail.com>
	<e27efe130811280456y6316f962o6ed67fe810b2b1c2@mail.gmail.com>
Message-ID: <200811281409.12899.victor.stinner@haypocalc.com>

Le Friday 28 November 2008 13:56:34 Amaury Forgeot d'Arc, vous avez ?crit?:
> Hrvoje Niksic wrote:
> > A friend pointed out that running python under valgrind (simply "valgrind
> > python") produces a lot of "invalid read" errors. (...) PyFree (...)
>
> Did you use the suppressions file as suggested in Misc/README.valgrind?
>
>    --suppressions=Misc/valgrind-python.supp

To be able to use the suppressions, Python have to be compiled with 
Py_USING_MEMORY_DEBUGGER. Edit Object/obmalloc.c near line 680.

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/

From hrvoje.niksic at avl.com  Fri Nov 28 14:51:48 2008
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Fri, 28 Nov 2008 14:51:48 +0100
Subject: [Python-Dev] Python under valgrind
In-Reply-To: <e27efe130811280456y6316f962o6ed67fe810b2b1c2@mail.gmail.com>
References: <492FE636.50905@avl.com>	<e27efe130811280455p2c037a77g367b81d099164f47@mail.gmail.com>
	<e27efe130811280456y6316f962o6ed67fe810b2b1c2@mail.gmail.com>
Message-ID: <492FF774.1050101@avl.com>

Amaury Forgeot d'Arc wrote:
> Did you use the suppressions file as suggested in Misc/README.valgrind?

Thanks for the suggestion (as well as to Gustavo and Victor), but my 
question wasn't about how to suppress the messages, but about why the 
messages appear in the first place.  I think my last paragraph answers 
my own question, but I'm not sure.

From mrts at mrts.pri.ee  Fri Nov 28 13:59:39 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Fri, 28 Nov 2008 14:59:39 +0200
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492FDAA3.8030706@gmail.com>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
	<492FC1AD.70106@mrts.pri.ee> <492FDAA3.8030706@gmail.com>
Message-ID: <492FEB3B.2040608@mrts.pri.ee>

Nick Coghlan wrote:

> As Hrvoje has pointed out, 'sys' is part of the internal interpreter
> machinery - it's there as soon as the interpreter starts. The import
> call for it just grabs it out of the module cache and creates a
> reference to it in the current namespace.
I understand that, but

Explicit is better than implicit.

 --> non-explicit import sys and __import__(modname) to access

     sys.modules[modname]

Simple is better than complex.

 --> three lines for a common use case instead of one

Readability counts.

 --> why is sys imported in this WSGI app (that shouldn't generally

     touch any of the common sys.argv, sys.stdout etc)?!

     (500 lines later) aha, to access sys.modules

There should be one-- and preferably only one --obvious way to do it.

--> should I use the 3-line idiom? should I use the getattr() idiom?

    Ideally, __import__() should support the only obvious way.

In short, the three-line version doesn't convey the intent in an obvious way and does more work than is necessary.


From steve at pearwood.info  Fri Nov 28 17:17:37 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 29 Nov 2008 03:17:37 +1100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492FEB3B.2040608@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee> <492FDAA3.8030706@gmail.com>
	<492FEB3B.2040608@mrts.pri.ee>
Message-ID: <200811290317.38431.steve@pearwood.info>

On Fri, 28 Nov 2008 11:59:39 pm Mart Somermaa wrote:
> Nick Coghlan wrote:
> > As Hrvoje has pointed out, 'sys' is part of the internal
> > interpreter machinery - it's there as soon as the interpreter
> > starts. The import call for it just grabs it out of the module
> > cache and creates a reference to it in the current namespace.
>
> I understand that, but
>
> Explicit is better than implicit.
>
>  --> non-explicit import sys and __import__(modname) to access
>
>      sys.modules[modname]

Importing sys is explicit. You actually say "import sys" -- how much 
more explicit can you get?


> Simple is better than complex.
>
>  --> three lines for a common use case instead of one

Simple doesn't mean shorter. The three line idiom is simple, because 
each line is simple and they combine in a simple way.

import sys  # does the same thing import sys always does
__import__(modname)  # does the same thing __import__ always does
modobj = sys.modules[modname]  # just a normal lookup

> Readability counts.
>
>  --> why is sys imported in this WSGI app (that shouldn't generally
>      touch any of the common sys.argv, sys.stdout etc)?!
>      (500 lines later) aha, to access sys.modules

sys is imported for the same reason that any module is imported: because 
some functionality provided by the module is needed. "I don't want to 
import a module" is not, on its own, a good enough reason to add extra 
complexity to a function.

There's no readability loss by importing a module at the top of the 
program and not using that module until some "500 lines" later. That's 
normal for modules.


> There should be one-- and preferably only one --obvious way to do it.
>
> --> should I use the 3-line idiom? should I use the getattr() idiom?

"Although that way may not be obvious at first unless you're Dutch."

I would say the three line idiom is obvious. At least, it's obvious now 
that I've learnt that there is a sys.modules.

>     Ideally, __import__() should support the only obvious way.
>
> In short, the three-line version doesn't convey the intent in an
> obvious way and does more work than is necessary.

I disagree with both those claims.

However, having disagreed with your arguments, I will say this: I think 
a "tail" argument to __import__ would be more elegant:

z = __import__('x.y.z', tail=True)  # or submodule in your initial post

I think that it is unintuitive that __import__(x.y.z) returns module x 
instead of z. When I say "get me the book inside the box inside the 
wardrobe", I expect to get the book, not the entire wardrobe! So even 
though I disagree with all your criticisms of the three-line idiom, I 
think there is an elegance to your idea that the three-line idiom 
doesn't have. I'm +0.5 on the idea of an extra argument that says, "no, 
really, just give me the book, I don't need the wardrobe".

But I think if you want to convince others, you're going to need to come 
up with more positive reasons for preferring the extra argument rather 
than rely on dubious criticisms of the three-line idiom: the 
three-liner is short, simple, obvious, readable and explicit, but your 
solution would be even shorter, more readable, and more explicit. I 
think it loses on the obvious and possibly on simple. The advantage is 
you've already done the work rather than expecting somebody else to do 
it for you.

Good luck. 



-- 
Steven

From lists at cheimes.de  Fri Nov 28 17:30:49 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 28 Nov 2008 17:30:49 +0100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <200811290317.38431.steve@pearwood.info>
References: <492EB167.4010107@mrts.pri.ee>
	<492FDAA3.8030706@gmail.com>	<492FEB3B.2040608@mrts.pri.ee>
	<200811290317.38431.steve@pearwood.info>
Message-ID: <49301CB9.1040700@cheimes.de>

Steven D'Aprano wrote:
> However, having disagreed with your arguments, I will say this: I think 
> a "tail" argument to __import__ would be more elegant:
> 
> z = __import__('x.y.z', tail=True)  # or submodule in your initial post
> 
> I think that it is unintuitive that __import__(x.y.z) returns module x 
> instead of z. When I say "get me the book inside the box inside the 
> wardrobe", I expect to get the book, not the entire wardrobe! So even 
> though I disagree with all your criticisms of the three-line idiom, I 
> think there is an elegance to your idea that the three-line idiom 
> doesn't have. I'm +0.5 on the idea of an extra argument that says, "no, 
> really, just give me the book, I don't need the wardrobe".

May I point you to the two leading underscores? The name '__import__' 
clearly suggests that the function is part of Python's internals. By 
definition all attributes of the form __*__ are not meant to be used 
directly. Any suggestion to change the arguments of __import__() are 
futile. It's not going to happen unless the feature is required by 
Python's internal import system.

However I agree that something should be done here. I suggest to add a 
function to the imp module that does what you are trying to archive with 
tail=True.

Christian


From torne-pythondev at wolfpuppy.org.uk  Fri Nov 28 17:48:07 2008
From: torne-pythondev at wolfpuppy.org.uk (Torne Wuff)
Date: Fri, 28 Nov 2008 16:48:07 +0000
Subject: [Python-Dev] Python 2.6/3.0,
	IEEE 754 floating point semantics	and S60
In-Reply-To: <5c6f2a5d0811271127y285b234bw4420a8ef9c94c0e4@mail.gmail.com>
References: <492EB5A3.1000608@cheimes.de>
	<20081127183139.GA2027@wolfpuppy.org.uk>
	<5c6f2a5d0811271127y285b234bw4420a8ef9c94c0e4@mail.gmail.com>
Message-ID: <20081128164807.GA11345@wolfpuppy.org.uk>

On Thu, Nov 27 08 at  7:27:03PM +0000, Mark Dickinson wrote:
> Many thanks for your input!

No problem! I don't like hearing folks from Nokia implying that our code
is rubbish :)

> I think IEEE 754 is relevant. :-)

Thanks for hunting down all that information! I'm in agreement: we
should fix this so that it works for 17 digits. I've had a chat with our
team's architect and it's not as awkward as I had assumed: the hardware
floating point doesn't actually get in the way as the function in
question only uses our software-implemented high precision real class,
not float/double types.

Whether our implementation is actually *correct* if you ask it for more
digits is an interesting question, though. It might be already :)

Are you aware of any compliance suite, test vectors, etc we could
'borrow' to verify our implementation? We would certainly like to verify
this. I'm going to raise a defect for the binary<->decimal conversion
issue, and see what we can do about sorting this out for future OS
versions.

-- 
Torne Wuff
torne at wolfpuppy.org.uk

From status at bugs.python.org  Fri Nov 28 18:06:44 2008
From: status at bugs.python.org (Python tracker)
Date: Fri, 28 Nov 2008 18:06:44 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20081128170644.3191F78346@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (11/21/08 - 11/28/08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 2217 open (+42) / 14059 closed (+33) / 16276 total (+75)

Open issues with patches:   735

Average duration of open issues: 710 days.
Median duration of open issues: 2108 days.

Open Issues Breakdown
   open  2199 (+42)
pending    18 ( +0)

Issues Created Or Reopened (77)
_______________________________

Thread local storage and PyGILState_* mucked up by os.fork()     11/28/08
       http://bugs.python.org/issue1683    reopened christian.heimes          
                                                                               

Py_SAFE_DOWNCAST in FILE_TIME_to_time_t_nsec failing             11/21/08
       http://bugs.python.org/issue4379    created  kevinwatters              
       patch                                                                   

Deepcopy of functools.partial gives wierd exception              11/21/08
       http://bugs.python.org/issue4380    created  kfitch                    
                                                                               

Cannot declare multiple classes via exec when inside a function. 11/21/08
CLOSED http://bugs.python.org/issue4381    created  kfitch                    
                                                                               

test_dbm_dumb fails due to character encoding issue on Mac OS X  11/21/08
CLOSED http://bugs.python.org/issue4382    created  oefe                      
                                                                               

UnboundLocalError when IDLE cannot connect to its subprocess     11/21/08
CLOSED http://bugs.python.org/issue4383    created  amaury.forgeotdarc        
       patch, needs review                                                     

Add a warnings.showwarning replacement for logging               11/21/08
       http://bugs.python.org/issue4384    created  brett.cannon              
       easy                                                                    

Py_Object_HEAD_INIT in Py3k                                      11/21/08
       http://bugs.python.org/issue4385    created  ncoghlan                  
                                                                               

A binary file should show "b" in its mode                        11/22/08
CLOSED http://bugs.python.org/issue4386    created  amaury.forgeotdarc        
       patch, needs review                                                     

binascii b2a functions accept strings (unicode) as data          11/22/08
       http://bugs.python.org/issue4387    created  tjreedy                   
                                                                               

test_cmd_line fails on MacOS X                                   11/22/08
       http://bugs.python.org/issue4388    created  MrJean1                   
                                                                               

Uninstaller Lacks an Icon                                        11/22/08
       http://bugs.python.org/issue4389    created  Retro                     
                                                                               

Using subprocess.STDOUT causes AttributeError                    11/22/08
CLOSED http://bugs.python.org/issue4390    created  nelchael                  
                                                                               

optparse: use proper gettext plurals forms                       11/22/08
       http://bugs.python.org/issue4391    created  dwayne                    
       patch                                                                   

incorrect parameter name for collections.namedtuple              11/23/08
CLOSED http://bugs.python.org/issue4392    created  none_00                   
                                                                               

Portability fixes in longobject.c                                11/23/08
       http://bugs.python.org/issue4393    created  marketdickinson           
       patch                                                                   

make the storage of the password optional in .pypirc (using the  11/23/08
       http://bugs.python.org/issue4394    created  tarek                     
       patch                                                                   

Document auto __ne__ generation; provide a use case for non-triv 11/23/08
       http://bugs.python.org/issue4395    created  tjreedy                   
                                                                               

parser module fails to validate "with" statements.               11/23/08
CLOSED http://bugs.python.org/issue4396    created  dbinger                   
       patch                                                                   

test_socket fails occassionaly in teardown: AssertionError: [Err 11/23/08
       http://bugs.python.org/issue4397    created  oefe                      
                                                                               

erewer                                                           11/23/08
CLOSED http://bugs.python.org/issue4398    created  der74hva3                 
                                                                               

"shard" appears where "shared" intended                          11/23/08
CLOSED http://bugs.python.org/issue4399    created  LambertDW                 
                                                                               

pypirc default is not at the right format                        11/24/08
       http://bugs.python.org/issue4400    created  tarek                     
       patch                                                                   

os.extsep status? doc or os bug?                                 11/24/08
       http://bugs.python.org/issue4401    created  tjreedy                   
                                                                               

os.getenv('PATH')   return different result between 2.5 and 3.0r 11/24/08
       http://bugs.python.org/issue4402    created  clive                     
                                                                               

regression from 2.6: smtplib.py requiring ascii for sending mess 11/24/08
       http://bugs.python.org/issue4403    created  ccgus                     
                                                                               

os.listdir() documentation error                                 11/24/08
CLOSED http://bugs.python.org/issue4404    created  steve21                   
                                                                               

fix_metaclass broken                                             11/24/08
CLOSED http://bugs.python.org/issue4405    created  loewis                    
                                                                               

In Lib\tkinter\filedialog.py, class Directory define loss a"_"   11/24/08
       http://bugs.python.org/issue4406    created  lion.guo                  
       patch                                                                   

Windows Installer Error 1722 when opting for compilation at inst 11/24/08
       http://bugs.python.org/issue4407    created  keldonin                  
       patch                                                                   

re.compile(regexp).groups not documented                         11/24/08
       http://bugs.python.org/issue4408    created  fw                        
                                                                               

Dangling asterisks in Python 3.0 subprocess docs                 11/24/08
       http://bugs.python.org/issue4409    created  skip.montanaro            
                                                                               

IDLE string problem in Run/Run Module                            11/24/08
       http://bugs.python.org/issue4410    created  andre                     
                                                                               

__mro__ documentation                                            11/24/08
       http://bugs.python.org/issue4411    created  lunar                     
                                                                               

Failure to test return value of PyUnicode_AsUTF8String() for NUL 11/25/08
CLOSED http://bugs.python.org/issue4412    created  CWRU_Researcher1          
                                                                               

Failure to check PyUnicode_AsUTF8String() return value for NULL  11/25/08
CLOSED http://bugs.python.org/issue4413    created  CWRU_Researcher1          
                                                                               

Failure to check return value of PyBool_FromLong for NULL        11/25/08
CLOSED http://bugs.python.org/issue4414    created  CWRU_Researcher1          
                                                                               

Failure to check return value of PyBool_FromLong for NULL        11/25/08
CLOSED http://bugs.python.org/issue4415    created  CWRU_Researcher1          
                                                                               

state_reset not called on 'state' before sre_search invoked      11/25/08
       http://bugs.python.org/issue4416    created  CWRU_Researcher1          
                                                                               

PySequence_List is not checked for NULL return value             11/25/08
CLOSED http://bugs.python.org/issue4417    created  CWRU_Researcher1          
                                                                               

PyObject_CallObject is not checked for NULL return value         11/25/08
CLOSED http://bugs.python.org/issue4418    created  CWRU_Researcher1          
                                                                               

PyUnicode_DecodeUTF8 is not checked for NULL return value        11/25/08
CLOSED http://bugs.python.org/issue4419    created  CWRU_Researcher1          
                                                                               

PyUnicode_Decode is not checked for NULL return value            11/25/08
CLOSED http://bugs.python.org/issue4420    created  CWRU_Researcher1          
                                                                               

Failed to check 3rd Param of PyArena_AddPyObject() to ensure it  11/25/08
CLOSED http://bugs.python.org/issue4421    created  CWRU_Researcher1          
                                                                               

Failed to check 3rd Param of PyArena_AddPyObject() to ensure it  11/25/08
CLOSED http://bugs.python.org/issue4422    created  CWRU_Researcher1          
                                                                               

2to3 replaces "arbitrary" variables                              11/25/08
CLOSED http://bugs.python.org/issue4423    created  loewis                    
                                                                               

Add support for a cmp, or key argument to heapq functions.       11/25/08
CLOSED http://bugs.python.org/issue4424    created  alex                      
                                                                               

UTF7 encoding of slash (character 47) is incorrect               11/25/08
       http://bugs.python.org/issue4425    created  Nick Barnes               
                                                                               

UTF7 decoding is far too strict                                  11/25/08
       http://bugs.python.org/issue4426    created  Nick Barnes               
                                                                               

Docs for 'y' Py_BuildValue tag are wrong                         11/25/08
       http://bugs.python.org/issue4427    created  theller                   
       patch, patch                                                            

io.BufferedWriter does not observe buffer size limits            11/25/08
       http://bugs.python.org/issue4428    created  beazley                   
                                                                               

ctypes.FormatError raises when the string to return contains non 11/25/08
CLOSED http://bugs.python.org/issue4429    created  theller                   
       patch                                                                   

time.strptime does not allow same format directive twice         11/25/08
       http://bugs.python.org/issue4430    created  sil                       
                                                                               

Distutils MSVC doesn't create manifest file (with fix)           11/25/08
CLOSED http://bugs.python.org/issue4431    created  dschnur                   
                                                                               

IDLE.app (Mac) File Menu MIssing Options                         11/26/08
       http://bugs.python.org/issue4432    created  LZ                        
                                                                               

_ctypes.COMError crash                                           11/26/08
CLOSED http://bugs.python.org/issue4433    created  theller                   
       patch, patch                                                            

Embedding into a shared library fails                            11/26/08
       http://bugs.python.org/issue4434    created  rb                        
                                                                               

Sphinx does not show failed doctests in quiet mode               11/26/08
CLOSED http://bugs.python.org/issue4435    created  schuppenies               
                                                                               

Sphinx latex writer crashes when encountering deep section level 11/26/08
CLOSED http://bugs.python.org/issue4436    created  schuppenies               
                                                                               

Windows 32-bit installer crash on "cancel" in "advanced" setting 11/26/08
CLOSED http://bugs.python.org/issue4437    created  fgrieu                    
                                                                               

Add an easy way to __import___ submodules                        11/26/08
       http://bugs.python.org/issue4438    created  mrts                      
       patch                                                                   

Typo in sqlite3 documentation.                                   11/26/08
CLOSED http://bugs.python.org/issue4439    created  olt                       
                                                                               

"sort" command doesn't work in pstats if run interactively       11/26/08
       http://bugs.python.org/issue4440    created  marcin.bachry             
       patch                                                                   

Improve os open flag options doc                                 11/26/08
       http://bugs.python.org/issue4441    created  tjreedy                   
                                                                               

datetime not subclassable in the usual way                       11/26/08
       http://bugs.python.org/issue4442    created  cjw296                    
                                                                               

Python/pystate.c:561 :PyGILState: The assertion 'autoInterpreter 11/26/08
CLOSED http://bugs.python.org/issue4443    created  lcarrionr                 
                                                                               

unittest - use contexts to assert exceptions                     11/27/08
       http://bugs.python.org/issue4444    created  dleonard0                 
       patch                                                                   

String allocations waste 3 bytes of memory on average.           11/27/08
       http://bugs.python.org/issue4445    created  marketdickinson           
       patch                                                                   

Distutils Metadata Documentation Missing "platforms" Keyword     11/27/08
       http://bugs.python.org/issue4446    created  hodgestar                 
       patch                                                                   

exec inside a function                                           11/27/08
CLOSED http://bugs.python.org/issue4447    created  beazley                   
                                                                               

should socket readline() use default_bufsize instead of _rbufsiz 11/27/08
       http://bugs.python.org/issue4448    created  gregory.p.smith           
                                                                               

AssertionError in Doc/includes/mp_benchmarks.py                  11/28/08
       http://bugs.python.org/issue4449    created  christian.heimes          
                                                                               

2to3 run changed multiprocessing.Queue() to multiprocessing.queu 11/28/08
       http://bugs.python.org/issue4450    created  christian.heimes          
                                                                               

multiprocessing fails with "Invalid thread state for this thread 11/28/08
CLOSED http://bugs.python.org/issue4451    created  christian.heimes          
                                                                               

Incorrect docstring of os.setpgrp                                11/28/08
       http://bugs.python.org/issue4452    created  eggy                      
                                                                               

MSI installer shows error message if "Compile .py files to bytec 11/28/08
       http://bugs.python.org/issue4453    created  suraj                     
                                                                               

urllib2's urlopen() method causes a memory leak                  11/24/08
       http://bugs.python.org/issue1208304 reopened amaury.forgeotdarc        
                                                                               



Issues Now Closed (56)
______________________

__import__ with fromlist=[''] causes double initialization of mo  288 days
       http://bugs.python.org/issue2090    mrts                      
                                                                               

Merge audio modules                                               249 days
       http://bugs.python.org/issue2433    brett.cannon              
                                                                               

io.open() handles errors differently on different platforms       245 days
       http://bugs.python.org/issue2491    amaury.forgeotdarc        
                                                                               

DISTUTILS_USE_SDK set causes msvc9compiler.py to raise an except   89 days
       http://bugs.python.org/issue3741    christian.heimes          
       patch                                                                   

Byte/string inconsistencies between different dbm modules           5 days
       http://bugs.python.org/issue3799    brett.cannon              
       patch, needs review                                                     

import fixer misses some symbols                                   58 days
       http://bugs.python.org/issue3994    benjamin.peterson         
       patch                                                                   

PyOS_CheckStack does not work                                      54 days
       http://bugs.python.org/issue3996    amaury.forgeotdarc        
       patch                                                                   

os.getenv silently discards env variables with non-UTF-8 values    54 days
       http://bugs.python.org/issue4006    a.badger                  
                                                                               

Unicode HOWTO up to date?                                          33 days
       http://bugs.python.org/issue4153    georg.brandl              
                                                                               

multiprocessing docs                                               27 days
       http://bugs.python.org/issue4206    georg.brandl              
                                                                               

some missing links in html help index pane                         20 days
       http://bugs.python.org/issue4252    georg.brandl              
       patch                                                                   

base64 does not properly handle unicode strings                    11 days
       http://bugs.python.org/issue4329    mbecker                   
                                                                               

Iteration over a map object with list()                             4 days
       http://bugs.python.org/issue4337    tjreedy                   
                                                                               

Add "key" argument to "bisect" module functions                     2 days
       http://bugs.python.org/issue4356    tjreedy                   
                                                                               

Docstring for "Lib/string.py" is outdated                           2 days
       http://bugs.python.org/issue4361    georg.brandl              
       patch                                                                   

Make uuid module functions usable without ctypes                    1 days
       http://bugs.python.org/issue4363    amaury.forgeotdarc        
       patch                                                                   

error in multiprocessing docs - rawvalue                            2 days
       http://bugs.python.org/issue4364    georg.brandl              
                                                                               

segmentation fault in ast_for_atom                                  1 days
       http://bugs.python.org/issue4367    benjamin.peterson         
       patch                                                                   

coerce gone---but not from docs                                     1 days
       http://bugs.python.org/issue4371    benjamin.peterson         
                                                                               

Reference leaks in Python 3.0rc3                                    5 days
       http://bugs.python.org/issue4373    christian.heimes          
       patch                                                                   

howto doc update                                                    1 days
       http://bugs.python.org/issue4378    georg.brandl              
                                                                               

Cannot declare multiple classes via exec when inside a function.    0 days
       http://bugs.python.org/issue4381    amaury.forgeotdarc        
                                                                               

test_dbm_dumb fails due to character encoding issue on Mac OS X     3 days
       http://bugs.python.org/issue4382    brett.cannon              
                                                                               

UnboundLocalError when IDLE cannot connect to its subprocess        0 days
       http://bugs.python.org/issue4383    amaury.forgeotdarc        
       patch, needs review                                                     

A binary file should show "b" in its mode                           0 days
       http://bugs.python.org/issue4386    benjamin.peterson         
       patch, needs review                                                     

Using subprocess.STDOUT causes AttributeError                       0 days
       http://bugs.python.org/issue4390    amaury.forgeotdarc        
                                                                               

incorrect parameter name for collections.namedtuple                 1 days
       http://bugs.python.org/issue4392    georg.brandl              
                                                                               

parser module fails to validate "with" statements.                  1 days
       http://bugs.python.org/issue4396    benjamin.peterson         
       patch                                                                   

erewer                                                              0 days
       http://bugs.python.org/issue4398    loewis                    
                                                                               

"shard" appears where "shared" intended                             0 days
       http://bugs.python.org/issue4399    georg.brandl              
                                                                               

os.listdir() documentation error                                    1 days
       http://bugs.python.org/issue4404    georg.brandl              
                                                                               

fix_metaclass broken                                                1 days
       http://bugs.python.org/issue4405    benjamin.peterson         
                                                                               

Failure to test return value of PyUnicode_AsUTF8String() for NUL    0 days
       http://bugs.python.org/issue4412    benjamin.peterson         
                                                                               

Failure to check PyUnicode_AsUTF8String() return value for NULL     0 days
       http://bugs.python.org/issue4413    benjamin.peterson         
                                                                               

Failure to check return value of PyBool_FromLong for NULL           0 days
       http://bugs.python.org/issue4414    benjamin.peterson         
                                                                               

Failure to check return value of PyBool_FromLong for NULL           0 days
       http://bugs.python.org/issue4415    benjamin.peterson         
                                                                               

PySequence_List is not checked for NULL return value                0 days
       http://bugs.python.org/issue4417    benjamin.peterson         
                                                                               

PyObject_CallObject is not checked for NULL return value            0 days
       http://bugs.python.org/issue4418    benjamin.peterson         
                                                                               

PyUnicode_DecodeUTF8 is not checked for NULL return value           0 days
       http://bugs.python.org/issue4419    benjamin.peterson         
                                                                               

PyUnicode_Decode is not checked for NULL return value               0 days
       http://bugs.python.org/issue4420    benjamin.peterson         
                                                                               

Failed to check 3rd Param of PyArena_AddPyObject() to ensure it     1 days
       http://bugs.python.org/issue4421    benjamin.peterson         
                                                                               

Failed to check 3rd Param of PyArena_AddPyObject() to ensure it     1 days
       http://bugs.python.org/issue4422    benjamin.peterson         
                                                                               

2to3 replaces "arbitrary" variables                                 2 days
       http://bugs.python.org/issue4423    benjamin.peterson         
                                                                               

Add support for a cmp, or key argument to heapq functions.          0 days
       http://bugs.python.org/issue4424    alex                      
                                                                               

ctypes.FormatError raises when the string to return contains non    1 days
       http://bugs.python.org/issue4429    theller                   
       patch                                                                   

Distutils MSVC doesn't create manifest file (with fix)              0 days
       http://bugs.python.org/issue4431    loewis                    
                                                                               

_ctypes.COMError crash                                              0 days
       http://bugs.python.org/issue4433    theller                   
       patch, patch                                                            

Sphinx does not show failed doctests in quiet mode                  1 days
       http://bugs.python.org/issue4435    georg.brandl              
                                                                               

Sphinx latex writer crashes when encountering deep section level    1 days
       http://bugs.python.org/issue4436    georg.brandl              
                                                                               

Windows 32-bit installer crash on "cancel" in "advanced" setting    0 days
       http://bugs.python.org/issue4437    amaury.forgeotdarc        
                                                                               

Typo in sqlite3 documentation.                                      0 days
       http://bugs.python.org/issue4439    benjamin.peterson         
                                                                               

Python/pystate.c:561 :PyGILState: The assertion 'autoInterpreter    1 days
       http://bugs.python.org/issue4443    amaury.forgeotdarc        
                                                                               

exec inside a function                                              0 days
       http://bugs.python.org/issue4447    beazley                   
                                                                               

multiprocessing fails with "Invalid thread state for this thread    0 days
       http://bugs.python.org/issue4451    jnoller                   
                                                                               

about shift key                                                  1306 days
       http://bugs.python.org/issue1191726 gpolo                     
                                                                               

Problem with socket.gethostbyaddr() and KeyboardInterrupt         499 days
       http://bugs.python.org/issue1755388 gagenellina               
                                                                               



Top Issues Most Discussed (10)
______________________________

 10 2to3 replaces "arbitrary" variables                                2 days
closed  http://bugs.python.org/issue4423   

 10 os.getenv silently discards env variables with non-UTF-8 values   54 days
closed  http://bugs.python.org/issue4006   

 10 Byte/string inconsistencies between different dbm modules          5 days
closed  http://bugs.python.org/issue3799   

 10 datetime needs and "epoch" method                                211 days
open    http://bugs.python.org/issue2736   

  9 Embedding into a shared library fails                              2 days
open    http://bugs.python.org/issue4434   

  8 test_cmd_line fails on MacOS X                                     7 days
open    http://bugs.python.org/issue4388   

  8 Fix performance issues in xmlrpclib                               11 days
pending http://bugs.python.org/issue4336   

  7 UTF7 decoding is far too strict                                    3 days
open    http://bugs.python.org/issue4426   

  7 os.getenv('PATH')   return different result between 2.5 and 3.0    5 days
open    http://bugs.python.org/issue4402   

  7 warning: unknown conversion type character `z' in format           8 days
open    http://bugs.python.org/issue4370   




From dickinsm at gmail.com  Fri Nov 28 18:47:25 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 28 Nov 2008 17:47:25 +0000
Subject: [Python-Dev] Python 2.6/3.0,
	IEEE 754 floating point semantics and S60
In-Reply-To: <20081128164807.GA11345@wolfpuppy.org.uk>
References: <492EB5A3.1000608@cheimes.de>
	<20081127183139.GA2027@wolfpuppy.org.uk>
	<5c6f2a5d0811271127y285b234bw4420a8ef9c94c0e4@mail.gmail.com>
	<20081128164807.GA11345@wolfpuppy.org.uk>
Message-ID: <5c6f2a5d0811280947y5b9345d3wa1d46657b39aa267@mail.gmail.com>

On Fri, Nov 28, 2008 at 4:48 PM, Torne Wuff
<torne-pythondev at wolfpuppy.org.uk> wrote:
> Thanks for hunting down all that information! I'm in agreement: we
> should fix this so that it works for 17 digits.

Cool.  I think this should make porting easier (and not just for
Python, either).

> Are you aware of any compliance suite, test vectors, etc we could
> 'borrow' to verify our implementation?

Not really. Some googling turns up a few links, notably this one:

http://www.cant.ua.ac.be/ieeecc754.html

It looks a little old, and possibly unmaintained, but even if you can't
get the C driver to compile then the test vectors might still be usable.

Python doesn't require that the platform is IEEE 754 compliant,
so there isn't much IEEE 754-specific stuff in the Python test suite.

Thank you again for your messages;  it's *very* useful to know that
(1) IEEE 754 compliance is intended, and (2) there are people at
Symbian who care about fixing non-compliance issues.  It makes the
currently fairly insane activity of trying to write cross-platform
floating-point C code that's going to work on any system that
Python might ever be run on just that little bit saner. :)

Mark

From steve at pearwood.info  Fri Nov 28 18:47:24 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 29 Nov 2008 04:47:24 +1100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <49301CB9.1040700@cheimes.de>
References: <492EB167.4010107@mrts.pri.ee>
	<200811290317.38431.steve@pearwood.info>
	<49301CB9.1040700@cheimes.de>
Message-ID: <200811290447.24871.steve@pearwood.info>

On Sat, 29 Nov 2008 03:30:49 am Christian Heimes wrote:
...
> May I point you to the two leading underscores? The name '__import__'
> clearly suggests that the function is part of Python's internals. By
> definition all attributes of the form __*__ are not meant to be used
> directly. 

What about __doc__, __name__ and __slots__, to mention just a few?

I believe that __import__() is the recommended way to import a module 
known only at runtime. The docs also don't seem to agree with your 
assertion that __import__ is not to be touched. On the contrary:

"The function is invoked by the import statement. It mainly exists so 
that you can replace it with another function that has a compatible 
interface..."
http://docs.python.org/library/functions.html

Far from saying "Do Not Touch", the docs say "Please, go ahead and 
replace __import__ if needed". If the OP wants "import x.y.z" to load 
module z instead of x, it seems to me that is exactly the problem 
__import__ is designed to allow him to do.


> Any suggestion to change the arguments of __import__() are 
> futile. It's not going to happen unless the feature is required by
> Python's internal import system.

That may very well be true, but I don't know why you're dumping on me. 
It wasn't my suggestion to change __import__. I merely said I could see 
some advantages to it. I spent most of my post telling the OP why I 
thought his arguments were wrong!


-- 
Steven

From dickinsm at gmail.com  Fri Nov 28 18:59:00 2008
From: dickinsm at gmail.com (Mark Dickinson)
Date: Fri, 28 Nov 2008 17:59:00 +0000
Subject: [Python-Dev] Python 2.6/3.0,
	IEEE 754 floating point semantics and S60
In-Reply-To: <20081128164807.GA11345@wolfpuppy.org.uk>
References: <492EB5A3.1000608@cheimes.de>
	<20081127183139.GA2027@wolfpuppy.org.uk>
	<5c6f2a5d0811271127y285b234bw4420a8ef9c94c0e4@mail.gmail.com>
	<20081128164807.GA11345@wolfpuppy.org.uk>
Message-ID: <5c6f2a5d0811280959k1abf34e8gd5f303ca68e39969@mail.gmail.com>

On Fri, Nov 28, 2008 at 4:48 PM, Torne Wuff
<torne-pythondev at wolfpuppy.org.uk> wrote:
> Are you aware of any compliance suite, test vectors, etc we could
> 'borrow' to verify our implementation?

See also the ucbtest package, available from http://www.netlib.org/fp/

From aleaxit at gmail.com  Fri Nov 28 19:20:01 2008
From: aleaxit at gmail.com (Alex Martelli)
Date: Fri, 28 Nov 2008 10:20:01 -0800
Subject: [Python-Dev] __import__ problems
In-Reply-To: <200811290447.24871.steve@pearwood.info>
References: <492EB167.4010107@mrts.pri.ee>
	<200811290317.38431.steve@pearwood.info> <49301CB9.1040700@cheimes.de>
	<200811290447.24871.steve@pearwood.info>
Message-ID: <e8a0972d0811281020u330400b0x4b307619312c22c@mail.gmail.com>

On Fri, Nov 28, 2008 at 9:47 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Sat, 29 Nov 2008 03:30:49 am Christian Heimes wrote:
> ...
>> May I point you to the two leading underscores? The name '__import__'
>> clearly suggests that the function is part of Python's internals. By
>> definition all attributes of the form __*__ are not meant to be used
>> directly.
>
> What about __doc__, __name__ and __slots__, to mention just a few?

Excellent examples of "not meant to be used directly".  __slots__,
like most other special names (though not all) is meant to be *bound*
directly when you need to (which for __slots__ is much more rarely
than for many other special names, mostly those of special methods
that often need to be overridden); but none of these are meant to be
*used* in normal cases -- introspection should normally be done via
provided tools such as the help built-in and standard library modules
such as pydoc and inspect.  It's pretty rare to have to write your own
introspection framework, and the dunder-names are internal machinery
exposed for those rare use cases.


> I believe that __import__() is the recommended way to import a module
> known only at runtime. The docs also don't seem to agree with your
> assertion that __import__ is not to be touched. On the contrary:
>
> "The function is invoked by the import statement. It mainly exists so
> that you can replace it with another function that has a compatible
> interface..."
> http://docs.python.org/library/functions.html

Yes, that documentation predates the current generation of import
hooks which almost remove the need for such trickery.


> Far from saying "Do Not Touch", the docs say "Please, go ahead and
> replace __import__ if needed". If the OP wants "import x.y.z" to load
> module z instead of x, it seems to me that is exactly the problem
> __import__ is designed to allow him to do.

"import x.y.z", the import statement, has well defined semantics
relying on __import__'s interface -- it does load z as well as,
inevitably, its parent y and grandparent x.  If __import__ was
replaced with a version with NON compatible interface, "import x.y.z"
would break.


>> Any suggestion to change the arguments of __import__() are
>> futile. It's not going to happen unless the feature is required by
>> Python's internal import system.
>
> That may very well be true, but I don't know why you're dumping on me.
> It wasn't my suggestion to change __import__. I merely said I could see
> some advantages to it. I spent most of my post telling the OP why I
> thought his arguments were wrong!

And now we're telling you why we think your arguments are also wrong
-- what else do you expect from python-dev after all?


Alex

From martin at v.loewis.de  Fri Nov 28 20:03:01 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 28 Nov 2008 20:03:01 +0100
Subject: [Python-Dev] Python under valgrind
In-Reply-To: <492FE636.50905@avl.com>
References: <492FE636.50905@avl.com>
Message-ID: <49304065.6060400@v.loewis.de>

> I suppose valgrind could be confused by PyFree's pool address validation
> that intentionally reads the memory just before the allocated block, and
> incorrectly attributes it to a previously allocated (and hence freed)
> block, but I can't prove that. 

I agree this is the likely cause. To prove it, you will need to find
the line number associated with 0x808EBDF. Either you available debug
information, or disassemble PyObject_Free to determine what specific
read operation is causing the report. Most likely, you find it is
inside Py_ADDRESS_IN_RANGE, in the attempt to read pool->arenaindex.

Regards,
Martin

From mrts at mrts.pri.ee  Fri Nov 28 20:07:17 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Fri, 28 Nov 2008 21:07:17 +0200
Subject: [Python-Dev] __import__ problems
In-Reply-To: <492FDAA3.8030706@gmail.com>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
	<492FC1AD.70106@mrts.pri.ee> <492FDAA3.8030706@gmail.com>
Message-ID: <49304165.50708@mrts.pri.ee>

 > If __import__ was replaced with a version with NON compatible interface,
 > "import x.y.z" would break.

But it is not. The proposed __import__(name, submodule=True) has
a compatible interface. All tests pass with
http://bugs.python.org/file12136/issue4438.diff .

As for the imp approach, I've alternatively implemented
imp.import_module() that imports tail modules in
http://bugs.python.org/file12147/imp_import_module.diff
(colour diff in http://dpaste.com/hold/94431/).

IMHO the functionality duplication with __import__ is ugly, but
if it is desired that __import__ does not change (even in
backwards-compatible way), so be it.

The signature and behaviour is as follows:

---

import_module(name, globals, level) -> module

Import the tail module, given dotted module hierarchy in 'name'.
'globals' only determines the package context and is not
modified. 'level' specifies whether to use absolute or relative
imports.

 >>> from imp import import_module
 >>> import_module('foo.bar.baz')
<module 'foo.bar.baz' from 'foo/bar/baz/__init__.py'>

---

But I'm still +1 for adding 'tail'/'submodule'/'tailmodule'
argument to __import__ instead of providing an almost identical
copy in imp.import_module().

Let me know which of the approaches is desired (if any) and I'll
add tests and update docs.

From brett at python.org  Fri Nov 28 21:17:08 2008
From: brett at python.org (Brett Cannon)
Date: Fri, 28 Nov 2008 12:17:08 -0800
Subject: [Python-Dev] __import__ problems
In-Reply-To: <49304165.50708@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
	<492FC1AD.70106@mrts.pri.ee> <492FDAA3.8030706@gmail.com>
	<49304165.50708@mrts.pri.ee>
Message-ID: <bbaeab100811281217p2c8386b6k3180dcd57afbc9ec@mail.gmail.com>

On Fri, Nov 28, 2008 at 11:07, Mart Somermaa <mrts at mrts.pri.ee> wrote:
>> If __import__ was replaced with a version with NON compatible interface,
>> "import x.y.z" would break.
>
> But it is not. The proposed __import__(name, submodule=True) has
> a compatible interface. All tests pass with
> http://bugs.python.org/file12136/issue4438.diff .
>

But as people have pointed  out, __import__ is not meant to be used
directly, and so adding keyword arguments just  to make it so it can
be used  directly is the wrong approach.

> As for the imp approach, I've alternatively implemented
> imp.import_module() that imports tail modules in
> http://bugs.python.org/file12147/imp_import_module.diff
> (colour diff in http://dpaste.com/hold/94431/).
>
> IMHO the functionality duplication with __import__ is ugly, but
> if it is desired that __import__ does not change (even in
> backwards-compatible way), so be it.
>
> The signature and behaviour is as follows:
>
> ---
>
> import_module(name, globals, level) -> module
>
> Import the tail module, given dotted module hierarchy in 'name'.
> 'globals' only determines the package context and is not
> modified. 'level' specifies whether to use absolute or relative
> imports.
>
>>>> from imp import import_module
>>>> import_module('foo.bar.baz')
> <module 'foo.bar.baz' from 'foo/bar/baz/__init__.py'>
>
> ---
>
> But I'm still +1 for adding 'tail'/'submodule'/'tailmodule'
> argument to __import__ instead of providing an almost identical
> copy in imp.import_module().
>

But  you are assuming you need to keep those arguments like global and
level, and you really don't if you have the API handle the low-level
details as needed. This could really turn into:

import_module(name, package)

and that gives the machinery everything it needs to do an import,
including relative imports. Then all you need to do is:

import_module('.baz', __package__)

and everything works.

> Let me know which of the approaches is desired (if any) and I'll
> add tests and update docs.

The old-hands on python-dev know this is where  I plug  my import
rewrite vaporware. It will be in 3.1, and as part of it there will be
a new API for handling direct imports. Jacob Kaplan-Moss and I have
talked about Django's need for this as PyCon so I am very aware of
needing this API and it will be addressed in the simplest way possible
(heck, the __import__ API might actually become a wrapper around the
simpler API in the end). And since nothing can go into 3.0 anyway,
there is no need to rush into solving this right now.

-Brett

From lists at cheimes.de  Fri Nov 28 21:45:41 2008
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 28 Nov 2008 21:45:41 +0100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <49304165.50708@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee>
	<492F0FA2.10905@gmail.com>	<492FC1AD.70106@mrts.pri.ee>
	<492FDAA3.8030706@gmail.com> <49304165.50708@mrts.pri.ee>
Message-ID: <49305875.8060106@cheimes.de>

Mart Somermaa wrote:
> But it is not. The proposed __import__(name, submodule=True) has
> a compatible interface. All tests pass with
> http://bugs.python.org/file12136/issue4438.diff .

Your approach complicates the interface. Every implementation of Python 
and every replacement __import__ would have to implement the additional 
argument, too. There is simply no need to make the interface more 
complex when an additional, trivial function does the same job.
> 
> As for the imp approach, I've alternatively implemented
> imp.import_module() that imports tail modules in
> http://bugs.python.org/file12147/imp_import_module.diff
> (colour diff in http://dpaste.com/hold/94431/).
> 
> IMHO the functionality duplication with __import__ is ugly, but
> if it is desired that __import__ does not change (even in
> backwards-compatible way), so be it.

It's not a duplication of functionality. __import__() implements exactly 
the functionality that is required by Python's import system. Anything 
else can be implemented on top of it. We simply prefer divide 'n conquer 
over mingle 'n mix. :)

Christian

From ncoghlan at gmail.com  Fri Nov 28 23:25:26 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 29 Nov 2008 08:25:26 +1000
Subject: [Python-Dev] __import__ problems
In-Reply-To: <49304165.50708@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee>
	<492F0FA2.10905@gmail.com>	<492FC1AD.70106@mrts.pri.ee>
	<492FDAA3.8030706@gmail.com> <49304165.50708@mrts.pri.ee>
Message-ID: <49306FD6.9030505@gmail.com>

Mart Somermaa wrote:
> But I'm still +1 for adding 'tail'/'submodule'/'tailmodule'
> argument to __import__ instead of providing an almost identical
> copy in imp.import_module().
> 
> Let me know which of the approaches is desired (if any) and I'll
> add tests and update docs.

Have you considered using runpy.run_module() instead? i.e. do you
actually need the resulting module object, or just the global namespace
produced by running its code?

If the latter, then run_module() is a much better way of going about it
as it has (slightly) fewer side effects on the import system's internals.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steve at pearwood.info  Sat Nov 29 02:18:01 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 29 Nov 2008 12:18:01 +1100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <e8a0972d0811281020u330400b0x4b307619312c22c@mail.gmail.com>
References: <492EB167.4010107@mrts.pri.ee>
	<200811290447.24871.steve@pearwood.info>
	<e8a0972d0811281020u330400b0x4b307619312c22c@mail.gmail.com>
Message-ID: <200811291218.01658.steve@pearwood.info>

On Sat, 29 Nov 2008 05:20:01 am Alex Martelli wrote:
> On Fri, Nov 28, 2008 at 9:47 AM, Steven D'Aprano <steve at pearwood.info> 
wrote:
> > On Sat, 29 Nov 2008 03:30:49 am Christian Heimes wrote:
> > ...
> >
> >> May I point you to the two leading underscores? The name
> >> '__import__' clearly suggests that the function is part of
> >> Python's internals. By definition all attributes of the form __*__
> >> are not meant to be used directly.
> >
> > What about __doc__, __name__ and __slots__, to mention just a few?
>
> Excellent examples of "not meant to be used directly".
[snip]

We obviously have different ideas about what that means, because I agree 
with everything you say but disagree with the conclusion. The idiom "if 
__name__ == '__main__'" is very common; and I frequently have scripts 
that "print __doc__" if given a --help command line option. And as you 
say, __slots__ is meant to be bound directly.


> > I believe that __import__() is the recommended way to import a
> > module known only at runtime. The docs also don't seem to agree 
> > with your assertion that __import__ is not to be touched. On the
> > contrary:
> >
> > "The function is invoked by the import statement. It mainly exists
> > so that you can replace it with another function that has a
> > compatible interface..."
> > http://docs.python.org/library/functions.html
>
> Yes, that documentation predates the current generation of import
> hooks which almost remove the need for such trickery.

Should this be reported as a documentation bug? Given the new import 
hooks, would it be fair to say that the main reason for __import__ is 
to use it to import a module whose name is only known at runtime?



-- 
Steven

From ncoghlan at gmail.com  Sat Nov 29 03:56:00 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 29 Nov 2008 12:56:00 +1000
Subject: [Python-Dev] __import__ problems
In-Reply-To: <200811291218.01658.steve@pearwood.info>
References: <492EB167.4010107@mrts.pri.ee>	<200811290447.24871.steve@pearwood.info>	<e8a0972d0811281020u330400b0x4b307619312c22c@mail.gmail.com>
	<200811291218.01658.steve@pearwood.info>
Message-ID: <4930AF40.6000504@gmail.com>

Steven D'Aprano wrote:
> Should this be reported as a documentation bug? Given the new import 
> hooks, would it be fair to say that the main reason for __import__ is 
> to use it to import a module whose name is only known at runtime?

Only known at runtime, and for some reason you want an actual module
object rather than just the module's global namespace (since you can use
runpy.run_module() if you only need the latter).

At the very least, the __import__ docs should probably be updated to
point to run_module() as an alternative approach, so a doc issue is
probably a good idea.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From guido at python.org  Sat Nov 29 04:15:18 2008
From: guido at python.org (Guido van Rossum)
Date: Fri, 28 Nov 2008 19:15:18 -0800
Subject: [Python-Dev] __import__ problems
In-Reply-To: <4930AF40.6000504@gmail.com>
References: <492EB167.4010107@mrts.pri.ee>
	<200811290447.24871.steve@pearwood.info>
	<e8a0972d0811281020u330400b0x4b307619312c22c@mail.gmail.com>
	<200811291218.01658.steve@pearwood.info> <4930AF40.6000504@gmail.com>
Message-ID: <ca471dc20811281915pf4505e1q4bec8e540b28ec8c@mail.gmail.com>

On Fri, Nov 28, 2008 at 6:56 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Steven D'Aprano wrote:
>> Should this be reported as a documentation bug? Given the new import
>> hooks, would it be fair to say that the main reason for __import__ is
>> to use it to import a module whose name is only known at runtime?
>
> Only known at runtime, and for some reason you want an actual module
> object rather than just the module's global namespace (since you can use
> runpy.run_module() if you only need the latter).
>
> At the very least, the __import__ docs should probably be updated to
> point to run_module() as an alternative approach, so a doc issue is
> probably a good idea.

This sounds wrong to me. run_module() runs the module each time it is
called. __import__ has all the semantics of the import statement (by
definition -- it is almost a tautology :-) in that it first tries to
see if the module is already imported. Pointing to run_module() as an
alternative just perpetuates the misunderstanding (alas fairly common
amongst casual users) about what exactly import does.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Sat Nov 29 05:17:06 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 29 Nov 2008 14:17:06 +1000
Subject: [Python-Dev] __import__ problems
In-Reply-To: <ca471dc20811281915pf4505e1q4bec8e540b28ec8c@mail.gmail.com>
References: <492EB167.4010107@mrts.pri.ee>	
	<200811290447.24871.steve@pearwood.info>	
	<e8a0972d0811281020u330400b0x4b307619312c22c@mail.gmail.com>	
	<200811291218.01658.steve@pearwood.info>
	<4930AF40.6000504@gmail.com>
	<ca471dc20811281915pf4505e1q4bec8e540b28ec8c@mail.gmail.com>
Message-ID: <4930C242.9010704@gmail.com>

Guido van Rossum wrote:
> On Fri, Nov 28, 2008 at 6:56 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Steven D'Aprano wrote:
>>> Should this be reported as a documentation bug? Given the new import
>>> hooks, would it be fair to say that the main reason for __import__ is
>>> to use it to import a module whose name is only known at runtime?
>> Only known at runtime, and for some reason you want an actual module
>> object rather than just the module's global namespace (since you can use
>> runpy.run_module() if you only need the latter).
>>
>> At the very least, the __import__ docs should probably be updated to
>> point to run_module() as an alternative approach, so a doc issue is
>> probably a good idea.
> 
> This sounds wrong to me. run_module() runs the module each time it is
> called. __import__ has all the semantics of the import statement (by
> definition -- it is almost a tautology :-) in that it first tries to
> see if the module is already imported. Pointing to run_module() as an
> alternative just perpetuates the misunderstanding (alas fairly common
> amongst casual users) about what exactly import does.

Ah, good point. I guess it depends on the specific use case...

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From steve at pearwood.info  Sat Nov 29 05:49:40 2008
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 29 Nov 2008 15:49:40 +1100
Subject: [Python-Dev] __import__ problems
In-Reply-To: <4930AF40.6000504@gmail.com>
References: <492EB167.4010107@mrts.pri.ee>
	<200811291218.01658.steve@pearwood.info>
	<4930AF40.6000504@gmail.com>
Message-ID: <200811291549.40459.steve@pearwood.info>

On Sat, 29 Nov 2008 01:56:00 pm Nick Coghlan wrote:
> Steven D'Aprano wrote:
> > Should this be reported as a documentation bug? Given the new
> > import hooks, would it be fair to say that the main reason for
> > __import__ is to use it to import a module whose name is only known
> > at runtime?
>
> Only known at runtime, and for some reason you want an actual module
> object rather than just the module's global namespace (since you can
> use runpy.run_module() if you only need the latter).
>
> At the very least, the __import__ docs should probably be updated to
> point to run_module() as an alternative approach, so a doc issue is
> probably a good idea.

Bug report is here:
http://bugs.python.org/issue4457

Based on Guido's comments, I haven't mentioned run_module().


-- 
Steven

From greg.ewing at canterbury.ac.nz  Sat Nov 29 07:57:22 2008
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 29 Nov 2008 19:57:22 +1300
Subject: [Python-Dev] __import__ problems
In-Reply-To: <49304165.50708@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
	<492FC1AD.70106@mrts.pri.ee> <492FDAA3.8030706@gmail.com>
	<49304165.50708@mrts.pri.ee>
Message-ID: <4930E7D2.8000208@canterbury.ac.nz>

Mart Somermaa wrote:

> But it is not. The proposed __import__(name, submodule=True) has
> a compatible interface.

Actually, it's not. Keep in mind that __import__ isn't
a particular function, it's a defined interface to a
family of functions. If that interface is extended, any
replacement __import__ functions will be required to
conform to the new interface, and existing ones will
no longer be valid.

-- 
Greg

From skip at pobox.com  Sun Nov 30 05:00:24 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 29 Nov 2008 22:00:24 -0600
Subject: [Python-Dev] What does this mean?
Message-ID: <18738.4056.266531.278814@montanaro-dyndns-org.local>

I just ran "svn up" in my 2.6 sandbox and got this warning about the
configure script:

    svn: File 'configure' has inconsistent newlines
    svn: Inconsistent line ending style

I don't see any newlines other than just LF.  Is some property on the file
hosed?

Skip


From skip at pobox.com  Sun Nov 30 05:02:49 2008
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 29 Nov 2008 22:02:49 -0600
Subject: [Python-Dev] What does this mean?
Message-ID: <18738.4201.239693.951635@montanaro-dyndns-org.local>

    I just ran "svn up" in my 2.6 sandbox and got this warning about the
    configure script:

        svn: File 'configure' has inconsistent newlines
        svn: Inconsistent line ending style

    I don't see any newlines other than just LF.  Is some property on the
    file hosed?

I should point out that it is 'M'odified in my sandbox, but I just ran
autoconf (v2.63) to recreate it from a modified configure.in.

Skip


From g.brandl at gmx.net  Sun Nov 30 12:06:36 2008
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 30 Nov 2008 12:06:36 +0100
Subject: [Python-Dev] What does this mean?
In-Reply-To: <18738.4201.239693.951635@montanaro-dyndns-org.local>
References: <18738.4201.239693.951635@montanaro-dyndns-org.local>
Message-ID: <ggts4m$5hk$1@ger.gmane.org>

skip at pobox.com schrieb:
>     I just ran "svn up" in my 2.6 sandbox and got this warning about the
>     configure script:
> 
>         svn: File 'configure' has inconsistent newlines
>         svn: Inconsistent line ending style
> 
>     I don't see any newlines other than just LF.  Is some property on the
>     file hosed?
> 
> I should point out that it is 'M'odified in my sandbox, but I just ran
> autoconf (v2.63) to recreate it from a modified configure.in.

Happens for me too. I investigated a bit and found that autoconf inserts a
literal \r, i.e. chr(13), into the generated configure, and that trips up
svn. It occurs at the definition of "ac_cr".

I've since always asked Benjamin regenerate configure, who seems to have
a setup where ac_cr isn't inserted into the generated file.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From martin at v.loewis.de  Sun Nov 30 12:22:13 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 30 Nov 2008 12:22:13 +0100
Subject: [Python-Dev] autoconf version (Was: What does this mean?)
In-Reply-To: <ggts4m$5hk$1@ger.gmane.org>
References: <18738.4201.239693.951635@montanaro-dyndns-org.local>
	<ggts4m$5hk$1@ger.gmane.org>
Message-ID: <49327765.7010201@v.loewis.de>

>> I should point out that it is 'M'odified in my sandbox, but I just ran
>> autoconf (v2.63) to recreate it from a modified configure.in.
> 
> Happens for me too. I investigated a bit and found that autoconf inserts a
> literal \r, i.e. chr(13), into the generated configure, and that trips up
> svn. It occurs at the definition of "ac_cr".
> 
> I've since always asked Benjamin regenerate configure, who seems to have
> a setup where ac_cr isn't inserted into the generated file.

For that very reason, we should continue to use autoconf 2.61. Not sure
why Benjamin put in the comment that 2.63 is being used. If you look at
the configure files that Benjamin has checked in, you'll notice that he
has been using 2.61.

Regards,
Martin

From mrts at mrts.pri.ee  Sun Nov 30 14:02:16 2008
From: mrts at mrts.pri.ee (Mart Somermaa)
Date: Sun, 30 Nov 2008 15:02:16 +0200
Subject: [Python-Dev] __import__ problems
In-Reply-To: <bbaeab100811281217p2c8386b6k3180dcd57afbc9ec@mail.gmail.com>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>	
	<492FC1AD.70106@mrts.pri.ee> <492FDAA3.8030706@gmail.com>	
	<49304165.50708@mrts.pri.ee>
	<bbaeab100811281217p2c8386b6k3180dcd57afbc9ec@mail.gmail.com>
Message-ID: <49328ED8.2060500@mrts.pri.ee>

Brett Cannon wrote:

 > The old-hands on python-dev know this is where  I plug  my import
 > rewrite vaporware. It will be in 3.1, and as part of it there will be
 > a new API for handling direct imports. Jacob Kaplan-Moss and I have

Sounds good. I can finally rest in peace :) . May I suggest
that you disable the hack while you are at it as well so that
people will be aware of their misdoings?

 > talked about Django's need for this as PyCon so I am very aware of
 > needing this API and it will be addressed in the simplest way possible
 > (heck, the __import__ API might actually become a wrapper around the
 > simpler API in the end).

Even better (the bracketed part).

 > And since nothing can go into 3.0 anyway,
 > there is no need to rush into solving this right now.

Agreed, I just wanted to get the ball rolling.

Let me know if you want me to do some gruntwork (tests,
documentation) once the improved implementation is taking shape.

---

As for the other comments, I'll try to wrap things up:

 * I did get the impression from some posts that it was assumed
   to be somehow "my problem" -- although indeed seeing both
   'foo' and 'foo.' when printing sys.modules in a popular web
   framework I frequently use makes me wince in discomfort, the
   hack is present in 2000 lines in various projects as seen in
   the Google Code Search.

 * runpy.run_module() is not the solution as it runs the module
   each time it is called and particularly because access to the
   submodule object is generally needed (again, look around in
   the Google Code Search results).

 * changing the signature of __import__ is out of question both
   because it would break the existing __import__ replacements
   and would perpetuate the wrong assumption that it should be
   directly used (instead of the presently missing simpler
   interface).

---

It looks that the

__import__(modname)
mod = sys.modules[modname]

idiom is the clear winner for the import submodule use case.
I've updated http://bugs.python.org/issue4457 with proposed
additions to current __import__ docs. Once the simpler interface
emerges, the docs should be updated again and __import__
use should be discouraged.

From musiccomposition at gmail.com  Sun Nov 30 15:54:53 2008
From: musiccomposition at gmail.com (Benjamin Peterson)
Date: Sun, 30 Nov 2008 08:54:53 -0600
Subject: [Python-Dev] autoconf version (Was: What does this mean?)
In-Reply-To: <49327765.7010201@v.loewis.de>
References: <18738.4201.239693.951635@montanaro-dyndns-org.local>
	<ggts4m$5hk$1@ger.gmane.org> <49327765.7010201@v.loewis.de>
Message-ID: <1afaf6160811300654x3bda0b19m231d96777600b2e4@mail.gmail.com>

On Sun, Nov 30, 2008 at 5:22 AM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>>> I should point out that it is 'M'odified in my sandbox, but I just ran
>>> autoconf (v2.63) to recreate it from a modified configure.in.
>>
>> Happens for me too. I investigated a bit and found that autoconf inserts a
>> literal \r, i.e. chr(13), into the generated configure, and that trips up
>> svn. It occurs at the definition of "ac_cr".
>>
>> I've since always asked Benjamin regenerate configure, who seems to have
>> a setup where ac_cr isn't inserted into the generated file.
>
> For that very reason, we should continue to use autoconf 2.61. Not sure
> why Benjamin put in the comment that 2.63 is being used. If you look at
> the configure files that Benjamin has checked in, you'll notice that he
> has been using 2.61.

I've been trying to find an autoconf that works with all my projects,
so when I wrote, that I may have been using 1.63. Anyway, I've updated
the comment to say 1.61.



-- 
Cheers,
Benjamin Peterson
"There's nothing quite as beautiful as an oboe... except a chicken
stuck in a vacuum cleaner."

From gruszczy at gmail.com  Sun Nov 30 19:41:55 2008
From: gruszczy at gmail.com (=?UTF-8?Q?Filip_Gruszczy=C5=84ski?=)
Date: Sun, 30 Nov 2008 19:41:55 +0100
Subject: [Python-Dev] Attribute error: providing type name
Message-ID: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>

This is my first message in this list, therefore I would like to say
hello to everyone. I also hope, that I am not breaking any rules or
guidelines for sending proposals.

I would like to ask, if it is possible to provide type name of the
object invoking the exception, when Attribute error is catched. It is
done for functions, like:

AttributeError: 'function' object has no attribute 'getValue'

but for some objects there is only:

AttributeError: connectToBases

This is cool, when you know exactly what type of object cast the
exception. But if there might be many of them, you must do one of two
things: add print statement just before the line with the exception
and check the type or iterate over all classes that might appear them.
Showing the class name would solve this problem and could save a lot
of time.

-- 
Filip Gruszczy?ski

From rhamph at gmail.com  Sun Nov 30 20:06:43 2008
From: rhamph at gmail.com (Adam Olsen)
Date: Sun, 30 Nov 2008 12:06:43 -0700
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
Message-ID: <aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>

On Sun, Nov 30, 2008 at 11:41 AM, Filip Gruszczy?ski <gruszczy at gmail.com> wrote:
> This is my first message in this list, therefore I would like to say
> hello to everyone. I also hope, that I am not breaking any rules or
> guidelines for sending proposals.
>
> I would like to ask, if it is possible to provide type name of the
> object invoking the exception, when Attribute error is catched. It is
> done for functions, like:
>
> AttributeError: 'function' object has no attribute 'getValue'
>
> but for some objects there is only:
>
> AttributeError: connectToBases
>
> This is cool, when you know exactly what type of object cast the
> exception. But if there might be many of them, you must do one of two
> things: add print statement just before the line with the exception
> and check the type or iterate over all classes that might appear them.
> Showing the class name would solve this problem and could save a lot
> of time.

I'm sure you'll get support for this, unless it's a really
inconvenient spot that requires a gross hack to print the type name.
Post a patch on the bug tracker.


-- 
Adam Olsen, aka Rhamphoryncus

From brett at python.org  Sun Nov 30 20:13:49 2008
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Nov 2008 11:13:49 -0800
Subject: [Python-Dev] __import__ problems
In-Reply-To: <49328ED8.2060500@mrts.pri.ee>
References: <492EB167.4010107@mrts.pri.ee> <492F0FA2.10905@gmail.com>
	<492FC1AD.70106@mrts.pri.ee> <492FDAA3.8030706@gmail.com>
	<49304165.50708@mrts.pri.ee>
	<bbaeab100811281217p2c8386b6k3180dcd57afbc9ec@mail.gmail.com>
	<49328ED8.2060500@mrts.pri.ee>
Message-ID: <bbaeab100811301113s4f8dc709q154918d939f4e079@mail.gmail.com>

On Sun, Nov 30, 2008 at 05:02, Mart Somermaa <mrts at mrts.pri.ee> wrote:
> Brett Cannon wrote:
>
>> The old-hands on python-dev know this is where  I plug  my import
>> rewrite vaporware. It will be in 3.1, and as part of it there will be
>> a new API for handling direct imports. Jacob Kaplan-Moss and I have
>
> Sounds good. I can finally rest in peace :) . May I suggest
> that you disable the hack while you are at it as well so that
> people will be aware of their misdoings?
>

Possibly since 'warnings' was re-written in C partially so it could be
used by the import rewrite. I will worry about that more when the code
actually lands.

>> talked about Django's need for this as PyCon so I am very aware of
>> needing this API and it will be addressed in the simplest way possible
>> (heck, the __import__ API might actually become a wrapper around the
>> simpler API in the end).
>
> Even better (the bracketed part).
>

Well, it's only a maybe. There  is the balance between a good API for
people versus a good API for the compiler to compile to. Not sure if
they will line up nicely or not.

>> And since nothing can go into 3.0 anyway,
>> there is no need to rush into solving this right now.
>
> Agreed, I just wanted to get the ball rolling.
>
> Let me know if you want me to do some gruntwork (tests,
> documentation) once the improved implementation is taking shape.
>

As of right now I am rewriting the tests to use more mock objects as I
was originally testing too often against actual files. I also needed
to untangle the tests as I was testing too much at once. Unfortunately
it's not grunt work as I am purposefully not looking at the old tests
to make sure that I get as much test coverage as possible in terms of
semantics (they are rather involved as I am sure you have noticed).

After that it's little compatibility things and then cleaning up the
bootstrapping code.

As for the docs, once the code  is in the docs will need to be
thoroughly rewritten so they are finally updated to include PEP 302.
But one thing at a time. =)

-Brett

From gruszczy at gmail.com  Sun Nov 30 20:17:31 2008
From: gruszczy at gmail.com (=?UTF-8?Q?Filip_Gruszczy=C5=84ski?=)
Date: Sun, 30 Nov 2008 20:17:31 +0100
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
	<aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
Message-ID: <1be78d220811301117i489e9aabmb449318504c0096f@mail.gmail.com>

Well, if this is not an inconvenient spot that requires a gross hack
to print the type name I'd love to get some instructions or direction,
and try to change it myself. Unfortunately it's the first time I
looked into Python source and it's pretty big - I tried browsing
exceptions.c and errors.c, but couldn't find any simple place, where I
could place the required type information.

I can of course post it on the bug tracker, but I believe I could
learn something on this ;-) And I don't like missing a chance to get a
glance at something new.

> I'm sure you'll get support for this, unless it's a really
> inconvenient spot that requires a gross hack to print the type name.
> Post a patch on the bug tracker.
>
>
> --
> Adam Olsen, aka Rhamphoryncus
>



-- 
Filip Gruszczy?ski

From lists at cheimes.de  Sun Nov 30 20:43:27 2008
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 30 Nov 2008 20:43:27 +0100
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
	<aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
Message-ID: <gguqcv$q87$1@ger.gmane.org>

Adam Olsen wrote:
> I'm sure you'll get support for this, unless it's a really
> inconvenient spot that requires a gross hack to print the type name.
> Post a patch on the bug tracker.

So far I can see only one argument against the proposed idea: doc tests. 
  The modified exception message would break existing doc tests.

Christian


From martin at v.loewis.de  Sun Nov 30 20:47:34 2008
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sun, 30 Nov 2008 20:47:34 +0100
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <1be78d220811301117i489e9aabmb449318504c0096f@mail.gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>	<aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
	<1be78d220811301117i489e9aabmb449318504c0096f@mail.gmail.com>
Message-ID: <4932EDD6.9040507@v.loewis.de>

Filip Gruszczy?ski wrote:
> Well, if this is not an inconvenient spot that requires a gross hack
> to print the type name I'd love to get some instructions or direction,
> and try to change it myself. Unfortunately it's the first time I
> looked into Python source and it's pretty big - I tried browsing
> exceptions.c and errors.c, but couldn't find any simple place, where I
> could place the required type information.

You need to look at the places that raise AttributeError; grep the
entire source for it (both C and .py files). In PyObject_GetAttr
(or PyObject_GenericGetAttr), you'll find the place where the case
of functions is dealt with.

Alternatively, pick a specific test case, and try to find out where
it raises the exception. E.g. set a debugger breakpoint, on
PyErr_SetString and PyErr_Format, and see whether your test case
triggers that breakpoint. Then walk up the stack to find out where
the exception is raised.

> I can of course post it on the bug tracker, but I believe I could
> learn something on this ;-) 

I sure hope so.

> And I don't like missing a chance to get a
> glance at something new.

Good luck!

Martin

From nd at perlig.de  Sun Nov 30 20:48:58 2008
From: nd at perlig.de (=?iso-8859-1?q?Andr=E9_Malo?=)
Date: Sun, 30 Nov 2008 20:48:58 +0100
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <gguqcv$q87$1@ger.gmane.org>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
	<aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
	<gguqcv$q87$1@ger.gmane.org>
Message-ID: <200811302048.58269@news.perlig.de>

* Christian Heimes wrote:

> Adam Olsen wrote:
> > I'm sure you'll get support for this, unless it's a really
> > inconvenient spot that requires a gross hack to print the type name.
> > Post a patch on the bug tracker.
>
> So far I can see only one argument against the proposed idea: doc tests.
>   The modified exception message would break existing doc tests.

As the exception text is officially not part of the API, I'd say, let them.

nd
-- 
Winnetous Erbe: <http://pub.perlig.de/books.html#apache2>

From ggpolo at gmail.com  Sun Nov 30 20:58:59 2008
From: ggpolo at gmail.com (Guilherme Polo)
Date: Sun, 30 Nov 2008 17:58:59 -0200
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
Message-ID: <ac2200130811301158j2e9ca98ct26af39230661b5c6@mail.gmail.com>

On Sun, Nov 30, 2008 at 4:41 PM, Filip Gruszczy?ski <gruszczy at gmail.com> wrote:
> This is my first message in this list, therefore I would like to say
> hello to everyone. I also hope, that I am not breaking any rules or
> guidelines for sending proposals.
>
> I would like to ask, if it is possible to provide type name of the
> object invoking the exception, when Attribute error is catched. It is
> done for functions, like:
>
> AttributeError: 'function' object has no attribute 'getValue'
>
> but for some objects there is only:
>
> AttributeError: connectToBases

I would say this is due to Py_FindMethod being used.
It is a nice and easy function to use, but it fails to set a good
error message so maybe you are experiencing these error messages from
modules that are using it.

I have added a patch in http://bugs.python.org/issue4475

>
> This is cool, when you know exactly what type of object cast the
> exception. But if there might be many of them, you must do one of two
> things: add print statement just before the line with the exception
> and check the type or iterate over all classes that might appear them.
> Showing the class name would solve this problem and could save a lot
> of time.
>
> --
> Filip Gruszczy?ski
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ggpolo%40gmail.com
>



-- 
-- Guilherme H. Polo Goncalves

From gruszczy at gmail.com  Sun Nov 30 21:11:00 2008
From: gruszczy at gmail.com (=?UTF-8?Q?Filip_Gruszczy=C5=84ski?=)
Date: Sun, 30 Nov 2008 21:11:00 +0100
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <ac2200130811301158j2e9ca98ct26af39230661b5c6@mail.gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
	<ac2200130811301158j2e9ca98ct26af39230661b5c6@mail.gmail.com>
Message-ID: <1be78d220811301211u2b342529u40f1ae9faecd707f@mail.gmail.com>

> I would say this is due to Py_FindMethod being used.
> It is a nice and easy function to use, but it fails to set a good
> error message so maybe you are experiencing these error messages from
> modules that are using it.

Yep, found it and it does call PyErr_SetString with proper values.
I'll try to change this, without breaking anything. Thanks for help :)

-- 
Filip Gruszczy?ski

From ncoghlan at gmail.com  Sun Nov 30 21:35:13 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 01 Dec 2008 06:35:13 +1000
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <gguqcv$q87$1@ger.gmane.org>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>	<aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
	<gguqcv$q87$1@ger.gmane.org>
Message-ID: <4932F901.6070803@gmail.com>

Christian Heimes wrote:
> Adam Olsen wrote:
>> I'm sure you'll get support for this, unless it's a really
>> inconvenient spot that requires a gross hack to print the type name.
>> Post a patch on the bug tracker.
> 
> So far I can see only one argument against the proposed idea: doc tests.
>  The modified exception message would break existing doc tests.

It wouldn't be the first time we've broken doc tests that way. Since the
details of the error messages aren't guaranteed to remain the same
across releases, doctests that aren't part of Python's own test suite
really should be using IGNORE_EXCEPTION_DETAIL when checking for
exceptions that are raised directly by the interpreter or standard library.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From gruszczy at gmail.com  Sun Nov 30 22:39:12 2008
From: gruszczy at gmail.com (=?UTF-8?Q?Filip_Gruszczy=C5=84ski?=)
Date: Sun, 30 Nov 2008 22:39:12 +0100
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <4932F901.6070803@gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
	<aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
	<gguqcv$q87$1@ger.gmane.org> <4932F901.6070803@gmail.com>
Message-ID: <1be78d220811301339l407ba8advfe146dc8c1511370@mail.gmail.com>

I have done some testing and it seems, that it might not be Python
problem. Well, when I use only pure Python objects, I get really nice
description of the object (which means the type). But I am using PyQt
and it seems, that when an object is subclassing QObject (or possibly
some other class from qt, that can be not derived from QObject) it can
only display information about the name of the function. PyQt are
python bindings for C++ qt library. Can this be the reason for not
displaying type of the object?

2008/11/30 Nick Coghlan <ncoghlan at gmail.com>:
> Christian Heimes wrote:
>> Adam Olsen wrote:
>>> I'm sure you'll get support for this, unless it's a really
>>> inconvenient spot that requires a gross hack to print the type name.
>>> Post a patch on the bug tracker.
>>
>> So far I can see only one argument against the proposed idea: doc tests.
>>  The modified exception message would break existing doc tests.
>
> It wouldn't be the first time we've broken doc tests that way. Since the
> details of the error messages aren't guaranteed to remain the same
> across releases, doctests that aren't part of Python's own test suite
> really should be using IGNORE_EXCEPTION_DETAIL when checking for
> exceptions that are raised directly by the interpreter or standard library.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> ---------------------------------------------------------------
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/gruszczy%40gmail.com
>



-- 
Filip Gruszczy?ski

From ncoghlan at gmail.com  Sun Nov 30 22:50:33 2008
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 01 Dec 2008 07:50:33 +1000
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <1be78d220811301339l407ba8advfe146dc8c1511370@mail.gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>	<aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>	<gguqcv$q87$1@ger.gmane.org>
	<4932F901.6070803@gmail.com>
	<1be78d220811301339l407ba8advfe146dc8c1511370@mail.gmail.com>
Message-ID: <49330AA9.7070005@gmail.com>

Filip Gruszczy?ski wrote:
> I have done some testing and it seems, that it might not be Python
> problem. Well, when I use only pure Python objects, I get really nice
> description of the object (which means the type). But I am using PyQt
> and it seems, that when an object is subclassing QObject (or possibly
> some other class from qt, that can be not derived from QObject) it can
> only display information about the name of the function. PyQt are
> python bindings for C++ qt library. Can this be the reason for not
> displaying type of the object?

Yeah, any time someone implements their own attribute lookup process for
a class (be it via __getattr__, __getattribute__ or the C equivalents),
it is up to the reimplementation to appropriately format their error
message if they raise AttributeError directly.

This could possibly be made easier to do correctly via a specific
AttributeError class method (also exposed through the C API) that
accepted a type object and an attribute name and then produced a nicely
formatted error message the way the builtin types do.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------

From gruszczy at gmail.com  Sun Nov 30 23:02:09 2008
From: gruszczy at gmail.com (=?UTF-8?Q?Filip_Gruszczy=C5=84ski?=)
Date: Sun, 30 Nov 2008 23:02:09 +0100
Subject: [Python-Dev] Attribute error: providing type name
In-Reply-To: <49330AA9.7070005@gmail.com>
References: <1be78d220811301041o6f737b6q9088b4b8266cf56f@mail.gmail.com>
	<aac2c7cb0811301106y3a1bbcbbt705365f37be4f548@mail.gmail.com>
	<gguqcv$q87$1@ger.gmane.org> <4932F901.6070803@gmail.com>
	<1be78d220811301339l407ba8advfe146dc8c1511370@mail.gmail.com>
	<49330AA9.7070005@gmail.com>
Message-ID: <1be78d220811301402p4281e8b3wd05122dd4ea87a6@mail.gmail.com>

> Yeah, any time someone implements their own attribute lookup process for
> a class (be it via __getattr__, __getattribute__ or the C equivalents),
> it is up to the reimplementation to appropriately format their error
> message if they raise AttributeError directly.

I guess, this means that I have to go to Phil Thompson at Riverbank
and try to convince him to change the message.

-- 
Filip Gruszczy?ski

From mhammond at skippinet.com.au  Sun Nov 30 23:06:58 2008
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Mon, 1 Dec 2008 09:06:58 +1100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <492FBB2C.5000309@gmail.com>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>	<042401c95012$3bff99d0$b3fecd70$@com.au>	<492DCE5E.5080602@v.loewis.de>	<043e01c95019$9955a0a0$cc00e1e0$@com.au>	<492DDE40.2040206@v.loewis.de>	<045801c9503a$8e85d2f0$ab9178d0$@com.au>	<492F2788.7040300@canterbury.ac.nz>
	<04ce01c9510d$3132d750$939885f0$@com.au>
	<492FBB2C.5000309@gmail.com>
Message-ID: <053201c95337$fa09e930$ee1dbb90$@com.au>

> > [Hrm - looking closer at that HTML link I sent before, it says
> > specifically "Per-machine installs must install to Program Files 
> > by default in order to
> > pass this test case" - that seems pretty clear...]
> 
> Given that the links in Gerald's examples were under Program Files, I
> had assumed that HP were tweaking the installer at least that much...

But wouldn't that leave us in the situation where Python's installer is
*not* "OEM Ready", nor has it been certified as such - but an installer
based on ours, but with unspecified changes has?

Of course, I don't object to that and still think we should help where we
can, but if that is true it would make the premise of this thread a little
misleading, as obviously HP could then make *any* necessary changes without
our agreement or even knowledge.

Cheers,

Mark


From jyasskin at gmail.com  Sun Nov 30 23:19:32 2008
From: jyasskin at gmail.com (Jeffrey Yasskin)
Date: Sun, 30 Nov 2008 14:19:32 -0800
Subject: [Python-Dev] [ANN] VPython 0.1
In-Reply-To: <20081023014814.GA14030@vidar.dreamhost.com>
References: <20081023014814.GA14030@vidar.dreamhost.com>
Message-ID: <5d44f72f0811301419u6175efddma2af264089cc9601@mail.gmail.com>

Here's another data point. My results are similar to Skip's
(unsurprising since I'm also using a mac). My wild guess is that the
30% vs 10% improvement is an AMD vs. Intel thing? It's not 32-bit vs.
64-bit since both David and Jakob got a 30% speedup, but David had a
32-bit build while Jakob had a 64-bit build.

There's also a crashing bug on:
  def f():
    a+=1
  f()
I have a fix by changing the load_fast opcode to adjust the stack on
error, but it requires removing all the superinstructions involving
load_fast, which costs me 1-2% in performance. The fix is not included
in these numbers.

-------------------------------------------------------------------------------
PYBENCH 2.0
-------------------------------------------------------------------------------
* using CPython 2.6+ (unknown, Nov 19 2008, 09:14:51) [GCC 4.0.1
(Apple Inc. build 5484)]
* disabled garbage collection
* system check interval set to maximum: 2147483647
* using timer: time.time

-------------------------------------------------------------------------------
Benchmark: /Users/jyasskin/src/python/bzr/2.6_cxx/build_c4.0/pybench.out
-------------------------------------------------------------------------------

   Rounds: 10
   Warp:   10
   Timer:  time.time

   Machine Details:
      Platform ID:    Darwin-9.5.0-i386-32bit
      Processor:      i386

   Python:
      Implementation: CPython
      Executable:
/Users/jyasskin/src/python/bzr/2.6_cxx/build_c4.0/python.exe
      Version:        2.6.0
      Compiler:       GCC 4.0.1 (Apple Inc. build 5484)
      Bits:           32bit
      Build:          Nov 19 2008 09:14:51 (#unknown)
      Unicode:        UCS2


-------------------------------------------------------------------------------
Comparing with: /Users/jyasskin/src/python/bzr/2.6_vmgen/build/pybench.out
-------------------------------------------------------------------------------

   Rounds: 10
   Warp:   10
   Timer:  time.time

   Machine Details:
      Platform ID:    Darwin-9.5.0-i386-32bit
      Processor:      i386

   Python:
      Implementation: CPython
      Executable:     /Users/jyasskin/src/python/bzr/2.6_vmgen/build/python.exe
      Version:        2.6.0
      Compiler:       GCC 4.0.1 (Apple Inc. build 5488)
      Bits:           32bit
      Build:          Nov 24 2008 20:20:04 (#unknown)
      Unicode:        UCS2


Test                             minimum run-time        average  run-time
                                this    other   diff    this    other   diff
-------------------------------------------------------------------------------
         BuiltinFunctionCalls:   131ms   118ms  +10.9%   134ms   120ms  +11.3%
          BuiltinMethodLookup:   109ms    90ms  +20.9%   111ms    96ms  +15.7%
                CompareFloats:    91ms    65ms  +40.4%    92ms    66ms  +39.2%
        CompareFloatsIntegers:    99ms    85ms  +16.5%    99ms    85ms  +16.4%
              CompareIntegers:    83ms    49ms  +67.3%    83ms    50ms  +67.2%
       CompareInternedStrings:    93ms    72ms  +30.3%    95ms    73ms  +29.3%
                 CompareLongs:    84ms    62ms  +36.6%    86ms    63ms  +37.3%
               CompareStrings:    82ms    68ms  +20.2%    84ms    71ms  +17.7%
               CompareUnicode:   104ms    89ms  +17.5%   109ms    94ms  +15.1%
   ComplexPythonFunctionCalls:   139ms   126ms  +11.1%   142ms   127ms  +11.4%
                ConcatStrings:   149ms   138ms   +8.0%   154ms   148ms   +3.8%
                ConcatUnicode:    88ms    84ms   +4.7%    90ms    85ms   +5.8%
              CreateInstances:   142ms   130ms   +9.5%   143ms   131ms   +9.0%
           CreateNewInstances:   106ms    99ms   +7.4%   107ms    99ms   +7.6%
      CreateStringsWithConcat:   116ms    94ms  +23.3%   118ms    95ms  +25.0%
      CreateUnicodeWithConcat:    91ms    83ms  +10.3%    92ms    84ms   +9.6%
                 DictCreation:    92ms    80ms  +14.8%    93ms    81ms  +14.8%
            DictWithFloatKeys:    95ms    90ms   +5.2%    98ms    91ms   +6.7%
          DictWithIntegerKeys:    99ms    91ms   +9.1%   104ms    92ms  +13.8%
           DictWithStringKeys:    83ms    73ms  +13.8%    87ms    76ms  +14.9%
                     ForLoops:    77ms    62ms  +23.2%    79ms    63ms  +24.5%
                   IfThenElse:    78ms    55ms  +41.6%    79ms    56ms  +42.7%
                  ListSlicing:   115ms   185ms  -37.7%   120ms   187ms  -36.1%
               NestedForLoops:   135ms   100ms  +35.0%   136ms   102ms  +33.8%
         NormalClassAttribute:   105ms    98ms   +6.9%   106ms    99ms   +6.8%
      NormalInstanceAttribute:    93ms    84ms  +11.2%    94ms    85ms  +10.8%
          PythonFunctionCalls:   102ms    90ms  +13.5%   105ms    93ms  +13.4%
            PythonMethodCalls:   147ms   133ms  +10.5%   148ms   135ms   +9.7%
                    Recursion:   142ms   118ms  +20.2%   147ms   119ms  +22.9%
                 SecondImport:    99ms    98ms   +1.3%   100ms   100ms   +0.1%
          SecondPackageImport:   102ms   101ms   +1.2%   104ms   102ms   +1.8%
        SecondSubmoduleImport:   133ms   133ms   +0.4%   135ms   134ms   +1.0%
      SimpleComplexArithmetic:   100ms    93ms   +7.3%   101ms    94ms   +7.8%
       SimpleDictManipulation:   110ms    93ms  +18.3%   111ms    94ms  +18.3%
        SimpleFloatArithmetic:    92ms    76ms  +19.9%    94ms    82ms  +15.5%
     SimpleIntFloatArithmetic:    73ms    62ms  +16.8%    73ms    63ms  +16.4%
      SimpleIntegerArithmetic:    73ms    64ms  +13.5%    74ms    65ms  +13.0%
       SimpleListManipulation:    79ms    67ms  +18.6%    80ms    69ms  +15.6%
         SimpleLongArithmetic:   111ms    98ms  +13.3%   112ms    99ms  +13.3%
                   SmallLists:   126ms   112ms  +12.9%   129ms   114ms  +12.7%
                  SmallTuples:   123ms   104ms  +18.5%   125ms   105ms  +18.7%
        SpecialClassAttribute:   101ms    95ms   +6.5%   102ms    97ms   +5.2%
     SpecialInstanceAttribute:   173ms   154ms  +12.8%   175ms   158ms  +10.7%
               StringMappings:   165ms   163ms   +1.1%   166ms   164ms   +1.3%
             StringPredicates:   126ms   121ms   +4.3%   130ms   124ms   +5.0%
                StringSlicing:   125ms   107ms  +17.1%   130ms   111ms  +16.4%
                    TryExcept:    83ms    57ms  +44.6%    84ms    58ms  +45.3%
                   TryFinally:   102ms   104ms   -1.8%   107ms   105ms   +2.2%
               TryRaiseExcept:    98ms    95ms   +2.9%    99ms    97ms   +2.7%
                 TupleSlicing:   124ms   141ms  -12.5%   138ms   144ms   -4.4%
              UnicodeMappings:   142ms   142ms   -0.2%   143ms   143ms   +0.1%
            UnicodePredicates:   107ms   100ms   +7.4%   108ms   101ms   +7.2%
            UnicodeProperties:   109ms   101ms   +7.8%   111ms   102ms   +8.3%
               UnicodeSlicing:   107ms    84ms  +27.8%   111ms    89ms  +24.4%
                  WithFinally:   156ms   151ms   +3.4%   157ms   151ms   +3.9%
              WithRaiseExcept:   124ms   120ms   +3.1%   125ms   121ms   +3.0%
-------------------------------------------------------------------------------
Totals:                          6137ms  5548ms  +10.6%  6258ms  5653ms  +10.7%

(this=/Users/jyasskin/src/python/bzr/2.6_cxx/build_c4.0/pybench.out,
other=/Users/jyasskin/src/python/bzr/2.6_vmgen/build/pybench.out)

On Wed, Oct 22, 2008 at 5:48 PM, David Ripton <dripton at ripton.net> wrote:
>> Feedback is, of course, very welcome and it'd be great to have some pybench
>> results from different machines.
>
> My results are very similar to Jakob's.
>
> Gentoo Linux, 32-bit x86, Athlon 6400+ underclocked to 3.0 GHz.
>
> make test:
> 282 tests OK.
> 5 tests failed:
>    test_doctest test_hotshot test_inspect test_subprocess test_trace
>
> -------------------------------------------------------------------------------
> PYBENCH 2.0
> -------------------------------------------------------------------------------
> * using Python 2.5.2 (r252:60911, Oct 22 2008, 13:47:58) [GCC 4.1.2 20070214 ( (gdc 0.24, using dmd 1.020)) (Gentoo 4.1.2 p1.0.2)]
> * disabled garbage collection
> * system check interval set to maximum: 2147483647
> * using timer: time.time
>
> Calibrating tests. Please wait... done.
>
> Running 10 round(s) of the suite at warp factor 10:
>
> * Round 1 done in 8.474 seconds.
> * Round 2 done in 8.389 seconds.
> * Round 3 done in 8.438 seconds.
> * Round 4 done in 8.411 seconds.
> * Round 5 done in 8.484 seconds.
> * Round 6 done in 8.471 seconds.
> * Round 7 done in 8.492 seconds.
> * Round 8 done in 8.549 seconds.
> * Round 9 done in 8.429 seconds.
> * Round 10 done in 8.542 seconds.
>
> -------------------------------------------------------------------------------
> Benchmark: 2008-10-22 20:45:22
> -------------------------------------------------------------------------------
>
>    Rounds: 10
>    Warp:   10
>    Timer:  time.time
>
>    Machine Details:
>       Platform ID:    Linux-2.6.26-gentoo-r1-i686-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_6400+-with-glibc2.3
>       Processor:      AMD Athlon(tm) 64 X2 Dual Core Processor 6400+
>
>    Python:
>       Implementation: n/a
>       Executable:     /var/tmp/VPython-2.5.2/python
>       Version:        2.5.2
>       Compiler:       GCC 4.1.2 20070214 (  (gdc 0.24, using dmd 1.020)) (Gentoo 4.1.2 p1.0.2)
>       Bits:           32bit
>       Build:          Oct 22 2008 13:47:58 (#r252:60911)
>       Unicode:        UCS2
>
>
> -------------------------------------------------------------------------------
> Comparing with: /tmp/vanilla252.pybench
> -------------------------------------------------------------------------------
>
>    Rounds: 10
>    Warp:   10
>    Timer:  time.time
>
>    Machine Details:
>       Platform ID:    Linux-2.6.26-gentoo-r1-i686-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_6400+-with-glibc2.3
>       Processor:      AMD Athlon(tm) 64 X2 Dual Core Processor 6400+
>
>    Python:
>       Implementation: n/a
>       Executable:     /usr/local/bin/python2.5
>       Version:        2.5.2
>       Compiler:       GCC 4.1.2 20070214 (  (gdc 0.24, using dmd 1.020)) (Gentoo 4.1.2 p1.0.2)
>       Bits:           32bit
>       Build:          Oct 22 2008 20:39:10 (#r252:60911)
>       Unicode:        UCS2
>
>
> Test                             minimum run-time        average  run-time
>                                 this    other   diff    this    other   diff
> -------------------------------------------------------------------------------
>          BuiltinFunctionCalls:   144ms   205ms  -30.1%   162ms   240ms  -32.5%
>           BuiltinMethodLookup:   164ms   222ms  -26.2%   167ms   236ms  -29.2%
>                 CompareFloats:    90ms   211ms  -57.5%   103ms   222ms  -53.7%
>         CompareFloatsIntegers:    88ms   182ms  -51.4%   107ms   200ms  -46.6%
>               CompareIntegers:    63ms   258ms  -75.5%    84ms   272ms  -69.1%
>        CompareInternedStrings:    93ms   252ms  -63.0%   103ms   261ms  -60.5%
>                  CompareLongs:    65ms   180ms  -63.9%    87ms   203ms  -57.1%
>                CompareStrings:   113ms   211ms  -46.5%   120ms   218ms  -44.9%
>                CompareUnicode:   187ms   273ms  -31.7%   228ms   290ms  -21.4%
>    ComplexPythonFunctionCalls:   261ms   330ms  -20.9%   277ms   336ms  -17.5%
>                 ConcatStrings:   204ms   255ms  -20.2%   209ms   297ms  -29.7%
>                 ConcatUnicode:   143ms   118ms  +20.3%   159ms   228ms  -30.0%
>               CreateInstances:   172ms   112ms  +53.0%   187ms   211ms  -11.5%
>            CreateNewInstances:   165ms   100ms  +65.0%   171ms   196ms  -12.6%
>       CreateStringsWithConcat:   141ms   133ms   +5.8%   160ms   256ms  -37.3%
>       CreateUnicodeWithConcat:   145ms   126ms  +14.8%   167ms   242ms  -30.9%
>                  DictCreation:   129ms    98ms  +31.6%   131ms   184ms  -28.8%
>             DictWithFloatKeys:   185ms   143ms  +29.6%   216ms   268ms  -19.6%
>           DictWithIntegerKeys:   122ms   115ms   +6.0%   126ms   227ms  -44.4%
>            DictWithStringKeys:    92ms   112ms  -17.6%   104ms   216ms  -51.8%
>                      ForLoops:    98ms   224ms  -56.2%   117ms   243ms  -52.0%
>                    IfThenElse:    89ms   221ms  -59.9%    97ms   237ms  -59.1%
>                   ListSlicing:   123ms   111ms  +10.8%   131ms   141ms   -6.8%
>                NestedForLoops:   138ms   234ms  -41.1%   153ms   262ms  -41.6%
>          NormalClassAttribute:   131ms   225ms  -41.5%   139ms   243ms  -42.9%
>       NormalInstanceAttribute:   121ms   191ms  -36.9%   121ms   210ms  -42.5%
>           PythonFunctionCalls:   134ms   200ms  -32.6%   144ms   219ms  -34.2%
>             PythonMethodCalls:   173ms   228ms  -23.9%   185ms   251ms  -26.5%
>                     Recursion:   177ms   298ms  -40.5%   187ms   316ms  -40.8%
>                  SecondImport:   135ms   133ms   +1.5%   160ms   147ms   +8.9%
>           SecondPackageImport:   148ms   141ms   +5.0%   166ms   162ms   +2.7%
>         SecondSubmoduleImport:   209ms   188ms  +11.4%   221ms   203ms   +8.6%
>       SimpleComplexArithmetic:   131ms   219ms  -40.0%   139ms   239ms  -41.7%
>        SimpleDictManipulation:   105ms   210ms  -49.9%   123ms   233ms  -47.1%
>         SimpleFloatArithmetic:    93ms   224ms  -58.6%   109ms   246ms  -55.8%
>      SimpleIntFloatArithmetic:    84ms   190ms  -56.0%    89ms   213ms  -58.4%
>       SimpleIntegerArithmetic:    82ms   191ms  -57.1%    84ms   218ms  -61.5%
>        SimpleListManipulation:    85ms   188ms  -54.6%    90ms   207ms  -56.7%
>          SimpleLongArithmetic:   111ms   198ms  -44.0%   134ms   215ms  -37.6%
>                    SmallLists:   126ms   182ms  -30.7%   143ms   202ms  -28.9%
>                   SmallTuples:   132ms   193ms  -31.3%   143ms   210ms  -31.7%
>         SpecialClassAttribute:   110ms   221ms  -50.4%   144ms   241ms  -40.1%
>      SpecialInstanceAttribute:   146ms   236ms  -38.2%   165ms   258ms  -36.1%
>                StringMappings:   177ms   209ms  -15.2%   186ms   218ms  -14.5%
>              StringPredicates:   169ms   219ms  -22.9%   178ms   238ms  -25.0%
>                 StringSlicing:   130ms   206ms  -37.0%   151ms   223ms  -32.4%
>                     TryExcept:    92ms   230ms  -59.9%    94ms   258ms  -63.5%
>                    TryFinally:   139ms   183ms  -23.6%   160ms   204ms  -21.8%
>                TryRaiseExcept:   139ms   147ms   -5.0%   151ms   162ms   -6.7%
>                  TupleSlicing:   135ms   174ms  -22.0%   151ms   190ms  -20.7%
>               UnicodeMappings:   222ms   244ms   -8.9%   241ms   257ms   -6.3%
>             UnicodePredicates:   170ms   214ms  -20.6%   179ms   227ms  -21.2%
>             UnicodeProperties:   136ms   159ms  -14.9%   154ms   206ms  -25.3%
>                UnicodeSlicing:   142ms   215ms  -34.1%   171ms   248ms  -31.3%
>                   WithFinally:   208ms   260ms  -20.1%   212ms   271ms  -21.9%
>               WithRaiseExcept:   175ms   193ms   -9.0%   186ms   209ms  -11.0%
> -------------------------------------------------------------------------------
> Totals:                          7682ms 10935ms  -29.8%  8468ms 12832ms  -34.0%
>
> (this=2008-10-22 20:45:22, other=/tmp/vanilla252.pybench)
>
> --
> David Ripton    dripton at ripton.net
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jyasskin%40gmail.com
>



-- 
Namast?,
Jeffrey Yasskin
http://jeffrey.yasskin.info/

From martin at v.loewis.de  Sun Nov 30 23:23:51 2008
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 30 Nov 2008 23:23:51 +0100
Subject: [Python-Dev] Python for windows.
In-Reply-To: <053201c95337$fa09e930$ee1dbb90$@com.au>
References: <90bb445a0811260928u5a6b5c36ib4b6947472d2b2be@mail.gmail.com>	<238A96A773B3934685A7269CC8A8D0423F7FBF71E9@GVW0436EXB.americas.hpqcorp.net>	<ggkat4$2q5$1@ger.gmane.org>	<238A96A773B3934685A7269CC8A8D0423F7FBF728C@GVW0436EXB.americas.hpqcorp.net>	<042401c95012$3bff99d0$b3fecd70$@com.au>	<492DCE5E.5080602@v.loewis.de>	<043e01c95019$9955a0a0$cc00e1e0$@com.au>	<492DDE40.2040206@v.loewis.de>	<045801c9503a$8e85d2f0$ab9178d0$@com.au>	<492F2788.7040300@canterbury.ac.nz>	<04ce01c9510d$3132d750$939885f0$@com.au>	<492FBB2C.5000309@gmail.com>
	<053201c95337$fa09e930$ee1dbb90$@com.au>
Message-ID: <49331277.10003@v.loewis.de>

> Of course, I don't object to that and still think we should help where we
> can, but if that is true it would make the premise of this thread a little
> misleading, as obviously HP could then make *any* necessary changes without
> our agreement or even knowledge.

Perhaps. However, "help where we can" is about right. If its only the
changes HP discussed so far, I think we should be able to help.
For the Program Files issue, without going into the discussion whether
Python's defaults are good or not, I think there would be still a number
of technical solutions (such as providing a merge module which changes
the default).

Regards,
Martin