From richies at gmail.com  Mon Oct  1 01:11:11 2007
From: richies at gmail.com (Richie Ward)
Date: Mon, 1 Oct 2007 00:11:11 +0100
Subject: [Python-Dev] OpenSSL httplib bug
Message-ID: <6b33c0ac0709301611sde8b92fvfb892550935d7f99@mail.gmail.com>

I was using httplib to power my xml rpc script.

I had problems when I wanted to use SSL and I got this error:
  File "/usr/lib/python2.5/httplib.py", line 1109, in recv
    return self._ssl.read(len)
socket.sslerror: (8, 'EOF occurred in violation of protocol')

I figured out this was because of poor error handling in python.

May I suggest this as a fix to this bug:
$ diff /usr/lib/python2.5/httplib.py /usr/lib/python2.5/httplib.py~
1109,1112c1109
<         try:
<             return self._ssl.read(len)
<         except socket.sslerror:
<             return
---
>         return self._ssl.read(len)

Just a note. I am by no means a python expert, just good enough to get
my work done.
I use Ubuntu gutsy.
-- 
Thanks, Richie Ward

From greg.ewing at canterbury.ac.nz  Mon Oct  1 02:17:07 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 01 Oct 2007 13:17:07 +1300
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <E1IbvCU-0000Jv-Pb@draco.cus.cam.ac.uk>
References: <E1IbvCU-0000Jv-Pb@draco.cus.cam.ac.uk>
Message-ID: <47003C83.5050904@canterbury.ac.nz>

Nick Maclaren wrote:
> I don't know PRECISELY what you mean by "universal newlines mode"

I mean precisely what Python means by the term: any of
"\r", "\n" or "\r\n" represent a newline, and no distinction
is made between them.

You only need to use that if you don't know what convention
is being used by the file you're reading. And if you don't
know that, you've already lost information about what the
contents of the file means, and there's nothing that any
I/O system can do to get it back.

> Model 1:  certain characters can be used only in combination.
 > ...

That's all fine if you know the file adheres to those
conventions. Just open it in binary mode and go for it.

The I/O systems of C and/or Python are designed for
environments where the files *don't* adhere to conventions
as helpful as that. They're making the best of what they're
given.

> Note that the above is what the program sees - what is written
> to the outside world and how input is read is another matter.
> 
> But I can assure you, from my own and many other people's experience,
> that neither of the above models cause the confusion being shown by
> the postings in this thread.

There's no confusion about how newlines are represented
*inside* a Python program. The convention is quite clear -
a newline is "\n" and only "\n". Confusion only arises when
people try to process strings internally that don't adhere
to that convention.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From brett at python.org  Mon Oct  1 02:17:17 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 30 Sep 2007 17:17:17 -0700
Subject: [Python-Dev] OpenSSL httplib bug
In-Reply-To: <6b33c0ac0709301611sde8b92fvfb892550935d7f99@mail.gmail.com>
References: <6b33c0ac0709301611sde8b92fvfb892550935d7f99@mail.gmail.com>
Message-ID: <bbaeab100709301717k7231497fw87a80b91c629be44@mail.gmail.com>

On 9/30/07, Richie Ward <richies at gmail.com> wrote:
> I was using httplib to power my xml rpc script.
>
> I had problems when I wanted to use SSL and I got this error:
>   File "/usr/lib/python2.5/httplib.py", line 1109, in recv
>     return self._ssl.read(len)
> socket.sslerror: (8, 'EOF occurred in violation of protocol')
>
> I figured out this was because of poor error handling in python.
>
> May I suggest this as a fix to this bug:
> $ diff /usr/lib/python2.5/httplib.py /usr/lib/python2.5/httplib.py~
> 1109,1112c1109
> <         try:
> <             return self._ssl.read(len)
> <         except socket.sslerror:
> <             return
> ---
> >         return self._ssl.read(len)
>
> Just a note. I am by no means a python expert, just good enough to get
> my work done.
> I use Ubuntu gutsy.

If you could, Richie, please open a bug report at bugs.python.org and
attach your patch to it?  That way it won't get lost and it can be
attended to properly.

-Brett

From greg.ewing at canterbury.ac.nz  Mon Oct  1 02:37:37 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 01 Oct 2007 13:37:37 +1300
Subject: [Python-Dev] New lines, carriage returns, and Windows
In-Reply-To: <E1IbvQy-0000RO-Cj@draco.cus.cam.ac.uk>
References: <E1IbvQy-0000RO-Cj@draco.cus.cam.ac.uk>
Message-ID: <47004151.60200@canterbury.ac.nz>

Nick Maclaren wrote:
> I have implemented both of those two models
> on systems that are FAR more different than most people can imagine.
> Both work, and neither causes confusion.  The C/Unix/Python one does.

Now I'm not sure what *you* mean by the C/Unix/Python
model. As far as newlines are concerned, the internal
model is fine as far as I can see.

> a mismatch between
>>the world of Python strings which use "\n" and .NET
>>library code expecting strings which use "\r\n".
> 
> That's an I/O problem :-)

If you define passing a string to/from any .NET function
as I/O, I suppose that's true, but it's not what people
normally mean by the term.

> the REASON it causes trouble is the inconsistency
> in the basic C/Unix/Python text I/O model.  Let's consider just
> \f, \r and \n,

But we're not talking about \f or anything else here, only
newlines. I've never heard anyone complain about getting
confused over the handling of \f in Python. That may be
because nobody uses \f for anything these days, but whatever
the reason, it seems to be a non-issue.

In any case, it still doesn't mean that you "don't get
back what you wrote". If you write "\f\n" to a file using
Python and read it back, you get "\f\n". If you write just
"\f", you get back "\f". What the \f *means* is a separate
issue.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Mon Oct  1 02:48:02 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 01 Oct 2007 13:48:02 +1300
Subject: [Python-Dev] New lines, carriage returns, and Windows
In-Reply-To: <18175.42108.40732.660470@montanaro.dyndns.org>
References: <7AD436E4270DD54A94238001769C2227CCBD18CB33@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
	<46FB0F9D.6010303@canterbury.ac.nz>
	<18171.6511.526695.684154@montanaro.dyndns.org>
	<ca471dc20709271035h29948358xe6bba5466edb1963@mail.gmail.com>
	<18175.42108.40732.660470@montanaro.dyndns.org>
Message-ID: <470043C2.5010202@canterbury.ac.nz>

skip at pobox.com wrote:
> I've been thinking about this some more (in lieu of actually writing up any
> sort of proposal ;-) and I'm not so sure it would be all that useful.

Yes, despite being the one who suggested it, I've come to
the same conclusion myself. The problem should really be
addressed at the source, which is the Python/.NET boundary.
Anything else would just lead to ambiguity.

So I'm voting -1 on my own proposal here.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Mon Oct  1 02:57:00 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 01 Oct 2007 13:57:00 +1300
Subject: [Python-Dev] [python] Re:  New lines, carriage returns,
	and Windows
In-Reply-To: <46FFA984.9060602@voidspace.org.uk>
References: <E1IbYZl-0000pk-HS@libra.cus.cam.ac.uk>
	<ca471dc20709290807l351b87d7k2dd651e5ef18a83c@mail.gmail.com>
	<46FE6F92.40601@voidspace.org.uk> <fdm5l4$rok$1@sea.gmane.org>
	<46FE9B09.8000800@voidspace.org.uk>
	<18175.42782.873060.154910@montanaro.dyndns.org>
	<46FFA984.9060602@voidspace.org.uk>
Message-ID: <470045DC.5040903@canterbury.ac.nz>

Michael Foord wrote:
> We stick to using the .NET file I/O and so don't 
> have a problem. The only time it is an issue for us is our tests, where 
> we have string literals in our test code (where new lines are obviously 
> '\n')

If you're going to do that, you really need to be consistent
about and have IronPython use \r\n internally for line endings
*everywhere*, including string literals.

> It is just slightly ironic that the time Python 'gets it wrong' (for 
> some value of wrong) is when you are using text mode for I/O :-)

I would say IronPython is getting it wrong by using inconsistent
internal representations of line endings.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Mon Oct  1 04:14:59 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 30 Sep 2007 19:14:59 -0700
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <470045DC.5040903@canterbury.ac.nz>
References: <E1IbYZl-0000pk-HS@libra.cus.cam.ac.uk>
	<ca471dc20709290807l351b87d7k2dd651e5ef18a83c@mail.gmail.com>
	<46FE6F92.40601@voidspace.org.uk> <fdm5l4$rok$1@sea.gmane.org>
	<46FE9B09.8000800@voidspace.org.uk>
	<18175.42782.873060.154910@montanaro.dyndns.org>
	<46FFA984.9060602@voidspace.org.uk>
	<470045DC.5040903@canterbury.ac.nz>
Message-ID: <ca471dc20709301914u3f972abdpbab870bcf0114d64@mail.gmail.com>

On 9/30/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Michael Foord wrote:
> > We stick to using the .NET file I/O and so don't
> > have a problem. The only time it is an issue for us is our tests, where
> > we have string literals in our test code (where new lines are obviously
> > '\n')
>
> If you're going to do that, you really need to be consistent
> about and have IronPython use \r\n internally for line endings
> *everywhere*, including string literals.

I don't know what you mean by "internally". There's lots of portable
code that uses the \n character in string literals (either to generate
line endings or to recognize them). That code can't suddenly be made
invalid. And changing all string literals that say "\n" to secretly
become "\r\n" would be worse than the \r <--> \n swap that some old
Apple tools used to do. (If len("\n") == 2, what would len("\r\n")
be?)

> > It is just slightly ironic that the time Python 'gets it wrong' (for
> > some value of wrong) is when you are using text mode for I/O :-)
>
> I would say IronPython is getting it wrong by using inconsistent
> internal representations of line endings.

Honestly, I find it hard to see much merit in this discussion. A
number of Python libraries, including print() and io.py, use \n to
represent line endings in memory, and translate these to/from
platform-appropriate line endings when reading/writing text files.
OTOH, some other APIs, for example, sockets talking various internet
protocols (from SMTP to HTTP) as well as most (all?) native .NET APIs,
use \r\n to represent line endings. There are any number of ways to
convert between these conversions, including various invocations of
s.replace() and s.splitlines() (the latter does a
universal-newlines-like thing). Applications can take care of this,
and APIs can choose to use either convention for line endings (or
both, in the case of input).

Yes, occasionally users get confused. Too bad. They'll have to learn
about this issue. The issue isn't going away by wishing it to go away;
it is a fundamental difference between Windows and Unix, and neither
is likely to change or disappear. Changing Python to use the Windows
convention internally isn't going to help one bit. Changing Python to
use the platforn's convention is impossible without introducing a new
string escape that would mean \r\n on Windows and \n on Unix; and
given that there are legitimate reasons to sometimes deal with \r\n
explicitly even on Unix (and with just \n even on Windows) we wouldn't
be completely isolated from the issue. Changing APIs to not represent
the line ending as a character (as the Java I/O libraries do) would be
too big a change (and how would we distinguish between readline()
returning an empty line and EOF?) -- and I'm sure the issue still pops
up in plenty of places in Java.

The best solution for IronPython is probably to have the occasional
wrapper around .NET APIs that translates between \r\n and \n on the
boundary between Python and .NET; but one must be able to turn this
off or bypass the wrappers in cases where the data retrieved from one
.NET API is just passed straight on to another .NET API (and the
translation would just cause two redundant copies being made).

Get used to it. End of discussion.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tulloss2 at uiuc.edu  Mon Oct  1 08:43:04 2007
From: tulloss2 at uiuc.edu (Justin Tulloss)
Date: Mon, 1 Oct 2007 01:43:04 -0500
Subject: [Python-Dev] GC Changes
In-Reply-To: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
Message-ID: <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>

Hello,

I've been doing some tests on removing the GIL, and it's becoming clear that
some basic changes to the garbage collector may be needed in order for this
to happen efficiently. Reference counting as it stands today is not very
scalable.

I've been looking into a few options, and I'm leaning towards the
implementing IBMs recycler GC
(http://www.research.ibm.com/people/d/dfb/recycler-publications.html
) since it is very similar to what is in place now from the users'
perspective. However, I haven't been around the list long enough to really
understand the feeling in the community on GC in the future of the
interpreter. It seems that a full GC might have a lot of benefits in terms
of performance and scalability, and I think that the current gc module is of
the mark-and-sweep variety. Is the trend going to be to move away from
reference counting and towards the mark-and-sweep implementation that
currently exists, or is reference counting a firmly ingrained tradition?

On a more immediately relevant note, I'm not certain I understand the full
extent of the gc module. From what I've read, it sounds like it's fairly
close to a fully functional GC, yet it seems to exist only as a
cycle-detecting backup to the reference counting mechanism. Would somebody
care to give me a brief overview on how the current gc module interacts with
the interpreter, or point me to a place where that is done? Why isn't the
mark-and-sweep mechanism used for all memory management?

Thanks a lot!

Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071001/08f40ae5/attachment.htm 

From nmm1 at cus.cam.ac.uk  Mon Oct  1 10:27:10 2007
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Mon, 01 Oct 2007 09:27:10 +0100
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
Message-ID: <E1IcGcQ-0002Hf-JZ@virgo.cus.cam.ac.uk>

Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> > I don't know PRECISELY what you mean by "universal newlines mode"
> 
> I mean precisely what Python means by the term: any of
> "\r", "\n" or "\r\n" represent a newline, and no distinction
> is made between them.

Excellent.  While this over-simplifies the issue, let's stick to
the over-simplified form, as we may be able to get somewhere.

The question is independent of what the outside system believes a
text file should look like, and is solely what Python believes a
sequence of characters should mean.  For example, does 'A\r\nB'
mean that B is separated from A by one newline or two?

The point is that, once we know that, we can design a translator
to and from Python's conventions to any reasonable system (and,
as I say, I have done it many times).  But, if Python's own
interpretation is ambiguous, it is a sure recipe for different
translators being incompatible, even on the same system.  Which
is what has happened here.

So, damn the outside system, EXACTLY what does Python mean by
such characters, and EXACTLY what uses of them are discouraged
as having unspecified meanings?  If we could get an answer to
that precisely enough to write a parse tree with all terminals
explicit, this problem would go away.

And that is all that I say can or should be done.  The details
of how to write the translators to other file systems are then
a separate matter.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From gjcarneiro at gmail.com  Mon Oct  1 13:10:17 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Mon, 1 Oct 2007 12:10:17 +0100
Subject: [Python-Dev] GC Changes
In-Reply-To: <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
Message-ID: <a467ca4f0710010410q15010b03u15658c7bab20db4b@mail.gmail.com>

On 01/10/2007, Justin Tulloss <tulloss2 at uiuc.edu> wrote:
>
> Hello,
>
> I've been doing some tests on removing the GIL, and it's becoming clear
> that some basic changes to the garbage collector may be needed in order for
> this to happen efficiently. Reference counting as it stands today is not
> very scalable.
>
> I've been looking into a few options, and I'm leaning towards the
> implementing IBMs recycler GC (http://www.research.ibm.com/people/d/dfb/recycler-publications.html )
> since it is very similar to what is in place now from the users'
> perspective. However, I haven't been around the list long enough to really
> understand the feeling in the community on GC in the future of the
> interpreter. It seems that a full GC might have a lot of benefits in terms
> of performance and scalability, and I think that the current gc module is of
> the mark-and-sweep variety. Is the trend going to be to move away from
> reference counting and towards the mark-and-sweep implementation that
> currently exists, or is reference counting a firmly ingrained tradition?
>
> On a more immediately relevant note, I'm not certain I understand the full
> extent of the gc module. From what I've read, it sounds like it's fairly
> close to a fully functional GC, yet it seems to exist only as a
> cycle-detecting backup to the reference counting mechanism. Would somebody
> care to give me a brief overview on how the current gc module interacts with
> the interpreter, or point me to a place where that is done? Why isn't the
> mark-and-sweep mechanism used for all memory management?


The cyclic GC is just too slow to react and makes programmers mad.

For instance, in PyGtk we had a traditional problem with gtk.gdk.Pixbuf,
which is basically an object that wraps a raw RGB image.  When users deleted
such an object, which could sometimes comprise tens or hundreds of
megabytes, the memory was not relased until much much later.  That kind of
code ended up having to manually call gc.collect() to fix what was perceived
by most programmers as a "memory leak", which kind of defeats the purpose of
a garbage collector.  This happened because PyGtk used to rely on the cyclic
GC doing its work.  Thankfully we moved away from that and now simple
reference counting can free a Pixbuf in most cases.

The cyclic GC is a very useful system, but it should only be used in
addition to, not instead of, reference counting.  At least that's my
personal opinion...

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071001/ee5746af/attachment.htm 

From p.f.moore at gmail.com  Mon Oct  1 14:35:28 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 1 Oct 2007 13:35:28 +0100
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <E1IcGcQ-0002Hf-JZ@virgo.cus.cam.ac.uk>
References: <E1IcGcQ-0002Hf-JZ@virgo.cus.cam.ac.uk>
Message-ID: <79990c6b0710010535q678afa0ese30af334e202f8e8@mail.gmail.com>

On 01/10/2007, Nick Maclaren <nmm1 at cus.cam.ac.uk> wrote:
> So, damn the outside system, EXACTLY what does Python mean by
> such characters, and EXACTLY what uses of them are discouraged
> as having unspecified meanings?  If we could get an answer to
> that precisely enough to write a parse tree with all terminals
> explicit, this problem would go away.

Python, the language, means nothing by the characters. They are bytes
with defined values in a byte string (in 2.x, in 3.0 they are Unicode
characters, but otherwise no difference). The *language* places no
interpretation on them.

Certain library functions place an interpretation on the byte values,
but you need to read the function definition for that. And (a) they
may not all be consistent, and (b) they may say "follows platform
behaviour", but that's the way it is, so you have to live with it.

Paul.

From nmm1 at cus.cam.ac.uk  Mon Oct  1 14:49:32 2007
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Mon, 01 Oct 2007 13:49:32 +0100
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
Message-ID: <E1IcKiK-0003Q1-6Q@draco.cus.cam.ac.uk>

"Paul Moore" <p.f.moore at gmail.com> wrote:
> 
> > So, damn the outside system, EXACTLY what does Python mean by
> > such characters, and EXACTLY what uses of them are discouraged
> > as having unspecified meanings?  If we could get an answer to
> > that precisely enough to write a parse tree with all terminals
> > explicit, this problem would go away.
> 
> Python, the language, means nothing by the characters. They are bytes
> with defined values in a byte string (in 2.x, in 3.0 they are Unicode
> characters, but otherwise no difference). The *language* places no
> interpretation on them.

Actually, it's not that simple, because of the "universal newline"
rule and the fact that both Unix/C ASCII and Unicode DO provide
meanings for their characters, but let that pass.  Your statement
is not far off the situation.

> Certain library functions place an interpretation on the byte values,
> but you need to read the function definition for that. And (a) they
> may not all be consistent, and (b) they may say "follows platform
> behaviour", but that's the way it is, so you have to live with it.

And that is why there will continue to be confusion and inconsistency,
and why there will be similar threads to this for the foreseeable
future.  If you regard continuing problems of this sort as acceptable,
then fine, but I am pointing out that they are fairly easy to avoid.
But only by specifying a precise Python model.

Incidentally, the response (b) you give is a common one, but isn't
usually correct when it is given.  It is, after all, the cause of
the problem that started this thread.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From jeremy at alum.mit.edu  Mon Oct  1 15:45:04 2007
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 1 Oct 2007 09:45:04 -0400
Subject: [Python-Dev] GC Changes
In-Reply-To: <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
Message-ID: <e8bf7a530710010645w3b6e99ddj3733c0a5b29145e7@mail.gmail.com>

On 10/1/07, Justin Tulloss <tulloss2 at uiuc.edu> wrote:
> Hello,
>
> I've been doing some tests on removing the GIL, and it's becoming clear that
> some basic changes to the garbage collector may be needed in order for this
> to happen efficiently. Reference counting as it stands today is not very
> scalable.
>
> I've been looking into a few options, and I'm leaning towards the
> implementing IBMs recycler GC (
> http://www.research.ibm.com/people/d/dfb/recycler-publications.html
> ) since it is very similar to what is in place now from the users'
> perspective. However, I haven't been around the list long enough to really
> understand the feeling in the community on GC in the future of the
> interpreter. It seems that a full GC might have a lot of benefits in terms
> of performance and scalability, and I think that the current gc module is of
> the mark-and-sweep variety. Is the trend going to be to move away from
> reference counting and towards the mark-and-sweep implementation that
> currently exists, or is reference counting a firmly ingrained tradition?
>
> On a more immediately relevant note, I'm not certain I understand the full
> extent of the gc module. From what I've read, it sounds like it's fairly
> close to a fully functional GC, yet it seems to exist only as a
> cycle-detecting backup to the reference counting mechanism. Would somebody
> care to give me a brief overview on how the current gc module interacts with
> the interpreter, or point me to a place where that is done? Why isn't the
> mark-and-sweep mechanism used for all memory management?

There are probably lots of interesting things to say about the gc
cycle collector, but I'll just pick a few things that seem relevant.
First off, the gc doesn't have a root set.  It traces all the
container objects, subtracting known references from the refcount, and
is left with a root set, i.e. those objects that have some references
that can't be accounted for among the known container objects.  (see
update_refs and substract_refs)  In the end, we make three traversals
of the heap to detect the objects that appear to be unreachable.
(move_unreachable is the third.)

The cycle detection relies on having the reference counts correct, so
it doesn't really represent a move away from refcounting.

I skipped the generations.  The GC divides the heap into three
generations and tends to focus on the youngest generation.  So that
limits the portion of the heap that is scanned, but I don't understand
the magnitude of that effect in practice.

The current collector works in collaboration with ref counting.  In
particular, refcounting probably handles the majority of
deallocations.  If the cycle detection system were responsible for all
deallocations, the gc module would have a lot more work to do.

I do think it would be interesting to experiment with the recycler
approach, but I think it would take a lot of work to do a decent
experiment.  But please give it a shot!

Jeremy

From arigo at tunes.org  Mon Oct  1 16:10:08 2007
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 1 Oct 2007 16:10:08 +0200
Subject: [Python-Dev] building with -Wwrite-strings
In-Reply-To: <46FD6DA2.1060107@v.loewis.de>
References: <e7ba66e40709271518v1f10cdd4h8643f74048bf0b82@mail.gmail.com>
	<46FD6DA2.1060107@v.loewis.de>
Message-ID: <20071001141007.GA20122@code0.codespeak.net>

Hi Martin,

On Fri, Sep 28, 2007 at 11:09:54PM +0200, "Martin v. L?wis" wrote:
> What's wrong with
> 
> static const char *kwlist[] = {"x", "base", 0};

The following goes wrong if we try again to walk this path:
http://mail.python.org/pipermail/python-dev/2006-February/060689.html


Armin

From dalcinl at gmail.com  Mon Oct  1 17:00:11 2007
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 1 Oct 2007 12:00:11 -0300
Subject: [Python-Dev] building with -Wwrite-strings
In-Reply-To: <20071001141007.GA20122@code0.codespeak.net>
References: <e7ba66e40709271518v1f10cdd4h8643f74048bf0b82@mail.gmail.com>
	<46FD6DA2.1060107@v.loewis.de>
	<20071001141007.GA20122@code0.codespeak.net>
Message-ID: <e7ba66e40710010800l68d546fbke5bbd8721bab132b@mail.gmail.com>

Yes, you are completely right. I ended up realizing that a change like
this would break almost all third-party extension.

But... What about of doing this for Py3K? Third-party extension have
to be fixed anyway.


On 10/1/07, Armin Rigo <arigo at tunes.org> wrote:
> Hi Martin,
>
> On Fri, Sep 28, 2007 at 11:09:54PM +0200, "Martin v. L?wis" wrote:
> > What's wrong with
> >
> > static const char *kwlist[] = {"x", "base", 0};
>
> The following goes wrong if we try again to walk this path:
> http://mail.python.org/pipermail/python-dev/2006-February/060689.html
>
>
> Armin
>


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From steve at holdenweb.com  Mon Oct  1 19:26:57 2007
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 01 Oct 2007 13:26:57 -0400
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <46FE9DB8.9000604@voidspace.org.uk>
References: <E1IbYZl-0000pk-HS@libra.cus.cam.ac.uk>		<ca471dc20709290807l351b87d7k2dd651e5ef18a83c@mail.gmail.com>		<46FE6F92.40601@voidspace.org.uk>
	<fdm5l4$rok$1@sea.gmane.org>		<46FE9B09.8000800@voidspace.org.uk>	<d11dcfba0709291142u1fb3aa68vf81d1125e95e6a90@mail.gmail.com>
	<46FE9DB8.9000604@voidspace.org.uk>
Message-ID: <fdraku$mlm$1@sea.gmane.org>

Michael Foord wrote:
> Steven Bethard wrote:
>> On 9/29/07, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>>   
>>> Terry Reedy wrote:
>>>     
>>>> There are two normal ways for internal Python text to have \r\n:
>>>> 1. Read from a file with \r\r\n.  Then \r\r\n is correct output (on the
>>>> same platform).
>>>> 2. Intentially put there by a programmer.  If s/he also chooses default \n
>>>> translation on output, \r<translation of \n> is correct.
>>>>
>>>>       
>>> Actually, I usually get these strings from Windows UI components. A file
>>> containing '\r\n' is read in with '\r\n' being translated to '\n'. New
>>> user input is added containing '\r\n' line endings. The file is written
>>> out and now contains a mix of '\r\n' and '\r\r\n'.
>>>     
>> Out of curiosity, why don't the Python wrappers for your Windows UI
>> components do the appropriate '\r\n' -> '\n' conversions?
>>   
> 
> One of the great things about IronPython is that you don't *need* any 
> wrappers - you access .NET objects natively (which in fact wrap the 
> lower level win32 API) - and the .NET APIs are usually not as bad as you 
> probably assume. ;-)
> 
This thread might represent an argument that you *do* need wrappers ...

> You just have to be aware that line endings are '\r\n'. I'm not sure how 
> or if pywin32 handles this.
> 
Presumably that awareness should be implemented by the "unnecessary" 
wrappers.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd           http://www.holdenweb.com
Skype: holdenweb      http://del.icio.us/steve.holden

Sorry, the dog ate my .sigline


From fuzzyman at voidspace.org.uk  Mon Oct  1 20:47:28 2007
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 01 Oct 2007 19:47:28 +0100
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <fdraku$mlm$1@sea.gmane.org>
References: <E1IbYZl-0000pk-HS@libra.cus.cam.ac.uk>		<ca471dc20709290807l351b87d7k2dd651e5ef18a83c@mail.gmail.com>		<46FE6F92.40601@voidspace.org.uk>	<fdm5l4$rok$1@sea.gmane.org>		<46FE9B09.8000800@voidspace.org.uk>	<d11dcfba0709291142u1fb3aa68vf81d1125e95e6a90@mail.gmail.com>	<46FE9DB8.9000604@voidspace.org.uk>
	<fdraku$mlm$1@sea.gmane.org>
Message-ID: <470140C0.4010806@voidspace.org.uk>

Steve Holden wrote:
> Michael Foord wrote:
>   
>> Steven Bethard wrote:
>>     
>>> On 9/29/07, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>>>   
>>>       
>>>> Terry Reedy wrote:
>>>>     
>>>>         
>>>>> There are two normal ways for internal Python text to have \r\n:
>>>>> 1. Read from a file with \r\r\n.  Then \r\r\n is correct output (on the
>>>>> same platform).
>>>>> 2. Intentially put there by a programmer.  If s/he also chooses default \n
>>>>> translation on output, \r<translation of \n> is correct.
>>>>>
>>>>>       
>>>>>           
>>>> Actually, I usually get these strings from Windows UI components. A file
>>>> containing '\r\n' is read in with '\r\n' being translated to '\n'. New
>>>> user input is added containing '\r\n' line endings. The file is written
>>>> out and now contains a mix of '\r\n' and '\r\r\n'.
>>>>     
>>>>         
>>> Out of curiosity, why don't the Python wrappers for your Windows UI
>>> components do the appropriate '\r\n' -> '\n' conversions?
>>>   
>>>       
>> One of the great things about IronPython is that you don't *need* any 
>> wrappers - you access .NET objects natively (which in fact wrap the 
>> lower level win32 API) - and the .NET APIs are usually not as bad as you 
>> probably assume. ;-)
>>
>>     
> This thread might represent an argument that you *do* need wrappers ...
>
>   
>> You just have to be aware that line endings are '\r\n'. I'm not sure how 
>> or if pywin32 handles this.
>>
>>     
> Presumably that awareness should be implemented by the "unnecessary" 
> wrappers.
>   

Well, it's an OS level difference and I thought that in general Python 
*doesn't* try to protect you from OS differences.

These different line endings are returned by the components - and making 
the string type aware of where it comes from and transform itself 
accordingly seems odd. It also leaves you with all sorts of other 
problems like string comparison (do you ignore difference in line 
endings?), string length (on different sides of the .NET / IronPython 
strings would report different lengths).

It is also different from how libraries like wxPython behave - where 
they *don't* protect you from OS differences and if a textbox has '\r\n' 
line endings - that is what you get...

Michael
http://www.manning.com/foord


> regards
>   Steve
>   


From janssen at parc.com  Mon Oct  1 20:59:46 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 1 Oct 2007 11:59:46 PDT
Subject: [Python-Dev] OpenSSL httplib bug
In-Reply-To: <6b33c0ac0709301611sde8b92fvfb892550935d7f99@mail.gmail.com> 
References: <6b33c0ac0709301611sde8b92fvfb892550935d7f99@mail.gmail.com>
Message-ID: <07Oct1.115954pdt."57996"@synergy1.parc.xerox.com>

I believe this is already fixed in 2.6 with the new SSL code (I got
the same error writing the unit tests and fixed it).  Thanks for
reporting it, though.

Bill

From janssen at parc.com  Tue Oct  2 01:21:28 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 1 Oct 2007 16:21:28 PDT
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <470140C0.4010806@voidspace.org.uk> 
References: <E1IbYZl-0000pk-HS@libra.cus.cam.ac.uk>
	<ca471dc20709290807l351b87d7k2dd651e5ef18a83c@mail.gmail.com>
	<46FE6F92.40601@voidspace.org.uk> <fdm5l4$rok$1@sea.gmane.org>
	<46FE9B09.8000800@voidspace.org.uk>
	<d11dcfba0709291142u1fb3aa68vf81d1125e95e6a90@mail.gmail.com>
	<46FE9DB8.9000604@voidspace.org.uk> <fdraku$mlm$1@sea.gmane.org>
	<470140C0.4010806@voidspace.org.uk>
Message-ID: <07Oct1.162129pdt."57996"@synergy1.parc.xerox.com>

> Well, it's an OS level difference and I thought that in general Python 
> *doesn't* try to protect you from OS differences.

I think that's the key point.  In general, Python tries to present a
"translucent" interface to the OS in which OS differences can show
through, in contrast to other languages (Java?) which try to present a
complete abstraction of the underlying environment.  This makes Python
in general more useful, thought it also makes it harder to write
portable code in Python, because you have to be aware of the potential
differences (and they aren't particularly well documented -- it's not
clear that they can be).

Bill

From greg.ewing at canterbury.ac.nz  Tue Oct  2 02:10:40 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 02 Oct 2007 13:10:40 +1300
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <ca471dc20709301914u3f972abdpbab870bcf0114d64@mail.gmail.com>
References: <E1IbYZl-0000pk-HS@libra.cus.cam.ac.uk>
	<ca471dc20709290807l351b87d7k2dd651e5ef18a83c@mail.gmail.com>
	<46FE6F92.40601@voidspace.org.uk> <fdm5l4$rok$1@sea.gmane.org>
	<46FE9B09.8000800@voidspace.org.uk>
	<18175.42782.873060.154910@montanaro.dyndns.org>
	<46FFA984.9060602@voidspace.org.uk> <470045DC.5040903@canterbury.ac.nz>
	<ca471dc20709301914u3f972abdpbab870bcf0114d64@mail.gmail.com>
Message-ID: <47018C80.1050007@canterbury.ac.nz>

Guido van Rossum wrote:
> The best solution for IronPython is probably to have the occasional
> wrapper around .NET APIs that translates between \r\n and \n on the
> boundary between Python and .NET;

That's probably true. I was responding to the notion
that IronPython shouldn't need any wrappers. To make
that really true would require IronPython to become
a different language that has a different canonical
representation of newlines.

It's fine with me to keep things as they are.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Tue Oct  2 02:37:43 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 02 Oct 2007 13:37:43 +1300
Subject: [Python-Dev] GC Changes
In-Reply-To: <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
Message-ID: <470192D7.5020504@canterbury.ac.nz>

Justin Tulloss wrote:
> Is the trend going to be to 
> move away from reference counting and towards the mark-and-sweep 
> implementation that currently exists, or is reference counting a firmly 
> ingrained tradition?

It's hard to predict the future, but the general feeling
I get is that many people would like to keep the current
reference counting behaviour, because it has a number of
nice properties:

* It's cache-friendly - it doesn't periodically go
   rampaging through large chunks of memory like
   mark-and-sweep tends to do.

* It tends to reclaim things very promptly. This is
   important in a language like Python that uses
   dynamic allocation extremely heavily, even for
   trivial things like integers. It also helps with
   cacheing.

* It's easy to make it interoperate with external
   libraries that have their own ways of managing
   memory. You don't have to take special steps to
   "protect" things from the garbage collector.

> Would 
> somebody care to give me a brief overview on how the current gc module 
> interacts with the interpreter

The cyclic GC kicks in when memory is running low. Since
the reference counting takes care of any data structures
that don't contain cycles, the GC only has to deal with
cycles. It goes through all currently allocated objects
trying to find sets whose reference counts are all
accounted for by references within the set. Such a set
must constitute a cycle that is not referenced anywhere
from outside. It then picks an arbitrary object from the
set and decrements the reference counts of all the objects
it references. This breaks the cycle, and allows the
reference counting mechanism to reclaim the memory.

Although the cyclic GC requires passing over large
chunks of memory like mark-and-sweep, it happens far less
frequently than would happen if mark-and-sweep were used
for all memory management. Also, the programmer can
minimise the need for it by manually breaking cycles
where they are known to occur.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Tue Oct  2 02:42:29 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 02 Oct 2007 13:42:29 +1300
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <E1IcGcQ-0002Hf-JZ@virgo.cus.cam.ac.uk>
References: <E1IcGcQ-0002Hf-JZ@virgo.cus.cam.ac.uk>
Message-ID: <470193F5.1020309@canterbury.ac.nz>

Nick Maclaren wrote:
> if Python's own
> interpretation is ambiguous, it is a sure recipe for different
> translators being incompatible,

Python's own interpretation is not ambiguous. The
problem at hand is people wanting to use some random
mixture of Python and .NET conventions.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Tue Oct  2 03:07:17 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 02 Oct 2007 14:07:17 +1300
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <470140C0.4010806@voidspace.org.uk>
References: <E1IbYZl-0000pk-HS@libra.cus.cam.ac.uk>
	<ca471dc20709290807l351b87d7k2dd651e5ef18a83c@mail.gmail.com>
	<46FE6F92.40601@voidspace.org.uk> <fdm5l4$rok$1@sea.gmane.org>
	<46FE9B09.8000800@voidspace.org.uk>
	<d11dcfba0709291142u1fb3aa68vf81d1125e95e6a90@mail.gmail.com>
	<46FE9DB8.9000604@voidspace.org.uk> <fdraku$mlm$1@sea.gmane.org>
	<470140C0.4010806@voidspace.org.uk>
Message-ID: <470199C5.4050605@canterbury.ac.nz>

Michael Foord wrote:
> It is also different from how libraries like wxPython behave - where 
> they *don't* protect you from OS differences and if a textbox has '\r\n' 
> line endings - that is what you get...

That sounds like an undesirable deficiency of those library
wrappers, especially cross-platform ones like wxPython.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From tulloss2 at uiuc.edu  Tue Oct  2 03:14:04 2007
From: tulloss2 at uiuc.edu (Justin Tulloss)
Date: Mon, 1 Oct 2007 20:14:04 -0500
Subject: [Python-Dev] GC Changes
In-Reply-To: <470192D7.5020504@canterbury.ac.nz>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
Message-ID: <2cfeb93c0710011814m2544e2b4yf378b9199aaee7cc@mail.gmail.com>

> The cyclic GC kicks in when memory is running low.


When what memory is running low? Its default pool? System memory?

Justin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071001/c79b64f6/attachment.htm 

From rhamph at gmail.com  Tue Oct  2 03:18:18 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 1 Oct 2007 19:18:18 -0600
Subject: [Python-Dev] GC Changes
In-Reply-To: <470192D7.5020504@canterbury.ac.nz>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
Message-ID: <aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>

On 10/1/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Justin Tulloss wrote:
> > Would
> > somebody care to give me a brief overview on how the current gc module
> > interacts with the interpreter
>
> The cyclic GC kicks in when memory is running low. Since

This isn't true at all.  It's triggered by heuristics based on the
total number of allocated objects.  It doesn't know how much memory is
available and is not called if an allocation fails.


-- 
Adam Olsen, aka Rhamphoryncus

From greg.ewing at canterbury.ac.nz  Tue Oct  2 03:31:10 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 02 Oct 2007 14:31:10 +1300
Subject: [Python-Dev] GC Changes
In-Reply-To: <2cfeb93c0710011814m2544e2b4yf378b9199aaee7cc@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<2cfeb93c0710011814m2544e2b4yf378b9199aaee7cc@mail.gmail.com>
Message-ID: <47019F5E.3090506@canterbury.ac.nz>

Justin Tulloss wrote:

> When what memory is running low? Its default pool? System memory?

I'm not sure of the details, but I think it keeps
a high-water mark of the amount of memory allocated
for Python objects so far. When that is reached, it
tries to free up memory by cyclic GC, and only
mallocs more if that fails.

I think it also counts the number of allocations
made since the last GC and does a GC when it gets
up to some threshold, so that things get cleaned
out periodically and the processing is spread
out somewhat.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From greg.ewing at canterbury.ac.nz  Tue Oct  2 03:33:25 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 02 Oct 2007 14:33:25 +1300
Subject: [Python-Dev] GC Changes
In-Reply-To: <aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
Message-ID: <47019FE5.3070703@canterbury.ac.nz>

Adam Olsen wrote:
> This isn't true at all.  It's triggered by heuristics based on the
> total number of allocated objects.

Hmmm, all right, it seems I don't know what I'm
talking about. I'll shut up now before I spread
any more misinformation. Sorry.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From rhamph at gmail.com  Tue Oct  2 03:42:14 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 1 Oct 2007 19:42:14 -0600
Subject: [Python-Dev] GC Changes
In-Reply-To: <47019FE5.3070703@canterbury.ac.nz>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
	<47019FE5.3070703@canterbury.ac.nz>
Message-ID: <aac2c7cb0710011842w2f99c215u7af115c03b599636@mail.gmail.com>

On 10/1/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Adam Olsen wrote:
> > This isn't true at all.  It's triggered by heuristics based on the
> > total number of allocated objects.
>
> Hmmm, all right, it seems I don't know what I'm
> talking about. I'll shut up now before I spread
> any more misinformation. Sorry.

Hey, no worries.  I half expect someone to correct me. ;)

-- 
Adam Olsen, aka Rhamphoryncus

From aahz at pythoncraft.com  Tue Oct  2 04:47:45 2007
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 1 Oct 2007 19:47:45 -0700
Subject: [Python-Dev] GC Changes
In-Reply-To: <470192D7.5020504@canterbury.ac.nz>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
Message-ID: <20071002024745.GA21172@panix.com>

[xposted to python-ideas, reply-to python-ideas, leaving python-dev in
to correct misinformation]

On Tue, Oct 02, 2007, Greg Ewing wrote:
>
> The cyclic GC kicks in when memory is running low. 

Not at all.  The sole and only basis for GC is number of allocations
compared to number of de-allocations.  See
http://docs.python.org/lib/module-gc.html
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.

From tjreedy at udel.edu  Tue Oct  2 05:20:15 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 1 Oct 2007 23:20:15 -0400
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
References: <E1IcGcQ-0002Hf-JZ@virgo.cus.cam.ac.uk>
Message-ID: <fdsddf$399$1@sea.gmane.org>


"Nick Maclaren" <nmm1 at cus.cam.ac.uk> wrote in message 
news:E1IcGcQ-0002Hf-JZ at virgo.cus.cam.ac.uk...
| The question is independent of what the outside system believes a
| text file should look like, and is solely what Python believes a
| sequence of characters should mean.  For example, does 'A\r\nB'
| mean that B is separated from A by one newline or two?

The grammar presupposes that Python code is divided into lines.  Any 
successful interpreter must adjust to the external source's idea of line 
endings.  This is implementation, not language definition.

The grammar itself has no notion of structure within Python string objects. 
The split method lets one define anything as chunk separators.

The builtin compile method that uses strings as code input specifies \n and 
only \n as a line ending.  The universal line-ending model of string output 
to files does the same.  So from either viewpoint, the unambiguous answer 
to your question is 'one'.

Terry Jan Reedy




From guido at python.org  Tue Oct  2 05:35:36 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 1 Oct 2007 20:35:36 -0700
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
In-Reply-To: <fdsddf$399$1@sea.gmane.org>
References: <E1IcGcQ-0002Hf-JZ@virgo.cus.cam.ac.uk>
	<fdsddf$399$1@sea.gmane.org>
Message-ID: <ca471dc20710012035g5a23304t61fa632b37c4276c@mail.gmail.com>

Does anyone else have the feeling that discussions with Mr. MacLaren
don't usually bear any fruit?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nmm1 at cus.cam.ac.uk  Tue Oct  2 11:28:43 2007
From: nmm1 at cus.cam.ac.uk (Nick Maclaren)
Date: Tue, 02 Oct 2007 10:28:43 +0100
Subject: [Python-Dev] [python] Re: New lines, carriage returns,
	and Windows
Message-ID: <E1Ice3X-00069a-Qh@virgo.cus.cam.ac.uk>

"Guido van Rossum" <guido at python.org> wrote:
>
> Does anyone else have the feeling that discussions with Mr. MacLaren
> don't usually bear any fruit?

Yes.  I do.  My ability to predict the (technical) future is good;
my ability to persuade people of it is almost non-existent.

However, when an almost identical thread to this one occurs in a
decade's time, I shall be safely retired.  And, assuming no changes
to the basic models, when one occurs in twenty years' time, I shall
be forgotten.  Such is life.


Regards,
Nick Maclaren,
University of Cambridge Computing Service,
New Museums Site, Pembroke Street, Cambridge CB2 3QH, England.
Email:  nmm1 at cam.ac.uk
Tel.:  +44 1223 334761    Fax:  +44 1223 334679

From gjcarneiro at gmail.com  Tue Oct  2 11:50:36 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Tue, 2 Oct 2007 10:50:36 +0100
Subject: [Python-Dev] GC Changes
In-Reply-To: <aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
Message-ID: <a467ca4f0710020250g4fc97b68v5e5616c645b2cffd@mail.gmail.com>

On 02/10/2007, Adam Olsen <rhamph at gmail.com> wrote:
>
> On 10/1/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> > Justin Tulloss wrote:
> > > Would
> > > somebody care to give me a brief overview on how the current gc module
> > > interacts with the interpreter
> >
> > The cyclic GC kicks in when memory is running low. Since
>
> This isn't true at all.  It's triggered by heuristics based on the
> total number of allocated objects.  It doesn't know how much memory is
> available and is not called if an allocation fails.


Correct.  And that reminds me of the limitation of the the Python GC: it
doesn't take into account how much memory is being indirectly retained by a
Python Object.  Like in the example I already gave, gtk.gdk.Pixbuf can
easily hold hundreds of megabytes, yet the GC gives it as much consideration
as it does to a simple python integer object which is several orders of
magnitude smaller.

If someone wanted to improve the GC, that person should consider adding a
protocol for the GC to retrieve the ammount of memory indirectly held by a
python object, and take such memory into consideration in its heuristics.

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071002/38ad21d7/attachment.htm 

From hrvoje.niksic at avl.com  Tue Oct  2 12:24:11 2007
From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=)
Date: Tue, 02 Oct 2007 12:24:11 +0200
Subject: [Python-Dev] GC Changes
In-Reply-To: <a467ca4f0710020250g4fc97b68v5e5616c645b2cffd@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
	<a467ca4f0710020250g4fc97b68v5e5616c645b2cffd@mail.gmail.com>
Message-ID: <1191320651.11322.416.camel@localhost>

On Tue, 2007-10-02 at 10:50 +0100, Gustavo Carneiro wrote:
> Correct.  And that reminds me of the limitation of the the Python GC:
> it doesn't take into account how much memory is being indirectly
> retained by a Python Object.  Like in the example I already gave,
> gtk.gdk.Pixbuf can easily hold hundreds of megabytes, yet the GC gives
> it as much consideration as it does to a simple python integer object
> which is several orders of magnitude smaller.

That sounds like a case for the Pixbuf object to have a "close" method
(not necessarily called that) that releases the resources.  The point of
GC is that you normally don't care if memory is released sooner or
later; for stuff you do care about, such as files, shared memory, or
even large memory objects, there is always explicit management.
cStringIO's "close" method provides a precedent.




From gjcarneiro at gmail.com  Tue Oct  2 12:30:35 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Tue, 2 Oct 2007 11:30:35 +0100
Subject: [Python-Dev] GC Changes
In-Reply-To: <1191320651.11322.416.camel@localhost>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
	<a467ca4f0710020250g4fc97b68v5e5616c645b2cffd@mail.gmail.com>
	<1191320651.11322.416.camel@localhost>
Message-ID: <a467ca4f0710020330s5b805943t582d663c47571a67@mail.gmail.com>

On 02/10/2007, Hrvoje Nik?i? <hrvoje.niksic at avl.com> wrote:
>
> On Tue, 2007-10-02 at 10:50 +0100, Gustavo Carneiro wrote:
> > Correct.  And that reminds me of the limitation of the the Python GC:
> > it doesn't take into account how much memory is being indirectly
> > retained by a Python Object.  Like in the example I already gave,
> > gtk.gdk.Pixbuf can easily hold hundreds of megabytes, yet the GC gives
> > it as much consideration as it does to a simple python integer object
> > which is several orders of magnitude smaller.
>
> That sounds like a case for the Pixbuf object to have a "close" method
> (not necessarily called that) that releases the resources.  The point of
> GC is that you normally don't care if memory is released sooner or
> later; for stuff you do care about, such as files, shared memory, or
> even large memory objects, there is always explicit management.
> cStringIO's "close" method provides a precedent.


I think close in real files is needed  not so much because you want to free
memory, but that you want to prevent data loss by flushing the file buffer
into the actual file at a certain point in time.  And I suspect that
cStringIO just added a close() method for compatibility with real files.
But of course I speculating here...

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071002/097cd4ac/attachment.htm 

From hrvoje.niksic at avl.com  Tue Oct  2 12:58:50 2007
From: hrvoje.niksic at avl.com (Hrvoje =?UTF-8?Q?Nik=C5=A1i=C4=87?=)
Date: Tue, 02 Oct 2007 12:58:50 +0200
Subject: [Python-Dev] GC Changes
In-Reply-To: <a467ca4f0710020330s5b805943t582d663c47571a67@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
	<a467ca4f0710020250g4fc97b68v5e5616c645b2cffd@mail.gmail.com>
	<1191320651.11322.416.camel@localhost>
	<a467ca4f0710020330s5b805943t582d663c47571a67@mail.gmail.com>
Message-ID: <1191322730.11322.436.camel@localhost>

On Tue, 2007-10-02 at 11:30 +0100, Gustavo Carneiro wrote:
>         even large memory objects, there is always explicit
>         management.
>         cStringIO's "close" method provides a precedent.
> 
> I think close in real files is needed  not so much because you want to
> free memory, but that you want to prevent data loss by flushing the
> file buffer into the actual file at a certain point in time.

You can also do that with flush.  What is specific for close is that it
frees up the system resources occupied by the open file.  Calling it at
a known point in time ensures that these external resources aren't
occupied any longer than necessary, regardless of the object
deallocator's policies.  That is why open(filename).read() is
discouraged, despite being a useful idiom in throwaway scripts and
working just fine in CPython.

The fact that programmers need Pixbuf's memory released sooner rather
than later might indicate that Pixbuf would benefit from a "close"-like
method.  In any case, I trust your judgment on Pixbufs, but merely point
out that in general it is unwise to rely on the deallocator to release
resources where timing of the release matters.

>   And I suspect that cStringIO just added a close() method for
> compatibility with real files.  But of course I speculating here... 

That could be the case, but then the method could also be a no-op.  I
only brought up cStringIO as a precedent for a close method that works
with a memory-only object.  (StringIO's close also tries to arrange for
the buffer to be freed immediately, inasmuch as that is possible to
achieve in pure Python.)




From greg at krypto.org  Tue Oct  2 18:25:17 2007
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 2 Oct 2007 09:25:17 -0700
Subject: [Python-Dev] GC Changes
In-Reply-To: <1191320651.11322.416.camel@localhost>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
	<a467ca4f0710020250g4fc97b68v5e5616c645b2cffd@mail.gmail.com>
	<1191320651.11322.416.camel@localhost>
Message-ID: <52dc1c820710020925y3aa07c2x952736baf032af00@mail.gmail.com>

On 10/2/07, Hrvoje Nik?i? <hrvoje.niksic at avl.com> wrote:
>
> On Tue, 2007-10-02 at 10:50 +0100, Gustavo Carneiro wrote:
> > Correct.  And that reminds me of the limitation of the the Python GC:
> > it doesn't take into account how much memory is being indirectly
> > retained by a Python Object.  Like in the example I already gave,
> > gtk.gdk.Pixbuf can easily hold hundreds of megabytes, yet the GC gives
> > it as much consideration as it does to a simple python integer object
> > which is several orders of magnitude smaller.
>
> That sounds like a case for the Pixbuf object to have a "close" method
> (not necessarily called that) that releases the resources.  The point of
> GC is that you normally don't care if memory is released sooner or
> later; for stuff you do care about, such as files, shared memory, or
> even large memory objects, there is always explicit management.
> cStringIO's "close" method provides a precedent.


Agreed, objects that consume a lot of resources of any type should have a
method to clean themselves up or the language should take special notice of
'del myObject' and call the destructor immediately if nothing else refers to
the object at that point rather than leaving it up to a GC.  Its just good
programming practice to do such things on "large" objects regardless of the
underlying allocation implementation.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071002/4e55dcd5/attachment.htm 

From facundobatista at gmail.com  Tue Oct  2 18:26:48 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 2 Oct 2007 13:26:48 -0300
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <46F1B6FD.70007@ronadam.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>
	<e04bdf310709191341l289c034fjb4bf6d6b88ae4ef9@mail.gmail.com>
	<46F1B6FD.70007@ronadam.com>
Message-ID: <e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>

2007/9/19, Ron Adam <rrr at ronadam.com>:

> I noticed that there is a background of light blue between marks.  That is
> hard to see on my computer because it is so close to the grey tone.

Made it a little darker, now it's easier to look.


> Also shouldn't the light blue background bar extend all the way to the end
> for all open items?

No, because this "light blue bar" is the span of time from opened to
last comment.

Note that these items are *all* open.

Thanks for the feedback!

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From facundobatista at gmail.com  Tue Oct  2 19:56:40 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 2 Oct 2007 14:56:40 -0300
Subject: [Python-Dev] Backporting Decimal
Message-ID: <e04bdf310710021056pfc29083j6be252a1276f4ed6@mail.gmail.com>

People:

I don't decide myself what to do in this case.

The Decimal module appeared in 2.4, and received just slight
modifications for 2.5. Since it appeared, a "just download and use it"
version was available for Python 2.3 users.

But for 2.6, it was fully renewed. Not only was updated to the last
spec from Cawlishaw, but also a lot of (very slightly, arithmetic
corner cases) bugs were fixed. Also, it received a lot of code
simplifications and speedups. All this is, in most part, because Mark
Dickinson was involved here, and he did (and is doing) a great work.

When 2.6 is out, I'll update that downloadable for Py2.3. But what to
do with Py2.5?

Decimal is a pretty stand alone module, and I'm absolutely sure that
just backporting the whole module and its testcases will fix a lot of
problems, and Py2.5 users will have new functionality, but is this ok?
(of course, I can not be 100% sure that any bugs were introduced in
these modifications).

Recommendations?

Thank you very much!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From python at rcn.com  Tue Oct  2 20:32:50 2007
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 2 Oct 2007 11:32:50 -0700
Subject: [Python-Dev] Backporting Decimal
References: <e04bdf310710021056pfc29083j6be252a1276f4ed6@mail.gmail.com>
Message-ID: <002101c80522$a4dbe300$69196b0a@RaymondLaptop1>

> Decimal is a pretty stand alone module, and I'm absolutely sure that
> just backporting the whole module and its testcases will fix a lot of
> problems, and Py2.5 users will have new functionality, but is this ok?

Yes!  We have guaranteed that spec updates are to be treated as bug fixes and backported.  This is especially important in this case 
because other errors have been fixed and the test cases have grown.

Raymond

From facundobatista at gmail.com  Tue Oct  2 20:56:06 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 2 Oct 2007 15:56:06 -0300
Subject: [Python-Dev] Backporting Decimal
In-Reply-To: <002101c80522$a4dbe300$69196b0a@RaymondLaptop1>
References: <e04bdf310710021056pfc29083j6be252a1276f4ed6@mail.gmail.com>
	<002101c80522$a4dbe300$69196b0a@RaymondLaptop1>
Message-ID: <e04bdf310710021156i1aed6b09xc3985415d70c27a@mail.gmail.com>

2007/10/2, Raymond Hettinger <python at rcn.com>:

> Yes!  We have guaranteed that spec updates are to be treated as bug fixes and backported.  This is especially important in this case
> because other errors have been fixed and the test cases have grown.

Perfect! I'll backport it to 2.5... what about 2.4?

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From python at rcn.com  Tue Oct  2 21:30:51 2007
From: python at rcn.com (Raymond Hettinger)
Date: Tue, 2 Oct 2007 12:30:51 -0700
Subject: [Python-Dev] Backporting Decimal
References: <e04bdf310710021056pfc29083j6be252a1276f4ed6@mail.gmail.com>
	<002101c80522$a4dbe300$69196b0a@RaymondLaptop1>
	<e04bdf310710021156i1aed6b09xc3985415d70c27a@mail.gmail.com>
Message-ID: <003d01c8052a$be8f8060$69196b0a@RaymondLaptop1>

>> Yes!  We have guaranteed that spec updates are to be treated as bug fixes and backported.  This is especially important in this 
>> case
>> because other errors have been fixed and the test cases have grown.
>
> Perfect! I'll backport it to 2.5... what about 2.4?

If there are any plans for another 2.4 release, then yes; otherwise, why bother.


Raymond

From martin at v.loewis.de  Tue Oct  2 21:54:14 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 02 Oct 2007 21:54:14 +0200
Subject: [Python-Dev] GC Changes
In-Reply-To: <2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
Message-ID: <4702A1E6.2010001@v.loewis.de>

Jeremy covered already most of it, so I'll only address specific points:

> and I think that the current gc
> module is of the mark-and-sweep variety.

That is incorrect. It's two phases, but in the first phase, it isn't
"mark", but "count", and in the second phase, it's not "sweep" but
"break". To "count" means to count the number of references to an
object, and to "break" means to break the cycles at selected places,
expecting that reference counting will actually cause objects to
become released.

> Is the trend going to be to
> move away from reference counting and towards the mark-and-sweep
> implementation that currently exists, or is reference counting a firmly
> ingrained tradition?

For the CPython implementation, there is no trend. The language spec
explicitly states that automatic memory management is unspecified
and implementation-defined, yet people rely on reference counting in
many programs. That is not the reason to kick it out, though - it
stays around because a true mark-and-sweep algorithm cannot be
implemented on top of the current object API (as Jeremy explains,
there is no notion of root objects in the VM).

> On a more immediately relevant note, I'm not certain I understand the
> full extent of the gc module. From what I've read, it sounds like it's
> fairly close to a fully functional GC, yet it seems to exist only as a
> cycle-detecting backup to the reference counting mechanism. Would
> somebody care to give me a brief overview on how the current gc module
> interacts with the interpreter, or point me to a place where that is
> done?

What is "this" that is done? The collector? I think you found
Modules/gcmodule.c already. Take particular notice of the tp_traverse
and tp_clear type methods.

> Why isn't the mark-and-sweep mechanism used for all memory
> management?

See above - it's not implementable, because the root objects get not
tracked.

Regards,
Martin

From martin at v.loewis.de  Tue Oct  2 21:56:55 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 02 Oct 2007 21:56:55 +0200
Subject: [Python-Dev] Backporting Decimal
In-Reply-To: <003d01c8052a$be8f8060$69196b0a@RaymondLaptop1>
References: <e04bdf310710021056pfc29083j6be252a1276f4ed6@mail.gmail.com>	<002101c80522$a4dbe300$69196b0a@RaymondLaptop1>	<e04bdf310710021156i1aed6b09xc3985415d70c27a@mail.gmail.com>
	<003d01c8052a$be8f8060$69196b0a@RaymondLaptop1>
Message-ID: <4702A287.7020908@v.loewis.de>

> If there are any plans for another 2.4 release, then yes; otherwise, why bother.

Please don't make any functional changes to 2.4. There may be additional
releases, but (IMO) they should be security releases only, i.e. contain
no functional changes whatsoever (unless they fix a security issue).

I'm still trying to work on the PEP that formalizes this notion.

Regards,
Martin

From dickinsm at gmail.com  Tue Oct  2 22:34:08 2007
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 2 Oct 2007 16:34:08 -0400
Subject: [Python-Dev] Backporting Decimal
In-Reply-To: <e04bdf310710021156i1aed6b09xc3985415d70c27a@mail.gmail.com>
References: <e04bdf310710021056pfc29083j6be252a1276f4ed6@mail.gmail.com>
	<002101c80522$a4dbe300$69196b0a@RaymondLaptop1>
	<e04bdf310710021156i1aed6b09xc3985415d70c27a@mail.gmail.com>
Message-ID: <5c6f2a5d0710021334o4ba282e9h834b2c62e4986d19@mail.gmail.com>

On 10/2/07, Facundo Batista <facundobatista at gmail.com> wrote:
>
> 2007/10/2, Raymond Hettinger <python at rcn.com>:
>
> > Yes!  We have guaranteed that spec updates are to be treated as bug
> fixes and backported.  This is especially important in this case
> > because other errors have been fixed and the test cases have grown.
>
> Perfect! I'll backport it to 2.5... what about 2.4?


There's one potential breakage that I see here:  the Decimal.__pow__
function has different semantics in the new version of Decimal (nonintegers
are now allowed as exponents; some previous restrictions on argument sizes
no longer exist), so backporting the new version of __pow__ might cause
difficulties.  In particular, some cases of three-argument pow that
previously worked (giving arguably nonsensical results) will now raise an
exception.  To be honest, I'd be quite surprised to find that *anyone* was
using the three-argument pow, but perhaps we should be careful here?

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071002/57339aa9/attachment.htm 

From nas at arctrix.com  Tue Oct  2 23:43:23 2007
From: nas at arctrix.com (Neil Schemenauer)
Date: Tue, 2 Oct 2007 21:43:23 +0000 (UTC)
Subject: [Python-Dev] GC Changes
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<4702A1E6.2010001@v.loewis.de>
Message-ID: <fdue1r$c9v$1@sea.gmane.org>

Martin v. L?wis <martin at v.loewis.de> wrote:
>> Why isn't the mark-and-sweep mechanism used for all memory
>> management?
>
> See above - it's not implementable, because the root objects get not
> tracked.

To further elaborate, the main obstacle is with extension modules.
Most of them create roots and there is no defined API for the Python
interpreter to find them.

Perhaps a more serious problem is that CPython and 3rd party
extension modules rely heavily on the fact that the GC does not move
objects.  There are GC strategies that perform well when faced with
high object allocation rates but, AFAIK, all of them rely on moving
objects.

  Neil


From facundobatista at gmail.com  Wed Oct  3 00:47:53 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 2 Oct 2007 19:47:53 -0300
Subject: [Python-Dev] Backporting Decimal
In-Reply-To: <5c6f2a5d0710021334o4ba282e9h834b2c62e4986d19@mail.gmail.com>
References: <e04bdf310710021056pfc29083j6be252a1276f4ed6@mail.gmail.com>
	<002101c80522$a4dbe300$69196b0a@RaymondLaptop1>
	<e04bdf310710021156i1aed6b09xc3985415d70c27a@mail.gmail.com>
	<5c6f2a5d0710021334o4ba282e9h834b2c62e4986d19@mail.gmail.com>
Message-ID: <e04bdf310710021547r5e4b4d67p867cdfe5852815d7@mail.gmail.com>

2007/10/2, Mark Dickinson <dickinsm at gmail.com>:

> difficulties.  In particular, some cases of three-argument pow that
> previously worked (giving arguably nonsensical results) will now raise an
> exception.  To be honest, I'd be quite surprised to find that *anyone* was

If previously it gave a arguably nonsensical results, and now it
raises an exception, I consider it a fixed bug, :)

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From greg.ewing at canterbury.ac.nz  Wed Oct  3 01:11:09 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 03 Oct 2007 11:11:09 +1200
Subject: [Python-Dev] GC Changes
In-Reply-To: <1191320651.11322.416.camel@localhost>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<470192D7.5020504@canterbury.ac.nz>
	<aac2c7cb0710011818j28cab3ecj96554fa575072ef2@mail.gmail.com>
	<a467ca4f0710020250g4fc97b68v5e5616c645b2cffd@mail.gmail.com>
	<1191320651.11322.416.camel@localhost>
Message-ID: <4702D00D.4020805@canterbury.ac.nz>

Hrvoje Nik?i? wrote:
> That sounds like a case for the Pixbuf object to have a "close" method
> (not necessarily called that) that releases the resources.  The point of
> GC is that you normally don't care if memory is released sooner or
> later;

I think the problem here is that the GC's lack of knowledge
about how much memory is being used means that you need to care
more than you should have to.

--
Greg


From martin at v.loewis.de  Wed Oct  3 08:27:04 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 03 Oct 2007 08:27:04 +0200
Subject: [Python-Dev] GC Changes
In-Reply-To: <fdue1r$c9v$1@sea.gmane.org>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>	<4702A1E6.2010001@v.loewis.de>
	<fdue1r$c9v$1@sea.gmane.org>
Message-ID: <47033638.7010202@v.loewis.de>

> To further elaborate, the main obstacle is with extension modules.
> Most of them create roots and there is no defined API for the Python
> interpreter to find them.

That is a problem, but furthermore, I feel that local variables stored
in stack frames of threads are even more difficult to integrate. For
the extension-module globals, some sort of registration could be done
(and PEP 3121 provides an infrastructure for that). For stack frames,
such a registration is difficult to make efficient.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Thu Oct  4 00:31:01 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 04 Oct 2007 10:31:01 +1200
Subject: [Python-Dev] GC Changes
In-Reply-To: <47033638.7010202@v.loewis.de>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<4702A1E6.2010001@v.loewis.de> <fdue1r$c9v$1@sea.gmane.org>
	<47033638.7010202@v.loewis.de>
Message-ID: <47041825.7020300@canterbury.ac.nz>

Martin v. L?wis wrote:
> For stack frames,
> such a registration is difficult to make efficient.

Also very error-prone if you happen to miss one. Although
maybe no more error-prone than getting the reference
counting right.

--
Greg

From draghuram at gmail.com  Thu Oct  4 16:34:27 2007
From: draghuram at gmail.com (Raghuram Devarakonda)
Date: Thu, 4 Oct 2007 10:34:27 -0400
Subject: [Python-Dev] Returning exception stack trace in SimpleXMLRPCServer
	response
Message-ID: <2c51ecee0710040734q27d17837m253f0fa1d8ef4b75@mail.gmail.com>

Hi,

When an exception occurs in a function, SimpleXMLRPCServer currently sends
exception type and value back to the client (in <faultString>) . So the
client sees some thing like:

"'<type \'exceptions.ValueError\'>:invalid literal for int() with base 10"

This string is constructed in _marshaled_dispatch() as ""%s:%s" % (exc_type,
exc_value)". How about adding the stack traceback to this string? I found it
very useful to have the server side traceback in case of failures. I will
come up with a patch if there is any interest.

Some thing similar has been done in case of internal server errors (in
r57158).

Thanks,
Raghu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071004/c9a8b4fc/attachment.htm 

From janssen at parc.com  Thu Oct  4 20:57:48 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 4 Oct 2007 11:57:48 PDT
Subject: [Python-Dev] Windows builds for backported SSL module, please?
Message-ID: <07Oct4.115757pdt."57996"@synergy1.parc.xerox.com>

I've uploaded the source version of the backported SSL module to the
Cheeseshop.  It would be nice to include Windows builds for it for
Python 2.3, 2.4, and 2.5, as well, and I'd be happy to upload them if
someone could create them.

http://pypi.python.org/pypi?name=ssl&version=1.4&:action=display

Now, for the Python 3K version...

Bill

From janssen at parc.com  Fri Oct  5 01:30:20 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 4 Oct 2007 16:30:20 PDT
Subject: [Python-Dev] Windows builds for backported SSL module, please?
In-Reply-To: <07Oct4.115757pdt."57996"@synergy1.parc.xerox.com> 
References: <07Oct4.115757pdt."57996"@synergy1.parc.xerox.com>
Message-ID: <07Oct4.163028pdt."57996"@synergy1.parc.xerox.com>

I found a bug -- setup.py wasn't detecting the difference between
2.5.0 and 2.5.1 properly.  I've updated the release to 1.5.

> http://pypi.python.org/pypi?name=ssl&version=1.5&

Bill



From fdrake at acm.org  Fri Oct  5 05:23:12 2007
From: fdrake at acm.org (Fred Drake)
Date: Thu, 4 Oct 2007 23:23:12 -0400
Subject: [Python-Dev] Removing hotshot profiler?
Message-ID: <5E900C95-DF1D-496E-91FA-1FC48BBAB087@acm.org>

I vaguely recall some discussion about removing hotshot in favor of a  
better maintained profiler that has better thread support as well.   
Does anyone remember the decision?  I don't see anything about it in  
PEP 4 or the Python 3000 PEPs I checked.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>




From brett at python.org  Fri Oct  5 06:07:25 2007
From: brett at python.org (Brett Cannon)
Date: Thu, 4 Oct 2007 21:07:25 -0700
Subject: [Python-Dev] Removing hotshot profiler?
In-Reply-To: <5E900C95-DF1D-496E-91FA-1FC48BBAB087@acm.org>
References: <5E900C95-DF1D-496E-91FA-1FC48BBAB087@acm.org>
Message-ID: <bbaeab100710042107g5a973338k3798282d2c75f461@mail.gmail.com>

On 10/4/07, Fred Drake <fdrake at acm.org> wrote:
> I vaguely recall some discussion about removing hotshot in favor of a
> better maintained profiler that has better thread support as well.
> Does anyone remember the decision?  I don't see anything about it in
> PEP 4 or the Python 3000 PEPs I checked.

There was discussion, but I don't think a specific resolution was
reached (could be wrong).

-Brett

From skip at pobox.com  Fri Oct  5 12:02:58 2007
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 5 Oct 2007 05:02:58 -0500
Subject: [Python-Dev] Removing hotshot profiler?
In-Reply-To: <5E900C95-DF1D-496E-91FA-1FC48BBAB087@acm.org>
References: <5E900C95-DF1D-496E-91FA-1FC48BBAB087@acm.org>
Message-ID: <18182.3026.343975.979095@montanaro.dyndns.org>

Fred,

Search the python-3000 archives for a subject started by Neal Norwitz in
late August with a subject of "what to do with profilers in the stdlib".  I
was the one who suggested removing hotshot (Armin's cprofile covers
basically the same ground).

Skip

From p.f.moore at gmail.com  Fri Oct  5 12:22:30 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 5 Oct 2007 11:22:30 +0100
Subject: [Python-Dev] Windows builds for backported SSL module, please?
In-Reply-To: <-7691212560301951594@unknownmsgid>
References: <-7691212560301951594@unknownmsgid>
Message-ID: <79990c6b0710050322v221f29c9i77f3b629cea57661@mail.gmail.com>

On 05/10/2007, Bill Janssen <janssen at parc.com> wrote:
> I found a bug -- setup.py wasn't detecting the difference between
> 2.5.0 and 2.5.1 properly.  I've updated the release to 1.5.
>
> > http://pypi.python.org/pypi?name=ssl&version=1.5&

I'll do Windows binaries for you. The URL you gave is broken, though,
and I can't find the package on PyPI... :-(

Paul.

From janssen at parc.com  Fri Oct  5 18:56:52 2007
From: janssen at parc.com (Bill Janssen)
Date: Fri, 5 Oct 2007 09:56:52 PDT
Subject: [Python-Dev] Windows builds for backported SSL module, please?
In-Reply-To: <79990c6b0710050322v221f29c9i77f3b629cea57661@mail.gmail.com> 
References: <-7691212560301951594@unknownmsgid>
	<79990c6b0710050322v221f29c9i77f3b629cea57661@mail.gmail.com>
Message-ID: <07Oct5.095654pdt."57996"@synergy1.parc.xerox.com>

For some reason, it was set to be 'hidden'.

Try 

  http://pypi.python.org/pypi/ssl/1.5

Bill

From status at bugs.python.org  Fri Oct  5 19:37:00 2007
From: status at bugs.python.org (Tracker)
Date: Fri,  5 Oct 2007 17:37:00 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071005173700.0665C782DC@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (09/28/07 - 10/05/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1291 open (+18) / 11432 closed ( +3) / 12723 total (+21)

Open issues with patches:   413

Average duration of open issues: 680 days.
Median duration of open issues: 705 days.

Open Issues Breakdown
   open  1286 (+18)
pending     5 ( +0)

Issues Created Or Reopened (21)
_______________________________

popen3 website documentation inconsistency                       09/30/07
       http://bugs.python.org/issue1220    created  jbronn                   
                                                                               

email.Utils.parseaddr("a(WRONG)@b")                              09/30/07
       http://bugs.python.org/issue1221    created  helmut                   
                                                                               

locale.format bug if thousand separator is space (french separat 09/30/07
       http://bugs.python.org/issue1222    created  edlm10                   
                                                                               

httplib does not handle ssl end of file properly                 10/01/07
       http://bugs.python.org/issue1223    created  Richie                   
       patch                                                                   

SimpleHTTPServer doesn't understand // at beginning of path anym 10/01/07
       http://bugs.python.org/issue1224    created  philfr                   
                                                                               

IDLE - Fix: pressing Ctrl+C while printing exception -> stuck    10/01/07
       http://bugs.python.org/issue1225    created  taleinat                 
                                                                               

lib/sched.py superfluous code for removal                        10/01/07
       http://bugs.python.org/issue1226    created  lorph                    
                                                                               

csv docs say 'excel_tab'; code says 'excel-tab'                  10/02/07
CLOSED http://bugs.python.org/issue1227    created  dcelzinga                
                                                                               

3.0 tutorial/datastructures.rst patch                            10/02/07
       http://bugs.python.org/issue1228    created  scav                     
       patch                                                                   

3.0 library/stdtypes.rst                                         10/02/07
       http://bugs.python.org/issue1229    created  scav                     
                                                                               

Tix HList class missing method implementation for info_bbox      10/03/07
       http://bugs.python.org/issue1230    created  ron.longo                
                                                                               

PEPs 344 and 3134 contain incorrect hyperlinks.                  10/03/07
CLOSED http://bugs.python.org/issue1231    created  draghuram                
                                                                               

%f prints the wrong 6 decimal places                             10/03/07
CLOSED http://bugs.python.org/issue1232    created  kmckiou                  
                                                                               

bsddb.dbshelve.DbShelf.append doesn't work                       10/03/07
       http://bugs.python.org/issue1233    created  polaar                   
                                                                               

semaphore errors on AIX 5.2                                      10/04/07
       http://bugs.python.org/issue1234    created  sable                    
                                                                               

CGIHTTPRequestHandler.run_cgi() does not run on Windows if sys.e 10/04/07
       http://bugs.python.org/issue1235    created  schu                     
                                                                               

subprocess is not thread-safe                                    10/04/07
       http://bugs.python.org/issue1236    created  kjd at duda.org             
                                                                               

type_new doesn't allocate space for sentinal slot                10/05/07
       http://bugs.python.org/issue1237    created  rhamphoryncus            
                                                                               

dictobject and dictentry not used consistently in dictobject.c   10/05/07
       http://bugs.python.org/issue1238    created  anthon                   
                                                                               

openpty does not give bidirectional pipe                         10/05/07
       http://bugs.python.org/issue1239    created  mudyc                    
                                                                               

str.split bug when using sep = None and maxsplit                 10/05/07
       http://bugs.python.org/issue1240    created  QuantumTim               
                                                                               



Issues Now Closed (10)
______________________

Backport ABC to 2.6                                                40 days
       http://bugs.python.org/issue1026    baranguren               
       patch                                                                   

bogus attributes reported in asyncore doc                          37 days
       http://bugs.python.org/issue1053    fdrake                   
                                                                               

csv docs say 'excel_tab'; code says 'excel-tab'                     1 days
       http://bugs.python.org/issue1227    dcelzinga                
                                                                               

PEPs 344 and 3134 contain incorrect hyperlinks.                     1 days
       http://bugs.python.org/issue1231    georg.brandl             
                                                                               

%f prints the wrong 6 decimal places                                0 days
       http://bugs.python.org/issue1232    kmckiou                  
                                                                               

SimpleHTTPServer doesn't understand query arguments               639 days
       http://bugs.python.org/issue1394565 philfr                   
                                                                               

Move firewall warning to "about" menu                             435 days
       http://bugs.python.org/issue1529018 georg.brandl             
       patch                                                                   

Python SEGFAULT on tuple.__repr__ and str()                       192 days
       http://bugs.python.org/issue1686386 brett.cannon             
       patch                                                                   

textView code cleanup                                             144 days
       http://bugs.python.org/issue1718043 kbk                      
       patch                                                                   

IDLE - configDialog layout cleanup                                123 days
       http://bugs.python.org/issue1730217 kbk                      
       patch                                                                   




From p.f.moore at gmail.com  Fri Oct  5 20:02:58 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 5 Oct 2007 19:02:58 +0100
Subject: [Python-Dev] Windows builds for backported SSL module, please?
In-Reply-To: <8941209440109967756@unknownmsgid>
References: <-7691212560301951594@unknownmsgid>
	<79990c6b0710050322v221f29c9i77f3b629cea57661@mail.gmail.com>
	<8941209440109967756@unknownmsgid>
Message-ID: <79990c6b0710051102n166623an1fb20642663bf16c@mail.gmail.com>

On 05/10/2007, Bill Janssen <janssen at parc.com> wrote:
> For some reason, it was set to be 'hidden'.
>
> Try
>
>   http://pypi.python.org/pypi/ssl/1.5
>
> Bill

Tiny typo in setup.py (see below). I've fixed it in my copy - it won't
affect the Windows builds, unless you want to release a new version,
when the version number will change :-)

Paul.

--- setup.py.orig	2007-10-05 00:19:04.000000000 +0100
+++ setup.py	2007-10-05 19:01:27.359375000 +0100
@@ -160,7 +160,7 @@
     ssl_incs = [os.environ.get("C_INCLUDE_DIR") or
os.path.join(gnuwin32_dir, "include")]
     ssl_libs = [os.environ.get("C_LIB_DIR") or
os.path.join(gnuwin32_dir, "lib")]
     libs = ['ssl', 'crypto', 'wsock32']
-    if static:
+    if not dynamic:
 	libs = libs + ['gdi32', 'gw32c', 'ole32', 'uuid']
         link_args = ['-static']
 else:

From p.f.moore at gmail.com  Fri Oct  5 20:06:11 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 5 Oct 2007 19:06:11 +0100
Subject: [Python-Dev] Windows builds for backported SSL module, please?
In-Reply-To: <79990c6b0710051102n166623an1fb20642663bf16c@mail.gmail.com>
References: <-7691212560301951594@unknownmsgid>
	<79990c6b0710050322v221f29c9i77f3b629cea57661@mail.gmail.com>
	<8941209440109967756@unknownmsgid>
	<79990c6b0710051102n166623an1fb20642663bf16c@mail.gmail.com>
Message-ID: <79990c6b0710051106w21daedc4ue6750cb52797c19f@mail.gmail.com>

On 05/10/2007, Paul Moore <p.f.moore at gmail.com> wrote:
> On 05/10/2007, Bill Janssen <janssen at parc.com> wrote:
> > For some reason, it was set to be 'hidden'.
> >
> > Try
> >
> >   http://pypi.python.org/pypi/ssl/1.5
> >
> > Bill
>
> Tiny typo in setup.py (see below). I've fixed it in my copy - it won't
> affect the Windows builds, unless you want to release a new version,
> when the version number will change :-)

BTW, there's still an asyncore bug with Python 2.4. I can't recall if
that's a known issue. It was in 1.3 (I didn't ever get to 1.4).

Paul.

From janssen at parc.com  Fri Oct  5 21:09:39 2007
From: janssen at parc.com (Bill Janssen)
Date: Fri, 5 Oct 2007 12:09:39 PDT
Subject: [Python-Dev] Windows builds for backported SSL module, please?
In-Reply-To: <79990c6b0710051106w21daedc4ue6750cb52797c19f@mail.gmail.com> 
References: <-7691212560301951594@unknownmsgid>
	<79990c6b0710050322v221f29c9i77f3b629cea57661@mail.gmail.com>
	<8941209440109967756@unknownmsgid>
	<79990c6b0710051102n166623an1fb20642663bf16c@mail.gmail.com>
	<79990c6b0710051106w21daedc4ue6750cb52797c19f@mail.gmail.com>
Message-ID: <07Oct5.120945pdt."57996"@synergy1.parc.xerox.com>

> BTW, there's still an asyncore bug with Python 2.4. I can't recall if
> that's a known issue. It was in 1.3 (I didn't ever get to 1.4).

Hmmm, I just build Python 2.4.4 and tried it (on a Mac), and it seems
to work fine.  Must be a Windows issue, somehow.  Any ideas?

Bill

From tismer at stackless.com  Sun Oct  7 04:07:55 2007
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 07 Oct 2007 04:07:55 +0200
Subject: [Python-Dev] Where's the Starship's crew?
In-Reply-To: <1191576398.250642.71010@w3g2000hsg.googlegroups.com>
References: <mailman.1567.1191573488.2658.python-list@python.org>
	<1191576398.250642.71010@w3g2000hsg.googlegroups.com>
Message-ID: <47083F7B.2080401@stackless.com>

exhuma.twn wrote:
> On Oct 5, 10:31 am, Dick Moores <r... at rcblue.com> wrote:
>> <http://starship.python.net/crew/index.html>
>>
>> I didn't check on all of them, but the only one I found was Mark
>> Hammond <http://starship.python.net/crew/mhammond/>.
>>
>> Dick Moores
> 
> Doing 6 random clicks, revealed:
> 
> http://starship.python.net/crew/hooft/
> 
> all the others are dead links. I realised this fact as well some time
> ago already. I thought the page itself was still "in development" and
> did not find a single one page. These two obviously eluded me ;)

There are actually 20 active links, different from the error default.
Ok, this is not the best result for a total of 273 members.

> Strange to see that the situation did not yet improve. But then again,
> we all probably know how much of a hassle it is to keep 10 different
> web-pages with personal info's up-to-date. I suppose that the starship
> crew all have som homes elsewhere and don't find the time to update
> yet another page.

No, I think the Starship lost a lot of its popularity after a couple
of havaries, and its initiator failed to lead it and to push it
further. I think this is something that needs to be done by somebody.
There was a couple of very helpful supporters, but they were all
on their own, and the originally driving person did not care
any longer for many years. This has changed.

> I myself don't even bother anymore at all. I have
> about 2 hours of free time per day (given that I like to sleep a
> lot!). What makes things worse, the network at work is completely cut
> off from the rest of the world for security reasons. I am happy that I
> at least can access usenet by using google-groups *sigh*.

So you have HTTP access?
Then ask the starship to support that!
I will take care!

> An interesting task might be to find the homes of the starship crew,
> and post them here? Maybe? If lucky, the right person picks this up
> and updates the crew-quarter labels of the starship. Or would
> "Turbolift Buttons" be a better analogy for hyperlinks? Args.
> Supposing, that these turbolifts *do* have buttons. How retro! ;)

The Starship has gone through quite some problems and different hands.
They all were good, and I wish to thank Stefan Drees especially
for hosting it alone for four years from his personal budget.
Not to forget the years where we were hosted by zope.org, and
beopen.com, and some Service created by Mike Mc. Lay before.
Please excuse me, the I don't remember all the details, this
will go into a history page or blog ASAP.

After a long journey through space and time, I am getting back at
my roots, and I want to fill the gap of a driving person, again.
This is not selfish in any way, and not saying that it must be
me at all, but somehow I feel the wish to bring something to
a working future that I started long time ago.

I invented, created and hosted the Starship, ten years ago.
Now I think, after spending a lot of time with other projects
like Stackless Python and PyPy, time has come to redo the Starship
idea, which was my first contribution for the Python community,
a time where I was not able to really contribute by code.
Especially, I could not work on the Python core because I was no
C programmer. So I created Starship, learned C, understood all
the internals and invented Stackless.

But after all, I think I owe the Starship a lot, and I want it to
continue, and to become a valued resource, again. As a starter,
I moved the ship to my stackless server, this time using OpenVZ.

This transfer worked much smoother than expected; I consider
OpenVZ a great piece of software.

The Starship is therefore right now not very different from its
outside view, it was just moved and upgraded. That is just the
beginning. To turn it into a resource of desire, again, a row
of improvements are necessary.

My personal vision, amoung new projects, is to split my time
between Starship, Stackless and PyPy. If that works depends
on the goodwill of sponsors. But it has worked, so I believe it
will work, again.

--

Here is an incomplete list of things that I want to do or initiate.

First, I want to revive as many as crew homes that are archived.
This will fill the tiny list from above, reasonably, and probably
it will result in people updating their outdated pages.

Then, I want to split the Starship into many sub-sites, first
on my server, then by using a lot of more servers. I am seeking
for sponsors who are willing to support this.

In the end, my vision is turning python.net into a real network
of many servers, each running a very small number of services
through openVZ. Python.net should be something like a server farm,
which runs everything available in Python. Really everything,
including the myriads of possible PyPy builds.

I want to separate all of these things by utilizing OpenVZ,
as fine-grained as makes sense. I am investigating this these days.
Right now, the Starship is a single VE. It will turn into a growing
set of smaller VE's in the next weeks. And as soon as we get
sponsorship, python.net will split itself over multiple machines.

After all, my vision is to create the ultimate Python showdown,
running everything possible in isolated environments and allowing
people to play with different configurations.

python.net should become the ultimate python resource site
for interactive playing and trying, and providing a home for
people to show their development.

I will also try to get the PSF interested in that; maybe there
is also some PSF funding possible. But this is an option, I
will continue Starship without support as well.

I believe I can do that, with your help.

feeling stronger than before that stroke attack -- sincerely - chris

--

This is much more work than I can do alone. Therefore, I am asking
for people to help me with this.
I also don't want to miss any of the current supporters, and we will
name them all on the revised contributors pages to come.
In order to support really many Python projects, we will need
not only sponsors, but probably the support of the individual
project maintainers as well. I am open to make this my goal
of life, if there are enough people interested.
But they will, I know it.

I do believe in Python, Starship, PyPy and Stackless.
Please help me to make this life-dream into reality.

happily being back to the roots -- chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/

From nick.bastin at gmail.com  Mon Oct  8 02:18:02 2007
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Sun, 7 Oct 2007 20:18:02 -0400
Subject: [Python-Dev] GC Changes
In-Reply-To: <47041825.7020300@canterbury.ac.nz>
References: <2cfeb93c0709302333m4fe4599ejb2308ef9d881ae11@mail.gmail.com>
	<2cfeb93c0709302343s525ac174q319bc9609bf2ddb8@mail.gmail.com>
	<4702A1E6.2010001@v.loewis.de> <fdue1r$c9v$1@sea.gmane.org>
	<47033638.7010202@v.loewis.de> <47041825.7020300@canterbury.ac.nz>
Message-ID: <66d0a6e10710071718j31f4d51bx7fd645a77cdbbc44@mail.gmail.com>

On 10/3/07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Martin v. L?wis wrote:
> > For stack frames,
> > such a registration is difficult to make efficient.
>
> Also very error-prone if you happen to miss one. Although
> maybe no more error-prone than getting the reference
> counting right.

Maybe, but reference counting is really easy to debug if you screw it
up.  This is probably one of the primary benefits of the majority of
memory management being executed in reference counting - it's
deterministic and easy to debug.

I'm not opposed to memory management being done entirely through
garbage collection, but it would have to be vastly superior to the
current system in both memory efficiency and performance.

--
Nick

From lists at cheimes.de  Mon Oct  8 13:37:47 2007
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 08 Oct 2007 13:37:47 +0200
Subject: [Python-Dev] SVN down?
Message-ID: <fed4qc$uso$1@sea.gmane.org>

I'm having trouble to connect to the anonymous svn server at
http://svn.python.org/

$ LC_ALL=C svn up
svn: PROPFIND request failed on '/projects/python/branches/py3k'
svn: PROPFIND of '/projects/python/branches/py3k': could not connect to
server (http://svn.python.org)

$ nmap -v -p80 svn.python.org

Starting Nmap 4.20 ( http://insecure.org ) at 2007-10-08 13:37 CEST
Initiating Parallel DNS resolution of 1 host. at 13:37
Completed Parallel DNS resolution of 1 host. at 13:37, 0.00s elapsed
Initiating Connect() Scan at 13:37
Scanning svn.python.org (82.94.237.220) [1 port]
Completed Connect() Scan at 13:37, 0.06s elapsed (1 total ports)
Host svn.python.org (82.94.237.220) appears to be up ... good.
Interesting ports on svn.python.org (82.94.237.220):
PORT   STATE  SERVICE
80/tcp closed http

Nmap finished: 1 IP address (1 host up) scanned in 0.401 seconds

The apache server seems to be down.

Christian


From eli at courtwright.org  Mon Oct  8 17:07:56 2007
From: eli at courtwright.org (Eli Courtwright)
Date: Mon, 8 Oct 2007 11:07:56 -0400
Subject: [Python-Dev] possible string formatting bug
Message-ID: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>

Greetings,

I've found what might be a bug in Python's % string formatting operator.
Consider the following code:

"%%(%s)=%%s" % "hello"

On "Python 2.5.1 (r251:54863, May 18 2007, 16:56:43) [GCC 3.4.4 (cygming
special, gdc 0.12, using dmd 0.125)] on cygwin" this produces the string
"%(hello)s=%s" which is what I'd expect.

On "Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
(Intel)] on win32" this produces the following exception:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: not enough arguments for format string

Note that this is the exact same revision running on the same machine, but
it doesn't work for the win32 version and does work for the cygwin version.

I wish I had time to track down the source of the problem and submit a
patch.  But unfortunately I need to get back to work, and I won't even have
the free time after work to tackle this.  So I figured I'd kick it off to
the Python Dev list in case anyone deems this important enough to be worth
their time.

By the way, Python is my favorite language and I use it whenever possible
and have even gotten some co-workers to start using it.  Thanks for the
great language!

- Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071008/25d30a8c/attachment.htm 

From aahz at pythoncraft.com  Mon Oct  8 17:15:44 2007
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 8 Oct 2007 08:15:44 -0700
Subject: [Python-Dev] possible string formatting bug
In-Reply-To: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>
References: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>
Message-ID: <20071008151544.GA24948@panix.com>

On Mon, Oct 08, 2007, Eli Courtwright wrote:
>
> I've found what might be a bug in Python's % string formatting operator.
> Consider the following code:
> 
> "%%(%s)=%%s" % "hello"
> 
> On "Python 2.5.1 (r251:54863, May 18 2007, 16:56:43) [GCC 3.4.4 (cygming
> special, gdc 0.12, using dmd 0.125)] on cygwin" this produces the string
> "%(hello)s=%s" which is what I'd expect.
> 
> On "Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
> (Intel)] on win32" this produces the following exception:
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: not enough arguments for format string
> 
> Note that this is the exact same revision running on the same machine, but
> it doesn't work for the win32 version and does work for the cygwin version.

Please submit a bug report to http://bugs.python.org/ -- that way it
won't get lost.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

The best way to get information on Usenet is not to ask a question, but
to post the wrong information.

From g.brandl at gmx.net  Mon Oct  8 17:40:40 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 08 Oct 2007 17:40:40 +0200
Subject: [Python-Dev] possible string formatting bug
In-Reply-To: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>
References: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>
Message-ID: <fedj1j$o7v$1@sea.gmane.org>

Eli Courtwright schrieb:
> Greetings,
> 
> I've found what might be a bug in Python's % string formatting
> operator.  Consider the following code:
> 
> "%%(%s)=%%s" % "hello"
> 
> On "Python 2.5.1 (r251:54863, May 18 2007, 16:56:43) [GCC 3.4.4 (cygming
> special, gdc 0.12, using dmd 0.125)] on cygwin" this produces the string
> "%(hello)s=%s" which is what I'd expect.

There must be some confusion, because the above gives "%(hello)=%s"...
are you sure you tried the same format strings on both versions?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From theller at ctypes.org  Mon Oct  8 17:58:32 2007
From: theller at ctypes.org (Thomas Heller)
Date: Mon, 08 Oct 2007 17:58:32 +0200
Subject: [Python-Dev] anon svn down?
Message-ID: <fedk38$sup$1@sea.gmane.org>

It looks like anonymous SVN is down:

http://svn.python.org/projects/python/

Thomas


From facundobatista at gmail.com  Mon Oct  8 18:33:12 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Mon, 8 Oct 2007 13:33:12 -0300
Subject: [Python-Dev] possible string formatting bug
In-Reply-To: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>
References: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>
Message-ID: <e04bdf310710080933tc263663w191f5d1aba486264@mail.gmail.com>

2007/10/8, Eli Courtwright <eli at courtwright.org>:

> On "Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
> (Intel)] on win32" this produces the following exception:
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
> TypeError: not enough arguments for format string

Please check it again, because in my machine...

Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
(Intel)] on win32
>>> "%%(%s)=%%s" % "hello"
'%(hello)=%s'
>>>

If you still think that something is wrong somewhere, feel free to
post a bug in the tracker:

  http://bugs.python.org/

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From martin at v.loewis.de  Mon Oct  8 18:58:13 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 08 Oct 2007 18:58:13 +0200
Subject: [Python-Dev] SVN down?
In-Reply-To: <fed4qc$uso$1@sea.gmane.org>
References: <fed4qc$uso$1@sea.gmane.org>
Message-ID: <470A61A5.8000007@v.loewis.de>

> The apache server seems to be down.

I just restarted it.

Martin

From steve at holdenweb.com  Mon Oct  8 18:51:16 2007
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 08 Oct 2007 12:51:16 -0400
Subject: [Python-Dev] possible string formatting bug
In-Reply-To: <e04bdf310710080933tc263663w191f5d1aba486264@mail.gmail.com>
References: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>
	<e04bdf310710080933tc263663w191f5d1aba486264@mail.gmail.com>
Message-ID: <fedn5u$8om$1@sea.gmane.org>

Facundo Batista wrote:
> 2007/10/8, Eli Courtwright <eli at courtwright.org>:
> 
>> On "Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
>> (Intel)] on win32" this produces the following exception:
>> Traceback (most recent call last):
>>   File "<stdin>", line 1, in <module>
>> TypeError: not enough arguments for format string
> 
> Please check it again, because in my machine...
> 
> Python 2.5.1 (r251:54863, Apr 18 2007, 08:51:08) [MSC v.1310 32 bit
> (Intel)] on win32
>>>> "%%(%s)=%%s" % "hello"
> '%(hello)=%s'
> 
> If you still think that something is wrong somewhere, feel free to
> post a bug in the tracker:
> 
>   http://bugs.python.org/
> 
It even works on my Cygwin build:

$ python
Python 2.5.1 (r251:54863, May 18 2007, 16:56:43)
[GCC 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> "%%(%s)=%%s" % "hello"
'%(hello)=%s'
 >>>

I suspect the problem may lie somewhere else.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd           http://www.holdenweb.com
Skype: holdenweb      http://del.icio.us/steve.holden

Sorry, the dog ate my .sigline so I couldn't cat it


From eli at courtwright.org  Mon Oct  8 20:36:13 2007
From: eli at courtwright.org (Eli Courtwright)
Date: Mon, 8 Oct 2007 14:36:13 -0400
Subject: [Python-Dev] possible string formatting bug
In-Reply-To: <20071008181631.GA16847@panix.com>
References: <3f6c86f50710080807q167d235u95d7e850544a36a0@mail.gmail.com>
	<20071008151544.GA24948@panix.com>
	<3f6c86f50710080829l1b006dfbiacbfeb72f49bcf55@mail.gmail.com>
	<20071008181631.GA16847@panix.com>
Message-ID: <3f6c86f50710081136rb71b2ddgec19de3e0b077555@mail.gmail.com>

Thanks for the quick responses.  Embarrassingly, this problem turned out to
be a typo on my part.  I visually double-and-triple-checked my code before
posting to this list, but I still didn't notice the typo.  Sorry to send
everyone on a wild goose chase.

- Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071008/5ef94669/attachment.htm 

From skip at pobox.com  Mon Oct  8 22:23:03 2007
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 8 Oct 2007 15:23:03 -0500
Subject: [Python-Dev] sched module - still useful?
Message-ID: <18186.37287.443949.830546@montanaro.dyndns.org>


I apologize in advance for the somewhat provocative subject.  Is the sched
module still appropriate in an increasingly event-driven world?  If so, can
someone suggest how it might be used with a non-blocking delay function such
as gobject.timeout_add?  It would be nice to either remark in the
documentation that the sched module doesn't play nice with event-driven
architectures or provide an example of how it can.  It doesn't seem to be
possible without resorting to threads (not generally a big problem, but not
always the best choice either, depending on the environment in which you
find yourself).  The delayfunc is called and when it finishes, the action
function is called.

Thx,

Skip

From guido at python.org  Mon Oct  8 22:40:37 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 8 Oct 2007 13:40:37 -0700
Subject: [Python-Dev] sched module - still useful?
In-Reply-To: <18186.37287.443949.830546@montanaro.dyndns.org>
References: <18186.37287.443949.830546@montanaro.dyndns.org>
Message-ID: <ca471dc20710081340s39028e9dn8de6b56d293c87ca@mail.gmail.com>

It looks like it doesn't mind if delayfunc() returns early;
delayfunc() may insert new events into the queue.

I also think this could be used in a simulation, hence the
parameterization of timefunc() and delayfunc().

On 10/8/07, skip at pobox.com <skip at pobox.com> wrote:
>
> I apologize in advance for the somewhat provocative subject.  Is the sched
> module still appropriate in an increasingly event-driven world?  If so, can
> someone suggest how it might be used with a non-blocking delay function such
> as gobject.timeout_add?  It would be nice to either remark in the
> documentation that the sched module doesn't play nice with event-driven
> architectures or provide an example of how it can.  It doesn't seem to be
> possible without resorting to threads (not generally a big problem, but not
> always the best choice either, depending on the environment in which you
> find yourself).  The delayfunc is called and when it finishes, the action
> function is called.
>
> Thx,
>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Tue Oct  9 12:24:03 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 09 Oct 2007 23:24:03 +1300
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
Message-ID: <470B56C3.7060003@canterbury.ac.nz>

A while back I wrote about a problem I was having with
the ordering of -framework options in distutils compilation
commands. Well, now I've discovered something even stranger.

When distutils executes the following link command, I get
a bunch of undefined OpenGL-related symbols. But if I
copy and paste *exactly the same command* into the shell,
it succeeds!

What could possibly be going on here?

gcc -Wl,-F. -bundle -framework Python
build/temp.darwin-8.4.0-Power_Macintosh-2.3/_soya.o
build/temp.darwin-8.4.0-Power_Macintosh-2.3/matrix.o
build/temp.darwin-8.4.0-Power_Macintosh-2.3/chunk.o -L/usr/lib -L/usr/local/lib
-L/usr/X11R6/lib -lm -lGLEW -lfreetype -lcal3d -lstdc++ -lode -o
build/lib.darwin-8.4.0-Power_Macintosh-2.3/soya/_soya.so -framework OpenGL
-framework SDL -framework OpenAL

This is on MacOSX 10.4 with Python 2.3.

--
Greg

From ronaldoussoren at mac.com  Tue Oct  9 13:21:05 2007
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 09 Oct 2007 04:21:05 -0700
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
In-Reply-To: <470B56C3.7060003@canterbury.ac.nz>
References: <470B56C3.7060003@canterbury.ac.nz>
Message-ID: <65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>

 
On Tuesday, October 09, 2007, at 12:29PM, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote:
>A while back I wrote about a problem I was having with
>the ordering of -framework options in distutils compilation
>commands. Well, now I've discovered something even stranger.
>
>When distutils executes the following link command, I get
>a bunch of undefined OpenGL-related symbols. But if I
>copy and paste *exactly the same command* into the shell,
>it succeeds!
>
>What could possibly be going on here?

My guess is MACOSX_DEPLOYMENT_TARGET: this is set in the environment by distutils.

>
>gcc -Wl,-F. -bundle -framework Python
>build/temp.darwin-8.4.0-Power_Macintosh-2.3/_soya.o
>build/temp.darwin-8.4.0-Power_Macintosh-2.3/matrix.o
>build/temp.darwin-8.4.0-Power_Macintosh-2.3/chunk.o -L/usr/lib -L/usr/local/lib
>-L/usr/X11R6/lib -lm -lGLEW -lfreetype -lcal3d -lstdc++ -lode -o
>build/lib.darwin-8.4.0-Power_Macintosh-2.3/soya/_soya.so -framework OpenGL
>-framework SDL -framework OpenAL
>
>This is on MacOSX 10.4 with Python 2.3.

Is that the system supplied version of Python?  Distutils sets MACOSX_DEPLOYMENT_TARGET to 10.4 in that case, for home-build binaries the variable is set to whatever the value was at configure time, defaulting to 10.3.

Ronald

>
>--
>Greg
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com
>
>

From greg.ewing at canterbury.ac.nz  Tue Oct  9 23:11:26 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 10 Oct 2007 10:11:26 +1300
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
In-Reply-To: <65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
References: <470B56C3.7060003@canterbury.ac.nz>
	<65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
Message-ID: <470BEE7E.8070300@canterbury.ac.nz>

Ronald Oussoren wrote:
> Is that the system supplied version of Python?

No, it's my own installation of 2.3, but it's installed
as a framework in /Library/Frameworks.

> My guess is MACOSX_DEPLOYMENT_TARGET: this is set in the 
 > environment by distutils.

I wondered about that -- I've noticed that with later
Python versions, distutils linking commands *don't* work
in the shell unless I set MACOSX_DEPLOYMENT_TARGET to
10.3 (it's not set at all in my shell environment by
default).

But the problem here is the other way around. I would
have thought distutils would set it to something appropriate
for the Python being used.

Any suggestions on how I can find out what setting of
MACOSX_DEPLOYMENT_TARGET is being used by distutils, and
how to make it use something different if it's not right?

--
Greg



From ronaldoussoren at mac.com  Wed Oct 10 08:23:19 2007
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 10 Oct 2007 08:23:19 +0200
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
In-Reply-To: <470BEE7E.8070300@canterbury.ac.nz>
References: <470B56C3.7060003@canterbury.ac.nz>
	<65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
	<470BEE7E.8070300@canterbury.ac.nz>
Message-ID: <4FC003A8-D13E-4933-ACD0-72EABCC0E8F6@mac.com>


On 9 Oct, 2007, at 23:11, Greg Ewing wrote:

> Ronald Oussoren wrote:
>> Is that the system supplied version of Python?
>
> No, it's my own installation of 2.3, but it's installed
> as a framework in /Library/Frameworks.
>
>> My guess is MACOSX_DEPLOYMENT_TARGET: this is set in the
> > environment by distutils.
>
> I wondered about that -- I've noticed that with later
> Python versions, distutils linking commands *don't* work
> in the shell unless I set MACOSX_DEPLOYMENT_TARGET to
> 10.3 (it's not set at all in my shell environment by
> default).
>
> But the problem here is the other way around. I would
> have thought distutils would set it to something appropriate
> for the Python being used.

It is supposed to do that, but to be honest I don't remember if the  
code for that is present in the 2.3 tree and I'm too lazy to check the  
sources right now.

The deployment target does have an influence on how the compiler  
functions, which can explain when setting the target to a different  
value causes problems.

>
>
> Any suggestions on how I can find out what setting of
> MACOSX_DEPLOYMENT_TARGET is being used by distutils, and
> how to make it use something different if it's not right?

In current versions of python this is read from config/Makefile in the  
standard library (that is /Library/Frameworks/Python.framework/ 
Versions/2.3/lib/python2.3/config/Makefile)

Ronald

>
>
> --
> Greg
>
>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20071010/53b6b31a/attachment.bin 

From greg.ewing at canterbury.ac.nz  Wed Oct 10 13:36:13 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 11 Oct 2007 00:36:13 +1300
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
In-Reply-To: <4FC003A8-D13E-4933-ACD0-72EABCC0E8F6@mac.com>
References: <470B56C3.7060003@canterbury.ac.nz>
	<65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
	<470BEE7E.8070300@canterbury.ac.nz>
	<4FC003A8-D13E-4933-ACD0-72EABCC0E8F6@mac.com>
Message-ID: <470CB92D.6080100@canterbury.ac.nz>

Ronald Oussoren wrote:

> The deployment target does have an influence on how the compiler  
> functions, which can explain when setting the target to a different  
> value causes problems.

I did some more experimenting, and it doesn't seem to be
related to MACOSX_DEPLOYMENT TARGET. I tried setting it
to 10.1, 10.2, 10.3, 10.4 and leaving it unset, and in
all these cases the command works when run directly from
the shell.

So I'm not sure what to try next.

--
Greg

From graham.horler at gmail.com  Wed Oct 10 14:16:19 2007
From: graham.horler at gmail.com (Graham Horler)
Date: Wed, 10 Oct 2007 13:16:19 +0100
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
References: <470B56C3.7060003@canterbury.ac.nz>
	<65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
	<470BEE7E.8070300@canterbury.ac.nz>
	<4FC003A8-D13E-4933-ACD0-72EABCC0E8F6@mac.com>
	<470CB92D.6080100@canterbury.ac.nz>
Message-ID: <63jqje$3a5nam@venus.eclipse.kcom.com>

I would be inclined to move gcc to gcc-real (for example), and create a script
called gcc which dumps all environment variables, and command-line arguments
and a time-stamp to /tmp/gcc.<PID>, e.g.:

#!/bin/sh
FN=/tmp/gcc.$$
echo -n "date=" > $FN
date '+%Y/%m/%d %H:%M:%S.%N' >> $FN
echo -n "cmdline=" >> $FN
echo $0 $@ >> $FN
set >> $FN
gcc-real "$@"
echo "exitcode=$?" >> $FN

(I don't know if any of this will be different on OSX, I'm using Linux.)
Then run the command manually, and compare the dump files.
Here's hoping this helps,
Graham


On 10 Oct 2007, 12:36:13, Greg Ewing wrote:
> Ronald Oussoren wrote:
> 
> > The deployment target does have an influence on how the compiler  
> > functions, which can explain when setting the target to a different  
> > value causes problems.
> 
> I did some more experimenting, and it doesn't seem to be
> related to MACOSX_DEPLOYMENT TARGET. I tried setting it
> to 10.1, 10.2, 10.3, 10.4 and leaving it unset, and in
> all these cases the command works when run directly from
> the shell.
> 
> So I'm not sure what to try next.
> 
> --
> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/graham.horler%40gmail.com

From lists at cheimes.de  Thu Oct 11 16:15:32 2007
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 11 Oct 2007 16:15:32 +0200
Subject: [Python-Dev] SVN down?
In-Reply-To: <470A61A5.8000007@v.loewis.de>
References: <fed4qc$uso$1@sea.gmane.org> <470A61A5.8000007@v.loewis.de>
Message-ID: <470E3004.5060200@cheimes.de>

Martin v. L?wis wrote:
>> The apache server seems to be down.
> 
> I just restarted it.

The anon SVN server is down again :(

Christian


From facundobatista at gmail.com  Fri Oct 12 03:17:15 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 11 Oct 2007 22:17:15 -0300
Subject: [Python-Dev] UTF8 in the PEP branch
Message-ID: <e04bdf310710111817l35247b06md89264c1ba445e5a@mail.gmail.com>

There were two tracker issues that I want to solve that implies to
touch some PEP text.

So, I checked out the PEP branch...

  svn.python.org/peps/trunk

...and made a "make".

A lot of files started to fail because of not ASCII characters (the
standard Syntax Error of PEP-0263, I'm using Py2.5),  like...

  # Author: David Goodger
  # Contact: goodger at users.sourceforge.net
  # Revision: $Revision: 4152 $
  # Date: $Date: 2005-12-07 20:46:30 -0300 (mi?, 07 dic 2005) $
  # Copyright: This module has been placed in the public domain.


...(see the date) in the "./docutils/utils.py" program.

A *lot* of files failed this way. I started to fix them (-*- coding
-*- line at the start), but then I thought that maybe *I* was doing
something wrong, because it's strange that nobody noticed this
before...

So, shall I continue fixing these? Or what I'm doing wrong here?

Thank you very much!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From nnorwitz at gmail.com  Fri Oct 12 07:32:42 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 11 Oct 2007 22:32:42 -0700
Subject: [Python-Dev] 2.5.2 is coming
Message-ID: <ee2a432c0710112232u658f0addl91c391c4f5b0c1d3@mail.gmail.com>

We are planning the release of 2.5.2 for next week.  Unless there are
serious bugs, please hold off making big changes to the 2.5 branch
until after 2.5.2 final is released.  Anthony gets cranky when things
break and he scares me...a lot. :-)  Doc/test fixes as always are
still fine.

The plan is cut the release candidate around Tuesday/Wednesday next
week (Oct 16/17).  If all goes well, 2.5.2 final will follow a week
later.

Please test the 2.5 branch now and let us know if there are any
serious problems.  If there are serious bugs that you think should be
addressed, let us know.  Bugs are here:  http://bugs.python.org/

I see the bugs below as possibly needing attention.  Any comments on these?

http://bugs.python.org/issue1692335	Fix exception pickling: Move
initial args assignment to BaseException.__new__
http://bugs.python.org/issue815646	thread unsafe file objects cause crash
http://bugs.python.org/issue1179 	  Integer overflow in imageop module

I think Anthony might have a patch for 1179.

Cheers,
n

From theller at ctypes.org  Fri Oct 12 08:36:36 2007
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 12 Oct 2007 08:36:36 +0200
Subject: [Python-Dev] 2.5.2 is coming
In-Reply-To: <ee2a432c0710112232u658f0addl91c391c4f5b0c1d3@mail.gmail.com>
References: <ee2a432c0710112232u658f0addl91c391c4f5b0c1d3@mail.gmail.com>
Message-ID: <fen4ll$omo$1@sea.gmane.org>

Neal Norwitz schrieb:
> We are planning the release of 2.5.2 for next week.  Unless there are
> serious bugs, please hold off making big changes to the 2.5 branch
> until after 2.5.2 final is released.  Anthony gets cranky when things
> break and he scares me...a lot. :-)  Doc/test fixes as always are
> still fine.
> 
> The plan is cut the release candidate around Tuesday/Wednesday next
> week (Oct 16/17).  If all goes well, 2.5.2 final will follow a week
> later.
> 
> Please test the 2.5 branch now and let us know if there are any
> serious problems.  If there are serious bugs that you think should be
> addressed, let us know.  Bugs are here:  http://bugs.python.org/
> 

Neal, I have one patch for ctypes still sitting in some of my sandboxes.
ctypes doesn't work when configured with --with-system-ffi on machines
where 'sizeof(long long) != sizeof(long)', in other words on 32-bit boxes.
This is because I misinterpreted the meaning of the FFI_ type codes.

See also this bug: https://bugs.launchpad.net/bugs/72505

Here is the fairly simple patch, even a little bit simpler than the patch
posted on the page above, which I want to apply to thrunk AND
release25-maint branch:

theller at tubu610:~/devel/release25-maint$ svn diff
Index: Modules/_ctypes/cfield.c
===================================================================
--- Modules/_ctypes/cfield.c    (revision 52840)
+++ Modules/_ctypes/cfield.c    (working copy)
@@ -1541,18 +1541,22 @@
 /* XXX Hm, sizeof(int) == sizeof(long) doesn't hold on every platform */
 /* As soon as we can get rid of the type codes, this is no longer a problem */
 #if SIZEOF_LONG == 4
-       { 'l', l_set, l_get, &ffi_type_sint, l_set_sw, l_get_sw},
-       { 'L', L_set, L_get, &ffi_type_uint, L_set_sw, L_get_sw},
+       { 'l', l_set, l_get, &ffi_type_sint32, l_set_sw, l_get_sw},
+       { 'L', L_set, L_get, &ffi_type_uint32, L_set_sw, L_get_sw},
 #elif SIZEOF_LONG == 8
-       { 'l', l_set, l_get, &ffi_type_slong, l_set_sw, l_get_sw},
-       { 'L', L_set, L_get, &ffi_type_ulong, L_set_sw, L_get_sw},
+       { 'l', l_set, l_get, &ffi_type_sint64, l_set_sw, l_get_sw},
+       { 'L', L_set, L_get, &ffi_type_uint64, L_set_sw, L_get_sw},
 #else
 # error
 #endif
 #ifdef HAVE_LONG_LONG
-       { 'q', q_set, q_get, &ffi_type_slong, q_set_sw, q_get_sw},
-       { 'Q', Q_set, Q_get, &ffi_type_ulong, Q_set_sw, Q_get_sw},
+#if SIZEOF_LONG_LONG == 8
+       { 'q', q_set, q_get, &ffi_type_sint64, q_set_sw, q_get_sw},
+       { 'Q', Q_set, Q_get, &ffi_type_uint64, Q_set_sw, Q_get_sw},
+#else
+# error
 #endif
+#endif
        { 'P', P_set, P_get, &ffi_type_pointer},
        { 'z', z_set, z_get, &ffi_type_pointer},
 #ifdef CTYPES_UNICODE
theller at tubu610:~/devel/release25-maint$


Is is ok to apply this patch?

Thomas


From nnorwitz at gmail.com  Fri Oct 12 08:43:44 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 11 Oct 2007 23:43:44 -0700
Subject: [Python-Dev] 2.5.2 is coming
In-Reply-To: <fen4ll$omo$1@sea.gmane.org>
References: <ee2a432c0710112232u658f0addl91c391c4f5b0c1d3@mail.gmail.com>
	<fen4ll$omo$1@sea.gmane.org>
Message-ID: <ee2a432c0710112343i71eccd38ibbce3cc0a965358@mail.gmail.com>

On Oct 11, 2007 11:36 PM, Thomas Heller <theller at ctypes.org> wrote:
>
> Neal, I have one patch for ctypes still sitting in some of my sandboxes.
> ctypes doesn't work when configured with --with-system-ffi on machines
> where 'sizeof(long long) != sizeof(long)', in other words on 32-bit boxes.
>
> Is is ok to apply this patch?

The patch makes sense to me.  If you are comfortable that this will
fix more problems than it's likely to create, go for it.

n

From greg.ewing at canterbury.ac.nz  Fri Oct 12 10:21:15 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Fri, 12 Oct 2007 21:21:15 +1300
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
In-Reply-To: <63jqje$3a5nam@venus.eclipse.kcom.com>
References: <470B56C3.7060003@canterbury.ac.nz>
	<65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
	<470BEE7E.8070300@canterbury.ac.nz>
	<4FC003A8-D13E-4933-ACD0-72EABCC0E8F6@mac.com>
	<470CB92D.6080100@canterbury.ac.nz>
	<63jqje$3a5nam@venus.eclipse.kcom.com>
Message-ID: <470F2E7B.2070704@canterbury.ac.nz>

Graham Horler wrote:
> I would be inclined to move gcc to gcc-real (for example), and create a script
> called gcc which dumps all environment variables, and command-line arguments

Well, I tried that -- and things got even weirder still.

With the wrapper in place between distutils and gcc, the
linking command *worked*.

So then I cut the wrapper down so that the only thing it
does is call gcc, and it *still* works.

Just to make sure, I tried it again without the wrapper,
and it failed.

So the bizarre state of affairs is that if distutils
runs gcc through a shell script, it works, but if
it runs gcc directly, it doesn't work.

Anyone have any ideas on how that can happen?

--
Greg

From martin at v.loewis.de  Fri Oct 12 11:15:14 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 12 Oct 2007 11:15:14 +0200
Subject: [Python-Dev] UTF8 in the PEP branch
In-Reply-To: <e04bdf310710111817l35247b06md89264c1ba445e5a@mail.gmail.com>
References: <e04bdf310710111817l35247b06md89264c1ba445e5a@mail.gmail.com>
Message-ID: <470F3B22.50908@v.loewis.de>

>   # Date: $Date: 2005-12-07 20:46:30 -0300 (mi?, 07 dic 2005) $
> 
> A *lot* of files failed this way. I started to fix them (-*- coding
> -*- line at the start), but then I thought that maybe *I* was doing
> something wrong, because it's strange that nobody noticed this
> before...
> 
> So, shall I continue fixing these? Or what I'm doing wrong here?

I believe it's subversion that is doing something wrong. In my copy
of the file, I get

# Contact: goodger at users.sourceforge.net
# Revision: $Revision: 4152 $
# Date: $Date: 2005-12-08 00:46:30 +0100 (Do, 08 Dez 2005) $

which isn't any better, but happens to be ASCII (the only non-ASCII
dates that I may get are during March - M?rz).

IMO, it's a flaw in subversion that it translates the date (or even
provides a textual form of the date in the first place). It shouldn't
do that, or at least provide a way to get the date expanded according
to the "C" locale.

As a work-around, you can update your sources with

LANG=C svn up

which will prevent Date from being translated to your language.

Regards,
Martin

P.S. Notice that subversion also prints the time in local time, rather
than UTC, so our copies even disagree on the day-of-month on which
this revision was committed. So the proper command line would be

LANG=C TZ=GMT svn up

From phd at phd.pp.ru  Fri Oct 12 11:29:18 2007
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Fri, 12 Oct 2007 13:29:18 +0400
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
In-Reply-To: <470F2E7B.2070704@canterbury.ac.nz>
References: <470B56C3.7060003@canterbury.ac.nz>
	<65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
	<470BEE7E.8070300@canterbury.ac.nz>
	<4FC003A8-D13E-4933-ACD0-72EABCC0E8F6@mac.com>
	<470CB92D.6080100@canterbury.ac.nz>
	<63jqje$3a5nam@venus.eclipse.kcom.com>
	<470F2E7B.2070704@canterbury.ac.nz>
Message-ID: <20071012092918.GA28451@phd.pp.ru>

On Fri, Oct 12, 2007 at 09:21:15PM +1300, Greg Ewing wrote:
> So the bizarre state of affairs is that if distutils
> runs gcc through a shell script, it works, but if
> it runs gcc directly, it doesn't work.
> 
> Anyone have any ideas on how that can happen?

   Do you have a few different copies of gcc installed with setup.py called
the wrong one and your shell script the right one?

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From facundobatista at gmail.com  Fri Oct 12 14:45:05 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 12 Oct 2007 09:45:05 -0300
Subject: [Python-Dev] UTF8 in the PEP branch
In-Reply-To: <470F3B22.50908@v.loewis.de>
References: <e04bdf310710111817l35247b06md89264c1ba445e5a@mail.gmail.com>
	<470F3B22.50908@v.loewis.de>
Message-ID: <e04bdf310710120545g59af1d41oab55a2356d3014c6@mail.gmail.com>

2007/10/12, "Martin v. L?wis" <martin at v.loewis.de>:

> I believe it's subversion that is doing something wrong. In my copy
> of the file, I get
>
> # Contact: goodger at users.sourceforge.net
> # Revision: $Revision: 4152 $
> # Date: $Date: 2005-12-08 00:46:30 +0100 (Do, 08 Dez 2005) $

Looking the file through the web browser, I see:

  Version: $Revision$
  Last-Modified: $Date$

So, SVN is touching that. I guess that that is actually the idea, but
is breaking thing for us-non-7bits-speakers, :)

I see two solutions to this:

- Find a way to specify something in the original PEP file so SVN
translates always the dates in English.

- Touch the .py and adapt them to support this characters.

I prefer point 2.

What do you think? (if it's ok, I'll add the "coding" line in the
files where is needed).

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From martin at v.loewis.de  Fri Oct 12 14:55:54 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 12 Oct 2007 14:55:54 +0200
Subject: [Python-Dev] UTF8 in the PEP branch
In-Reply-To: <e04bdf310710120545g59af1d41oab55a2356d3014c6@mail.gmail.com>
References: <e04bdf310710111817l35247b06md89264c1ba445e5a@mail.gmail.com>	
	<470F3B22.50908@v.loewis.de>
	<e04bdf310710120545g59af1d41oab55a2356d3014c6@mail.gmail.com>
Message-ID: <470F6EDA.9030303@v.loewis.de>

> Looking the file through the web browser, I see:
> 
>   Version: $Revision$
>   Last-Modified: $Date$
> 
> So, SVN is touching that. I guess that that is actually the idea, but
> is breaking thing for us-non-7bits-speakers, :)

Please read about the svn:keywords feature, and yes, that is the idea.

> I see two solutions to this:
> 
> - Find a way to specify something in the original PEP file so SVN
> translates always the dates in English.

Are you sure you are talking about the file that contains the
PEP itself? Or are you perhaps talking about the source code of
docutils?

AFAICT, the problem is in docutils, not in the PEPs.

> - Touch the .py and adapt them to support this characters.
> 
> I prefer point 2.
> 
> What do you think? (if it's ok, I'll add the "coding" line in the
> files where is needed).

I cannot answer that question, because I don't understand your
proposal. What file would you like to touch, and are you sure
you have write permission to it?

Regards,
Martin


From anthon at mnt.org  Fri Oct 12 16:33:27 2007
From: anthon at mnt.org (Anthon van der Neut)
Date: Fri, 12 Oct 2007 16:33:27 +0200
Subject: [Python-Dev] what license for a module derived from
	Objects/dictobjec.c
Message-ID: <470F85B7.2000003@mnt.org>

I would like to get some advice on which license to include/refer to in 
my setup.py (and probably the source files).

This is for an extension module that is clearly (for the knowledgeable) 
derived from Python's Objects/dictobject.c and Include/dictobject.h.

I first of all don't want to infringe any original license on that code. 
  I second would like to make sure the license for code is not a problem 
to make it into Linux distributions like Ubuntu/Debian  (assuming of 
course the code is acceptable and fills enough of a gap that these 
distros want to pick it up in the first place).
And thirdly, I although I would like my name to be associated on further 
derivations of this module, that is not a strict requirement.

Should I refer to MIT, LGPL or BSD. Any particular copy of a license I 
should put up on my website and refer to with a URL?
Any examples or references with a brief explanation would be much 
appreciated.

I know I could look at what other modules use, but at this point I 
rather spent time on finishing implementing the "insert" method (and 
some tests) and getting my first release out (and hopefullysome 
feedback), than comparing other modules for the license they chose and 
try to deduct what I should use/specify for a license.

Thanks in advance
Anthon

For the curious: the module is called ordereddict. It is an 
implemenation in C of a dictionary with ordered keys based on Key 
Insertion Order. Value updating of existing Key/Value pairs does not 
reorder the keys, although I probably implement that as an 
(instantiation) option after the initial version.

orderdict() does all that dict() does, except for instantiation it will 
only take another ordereddict() (not a dict) or an ordered list of 
pairs. Some extras (reverse(), index()) are already implemented and work 
like Larosa/Foord's odict.OrderedDict() (theirs is also the example I 
took for the representation of the ordereddict).

ordereddict() is 5-10% slower than dict() (especially key deletion is 
'expensive') and it is 5-9 times faster than odict.OrderedDict()

I have done all of the development under Linux but will be proably be 
able to test things under Windows and may, at some point even dig up my 
old G4 to see how things go on OS X. I am fairly confident that the C 
code is still portable, or close to it.

From guido at python.org  Fri Oct 12 19:24:04 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 12 Oct 2007 10:24:04 -0700
Subject: [Python-Dev] Python developers are in demand
Message-ID: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>

I keep getting regular requests from people looking for Python coders
(and this is in addition to Google asking me to hand over my contacts
:-). This is good news because it suggests Python is on the uptake
(always good to know). At the same time it is disturbing because
apparently there aren't enough Python programmers out there. (At least
none of them looking for work.) What's up with that?

I wonder if we should start maintaining a list of Python developers
for hire somewhere on python.org, beyond the existing Jobs page. Is
anyone interested in organizing this?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From status at bugs.python.org  Fri Oct 12 19:37:06 2007
From: status at bugs.python.org (Tracker)
Date: Fri, 12 Oct 2007 17:37:06 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071012173706.5F3E17830F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/05/07 - 10/12/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1298 open (+26) / 11458 closed ( +7) / 12756 total (+33)

Open issues with patches:   420

Average duration of open issues: 681 days.
Median duration of open issues: 727 days.

Open Issues Breakdown
   open  1293 (+26)
pending     5 ( +0)

Issues Created Or Reopened (34)
_______________________________

split(None, maxsplit) does not strip whitespace correctly        10/07/07
CLOSED http://bugs.python.org/issue1123    reopened brett.cannon             
                                                                               

subprocess.py stdout of childprocess always buffered.            10/05/07
CLOSED http://bugs.python.org/issue1241    created  jason.w.kim              
                                                                               

test                                                             10/06/07
CLOSED http://bugs.python.org/issue1242    created  gopiyadav26              
                                                                               

option.dest not set when callback called with optparse           10/06/07
       http://bugs.python.org/issue1243    created  cjw296                   
                                                                               

Error on 'raise' does not show correct line number and Traceback 10/08/07
CLOSED http://bugs.python.org/issue1244    created  alexander.shirokov       
                                                                               

Document PySys_* API functions                                   10/08/07
       http://bugs.python.org/issue1245    created  brett.cannon             
                                                                               

Document PySys_GetObject()                                       10/08/07
CLOSED http://bugs.python.org/issue1246    created  brett.cannon             
                                                                               

PEP 3137 patch (repr, names, parser)                             10/08/07
       http://bugs.python.org/issue1247    created  tiran                    
       patch                                                                   

ftplib needs a rewrite to use bytes/buffers                      10/08/07
       http://bugs.python.org/issue1248    created  brett.cannon             
       py3k                                                                    

PEP 3137 patch: PyBytes/PyUnicode comparisons                    10/09/07
CLOSED http://bugs.python.org/issue1249    created  thomas.lee               
                                                                               

Building external modules using Sun Studio 12                    10/09/07
       http://bugs.python.org/issue1250    created  anilj                    
                                                                               

ssl module doesn't support non-blocking handshakes               10/09/07
       http://bugs.python.org/issue1251    created  chris.stawarz            
                                                                               

IDLE - patch Delegator to support callables                      10/10/07
       http://bugs.python.org/issue1252    created  taleinat                 
       patch                                                                   

IDLE - Percolator overhaul                                       10/10/07
       http://bugs.python.org/issue1253    created  taleinat                 
       patch                                                                   

pdb fails to launch some script.                                 10/10/07
       http://bugs.python.org/issue1254    created  romain_jacquet           
                                                                               

Strange Python hangup                                            10/10/07
       http://bugs.python.org/issue1255    created  kakacek                  
                                                                               

subprocess Popen wait() function hangs when stdout is > 20480    10/10/07
       http://bugs.python.org/issue1256    created  superwesman              
                                                                               

atexit errors should result in nonzero exit code                 10/10/07
       http://bugs.python.org/issue1257    created  aminusfu                 
                                                                               

Removal of basestring type                                       10/10/07
       http://bugs.python.org/issue1258    created  tiran                    
       patch                                                                   

string find and rfind methods give a TypeError that is misleadin 10/10/07
       http://bugs.python.org/issue1259    created  rbcollins                
                                                                               

PEP 3137: Remove the buffer API from PyUnicode                   10/11/07
       http://bugs.python.org/issue1260    created  alexandre.vassalotti     
       patch                                                                   

PEP 3137: make bytesobject.c methods                             10/11/07
       http://bugs.python.org/issue1261    created  gregory.p.smith          
       py3k, patch                                                             

IDLE does not start if windows environment variable containing ' 10/11/07
       http://bugs.python.org/issue1262    created  reto                     
                                                                               

PEP 3137 patch - str8/str comparison should return false         10/11/07
       http://bugs.python.org/issue1263    created  thomas.lee               
       patch                                                                   

__file__ and co_filename as unicode                              10/11/07
       http://bugs.python.org/issue1264    created  tiran                    
       patch                                                                   

pdb bug with "with" statement                                    10/11/07
       http://bugs.python.org/issue1265    created  tiran                    
                                                                               

segfault in curses when calling redrawwin() before refresh()     10/11/07
       http://bugs.python.org/issue1266    created  Thorben                  
                                                                               

Py3K cannot run as ``python -S``                                 10/11/07
       http://bugs.python.org/issue1267    created  brett.cannon             
       py3k                                                                    

array unittest problems with UCS4 build                          10/12/07
       http://bugs.python.org/issue1268    created  tiran                    
       patch                                                                   

Exception in pstats print_callers()                              10/12/07
       http://bugs.python.org/issue1269    created  stromnov                 
                                                                               

MemoryError when working on large dictionaries on Windows 2003 S 10/12/07
CLOSED http://bugs.python.org/issue1270    created  amdescombes              
                                                                               

Raw string parsing fails with backslash as last character        10/12/07
CLOSED http://bugs.python.org/issue1271    created  QuantumTim               
                                                                               

Decode __file__ and co_filename to unicode using fs default      10/12/07
       http://bugs.python.org/issue1272    created  tiran                    
       patch                                                                   

email module example email-unpack.py error                       10/12/07
       http://bugs.python.org/issue1273    created  esv                      
                                                                               



Issues Now Closed (28)
______________________

split(None, maxsplit) does not strip whitespace correctly           0 days
       http://bugs.python.org/issue1123    georg.brandl             
                                                                               

Idle - Save (buffer) - closes IDLE and does not save file (Windo   32 days
       http://bugs.python.org/issue1130    taleinat                 
       py3k                                                                    

Documentation for tp_as_number... version 2.6                      14 days
       http://bugs.python.org/issue1199    georg.brandl             
       patch                                                                   

ctypes doesn't work on Mac with --disable-toolbox-glue             16 days
       http://bugs.python.org/issue1203    theller                  
                                                                               

Python2.5.1 fails to compile under  VC.NET2002 ( 7.0 )             15 days
       http://bugs.python.org/issue1216    loewis                   
                                                                               

IDLE - Fix: pressing Ctrl+C while printing exception -> stuck       8 days
       http://bugs.python.org/issue1225    kbk                      
                                                                               

lib/sched.py superfluous code for removal                           6 days
       http://bugs.python.org/issue1226    georg.brandl             
                                                                               

3.0 tutorial/datastructures.rst patch                               6 days
       http://bugs.python.org/issue1228    georg.brandl             
       patch                                                                   

3.0 library/stdtypes.rst                                            6 days
       http://bugs.python.org/issue1229    brett.cannon             
       py3k                                                                    

subprocess is not thread-safe                                       1 days
       http://bugs.python.org/issue1236    gvanrossum               
                                                                               

type_new doesn't allocate space for sentinal slot                   1 days
       http://bugs.python.org/issue1237    gvanrossum               
                                                                               

dictobject and dictentry not used consistently in dictobject.c      5 days
       http://bugs.python.org/issue1238    brett.cannon             
       patch                                                                   

str.split bug when using sep = None and maxsplit                    2 days
       http://bugs.python.org/issue1240    brett.cannon             
                                                                               

subprocess.py stdout of childprocess always buffered.               1 days
       http://bugs.python.org/issue1241    gvanrossum               
                                                                               

test                                                                0 days
       http://bugs.python.org/issue1242    georg.brandl             
                                                                               

Error on 'raise' does not show correct line number and Traceback    0 days
       http://bugs.python.org/issue1244    alexander.shirokov       
                                                                               

Document PySys_GetObject()                                          0 days
       http://bugs.python.org/issue1246    brett.cannon             
                                                                               

PEP 3137 patch: PyBytes/PyUnicode comparisons                       0 days
       http://bugs.python.org/issue1249    gvanrossum               
                                                                               

MemoryError when working on large dictionaries on Windows 2003 S    0 days
       http://bugs.python.org/issue1270    amdescombes              
                                                                               

Raw string parsing fails with backslash as last character           0 days
       http://bugs.python.org/issue1271    georg.brandl             
                                                                               

Recursion limit too high for MacOSX                              1576 days
       http://bugs.python.org/issue756576  brett.cannon             
                                                                               

Use correct encoding for printing SyntaxErrors                   1116 days
       http://bugs.python.org/issue1031213 gvanrossum               
       patch                                                                   

tools and demo missing from windows                               420 days
       http://bugs.python.org/issue1541420 loewis                   
                                                                               

Minor pasting patch                                               238 days
       http://bugs.python.org/issue1659326 kbk                      
       patch                                                                   

Ctrl+Shift block marking by words                                 161 days
       http://bugs.python.org/issue1710718 kbk                      
       patch                                                                   

character set in Japanese on Ubuntu distribution                  158 days
       http://bugs.python.org/issue1713252 kbk                      
       patch                                                                   

IDLE - cursor color configuration bug                             137 days
       http://bugs.python.org/issue1725576 kbk                      
       patch                                                                   

Display CallTips for classes using metaclasses.                    54 days
       http://bugs.python.org/issue1775388 kbk                      
                                                                               



Top Issues Most Discussed (10)
______________________________

  9 MemoryError when working on large dictionaries on Windows 2003     0 days
closed  http://bugs.python.org/issue1270   

  9 ssl module doesn't support non-blocking handshakes                 3 days
open    http://bugs.python.org/issue1251   

  7 Use correct encoding for printing SyntaxErrors                  1116 days
closed  http://bugs.python.org/issue1031213

  6 Removal of basestring type                                         2 days
open    http://bugs.python.org/issue1258   

  5 option.dest not set when callback called with optparse             6 days
open    http://bugs.python.org/issue1243   

  4 PEP 3137 patch (repr, names, parser)                               4 days
open    http://bugs.python.org/issue1247   

  4 dictobject and dictentry not used consistently in dictobject.c     5 days
closed  http://bugs.python.org/issue1238   

  3 implement warnings module in C                                   276 days
open    http://bugs.python.org/issue1631171

  3 Raw string parsing fails with backslash as last character          0 days
closed  http://bugs.python.org/issue1271   

  3 PEP 3137: make bytesobject.c methods                               1 days
open    http://bugs.python.org/issue1261   




From brett at python.org  Fri Oct 12 19:43:53 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 12 Oct 2007 10:43:53 -0700
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
Message-ID: <bbaeab100710121043t73bac79cx7614624e0051e31f@mail.gmail.com>

On 10/12/07, Guido van Rossum <guido at python.org> wrote:
> I keep getting regular requests from people looking for Python coders
> (and this is in addition to Google asking me to hand over my contacts
> :-). This is good news because it suggests Python is on the uptake
> (always good to know).

Yeah, I have noticed as well.  I have seen an increase in random
emails from people looking for Python developers.

> At the same time it is disturbing because
> apparently there aren't enough Python programmers out there. (At least
> none of them looking for work.) What's up with that?
>

It's still not taught in schools.  I still rarely see a new grad
student in my lab who has used Python before they meet me.  And even
after I give a Python tutorial to the lab and people realize it's a
great language, they wait until a project comes up where they need to
start from scratch to use it.  Unfortunately that project never comes
up.

So I would guess the only people becoming new Python programmers are
either the self-motivated people who are curious about the language or
people whose job requires it.

> I wonder if we should start maintaining a list of Python developers
> for hire somewhere on python.org, beyond the existing Jobs page. Is
> anyone interested in organizing this?

You thinking like contractors or anyone out of work?

-Brett

From shane at hathawaymix.org  Fri Oct 12 19:50:03 2007
From: shane at hathawaymix.org (Shane Hathaway)
Date: Fri, 12 Oct 2007 11:50:03 -0600
Subject: [Python-Dev] Explicit Tail Calls
Message-ID: <470FB3CB.7000007@hathawaymix.org>

Hello,

I'm interested in seeing a good way to write tail calls in Python.  Some
algorithms are more readable when expressed using tail recursion.

I know tail call optimization has been discussed before [1], but I would
like to consider a different approach.  The previous discussion centered
on implicit tail call optimization, which incurs the risk of changing
the behavior of currently working code.  (For example, is it safe to
optimize tail calls within try...finally blocks?  Probably not.  And I
generally want all stack frames to appear in tracebacks, unless I say
otherwise.)

I would like to suggest an explicit form of tail calls.  A new built-in
exception type called "Return" will be added, and it will be used like this:

def fact2(n, v):
    if n:
        raise Return(fact2, n-1, v*n)
    else:
        return v

The interpreter will catch Return exceptions and use them to call
something else.  The caller of a function that uses "raise Return" will
see the result of the tail call as the returned value, rather than the
Return exception.  I am not yet considering implementation details.

Not all algorithms are good candidates for this.  I used the fact2
example only because it's readily available.  I know there are other
people interested in tail call optimization in Python [2] [3]; perhaps
some of them are watching and can provide better examples.

Furthermore, there might be some explicit syntax that converts "return
f(...)" statements to "raise Return(f, ...)", such as a decorator.
However, I'm less interested in the syntax and more interested in the
basic capability.

Shane

[1] http://mail.python.org/pipermail/python-dev/2004-July/046150.html
[2] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/474088
[3] http://www.voidspace.org.uk/python/weblog/arch_d7_2007_09_22.shtml#e833

From martin at v.loewis.de  Fri Oct 12 20:00:29 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 12 Oct 2007 20:00:29 +0200
Subject: [Python-Dev] what license for a module derived
	from	Objects/dictobjec.c
In-Reply-To: <470F85B7.2000003@mnt.org>
References: <470F85B7.2000003@mnt.org>
Message-ID: <470FB63D.4060700@v.loewis.de>

> This is for an extension module that is clearly (for the knowledgeable) 
> derived from Python's Objects/dictobject.c and Include/dictobject.h.
> 
> 
> Should I refer to MIT, LGPL or BSD. Any particular copy of a license I 
> should put up on my website and refer to with a URL?

If this is work derived from dictobject.h, you are bound by the Python
license, see the file LICENSE for details. It is
fairly strict in what you have to do, see, in particular, clause 2
of the PSF license, the BeOpen License, and the the CNRI license.

You have to include the license file itself, as well as a certain magic
incantation.

HTH,
Martin


From shane at hathawaymix.org  Fri Oct 12 20:09:32 2007
From: shane at hathawaymix.org (Shane Hathaway)
Date: Fri, 12 Oct 2007 12:09:32 -0600
Subject: [Python-Dev] Explicit Tail Calls
In-Reply-To: <470FB3CB.7000007@hathawaymix.org>
References: <470FB3CB.7000007@hathawaymix.org>
Message-ID: <470FB85C.20204@hathawaymix.org>

Shane Hathaway wrote:
> I'm interested in seeing a good way to write tail calls in Python.  Some
> algorithms are more readable when expressed using tail recursion.

About ten seconds after I wrote the previous message, I realized two things:

- It's easy to write "return Return" instead of "raise Return".  So
"raise TailCall" is probably better.

- I can write a complete implementation of this idea with nothing but a
simple decorator.  Check it out!

Shane



class TailCall(Exception):
    def __init__(self, f, *args, **kwargs):
        self.f = f
        self.args = args
        self.kwargs = kwargs


def has_tail_calls(f):
    def tail_call_wrapper(*args, **kwargs):
        try:
            return f(*args, **kwargs)
        except TailCall, e:
            return e.f(*e.args, **e.kwargs)
    tail_call_wrapper.__doc__ = f.__doc__
    return tail_call_wrapper


@has_tail_calls
def fact2(n, v=1):
    """
    >>> fact2(1)
    1
    >>> fact2(2)
    2
    >>> fact2(3)
    6
    >>> fact2(4)
    24
    >>> fact2(20)
    2432902008176640000L
    """
    if not n:
        return v
    else:
        raise TailCall(fact2, n - 1, v * n)


if __name__ == '__main__':
    import doctest
    doctest.testmod()


From rhamph at gmail.com  Fri Oct 12 20:09:40 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 12 Oct 2007 12:09:40 -0600
Subject: [Python-Dev] Explicit Tail Calls
In-Reply-To: <470FB3CB.7000007@hathawaymix.org>
References: <470FB3CB.7000007@hathawaymix.org>
Message-ID: <aac2c7cb0710121109n7abbf4b4h8675eddb5e250573@mail.gmail.com>

On 10/12/07, Shane Hathaway <shane at hathawaymix.org> wrote:
> Hello,
>
> I'm interested in seeing a good way to write tail calls in Python.  Some
> algorithms are more readable when expressed using tail recursion.
>
> I know tail call optimization has been discussed before [1], but I would
> like to consider a different approach.  The previous discussion centered
> on implicit tail call optimization, which incurs the risk of changing
> the behavior of currently working code.  (For example, is it safe to
> optimize tail calls within try...finally blocks?  Probably not.  And I
> generally want all stack frames to appear in tracebacks, unless I say
> otherwise.)
>
> I would like to suggest an explicit form of tail calls.  A new built-in
> exception type called "Return" will be added, and it will be used like this:

So long as you're willing to make it explicit (which I strongly
encourage), you can accomplish nearly anything you'd like with
decorators and functions.  There doesn't seem to be strong enough use
cases to get anything into the core language anyway.

If you don't like the existing decorator recipes I can help you come
up with a better one in private, off the list.


> def fact2(n, v):
>     if n:
>         raise Return(fact2, n-1, v*n)
>     else:
>         return v

I hope your use cases are better than this. ;)


> The interpreter will catch Return exceptions and use them to call
> something else.  The caller of a function that uses "raise Return" will
> see the result of the tail call as the returned value, rather than the
> Return exception.  I am not yet considering implementation details.
>
> Not all algorithms are good candidates for this.  I used the fact2
> example only because it's readily available.  I know there are other
> people interested in tail call optimization in Python [2] [3]; perhaps
> some of them are watching and can provide better examples.
>
> Furthermore, there might be some explicit syntax that converts "return
> f(...)" statements to "raise Return(f, ...)", such as a decorator.
> However, I'm less interested in the syntax and more interested in the
> basic capability.
>
> Shane
>
> [1] http://mail.python.org/pipermail/python-dev/2004-July/046150.html
> [2] http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/474088
> [3] http://www.voidspace.org.uk/python/weblog/arch_d7_2007_09_22.shtml#e833
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/rhamph%40gmail.com
>


-- 
Adam Olsen, aka Rhamphoryncus

From shane at hathawaymix.org  Fri Oct 12 20:16:03 2007
From: shane at hathawaymix.org (Shane Hathaway)
Date: Fri, 12 Oct 2007 12:16:03 -0600
Subject: [Python-Dev] Explicit Tail Calls
In-Reply-To: <470FB85C.20204@hathawaymix.org>
References: <470FB3CB.7000007@hathawaymix.org> <470FB85C.20204@hathawaymix.org>
Message-ID: <470FB9E3.4020703@hathawaymix.org>

Shane Hathaway wrote:
> Shane Hathaway wrote:
>> I'm interested in seeing a good way to write tail calls in Python.  Some
>> algorithms are more readable when expressed using tail recursion.
> 
> About ten seconds after I wrote the previous message, I realized two things:
> 
> - It's easy to write "return Return" instead of "raise Return".  So
> "raise TailCall" is probably better.
> 
> - I can write a complete implementation of this idea with nothing but a
> simple decorator.  Check it out!

With yet another 10 seconds, I realized my quick implementation actually
does nothing to optimize tail calls.  Working on a fix.

Shane


From rhamph at gmail.com  Fri Oct 12 20:19:58 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 12 Oct 2007 12:19:58 -0600
Subject: [Python-Dev] Explicit Tail Calls
In-Reply-To: <470FB9E3.4020703@hathawaymix.org>
References: <470FB3CB.7000007@hathawaymix.org> <470FB85C.20204@hathawaymix.org>
	<470FB9E3.4020703@hathawaymix.org>
Message-ID: <aac2c7cb0710121119k29a4a56eucb55ee632c246ad6@mail.gmail.com>

On 10/12/07, Shane Hathaway <shane at hathawaymix.org> wrote:
> Shane Hathaway wrote:
> > Shane Hathaway wrote:
> >> I'm interested in seeing a good way to write tail calls in Python.  Some
> >> algorithms are more readable when expressed using tail recursion.
> >
> > About ten seconds after I wrote the previous message, I realized two things:
> >
> > - It's easy to write "return Return" instead of "raise Return".  So
> > "raise TailCall" is probably better.
> >
> > - I can write a complete implementation of this idea with nothing but a
> > simple decorator.  Check it out!
>
> With yet another 10 seconds, I realized my quick implementation actually
> does nothing to optimize tail calls.  Working on a fix.

Since they're not going to be accepted into python anyway, the
implementation is off-topic for python-dev.  Please take them up
elsewhere (such as my offer to discuss in private.)

-- 
Adam Olsen, aka Rhamphoryncus

From lists at cheimes.de  Fri Oct 12 20:21:20 2007
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 12 Oct 2007 20:21:20 +0200
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
Message-ID: <470FBB20.3020006@cheimes.de>

Guido van Rossum wrote:
> I keep getting regular requests from people looking for Python coders
> (and this is in addition to Google asking me to hand over my contacts
>  :-) . This is good news because it suggests Python is on the uptake
> (always good to know). At the same time it is disturbing because
> apparently there aren't enough Python programmers out there. (At least
> none of them looking for work.) What's up with that?

I know at least one Python developer who is looking for a Python job -
me. :]

The problem may be related to the fact that Python is rarely teached at
school or university. I know no school or university in Germany that is
teaching Python. It's mostly Pascal, Basic or C derivates at school and
C/C++, Java or nowadays C# at university. Some courses of studies even
teach PHP or Flash!
On the other hand Python is gaining big momentum in science. A friend of
mine who works at CERN once told me that they use Python as a glue and
front end language for their problems.

less teaching + growing demand = lack of available developers

> I wonder if we should start maintaining a list of Python developers
> for hire somewhere on python.org, beyond the existing Jobs page. Is
> anyone interested in organizing this?

There is definitely an interest in having a list of available
developers. What would it take to organize a list? How do we avoid to
create yet another XING (http://en.wikipedia.org/wiki/XING), LinkedIn
(http://en.wikipedia.org/wiki/LinkedIn) or SourceForge Marketplace
clone? Could we collaborate with one or multiple major players in the
job market?

Christian

From shane at hathawaymix.org  Fri Oct 12 21:26:43 2007
From: shane at hathawaymix.org (Shane Hathaway)
Date: Fri, 12 Oct 2007 13:26:43 -0600
Subject: [Python-Dev] Explicit Tail Calls
In-Reply-To: <aac2c7cb0710121119k29a4a56eucb55ee632c246ad6@mail.gmail.com>
References: <470FB3CB.7000007@hathawaymix.org>
	<470FB85C.20204@hathawaymix.org>	<470FB9E3.4020703@hathawaymix.org>
	<aac2c7cb0710121119k29a4a56eucb55ee632c246ad6@mail.gmail.com>
Message-ID: <470FCA73.3040201@hathawaymix.org>

Adam Olsen wrote:
> Since they're not going to be accepted into python anyway, the
> implementation is off-topic for python-dev.  Please take them up
> elsewhere (such as my offer to discuss in private.)

I was aware of Guido's earlier rejections, but I figured the rejection
was due to the risky implicit optimization proposed by others.  Perhaps
the objection is deeper than that; I suspect Guido specifically objects
to excessive use of recursion.  Anyway, I've posted a recipe:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/534104

Shane


From martin at v.loewis.de  Fri Oct 12 21:29:11 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 12 Oct 2007 21:29:11 +0200
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <470FBB20.3020006@cheimes.de>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
	<470FBB20.3020006@cheimes.de>
Message-ID: <470FCB07.2030807@v.loewis.de>

> The problem may be related to the fact that Python is rarely teached at
> school or university. I know no school or university in Germany that is
> teaching Python.

I teach Python to the first semester, at the Hasso-Plattner-Institut
in Potsdam, for the third year now.

Regards,
Martin

From greg.ewing at canterbury.ac.nz  Sat Oct 13 02:05:32 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 13 Oct 2007 13:05:32 +1300
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
In-Reply-To: <20071012092918.GA28451@phd.pp.ru>
References: <470B56C3.7060003@canterbury.ac.nz>
	<65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
	<470BEE7E.8070300@canterbury.ac.nz>
	<4FC003A8-D13E-4933-ACD0-72EABCC0E8F6@mac.com>
	<470CB92D.6080100@canterbury.ac.nz>
	<63jqje$3a5nam@venus.eclipse.kcom.com>
	<470F2E7B.2070704@canterbury.ac.nz> <20071012092918.GA28451@phd.pp.ru>
Message-ID: <47100BCC.50601@canterbury.ac.nz>

Oleg Broytmann wrote:
>    Do you have a few different copies of gcc installed with setup.py called
> the wrong one and your shell script the right one?

As far as I can tell, it's using my PATH to find gcc,
and that leads to /usr/bin/gcc, which is the same one
my wrapper calls.

--
Greg

From janssen at parc.com  Sat Oct 13 04:07:32 2007
From: janssen at parc.com (Bill Janssen)
Date: Fri, 12 Oct 2007 19:07:32 PDT
Subject: [Python-Dev] SSL 1.7
Message-ID: <07Oct12.190735pdt."57996"@synergy1.parc.xerox.com>

I've now got a large complex long-lived threaded server asyncore-based
(Medusa) multi-client server program running on OS X with Python 2.3.5
and the "older-Python" SSL module.  I had to make a number of changes
to the code; primarily fixing the way that certain "socket" methods
are called on the SSLSocket, and in the C code, allowing for changes
to the socket blocking attribute that are made out-of-band from use of
the SSL context.

I'll fold that back into the trunk next week, and produce a patch for
the 3K branch after that.

Meanwhile, I've updated the PyPI package to SSL 1.7.

Paul, this also fixes (I hope :-) the setup.py bug you mentioned in
1.6.

Chris, you might want to try this one with Twisted; I believe it will
work OK.

One thing to watch out for: ssl.SSLError can't inherit from
socket.error, as it does in 2.6+, so if you are catching exceptions,
you need to trap them like:

  except ssl.SSLError, x:
     ...
  except socket.error, x:
     ...

which idiom will continue to work for 2.6 and later.

Bill



From greg.ewing at canterbury.ac.nz  Sat Oct 13 05:24:58 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 13 Oct 2007 16:24:58 +1300
Subject: [Python-Dev] MacOSX -framework options and distutils weirdness
In-Reply-To: <47100BCC.50601@canterbury.ac.nz>
References: <470B56C3.7060003@canterbury.ac.nz>
	<65721D79-0115-1000-A555-6C7D007E1E12-Webmail-10010@mac.com>
	<470BEE7E.8070300@canterbury.ac.nz>
	<4FC003A8-D13E-4933-ACD0-72EABCC0E8F6@mac.com>
	<470CB92D.6080100@canterbury.ac.nz>
	<63jqje$3a5nam@venus.eclipse.kcom.com>
	<470F2E7B.2070704@canterbury.ac.nz> <20071012092918.GA28451@phd.pp.ru>
	<47100BCC.50601@canterbury.ac.nz>
Message-ID: <47103A8A.3030307@canterbury.ac.nz>

I wrote:

> As far as I can tell, it's using my PATH to find gcc,
> and that leads to /usr/bin/gcc, which is the same one
> my wrapper calls.

To make sure, I installed my wrapper in place of /usr/bin/gcc
and the same behaviour occurs.

--
Greg

From greg.ewing at canterbury.ac.nz  Sat Oct 13 11:19:43 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 13 Oct 2007 22:19:43 +1300
Subject: [Python-Dev] Explicit Tail Calls
Message-ID: <47108DAF.20007@canterbury.ac.nz>

Shane Hathaway wrote:

> I was aware of Guido's earlier rejections, but I figured the rejection
> was due to the risky implicit optimization proposed by others.  Perhaps
> the objection is deeper than that; I suspect Guido specifically objects
> to excessive use of recursion.

As I recall, one of the objections is that it makes
debugging difficult by removing intermediate stack
frames from the traceback.

However, I don't think that the sort of programming
style that requires tail call optimisation is really
part of the Python programming culture. It's used
in languages like Lisp and Haskell because the data
structures used to represent lists in those languages
are themselves recursive.

However, Python doesn't do things that way -- normally
it represents a list as a flat data structure, which
is more naturally iterated over than recursed over.

In my experience, the only time recursion is really
called for in Python is when you're operating on
tree-shaped data structures, in which case tail calls
don't really come into it.

When operating on something linear, I find that
a for-loop or list comprehension usually gets the
job done more clearly and concisely.

--
Greg


From facundobatista at gmail.com  Sun Oct 14 16:25:18 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Sun, 14 Oct 2007 11:25:18 -0300
Subject: [Python-Dev] UTF8 in the PEP branch
In-Reply-To: <470F6EDA.9030303@v.loewis.de>
References: <e04bdf310710111817l35247b06md89264c1ba445e5a@mail.gmail.com>
	<470F3B22.50908@v.loewis.de>
	<e04bdf310710120545g59af1d41oab55a2356d3014c6@mail.gmail.com>
	<470F6EDA.9030303@v.loewis.de>
Message-ID: <e04bdf310710140725r6631e96amce1cf5c3b846211a@mail.gmail.com>

2007/10/12, "Martin v. L?wis" <martin at v.loewis.de>:

> > - Find a way to specify something in the original PEP file so SVN
> > translates always the dates in English.
>
> Are you sure you are talking about the file that contains the
> PEP itself? Or are you perhaps talking about the source code of
> docutils?

In the PEP text (example:

  http://svn.python.org/view/peps/trunk/pep-0002.txt?rev=56077&view=auto

), it says "Last-Modified: $Date$". That $Date$ is being translated in
the checkout or update by subversion. Maybe there is a $C-locale-Date$
for subversion to automatically put the date there using C locale and
not the one of the user.

I don't know if it exists, though.


> AFAICT, the problem is in docutils, not in the PEPs.

Me too, that's why I prefer point 2.


> > - Touch the .py and adapt them to support this characters.
> >
> > I prefer point 2.
> >
> > What do you think? (if it's ok, I'll add the "coding" line in the
> > files where is needed).
>
> I cannot answer that question, because I don't understand your
> proposal. What file would you like to touch, and are you sure
> you have write permission to it?

Yes, I have permission.

Of course I need to raise this in docutils mail list.

Thanks!!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From martin at v.loewis.de  Sun Oct 14 16:48:01 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 14 Oct 2007 16:48:01 +0200
Subject: [Python-Dev] UTF8 in the PEP branch
In-Reply-To: <e04bdf310710140725r6631e96amce1cf5c3b846211a@mail.gmail.com>
References: <e04bdf310710111817l35247b06md89264c1ba445e5a@mail.gmail.com>	
	<470F3B22.50908@v.loewis.de>	
	<e04bdf310710120545g59af1d41oab55a2356d3014c6@mail.gmail.com>	
	<470F6EDA.9030303@v.loewis.de>
	<e04bdf310710140725r6631e96amce1cf5c3b846211a@mail.gmail.com>
Message-ID: <47122C21.7050102@v.loewis.de>

>>> - Find a way to specify something in the original PEP file so SVN
>>> translates always the dates in English.
>> Are you sure you are talking about the file that contains the
>> PEP itself? Or are you perhaps talking about the source code of
>> docutils?
> 
> In the PEP text (example:
> 
>   http://svn.python.org/view/peps/trunk/pep-0002.txt?rev=56077&view=auto
> 
> ), it says "Last-Modified: $Date$". That $Date$ is being translated in
> the checkout or update by subversion.

Certainly. However, why does that cause a problem *in the PEP file*?????

> Maybe there is a $C-locale-Date$
> for subversion to automatically put the date there using C locale and
> not the one of the user.
> 
> I don't know if it exists, though.

No, that's not supported. The Id keyword gets replaced with a
locale-independent ISO 8601 string, though.

Regards,
Martin


From lists at cheimes.de  Sun Oct 14 17:42:42 2007
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 14 Oct 2007 17:42:42 +0200
Subject: [Python-Dev] SVN down?
In-Reply-To: <470E3004.5060200@cheimes.de>
References: <fed4qc$uso$1@sea.gmane.org> <470A61A5.8000007@v.loewis.de>
	<470E3004.5060200@cheimes.de>
Message-ID: <471238F2.5070302@cheimes.de>

Guess what? It's down the third time this week ...

Christian

From martin at v.loewis.de  Sun Oct 14 18:21:40 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 14 Oct 2007 18:21:40 +0200
Subject: [Python-Dev] SVN down?
In-Reply-To: <471238F2.5070302@cheimes.de>
References: <fed4qc$uso$1@sea.gmane.org> <470A61A5.8000007@v.loewis.de>
	<470E3004.5060200@cheimes.de> <471238F2.5070302@cheimes.de>
Message-ID: <47124214.8030502@v.loewis.de>

> Guess what? It's down the third time this week ...

Guess what? I restarted it now only the second time this week.

Regards,
Martin

From scott+python-dev at scottdial.com  Mon Oct 15 12:39:12 2007
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Mon, 15 Oct 2007 06:39:12 -0400
Subject: [Python-Dev] UTF8 in the PEP branch
In-Reply-To: <e04bdf310710140725r6631e96amce1cf5c3b846211a@mail.gmail.com>
References: <e04bdf310710111817l35247b06md89264c1ba445e5a@mail.gmail.com>	<470F3B22.50908@v.loewis.de>	<e04bdf310710120545g59af1d41oab55a2356d3014c6@mail.gmail.com>	<470F6EDA.9030303@v.loewis.de>
	<e04bdf310710140725r6631e96amce1cf5c3b846211a@mail.gmail.com>
Message-ID: <47134350.6060107@scottdial.com>

Facundo Batista wrote:
> In the PEP text (example:
> 
>   http://svn.python.org/view/peps/trunk/pep-0002.txt?rev=56077&view=auto
> 
> ), it says "Last-Modified: $Date$". That $Date$ is being translated in
> the checkout or update by subversion. Maybe there is a $C-locale-Date$
> for subversion to automatically put the date there using C locale and
> not the one of the user.
> 
> I don't know if it exists, though.
> 

It currently does not. You are not the first person to notice this 
problem. I know of a number of projects that are currently avoiding the 
use of subversion keywords for this very reason. Anyways, the issue is 
open on subversion's tracker (since 2005):

http://subversion.tigris.org/issues/show_bug.cgi?id=2332

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From andrewm at object-craft.com.au  Mon Oct 15 15:25:55 2007
From: andrewm at object-craft.com.au (Andrew McNamara)
Date: Mon, 15 Oct 2007 23:25:55 +1000
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com> 
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
Message-ID: <20071015132555.91D65600ABA@longblack.object-craft.com.au>

>I wonder if we should start maintaining a list of Python developers
>for hire somewhere on python.org, beyond the existing Jobs page. Is
>anyone interested in organizing this?

What about something a little less formal - a mailing list such as
python-jobs?

-- 
Andrew McNamara, Senior Developer, Object Craft
http://www.object-craft.com.au/

From skip at pobox.com  Mon Oct 15 16:07:15 2007
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 15 Oct 2007 09:07:15 -0500
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <20071015132555.91D65600ABA@longblack.object-craft.com.au>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
	<20071015132555.91D65600ABA@longblack.object-craft.com.au>
Message-ID: <18195.29715.681230.724450@montanaro.dyndns.org>


    >> I wonder if we should start maintaining a list of Python developers
    >> for hire somewhere on python.org, beyond the existing Jobs page. Is
    >> anyone interested in organizing this?

    Andrew> What about something a little less formal - a mailing list such
    Andrew> as python-jobs?

How about just creating a page on the wiki and letting those people
participate who are interested?  

Skip

From mateusz.rukowicz at vp.pl  Mon Oct 15 18:00:24 2007
From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz)
Date: Mon, 15 Oct 2007 18:00:24 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
Message-ID: <47138E98.5060200@vp.pl>

Hi!

    I've been working on C decimal project during gSoC 2006. After year 
of idling (I had extremely busy first year on University, but well, most 
of us are extremely busy) I decided, that I will handle further 
developing (there is still much development needed, and updating to most 
recent standard is just the beginning). I understand, that chances of 
merging C Decimal with main tree are much lower than year ago, so I 
would like to know if there is still interest in C version of Decimal. 
If so - should I write PEP, or just code and 'we'll see later'?

I've made a little benchmark - given loan amount, assets and months that 
it's meant to be payed off, find minimal monthly loan cost (It's just 
first idea that came to me for use Decimal in financial 'application' 
:>) [This solution has complexity O(log(amount) * months) which is far 
from optimal, but it's meant to benchmark Decimal, not python itself].

Code:

from _decimal import *
import sys
gc = getcontext();

def check(loan, percent, monthly):
    ret = 0
    mult = 1 + (percent / 1200)
    if (loan - monthly) * mult >= loan:
        return -1               #you cannot payoff loan ;(

    while loan > 0:
        loan = loan - monthly
        loan = loan * mult
        ret += 1
    return ret

def minimize_monthly(loan, percent, months):
    lower = Decimal(0)
    upper = Decimal(loan)

    while(upper > lower + Decimal("1e-3")):
        mid = (upper + lower)/2
        ret = check(loan, percent, mid)
        if(ret > months or ret == -1):
            lower = mid
        else:
            upper = mid


    return lower



gc.prec = int(sys.argv[4])
gc.rounding = ROUND_UP
print minimize_monthly(Decimal(sys.argv[1]), Decimal(sys.argv[2]), 
int(sys.argv[3]))

and timings (1mln loan, for 15 years, 2% year assets, and precision = 10 
:>):
mateusz at MatLaps:~/programy/python/decimal/decimal-c$ time 
../../pyth/python/python loan.py  1000000 2 180 10
6424.37955

real    0m0.068s
user    0m0.064s
sys     0m0.004s
mateusz at MatLaps:~/programy/python/decimal/decimal-c$ time 
../../pyth/python/python loan2.py  1000000 2 180 10
6424.37955

real    0m2.168s
user    0m2.148s
sys     0m0.016s

Please don't misunderstand me - I don't want to show python Decimal is 
slow, I want to show that C Decimal is worth effort. I am also aware of 
simplicity of this benchmark. (This python have been of course compiled 
with -O3).

Best regards,
Mateusz Rukowicz.

From mateusz.rukowicz at vp.pl  Mon Oct 15 18:00:54 2007
From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz)
Date: Mon, 15 Oct 2007 18:00:54 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
Message-ID: <47138EB6.7060406@vp.pl>

Hi!

    I've been working on C decimal project during gSoC 2006. After year 
of idling (I had extremely busy first year on University, but well, most 
of us are extremely busy) I decided, that I will handle further 
developing (there is still much development needed, and updating to most 
recent standard is just the beginning). I understand, that chances of 
merging C Decimal with main tree are much lower than year ago, so I 
would like to know if there is still interest in C version of Decimal. 
If so - should I write PEP, or just code and 'we'll see later'?

I've made a little benchmark - given loan amount, assets and months that 
it's meant to be payed off, find minimal monthly loan cost (It's just 
first idea that came to me for use Decimal in financial 'application' 
:>) [This solution has complexity O(log(amount) * months) which is far 
from optimal, but it's meant to benchmark Decimal, not python itself].

Code:

from _decimal import *
import sys
gc = getcontext();

def check(loan, percent, monthly):
    ret = 0
    mult = 1 + (percent / 1200)
    if (loan - monthly) * mult >= loan:
        return -1               #you cannot payoff loan ;(

    while loan > 0:
        loan = loan - monthly
        loan = loan * mult
        ret += 1
    return ret

def minimize_monthly(loan, percent, months):
    lower = Decimal(0)
    upper = Decimal(loan)

    while(upper > lower + Decimal("1e-3")):
        mid = (upper + lower)/2
        ret = check(loan, percent, mid)
        if(ret > months or ret == -1):
            lower = mid
        else:
            upper = mid


    return lower



gc.prec = int(sys.argv[4])
gc.rounding = ROUND_UP
print minimize_monthly(Decimal(sys.argv[1]), Decimal(sys.argv[2]), 
int(sys.argv[3]))

and timings (1mln loan, for 15 years, 2% year assets, and precision = 10 
:>):
mateusz at MatLaps:~/programy/python/decimal/decimal-c$ time 
../../pyth/python/python loan.py  1000000 2 180 10
6424.37955

real    0m0.068s
user    0m0.064s
sys     0m0.004s
mateusz at MatLaps:~/programy/python/decimal/decimal-c$ time 
../../pyth/python/python loan2.py  1000000 2 180 10
6424.37955

real    0m2.168s
user    0m2.148s
sys     0m0.016s

Please don't misunderstand me - I don't want to show python Decimal is 
slow, I want to show that C Decimal is worth effort. I am also aware of 
simplicity of this benchmark. (This python have been of course compiled 
with -O3).

Best regards,
Mateusz Rukowicz.

From mateusz.rukowicz at vp.pl  Mon Oct 15 19:59:27 2007
From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz)
Date: Mon, 15 Oct 2007 19:59:27 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <47138EB6.7060406@vp.pl>
References: <47138EB6.7060406@vp.pl>
Message-ID: <4713AA7F.6090509@vp.pl>

Mateusz Rukowicz wrote:

>Hi!
>
>    I've been working on C decimal project during gSoC 2006. After year 
>of idling (I had extremely busy first year on University, but well, most 
>of us are extremely busy) I decided, that I will handle further 
>developing (there is still much development needed, and updating to most 
>recent standard is just the beginning). I understand, that chances of 
>merging C Decimal with main tree are much lower than year ago, so I 
>would like to know if there is still interest in C version of Decimal. 
>If so - should I write PEP, or just code and 'we'll see later'?
>
>I've made a little benchmark - given loan amount, assets and months that 
>it's meant to be payed off, find minimal monthly loan cost (It's just 
>first idea that came to me for use Decimal in financial 'application' 
>:>) [This solution has complexity O(log(amount) * months) which is far 
>from optimal, but it's meant to benchmark Decimal, not python itself].
>
>Code:
>
>from _decimal import *
>import sys
>gc = getcontext();
>
>def check(loan, percent, monthly):
>    ret = 0
>    mult = 1 + (percent / 1200)
>    if (loan - monthly) * mult >= loan:
>        return -1               #you cannot payoff loan ;(
>
>    while loan > 0:
>        loan = loan - monthly
>        loan = loan * mult
>        ret += 1
>    return ret
>
>def minimize_monthly(loan, percent, months):
>    lower = Decimal(0)
>    upper = Decimal(loan)
>
>    while(upper > lower + Decimal("1e-3")):
>        mid = (upper + lower)/2
>        ret = check(loan, percent, mid)
>        if(ret > months or ret == -1):
>            lower = mid
>        else:
>            upper = mid
>
>
>    return lower
>
>
>
>gc.prec = int(sys.argv[4])
>gc.rounding = ROUND_UP
>print minimize_monthly(Decimal(sys.argv[1]), Decimal(sys.argv[2]), 
>int(sys.argv[3]))
>
>and timings (1mln loan, for 15 years, 2% year assets, and precision = 10 
>:>):
>mateusz at MatLaps:~/programy/python/decimal/decimal-c$ time 
>../../pyth/python/python loan.py  1000000 2 180 10
>6424.37955
>
>real    0m0.068s
>user    0m0.064s
>sys     0m0.004s
>mateusz at MatLaps:~/programy/python/decimal/decimal-c$ time 
>../../pyth/python/python loan2.py  1000000 2 180 10
>6424.37955
>
>real    0m2.168s
>user    0m2.148s
>sys     0m0.016s
>
>Please don't misunderstand me - I don't want to show python Decimal is 
>slow, I want to show that C Decimal is worth effort. I am also aware of 
>simplicity of this benchmark. (This python have been of course compiled 
>with -O3).
>
>Best regards,
>Mateusz Rukowicz.
>_______________________________________________
>Python-Dev mailing list
>Python-Dev at python.org
>http://mail.python.org/mailman/listinfo/python-dev
>Unsubscribe: http://mail.python.org/mailman/options/python-dev/mateusz.rukowicz%40vp.pl
>
>  
>
Sorry for two messages, thunderbird told me first message hadn't been send.

From josepharmbruster at gmail.com  Tue Oct 16 02:33:37 2007
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Mon, 15 Oct 2007 20:33:37 -0400
Subject: [Python-Dev] XP Buildbot
Message-ID: <471406E1.6020701@gmail.com>

Greetings,

Long, long ago, I had mentioned dedicating a laptop of mine to the 
python project as a buildbot.  Well, I finally made it around to 
cleaning off my box and putting it all together.  I prepared my laptop 
with everything (I think...) needed to become a buildbot.  The system 
specifications are as follows:

OS Name: Microsoft Windows XP Professional
OS Version: 5.1.2600 Service Pack 2 Build 2600
Processor: x86 Family 15 Model 2 Stepping 4 GenuineIntel ~1195Mhz
Total Physical Memory: 1,023MB

I have the trunk built with VS2005 and I am doing a clean run of rt.bat 
right now to verify everything is reasonably good.

I would like to seek insight from the team as to what steps I could take 
next to verify my box is ready to go and make it be used.

Thank You,
Joseph Armbruster

From greg at krypto.org  Tue Oct 16 03:19:15 2007
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 15 Oct 2007 18:19:15 -0700
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <47138E98.5060200@vp.pl>
References: <47138E98.5060200@vp.pl>
Message-ID: <52dc1c820710151819k60d8e71t8eb9bd1171d33c62@mail.gmail.com>

+1 from me.  If you update it to the most recent Decimal standard I think
its worth it.

anyone else agree?

On 10/15/07, Mateusz Rukowicz <mateusz.rukowicz at vp.pl> wrote:
>
> Hi!
>
>     I've been working on C decimal project during gSoC 2006. After year
> of idling (I had extremely busy first year on University, but well, most
> of us are extremely busy) I decided, that I will handle further
> developing (there is still much development needed, and updating to most
> recent standard is just the beginning). I understand, that chances of
> merging C Decimal with main tree are much lower than year ago, so I
> would like to know if there is still interest in C version of Decimal.
> If so - should I write PEP, or just code and 'we'll see later'?
>
> I've made a little benchmark - given loan amount, assets and months that
> it's meant to be payed off, find minimal monthly loan cost (It's just
> first idea that came to me for use Decimal in financial 'application'
> :>) [This solution has complexity O(log(amount) * months) which is far
> from optimal, but it's meant to benchmark Decimal, not python itself].
>
> Code:
>
> from _decimal import *
> import sys
> gc = getcontext();
>
> def check(loan, percent, monthly):
>     ret = 0
>     mult = 1 + (percent / 1200)
>     if (loan - monthly) * mult >= loan:
>         return -1               #you cannot payoff loan ;(
>
>     while loan > 0:
>         loan = loan - monthly
>         loan = loan * mult
>         ret += 1
>     return ret
>
> def minimize_monthly(loan, percent, months):
>     lower = Decimal(0)
>     upper = Decimal(loan)
>
>     while(upper > lower + Decimal("1e-3")):
>         mid = (upper + lower)/2
>         ret = check(loan, percent, mid)
>         if(ret > months or ret == -1):
>             lower = mid
>         else:
>             upper = mid
>
>
>     return lower
>
>
>
> gc.prec = int(sys.argv[4])
> gc.rounding = ROUND_UP
> print minimize_monthly(Decimal(sys.argv[1]), Decimal(sys.argv[2]),
> int(sys.argv[3]))
>
> and timings (1mln loan, for 15 years, 2% year assets, and precision = 10
> :>):
> mateusz at MatLaps:~/programy/python/decimal/decimal-c$ time
> ../../pyth/python/python loan.py  1000000 2 180 10
> 6424.37955
>
> real    0m0.068s
> user    0m0.064s
> sys     0m0.004s
> mateusz at MatLaps:~/programy/python/decimal/decimal-c$ time
> ../../pyth/python/python loan2.py  1000000 2 180 10
> 6424.37955
>
> real    0m2.168s
> user    0m2.148s
> sys     0m0.016s
>
> Please don't misunderstand me - I don't want to show python Decimal is
> slow, I want to show that C Decimal is worth effort. I am also aware of
> simplicity of this benchmark. (This python have been of course compiled
> with -O3).
>
> Best regards,
> Mateusz Rukowicz.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071015/b7db70fb/attachment-0001.htm 

From brett at python.org  Tue Oct 16 03:47:10 2007
From: brett at python.org (Brett Cannon)
Date: Mon, 15 Oct 2007 18:47:10 -0700
Subject: [Python-Dev] Anyone using ctags with Vim?
Message-ID: <bbaeab100710151847s6a7a951i70b59f00eca0bb01@mail.gmail.com>

I just noticed that the 'tags' Makefile target does not work with
ctags 5.7 since the -t option no longer exists.

So question 1 is whether anyone is using the 'tags' target.

If the answer is no, then if you have your own way of generating the
tags, let me know and I will update the target.  Otherwise I might
just rip it out.

-Brett

From janssen at parc.com  Tue Oct 16 04:35:49 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 15 Oct 2007 19:35:49 PDT
Subject: [Python-Dev] SSL 1.8
Message-ID: <07Oct15.193550pdt."57996"@synergy1.parc.xerox.com>

I've added in some code that Chris Stawarz contributed to allow the
use of non-blocking sockets, with the program thread allowed to do
other things during the handshake while waiting for the peer to
respond.

http://pypi.python.org/pypi/ssl

If this is OK with everyone, I'd like to now port this back to 2.6
aand 3K.

Bill

From greg at krypto.org  Tue Oct 16 08:40:41 2007
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 15 Oct 2007 23:40:41 -0700
Subject: [Python-Dev] SSL 1.8
In-Reply-To: <-3913299298575626826@unknownmsgid>
References: <-3913299298575626826@unknownmsgid>
Message-ID: <52dc1c820710152340y5d1c46cdm9ed9f9df6b0a3c7c@mail.gmail.com>

+1 from me.  sounds like a good idea.

On 10/15/07, Bill Janssen <janssen at parc.com> wrote:
>
> I've added in some code that Chris Stawarz contributed to allow the
> use of non-blocking sockets, with the program thread allowed to do
> other things during the handshake while waiting for the peer to
> respond.
>
> http://pypi.python.org/pypi/ssl
>
> If this is OK with everyone, I'd like to now port this back to 2.6
> aand 3K.
>
> Bill
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071015/3c86cc30/attachment.htm 

From tds333+pydev at gmail.com  Tue Oct 16 09:49:57 2007
From: tds333+pydev at gmail.com (Wolfgang Langner)
Date: Tue, 16 Oct 2007 09:49:57 +0200
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <4c45c1530710160048y585e21a0laa6d550891c7b36f@mail.gmail.com>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
	<20071015132555.91D65600ABA@longblack.object-craft.com.au>
	<18195.29715.681230.724450@montanaro.dyndns.org>
	<4c45c1530710160048y585e21a0laa6d550891c7b36f@mail.gmail.com>
Message-ID: <4c45c1530710160049w492fbc40y5a567ada17b7022a@mail.gmail.com>

On 10/16/07, Wolfgang Langner <tds333+pydev at gmail.com> wrote:
> On 10/15/07, skip at pobox.com <skip at pobox.com> wrote:
> >
> >     >> I wonder if we should start maintaining a list of Python developers
> >     >> for hire somewhere on python.org, beyond the existing Jobs page. Is
> >     >> anyone interested in organizing this?
> >
> >     Andrew> What about something a little less formal - a mailing list such
> >     Andrew> as python-jobs?
> >
> > How about just creating a page on the wiki and letting those people
> > participate who are interested?
>

That's a good idea.
Post the page and I add my name.

From goodger at python.org  Tue Oct 16 15:30:50 2007
From: goodger at python.org (David Goodger)
Date: Tue, 16 Oct 2007 09:30:50 -0400
Subject: [Python-Dev] PyCon 2008: Call for Talk & Tutorial Proposals
In-Reply-To: <47140763.30009@python.org>
References: <47140763.30009@python.org>
Message-ID: <4335d2c40710160630p1f94e67am11c504f15ddeff42@mail.gmail.com>

Proposals for PyCon 2008 talks & tutorials are now being accepted.
The deadline for proposals is November 16.

PyCon 2008 will be held in Chicago, Illinois, USA, from March 13-20.

Please see the full announcement here:
http://pycon.blogspot.com/2007/10/call-for-talk-tutorial-proposals.html

-- 
David Goodger <http://python.net/~goodger>

From dickinsm at gmail.com  Tue Oct 16 16:05:42 2007
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 16 Oct 2007 10:05:42 -0400
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <47138E98.5060200@vp.pl>
References: <47138E98.5060200@vp.pl>
Message-ID: <5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>

On 10/15/07, Mateusz Rukowicz <mateusz.rukowicz at vp.pl> wrote:
> [...] I
> would like to know if there is still interest in C version of Decimal.
> If so - should I write PEP, or just code and 'we'll see later'?

I'd be happy to see decimal.py replaced by a C version giving
essentially the same functionality.  I think the current decimal would
certainly benefit from a speedup: it's probably `fast enough' for many
applications, but it suffers horribly at high precisions:  addition of
two n-digit decimals takes time quadratic in n, for example.

Mark

From mateusz.rukowicz at vp.pl  Tue Oct 16 17:24:30 2007
From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz)
Date: Tue, 16 Oct 2007 17:24:30 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
Message-ID: <4714D7AE.10504@vp.pl>

Mark Dickinson wrote:

>On 10/15/07, Mateusz Rukowicz <mateusz.rukowicz at vp.pl> wrote:
>  
>
>>[...] I
>>would like to know if there is still interest in C version of Decimal.
>>If so - should I write PEP, or just code and 'we'll see later'?
>>    
>>
>
>I'd be happy to see decimal.py replaced by a C version giving
>essentially the same functionality.  I think the current decimal would
>certainly benefit from a speedup: it's probably `fast enough' for many
>applications, but it suffers horribly at high precisions:  addition of
>two n-digit decimals takes time quadratic in n, for example.
>  
>
Well, I am pretty sure, that addition works in linear time in Python 
version :>.

Speaking of performance in high precision computation - a year ago that 
was my aim, to make high precision computation fast, but now I see it 
different way. That is - I am not really convinced, if ie. adding 2k+ 
lines just to make multiplying fast (some fft-like multiplication) is 
worth it - depends on how many people would like to perform computations 
with prec around 100k+ digits ;). So at the moment, pretty much 
everything will work in asymptotically the same time as in Python 
version, but will be faster by some constant (which is quite big 
anyway). It will make C version suitable for medium precision 
computation (let's say 1000-10k digits ;>) - of course, if there is 
demand for high speed high precision, I would love to implement that 
(that's what I wanted at the beginning), but in that case, project would 
really overgrow (it already is really big as for one C file ;).

All I am saying is I want to make situation with many low precision 
arithmetic operations works fast (just like my tiny and pretty useless 
benchmark ;P - but I was surprised by result ;>) - and because I write 
in C, it is not really hard ;).

PS I think, that Py and C version of Decimal may live together and 
cooperate ;>.
PSS Py Decimal uses Python multiplication, so this one is asymptotically 
better than mine.

Thanks for opinion :)

From dickinsm at gmail.com  Tue Oct 16 18:24:35 2007
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 16 Oct 2007 12:24:35 -0400
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <4714D7AE.10504@vp.pl>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl>
Message-ID: <5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>

On 10/16/07, Mateusz Rukowicz <mateusz.rukowicz at vp.pl> wrote:
> Well, I am pretty sure, that addition works in linear time in Python
> version :>.

Unfortunately not.  Here are some timings from my laptop:

>>> from timeit import Timer
>>> Timer("x+x", "from decimal import Decimal; x =
Decimal('1'*5000)").timeit(100)
8.1198890209197998
>>> Timer("x+x", "from decimal import Decimal; x =
Decimal('1'*10000)").timeit(100)
29.203926086425781
>>> Timer("x+x", "from decimal import Decimal; x =
Decimal('1'*20000)").timeit(100)
109.60141491889954
>>> Timer("x+x", "from decimal import Decimal; x =
Decimal('1'*40000)").timeit(100)
492.15129995346069

I'm almost sure that adding 40000 digit numbers together is not what
Decimal was intended to be used for, but it still seems unreasonable
that it takes almost 5 seconds to do such an addition.  The reason for
the quadratic behaviour is that almost all the arithmetic routines in
decimal.py, at some point, convert the coefficient of their
argument(s) from a tuple of digits to a Python integer, and then do
the reverse conversion to get a Decimal result;  both of these
conversions (tuple of digits <-> integer) take time quadratic in the
size of the tuple/integer.   This means that multiplication of
Decimals is also quadratic time, even though it makes use of Python's
subquadratic Karatsuba multiplication.

The alternative would be to implement addition digit-by-digit in
decimal.py;  this would be asymptotically linear but would be much
slower for the low precision ( < 50 digits, say)
decimals that almost everybody is going to be using in
practice---clearly not a good tradeoff.

So one attraction of the C version of decimal is that with little
effort it gets the best of both worlds:  addition *is* just
digit-by-digit (well, okay, limb-by-limb) but since it's coded in C
it's fast enough for regular users.  So you get fast addition in the
usual case, with good asymptotics.

> Speaking of performance in high precision computation - a year ago that
> was my aim, to make high precision computation fast, but now I see it
> different way. That is - I am not really convinced, if ie. adding 2k+
> lines just to make multiplying fast (some fft-like multiplication) is
> worth it - depends on how many people would like to perform computations
> with prec around 100k+ digits ;).

I quite agree:  people who want high-speed high-precision arithmetic
should probably be using some binary floating-point package like GMP
or MPFR.  It's difficult to believe that there's much of a market for
high-precision decimal floating point.

Mark

From mateusz.rukowicz at vp.pl  Tue Oct 16 18:55:31 2007
From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz)
Date: Tue, 16 Oct 2007 18:55:31 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
References: <47138E98.5060200@vp.pl>	
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>	
	<4714D7AE.10504@vp.pl>
	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
Message-ID: <4714ED03.1030403@vp.pl>

Mark Dickinson wrote:

>I'm almost sure that adding 40000 digit numbers together is not what
>Decimal was intended to be used for, but it still seems unreasonable
>that it takes almost 5 seconds to do such an addition.  The reason for
>the quadratic behaviour is that almost all the arithmetic routines in
>decimal.py, at some point, convert the coefficient of their
>argument(s) from a tuple of digits to a Python integer, and then do
>the reverse conversion to get a Decimal result;  both of these
>conversions (tuple of digits <-> integer) take time quadratic in the
>size of the tuple/integer.   This means that multiplication of
>Decimals is also quadratic time, even though it makes use of Python's
>subquadratic Karatsuba multiplication.
>  
>
Oh right, my mistake :> -- I looked at python code, but I forgot about 
conversion ;).


From fredrik.johansson at gmail.com  Tue Oct 16 19:19:39 2007
From: fredrik.johansson at gmail.com (Fredrik Johansson)
Date: Tue, 16 Oct 2007 19:19:39 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl>
	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
Message-ID: <3d0cebfb0710161019t2d8fe1c6t8ff1796f3084448f@mail.gmail.com>

On 10/16/07, Mark Dickinson <dickinsm at gmail.com> wrote:

> The alternative would be to implement addition digit-by-digit in
> decimal.py;  this would be asymptotically linear but would be much
> slower for the low precision ( < 50 digits, say)
> decimals that almost everybody is going to be using in
> practice---clearly not a good tradeoff.

There is another alternative, which is to use integers exclusively for
both representation and arithmetic, and only compute an explicit digit
tuple or string in special cases. I'm doing this in in mpmath
(http://code.google.com/p/mpmath/), which is about 10x faster than
decimal.py at low precision (and of course asymptotically much
faster). However, a significant part of that improvement may not be
possible to achieve when the rounding has to be done in decimal
instead of binary.

A more radical proposal would be to change Python's long type to use a
radix-10**n representation (Python 3000 or beyond?). An implementation
of decimal floating-point arithmetic on top of it, whether written in
C or pure Python (if some utility C functions such as for counting the
number of digits an integer were available), would be both
light-weight and efficient at high precision.

Fredrik

From dickinsm at gmail.com  Tue Oct 16 20:12:31 2007
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 16 Oct 2007 14:12:31 -0400
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <3d0cebfb0710161019t2d8fe1c6t8ff1796f3084448f@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl>
	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
	<3d0cebfb0710161019t2d8fe1c6t8ff1796f3084448f@mail.gmail.com>
Message-ID: <5c6f2a5d0710161112l71c57093p4d852784be1a3ca6@mail.gmail.com>

On 10/16/07, Fredrik Johansson <fredrik.johansson at gmail.com> wrote:
> There is another alternative, which is to use integers exclusively for
> both representation and arithmetic, and only compute an explicit digit
> tuple or string in special cases. I'm doing this in in mpmath
> (http://code.google.com/p/mpmath/), which is about 10x faster than
> decimal.py at low precision (and of course asymptotically much
> faster). However, a significant part of that improvement may not be
> possible to achieve when the rounding has to be done in decimal
> instead of binary.

This is something that I've also thought about on a number of
occasions.  Rounding would be a major concern here: a simple linear
operation (e.g. rounding a 2n-digit number to n digits) would become
quadratic, or at best subquadratic with fancy division algorithms.
There are also some esoteric functions in the decimal library that
require easy access to decimal digits, for example the logical
operations logical_and, logical_xor, etc., that would be slowed by
this approach.  But it may well be that for normal use this would be
the best way to go for decimal.py.  Perhaps some testing is in
order...

> A more radical proposal would be to change Python's long type to use a
> radix-10**n representation (Python 3000 or beyond?).

Mightn't this produce significant (constant factor) slowdowns for long
performance?  As I understand it, the main problem with a base 10**n
representation is that, for example, extracting the high and low limbs
of the 2-limb result of a 1-limb by 1-limb multiplication requires a
division, whereas for integers stored in binary that division can be
replaced by bit operations.

> An implementation
> of decimal floating-point arithmetic on top of it, whether written in
> C or pure Python (if some utility C functions such as for counting the
> number of digits an integer were available), would be both
> light-weight and efficient at high precision.

Agreed.  But it might be hard to sell a more efficient decimal module
at the expense of slower integer arithmetic in core Python.

Mark

From mateusz.rukowicz at vp.pl  Tue Oct 16 20:37:33 2007
From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz)
Date: Tue, 16 Oct 2007 20:37:33 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <5c6f2a5d0710161112l71c57093p4d852784be1a3ca6@mail.gmail.com>
References: <47138E98.5060200@vp.pl>	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>	<4714D7AE.10504@vp.pl>	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>	<3d0cebfb0710161019t2d8fe1c6t8ff1796f3084448f@mail.gmail.com>
	<5c6f2a5d0710161112l71c57093p4d852784be1a3ca6@mail.gmail.com>
Message-ID: <471504ED.3080509@vp.pl>

Mark Dickinson wrote:

>On 10/16/07, Fredrik Johansson <fredrik.johansson at gmail.com> wrote:
>  
>
>>A more radical proposal would be to change Python's long type to use a
>>radix-10**n representation (Python 3000 or beyond?).
>>    
>>
>
>Mightn't this produce significant (constant factor) slowdowns for long
>performance?  As I understand it, the main problem with a base 10**n
>representation is that, for example, extracting the high and low limbs
>of the 2-limb result of a 1-limb by 1-limb multiplication requires a
>division, whereas for integers stored in binary that division can be
>replaced by bit operations.
>
>  
>
>>An implementation
>>of decimal floating-point arithmetic on top of it, whether written in
>>C or pure Python (if some utility C functions such as for counting the
>>number of digits an integer were available), would be both
>>light-weight and efficient at high precision.
>>    
>>
>
>Agreed.  But it might be hard to sell a more efficient decimal module
>at the expense of slower integer arithmetic in core Python.
>
>  
>
My 2 cents -- integer division and modulo (needed for 10**n radix) is so 
slow, that in some extremal cases algorithm may slow down by a factor of 
4-5, so I guess it's not an option. Not to mention binary logical 
operations.


From mateusz.rukowicz at vp.pl  Tue Oct 16 22:19:20 2007
From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz)
Date: Tue, 16 Oct 2007 22:19:20 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <471507E7.9010802@trisoft.com.pl>
References: <47138E98.5060200@vp.pl>	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl> <471507E7.9010802@trisoft.com.pl>
Message-ID: <47151CC8.2010409@vp.pl>

Grzegorz Makarewicz wrote:

>
>Python in high aritmetic - nice, but in standard ? - using computation
>effective representation as in this "demand" should be put as *must*
>have ? in standard python ? - decimal with it's speed is sufficient - if
>U need more speed use more spohisticated data structures - numpy/scipy
>... - I dont like those additions to python standard.
>
>
>  
>
First of all C Decimal isn't meant to be substitute to numpy in any way.
Second of all, I don't think numpy/scipy is an alternative to Decimal -
it is more complicated, and I am not even sure if it supports Decimal
arithmetic (which is crucial for financial application ie.).

[I couldn't find on google whether scipy supports decimal arithmetic, so
I assumed it doesn't because of name '*scientific* tools for python'.
However, if it supports - ignore following text block]
I bet most computation using Decimal are made with precision < 20, in
that case, you might say 'if you need more performance, use built in
double' - it's performance is superior to Decimal, and precision is the
same, but that's not what all is about. So if one needs Decimal
arithmetic (mostly financial applications), and Py Decimal is not enough
- then pretty big problem arises. And that's why C Decimal is for IMHO :).

>if U need more speed use more spohisticated data structures - numpy/scipy

Why use more sophisticated data structure if less sophisticated, but
written in C is enough?

One more thing - C Decimal is being written such way, that moving from
Py Decimal to C Decimal needs very little to no effort. You cannot say
it about moving from Decimal to some other, more sophisticated libraries.

Anyway - thanks for opinion ;>







From robert.kern at gmail.com  Tue Oct 16 22:46:18 2007
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 16 Oct 2007 15:46:18 -0500
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <47151CC8.2010409@vp.pl>
References: <47138E98.5060200@vp.pl>	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>	<4714D7AE.10504@vp.pl>
	<471507E7.9010802@trisoft.com.pl> <47151CC8.2010409@vp.pl>
Message-ID: <ff37v0$bfp$1@ger.gmane.org>

Mateusz Rukowicz wrote:
> Grzegorz Makarewicz wrote:
> 
>> Python in high aritmetic - nice, but in standard ? - using computation
>> effective representation as in this "demand" should be put as *must*
>> have ? in standard python ? - decimal with it's speed is sufficient - if
>> U need more speed use more spohisticated data structures - numpy/scipy
>> ... - I dont like those additions to python standard.
>>
> First of all C Decimal isn't meant to be substitute to numpy in any way.
> Second of all, I don't think numpy/scipy is an alternative to Decimal -
> it is more complicated, and I am not even sure if it supports Decimal
> arithmetic (which is crucial for financial application ie.).
> 
> [I couldn't find on google whether scipy supports decimal arithmetic, so
> I assumed it doesn't because of name '*scientific* tools for python'.
> However, if it supports - ignore following text block]

You are right. We have no tools in numpy or scipy for decimal arithmetic. The
focus of numpy is arrays. This decimal module is complementary to numpy and vice
versa.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
 that is made terrible by our own mad attempt to interpret it as though it had
 an underlying truth."
  -- Umberto Eco


From facundobatista at gmail.com  Tue Oct 16 23:12:49 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 16 Oct 2007 18:12:49 -0300
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <47138E98.5060200@vp.pl>
References: <47138E98.5060200@vp.pl>
Message-ID: <e04bdf310710161412m7e2b8f77j90ab161f10c42fcc@mail.gmail.com>

2007/10/15, Mateusz Rukowicz <mateusz.rukowicz at vp.pl>:

>     I've been working on C decimal project during gSoC 2006. After year
> of idling (I had extremely busy first year on University, but well, most
> of us are extremely busy) I decided, that I will handle further

Welcome back, :)


> merging C Decimal with main tree are much lower than year ago, so I
> would like to know if there is still interest in C version of Decimal.
> If so - should I write PEP, or just code and 'we'll see later'?

First of all you need to address some issues raised by some people
here after you finished your work last year. I remember of Raymond
Hattinger, but maybe there were others.

After that, you should update the C version to comply the spec in its
last version.

Now that we have more prefixes in Py3k (0b110101, for example), we can
push to have something like 0d1.34. Or even that 1.34 is decimal by
default, and have a 0f1.34 if you want binary floating point. Or that
that behaviour is selectable system wide somehow. What people do you
think is the future here?


> I've made a little benchmark - given loan amount, assets and months that
> it's meant to be payed off, find minimal monthly loan cost (It's just

I will prepare, just for decimal.py, a benchmark that is a mix of all
operations and use (a mix of common operations like add, not so used
ones like log10, context switching, exceptions raised, etc).  You can
use this if you want, to measure also the difference in speed from Py
to C. Note, however, that you need to update it first for the last
spec.

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From steve at holdenweb.com  Tue Oct 16 23:56:10 2007
From: steve at holdenweb.com (Steve Holden)
Date: Tue, 16 Oct 2007 17:56:10 -0400
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <e04bdf310710161412m7e2b8f77j90ab161f10c42fcc@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<e04bdf310710161412m7e2b8f77j90ab161f10c42fcc@mail.gmail.com>
Message-ID: <ff3c1r$p60$1@ger.gmane.org>

Facundo Batista wrote:
> 2007/10/15, Mateusz Rukowicz <mateusz.rukowicz at vp.pl>:
> 
>>     I've been working on C decimal project during gSoC 2006. After year
>> of idling (I had extremely busy first year on University, but well, most
>> of us are extremely busy) I decided, that I will handle further
> 
> Welcome back, :)
> 
> 
>> merging C Decimal with main tree are much lower than year ago, so I
>> would like to know if there is still interest in C version of Decimal.
>> If so - should I write PEP, or just code and 'we'll see later'?
> 
> First of all you need to address some issues raised by some people
> here after you finished your work last year. I remember of Raymond
> Hattinger, but maybe there were others.
> 
> After that, you should update the C version to comply the spec in its
> last version.
> 
> Now that we have more prefixes in Py3k (0b110101, for example), we can
> push to have something like 0d1.34. Or even that 1.34 is decimal by
> default, and have a 0f1.34 if you want binary floating point. Or that
> that behaviour is selectable system wide somehow. What people do you
> think is the future here?
> 
I think you should forget any idea of making decimal the default numeric 
literal type (and anyway would you do that only for literals containing 
decimal points?).

Using a radix notation for literals would, IMHO, be acceptable if you 
can get the idea accepted (it is, after all, only syntactic sugar with a 
little memory saving and the transfer of some computation to compile time).
> 
>> I've made a little benchmark - given loan amount, assets and months that
>> it's meant to be payed off, find minimal monthly loan cost (It's just
> 
> I will prepare, just for decimal.py, a benchmark that is a mix of all
> operations and use (a mix of common operations like add, not so used
> ones like log10, context switching, exceptions raised, etc).  You can
> use this if you want, to measure also the difference in speed from Py
> to C. Note, however, that you need to update it first for the last
> spec.
> 
regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd           http://www.holdenweb.com
Skype: holdenweb      http://del.icio.us/steve.holden

Sorry, the dog ate my .sigline so I couldn't cat it


From mateusz.rukowicz at vp.pl  Tue Oct 16 23:57:42 2007
From: mateusz.rukowicz at vp.pl (Mateusz Rukowicz)
Date: Tue, 16 Oct 2007 23:57:42 +0200
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <e04bdf310710161412m7e2b8f77j90ab161f10c42fcc@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<e04bdf310710161412m7e2b8f77j90ab161f10c42fcc@mail.gmail.com>
Message-ID: <471533D6.3030503@vp.pl>

Facundo Batista wrote:

>2007/10/15, Mateusz Rukowicz <mateusz.rukowicz at vp.pl>:
>  
>
>Welcome back, :)
>  
>
Hi :>

>>merging C Decimal with main tree are much lower than year ago, so I
>>would like to know if there is still interest in C version of Decimal.
>>If so - should I write PEP, or just code and 'we'll see later'?
>>    
>>
>
>First of all you need to address some issues raised by some people
>here after you finished your work last year. I remember of Raymond
>Hattinger, but maybe there were others.
>  
>
Yes, I remember Raymond's points about C Decimal, and I keep them in 
mind. Mainly it was about implementation being too much python dependent 
- and because of that code complexity and performance. Fortunately, the 
code is written such way (mainly thanks to Georg Brandl and Jack 
Diedrich, who started project), that in my opinion, modifications of 
context, and other highly-python dependent parts are pretty easy - these 
changes are open to discussion (but not yet though). Complexity - I 
tried to do my best, but I am aware that there are parts which are hard 
to understand and/or are miscommented. I will gradually improve that.
I've been searching few weeks ago for other opinions, but I haven't 
found anything.

>After that, you should update the C version to comply the spec in its
>last version.
>  
>
That's what I am doing now.

>I will prepare, just for decimal.py, a benchmark that is a mix of all
>operations and use (a mix of common operations like add, not so used
>ones like log10, context switching, exceptions raised, etc).  You can
>use this if you want, to measure also the difference in speed from Py
>to C. Note, however, that you need to update it first for the last
>spec.
>  
>
Great :>.




From josepharmbruster at gmail.com  Wed Oct 17 00:13:48 2007
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Tue, 16 Oct 2007 18:13:48 -0400
Subject: [Python-Dev] Python Extension Building Network
In-Reply-To: <b3fa43300710161026n76205f05tc77e9f536a92b663@mail.gmail.com>
References: <b3fa43300710161026n76205f05tc77e9f536a92b663@mail.gmail.com>
Message-ID: <4715379C.3030400@gmail.com>


Mike,

I had been working on getting my box ready to function as one of these:
http://wiki.python.org/moin/BuildbotOnWindows.

However, I will help out in whatever way possible.  If the team feels it 
could be better utilized doing this, let's do it.

VS2005 only fits in because I happen to own a single license :-) It 
seems like most people in the community do not have ready-access to one, 
so I figured this could be a benefit.

Let me know,
Mr. Joseph Armbruster


Mike wrote:
> Dear Ms. Armbruster,
> 
> Steve Holden thought I should contact you. We are working on
> developing a network of volunteers to build binary Python extensions
> for Windows. In a thread on c.l.py, Lundh told us that we should
> compile against 2.3 on up.
> 
> I have been experimenting with compiling some extensions with open
> source tools only, namely the MingW compiler. I also have Visual
> Studio 2003 to use for comparison. I guess what I'm asking is if you'd
> like to be a part of the network. Right now, it's just me with
> Holden's backing.
> 
> How does Visual Studio 2005 fit in exactly? From my research, Python
> 2.4/2.5 were compiled with VS2003. Is VS2005 being  used for 2.6+?
> 
> Anyway, let me know what you think. You can see some of my stuff here:
> 
> http://www.pythonlibrary.org/python_modules.htm
> 
> Mike Driscoll
> 



From greg.ewing at canterbury.ac.nz  Wed Oct 17 00:26:37 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 17 Oct 2007 11:26:37 +1300
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <ff3c1r$p60$1@ger.gmane.org>
References: <47138E98.5060200@vp.pl>
	<e04bdf310710161412m7e2b8f77j90ab161f10c42fcc@mail.gmail.com>
	<ff3c1r$p60$1@ger.gmane.org>
Message-ID: <47153A9D.4090309@canterbury.ac.nz>

Steve Holden wrote:
> Using a radix notation for literals would, IMHO, be acceptable if you 
> can get the idea accepted

This would make decimal (or at least a part of it) a required
part of the Python core rather than an optional module. There
might be some resistance to that.

--
Greg

From dickinsm at gmail.com  Wed Oct 17 02:46:05 2007
From: dickinsm at gmail.com (Mark Dickinson)
Date: Tue, 16 Oct 2007 20:46:05 -0400
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <eae285400710161417m5803c489la324c8beeab27b47@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl>
	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
	<eae285400710161417m5803c489la324c8beeab27b47@mail.gmail.com>
Message-ID: <5c6f2a5d0710161746i3a0c1b82ka9d9ec8ab2ad49cd@mail.gmail.com>

On 10/16/07, Daniel Stutzbach <daniel.stutzbach at gmail.com> wrote:
> On 10/16/07, Mark Dickinson <dickinsm at gmail.com> wrote:
> > the reverse conversion to get a Decimal result;  both of these
> > conversions (tuple of digits <-> integer) take time quadratic in the
> > size of the tuple/integer.
>
> Instead of (or in addition to) porting to C, wouldn't it be better to
> improve the conversion algorithm?

I certainly wouldn't mind seeing subquadratic integer -> string and
string -> integer conversions in core Python.  But I guess somebody's
got to write them, and then persuade the powers-that-be that the extra
code and complexity are worth it... I'm not volunteering right now :).
 Though I do have Python string <-> integer code that's faster than
the builtins from around 8000 decimal digits onwards, if anyone wants
to use it as a starting point...

The desire for fast integer -> string has also come up on the bug
tracker recently: see

http://bugs.python.org/issue1746088

> Radix conversion can be done in O(n log**2 n) time using a divide and
> conquer algorithm.

Isn't there a log(log n) missing here?  In any case, it seems to me
that achieving this sort of complexity is probably best left to GMP
and the like.  But a basic subquadratic division based on Burnikel and
Ziegler's 'Fast Recursive Division' paper seems within reach---this
would give division and radix conversion essentially the same
complexity as  Karatsuba multiplication.

Mark

From sdeibel at wingware.com  Wed Oct 17 17:07:44 2007
From: sdeibel at wingware.com (Stephan Deibel (PSF))
Date: Wed, 17 Oct 2007 11:07:44 -0400
Subject: [Python-Dev] [PSF-Board] Python help desired
In-Reply-To: <9BB68F003ECAF34DAB58881735E89C7B01FBF895@mtk-sms-mail01.digi.com>
References: <9BB68F003ECAF34DAB58881735E89C7B01FBF895@mtk-sms-mail01.digi.com>
Message-ID: <47162540.4030104@wingware.com>

Hi,

I found one directory here:

http://www.opensourcexperts.com/Index/index_html/Python/index.html

I can't vouch for the completeness or quality of the list (in general
the PSF can't recommend any particular provider).  Not all of them are
focusing on programming services.  You'll probably want to start with
the above and develop your own list of those that you trust.

I also saw a few companies advertising on Google when I searched
on "python consultants".

The job board would also be a reasonable place to announce to get
applications from people that would like to be in your pool:

http://www.python.org/community/jobs/

Hope that helps.

-- 

Stephan Deibel
Chairman of the Board
Python Software Foundation

http://python.org/psf


Collis, Chris wrote:
> Guido van Rossum suggested I forward this email to you.  I contacted him earlier this week regarding the inquiry below.  Please read.
>  
>> Our company produces hardware for both wired and wireless
>> device networking.  We recently launched a wireless gateway that supports
>> Python for programmability.  Our customers use this gateway as part of an
>> end-to-end wireless sensor network and Python to add application-specific
>> data intelligence.  We are looking to assemble a group of third party Python
>> developers or Python shops that we can refer customers to who need support
>> in developing scripts to support their intended application.
>>
>> Do you have any organizations/individuals you might recommend who could
>> partner with us in such a capacity or another resource I might pursue to
>> find such partners?  We are a global company and ideally would like partners
>> who can support global opportunities or partners in North America, Europe
>> and Asia who specialize in those regions.
>>
>> Feel free to contact me directly if you would like further information or
>> clarification.
>>
>> Thanks much,
>> Chris Collis
>> Digi Int'l
>> Product Marketing
>> 952 912 3150
> 
> _______________________________________________
> PSF-Board mailing list
> PSF-Board at python.org
> http://mail.python.org/mailman/listinfo/psf-board


From jimjjewett at gmail.com  Wed Oct 17 20:44:55 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 17 Oct 2007 14:44:55 -0400
Subject: [Python-Dev] SSL 1.7
Message-ID: <fb6fbf560710171144y55c1bec8g7fdefdc5bc55ca34@mail.gmail.com>

Bill Janssen wrote:

> One thing to watch out for: ssl.SSLError can't
> inherit from socket.error, as it does in 2.6+,

Why not?

-jJ

From jimjjewett at gmail.com  Wed Oct 17 20:45:38 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 17 Oct 2007 14:45:38 -0400
Subject: [Python-Dev] SSL 1.7
Message-ID: <fb6fbf560710171145u6dd381a9u63c2bfe51fe7726c@mail.gmail.com>

Bill Janssen wrote:

> One thing to watch out for: ssl.SSLError can't
> inherit from socket.error, as it does in 2.6+,

Why not?

-jJ

From janssen at parc.com  Thu Oct 18 18:01:21 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 18 Oct 2007 09:01:21 PDT
Subject: [Python-Dev] SSL 1.7
In-Reply-To: <fb6fbf560710171145u6dd381a9u63c2bfe51fe7726c@mail.gmail.com> 
References: <fb6fbf560710171145u6dd381a9u63c2bfe51fe7726c@mail.gmail.com>
Message-ID: <07Oct18.090131pdt."57996"@synergy1.parc.xerox.com>

> > One thing to watch out for: ssl.SSLError can't
> > inherit from socket.error, as it does in 2.6+,
> 
> Why not?

Mainly because I don't see how to get my hands on the C version of
socket.error.  Patches gratefully accepted, though.

Bill

From brett at python.org  Thu Oct 18 19:57:42 2007
From: brett at python.org (Brett Cannon)
Date: Thu, 18 Oct 2007 10:57:42 -0700
Subject: [Python-Dev] SSL 1.7
In-Reply-To: <7311840145798673188@unknownmsgid>
References: <fb6fbf560710171145u6dd381a9u63c2bfe51fe7726c@mail.gmail.com>
	<7311840145798673188@unknownmsgid>
Message-ID: <bbaeab100710181057q66bed4afv6a4e47a3c577d19c@mail.gmail.com>

On 10/18/07, Bill Janssen <janssen at parc.com> wrote:
> > > One thing to watch out for: ssl.SSLError can't
> > > inherit from socket.error, as it does in 2.6+,
> >
> > Why not?
>
> Mainly because I don't see how to get my hands on the C version of
> socket.error.  Patches gratefully accepted, though.

For anyone who wants to write the patch, you can obviously either
expose the variable the exception is stored in globally, or you can
import the socket module and just get socket.error directly off of the
module itself.

-Brett

From janssen at parc.com  Thu Oct 18 20:22:11 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 18 Oct 2007 11:22:11 PDT
Subject: [Python-Dev] SSL 1.7
In-Reply-To: <bbaeab100710181057q66bed4afv6a4e47a3c577d19c@mail.gmail.com> 
References: <fb6fbf560710171145u6dd381a9u63c2bfe51fe7726c@mail.gmail.com>
	<7311840145798673188@unknownmsgid>
	<bbaeab100710181057q66bed4afv6a4e47a3c577d19c@mail.gmail.com>
Message-ID: <07Oct18.112221pdt."57996"@synergy1.parc.xerox.com>

> For anyone who wants to write the patch, you can obviously either
> expose the variable the exception is stored in globally,

Remember that this is an add-on module, so re-compiling the socket
module code to expose the variable globally would be a rather big
change.  Or so it seemed to me when I thought about it.

> you can
> import the socket module and just get socket.error directly off of the
> module itself.

This is feasible.

Bill



From janssen at parc.com  Thu Oct 18 23:08:07 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 18 Oct 2007 14:08:07 PDT
Subject: [Python-Dev] SSL 1.7
In-Reply-To: <07Oct18.112221pdt."57996"@synergy1.parc.xerox.com> 
References: <fb6fbf560710171145u6dd381a9u63c2bfe51fe7726c@mail.gmail.com>
	<7311840145798673188@unknownmsgid>
	<bbaeab100710181057q66bed4afv6a4e47a3c577d19c@mail.gmail.com>
	<07Oct18.112221pdt."57996"@synergy1.parc.xerox.com>
Message-ID: <07Oct18.140817pdt."57996"@synergy1.parc.xerox.com>

> > you can
> > import the socket module and just get socket.error directly off of the
> > module itself.
> 
> This is feasible.

In fact, so feasible I've done it. :-).

http://pypi.python.org/pypi/ssl/1.9/

Bill

From p.f.moore at gmail.com  Thu Oct 18 23:48:16 2007
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 18 Oct 2007 22:48:16 +0100
Subject: [Python-Dev] SSL 1.7
In-Reply-To: <445593758399817365@unknownmsgid>
References: <fb6fbf560710171145u6dd381a9u63c2bfe51fe7726c@mail.gmail.com>
	<7311840145798673188@unknownmsgid>
	<bbaeab100710181057q66bed4afv6a4e47a3c577d19c@mail.gmail.com>
	<445593758399817365@unknownmsgid>
Message-ID: <79990c6b0710181448y2c54d48fh6cef73001a4f8078@mail.gmail.com>

On 18/10/2007, Bill Janssen <janssen at parc.com> wrote:
> > > you can
> > > import the socket module and just get socket.error directly off of the
> > > module itself.
> >
> > This is feasible.
>
> In fact, so feasible I've done it. :-).
>
> http://pypi.python.org/pypi/ssl/1.9/

On a mostly unrelated note, I just checked a Python 2.4 build on
Windows. The asyncore error I was seeing previously hasn't gone away,
but it has changed. Rather than an exception (followed by the test
hanging) I now get the test hanging in what seems to be the same
place, but no exception :-(

I'm not sure if that's an improvement...

Python 2.3 and 2.5 still pass all tests fine.

Paul.

From janssen at parc.com  Fri Oct 19 00:05:35 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 18 Oct 2007 15:05:35 PDT
Subject: [Python-Dev] SSL 1.7
In-Reply-To: <79990c6b0710181448y2c54d48fh6cef73001a4f8078@mail.gmail.com> 
References: <fb6fbf560710171145u6dd381a9u63c2bfe51fe7726c@mail.gmail.com>
	<7311840145798673188@unknownmsgid>
	<bbaeab100710181057q66bed4afv6a4e47a3c577d19c@mail.gmail.com>
	<445593758399817365@unknownmsgid>
	<79990c6b0710181448y2c54d48fh6cef73001a4f8078@mail.gmail.com>
Message-ID: <07Oct18.150542pdt."57996"@synergy1.parc.xerox.com>

I'd love to know what's "different" about Python 2.4 on Windows that
isn't different in 2.3 and 2.5.

Bill

From daniel.stutzbach at gmail.com  Tue Oct 16 23:17:24 2007
From: daniel.stutzbach at gmail.com (Daniel Stutzbach)
Date: Tue, 16 Oct 2007 16:17:24 -0500
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl>
	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
Message-ID: <eae285400710161417m5803c489la324c8beeab27b47@mail.gmail.com>

On 10/16/07, Mark Dickinson <dickinsm at gmail.com> wrote:
> I'm almost sure that adding 40000 digit numbers together is not what
> Decimal was intended to be used for, but it still seems unreasonable
> that it takes almost 5 seconds to do such an addition.  The reason for
> the quadratic behaviour is that almost all the arithmetic routines in
> decimal.py, at some point, convert the coefficient of their
> argument(s) from a tuple of digits to a Python integer, and then do
> the reverse conversion to get a Decimal result;  both of these
> conversions (tuple of digits <-> integer) take time quadratic in the
> size of the tuple/integer.

Instead of (or in addition to) porting to C, wouldn't it be better to
improve the conversion algorithm?

Radix conversion can be done in O(n log**2 n) time using a divide and
conquer algorithm.

Such an algorithm can be found at the link below (search for "Radix
conversion"):
http://people.cis.ksu.edu/~rhowell/calculator/comparison.html

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From daniel.stutzbach at gmail.com  Wed Oct 17 04:22:13 2007
From: daniel.stutzbach at gmail.com (Daniel Stutzbach)
Date: Tue, 16 Oct 2007 21:22:13 -0500
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <5c6f2a5d0710161746i3a0c1b82ka9d9ec8ab2ad49cd@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl>
	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
	<eae285400710161417m5803c489la324c8beeab27b47@mail.gmail.com>
	<5c6f2a5d0710161746i3a0c1b82ka9d9ec8ab2ad49cd@mail.gmail.com>
Message-ID: <eae285400710161922nebf31aavdbf1c625a5606ca8@mail.gmail.com>

On 10/16/07, Mark Dickinson <dickinsm at gmail.com> wrote:
> > Radix conversion can be done in O(n log**2 n) time using a divide and
> > conquer algorithm.
>
> Isn't there a log(log n) missing here?

Possibly, but who's counting?  :-)

> In any case, it seems to me
> that achieving this sort of complexity is probably best left to GMP
> and the like.  But a basic subquadratic division based on Burnikel and
> Ziegler's 'Fast Recursive Division' paper seems within reach---this
> would give division and radix conversion essentially the same
> complexity as  Karatsuba multiplication.

I agree.  A basic subquadratic radix conversion algorithm isn't much
more complex than the existing quadratic code.  I just whipped
together a Python prototype and it's only 15 lines.

The quadratic algorithm is basically a divide-and-conquer algorithm,
too, except it's the bad kind that  breaks the input into pieces of
size O(1) and size O(n) instead of pieces of size O(n/2). :-)
(where n is number of digits)

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From Chris.Collis at digi.com  Tue Oct 16 23:01:08 2007
From: Chris.Collis at digi.com (Collis, Chris)
Date: Tue, 16 Oct 2007 16:01:08 -0500
Subject: [Python-Dev] Python help desired
Message-ID: <9BB68F003ECAF34DAB58881735E89C7B01FBF895@mtk-sms-mail01.digi.com>

Guido van Rossum suggested I forward this email to you.  I contacted him earlier this week regarding the inquiry below.  Please read.
 
> Our company produces hardware for both wired and wireless
> device networking.  We recently launched a wireless gateway that supports
> Python for programmability.  Our customers use this gateway as part of an
> end-to-end wireless sensor network and Python to add application-specific
> data intelligence.  We are looking to assemble a group of third party Python
> developers or Python shops that we can refer customers to who need support
> in developing scripts to support their intended application.
>
> Do you have any organizations/individuals you might recommend who could
> partner with us in such a capacity or another resource I might pursue to
> find such partners?  We are a global company and ideally would like partners
> who can support global opportunities or partners in North America, Europe
> and Asia who specialize in those regions.
>
> Feel free to contact me directly if you would like further information or
> clarification.
>
> Thanks much,
> Chris Collis
> Digi Int'l
> Product Marketing
> 952 912 3150


From facundobatista at gmail.com  Fri Oct 19 16:35:00 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 19 Oct 2007 11:35:00 -0300
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <eae285400710161922nebf31aavdbf1c625a5606ca8@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl>
	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
	<eae285400710161417m5803c489la324c8beeab27b47@mail.gmail.com>
	<5c6f2a5d0710161746i3a0c1b82ka9d9ec8ab2ad49cd@mail.gmail.com>
	<eae285400710161922nebf31aavdbf1c625a5606ca8@mail.gmail.com>
Message-ID: <e04bdf310710190735x4a394e05rab756c49334ebc70@mail.gmail.com>

2007/10/16, Daniel Stutzbach <daniel.stutzbach at gmail.com>:

> I agree.  A basic subquadratic radix conversion algorithm isn't much
> more complex than the existing quadratic code.  I just whipped
> together a Python prototype and it's only 15 lines.

Do you have a patch for decimal.py of current trunk?

Don't be afraid of submitting it in the Tracker and assign it to me...

Thank you!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From status at bugs.python.org  Fri Oct 19 19:37:11 2007
From: status at bugs.python.org (Tracker)
Date: Fri, 19 Oct 2007 17:37:11 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071019173711.36934782F7@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/12/07 - 10/19/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1312 open (+23) / 11472 closed ( +5) / 12784 total (+28)

Open issues with patches:   420

Average duration of open issues: 680 days.
Median duration of open issues: 752 days.

Open Issues Breakdown
   open  1307 (+23)
pending     5 ( +0)

Issues Created Or Reopened (28)
_______________________________

doctest fails to run file based tests with 8bit paths            10/12/07
       http://bugs.python.org/issue1274    created  bear                     
       patch                                                                   

bsddb closing a DB object before all DBCursors using it are clos 10/12/07
       http://bugs.python.org/issue1275    created  gregory.p.smith          
                                                                               

LookupError: unknown encoding: X-MAC-JAPANESE                    10/13/07
       http://bugs.python.org/issue1276    created  josm                     
       patch                                                                   

mailbox.Maildir: factory not used                                10/14/07
       http://bugs.python.org/issue1277    created  bwurst                   
                                                                               

imp.find_module() ignores -*- coding: Latin-1 -*-                10/15/07
       http://bugs.python.org/issue1278    created  tiran                    
                                                                               

os.system() oddity under Windows XP SP2                          10/15/07
       http://bugs.python.org/issue1279    created  pythonmeister            
                                                                               

PEP 3137: Make PyString's indexing and iteration return integers 10/15/07
       http://bugs.python.org/issue1280    created  alexandre.vassalotti     
       patch                                                                   

typo in documentation - lib ref section 14.3.3.4                 10/15/07
CLOSED http://bugs.python.org/issue1281    created  bsherman                 
                                                                               

re module needs to support bytes / memoryview well               10/15/07
       http://bugs.python.org/issue1282    created  gvanrossum               
                                                                               

PyBytes (buffer) .extend method needs to accept any iterable of  10/16/07
       http://bugs.python.org/issue1283    created  gregory.p.smith          
       py3k                                                                    

typo in lib doc 7.3.2.1 MaildirMessage                           10/16/07
       http://bugs.python.org/issue1284    created  encoded                  
                                                                               

setp.py error "The process cannot access the file ..."           10/16/07
       http://bugs.python.org/issue1285    created  uweber4711               
                                                                               

fileinput, StringIO, and cStringIO do not support the with proto 10/16/07
       http://bugs.python.org/issue1286    created  ygale                    
                                                                               

os.environ.pop doesn't work                                      10/16/07
       http://bugs.python.org/issue1287    created  niemeyer                 
                                                                               

dict.fromkeys - Odd logic when passing second dict.fromkeys as v 10/16/07
CLOSED http://bugs.python.org/issue1288    created  dohertywa                
                                                                               

Typo in Context Manager Types                                    10/18/07
CLOSED http://bugs.python.org/issue1289    created  nhaines                  
                                                                               

xml.dom.minidom not able to handle utf-8 data                    10/18/07
CLOSED http://bugs.python.org/issue1290    created  sharmila                 
                                                                               

test_resource fails on recent linux systems (                    10/18/07
       http://bugs.python.org/issue1291    created  doko                     
                                                                               

libffi needs an update to support mips64, arm and armeabi on lin 10/18/07
       http://bugs.python.org/issue1292    created  doko                     
                                                                               

Trailing slash in sys.path cause import failure                  10/18/07
       http://bugs.python.org/issue1293    created  guillaumegirard          
                                                                               

Management of KeyboardInterrupt in cmd.py                        10/18/07
       http://bugs.python.org/issue1294    created  stephbul                 
       patch                                                                   

logging records cache the result of formatException()            10/18/07
CLOSED http://bugs.python.org/issue1295    created  theller                  
       patch                                                                   

optparse's OptionGroup not described                             10/18/07
       http://bugs.python.org/issue1296    created  paulmelis                
                                                                               

pyconfig.h not compatible with MS VC++ Express Edition           10/18/07
       http://bugs.python.org/issue1297    created  weegreenblobbie          
                                                                               

Support for z/OS and EBCDIC.                                     10/18/07
       http://bugs.python.org/issue1298    created  lealanko                 
                                                                               

distutils.sysconfig is not cross-platform compatible             10/18/07
       http://bugs.python.org/issue1299    created  weegreenblobbie          
                                                                               

subprocess.list2cmdline doesn't do pipe symbols                  10/18/07
       http://bugs.python.org/issue1300    created  chops at demiurgestudios.com
                                                                               

Bad assertion in _tkinter.c                                      10/18/07
       http://bugs.python.org/issue1301    created  thyrsus                  
       patch                                                                   



Issues Now Closed (18)
______________________

Idle - Save (buffer) - closes IDLE and does not save file (Windo   37 days
       http://bugs.python.org/issue1130    kbk                      
       py3k                                                                    

bsddb.dbshelve.DbShelf.append doesn't work                          9 days
       http://bugs.python.org/issue1233    gregory.p.smith          
                                                                               

Removal of basestring type                                          6 days
       http://bugs.python.org/issue1258    gvanrossum               
       patch                                                                   

PEP 3137: Remove the buffer API from PyUnicode                      3 days
       http://bugs.python.org/issue1260    gvanrossum               
       patch                                                                   

PEP 3137: make bytesobject.c methods                                5 days
       http://bugs.python.org/issue1261    gregory.p.smith          
       py3k, patch                                                             

__file__ and co_filename as unicode                                 2 days
       http://bugs.python.org/issue1264    gvanrossum               
       patch                                                                   

array unittest problems with UCS4 build                             3 days
       http://bugs.python.org/issue1268    gvanrossum               
       patch                                                                   

Decode __file__ and co_filename to unicode using fs default         3 days
       http://bugs.python.org/issue1272    tiran                    
       patch                                                                   

typo in documentation - lib ref section 14.3.3.4                    3 days
       http://bugs.python.org/issue1281    georg.brandl             
                                                                               

dict.fromkeys - Odd logic when passing second dict.fromkeys as v    0 days
       http://bugs.python.org/issue1288    rhettinger               
                                                                               

Typo in Context Manager Types                                       0 days
       http://bugs.python.org/issue1289    facundobatista           
                                                                               

xml.dom.minidom not able to handle utf-8 data                       2 days
       http://bugs.python.org/issue1290    draghuram                
                                                                               

logging records cache the result of formatException()               0 days
       http://bugs.python.org/issue1295    theller                  
       patch                                                                   

Use correct encoding for printing SyntaxErrors                   1120 days
       http://bugs.python.org/issue1031213 gvanrossum               
       patch                                                                   

Irregular behavior of datetime.__str__()                         1051 days
       http://bugs.python.org/issue1074462 zooko                    
                                                                               

Add SSL certificate validation                                    985 days
       http://bugs.python.org/issue1114345 janssen                  
       patch                                                                   

httplib hangs reading too much data                               363 days
       http://bugs.python.org/issue1580738 facundobatista           
       patch                                                                   

Fix Bug 1362475 Text.edit_modified() doesn't work                 264 days
       http://bugs.python.org/issue1643641 georg.brandl             
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 30 Decode __file__ and co_filename to unicode using fs default        3 days
closed  http://bugs.python.org/issue1272   

 10 Removal of basestring type                                         6 days
closed  http://bugs.python.org/issue1258   

  9 Py3K cannot run as ``python -S``                                   8 days
open    http://bugs.python.org/issue1267   

  8 PEP 3137: make bytesobject.c methods                               5 days
closed  http://bugs.python.org/issue1261   

  7 imp.find_module() ignores -*- coding: Latin-1 -*-                  5 days
open    http://bugs.python.org/issue1278   

  6 xml.dom.minidom not able to handle utf-8 data                      2 days
closed  http://bugs.python.org/issue1290   

  5 array unittest problems with UCS4 build                            3 days
closed  http://bugs.python.org/issue1268   

  5 Irregular behavior of datetime.__str__()                        1051 days
closed  http://bugs.python.org/issue1074462

  4 implement warnings module in C                                   283 days
open    http://bugs.python.org/issue1631171

  4 Support for z/OS and EBCDIC.                                       1 days
open    http://bugs.python.org/issue1298   




From sw at gocept.com  Fri Oct 19 19:20:44 2007
From: sw at gocept.com (Sebastian Wehrmann)
Date: Fri, 19 Oct 2007 19:20:44 +0200
Subject: [Python-Dev] compile extensions under Windows Vista and VS8
Message-ID: <4718E76C.3030309@gocept.com>

Hi all,

I've got a tricky problem with a self-compiled python2.4.4 under Windows 
Vista. I compiled with Visual Studio 8 Standard (.NET 2005).

Python greets with the right header:

   Python 2.4.4 (#71, Oct 19 2007, 18:49:44) [MSC v.1400 32 bit (Intel)]
   on win32

I started the Visual Studio Shell and tried to get some extensions using 
setuptools (easy_install):

   python ez_setup.py readline
   [...]
   Best match: readline 2.4.2
   [..Downloading..]
   Running readline-2.4.2\setup.py -q bdist_egg --dist-dir
   c:\users\sweh\appdata\local\temp\easy_install-nnn9di\
   readline-2.4.2\egg-dist-tmp-fqezb_
   error: Setup script exited with error: Python was built with version
   8.0 of Visual Studio, and extensions need to be built with the same
   version of the compiler, but it isn't installed.

Does anyone know, why setuptools cannot find VS8? I'm sure the system 
variables are set correctly as in %VCDIR%\bin\vcvars32.bat.

regards,
-- 
Sebastian Wehrmann

gocept gmbh & co. kg ? forsterstrasse 29 ? 06112 halle/saale ? germany
www.gocept.com ? work. +49 345 122988912 ? fax. +49 345 12298891

From theller at ctypes.org  Fri Oct 19 20:26:08 2007
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 19 Oct 2007 20:26:08 +0200
Subject: [Python-Dev] Buildbot failure AMD64 trunk
Message-ID: <ffass0$ht3$1@ger.gmane.org>

The 64-bit windows trunk buildbot now only fails the test_winsound test.
This is because for whatever reasons the machine cannot play any sounds.
I have no idea why this is so, and I'm not too inclined to fix this.  The
buildbot is running Window XP 64-bit in a vmware image running under ubuntu.

Is there a way to display the winsound test on this machine?  I'm annoyed
by the red color ;-).

Thanks,
Thomas


From theller at ctypes.org  Fri Oct 19 20:38:35 2007
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 19 Oct 2007 20:38:35 +0200
Subject: [Python-Dev] Buildbot failure AMD64 trunk
In-Reply-To: <ffass0$ht3$1@ger.gmane.org>
References: <ffass0$ht3$1@ger.gmane.org>
Message-ID: <ffatjb$k75$1@ger.gmane.org>

Thomas Heller schrieb:
> The 64-bit windows trunk buildbot now only fails the test_winsound test.
> This is because for whatever reasons the machine cannot play any sounds.
> I have no idea why this is so, and I'm not too inclined to fix this.  The
> buildbot is running Window XP 64-bit in a vmware image running under ubuntu.
> 
> Is there a way to display the winsound test on this machine?  I'm annoyed
> by the red color ;-).

I mean: Is there a way to DISABLE the winsound test?

Sorry for the confusion...


From martin at v.loewis.de  Fri Oct 19 23:26:22 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 19 Oct 2007 23:26:22 +0200
Subject: [Python-Dev] Buildbot failure AMD64 trunk
In-Reply-To: <ffass0$ht3$1@ger.gmane.org>
References: <ffass0$ht3$1@ger.gmane.org>
Message-ID: <471920FE.3050501@v.loewis.de>

> The 64-bit windows trunk buildbot now only fails the test_winsound test.
> This is because for whatever reasons the machine cannot play any sounds.
> I have no idea why this is so, and I'm not too inclined to fix this.  The
> buildbot is running Window XP 64-bit in a vmware image running under ubuntu.
> 
> Is there a way to display the winsound test on this machine?  I'm annoyed
> by the red color ;-).

If you uninstall the sound driver, then the test should determine that
there is no sound card, and skip it.

Regards,
Martin

From martin at v.loewis.de  Fri Oct 19 23:28:09 2007
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 19 Oct 2007 23:28:09 +0200
Subject: [Python-Dev] compile extensions under Windows Vista and VS8
In-Reply-To: <4718E76C.3030309@gocept.com>
References: <4718E76C.3030309@gocept.com>
Message-ID: <47192169.6080206@v.loewis.de>

> Does anyone know, why setuptools cannot find VS8? 

Python 2.5 does not support being built with VS8, and distutils
"knows" that. The official compiler to build Python 2.5 is
VS 7.1.

There are environment variables to tell distutils otherwise;
see the documentation or the source for details. This question
is off-topic for python-dev.

Regards,
Martin

From facundobatista at gmail.com  Sat Oct 20 00:22:35 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 19 Oct 2007 19:22:35 -0300
Subject: [Python-Dev] Deadlock by a second import in a thread
Message-ID: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>

Hi!

I was looking to this bug:   http://bugs.python.org/issue1255

It basically creates a deadlock in Python by doing the following:

- aa.py imports bb.py
- bb.py imports time and generates a thread
- the thread uses time.strptime

The deadlock is because the strptime function imports another module,
line 517 of timemodule.c:

      PyObject *strptime_module = PyImport_ImportModule("_strptime");

This situation is well known, found a lot of references to this
import-thread-import problem in discussions and previous bugs (i.e.:
http://bugs.python.org/issue683658).

What I did *not* find, and why I'm asking here, is how to solve it.

Exists a known solution to this?

Thank you!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From exarkun at divmod.com  Sat Oct 20 00:45:47 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 19 Oct 2007 18:45:47 -0400
Subject: [Python-Dev] incompatible unittest changes
In-Reply-To: 0
Message-ID: <20071019224547.8162.1779249233.divmod.quotient.25351@ohm>

In trunk after 2.5, equality and hashing for TestCase were added, changing the behavior so that two instances of TestCase for the same test method hash the same and compare equal.  This means two instances of TestCase for the same test method cannot be added to a single set.

Here's the change:

http://svn.python.org/view/python/trunk/Lib/unittest.py?rev=54199&r1=42115&r2=54199

The implementations aren't even very good, since they prevent another type from deciding that it wants to customize comparison against TestCase (or TestSuite,
or FunctionTestCase) instances.

Is there a real use case for this functionality?  If not, I'd like it to be
removed to restore the old behavior.

Jean-Paul

From collinw at gmail.com  Sat Oct 20 00:51:51 2007
From: collinw at gmail.com (Collin Winter)
Date: Fri, 19 Oct 2007 15:51:51 -0700
Subject: [Python-Dev] incompatible unittest changes
In-Reply-To: <20071019224547.8162.1779249233.divmod.quotient.25351@ohm>
References: <20071019224547.8162.1779249233.divmod.quotient.25351@ohm>
Message-ID: <43aa6ff70710191551w75efd681x113c38ff1fe0280a@mail.gmail.com>

On 10/19/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
> In trunk after 2.5, equality and hashing for TestCase were added, changing the behavior so that two instances of TestCase for the same test method hash the same and compare equal.  This means two instances of TestCase for the same test method cannot be added to a single set.
>
> Here's the change:
>
> http://svn.python.org/view/python/trunk/Lib/unittest.py?rev=54199&r1=42115&r2=54199
>
> The implementations aren't even very good, since they prevent another type from deciding that it wants to customize comparison against TestCase (or TestSuite,
> or FunctionTestCase) instances.

The implementations have been changed in a more recent revision.

> Is there a real use case for this functionality?  If not, I'd like it to be
> removed to restore the old behavior.

The use-case was problems I encountered when writing the test suite
for unittest. If you can find a way to implement the functionality you
want *and* keep the test suite reasonably straightforward, I'll be
happy to review your patch.

Collin Winter

From rhamph at gmail.com  Sat Oct 20 00:52:44 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 19 Oct 2007 16:52:44 -0600
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
Message-ID: <aac2c7cb0710191552g3b84e3fbxf1cd6add441a6f20@mail.gmail.com>

On 10/19/07, Facundo Batista <facundobatista at gmail.com> wrote:
> Hi!
>
> I was looking to this bug:   http://bugs.python.org/issue1255
>
> It basically creates a deadlock in Python by doing the following:
>
> - aa.py imports bb.py
> - bb.py imports time and generates a thread
> - the thread uses time.strptime
>
> The deadlock is because the strptime function imports another module,
> line 517 of timemodule.c:
>
>       PyObject *strptime_module = PyImport_ImportModule("_strptime");
>
> This situation is well known, found a lot of references to this
> import-thread-import problem in discussions and previous bugs (i.e.:
> http://bugs.python.org/issue683658).
>
> What I did *not* find, and why I'm asking here, is how to solve it.
>
> Exists a known solution to this?

When python encounters a recursive import within a single thread it
allows you to get access to partially-imported modules, making the
assumption that you won't do any significant work until the entire
import process completes.

Only one thread is allowed to do an import at a time though, as
they'll do significant work with it immediately, so being
partially-imported would be a race condition.

Writing a python file as a script means you do significant work in the
body, rather than in a function called after importing completes.
Importing this python file then violates the assumption for
single-threaded recursive imports, and creating threads then violates
their safety assumptions.

The solution then is, if your python file will ever be imported, you
must write a main function and do all the work there instead.  Do not
write it in the style of a script (with significant work in the global
scope.)

-- 
Adam Olsen, aka Rhamphoryncus

From exarkun at divmod.com  Sat Oct 20 01:01:14 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Fri, 19 Oct 2007 19:01:14 -0400
Subject: [Python-Dev] incompatible unittest changes
In-Reply-To: <43aa6ff70710191551w75efd681x113c38ff1fe0280a@mail.gmail.com>
Message-ID: <20071019230114.8162.1279873014.divmod.quotient.25362@ohm>

On Fri, 19 Oct 2007 15:51:51 -0700, Collin Winter <collinw at gmail.com> wrote:
>On 10/19/07, Jean-Paul Calderone <exarkun at divmod.com> wrote:
>> In trunk after 2.5, equality and hashing for TestCase were added, changing the behavior so that two instances of TestCase for the same test method hash the same and compare equal.  This means two instances of TestCase for the same test method cannot be added to a single set.
>>
>> Here's the change:
>>
>> http://svn.python.org/view/python/trunk/Lib/unittest.py?rev=54199&r1=42115&r2=54199
>>
>> The implementations aren't even very good, since they prevent another type from deciding that it wants to customize comparison against TestCase (or TestSuite,
>> or FunctionTestCase) instances.
>
>The implementations have been changed in a more recent revision.

Not in http://svn.python.org/projects/python/trunk/Lib/unittest.py at 58552

>
>> Is there a real use case for this functionality?  If not, I'd like it to be
>> removed to restore the old behavior.
>
>The use-case was problems I encountered when writing the test suite
>for unittest. If you can find a way to implement the functionality you
>want *and* keep the test suite reasonably straightforward, I'll be
>happy to review your patch.

The test suite can implement the comparison which is currently on the
unittest classes and invoke that functionality instead of using == and
!=.

Jean-Paul

From facundobatista at gmail.com  Sat Oct 20 01:12:23 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 19 Oct 2007 20:12:23 -0300
Subject: [Python-Dev] Fwd:  Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710191611x505a18bdj84e20dd2ab16d67b@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<aac2c7cb0710191552g3b84e3fbxf1cd6add441a6f20@mail.gmail.com>
	<e04bdf310710191611x505a18bdj84e20dd2ab16d67b@mail.gmail.com>
Message-ID: <e04bdf310710191612i2b9e2b6ct5ec12fcb2610f863@mail.gmail.com>

2007/10/19, Adam Olsen <rhamph at gmail.com>:

> The solution then is, if your python file will ever be imported, you
> must write a main function and do all the work there instead.  Do not
> write it in the style of a script (with significant work in the global
> scope.)

I had this a as a good coding style, not so mandatory.

I agree with you that the OP shouldn't be doing that, but note that
the main problem arises here because it's completely unpredictable the
import in strptime for an external user.

Do you recommend to close the bug as "won't fix" saying something like...

    The deadlock happens because strptime has an import inside it, and
    recursive imports are not allowed in different threads.

    As a general rule and good coding style, don't run your code when the
    module is imported, but put it in a function like "main" in the second file,
    import it and call it from the first one. This will solve your problem.

    Note that this happens to you with strptime, but could happen with a lot
    of functions that do this internal import of something else. So,
you'll never
    be sure.

What do you think?

Thank you!

--
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From lists at cheimes.de  Sat Oct 20 01:13:53 2007
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 20 Oct 2007 01:13:53 +0200
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
Message-ID: <ffbdni$88d$1@ger.gmane.org>

Facundo Batista wrote:
> What I did *not* find, and why I'm asking here, is how to solve it.
> 
> Exists a known solution to this?

I know a possible solution. You could write a patch that moves the
imports in C code to the module init function and stores the modules in
a global static variable.

Christian


From rhamph at gmail.com  Sat Oct 20 01:22:33 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 19 Oct 2007 17:22:33 -0600
Subject: [Python-Dev] Fwd: Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710191612i2b9e2b6ct5ec12fcb2610f863@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<aac2c7cb0710191552g3b84e3fbxf1cd6add441a6f20@mail.gmail.com>
	<e04bdf310710191611x505a18bdj84e20dd2ab16d67b@mail.gmail.com>
	<e04bdf310710191612i2b9e2b6ct5ec12fcb2610f863@mail.gmail.com>
Message-ID: <aac2c7cb0710191622m2f8fd9e1r88faf24174b4f24@mail.gmail.com>

On 10/19/07, Facundo Batista <facundobatista at gmail.com> wrote:
> 2007/10/19, Adam Olsen <rhamph at gmail.com>:
>
> > The solution then is, if your python file will ever be imported, you
> > must write a main function and do all the work there instead.  Do not
> > write it in the style of a script (with significant work in the global
> > scope.)
>
> I had this a as a good coding style, not so mandatory.
>
> I agree with you that the OP shouldn't be doing that, but note that
> the main problem arises here because it's completely unpredictable the
> import in strptime for an external user.
>
> Do you recommend to close the bug as "won't fix" saying something like...
>
>     The deadlock happens because strptime has an import inside it, and
>     recursive imports are not allowed in different threads.
>
>     As a general rule and good coding style, don't run your code when the
>     module is imported, but put it in a function like "main" in the second file,
>     import it and call it from the first one. This will solve your problem.
>
>     Note that this happens to you with strptime, but could happen with a lot
>     of functions that do this internal import of something else. So,
> you'll never
>     be sure.
>
> What do you think?

Whether this is a minor problem due to poor style or a major problem
due to a language defect is a matter of perspective.  I'm working on
redesigning Python's threading support, expecting it to be used a
great deal more, which'd push it into the major problem category.

For now I'd leave it open.

-- 
Adam Olsen, aka Rhamphoryncus

From ncoghlan at gmail.com  Sat Oct 20 06:11:57 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 20 Oct 2007 14:11:57 +1000
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
Message-ID: <4719800D.8020208@gmail.com>

Facundo Batista wrote:
> Hi!
> 
> I was looking to this bug:   http://bugs.python.org/issue1255
> 
> It basically creates a deadlock in Python by doing the following:
> 
> - aa.py imports bb.py
> - bb.py imports time and generates a thread

bb.py is broken - importing a module should never spawn a new thread as 
a side effect (precisely because it will deadlock if the spawned thread 
tries to do an import, which can happen in a myriad of ways).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From facundobatista at gmail.com  Sat Oct 20 16:20:20 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Sat, 20 Oct 2007 11:20:20 -0300
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <ffbdni$88d$1@ger.gmane.org>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<ffbdni$88d$1@ger.gmane.org>
Message-ID: <e04bdf310710200720j35833ef4qbdc6a3be4b94f8db@mail.gmail.com>

2007/10/19, Christian Heimes <lists at cheimes.de>:

> I know a possible solution. You could write a patch that moves the
> imports in C code to the module init function and stores the modules in
> a global static variable.

I thought about this. It even will have the good side efect of not
shooting the whole "import" machinery to see that you already imported
it, every time you do an strptime.

One question: The program makes this:

    PyObject *strptime_module = PyImport_ImportModule("_strptime");
    ...
    Py_DECREF(strptime_module);

If I'd import it in the beggining of the file with the following...

    static PyObject *strptime_module = PyImport_ImportModule("_strptime");

... I'd never decref it, right?

Thank you!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From facundobatista at gmail.com  Sat Oct 20 16:22:34 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Sat, 20 Oct 2007 11:22:34 -0300
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <4719800D.8020208@gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<4719800D.8020208@gmail.com>
Message-ID: <e04bdf310710200722v3f7fe041jdfa7403a5f949bfa@mail.gmail.com>

2007/10/20, Nick Coghlan <ncoghlan at gmail.com>:

> bb.py is broken - importing a module should never spawn a new thread as
> a side effect (precisely because it will deadlock if the spawned thread
> tries to do an import, which can happen in a myriad of ways).

Agreed.

But if we move the import of the _strptime module to do it once when
"time" is imported, we enhance the "least surprise" situation (and has
a marginal performance benefit of not trying to import it again every
time.strptime() call).

And even solves this bug, which does not hurt enaybody, ;)

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From mrmakent at cox.net  Sun Oct 14 23:37:58 2007
From: mrmakent at cox.net (Michael Kent)
Date: Sun, 14 Oct 2007 21:37:58 +0000 (UTC)
Subject: [Python-Dev] Python developers are in demand
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
Message-ID: <loom.20071014T213627-256@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:

> I wonder if we should start maintaining a list of Python developers
> for hire somewhere on python.org, beyond the existing Jobs page. Is
> anyone interested in organizing this?

I would be definitely interested in putting my name on such a list. 





From g.brandl at gmx.net  Sat Oct 20 20:21:37 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 20 Oct 2007 20:21:37 +0200
Subject: [Python-Dev] New chapter "Using Python" in the docs
Message-ID: <ffdgvk$ggi$1@ger.gmane.org>

Hi,

as you may have seen in the checkins, I just created a new chapter
in the docs called "Using Python."

We, the doc team, would like that section to contain, roughly,
information how to install Python on different platforms, how to
configure it, how to invoke it, what extra things and quirks you
should know about, etc.

Currently, there are two sections: one about the Python command-line
options, newly distilled from the manpage and other sources docs, and the
old "Using Python on Mac" document which had been in the HOWTO section
since the migration.

Now it would of course be nice to have at least two other documents
for Unixy platforms and Windows...

I don't think this is not written down somewhere, but it isn't in the
official docs currently -- so it'd be wonderful if you could help us
to collect this information, give it a proper shape and add it to
the new chapter.

Even links to wiki pages, blog articles etc. can help!

Cheers,
Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From brett at python.org  Sat Oct 20 20:57:28 2007
From: brett at python.org (Brett Cannon)
Date: Sat, 20 Oct 2007 11:57:28 -0700
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710200720j35833ef4qbdc6a3be4b94f8db@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<ffbdni$88d$1@ger.gmane.org>
	<e04bdf310710200720j35833ef4qbdc6a3be4b94f8db@mail.gmail.com>
Message-ID: <bbaeab100710201157i25c313aage6f6e14764375bdb@mail.gmail.com>

On 10/20/07, Facundo Batista <facundobatista at gmail.com> wrote:
> 2007/10/19, Christian Heimes <lists at cheimes.de>:
>
> > I know a possible solution. You could write a patch that moves the
> > imports in C code to the module init function and stores the modules in
> > a global static variable.
>
> I thought about this. It even will have the good side efect of not
> shooting the whole "import" machinery to see that you already imported
> it, every time you do an strptime.
>
> One question: The program makes this:
>
>     PyObject *strptime_module = PyImport_ImportModule("_strptime");
>     ...
>     Py_DECREF(strptime_module);
>
> If I'd import it in the beggining of the file with the following...
>
>     static PyObject *strptime_module = PyImport_ImportModule("_strptime");
>
> ... I'd never decref it, right?

Right.  Otherwise, if the module is removed from sys.modules then it
will have a refcount of 0 and be collected, leaving your static
variable holding junk memory.

One issue with this approach, though, is that the import is a one-time
thing, and so replacing what is in sys.modules['_strptime'] will not
take affect in the module ever thanks to the caching of the module's
dict.

Granted this is a small issue as normal Python code does not pick up
changes in sys.modules, but it is something to be aware of.

-Brett

From daniel.stutzbach at gmail.com  Fri Oct 19 17:36:58 2007
From: daniel.stutzbach at gmail.com (Daniel Stutzbach)
Date: Fri, 19 Oct 2007 10:36:58 -0500
Subject: [Python-Dev] C Decimal - is there any interest?
In-Reply-To: <e04bdf310710190735x4a394e05rab756c49334ebc70@mail.gmail.com>
References: <47138E98.5060200@vp.pl>
	<5c6f2a5d0710160705i44431441ob55332fbdc0001e2@mail.gmail.com>
	<4714D7AE.10504@vp.pl>
	<5c6f2a5d0710160924y39ee39a5sfb99bc157437bf32@mail.gmail.com>
	<eae285400710161417m5803c489la324c8beeab27b47@mail.gmail.com>
	<5c6f2a5d0710161746i3a0c1b82ka9d9ec8ab2ad49cd@mail.gmail.com>
	<eae285400710161922nebf31aavdbf1c625a5606ca8@mail.gmail.com>
	<e04bdf310710190735x4a394e05rab756c49334ebc70@mail.gmail.com>
Message-ID: <eae285400710190836l15a17455ke78fb6e08c57c939@mail.gmail.com>

On 10/19/07, Facundo Batista <facundobatista at gmail.com> wrote:
> 2007/10/16, Daniel Stutzbach <daniel.stutzbach at gmail.com>:
> > I agree.  A basic subquadratic radix conversion algorithm isn't much
> > more complex than the existing quadratic code.  I just whipped
> > together a Python prototype and it's only 15 lines.
>
> Do you have a patch for decimal.py of current trunk?

I don't, though I could make one.

However, currently the int->Decimal conversion takes place in C via
str().  Here's the relevant line from decimal.py:

            self._int = tuple(map(int, str(abs(value))))

Doing some simple experiments, a Python subquadratic routine doesn't
make a big pay off until around 25,000 decimal digits.  On the plus
side, the extra overhead of the additional routine is small and I
didn't observe a noticeable performance penalty for small inputs (my
routine calls str() as a base case for values less than 2**31-1).

So... would the community rather have a Python patch for decimal.py or
a patch for subquadratic versions of long_format and PyLong_FromString
in longobject.c?  The later will be faster and will speed up some
non-Decimal operations as well.

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From derek.shockey at gmail.com  Sat Oct 20 23:00:38 2007
From: derek.shockey at gmail.com (Derek Shockey)
Date: Sat, 20 Oct 2007 17:00:38 -0400
Subject: [Python-Dev] Bug in smtpd.SMTPChannel.smtp_MAIL
Message-ID: <5728eb900710201400m4a78bebt7d6592b5facc02ac@mail.gmail.com>

smtpd.SMTPChannel contains a bug such that when connected to an SMTPServer
(or any subclass thereof), issuing a MAIL command with no argument closes
the socket and gives this error on the server:

error: uncaptured python exception, closing channel <smtpd.SMTPChannelconnected
127.0.0.1:58587 at 0x847d8> (<type 'exceptions.TypeError'>:'NoneType' object
is unsubscriptable
[/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py|read|68]
[/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py|handle_read_event|390]
[/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asynchat.py|handle_read|137]
[/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|found_terminator|158]
[/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|smtp_MAIL|224]
[/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|__getaddr|212])

The desired result is of course is to respond with a 501 Syntax error. The
problem arises because the arg parameter passed to each smtp_* command
handler function is None when there is no argument. arg is passed on to
__getaddr which attempts a slice on the None object.

I think the most elegant solution would be to insert a "if not arg" check in
__getaddr and return. The existing smtp_MAIL code will then issue the 501
Syntax error.


-Derek Shockey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071020/bca186ab/attachment.htm 

From guido at python.org  Sun Oct 21 17:57:10 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 21 Oct 2007 08:57:10 -0700
Subject: [Python-Dev] Bug in smtpd.SMTPChannel.smtp_MAIL
In-Reply-To: <5728eb900710201400m4a78bebt7d6592b5facc02ac@mail.gmail.com>
References: <5728eb900710201400m4a78bebt7d6592b5facc02ac@mail.gmail.com>
Message-ID: <ca471dc20710210857t2e5cddder2e7c8b32424a06ff@mail.gmail.com>

Could you submit a patch to the bug tracker (bugs.python.org)?

2007/10/20, Derek Shockey <derek.shockey at gmail.com>:
> smtpd.SMTPChannel contains a bug such that when connected to an SMTPServer
> (or any subclass thereof), issuing a MAIL command with no argument closes
> the socket and gives this error on the server:
>
> error: uncaptured python exception, closing channel < smtpd.SMTPChannel
> connected 127.0.0.1:58587 at 0x847d8> (<type
> 'exceptions.TypeError'>:'NoneType' object is unsubscriptable
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py|read|68]
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py|handle_read_event|390]
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asynchat.py|handle_read|137]
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|found_terminator|158]
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|smtp_MAIL|224]
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|__getaddr|212])
>
> The desired result is of course is to respond with a 501 Syntax error. The
> problem arises because the arg parameter passed to each smtp_* command
> handler function is None when there is no argument. arg is passed on to
> __getaddr which attempts a slice on the None object.
>
> I think the most elegant solution would be to insert a "if not arg" check in
> __getaddr and return. The existing smtp_MAIL code will then issue the 501
> Syntax error.
>
>
> -Derek Shockey
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From derek.shockey at gmail.com  Mon Oct 22 07:26:44 2007
From: derek.shockey at gmail.com (Derek Shockey)
Date: Mon, 22 Oct 2007 01:26:44 -0400
Subject: [Python-Dev] Bug in smtpd.SMTPChannel.smtp_MAIL
In-Reply-To: <ca471dc20710210857t2e5cddder2e7c8b32424a06ff@mail.gmail.com>
References: <5728eb900710201400m4a78bebt7d6592b5facc02ac@mail.gmail.com>
	<ca471dc20710210857t2e5cddder2e7c8b32424a06ff@mail.gmail.com>
Message-ID: <5728eb900710212226v20033e74x6905ed3fc1abdda3@mail.gmail.com>

I did actually submit a patch last night after I emailed the list. The issue
ID is 1307.

I used the ternary operator (because I like it and because it was the
shortest solution), but if this is not desirable for backwards
compatibility, I could obviously rewrite it another way.

-Derek Shockey


On 10/21/07, Guido van Rossum <guido at python.org> wrote:
>
> Could you submit a patch to the bug tracker (bugs.python.org)?
>
> 2007/10/20, Derek Shockey <derek.shockey at gmail.com>:
> > smtpd.SMTPChannel contains a bug such that when connected to an
> SMTPServer
> > (or any subclass thereof), issuing a MAIL command with no argument
> closes
> > the socket and gives this error on the server:
> >
> > error: uncaptured python exception, closing channel < smtpd.SMTPChannel
> > connected 127.0.0.1:58587 at 0x847d8> (<type
> > 'exceptions.TypeError'>:'NoneType' object is unsubscriptable
> >
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py|read|68]
> >
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asyncore.py|handle_read_event|390]
> >
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/asynchat.py|handle_read|137]
> >
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|found_terminator|158]
> >
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|smtp_MAIL|224]
> >
> [/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/smtpd.py|__getaddr|212])
> >
> > The desired result is of course is to respond with a 501 Syntax error.
> The
> > problem arises because the arg parameter passed to each smtp_* command
> > handler function is None when there is no argument. arg is passed on to
> > __getaddr which attempts a slice on the None object.
> >
> > I think the most elegant solution would be to insert a "if not arg"
> check in
> > __getaddr and return. The existing smtp_MAIL code will then issue the
> 501
> > Syntax error.
> >
> >
> > -Derek Shockey
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-dev/guido%40python.org
> >
> >
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071022/bf308018/attachment.htm 

From skip at pobox.com  Mon Oct 22 14:15:45 2007
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 22 Oct 2007 07:15:45 -0500
Subject: [Python-Dev] Does Python need a file locking module (slightly
	higher level)?
Message-ID: <18204.38001.562050.157197@montanaro.dyndns.org>


I'm always daunted by the prospect of trying to implement file locking.
This just came up again in SpamBayes where we have never protected our
pickle files from corruption when multiple processes access them
simultaneously.  The presence of networked file systems and
platform-independent locks make it a nasty little problem.  Maybe I'm just
showing my age.  Does fcntl.flock work over NFS and SMB and on Windows?  If
this is still as much of a mess as I remember, should Python provide a
simple file locking module in the standard distribution?

Side note: While reading the fcntl man page on my Mac I came across this
text in the description of F_GETLK/F_SETLK/F_SETLKW.

    This interface follows the completely stupid semantics of System V and
    IEEE Std 1003.1-1988 (``POSIX.1'') that require that all locks
    associated with a file for a given process are removed when any file
    descriptor for that file is closed by that process....  Flock(2) is
    recommended for applications that want to ensure the integrity of their
    locks when using library routines or wish to pass locks to their
    children.

I guess the BSD folks were a bit upset when they wrote that.

Skip

From lealanko at ssh.com  Mon Oct 22 14:45:56 2007
From: lealanko at ssh.com (Lauri Alanko)
Date: Mon, 22 Oct 2007 15:45:56 +0300
Subject: [Python-Dev] Python 2.5.1 ported to z/OS and EBCDIC
Message-ID: <20071022124556.GI30705@dhcp-54-30.hel.fi.ssh.com>

Hello.

Based on Jean-Yves Mengant's work on previous versions, I have ported
Python 2.5.1 to z/OS. A patch against current svn head is attached to
<http://bugs.python.org/issue1298>. The same patch should work with very
little changes also against pristine 2.5.1 sources. (The only failing
hunk is for Modules/makesetup, and it is quite trivial.)

I have no opinion on whether the patch should eventually be incorporated
into the main distribution. The port was motivated by internal reasons,
and I'm merely offering it as a community service to anyone else who
might be interested. If Jean-Yves wishes to distribute it from his
z/OS-page, that is fine with me. In general, anyone can do what they
want with the patch, but please give credit.

I'll describe some of the porting issues below.


CHARACTER SETS
==============

The biggest, major difficulty with z/OS is of course the character set.
There are lots of ASCII-dependencies in Python code, and z/OS uses
CP1047, an EBCDIC variant, which is utterly incompatible with ASCII.

There are two possible approaches in this situation. One is to keep on
using ASCII as the execution character set (and also as the default
encoding of string objects), and to add conversion support to everywhere
where we do text-based I/O, so that communication with the external
world still happens in EBCDIC. This was feasible since the z/OS C
compiler does support ASCII as the execution character set. (The source
character set would still remain EBCDIC, though. If you've ever wondered
why the C standard makes a distinction between these, here's a prime
example of a situation where they're different.)

However, I decided against this approach. The I/O conversions would have
been deeply magical, and would have required classic "text mode vs.
binary mode" -crap, which would be rather confusing.

Instead, I followed Jean-Yves' example and kept Python as a "native"
EBCDIC application: input, 8-bit data is treated by default as EBCDIC
everywhere. This only required fixing various ASCII-specific bits in the
code, e.g. stuff like this (in PyString_DecodeEscape):

-		else if (c < ' ' || c >= 0x7f)
+		else if (!isprint((unsigned char) c))

Of course, now this allows unescaped printing of characters if they are
printable in the platform's encoding even if they wouldn't be printable
in ASCII. I'm not sure if this is desirable or not. It would be simple
to fix this so that only characters in the ASCII _character set_ are
displayed varbatim.

A result of making strings EBCDIC-native is that it breaks any code that
depends on string literals being in ASCII. This probably applies to most
network protocol implementations written in Python. On the other hand,
making string literals use ASCII would break code that does ordinary
text processing on local files. Damned if you do, damned if you don't.

The real issue is that strings in Python are rather underspecified.
String objects are really just octet sequences without any _inherent_
textual interpretation for them. This is apparent from the fact that
strings are what are read from and written to binary files, and also
what unicode strings are encoded to and decoded from. However, Python
syntax allows specifying an octet sequence with a _character_ sequence
(i.e. a string literal), and the relationship between the source
characters and the resulting octets has been left implicit. So
programmers aren't really encouraged to think about character set issues
and the end result is code that only works on a platform that uses ASCII
everywhere.

Python already has the property that the meaning of a source file
depends on its encoding: if I write a string literal with some latin-1
characters, the resulting octet sequence depends on whether my source
was encoded in latin-1 or utf-8. I'm not sure if this is a good idea,
but my approach with the z/OS port continues the tradition: when your
source is in EBCDIC, the string literals get encoded in EBCDIC.

All this just shows that treating plain octet sequences as "strings"
simply won't work in the long run. You have to have separate type for
_textual_ data (i.e. Unicode strings, in Python), and encode and decode
between those and octet sequences using some _explicit_ encoding. Of
course, all non-English-speaking people have been keenly aware of this
already for ages. The relative universality of ASCII is an exception
amongst encodings rather than the norm. It's only reasonable to require
English text to require the same attention to encodings as all the other
languages.


UNICODE
-------

The biggest hurdle by far (at least LoC-wise) in the porting was
Unicode. The code assumed that the execution character set was not only
ASCII, but ISO-8859-1, since there was lots of casting back and forth
between Py_UNICODE and char. I added the following conversion operations
into unicodeobject.h:

#ifdef Py_CHARSET_ASCII
# define Py_UNICODE_FROM_CHAR(c) ((Py_UNICODE)(unsigned char)(c))
# define Py_UNICODE_AS_CHAR(u) (u < 0x80 ? (char)(unsigned char)(u) : '\0')
#else
# define Py_UNICODE_FROM_CHAR(c) _PyUnicode_FromChar(c)
# define Py_UNICODE_AS_CHAR(u) _PyUnicode_AsChar(u)
#endif

The Py_UNICODE_AS_CHAR operation maps a unicode character into a char in
the execution character set's encoding, or to '\0' if it's not
representable.

When on a non-ASCII platform, I used the simplest trick of all:

/* Map from ASCII codes to the platform's execution character set, or to
   '\0' if the corresponding character is not known. */
static const char unicode_ascii_table[128] =
    "\0\0\0\0\0\0\0\a\b\t\n\v\f\r\0\0"
    "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"
    " !\"#$%&'()*+,-./0123456789:;<=>?"
    "@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_"
    "`abcdefghijklmnopqrstuvwxyz{|}~\0";

(This is reasonably portable, as all the printable ASCII characters
except `, @ and $ are required by C to be present in any source or
execution character set, and of those, Python requires all but $.)

This, and the corresponding reverse index, are good enough for all
purposes in the Python core: converting unicode string literals into
unicode objects, detecting special escape characters, and calculating
digit values. It doesn't allow writing string or unicode literals that
directly contain characters that don't exist in ASCII, though. But since
such code wouldn't be portable across character sets anyway, this isn't
much of a problem.

I also added a Lib/encodings/cp1047.py that does proper recoding outside
the core. It was generated from jdk-1.5.0/CP1047.TXT (from
<http://haible.de/bruno/charsets/conversion-tables/CP1047.html>). This
map seems to best correspond to the actual conventions I have seen on a
z/OS machine.

Now, strings and unicode seem to work together fairly well, even though
the results may be a bit surprising to one used to ASCII and its
extensions:

>>> ord('a')
129

Here 129 is the EBCDIC value of the letter 'a'. The unicode literal
u'a', like all textual input, is itself represented in EBCDIC:

>>> map(ord,"u'a'")
[164, 125, 129, 125]

But when such a literal is parsed, the resulting unicode object has the 
correct value for the corresponding unicode character:

>>> ord(u'a')
97

And, of course, when this unicode literal is printed back or its repr is
taken, it is again encoded to EBCDIC so it shows correctly:

>>> map(ord,repr(u'a'))
[164, 125, 129, 125]

This seems to me to be the Right Thing. Now, as long as no exotic
characters are used directly in the source, source can be translated
between ASCII and EBCDIC so that strings and unicode strings retain
their correct semantic character values, even though the encoding of the
literals themselves is different. String objects have a
platform-dependent encoding, but unicode objects behave the same
everywhere.

One problem with this approach is that it is completely incompatible
with Python's UTF-8 support. The parser assumes that utf-8 (or latin-1)
are supersets of the platform's native encoding, and this of course
isn't true with EBCDIC.

A consequence is that the z/OS port cannot support eval of unicode
strings:

>>> eval('2+2')
4
>>> eval(u'2+2')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 1
    
    ^
SyntaxError: invalid syntax

This is because internally evaluation of unicode strings is implemented
by first encoding the unicode string as utf-8, and then trying to parse
that. And this of course fails.

This seems like a rather complicated and limited way of going about it.
It would be much cleaner and more portable to first decode input into
unicode by various means, and then to parse the unicode. Then unicode
strings would be the ones that don't need any special processing. But
this would require heavy changes to Python's parsing machinery, and I
tried to keep my changes as minimal as possible for now.


PICKLING
--------

One more character set issue arised with pickling. The pickle protocols
are a bit schitzophrenic in the sense that they can't quite decide
whether to be textual or binary protocols. A textual protocol should be
readable, and recodable across platforms to preserve the semantic
character values correctly, whereas a binary protocol should be based on
specific octet values whose readability is not an issue.

The original pickle protocol 0 can be seen either as a textual protocol
(all the pickles are readable), or a binary protocol (when characters
get mapped to their corresponding octet values in ASCII). The other
protocol versions, though extensions of protocol 0, are clearly binary,
since the pickled data is at least partially specified as specific octet
values.

Now, on an EBCDIC platform, it's impossible to have protocol 0 be
textual while still compatible with the other protocols. This is because
e.g. the following opcodes get the same value if we let 'a' be textual
(i.e. encoded in the host platforms's encoding):

APPEND          = 'a'   # append stack top to list below it
NEWOBJ          = '\x81'  # build object by applying cls.__new__ to argtuple

In the end, for now, I made protocol 0 textual, and disabled support for
protocol versions > 0 on non-ASCII platforms. This seems like the safest
choice. It's certainly possible to add support for the binary protocols
and make them explicitly use ASCII, but that again would require
non-trivial changes.

Incidentally, modified_EncodeRawUnicodeEscape in cPickle.c seems to be
out of sync with the one in unicodeobject.c, in that it lacks support
for Py_UNICODE_WIDE. Also, both versions generate a latin-1 string as
output, which doesn't seem portable enough. My patch recodes characters
in ASCII to the execution character set, and escapes everything else,
even characters in U+0080 - U+00FF -range. (Though strictly, all the
latin-1 characters happen to be representable in CP1047. But this is not
something that I think it's good to depend upon.)


INTEGER PARSING
---------------

There were quite a number of places where (hex) digits were parsed
nonportably. I added the following to longobject.h, and used that:

PyAPI_FUNC(int) _PyLong_DigitValue(char c);

This resulted in some nice cleanups. From PyString_DecodeEscape:

-				unsigned int x = 0;
-				c = Py_CHARMASK(*s);
-				s++;
-				if (isdigit(c))
-					x = c - '0';
-				else if (islower(c))
-					x = 10 + c - 'a';
-				else
-					x = 10 + c - 'A';
-				x = x << 4;
-				c = Py_CHARMASK(*s);
-				s++;
-				if (isdigit(c))
-					x += c - '0';
-				else if (islower(c))
-					x += 10 + c - 'a';
-				else
-					x += 10 + c - 'A';
-				*p++ = x;
+				int xh = _PyLong_DigitValue(*s++);
+				int xl = _PyLong_DigitValue(*s++);
+				*p++ = Py_CHARMASK(xh * 16 + xl);
 				break;


OTHER ISSUES
============

Most of the other changes are boring build-technical issues and tweaks
to make things compile on z/OS's very spartan support for Unix-like
facilities. I hard-coded various #ifdef __MVS__ bits here and there to
make things compile. I guess these things should properly be checked by
configure, but I'm not very good at autoconf magic, and besides, running
configure takes _ages_ on the machine I'm using, so I wasn't inclined to
tweak the scripts any more than I had to.

The dynamic loading support in dynload_mvs.c is verbatim from Jean-Yves'
modifications. I just cleaned it up a little.

I have only tested this with --enable-shared (which does what
--with-zdll did in Jean-Yves' version, i.e. enables shared libraries).
Without shared libraries the building of extensions may well fail
because of some linkage tweaks in Lib/distutils/unixccompiler.py. I hope
there is some way of deciding what to do depending on whether shared
libraries are enabled or not.

One nasty difficulty was that the makefile implicitly assumes that
shared libraries are named libpython2.x.dll only on Windows. However,
they have that name on z/OS, too. I resolved this with a simple "case
$(MACHDEP)" in the rule for building the library, but hopefully someone
can come up with a prettier solution.

Various wrappers for external libraries are untested. Certainly it might
be possible to install zlib, libbz2, openssl and various other nifty
libraries on z/OS, and see if the Python wrappers work, but that is an
undertaking that I will pass at least for now.

Quite a number of tests fail simply because they assume that strings are
encoded in ASCII. For instance, Lib/test/test_calendar.py fails because
the expected result is:

result_2004_html = """
<?xml version="1.0" encoding="ascii"?>
... """

And the real result begins with:

<?xml version="1.0" encoding="cp1047"?> ...

There were so many of these kinds of failures that there may be some
_actual_ problems amongs them that I've overlooked.

That is about all. Comments are welcome. I'd be especially interested in
hearing if my patch works on any other machine besides the one I was
using. :)

-- 
Lauri Alanko                                           Software Engineer
SSH Communications Security Corp                Mobile: +358-40-864-3037 
Valimotie 17, FI-00380, Helsinki, Finland          Tel: +358-20-500-7000
http://www.ssh.com/                                Fax: +358-20-500-7001

From barry at python.org  Mon Oct 22 15:37:03 2007
From: barry at python.org (Barry Warsaw)
Date: Mon, 22 Oct 2007 09:37:03 -0400
Subject: [Python-Dev] Does Python need a file locking module (slightly
	higher level)?
In-Reply-To: <18204.38001.562050.157197@montanaro.dyndns.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
Message-ID: <2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 22, 2007, at 8:15 AM, skip at pobox.com wrote:

> I'm always daunted by the prospect of trying to implement file  
> locking.
> This just came up again in SpamBayes where we have never protected our
> pickle files from corruption when multiple processes access them
> simultaneously.  The presence of networked file systems and
> platform-independent locks make it a nasty little problem.  Maybe  
> I'm just
> showing my age.  Does fcntl.flock work over NFS and SMB and on  
> Windows?  If
> this is still as much of a mess as I remember, should Python provide a
> simple file locking module in the standard distribution?

If you want something like this, you might start by looking at  
Mailman's LockFile.py.  It has a particular set of semantics (such as  
lock breaking) that you might not be interested in, but it is, or can  
be, mostly de-Mailmanized for use as a general library module.  In  
the particular use case it is designed for, it's been quite stable  
for many years.  Essentially it provides an NFS-safe lock file  
implementation.

- -Barry

http://codebrowse.launchpad.net/~mailman-coders/mailman/3.0/annotate/ 
barry%40python.org-20071011032203-w1j8qrmtlpkrvay4? 
file_id=mailmanlockfile.py-20070507165525-0o0kligrooe34vyc-172

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iQCVAwUBRxyngHEjvBPtnXfVAQKPOwP+JhuIC2LiOsHDPLtAft4bSMaYC1qfVJqG
q6SXFc8yJauE9zKttPcn9kkbgONj3RYbDJ9qW4aVA7fJfHEiRDbW8omp/e7rTELl
fIonBDnIk5XEo5bL/JslMudgInOa6BY7yGzCKjaRRy19wSmOZ8ptroXfOvLgqF+e
n7WVkh82sD8=
=aFzw
-----END PGP SIGNATURE-----

From benji at benjiyork.com  Mon Oct 22 17:07:48 2007
From: benji at benjiyork.com (Benji York)
Date: Mon, 22 Oct 2007 11:07:48 -0400
Subject: [Python-Dev] Does Python need a file locking module (slightly
 higher level)?
In-Reply-To: <2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
	<2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>
Message-ID: <471CBCC4.6000200@benjiyork.com>

Barry Warsaw wrote:
> On Oct 22, 2007, at 8:15 AM, skip at pobox.com wrote:
> 
>> I'm always daunted by the prospect of trying to implement file  
>> locking.

> If you want something like this, you might start by looking at  
> Mailman's LockFile.py.

Also related is the very simple zc.lockfile: 
http://pypi.python.org/pypi/zc.lockfile
-- 
Benji York
http://benjiyork.com

From oliphant.travis at ieee.org  Tue Oct 23 00:48:40 2007
From: oliphant.travis at ieee.org (Travis Oliphant)
Date: Mon, 22 Oct 2007 17:48:40 -0500
Subject: [Python-Dev] Patch for adding offset to mmap
Message-ID: <ffj9c7$am2$1@ger.gmane.org>

Hi all,

I think the latest patch for fixing Issue 708374 (adding offset to mmap)
should be committed to SVN.

I will do it, if nobody opposes the plan.  I think it is a very
important addition and greatly increases the capability of the mmap module.

Thanks,

-Travis Oliphant


P.S.  Initially sent this to the wrong group (I've been doing that a lot 
lately --- too many groups seen through gmane...).  Apologies for 
multiple postings.


From greg.ewing at canterbury.ac.nz  Tue Oct 23 01:16:41 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 23 Oct 2007 12:16:41 +1300
Subject: [Python-Dev] Does Python need a file locking module (slightly
 higher level)?
In-Reply-To: <18204.38001.562050.157197@montanaro.dyndns.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
Message-ID: <471D2F59.6000102@canterbury.ac.nz>

skip at pobox.com wrote:
>     This interface follows the completely stupid semantics of System V and
>     IEEE Std 1003.1-1988 (``POSIX.1'') that require that all locks
>     associated with a file for a given process are removed when any file
>     descriptor for that file is closed by that process....  Flock(2) is
>     recommended for applications that want to ensure the integrity of their
>     locks when using library routines or wish to pass locks to their
>     children.
> 
> I guess the BSD folks were a bit upset when they wrote that.

That sounds more like something written by the GNU folks than
the BSD folks.

--
Greg

From greg.ewing at canterbury.ac.nz  Tue Oct 23 01:29:35 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 23 Oct 2007 12:29:35 +1300
Subject: [Python-Dev] Does Python need a file locking module (slightly
 higher level)?
In-Reply-To: <18204.38001.562050.157197@montanaro.dyndns.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
Message-ID: <471D325F.2030806@canterbury.ac.nz>

skip at pobox.com wrote:
> Does fcntl.flock work over NFS and SMB and on Windows?

I don't think file locking will ever work over NFS, since
it's a stateless protocol by design, and locking would
require maintaining state on the server.

--
Greg

From greg.ewing at canterbury.ac.nz  Tue Oct 23 01:32:06 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 23 Oct 2007 12:32:06 +1300
Subject: [Python-Dev] Python 2.5.1 ported to z/OS and EBCDIC
In-Reply-To: <20071022124556.GI30705@dhcp-54-30.hel.fi.ssh.com>
References: <20071022124556.GI30705@dhcp-54-30.hel.fi.ssh.com>
Message-ID: <471D32F6.60506@canterbury.ac.nz>

Lauri Alanko wrote:
> In the end, for now, I made protocol 0 textual,

That could be a mistake. I believe there are some objects,
such as array.array, that use a binary format for pickling
even in protocol 0.

--
Greg

From tjreedy at udel.edu  Tue Oct 23 01:43:13 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 22 Oct 2007 19:43:13 -0400
Subject: [Python-Dev] Python 2.5.1 ported to z/OS and EBCDIC
References: <20071022124556.GI30705@dhcp-54-30.hel.fi.ssh.com>
Message-ID: <ffjcig$jdm$1@ger.gmane.org>


"Lauri Alanko" <lealanko at ssh.com> wrote in message 
news:20071022124556.GI30705 at dhcp-54-30.hel.fi.ssh.com...
| All this just shows that treating plain octet sequences as "strings"
| simply won't work in the long run. You have to have separate type for
| _textual_ data (i.e. Unicode strings, in Python), and encode and decode
| between those and octet sequences using some _explicit_ encoding. Of
| course, all non-English-speaking people have been keenly aware of this
| already for ages. The relative universality of ASCII is an exception
| amongst encodings rather than the norm. It's only reasonable to require
| English text to require the same attention to encodings as all the other
| languages.

Setting aside any debate over 'reasonable', I believe the above pretty well 
matchess the design for Py3, except that Guido prefers 'byte' to 'octet'. 
You may end up preferring Py3 for your port.  3.0a2 should be out in a few 
weeks.

| PICKLING
| In the end, for now, I made protocol 0 textual, and disabled support for
| protocol versions > 0 on non-ASCII platforms.

I believe that the plan for pickle in 3.0 is to read any protocol but only 
write the latest (2?), but perhaps this was only a suggestion.





From jon+python-dev at unequivocal.co.uk  Tue Oct 23 02:08:03 2007
From: jon+python-dev at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 23 Oct 2007 01:08:03 +0100
Subject: [Python-Dev] Does Python need a file locking module (slightly
	higher level)?
In-Reply-To: <471D2F59.6000102@canterbury.ac.nz>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
	<471D2F59.6000102@canterbury.ac.nz>
Message-ID: <20071023000803.GA11016@snowy.squish.net>

On Tue, Oct 23, 2007 at 12:16:41PM +1300, Greg Ewing wrote:
> skip at pobox.com wrote:
> >     This interface follows the completely stupid semantics of System V and
> >     IEEE Std 1003.1-1988 (``POSIX.1'') that require that all locks
> >     associated with a file for a given process are removed when any file
> >     descriptor for that file is closed by that process....  Flock(2) is
> >     recommended for applications that want to ensure the integrity of their
> >     locks when using library routines or wish to pass locks to their
> >     children.
> > 
> > I guess the BSD folks were a bit upset when they wrote that.
> 
> That sounds more like something written by the GNU folks than
> the BSD folks.

It's from BSD. The earliest I can find it is 4.4BSD. It's still in the
latest versions of OpenBSD, NetBSD and FreeBSD.

From jon+python-dev at unequivocal.co.uk  Tue Oct 23 02:11:39 2007
From: jon+python-dev at unequivocal.co.uk (Jon Ribbens)
Date: Tue, 23 Oct 2007 01:11:39 +0100
Subject: [Python-Dev] Does Python need a file locking module (slightly
	higher level)?
In-Reply-To: <471D325F.2030806@canterbury.ac.nz>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
	<471D325F.2030806@canterbury.ac.nz>
Message-ID: <20071023001139.GB11016@snowy.squish.net>

On Tue, Oct 23, 2007 at 12:29:35PM +1300, Greg Ewing wrote:
> skip at pobox.com wrote:
> > Does fcntl.flock work over NFS and SMB and on Windows?
> 
> I don't think file locking will ever work over NFS, since
> it's a stateless protocol by design, and locking would
> require maintaining state on the server.

You can do file locking over NFS, that's one of the reasons people
use fcntl. It uses an RPC side channel separate to the main NFS
protocol.

From exarkun at divmod.com  Tue Oct 23 03:52:00 2007
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 22 Oct 2007 21:52:00 -0400
Subject: [Python-Dev] Does Python need a file locking module (slightly
 higher level)?
In-Reply-To: <20071023001139.GB11016@snowy.squish.net>
Message-ID: <20071023015200.8162.1589024794.divmod.quotient.26634@ohm>

On Tue, 23 Oct 2007 01:11:39 +0100, Jon Ribbens <jon+python-dev at unequivocal.co.uk> wrote:
>On Tue, Oct 23, 2007 at 12:29:35PM +1300, Greg Ewing wrote:
>> skip at pobox.com wrote:
>> > Does fcntl.flock work over NFS and SMB and on Windows?
>>
>> I don't think file locking will ever work over NFS, since
>> it's a stateless protocol by design, and locking would
>> require maintaining state on the server.
>
>You can do file locking over NFS, that's one of the reasons people
>use fcntl. It uses an RPC side channel separate to the main NFS
>protocol.

You can do it.  It just doesn't work.  (You could say the same about
regular read and write operations for many NFS implementations, though)

Jean-Paul

From skip at pobox.com  Tue Oct 23 05:30:58 2007
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 22 Oct 2007 22:30:58 -0500
Subject: [Python-Dev] Does Python need a file locking module (slightly
 higher level)?
In-Reply-To: <2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
	<2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>
Message-ID: <18205.27378.432540.931896@montanaro.dyndns.org>


    >> I'm always daunted by the prospect of trying to implement file
    >> locking.

    Barry> If you want something like this, you might start by looking at
    Barry> Mailman's LockFile.py.

If I interpret the Python documentation for os.link correctly, the scheme
used by Mailman won't work on Windows (os.link isn't advertised as being
available there).  Nevertheless, the pointer to the Linux open(2) man page
was a good start.  Implementing something for Unix-y systems is not too
difficult using that advice.

Jean-Paul Calderone sent me a pointer to Twisted's file locking module.  I'm
still trying to figure out exactly what it does on Windows.  Something about
making and populating directories.  (os.mkdir is the replacement for
os.link?)

Benji York referred me to zc.lockfile.  That appears to use fcntl.flock.
Based on Jean-Paul's response it seems the jury is still out on whether
fcntl.flock works over NFS.  zc.lockfile at least has something specifically
for Windows.  Whether or not msvcrt.locking() works on networked file
systems remains to be seen.

Mailman's lockfile makes provision to block for a user-specified period of
time.  The other's push the waiting back onto the calling code.

It's not clear that any of these implementations is going to be perfect.
Maybe none ever will be.  In his reply Jean-Paul made this comment:

    It might be nice to have something like that in the standard library,
    but it's very simple once you know what to do.

I'm not so sure about the "very simple" part, especially if you aren't
familiar with all the ins and outs of the different platforms.  The fact
that the first three bits of code I was referred to were implemented by
three significant Python tools/platforms and that all are different in some
significant ways suggests that there is some both an underlying need for a
file locking mechanism but with a lack of consensus about the best way to
implement the mother-of-all-file-locking schemes for Python.  Maybe the best
place for this is in the distribution.  PEP?

Skip

From facundobatista at gmail.com  Tue Oct 23 22:44:27 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 23 Oct 2007 17:44:27 -0300
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <e04bdf310709131208v307bb273gd47a48779b4db61b@mail.gmail.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>
	<e04bdf310709131208v307bb273gd47a48779b4db61b@mail.gmail.com>
Message-ID: <e04bdf310710231344j166934ffy6b9194ef6f2e5c41@mail.gmail.com>

2007/9/13, Facundo Batista <facundobatista at gmail.com>:

> All the listings are accesible from the same pages, start here:
>
>   http://www.taniquetil.com.ar/facundo/py_tickets.html
>
> (remember to refresh)
>
> Any idea to improve these pages is welcomed.

Following an idea of John Lenton, now every page shows a graph of the
activity per day of all open tickets.

Look at the bottom:

  http://www.taniquetil.com.ar/facundo/py_tickets.html

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From theller at ctypes.org  Wed Oct 24 16:05:17 2007
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 24 Oct 2007 16:05:17 +0200
Subject: [Python-Dev] bug submitting procedure
Message-ID: <ffnjet$bnu$1@ger.gmane.org>

I've just received a private email from Christian Jacobsen (we were discussing
some ctypes bugs/deficiencies that do not matter in this context).  He wrote:

> [...] The bug
> reporting procedures for documentation is a big inconsistent:
> http://wiki.python.org/moin/SubmittingBugs, says: "If you find errors
> in the documentation, please use either the Add a comment or the
> Suggest a change features of the relevant page in the most recent
> online documentation at  http://docs.python.org/.", but the most
> recent online documentation points to the SF bugtracker or
> docs at python.org. The SF bugtracker in points back to bugs.python.org
> :)

I feel with him.  Further, there is no 'Add a comment' or 'Suggest a change' link
in the 2.5 documentation shown at http://docs.python.org.

Thomas


From g.brandl at gmx.net  Wed Oct 24 16:17:09 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 24 Oct 2007 16:17:09 +0200
Subject: [Python-Dev] bug submitting procedure
In-Reply-To: <ffnjet$bnu$1@ger.gmane.org>
References: <ffnjet$bnu$1@ger.gmane.org>
Message-ID: <ffnk1q$d4g$1@ger.gmane.org>

Thomas Heller schrieb:
> I've just received a private email from Christian Jacobsen (we were discussing
> some ctypes bugs/deficiencies that do not matter in this context).  He wrote:
> 
>> [...] The bug
>> reporting procedures for documentation is a big inconsistent:
>> http://wiki.python.org/moin/SubmittingBugs, says: "If you find errors
>> in the documentation, please use either the Add a comment or the
>> Suggest a change features of the relevant page in the most recent
>> online documentation at  http://docs.python.org/.", but the most
>> recent online documentation points to the SF bugtracker or
>> docs at python.org. The SF bugtracker in points back to bugs.python.org
>> :)
> 
> I feel with him.  Further, there is no 'Add a comment' or 'Suggest a change' link
> in the 2.5 documentation shown at http://docs.python.org.

Yes, that's unfortunate -- seems that somebody copied the information
from the development docs, but the feature is not present there yet.

At the moment, all bugs should be submitted to bugs.python.org.
The 2.5 docs still point to SF because they have not been rebuilt
since the move -- Fred said he'd do a rebuild some time ago.

Georg


From facundobatista at gmail.com  Wed Oct 24 16:14:48 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 24 Oct 2007 11:14:48 -0300
Subject: [Python-Dev] bug submitting procedure
In-Reply-To: <ffnjet$bnu$1@ger.gmane.org>
References: <ffnjet$bnu$1@ger.gmane.org>
Message-ID: <e04bdf310710240714t3c8015f6o6b238108d9feccdb@mail.gmail.com>

2007/10/24, Thomas Heller <theller at ctypes.org>:

> I've just received a private email from Christian Jacobsen (we were discussing
> some ctypes bugs/deficiencies that do not matter in this context).  He wrote:
>
> ...
>
> I feel with him.  Further, there is no 'Add a comment' or 'Suggest a change' link
> in the 2.5 documentation shown at http://docs.python.org.

+1 to change it and forward the reader to bugs.python.org for both
documentation issues and other kind of problems.

I can change it, but I'd prefer a english-speaker to change it. But
feel free to tell me to do it.

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From facundobatista at gmail.com  Wed Oct 24 16:22:24 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 24 Oct 2007 11:22:24 -0300
Subject: [Python-Dev] 2.5.2 is coming
In-Reply-To: <ee2a432c0710112232u658f0addl91c391c4f5b0c1d3@mail.gmail.com>
References: <ee2a432c0710112232u658f0addl91c391c4f5b0c1d3@mail.gmail.com>
Message-ID: <e04bdf310710240722m2b1a5ea5p3168a5e4caa8aaf4@mail.gmail.com>

2007/10/12, Neal Norwitz <nnorwitz at gmail.com>:

> The plan is cut the release candidate around Tuesday/Wednesday next
> week (Oct 16/17).  If all goes well, 2.5.2 final will follow a week
> later.

Hi Neal! Do you have any update of this schedule?

Thank you!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From rrr at ronadam.com  Wed Oct 24 18:53:21 2007
From: rrr at ronadam.com (Ron Adam)
Date: Wed, 24 Oct 2007 11:53:21 -0500
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>	
	<e04bdf310709191341l289c034fjb4bf6d6b88ae4ef9@mail.gmail.com>	
	<46F1B6FD.70007@ronadam.com>
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>
Message-ID: <471F7881.1070605@ronadam.com>



Facundo Batista wrote:
> 2007/9/19, Ron Adam <rrr at ronadam.com>:
> 
>> I noticed that there is a background of light blue between marks.  That is
>> hard to see on my computer because it is so close to the grey tone.
> 
> Made it a little darker, now it's easier to look.
> 
> 
>> Also shouldn't the light blue background bar extend all the way to the end
>> for all open items?
> 
> No, because this "light blue bar" is the span of time from opened to
> last comment.
> 
> Note that these items are *all* open.

I think the page title should reflect this.  Possible changing it from

     "Python tickets"

to
     "Python Open Tickets"



Ron

From facundobatista at gmail.com  Wed Oct 24 22:23:55 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 24 Oct 2007 17:23:55 -0300
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
Message-ID: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>

Hi, people!

I'm following the issue 1311: http://bugs.python.org/issue1311

There (and always talking in windows), the OP says the in Py2.4
os.path.exists("nul") returned True and now in 2.5 returns False. Note
that "nul" is an special file, something like /dev/null.

We made some tests, and we have inconsisten behaviour in previous
Python versions. For example, in Py2.3.5 in my machine I get a False,
as in Py2.5. But other person in the bug, gets True in 2.3.3 and
False in 2.5.

Even the OP has differents result for the same Python 2.4 in different machines.

Right now (but don't know exactly since when), Python relies in
kernel32.dll functions to make the stat on the file (if stat raises an
error, os.path.exists says that the file does not exist). Of course,
if I call to this function separately, I have the same behaviour.

So, the question is what we should do?:

1. Rely on the kernel32 function and behaves like it says?

2. Return a fixed response for this special file "nul"?

Personally, I prefer the first one, but it changed the semantic of
os.path.exists("nul") (but this semantic is not clear, as we get
different behaviour in different Python versions and windows
versions).

Thank you very much!

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From fdrake at acm.org  Wed Oct 24 23:34:45 2007
From: fdrake at acm.org (Fred Drake)
Date: Wed, 24 Oct 2007 17:34:45 -0400
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
Message-ID: <C59810A4-6440-46CA-AAB4-5D0E894D2A08@acm.org>

On Oct 24, 2007, at 4:23 PM, Facundo Batista wrote:
> There (and always talking in windows), the OP says the in Py2.4
> os.path.exists("nul") returned True and now in 2.5 returns False. Note
> that "nul" is an special file, something like /dev/null.

It's special, but in a different way.  /dev/null really exists in the  
Unix filesystem; "nul" is more magical than that.

What's more, it has peers: "prn", "com1" and others like that.

I don't know what the right way to handle these is (I'm no Windows  
guru, or even regular user), but it's important to realize that the  
pain of the specialness isn't limited.  :-)


   -Fred

-- 
Fred Drake   <fdrake at acm.org>




From nick.bastin at gmail.com  Wed Oct 24 23:43:25 2007
From: nick.bastin at gmail.com (Nicholas Bastin)
Date: Wed, 24 Oct 2007 15:43:25 -0600
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
Message-ID: <66d0a6e10710241443j127cb3bcwb96e7d82e2f6f12b@mail.gmail.com>

On 10/12/07, Guido van Rossum <guido at python.org> wrote:
> I keep getting regular requests from people looking for Python coders
> (and this is in addition to Google asking me to hand over my contacts
> :-). This is good news because it suggests Python is on the uptake
> (always good to know). At the same time it is disturbing because
> apparently there aren't enough Python programmers out there. (At least
> none of them looking for work.) What's up with that?

At least from my perspective, all the jobs are in web applications,
and all the Python developers I know are traditional applications
programmers, not web developers.

--
Nick

From mhammond at skippinet.com.au  Wed Oct 24 23:42:58 2007
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Thu, 25 Oct 2007 07:42:58 +1000
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
Message-ID: <048e01c81686$da84e3b0$8f8eab10$@com.au>

> So, the question is what we should do?:
> 
> 1. Rely on the kernel32 function and behaves like it says?
> 
> 2. Return a fixed response for this special file "nul"?
> 
> Personally, I prefer the first one, but it changed the semantic of
> os.path.exists("nul") (but this semantic is not clear, as we get
> different behaviour in different Python versions and windows
> versions).

Note that the same issue would exist for 'aux', 'con' and 'prn' too -
'comXX' 'lptXX' 'clock$' also seem to get special treatment.  I agree it is
unfortunate that the behaviour has changed, but these special names are
broken enough on Windows that (1) seems the sanest thing to do.

Mark


From eric+python-dev at trueblade.com  Thu Oct 25 00:06:22 2007
From: eric+python-dev at trueblade.com (Eric Smith)
Date: Wed, 24 Oct 2007 18:06:22 -0400
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <C59810A4-6440-46CA-AAB4-5D0E894D2A08@acm.org>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
	<C59810A4-6440-46CA-AAB4-5D0E894D2A08@acm.org>
Message-ID: <471FC1DE.8060903@trueblade.com>

Fred Drake wrote:
> On Oct 24, 2007, at 4:23 PM, Facundo Batista wrote:
>> There (and always talking in windows), the OP says the in Py2.4
>> os.path.exists("nul") returned True and now in 2.5 returns False. Note
>> that "nul" is an special file, something like /dev/null.
> 
> It's special, but in a different way.  /dev/null really exists in the  
> Unix filesystem; "nul" is more magical than that.
> 
> What's more, it has peers: "prn", "com1" and others like that.

It's even worse than that, because file extensions are ignored in this 
magical-ness:

C:\Documents and Settings\User>type nul

C:\Documents and Settings\User>type nul.lst

C:\Documents and Settings\User>type foo.lst
The system cannot find the file specified.

> I don't know what the right way to handle these is (I'm no Windows  
> guru, or even regular user), but it's important to realize that the  
> pain of the specialness isn't limited.  :-)

http://www.microsoft.com/technet/prodtechnol/Windows2000Pro/reskit/part3/proch17.mspx?mfr=true 
gives the list as CON, AUX, COM1, COM2, COM3, COM4, LPT1, LPT2, LPT3, 
PRN, NUL; but I can't imagine testing against that list would be the 
best idea.  For example, 
http://www.microsoft.com/technet/solutionaccelerators/cits/interopmigration/unix/unixbld/unixbld4.mspx 
adds CLOCK$, among others (although I don't find CLOCK$ to be special, 
it's rumored to be an NT only thing, and I'm running XP).  So I think 
implementing Facundo's option 2 (test for "nul") will not work in the 
general case for finding "special files" (don't forget to throw in mixed 
case names).  I hate to think of trying to match Windows' behavior if 
there are multiple dots in the name.

I think I'd leave the current behavior of calling the kernel function, 
even though it varies based on Windows version (if I'm reading the issue 
correctly).

Eric.



From scott+python-dev at scottdial.com  Thu Oct 25 00:07:46 2007
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Wed, 24 Oct 2007 18:07:46 -0400
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <C59810A4-6440-46CA-AAB4-5D0E894D2A08@acm.org>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
	<C59810A4-6440-46CA-AAB4-5D0E894D2A08@acm.org>
Message-ID: <471FC232.70005@scottdial.com>

Fred Drake wrote:
> It's special, but in a different way.  /dev/null really exists in the  
> Unix filesystem; "nul" is more magical than that.
> 
> What's more, it has peers: "prn", "com1" and others like that.
> 

For the record, the fixed names 'aux', 'con', 'nul', and 'prn' along 
with the set of 'com[0-9]' and 'lpt[0-9]' names that are reserved. And 
for that matter, any of those with an extension is reserved as well. 
These files always exist as far as I am concerned (where existence is 
defined by your ability to open() them).

def is_special_on_win32(name):
     import os.path, re
     name = os.path.basename(name)
     return (re.match('(nul|prn|aux|con|com[0-9]|lpt[0-9])(\..*)?$',
                      name) is not None)

-- 
Scott Dial
scott at scottdial.com
scodial at cs.indiana.edu

From guido at python.org  Thu Oct 25 00:15:14 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 24 Oct 2007 15:15:14 -0700
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <66d0a6e10710241443j127cb3bcwb96e7d82e2f6f12b@mail.gmail.com>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
	<66d0a6e10710241443j127cb3bcwb96e7d82e2f6f12b@mail.gmail.com>
Message-ID: <ca471dc20710241515q5054d840ha255a3b5d5eee9ba@mail.gmail.com>

2007/10/24, Nicholas Bastin <nick.bastin at gmail.com>:
> On 10/12/07, Guido van Rossum <guido at python.org> wrote:
> > I keep getting regular requests from people looking for Python coders
> > (and this is in addition to Google asking me to hand over my contacts
> > :-). This is good news because it suggests Python is on the uptake
> > (always good to know). At the same time it is disturbing because
> > apparently there aren't enough Python programmers out there. (At least
> > none of them looking for work.) What's up with that?
>
> At least from my perspective, all the jobs are in web applications,
> and all the Python developers I know are traditional applications
> programmers, not web developers.

Get a new set of friends. :-)

You can find them on web-sig at python.org.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aleaxit at gmail.com  Thu Oct 25 01:09:38 2007
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 24 Oct 2007 16:09:38 -0700
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <470FCB07.2030807@v.loewis.de>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
	<470FBB20.3020006@cheimes.de> <470FCB07.2030807@v.loewis.de>
Message-ID: <e8a0972d0710241609r226f7c93ua3abc4ca5f7158f6@mail.gmail.com>

On 10/12/07, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > The problem may be related to the fact that Python is rarely teached at
> > school or university. I know no school or university in Germany that is
> > teaching Python.
>
> I teach Python to the first semester, at the Hasso-Plattner-Institut
> in Potsdam, for the third year now.

My wife (and fellow PSF member;-) Anna, a major in Symbolic Systems at
Stanford, has noticed the interesting coincidence that TWO of her
courses this quarter suggest Python as the preferred language to do
the course assignments (although both courses accept other languages
as well, focusing on the results and not the language used to achieve
them, the teachers think that Python is the best language to "get out
of your way" and let you focus on the courses' specific subjects
rather than on programming problems).  The two courses are
Computational Linguistics and Computer-Human Interactions.

The CHI course also offers a short optional Python seminar for
students that want help learning it (I believe the exercises
specifically use Nokia phones, so I assume the seminar will also cover
the specifics of the Nokia Python development environment); Anna
volunteered to do a similar short seminar for the CL course (I helped
out -- took us a Saturday).  All students taking CHI and/or CL have
already taken programming courses (typically more than one), mostly
using C++ and Java (and often C), but as far as I know there is no
Stanford course (at least not within Symbolic Systems) that focuses
specifically and exclusively on Python (there IS one course,
Programming Paradigms, that covers Python as well as Lisp, Prolog and
some FP language).

Of course, Symbolic Systems majors typically don't think of themselves
as "developers"; they're more likely to end up, say, as "CHI experts",
"computational linguists", and the like...


Alex

From skip at pobox.com  Thu Oct 25 04:12:40 2007
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 24 Oct 2007 21:12:40 -0500
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <66d0a6e10710241443j127cb3bcwb96e7d82e2f6f12b@mail.gmail.com>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
	<66d0a6e10710241443j127cb3bcwb96e7d82e2f6f12b@mail.gmail.com>
Message-ID: <18207.64408.368072.465602@montanaro.dyndns.org>


    Nick> At least from my perspective, all the jobs are in web
    Nick> applications, and all the Python developers I know are traditional
    Nick> applications programmers, not web developers.

I find almost the opposite to be true.  Most resumes I see with Python
experience are quite web-focused.  For the open positions in our group I'd
much rather see experience doing scientific or database programming with
Python than web apps.

Skip

From janssen at parc.com  Thu Oct 25 05:02:53 2007
From: janssen at parc.com (Bill Janssen)
Date: Wed, 24 Oct 2007 20:02:53 PDT
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <e8a0972d0710241609r226f7c93ua3abc4ca5f7158f6@mail.gmail.com> 
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
	<470FBB20.3020006@cheimes.de> <470FCB07.2030807@v.loewis.de>
	<e8a0972d0710241609r226f7c93ua3abc4ca5f7158f6@mail.gmail.com>
Message-ID: <07Oct24.200256pdt."57996"@synergy1.parc.xerox.com>

Application programmers...  Web programmers...

I can't resist chiming in that I'm running a 4000-line Python
application on my iPhone that is both a full-blown application, and a
Web server, because it uses the phone's browser as its application
GUI.  (By the way, thanks to whoever pushed through the addition of
SQLite to Python 2.5.  It's been useful on the phone.)

I teach an annual short course on Python at PARC; two hours, one for
the basics, the other for tricks.  That's about all it takes, for
folks who already know how to program.

Bill

From martin at v.loewis.de  Thu Oct 25 06:05:51 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 25 Oct 2007 06:05:51 +0200
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
Message-ID: <4720161F.7040306@v.loewis.de>

> So, the question is what we should do?:

Before this question can be answered, I think we need to fully
understand what precisely is happening in 2.4, and what precisely
is happening in 2.5.

AFAICT, it is *not* the case that Python 2.4 (indirectly) has
hard-coded the names CON, PRN, NUL etc. in the C library. Instead,
Python 2.4 *also* relies on kernel32 functions to determine that
these files "exist".

My question now is what specific kernel32 functions Python 2.4
calls to determine that NUL is a file; before that question
is sufficiently answered, I don't think any action should be
taken.

Regards,
Martin

From nde at comp.leeds.ac.uk  Thu Oct 25 12:58:46 2007
From: nde at comp.leeds.ac.uk (Nick Efford)
Date: Thu, 25 Oct 2007 11:58:46 +0100
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <mailman.55.1193306415.32757.python-dev@python.org>
References: <mailman.55.1193306415.32757.python-dev@python.org>
Message-ID: <472076E6.3090307@comp.leeds.ac.uk>

Interesting to see discussion on supply and demand issues for
Python programmers.  You might be interested to learn that,
after a few years of flirting with Python in various ways, the
School of Computing at the University of Leeds has recently
switched to teaching Python as the first and primary programming
language for undergraduates on all of our degree programmes.

I know we're not the only ones doing this, so perhaps the
supply will rise to meet the demand in a few years!


Nick

-- 
Dr Nick Efford, School of | E: nde at comp.leeds.ac.uk
Computing, University of  | T: +44 113 343 6809
Leeds, Leeds, LS2 9JT, UK | W: http://www.comp.leeds.ac.uk/nde/
--------------------------+-----------------------------------------
PGP fingerprint: 6ADF 16C2 4E2D 320B F537  8F3C 402D 1C78 A668 8492


From facundobatista at gmail.com  Thu Oct 25 14:13:48 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 25 Oct 2007 09:13:48 -0300
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <e8a0972d0710241609r226f7c93ua3abc4ca5f7158f6@mail.gmail.com>
References: <ca471dc20710121024m249a7162t71eb82a938b7bea3@mail.gmail.com>
	<470FBB20.3020006@cheimes.de> <470FCB07.2030807@v.loewis.de>
	<e8a0972d0710241609r226f7c93ua3abc4ca5f7158f6@mail.gmail.com>
Message-ID: <e04bdf310710250513j75099e7cj7e9d6fb662902472@mail.gmail.com>

2007/10/24, Alex Martelli <aleaxit at gmail.com>:

> using C++ and Java (and often C), but as far as I know there is no
> Stanford course (at least not within Symbolic Systems) that focuses
> specifically and exclusively on Python (there IS one course,

In my constant try-to-push-Python-everywhere-I-go, I offered several
times Python courses to educational institutions (sometimes even
free).

I succeeded some times, but then these courses not thrived year after
year. Normally, this is because the people that is actually taking the
decision of which language to teach in the courses do not know Python,
so is easier to them to keep teaching C.  And this happens even if
it's not the better to the students, and even witht the students
asking for the change.

But this is a problem of educative system here in Argentina, not of
Python itself (it surely get affected, though).

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From facundobatista at gmail.com  Thu Oct 25 15:11:28 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 25 Oct 2007 10:11:28 -0300
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <471F7881.1070605@ronadam.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>
	<e04bdf310709191341l289c034fjb4bf6d6b88ae4ef9@mail.gmail.com>
	<46F1B6FD.70007@ronadam.com>
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>
	<471F7881.1070605@ronadam.com>
Message-ID: <e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>

2007/10/24, Ron Adam <rrr at ronadam.com>:

> > Note that these items are *all* open.
>
> I think the page title should reflect this.  Possible changing it from
>
>      "Python tickets"
>
> to
>      "Python Open Tickets"

Good point! It's fixed now.

Thank you!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From home at antroy.co.uk  Thu Oct 25 14:40:45 2007
From: home at antroy.co.uk (Anthony Roy)
Date: Thu, 25 Oct 2007 13:40:45 +0100 (BST)
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <472076E6.3090307@comp.leeds.ac.uk>
References: <mailman.55.1193306415.32757.python-dev@python.org>
	<472076E6.3090307@comp.leeds.ac.uk>
Message-ID: <9558.194.200.199.193.1193316045.squirrel@webmail.plus.net>


> Interesting to see discussion on supply and demand issues for
> Python programmers.  You might be interested to learn that,
> after a few years of flirting with Python in various ways, the
> School of Computing at the University of Leeds has recently
> switched to teaching Python as the first and primary programming
> language for undergraduates on all of our degree programmes.
>
> I know we're not the only ones doing this, so perhaps the
> supply will rise to meet the demand in a few years!

I was a researcher in the School of Computing at Leeds Uni about 4 years
ago. Good to see them pushing Python!

I keep my eyes open for Python Developer roles in the UK (particularly the
North), since I would far prefer to develop in Python than Java. However,
in the UK Python Jobs seem to be few and far between, and most of the ones
that there are are either low paid sys admin type roles, or are based in
London.

Cheers,

-- 
Anthony Roy


From bioinformed at gmail.com  Thu Oct 25 16:18:04 2007
From: bioinformed at gmail.com (Kevin Jacobs <jacobs@bioinformed.com>)
Date: Thu, 25 Oct 2007 10:18:04 -0400
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <9558.194.200.199.193.1193316045.squirrel@webmail.plus.net>
References: <mailman.55.1193306415.32757.python-dev@python.org>
	<472076E6.3090307@comp.leeds.ac.uk>
	<9558.194.200.199.193.1193316045.squirrel@webmail.plus.net>
Message-ID: <2e1434c10710250718v3b9d72esf5ee79be7d96a51d@mail.gmail.com>

Just to chime in from the other side of the coin.

I'm actively trying to hire qualified scientific programmers with strong
Python experience.  Unfortunately, I've had little success finding
candidates with actual Python knowledge, resorting mainly to hiring those
who've seen it and can readily learn it on the job.  So while it is
encouraging that Python is being used as an introductory language, that
trend has yet to "trickle up" to general availability of more advanced
practitioners.

(The other reason I am having trouble recruiting Pythonistas is that my
field -- statistical genetics -- tends to be saturated with Perl folk.
Retraining them is a blast...)

~Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071025/d21b98df/attachment.htm 

From annaraven at gmail.com  Thu Oct 25 16:59:58 2007
From: annaraven at gmail.com (Anna Ravenscroft)
Date: Thu, 25 Oct 2007 07:59:58 -0700
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <2e1434c10710250718v3b9d72esf5ee79be7d96a51d@mail.gmail.com>
References: <mailman.55.1193306415.32757.python-dev@python.org>
	<472076E6.3090307@comp.leeds.ac.uk>
	<9558.194.200.199.193.1193316045.squirrel@webmail.plus.net>
	<2e1434c10710250718v3b9d72esf5ee79be7d96a51d@mail.gmail.com>
Message-ID: <cb361a610710250759q2afe1f18j895912c7ae2c7f26@mail.gmail.com>

I noticed at the Grace Hopper Celebration of Women in Computing that
several major universities in the US are starting to offer intro (CS1)
courses based on Python, among them:
Georgia Tech
CMU
Bryn Mawr

Some of them are using:
Introduction-Computing-Programming-Multimedia-Approach

So, it's starting to get out there...


-- 
cordially,
Anna
--
Walking through the water. Trying to get across.
Just like everybody else.

From annaraven at gmail.com  Thu Oct 25 17:00:28 2007
From: annaraven at gmail.com (Anna Ravenscroft)
Date: Thu, 25 Oct 2007 08:00:28 -0700
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <cb361a610710250759q2afe1f18j895912c7ae2c7f26@mail.gmail.com>
References: <mailman.55.1193306415.32757.python-dev@python.org>
	<472076E6.3090307@comp.leeds.ac.uk>
	<9558.194.200.199.193.1193316045.squirrel@webmail.plus.net>
	<2e1434c10710250718v3b9d72esf5ee79be7d96a51d@mail.gmail.com>
	<cb361a610710250759q2afe1f18j895912c7ae2c7f26@mail.gmail.com>
Message-ID: <cb361a610710250800p7add7dc3p84c2a0a6710fab92@mail.gmail.com>

On Oct 25, 2007 7:59 AM, Anna Ravenscroft <annaraven at gmail.com> wrote:
> I noticed at the Grace Hopper Celebration of Women in Computing that
> several major universities in the US are starting to offer intro (CS1)
> courses based on Python, among them:
> Georgia Tech
> CMU
> Bryn Mawr
>
> Some of them are using:

Introduction to Computing and Programming in Python, A Multimedia
Approach (Paperback)
by Mark Guzdial (Author)
>
> So, it's starting to get out there...
>
>
> --
> cordially,
> Anna
> --
> Walking through the water. Trying to get across.
> Just like everybody else.
>



-- 
cordially,
Anna
--
Walking through the water. Trying to get across.
Just like everybody else.

From titus at caltech.edu  Thu Oct 25 19:00:48 2007
From: titus at caltech.edu (Titus Brown)
Date: Thu, 25 Oct 2007 10:00:48 -0700
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <cb361a610710250759q2afe1f18j895912c7ae2c7f26@mail.gmail.com>
References: <mailman.55.1193306415.32757.python-dev@python.org>
	<472076E6.3090307@comp.leeds.ac.uk>
	<9558.194.200.199.193.1193316045.squirrel@webmail.plus.net>
	<2e1434c10710250718v3b9d72esf5ee79be7d96a51d@mail.gmail.com>
	<cb361a610710250759q2afe1f18j895912c7ae2c7f26@mail.gmail.com>
Message-ID: <20071025170048.GE19808@caltech.edu>

On Thu, Oct 25, 2007 at 07:59:58AM -0700, Anna Ravenscroft wrote:
-> I noticed at the Grace Hopper Celebration of Women in Computing that
-> several major universities in the US are starting to offer intro (CS1)
-> courses based on Python, among them:
-> Georgia Tech
-> CMU
-> Bryn Mawr

It's been adopted at Michigan State U. this past year, and I'll be
teaching a Web dev followup course *next* year.

Python is also being used for bioinformatics at Caltech (not just me)
and at Michigan State (with no connection to the intro course).

The SciPy conferences have been eye opening as well: adoption here,
there, and everywhere.

Good to finally see this happening ;)

--titus

From rrr at ronadam.com  Thu Oct 25 21:01:19 2007
From: rrr at ronadam.com (Ron Adam)
Date: Thu, 25 Oct 2007 14:01:19 -0500
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>	
	<e04bdf310709191341l289c034fjb4bf6d6b88ae4ef9@mail.gmail.com>	
	<46F1B6FD.70007@ronadam.com>	
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>	
	<471F7881.1070605@ronadam.com>
	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>
Message-ID: <4720E7FF.7080404@ronadam.com>



Facundo Batista wrote:
> 2007/10/24, Ron Adam <rrr at ronadam.com>:
> 
>>> Note that these items are *all* open.
>> I think the page title should reflect this.  Possible changing it from
>>
>>      "Python tickets"
>>
>> to
>>      "Python Open Tickets"
> 
> Good point! It's fixed now.
> 
> Thank you!
> 


Clicking on one of the filter links changes the title back.  (No Keyword,
Patch, P3K)

I think the keyword and keywords interface can be improved.  Do you have
any plans in that direction?

How do you get the data the page is built from?

Cheers,
    Ron


From facundobatista at gmail.com  Fri Oct 26 04:27:57 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 25 Oct 2007 23:27:57 -0300
Subject: [Python-Dev] Fwd: Deadlock by a second import in a thread
In-Reply-To: <aac2c7cb0710191622m2f8fd9e1r88faf24174b4f24@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<aac2c7cb0710191552g3b84e3fbxf1cd6add441a6f20@mail.gmail.com>
	<e04bdf310710191611x505a18bdj84e20dd2ab16d67b@mail.gmail.com>
	<e04bdf310710191612i2b9e2b6ct5ec12fcb2610f863@mail.gmail.com>
	<aac2c7cb0710191622m2f8fd9e1r88faf24174b4f24@mail.gmail.com>
Message-ID: <e04bdf310710251927k64995197v3af332c4dbf75f5f@mail.gmail.com>

2007/10/19, Adam Olsen <rhamph at gmail.com>:

> Whether this is a minor problem due to poor style or a major problem
> due to a language defect is a matter of perspective.  I'm working on
> redesigning Python's threading support, expecting it to be used a
> great deal more, which'd push it into the major problem category.
>
> For now I'd leave it open.

It's a matter of perspective, yes. But I'll close this bug, because
he's hitting the problem through a weird way, doing something that he
shouldn't.

The real problem here, if any, is that you can not make a second
import in another thread. Feel free to open a bug for this, but
addressing this specifically.

I'd prefer a PEP, though, ;)

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From facundobatista at gmail.com  Fri Oct 26 04:34:56 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 25 Oct 2007 23:34:56 -0300
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <4719800D.8020208@gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<4719800D.8020208@gmail.com>
Message-ID: <e04bdf310710251934t3867fb71he4feaeb1a9d6f393@mail.gmail.com>

2007/10/20, Nick Coghlan <ncoghlan at gmail.com>:

> bb.py is broken - importing a module should never spawn a new thread as
> a side effect (precisely because it will deadlock if the spawned thread
> tries to do an import, which can happen in a myriad of ways).

Exactly, :(.

I changed timeobject.c to import _strptime once at init time. But the
problem still happened, because the function strptime needs to parse
the format, and this involves a regex sub() call, which finish in a
pattern_subx() call in _srec.c, which needs to call to _subx() in the
re module through the call() function, which finally issues another
import, creating the deadlock again.

In short: you can not avoid a second import unless you rewrite *a lot*
of internal code.

BTW, I'll leave the optimization of importing strptime one time,
there's no reason to try to import it everytime strptime() is called.

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From lists at cheimes.de  Fri Oct 26 04:51:12 2007
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 26 Oct 2007 04:51:12 +0200
Subject: [Python-Dev] Fwd: Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710251927k64995197v3af332c4dbf75f5f@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>	<aac2c7cb0710191552g3b84e3fbxf1cd6add441a6f20@mail.gmail.com>	<e04bdf310710191611x505a18bdj84e20dd2ab16d67b@mail.gmail.com>	<e04bdf310710191612i2b9e2b6ct5ec12fcb2610f863@mail.gmail.com>	<aac2c7cb0710191622m2f8fd9e1r88faf24174b4f24@mail.gmail.com>
	<e04bdf310710251927k64995197v3af332c4dbf75f5f@mail.gmail.com>
Message-ID: <ffrkn3$ne3$1@ger.gmane.org>

Facundo Batista wrote:
> It's a matter of perspective, yes. But I'll close this bug, because
> he's hitting the problem through a weird way, doing something that he
> shouldn't.
> 
> The real problem here, if any, is that you can not make a second
> import in another thread. Feel free to open a bug for this, but
> addressing this specifically.

I had a look into the code. I think it's possible to get rid of most
imports by caching the import in a static variable. For warnings, time
and resource it's even possible to import the module in module
initializer but not for strptime. It depends on a Python module that
imports datetime and time.

I could look into the matter and provide a patch for the trunk.

Christian


From facundobatista at gmail.com  Fri Oct 26 04:56:19 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Thu, 25 Oct 2007 23:56:19 -0300
Subject: [Python-Dev] Fwd: Deadlock by a second import in a thread
In-Reply-To: <ffrkn3$ne3$1@ger.gmane.org>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<aac2c7cb0710191552g3b84e3fbxf1cd6add441a6f20@mail.gmail.com>
	<e04bdf310710191611x505a18bdj84e20dd2ab16d67b@mail.gmail.com>
	<e04bdf310710191612i2b9e2b6ct5ec12fcb2610f863@mail.gmail.com>
	<aac2c7cb0710191622m2f8fd9e1r88faf24174b4f24@mail.gmail.com>
	<e04bdf310710251927k64995197v3af332c4dbf75f5f@mail.gmail.com>
	<ffrkn3$ne3$1@ger.gmane.org>
Message-ID: <e04bdf310710251956h1bf0df65w8e93d8dc809ee6f2@mail.gmail.com>

2007/10/25, Christian Heimes <lists at cheimes.de>:

> I could look into the matter and provide a patch for the trunk.

Feel free to do it. But note, that some imports are inside the call()
function, this could have more implications that you see (at least I
saw) at first glance.

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From facundobatista at gmail.com  Fri Oct 26 05:12:07 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 26 Oct 2007 00:12:07 -0300
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710251934t3867fb71he4feaeb1a9d6f393@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<4719800D.8020208@gmail.com>
	<e04bdf310710251934t3867fb71he4feaeb1a9d6f393@mail.gmail.com>
Message-ID: <e04bdf310710252012r6e044c20uc8ca2b27417c2e4f@mail.gmail.com>

2007/10/25, Facundo Batista <facundobatista at gmail.com>:

> BTW, I'll leave the optimization of importing strptime one time,
> there's no reason to try to import it everytime strptime() is called.

No, I'm not. In consideration to the possible warning raised by Brett,
I won't commit the change (it does not fix anything, just a marginal
optimization, so there's no enough reason to bring a possible issue).

For the record, I'm attaching the would-be patch.

Thank you all!

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: timemodule.c.patch
Type: text/x-diff
Size: 1013 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20071026/76c9f7b8/attachment.patch 

From barry at python.org  Fri Oct 26 05:41:20 2007
From: barry at python.org (Barry Warsaw)
Date: Thu, 25 Oct 2007 23:41:20 -0400
Subject: [Python-Dev] Does Python need a file locking module (slightly
	higher level)?
In-Reply-To: <18205.27378.432540.931896@montanaro.dyndns.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
	<2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>
	<18205.27378.432540.931896@montanaro.dyndns.org>
Message-ID: <95064092-825F-4D3E-A2D4-6309248A6247@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 22, 2007, at 11:30 PM, skip at pobox.com wrote:

> It's not clear that any of these implementations is going to be  
> perfect.
> Maybe none ever will be.

I would agree with this.  You write a program and know you need to  
implement some kind of resource locking, so you start looking for  
some OTS solution.  But then you realize that your application needs  
somewhat different semantics or needs to work in platforms or  
environments that the OTS code doesn't handle.  Just a few days ago,  
I was looking at some locking code that needed to work across  
multiple invocations of a script on multiple machines, and the only  
thing they shared was a PostgreSQL connection, so we ended up wanting  
to use its advisory locks.

> In his reply Jean-Paul made this comment:
>
>     It might be nice to have something like that in the standard  
> library,
>     but it's very simple once you know what to do.
>
> I'm not so sure about the "very simple" part, especially if you aren't
> familiar with all the ins and outs of the different platforms.

I'd totally agree with this.  Locking seems simple, but it's got some  
really tricky aspects that need to be coded just right or you'll be  
in a world of hurt.  Mailman's LockFile.py (which you're right is  
*nix only) is stable now, but has had some really subtle bugs in the  
past.

> The fact
> that the first three bits of code I was referred to were  
> implemented by
> three significant Python tools/platforms and that all are different  
> in some
> significant ways suggests that there is some both an underlying  
> need for a
> file locking mechanism but with a lack of consensus about the best  
> way to
> implement the mother-of-all-file-locking schemes for Python.  Maybe  
> the best
> place for this is in the distribution.  PEP?

I don't think any one solution will work for everybody.  I'm not even  
sure we can define a common API a la the DBAPI, but if something were  
to make it into the standard distribution, that's the direction I'd  
go in.  Then we can provide various implementations that support the  
LockingAPI under various environments, constraints, and platforms.   
If we wanted to distribute them in the stdlib, we could put them all  
in a package and let the user decide which features they need.

I'm still planning on de-Mailman-ifying LockFile.py sometime soon.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iQCVAwUBRyFh4XEjvBPtnXfVAQIAgwQAk0Hf8df6zVGE0sMEfDGFqw6U5/w4TN07
Wiw4Gxq5mRh7jUGoscMrs7L0mjppC/yrv0xIey0u3uQAZqGKLvK2LRBSdC6vyaGY
v9ExnI+q59ffe3oL6UTAmuiouTZspYxSv88wl6ATIPpK0SveAzlwu1c2Xnmw1MaR
5m0Mp+VUR9Q=
=6FrA
-----END PGP SIGNATURE-----

From martin at v.loewis.de  Fri Oct 26 07:51:00 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 26 Oct 2007 07:51:00 +0200
Subject: [Python-Dev] Does Python need a file locking module (slightly
 higher level)?
In-Reply-To: <471D325F.2030806@canterbury.ac.nz>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
	<471D325F.2030806@canterbury.ac.nz>
Message-ID: <47218044.8060103@v.loewis.de>

> I don't think file locking will ever work over NFS, since
> it's a stateless protocol by design

NFS is stateless, but the NFS locking protocol (NLM) is not.

Regards,
Martin

From mal at egenix.com  Fri Oct 26 12:03:15 2007
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 26 Oct 2007 12:03:15 +0200
Subject: [Python-Dev] Does Python need a file locking module (slightly
 higher level)?
In-Reply-To: <95064092-825F-4D3E-A2D4-6309248A6247@python.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>	<2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>	<18205.27378.432540.931896@montanaro.dyndns.org>
	<95064092-825F-4D3E-A2D4-6309248A6247@python.org>
Message-ID: <4721BB63.7090505@egenix.com>

On 2007-10-26 05:41, Barry Warsaw wrote:
> On Oct 22, 2007, at 11:30 PM, skip at pobox.com wrote:
> 
>> It's not clear that any of these implementations is going to be  
>> perfect.
>> Maybe none ever will be.
> 
> I would agree with this.  You write a program and know you need to  
> implement some kind of resource locking, so you start looking for  
> some OTS solution.  But then you realize that your application needs  
> somewhat different semantics or needs to work in platforms or  
> environments that the OTS code doesn't handle.  Just a few days ago,  
> I was looking at some locking code that needed to work across  
> multiple invocations of a script on multiple machines, and the only  
> thing they shared was a PostgreSQL connection, so we ended up wanting  
> to use its advisory locks.
> 
>> In his reply Jean-Paul made this comment:
> 
>>     It might be nice to have something like that in the standard  
>> library,
>>     but it's very simple once you know what to do.
> 
>> I'm not so sure about the "very simple" part, especially if you aren't
>> familiar with all the ins and outs of the different platforms.
> 
> I'd totally agree with this.  Locking seems simple, but it's got some  
> really tricky aspects that need to be coded just right or you'll be  
> in a world of hurt.  Mailman's LockFile.py (which you're right is  
> *nix only) is stable now, but has had some really subtle bugs in the  
> past.

You might want to take a look at the FileLock.py module that's
part of the eGenix mx Base distribution (mx.Misc.FileLock).

It works reliably on Unix and Windows, doesn't rely on fcntl and
has been in use for years.

The only downside is that it's application specific,
ie. only applications using the module for locking will
detect the locks - but then again: this is exactly the problem
you typically want to solve.

>> The fact
>> that the first three bits of code I was referred to were  
>> implemented by
>> three significant Python tools/platforms and that all are different  
>> in some
>> significant ways suggests that there is some both an underlying  
>> need for a
>> file locking mechanism but with a lack of consensus about the best  
>> way to
>> implement the mother-of-all-file-locking schemes for Python.  Maybe  
>> the best
>> place for this is in the distribution.  PEP?
> 
> I don't think any one solution will work for everybody.  I'm not even  
> sure we can define a common API a la the DBAPI, but if something were  
> to make it into the standard distribution, that's the direction I'd  
> go in.  Then we can provide various implementations that support the  
> LockingAPI under various environments, constraints, and platforms.   
> If we wanted to distribute them in the stdlib, we could put them all  
> in a package and let the user decide which features they need.
> 
> I'm still planning on de-Mailman-ifying LockFile.py sometime soon.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 26 2007)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611

From lists at cheimes.de  Fri Oct 26 13:30:28 2007
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 26 Oct 2007 13:30:28 +0200
Subject: [Python-Dev] Fwd: Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710251956h1bf0df65w8e93d8dc809ee6f2@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>	<aac2c7cb0710191552g3b84e3fbxf1cd6add441a6f20@mail.gmail.com>	<e04bdf310710191611x505a18bdj84e20dd2ab16d67b@mail.gmail.com>	<e04bdf310710191612i2b9e2b6ct5ec12fcb2610f863@mail.gmail.com>	<aac2c7cb0710191622m2f8fd9e1r88faf24174b4f24@mail.gmail.com>	<e04bdf310710251927k64995197v3af332c4dbf75f5f@mail.gmail.com>	<ffrkn3$ne3$1@ger.gmane.org>
	<e04bdf310710251956h1bf0df65w8e93d8dc809ee6f2@mail.gmail.com>
Message-ID: <4721CFD4.9040808@cheimes.de>

Facundo Batista wrote:
> Feel free to do it. But note, that some imports are inside the call()
> function, this could have more implications that you see (at least I
> saw) at first glance.

CC to get Guido's attention

First of all I don't understand what you mean with "that some imports
are inside the call() function". Please elaborate on it.

I skimmed through the code and found just a handful of modules that are
important by name as well as imported more than once through the life
time of a python process. The most noticeable modules are time,
_strptime, resource, unicodedata and warnings. The other modules like
__builtin__, zlib and warnings are just loaded once or twice during the
boot strapping of the interpreter.

Guido:
What do you think about storing the modules in an "extern PyObject
*PyMod_Spam" variable? I could either store them when they are used the
first time or I could load them in Py_InitializeEx.

Christian

$ find -name '*.c' | xargs grep PyImport_ImportModule\(\"
./Objects/unicodeobject.c:                m =
PyImport_ImportModule("unicodedata");
./Objects/exceptions.c:    bltinmod = PyImport_ImportModule("__builtin__");
./PC/bdist_wininst/install.c:   mod = PyImport_ImportModule("__builtin__");
./Modules/_ctypes/callbacks.c:  mod = PyImport_ImportModule("ctypes");
./Modules/_ctypes/callbacks.c:  mod = PyImport_ImportModule("ctypes");
./Modules/cPickle.c:    if (!( copy_reg =
PyImport_ImportModule("copy_reg")))
./Modules/cPickle.c:    if (!( t=PyImport_ImportModule("__builtin__")))
 return -1;
./Modules/posixmodule.c:                PyObject *m =
PyImport_ImportModule("resource");
./Modules/zipimport.c:          zlib = PyImport_ImportModule("zlib");
/* import zlib */
./Modules/datetimemodule.c:             PyObject *time =
PyImport_ImportModule("time");
./Modules/datetimemodule.c:     PyObject *time =
PyImport_ImportModule("time");
./Modules/datetimemodule.c:     time = PyImport_ImportModule("time");
./Modules/datetimemodule.c:     if ((module =
PyImport_ImportModule("time")) == NULL)
./Modules/timemodule.c:    PyObject *strptime_module =
PyImport_ImportModule("_strptime");
./Modules/timemodule.c: m = PyImport_ImportModule("time");
./Modules/gcmodule.c:           tmod = PyImport_ImportModule("time");
./Modules/main.c:       runpy = PyImport_ImportModule("runpy");
./Modules/main.c:               v = PyImport_ImportModule("readline");
./Modules/parsermodule.c:    copyreg = PyImport_ImportModule("copy_reg");
./Modules/_cursesmodule.c:  PyObject *m = PyImport_ImportModule("curses");
./Python/mactoolboxglue.c:      m = PyImport_ImportModule("MacOS");
./Python/pythonrun.c:   warnings_module = PyImport_ImportModule("warnings");
./Python/pythonrun.c:           PyObject *bimod =
PyImport_ImportModule("__builtin__");
./Python/pythonrun.c:   m = PyImport_ImportModule("site");
./Python/errors.c:      mod = PyImport_ImportModule("warnings");
./Python/import.c:      zimpimport = PyImport_ImportModule("zipimport");
./Doc/tools/sphinx/jinja/_speedups.c:   PyObject *datastructure =
PyImport_ImportModule("jinja.datastructure");
./Mac/Modules/MacOS.c:          m = PyImport_ImportModule("macresource");

From facundobatista at gmail.com  Fri Oct 26 15:50:31 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 26 Oct 2007 10:50:31 -0300
Subject: [Python-Dev] Fwd: Deadlock by a second import in a thread
In-Reply-To: <4721CFD4.9040808@cheimes.de>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<aac2c7cb0710191552g3b84e3fbxf1cd6add441a6f20@mail.gmail.com>
	<e04bdf310710191611x505a18bdj84e20dd2ab16d67b@mail.gmail.com>
	<e04bdf310710191612i2b9e2b6ct5ec12fcb2610f863@mail.gmail.com>
	<aac2c7cb0710191622m2f8fd9e1r88faf24174b4f24@mail.gmail.com>
	<e04bdf310710251927k64995197v3af332c4dbf75f5f@mail.gmail.com>
	<ffrkn3$ne3$1@ger.gmane.org>
	<e04bdf310710251956h1bf0df65w8e93d8dc809ee6f2@mail.gmail.com>
	<4721CFD4.9040808@cheimes.de>
Message-ID: <e04bdf310710260650t687122d1pc287586d6680ccc8@mail.gmail.com>

2007/10/26, Christian Heimes <lists at cheimes.de>:

> First of all I don't understand what you mean with "that some imports
> are inside the call() function". Please elaborate on it.

I'm talking about the "call" function defined in the _sre.c file. This
function has a call to PyImport_Import() inside it.

In the "bug" I was persuing, this call to PyImport_Import() causes the
deadlock after "call" being called from line 2314:

    filter = call(
        SRE_PY_MODULE, "_subx",
        PyTuple_Pack(2, self, ptemplate)
        );

I just was saying the calls to "imports" are so deep, that you need to
touch a lot of code to make sure that they're executed in a sane way.

With "sane way", I mean that the imports should be executed when the
user makes "import foobar". Importing thing when the user makes
time.strptime() or re.sub() is breaking the "least surprise" rule,
which is a bad thing especially when talking of imports, that can
cause (and actually do!) deadlocks.


> I skimmed through the code and found just a handful of modules that are
> important by name as well as imported more than once through the life
> time of a python process. The most noticeable modules are time,
> _strptime, resource, unicodedata and warnings. The other modules like
> __builtin__, zlib and warnings are just loaded once or twice during the
> boot strapping of the interpreter.

This goes along what I said in my first mail: everytime you do
time.strptime(), that function tries to import "_strptime.py", which
is, at least, inefficient.


> What do you think about storing the modules in an "extern PyObject
> *PyMod_Spam" variable? I could either store them when they are used the
> first time or I could load them in Py_InitializeEx.

You shouldn't import them when they're used in the first time, are you
still are breaking the least surprise rule (see last parragraph).

However, one possible way to solve this problems is to, in every
module, import everything the module will ever need at init time. Note
that you actually do not need ane extern variable here, as you won't
be accessing it from other files, making it global and static would be
ok.

I was about to commit the following change in timemodule.c, that is a
good example of what I just said (I didn't commit it, because I'm not
full aware of the issue that Brett described):

1. I extracted the import from the strptime function:

@@ -514,13 +517,11 @@
 static PyObject *
 time_strptime(PyObject *self, PyObject *args)
 {
-    PyObject *strptime_module = PyImport_ImportModule("_strptime");
     PyObject *strptime_result;

     if (!strptime_module)
         return NULL;
     strptime_result = PyObject_CallMethod(strptime_module,
"strptime", "O", args);
-    Py_DECREF(strptime_module);
     return strptime_result;
 }


2. Created a global static variable that will hold the module:

@@ -98,6 +98,9 @@
 /* For Y2K check */
 static PyObject *moddict;

+/* This will be initializied at module init time */
+static PyObject *strptime_module;
+
 /* Exposed in timefuncs.h. */
 time_t
 _PyTime_DoubleToTimet(double x)


3. Imported the module at init time:

@@ -848,6 +849,8 @@
 	Py_INCREF(&StructTimeType);
 	PyModule_AddObject(m, "struct_time", (PyObject*) &StructTimeType);
 	initialized = 1;
+
+    strptime_module = PyImport_ImportModule("_strptime");
 }

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From janssen at parc.com  Fri Oct 26 19:16:57 2007
From: janssen at parc.com (Bill Janssen)
Date: Fri, 26 Oct 2007 10:16:57 PDT
Subject: [Python-Dev] SSL 1.10, to fix client-side bug
Message-ID: <07Oct26.101659pdt."57996"@synergy1.parc.xerox.com>

I've updated the PyPI SSL package to 1.10, to fix a fairly serious bug
in sendall() I found while updating the trunk code.

http://pypi.python.org/pypi/ssl/

Bill

From status at bugs.python.org  Fri Oct 26 19:37:24 2007
From: status at bugs.python.org (Tracker)
Date: Fri, 26 Oct 2007 17:37:24 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071026173724.46D6E78280@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/19/07 - 10/26/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1311 open (+18) / 11509 closed (+18) / 12820 total (+36)

Open issues with patches:   414

Average duration of open issues: 682 days.
Median duration of open issues: 759 days.

Open Issues Breakdown
   open  1307 (+18)
pending     4 ( +0)

Issues Created Or Reopened (37)
_______________________________

Py3K cannot run as ``python -S``                                 10/21/07
CLOSED http://bugs.python.org/issue1267    reopened gvanrossum               
       py3k, patch                                                             

Fixes for profile/cprofile                                       10/19/07
CLOSED http://bugs.python.org/issue1302    created  tiran                    
       py3k, patch                                                             

adapt str8 constructor to bytes constructor                      10/19/07
CLOSED http://bugs.python.org/issue1303    created  georg.brandl             
       py3k, patch                                                             

py3k compilation on Windows                                      10/19/07
CLOSED http://bugs.python.org/issue1304    created  amaury.forgeotdarc       
       patch                                                                   

socket.py hangs on Mac OS X                                      10/20/07
CLOSED http://bugs.python.org/issue1305    created  andres                   
                                                                               

Embedded python reinitialization                                 10/20/07
       http://bugs.python.org/issue1306    created  Karemir                  
                                                                               

smtpd.SMTPServer throws exception on MAIL command with no arg    10/23/07
CLOSED http://bugs.python.org/issue1307    reopened gvanrossum               
       patch                                                                   

unicode(None) anomaly                                            10/21/07
CLOSED http://bugs.python.org/issue1308    created  jgsack                   
                                                                               

windows build fix                                                10/22/07
CLOSED http://bugs.python.org/issue1309    created  tiran                    
       patch                                                                   

tempfile breaks on Windows                                       10/22/07
CLOSED http://bugs.python.org/issue1310    created  tiran                    
       patch                                                                   

os.path.exists(os.devnull) regression on windows                 10/22/07
       http://bugs.python.org/issue1311    created  d_kagedal                
                                                                               

doctest EXCEPTION_RE can't handle preceding output               10/22/07
       http://bugs.python.org/issue1312    created  nouri                    
       patch                                                                   

Typo in Python Library Reference documentation: logging module   10/22/07
CLOSED http://bugs.python.org/issue1313    created  nhaines                  
                                                                               

Patch for strftime problem on German Windows                     10/23/07
CLOSED http://bugs.python.org/issue1314    created  tiran                    
       patch                                                                   

Bug in pstats.print_callers                                      10/23/07
       http://bugs.python.org/issue1315    created  JoelSchaerer             
                                                                               

fix a few PyInt_Check vs PyLong_Check problems                   10/23/07
CLOSED http://bugs.python.org/issue1316    created  georg.brandl             
       py3k, patch                                                             

smtplib.SMTP docs                                                10/23/07
       http://bugs.python.org/issue1317    created  fdrake                   
                                                                               

Remove os.tmpnam() and os.tempnam()                              10/23/07
CLOSED http://bugs.python.org/issue1318    created  tiran                    
       patch                                                                   

py3k: fixes for test_ctypes                                      10/23/07
CLOSED http://bugs.python.org/issue1319    created  amaury.forgeotdarc       
       patch                                                                   

PCBuild8 Solution Support Changes                                10/24/07
       http://bugs.python.org/issue1320    created  JosephArmbruster         
                                                                               

TimedRotatingFileHandler logic for day of week is wrong          10/24/07
CLOSED http://bugs.python.org/issue1321    created  mosheco                  
                                                                               

platform.dist() has unpredictable result under Linux             10/24/07
       http://bugs.python.org/issue1322    created  sapetnioc                
                                                                               

py3k: file.truncate() changes the file position                  10/24/07
CLOSED http://bugs.python.org/issue1323    created  amaury.forgeotdarc       
       patch                                                                   

r58034 breaks building _ctypes with the upstream libffi.         10/25/07
CLOSED http://bugs.python.org/issue1324    created  doko                     
       patch                                                                   

zipimport.zipimporter.archive undocumented and untested          10/25/07
       http://bugs.python.org/issue1325    created  exarkun                  
                                                                               

"internal" zipimport.zipimporter feature untested                10/25/07
       http://bugs.python.org/issue1326    created  exarkun                  
                                                                               

Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmcl 10/25/07
       http://bugs.python.org/issue1327    created  eanxgeek                 
                                                                               

feature request: force BOM option                                10/25/07
       http://bugs.python.org/issue1328    created  jgsack                   
                                                                               

Different 3.0a1 exit behavior                                    10/25/07
       http://bugs.python.org/issue1329    created  MrJean1                  
                                                                               

Fix truncate on Windows, this time for real                      10/26/07
CLOSED http://bugs.python.org/issue1330    created  tiran                    
       py3k, patch                                                             

More fixes for Windows                                           10/26/07
CLOSED http://bugs.python.org/issue1331    created  tiran                    
       py3k, patch                                                             

threading.RLock().aquire(0) fails with python-2.5.1.amd64.msi    10/26/07
       http://bugs.python.org/issue1332    created  delwarl                  
                                                                               

merge urllib and urlparse functionality                          10/26/07
       http://bugs.python.org/issue1333    created  techtonik                
                                                                               

IDLE - Fix several highlighting bugs                             10/26/07
       http://bugs.python.org/issue1334    created  taleinat                 
                                                                               

bytesiterator patch                                              10/26/07
       http://bugs.python.org/issue1335    created  tiran                    
                                                                               

subprocess.Popen hangs when child writes to stderr               10/26/07
       http://bugs.python.org/issue1336    created  jba                      
                                                                               

Tools/msi/msi.py does not work with PCBuild8                     10/26/07
       http://bugs.python.org/issue1337    created  kevinwatters             
                                                                               



Issues Now Closed (39)
______________________

use bytes for code objects                                         60 days
       http://bugs.python.org/issue1022    gvanrossum               
       patch                                                                   

ABC caches should use weak refs                                    54 days
       http://bugs.python.org/issue1061    georg.brandl             
       patch                                                                   

unicode.translate() doesn't error out on invalid translation tab   53 days
       http://bugs.python.org/issue1071    georg.brandl             
       patch                                                                   

itertools missing, causes interactive help to break                54 days
       http://bugs.python.org/issue1077    georg.brandl             
                                                                               

"TypeError: expected string, bytes found" instead of KeyboardInt   38 days
       http://bugs.python.org/issue1151    gvanrossum               
                                                                               

Garbled chars in offending line of SyntaxError traceback           36 days
       http://bugs.python.org/issue1161    gvanrossum               
                                                                               

Strange Python hangup                                              16 days
       http://bugs.python.org/issue1255    kakacek                  
                                                                               

Removal of basestring type                                         14 days
       http://bugs.python.org/issue1258    gvanrossum               
       patch                                                                   

PEP 3137 patch - str8/str comparison should return false           11 days
       http://bugs.python.org/issue1263    brett.cannon             
       py3k, patch                                                             

Py3K cannot run as ``python -S``                                    0 days
       http://bugs.python.org/issue1267    tiran                    
       py3k, patch                                                             

imp.find_module() ignores -*- coding: Latin-1 -*-                   5 days
       http://bugs.python.org/issue1278    gvanrossum               
                                                                               

PEP 3137: Make PyString's indexing and iteration return integers    4 days
       http://bugs.python.org/issue1280    gvanrossum               
       patch                                                                   

typo in lib doc 7.3.2.1 MaildirMessage                              4 days
       http://bugs.python.org/issue1284    georg.brandl             
                                                                               

os.environ.pop doesn't work                                         9 days
       http://bugs.python.org/issue1287    draghuram                
                                                                               

xml.dom.minidom not able to handle utf-8 data                       7 days
       http://bugs.python.org/issue1290    facundobatista           
                                                                               

Support for z/OS and EBCDIC.                                        6 days
       http://bugs.python.org/issue1298    gvanrossum               
                                                                               

Fixes for profile/cprofile                                          6 days
       http://bugs.python.org/issue1302    tiran                    
       py3k, patch                                                             

adapt str8 constructor to bytes constructor                         5 days
       http://bugs.python.org/issue1303    georg.brandl             
       py3k, patch                                                             

py3k compilation on Windows                                         3 days
       http://bugs.python.org/issue1304    gvanrossum               
       patch                                                                   

socket.py hangs on Mac OS X                                         3 days
       http://bugs.python.org/issue1305    facundobatista           
                                                                               

smtpd.SMTPServer throws exception on MAIL command with no arg       0 days
       http://bugs.python.org/issue1307    nnorwitz                 
       patch                                                                   

unicode(None) anomaly                                               0 days
       http://bugs.python.org/issue1308    jgsack                   
                                                                               

windows build fix                                                   1 days
       http://bugs.python.org/issue1309    gvanrossum               
       patch                                                                   

tempfile breaks on Windows                                          0 days
       http://bugs.python.org/issue1310    georg.brandl             
       patch                                                                   

Typo in Python Library Reference documentation: logging module      0 days
       http://bugs.python.org/issue1313    nnorwitz                 
                                                                               

Patch for strftime problem on German Windows                        0 days
       http://bugs.python.org/issue1314    gvanrossum               
       patch                                                                   

fix a few PyInt_Check vs PyLong_Check problems                      0 days
       http://bugs.python.org/issue1316    georg.brandl             
       py3k, patch                                                             

Remove os.tmpnam() and os.tempnam()                                 2 days
       http://bugs.python.org/issue1318    gvanrossum               
       patch                                                                   

py3k: fixes for test_ctypes                                         1 days
       http://bugs.python.org/issue1319    theller                  
       patch                                                                   

TimedRotatingFileHandler logic for day of week is wrong             0 days
       http://bugs.python.org/issue1321    vsajip                   
                                                                               

py3k: file.truncate() changes the file position                     1 days
       http://bugs.python.org/issue1323    gvanrossum               
       patch                                                                   

r58034 breaks building _ctypes with the upstream libffi.            1 days
       http://bugs.python.org/issue1324    theller                  
       patch                                                                   

Fix truncate on Windows, this time for real                         1 days
       http://bugs.python.org/issue1330    gvanrossum               
       py3k, patch                                                             

More fixes for Windows                                              0 days
       http://bugs.python.org/issue1331    gvanrossum               
       py3k, patch                                                             

Non-contiguous indexing and slicing                              2369 days
       http://bugs.python.org/issue419903  facundobatista           
                                                                               

python should obey the FHS                                       1912 days
       http://bugs.python.org/issue588756  facundobatista           
                                                                               

add offset to mmap                                               1675 days
       http://bugs.python.org/issue708374  gvanrossum               
       patch                                                                   

mmap missing offset parameter                                    1034 days
       http://bugs.python.org/issue1089974 georg.brandl             
                                                                               

Open always create new tab or new browser                         103 days
       http://bugs.python.org/issue1753371 georg.brandl             
                                                                               



Top Issues Most Discussed (10)
______________________________

 38 Py3K cannot run as ``python -S``                                   0 days
closed  http://bugs.python.org/issue1267   

 12 Fixes for profile/cprofile                                         6 days
closed  http://bugs.python.org/issue1302   

 12 PEP 3137 patch - str8/str comparison should return false          11 days
closed  http://bugs.python.org/issue1263   

 10 platform.dist() has unpredictable result under Linux               2 days
open    http://bugs.python.org/issue1322   

 10 Remove os.tmpnam() and os.tempnam()                                2 days
closed  http://bugs.python.org/issue1318   

  9 os.path.exists(os.devnull) regression on windows                   4 days
open    http://bugs.python.org/issue1311   

  8 py3k: fixes for test_ctypes                                        1 days
closed  http://bugs.python.org/issue1319   

  7 smtpd.SMTPServer throws exception on MAIL command with no arg      0 days
closed  http://bugs.python.org/issue1307   

  7 Support for z/OS and EBCDIC.                                       6 days
closed  http://bugs.python.org/issue1298   

  6 adapt str8 constructor to bytes constructor                        5 days
closed  http://bugs.python.org/issue1303   




From lovingfamily511 at sbcglobal.net  Fri Oct 26 20:16:42 2007
From: lovingfamily511 at sbcglobal.net (michael macdonald)
Date: Fri, 26 Oct 2007 11:16:42 -0700
Subject: [Python-Dev] Python developers are in demand
In-Reply-To: <mailman.1828.1193365684.13604.python-dev@python.org>
References: <mailman.1828.1193365684.13604.python-dev@python.org>
Message-ID: <1193422602.13378.19.camel@DellDesktop511>

I come from a corporate IT background working as a drone in large
businesses, starting about 30 years ago (argh!).  I come from an IBM
mainframe background, but with constant exposure and interfaces with
other platforms, and writing code with a variety of languages and
scripting languages throughout my career.  I recently completed a very
long and painful transition to becoming at least a part time Java
developer.  For the past several years I have had a background
conversation with myself on the topic of the Perfect Programming
Language; what would it look like/feel like.  Then I stumbled on Python,
and I am totally hooked.

I gather this experience is similar to others who are now Python
advocates, who are using Python for their personal projects, etc.

In the short term, though, for the corporate world, it'll probably be an
uphill battle to have major projects developed in Python (or even
Jython, which would be the logical choice for a transition out of j2ee
to the lamp world).  There is some amount of resistance to open source
software.  The same resistance to linux was overcome by 'respectable'
companies supporting and interacting on a corporate level, like Red Hat.
IBM certainly helped there, too, like it or not.  Python would probably
do well to have similar corporate advocates, but that would be an
expensive and risky business proposition.

Short term, I wouldn't expect to be able to make a big paycheck out of
Python.  As time goes on and the younger programmers who are using
Python on their personal and open source projects get into positions of
influence in the corporate world that will change. 

My thoughts, for what it's worth.
Fight the good fight!
Michael 



On Fri, 2007-10-26 at 04:28 +0200, python-dev-request at python.org wrote:
> Send Python-Dev mailing list submissions to
> 	python-dev at python.org
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://mail.python.org/mailman/listinfo/python-dev
> or, via email, send a message with subject or body 'help' to
> 	python-dev-request at python.org
> 
> You can reach the person managing the list at
> 	python-dev-owner at python.org
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Python-Dev digest..."
> 
> 
> Today's Topics:
> 
>    1. Re: Python developers are in demand (Nick Efford)
>    2. Re: Python developers are in demand (Facundo Batista)
>    3. Re: Python tickets summary (Facundo Batista)
>    4. Re: Python developers are in demand (Anthony Roy)
>    5. Re: Python developers are in demand
>       (Kevin Jacobs <jacobs at bioinformed.com>)
>    6. Re: Python developers are in demand (Anna Ravenscroft)
>    7. Re: Python developers are in demand (Anna Ravenscroft)
>    8. Re: Python developers are in demand (Titus Brown)
>    9. Re: Python tickets summary (Ron Adam)
>   10. Re: Fwd: Deadlock by a second import in a thread (Facundo Batista)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Thu, 25 Oct 2007 11:58:46 +0100
> From: Nick Efford <nde at comp.leeds.ac.uk>
> Subject: Re: [Python-Dev] Python developers are in demand
> To: python-dev at python.org
> Message-ID: <472076E6.3090307 at comp.leeds.ac.uk>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Interesting to see discussion on supply and demand issues for
> Python programmers.  You might be interested to learn that,
> after a few years of flirting with Python in various ways, the
> School of Computing at the University of Leeds has recently
> switched to teaching Python as the first and primary programming
> language for undergraduates on all of our degree programmes.
> 
> I know we're not the only ones doing this, so perhaps the
> supply will rise to meet the demand in a few years!
> 
> 
> Nick
> 
> -- 
> Dr Nick Efford, School of | E: nde at comp.leeds.ac.uk
> Computing, University of  | T: +44 113 343 6809
> Leeds, Leeds, LS2 9JT, UK | W: http://www.comp.leeds.ac.uk/nde/
> --------------------------+-----------------------------------------
> PGP fingerprint: 6ADF 16C2 4E2D 320B F537  8F3C 402D 1C78 A668 8492
> 
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Thu, 25 Oct 2007 09:13:48 -0300
> From: "Facundo Batista" <facundobatista at gmail.com>
> Subject: Re: [Python-Dev] Python developers are in demand
> To: "Alex Martelli" <aleaxit at gmail.com>
> Cc: " Martin v. L?wis " <martin at v.loewis.de>,	Christian Heimes
> 	<lists at cheimes.de>, Python Dev <python-dev at python.org>
> Message-ID:
> 	<e04bdf310710250513j75099e7cj7e9d6fb662902472 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> 2007/10/24, Alex Martelli <aleaxit at gmail.com>:
> 
> > using C++ and Java (and often C), but as far as I know there is no
> > Stanford course (at least not within Symbolic Systems) that focuses
> > specifically and exclusively on Python (there IS one course,
> 
> In my constant try-to-push-Python-everywhere-I-go, I offered several
> times Python courses to educational institutions (sometimes even
> free).
> 
> I succeeded some times, but then these courses not thrived year after
> year. Normally, this is because the people that is actually taking the
> decision of which language to teach in the courses do not know Python,
> so is easier to them to keep teaching C.  And this happens even if
> it's not the better to the students, and even witht the students
> asking for the change.
> 
> But this is a problem of educative system here in Argentina, not of
> Python itself (it surely get affected, though).
> 
> Regards,
> 
> -- 
> .    Facundo
> 
> Blog: http://www.taniquetil.com.ar/plog/
> PyAr: http://www.python.org/ar/
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Thu, 25 Oct 2007 10:11:28 -0300
> From: "Facundo Batista" <facundobatista at gmail.com>
> Subject: Re: [Python-Dev] Python tickets summary
> To: rrr at ronadam.com
> Cc: python-dev at python.org
> Message-ID:
> 	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> 2007/10/24, Ron Adam <rrr at ronadam.com>:
> 
> > > Note that these items are *all* open.
> >
> > I think the page title should reflect this.  Possible changing it from
> >
> >      "Python tickets"
> >
> > to
> >      "Python Open Tickets"
> 
> Good point! It's fixed now.
> 
> Thank you!
> 
> -- 
> .    Facundo
> 
> Blog: http://www.taniquetil.com.ar/plog/
> PyAr: http://www.python.org/ar/
> 
> 
> ------------------------------
> 
> Message: 4
> Date: Thu, 25 Oct 2007 13:40:45 +0100 (BST)
> From: "Anthony Roy" <home at antroy.co.uk>
> Subject: Re: [Python-Dev] Python developers are in demand
> To: "Nick Efford" <nde at comp.leeds.ac.uk>
> Cc: python-dev at python.org
> Message-ID:
> 	<9558.194.200.199.193.1193316045.squirrel at webmail.plus.net>
> Content-Type: text/plain;charset=iso-8859-1
> 
> 
> > Interesting to see discussion on supply and demand issues for
> > Python programmers.  You might be interested to learn that,
> > after a few years of flirting with Python in various ways, the
> > School of Computing at the University of Leeds has recently
> > switched to teaching Python as the first and primary programming
> > language for undergraduates on all of our degree programmes.
> >
> > I know we're not the only ones doing this, so perhaps the
> > supply will rise to meet the demand in a few years!
> 
> I was a researcher in the School of Computing at Leeds Uni about 4 years
> ago. Good to see them pushing Python!
> 
> I keep my eyes open for Python Developer roles in the UK (particularly the
> North), since I would far prefer to develop in Python than Java. However,
> in the UK Python Jobs seem to be few and far between, and most of the ones
> that there are are either low paid sys admin type roles, or are based in
> London.
> 
> Cheers,
> 
> -- 
> Anthony Roy
> 
> 
> 
> ------------------------------
> 
> Message: 5
> Date: Thu, 25 Oct 2007 10:18:04 -0400
> From: "Kevin Jacobs <jacobs at bioinformed.com>" <bioinformed at gmail.com>
> Subject: Re: [Python-Dev] Python developers are in demand
> To: python-dev at python.org
> Message-ID:
> 	<2e1434c10710250718v3b9d72esf5ee79be7d96a51d at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-1"
> 
> Just to chime in from the other side of the coin.
> 
> I'm actively trying to hire qualified scientific programmers with strong
> Python experience.  Unfortunately, I've had little success finding
> candidates with actual Python knowledge, resorting mainly to hiring those
> who've seen it and can readily learn it on the job.  So while it is
> encouraging that Python is being used as an introductory language, that
> trend has yet to "trickle up" to general availability of more advanced
> practitioners.
> 
> (The other reason I am having trouble recruiting Pythonistas is that my
> field -- statistical genetics -- tends to be saturated with Perl folk.
> Retraining them is a blast...)
> 
> ~Kevin
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.python.org/pipermail/python-dev/attachments/20071025/d21b98df/attachment-0001.htm 
> 
> ------------------------------
> 
> Message: 6
> Date: Thu, 25 Oct 2007 07:59:58 -0700
> From: "Anna Ravenscroft" <annaraven at gmail.com>
> Subject: Re: [Python-Dev] Python developers are in demand
> To: jacobs at bioinformed.com
> Cc: python-dev at python.org
> Message-ID:
> 	<cb361a610710250759q2afe1f18j895912c7ae2c7f26 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> I noticed at the Grace Hopper Celebration of Women in Computing that
> several major universities in the US are starting to offer intro (CS1)
> courses based on Python, among them:
> Georgia Tech
> CMU
> Bryn Mawr
> 
> Some of them are using:
> Introduction-Computing-Programming-Multimedia-Approach
> 
> So, it's starting to get out there...
> 
> 
> -- 
> cordially,
> Anna
> --
> Walking through the water. Trying to get across.
> Just like everybody else.
> 
> 
> ------------------------------
> 
> Message: 7
> Date: Thu, 25 Oct 2007 08:00:28 -0700
> From: "Anna Ravenscroft" <annaraven at gmail.com>
> Subject: Re: [Python-Dev] Python developers are in demand
> To: jacobs at bioinformed.com
> Cc: python-dev at python.org
> Message-ID:
> 	<cb361a610710250800p7add7dc3p84c2a0a6710fab92 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> On Oct 25, 2007 7:59 AM, Anna Ravenscroft <annaraven at gmail.com> wrote:
> > I noticed at the Grace Hopper Celebration of Women in Computing that
> > several major universities in the US are starting to offer intro (CS1)
> > courses based on Python, among them:
> > Georgia Tech
> > CMU
> > Bryn Mawr
> >
> > Some of them are using:
> 
> Introduction to Computing and Programming in Python, A Multimedia
> Approach (Paperback)
> by Mark Guzdial (Author)
> >
> > So, it's starting to get out there...
> >
> >
> > --
> > cordially,
> > Anna
> > --
> > Walking through the water. Trying to get across.
> > Just like everybody else.
> >
> 
> 
> 
> -- 
> cordially,
> Anna
> --
> Walking through the water. Trying to get across.
> Just like everybody else.
> 
> 
> ------------------------------
> 
> Message: 8
> Date: Thu, 25 Oct 2007 10:00:48 -0700
> From: Titus Brown <titus at caltech.edu>
> Subject: Re: [Python-Dev] Python developers are in demand
> To: Anna Ravenscroft <annaraven at gmail.com>
> Cc: python-dev at python.org
> Message-ID: <20071025170048.GE19808 at caltech.edu>
> Content-Type: text/plain; charset=us-ascii
> 
> On Thu, Oct 25, 2007 at 07:59:58AM -0700, Anna Ravenscroft wrote:
> -> I noticed at the Grace Hopper Celebration of Women in Computing that
> -> several major universities in the US are starting to offer intro (CS1)
> -> courses based on Python, among them:
> -> Georgia Tech
> -> CMU
> -> Bryn Mawr
> 
> It's been adopted at Michigan State U. this past year, and I'll be
> teaching a Web dev followup course *next* year.
> 
> Python is also being used for bioinformatics at Caltech (not just me)
> and at Michigan State (with no connection to the intro course).
> 
> The SciPy conferences have been eye opening as well: adoption here,
> there, and everywhere.
> 
> Good to finally see this happening ;)
> 
> --titus
> 
> 
> ------------------------------
> 
> Message: 9
> Date: Thu, 25 Oct 2007 14:01:19 -0500
> From: Ron Adam <rrr at ronadam.com>
> Subject: Re: [Python-Dev] Python tickets summary
> To: python-dev at python.org
> Message-ID: <4720E7FF.7080404 at ronadam.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> 
> 
> Facundo Batista wrote:
> > 2007/10/24, Ron Adam <rrr at ronadam.com>:
> > 
> >>> Note that these items are *all* open.
> >> I think the page title should reflect this.  Possible changing it from
> >>
> >>      "Python tickets"
> >>
> >> to
> >>      "Python Open Tickets"
> > 
> > Good point! It's fixed now.
> > 
> > Thank you!
> > 
> 
> 
> Clicking on one of the filter links changes the title back.  (No Keyword,
> Patch, P3K)
> 
> I think the keyword and keywords interface can be improved.  Do you have
> any plans in that direction?
> 
> How do you get the data the page is built from?
> 
> Cheers,
>     Ron
> 
> 
> 
> ------------------------------
> 
> Message: 10
> Date: Thu, 25 Oct 2007 23:27:57 -0300
> From: "Facundo Batista" <facundobatista at gmail.com>
> Subject: Re: [Python-Dev] Fwd: Deadlock by a second import in a thread
> To: "Adam Olsen" <rhamph at gmail.com>
> Cc: python-dev <python-dev at python.org>
> Message-ID:
> 	<e04bdf310710251927k64995197v3af332c4dbf75f5f at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> 2007/10/19, Adam Olsen <rhamph at gmail.com>:
> 
> > Whether this is a minor problem due to poor style or a major problem
> > due to a language defect is a matter of perspective.  I'm working on
> > redesigning Python's threading support, expecting it to be used a
> > great deal more, which'd push it into the major problem category.
> >
> > For now I'd leave it open.
> 
> It's a matter of perspective, yes. But I'll close this bug, because
> he's hitting the problem through a weird way, doing something that he
> shouldn't.
> 
> The real problem here, if any, is that you can not make a second
> import in another thread. Feel free to open a bug for this, but
> addressing this specifically.
> 
> I'd prefer a PEP, though, ;)
> 
> Regards,
> 


From brett at python.org  Fri Oct 26 21:55:46 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 26 Oct 2007 12:55:46 -0700
Subject: [Python-Dev] Deadlock by a second import in a thread
In-Reply-To: <e04bdf310710252012r6e044c20uc8ca2b27417c2e4f@mail.gmail.com>
References: <e04bdf310710191522p31bb61eay8390e12942582357@mail.gmail.com>
	<4719800D.8020208@gmail.com>
	<e04bdf310710251934t3867fb71he4feaeb1a9d6f393@mail.gmail.com>
	<e04bdf310710252012r6e044c20uc8ca2b27417c2e4f@mail.gmail.com>
Message-ID: <bbaeab100710261255y5df17e54v25e3693a1f90e9ff@mail.gmail.com>

On 10/25/07, Facundo Batista <facundobatista at gmail.com> wrote:
> 2007/10/25, Facundo Batista <facundobatista at gmail.com>:
>
> > BTW, I'll leave the optimization of importing strptime one time,
> > there's no reason to try to import it everytime strptime() is called.
>
> No, I'm not. In consideration to the possible warning raised by Brett,
> I won't commit the change (it does not fix anything, just a marginal
> optimization, so there's no enough reason to bring a possible issue).
>
> For the record, I'm attaching the would-be patch.
>

I honestly have no problem if you do the import once, Facundo.  My
warning is so marginal I wouldn't worry about it.  Plus almost all
other code, C and Python alike, assume the import happens only once at
import so it shouldn't be a surprise to anyone.

> Thank you all!

Thanks for putting the time in to figure this out!

-Brett

From skip at pobox.com  Fri Oct 26 22:10:18 2007
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 26 Oct 2007 15:10:18 -0500
Subject: [Python-Dev] Does Python need a file locking module (slightly
 higher level)?
In-Reply-To: <95064092-825F-4D3E-A2D4-6309248A6247@python.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
	<2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>
	<18205.27378.432540.931896@montanaro.dyndns.org>
	<95064092-825F-4D3E-A2D4-6309248A6247@python.org>
Message-ID: <18210.18858.361196.95168@montanaro.dyndns.org>


    Barry> I don't think any one solution will work for everybody.  I'm not
    Barry> even sure we can define a common API a la the DBAPI, but if
    Barry> something were to make it into the standard distribution, that's
    Barry> the direction I'd go in.

I've been working on a lockfile module the past few days on the train.  I
have something that passes a boatload of doctest test cases on my Mac, works
for threads as well as processes (acquire the lock in thread one, then block
in thread two until thread one releases the lock).  The Unix version relies
on the atomic nature of the link(2) system call.  The Windows version (not
yet tested on that platform) relies on mkdir(2) being atomic.  (I think it
is.)  In theory, I suppose the mkdir version will work for both platforms,
so it's possible that you could have file locking between Windows and Unix
should you want it.

The current implementation provides a FileLock class with these methods:

    acquire
    release
    is_locked
    break_lock
    __enter__
    __exit__

The acquire method takes an optional timeout parameter (None => wait
forever, +ive value => block for awhile, zero or -ive => give up
immediately).  The others all take no arguments.  I'm working on ReST
documentation now and hope to have that finished over the weekend.  After
that I'll write a simple setup.py, toss it out on PyPI, then announce it
more broadly.  If anyone would like to try it out and/or review the code
sooner (especially if you have access to Windows) let me know.  I'll shoot
you a copy as it currently exists.

The API and almost all test cases are defined in a _FileLock base class.
You could (in theory at least) subclass it to provide locking through some
other shared resource like a database and not have to write and or many
other test cases.

Skip

From skip at pobox.com  Sat Oct 27 06:45:04 2007
From: skip at pobox.com (skip at pobox.com)
Date: Fri, 26 Oct 2007 23:45:04 -0500
Subject: [Python-Dev] resurrected modindex
Message-ID: <18210.49744.191842.622583@montanaro.dyndns.org>

A long time ago I wrote a redirector to note and display the most frequently
requested modules from the library reference manual's global module index.
When Georg Brandl did his massive rewrite of the Python manuals that broke.
I finally took the time this evening to resurrect that function.  The result
is here:

    http://www.webfast.com/modindex/

I reset all the counters and seeded the cache with a few modules.  Click
around in the index a few times then reload the above page.  You'll get the
idea.

Skip


From guido at python.org  Sat Oct 27 18:34:23 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 27 Oct 2007 09:34:23 -0700
Subject: [Python-Dev] resurrected modindex
In-Reply-To: <18210.49744.191842.622583@montanaro.dyndns.org>
References: <18210.49744.191842.622583@montanaro.dyndns.org>
Message-ID: <ca471dc20710270934p4dde3b24p6ecc9771d197bbc7@mail.gmail.com>

Cool! Is it just a matter of checking this in and it will work on the site?

--Guido

2007/10/26, skip at pobox.com <skip at pobox.com>:
> A long time ago I wrote a redirector to note and display the most frequently
> requested modules from the library reference manual's global module index.
> When Georg Brandl did his massive rewrite of the Python manuals that broke.
> I finally took the time this evening to resurrect that function.  The result
> is here:
>
>     http://www.webfast.com/modindex/
>
> I reset all the counters and seeded the cache with a few modules.  Click
> around in the index a few times then reload the above page.  You'll get the
> idea.
>
> Skip
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Sat Oct 27 18:58:12 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 27 Oct 2007 18:58:12 +0200
Subject: [Python-Dev] resurrected modindex
In-Reply-To: <ca471dc20710270934p4dde3b24p6ecc9771d197bbc7@mail.gmail.com>
References: <18210.49744.191842.622583@montanaro.dyndns.org>
	<ca471dc20710270934p4dde3b24p6ecc9771d197bbc7@mail.gmail.com>
Message-ID: <ffvqnb$n13$1@ger.gmane.org>

Guido van Rossum schrieb:
> Cool! Is it just a matter of checking this in and it will work on the site?

No. The current newstyle docs (2.6, 3.0) are static HTML pages.
However, Sphinx can also generate slightly different output which
is then used by a web app that has all the dynamic features for docs
like comments, and among them is also already something like Skip's
"frequently requested modules" list.

I wanted to keep the development docs static, to minimize load on the
docs server, and begin serving the 2.6 or 3.0 docs via the web app as
they are the "official" ones, that is, as soon as one of them is released.

Georg


-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From skip at pobox.com  Sat Oct 27 20:21:27 2007
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 27 Oct 2007 13:21:27 -0500
Subject: [Python-Dev] resurrected modindex
In-Reply-To: <ca471dc20710270934p4dde3b24p6ecc9771d197bbc7@mail.gmail.com>
References: <18210.49744.191842.622583@montanaro.dyndns.org>
	<ca471dc20710270934p4dde3b24p6ecc9771d197bbc7@mail.gmail.com>
Message-ID: <18211.33191.801112.652937@montanaro.dyndns.org>

>>>>> "Guido" == Guido van Rossum <guido at python.org> writes:

    Guido> Cool! Is it just a matter of checking this in and it will work on
    Guido> the site?

Alas, no.  I wrote this awhile ago when the site was built w/ ht2html.  My
thing is a simple front-end using Myghty.  As Georg indicated in his
response he has some dynamic stuff in Sphinx which will do this and more.  I
suspect once he goes fully dynamic my little hack won't be necessary any
longer, but I missed it, especially with the new layout where less of the
index is visible without paging down.

I got tired of my modindex URL doing the work necessary to process the index
(and thus being a little slower than just going straight to the index
itself) but not seeing the proper anchors in the file which allowed it to
insert the table and rewrite URLs to allow counting clicks.

Skip

From python at rcn.com  Sun Oct 28 20:14:38 2007
From: python at rcn.com (Raymond Hettinger)
Date: Sun, 28 Oct 2007 12:14:38 -0700
Subject: [Python-Dev] Overflow on enumerate() and itertools.count()
Message-ID: <00b301c81996$cd210060$6501a8c0@RaymondLaptop1>

For Py2.6, I fixed a long standing irritant where the count raised an OverflowError when it reached LONG_MAX.

If you guys agree that is a bug fix, I'll backport it to Py2.5.


Raymond

From guido at python.org  Mon Oct 29 01:58:51 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 28 Oct 2007 17:58:51 -0700
Subject: [Python-Dev] Overflow on enumerate() and itertools.count()
In-Reply-To: <00b301c81996$cd210060$6501a8c0@RaymondLaptop1>
References: <00b301c81996$cd210060$6501a8c0@RaymondLaptop1>
Message-ID: <ca471dc20710281758u5f25ba5drb3241a31e192a8e9@mail.gmail.com>

2007/10/28, Raymond Hettinger <python at rcn.com>:
> For Py2.6, I fixed a long standing irritant where the count raised an OverflowError when it reached LONG_MAX.
>
> If you guys agree that is a bug fix, I'll backport it to Py2.5.

Smells like a feature to me.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Mon Oct 29 15:30:32 2007
From: barry at python.org (Barry Warsaw)
Date: Mon, 29 Oct 2007 10:30:32 -0400
Subject: [Python-Dev] Does Python need a file locking module (slightly
	higher level)?
In-Reply-To: <18210.18858.361196.95168@montanaro.dyndns.org>
References: <18204.38001.562050.157197@montanaro.dyndns.org>
	<2C9CF122-7954-434C-BFCA-CB5AC0B229F3@python.org>
	<18205.27378.432540.931896@montanaro.dyndns.org>
	<95064092-825F-4D3E-A2D4-6309248A6247@python.org>
	<18210.18858.361196.95168@montanaro.dyndns.org>
Message-ID: <D6087965-1077-495C-9965-810A56A95C89@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Oct 26, 2007, at 4:10 PM, skip at pobox.com wrote:

>
>     Barry> I don't think any one solution will work for everybody.   
> I'm not
>     Barry> even sure we can define a common API a la the DBAPI, but if
>     Barry> something were to make it into the standard  
> distribution, that's
>     Barry> the direction I'd go in.
>
> I've been working on a lockfile module the past few days on the  
> train.  I
> have something that passes a boatload of doctest test cases on my  
> Mac, works
> for threads as well as processes (acquire the lock in thread one,  
> then block
> in thread two until thread one releases the lock).  The Unix  
> version relies
> on the atomic nature of the link(2) system call.  The Windows  
> version (not
> yet tested on that platform) relies on mkdir(2) being atomic.  (I  
> think it
> is.)  In theory, I suppose the mkdir version will work for both  
> platforms,
> so it's possible that you could have file locking between Windows  
> and Unix
> should you want it.
>
> The current implementation provides a FileLock class with these  
> methods:
>
>     acquire
>     release
>     is_locked
>     break_lock
>     __enter__
>     __exit__
>
> The acquire method takes an optional timeout parameter (None => wait
> forever, +ive value => block for awhile, zero or -ive => give up
> immediately).  The others all take no arguments.  I'm working on ReST
> documentation now and hope to have that finished over the weekend.   
> After
> that I'll write a simple setup.py, toss it out on PyPI, then  
> announce it
> more broadly.  If anyone would like to try it out and/or review the  
> code
> sooner (especially if you have access to Windows) let me know.   
> I'll shoot
> you a copy as it currently exists.
>
> The API and almost all test cases are defined in a _FileLock base  
> class.
> You could (in theory at least) subclass it to provide locking  
> through some
> other shared resource like a database and not have to write and or  
> many
> other test cases.

It sounds pretty interesting Skip, though I won't have time to look  
at that for a couple of weeks.  I did throw mine into PyPI:

http://pypi.python.org/pypi/locknix/1.0

Mostly did it just to get it off my plate.  You're being more  
diligent with tests and documentation than I am, and if your stuff  
works on Windows too, more thorough in your implementation.  There is  
one feature locknix has which probably isn't very interesting to  
other applications, and that is the ability to transfer a lock  
between processes across a fork.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iQCVAwUBRyXuiHEjvBPtnXfVAQIEgwP/dn7GXdT5tV3PGpVtU09nkfSPdRTBQQSz
ZZ3wFbSH6J+zMlM+esHLUBm+ZD1cqrlOLLvUIs7J5eRdryF3tRFjbZXqWV+a9/f+
gfLRS0E/80bX5EoJkGesRURtncY00E/cLGdTok4M1ZAWi7oO+K5l812uvrZzblOm
2hb7N/j1zYM=
=0Xq0
-----END PGP SIGNATURE-----

From status at bugs.python.org  Tue Oct 30 19:06:08 2007
From: status at bugs.python.org (Tracker)
Date: Tue, 30 Oct 2007 18:06:08 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071030180608.E12CF7830B@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/23/07 - 10/30/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1321 open (+28) / 11524 closed (+19) / 12845 total (+47)

Open issues with patches:   420

Average duration of open issues: 679 days.
Median duration of open issues: 772 days.

Open Issues Breakdown
   open  1317 (+28)
pending     4 ( +0)

Issues Created Or Reopened (48)
_______________________________

smtpd.SMTPServer throws exception on MAIL command with no arg    10/23/07
CLOSED http://bugs.python.org/issue1307    reopened gvanrossum               
       patch                                                                   

fix a few PyInt_Check vs PyLong_Check problems                   10/23/07
CLOSED http://bugs.python.org/issue1316    created  georg.brandl             
       py3k, patch                                                             

smtplib.SMTP docs                                                10/23/07
       http://bugs.python.org/issue1317    created  fdrake                   
                                                                               

Remove os.tmpnam() and os.tempnam()                              10/23/07
CLOSED http://bugs.python.org/issue1318    created  tiran                    
       patch                                                                   

py3k: fixes for test_ctypes                                      10/23/07
CLOSED http://bugs.python.org/issue1319    created  amaury.forgeotdarc       
       patch                                                                   

PCBuild8 Solution Support Changes                                10/24/07
       http://bugs.python.org/issue1320    created  JosephArmbruster         
                                                                               

TimedRotatingFileHandler logic for day of week is wrong          10/24/07
CLOSED http://bugs.python.org/issue1321    created  mosheco                  
                                                                               

platform.dist() has unpredictable result under Linux             10/24/07
       http://bugs.python.org/issue1322    created  sapetnioc                
                                                                               

py3k: file.truncate() changes the file position                  10/24/07
CLOSED http://bugs.python.org/issue1323    created  amaury.forgeotdarc       
       patch                                                                   

r58034 breaks building _ctypes with the upstream libffi.         10/25/07
CLOSED http://bugs.python.org/issue1324    created  doko                     
       patch                                                                   

zipimport.zipimporter.archive undocumented and untested          10/25/07
       http://bugs.python.org/issue1325    created  exarkun                  
                                                                               

"internal" zipimport.zipimporter feature untested                10/25/07
       http://bugs.python.org/issue1326    created  exarkun                  
                                                                               

Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmcl 10/25/07
       http://bugs.python.org/issue1327    created  eanxgeek                 
                                                                               

feature request: force BOM option                                10/25/07
       http://bugs.python.org/issue1328    created  jgsack                   
                                                                               

Different 3.0a1 exit behavior                                    10/25/07
CLOSED http://bugs.python.org/issue1329    created  MrJean1                  
                                                                               

Fix truncate on Windows, this time for real                      10/26/07
CLOSED http://bugs.python.org/issue1330    created  tiran                    
       py3k, patch                                                             

More fixes for Windows                                           10/26/07
CLOSED http://bugs.python.org/issue1331    created  tiran                    
       py3k, patch                                                             

threading.RLock().aquire(0) fails with python-2.5.1.amd64.msi    10/26/07
       http://bugs.python.org/issue1332    created  delwarl                  
                                                                               

merge urllib and urlparse functionality                          10/26/07
       http://bugs.python.org/issue1333    created  techtonik                
                                                                               

IDLE - Fix several highlighting bugs                             10/26/07
       http://bugs.python.org/issue1334    created  taleinat                 
       patch                                                                   

bytesiterator patch                                              10/26/07
CLOSED http://bugs.python.org/issue1335    created  tiran                    
                                                                               

subprocess.Popen hangs when child writes to stderr               10/26/07
       http://bugs.python.org/issue1336    created  jba                      
                                                                               

Tools/msi/msi.py does not work with PCBuild8                     10/26/07
       http://bugs.python.org/issue1337    created  kevinwatters             
                                                                               

pickling bytes?                                                  10/26/07
       http://bugs.python.org/issue1338    created  gvanrossum               
       py3k                                                                    

smtplib starttls() should ehlo() if it needs to                  10/26/07
       http://bugs.python.org/issue1339    created  fenner                   
                                                                               

correction for test_tempfile in py3k on Windows                  10/28/07
CLOSED http://bugs.python.org/issue1340    reopened gvanrossum               
                                                                               

correction for test_fileinput in py3k on Windows                 10/26/07
CLOSED http://bugs.python.org/issue1341    created  amaury.forgeotdarc       
       patch                                                                   

Crash on Windows if Python runs from a directory with umlauts    10/27/07
       http://bugs.python.org/issue1342    created  tiran                    
                                                                               

XMLGenerator: nice <empty/> elements                             10/27/07
       http://bugs.python.org/issue1343    created  panzi                    
                                                                               

subprocess.communication doc could use clarification             10/27/07
       http://bugs.python.org/issue1344    created  dsturtevant              
                                                                               

Fix for test_netrc on Windows                                    10/27/07
       http://bugs.python.org/issue1345    created  tiran                    
       patch                                                                   

Error using >>> from OpenGL.GLUT import *                        10/27/07
       http://bugs.python.org/issue1346    created  neuralsensor             
                                                                               

BaseHTTPServer writing strings to bytes interface                10/28/07
       http://bugs.python.org/issue1347    created  janssen                  
       patch                                                                   

httplib closes socket, then tries to read from it                10/28/07
       http://bugs.python.org/issue1348    created  janssen                  
       py3k, patch                                                             

more uses of ord() in plat-mac/ic.py                             10/28/07
       http://bugs.python.org/issue1349    created  janssen                  
       py3k, patch                                                             

IDLE - CallTips enhancement - show full doc-string in new window 10/28/07
       http://bugs.python.org/issue1350    created  taleinat                 
       patch                                                                   

Add getsize() to io instances                                    10/28/07
       http://bugs.python.org/issue1351    created  tiran                    
                                                                               

Preliminary stderr patch                                         10/28/07
       http://bugs.python.org/issue1352    created  tiran                    
                                                                               

mp4 missing from mimetypes.py                                    10/29/07
CLOSED http://bugs.python.org/issue1353    created  kraft                    
                                                                               

windows installer problem                                        10/29/07
       http://bugs.python.org/issue1354    created  rajar                    
                                                                               

xml.dom refers to PyXML, which is no longer maintained           10/29/07
       http://bugs.python.org/issue1355    created  whooey1830               
                                                                               

3.0a1 Solaris w/ SUN C/C++                                       10/29/07
CLOSED http://bugs.python.org/issue1356    created  MrJean1                  
                                                                               

3.0a1 make test Error on  Solaris w/ SUN C/C++                   10/29/07
CLOSED http://bugs.python.org/issue1357    created  MrJean1                  
                                                                               

Compile error on OS X 10.5                                       10/29/07
       http://bugs.python.org/issue1358    created  andres                   
                                                                               

py3k: out of bounds read in PyUnicode_DecodeUnicodeEscape        10/29/07
CLOSED http://bugs.python.org/issue1359    reopened gvanrossum               
                                                                               

Queue.get() can't be interrupted with Ctrl-C unless timed out    10/30/07
CLOSED http://bugs.python.org/issue1360    created  piro                     
                                                                               

please close: hashlib uses OpenSSL which is much slower than Aar 10/30/07
CLOSED http://bugs.python.org/issue1361    created  Omnifarious              
                                                                               

Simple mistake in http://docs.python.org/tut/node6.html          10/30/07
CLOSED http://bugs.python.org/issue1362    created  dmazz                    
                                                                               



Issues Now Closed (37)
______________________

use bytes for code objects                                         60 days
       http://bugs.python.org/issue1022    gvanrossum               
       patch                                                                   

itertools missing, causes interactive help to break                54 days
       http://bugs.python.org/issue1077    georg.brandl             
                                                                               

Strange Python hangup                                              16 days
       http://bugs.python.org/issue1255    kakacek                  
                                                                               

Removal of basestring type                                         14 days
       http://bugs.python.org/issue1258    gvanrossum               
       patch                                                                   

IDLE does not start if windows environment variable containing '   17 days
       http://bugs.python.org/issue1262    kbk                      
                                                                               

os.environ.pop doesn't work                                        10 days
       http://bugs.python.org/issue1287    georg.brandl             
                                                                               

xml.dom.minidom not able to handle utf-8 data                       7 days
       http://bugs.python.org/issue1290    facundobatista           
                                                                               

Support for z/OS and EBCDIC.                                        6 days
       http://bugs.python.org/issue1298    gvanrossum               
                                                                               

Fixes for profile/cprofile                                          6 days
       http://bugs.python.org/issue1302    tiran                    
       py3k, patch                                                             

adapt str8 constructor to bytes constructor                         5 days
       http://bugs.python.org/issue1303    georg.brandl             
       py3k, patch                                                             

smtpd.SMTPServer throws exception on MAIL command with no arg       0 days
       http://bugs.python.org/issue1307    nnorwitz                 
       patch                                                                   

Patch for strftime problem on German Windows                        0 days
       http://bugs.python.org/issue1314    gvanrossum               
       patch                                                                   

fix a few PyInt_Check vs PyLong_Check problems                      0 days
       http://bugs.python.org/issue1316    georg.brandl             
       py3k, patch                                                             

Remove os.tmpnam() and os.tempnam()                                 2 days
       http://bugs.python.org/issue1318    gvanrossum               
       patch                                                                   

py3k: fixes for test_ctypes                                         1 days
       http://bugs.python.org/issue1319    theller                  
       patch                                                                   

TimedRotatingFileHandler logic for day of week is wrong             0 days
       http://bugs.python.org/issue1321    vsajip                   
                                                                               

py3k: file.truncate() changes the file position                     1 days
       http://bugs.python.org/issue1323    gvanrossum               
       patch                                                                   

r58034 breaks building _ctypes with the upstream libffi.            1 days
       http://bugs.python.org/issue1324    theller                  
       patch                                                                   

Different 3.0a1 exit behavior                                       5 days
       http://bugs.python.org/issue1329    gvanrossum               
                                                                               

Fix truncate on Windows, this time for real                         1 days
       http://bugs.python.org/issue1330    gvanrossum               
       py3k, patch                                                             

More fixes for Windows                                              0 days
       http://bugs.python.org/issue1331    gvanrossum               
       py3k, patch                                                             

bytesiterator patch                                                 0 days
       http://bugs.python.org/issue1335    gvanrossum               
                                                                               

correction for test_tempfile in py3k on Windows                     1 days
       http://bugs.python.org/issue1340    gvanrossum               
                                                                               

correction for test_fileinput in py3k on Windows                    3 days
       http://bugs.python.org/issue1341    gvanrossum               
       patch                                                                   

mp4 missing from mimetypes.py                                       0 days
       http://bugs.python.org/issue1353    gvanrossum               
                                                                               

3.0a1 Solaris w/ SUN C/C++                                          0 days
       http://bugs.python.org/issue1356    gvanrossum               
                                                                               

3.0a1 make test Error on  Solaris w/ SUN C/C++                      0 days
       http://bugs.python.org/issue1357    gvanrossum               
                                                                               

py3k: out of bounds read in PyUnicode_DecodeUnicodeEscape           0 days
       http://bugs.python.org/issue1359    gvanrossum               
                                                                               

Queue.get() can't be interrupted with Ctrl-C unless timed out       0 days
       http://bugs.python.org/issue1360    piro                     
                                                                               

please close: hashlib uses OpenSSL which is much slower than Aar    0 days
       http://bugs.python.org/issue1361    facundobatista           
                                                                               

Simple mistake in http://docs.python.org/tut/node6.html             1 days
       http://bugs.python.org/issue1362    georg.brandl             
                                                                               

Non-contiguous indexing and slicing                              2369 days
       http://bugs.python.org/issue419903  facundobatista           
                                                                               

python should obey the FHS                                       1912 days
       http://bugs.python.org/issue588756  facundobatista           
                                                                               

threading.Thread.join() cannot be interrupted by a Ctrl-C         952 days
       http://bugs.python.org/issue1167930 gvanrossum               
                                                                               

Ctrl+C for copy does not work when caps-lock is on                713 days
       http://bugs.python.org/issue1356720 kbk                      
                                                                               

"Really  print?" Dialog                                           170 days
       http://bugs.python.org/issue1717170 taleinat                 
       patch                                                                   

Open always create new tab or new browser                         103 days
       http://bugs.python.org/issue1753371 georg.brandl             
                                                                               



Top Issues Most Discussed (10)
______________________________

 36 Different 3.0a1 exit behavior                                      5 days
closed  http://bugs.python.org/issue1329   

 12 threading.RLock().aquire(0) fails with python-2.5.1.amd64.msi      5 days
open    http://bugs.python.org/issue1332   

 12 platform.dist() has unpredictable result under Linux               6 days
open    http://bugs.python.org/issue1322   

 10 Remove os.tmpnam() and os.tempnam()                                2 days
closed  http://bugs.python.org/issue1318   

  8 py3k: fixes for test_ctypes                                        1 days
closed  http://bugs.python.org/issue1319   

  7 Fixes for profile/cprofile                                         6 days
closed  http://bugs.python.org/issue1302   

  6 feature request: force BOM option                                  5 days
open    http://bugs.python.org/issue1328   

  6 os.path.exists(os.devnull) regression on windows                   8 days
open    http://bugs.python.org/issue1311   

  6 IDLE - patch Delegator to support callables                       21 days
open    http://bugs.python.org/issue1252   

  5 Enhanced tabbed pane widget                                      324 days
open    http://bugs.python.org/issue1612746




From arkanes at gmail.com  Tue Oct 30 19:56:56 2007
From: arkanes at gmail.com (Chris Mellon)
Date: Tue, 30 Oct 2007 13:56:56 -0500
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <4720161F.7040306@v.loewis.de>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
	<4720161F.7040306@v.loewis.de>
Message-ID: <4866bea60710301156k6a4ac17as7dcdcb4949ca4c08@mail.gmail.com>

On Oct 24, 2007 11:05 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > So, the question is what we should do?:
>
> Before this question can be answered, I think we need to fully
> understand what precisely is happening in 2.4, and what precisely
> is happening in 2.5.
>
> AFAICT, it is *not* the case that Python 2.4 (indirectly) has
> hard-coded the names CON, PRN, NUL etc. in the C library. Instead,
> Python 2.4 *also* relies on kernel32 functions to determine that
> these files "exist".
>
> My question now is what specific kernel32 functions Python 2.4
> calls to determine that NUL is a file; before that question
> is sufficiently answered, I don't think any action should be
> taken.
>


os.path.exist() in win32 just calls os.stat() and decides it doesn't
exist if an error is returned. os.stat() uses the vcrt stat()in 2.4,
but 2.5 implements it directly in terms of win32 api to deal with
limitations in the vcrt implementation.

The hand-rolled stat uses GetFileAttributesEx, which returns 0 for the
special filenames, with an error code of "The parameter is incorrect"
(87), which is why os.path.exists() claims it doesn't exist.

Interestingly, plain old GetFileAttributes() works, and returns
FILE_ATTRIBUTE_ARCHIVE for them.

From theller at ctypes.org  Tue Oct 30 21:41:22 2007
From: theller at ctypes.org (Thomas Heller)
Date: Tue, 30 Oct 2007 21:41:22 +0100
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <4866bea60710301156k6a4ac17as7dcdcb4949ca4c08@mail.gmail.com>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>	<4720161F.7040306@v.loewis.de>
	<4866bea60710301156k6a4ac17as7dcdcb4949ca4c08@mail.gmail.com>
Message-ID: <fg84tj$33c$1@ger.gmane.org>

Chris Mellon schrieb:
> On Oct 24, 2007 11:05 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> > So, the question is what we should do?:
>>
>> Before this question can be answered, I think we need to fully
>> understand what precisely is happening in 2.4, and what precisely
>> is happening in 2.5.
>>
>> AFAICT, it is *not* the case that Python 2.4 (indirectly) has
>> hard-coded the names CON, PRN, NUL etc. in the C library. Instead,
>> Python 2.4 *also* relies on kernel32 functions to determine that
>> these files "exist".
>>
>> My question now is what specific kernel32 functions Python 2.4
>> calls to determine that NUL is a file; before that question
>> is sufficiently answered, I don't think any action should be
>> taken.
>>
> 
> 
> os.path.exist() in win32 just calls os.stat() and decides it doesn't
> exist if an error is returned. os.stat() uses the vcrt stat()in 2.4,
> but 2.5 implements it directly in terms of win32 api to deal with
> limitations in the vcrt implementation.
> 
> The hand-rolled stat uses GetFileAttributesEx, which returns 0 for the
> special filenames, with an error code of "The parameter is incorrect"
> (87), which is why os.path.exists() claims it doesn't exist.
> 
> Interestingly, plain old GetFileAttributes() works, and returns
> FILE_ATTRIBUTE_ARCHIVE for them.

See also a recent blog entry of Raymond Chen at
http://blogs.msdn.com/oldnewthing/archive/2007/10/23/5612082.aspx

Thomas


From martin at v.loewis.de  Tue Oct 30 22:10:18 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 30 Oct 2007 22:10:18 +0100
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <4866bea60710301156k6a4ac17as7dcdcb4949ca4c08@mail.gmail.com>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>	<4720161F.7040306@v.loewis.de>
	<4866bea60710301156k6a4ac17as7dcdcb4949ca4c08@mail.gmail.com>
Message-ID: <47279DBA.1090400@v.loewis.de>

>> My question now is what specific kernel32 functions Python 2.4
>> calls to determine that NUL is a file; before that question
>> is sufficiently answered, I don't think any action should be
>> taken.
>>
> 
> os.path.exist() in win32 just calls os.stat() and decides it doesn't
> exist if an error is returned. os.stat() uses the vcrt stat()in 2.4,
> but 2.5 implements it directly in terms of win32 api to deal with
> limitations in the vcrt implementation.

That doesn't really answer the question, though - you merely state
that Python 2.4 calls the CRT, but then my question is still what
kernel32 functions are called to have stat on NUL succeed.

> Interestingly, plain old GetFileAttributes() works, and returns
> FILE_ATTRIBUTE_ARCHIVE for them.

What about the other attributes (like modification time, size, etc)?

Regards,
Martin

From nnorwitz at gmail.com  Wed Oct 31 07:19:26 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 30 Oct 2007 23:19:26 -0700
Subject: [Python-Dev] 2.5.2 is coming
In-Reply-To: <e04bdf310710240722m2b1a5ea5p3168a5e4caa8aaf4@mail.gmail.com>
References: <ee2a432c0710112232u658f0addl91c391c4f5b0c1d3@mail.gmail.com>
	<e04bdf310710240722m2b1a5ea5p3168a5e4caa8aaf4@mail.gmail.com>
Message-ID: <ee2a432c0710302319h31b45446l8672a617028f6fc1@mail.gmail.com>

On Oct 24, 2007 7:22 AM, Facundo Batista <facundobatista at gmail.com> wrote:
> 2007/10/12, Neal Norwitz <nnorwitz at gmail.com>:
>
> > The plan is cut the release candidate around Tuesday/Wednesday next
> > week (Oct 16/17).  If all goes well, 2.5.2 final will follow a week
> > later.
>
> Hi Neal! Do you have any update of this schedule?

Sorry, the various schedules of everyone involved didn't allow us to
get a release out as planned.  We are hoping to get out a 2.5.2
release candidate in a couple of weeks.

n

From nnorwitz at gmail.com  Wed Oct 31 08:01:48 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 31 Oct 2007 00:01:48 -0700
Subject: [Python-Dev] Windows amd64 warnings
Message-ID: <ee2a432c0710310001o3427de60x7f0636b11fcdecf@mail.gmail.com>

There are many warnings from the trunk (and 2.5) builds on Win64.  I
suspect quite a few of these are real problems.  Win64 is the most
unusual platform wrt 64-bits due to long being 32-bit.  If we need
64-bits, we need to use Py_ssize_t.

IIRC, we've had some bug reports for Win64.  I know I've fixed some
problems found in py3k.  It would be great if someone could create a
patch to fix the warnings.

The link below shows the warnings from a recent checkin.

  http://python.org/dev/buildbot/all/amd64%20XP%20trunk/builds/297/step-compile/0

n

From guido at python.org  Wed Oct 31 18:08:03 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 31 Oct 2007 10:08:03 -0700
Subject: [Python-Dev] Declaring setters with getters
Message-ID: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>

I've come up with a relatively unobtrusive pattern for defining
setters. Given the following definition:

def propset(prop):
    assert isinstance(prop, property)
    def helper(func):
        return property(prop.__get__, func, func, prop.__doc__)
    return helper

we can declare getters and setters as follows:

class C(object):

    _encoding = None

    @property
    def encoding(self):
        return self._encoding

    @propset(encoding)
    def encoding(self, value=None):
        if value is not None:
            unicode("0", value)  # Test it
        self._encoding = value

c = C()
print(c.encoding)
c.encoding = "ascii"
print(c.encoding)
try:
    c.encoding = "invalid"  # Fails
except:
    pass
print(c.encoding)

I'd like to make this a standard built-in, in the hope the debate on
how to declare settable properties.

I'd also like to change property so that the doc string defaults to
the doc string of the getter.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Wed Oct 31 18:48:13 2007
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 31 Oct 2007 12:48:13 -0500
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
Message-ID: <18216.49117.401434.718065@montanaro.dyndns.org>


    Guido> I've come up with a relatively unobtrusive pattern for defining
    Guido> setters. Given the following definition:

    ...

I'm a pretty naive user of properties, so can you explain to me how what you
propose is better than

    class C(object):
        def __init__(self):
            self._encoding = None

        def get_encoding(self):
            return self._encoding

        def set_encoding(self, value):
            if value is not None:
                unicode("0", value)  # Test it
            self._encoding = value
        encoding = property(get_encoding, set_encoding)

?

    Guido> I'd also like to change property so that the doc string defaults
    Guido> to the doc string of the getter.

How about defaulting it to the first argument to property() which has a doc
string?

Skip

From pje at telecommunity.com  Wed Oct 31 18:51:37 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 31 Oct 2007 13:51:37 -0400
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.co
 m>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
Message-ID: <20071031174852.9CAB33A40A4@sparrow.telecommunity.com>

At 10:08 AM 10/31/2007 -0700, Guido van Rossum wrote:
>I've come up with a relatively unobtrusive pattern for defining
>setters. Given the following definition:
>
>def propset(prop):
>     assert isinstance(prop, property)
>     def helper(func):
>         return property(prop.__get__, func, func, prop.__doc__)
>     return helper

Shouldn't that be property(prop.fget, func, prop.fdel, prop.__doc__), 
and have a matching propdel decorator?

Apart from those extremely small nits, a big +1, and I expect to 
swipe this idea more or less immediately.  :)


>I'd also like to change property so that the doc string defaults to
>the doc string of the getter.

+1 also; I thought it did this already, but am now disappointed to 
find it doesn't.  :)


From python at rcn.com  Wed Oct 31 18:57:43 2007
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 31 Oct 2007 13:57:43 -0400 (EDT)
Subject: [Python-Dev] Declaring setters with getters
Message-ID: <20071031135743.AAO18248@ms19.lnh.mail.rcn.net>

> I'd like to make this [propset] a standard built-in, 

+1 -- I find this to be an attractive syntax

> I'd also like to change property so that the doc 
> string defaults to the doc string of the getter.

+1 -- This should also be done for classmethod,
staticmethod, and anything else that wraps functions.

Alternatively, the doc string could be made a little
more informative:

__doc__ = 'property of function %s: %s' % (f.__name__, f.__doc__) 


Raymond

From guido at python.org  Wed Oct 31 19:05:32 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 31 Oct 2007 11:05:32 -0700
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <18216.49117.401434.718065@montanaro.dyndns.org>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
	<18216.49117.401434.718065@montanaro.dyndns.org>
Message-ID: <ca471dc20710311105h6d1e65ffj5acea474ff084240@mail.gmail.com>

On 10/31/07, skip at pobox.com <skip at pobox.com> wrote:
>
>     Guido> I've come up with a relatively unobtrusive pattern for defining
>     Guido> setters. Given the following definition:
>
>     ...
>
> I'm a pretty naive user of properties, so can you explain to me how what you
> propose is better than
>
>     class C(object):
>         def __init__(self):
>             self._encoding = None
>
>         def get_encoding(self):
>             return self._encoding
>
>         def set_encoding(self, value):
>             if value is not None:
>                 unicode("0", value)  # Test it
>             self._encoding = value
>         encoding = property(get_encoding, set_encoding)
>
> ?

Mostly so it's easy to add a setter to a property that originally had
only a getter. For read-only properties, using

  @property
  def name(self):
      return <whatever>

is preferred over

  def get_name(self):
      return <whatever>
  name = property(get_name)

>     Guido> I'd also like to change property so that the doc string defaults
>     Guido> to the doc string of the getter.
>
> How about defaulting it to the first argument to property() which has a doc
> string?

I considered that briefly, but decided against it -- it could lead to
odd results if the getter has no docstring and the setter's docstring
is about setting only.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From status at bugs.python.org  Wed Oct 31 19:06:08 2007
From: status at bugs.python.org (Tracker)
Date: Wed, 31 Oct 2007 18:06:08 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071031180608.ADD5E7833F@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (10/24/07 - 10/31/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1320 open (+25) / 11530 closed (+20) / 12850 total (+45)

Open issues with patches:   418

Average duration of open issues: 681 days.
Median duration of open issues: 773 days.

Open Issues Breakdown
   open  1316 (+25)
pending     4 ( +0)

Issues Created Or Reopened (45)
_______________________________

py3k: file.truncate() changes the file position                  10/24/07
CLOSED http://bugs.python.org/issue1323    created  amaury.forgeotdarc       
       patch                                                                   

r58034 breaks building _ctypes with the upstream libffi.         10/25/07
CLOSED http://bugs.python.org/issue1324    created  doko                     
       patch                                                                   

zipimport.zipimporter.archive undocumented and untested          10/25/07
       http://bugs.python.org/issue1325    created  exarkun                  
                                                                               

"internal" zipimport.zipimporter feature untested                10/25/07
       http://bugs.python.org/issue1326    created  exarkun                  
                                                                               

Python 2.4+ spends too much time in PyEval_EvalFrame w/ xmlrpmcl 10/25/07
       http://bugs.python.org/issue1327    created  eanxgeek                 
                                                                               

feature request: force BOM option                                10/25/07
       http://bugs.python.org/issue1328    created  jgsack                   
                                                                               

Different 3.0a1 exit behavior                                    10/25/07
CLOSED http://bugs.python.org/issue1329    created  MrJean1                  
                                                                               

Fix truncate on Windows, this time for real                      10/26/07
CLOSED http://bugs.python.org/issue1330    created  tiran                    
       py3k, patch                                                             

More fixes for Windows                                           10/26/07
CLOSED http://bugs.python.org/issue1331    created  tiran                    
       py3k, patch                                                             

threading.RLock().aquire(0) fails with python-2.5.1.amd64.msi    10/26/07
       http://bugs.python.org/issue1332    created  delwarl                  
                                                                               

merge urllib and urlparse functionality                          10/26/07
       http://bugs.python.org/issue1333    created  techtonik                
                                                                               

IDLE - Fix several highlighting bugs                             10/26/07
       http://bugs.python.org/issue1334    created  taleinat                 
       patch                                                                   

bytesiterator patch                                              10/26/07
CLOSED http://bugs.python.org/issue1335    created  tiran                    
                                                                               

subprocess.Popen hangs when child writes to stderr               10/26/07
       http://bugs.python.org/issue1336    created  jba                      
                                                                               

Tools/msi/msi.py does not work with PCBuild8                     10/26/07
       http://bugs.python.org/issue1337    created  kevinwatters             
                                                                               

pickling bytes?                                                  10/26/07
       http://bugs.python.org/issue1338    created  gvanrossum               
       py3k                                                                    

smtplib starttls() should ehlo() if it needs to                  10/26/07
       http://bugs.python.org/issue1339    created  fenner                   
                                                                               

correction for test_tempfile in py3k on Windows                  10/28/07
CLOSED http://bugs.python.org/issue1340    reopened gvanrossum               
                                                                               

correction for test_fileinput in py3k on Windows                 10/26/07
CLOSED http://bugs.python.org/issue1341    created  amaury.forgeotdarc       
       patch                                                                   

Crash on Windows if Python runs from a directory with umlauts    10/27/07
       http://bugs.python.org/issue1342    created  tiran                    
                                                                               

XMLGenerator: nice <empty/> elements                             10/27/07
       http://bugs.python.org/issue1343    created  panzi                    
                                                                               

subprocess.communication doc could use clarification             10/27/07
       http://bugs.python.org/issue1344    created  dsturtevant              
                                                                               

Fix for test_netrc on Windows                                    10/27/07
       http://bugs.python.org/issue1345    created  tiran                    
       patch                                                                   

Error using >>> from OpenGL.GLUT import *                        10/27/07
CLOSED http://bugs.python.org/issue1346    created  neuralsensor             
                                                                               

BaseHTTPServer writing strings to bytes interface                10/28/07
CLOSED http://bugs.python.org/issue1347    created  janssen                  
       py3k, patch                                                             

httplib closes socket, then tries to read from it                10/28/07
       http://bugs.python.org/issue1348    created  janssen                  
       py3k, patch                                                             

more uses of ord() in plat-mac/ic.py                             10/28/07
CLOSED http://bugs.python.org/issue1349    created  janssen                  
       py3k, patch                                                             

IDLE - CallTips enhancement - show full doc-string in new window 10/28/07
       http://bugs.python.org/issue1350    created  taleinat                 
       patch                                                                   

Add getsize() to io instances                                    10/28/07
       http://bugs.python.org/issue1351    created  tiran                    
                                                                               

Preliminary stderr patch                                         10/28/07
CLOSED http://bugs.python.org/issue1352    created  tiran                    
                                                                               

mp4 missing from mimetypes.py                                    10/29/07
CLOSED http://bugs.python.org/issue1353    created  kraft                    
                                                                               

windows installer problem                                        10/29/07
       http://bugs.python.org/issue1354    created  rajar                    
                                                                               

xml.dom refers to PyXML, which is no longer maintained           10/29/07
       http://bugs.python.org/issue1355    created  whooey1830               
                                                                               

3.0a1 Solaris w/ SUN C/C++                                       10/29/07
CLOSED http://bugs.python.org/issue1356    created  MrJean1                  
                                                                               

3.0a1 make test Error on  Solaris w/ SUN C/C++                   10/29/07
CLOSED http://bugs.python.org/issue1357    created  MrJean1                  
                                                                               

Compile error on OS X 10.5                                       10/29/07
       http://bugs.python.org/issue1358    created  andres                   
                                                                               

py3k: out of bounds read in PyUnicode_DecodeUnicodeEscape        10/29/07
CLOSED http://bugs.python.org/issue1359    reopened gvanrossum               
                                                                               

Queue.get() can't be interrupted with Ctrl-C unless timed out    10/30/07
CLOSED http://bugs.python.org/issue1360    created  piro                     
                                                                               

please close: hashlib uses OpenSSL which is much slower than Aar 10/30/07
CLOSED http://bugs.python.org/issue1361    created  Omnifarious              
                                                                               

Simple mistake in http://docs.python.org/tut/node6.html          10/30/07
CLOSED http://bugs.python.org/issue1362    created  dmazz                    
                                                                               

python 2.4.4 fails on solaris (sun4u sparc SUNW,Sun-Fire-880)    10/30/07
       http://bugs.python.org/issue1363    created  theoryno3                
                                                                               

os.lstat documentation error                                     10/31/07
       http://bugs.python.org/issue1364    created  billiejoex               
                                                                               

bytes(<int>) constructor                                         10/31/07
CLOSED http://bugs.python.org/issue1365    created  tiran                    
                                                                               

popen spawned process may not write to stdout under windows      10/31/07
       http://bugs.python.org/issue1366    created  pmezard                  
                                                                               

mkdir+chdir problem in multiple threads                          10/31/07
       http://bugs.python.org/issue1367    created  anonyprog                
                                                                               



Issues Now Closed (36)
______________________

use bytes for code objects                                         60 days
       http://bugs.python.org/issue1022    gvanrossum               
       patch                                                                   

itertools missing, causes interactive help to break                54 days
       http://bugs.python.org/issue1077    georg.brandl             
                                                                               

Strange Python hangup                                              16 days
       http://bugs.python.org/issue1255    kakacek                  
                                                                               

Removal of basestring type                                         14 days
       http://bugs.python.org/issue1258    gvanrossum               
       patch                                                                   

IDLE does not start if windows environment variable containing '   17 days
       http://bugs.python.org/issue1262    kbk                      
                                                                               

os.environ.pop doesn't work                                        10 days
       http://bugs.python.org/issue1287    georg.brandl             
                                                                               

xml.dom.minidom not able to handle utf-8 data                       7 days
       http://bugs.python.org/issue1290    facundobatista           
                                                                               

Fixes for profile/cprofile                                          6 days
       http://bugs.python.org/issue1302    tiran                    
       py3k, patch                                                             

adapt str8 constructor to bytes constructor                         5 days
       http://bugs.python.org/issue1303    georg.brandl             
       py3k, patch                                                             

Remove os.tmpnam() and os.tempnam()                                 2 days
       http://bugs.python.org/issue1318    gvanrossum               
       patch                                                                   

py3k: fixes for test_ctypes                                         1 days
       http://bugs.python.org/issue1319    theller                  
       patch                                                                   

py3k: file.truncate() changes the file position                     1 days
       http://bugs.python.org/issue1323    gvanrossum               
       patch                                                                   

r58034 breaks building _ctypes with the upstream libffi.            1 days
       http://bugs.python.org/issue1324    theller                  
       patch                                                                   

Different 3.0a1 exit behavior                                       5 days
       http://bugs.python.org/issue1329    gvanrossum               
                                                                               

Fix truncate on Windows, this time for real                         1 days
       http://bugs.python.org/issue1330    gvanrossum               
       py3k, patch                                                             

More fixes for Windows                                              0 days
       http://bugs.python.org/issue1331    gvanrossum               
       py3k, patch                                                             

bytesiterator patch                                                 0 days
       http://bugs.python.org/issue1335    gvanrossum               
                                                                               

correction for test_tempfile in py3k on Windows                     1 days
       http://bugs.python.org/issue1340    gvanrossum               
                                                                               

correction for test_fileinput in py3k on Windows                    3 days
       http://bugs.python.org/issue1341    gvanrossum               
       patch                                                                   

Error using >>> from OpenGL.GLUT import *                           3 days
       http://bugs.python.org/issue1346    nnorwitz                 
                                                                               

BaseHTTPServer writing strings to bytes interface                   3 days
       http://bugs.python.org/issue1347    janssen                  
       py3k, patch                                                             

more uses of ord() in plat-mac/ic.py                                3 days
       http://bugs.python.org/issue1349    janssen                  
       py3k, patch                                                             

Preliminary stderr patch                                            2 days
       http://bugs.python.org/issue1352    tiran                    
                                                                               

mp4 missing from mimetypes.py                                       0 days
       http://bugs.python.org/issue1353    gvanrossum               
                                                                               

3.0a1 Solaris w/ SUN C/C++                                          0 days
       http://bugs.python.org/issue1356    gvanrossum               
                                                                               

3.0a1 make test Error on  Solaris w/ SUN C/C++                      0 days
       http://bugs.python.org/issue1357    gvanrossum               
                                                                               

py3k: out of bounds read in PyUnicode_DecodeUnicodeEscape           0 days
       http://bugs.python.org/issue1359    gvanrossum               
                                                                               

Queue.get() can't be interrupted with Ctrl-C unless timed out       0 days
       http://bugs.python.org/issue1360    piro                     
                                                                               

please close: hashlib uses OpenSSL which is much slower than Aar    0 days
       http://bugs.python.org/issue1361    facundobatista           
                                                                               

Simple mistake in http://docs.python.org/tut/node6.html             1 days
       http://bugs.python.org/issue1362    georg.brandl             
                                                                               

bytes(<int>) constructor                                            0 days
       http://bugs.python.org/issue1365    gvanrossum               
                                                                               

threading.Thread.join() cannot be interrupted by a Ctrl-C         952 days
       http://bugs.python.org/issue1167930 gvanrossum               
                                                                               

Ctrl+C for copy does not work when caps-lock is on                713 days
       http://bugs.python.org/issue1356720 kbk                      
                                                                               

"Really  print?" Dialog                                           170 days
       http://bugs.python.org/issue1717170 taleinat                 
       patch                                                                   

Open always create new tab or new browser                         103 days
       http://bugs.python.org/issue1753371 georg.brandl             
                                                                               

Errors in site.py not reported properly                            82 days
       http://bugs.python.org/issue1771260 gvanrossum               
       py3k                                                                    



Top Issues Most Discussed (10)
______________________________

 38 Different 3.0a1 exit behavior                                      5 days
closed  http://bugs.python.org/issue1329   

 12 threading.RLock().aquire(0) fails with python-2.5.1.amd64.msi      6 days
open    http://bugs.python.org/issue1332   

 12 platform.dist() has unpredictable result under Linux               7 days
open    http://bugs.python.org/issue1322   

  8 IDLE - patch Delegator to support callables                       22 days
open    http://bugs.python.org/issue1252   

  7 py3k: fixes for test_ctypes                                        1 days
closed  http://bugs.python.org/issue1319   

  7 Fixes for profile/cprofile                                         6 days
closed  http://bugs.python.org/issue1302   

  6 feature request: force BOM option                                  6 days
open    http://bugs.python.org/issue1328   

  6 Remove os.tmpnam() and os.tempnam()                                2 days
closed  http://bugs.python.org/issue1318   

  5 Enhanced tabbed pane widget                                      325 days
open    http://bugs.python.org/issue1612746

  5 Compile error on OS X 10.5                                         2 days
open    http://bugs.python.org/issue1358   




From python at rcn.com  Wed Oct 31 19:12:11 2007
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 31 Oct 2007 14:12:11 -0400 (EDT)
Subject: [Python-Dev] Declaring setters with getters
Message-ID: <20071031141211.AAO19441@ms19.lnh.mail.rcn.net>

> and have a matching propdel decorator?

-1. That would be a complete waste of builtin space.
Put stuff in when it is really needed.  Ideas are
not required to automatically propagate from the
commonly used cases to the rarely used cases.


Raymond

From g.brandl at gmx.net  Wed Oct 31 19:14:42 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 31 Oct 2007 19:14:42 +0100
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
Message-ID: <fgagmj$6k5$1@ger.gmane.org>

Guido van Rossum schrieb:

> I'd like to make this a standard built-in, in the hope the debate on
> how to declare settable properties.

+1. I'm waiting for the obligatory cry for @propdel though :)

> I'd also like to change property so that the doc string defaults to
> the doc string of the getter.

I may be misunderstanding, but isn't that what patch #1434038 already
implemented in 2.5?

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From guido at python.org  Wed Oct 31 19:14:03 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 31 Oct 2007 11:14:03 -0700
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <20071031135743.AAO18248@ms19.lnh.mail.rcn.net>
References: <20071031135743.AAO18248@ms19.lnh.mail.rcn.net>
Message-ID: <ca471dc20710311114wd2fdf50t4577d98968ddd799@mail.gmail.com>

On 10/31/07, Raymond Hettinger <python at rcn.com> wrote:
> > I'd like to make this [propset] a standard built-in,
>
> +1 -- I find this to be an attractive syntax
>
> > I'd also like to change property so that the doc
> > string defaults to the doc string of the getter.
>
> +1 -- This should also be done for classmethod,
> staticmethod, and anything else that wraps functions.

With this level of support, it could be implemented by this afternoon!
(and backported to 2.4 within a week. ;-)

> Alternatively, the doc string could be made a little
> more informative:
>
> __doc__ = 'property of function %s: %s' % (f.__name__, f.__doc__)

Hm, not clear that helps. It's not a property of the function. We
should just teach users to write docstrings that read well for this
use case.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Oct 31 19:25:30 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 31 Oct 2007 11:25:30 -0700
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <fgagmj$6k5$1@ger.gmane.org>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
	<fgagmj$6k5$1@ger.gmane.org>
Message-ID: <ca471dc20710311125y6d212bc6yeed4a668daf57fdd@mail.gmail.com>

On 10/31/07, Georg Brandl <g.brandl at gmx.net> wrote:
> Guido van Rossum schrieb:
>
> > I'd like to make this a standard built-in, in the hope the debate on
> > how to declare settable properties.
>
> +1. I'm waiting for the obligatory cry for @propdel though :)

propdel: just say no.

> > I'd also like to change property so that the doc string defaults to
> > the doc string of the getter.
>
> I may be misunderstanding, but isn't that what patch #1434038 already
> implemented in 2.5?

D'oh, you're right. I tested this with 2.4 (which is the default
Python at Google). +1 on this proposal then. :-)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Oct 31 19:09:06 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 31 Oct 2007 11:09:06 -0700
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <20071031174852.9CAB33A40A4@sparrow.telecommunity.com>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
	<20071031174852.9CAB33A40A4@sparrow.telecommunity.com>
Message-ID: <ca471dc20710311109h4e54a3f2t4955b777073ea2e4@mail.gmail.com>

On 10/31/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:08 AM 10/31/2007 -0700, Guido van Rossum wrote:
> >I've come up with a relatively unobtrusive pattern for defining
> >setters. Given the following definition:
> >
> >def propset(prop):
> >     assert isinstance(prop, property)
> >     def helper(func):
> >         return property(prop.__get__, func, func, prop.__doc__)
> >     return helper
>
> Shouldn't that be property(prop.fget, func, prop.fdel, prop.__doc__),

Of course. (Though my code works too. :-)

> and have a matching propdel decorator?

Yes, though I think that if prop.fdel is None, we could use func in both slots.

> Apart from those extremely small nits, a big +1, and I expect to
> swipe this idea more or less immediately.  :)

Be my guest. I should patent this; it's apparently non-obvious. :-) (I
can't recall why I didn't think of this sooner, and why nobody else
did.)

> >I'd also like to change property so that the doc string defaults to
> >the doc string of the getter.
>
> +1 also; I thought it did this already, but am now disappointed to
> find it doesn't.  :)

Patch for 2.6 anyone?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Oct 31 20:01:01 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 31 Oct 2007 15:01:01 -0400
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <ca471dc20710311109h4e54a3f2t4955b777073ea2e4@mail.gmail.co
 m>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
	<20071031174852.9CAB33A40A4@sparrow.telecommunity.com>
	<ca471dc20710311109h4e54a3f2t4955b777073ea2e4@mail.gmail.com>
Message-ID: <20071031185816.2A9AA3A405B@sparrow.telecommunity.com>

At 11:09 AM 10/31/2007 -0700, Guido van Rossum wrote:
>Yes, though I think that if prop.fdel is None, we could use func in 
>both slots.

Eh?  Isn't prop.fdel supposed to be a single-argument function?  ISTM 
that C is the only place where fset and fdel are the same thing.


From arkanes at gmail.com  Wed Oct 31 20:31:37 2007
From: arkanes at gmail.com (Chris Mellon)
Date: Wed, 31 Oct 2007 14:31:37 -0500
Subject: [Python-Dev] Special file "nul" in Windows and os.stat
In-Reply-To: <47279DBA.1090400@v.loewis.de>
References: <e04bdf310710241323re978801h1b9dd05a157d09fa@mail.gmail.com>
	<4720161F.7040306@v.loewis.de>
	<4866bea60710301156k6a4ac17as7dcdcb4949ca4c08@mail.gmail.com>
	<47279DBA.1090400@v.loewis.de>
Message-ID: <4866bea60710311231k59ae883dy6ff984e8b9e7ab35@mail.gmail.com>

On Oct 30, 2007 4:10 PM, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> >> My question now is what specific kernel32 functions Python 2.4
> >> calls to determine that NUL is a file; before that question
> >> is sufficiently answered, I don't think any action should be
> >> taken.
> >>
> >
> > os.path.exist() in win32 just calls os.stat() and decides it doesn't
> > exist if an error is returned. os.stat() uses the vcrt stat()in 2.4,
> > but 2.5 implements it directly in terms of win32 api to deal with
> > limitations in the vcrt implementation.
>
> That doesn't really answer the question, though - you merely state
> that Python 2.4 calls the CRT, but then my question is still what
> kernel32 functions are called to have stat on NUL succeed.
>

I'm not 100% (it calls it through a function pointer and I'm not sure
I tracked it down correctly), but I think it calls it through the C
stat() function. In other words, it doesn't use any kernel32 functions
directly, it calls the stat() that's exported from the MSVCRT.

> > Interestingly, plain old GetFileAttributes() works, and returns
> > FILE_ATTRIBUTE_ARCHIVE for them.
>
> What about the other attributes (like modification time, size, etc)?
>

GetFileAttributes() doesn't return those, just the FAT filesystem
attributes. GetFileSize and GetFileTime fail.

> Regards,
> Martin
>

From lists at cheimes.de  Wed Oct 31 20:33:22 2007
From: lists at cheimes.de (Christian Heimes)
Date: Wed, 31 Oct 2007 20:33:22 +0100
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
Message-ID: <4728D882.7030805@cheimes.de>

Guido van Rossum wrote:
> I'd like to make this a standard built-in, in the hope the debate on
> how to declare settable properties.

+0.75

It's still not as good as the C# way to create a property but it's so
simple and obvious that it's ingenious again. Every other implementation
would either require support from the parser or some metaclass magic.
Your KISS approach rocks! :)

Christian


From theller at ctypes.org  Wed Oct 31 21:24:32 2007
From: theller at ctypes.org (Thomas Heller)
Date: Wed, 31 Oct 2007 21:24:32 +0100
Subject: [Python-Dev] ctypes and libffi
Message-ID: <fgaoa0$15s$1@ger.gmane.org>

The ctypes sources contain the source code for libffi, in Modules/_ctypes/libffi.
These sources were pulled from GCC cvs some time ago, and a new configure system
was written by Perky iirc.

Now, it seems that these sources are showing their age and a newer libffi version
should be used instead.

There are issues on certain platforms like mips64, arm, armeabi; also on other systems
to correctly use the c_longdouble data type that I added recently.

See also http://bugs.python.org/issue1292.

Pythons configure command has a --with-system-ffi option that, if specified, uses
a system installed libffi package (if present).  By default, the bundled sources are
used.

The cleanest way to fix this problem probably would be to remove the outdated libffi
sources from Python svn (in trunk and py3k), and require a system installed libffi
for building ctypes.  However, I have no idea if there are libffi packages available
for systems like solaris, BSD, or OS X (to name a few that come to mind).

Another way would be to leave the sources in place, and only use them when no
system libffi installation is present - in other words to change the default
of the --with-system-ffi option.

Matthias Klose has already committed a change to configure.in that sets
--with-system-ffi="yes" for Linux/arm* systems.  This would be a third way:
do this for all systems that require newer libffi.

Any thoughts?

Thanks,
Thomas

PS: As a test, I changed setup.py (in branches/ctypes-branch) so that always
a system installed libffi is used, and manually triggered the build on the
bots that failed or even crashed before: ppc Debian unstable, and S-390 Debian.

PPS:  The above does not apply for Windows; the windows libffi suorces work fine
and are in different subdirectories.  Also, configure is not used on Windows.


From guido at python.org  Wed Oct 31 21:28:37 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 31 Oct 2007 13:28:37 -0700
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <20071031185816.2A9AA3A405B@sparrow.telecommunity.com>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
	<20071031174852.9CAB33A40A4@sparrow.telecommunity.com>
	<ca471dc20710311109h4e54a3f2t4955b777073ea2e4@mail.gmail.com>
	<20071031185816.2A9AA3A405B@sparrow.telecommunity.com>
Message-ID: <ca471dc20710311328k69af3ee0qef565ebeb6e366ca@mail.gmail.com>

On 10/31/07, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 11:09 AM 10/31/2007 -0700, Guido van Rossum wrote:
> >Yes, though I think that if prop.fdel is None, we could use func in
> >both slots.
>
> Eh?  Isn't prop.fdel supposed to be a single-argument function?  ISTM
> that C is the only place where fset and fdel are the same thing.

Given how rarely supporting deletions matters enough to write extra
code, we can just say that *when using @propset* your setter function
needs to have a default value for the argument or otherwise support
being called with one or two arguments.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Wed Oct 31 23:48:43 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 01 Nov 2007 11:48:43 +1300
Subject: [Python-Dev] Declaring setters with getters
In-Reply-To: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
References: <ca471dc20710311008s77963218s51adbf6cb9f4c5c7@mail.gmail.com>
Message-ID: <4729064B.80802@canterbury.ac.nz>

Guido van Rossum wrote:
>     @property
>     def encoding(self):
>         return self._encoding
> 
>     @propset(encoding)
>     def encoding(self, value=None):
>         if value is not None:
>             unicode("0", value)  # Test it
>         self._encoding = value

That's reasonably nice, although I'm not sure about the
name "propset" -- it sounds like a repair kit for an
aircraft. Maybe something like "setproperty" would be
better.

But there's something that bothers me about the whole
propery mechanism -- there's no straightforward way for
a property accessor to call the inherited version from
a base class.

Wrapping property accessors up in decorators makes this
worse. At least when the accessors are declared as separate
functions you have a chance of grabbing an inherited one
and calling it. But if they're buried inside property
descriptors with no other reference to them, it's a lot
harder to get hold of one.

--
Greg

From mike.klaas at gmail.com  Wed Oct 31 22:36:25 2007
From: mike.klaas at gmail.com (Mike Klaas)
Date: Wed, 31 Oct 2007 14:36:25 -0700
Subject: [Python-Dev] Request for inclusion in 2.5.2 (5-for-1)
Message-ID: <19C5D86A-8534-4DB7-934C-CEE94003BB79@gmail.com>

Issue http://bugs.python.org/issue1663329 details an annoyance in the  
subprocess module that has affected several users, including me.   
Essentially, closing hundreds of thousands of file descriptors by  
round-tripping through the python exception machinery is very slow,  
taking hundreds of milliseconds and at times many seconds.  The  
proposed fix is to write this loop in c.  The c function is but a  
handful of lines long.  I purposefully kept the implementation  
trivial so that it will work on all unix variants (there is another  
issue that contains a super-duper optimization for AIX, and other  
possibilities exist for Solaris, but the simple fix yields a ten-fold  
speedup everywhere but windows, so I didn't think that it was worth  
the complexity).

Though technically relating only to performance, I consider this a  
bug-fix candidate as mysterious multi-second delays when launching a  
subprocess end up making the functionality of close_fds unusable on  
some platform configurations (namely, those with high MAX_FD set).

It would be great to see this is 2.5.2.  Understanding that issue  
evaluation takes significant effort, I've done some evaluation/triage  
on other open tickets:
----------------------------------------
See issues for detailed comments.

http://bugs.python.org/issue1516330:  No clear problem, invalid  
patch.  Recommend rejection.

http://bugs.python.org/issue1516327:  No clear problem, no patch.  
Recommend closing.

http://bugs.python.org/issue1705170:  reproduced.  Conjecture as to  
why it is occurring, but I don't know the guts well enough to propose  
a decent fix.

http://bugs.python.org/issue1773632: tested patch.  Recommend  
accepting unless there are things I don't know about this mysterious  
_xmlrpclib extension (which there doubtlessly are)

http://bugs.python.org/issue738948: Rather old PEP that has gathered  
no comments.  Calling it a "PEP" is generous--it is really just a  
link to an academic paper with a note about how this might be  
integrated into Stackless.

Thanks,
-Mike