From guido at python.org  Sat Dec  1 00:05:18 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 30 Nov 2007 15:05:18 -0800
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
References: <474D8725.8000706@cheimes.de> <fik6h2$q3$1@ger.gmane.org>
	<474DF68D.6010701@canterbury.ac.nz> <474E9999.5090401@cheimes.de>
	<474E9D45.8030509@cs.byu.edu> <474EA42F.70309@cheimes.de>
	<6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
Message-ID: <ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>

On Nov 30, 2007 2:17 PM, Nicko van Someren <nicko at nicko.org> wrote:
> +1 for __universal__

It's almost as if nobody has seen my proposal to leave __builtins__
alone and rename the __builtin__ module instead.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From cmason at vcentertainment.com  Sat Dec  1 00:09:10 2007
From: cmason at vcentertainment.com (Chuck Mason (Visual Concepts))
Date: Fri, 30 Nov 2007 15:09:10 -0800
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
Message-ID: <8D590944234B4047B01E464FACB35A3516B802@2KGNOVEXG01.2kgames.t2.corp>

Thing is that "universal" is an adjective and we tend to use nouns
(maybe not by intention) for our modules/objects:

Sys, os, builtins, etc, are all nouns: maybe +1 for __universe__ ?  But
when you phrase it that way, it doesn't quite make sense.

Have we considered special syntax for universal py methods? *.open() ?
::open() (ew!, by the way) ?  

I'm a huge fan of keeping things legible to people who don't know
python, but for some weird reason __universal__ makes me feel like I am
writing non-professional software.

C


-----Original Message-----
From: python-dev-bounces+cmason=vcentertainment.com at python.org
[mailto:python-dev-bounces+cmason=vcentertainment.com at python.org] On
Behalf Of Nicko van Someren
Sent: Friday, November 30, 2007 2:17 PM
To: Isaac Morland
Cc: python-dev at python.org
Subject: Re: [Python-Dev] [poll] New name for __builtins__

On 29 Nov 2007, at 14:06, Isaac Morland wrote:
>
> I wonder how much you could sell the naming rights for?  i.e. call it
> __[name of sponsor]__.  Python's pretty popular, such advertising  
> should
> be worth something....

I'm sorry, but if you call it __Microsoft_Office_2007__ I shall never  
write a program that uses what we now call __builtins__ ever again!

> PS: I actually do like __universal__.

Me too.

+1 for __universal__

	Nicko

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/cmason%40vcentertainme
nt.com

From fdrake at acm.org  Sat Dec  1 00:16:10 2007
From: fdrake at acm.org (Fred Drake)
Date: Fri, 30 Nov 2007 18:16:10 -0500
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
References: <474D8725.8000706@cheimes.de> <fik6h2$q3$1@ger.gmane.org>
	<474DF68D.6010701@canterbury.ac.nz>
	<474E9999.5090401@cheimes.de> <474E9D45.8030509@cs.byu.edu>
	<474EA42F.70309@cheimes.de> <6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
Message-ID: <B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>

On Nov 30, 2007, at 6:05 PM, Guido van Rossum wrote:
> It's almost as if nobody has seen my proposal to leave __builtins__
> alone and rename the __builtin__ module instead.


I suspect that's indistinguishable from everyone being tired of the  
discussion, knowing that you're going to pick something reasonable in  
spite of our yammering.

+1 for a module named "builtin", or something similarly obscure.


   -Fred

-- 
Fred Drake   <fdrake at acm.org>





From barry at python.org  Sat Dec  1 00:18:46 2007
From: barry at python.org (Barry Warsaw)
Date: Fri, 30 Nov 2007 18:18:46 -0500
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
References: <474D8725.8000706@cheimes.de> <fik6h2$q3$1@ger.gmane.org>
	<474DF68D.6010701@canterbury.ac.nz>
	<474E9999.5090401@cheimes.de> <474E9D45.8030509@cs.byu.edu>
	<474EA42F.70309@cheimes.de> <6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
Message-ID: <9A00CAC8-DB1F-4308-9EE0-EBD487DA6EEC@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Nov 30, 2007, at 6:05 PM, Guido van Rossum wrote:

> On Nov 30, 2007 2:17 PM, Nicko van Someren <nicko at nicko.org> wrote:
>> +1 for __universal__
>
> It's almost as if nobody has seen my proposal to leave __builtins__
> alone and rename the __builtin__ module instead.

Hi Guido,

I saw it, and I like that change much better!  There's no real need  
to underscorify the module name.

- -Barry

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)

iQCVAwUBR1CaVnEjvBPtnXfVAQJk6QP/Z02maduXygFtbObY8LhdwW7sZbkvEkuW
Xc0OM11xa6zPV0gAuCgrNgLS7AXJ43FgEyvmbHmQMeicu7V2Ft3xyM7riTz0AqE6
EU22PR1wbPJBWxMFB+D8ufsKAgTtYdLloqDSHrDhMpIKhETA9qUeYZOL+VM2BFaN
rkaOuCo5S+4=
=wG9Q
-----END PGP SIGNATURE-----

From phd at phd.pp.ru  Sat Dec  1 00:40:36 2007
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Sat, 1 Dec 2007 02:40:36 +0300
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
References: <fik6h2$q3$1@ger.gmane.org> <474DF68D.6010701@canterbury.ac.nz>
	<474E9999.5090401@cheimes.de> <474E9D45.8030509@cs.byu.edu>
	<474EA42F.70309@cheimes.de> <6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
Message-ID: <20071130234036.GA20061@phd.pp.ru>

On Fri, Nov 30, 2007 at 03:05:18PM -0800, Guido van Rossum wrote:
> On Nov 30, 2007 2:17 PM, Nicko van Someren <nicko at nicko.org> wrote:
> > +1 for __universal__
> 
> It's almost as if nobody has seen my proposal to leave __builtins__
> alone and rename the __builtin__ module instead.

   I saw it, and I think it'd be the best.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From steven.bethard at gmail.com  Sat Dec  1 00:41:35 2007
From: steven.bethard at gmail.com (Steven Bethard)
Date: Fri, 30 Nov 2007 16:41:35 -0700
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <20071130234036.GA20061@phd.pp.ru>
References: <fik6h2$q3$1@ger.gmane.org> <474E9999.5090401@cheimes.de>
	<474E9D45.8030509@cs.byu.edu> <474EA42F.70309@cheimes.de>
	<6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
	<20071130234036.GA20061@phd.pp.ru>
Message-ID: <d11dcfba0711301541u105ff9ceh330e46149c8c9a71@mail.gmail.com>

On Nov 30, 2007 4:40 PM, Oleg Broytmann <phd at phd.pp.ru> wrote:
> On Fri, Nov 30, 2007 at 03:05:18PM -0800, Guido van Rossum wrote:
> > On Nov 30, 2007 2:17 PM, Nicko van Someren <nicko at nicko.org> wrote:
> > > +1 for __universal__
> >
> > It's almost as if nobody has seen my proposal to leave __builtins__
> > alone and rename the __builtin__ module instead.
>
>    I saw it, and I think it'd be the best.

I'd like to jump on the __builtin__ -> builtin bandwagon too.  ;-)

Steve
-- 
I'm not *in*-sane. Indeed, I am so far *out* of sane that you appear a
tiny blip on the distant coast of sanity.
        --- Bucky Katt, Get Fuzzy

From ntoronto at cs.byu.edu  Sat Dec  1 00:42:47 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 30 Nov 2007 16:42:47 -0700
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
References: <474D8725.8000706@cheimes.de>
	<fik6h2$q3$1@ger.gmane.org>	<474DF68D.6010701@canterbury.ac.nz>	<474E9999.5090401@cheimes.de>
	<474E9D45.8030509@cs.byu.edu>	<474EA42F.70309@cheimes.de>
	<6framq$5ce6af@earth.karoo.kcom.com>	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
Message-ID: <47509FF7.2060803@cs.byu.edu>

Fred Drake wrote:
> On Nov 30, 2007, at 6:05 PM, Guido van Rossum wrote:
>> It's almost as if nobody has seen my proposal to leave __builtins__
>> alone and rename the __builtin__ module instead.
> 
> 
> I suspect that's indistinguishable from everyone being tired of the  
> discussion, knowing that you're going to pick something reasonable in  
> spite of our yammering.
> 
> +1 for a module named "builtin", or something similarly obscure.

Would it be too confusing to call it "builtins"?

Neil

From guido at python.org  Sat Dec  1 00:48:27 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 30 Nov 2007 15:48:27 -0800
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <47509FF7.2060803@cs.byu.edu>
References: <474D8725.8000706@cheimes.de> <474E9D45.8030509@cs.byu.edu>
	<474EA42F.70309@cheimes.de> <6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
	<47509FF7.2060803@cs.byu.edu>
Message-ID: <ca471dc20711301548x647a330fle6acb0f26652776f@mail.gmail.com>

> > On Nov 30, 2007, at 6:05 PM, Guido van Rossum wrote:
> >> It's almost as if nobody has seen my proposal to leave __builtins__
> >> alone and rename the __builtin__ module instead.

> Fred Drake wrote:
> > +1 for a module named "builtin", or something similarly obscure.

On Nov 30, 2007 3:42 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> Would it be too confusing to call it "builtins"?

To the contrary, I like that best (and want to note for the record
that that's what I proposed when I first brought it up here :-).

If someone wants to prepare a patch, be my guest!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Sat Dec  1 00:59:08 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sat, 01 Dec 2007 12:59:08 +1300
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <finh1j$tng$1@ger.gmane.org>
References: <474D8725.8000706@cheimes.de> <fik6h2$q3$1@ger.gmane.org>
	<474DF68D.6010701@canterbury.ac.nz> <474E9999.5090401@cheimes.de>
	<474E9D45.8030509@cs.byu.edu> <474EA42F.70309@cheimes.de>
	<474EBBBE.2010808@gmail.com>
	<20071129161543.A98703A405E@sparrow.telecommunity.com>
	<ca471dc20711290954v173cf2ebx9427de211ba9afbd@mail.gmail.com>
	<finh1j$tng$1@ger.gmane.org>
Message-ID: <4750A3CC.2000207@canterbury.ac.nz>

Terry Reedy wrote:
> The only problem would be if someone put 
> the incantation into a non-main module named 'main.py', but the same is 
> true today of '__main__.py'.  And I would consider either a buggy practice.

I often put the "real" main code into a separate module, so
that it gets compiled to a .pyc, and sometimes I call that
module "main.py". So this change would break a lot of my
programs, and perhaps other people's as well.

I think the situation with __main__ is different from __builtin__,
because __builtin__ is an actual, specific module, whereas
__main__ is a pseudo-module which stands in for a different thing
in every Python program. So I'm in favour of keeping an
__xxx__ name for it, to emphasise its magic nature. I'm much
less likely to accidentally stomp on a magic name than a
non-magic one.

--
Greg

From guido at python.org  Sat Dec  1 01:21:56 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 30 Nov 2007 16:21:56 -0800
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <4750A3CC.2000207@canterbury.ac.nz>
References: <474D8725.8000706@cheimes.de> <474DF68D.6010701@canterbury.ac.nz>
	<474E9999.5090401@cheimes.de> <474E9D45.8030509@cs.byu.edu>
	<474EA42F.70309@cheimes.de> <474EBBBE.2010808@gmail.com>
	<20071129161543.A98703A405E@sparrow.telecommunity.com>
	<ca471dc20711290954v173cf2ebx9427de211ba9afbd@mail.gmail.com>
	<finh1j$tng$1@ger.gmane.org> <4750A3CC.2000207@canterbury.ac.nz>
Message-ID: <ca471dc20711301621v8af094ft37377a65ccd1d60d@mail.gmail.com>

On Nov 30, 2007 3:59 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Terry Reedy wrote:
> > The only problem would be if someone put
> > the incantation into a non-main module named 'main.py', but the same is
> > true today of '__main__.py'.  And I would consider either a buggy practice.
>
> I often put the "real" main code into a separate module, so
> that it gets compiled to a .pyc, and sometimes I call that
> module "main.py". So this change would break a lot of my
> programs, and perhaps other people's as well.
>
> I think the situation with __main__ is different from __builtin__,
> because __builtin__ is an actual, specific module, whereas
> __main__ is a pseudo-module which stands in for a different thing
> in every Python program. So I'm in favour of keeping an
> __xxx__ name for it, to emphasise its magic nature. I'm much
> less likely to accidentally stomp on a magic name than a
> non-magic one.

Right. Rest assured, __main__ is not going to change.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Sat Dec  1 04:16:20 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 30 Nov 2007 19:16:20 -0800
Subject: [Python-Dev] -O2 faster than -O3?
In-Reply-To: <47506C3F.7040709@cs.byu.edu>
References: <47506C3F.7040709@cs.byu.edu>
Message-ID: <bbaeab100711301916i6129f349x79a3489bcf2cebf3@mail.gmail.com>

On Nov 30, 2007 12:02 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> On both of my systems, using -O2 reduces execution time in pystone by 9%
> and in pybench by 8%. It's function inlining: "-O3
> -fno-inline-functions" works just as well as "-O2". Removing "-g" has
> little effect on the result.
>
> Systems:
>   - AMD Athlon 64 X2 Dual Core 4600+, 512 KB cache (desktop)
>   - Intel T2300 Dual Core 1.66GHz, 512 KB cache (laptop)
>
> Both are Ubuntu 7.04, GCC 4.1.2.
>
> Does anybody else see this?
>
> It may be GCC being stupid (which has happened before) or not enough
> cache on my systems (definitely possible). If it's not one of those, I'd
> say it's because CPython core functions are already very large, and
> almost everything that ought to be inlined is already a macro.
>

That's quite possible.  Previous benchmarks by AMK have shown that
perhaps -0m (or whatever the flag is to optimize for size) sometimes
is the best solution.  It has always been believed that the eval loop
is already large and manages to hit some cache sweet spot.

-Brett

From brett at python.org  Sat Dec  1 04:18:42 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 30 Nov 2007 19:18:42 -0800
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
References: <474D8725.8000706@cheimes.de> <474E9999.5090401@cheimes.de>
	<474E9D45.8030509@cs.byu.edu> <474EA42F.70309@cheimes.de>
	<6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
Message-ID: <bbaeab100711301918i48e5951ak266fbda4c8f42a56@mail.gmail.com>

On Nov 30, 2007 3:16 PM, Fred Drake <fdrake at acm.org> wrote:
> On Nov 30, 2007, at 6:05 PM, Guido van Rossum wrote:
> > It's almost as if nobody has seen my proposal to leave __builtins__
> > alone and rename the __builtin__ module instead.
>
>
> I suspect that's indistinguishable from everyone being tired of the
> discussion, knowing that you're going to pick something reasonable in
> spite of our yammering.
>

Yeah.  I figured your rambling tolerance had already been hit on this
and you were just going to pick something.

> +1 for a module named "builtin", or something similarly obscure.

+1 for your (Guido's) 'builtins' suggestion so that there is symmetry
with __builtins__ and 'builtins'.

-Brett



>
>
>    -Fred
>
> --
> Fred Drake   <fdrake at acm.org>
>
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From pje at telecommunity.com  Sat Dec  1 02:16:11 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 30 Nov 2007 20:16:11 -0500
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
References: <474D8725.8000706@cheimes.de> <fik6h2$q3$1@ger.gmane.org>
	<474DF68D.6010701@canterbury.ac.nz> <474E9999.5090401@cheimes.de>
	<474E9D45.8030509@cs.byu.edu> <474EA42F.70309@cheimes.de>
	<6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
Message-ID: <20071201040048.BFC7A3A408C@sparrow.telecommunity.com>

At 06:16 PM 11/30/2007 -0500, Fred Drake wrote:
>On Nov 30, 2007, at 6:05 PM, Guido van Rossum wrote:
> > It's almost as if nobody has seen my proposal to leave __builtins__
> > alone and rename the __builtin__ module instead.
>
>
>I suspect that's indistinguishable from everyone being tired of the
>discussion, knowing that you're going to pick something reasonable in
>spite of our yammering.
>
>+1 for a module named "builtin", or something similarly obscure.

What he said - but "builtins", i.e. plural.


From nnorwitz at gmail.com  Sat Dec  1 06:32:29 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 30 Nov 2007 21:32:29 -0800
Subject: [Python-Dev] -O2 faster than -O3?
In-Reply-To: <bbaeab100711301916i6129f349x79a3489bcf2cebf3@mail.gmail.com>
References: <47506C3F.7040709@cs.byu.edu>
	<bbaeab100711301916i6129f349x79a3489bcf2cebf3@mail.gmail.com>
Message-ID: <ee2a432c0711302132k720b636et3ae21e348af5dda5@mail.gmail.com>

On Nov 30, 2007 7:16 PM, Brett Cannon <brett at python.org> wrote:
> On Nov 30, 2007 12:02 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> > On both of my systems, using -O2 reduces execution time in pystone by 9%
> > and in pybench by 8%. It's function inlining: "-O3
> > -fno-inline-functions" works just as well as "-O2". Removing "-g" has
> > little effect on the result.
> >
> > Systems:
> >   - AMD Athlon 64 X2 Dual Core 4600+, 512 KB cache (desktop)
> >   - Intel T2300 Dual Core 1.66GHz, 512 KB cache (laptop)
> >
> > Both are Ubuntu 7.04, GCC 4.1.2.
> >
> > Does anybody else see this?
> >
> > It may be GCC being stupid (which has happened before) or not enough
> > cache on my systems (definitely possible). If it's not one of those, I'd
> > say it's because CPython core functions are already very large, and
> > almost everything that ought to be inlined is already a macro.
> >
>
> That's quite possible.  Previous benchmarks by AMK have shown that
> perhaps -0m (or whatever the flag is to optimize for size) sometimes
> is the best solution.  It has always been believed that the eval loop
> is already large and manages to hit some cache sweet spot.

The flag is -Os.  I suspect you will do better to limit the size of
inlining rather disabling it completely.  The option is
-finline-limit=number.  I don't know the default value or what you
should try.  I would be interested to hear more results though.

n

From tjreedy at udel.edu  Sat Dec  1 07:05:23 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 1 Dec 2007 01:05:23 -0500
Subject: [Python-Dev] [poll] New name for __builtins__
References: <474D8725.8000706@cheimes.de>
	<fik6h2$q3$1@ger.gmane.org><474DF68D.6010701@canterbury.ac.nz>
	<474E9999.5090401@cheimes.de><474E9D45.8030509@cs.byu.edu>
	<474EA42F.70309@cheimes.de><474EBBBE.2010808@gmail.com><20071129161543.A98703A405E@sparrow.telecommunity.com><ca471dc20711290954v173cf2ebx9427de211ba9afbd@mail.gmail.com><finh1j$tng$1@ger.gmane.org>
	<4750A3CC.2000207@canterbury.ac.nz>
Message-ID: <fiqtj2$f3e$1@ger.gmane.org>


"Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote in message 
news:4750A3CC.2000207 at canterbury.ac.nz...
| I think the situation with __main__ is different from __builtin__,

I effectively agreed by not disputing Guido's response ;-) 




From ntoronto at cs.byu.edu  Sat Dec  1 07:35:09 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 30 Nov 2007 23:35:09 -0700
Subject: [Python-Dev] -O2 faster than -O3?
In-Reply-To: <ee2a432c0711302132k720b636et3ae21e348af5dda5@mail.gmail.com>
References: <47506C3F.7040709@cs.byu.edu>	
	<bbaeab100711301916i6129f349x79a3489bcf2cebf3@mail.gmail.com>
	<ee2a432c0711302132k720b636et3ae21e348af5dda5@mail.gmail.com>
Message-ID: <4751009D.1070408@cs.byu.edu>

Neal Norwitz wrote:
> On Nov 30, 2007 7:16 PM, Brett Cannon <brett at python.org> wrote:
>> On Nov 30, 2007 12:02 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
>>> On both of my systems, using -O2 reduces execution time in pystone by 9%
>>> and in pybench by 8%. It's function inlining: "-O3
>>> -fno-inline-functions" works just as well as "-O2". Removing "-g" has
>>> little effect on the result.
>>>
>>> Systems:
>>>   - AMD Athlon 64 X2 Dual Core 4600+, 512 KB cache (desktop)
>>>   - Intel T2300 Dual Core 1.66GHz, 512 KB cache (laptop)
>>>
>>> Both are Ubuntu 7.04, GCC 4.1.2.
>>>
>>> Does anybody else see this?
>>>
>>> It may be GCC being stupid (which has happened before) or not enough
>>> cache on my systems (definitely possible). If it's not one of those, I'd
>>> say it's because CPython core functions are already very large, and
>>> almost everything that ought to be inlined is already a macro.
>>>
>> That's quite possible.  Previous benchmarks by AMK have shown that
>> perhaps -0m (or whatever the flag is to optimize for size) sometimes
>> is the best solution.  It has always been believed that the eval loop
>> is already large and manages to hit some cache sweet spot.
> 
> The flag is -Os.  I suspect you will do better to limit the size of
> inlining rather disabling it completely.  The option is
> -finline-limit=number.  I don't know the default value or what you
> should try.  I would be interested to hear more results though.

I've got some pystones (500000) results for the Athlon. The default for 
-finline-limit is 600. This is for the current trunk.

Global options                    pystones/sec (median of 3)
--------------                    ------------
-O3                                  50454.1
-O2                                  57273.8
-Os                                  52798.3
-O3 -fno-inline-functions            54824.6
-O3 -finline-limit=300               51229.7
-O3 -finline-limit=150               51177.7
-O3 -finline-limit=75                51759.8
-O3 -finline-limit=25                53821.3

ceval.c options (-O3 for others)  pystones/sec (median of 3)
---------------                   ------------
-O2                                  55066.1
-Os                                  57012.5
-O3 -fno-inline-functions            55679.3
-O3 -finline-limit=300               51440.3
-O3 -finline-limit=150               50916.5
-O3 -finline-limit=75                51387.5
-O3 -finline-limit=25                52631.6

Now that's interesting. -O2 seems to be the best global option, and -Os 
seems to be best for ceval.c. One more test then:

Global -O2, ceval.c -Os              56753.7

Weird.

If you're going to run these benchmarks yourself, make sure you "make 
clean" before building with different options. (I don't know why it's 
necessary, but it is.) To change options for just ceval.c, add this to 
Makefile.pre.in under "Special rules":

Python/ceval.o: $(srcdir)/Python/ceval.c
         $(CC) -c $(PY_CFLAGS) -Os \
         -o $@ $(srcdir)/Python/ceval.c

The last -O flag should override any other.

Neil

From ntoronto at cs.byu.edu  Sat Dec  1 07:39:39 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Fri, 30 Nov 2007 23:39:39 -0700
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <fiqtj2$f3e$1@ger.gmane.org>
References: <474D8725.8000706@cheimes.de>	<fik6h2$q3$1@ger.gmane.org><474DF68D.6010701@canterbury.ac.nz>	<474E9999.5090401@cheimes.de><474E9D45.8030509@cs.byu.edu>	<474EA42F.70309@cheimes.de><474EBBBE.2010808@gmail.com><20071129161543.A98703A405E@sparrow.telecommunity.com><ca471dc20711290954v173cf2ebx9427de211ba9afbd@mail.gmail.com><finh1j$tng$1@ger.gmane.org>	<4750A3CC.2000207@canterbury.ac.nz>
	<fiqtj2$f3e$1@ger.gmane.org>
Message-ID: <475101AB.1090806@cs.byu.edu>

Terry Reedy wrote:
> "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote in message 
> news:4750A3CC.2000207 at canterbury.ac.nz...
> | I think the situation with __main__ is different from __builtin__,
> 
> I effectively agreed by not disputing Guido's response ;-) 

Very cunning. But I was even more cunning, and didn't even *consider* 
disputing Guido's response.

Neil


From steve at holdenweb.com  Sat Dec  1 07:50:19 2007
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 01 Dec 2007 01:50:19 -0500
Subject: [Python-Dev] -O2 faster than -O3?
In-Reply-To: <4751009D.1070408@cs.byu.edu>
References: <47506C3F.7040709@cs.byu.edu>		<bbaeab100711301916i6129f349x79a3489bcf2cebf3@mail.gmail.com>	<ee2a432c0711302132k720b636et3ae21e348af5dda5@mail.gmail.com>
	<4751009D.1070408@cs.byu.edu>
Message-ID: <fir078$k7a$1@ger.gmane.org>

Neil Toronto wrote:
[...]
> If you're going to run these benchmarks yourself, make sure you "make 
> clean" before building with different options. (I don't know why it's 
> necessary, but it is.) 

Because the dependencies evaluated by "make" don't take into account the 
different options that were used to compile existing object files.

Without the "make clean" it says "aha, here's a nice up-to-date object 
file, I'll use that" and you don't get a freshly-compiled version.

Touching the sources should also work, and is a little quicker.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Sat Dec  1 07:51:49 2007
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 01 Dec 2007 01:51:49 -0500
Subject: [Python-Dev] [Fwd: Re: -O2 faster than -O3?]
Message-ID: <47510485.4080807@holdenweb.com>

[Sorry, the send key pressed itself there]

Touching the sources should also work, and is a little quicker (but this 
is usually only practical for small projects).

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/

From g.brandl at gmx.net  Sat Dec  1 14:28:36 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 01 Dec 2007 14:28:36 +0100
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <ca471dc20711301548x647a330fle6acb0f26652776f@mail.gmail.com>
References: <474D8725.8000706@cheimes.de>
	<474E9D45.8030509@cs.byu.edu>	<474EA42F.70309@cheimes.de>
	<6framq$5ce6af@earth.karoo.kcom.com>	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>	<47509FF7.2060803@cs.byu.edu>
	<ca471dc20711301548x647a330fle6acb0f26652776f@mail.gmail.com>
Message-ID: <firni7$am5$1@ger.gmane.org>

Guido van Rossum schrieb:
>> > On Nov 30, 2007, at 6:05 PM, Guido van Rossum wrote:
>> >> It's almost as if nobody has seen my proposal to leave __builtins__
>> >> alone and rename the __builtin__ module instead.
> 
>> Fred Drake wrote:
>> > +1 for a module named "builtin", or something similarly obscure.
> 
> On Nov 30, 2007 3:42 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
>> Would it be too confusing to call it "builtins"?
> 
> To the contrary, I like that best (and want to note for the record
> that that's what I proposed when I first brought it up here :-).
> 
> If someone wants to prepare a patch, be my guest!

Done, see #1535.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From lists at cheimes.de  Sat Dec  1 15:00:52 2007
From: lists at cheimes.de (Christian Heimes)
Date: Sat, 01 Dec 2007 15:00:52 +0100
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <firni7$am5$1@ger.gmane.org>
References: <474D8725.8000706@cheimes.de>	<474E9D45.8030509@cs.byu.edu>	<474EA42F.70309@cheimes.de>	<6framq$5ce6af@earth.karoo.kcom.com>	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>	<47509FF7.2060803@cs.byu.edu>	<ca471dc20711301548x647a330fle6acb0f26652776f@mail.gmail.com>
	<firni7$am5$1@ger.gmane.org>
Message-ID: <47516914.108@cheimes.de>

Georg Brandl wrote:
> Done, see #1535.

I've written a 2to3 fixer, see #1535.

Christian

From chad at imvu.com  Sat Dec  1 23:38:23 2007
From: chad at imvu.com (Chad Austin)
Date: Sat, 01 Dec 2007 14:38:23 -0800
Subject: [Python-Dev] Problems with GeneratorExit deriving from Exception
Message-ID: <4751E25F.9020107@imvu.com>

Hello Python-Dev,

Here at IMVU, we love Python 2.5's generators-as-coroutines.  That feature has
let us succinctly express algorithms that intermix asynchronous network requests
and UI operations without writing complicated state machines, and, perhaps most
importantly, get high-quality unit tests around these algorithms.

However, we've been having a problem with the way GeneratorExit interacts with
our coroutine system.  Let's take a bit of simplified example code from our system:

	@task
	def pollForChatInvites(chatGateway, userId):
		while True:
			try:
				# Network call.
				result = yield chatGateway.checkForInvite({'userId': userId})
			except Exception:  # An XML-RPC call can fail for many reasons.
				result = None
			# ... handle result here
			yield Sleep(10)

If a task (coroutine) is cancelled while it's waiting for the result from
checkForInvite, a GeneratorExit will be raised from that point in the generator,
which will be caught and ignored by the "except Exception:" clause, causing a
RuntimeError to be raised from whoever tried to close the generator.  Moreover,
any finally: clauses or with-statement contexts don't run.

We have also run into problems where a task tries to "return" (yield Return())
from within a try: except Exception: block.  Since returning from a coroutine is
roughly equivalent to "raise GeneratorExit", the exception can be caught and
ignored, with the same consequences as above.

This problem could be solved in several ways:

1) Make GeneratorExit derive from BaseException, just like SystemExit.

2) Add "except GeneratorExit: raise" to every usage of except Exception: in tasks.

3) Change the semantics of except clauses so that you can use any container as
an exception filter.  You could have a custom exception filter object that would
catch any Exception-derived exception except for GeneratorExit.  Then we'd have
to teach the team to use "except ImvuExceptionFilter:" rather than "except
Exception:".

I prefer option #1, because it matches SystemExit and I haven't ever seen a case
where I wanted to catch GeneratorExit.  When a generator is closed, I just want
finally: clauses to run, like a normal return statement would.  In fact, we have
already implemented option #1 locally, but would like it to be standard.

Option #2 would add needless noise throughout the system,

You could argue that it's bad style to catch Exception, but there are many
situations where that's exactly what I want.  I don't actually care _how_ the
xml-rpc call failed, just that the error is logged and the call is retried
later.  Same with loading caches from disk.

Proposals for changing GeneratorExit to be a BaseException have come up on this
list in the past [1] [2], but were rejected as being too "theoretical".  A
significant portion of the IMVU client is now specified with coroutines, so I
hope to resume this conversation.

Thoughts?

Chad

[1] http://mail.python.org/pipermail/python-dev/2006-March/062654.html
[2] http://mail.python.org/pipermail/python-dev/2006-March/062825.html

-- 
http://imvu.com/technology



From ntoronto at cs.byu.edu  Sun Dec  2 04:09:22 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Sat, 01 Dec 2007 20:09:22 -0700
Subject: [Python-Dev] Non-string keys in namespace dicts
Message-ID: <475221E2.4010800@cs.byu.edu>

Are there any use-cases for allowing namespace dicts (such as globals, 
builtins and classes) to have non-string keys? I'm asking because I'm 
planning on accelerating method lookups next, and the possibility of a 
key compare changing the underlying dict could be a major pain. (It was 
a minor pain for globals.)

Neil

From guido at python.org  Sun Dec  2 05:45:23 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 1 Dec 2007 20:45:23 -0800
Subject: [Python-Dev] Problems with GeneratorExit deriving from Exception
In-Reply-To: <4751E25F.9020107@imvu.com>
References: <4751E25F.9020107@imvu.com>
Message-ID: <ca471dc20712012045u608735bdh7744e7762f733316@mail.gmail.com>

On Dec 1, 2007 2:38 PM, Chad Austin <chad at imvu.com> wrote:
> Here at IMVU, we love Python 2.5's generators-as-coroutines.  That feature has
> let us succinctly express algorithms that intermix asynchronous network requests
> and UI operations without writing complicated state machines, and, perhaps most
> importantly, get high-quality unit tests around these algorithms.
>
> However, we've been having a problem with the way GeneratorExit interacts with
> our coroutine system.  Let's take a bit of simplified example code from our system:
>
>         @task
>         def pollForChatInvites(chatGateway, userId):
>                 while True:
>                         try:
>                                 # Network call.
>                                 result = yield chatGateway.checkForInvite({'userId': userId})
>                         except Exception:  # An XML-RPC call can fail for many reasons.
>                                 result = None
>                         # ... handle result here
>                         yield Sleep(10)
>
> If a task (coroutine) is cancelled while it's waiting for the result from
> checkForInvite, a GeneratorExit will be raised from that point in the generator,
> which will be caught and ignored by the "except Exception:" clause, causing a
> RuntimeError to be raised from whoever tried to close the generator.  Moreover,
> any finally: clauses or with-statement contexts don't run.
>
> We have also run into problems where a task tries to "return" (yield Return())
> from within a try: except Exception: block.  Since returning from a coroutine is
> roughly equivalent to "raise GeneratorExit", the exception can be caught and
> ignored, with the same consequences as above.
>
> This problem could be solved in several ways:
>
> 1) Make GeneratorExit derive from BaseException, just like SystemExit.
>
> 2) Add "except GeneratorExit: raise" to every usage of except Exception: in tasks.
>
> 3) Change the semantics of except clauses so that you can use any container as
> an exception filter.  You could have a custom exception filter object that would
> catch any Exception-derived exception except for GeneratorExit.  Then we'd have
> to teach the team to use "except ImvuExceptionFilter:" rather than "except
> Exception:".
>
> I prefer option #1, because it matches SystemExit and I haven't ever seen a case
> where I wanted to catch GeneratorExit.  When a generator is closed, I just want
> finally: clauses to run, like a normal return statement would.  In fact, we have
> already implemented option #1 locally, but would like it to be standard.
>
> Option #2 would add needless noise throughout the system,
>
> You could argue that it's bad style to catch Exception, but there are many
> situations where that's exactly what I want.  I don't actually care _how_ the
> xml-rpc call failed, just that the error is logged and the call is retried
> later.  Same with loading caches from disk.
>
> Proposals for changing GeneratorExit to be a BaseException have come up on this
> list in the past [1] [2], but were rejected as being too "theoretical".  A
> significant portion of the IMVU client is now specified with coroutines, so I
> hope to resume this conversation.
>
> Thoughts?
>
> Chad
>
> [1] http://mail.python.org/pipermail/python-dev/2006-March/062654.html
> [2] http://mail.python.org/pipermail/python-dev/2006-March/062825.html

Well argued. I suggest to go for option (1) -- make GeneratorExit
inherit from BaseException. We can do this starting 2.6. Feel free to
upload a patch to bugs.python.org.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Dec  2 05:52:52 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 1 Dec 2007 20:52:52 -0800
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <475221E2.4010800@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>
Message-ID: <ca471dc20712012052g662a1901hfa571d72e057262f@mail.gmail.com>

On Dec 1, 2007 7:09 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> Are there any use-cases for allowing namespace dicts (such as globals,
> builtins and classes) to have non-string keys? I'm asking because I'm
> planning on accelerating method lookups next, and the possibility of a
> key compare changing the underlying dict could be a major pain. (It was
> a minor pain for globals.)

Since this has always worked, I suspect there are probably some
packages that depend on this. We could deprecate this however in 2.6
and forbid it in 3.0; it's easy enough to switch to string keys,
either using a unique prefix or even non-identifier characters like
'.' or '$'.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Sun Dec  2 07:02:17 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 2 Dec 2007 01:02:17 -0500
Subject: [Python-Dev] Non-string keys in namespace dicts
References: <475221E2.4010800@cs.byu.edu>
	<ca471dc20712012052g662a1901hfa571d72e057262f@mail.gmail.com>
Message-ID: <fithp5$pil$1@ger.gmane.org>


"Guido van Rossum" <guido at python.org> wrote in message 
news:ca471dc20712012052g662a1901hfa571d72e057262f at mail.gmail.com...
| On Dec 1, 2007 7:09 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
| > Are there any use-cases for allowing namespace dicts (such as globals,
| > builtins and classes) to have non-string keys? I'm asking because I'm
| > planning on accelerating method lookups next, and the possibility of a
| > key compare changing the underlying dict could be a major pain. (It was
| > a minor pain for globals.)
|
| Since this has always worked, I suspect there are probably some
| packages that depend on this.

My impression from a decade of reading c.l.p. is that most people posting 
there regard the possibility, when bypassing the normal identifier access 
mechanisms, of putting non-identifiers, let alone non-string keys into 
namespace dicts, as

1. Surprising (because they assumed a string- or name-only dict was used).
or
2. A side effect (implementation dependent) of Python economizing on types. 
(But with the result of spurring optimization of dicts, which in turn 
reduced the motivation for a specialized type [until now].)

In any case, it seemed pretty useless, since naming and using a dict was/is 
both easier and safer (more future proof).

But there may have been a person or two who liked having semi-hidden 
variables or attributes.

|We could deprecate this however in 2.6
| and forbid it in 3.0; it's easy enough to switch to string keys,
| either using a unique prefix or even non-identifier characters like
| '.' or '$'.

Or use a separate, unrestricted, regular dict.

+1

tjr





From chad at imvu.com  Sun Dec  2 08:14:03 2007
From: chad at imvu.com (Chad Austin)
Date: Sat, 01 Dec 2007 23:14:03 -0800
Subject: [Python-Dev] Problems with GeneratorExit deriving from Exception
In-Reply-To: <ca471dc20712012045u608735bdh7744e7762f733316@mail.gmail.com>
References: <4751E25F.9020107@imvu.com>
	<ca471dc20712012045u608735bdh7744e7762f733316@mail.gmail.com>
Message-ID: <47525B3B.3080700@imvu.com>

Guido van Rossum wrote:
> On Dec 1, 2007 2:38 PM, Chad Austin <chad at imvu.com> wrote:
>> This problem could be solved in several ways:
>>
>> 1) Make GeneratorExit derive from BaseException, just like SystemExit.
> 
> Well argued. I suggest to go for option (1) -- make GeneratorExit
> inherit from BaseException. We can do this starting 2.6. Feel free to
> upload a patch to bugs.python.org.

Great!  Patch is uploaded at http://bugs.python.org/issue1537

The patch changes the definition of GeneratorExit so that it extends 
BaseException, adds a generator test, updates exception_hierarchy.txt, and 
updates the exceptions page in the documentation.  This is my first patch to 
Python -- did I miss anything?

Thanks,
Chad

From solipsis at pitrou.net  Sun Dec  2 14:56:13 2007
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 02 Dec 2007 14:56:13 +0100
Subject: [Python-Dev]  Problems with GeneratorExit deriving from Exception
Message-ID: <1196603773.7448.13.camel@fsol>


Hi,

(I was asked to forward this from the bug tracker)

> We have also run into problems where a task tries to "return" (yield Return())
> from within a try: except Exception: block.  Since returning from a coroutine is
> roughly equivalent to "raise GeneratorExit", the exception can be caught and
> ignored, with the same consequences as above.

I may be missing something but why don't you just catch StandardError
instead? If I believe Python 2.5's exception hierarchy it would catch
anything under Exception except for GeneratorExit, StopIteration and the
various Warnings.

Also it seems to me that muting any Exception is bad practice since it
can lead you to overlook errors in your code. It's better to catch a
specific Exception subclass, like IOError (or XMLRPCError, or whatever
it is called).


Antoine.



From ncoghlan at gmail.com  Sun Dec  2 16:21:39 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 03 Dec 2007 01:21:39 +1000
Subject: [Python-Dev] Problems with GeneratorExit deriving from Exception
In-Reply-To: <ca471dc20712012045u608735bdh7744e7762f733316@mail.gmail.com>
References: <4751E25F.9020107@imvu.com>
	<ca471dc20712012045u608735bdh7744e7762f733316@mail.gmail.com>
Message-ID: <4752CD83.3080408@gmail.com>

Guido van Rossum wrote:
> Well argued. I suggest to go for option (1) -- make GeneratorExit
> inherit from BaseException. We can do this starting 2.6. Feel free to
> upload a patch to bugs.python.org.

It actually took me a while to figure out why this use case was 
convincing, when the same idea had been rejected for 2.5.

For anyone else as slow as me: in the hypothetical examples I posted 
before the release of 2.5, the yield could be moved to an else clause on 
the try-except statement without adversely affecting the semantics.

The use case Chad presented here is different, because the exceptions to 
be handled are being passed back in via the yield expression - moving it 
would defeat the whole purpose of the exception handling. I'm sure the 
fact that the example comes from a real application rather than the 
'what-if' generator in my brain helps a lot too :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sun Dec  2 16:27:03 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 03 Dec 2007 01:27:03 +1000
Subject: [Python-Dev] Update to PEP 366 (Relative imports from the main
 module)
In-Reply-To: <ca471dc20711301238x71af1c8bnc09cc1ffb7bad121@mail.gmail.com>
References: <47459613.7060301@gmail.com>
	<ca471dc20711301238x71af1c8bnc09cc1ffb7bad121@mail.gmail.com>
Message-ID: <4752CEC7.8020501@gmail.com>

Guido van Rossum wrote:
> This looks good. Please make the appropriate changes to the PEP and to
> PEP 0 to mark it as accepted.

I should get to that in the next day or two. Thanks.

> I think the implementation is fine too (others will have to check it
> more carefully) but I noticed that the promised functionality of -m
> doesn't work yet

It turns out one of the code paths through runpy wasn't getting tested 
properly, and naturally it was the path that the -m switch relies on.

I've posted a new patch which both fixes that code path and adds 
additional tests to make sure it is doing the right thing.

Cheers,
Nick.

PEP 366 implementation patch
(http://bugs.python.org/issue1487)

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sun Dec  2 16:40:24 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 03 Dec 2007 01:40:24 +1000
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
References: <474D8725.8000706@cheimes.de>
	<fik6h2$q3$1@ger.gmane.org>	<474DF68D.6010701@canterbury.ac.nz>	<474E9999.5090401@cheimes.de>
	<474E9D45.8030509@cs.byu.edu>	<474EA42F.70309@cheimes.de>
	<6framq$5ce6af@earth.karoo.kcom.com>	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
Message-ID: <4752D1E8.90301@gmail.com>

Fred Drake wrote:
> I suspect that's indistinguishable from everyone being tired of the  
> discussion, knowing that you're going to pick something reasonable in  
> spite of our yammering.

What Fred said. It's Guido's bikeshed, he can choose the colour :)

Just for the record, I also like the idea of __builtins__ being a magic 
alias for the boringly-but-practically named builtins module.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sun Dec  2 16:43:37 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 03 Dec 2007 01:43:37 +1000
Subject: [Python-Dev] Decimal news: speedup and stabilization
In-Reply-To: <e04bdf310711300918m770c06ddr67d970859dd179b9@mail.gmail.com>
References: <e04bdf310711231019l27951163j4da2cb65290620bb@mail.gmail.com>	
	<4747AEEF.2010104@gmail.com>
	<e04bdf310711300918m770c06ddr67d970859dd179b9@mail.gmail.com>
Message-ID: <4752D2A9.7040601@gmail.com>

Facundo Batista wrote:
> 2007/11/24, Nick Coghlan <ncoghlan at gmail.com>:
> 
>> Did you change the Decimal repr to use the same format for the mantissa?
> 
> I don't understand the question. The output of repr() does not show
> this internals...

Yeah, um... can we just forget I asked that question? (I blame lack of 
sleep, or coffee, or something...)

>> Could you also check the performance gain against the telco benchmark
>> which is in the sandbox? [1]
> 
> I tested different versions of Decimal with this telco.py.
> 
> With Python from the trunk (r58550), which is the last decimal.py
> version previous to this change: 1241.8
> 
> After changed it to keep _int as string: 869.3 (30% faster!)
> 
> But still there're a lot of room for improvements. I just commited
> other patch from Mark, where he reordered __new__, to minimize
> isinstance() checks for the most used types.}
> 
> After new reordering patch: 672.9 (22% faster!)

Great news!

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From g.brandl at gmx.net  Sun Dec  2 19:43:01 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 02 Dec 2007 19:43:01 +0100
Subject: [Python-Dev] Problems with GeneratorExit deriving from Exception
In-Reply-To: <1196603773.7448.13.camel@fsol>
References: <1196603773.7448.13.camel@fsol>
Message-ID: <fiuuc3$fj2$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Hi,
> 
> (I was asked to forward this from the bug tracker)
> 
>> We have also run into problems where a task tries to "return" (yield Return())
>> from within a try: except Exception: block.  Since returning from a coroutine is
>> roughly equivalent to "raise GeneratorExit", the exception can be caught and
>> ignored, with the same consequences as above.
> 
> I may be missing something but why don't you just catch StandardError
> instead? If I believe Python 2.5's exception hierarchy it would catch
> anything under Exception except for GeneratorExit, StopIteration and the
> various Warnings.

Problem is, many (most?) third-party modules derive their exceptions from
Exception, not StandardError, so you'd have to add special cases for them
too.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From nicko at nicko.org  Sun Dec  2 20:06:29 2007
From: nicko at nicko.org (Nicko van Someren)
Date: Sun, 2 Dec 2007 19:06:29 +0000
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <475221E2.4010800@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>
Message-ID: <BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>

On 2 Dec 2007, at 03:09, Neil Toronto wrote:

> Are there any use-cases for allowing namespace dicts (such as globals,
> builtins and classes) to have non-string keys? I'm asking because I'm
> planning on accelerating method lookups next, and the possibility of a
> key compare changing the underlying dict could be a major pain. (It  
> was
> a minor pain for globals.)

The only plausible use case I can think of might be wanting to use  
ints or longs as keys, though I've never seen it done.  Of course this  
would be trivial to code around and it seems very much a fringe case,  
so I'd be in favour of deprecating non-string namespace keys if it's  
going to make look-ups go faster.

	Cheers,
		Nicko


From chad at imvu.com  Sun Dec  2 20:29:37 2007
From: chad at imvu.com (Chad Austin)
Date: Sun, 02 Dec 2007 11:29:37 -0800
Subject: [Python-Dev] Problems with GeneratorExit deriving from Exception
In-Reply-To: <1196603773.7448.13.camel@fsol>
References: <1196603773.7448.13.camel@fsol>
Message-ID: <475307A1.1090200@imvu.com>

Hi Antoine,

Antoine Pitrou wrote:
> Hi,
> 
> (I was asked to forward this from the bug tracker)
> 
>> We have also run into problems where a task tries to "return" (yield Return())
>> from within a try: except Exception: block.  Since returning from a coroutine is
>> roughly equivalent to "raise GeneratorExit", the exception can be caught and
>> ignored, with the same consequences as above.
> 
> I may be missing something but why don't you just catch StandardError
> instead? If I believe Python 2.5's exception hierarchy it would catch
> anything under Exception except for GeneratorExit, StopIteration and the
> various Warnings.

If socket.error, xmlrpclib.Fault, httplib.HTTPException, etc. all extended 
StandardError, then we would probably be fine with that approach.  But I think 
the majority of exceptions, both in the standard library and our code, extend 
Exception directly.

> Also it seems to me that muting any Exception is bad practice since it
> can lead you to overlook errors in your code. It's better to catch a
> specific Exception subclass, like IOError (or XMLRPCError, or whatever
> it is called).

Yes, in general, it's better to catch specific errors, but sometimes it really 
is the case that it doesn't matter why the call failed, as long as your unit and 
acceptance tests verify that the code behaves as expected and you log any 
exceptions that do occur.  In fact, our logger remembers the last N error or 
exception log entries and automatically sends them back to our servers for 
analysis.  So think of it as protecting the application from intermittent 
failures rather than silently dropping exceptions.  :)

Chad

From guido at python.org  Sun Dec  2 21:07:15 2007
From: guido at python.org (Guido van Rossum)
Date: Sun, 2 Dec 2007 12:07:15 -0800
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <4752D1E8.90301@gmail.com>
References: <474D8725.8000706@cheimes.de> <474E9D45.8030509@cs.byu.edu>
	<474EA42F.70309@cheimes.de> <6framq$5ce6af@earth.karoo.kcom.com>
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>
	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>
	<4752D1E8.90301@gmail.com>
Message-ID: <ca471dc20712021207n40a79822u106f351b194711a5@mail.gmail.com>

On Dec 2, 2007 7:40 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Just for the record, I also like the idea of __builtins__ being a magic
> alias for the boringly-but-practically named builtins module.

[Imagine me jumping up and down and screaming at the top of my lungs
out of frustration:]

BUT THAT'S NOT WHAT IT IS! IT'S A HOOK FOR SANDBOXING! YOU SHOULD
NEVER BE USING __builtins__ DIRECTLY EXCEPT WHEN CONTROLLING THE SET
OF BUILTINS AVAILABLE TO UNTRUSTED CODE!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nnorwitz at gmail.com  Sun Dec  2 21:08:40 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 2 Dec 2007 12:08:40 -0800
Subject: [Python-Dev] Problems with GeneratorExit deriving from Exception
In-Reply-To: <475307A1.1090200@imvu.com>
References: <1196603773.7448.13.camel@fsol> <475307A1.1090200@imvu.com>
Message-ID: <ee2a432c0712021208v19b39df7t40dc40b443881c@mail.gmail.com>

On Dec 2, 2007 11:29 AM, Chad Austin <chad at imvu.com> wrote:
> Hi Antoine,
>
> Antoine Pitrou wrote:
> > Hi,
> >
> > (I was asked to forward this from the bug tracker)
> >
> >> We have also run into problems where a task tries to "return" (yield Return())
> >> from within a try: except Exception: block.  Since returning from a coroutine is
> >> roughly equivalent to "raise GeneratorExit", the exception can be caught and
> >> ignored, with the same consequences as above.
> >
> > I may be missing something but why don't you just catch StandardError
> > instead? If I believe Python 2.5's exception hierarchy it would catch
> > anything under Exception except for GeneratorExit, StopIteration and the
> > various Warnings.
>
> If socket.error, xmlrpclib.Fault, httplib.HTTPException, etc. all extended
> StandardError, then we would probably be fine with that approach.  But I think
> the majority of exceptions, both in the standard library and our code, extend
> Exception directly.

Note that StandardError was removed from py3k.

n

From g.brandl at gmx.net  Sun Dec  2 21:17:18 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 02 Dec 2007 21:17:18 +0100
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <ca471dc20712021207n40a79822u106f351b194711a5@mail.gmail.com>
References: <474D8725.8000706@cheimes.de>
	<474E9D45.8030509@cs.byu.edu>	<474EA42F.70309@cheimes.de>
	<6framq$5ce6af@earth.karoo.kcom.com>	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>	<4752D1E8.90301@gmail.com>
	<ca471dc20712021207n40a79822u106f351b194711a5@mail.gmail.com>
Message-ID: <fiv3sf$1tp$1@ger.gmane.org>

Guido van Rossum schrieb:
> On Dec 2, 2007 7:40 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Just for the record, I also like the idea of __builtins__ being a magic
>> alias for the boringly-but-practically named builtins module.
> 
> [Imagine me jumping up and down and screaming at the top of my lungs
> out of frustration:]
> 
> BUT THAT'S NOT WHAT IT IS! IT'S A HOOK FOR SANDBOXING! YOU SHOULD
> NEVER BE USING __builtins__ DIRECTLY EXCEPT WHEN CONTROLLING THE SET
> OF BUILTINS AVAILABLE TO UNTRUSTED CODE!

You've scared me.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From janssen at parc.com  Sun Dec  2 21:23:01 2007
From: janssen at parc.com (Bill Janssen)
Date: Sun, 2 Dec 2007 12:23:01 PST
Subject: [Python-Dev] blocking a non-blocking socket
Message-ID: <07Dec2.122308pst."58696"@synergy1.parc.xerox.com>

An interesting question has come up in the development of the SSL module.

The function ssl.wrap_socket() takes a flag, do_handshake_on_connect,
which tells it whether to do the SSL handshake before returning an
SSLSocket object to the caller.  If the socket being wrapped is
non-blocking, the code in wrap_socket() must invoke do_handshake()
multiple times, and effectively block until the handshake is done.

Right now, I'm doing it with this loop:

            if do_handshake_on_connect:
                # have to loop to support non-blocking sockets
                while True:
                    try:
                        self.do_handshake()
                        break
                    except SSLError, err:
                        if err.args[0] == SSL_ERROR_WANT_READ:
                            select.select([self], [], [])
                        elif err.args[0] == SSL_ERROR_WANT_WRITE:
                            select.select([], [self], [])
                        else:
                            raise

But this seems fragile to me.  "select" and/or "poll" is awfully
system-dependent.  What I'd like to do is just use the socket API,
something like:

    blocking = self.getblocking()
    try:
        self.setblocking(1)
	self.do_handshake()
    finally:
        self.setblocking(blocking)

But there's no "getblocking" method on sockets.  Instead, there's
"gettimeout".  So I'd write something like
	
    timeout = self.gettimeout()
    try:
        self.setblocking(1)
	self.do_handshake()
    finally:
        if (timeout == 0.0):
	    self.setblocking(0)

But my mother taught me never to test for equality against
floating-point zero.  But in this case it might just fly...

Or, should I just set the timeout:

    timeout = self.gettimeout()
    try:
        self.settimeout(None)
	self.do_handshake()
    finally:
        self.settimeout(timeout)

This is the solution I'm leaning towards...

Any recommendations?

Bill

From phd at phd.pp.ru  Sun Dec  2 21:31:44 2007
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Sun, 2 Dec 2007 23:31:44 +0300
Subject: [Python-Dev] blocking a non-blocking socket
In-Reply-To: <07Dec2.122308pst."58696"@synergy1.parc.xerox.com>
References: <07Dec2.122308pst."58696"@synergy1.parc.xerox.com>
Message-ID: <20071202203144.GA19899@phd.pp.ru>

On Sun, Dec 02, 2007 at 12:23:01PM -0800, Bill Janssen wrote:
[skip]
> Or, should I just set the timeout:
> 
>     timeout = self.gettimeout()
>     try:
>         self.settimeout(None)
> 	self.do_handshake()
>     finally:
>         self.settimeout(timeout)

   Yes, this is the correct solution for all cases: if the timeout is None
(socket is blocking) or 0 (non-blocking) or not-0 (blocking with timeout)
- just set it back.

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From solipsis at pitrou.net  Sun Dec  2 21:39:50 2007
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 02 Dec 2007 21:39:50 +0100
Subject: [Python-Dev] StandardError removed from py3k
In-Reply-To: <ee2a432c0712021208v19b39df7t40dc40b443881c@mail.gmail.com>
References: <1196603773.7448.13.camel@fsol> <475307A1.1090200@imvu.com>
	<ee2a432c0712021208v19b39df7t40dc40b443881c@mail.gmail.com>
Message-ID: <1196627990.7448.36.camel@fsol>


Le dimanche 02 d?cembre 2007 ? 12:08 -0800, Neal Norwitz a ?crit :
> Note that StandardError was removed from py3k.

Out of curiosity, what is the reason for this? Another exception tree
rearrangement?




From solipsis at pitrou.net  Sun Dec  2 21:41:43 2007
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 02 Dec 2007 21:41:43 +0100
Subject: [Python-Dev] Problems with GeneratorExit deriving from	Exception
In-Reply-To: <475307A1.1090200@imvu.com>
References: <1196603773.7448.13.camel@fsol>  <475307A1.1090200@imvu.com>
Message-ID: <1196628103.7448.40.camel@fsol>


Le dimanche 02 d?cembre 2007 ? 11:29 -0800, Chad Austin a ?crit :
> If socket.error, xmlrpclib.Fault, httplib.HTTPException, etc. all extended 
> StandardError, then we would probably be fine with that approach.  But I think 
> the majority of exceptions, both in the standard library and our code, extend 
> Exception directly.

Ok, understood. :)

To me it's a shame people don't try to make their exceptions inherit
from a proper base class (be it IOError, ValueError, etc.), but
anyway...

cheers

Antoine.



From ntoronto at cs.byu.edu  Sun Dec  2 21:49:35 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Sun, 02 Dec 2007 13:49:35 -0700
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
Message-ID: <47531A5F.5020308@cs.byu.edu>

Nicko van Someren wrote:
> On 2 Dec 2007, at 03:09, Neil Toronto wrote:
> 
>> Are there any use-cases for allowing namespace dicts (such as globals,
>> builtins and classes) to have non-string keys? I'm asking because I'm
>> planning on accelerating method lookups next, and the possibility of a
>> key compare changing the underlying dict could be a major pain. (It was
>> a minor pain for globals.)
> 
> The only plausible use case I can think of might be wanting to use ints 
> or longs as keys, though I've never seen it done.  Of course this would 
> be trivial to code around and it seems very much a fringe case, so I'd 
> be in favour of deprecating non-string namespace keys if it's going to 
> make look-ups go faster.

If you insert non-string keys into a namespace dict it'll slow down 
lookups already. :) The dict will switch to the more general lookdict 
from lookdict_string. Looks like it's just a bad idea all around.

It turned out not *that* hard to code around for attribute caching, and 
the extra cruft only gets invoked on a cache miss. The biggest problem 
isn't speed - it's that it's possible (though extremely unlikely), while 
testing keys for equality, that a rich compare alters the underlying 
dict. This causes the caching lookup to have to try to get an entry 
pointer again, which could invoke the rich compare, which might alter 
the underlying dict...

This problem already exists for general dicts with non-string keys (it 
can blow the C stack) and attribute caching makes it a bit more likely 
(the compare only has to insert or delete an item rather than cause a 
resize), so it'd be nice if it didn't apply to identifiers.

As far as I know, though, the only way to get non-string keys into a 
class dict is by using a metaclass.

Anyway, report: I've got an initial working attribute cache, using the 
conveniently-named-and-left-NULL tp_cache. It's a nice speedup - except 
on everything the standard benchmarks test, because their class 
hierarchies are very shallow. :p  If an MRO has more than two classes in 
it, every kind of lookup (class method, object method, object attribute) 
is faster. Having more than four or five makes things like self.variable 
take less than half the time.

It'd be nice to have a benchmark with a deep class hierarchy. Does 
anybody know of one?

I'm working on making it as fast as the original when the MRO is short. 
Question for Guido: should I roll this into the fastglobals patch?

Neil

From aahz at pythoncraft.com  Sun Dec  2 22:01:09 2007
From: aahz at pythoncraft.com (Aahz)
Date: Sun, 2 Dec 2007 13:01:09 -0800
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <47531A5F.5020308@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
Message-ID: <20071202210109.GA7926@panix.com>

On Sun, Dec 02, 2007, Neil Toronto wrote:
>
> Anyway, report: I've got an initial working attribute cache, using the 
> conveniently-named-and-left-NULL tp_cache. It's a nice speedup - except 
> on everything the standard benchmarks test, because their class 
> hierarchies are very shallow. :p  If an MRO has more than two classes in 
> it, every kind of lookup (class method, object method, object attribute) 
> is faster. Having more than four or five makes things like self.variable 
> take less than half the time.

This patch is probably a non-starter, then: that's what killed the
CACHE_ATTR patch several years ago (I was sprinting on that with Guido
and Ping):

http://mail.python.org/pipermail/python-dev/2007-June/073604.html
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith

From g.brandl at gmx.net  Sun Dec  2 22:49:08 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 02 Dec 2007 22:49:08 +0100
Subject: [Python-Dev] StandardError removed from py3k
In-Reply-To: <1196627990.7448.36.camel@fsol>
References: <1196603773.7448.13.camel@fsol>
	<475307A1.1090200@imvu.com>	<ee2a432c0712021208v19b39df7t40dc40b443881c@mail.gmail.com>
	<1196627990.7448.36.camel@fsol>
Message-ID: <fiv98k$h8h$1@ger.gmane.org>

Antoine Pitrou schrieb:
> Le dimanche 02 d?cembre 2007 ? 12:08 -0800, Neal Norwitz a ?crit :
>> Note that StandardError was removed from py3k.
> 
> Out of curiosity, what is the reason for this? Another exception tree
> rearrangement?

I think it was found not to serve a real purpose.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From brett at python.org  Sun Dec  2 22:56:46 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 2 Dec 2007 13:56:46 -0800
Subject: [Python-Dev] Problems with GeneratorExit deriving from Exception
In-Reply-To: <47525B3B.3080700@imvu.com>
References: <4751E25F.9020107@imvu.com>
	<ca471dc20712012045u608735bdh7744e7762f733316@mail.gmail.com>
	<47525B3B.3080700@imvu.com>
Message-ID: <bbaeab100712021356q68e42b01hfa63c187987cafd@mail.gmail.com>

On Dec 1, 2007 11:14 PM, Chad Austin <chad at imvu.com> wrote:
> Guido van Rossum wrote:
> > On Dec 1, 2007 2:38 PM, Chad Austin <chad at imvu.com> wrote:
> >> This problem could be solved in several ways:
> >>
> >> 1) Make GeneratorExit derive from BaseException, just like SystemExit.
> >
> > Well argued. I suggest to go for option (1) -- make GeneratorExit
> > inherit from BaseException. We can do this starting 2.6. Feel free to
> > upload a patch to bugs.python.org.
>
> Great!  Patch is uploaded at http://bugs.python.org/issue1537
>
> The patch changes the definition of GeneratorExit so that it extends
> BaseException, adds a generator test, updates exception_hierarchy.txt, and
> updates the exceptions page in the documentation.  This is my first patch to
> Python -- did I miss anything?

I have not looked at the patch, so take what I say with a grain of salt.  =)

First, a generator test is not necessary.  The patch changes the
inheritance of exceptions, nothing more.  While its usefulness is
manifested for generators, this is really an exception detail.  And
changing exception_hierarchy.txt gives you the exception test you
need.

Second, the docs will need to be changed.  I know that
Doc/library/exceptions.rst needs a tweak.  Not sure if anywhere else
does.

-Brett

From brett at python.org  Sun Dec  2 23:00:09 2007
From: brett at python.org (Brett Cannon)
Date: Sun, 2 Dec 2007 14:00:09 -0800
Subject: [Python-Dev] StandardError removed from py3k
In-Reply-To: <fiv98k$h8h$1@ger.gmane.org>
References: <1196603773.7448.13.camel@fsol> <475307A1.1090200@imvu.com>
	<ee2a432c0712021208v19b39df7t40dc40b443881c@mail.gmail.com>
	<1196627990.7448.36.camel@fsol> <fiv98k$h8h$1@ger.gmane.org>
Message-ID: <bbaeab100712021400q1be95991rd0d06f8ee4f9f310@mail.gmail.com>

On Dec 2, 2007 1:49 PM, Georg Brandl <g.brandl at gmx.net> wrote:
> Antoine Pitrou schrieb:
> > Le dimanche 02 d?cembre 2007 ? 12:08 -0800, Neal Norwitz a ?crit :
> >> Note that StandardError was removed from py3k.
> >
> > Out of curiosity, what is the reason for this? Another exception tree
> > rearrangement?
>
> I think it was found not to serve a real purpose.
>

That's right.  Since we introduced BaseException StandardException was
no longer needed.

-Brett



> Georg
>
> --
> Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
> Four shall be the number of spaces thou shalt indent, and the number of thy
> indenting shall be four. Eight shalt thou not indent, nor either indent thou
> two, excepting that thou then proceed to four. Tabs are right out.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From lists at cheimes.de  Sun Dec  2 23:29:37 2007
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 02 Dec 2007 23:29:37 +0100
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <47531A5F.5020308@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
Message-ID: <475331D1.9090501@cheimes.de>

Neil Toronto wrote:
> It'd be nice to have a benchmark with a deep class hierarchy. Does 
> anybody know of one?

Zope has some very complex and deep class hierarchies, especially when
it comes down to Plone and Archetypes. Unfortunately Zope is still stuck
with Python 2.4.

Christian

From greg.ewing at canterbury.ac.nz  Mon Dec  3 01:10:44 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 03 Dec 2007 13:10:44 +1300
Subject: [Python-Dev] blocking a non-blocking socket
In-Reply-To: <07Dec2.122308pst.58696@synergy1.parc.xerox.com>
References: <07Dec2.122308pst.58696@synergy1.parc.xerox.com>
Message-ID: <47534984.3070403@canterbury.ac.nz>

Bill Janssen wrote:
> What I'd like to do is just use the socket API,
> something like:
> 
>     blocking = self.getblocking()
>     try:
>         self.setblocking(1)
> 	self.do_handshake()
>     finally:
>         self.setblocking(blocking)

I'm not sure this is the right  approach. If the calling code
has made the socket non-blocking, then it doesn't want any
operations on it to block. Rather than temporarily
making it blocking by whatever means, some indication needs
to be returned that the operation would block, and a way
provided for the calling code to re-try later.

If that can't reasonably be done, then passing a non-blocking
socket here should be an error.

> But my mother taught me never to test for equality against
> floating-point zero.

That doesn't apply here. If a float is explicitly set to 0.0
you can reasonably expect it to test equal to 0.0. The caveat
only applies to results of a calculation, which may incorporate
roundoff errors.

--
Greg

From janssen at parc.com  Mon Dec  3 03:04:34 2007
From: janssen at parc.com (Bill Janssen)
Date: Sun, 2 Dec 2007 18:04:34 PST
Subject: [Python-Dev] blocking a non-blocking socket
In-Reply-To: <47534984.3070403@canterbury.ac.nz> 
References: <07Dec2.122308pst.58696@synergy1.parc.xerox.com>
	<47534984.3070403@canterbury.ac.nz>
Message-ID: <07Dec2.180444pst."58696"@synergy1.parc.xerox.com>

> Rather than temporarily
> making it blocking by whatever means, some indication needs
> to be returned that the operation would block, and a way
> provided for the calling code to re-try later.
> 
> If that can't reasonably be done, then passing a non-blocking
> socket here should be an error.

I tend to agree with you.  At this point, we're executing bad code,
because passing in a non-blocking socket and asking the routine to do
the handshake is self-contradictory.  Checking for this condition and
raising an exception would probably be best.  Other opinions.

> > But my mother taught me never to test for equality against
> > floating-point zero.
> 
> That doesn't apply here. If a float is explicitly set to 0.0
> you can reasonably expect it to test equal to 0.0. The caveat
> only applies to results of a calculation, which may incorporate
> roundoff errors.

Yep.  Sorry, meant to imply that with the next sentence.

Bill


From pje at telecommunity.com  Mon Dec  3 03:28:18 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 02 Dec 2007 21:28:18 -0500
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <475221E2.4010800@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>
Message-ID: <20071203023202.EBFB63A40D9@sparrow.telecommunity.com>

At 08:09 PM 12/1/2007 -0700, Neil Toronto wrote:
>Are there any use-cases for allowing namespace dicts (such as globals,
>builtins and classes) to have non-string keys?

Yes.  See http://pypi.python.org/pypi/AddOns


>  I'm asking because I'm
>planning on accelerating method lookups next, and the possibility of a
>key compare changing the underlying dict could be a major pain. (It was
>a minor pain for globals.)

For what it's worth, the AddOns package recommends the use of 
instances of built-in types (or tuples thereof) as add-on keys, so 
they would not have that problem in normal use.

I don't see a problem with requiring dictionary key comparisons to be 
side-effect-free - even in the general case of dictionaries, not just 
namespace ones.


From ncoghlan at gmail.com  Mon Dec  3 13:01:15 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 03 Dec 2007 22:01:15 +1000
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <ca471dc20712021207n40a79822u106f351b194711a5@mail.gmail.com>
References: <474D8725.8000706@cheimes.de> <474E9D45.8030509@cs.byu.edu>	
	<474EA42F.70309@cheimes.de> <6framq$5ce6af@earth.karoo.kcom.com>	
	<6fsfi8$5bl9rn@venus.eclipse.kcom.com>	
	<Pine.GSO.4.64.0711290853060.23564@cpu102.cs.uwaterloo.ca>	
	<C6BB1B56-EA41-47DB-95D2-CEB76361D67F@nicko.org>	
	<ca471dc20711301505h1174f6f5la07cd98d0dcfe7be@mail.gmail.com>	
	<B96CE3B2-7669-4814-88B2-BB6A75F4815E@acm.org>	
	<4752D1E8.90301@gmail.com>
	<ca471dc20712021207n40a79822u106f351b194711a5@mail.gmail.com>
Message-ID: <4753F00B.5050101@gmail.com>

Guido van Rossum wrote:
> On Dec 2, 2007 7:40 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Just for the record, I also like the idea of __builtins__ being a magic
>> alias for the boringly-but-practically named builtins module.
> 
> [Imagine me jumping up and down and screaming at the top of my lungs
> out of frustration:]
> 
> BUT THAT'S NOT WHAT IT IS! IT'S A HOOK FOR SANDBOXING! YOU SHOULD
> NEVER BE USING __builtins__ DIRECTLY EXCEPT WHEN CONTROLLING THE SET
> OF BUILTINS AVAILABLE TO UNTRUSTED CODE!
> 

I never mess with the builtin definitions under either name, but I agree 
that my description was highly inaccurate. It's not a topic I've spent 
much time considering :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From janssen at parc.com  Mon Dec  3 18:40:45 2007
From: janssen at parc.com (Bill Janssen)
Date: Mon, 3 Dec 2007 09:40:45 PST
Subject: [Python-Dev] blocking a non-blocking socket
In-Reply-To: <443039046E52EB4CABDCB50638B9A84C8D0465@cernxchg42.cern.ch> 
References: <07Dec2.122308pst."58696"@synergy1.parc.xerox.com>
	<443039046E52EB4CABDCB50638B9A84C8D0465@cernxchg42.cern.ch>
Message-ID: <07Dec3.094050pst."58696"@synergy1.parc.xerox.com>

Thanks, Audun.  If you look at the code, you'll see that both a
connect method and a do_handshake method already exist, and work
pretty much as you describe.  The issue is what to do when the user
doesn't use them -- specifies do_handshake_on_connect=True.

> Another way of doing it could be to expose a connect() method on the ssl
> objects.  It changes the socket.ssl api, but I'd say it is in the same
> spirit as the do_handshake_on_connect parameter since no existing code
> will break.  The caller then calls connect() until it does not return

Bill

From guido at python.org  Mon Dec  3 19:51:58 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 3 Dec 2007 10:51:58 -0800
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <20071203023202.EBFB63A40D9@sparrow.telecommunity.com>
References: <475221E2.4010800@cs.byu.edu>
	<20071203023202.EBFB63A40D9@sparrow.telecommunity.com>
Message-ID: <ca471dc20712031051y384b219bx9282a58459dfc443@mail.gmail.com>

On Dec 2, 2007 6:28 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> I don't see a problem with requiring dictionary key comparisons to be
> side-effect-free - even in the general case of dictionaries, not just
> namespace ones.

Me neither -- but the problem is enforcement.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec  3 19:53:04 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 3 Dec 2007 10:53:04 -0800
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <47531A5F.5020308@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
Message-ID: <ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>

On Dec 2, 2007 12:49 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> It turned out not *that* hard to code around for attribute caching, and
> the extra cruft only gets invoked on a cache miss. The biggest problem
> isn't speed - it's that it's possible (though extremely unlikely), while
> testing keys for equality, that a rich compare alters the underlying
> dict. This causes the caching lookup to have to try to get an entry
> pointer again, which could invoke the rich compare, which might alter
> the underlying dict..

How about subclasses of str? These have all the same issues...

> I'm working on making it as fast as the original when the MRO is short.
> Question for Guido: should I roll this into the fastglobals patch?

No, please keep them separate.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ntoronto at cs.byu.edu  Mon Dec  3 20:27:25 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 03 Dec 2007 12:27:25 -0700
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
References: <475221E2.4010800@cs.byu.edu>	
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>	
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
Message-ID: <4754589D.4020308@cs.byu.edu>

Guido van Rossum wrote:
> On Dec 2, 2007 12:49 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
>> It turned out not *that* hard to code around for attribute caching, and
>> the extra cruft only gets invoked on a cache miss. The biggest problem
>> isn't speed - it's that it's possible (though extremely unlikely), while
>> testing keys for equality, that a rich compare alters the underlying
>> dict. This causes the caching lookup to have to try to get an entry
>> pointer again, which could invoke the rich compare, which might alter
>> the underlying dict..
> 
> How about subclasses of str? These have all the same issues...

Yeah. I ended up having it, per class, permanently revert to uncached 
lookups when it detects that a class dict in the MRO has non-string 
keys. That's flagged by lookdict_string, which uses PyString_CheckExact.

Neil

From pje at telecommunity.com  Mon Dec  3 21:14:40 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 03 Dec 2007 15:14:40 -0500
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <4754589D.4020308@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
Message-ID: <20071203201438.75F083A40A5@sparrow.telecommunity.com>

At 12:27 PM 12/3/2007 -0700, Neil Toronto wrote:
>Guido van Rossum wrote:
> > On Dec 2, 2007 12:49 PM, Neil Toronto <ntoronto at cs.byu.edu> wrote:
> >> It turned out not *that* hard to code around for attribute caching, and
> >> the extra cruft only gets invoked on a cache miss. The biggest problem
> >> isn't speed - it's that it's possible (though extremely unlikely), while
> >> testing keys for equality, that a rich compare alters the underlying
> >> dict. This causes the caching lookup to have to try to get an entry
> >> pointer again, which could invoke the rich compare, which might alter
> >> the underlying dict..
> >
> > How about subclasses of str? These have all the same issues...
>
>Yeah. I ended up having it, per class, permanently revert to uncached
>lookups when it detects that a class dict in the MRO has non-string
>keys. That's flagged by lookdict_string, which uses PyString_CheckExact.

I'm a bit confused here.  Isn't the simplest way to cache attribute 
lookups to just have a cache dictionary in the type, and update that 
dictionary whenever a change is made to a superclass?  That's 
essentially how __slotted__ attribute changes on base classes work 
now, isn't it?  Why do we need to mess around with the dictionary 
entries themselves in order to do that?


From ntoronto at cs.byu.edu  Mon Dec  3 23:26:09 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 03 Dec 2007 15:26:09 -0700
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <20071203201438.75F083A40A5@sparrow.telecommunity.com>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
	<20071203201438.75F083A40A5@sparrow.telecommunity.com>
Message-ID: <47548281.3010404@cs.byu.edu>

Phillip J. Eby wrote:
> At 12:27 PM 12/3/2007 -0700, Neil Toronto wrote:
>> Guido van Rossum wrote:
>> > How about subclasses of str? These have all the same issues...
>>
>> Yeah. I ended up having it, per class, permanently revert to uncached
>> lookups when it detects that a class dict in the MRO has non-string
>> keys. That's flagged by lookdict_string, which uses PyString_CheckExact.
> 
> I'm a bit confused here.  Isn't the simplest way to cache attribute 
> lookups to just have a cache dictionary in the type, and update that 
> dictionary whenever a change is made to a superclass?  That's 
> essentially how __slotted__ attribute changes on base classes work now, 
> isn't it?  Why do we need to mess around with the dictionary entries 
> themselves in order to do that?

The nice thing about caching pointers to dict entries is that they don't 
change as often as values do. There are fewer ways to invalidate an 
entry pointer: inserting set, resize, clear, and delete. If you cache 
values, non-inserting set could invalidate as well.

Because inserting into namespace dicts should be very rare, caching 
entries rather than values should reduce the number of times cache 
entries are invalidated to near zero. Updating is expensive, so that's 
good for performance.

Rare updating also means it's okay to invalidate the entire cache rather 
than single entries, so the footprint of the caching mechanism in the 
dict can be very small. For example, I've got a single 64-bit counter in 
each dict that gets incremented after every potentially invalidating 
operation. That comes down to 8 bytes of storage and two extra machine 
instructions (currently) per invalidating operation. The cache checks it 
against its own counter, and updating ensures that it's synced.

Some version of the non-string keys problem would exist with any caching 
mechanism, though. An evil rich compare can always monkey about with 
class dicts in the MRO. If a caching scheme caches values and doesn't 
account for that, it could return stale values. If it caches entries and 
doesn't account for that, it could segfault. I suppose you could argue 
that returning stale values is fitting punishment for using an evil rich 
compare, though the punishee isn't always the same person as the punisher.

Neil


From pje at telecommunity.com  Tue Dec  4 00:48:47 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon, 03 Dec 2007 18:48:47 -0500
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <47548281.3010404@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
	<20071203201438.75F083A40A5@sparrow.telecommunity.com>
	<47548281.3010404@cs.byu.edu>
Message-ID: <20071203234846.274253A40A5@sparrow.telecommunity.com>

At 03:26 PM 12/3/2007 -0700, Neil Toronto wrote:
>Phillip J. Eby wrote:
> > At 12:27 PM 12/3/2007 -0700, Neil Toronto wrote:
> >> Guido van Rossum wrote:
> >> > How about subclasses of str? These have all the same issues...
> >>
> >> Yeah. I ended up having it, per class, permanently revert to uncached
> >> lookups when it detects that a class dict in the MRO has non-string
> >> keys. That's flagged by lookdict_string, which uses PyString_CheckExact.
> >
> > I'm a bit confused here.  Isn't the simplest way to cache attribute
> > lookups to just have a cache dictionary in the type, and update that
> > dictionary whenever a change is made to a superclass?  That's
> > essentially how __slotted__ attribute changes on base classes work now,
> > isn't it?  Why do we need to mess around with the dictionary entries
> > themselves in order to do that?
>
>The nice thing about caching pointers to dict entries is that they don't
>change as often as values do. There are fewer ways to invalidate an
>entry pointer: inserting set, resize, clear, and delete. If you cache
>values, non-inserting set could invalidate as well.
>
>Because inserting into namespace dicts should be very rare, caching
>entries rather than values should reduce the number of times cache
>entries are invalidated to near zero. Updating is expensive, so that's
>good for performance.
>
>Rare updating also means it's okay to invalidate the entire cache rather
>than single entries, so the footprint of the caching mechanism in the
>dict can be very small. For example, I've got a single 64-bit counter in
>each dict that gets incremented after every potentially invalidating
>operation. That comes down to 8 bytes of storage and two extra machine
>instructions (currently) per invalidating operation. The cache checks it
>against its own counter, and updating ensures that it's synced.
>
>Some version of the non-string keys problem would exist with any caching
>mechanism, though. An evil rich compare can always monkey about with
>class dicts in the MRO. If a caching scheme caches values and doesn't
>account for that, it could return stale values. If it caches entries and
>doesn't account for that, it could segfault. I suppose you could argue
>that returning stale values is fitting punishment for using an evil rich
>compare, though the punishee isn't always the same person as the punisher.

Actually, you're missing the part where such evil code *can't* muck 
things up for class dictionaries.  Type dicts aren't reachable via 
ordinary Python code; you *have* to modify them via setattr.  (The 
__dict__ of types returns a read-only proxy object, so the most evil 
rich compare you can imagine still can't touch it.)

This means that MRO cache invalidation can already be detected using 
"type"'s tp_setattro implementation.  And setting attributes on types 
is already extremely rare.  It doesn't seem to me that there's any 
need to use the same namespace speedup mechanism here: capturing 
setattr operations on a type should be sufficient to implement 
invalidation, without mucking about with dictionary entries.  An 
ordinary dict should suffice.

Of course, I suppose there are use cases where somebody uses a class 
attribute as a "global" of sorts, and those use cases would be slowed 
down.  However, if you want to use the entry caching approach, you 
wouldn't need to worry about the segfault case.  (Since somebody 
would have to use C to get at the "real" dictionary.)


From guido at python.org  Tue Dec  4 00:51:17 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 3 Dec 2007 15:51:17 -0800
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <20071203234846.274253A40A5@sparrow.telecommunity.com>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
	<20071203201438.75F083A40A5@sparrow.telecommunity.com>
	<47548281.3010404@cs.byu.edu>
	<20071203234846.274253A40A5@sparrow.telecommunity.com>
Message-ID: <ca471dc20712031551x4ebc313k659aa45ef32d57ad@mail.gmail.com>

On Dec 3, 2007 3:48 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> Actually, you're missing the part where such evil code *can't* muck
> things up for class dictionaries.  Type dicts aren't reachable via
> ordinary Python code; you *have* to modify them via setattr.  (The
> __dict__ of types returns a read-only proxy object, so the most evil
> rich compare you can imagine still can't touch it.)

What's to prevent that evil comparison to call setattr on the class?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ntoronto at cs.byu.edu  Tue Dec  4 06:17:25 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 03 Dec 2007 22:17:25 -0700
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <20071203234846.274253A40A5@sparrow.telecommunity.com>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
	<20071203201438.75F083A40A5@sparrow.telecommunity.com>
	<47548281.3010404@cs.byu.edu>
	<20071203234846.274253A40A5@sparrow.telecommunity.com>
Message-ID: <4754E2E5.9000304@cs.byu.edu>

Phillip J. Eby wrote:
> At 03:26 PM 12/3/2007 -0700, Neil Toronto wrote:
>> Phillip J. Eby wrote:
>> > At 12:27 PM 12/3/2007 -0700, Neil Toronto wrote:
>> Some version of the non-string keys problem would exist with any caching
>> mechanism, though. An evil rich compare can always monkey about with
>> class dicts in the MRO. If a caching scheme caches values and doesn't
>> account for that, it could return stale values. If it caches entries and
>> doesn't account for that, it could segfault. I suppose you could argue
>> that returning stale values is fitting punishment for using an evil rich
>> compare, though the punishee isn't always the same person as the 
>> punisher.
> 
> Actually, you're missing the part where such evil code *can't* muck 
> things up for class dictionaries.  Type dicts aren't reachable via 
> ordinary Python code; you *have* to modify them via setattr.  (The 
> __dict__ of types returns a read-only proxy object, so the most evil 
> rich compare you can imagine still can't touch it.)

Interesting. But I'm going to have to say it probably wouldn't work as 
well, since C code can and does alter tp_dict directly. Those places in 
the core would have to be altered to invalidate the cache. There's also 
the issue of extensions, which so far have been able to alter any 
tp_dict without problems. It'd also be really annoying for a class to 
have to notify all of its subclasses when one of its attributes changed.

In other words, I can see the footprint being rather large and difficult 
to manage. By hooking right into dicts and letting them track when 
things change, every other piece of code in the system can happily 
continue doing whatever it likes without needing to worry that it might 
invalidate some cache entry somewhere. I'm confident that's the right 
design choice whether it's best to cache entries or not.

I hope you don't feel that I'm just trying to be contradictory. I'm 
actually enjoying the discussion a lot. I'd rather have my grand ideas 
tested now than discover I was wrong later.

Neil

From pje at telecommunity.com  Tue Dec  4 07:11:22 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 04 Dec 2007 01:11:22 -0500
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <ca471dc20712031551x4ebc313k659aa45ef32d57ad@mail.gmail.com
 >
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
	<20071203201438.75F083A40A5@sparrow.telecommunity.com>
	<47548281.3010404@cs.byu.edu>
	<20071203234846.274253A40A5@sparrow.telecommunity.com>
	<ca471dc20712031551x4ebc313k659aa45ef32d57ad@mail.gmail.com>
Message-ID: <20071204061122.C4DEA3A40A5@sparrow.telecommunity.com>

At 03:51 PM 12/3/2007 -0800, Guido van Rossum wrote:
>On Dec 3, 2007 3:48 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> > Actually, you're missing the part where such evil code *can't* muck
> > things up for class dictionaries.  Type dicts aren't reachable via
> > ordinary Python code; you *have* to modify them via setattr.  (The
> > __dict__ of types returns a read-only proxy object, so the most evil
> > rich compare you can imagine still can't touch it.)
>
>What's to prevent that evil comparison to call setattr on the class?

If you're caching values, it should be sufficient to have setattr 
trigger the invalidation.  For entries, I have to admit I don't 
understand the approach well enough to make a specific proposal.


From ntoronto at cs.byu.edu  Tue Dec  4 07:12:48 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 03 Dec 2007 23:12:48 -0700
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <4754E2E5.9000304@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>	<47531A5F.5020308@cs.byu.edu>	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>	<4754589D.4020308@cs.byu.edu>	<20071203201438.75F083A40A5@sparrow.telecommunity.com>	<47548281.3010404@cs.byu.edu>	<20071203234846.274253A40A5@sparrow.telecommunity.com>
	<4754E2E5.9000304@cs.byu.edu>
Message-ID: <4754EFE0.1040106@cs.byu.edu>

I apologize - I had forgotten what you were telling me by the time I 
replied. Here's a better answer.

> Phillip J. Eby wrote:
>> At 03:26 PM 12/3/2007 -0700, Neil Toronto wrote:
>> Actually, you're missing the part where such evil code *can't* muck 
>> things up for class dictionaries.  Type dicts aren't reachable via 
>> ordinary Python code; you *have* to modify them via setattr.  (The 
>> __dict__ of types returns a read-only proxy object, so the most evil 
>> rich compare you can imagine still can't touch it.)

C code can and does alter tp_dict directly already. If caching were 
implemented within type's setattr, all these places would have to be 
changed to use setattr only. That doesn't seem so bad at first. It's a 
change in convention, certainly: a new informal rule that says "no 
monkeying with a PyTypeObject's tp_dict, period". Lack of observance 
could be difficult to debug, as a PyDict_SetItem would appear to have 
worked just fine to C code but not show up to Python code.

Neil


From pje at telecommunity.com  Tue Dec  4 07:20:32 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue, 04 Dec 2007 01:20:32 -0500
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <4754E2E5.9000304@cs.byu.edu>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
	<20071203201438.75F083A40A5@sparrow.telecommunity.com>
	<47548281.3010404@cs.byu.edu>
	<20071203234846.274253A40A5@sparrow.telecommunity.com>
	<4754E2E5.9000304@cs.byu.edu>
Message-ID: <20071204062029.D6F883A40A5@sparrow.telecommunity.com>

At 10:17 PM 12/3/2007 -0700, Neil Toronto wrote:
>Phillip J. Eby wrote:
> > Actually, you're missing the part where such evil code *can't* muck
> > things up for class dictionaries.  Type dicts aren't reachable via
> > ordinary Python code; you *have* to modify them via setattr.  (The
> > __dict__ of types returns a read-only proxy object, so the most evil
> > rich compare you can imagine still can't touch it.)
>
>Interesting. But I'm going to have to say it probably wouldn't work as
>well, since C code can and does alter tp_dict directly. Those places in
>the core would have to be altered to invalidate the cache.

Eh?  Where is the type dictionary altered outside of setattr and 
class creation?


>  There's also
>the issue of extensions, which so far have been able to alter any
>tp_dict without problems.

Do you have any actual examples?

Believe me, I'm the last person to suggest removing useful hack, er, 
hooks.  :)  But I don't think that type __dict__ munging is actually 
common at all.


>It'd also be really annoying for a class to
>have to notify all of its subclasses when one of its attributes changed.

It's not all subclasses - only those subclasses that don't shadow the 
attribute.  Also, it's not necessarily the case that notification 
would be O(subclasses) - it could be done via a version counter, as 
in your approach.  Admittedly, that would require an extra bit of 
indirection, since you'd need to keep (and check) counters for each descriptor.


From ntoronto at cs.byu.edu  Tue Dec  4 07:23:10 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Mon, 03 Dec 2007 23:23:10 -0700
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <20071204061122.C4DEA3A40A5@sparrow.telecommunity.com>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
	<20071203201438.75F083A40A5@sparrow.telecommunity.com>
	<47548281.3010404@cs.byu.edu>
	<20071203234846.274253A40A5@sparrow.telecommunity.com>
	<ca471dc20712031551x4ebc313k659aa45ef32d57ad@mail.gmail.com>
	<20071204061122.C4DEA3A40A5@sparrow.telecommunity.com>
Message-ID: <4754F24E.108@cs.byu.edu>

Phillip J. Eby wrote:
> At 03:51 PM 12/3/2007 -0800, Guido van Rossum wrote:
>> On Dec 3, 2007 3:48 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
>> > Actually, you're missing the part where such evil code *can't* muck
>> > things up for class dictionaries.  Type dicts aren't reachable via
>> > ordinary Python code; you *have* to modify them via setattr.  (The
>> > __dict__ of types returns a read-only proxy object, so the most evil
>> > rich compare you can imagine still can't touch it.)
>>
>> What's to prevent that evil comparison to call setattr on the class?
> 
> If you're caching values, it should be sufficient to have setattr 
> trigger the invalidation.  For entries, I have to admit I don't 
> understand the approach well enough to make a specific proposal.

As long as you could determine whether PyDict_SetItem inserted a new 
key, it would make sense. (If it only updates a value, the cache doesn't 
need to change because the pointer to the entry is still valid and the 
entry points to the new value.) The PyDict_SetItem API would have to 
change, or the dict would have to somehow pass the information 
out-of-bound. Neither option sounds great to me, so I'd go with caching 
values from setattr.

Neil


From ntoronto at cs.byu.edu  Tue Dec  4 08:08:53 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Tue, 04 Dec 2007 00:08:53 -0700
Subject: [Python-Dev] Non-string keys in namespace dicts
In-Reply-To: <20071204062029.D6F883A40A5@sparrow.telecommunity.com>
References: <475221E2.4010800@cs.byu.edu>
	<BD4E7F4A-A79D-4E50-83E6-31EB916731F9@nicko.org>
	<47531A5F.5020308@cs.byu.edu>
	<ca471dc20712031053k46111ec6kf748246fc2055b50@mail.gmail.com>
	<4754589D.4020308@cs.byu.edu>
	<20071203201438.75F083A40A5@sparrow.telecommunity.com>
	<47548281.3010404@cs.byu.edu>
	<20071203234846.274253A40A5@sparrow.telecommunity.com>
	<4754E2E5.9000304@cs.byu.edu>
	<20071204062029.D6F883A40A5@sparrow.telecommunity.com>
Message-ID: <4754FD05.3090404@cs.byu.edu>

Phillip J. Eby wrote:
> At 10:17 PM 12/3/2007 -0700, Neil Toronto wrote:
>> Interesting. But I'm going to have to say it probably wouldn't work as
>> well, since C code can and does alter tp_dict directly. Those places in
>> the core would have to be altered to invalidate the cache.
> 
> Eh?  Where is the type dictionary altered outside of setattr and class 
> creation?

You're right - my initial grep turned up stuff that looked like tp_dict 
monkeying out of context. The ctypes module does it a lot, but only in 
its various *_new functions.

>> It'd also be really annoying for a class to
>> have to notify all of its subclasses when one of its attributes changed.
> 
> It's not all subclasses - only those subclasses that don't shadow the 
> attribute.  Also, it's not necessarily the case that notification would 
> be O(subclasses) - it could be done via a version counter, as in your 
> approach.  Admittedly, that would require an extra bit of indirection, 
> since you'd need to keep (and check) counters for each descriptor.

And the extra overhead comes back to bite us again, and probably in a 
critical path. (I'm sure you've been bitten in a critical path before.) 
That's been the issue with all of these caching schemes so far - Python 
is just too durned dynamic to guarantee them anything they can exploit 
for efficiency, so they end up slowing down common operations. (Not that 
I'd change a bit of Python, mind you.)

For example, almost everything I've tried slows down attribute lookups 
on built-in types. Adding one 64-bit version counter check and a branch 
on failure incurs a 3-5% penalty. That's not the end of the world, but 
it makes pybench take about 0.65% longer.

I finally overcame that by making a custom dictionary type to use as the 
cache. I haven't yet tested something my caching lookups are slower at - 
they're all faster so far for builtins and Python objects with any size 
MRO - but I haven't tested exhaustively and I haven't done failing 
hasattr-style lookups. Turns out that not finding an attribute all the 
way up the MRO (which can lead to a persistent cache miss if done with 
the same name) is rather frequent in Python and is expected to be fast. 
I can cache missing attributes as easily as present attributes, but they 
could pile up if someone decides to hasattr an object with a zillion 
different names.

I have a cunning plan, though, which is probably best explained using a 
patch.

At any rate, I'm warming to this setattr idea, and I'll likely try that 
next whether my current approach works out or not.

Neil

From skip at pobox.com  Tue Dec  4 02:16:22 2007
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 3 Dec 2007 19:16:22 -0600
Subject: [Python-Dev] Slow tests involving bsddb - timeout
Message-ID: <18260.43622.615423.63804@montanaro.dyndns.org>


I noticed that test_anydbm and test_bsddb seemed to hang, so I -x'd them.
Later on while test_whichdb was running and I was off doing something else
(so didn't notice the delay), it eventually spewed this traceback:

    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_whichdb.py", line 49, in test_whichdb_name
        f = mod.open(_fname, 'c')
      File "/Users/skip/src/python/trunk/Lib/dbhash.py", line 16, in open
        return bsddb.hashopen(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/bsddb/__init__.py", line 310, in hashopen
        d.open(file, db.DB_HASH, flags, mode)
    DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit (100) exceeded')

Looking at _bsddb.so I see it's linked against Berkeley DB 4.5.  This is on
Mac OSX 10.4.11 using the MacPorts version of Berkeley DB (4.5.20).  Have I
somehow strayed out of the support cocoon without realizing it?  I wouldn't
have thought so, since the max version listed in setup.py is 4.6.

Thx,

Skip

From alexandre at peadrop.com  Tue Dec  4 18:49:32 2007
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Tue, 4 Dec 2007 12:49:32 -0500
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <474E9F40.5050707@gmail.com>
References: <474D8725.8000706@cheimes.de> <474E9F40.5050707@gmail.com>
Message-ID: <acd65fa20712040949v1574ba91h620d5e43d42411af@mail.gmail.com>

I just want to let you all know that the name issue was settled and
committed to py3k branch a few days ago. It was chosen to simply
rename the module __builtin__ to builtins.

-- Alexandre

On Nov 29, 2007 6:15 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Given that the *effect* of __builtins__ is to make the contents of the
> __builtin__ module implicitly available in every module's global
> namespace, why not call it __implicit__?
>
> I really don't like all of these __root__ inspired names, because
> __builtin__ isn't the root of any Python hierarchy that I know of.
>
>  >>> import sys
>  >>> import __builtin__
>  >>> __builtin__.sys
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> AttributeError: 'module' object has no attribute 'sys'
>
> The builtin namespace doesn't know anything about other modules, the
> current module's global namespace, the current function's local
> variables, or much of anything really. To me, the concept of "root" in a
> computing sense implies a node from which you can reach every other node
> - from the root of the filesystem you can get to every other directory,
> as the root user you can access any other account, etc. To those that
> like these names, what do you consider __root__ to be the root of?
>

From alexandre at peadrop.com  Tue Dec  4 18:57:54 2007
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Tue, 4 Dec 2007 12:57:54 -0500
Subject: [Python-Dev] [poll] New name for __builtins__
In-Reply-To: <acd65fa20712040949v1574ba91h620d5e43d42411af@mail.gmail.com>
References: <474D8725.8000706@cheimes.de> <474E9F40.5050707@gmail.com>
	<acd65fa20712040949v1574ba91h620d5e43d42411af@mail.gmail.com>
Message-ID: <acd65fa20712040957g71552a20s5a0ec90deea408de@mail.gmail.com>

Oh, sorry for the noise. I thought people were still arguing about the
name issue, but it was in fact 5-day late emails that I am still
receiving. (Gmail seems to have delivery issues lately...)

-- Alexandre

On Dec 4, 2007 12:49 PM, Alexandre Vassalotti <alexandre at peadrop.com> wrote:
> I just want to let you all know that the name issue was settled and
> committed to py3k branch a few days ago. It was chosen to simply
> rename the module __builtin__ to builtins.
>

From jimjjewett at gmail.com  Tue Dec  4 20:17:53 2007
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 4 Dec 2007 14:17:53 -0500
Subject: [Python-Dev] Non-string keys in namespace dicts
Message-ID: <fb6fbf560712041117j5e8277e5v556fc34ada260e31@mail.gmail.com>

PJE wrote:
> Isn't the simplest way to cache attribute
> lookups to just have a cache dictionary in the type,
> and update that  dictionary whenever a change is
> made to a superclass?  That's  essentially how
> __slotted__ attribute changes on base classes
> work now, isn't it?

Neil Toronto wrote:

> The nice thing about caching pointers to dict
> entries is that they don't change as often as
> values do.

Is this really true for namespaces?

I was thinking that the typical namespace usage is a bunch of inserts
(possibly with lookups mixed in), followed by never changing it again
until it is deallocated.

> There are fewer ways to invalidate an
> entry pointer: inserting set, resize, clear, and delete.

I'm not sure how to resize without an inserting set.

I'm not sure I've ever seen clear on a namespace.  (I have seen it on
regular dicts being used as a namespace, such as tcl config options.)

I have seen deletes (deleting a temp name) and non-inserting sets ...
but they're both rare enough that letting them force the slow path
might be a good trade, if the optimization is otherwise simpler.

> Rare updating also means it's okay to invalidate the
> entire cache rather than single entries

Changing __bases__ seems to do that already.

(See http://svn.python.org/view/python/trunk/Objects/typeobject.c?rev=59106&view=markup
functions like update_subclasses.)

So I think an alternate version PJE's question would be:

Why not just extend that existing mechanism to work on non-slot,
non-method attributes?

-jJ

From guido at python.org  Tue Dec  4 22:49:45 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 4 Dec 2007 13:49:45 -0800
Subject: [Python-Dev] Mac _OSA extension doesn't build on Leopard
Message-ID: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>

On OSX 10.5 with Xcode 3.0, whenever I build either the trunk or the
py3k or 25 branches, I get a series of errors when setup.py tries to
build the _OSA module. On OSX 10.4 it builds fine. Can anybody help?
I don't even know what OSA is!

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Tue Dec  4 23:29:18 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 04 Dec 2007 23:29:18 +0100
Subject: [Python-Dev] Mac _OSA extension doesn't build on Leopard
In-Reply-To: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
References: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
Message-ID: <4755D4BE.4070902@v.loewis.de>

> Can anybody help?
> I don't even know what OSA is!

I can help with that: It's the Open Scripting Architecture,

http://developer.apple.com/documentation/AppleScript/Conceptual/AppleScriptX/Concepts/osa.html

(Probably not the kind of help you were asking for :-)

Regards,
Martin

From janssen at parc.com  Wed Dec  5 04:20:22 2007
From: janssen at parc.com (Bill Janssen)
Date: Tue, 4 Dec 2007 19:20:22 PST
Subject: [Python-Dev] Mac _OSA extension doesn't build on Leopard
In-Reply-To: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com> 
References: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
Message-ID: <07Dec4.192029pst."58696"@synergy1.parc.xerox.com>

I shifted to Leopard a couple of weeks ago.  Seems to build and test
fine, but I disable all the various poorly documented/maintained Mac
modules, so my configure looks like this:

./configure --disable-universalsdk --disable-framework --disable-toolbox-glue

I believe OSA is "toolbox glue", so I'll see what happens.

Sure enough, looks like the API to the OSA changed with 10.5.  Not
unreasonable.  I'd say, just add "--disable-toobox-glue".

Bill

From ronaldoussoren at mac.com  Wed Dec  5 08:19:02 2007
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 5 Dec 2007 08:19:02 +0100
Subject: [Python-Dev] Mac _OSA extension doesn't build on Leopard
In-Reply-To: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
References: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
Message-ID: <2FD319C6-6872-4BEB-8D4C-B772F822398D@mac.com>


On 4 Dec, 2007, at 22:49, Guido van Rossum wrote:

> On OSX 10.5 with Xcode 3.0, whenever I build either the trunk or the
> py3k or 25 branches, I get a series of errors when setup.py tries to
> build the _OSA module. On OSX 10.4 it builds fine. Can anybody help?
> I don't even know what OSA is!

I'll try to do a crude fix later this week. The Carbon bindings also  
wrap some deprecated API's, some of which were dropped in Leopard.  
We're kind of lucky that _OSA is the only one that no longer compiles.

A correct fix will take some more time, as this will require  
retargeting the bgen tool to use System headers instead of the OS9 (!)  
headers that were used to generate the current Carbon bindings.

Ronald


From guido at python.org  Wed Dec  5 17:56:32 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 08:56:32 -0800
Subject: [Python-Dev] Mac _OSA extension doesn't build on Leopard
In-Reply-To: <2FD319C6-6872-4BEB-8D4C-B772F822398D@mac.com>
References: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
	<2FD319C6-6872-4BEB-8D4C-B772F822398D@mac.com>
Message-ID: <ca471dc20712050856j53b1d2e6r7281e999c627076e@mail.gmail.com>

Thanks! The sooner the better given that tonight (PST) I plan to do
the code freeze for the 3.0a2 release, and Anthony is also making
noises about 2.5.2 again.

--Guido

On Dec 4, 2007 11:19 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 4 Dec, 2007, at 22:49, Guido van Rossum wrote:
>
> > On OSX 10.5 with Xcode 3.0, whenever I build either the trunk or the
> > py3k or 25 branches, I get a series of errors when setup.py tries to
> > build the _OSA module. On OSX 10.4 it builds fine. Can anybody help?
> > I don't even know what OSA is!
>
> I'll try to do a crude fix later this week. The Carbon bindings also
> wrap some deprecated API's, some of which were dropped in Leopard.
> We're kind of lucky that _OSA is the only one that no longer compiles.
>
> A correct fix will take some more time, as this will require
> retargeting the bgen tool to use System headers instead of the OS9 (!)
> headers that were used to generate the current Carbon bindings.
>
> Ronald
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec  5 18:19:27 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 09:19:27 -0800
Subject: [Python-Dev] Does anyone care enough about asyncore and asynchat to
	help adapt their APIs for Py3k?
Message-ID: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>

The asyncore and asynchat modules are in a difficult position when it
comes to Python 3000. None of the core developers use it or
particularly care about it (AFAIK), and the API has problems because
it wasn't written to deal with bytes vs. unicode. E.g. in
http://bugs.python.org/issue1067, Thomas suggests that these modules
need to be rewritten to use bytes internally and have separate APIs to
handle (unicode) text as desired, similar to the way file I/O was
redesigned. Another alternative would be to make these modules deal
strictly in bytes, but that would probably vastly reduce their
usefulness (though I don't know -- as I said, I don't use them).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fuzzyman at voidspace.org.uk  Wed Dec  5 19:17:34 2007
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 05 Dec 2007 18:17:34 +0000
Subject: [Python-Dev] New Standard Library Module
Message-ID: <4756EB3E.9000201@voidspace.org.uk>

Hello all,

Can I suggest a new module for the standard library: 'antigravity.py'.

Perhaps it could display a particular image on import...


Michael
http://www.manning.com/foord

From guido at python.org  Wed Dec  5 19:20:18 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 10:20:18 -0800
Subject: [Python-Dev] Does anyone care enough about asyncore and
	asynchat to help adapt their APIs for Py3k?
In-Reply-To: <e6511dbf0712051004g3a67c1d9j183af1d8fc1bcba5@mail.gmail.com>
References: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
	<e6511dbf0712051004g3a67c1d9j183af1d8fc1bcba5@mail.gmail.com>
Message-ID: <ca471dc20712051020n18fce184y32e7116968cd7e07@mail.gmail.com>

On Dec 5, 2007 10:04 AM, Josiah Carlson <josiah.carlson at gmail.com> wrote:
> On Dec 5, 2007 9:19 AM, Guido van Rossum <guido at python.org> wrote:
> > The asyncore and asynchat modules are in a difficult position when it
> > comes to Python 3000. None of the core developers use it or
> > particularly care about it (AFAIK), and the API has problems because
> > it wasn't written to deal with bytes vs. unicode. E.g. in
> > http://bugs.python.org/issue1067, Thomas suggests that these modules
> > need to be rewritten to use bytes internally and have separate APIs to
> > handle (unicode) text as desired, similar to the way file I/O was
> > redesigned. Another alternative would be to make these modules deal
> > strictly in bytes, but that would probably vastly reduce their
> > usefulness (though I don't know -- as I said, I don't use them).
>
> I can look into it later this month, but for the short term, I'm a
> little squeezed for time (work, finishing school, etc.).

That's fine; I see this as part of the library reorg effort, which is
still in the starting blocks.

> I am a bit
> curious, however, I could have sworn that bytes were to become,
> essentially, the 2.x str type (without some methods).  If that is the
> case, no changes should really be necessary, except for a few small
> changes to asynchat with regards to line terminators.

Well, yes, that would be the easiest thing (and may be right for
asyncore), but most of the code that *uses* asynchat (that I've seen
anyway) passes it text strings, not byte strings. And in 3.0, all text
is unicode, and you *must* specify an encoding *somewhere* in order to
convert it to bytes correctly -- otherwise it will blow up immediately
with a TypeError.

> Then again, I haven't really been keeping up in the p3k/pydev lists
> for the last 6-8 months, and only the subject line of this posting
> caught my eye (because I use asyncore/asynchat, and support users of
> asyncore/asynchat that contact me directly).

Good to know there are users. Perhaps you could make them aware of this request.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From phd at phd.pp.ru  Wed Dec  5 19:36:35 2007
From: phd at phd.pp.ru (Oleg Broytmann)
Date: Wed, 5 Dec 2007 21:36:35 +0300
Subject: [Python-Dev] New Standard Library Module
In-Reply-To: <4756EB3E.9000201@voidspace.org.uk>
References: <4756EB3E.9000201@voidspace.org.uk>
Message-ID: <20071205183635.GA13122@phd.pp.ru>

On Wed, Dec 05, 2007 at 06:17:34PM +0000, Michael Foord wrote:
> Can I suggest a new module for the standard library: 'antigravity.py'.

   A friend of mine (the person who has suggested "raise without
arguments") recommends implementing it in two phases. The first should be

from __future__ import antigravity

XKCD'ly yours ;)
Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From janssen at parc.com  Wed Dec  5 20:27:38 2007
From: janssen at parc.com (Bill Janssen)
Date: Wed, 5 Dec 2007 11:27:38 PST
Subject: [Python-Dev] Does anyone care enough about asyncore and
	asynchat to help adapt their APIs for Py3k?
In-Reply-To: <ca471dc20712051020n18fce184y32e7116968cd7e07@mail.gmail.com> 
References: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
	<e6511dbf0712051004g3a67c1d9j183af1d8fc1bcba5@mail.gmail.com>
	<ca471dc20712051020n18fce184y32e7116968cd7e07@mail.gmail.com>
Message-ID: <07Dec5.112745pst."58696"@synergy1.parc.xerox.com>

> Good to know there are users.

And I use Medusa, which is built on top of asyncore.

Bill

From ronaldoussoren at mac.com  Wed Dec  5 21:08:41 2007
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 5 Dec 2007 21:08:41 +0100
Subject: [Python-Dev] Mac _OSA extension doesn't build on Leopard
In-Reply-To: <ca471dc20712050856j53b1d2e6r7281e999c627076e@mail.gmail.com>
References: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
	<2FD319C6-6872-4BEB-8D4C-B772F822398D@mac.com>
	<ca471dc20712050856j53b1d2e6r7281e999c627076e@mail.gmail.com>
Message-ID: <B35CE359-6480-4D79-B500-C8D9B4854F53@mac.com>


On 5 Dec, 2007, at 17:56, Guido van Rossum wrote:

> Thanks! The sooner the better given that tonight (PST) I plan to do
> the code freeze for the 3.0a2 release, and Anthony is also making
> noises about 2.5.2 again.

I'm working on it right now. I would like a pronouncement on a  
backward incompatible change though:

The _OSA module doesn't compile anymore because it wraps an API that  
was unsupported in 10.4 and was removed in  10.5. I can probably add  
some configury-logic  to detect if the API is still present, but would  
prefer to remove those wrappers completely. That's no problem for 2.6  
and 3.0, but strictly speaking this would introduce a backward  
incompatibility in 2.5.2.

The wrappers are for debugging functionality and unlikely to be used  
by anyone.

BTW. the wrappers for OSADebug* functions are now gone in the trunk  
(as of revision 59369).

Ronald

>
>
> --Guido
>
> On Dec 4, 2007 11:19 PM, Ronald Oussoren <ronaldoussoren at mac.com>  
> wrote:
>>
>> On 4 Dec, 2007, at 22:49, Guido van Rossum wrote:
>>
>>> On OSX 10.5 with Xcode 3.0, whenever I build either the trunk or the
>>> py3k or 25 branches, I get a series of errors when setup.py tries to
>>> build the _OSA module. On OSX 10.4 it builds fine. Can anybody help?
>>> I don't even know what OSA is!
>>
>> I'll try to do a crude fix later this week. The Carbon bindings also
>> wrap some deprecated API's, some of which were dropped in Leopard.
>> We're kind of lucky that _OSA is the only one that no longer  
>> compiles.
>>
>> A correct fix will take some more time, as this will require
>> retargeting the bgen tool to use System headers instead of the OS9  
>> (!)
>> headers that were used to generate the current Carbon bindings.
>>
>> Ronald
>>
>>
>
>
>
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)


From guido at python.org  Wed Dec  5 21:25:27 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 12:25:27 -0800
Subject: [Python-Dev] Mac _OSA extension doesn't build on Leopard
In-Reply-To: <B35CE359-6480-4D79-B500-C8D9B4854F53@mac.com>
References: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
	<2FD319C6-6872-4BEB-8D4C-B772F822398D@mac.com>
	<ca471dc20712050856j53b1d2e6r7281e999c627076e@mail.gmail.com>
	<B35CE359-6480-4D79-B500-C8D9B4854F53@mac.com>
Message-ID: <ca471dc20712051225i73899a1s26f228b8a0b3a507@mail.gmail.com>

How about this: delete them in 2.6 (3.0 will follow after a merge); in
2.5.2, put them inside an #if or #ifdef. Bonus points if you can use a
condition that's true on 10.4 and false on 10.5, but always false is
okay with me too, as long as there's a comment explaining it.

--Guido

On Dec 5, 2007 12:08 PM, Ronald Oussoren <ronaldoussoren at mac.com> wrote:
>
> On 5 Dec, 2007, at 17:56, Guido van Rossum wrote:
>
> > Thanks! The sooner the better given that tonight (PST) I plan to do
> > the code freeze for the 3.0a2 release, and Anthony is also making
> > noises about 2.5.2 again.
>
> I'm working on it right now. I would like a pronouncement on a
> backward incompatible change though:
>
> The _OSA module doesn't compile anymore because it wraps an API that
> was unsupported in 10.4 and was removed in  10.5. I can probably add
> some configury-logic  to detect if the API is still present, but would
> prefer to remove those wrappers completely. That's no problem for 2.6
> and 3.0, but strictly speaking this would introduce a backward
> incompatibility in 2.5.2.
>
> The wrappers are for debugging functionality and unlikely to be used
> by anyone.
>
> BTW. the wrappers for OSADebug* functions are now gone in the trunk
> (as of revision 59369).
>
> Ronald
>
>
> >
> >
> > --Guido
> >
> > On Dec 4, 2007 11:19 PM, Ronald Oussoren <ronaldoussoren at mac.com>
> > wrote:
> >>
> >> On 4 Dec, 2007, at 22:49, Guido van Rossum wrote:
> >>
> >>> On OSX 10.5 with Xcode 3.0, whenever I build either the trunk or the
> >>> py3k or 25 branches, I get a series of errors when setup.py tries to
> >>> build the _OSA module. On OSX 10.4 it builds fine. Can anybody help?
> >>> I don't even know what OSA is!
> >>
> >> I'll try to do a crude fix later this week. The Carbon bindings also
> >> wrap some deprecated API's, some of which were dropped in Leopard.
> >> We're kind of lucky that _OSA is the only one that no longer
> >> compiles.
> >>
> >> A correct fix will take some more time, as this will require
> >> retargeting the bgen tool to use System headers instead of the OS9
> >> (!)
> >> headers that were used to generate the current Carbon bindings.
> >>
> >> Ronald
> >>
> >>
> >
> >
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gnewsg at gmail.com  Wed Dec  5 21:32:46 2007
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Wed, 5 Dec 2007 12:32:46 -0800 (PST)
Subject: [Python-Dev] Does anyone care enough about asyncore and
	asynchat to help adapt their APIs for Py3k?
In-Reply-To: <07Dec5.112745pst."58696"@synergy1.parc.xerox.com>
References: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com> 
	<e6511dbf0712051004g3a67c1d9j183af1d8fc1bcba5@mail.gmail.com> 
	<ca471dc20712051020n18fce184y32e7116968cd7e07@mail.gmail.com> 
	<07Dec5.112745pst."58696"@synergy1.parc.xerox.com>
Message-ID: <a4932068-1594-4c8a-8d64-77d5bc7a901c@s36g2000prg.googlegroups.com>

On 5 Dic, 20:27, Bill Janssen <jans... at parc.com> wrote:
> > Good to know there are users.
>
> And I use Medusa, which is built on top of asyncore.
>
> Bill

+1.
I use asyncore/asynchat modules in a public project of mine.

From ronaldoussoren at mac.com  Wed Dec  5 21:48:24 2007
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 5 Dec 2007 21:48:24 +0100
Subject: [Python-Dev] Mac _OSA extension doesn't build on Leopard
In-Reply-To: <ca471dc20712051225i73899a1s26f228b8a0b3a507@mail.gmail.com>
References: <ca471dc20712041349y7d09ae95wd4e62eefab146385@mail.gmail.com>
	<2FD319C6-6872-4BEB-8D4C-B772F822398D@mac.com>
	<ca471dc20712050856j53b1d2e6r7281e999c627076e@mail.gmail.com>
	<B35CE359-6480-4D79-B500-C8D9B4854F53@mac.com>
	<ca471dc20712051225i73899a1s26f228b8a0b3a507@mail.gmail.com>
Message-ID: <15E99C21-D51D-4B28-9FD1-070D81F49C23@mac.com>


On 5 Dec, 2007, at 21:25, Guido van Rossum wrote:

> How about this: delete them in 2.6 (3.0 will follow after a merge); in
> 2.5.2, put them inside an #if or #ifdef. Bonus points if you can use a
> condition that's true on 10.4 and false on 10.5, but always false is
> okay with me too, as long as there's a comment explaining it.

There's now a configure test of this in the 2.5 branch (as of revision  
59372), the wrappers for OSADebug API's were removed in 2.6 in  
revision 59369 (earlier tonight) and should be gone in 3.0 after the  
next merge.

Both patches update a generated file, but as it is virtually  
impossible to regenerate these files at the moment that should be  
perfectly safe.

Ronald

>
>
> --Guido
>
> On Dec 5, 2007 12:08 PM, Ronald Oussoren <ronaldoussoren at mac.com>  
> wrote:
>>
>> On 5 Dec, 2007, at 17:56, Guido van Rossum wrote:
>>
>>> Thanks! The sooner the better given that tonight (PST) I plan to do
>>> the code freeze for the 3.0a2 release, and Anthony is also making
>>> noises about 2.5.2 again.
>>
>> I'm working on it right now. I would like a pronouncement on a
>> backward incompatible change though:
>>
>> The _OSA module doesn't compile anymore because it wraps an API that
>> was unsupported in 10.4 and was removed in  10.5. I can probably add
>> some configury-logic  to detect if the API is still present, but  
>> would
>> prefer to remove those wrappers completely. That's no problem for 2.6
>> and 3.0, but strictly speaking this would introduce a backward
>> incompatibility in 2.5.2.
>>
>> The wrappers are for debugging functionality and unlikely to be used
>> by anyone.
>>
>> BTW. the wrappers for OSADebug* functions are now gone in the trunk
>> (as of revision 59369).
>>
>> Ronald
>>
>>
>>>
>>>
>>> --Guido
>>>
>>> On Dec 4, 2007 11:19 PM, Ronald Oussoren <ronaldoussoren at mac.com>
>>> wrote:
>>>>
>>>> On 4 Dec, 2007, at 22:49, Guido van Rossum wrote:
>>>>
>>>>> On OSX 10.5 with Xcode 3.0, whenever I build either the trunk or  
>>>>> the
>>>>> py3k or 25 branches, I get a series of errors when setup.py  
>>>>> tries to
>>>>> build the _OSA module. On OSX 10.4 it builds fine. Can anybody  
>>>>> help?
>>>>> I don't even know what OSA is!
>>>>
>>>> I'll try to do a crude fix later this week. The Carbon bindings  
>>>> also
>>>> wrap some deprecated API's, some of which were dropped in Leopard.
>>>> We're kind of lucky that _OSA is the only one that no longer
>>>> compiles.
>>>>
>>>> A correct fix will take some more time, as this will require
>>>> retargeting the bgen tool to use System headers instead of the OS9
>>>> (!)
>>>> headers that were used to generate the current Carbon bindings.
>>>>
>>>> Ronald
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> --Guido van Rossum (home page: http://www.python.org/~guido/)
>>
>>
>
>
>
> -- 
> --Guido van Rossum (home page: http://www.python.org/~guido/)


From gherron at islandtraining.com  Wed Dec  5 21:54:49 2007
From: gherron at islandtraining.com (Gary Herron)
Date: Wed, 05 Dec 2007 12:54:49 -0800
Subject: [Python-Dev] Does anyone care enough about asyncore and
 asynchat to	help adapt their APIs for Py3k?
In-Reply-To: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
References: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
Message-ID: <47571019.50809@islandtraining.com>

Guido van Rossum wrote:
> The asyncore and asynchat modules are in a difficult position when it
> comes to Python 3000. None of the core developers use it or
> particularly care about it (AFAIK), and the API has problems because
> it wasn't written to deal with bytes vs. unicode. E.g. in
> http://bugs.python.org/issue1067, Thomas suggests that these modules
> need to be rewritten to use bytes internally and have separate APIs to
> handle (unicode) text as desired, similar to the way file I/O was
> redesigned. Another alternative would be to make these modules deal
> strictly in bytes, but that would probably vastly reduce their
> usefulness (though I don't know -- as I said, I don't use them).
>
>   
I use asyncore/asynchat in one (proprietary) project of mine.  However,
since the only thing I use them for is bytes, your suggested alternative
(of bytes instead of strings) is fine with me, and seems the most
natural choice. 

(In fact what I'm currently passing around is strings produced by
cPickle,  but I'm assuming that the Python3 version of cPickle will
create/consume bytes.  True?)

Gary Herron




From ntoronto at cs.byu.edu  Wed Dec  5 22:24:21 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Wed, 05 Dec 2007 14:24:21 -0700
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
Message-ID: <47571705.9080309@cs.byu.edu>

So Jim and PJE finally convinced me to do it the right way. :) Thanks 
guys - it turned out very nice.

http://bugs.python.org/issue1560

http://spreadsheets.google.com/ccc?key=pHIJrYc_pnIUpTm6QSG2gZg&hl=en_US

It caches type/metatype attribute lookups, including missing attributes. 
Summary of the bug tracker issue:

- Successful attribute lookups are 20% faster, even for classes with 
short MROs and (probably most) builtins - haven't tested unsuccessful 
lookups

- Successful hasattr is 5-10% faster, unsuccessful is 5% faster (less 
impressive than above, and likely due to overhead - internally, all 
lookups are the same)

- list.__init__ and list().__init__ are slower, and I can't figure out 
why (creating instances of subclasses of list will be a little slower, 
and this may show up in other builtin types)

- I haven't benchmarked type attribute sets (how much do we care?) - it 
should be quite a bit faster than updating a slot, though

- Caching missing attributes is crucial for good performance

- The CreateNewInstances benchmark uncovered an issue that needs fixing; 
please see the tracker for details

All kinds of commentary and feedback is most welcome.

Neil

From g.brandl at gmx.net  Wed Dec  5 22:48:37 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 05 Dec 2007 22:48:37 +0100
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <47571705.9080309@cs.byu.edu>
References: <47571705.9080309@cs.byu.edu>
Message-ID: <fj767v$9h4$1@ger.gmane.org>

Neil Toronto schrieb:
> So Jim and PJE finally convinced me to do it the right way. :) Thanks 
> guys - it turned out very nice.

How does this relate to Armin Rigo's method cache patch?

(http://bugs.python.org/issue1685986)

Georg


From pje at telecommunity.com  Wed Dec  5 23:50:28 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 05 Dec 2007 17:50:28 -0500
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <fj767v$9h4$1@ger.gmane.org>
References: <47571705.9080309@cs.byu.edu>
 <fj767v$9h4$1@ger.gmane.org>
Message-ID: <20071205225030.B23BE3A405E@sparrow.telecommunity.com>

At 10:48 PM 12/5/2007 +0100, Georg Brandl wrote:
>Neil Toronto schrieb:
> > So Jim and PJE finally convinced me to do it the right way. :) Thanks
> > guys - it turned out very nice.
>
>How does this relate to Armin Rigo's method cache patch?
>
>(http://bugs.python.org/issue1685986)

Interesting.  Armin's approach uses a single global cache of up to 
1024 descriptors.  That seems a lot simpler than anything I thought 
of, and might perform better by encouraging the processor to keep the 
descriptors in cache.  It has a lot less pointer indirection, and has 
a dirt-simple way of invalidating a class' entries when something changes.

Was there any reason (aside from the usual lack of volunteers for 
review) why it didn't go in already?


From bioinformed at gmail.com  Thu Dec  6 01:08:14 2007
From: bioinformed at gmail.com (Kevin Jacobs <jacobs@bioinformed.com>)
Date: Wed, 5 Dec 2007 19:08:14 -0500
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <20071205225030.B23BE3A405E@sparrow.telecommunity.com>
References: <47571705.9080309@cs.byu.edu> <fj767v$9h4$1@ger.gmane.org>
	<20071205225030.B23BE3A405E@sparrow.telecommunity.com>
Message-ID: <2e1434c10712051608x534d88cdg23a4b8af7ecd6f29@mail.gmail.com>

On Dec 5, 2007 5:50 PM, Phillip J. Eby <pje at telecommunity.com> wrote:

> At 10:48 PM 12/5/2007 +0100, Georg Brandl wrote:
> >Neil Toronto schrieb:
> > > So Jim and PJE finally convinced me to do it the right way. :) Thanks
> > > guys - it turned out very nice.
> >
> >How does this relate to Armin Rigo's method cache patch?
> >
> >(http://bugs.python.org/issue1685986)
>
> [...]


> Was there any reason (aside from the usual lack of volunteers for
> review) why it didn't go in already?
>


I'm not sure-- I think folks were waiting for Raymond H. to evaluate it.  I
did my part to update Armin's patch against 2.4 to HEAD at the time and put
in what cleanups seemed sensible.  FWIW, I've been using an interpreter with
this patch ever since without problems.

-Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071205/29755699/attachment.htm 

From guido at python.org  Thu Dec  6 01:11:26 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 16:11:26 -0800
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <2e1434c10712051608x534d88cdg23a4b8af7ecd6f29@mail.gmail.com>
References: <47571705.9080309@cs.byu.edu> <fj767v$9h4$1@ger.gmane.org>
	<20071205225030.B23BE3A405E@sparrow.telecommunity.com>
	<2e1434c10712051608x534d88cdg23a4b8af7ecd6f29@mail.gmail.com>
Message-ID: <ca471dc20712051611s552e0141oc02ae7258729b9c2@mail.gmail.com>

On Dec 5, 2007 4:08 PM, Kevin Jacobs <jacobs at bioinformed.com>
<bioinformed at gmail.com> wrote:
> On Dec 5, 2007 5:50 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> > At 10:48 PM 12/5/2007 +0100, Georg Brandl wrote:
> > >Neil Toronto schrieb:
> > > > So Jim and PJE finally convinced me to do it the right way. :) Thanks
> > > > guys - it turned out very nice.
> > >
> > >How does this relate to Armin Rigo's method cache patch?
> > >
> > >(http://bugs.python.org/issue1685986)

> > Was there any reason (aside from the usual lack of volunteers for
> > review) why it didn't go in already?

> I'm not sure-- I think folks were waiting for Raymond H. to evaluate it.  I
> did my part to update Armin's patch against 2.4 to HEAD at the time and put
> in what cleanups seemed sensible.  FWIW, I've been using an interpreter with
> this patch ever since without problems.

I never even saw that one. I'm hoping Raymond will have another look.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec  6 01:12:54 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 16:12:54 -0800
Subject: [Python-Dev] Does anyone care enough about asyncore and
	asynchat to help adapt their APIs for Py3k?
In-Reply-To: <47571019.50809@islandtraining.com>
References: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
	<47571019.50809@islandtraining.com>
Message-ID: <ca471dc20712051612i6f7e3dch2f622be78dda92f8@mail.gmail.com>

On Dec 5, 2007 12:54 PM, Gary Herron <gherron at islandtraining.com> wrote:
> Guido van Rossum wrote:
> > The asyncore and asynchat modules are in a difficult position when it
> > comes to Python 3000. None of the core developers use it or
> > particularly care about it (AFAIK), and the API has problems because
> > it wasn't written to deal with bytes vs. unicode. E.g. in
> > http://bugs.python.org/issue1067, Thomas suggests that these modules
> > need to be rewritten to use bytes internally and have separate APIs to
> > handle (unicode) text as desired, similar to the way file I/O was
> > redesigned. Another alternative would be to make these modules deal
> > strictly in bytes, but that would probably vastly reduce their
> > usefulness (though I don't know -- as I said, I don't use them).

> I use asyncore/asynchat in one (proprietary) project of mine.  However,
> since the only thing I use them for is bytes, your suggested alternative
> (of bytes instead of strings) is fine with me, and seems the most
> natural choice.
>
> (In fact what I'm currently passing around is strings produced by
> cPickle,  but I'm assuming that the Python3 version of cPickle will
> create/consume bytes.  True?)

Right. Although it'll just be "pickle" -- if there's a C accelerator,
it'll be hidden from view, so you won't have to change the imported
name to get it.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec  6 01:14:17 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 16:14:17 -0800
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <ca471dc20712051611s552e0141oc02ae7258729b9c2@mail.gmail.com>
References: <47571705.9080309@cs.byu.edu> <fj767v$9h4$1@ger.gmane.org>
	<20071205225030.B23BE3A405E@sparrow.telecommunity.com>
	<2e1434c10712051608x534d88cdg23a4b8af7ecd6f29@mail.gmail.com>
	<ca471dc20712051611s552e0141oc02ae7258729b9c2@mail.gmail.com>
Message-ID: <ca471dc20712051614m3424daf1u832308983fa33d61@mail.gmail.com>

Hm... rhettinger at ewtllc.com bounced. I wonder what's going on there...

On Dec 5, 2007 4:11 PM, Guido van Rossum <guido at python.org> wrote:
> On Dec 5, 2007 4:08 PM, Kevin Jacobs <jacobs at bioinformed.com>
>
> <bioinformed at gmail.com> wrote:
> > On Dec 5, 2007 5:50 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> > > At 10:48 PM 12/5/2007 +0100, Georg Brandl wrote:
> > > >Neil Toronto schrieb:
> > > > > So Jim and PJE finally convinced me to do it the right way. :) Thanks
> > > > > guys - it turned out very nice.
> > > >
> > > >How does this relate to Armin Rigo's method cache patch?
> > > >
> > > >(http://bugs.python.org/issue1685986)
>
> > > Was there any reason (aside from the usual lack of volunteers for
> > > review) why it didn't go in already?
>
> > I'm not sure-- I think folks were waiting for Raymond H. to evaluate it.  I
> > did my part to update Armin's patch against 2.4 to HEAD at the time and put
> > in what cleanups seemed sensible.  FWIW, I've been using an interpreter with
> > this patch ever since without problems.
>
> I never even saw that one. I'm hoping Raymond will have another look.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at krypto.org  Thu Dec  6 02:24:42 2007
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 5 Dec 2007 17:24:42 -0800
Subject: [Python-Dev] Slow tests involving bsddb - timeout
In-Reply-To: <18260.43622.615423.63804@montanaro.dyndns.org>
References: <18260.43622.615423.63804@montanaro.dyndns.org>
Message-ID: <52dc1c820712051724l605893cdq14746a8585138538@mail.gmail.com>

I'd expect 4.5 to work fine but I don't know why you're getting such a
strange error, i've never seen that.  fwiw i suggest people avoid
berkeleydb 4.6 for now.

On 12/3/07, skip at pobox.com <skip at pobox.com> wrote:
>
> I noticed that test_anydbm and test_bsddb seemed to hang, so I -x'd them.
> Later on while test_whichdb was running and I was off doing something else
> (so didn't notice the delay), it eventually spewed this traceback:
>
>     Traceback (most recent call last):
>       File "/Users/skip/src/python/trunk/Lib/test/test_whichdb.py", line 49, in test_whichdb_name
>         f = mod.open(_fname, 'c')
>       File "/Users/skip/src/python/trunk/Lib/dbhash.py", line 16, in open
>         return bsddb.hashopen(file, flag, mode)
>       File "/Users/skip/src/python/trunk/Lib/bsddb/__init__.py", line 310, in hashopen
>         d.open(file, db.DB_HASH, flags, mode)
>     DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit (100) exceeded')
>
> Looking at _bsddb.so I see it's linked against Berkeley DB 4.5.  This is on
> Mac OSX 10.4.11 using the MacPorts version of Berkeley DB (4.5.20).  Have I
> somehow strayed out of the support cocoon without realizing it?  I wouldn't
> have thought so, since the max version listed in setup.py is 4.6.
>
> Thx,
>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>

From guido at python.org  Thu Dec  6 02:30:40 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 17:30:40 -0800
Subject: [Python-Dev] Slow tests involving bsddb - timeout
In-Reply-To: <52dc1c820712051724l605893cdq14746a8585138538@mail.gmail.com>
References: <18260.43622.615423.63804@montanaro.dyndns.org>
	<52dc1c820712051724l605893cdq14746a8585138538@mail.gmail.com>
Message-ID: <ca471dc20712051730q320377e9i98ab4a2acdad8956@mail.gmail.com>

I think I've seen this too when running the bsddb3 unittest. I think
it's caused by a previous test ending badly and leaving junk behind
that the test suite doesn't properly remove before starting. But I
don't recall the details.

--Guido

On Dec 5, 2007 5:24 PM, Gregory P. Smith <greg at krypto.org> wrote:
> I'd expect 4.5 to work fine but I don't know why you're getting such a
> strange error, i've never seen that.  fwiw i suggest people avoid
> berkeleydb 4.6 for now.
>
>
> On 12/3/07, skip at pobox.com <skip at pobox.com> wrote:
> >
> > I noticed that test_anydbm and test_bsddb seemed to hang, so I -x'd them.
> > Later on while test_whichdb was running and I was off doing something else
> > (so didn't notice the delay), it eventually spewed this traceback:
> >
> >     Traceback (most recent call last):
> >       File "/Users/skip/src/python/trunk/Lib/test/test_whichdb.py", line 49, in test_whichdb_name
> >         f = mod.open(_fname, 'c')
> >       File "/Users/skip/src/python/trunk/Lib/dbhash.py", line 16, in open
> >         return bsddb.hashopen(file, flag, mode)
> >       File "/Users/skip/src/python/trunk/Lib/bsddb/__init__.py", line 310, in hashopen
> >         d.open(file, db.DB_HASH, flags, mode)
> >     DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit (100) exceeded')
> >
> > Looking at _bsddb.so I see it's linked against Berkeley DB 4.5.  This is on
> > Mac OSX 10.4.11 using the MacPorts version of Berkeley DB (4.5.20).  Have I
> > somehow strayed out of the support cocoon without realizing it?  I wouldn't
> > have thought so, since the max version listed in setup.py is 4.6.
> >
> > Thx,
> >
> > Skip
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
> >
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec  6 02:46:40 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 17:46:40 -0800
Subject: [Python-Dev] Py3k code freeze imminent; 3.0a2 release Friday
Message-ID: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>

I'm planning to freeze the py3k branch in 2-3 hours, some time
after/around 8pm PST (midnight UTC).

If someone wants to do another svnmerge from the trunk please do it
before then -- though we're nearly current so I don't mind not having
the last few changes merged into this release (it's only Raymond's
refactoring of __length_hint__ implementations).

If there's anything you think really should be in this release, please
speak up ASAP. Filing a high priority bug and assigning it to me would
be a great way to get my attention.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Thu Dec  6 03:11:04 2007
From: python at rcn.com (Raymond Hettinger)
Date: Wed,  5 Dec 2007 21:11:04 -0500 (EST)
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
Message-ID: <20071205211104.ACT45503@ms19.lnh.mail.rcn.net>

> I never even saw that one. I'm hoping Raymond will have another look.

Great.  Will review it this week.


Raymond

From python at rcn.com  Thu Dec  6 03:28:12 2007
From: python at rcn.com (Raymond Hettinger)
Date: Wed,  5 Dec 2007 21:28:12 -0500 (EST)
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
Message-ID: <20071205212812.ACT48257@ms19.lnh.mail.rcn.net>

> Hm... rhettinger at ewtllc.com bounced. I wonder what's going on there..

I'm now in an EWT spin-off company.  The new email address is rhettinger at fattoc.com.  Also, I frequently check the python at rcn.com account too.


Raymond

From djarb at highenergymagic.org  Thu Dec  6 03:34:31 2007
From: djarb at highenergymagic.org (Daniel Arbuckle)
Date: Wed, 5 Dec 2007 18:34:31 -0800
Subject: [Python-Dev] Does anyone care enough about asyncore and
	asynchat to help adapt their APIs for Py3k?
Message-ID: <ddd408be0712051834x7273988et16195b366933ccf5@mail.gmail.com>

Guido van Rossum wrote:
> The asyncore and asynchat modules are in a difficult position when it
> comes to Python 3000. None of the core developers use it or
> particularly care about it (AFAIK), and the API has problems because
> it wasn't written to deal with bytes vs. unicode. E.g. in
> http://bugs.python.org/issue1067, Thomas suggests that these modules
> need to be rewritten to use bytes internally and have separate APIs to
> handle (unicode) text as desired, similar to the way file I/O was
> redesigned. Another alternative would be to make these modules deal
> strictly in bytes, but that would probably vastly reduce their
> usefulness (though I don't know -- as I said, I don't use them).

I guess that would be me, if you'll have me. I think asyncore and
asynchat are valuable, and I'm willing to do what needs to be done to
ensure that they remain in Py3k.

From ntoronto at cs.byu.edu  Thu Dec  6 03:43:22 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Wed, 05 Dec 2007 19:43:22 -0700
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <20071205225030.B23BE3A405E@sparrow.telecommunity.com>
References: <47571705.9080309@cs.byu.edu> <fj767v$9h4$1@ger.gmane.org>
	<20071205225030.B23BE3A405E@sparrow.telecommunity.com>
Message-ID: <475761CA.2070200@cs.byu.edu>

Phillip J. Eby wrote:
> At 10:48 PM 12/5/2007 +0100, Georg Brandl wrote:
>> Neil Toronto schrieb:
>>> So Jim and PJE finally convinced me to do it the right way. :) Thanks
>>> guys - it turned out very nice.
>> How does this relate to Armin Rigo's method cache patch?
>>
>> (http://bugs.python.org/issue1685986)
> 
> Interesting.  Armin's approach uses a single global cache of up to 
> 1024 descriptors.  That seems a lot simpler than anything I thought 
> of, and might perform better by encouraging the processor to keep the 
> descriptors in cache.  It has a lot less pointer indirection, and has 
> a dirt-simple way of invalidating a class' entries when something changes.

Hey, I took out all my extra pointer indirection. :p

FWIW, I like it. Though the hash should really incorporate the hash of 
the type name as well as the attribute's so that sometype.method calling 
othertype.method doesn't invalidate the cache. Locality makes the global 
cache work, but locality also often means re-using the same names.

Neil

From guido at python.org  Thu Dec  6 04:01:42 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 19:01:42 -0800
Subject: [Python-Dev] Does anyone care enough about asyncore and
	asynchat to help adapt their APIs for Py3k?
In-Reply-To: <ddd408be0712051834x7273988et16195b366933ccf5@mail.gmail.com>
References: <ddd408be0712051834x7273988et16195b366933ccf5@mail.gmail.com>
Message-ID: <ca471dc20712051901v3db0f957s917a30cf7007179d@mail.gmail.com>

On Dec 5, 2007 6:34 PM, Daniel Arbuckle <djarb at highenergymagic.org> wrote:
> Guido van Rossum wrote:
> > The asyncore and asynchat modules are in a difficult position when it
> > comes to Python 3000. None of the core developers use it or
> > particularly care about it (AFAIK), and the API has problems because
> > it wasn't written to deal with bytes vs. unicode. E.g. in
> > http://bugs.python.org/issue1067, Thomas suggests that these modules
> > need to be rewritten to use bytes internally and have separate APIs to
> > handle (unicode) text as desired, similar to the way file I/O was
> > redesigned. Another alternative would be to make these modules deal
> > strictly in bytes, but that would probably vastly reduce their
> > usefulness (though I don't know -- as I said, I don't use them).
>
> I guess that would be me, if you'll have me. I think asyncore and
> asynchat are valuable, and I'm willing to do what needs to be done to
> ensure that they remain in Py3k.

Excellent!

Are you at all familiar with how Py3k deals with bytes vs strings? If
not, perhaps you could start by reading PEP 3137. I'd be happy to
answer any questions you have (possibly off-list).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec  6 05:43:10 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 5 Dec 2007 20:43:10 -0800
Subject: [Python-Dev] Py3k code freeze imminent; 3.0a2 release Friday
In-Reply-To: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>
References: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>
Message-ID: <ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>

I've built and tested the latest py3k from scratch on Ubuntu, Fedora
7, OSX 10.4 and OSX 10.5, and found no issues.

So the code freeze is a fact. Don't check anything into the py3k
branch unless I tell you to. Please file high-priority bugs and assign
them to me if you think you've found a showstopper.

The buildbot is green for Solaris also, so I think we're in good
shape. I don't see any green buildbots for Windows though, but Windows
is always a flakey situation; Christian, what's your assessment?

I see a few tests leaking; in particular test_ssl (1522 refs leaned
per run!) and test_urllib2_localnet (3 per run). Anyone interested in
researching these feel free to do so; just upload a patch and file a
bug if you've squashed the leaks (or some).

We're on for a 3.0a2 release Friday!

--Guido

On Dec 5, 2007 5:46 PM, Guido van Rossum <guido at python.org> wrote:
> I'm planning to freeze the py3k branch in 2-3 hours, some time
> after/around 8pm PST (midnight UTC).
>
> If someone wants to do another svnmerge from the trunk please do it
> before then -- though we're nearly current so I don't mind not having
> the last few changes merged into this release (it's only Raymond's
> refactoring of __length_hint__ implementations).
>
> If there's anything you think really should be in this release, please
> speak up ASAP. Filing a high priority bug and assigning it to me would
> be a great way to get my attention.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Dec  6 06:08:09 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 06 Dec 2007 00:08:09 -0500
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <475761CA.2070200@cs.byu.edu>
References: <47571705.9080309@cs.byu.edu> <fj767v$9h4$1@ger.gmane.org>
	<20071205225030.B23BE3A405E@sparrow.telecommunity.com>
	<475761CA.2070200@cs.byu.edu>
Message-ID: <20071206050807.7439A3A405E@sparrow.telecommunity.com>

At 07:43 PM 12/5/2007 -0700, Neil Toronto wrote:
>Phillip J. Eby wrote:
> > At 10:48 PM 12/5/2007 +0100, Georg Brandl wrote:
> >> Neil Toronto schrieb:
> >>> So Jim and PJE finally convinced me to do it the right way. :) Thanks
> >>> guys - it turned out very nice.
> >> How does this relate to Armin Rigo's method cache patch?
> >>
> >> (http://bugs.python.org/issue1685986)
> >
> > Interesting.  Armin's approach uses a single global cache of up to
> > 1024 descriptors.  That seems a lot simpler than anything I thought
> > of, and might perform better by encouraging the processor to keep the
> > descriptors in cache.  It has a lot less pointer indirection, and has
> > a dirt-simple way of invalidating a class' entries when something changes.
>
>Hey, I took out all my extra pointer indirection. :p
>
>FWIW, I like it. Though the hash should really incorporate the hash of
>the type name as well as the attribute's so that sometype.method calling
>othertype.method doesn't invalidate the cache. Locality makes the global
>cache work, but locality also often means re-using the same names.

Look at the patch more closely.  The hash function uses a version 
number times the method name's hash.  "Version" numbers are assigned 
one per class, so unless there are 2**32 classes in the system, they 
are uniquely numbered.  The multiplication and use of the high bits 
should tend to spread the hash locations around and avoid same-name collisions.

Of course, it's still always possible to have pathological cases, but 
even these shouldn't be much slower than the way things work now.


From skip at pobox.com  Thu Dec  6 04:09:27 2007
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 5 Dec 2007 21:09:27 -0600
Subject: [Python-Dev] Slow tests involving bsddb - timeout
In-Reply-To: <ca471dc20712051730q320377e9i98ab4a2acdad8956@mail.gmail.com>
References: <18260.43622.615423.63804@montanaro.dyndns.org>
	<52dc1c820712051724l605893cdq14746a8585138538@mail.gmail.com>
	<ca471dc20712051730q320377e9i98ab4a2acdad8956@mail.gmail.com>
Message-ID: <18263.26599.386184.603421@montanaro.dyndns.org>


    Guido> I think I've seen this too when running the bsddb3 unittest. I
    Guido> think it's caused by a previous test ending badly and leaving
    Guido> junk behind that the test suite doesn't properly remove before
    Guido> starting. But I don't recall the details.

Thanks, that at least gives me some hope that I can try to narrow down the
problem with a little binary searching.

Skip

From ntoronto at cs.byu.edu  Thu Dec  6 07:35:10 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Wed, 05 Dec 2007 23:35:10 -0700
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <20071206050807.7439A3A405E@sparrow.telecommunity.com>
References: <47571705.9080309@cs.byu.edu> <fj767v$9h4$1@ger.gmane.org>
	<20071205225030.B23BE3A405E@sparrow.telecommunity.com>
	<475761CA.2070200@cs.byu.edu>
	<20071206050807.7439A3A405E@sparrow.telecommunity.com>
Message-ID: <4757981E.1010006@cs.byu.edu>

Phillip J. Eby wrote:
> At 07:43 PM 12/5/2007 -0700, Neil Toronto wrote:
>> FWIW, I like it. Though the hash should really incorporate the hash of
>> the type name as well as the attribute's so that sometype.method calling
>> othertype.method doesn't invalidate the cache. Locality makes the global
>> cache work, but locality also often means re-using the same names.
> 
> Look at the patch more closely.  The hash function uses a version number 
> times the method name's hash.  "Version" numbers are assigned one per 
> class, so unless there are 2**32 classes in the system, they are 
> uniquely numbered.  The multiplication and use of the high bits should 
> tend to spread the hash locations around and avoid same-name collisions.

Good grief - how did I miss that? I plead parenthesis. They threw me off.

So I've applied Armin's patch to 2.6 (it was nearly clean) and am 
playing with it. cls.name lookups are 15-20% faster than mine, and 
inst.name lookups are 5-10% faster. His is also winning on hasattr calls 
(succeeding and failing) on classes, but mine is winning on hasattr 
calls on instances. I want to poke at it a bit to find out why.

On pybench, his is faster at BuiltinMethodLookups, significantly faster 
at CreateNewInstances, and a bit faster at almost everything else. 
BuiltinFunctionCalls is slower - slower than stock - it might need 
poking there, too.

In all, it's a lovely piece of work.

Neil

From lists at cheimes.de  Thu Dec  6 09:46:47 2007
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 06 Dec 2007 09:46:47 +0100
Subject: [Python-Dev] Py3k code freeze imminent; 3.0a2 release Friday
In-Reply-To: <ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
References: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>
	<ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
Message-ID: <4757B6F7.8040501@cheimes.de>

Guido van Rossum wrote:
> The buildbot is green for Solaris also, so I think we're in good
> shape. I don't see any green buildbots for Windows though, but Windows
> is always a flakey situation; Christian, what's your assessment?

test_mailbox is still failing with lots of errors. The module needs
extra embracement and love on Windows. Most to all of the problems are
related to the newline separator \r\n.

Some modules are also failing when I run a refleak regression test, see
http://bugs.python.org/issue1540

I've ironed out the last cosmetic problems in the PCbuild9 directory and
profile guided optimization builds. A PGO version can be build with
"build_pgo -2" (runs the complete unit test suite) or "build_pgo" (runs
PyBench) in a VS 2008 command shell. The x64 builds are looking fine
except of tkinter (it doesn't build) but I'm not able to test the x64
version on my computer.

Could you please add two comments to the release notes for Windows users?

* On Windows Python can't be run from a directory with non ASCII chars
in its path name. #1342

* The current releases of MinGW and Cygwin can't build Python extensions
since they don't support msvcr90.dll. The necessary bits and pieces are
already in Python and cygwin cvs.

> I see a few tests leaking; in particular test_ssl (1522 refs leaned
> per run!) and test_urllib2_localnet (3 per run). Anyone interested in
> researching these feel free to do so; just upload a patch and file a
> bug if you've squashed the leaks (or some).

The test_ssl tests are only leaking with the -unetwork option. On my
Ubuntu box they are leaking 1536 references per turn. For heaven's sake
I can't remember how I found the leaking code lines the last time.
Py_DUMP_REFS dumps too many information.

Christian

From bioinformed at gmail.com  Thu Dec  6 13:08:47 2007
From: bioinformed at gmail.com (Kevin Jacobs <jacobs@bioinformed.com>)
Date: Thu, 6 Dec 2007 07:08:47 -0500
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <4757981E.1010006@cs.byu.edu>
References: <47571705.9080309@cs.byu.edu> <fj767v$9h4$1@ger.gmane.org>
	<20071205225030.B23BE3A405E@sparrow.telecommunity.com>
	<475761CA.2070200@cs.byu.edu>
	<20071206050807.7439A3A405E@sparrow.telecommunity.com>
	<4757981E.1010006@cs.byu.edu>
Message-ID: <2e1434c10712060408o7320364dl309e36eddfb25220@mail.gmail.com>

On Dec 6, 2007 1:35 AM, Neil Toronto <ntoronto at cs.byu.edu> wrote:

> So I've applied Armin's patch to 2.6 (it was nearly clean) and am
> playing with it. cls.name lookups are 15-20% faster than mine, and
> inst.name lookups are 5-10% faster. His is also winning on hasattr calls
> (succeeding and failing) on classes, but mine is winning on hasattr
> calls on instances. I want to poke at it a bit to find out why.
>

I hope folks have noticed that I've done some significant cleanup and
forward porting some months ago at

http://bugs.python.org/issue1700288

-Kevin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071206/bd63dcc4/attachment.htm 

From lists at cheimes.de  Thu Dec  6 15:58:35 2007
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 06 Dec 2007 15:58:35 +0100
Subject: [Python-Dev] Py3k code freeze imminent; 3.0a2 release Friday
In-Reply-To: <ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
References: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>
	<ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
Message-ID: <47580E1B.10103@cheimes.de>

Guido van Rossum wrote:
> I see a few tests leaking; in particular test_ssl (1522 refs leaned
> per run!) and test_urllib2_localnet (3 per run). Anyone interested in
> researching these feel free to do so; just upload a patch and file a
> bug if you've squashed the leaks (or some).

I see the reference leaks, too. I didn't notice the ssl leaks before
because I usually don't run the refleak test with -unetwork or -uall.

./python Lib/test/regrtest.py -R1:2 -unetwork test_ssl

I've started to work on a patch that adds GC support to Modules/_ssl.c
PySSLObject but I don't have time to finish it until tonight.
http://bugs.python.org/issue1469

Christian

From lists at cheimes.de  Thu Dec  6 15:58:35 2007
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 06 Dec 2007 15:58:35 +0100
Subject: [Python-Dev] Py3k code freeze imminent; 3.0a2 release Friday
In-Reply-To: <ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
References: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>
	<ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
Message-ID: <47580E1B.10103@cheimes.de>

Guido van Rossum wrote:
> I see a few tests leaking; in particular test_ssl (1522 refs leaned
> per run!) and test_urllib2_localnet (3 per run). Anyone interested in
> researching these feel free to do so; just upload a patch and file a
> bug if you've squashed the leaks (or some).

I see the reference leaks, too. I didn't notice the ssl leaks before
because I usually don't run the refleak test with -unetwork or -uall.

./python Lib/test/regrtest.py -R1:2 -unetwork test_ssl

I've started to work on a patch that adds GC support to Modules/_ssl.c
PySSLObject but I don't have time to finish it until tonight.
http://bugs.python.org/issue1469

Christian


From janssen at parc.com  Thu Dec  6 18:46:52 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 6 Dec 2007 09:46:52 PST
Subject: [Python-Dev] Py3k code freeze imminent; 3.0a2 release Friday
In-Reply-To: <ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com> 
References: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>
	<ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
Message-ID: <07Dec6.094654pst."58696"@synergy1.parc.xerox.com>

> I see a few tests leaking; in particular test_ssl (1522 refs leaned
> per run!)

I'm looking at this, but I haven't found anything in the last week...

Bill

From janssen at parc.com  Thu Dec  6 18:48:45 2007
From: janssen at parc.com (Bill Janssen)
Date: Thu, 6 Dec 2007 09:48:45 PST
Subject: [Python-Dev] [Python-3000] Py3k code freeze imminent;
	3.0a2 release Friday
In-Reply-To: <4757B6F7.8040501@cheimes.de> 
References: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>
	<ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
	<4757B6F7.8040501@cheimes.de>
Message-ID: <07Dec6.094849pst."58696"@synergy1.parc.xerox.com>

> The test_ssl tests are only leaking with the -unetwork option. On my
> Ubuntu box they are leaking 1536 references per turn. For heaven's sake
> I can't remember how I found the leaking code lines the last time.
> Py_DUMP_REFS dumps too many information.

I found the leak the last time by narrowing down the tests, test by
test.

It was leaking sockets because they got dropped on the floor when a
connect failed.  I'll look at this some more this weekend.

Bill

From greg at krypto.org  Fri Dec  7 03:49:49 2007
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 6 Dec 2007 18:49:49 -0800
Subject: [Python-Dev] tstate_delete_current infinite loop from
	PyThreadState_DeleteCurrent
Message-ID: <52dc1c820712061849u2636189eoc9e2bc18d18fd9d4@mail.gmail.com>

Has anyone else ever encountered a situation where a python process gets
stuck in an infinite loop within Python/pystate.c tstate_delete_current()
called from PyThreadState_DeleteCurrent() when a thread is
exiting?  (revealed by attaching to the looping process with a debugger)

I'm seeing this (very rarely, reproducing it is difficult).  In this case
its python 2.4.3 but that code does not appear to have changed since then.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071206/407b0b81/attachment.htm 

From ntoronto at cs.byu.edu  Fri Dec  7 04:50:47 2007
From: ntoronto at cs.byu.edu (Neil Toronto)
Date: Thu, 06 Dec 2007 20:50:47 -0700
Subject: [Python-Dev] PATCH: attribute lookup caching for 2.6
In-Reply-To: <2e1434c10712060408o7320364dl309e36eddfb25220@mail.gmail.com>
References: <47571705.9080309@cs.byu.edu> <fj767v$9h4$1@ger.gmane.org>	
	<20071205225030.B23BE3A405E@sparrow.telecommunity.com>	
	<475761CA.2070200@cs.byu.edu>	
	<20071206050807.7439A3A405E@sparrow.telecommunity.com>	
	<4757981E.1010006@cs.byu.edu>
	<2e1434c10712060408o7320364dl309e36eddfb25220@mail.gmail.com>
Message-ID: <4758C317.7080509@cs.byu.edu>

Kevin Jacobs <jacobs at bioinformed.com> wrote:
> On Dec 6, 2007 1:35 AM, Neil Toronto <ntoronto at cs.byu.edu 
> <mailto:ntoronto at cs.byu.edu>> wrote:
> 
>     So I've applied Armin's patch to 2.6 (it was nearly clean) and am
>     playing with it. cls.name <http://cls.name> lookups are 15-20%
>     faster than mine, and
>     inst.name <http://inst.name> lookups are 5-10% faster. His is also
>     winning on hasattr calls
>     (succeeding and failing) on classes, but mine is winning on hasattr
>     calls on instances. I want to poke at it a bit to find out why.
> 
> 
> I hope folks have noticed that I've done some significant cleanup and 
> forward porting some months ago at
> 
> http://bugs.python.org/issue1700288

Excellent. I tried this as well. This is guarding cache access with 
PyString_CheckExact (as it should) rather than asserting PyString_Check, 
plus a few other cleanups. It runs nearly as fast as Armin's, and still 
faster than mine and much faster than without.

Neil


From jafo-python-dev at tummy.com  Fri Dec  7 05:56:06 2007
From: jafo-python-dev at tummy.com (Sean Reifschneider)
Date: Thu, 6 Dec 2007 21:56:06 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
Message-ID: <20071207045606.GA13636@tummy.com>

Overview (my reading of it):

   PyGTK wakes up 10x a second in it's main loop because signals may be
   delivered to another thread and will only get picked up if the mainloop
   wakes up.

In the following thread:

   http://mail.python.org/pipermail/python-dev/2006-September/068569.html

it sounds like the patch at:

   http://bugs.python.org/issue1564547

doesn't solve the problem.  A recent gnome bug brings this issue back up:

   http://bugzilla.gnome.org/show_bug.cgi?id=481569

I went ahead and closed the python issue as "rejected" to hopefully get
some more activity on it.

I thought about this some, and I wondered if there was some way we could
signal the sleeping thread when a signal came in on another thread.  Like
perhaps we could make some code to create a pipe, and put it someplace that
all threads could get access to.  Then, if a thread gets a signal, write on
this pipe.  The mainloop could include this file descriptor in the set it's
watching, so it would wake up when the signal came in.

Is this something Python should provide, or something PyGTK should do?  If
an approach like the above would work, we could make it so that select()
always created this file descriptor and added it to one of the FD sets, so
that it would do the right thing behind the scenes.

I have no idea if this is a reasonable approach, but it's something that
came to mind when I thought about the problem and was an approach I didn't
see mentioned before in the discussion.

Thanks,
Sean
-- 
 I live my life like I type; Fast with lots of mistakes.
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability
      Back off man. I'm a scientist.   http://HackingSociety.org/


From guido at python.org  Fri Dec  7 06:08:54 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 6 Dec 2007 21:08:54 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071207045606.GA13636@tummy.com>
References: <20071207045606.GA13636@tummy.com>
Message-ID: <ca471dc20712062108j7f55fa38h82836bb1c985faf4@mail.gmail.com>

The OLPC project is also interested in getting this solved.

Can you explain how Gtk uses signals and threads together? The
combination strikes me as a recipe for disaster, but I'm probably
missing something.

--Guido

On Dec 6, 2007 8:56 PM, Sean Reifschneider <jafo-python-dev at tummy.com> wrote:
> Overview (my reading of it):
>
>    PyGTK wakes up 10x a second in it's main loop because signals may be
>    delivered to another thread and will only get picked up if the mainloop
>    wakes up.
>
> In the following thread:
>
>    http://mail.python.org/pipermail/python-dev/2006-September/068569.html
>
> it sounds like the patch at:
>
>    http://bugs.python.org/issue1564547
>
> doesn't solve the problem.  A recent gnome bug brings this issue back up:
>
>    http://bugzilla.gnome.org/show_bug.cgi?id=481569
>
> I went ahead and closed the python issue as "rejected" to hopefully get
> some more activity on it.
>
> I thought about this some, and I wondered if there was some way we could
> signal the sleeping thread when a signal came in on another thread.  Like
> perhaps we could make some code to create a pipe, and put it someplace that
> all threads could get access to.  Then, if a thread gets a signal, write on
> this pipe.  The mainloop could include this file descriptor in the set it's
> watching, so it would wake up when the signal came in.
>
> Is this something Python should provide, or something PyGTK should do?  If
> an approach like the above would work, we could make it so that select()
> always created this file descriptor and added it to one of the FD sets, so
> that it would do the right thing behind the scenes.
>
> I have no idea if this is a reasonable approach, but it's something that
> came to mind when I thought about the problem and was an approach I didn't
> see mentioned before in the discussion.
>
> Thanks,
> Sean
> --
>  I live my life like I type; Fast with lots of mistakes.
> Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
> tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability
>       Back off man. I'm a scientist.   http://HackingSociety.org/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rhamph at gmail.com  Fri Dec  7 06:55:12 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Thu, 6 Dec 2007 22:55:12 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071207045606.GA13636@tummy.com>
References: <20071207045606.GA13636@tummy.com>
Message-ID: <aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>

On Dec 6, 2007 9:56 PM, Sean Reifschneider <jafo-python-dev at tummy.com> wrote:
> Overview (my reading of it):
>
>    PyGTK wakes up 10x a second in it's main loop because signals may be
>    delivered to another thread and will only get picked up if the mainloop
>    wakes up.
>
> In the following thread:
>
>    http://mail.python.org/pipermail/python-dev/2006-September/068569.html
>
> it sounds like the patch at:
>
>    http://bugs.python.org/issue1564547
>
> doesn't solve the problem.  A recent gnome bug brings this issue back up:
>
>    http://bugzilla.gnome.org/show_bug.cgi?id=481569
>
> I went ahead and closed the python issue as "rejected" to hopefully get
> some more activity on it.
>
> I thought about this some, and I wondered if there was some way we could
> signal the sleeping thread when a signal came in on another thread.  Like
> perhaps we could make some code to create a pipe, and put it someplace that
> all threads could get access to.  Then, if a thread gets a signal, write on
> this pipe.  The mainloop could include this file descriptor in the set it's
> watching, so it would wake up when the signal came in.
>
> Is this something Python should provide, or something PyGTK should do?  If
> an approach like the above would work, we could make it so that select()
> always created this file descriptor and added it to one of the FD sets, so
> that it would do the right thing behind the scenes.
>
> I have no idea if this is a reasonable approach, but it's something that
> came to mind when I thought about the problem and was an approach I didn't
> see mentioned before in the discussion.

That's pretty much what issue1564547 does.  I think there's two marks
against it:
* Using poll and fd's is pretty platform specific for what should be a
general-purpose API
* Handling signals is icky, hard to get right, and nobody trusts it

Since I don't think there's any more immediate solutions I'll provide
a plan B: my threading patch[1] will have a dedicated signal handler
thread, allowing them to be processed regardless of one blocked
thread.  I'm also providing an interrupt API the gtk bindings could
use to support wakeups, while keeping the poll+fd details private.


[1] http://code.google.com/p/python-safethread/
  The patch is, of course, out of date.

-- 
Adam Olsen, aka Rhamphoryncus

From lists at cheimes.de  Fri Dec  7 12:41:07 2007
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 07 Dec 2007 12:41:07 +0100
Subject: [Python-Dev] Split up the c-api documentation in smaller files?
Message-ID: <fjbbgj$s6n$1@ger.gmane.org>

Good afternoon everybody!

The new C API documentation contains some large files:

105K abstract.html
300K concrete.html
183K newtypes.html

The concrete.html takes noticeable time to render on my computer (P4 2.4
with 1GB RAM, Firefox 2.0 and Ubuntu Linux). I estimate the load and
rendering time is about 4-5 seconds! It also takes long to search for a
string and scrolling isn't smooth, too.

Can we split the files in smaller chunks, please?

Christian


From steve at holdenweb.com  Fri Dec  7 13:24:04 2007
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 07 Dec 2007 07:24:04 -0500
Subject: [Python-Dev] Split up the c-api documentation in smaller files?
In-Reply-To: <fjbbgj$s6n$1@ger.gmane.org>
References: <fjbbgj$s6n$1@ger.gmane.org>
Message-ID: <47593B64.7050704@holdenweb.com>

Christian Heimes wrote:
> Good afternoon everybody!
> 
> The new C API documentation contains some large files:
> 
> 105K abstract.html
> 300K concrete.html
> 183K newtypes.html
> 
> The concrete.html takes noticeable time to render on my computer (P4 2.4
> with 1GB RAM, Firefox 2.0 and Ubuntu Linux). I estimate the load and
> rendering time is about 4-5 seconds! It also takes long to search for a
> string and scrolling isn't smooth, too.
> 
> Can we split the files in smaller chunks, please?
> 
This might make a good GHOP project, though as Titus has just uploaded 
about seventy (70!) new projects I am unsure as to how long it would 
have to wait ti get in the queue.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/

From steve at holdenweb.com  Fri Dec  7 13:24:04 2007
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 07 Dec 2007 07:24:04 -0500
Subject: [Python-Dev] Split up the c-api documentation in smaller files?
In-Reply-To: <fjbbgj$s6n$1@ger.gmane.org>
References: <fjbbgj$s6n$1@ger.gmane.org>
Message-ID: <47593B64.7050704@holdenweb.com>

Christian Heimes wrote:
> Good afternoon everybody!
> 
> The new C API documentation contains some large files:
> 
> 105K abstract.html
> 300K concrete.html
> 183K newtypes.html
> 
> The concrete.html takes noticeable time to render on my computer (P4 2.4
> with 1GB RAM, Firefox 2.0 and Ubuntu Linux). I estimate the load and
> rendering time is about 4-5 seconds! It also takes long to search for a
> string and scrolling isn't smooth, too.
> 
> Can we split the files in smaller chunks, please?
> 
This might make a good GHOP project, though as Titus has just uploaded 
about seventy (70!) new projects I am unsure as to how long it would 
have to wait ti get in the queue.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From jafo at tummy.com  Fri Dec  7 15:23:50 2007
From: jafo at tummy.com (Sean Reifschneider)
Date: Fri, 7 Dec 2007 07:23:50 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712062108j7f55fa38h82836bb1c985faf4@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<ca471dc20712062108j7f55fa38h82836bb1c985faf4@mail.gmail.com>
Message-ID: <20071207142350.GB13636@tummy.com>

On Thu, Dec 06, 2007 at 09:08:54PM -0800, Guido van Rossum wrote:
>Can you explain how Gtk uses signals and threads together? The

Me?  No.  I've updated the bug at gnome.org asking someone there to answer
this.

FYI: I have no real interest in this, a friend of mine is interested in
this, just from a "why is powertop saying pygtk is waking up 10 times a
second on my laptop?" standpoint.  So I'm just trying to shepherd this.

Sean
-- 
 "Sometimes Omaha can't be avoided."
                 -- Howard Borden the navigator, _Bob_Newhart_
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability


From jafo-python-dev at tummy.com  Fri Dec  7 15:26:42 2007
From: jafo-python-dev at tummy.com (Sean Reifschneider)
Date: Fri, 7 Dec 2007 07:26:42 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
Message-ID: <20071207142642.GC13636@tummy.com>

On Thu, Dec 06, 2007 at 10:55:12PM -0700, Adam Olsen wrote:
>That's pretty much what issue1564547 does.  I think there's two marks

Good point, I hadn't seen that before.

>* Using poll and fd's is pretty platform specific for what should be a
>general-purpose API

I would say that this is an optimization that helps a specific set of
platforms, including one that I think we really care about, the OLPC which
needs it for decreased battery use.  It doesn't cause breakage of
other platforms, it just may not help them.

Thanks,
Sean
-- 
 Linux, because eventually you grow up enough to be trusted with a fork().
Sean Reifschneider, Member of Technical Staff <jafo at tummy.com>
tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability
      Back off man. I'm a scientist.   http://HackingSociety.org/


From facundobatista at gmail.com  Fri Dec  7 15:35:25 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 7 Dec 2007 11:35:25 -0300
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071207142350.GB13636@tummy.com>
References: <20071207045606.GA13636@tummy.com>
	<ca471dc20712062108j7f55fa38h82836bb1c985faf4@mail.gmail.com>
	<20071207142350.GB13636@tummy.com>
Message-ID: <e04bdf310712070635n3ce684ddq630c8bf1a66cd979@mail.gmail.com>

2007/12/7, Sean Reifschneider <jafo at tummy.com>:

> FYI: I have no real interest in this, a friend of mine is interested in
> this, just from a "why is powertop saying pygtk is waking up 10 times a
> second on my laptop?" standpoint.  So I'm just trying to shepherd this.

As a Gnome user, I'm personally +1 for these improvements.

But, in a general point of view, I'd hate to see a article somewhere
in the future saying something like "using GTK is ok, but if you want
to be power-friendly, avoid using it from Python"...

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From g.brandl at gmx.net  Fri Dec  7 15:42:44 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 07 Dec 2007 15:42:44 +0100
Subject: [Python-Dev] Split up the c-api documentation in smaller files?
In-Reply-To: <47593B64.7050704@holdenweb.com>
References: <fjbbgj$s6n$1@ger.gmane.org> <47593B64.7050704@holdenweb.com>
Message-ID: <fjbm50$p9$1@ger.gmane.org>

Steve Holden schrieb:
> Christian Heimes wrote:
>> Good afternoon everybody!
>> 
>> The new C API documentation contains some large files:
>> 
>> 105K abstract.html
>> 300K concrete.html
>> 183K newtypes.html
>> 
>> The concrete.html takes noticeable time to render on my computer (P4 2.4
>> with 1GB RAM, Firefox 2.0 and Ubuntu Linux). I estimate the load and
>> rendering time is about 4-5 seconds! It also takes long to search for a
>> string and scrolling isn't smooth, too.
>> 
>> Can we split the files in smaller chunks, please?

Yes! It's in the TODO list since the beginning. Other than the C API, there's
also stdtypes.rst which needs quite a bit of splitting and refactoring.

> This might make a good GHOP project, though as Titus has just uploaded 
> about seventy (70!) new projects I am unsure as to how long it would 
> have to wait ti get in the queue.

Oh, provided the students stay as eager as they are at the moment, the
task won't be unclaimed for long. However, I'm not sure if that isn't too
dull for GHOP though... remember, we want to show them how interesting
Open Source development is ;)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From gjcarneiro at gmail.com  Fri Dec  7 15:48:51 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Fri, 7 Dec 2007 14:48:51 +0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071207142642.GC13636@tummy.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
Message-ID: <a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>

On 07/12/2007, Sean Reifschneider <jafo-python-dev at tummy.com> wrote:
>
> On Thu, Dec 06, 2007 at 10:55:12PM -0700, Adam Olsen wrote:
> >That's pretty much what issue1564547 does.  I think there's two marks
>
> Good point, I hadn't seen that before.
>
> >* Using poll and fd's is pretty platform specific for what should be a
> >general-purpose API
>
> I would say that this is an optimization that helps a specific set of
> platforms, including one that I think we really care about, the OLPC which
> needs it for decreased battery use.  It doesn't cause breakage of
> other platforms, it just may not help them.


Not only that, but current python signal handling is not theorethically
async safe; there are race conditions in the Py_AddPendingCalls API, and it
just happens to work most of the time.

BTW, the problem is described here:
http://mail.python.org/pipermail/python-dev/2006-September/068569.html

Think of even python select.poll and multiple threads.  If you have dozens
of threads, each blocked in some system call, when a signal arrives it will
interrupt one of the system calls, causing it to return EINTR, and then
python checks for signals.  Now imagine all the non-main threads are not
created by python; then one of the threads that wakes up could very well be
non-python one, and so python will never realize a signal was delivered.

The solution of blocking signals in all but the python thread(s) is not
feasible as we cannot control all threads that are created, some are created
by C libraries...

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071207/3eafef5c/attachment.htm 

From guido at python.org  Fri Dec  7 19:03:31 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Dec 2007 10:03:31 -0800
Subject: [Python-Dev] Py3k code freeze imminent; 3.0a2 release Friday
In-Reply-To: <ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
References: <ca471dc20712051746gc8666eft1ecba706ab400465@mail.gmail.com>
	<ca471dc20712052043o2eaf29b8m40a58e49345ddce1@mail.gmail.com>
Message-ID: <ca471dc20712071003g1c26ebb1vb938c7e4e372be0e@mail.gmail.com>

As people have been disregarding the freeze anyway, I declare the py3k
branch back open. I tagged it with r30a2 yesterday morning and that's
the version that I'll be releasing shortly (waiting for Crys & me to
sort out some things around the Windows MSI installer).

--Guido

On Dec 5, 2007 8:43 PM, Guido van Rossum <guido at python.org> wrote:
> I've built and tested the latest py3k from scratch on Ubuntu, Fedora
> 7, OSX 10.4 and OSX 10.5, and found no issues.
>
> So the code freeze is a fact. Don't check anything into the py3k
> branch unless I tell you to. Please file high-priority bugs and assign
> them to me if you think you've found a showstopper.
>
> The buildbot is green for Solaris also, so I think we're in good
> shape. I don't see any green buildbots for Windows though, but Windows
> is always a flakey situation; Christian, what's your assessment?
>
> I see a few tests leaking; in particular test_ssl (1522 refs leaned
> per run!) and test_urllib2_localnet (3 per run). Anyone interested in
> researching these feel free to do so; just upload a patch and file a
> bug if you've squashed the leaks (or some).
>
> We're on for a 3.0a2 release Friday!
>
> --Guido
>
>
> On Dec 5, 2007 5:46 PM, Guido van Rossum <guido at python.org> wrote:
> > I'm planning to freeze the py3k branch in 2-3 hours, some time
> > after/around 8pm PST (midnight UTC).
> >
> > If someone wants to do another svnmerge from the trunk please do it
> > before then -- though we're nearly current so I don't mind not having
> > the last few changes merged into this release (it's only Raymond's
> > refactoring of __length_hint__ implementations).
> >
> > If there's anything you think really should be in this release, please
> > speak up ASAP. Filing a high priority bug and assigning it to me would
> > be a great way to get my attention.
> >
> > --
> > --Guido van Rossum (home page: http://www.python.org/~guido/)
> >
>
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From status at bugs.python.org  Fri Dec  7 19:06:06 2007
From: status at bugs.python.org (Tracker)
Date: Fri,  7 Dec 2007 18:06:06 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071207180606.E071278097@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (11/30/07 - 12/07/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1324 open (+22) / 11727 closed (+19) / 13051 total (+41)

Open issues with patches:   417

Average duration of open issues: 700 days.
Median duration of open issues: 858 days.

Open Issues Breakdown
   open  1317 (+22)
pending     7 ( +0)

Issues Created Or Reopened (42)
_______________________________

Management of KeyboardInterrupt in cmd.py                        12/04/07
       http://bugs.python.org/issue1294    reopened gvanrossum               
       patch                                                                   

Add os.fchmod                                                    11/30/07
CLOSED http://bugs.python.org/issue1528    created  gvanrossum               
                                                                               

Error when passing a file object to tarfile.open()               11/30/07
CLOSED http://bugs.python.org/issue1529    created  GeorgeNotaras            
                                                                               

doctest should return error if not all tests passed              11/30/07
       http://bugs.python.org/issue1530    created  tebeka                   
                                                                               

tarfile.open(fileobj=f) and bad metadata of the first file withi 11/30/07
CLOSED http://bugs.python.org/issue1531    created  GeorgeNotaras            
                                                                               

Refleak run of test_tcl fails                                    11/30/07
       http://bugs.python.org/issue1532    created  tiran                    
       py3k                                                                    

Bug in range() function for large values                         12/01/07
       http://bugs.python.org/issue1533    created  robertwb                 
                                                                               

sys.maxfloat patch                                               12/01/07
CLOSED http://bugs.python.org/issue1534    created  tiran                    
       py3k, patch                                                             

Rename __builtin__ to builtins                                   12/01/07
CLOSED http://bugs.python.org/issue1535    created  georg.brandl             
       py3k, patch                                                             

pickle's documentation is severely outdated                      12/01/07
       http://bugs.python.org/issue1536    created  alexandre.vassalotti     
                                                                               

Change GeneratorExit's base class from Exception to BaseExceptio 12/02/07
CLOSED http://bugs.python.org/issue1537    created  aegis                    
       patch                                                                   

Avoid string copy when split char doesn't match                  12/02/07
       http://bugs.python.org/issue1538    created  skip.montanaro           
       patch                                                                   

test_collections: failing refleak test                           12/02/07
CLOSED http://bugs.python.org/issue1539    created  tiran                    
       py3k                                                                    

Refleak tests: test_doctest and test_gc are failing              12/02/07
       http://bugs.python.org/issue1540    created  tiran                    
       py3k                                                                    

Bad OOB data management when using asyncore with select.poll()   12/02/07
       http://bugs.python.org/issue1541    created  billiejoex               
                                                                               

Ship 32 and 64bit libs with MSI installer                        12/02/07
       http://bugs.python.org/issue1542    created  tiran                    
                                                                               

MSI installer needs to be updated to install x86 and x64 version 12/02/07
CLOSED http://bugs.python.org/issue1543    created  Dude-X                   
                                                                               

IDLE installation problems and no message errors                 12/03/07
       http://bugs.python.org/issue1544    created  danicyber                
                                                                               

shutil fails when copying to NTFS in Linux                       12/03/07
       http://bugs.python.org/issue1545    reopened gvanrossum               
       patch                                                                   

Win32 Platform SDK conflict                                      12/03/07
CLOSED http://bugs.python.org/issue1546    created  adal                     
                                                                               

Minor typos in whatsnew26                                        12/03/07
CLOSED http://bugs.python.org/issue1547    created  tim.golden               
       patch                                                                   

Tiny typo in doc\using\cmdline.rst                               12/03/07
CLOSED http://bugs.python.org/issue1548    created  tim.golden               
       patch                                                                   

Regression with __hash__ definition and rich comparison operator 12/03/07
       http://bugs.python.org/issue1549    created  therve                   
                                                                               

help('modules') broken by several 3rd party libraries (svn patch 12/03/07
       http://bugs.python.org/issue1550    created  benjhayden               
                                                                               

Port Python/import.c to py3k branch                              12/03/07
CLOSED http://bugs.python.org/issue1551    created  tiran                    
       py3k                                                                    

fromfd() and socketpair() should return wrapped sockets          12/04/07
       http://bugs.python.org/issue1552    created  ts1                      
                                                                               

An errornous __length_hint__ can make list() raise a SystemError 12/04/07
CLOSED http://bugs.python.org/issue1553    created  alexandre.vassalotti     
                                                                               

[patch] socketmodule cleanups: allow the use of keywords in sock 12/04/07
       http://bugs.python.org/issue1554    created  therve                   
                                                                               

Print-media stylesheet for sphinx docs incomplete                12/04/07
CLOSED http://bugs.python.org/issue1555    created  tim.golden               
                                                                               

Failure when calling __str__ for MIMEBase(message, rfc822) objec 12/05/07
       http://bugs.python.org/issue1556    created  hsp                      
                                                                               

import distutils.msvccompiler hangs when the environment is too  12/05/07
CLOSED http://bugs.python.org/issue1557    created  theller                  
       py3k                                                                    

x64 platform doesn't define _WIN64                               12/05/07
CLOSED http://bugs.python.org/issue1558    created  tiran                    
       py3k, 64bit                                                             

round() does not                                                 12/05/07
CLOSED http://bugs.python.org/issue1559    created  travelgirl               
                                                                               

PATCH: Attribute lookup caching                                  12/05/07
       http://bugs.python.org/issue1560    created  ntoronto                 
       patch                                                                   

py3k: test_mailbox fails on Windows                              12/06/07
       http://bugs.python.org/issue1561    created  amaury.forgeotdarc       
                                                                               

Decimal can't be subclassed useful                               12/06/07
       http://bugs.python.org/issue1562    created  poelzi                   
                                                                               

asyncore and asynchat incompatible with Py3k str and bytes       12/06/07
       http://bugs.python.org/issue1563    created  djarb                    
       py3k                                                                    

The set implementation should special-case PyUnicode instead of  12/06/07
       http://bugs.python.org/issue1564    created  gvanrossum               
       py3k                                                                    

round(x,y) doesn't behave as expected, round error               12/07/07
CLOSED http://bugs.python.org/issue1565    created  shlomoa                  
                                                                               

sock_type doesn't have GC although it can contain a PyObject*    12/07/07
CLOSED http://bugs.python.org/issue1566    created  tiran                    
                                                                               

Patch for new API method _PyImport_ImportModuleNoLock(char *name 12/07/07
       http://bugs.python.org/issue1567    created  tiran                    
                                                                               

PATCH: Armin's attribute lookup caching for 3.0                  12/07/07
       http://bugs.python.org/issue1568    created  ntoronto                 
       py3k, patch                                                             



Issues Now Closed (41)
______________________

test_smtplib failures (caused by asyncore)                         96 days
       http://bugs.python.org/issue1067    georg.brandl             
       py3k, patch                                                             

''.find() gives wrong result in Python built with ICC              94 days
       http://bugs.python.org/issue1084    sanders_muc              
                                                                               

Document PySys_* API functions                                     56 days
       http://bugs.python.org/issue1245    georg.brandl             
                                                                               

PyBytes (buffer) .extend method needs to accept any iterable of    49 days
       http://bugs.python.org/issue1283    alexandre.vassalotti     
       py3k, patch                                                             

Compile error on OS X 10.5                                         33 days
       http://bugs.python.org/issue1358    ronaldoussoren           
                                                                               

Py3k's print() flushing problem                                    28 days
       http://bugs.python.org/issue1400    gvanrossum               
       py3k                                                                    

Fix for refleak tests                                              21 days
       http://bugs.python.org/issue1414    tiran                    
       py3k, patch                                                             

Calling base class methods is slow due to __instancecheck__ over   18 days
       http://bugs.python.org/issue1438    tiran                    
       py3k                                                                    

VS2008, quick hack for distutils.msvccompiler                      16 days
       http://bugs.python.org/issue1455    tiran                    
       py3k, patch                                                             

PEP 366 implementation                                             11 days
       http://bugs.python.org/issue1487    ncoghlan                 
       patch                                                                   

Rename __builtins__ to __root_namespace__?                          5 days
       http://bugs.python.org/issue1498    tiran                    
       py3k                                                                    

'without make' documentation build anomaly                          4 days
       http://bugs.python.org/issue1520    georg.brandl             
                                                                               

string.decode() fails on long strings                               1 days
       http://bugs.python.org/issue1521    amaury.forgeotdarc       
                                                                               

Add os.fchmod                                                       0 days
       http://bugs.python.org/issue1528    tiran                    
                                                                               

Error when passing a file object to tarfile.open()                  0 days
       http://bugs.python.org/issue1529    amaury.forgeotdarc       
                                                                               

tarfile.open(fileobj=f) and bad metadata of the first file withi    1 days
       http://bugs.python.org/issue1531    GeorgeNotaras            
                                                                               

sys.maxfloat patch                                                  1 days
       http://bugs.python.org/issue1534    marketdickinson          
       py3k, patch                                                             

Rename __builtin__ to builtins                                      1 days
       http://bugs.python.org/issue1535    georg.brandl             
       py3k, patch                                                             

Change GeneratorExit's base class from Exception to BaseExceptio    2 days
       http://bugs.python.org/issue1537    aegis                    
       patch                                                                   

test_collections: failing refleak test                              3 days
       http://bugs.python.org/issue1539    tiran                    
       py3k                                                                    

MSI installer needs to be updated to install x86 and x64 version    0 days
       http://bugs.python.org/issue1543    loewis                   
                                                                               

Win32 Platform SDK conflict                                         0 days
       http://bugs.python.org/issue1546    tiran                    
                                                                               

Minor typos in whatsnew26                                           0 days
       http://bugs.python.org/issue1547    facundobatista           
       patch                                                                   

Tiny typo in doc\using\cmdline.rst                                  0 days
       http://bugs.python.org/issue1548    georg.brandl             
       patch                                                                   

Port Python/import.c to py3k branch                                 1 days
       http://bugs.python.org/issue1551    tiran                    
       py3k                                                                    

An errornous __length_hint__ can make list() raise a SystemError    1 days
       http://bugs.python.org/issue1553    tiran                    
                                                                               

Print-media stylesheet for sphinx docs incomplete                   0 days
       http://bugs.python.org/issue1555    georg.brandl             
                                                                               

import distutils.msvccompiler hangs when the environment is too     0 days
       http://bugs.python.org/issue1557    tiran                    
       py3k                                                                    

x64 platform doesn't define _WIN64                                  0 days
       http://bugs.python.org/issue1558    tiran                    
       py3k, 64bit                                                             

round() does not                                                    0 days
       http://bugs.python.org/issue1559    gvanrossum               
                                                                               

round(x,y) doesn't behave as expected, round error                  0 days
       http://bugs.python.org/issue1565    amaury.forgeotdarc       
                                                                               

sock_type doesn't have GC although it can contain a PyObject*       0 days
       http://bugs.python.org/issue1566    tiran                    
                                                                               

Write 'Using Python on Platform X' documents                     2246 days
       http://bugs.python.org/issue469773  georg.brandl             
                                                                               

cPickle.Pickler: in list mode, no way to set protocol            1319 days
       http://bugs.python.org/issue939395  georg.brandl             
                                                                               

socket intial recv() latency                                      818 days
       http://bugs.python.org/issue1285940 arigo                    
                                                                               

add os.chflags() and os.lchflags() where available                566 days
       http://bugs.python.org/issue1490190 loewis                   
       patch                                                                   

Absolute/relative import not working?                             530 days
       http://bugs.python.org/issue1510172 ncoghlan                 
                                                                               

String methods don't support explicit None arguments              465 days
       http://bugs.python.org/issue1546585 ncoghlan                 
                                                                               

Py_signal_pipe                                                    439 days
       http://bugs.python.org/issue1564547 gustavo                  
       patch                                                                   

No docs for PyEval_EvalCode and related functions                 200 days
       http://bugs.python.org/issue1719933 georg.brandl             
                                                                               

64/32-bit issue when unpickling random.Random                     188 days
       http://bugs.python.org/issue1727780 loewis                   
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 26 shutil fails when copying to NTFS in Linux                         4 days
open    http://bugs.python.org/issue1545   

 14 SSL tests leak memory                                             18 days
open    http://bugs.python.org/issue1469   

 12 'without make' documentation build anomaly                         4 days
closed  http://bugs.python.org/issue1520   

 10 Regression with __hash__ definition and rich comparison operato    4 days
open    http://bugs.python.org/issue1549   

 10 Change GeneratorExit's base class from Exception to BaseExcepti    2 days
closed  http://bugs.python.org/issue1537   

  9 Add 2to3 fixer for (un)bound methods                              10 days
open    http://bugs.python.org/issue1504   

  8 sys.maxfloat patch                                                 1 days
closed  http://bugs.python.org/issue1534   

  8 missing constants in socket module                                 9 days
open    http://bugs.python.org/issue1514   

  8 PyBytes (buffer) .extend method needs to accept any iterable of   49 days
closed  http://bugs.python.org/issue1283   

  6 Rename __builtin__ to builtins                                     1 days
closed  http://bugs.python.org/issue1535   




From lists at cheimes.de  Fri Dec  7 19:43:13 2007
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 07 Dec 2007 19:43:13 +0100
Subject: [Python-Dev] Python 3.0a2 is out
Message-ID: <47599441.5000804@cheimes.de>

A new alpha of Python 3000 was released a few minutes ago!

http://www.python.org/download/releases/3.0/

Have fun and don't forget to report bugs at http://bugs.python.org/

Christian

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 252 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20071207/cb62754f/attachment.pgp 

From titus at caltech.edu  Fri Dec  7 19:43:28 2007
From: titus at caltech.edu (Titus Brown)
Date: Fri, 7 Dec 2007 10:43:28 -0800
Subject: [Python-Dev] Split up the c-api documentation in smaller files?
In-Reply-To: <47593B64.7050704@holdenweb.com>
References: <fjbbgj$s6n$1@ger.gmane.org> <47593B64.7050704@holdenweb.com>
Message-ID: <20071207184328.GE22597@caltech.edu>

On Fri, Dec 07, 2007 at 07:24:04AM -0500, Steve Holden wrote:
-> Christian Heimes wrote:
-> > Good afternoon everybody!
-> > 
-> > The new C API documentation contains some large files:
-> > 
-> > 105K abstract.html
-> > 300K concrete.html
-> > 183K newtypes.html
-> > 
-> > The concrete.html takes noticeable time to render on my computer (P4 2.4
-> > with 1GB RAM, Firefox 2.0 and Ubuntu Linux). I estimate the load and
-> > rendering time is about 4-5 seconds! It also takes long to search for a
-> > string and scrolling isn't smooth, too.
-> > 
-> > Can we split the files in smaller chunks, please?
-> > 
-> This might make a good GHOP project, though as Titus has just uploaded 
-> about seventy (70!) new projects I am unsure as to how long it would 
-> have to wait ti get in the queue.

At the current rate, approximately 10 days.

;)

--titus

From facundobatista at gmail.com  Fri Dec  7 21:27:05 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 7 Dec 2007 17:27:05 -0300
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <e04bdf310711011213u68d88d35t96b7e9c330fd9c53@mail.gmail.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>
	<e04bdf310709191341l289c034fjb4bf6d6b88ae4ef9@mail.gmail.com>
	<46F1B6FD.70007@ronadam.com>
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>
	<471F7881.1070605@ronadam.com>
	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>
	<4720E7FF.7080404@ronadam.com>
	<e04bdf310711011213u68d88d35t96b7e9c330fd9c53@mail.gmail.com>
Message-ID: <e04bdf310712071227w12dd74e2l2b14809160e780f6@mail.gmail.com>

2007/11/1, Facundo Batista <facundobatista at gmail.com>:

> > I think the keyword and keywords interface can be improved.  Do you have
> > any plans in that direction?
>
> Surely!
>
> But, no, I have no plans to do it, as I can not make cgi scripts in my
> hosting, so these pages are statics, generated every night, and that's
> all...

Well, after my hosting allowing CGI, I now improved *a lot* the
interface of this page.

Now you have more columns:

- Id
- Summary
- Priority
- Severity
- Components
- Versions
- Keywords
- Opened by (when)
- Temporal location
- Last update by (when)

And, the biggest enhancement, you can filter by any combination of:

- Priority
- Severity
- Component
- Version
- Keyword

As before, you have everything paged, and with a graph of activity per
day at the bottom.

Enjoy it!:

    http://www.taniquetil.com.ar/facundo/py_tickets.html

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From g.brandl at gmx.net  Fri Dec  7 22:04:38 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 07 Dec 2007 22:04:38 +0100
Subject: [Python-Dev] Bug Day in January?
Message-ID: <fjcch2$mlt$1@ger.gmane.org>

Hi,

there wasn't much response to the bug day proposal, but I still think it's
a good idea. I'd propose a date in January, when Christmas etc. is over.
It would also be nice if someone did the organizing who hasn't got a
daily batch of GHOP tasks to review and commit :)

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From glyph at divmod.com  Fri Dec  7 22:35:19 2007
From: glyph at divmod.com (glyph at divmod.com)
Date: Fri, 07 Dec 2007 21:35:19 -0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
Message-ID: <20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>


On 02:48 pm, gjcarneiro at gmail.com wrote:
>Not only that, but current python signal handling is not theorethically
>async safe; there are race conditions in the Py_AddPendingCalls API, 
>and it
>just happens to work most of the time.

Twisted has encountered one such issue, described here:

    http://twistedmatrix.com/trac/ticket/1997#comment:12

Unfortunately, I don't know enough about signals to suggest or comment 
on the solution.  Any Python/C wrapper around a syscall which can be 
interrupted needs to somehow atomically check for the presence of 
pending python signal handlers; I don't know of any POSIX API to do 
that.

From rhamph at gmail.com  Sat Dec  8 00:27:56 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 7 Dec 2007 16:27:56 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
Message-ID: <aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>

On Dec 7, 2007 2:35 PM,  <glyph at divmod.com> wrote:
>
> On 02:48 pm, gjcarneiro at gmail.com wrote:
> >Not only that, but current python signal handling is not theorethically
> >async safe; there are race conditions in the Py_AddPendingCalls API,
> >and it
> >just happens to work most of the time.

[This refers to the internal datastructures used by
Py_AddPendingCalls, which aren't updated in a safe way.
Hard/impossible to fix in C, but fairly easy with embedded assembly.]


> Twisted has encountered one such issue, described here:
>
>     http://twistedmatrix.com/trac/ticket/1997#comment:12

[This refers to the overall design, which is inherently racey.]


> Unfortunately, I don't know enough about signals to suggest or comment
> on the solution.  Any Python/C wrapper around a syscall which can be
> interrupted needs to somehow atomically check for the presence of
> pending python signal handlers; I don't know of any POSIX API to do
> that.

Overall, what you'd need to do is register a wakeup function (to be
called by a signal handler or another thread), and have that wakeup
function cancel whatever you're doing.  The hard part is it needs to
work at *ANY* time while it's registered, before you've even called
the library function or syscall you intend to cancel!

I currently know of two methods of achieving this:
1) If reading a file or socket, first poll the fd, then do a
non-blocking read.  The wakeup function writes to a wakeup pipe you
also poll, which then wakes you up.  A wakeup after poll completes is
ignored, but the non-blocking read will finish promptly enough anyway.
2) Use sigsetjmp before a syscall (I wouldn't trust a library call),
then have the signal handler jump completely out of the operation.
This is evil and unportable, but probably works.

Additionally, this only gets SIGINT with the default behaviour to work
right, as it can be entirely implemented in C.  If you want to handle
arbitrary signals running arbitrary python code you really need a
second thread to run them in.

-- 
Adam Olsen, aka Rhamphoryncus

From gnewsg at gmail.com  Sat Dec  8 03:47:20 2007
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Fri, 7 Dec 2007 18:47:20 -0800 (PST)
Subject: [Python-Dev] Bug Day in January?
In-Reply-To: <fjcch2$mlt$1@ger.gmane.org>
References: <fjcch2$mlt$1@ger.gmane.org>
Message-ID: <0fac6cf4-e45e-4d26-8e8d-30fe017c646d@e6g2000prf.googlegroups.com>

O.T. - I noticed that the PEP-3 still refers to the old sourceforge
bug tracker. Shouldn't it be rewritten?

From alexandre at peadrop.com  Sat Dec  8 05:35:14 2007
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Fri, 7 Dec 2007 23:35:14 -0500
Subject: [Python-Dev] Subversion forbidden error while committing to trunk
Message-ID: <acd65fa20712072035p102369e7ob5bd445acc997bec@mail.gmail.com>

Hi,

I tried a few times to commit a patch (for issue #1530) to the trunk,
but I always get this error:

  alex:python% svn commit Lib/doctest.py --file svn-commit.tmp
  svn: Commit failed (details follow):
  svn: MKACTIVITY of
'/projects/!svn/act/53683b5b-99d8-497e-bc98-6d07f9401f50': 403
Forbidden (http://svn.python.org)

I first thought that was related to the Py3k freeze. However, I tried
again a few minutes ago and I still got this error. Is possible that
my commit rights are limited to the py3k branches? Or this is a
genuine error?

Thanks,
--  Alexandre

From guido at python.org  Sat Dec  8 05:40:10 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 7 Dec 2007 20:40:10 -0800
Subject: [Python-Dev] Subversion forbidden error while committing to
	trunk
In-Reply-To: <acd65fa20712072035p102369e7ob5bd445acc997bec@mail.gmail.com>
References: <acd65fa20712072035p102369e7ob5bd445acc997bec@mail.gmail.com>
Message-ID: <ca471dc20712072040i65ad099dp9f39a57e6900e384@mail.gmail.com>

On Dec 7, 2007 8:35 PM, Alexandre Vassalotti <alexandre at peadrop.com> wrote:
> I tried a few times to commit a patch (for issue #1530) to the trunk,
> but I always get this error:
>
>   alex:python% svn commit Lib/doctest.py --file svn-commit.tmp
>   svn: Commit failed (details follow):
>   svn: MKACTIVITY of
> '/projects/!svn/act/53683b5b-99d8-497e-bc98-6d07f9401f50': 403
> Forbidden (http://svn.python.org)
>
> I first thought that was related to the Py3k freeze. However, I tried
> again a few minutes ago and I still got this error. Is possible that
> my commit rights are limited to the py3k branches? Or this is a
> genuine error?

I just successfully committed something to the trunk, so the server is
not screwed.

I'm not aware of an access control mechanism that would prevent anyone
from checking in to the trunk while allowing them to check in to a
branch.

I suspect your workspace may be corrupt.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From alexandre at peadrop.com  Sat Dec  8 05:45:39 2007
From: alexandre at peadrop.com (Alexandre Vassalotti)
Date: Fri, 7 Dec 2007 23:45:39 -0500
Subject: [Python-Dev] Subversion forbidden error while committing to
	trunk
In-Reply-To: <ca471dc20712072040i65ad099dp9f39a57e6900e384@mail.gmail.com>
References: <acd65fa20712072035p102369e7ob5bd445acc997bec@mail.gmail.com>
	<ca471dc20712072040i65ad099dp9f39a57e6900e384@mail.gmail.com>
Message-ID: <acd65fa20712072045t13d25262qfd755270ad59563d@mail.gmail.com>

Thanks Guido.

I just found what was the problem. My checkout of the trunk was the
read-only one (i.e., over http).

-- Alexandre

On Dec 7, 2007 11:40 PM, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 7, 2007 8:35 PM, Alexandre Vassalotti <alexandre at peadrop.com> wrote:
> > I tried a few times to commit a patch (for issue #1530) to the trunk,
> > but I always get this error:
> >
> >   alex:python% svn commit Lib/doctest.py --file svn-commit.tmp
> >   svn: Commit failed (details follow):
> >   svn: MKACTIVITY of
> > '/projects/!svn/act/53683b5b-99d8-497e-bc98-6d07f9401f50': 403
> > Forbidden (http://svn.python.org)
> >
> > I first thought that was related to the Py3k freeze. However, I tried
> > again a few minutes ago and I still got this error. Is possible that
> > my commit rights are limited to the py3k branches? Or this is a
> > genuine error?
>
> I just successfully committed something to the trunk, so the server is
> not screwed.
>
> I'm not aware of an access control mechanism that would prevent anyone
> from checking in to the trunk while allowing them to check in to a
> branch.
>
> I suspect your workspace may be corrupt.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>

From rrr at ronadam.com  Sat Dec  8 10:54:28 2007
From: rrr at ronadam.com (Ron Adam)
Date: Sat, 08 Dec 2007 03:54:28 -0600
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <e04bdf310712071227w12dd74e2l2b14809160e780f6@mail.gmail.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>	
	<e04bdf310709191341l289c034fjb4bf6d6b88ae4ef9@mail.gmail.com>	
	<46F1B6FD.70007@ronadam.com>	
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>	
	<471F7881.1070605@ronadam.com>	
	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>	
	<4720E7FF.7080404@ronadam.com>	
	<e04bdf310711011213u68d88d35t96b7e9c330fd9c53@mail.gmail.com>
	<e04bdf310712071227w12dd74e2l2b14809160e780f6@mail.gmail.com>
Message-ID: <475A69D4.9070204@ronadam.com>



Facundo Batista wrote:
> 2007/11/1, Facundo Batista <facundobatista at gmail.com>:
> 
>>> I think the keyword and keywords interface can be improved.  Do you have
>>> any plans in that direction?
>> Surely!
>>
>> But, no, I have no plans to do it, as I can not make cgi scripts in my
>> hosting, so these pages are statics, generated every night, and that's
>> all...
> 
> Well, after my hosting allowing CGI, I now improved *a lot* the
> interface of this page.

Looks much improved!  :-)



> Now you have more columns:
> 
> - Id
> - Summary
> - Priority
> - Severity
> - Components
> - Versions
> - Keywords
> - Opened by (when)
> - Temporal location
> - Last update by (when)
> 
> And, the biggest enhancement, you can filter by any combination of:
> 
> - Priority
> - Severity
> - Component
> - Version
> - Keyword


Maybe components and keywords could be combined together and use check 
boxes so more than one item at a time can be selected?




The following suggestions will probably need changes to the data base. 
Maybe this be done at some point in the future.


Ideally the temporal bar could be more like a mini gant/status chart which 
indicates the status as well as th activity.  Maybe the background color 
can change when the status changes.

The possible status sequences might be something like the following.

Bugs...

     1. bug > bug_confirmed > fix_provided > fix_accepted
          > fix_applied > closed

     2. bug > invalid > closed


Features...

     3. feature > patch_provided > patch_accepted > patch_applied
	> closed

     4. feature > patch_provided > rejected > closed

     5. feature > rejected > closed


Filtering on the above status keywords would give good results I think.


The fix-provided and patch_provided status items might be split into tests, 
docs, and patch/fix provided.  But that may not be needed.

Cheers,
    Ron



> As before, you have everything paged, and with a graph of activity per
> day at the bottom.
> 
> Enjoy it!:
> 
>     http://www.taniquetil.com.ar/facundo/py_tickets.html
> 
> Regards,




From guido at python.org  Sat Dec  8 11:01:58 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 02:01:58 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
Message-ID: <ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>

Adam, perhaps at some point (Monday?) we could get together on
#python-dev and interact in real time on this issue. Probably even
better on the phone. This offer is open to anyone who is serious about
getting this resolved. Someone please take it -- I'm offering free
consulting here!

I'm curious -- is there anyone here who understands why [Py]GTK is
using signals anyway? It's not like writing robust signal handling
code in C is at all easy or obvious. If instead of a signal a file
descriptor could be used, all problems would likely be gone.

--Guido

On Dec 7, 2007 3:27 PM, Adam Olsen <rhamph at gmail.com> wrote:
> On Dec 7, 2007 2:35 PM,  <glyph at divmod.com> wrote:
> >
> > On 02:48 pm, gjcarneiro at gmail.com wrote:
> > >Not only that, but current python signal handling is not theorethically
> > >async safe; there are race conditions in the Py_AddPendingCalls API,
> > >and it
> > >just happens to work most of the time.
>
> [This refers to the internal datastructures used by
> Py_AddPendingCalls, which aren't updated in a safe way.
> Hard/impossible to fix in C, but fairly easy with embedded assembly.]
>
>
> > Twisted has encountered one such issue, described here:
> >
> >     http://twistedmatrix.com/trac/ticket/1997#comment:12
>
> [This refers to the overall design, which is inherently racey.]
>
>
> > Unfortunately, I don't know enough about signals to suggest or comment
> > on the solution.  Any Python/C wrapper around a syscall which can be
> > interrupted needs to somehow atomically check for the presence of
> > pending python signal handlers; I don't know of any POSIX API to do
> > that.
>
> Overall, what you'd need to do is register a wakeup function (to be
> called by a signal handler or another thread), and have that wakeup
> function cancel whatever you're doing.  The hard part is it needs to
> work at *ANY* time while it's registered, before you've even called
> the library function or syscall you intend to cancel!
>
> I currently know of two methods of achieving this:
> 1) If reading a file or socket, first poll the fd, then do a
> non-blocking read.  The wakeup function writes to a wakeup pipe you
> also poll, which then wakes you up.  A wakeup after poll completes is
> ignored, but the non-blocking read will finish promptly enough anyway.
> 2) Use sigsetjmp before a syscall (I wouldn't trust a library call),
> then have the signal handler jump completely out of the operation.
> This is evil and unportable, but probably works.
>
> Additionally, this only gets SIGINT with the default behaviour to work
> right, as it can be entirely implemented in C.  If you want to handle
> arbitrary signals running arbitrary python code you really need a
> second thread to run them in.
>
> --
> Adam Olsen, aka Rhamphoryncus
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From johan at gnome.org  Sat Dec  8 11:17:29 2007
From: johan at gnome.org (Johan Dahlin)
Date: Sat, 08 Dec 2007 11:17:29 +0100
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>	<20071207142642.GC13636@tummy.com>	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
Message-ID: <475A6F39.4090905@gnome.org>

Guido van Rossum wrote:
> Adam, perhaps at some point (Monday?) we could get together on
> #python-dev and interact in real time on this issue. Probably even
> better on the phone. This offer is open to anyone who is serious about
> getting this resolved. Someone please take it -- I'm offering free
> consulting here!
> 
> I'm curious -- is there anyone here who understands why [Py]GTK is
> using signals anyway? It's not like writing robust signal handling
> code in C is at all easy or obvious. If instead of a signal a file
> descriptor could be used, all problems would likely be gone.

The timeout handler was added for KeyboardInterrupt to be able to work when 
you want to Ctrl-C yourself out of the gtk.main() loop.

Johan


From g.brandl at gmx.net  Sat Dec  8 11:48:42 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Sat, 08 Dec 2007 11:48:42 +0100
Subject: [Python-Dev] Bug Day in January?
In-Reply-To: <0fac6cf4-e45e-4d26-8e8d-30fe017c646d@e6g2000prf.googlegroups.com>
References: <fjcch2$mlt$1@ger.gmane.org>
	<0fac6cf4-e45e-4d26-8e8d-30fe017c646d@e6g2000prf.googlegroups.com>
Message-ID: <fjdsq6$bq5$1@ger.gmane.org>

Giampaolo Rodola' schrieb:
> O.T. - I noticed that the PEP-3 still refers to the old sourceforge
> bug tracker. Shouldn't it be rewritten?

Yes, thanks! I did a minimal change, but still someone could elaborate
it, describing current practices.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From gjcarneiro at gmail.com  Sat Dec  8 12:58:26 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Sat, 8 Dec 2007 11:58:26 +0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <475A6F39.4090905@gnome.org>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
Message-ID: <a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>

On 08/12/2007, Johan Dahlin <johan at gnome.org> wrote:
>
> Guido van Rossum wrote:
> > Adam, perhaps at some point (Monday?) we could get together on
> > #python-dev and interact in real time on this issue. Probably even
> > better on the phone. This offer is open to anyone who is serious about
> > getting this resolved. Someone please take it -- I'm offering free
> > consulting here!
> >
> > I'm curious -- is there anyone here who understands why [Py]GTK is
> > using signals anyway? It's not like writing robust signal handling
> > code in C is at all easy or obvious. If instead of a signal a file
> > descriptor could be used, all problems would likely be gone.
>
> The timeout handler was added for KeyboardInterrupt to be able to work
> when
> you want to Ctrl-C yourself out of the gtk.main() loop.


Not only that, but pygtk is a generic module; who are we to forbid the usage
of signals if python itself allows it?  If we were to ignore signals then
sooner or later someone would come along and shout "hey, signals work just
fine in pure python, so why did pygtk have to break my signals?".

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071208/1738bc7d/attachment.htm 

From skip at pobox.com  Sat Dec  8 14:38:40 2007
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 8 Dec 2007 07:38:40 -0600
Subject: [Python-Dev] Slow tests involving bsddb - timeout
In-Reply-To: <18263.26599.386184.603421@montanaro.dyndns.org>
References: <18260.43622.615423.63804@montanaro.dyndns.org>
	<52dc1c820712051724l605893cdq14746a8585138538@mail.gmail.com>
	<ca471dc20712051730q320377e9i98ab4a2acdad8956@mail.gmail.com>
	<18263.26599.386184.603421@montanaro.dyndns.org>
Message-ID: <18266.40544.732146.183061@montanaro.dyndns.org>


    Guido> I think I've seen this too when running the bsddb3 unittest. I
    Guido> think it's caused by a previous test ending badly and leaving
    Guido> junk behind that the test suite doesn't properly remove before
    Guido> starting. But I don't recall the details.

    skip> Thanks, that at least gives me some hope that I can try to narrow
    skip> down the problem with a little binary searching.

The binary search wasn't very difficult.  Ran regrtest with the -v and -f
flags.   test_anydbm was the only test.  Here's the output:

    % ./python.exe -E -tt ../Lib/test/regrtest.py -f tests -v -uall
    test_anydbm
    test_anydbm_creation (test.test_anydbm.AnyDBMTestCase) ... ERROR
    test_anydbm_keys (test.test_anydbm.AnyDBMTestCase) ... ERROR
    test_anydbm_modification (test.test_anydbm.AnyDBMTestCase) ... ERROR
    test_anydbm_read (test.test_anydbm.AnyDBMTestCase) ... ERROR

    ======================================================================
    ERROR: test_anydbm_creation (test.test_anydbm.AnyDBMTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_anydbm.py", line 37, in test_anydbm_creation
        f = anydbm.open(_fname, 'c')
      File "/Users/skip/src/python/trunk/Lib/anydbm.py", line 83, in open
        return mod.open(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/dbhash.py", line 16, in open
        return bsddb.hashopen(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/bsddb/__init__.py", line 310, in hashopen
        d.open(file, db.DB_HASH, flags, mode)
    DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit (100) exceeded')

    ======================================================================
    ERROR: test_anydbm_keys (test.test_anydbm.AnyDBMTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_anydbm.py", line 58, in test_anydbm_keys
        self.init_db()
      File "/Users/skip/src/python/trunk/Lib/test/test_anydbm.py", line 69, in init_db
        f = anydbm.open(_fname, 'n')
      File "/Users/skip/src/python/trunk/Lib/anydbm.py", line 83, in open
        return mod.open(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/dbhash.py", line 16, in open
        return bsddb.hashopen(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/bsddb/__init__.py", line 310, in hashopen
        d.open(file, db.DB_HASH, flags, mode)
    DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit (100) exceeded')

    ======================================================================
    ERROR: test_anydbm_modification (test.test_anydbm.AnyDBMTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_anydbm.py", line 45, in test_anydbm_modification
        self.init_db()
      File "/Users/skip/src/python/trunk/Lib/test/test_anydbm.py", line 69, in init_db
        f = anydbm.open(_fname, 'n')
      File "/Users/skip/src/python/trunk/Lib/anydbm.py", line 83, in open
        return mod.open(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/dbhash.py", line 16, in open
        return bsddb.hashopen(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/bsddb/__init__.py", line 310, in hashopen
        d.open(file, db.DB_HASH, flags, mode)
    DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit (100) exceeded')

    ======================================================================
    ERROR: test_anydbm_read (test.test_anydbm.AnyDBMTestCase)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/skip/src/python/trunk/Lib/test/test_anydbm.py", line 52, in test_anydbm_read
        self.init_db()
      File "/Users/skip/src/python/trunk/Lib/test/test_anydbm.py", line 69, in init_db
        f = anydbm.open(_fname, 'n')
      File "/Users/skip/src/python/trunk/Lib/anydbm.py", line 83, in open
        return mod.open(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/dbhash.py", line 16, in open
        return bsddb.hashopen(file, flag, mode)
      File "/Users/skip/src/python/trunk/Lib/bsddb/__init__.py", line 310, in hashopen
        d.open(file, db.DB_HASH, flags, mode)
    DBFileExistsError: (17, 'File exists -- __fop_file_setup:  Retry limit (100) exceeded')

    ----------------------------------------------------------------------
    Ran 4 tests in 404.353s

    FAILED (errors=4)
    test test_anydbm failed -- errors occurred; run in verbose mode for details
    1 test failed:
        test_anydbm

Skip

From guido at python.org  Sat Dec  8 18:06:23 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 09:06:23 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <475A6F39.4090905@gnome.org>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
Message-ID: <ca471dc20712080906g23f43363g1f727415370a95e6@mail.gmail.com>

On Dec 8, 2007 2:17 AM, Johan Dahlin <johan at gnome.org> wrote:
> Guido van Rossum wrote:
> > Adam, perhaps at some point (Monday?) we could get together on
> > #python-dev and interact in real time on this issue. Probably even
> > better on the phone. This offer is open to anyone who is serious about
> > getting this resolved. Someone please take it -- I'm offering free
> > consulting here!
> >
> > I'm curious -- is there anyone here who understands why [Py]GTK is
> > using signals anyway? It's not like writing robust signal handling
> > code in C is at all easy or obvious. If instead of a signal a file
> > descriptor could be used, all problems would likely be gone.
>
> The timeout handler was added for KeyboardInterrupt to be able to work when
> you want to Ctrl-C yourself out of the gtk.main() loop.

Hm. How about making that an option? I don't think on the OLPC XO this
is a valid use case (end users never have a console where they might
enter ^C).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Dec  8 18:20:06 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 09:20:06 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
Message-ID: <ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>

On Dec 8, 2007 3:58 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> Not only that, but pygtk is a generic module;

What does "generic" mean in this context? Surely not that it applies
to other libraries than GTK?

> who are we to forbid the usage
> of signals if python itself allows it?  If we were to ignore signals then
> sooner or later someone would come along and shout "hey, signals work just
> fine in pure python, so why did pygtk have to break my signals?".

Um, signals don't "work just fine" in pure Python. And I would argue
they don't either in C. They are extremely subtle, and most code using
signals is broken in some way. Just try to read the sigaction() man
page and claim you understand it.

Unfortunately, in Unix there are some conditions that can only be
delivered using signals (e.g. SIGINTR, SIGTERM) and others for which
your choices are either polling or signals (SIGCHILD, SIGWINCH).
Traditionally, solutions based on select() or poll() with a short
timeout (e.g. 20 or 100 ms) have worked well, as the number of
instructions executed each time is really negligeable, and the
response time is still reasonable on a human time scale. Unfortunately
it seems recent developments in power management for ultra-low power
devices have made this an issue again.

Windows solves this more elegantly by having a unified "wait for
multiple conditions" system call which can wait on I/O, semaphores,
and other events (within the same process or coming from other
processes). Unfortunately, in Unix, some events don't have a file
descriptor associated with them, and for those select()/poll() cannot
be used.

The best solution I can think of is to add a new API that takes a
signal and a file descriptor and registers a C-level handler for that
signal which writes a byte to the file descriptor. You can then create
a pipe, connect the signal handler to the write end, and add the read
end to your list of file descriptors passed to select() or poll(). The
handler must be written in C in order to avoid the race condition
referred to by Glyph (signals arriving after the signal check in the
VM main loop but before the select()/poll() system call is entered
will not be noticed until the select()/poll() call completes).

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Dec  8 18:25:28 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 09:25:28 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
Message-ID: <ca471dc20712080925g36599034ie27a2ade3f50141f@mail.gmail.com>

On Dec 8, 2007 9:20 AM, Guido van Rossum <guido at python.org> wrote:
> The
> handler must be written in C in order to avoid the race condition
> referred to by Glyph (signals arriving after the signal check in the
> VM main loop but before the select()/poll() system call is entered
> will not be noticed until the select()/poll() call completes).

Sorry, I misattributed this; Gustavo Carneiro pointed this out earlier
in this thread.

BTW, in the referenced post (also by Gustavo), I found this:

"""
According to [1], all python needs to do to avoid this problem is
block all signals in all but the main thread; then we can guarantee
signal handlers are always called from the main thread, and pygtk
doesn't need a timeout.

[1] https://bugzilla.redhat.com/bugzilla/process_bug.cgi#c3
"""

Unfortunately I can't read [1] without having registered, so I can
only guess what it says. But I know that Python already ensures that
signals are only delivered to the main thread. What am I missing?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tonynelson at georgeanelson.com  Sat Dec  8 17:31:56 2007
From: tonynelson at georgeanelson.com (Tony Nelson)
Date: Sat, 8 Dec 2007 11:31:56 -0500
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
Message-ID: <p04330102c38070136b30@[192.168.123.162]>

At 2:01 AM -0800 12/8/07, Guido van Rossum wrote:
 ...
>I'm curious -- is there anyone here who understands why [Py]GTK is
>using signals anyway? It's not like writing robust signal handling
>code in C is at all easy or obvious. If instead of a signal a file
>descriptor could be used, all problems would likely be gone.

I don't think PyGTK does for GTK2 signal emission -- though Johan Dahlin is
authoritative here.  See
<http://www.pygtk.org/pygtk2tutorial/sec-TheoryOfSignalsAndCallbacks.html> .
-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>

From tonynelson at georgeanelson.com  Sat Dec  8 17:28:08 2007
From: tonynelson at georgeanelson.com (Tony Nelson)
Date: Sat, 8 Dec 2007 11:28:08 -0500
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <475A6F39.4090905@gnome.org>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
Message-ID: <p04330103c380709c8b5d@[192.168.123.162]>

At 11:17 AM +0100 12/8/07, Johan Dahlin wrote:
>Guido van Rossum wrote:
>> Adam, perhaps at some point (Monday?) we could get together on
>> #python-dev and interact in real time on this issue. Probably even
>> better on the phone. This offer is open to anyone who is serious about
>> getting this resolved. Someone please take it -- I'm offering free
>> consulting here!
>>
>> I'm curious -- is there anyone here who understands why [Py]GTK is
>> using signals anyway? It's not like writing robust signal handling
>> code in C is at all easy or obvious. If instead of a signal a file
>> descriptor could be used, all problems would likely be gone.
>
>The timeout handler was added for KeyboardInterrupt to be able to work when
>you want to Ctrl-C yourself out of the gtk.main() loop.

Is that always required (with threads), or are things better now that
Ctrl-C handling is improved (at least in the Socket module, which doesn't
lose signals anymore)?
-- 
____________________________________________________________________
TonyN.:'                       <mailto:tonynelson at georgeanelson.com>
      '                              <http://www.georgeanelson.com/>

From gjcarneiro at gmail.com  Sat Dec  8 18:57:57 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Sat, 8 Dec 2007 17:57:57 +0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
Message-ID: <a467ca4f0712080957k51f6f6cs5e3be19149148871@mail.gmail.com>

On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 8, 2007 3:58 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > Not only that, but pygtk is a generic module;
>
> What does "generic" mean in this context? Surely not that it applies
> to other libraries than GTK?


Well, actually this is also a PyGObject issue, not only PyGtk.  PyGObject
wraps GLib.  GLib was created to serve the needs of Gtk+, but is useful by
itself for writing portable programs.  Among other things, GLib offers a
generic "main loop", which programs can use to do generic event-driven
programming, such as timeouts, polling file descriptors, and doing other
work when the main loop becomes idle.

You could argue that non-gtk programs are rare and we shouldn't worry too
much about them.  Maybe it's true, I don't know.

> who are we to forbid the usage
> > of signals if python itself allows it?  If we were to ignore signals
> then
> > sooner or later someone would come along and shout "hey, signals work
> just
> > fine in pure python, so why did pygtk have to break my signals?".
>
> Um, signals don't "work just fine" in pure Python. And I would argue
> they don't either in C. They are extremely subtle, and most code using
> signals is broken in some way. Just try to read the sigaction() man
> page and claim you understand it.
>
> Unfortunately, in Unix there are some conditions that can only be
> delivered using signals (e.g. SIGINTR, SIGTERM) and others for which
> your choices are either polling or signals (SIGCHILD, SIGWINCH).
> Traditionally, solutions based on select() or poll() with a short
> timeout (e.g. 20 or 100 ms) have worked well, as the number of
> instructions executed each time is really negligeable, and the
> response time is still reasonable on a human time scale. Unfortunately
> it seems recent developments in power management for ultra-low power
> devices have made this an issue again.
>
> Windows solves this more elegantly by having a unified "wait for
> multiple conditions" system call which can wait on I/O, semaphores,
> and other events (within the same process or coming from other
> processes). Unfortunately, in Unix, some events don't have a file
> descriptor associated with them, and for those select()/poll() cannot
> be used.
>
> The best solution I can think of is to add a new API that takes a
> signal and a file descriptor and registers a C-level handler for that
> signal which writes a byte to the file descriptor. You can then create
> a pipe, connect the signal handler to the write end, and add the read
> end to your list of file descriptors passed to select() or poll(). The
> handler must be written in C in order to avoid the race condition
> referred to by Glyph (signals arriving after the signal check in the
> VM main loop but before the select()/poll() system call is entered
> will not be noticed until the select()/poll() call completes).


Funny that everyone mentions this solution, as it is the solution
implemented by my patch :-)

Well, to be fair, it might not be _exactly_ what is implemented by the
patch.  Reading between the lines, I think what you mean is to have python's
C signal handler mostly untouched; it would only write a byte to a pipe _in
addition to_ the normal setting the flag and Py_AddPendingCall.

The patch I submitted, on the other hand, completely removes the vector of
flags and Py_AddPendingCall, and instead writes the number of the signal
that was raised into the pipe, and reads it back from the pipe in the Python
main loop.

Which is the best solution?  I think my patch fixes two problems: 1. the
need to have a FD to wake up poll() (t o fix the problem with what we are
discussing in this thread), and 2. make Python's signal handling more
reliable (not 100% reliable because it doesn't handle longer bursts of
signals than the pipe buffer can take, but at least is race free).

My solution is being reject because people are afraid to touch the signal
handling code, which has its faults, but well know faults.  But if I
refactor the patch to keep the crappy-but-sort-of-working signal code, and
only enhance it with the pipe, maybe it will more likely get accepted.

Do I understand correctly? :-)

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071208/66fe9718/attachment.htm 

From martin at v.loewis.de  Sat Dec  8 19:09:12 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 08 Dec 2007 19:09:12 +0100
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080925g36599034ie27a2ade3f50141f@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>	<20071207142642.GC13636@tummy.com>	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>	<475A6F39.4090905@gnome.org>	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<ca471dc20712080925g36599034ie27a2ade3f50141f@mail.gmail.com>
Message-ID: <475ADDC8.5090305@v.loewis.de>

> """
> According to [1], all python needs to do to avoid this problem is
> block all signals in all but the main thread; then we can guarantee
> signal handlers are always called from the main thread, and pygtk
> doesn't need a timeout.
> 
> [1] https://bugzilla.redhat.com/bugzilla/process_bug.cgi#c3
> """
> 
> Unfortunately I can't read [1] without having registered, so I can
> only guess what it says. But I know that Python already ensures that
> signals are only delivered to the main thread. What am I missing?

Your notion of "delivers" differs. Python does receive signals in
threads other than the main thread. However, it will only invoke the
*Python* signal handler in the main thread.
signalmodule.c:signal_handler calls Py_AddPendingCall, which is
only ever considered in the main thread (although perhaps not
in a timely manner - this is precisely where the busy-waiting in gtk
comes from).

Python does not attempt to block any signals, let alone using
pthread_sigmask.

Regards,
Martin

From gjcarneiro at gmail.com  Sat Dec  8 19:20:15 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Sat, 8 Dec 2007 18:20:15 +0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080925g36599034ie27a2ade3f50141f@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<ca471dc20712080925g36599034ie27a2ade3f50141f@mail.gmail.com>
Message-ID: <a467ca4f0712081020g6d3efca7l3a926fb132a5057c@mail.gmail.com>

On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 8, 2007 9:20 AM, Guido van Rossum <guido at python.org> wrote:
> > The
> > handler must be written in C in order to avoid the race condition
> > referred to by Glyph (signals arriving after the signal check in the
> > VM main loop but before the select()/poll() system call is entered
> > will not be noticed until the select()/poll() call completes).
>
> Sorry, I misattributed this; Gustavo Carneiro pointed this out earlier
> in this thread.
>
> BTW, in the referenced post (also by Gustavo), I found this:
>
> """
> According to [1], all python needs to do to avoid this problem is
> block all signals in all but the main thread; then we can guarantee
> signal handlers are always called from the main thread, and pygtk
> doesn't need a timeout.
>
> [1] https://bugzilla.redhat.com/bugzilla/process_bug.cgi#c3
> """
>
> Unfortunately I can't read [1] without having registered, so I can
> only guess what it says.


I can't read [1] either because the URL is not correct.  Sorry :P

But I know that Python already ensures that
> signals are only delivered to the main thread.


Are you sure?  In python code I see pthread_sigmask being checked for in
configure.in, but I can't find it being used anywhere (unless I'm grepping
for the wrong thing).

What you probably meant to say was "python ensures that signals are only
processed from the main thread".

Except when python uses "GNU PTH threads"; there I do see signals being
delivered to the main thread.  But GNU pth are not real kernel threads,
anyway.

What am I missing?


I think Python could make sure (via pthread_sigmask) that signals are
blocked for all non-main threads created by Python.  Unfortunately it cannot
do the same for threads not created by Python.  I know at least one case
where threads are created behing Python's back: the GnomeVFS library.

Best regards,

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071208/f512ac6c/attachment.htm 

From ceronman at gmail.com  Sat Dec  8 19:49:54 2007
From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Alejandro_Cer=F3n_Estrada?=)
Date: Sat, 8 Dec 2007 13:49:54 -0500
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
Message-ID: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>

Hello Python developers.

This is my first post on the list. I have been using Python for a
while and I have been thinking about one feature I would like to see
integrated into the language. I thought it could be a good topic for a
PEP, so I decided to join the list and write about it.

First problem: empty generators.

Suppose you have to write an empty generator. You could do something like:

def foo():
    return
    yield 'never'

Or something like:

def foo():
    yield iter([]).next()

Or:

def foo():
    raise StopIteration()
    yield "never"

There is an old thread discussing the diferent alternatives:
http://mail.python.org/pipermail/python-list/2002-November/171588.html

Of curse this is unpythonic. It violates the principle of: "There
should be one-- and preferably only one --obvious way to do it".
Instead, we have a bunch of inelegant solutions, and no one is the
obvious one.

Second problem: Explicit raise without explicit try-except.

Take a look at this example:

def lines():
    for line in my_file:
        if some_error():
            raise StopIteration()
        yield line
    yield 'end'

for line in lines():
    do_something()

Even when the lines function contains an explicit raise statement,
there is no explicit try-except block. Of curse, the StopIteration
exception is implicitly caught when the generator is called, but this
looks a bit confusing. In my opinion, every explicitly raised
exception should be explicitly caught by a try-except block.

The solution: yield break.

The solution used in C# for these problems is the 'yield break'
statement.  In this way, the empty generator would look like:

def foo():
    yield break

This would be the pythonic way of writing an empty generator.

In the same way, every time you want to stop the generation you should
call 'yield break', for example:

def lines():
    for line in my_file:
        if some_error():
            yield break
        yield line
    yield 'end'

Note that 'yield break' resembles the 'break' statement used in loops,
while 'StopIteration' doesn't. 'yield break' is more orthogonal to the
rest of the language.

I am looking forward to seeing your opinions.

Manuel Cer?n.

From python at rcn.com  Sat Dec  8 20:17:28 2007
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 8 Dec 2007 11:17:28 -0800
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
Message-ID: <002c01c839ce$fa13d8b0$9200a8c0@RaymondLaptop1>

> Note that 'yield break' resembles the 'break' statement used in loops,
> while 'StopIteration' doesn't. 'yield break' is more orthogonal to the
> rest of the language.
>
> I am looking forward to seeing your opinions.

-1

I do not find the meaning to be transparent and the proposal adds new syntax without adding functionality.

Also, I do not find the "emtpy generator" use case to be even slightly motivating.  If an empty iterator were needed for some 
reason, I would just write: iter([]) and be done with it.


Raymond 

From guido at python.org  Sat Dec  8 20:31:03 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 11:31:03 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <475ADA8F.600@gnome.org>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<ca471dc20712080906g23f43363g1f727415370a95e6@mail.gmail.com>
	<475ADA8F.600@gnome.org>
Message-ID: <ca471dc20712081131o540d9df6g7b6a1b419f3faa4e@mail.gmail.com>

On Dec 8, 2007 9:55 AM, Johan Dahlin <johan at gnome.org> wrote:
> Guido van Rossum wrote:
> > On Dec 8, 2007 2:17 AM, Johan Dahlin <johan at gnome.org> wrote:
> >> Guido van Rossum wrote:
> >>> Adam, perhaps at some point (Monday?) we could get together on
> >>> #python-dev and interact in real time on this issue. Probably even
> >>> better on the phone. This offer is open to anyone who is serious about
> >>> getting this resolved. Someone please take it -- I'm offering free
> >>> consulting here!
> >>>
> >>> I'm curious -- is there anyone here who understands why [Py]GTK is
> >>> using signals anyway? It's not like writing robust signal handling
> >>> code in C is at all easy or obvious. If instead of a signal a file
> >>> descriptor could be used, all problems would likely be gone.
> >> The timeout handler was added for KeyboardInterrupt to be able to work when
> >> you want to Ctrl-C yourself out of the gtk.main() loop.
> >
> > Hm. How about making that an option? I don't think on the OLPC XO this
> > is a valid use case (end users never have a console where they might
> > enter ^C).
> >
>
> It could easily be made into a compilation option which would solve the
> problem specifically for OLPC, but it would still be problematic for other
> platforms important to PyGTK (linux/gnome) where console based development
> is more common.

But do those other platforms care about the extra CPU cycles and power
used? I suspect not, at least not to the extent that OLPC cares.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Dec  8 20:43:56 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 11:43:56 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <a467ca4f0712080957k51f6f6cs5e3be19149148871@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<a467ca4f0712080957k51f6f6cs5e3be19149148871@mail.gmail.com>
Message-ID: <ca471dc20712081143lcc92e50m1ffa23aebeaf402a@mail.gmail.com>

On Dec 8, 2007 9:57 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> > On Dec 8, 2007 3:58 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > > Not only that, but pygtk is a generic module;
> >
> > What does "generic" mean in this context? Surely not that it applies
> > to other libraries than GTK?
>
> Well, actually this is also a PyGObject issue, not only PyGtk.  PyGObject
> wraps GLib.  GLib was created to serve the needs of Gtk+, but is useful by
> itself for writing portable programs.  Among other things, GLib offers a
> generic "main loop", which programs can use to do generic event-driven
> programming, such as timeouts, polling file descriptors, and doing other
> work when the main loop becomes idle.
>
> You could argue that non-gtk programs are rare and we shouldn't worry too
> much about them.  Maybe it's true, I don't know.
>
>
> > > who are we to forbid the usage
> > > of signals if python itself allows it?  If we were to ignore signals
> then
> > > sooner or later someone would come along and shout "hey, signals work
> just
> > > fine in pure python, so why did pygtk have to break my signals?".
> >
> > Um, signals don't "work just fine" in pure Python. And I would argue
> > they don't either in C. They are extremely subtle, and most code using
> > signals is broken in some way. Just try to read the sigaction() man
> > page and claim you understand it.
> >
> > Unfortunately, in Unix there are some conditions that can only be
> > delivered using signals (e.g. SIGINTR, SIGTERM) and others for which
> > your choices are either polling or signals (SIGCHILD, SIGWINCH).
> > Traditionally, solutions based on select() or poll() with a short
> > timeout (e.g. 20 or 100 ms) have worked well, as the number of
> > instructions executed each time is really negligeable, and the
> > response time is still reasonable on a human time scale. Unfortunately
> > it seems recent developments in power management for ultra-low power
> > devices have made this an issue again.
> >
> > Windows solves this more elegantly by having a unified "wait for
> > multiple conditions" system call which can wait on I/O, semaphores,
> > and other events (within the same process or coming from other
> > processes). Unfortunately, in Unix, some events don't have a file
> > descriptor associated with them, and for those select()/poll() cannot
> > be used.
> >
> > The best solution I can think of is to add a new API that takes a
> > signal and a file descriptor and registers a C-level handler for that
> > signal which writes a byte to the file descriptor. You can then create
> > a pipe, connect the signal handler to the write end, and add the read
> > end to your list of file descriptors passed to select() or poll(). The
> > handler must be written in C in order to avoid the race condition
> > referred to by Glyph (signals arriving after the signal check in the
> > VM main loop but before the select()/poll() system call is entered
> > will not be noticed until the select()/poll() call completes).
>
> Funny that everyone mentions this solution, as it is the solution
> implemented by my patch :-)

Mind pointing me to it again? I can't find it in the Python bug tracker.

> Well, to be fair, it might not be _exactly_ what is implemented by the
> patch.  Reading between the lines, I think what you mean is to have python's
> C signal handler mostly untouched; it would only write a byte to a pipe _in
> addition to_ the normal setting the flag and Py_AddPendingCall.

Well, in many cases I see no problems with the current signal handler,
and people are used to it, so I'm not sure what is gained by getting
rid of it.

> The patch I submitted, on the other hand, completely removes the vector of
> flags and Py_AddPendingCall, and instead writes the number of the signal
> that was raised into the pipe, and reads it back from the pipe in the Python
> main loop.

I believe Py_AddPendingCall was meant to be used by other places
besides the signal handling.

> Which is the best solution?  I think my patch fixes two problems: 1. the
> need to have a FD to wake up poll() (t o fix the problem with what we are
> discussing in this thread), and 2. make Python's signal handling more
> reliable (not 100% reliable because it doesn't handle longer bursts of
> signals than the pipe buffer can take, but at least is race free).

I think it's okay to drop signals if too many come. The FD should be
put in non-blocking mode so the signal handler won't block forever.
Does Unix even promise that a signal gets delivered twice if it gets
sent quickly twice in a row?

> My solution is being reject because people are afraid to touch the signal
> handling code, which has its faults, but well know faults.  But if I
> refactor the patch to keep the crappy-but-sort-of-working signal code, and
> only enhance it with the pipe, maybe it will more likely get accepted.
>
> Do I understand correctly? :-)

Not really; I don't recall seeing your patch. And I object to your
subjective description of the existing signal handling; it has served
us well enough.

I have a worry though -- if signal handlers *always* and *only*
communicate by writing a byte to a FD, does that mean that the VM main
loop will have to attempt to read a byte from a pipe every time it
checks for signals? That sounds very expensive for something that's
not needed by the vast majority of Python applications. Plus, you will
have to expose the FD to the user so that it can be included in
select() and poll() calls.

Anyway, let's see the patch first.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at krypto.org  Sat Dec  8 21:00:48 2007
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 8 Dec 2007 12:00:48 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081131o540d9df6g7b6a1b419f3faa4e@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<ca471dc20712080906g23f43363g1f727415370a95e6@mail.gmail.com>
	<475ADA8F.600@gnome.org>
	<ca471dc20712081131o540d9df6g7b6a1b419f3faa4e@mail.gmail.com>
Message-ID: <52dc1c820712081200n1ac8a8baqf074448c1fd4b3d9@mail.gmail.com>

On 12/8/07, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 8, 2007 9:55 AM, Johan Dahlin <johan at gnome.org> wrote:
> > Guido van Rossum wrote:
>
> > > Hm. How about making that an option? I don't think on the OLPC XO this
> > > is a valid use case (end users never have a console where they might
> > > enter ^C).
> > >
> >
> > It could easily be made into a compilation option which would solve the
> > problem specifically for OLPC, but it would still be problematic for
> other
> > platforms important to PyGTK (linux/gnome) where console based
> development
> > is more common.
>
> But do those other platforms care about the extra CPU cycles and power
> used? I suspect not, at least not to the extent that OLPC cares.


The OLPC project should go ahead with a hackish or otherwise unacceptable to
mainstream fix for their issue while the better solution is worked on.  I
suspect even changing the evil check for signal loop delay to several
seconds would be enough of a hack for them to save power.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071208/143ac540/attachment.htm 

From gjcarneiro at gmail.com  Sat Dec  8 21:38:30 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Sat, 8 Dec 2007 20:38:30 +0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081143lcc92e50m1ffa23aebeaf402a@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<a467ca4f0712080957k51f6f6cs5e3be19149148871@mail.gmail.com>
	<ca471dc20712081143lcc92e50m1ffa23aebeaf402a@mail.gmail.com>
Message-ID: <a467ca4f0712081238h182f92ebj51256305c78620d1@mail.gmail.com>

On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 8, 2007 9:57 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
> >
> > > On Dec 8, 2007 3:58 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > > > Not only that, but pygtk is a generic module;
> > >
> > > What does "generic" mean in this context? Surely not that it applies
> > > to other libraries than GTK?
> >
> > Well, actually this is also a PyGObject issue, not only
> PyGtk.  PyGObject
> > wraps GLib.  GLib was created to serve the needs of Gtk+, but is useful
> by
> > itself for writing portable programs.  Among other things, GLib offers a
> > generic "main loop", which programs can use to do generic event-driven
> > programming, such as timeouts, polling file descriptors, and doing other
> > work when the main loop becomes idle.
> >
> > You could argue that non-gtk programs are rare and we shouldn't worry
> too
> > much about them.  Maybe it's true, I don't know.
> >
> >
> > > > who are we to forbid the usage
> > > > of signals if python itself allows it?  If we were to ignore signals
> > then
> > > > sooner or later someone would come along and shout "hey, signals
> work
> > just
> > > > fine in pure python, so why did pygtk have to break my signals?".
> > >
> > > Um, signals don't "work just fine" in pure Python. And I would argue
> > > they don't either in C. They are extremely subtle, and most code using
> > > signals is broken in some way. Just try to read the sigaction() man
> > > page and claim you understand it.
> > >
> > > Unfortunately, in Unix there are some conditions that can only be
> > > delivered using signals (e.g. SIGINTR, SIGTERM) and others for which
> > > your choices are either polling or signals (SIGCHILD, SIGWINCH).
> > > Traditionally, solutions based on select() or poll() with a short
> > > timeout (e.g. 20 or 100 ms) have worked well, as the number of
> > > instructions executed each time is really negligeable, and the
> > > response time is still reasonable on a human time scale. Unfortunately
> > > it seems recent developments in power management for ultra-low power
> > > devices have made this an issue again.
> > >
> > > Windows solves this more elegantly by having a unified "wait for
> > > multiple conditions" system call which can wait on I/O, semaphores,
> > > and other events (within the same process or coming from other
> > > processes). Unfortunately, in Unix, some events don't have a file
> > > descriptor associated with them, and for those select()/poll() cannot
> > > be used.
> > >
> > > The best solution I can think of is to add a new API that takes a
> > > signal and a file descriptor and registers a C-level handler for that
> > > signal which writes a byte to the file descriptor. You can then create
> > > a pipe, connect the signal handler to the write end, and add the read
> > > end to your list of file descriptors passed to select() or poll(). The
> > > handler must be written in C in order to avoid the race condition
> > > referred to by Glyph (signals arriving after the signal check in the
> > > VM main loop but before the select()/poll() system call is entered
> > > will not be noticed until the select()/poll() call completes).
> >
> > Funny that everyone mentions this solution, as it is the solution
> > implemented by my patch :-)
>
> Mind pointing me to it again? I can't find it in the Python bug tracker.
>
> > Well, to be fair, it might not be _exactly_ what is implemented by the
> > patch.  Reading between the lines, I think what you mean is to have
> python's
> > C signal handler mostly untouched; it would only write a byte to a pipe
> _in
> > addition to_ the normal setting the flag and Py_AddPendingCall.
>
> Well, in many cases I see no problems with the current signal handler,
> and people are used to it, so I'm not sure what is gained by getting
> rid of it.
>
> > The patch I submitted, on the other hand, completely removes the vector
> of
> > flags and Py_AddPendingCall, and instead writes the number of the signal
> > that was raised into the pipe, and reads it back from the pipe in the
> Python
> > main loop.
>
> I believe Py_AddPendingCall was meant to be used by other places
> besides the signal handling.


True.  The patch does not remove Py_AddPendingCall, only stops using it for
signals.

> Which is the best solution?  I think my patch fixes two problems: 1. the
> > need to have a FD to wake up poll() (t o fix the problem with what we
> are
> > discussing in this thread), and 2. make Python's signal handling more
> > reliable (not 100% reliable because it doesn't handle longer bursts of
> > signals than the pipe buffer can take, but at least is race free).
>
> I think it's okay to drop signals if too many come. The FD should be
> put in non-blocking mode so the signal handler won't block forever.
> Does Unix even promise that a signal gets delivered twice if it gets
> sent quickly twice in a row?


Good point.

> My solution is being reject because people are afraid to touch the signal
> > handling code, which has its faults, but well know faults.  But if I
> > refactor the patch to keep the crappy-but-sort-of-working signal code,
> and
> > only enhance it with the pipe, maybe it will more likely get accepted.
> >
> > Do I understand correctly? :-)
>
> Not really; I don't recall seeing your patch. And I object to your
> subjective description of the existing signal handling; it has served
> us well enough.


Sorry.  I have to say existing code is very bad in order to convince people
it is worth changing... :P

Anyway, the approach of writing to a pipe and reading it back later appears
(to me) simpler to read and understand.  That should also count for
something, right?

I have a worry though -- if signal handlers *always* and *only*
> communicate by writing a byte to a FD, does that mean that the VM main
> loop will have to attempt to read a byte from a pipe every time it
> checks for signals? That sounds very expensive for something that's
> not needed by the vast majority of Python applications.


Yes, it would be expensive.  Instead the patch uses a simple global boolean
value, set to true when a signal is received, after writing to the pipe.
PyErr_CheckSignals() does not try to read from the pipe unless that value is
true.

Plus, you will
> have to expose the FD to the user so that it can be included in
> select() and poll() calls.


Correct.

Anyway, let's see the patch first.


http://bugs.python.org/issue1564547

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071208/94a9727d/attachment-0001.htm 

From tjreedy at udel.edu  Sat Dec  8 21:59:55 2007
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 8 Dec 2007 15:59:55 -0500
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
Message-ID: <fjf0ka$d96$1@ger.gmane.org>

I would prefer plain 'yield' to 'yield break' as a synonym for 'raise 
StopIteration'.
But the raise stands out better visually as an exit point.

The point about the raise not being explicitly caught is not valid since 
any generator could be used in the following code

g = some_gen
try:
    while True:
        process(g.next())
except StopIteration:
    whatever()

where it is caught.  Then under your proposal, there would be a catch 
without a throw ;-(.

If there were a use case for empty generators, then iter() could be made to 
produce one, in analogy with int() returning 0, etc.  But without a use 
case, 'iter()' is much more likely to be a bug and should be flagged as 
such.  In the meanwhile, 'iter(())' is trivial to type.

Terry Jan Reedy




From ceronman at gmail.com  Sat Dec  8 22:06:40 2007
From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Alejandro_Cer=F3n_Estrada?=)
Date: Sat, 8 Dec 2007 16:06:40 -0500
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <002c01c839ce$fa13d8b0$9200a8c0@RaymondLaptop1>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<002c01c839ce$fa13d8b0$9200a8c0@RaymondLaptop1>
Message-ID: <796874fc0712081306p164b5858u7b99fe4a4f3ce5b6@mail.gmail.com>

2007/12/8, Raymond Hettinger <python at rcn.com>:
>...the proposal adds new syntax without adding functionality.

That is indeed the definition of syntactic sugar [1]. Python is full
of that, for example the import statement.

2007/12/8, Paul Svensson <paul at svensson.org>:
> What is the problem that is not solved by iter(()) ?

'yield iter(()).next()'  it's obscure, you can't know its purpose at
first sight. There are other alternatives but none of them seems to be
the "obvious way to do it".

But that is the less important problem. The real problem is that
raising StopIteration is not orthogonal. It is confusing for the
reasons I exposed. Generators are something in the middle between a
language feature and a framework/library. With 'yield break' they will
be completely a language feature.

[1] http://en.wikipedia.org/wiki/Syntactic_sugar

Manuel.

From python at rcn.com  Sat Dec  8 22:16:47 2007
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 8 Dec 2007 13:16:47 -0800
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
Message-ID: <005701c839df$a4ff63b0$9200a8c0@RaymondLaptop1>

In your example, why do you "raise StopIteration" instead just writing "return"?

----- Original Message ----- 
From: "Manuel Alejandro Cer?n Estrada" <ceronman at gmail.com>

Take a look at this example:

def lines():
    for line in my_file:
        if some_error():
            raise StopIteration()
        yield line
    yield 'end'

for line in lines():
    do_something()

From rhamph at gmail.com  Sat Dec  8 22:33:26 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 8 Dec 2007 14:33:26 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <a467ca4f0712081238h182f92ebj51256305c78620d1@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<a467ca4f0712080957k51f6f6cs5e3be19149148871@mail.gmail.com>
	<ca471dc20712081143lcc92e50m1ffa23aebeaf402a@mail.gmail.com>
	<a467ca4f0712081238h182f92ebj51256305c78620d1@mail.gmail.com>
Message-ID: <aac2c7cb0712081333l178d8acuade2c7b1a6dc51c4@mail.gmail.com>

On Dec 8, 2007 1:38 PM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> On 08/12/2007, Guido van Rossum <guido at python.org> wrote:
> > On Dec 8, 2007 9:57 AM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > > Which is the best solution?  I think my patch fixes two problems: 1. the
> > > need to have a FD to wake up poll() (t o fix the problem with what we
> are
> > > discussing in this thread), and 2. make Python's signal handling more
> > > reliable (not 100% reliable because it doesn't handle longer bursts of
> > > signals than the pipe buffer can take, but at least is race free).
> >
> > I think it's okay to drop signals if too many come. The FD should be
> > put in non-blocking mode so the signal handler won't block forever.
> > Does Unix even promise that a signal gets delivered twice if it gets
> > sent quickly twice in a row?
>
> Good point.

Note that we may drop a new signal, not the same one we got several
times.  I don't know if Unix will do that.  Then again, I've been
unable to find documentation promising it'd deliver any signal at all.

-- 
Adam Olsen, aka Rhamphoryncus

From glyph at divmod.com  Sat Dec  8 22:56:33 2007
From: glyph at divmod.com (glyph at divmod.com)
Date: Sat, 08 Dec 2007 21:56:33 -0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
Message-ID: <20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>

On 05:20 pm, guido at python.org wrote:
>The best solution I can think of is to add a new API that takes a
>signal and a file descriptor and registers a C-level handler for that
>signal which writes a byte to the file descriptor. You can then create
>a pipe, connect the signal handler to the write end, and add the read
>end to your list of file descriptors passed to select() or poll(). The
>handler must be written in C in order to avoid the race condition
>referred to by Glyph (signals arriving after the signal check in the
>VM main loop but before the select()/poll() system call is entered
>will not be noticed until the select()/poll() call completes).

This paragraph jogged my memory.  I remember this exact solution being 
discussed now, a year ago when I was last talking about these issues.

There's another benefit to implementing a write-a-byte C signal handler. 
Without this feature, it wouldn't make sense to have passed the 
SA_RESTART flag to sigaction, because and GUIs written in Python could 
have spent an indefinite amount of time waiting to deliver their signal 
to Python code.  So, if you had to handle SIGCHLD in Python, for 
example, calls like file().write() would suddenly start raising a new 
exception (EINTR).  With it, you could avoid a whole class of subtle 
error-handling code in Twisted programs.

From steve at holdenweb.com  Sat Dec  8 23:04:05 2007
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 08 Dec 2007 17:04:05 -0500
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <796874fc0712081306p164b5858u7b99fe4a4f3ce5b6@mail.gmail.com>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>	<002c01c839ce$fa13d8b0$9200a8c0@RaymondLaptop1>
	<796874fc0712081306p164b5858u7b99fe4a4f3ce5b6@mail.gmail.com>
Message-ID: <fjf4cj$mre$1@ger.gmane.org>

Manuel Alejandro Cer?n Estrada wrote:
> 2007/12/8, Raymond Hettinger <python at rcn.com>:
>> ...the proposal adds new syntax without adding functionality.
> 
> That is indeed the definition of syntactic sugar [1]. Python is full
> of that, for example the import statement.
> 
> 2007/12/8, Paul Svensson <paul at svensson.org>:
>> What is the problem that is not solved by iter(()) ?
> 
> 'yield iter(()).next()'  it's obscure, you can't know its purpose at
> first sight. There are other alternatives but none of them seems to be
> the "obvious way to do it".
> 
But surely iter(()) itself (and indeed iter(any empty sequence) is 
precisely the empty generator you seek, without having to use a def 
statement to create your own in Python.

 >>> for i in iter(()):
...   print "Result"
...
 >>>

So I don't see what's so magical about creating your own empty iterator, 
nor why you consider it necessary. The 2002 discussion to which you 
refer is clearly outdated.

> But that is the less important problem. The real problem is that
> raising StopIteration is not orthogonal. It is confusing for the
> reasons I exposed. Generators are something in the middle between a
> language feature and a framework/library. With 'yield break' they will
> be completely a language feature.
> 
> [1] http://en.wikipedia.org/wiki/Syntactic_sugar
> 
It's necessary, in discussions of syntactic sugar, to admit from the 
start that a certain amount of sweetness is desirable. If you would 
really like to remove the import statement you might consider 
programming in brainfuck, one of the least sugary languages around.

   http://en.wikipedia.org/wiki/Brainfuck

Of course, Python is popular because it hits many people's "sweet spot" 
for the desirable balance between syntactic sugar and minimalism, making 
it possible to write compact solutions to advanced problems. There are 
definitely limits, though. "yield break" seems ugly and unnecessary to me.

I'd rather consider Icon's "break break" to exit two levels of looping, 
but I don't see that flying either.

You still haven't made a convincing use case for "yield break", IMHO.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From ceronman at gmail.com  Sat Dec  8 23:13:52 2007
From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Alejandro_Cer=F3n_Estrada?=)
Date: Sat, 8 Dec 2007 17:13:52 -0500
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <005701c839df$a4ff63b0$9200a8c0@RaymondLaptop1>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<005701c839df$a4ff63b0$9200a8c0@RaymondLaptop1>
Message-ID: <796874fc0712081413t6e9ef92fr227a906d283b8636@mail.gmail.com>

2007/12/8, Raymond Hettinger <python at rcn.com>:
> In your example, why do you "raise StopIteration" instead just writing "return"?

In the example would be better to use 'return' rather than 'raise
StopIteration'. In most cases 'return' will have the same behavior
than 'raise StopIteration', but not always.

Acording to PEP 255:

    Note that return isn't always equivalent to raising StopIteration:  the
    difference lies in how enclosing try/except constructs are treated.

The trivial example is the empty generator where return is ambiguous
with the same statement for normal functions.

Maybe my proposal can change to "Replace 'return' with 'yield break'
in generators"

Again, PEP 255 talks about the possibility of removing 'return' from generators

    Q. Why allow "return" at all?  Why not force termination to be spelled
       "raise StopIteration"?

    A. The mechanics of StopIteration are low-level details, much like the
       mechanics of IndexError in Python 2.1:  the implementation needs to
       do *something* well-defined under the covers, and Python exposes
       these mechanisms for advanced users.  That's not an argument for
       forcing everyone to work at that level, though.  "return" means "I'm
       done" in any kind of function, and that's easy to explain and to use.
       Note that "return" isn't always equivalent to "raise StopIteration"
       in try/except construct, either (see the "Specification: Return"
       section).

Of curse, the problem of low level details it's solved by 'yield break'

Manuel.

From Audun.Ostrem.Nordal at cern.ch  Mon Dec  3 14:01:04 2007
From: Audun.Ostrem.Nordal at cern.ch (Audun Ostrem Nordal)
Date: Mon, 3 Dec 2007 14:01:04 +0100
Subject: [Python-Dev] blocking a non-blocking socket
In-Reply-To: <07Dec2.122308pst."58696"@synergy1.parc.xerox.com>
References: <07Dec2.122308pst."58696"@synergy1.parc.xerox.com>
Message-ID: <443039046E52EB4CABDCB50638B9A84C8D0465@cernxchg42.cern.ch>

> An interesting question has come up in the development of the 
> SSL module.
> 
> The function ssl.wrap_socket() takes a flag, 
> do_handshake_on_connect, which tells it whether to do the SSL 
> handshake before returning an SSLSocket object to the caller. 
>  If the socket being wrapped is non-blocking, the code in 
> wrap_socket() must invoke do_handshake() multiple times, and 
> effectively block until the handshake is done.
> 
> Right now, I'm doing it with this loop:
> 
>             if do_handshake_on_connect:
>                 # have to loop to support non-blocking sockets
>                 while True:
>                     try:
>                         self.do_handshake()
>                         break
>                     except SSLError, err:
>                         if err.args[0] == SSL_ERROR_WANT_READ:
>                             select.select([self], [], [])
>                         elif err.args[0] == SSL_ERROR_WANT_WRITE:
>                             select.select([], [self], [])
>                         else:
>                             raise

Hello Bill,

Another way of doing it could be to expose a connect() method on the ssl
objects.  It changes the socket.ssl api, but I'd say it is in the same
spirit as the do_handshake_on_connect parameter since no existing code
will break.  The caller then calls connect() until it does not return
SSL_ERROR_WANT_[WRITE|READ].  This only applies when the underlying
socket is non-blocking and the do_handshake_on_connect parameter is
false.

Arguably, this is similar to how normal sockets are treated in asyncore,
only that he caller must be prepared to respond to (multiple?) readable
and writable events for the connection to be established.


Cheers

Audun

From demitry.meerovich at intel.com  Wed Dec  5 10:21:52 2007
From: demitry.meerovich at intel.com (Meerovich, Demitry)
Date: Wed, 5 Dec 2007 11:21:52 +0200
Subject: [Python-Dev] python25 on "Windows 2003 Enterprise Server OS"
Message-ID: <3F7160719EE8BB4583803E15D600E2B602DE4268@hasmsx414.ger.corp.intel.com>

Hello,

 

Does python25 supports on the "Windows 2003 Enterprise Server OS" ?

 

Thanks

 

Meerovich Dima,

Software

Intel

 

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071205/2cad70de/attachment-0001.htm 

From demitry.meerovich at intel.com  Wed Dec  5 10:41:20 2007
From: demitry.meerovich at intel.com (Meerovich, Demitry)
Date: Wed, 5 Dec 2007 11:41:20 +0200
Subject: [Python-Dev] python25 on "Windows 2003 Enterprise Server OS"
Message-ID: <3F7160719EE8BB4583803E15D600E2B602DE428B@hasmsx414.ger.corp.intel.com>

Hello,

 

Does python25 supports on the "Windows 2003 Enterprise Server OS" ?

 

Thanks

 

Meerovich Dima,

Software

Intel

 

---------------------------------------------------------------------
Intel Israel (74) Limited

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071205/b4705f04/attachment-0001.htm 

From josiah.carlson at gmail.com  Wed Dec  5 19:04:59 2007
From: josiah.carlson at gmail.com (Josiah Carlson)
Date: Wed, 5 Dec 2007 10:04:59 -0800
Subject: [Python-Dev] Does anyone care enough about asyncore and
	asynchat to help adapt their APIs for Py3k?
In-Reply-To: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
References: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
Message-ID: <e6511dbf0712051004g3a67c1d9j183af1d8fc1bcba5@mail.gmail.com>

On Dec 5, 2007 9:19 AM, Guido van Rossum <guido at python.org> wrote:
> The asyncore and asynchat modules are in a difficult position when it
> comes to Python 3000. None of the core developers use it or
> particularly care about it (AFAIK), and the API has problems because
> it wasn't written to deal with bytes vs. unicode. E.g. in
> http://bugs.python.org/issue1067, Thomas suggests that these modules
> need to be rewritten to use bytes internally and have separate APIs to
> handle (unicode) text as desired, similar to the way file I/O was
> redesigned. Another alternative would be to make these modules deal
> strictly in bytes, but that would probably vastly reduce their
> usefulness (though I don't know -- as I said, I don't use them).

I can look into it later this month, but for the short term, I'm a
little squeezed for time (work, finishing school, etc.).  I am a bit
curious, however, I could have sworn that bytes were to become,
essentially, the 2.x str type (without some methods).  If that is the
case, no changes should really be necessary, except for a few small
changes to asynchat with regards to line terminators.

Then again, I haven't really been keeping up in the p3k/pydev lists
for the last 6-8 months, and only the subject line of this posting
caught my eye (because I use asyncore/asynchat, and support users of
asyncore/asynchat that contact me directly).

 - Josiah

From drallison at gmail.com  Wed Dec  5 20:51:56 2007
From: drallison at gmail.com (Dennis Allison)
Date: Wed, 5 Dec 2007 12:51:56 -0700
Subject: [Python-Dev] Does anyone care enough about asyncore and
	asynchat to help adapt their APIs for Py3k?
In-Reply-To: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
References: <ca471dc20712050919n6afebc24s43f8736a4282ab81@mail.gmail.com>
Message-ID: <54523c4d0712051151k7105598u78d745b2b8f957e8@mail.gmail.com>

Guido,

These modules provide the server component of many existing versions of the
Zope system where they have served well (pun intended).    I would be
concerned were they disappear in Python 3000.

On Dec 5, 2007 10:19 AM, Guido van Rossum <guido at python.org> wrote:

> The asyncore and asynchat modules are in a difficult position when it
> comes to Python 3000. None of the core developers use it or
> particularly care about it (AFAIK), and the API has problems because
> it wasn't written to deal with bytes vs. unicode. E.g. in
> http://bugs.python.org/issue1067, Thomas suggests that these modules
> need to be rewritten to use bytes internally and have separate APIs to
> handle (unicode) text as desired, similar to the way file I/O was
> redesigned. Another alternative would be to make these modules deal
> strictly in bytes, but that would probably vastly reduce their
> usefulness (though I don't know -- as I said, I don't use them).
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/>
> )
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/allison%40shasta.stanford.edu
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071205/e44178dc/attachment-0001.htm 

From johan at gnome.org  Sat Dec  8 11:17:29 2007
From: johan at gnome.org (Johan Dahlin)
Date: Sat, 08 Dec 2007 11:17:29 +0100
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>	<20071207142642.GC13636@tummy.com>	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
Message-ID: <475A6F39.4090905@gnome.org>

Guido van Rossum wrote:
> Adam, perhaps at some point (Monday?) we could get together on
> #python-dev and interact in real time on this issue. Probably even
> better on the phone. This offer is open to anyone who is serious about
> getting this resolved. Someone please take it -- I'm offering free
> consulting here!
> 
> I'm curious -- is there anyone here who understands why [Py]GTK is
> using signals anyway? It's not like writing robust signal handling
> code in C is at all easy or obvious. If instead of a signal a file
> descriptor could be used, all problems would likely be gone.

The timeout handler was added for KeyboardInterrupt to be able to work when 
you want to Ctrl-C yourself out of the gtk.main() loop.

Johan


From johan at gnome.org  Sat Dec  8 18:55:27 2007
From: johan at gnome.org (Johan Dahlin)
Date: Sat, 08 Dec 2007 18:55:27 +0100
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712080906g23f43363g1f727415370a95e6@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>	
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>	
	<20071207142642.GC13636@tummy.com>	
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>	
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>	
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>	
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>	
	<475A6F39.4090905@gnome.org>
	<ca471dc20712080906g23f43363g1f727415370a95e6@mail.gmail.com>
Message-ID: <475ADA8F.600@gnome.org>

Guido van Rossum wrote:
> On Dec 8, 2007 2:17 AM, Johan Dahlin <johan at gnome.org> wrote:
>> Guido van Rossum wrote:
>>> Adam, perhaps at some point (Monday?) we could get together on
>>> #python-dev and interact in real time on this issue. Probably even
>>> better on the phone. This offer is open to anyone who is serious about
>>> getting this resolved. Someone please take it -- I'm offering free
>>> consulting here!
>>>
>>> I'm curious -- is there anyone here who understands why [Py]GTK is
>>> using signals anyway? It's not like writing robust signal handling
>>> code in C is at all easy or obvious. If instead of a signal a file
>>> descriptor could be used, all problems would likely be gone.
>> The timeout handler was added for KeyboardInterrupt to be able to work when
>> you want to Ctrl-C yourself out of the gtk.main() loop.
> 
> Hm. How about making that an option? I don't think on the OLPC XO this
> is a valid use case (end users never have a console where they might
> enter ^C).
> 

It could easily be made into a compilation option which would solve the 
problem specifically for OLPC, but it would still be problematic for other 
platforms important to PyGTK (linux/gnome) where console based development
is more common.

Johan

From johan at gnome.org  Sat Dec  8 20:40:49 2007
From: johan at gnome.org (Johan Dahlin)
Date: Sat, 08 Dec 2007 20:40:49 +0100
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081131o540d9df6g7b6a1b419f3faa4e@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>	
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>	
	<20071207142642.GC13636@tummy.com>	
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>	
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>	
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>	
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>	
	<475A6F39.4090905@gnome.org>	
	<ca471dc20712080906g23f43363g1f727415370a95e6@mail.gmail.com>	
	<475ADA8F.600@gnome.org>
	<ca471dc20712081131o540d9df6g7b6a1b419f3faa4e@mail.gmail.com>
Message-ID: <475AF341.30604@gnome.org>

Guido van Rossum wrote:
> On Dec 8, 2007 9:55 AM, Johan Dahlin <johan at gnome.org> wrote:
>> Guido van Rossum wrote:
>>> On Dec 8, 2007 2:17 AM, Johan Dahlin <johan at gnome.org> wrote:
>>>> Guido van Rossum wrote:
>>>>> Adam, perhaps at some point (Monday?) we could get together on
>>>>> #python-dev and interact in real time on this issue. Probably even
>>>>> better on the phone. This offer is open to anyone who is serious about
>>>>> getting this resolved. Someone please take it -- I'm offering free
>>>>> consulting here!
>>>>>
>>>>> I'm curious -- is there anyone here who understands why [Py]GTK is
>>>>> using signals anyway? It's not like writing robust signal handling
>>>>> code in C is at all easy or obvious. If instead of a signal a file
>>>>> descriptor could be used, all problems would likely be gone.
>>>> The timeout handler was added for KeyboardInterrupt to be able to work when
>>>> you want to Ctrl-C yourself out of the gtk.main() loop.
>>> Hm. How about making that an option? I don't think on the OLPC XO this
>>> is a valid use case (end users never have a console where they might
>>> enter ^C).
>>>
>> It could easily be made into a compilation option which would solve the
>> problem specifically for OLPC, but it would still be problematic for other
>> platforms important to PyGTK (linux/gnome) where console based development
>> is more common.
> 
> But do those other platforms care about the extra CPU cycles and power
> used? I suspect not, at least not to the extent that OLPC cares.

They most certainly do, pygtk applications are used in desktop software used 
on laptops where battery time is increasingly important.

Johan

From rhamph at gmail.com  Sat Dec  8 23:36:24 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 8 Dec 2007 15:36:24 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
References: <20071207045606.GA13636@tummy.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
Message-ID: <aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>

On Dec 8, 2007 2:56 PM,  <glyph at divmod.com> wrote:
> On 05:20 pm, guido at python.org wrote:
> >The best solution I can think of is to add a new API that takes a
> >signal and a file descriptor and registers a C-level handler for that
> >signal which writes a byte to the file descriptor. You can then create
> >a pipe, connect the signal handler to the write end, and add the read
> >end to your list of file descriptors passed to select() or poll(). The
> >handler must be written in C in order to avoid the race condition
> >referred to by Glyph (signals arriving after the signal check in the
> >VM main loop but before the select()/poll() system call is entered
> >will not be noticed until the select()/poll() call completes).
>
> This paragraph jogged my memory.  I remember this exact solution being
> discussed now, a year ago when I was last talking about these issues.
>
> There's another benefit to implementing a write-a-byte C signal handler.
> Without this feature, it wouldn't make sense to have passed the
> SA_RESTART flag to sigaction, because and GUIs written in Python could
> have spent an indefinite amount of time waiting to deliver their signal
> to Python code.  So, if you had to handle SIGCHLD in Python, for
> example, calls like file().write() would suddenly start raising a new
> exception (EINTR).  With it, you could avoid a whole class of subtle
> error-handling code in Twisted programs.

SA_RESTART still isn't useful.  The low-level poll call (not write!)
must stop and call back into python.  If that doesn't indicate an
error you can safely restart your poll call though, and follow it with
a (probably non-blocking) write.

Note that the only reason to use C for a low-level handler here is
give access to sigatomic_t and avoid needing locks.  If you ran the
signal handler in a background thread (using sigwait to trigger them)
you could use a python handler.


-- 
Adam Olsen, aka Rhamphoryncus

From guido at python.org  Sun Dec  9 00:28:03 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 15:28:03 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
Message-ID: <ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>

On Dec 8, 2007 2:36 PM, Adam Olsen <rhamph at gmail.com> wrote:
> On Dec 8, 2007 2:56 PM,  <glyph at divmod.com> wrote:
> > On 05:20 pm, guido at python.org wrote:
> > >The best solution I can think of is to add a new API that takes a
> > >signal and a file descriptor and registers a C-level handler for that
> > >signal which writes a byte to the file descriptor. You can then create
> > >a pipe, connect the signal handler to the write end, and add the read
> > >end to your list of file descriptors passed to select() or poll(). The
> > >handler must be written in C in order to avoid the race condition
> > >referred to by Glyph (signals arriving after the signal check in the
> > >VM main loop but before the select()/poll() system call is entered
> > >will not be noticed until the select()/poll() call completes).
> >
> > This paragraph jogged my memory.  I remember this exact solution being
> > discussed now, a year ago when I was last talking about these issues.
> >
> > There's another benefit to implementing a write-a-byte C signal handler.
> > Without this feature, it wouldn't make sense to have passed the
> > SA_RESTART flag to sigaction, because and GUIs written in Python could
> > have spent an indefinite amount of time waiting to deliver their signal
> > to Python code.  So, if you had to handle SIGCHLD in Python, for
> > example, calls like file().write() would suddenly start raising a new
> > exception (EINTR).  With it, you could avoid a whole class of subtle
> > error-handling code in Twisted programs.
>
> SA_RESTART still isn't useful.  The low-level poll call (not write!)
> must stop and call back into python.  If that doesn't indicate an
> error you can safely restart your poll call though, and follow it with
> a (probably non-blocking) write.

Can't say I understand all of this, but it does reiterate that there
are more problems with signals than just the issue that Gustavo is
trying to squash. The possibility of having *any* I/O interrupted is
indeed a big worry. Though perhaps this could be alleviated by rigging
things so that signals get delivered (at the C level) to the main
thread and the rest of the code runs in a non-main thread?

> Note that the only reason to use C for a low-level handler here is
> give access to sigatomic_t and avoid needing locks.  If you ran the
> signal handler in a background thread (using sigwait to trigger them)
> you could use a python handler.

I haven't seen Gustavo's patch yet, but *my* reason for using a C
handler was different -- it was because writing a byte to a pipe in
Python would do nothing to fix Gustavo's issue.

Looking at the man page for sigwait()  it could be an alternative
solution, but I'm not sure how it would actually allow PyGTK to catch
KeyboardInterrupt.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Sun Dec  9 00:27:50 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 09 Dec 2007 12:27:50 +1300
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081143lcc92e50m1ffa23aebeaf402a@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<a467ca4f0712080957k51f6f6cs5e3be19149148871@mail.gmail.com>
	<ca471dc20712081143lcc92e50m1ffa23aebeaf402a@mail.gmail.com>
Message-ID: <475B2876.3080008@canterbury.ac.nz>

Guido van Rossum wrote:

> Does Unix even promise that a signal gets delivered twice if it gets
> sent quickly twice in a row?

No, it doesn't.

However, this doesn't necessarily mean that you can just
drop them when the pipe becomes full. If more than one
type of signal is sharing a pipe, you don't want it
getting filled up with signals of one type and causing
signals of a different type to get lost.

To guarantee that won't happen, you would have to either
use a separate pipe for each signal type, or have some
other scheme such as a bitmask keeping track of which
signal types are in the pipe.

> I have a worry though -- if signal handlers *always* and *only*
> communicate by writing a byte to a FD, does that mean that the VM main
> loop will have to attempt to read a byte from a pipe every time it
> checks for signals?

That sounds like a good reason for having the fd be an
additional mechanism that you can turn on when you want
it, but not the core mechanism for dealing with signals.

--
Greg

From greg.ewing at canterbury.ac.nz  Sun Dec  9 00:30:45 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 09 Dec 2007 12:30:45 +1300
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <fjf0ka$d96$1@ger.gmane.org>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<fjf0ka$d96$1@ger.gmane.org>
Message-ID: <475B2925.5060303@canterbury.ac.nz>

Terry Reedy wrote:
> I would prefer plain 'yield' to 'yield break' as a synonym for 'raise 
> StopIteration'.

Don't we already have a plain yield these days meaning
something else?

--
Greg


From python at rcn.com  Sun Dec  9 00:42:11 2007
From: python at rcn.com (Raymond Hettinger)
Date: Sat, 8 Dec 2007 15:42:11 -0800
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com><002c01c839ce$fa13d8b0$9200a8c0@RaymondLaptop1>
	<796874fc0712081306p164b5858u7b99fe4a4f3ce5b6@mail.gmail.com>
Message-ID: <002501c839f3$f4d2a1e0$9900a8c0@RaymondLaptop1>

> 2007/12/8, Raymond Hettinger <python at rcn.com>:
>>...the proposal adds new syntax without adding functionality.
>
> That is indeed the definition of syntactic sugar [1]. Python is full
> of that, for example the import statement.
 . . .
> The real problem is that
> raising StopIteration is not orthogonal. It is confusing for the
> reasons I exposed. Generators are something in the middle between a
> language feature and a framework/library. With 'yield break' they will
> be completely a language feature.

This personal notion of purity carries almost zero weight and is more a matter of personal taste/style than anything else. It 
certainly does not warrant a change to the grammar.

I've been a big fan of generators and tend to use them everywhere. I've *never* had a use case for "yield break" and find it 
uncommon to even "raise StopIteration".

The tone of your messages suggests that you're going to stick to this idea like glue, so to avoid further time wasting, this will 
likely be my last post on the subject.

If for some reason, you persist through to writing a PEP, it will be almost mandatory to scan existing, published code to find 
examples of code fragments that would be improved by 'yield break' .  My bet is that the examples will be rare and any "improvement" 
will be marginal at best and simply a matter of taste at worst.


Raymond 

From greg.ewing at canterbury.ac.nz  Sun Dec  9 00:44:04 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 09 Dec 2007 12:44:04 +1300
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <796874fc0712081306p164b5858u7b99fe4a4f3ce5b6@mail.gmail.com>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<002c01c839ce$fa13d8b0$9200a8c0@RaymondLaptop1>
	<796874fc0712081306p164b5858u7b99fe4a4f3ce5b6@mail.gmail.com>
Message-ID: <475B2C44.8020004@canterbury.ac.nz>

Manuel Alejandro Cer?n Estrada wrote:
> The real problem is that
> raising StopIteration is not orthogonal.

This is a non-problem as far as I can see. In a generator,
the way to stop the iteration is simply to return. In a
non-generator iterator, you're still going to have to
raise StopIteration. So I see no advantage at all to this
proposal.

--
Greg

From rhamph at gmail.com  Sun Dec  9 00:57:25 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 8 Dec 2007 16:57:25 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
Message-ID: <aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>

On Dec 8, 2007 4:28 PM, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 8, 2007 2:36 PM, Adam Olsen <rhamph at gmail.com> wrote:
> > On Dec 8, 2007 2:56 PM,  <glyph at divmod.com> wrote:
> > > On 05:20 pm, guido at python.org wrote:
> > > >The best solution I can think of is to add a new API that takes a
> > > >signal and a file descriptor and registers a C-level handler for that
> > > >signal which writes a byte to the file descriptor. You can then create
> > > >a pipe, connect the signal handler to the write end, and add the read
> > > >end to your list of file descriptors passed to select() or poll(). The
> > > >handler must be written in C in order to avoid the race condition
> > > >referred to by Glyph (signals arriving after the signal check in the
> > > >VM main loop but before the select()/poll() system call is entered
> > > >will not be noticed until the select()/poll() call completes).
> > >
> > > This paragraph jogged my memory.  I remember this exact solution being
> > > discussed now, a year ago when I was last talking about these issues.
> > >
> > > There's another benefit to implementing a write-a-byte C signal handler.
> > > Without this feature, it wouldn't make sense to have passed the
> > > SA_RESTART flag to sigaction, because and GUIs written in Python could
> > > have spent an indefinite amount of time waiting to deliver their signal
> > > to Python code.  So, if you had to handle SIGCHLD in Python, for
> > > example, calls like file().write() would suddenly start raising a new
> > > exception (EINTR).  With it, you could avoid a whole class of subtle
> > > error-handling code in Twisted programs.
> >
> > SA_RESTART still isn't useful.  The low-level poll call (not write!)
> > must stop and call back into python.  If that doesn't indicate an
> > error you can safely restart your poll call though, and follow it with
> > a (probably non-blocking) write.
>
> Can't say I understand all of this, but it does reiterate that there
> are more problems with signals than just the issue that Gustavo is
> trying to squash. The possibility of having *any* I/O interrupted is
> indeed a big worry. Though perhaps this could be alleviated by rigging
> things so that signals get delivered (at the C level) to the main
> thread and the rest of the code runs in a non-main thread?

That's the approach my threading patch will take, although reversed
(signals are handled by a background thread, leaving the main thread
as the *main* thread.)

I share your concern about interrupting whatever random syscalls (not
even limited to I/O!) that a library happens to use.


> > Note that the only reason to use C for a low-level handler here is
> > give access to sigatomic_t and avoid needing locks.  If you ran the
> > signal handler in a background thread (using sigwait to trigger them)
> > you could use a python handler.
>
> I haven't seen Gustavo's patch yet, but *my* reason for using a C
> handler was different -- it was because writing a byte to a pipe in
> Python would do nothing to fix Gustavo's issue.
>
> Looking at the man page for sigwait()  it could be an alternative
> solution, but I'm not sure how it would actually allow PyGTK to catch
> KeyboardInterrupt.

My mail at [1] was referring to this.  Option 1 involved writing to a
pipe that gets polled while option 2 requires we generate a new signal
targeting the specific thread we want to interrupt.

I'd like to propose an interim solution though: pygtk could install
their own SIGINT handler during the gtk mainloop (or all gtk code?),
have it write to a pipe monitored by gtk, and have gtk raise
KeyboardInterrupt if it gets used.  This won't allow custom SIGINT
handlers or any other signal handlers to run promptly, but it should
be good enough for OLPC's use case.


[1] http://mail.python.org/pipermail/python-dev/2007-December/075607.html

-- 
Adam Olsen, aka Rhamphoryncus

From pje at telecommunity.com  Sun Dec  9 01:06:19 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 08 Dec 2007 19:06:19 -0500
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <475B2925.5060303@canterbury.ac.nz>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<fjf0ka$d96$1@ger.gmane.org> <475B2925.5060303@canterbury.ac.nz>
Message-ID: <20071209000616.B367B3A405E@sparrow.telecommunity.com>

At 12:30 PM 12/9/2007 +1300, Greg Ewing wrote:
>Terry Reedy wrote:
> > I would prefer plain 'yield' to 'yield break' as a synonym for 'raise
> > StopIteration'.
>
>Don't we already have a plain yield these days meaning
>something else?

Yes.  It's equivalent to 'yield None'. 


From greg.ewing at canterbury.ac.nz  Sun Dec  9 01:09:15 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 09 Dec 2007 13:09:15 +1300
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <796874fc0712081413t6e9ef92fr227a906d283b8636@mail.gmail.com>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<005701c839df$a4ff63b0$9200a8c0@RaymondLaptop1>
	<796874fc0712081413t6e9ef92fr227a906d283b8636@mail.gmail.com>
Message-ID: <475B322B.8030801@canterbury.ac.nz>

Manuel Alejandro Cer?n Estrada wrote:

 > Acording to PEP 255:
 >
>     Note that return isn't always equivalent to raising StopIteration:  the
>     difference lies in how enclosing try/except constructs are treated.

All that means is that

   def g():
     try:
       if 0:
         yield
       return
     except StopIteration:
       print "Spam"

won't print "Spam". But since this involves catching the
exception with an explicit try-except, it doesn't fall
under the scope of your complaint.

> Of curse, the problem of low level details it's solved by 'yield break'

I would put it the other way around -- the problem
that 'yield break' is meant to solve is already solved
by 'return'. So there's no need for change.

--
Greg

From guido at python.org  Sun Dec  9 01:21:14 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 16:21:14 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
Message-ID: <ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>

On Dec 8, 2007 3:57 PM, Adam Olsen <rhamph at gmail.com> wrote:
>
> On Dec 8, 2007 4:28 PM, Guido van Rossum <guido at python.org> wrote:
> >
> > On Dec 8, 2007 2:36 PM, Adam Olsen <rhamph at gmail.com> wrote:
> > > On Dec 8, 2007 2:56 PM,  <glyph at divmod.com> wrote:
> > > > On 05:20 pm, guido at python.org wrote:
> > > > >The best solution I can think of is to add a new API that takes a
> > > > >signal and a file descriptor and registers a C-level handler for that
> > > > >signal which writes a byte to the file descriptor. You can then create
> > > > >a pipe, connect the signal handler to the write end, and add the read
> > > > >end to your list of file descriptors passed to select() or poll(). The
> > > > >handler must be written in C in order to avoid the race condition
> > > > >referred to by Glyph (signals arriving after the signal check in the
> > > > >VM main loop but before the select()/poll() system call is entered
> > > > >will not be noticed until the select()/poll() call completes).
> > > >
> > > > This paragraph jogged my memory.  I remember this exact solution being
> > > > discussed now, a year ago when I was last talking about these issues.
> > > >
> > > > There's another benefit to implementing a write-a-byte C signal handler.
> > > > Without this feature, it wouldn't make sense to have passed the
> > > > SA_RESTART flag to sigaction, because and GUIs written in Python could
> > > > have spent an indefinite amount of time waiting to deliver their signal
> > > > to Python code.  So, if you had to handle SIGCHLD in Python, for
> > > > example, calls like file().write() would suddenly start raising a new
> > > > exception (EINTR).  With it, you could avoid a whole class of subtle
> > > > error-handling code in Twisted programs.
> > >
> > > SA_RESTART still isn't useful.  The low-level poll call (not write!)
> > > must stop and call back into python.  If that doesn't indicate an
> > > error you can safely restart your poll call though, and follow it with
> > > a (probably non-blocking) write.
> >
> > Can't say I understand all of this, but it does reiterate that there
> > are more problems with signals than just the issue that Gustavo is
> > trying to squash. The possibility of having *any* I/O interrupted is
> > indeed a big worry. Though perhaps this could be alleviated by rigging
> > things so that signals get delivered (at the C level) to the main
> > thread and the rest of the code runs in a non-main thread?
>
> That's the approach my threading patch will take, although reversed
> (signals are handled by a background thread, leaving the main thread
> as the *main* thread.)

Hm... Does this mean you're *always* creating an extra thread to handle signals?

> I share your concern about interrupting whatever random syscalls (not
> even limited to I/O!) that a library happens to use.
>
>
> > > Note that the only reason to use C for a low-level handler here is
> > > give access to sigatomic_t and avoid needing locks.  If you ran the
> > > signal handler in a background thread (using sigwait to trigger them)
> > > you could use a python handler.
> >
> > I haven't seen Gustavo's patch yet, but *my* reason for using a C
> > handler was different -- it was because writing a byte to a pipe in
> > Python would do nothing to fix Gustavo's issue.
> >
> > Looking at the man page for sigwait()  it could be an alternative
> > solution, but I'm not sure how it would actually allow PyGTK to catch
> > KeyboardInterrupt.
>
> My mail at [1] was referring to this.  Option 1 involved writing to a
> pipe that gets polled while option 2 requires we generate a new signal
> targeting the specific thread we want to interrupt.
>
> I'd like to propose an interim solution though: pygtk could install
> their own SIGINT handler during the gtk mainloop (or all gtk code?),
> have it write to a pipe monitored by gtk, and have gtk raise
> KeyboardInterrupt if it gets used.  This won't allow custom SIGINT
> handlers or any other signal handlers to run promptly, but it should
> be good enough for OLPC's use case.
>
>
> [1] http://mail.python.org/pipermail/python-dev/2007-December/075607.html

Since OLPC has to use 2.5 they don't really have another choice
besides this or making the timeout (perhaps much) larger -- I'm not
going to risk a change as big as anything proposed here for 2.5.2, so
nothing will change before 2.6.

I've got to say that all the cross-referencing and asynchronous
discussion here makes it *very* difficult to wrap my head around the
various proposals. It also doesn't help that different participants
appear to have different use cases in mind. E.g. do we care about
threads started directly from C++ code? (These happen all the time at
Google, but we don't care much about signals.) And what about
restarting system calls (like Glyph brought up)?

I've seen references to bug #1643738 which got a thumbs up from Tim
Peters -- Adam, what do you think of that? I know it doesn't address
Gustavo's issue but it seems useful in its own right.

Gustavo, at some point you suggested making changes to Python so that
all signals are blocked in all threads except for the main thread. I
think I'd be more inclined to give that the green light than the patch
using pipes for all signal handling, as long as we can make sure that
this blocking of all signals isn't inherited by fork()'ed children --
we had serious problems with that in 2.4 where child processes were
unkillable (except for SIGKILL). I'd also be OK with a patch that
leaves the existing signal handling code intact but *adds* a way to
have a signal handler written in C that writes one byte to one end of
a pipe -- where the pipe is provided by Python code.

Does any of this make sense still?

Anyway, I would still like to discuss this on #python-dev Monday.
Adam, in what time zone are you? (I'm PST.) Who else is interested?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ceronman at gmail.com  Sun Dec  9 01:55:55 2007
From: ceronman at gmail.com (=?ISO-8859-1?Q?Manuel_Alejandro_Cer=F3n_Estrada?=)
Date: Sat, 8 Dec 2007 19:55:55 -0500
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <475B322B.8030801@canterbury.ac.nz>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<005701c839df$a4ff63b0$9200a8c0@RaymondLaptop1>
	<796874fc0712081413t6e9ef92fr227a906d283b8636@mail.gmail.com>
	<475B322B.8030801@canterbury.ac.nz>
Message-ID: <796874fc0712081655h27461609l3ec9b88f4a7e41b9@mail.gmail.com>

2007/12/8, Greg Ewing <greg.ewing at canterbury.ac.nz>:
> I would put it the other way around -- the problem
> that 'yield break' is meant to solve is already solved
> by 'return'. So there's no need for change.

I have been re-thinking the problem and this is true. The only
exception would be empty generators, but they are rare. 'return' works
well for 99.99% of the cases. And the rest 0.01% does not worth a
syntax change.

Thank you very much for all your comments.

Manuel.

From skip at pobox.com  Sun Dec  9 02:15:17 2007
From: skip at pobox.com (skip at pobox.com)
Date: Sat, 8 Dec 2007 19:15:17 -0600
Subject: [Python-Dev] bsddb185 v1.0 for Python 2.6 and 3.0
Message-ID: <18267.16805.948614.915298@montanaro.dyndns.org>


Python 3.0 will dispense with the rarely used, but occasionally
indispensible, bsddb185 module.  I extracted the source code and unit tests
from the current Python trunk, wrote a setup.py, made a couple slight mods
so it would build and pass tests under both Python 2.6 and 3.0.  It's
available from the Python Package Index:

    http://pypi.python.org/pypi/bsddb185

Cheers,

-- 
Skip Montanaro - skip at pobox.com - http://www.webfast.com/~skip/

From pje at telecommunity.com  Sun Dec  9 02:19:29 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 08 Dec 2007 20:19:29 -0500
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <796874fc0712081655h27461609l3ec9b88f4a7e41b9@mail.gmail.co
 m>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<005701c839df$a4ff63b0$9200a8c0@RaymondLaptop1>
	<796874fc0712081413t6e9ef92fr227a906d283b8636@mail.gmail.com>
	<475B322B.8030801@canterbury.ac.nz>
	<796874fc0712081655h27461609l3ec9b88f4a7e41b9@mail.gmail.com>
Message-ID: <20071209012017.CDB913A4068@sparrow.telecommunity.com>

At 07:55 PM 12/8/2007 -0500, Manuel Alejandro Cer?n Estrada wrote:
>2007/12/8, Greg Ewing <greg.ewing at canterbury.ac.nz>:
> > I would put it the other way around -- the problem
> > that 'yield break' is meant to solve is already solved
> > by 'return'. So there's no need for change.
>
>I have been re-thinking the problem and this is true. The only
>exception would be empty generators, but they are rare.

Note that:

    def g():
        return
        yield

Is a valid empty generator.  :)


From rhamph at gmail.com  Sun Dec  9 02:30:09 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 8 Dec 2007 18:30:09 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
Message-ID: <aac2c7cb0712081730q17f97000ofae248dd7ad49957@mail.gmail.com>

On Dec 8, 2007 5:21 PM, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 8, 2007 3:57 PM, Adam Olsen <rhamph at gmail.com> wrote:
> >
> > On Dec 8, 2007 4:28 PM, Guido van Rossum <guido at python.org> wrote:
> > >
> > > On Dec 8, 2007 2:36 PM, Adam Olsen <rhamph at gmail.com> wrote:
> > > > On Dec 8, 2007 2:56 PM,  <glyph at divmod.com> wrote:
> > > > > On 05:20 pm, guido at python.org wrote:
> > > > > >The best solution I can think of is to add a new API that takes a
> > > > > >signal and a file descriptor and registers a C-level handler for that
> > > > > >signal which writes a byte to the file descriptor. You can then create
> > > > > >a pipe, connect the signal handler to the write end, and add the read
> > > > > >end to your list of file descriptors passed to select() or poll(). The
> > > > > >handler must be written in C in order to avoid the race condition
> > > > > >referred to by Glyph (signals arriving after the signal check in the
> > > > > >VM main loop but before the select()/poll() system call is entered
> > > > > >will not be noticed until the select()/poll() call completes).
> > > > >
> > > > > This paragraph jogged my memory.  I remember this exact solution being
> > > > > discussed now, a year ago when I was last talking about these issues.
> > > > >
> > > > > There's another benefit to implementing a write-a-byte C signal handler.
> > > > > Without this feature, it wouldn't make sense to have passed the
> > > > > SA_RESTART flag to sigaction, because and GUIs written in Python could
> > > > > have spent an indefinite amount of time waiting to deliver their signal
> > > > > to Python code.  So, if you had to handle SIGCHLD in Python, for
> > > > > example, calls like file().write() would suddenly start raising a new
> > > > > exception (EINTR).  With it, you could avoid a whole class of subtle
> > > > > error-handling code in Twisted programs.
> > > >
> > > > SA_RESTART still isn't useful.  The low-level poll call (not write!)
> > > > must stop and call back into python.  If that doesn't indicate an
> > > > error you can safely restart your poll call though, and follow it with
> > > > a (probably non-blocking) write.
> > >
> > > Can't say I understand all of this, but it does reiterate that there
> > > are more problems with signals than just the issue that Gustavo is
> > > trying to squash. The possibility of having *any* I/O interrupted is
> > > indeed a big worry. Though perhaps this could be alleviated by rigging
> > > things so that signals get delivered (at the C level) to the main
> > > thread and the rest of the code runs in a non-main thread?
> >
> > That's the approach my threading patch will take, although reversed
> > (signals are handled by a background thread, leaving the main thread
> > as the *main* thread.)
>
> Hm... Does this mean you're *always* creating an extra thread to handle signals?

Yup, Py_Initialize will do it.


> > I share your concern about interrupting whatever random syscalls (not
> > even limited to I/O!) that a library happens to use.
> >
> >
> > > > Note that the only reason to use C for a low-level handler here is
> > > > give access to sigatomic_t and avoid needing locks.  If you ran the
> > > > signal handler in a background thread (using sigwait to trigger them)
> > > > you could use a python handler.
> > >
> > > I haven't seen Gustavo's patch yet, but *my* reason for using a C
> > > handler was different -- it was because writing a byte to a pipe in
> > > Python would do nothing to fix Gustavo's issue.
> > >
> > > Looking at the man page for sigwait()  it could be an alternative
> > > solution, but I'm not sure how it would actually allow PyGTK to catch
> > > KeyboardInterrupt.
> >
> > My mail at [1] was referring to this.  Option 1 involved writing to a
> > pipe that gets polled while option 2 requires we generate a new signal
> > targeting the specific thread we want to interrupt.
> >
> > I'd like to propose an interim solution though: pygtk could install
> > their own SIGINT handler during the gtk mainloop (or all gtk code?),
> > have it write to a pipe monitored by gtk, and have gtk raise
> > KeyboardInterrupt if it gets used.  This won't allow custom SIGINT
> > handlers or any other signal handlers to run promptly, but it should
> > be good enough for OLPC's use case.
> >
> >
> > [1] http://mail.python.org/pipermail/python-dev/2007-December/075607.html
>
> Since OLPC has to use 2.5 they don't really have another choice
> besides this or making the timeout (perhaps much) larger -- I'm not
> going to risk a change as big as anything proposed here for 2.5.2, so
> nothing will change before 2.6.
>
> I've got to say that all the cross-referencing and asynchronous
> discussion here makes it *very* difficult to wrap my head around the
> various proposals. It also doesn't help that different participants
> appear to have different use cases in mind. E.g. do we care about
> threads started directly from C++ code? (These happen all the time at
> Google, but we don't care much about signals.) And what about
> restarting system calls (like Glyph brought up)?
>
> I've seen references to bug #1643738 which got a thumbs up from Tim
> Peters -- Adam, what do you think of that? I know it doesn't address
> Gustavo's issue but it seems useful in its own right.

It's a step in the right direction (and I don't think it will break
anything), but I don't think it's enough to make anything entirely
correct either.

Hrm.  If we replaced Py_AddPendingCall with a single flag (what
is_tripped is now), and had the main thread check it directly, I think
that'd avoid the corruption risks.  That's with bug #1643738, and
assuming sig_atomic_t functions sanely.

To summarize, there's two problems to be solved:
1) low-level corruption in the signal handlers as they record a new
signal, such as in Py_AddPendingCalls
2) high-level wakeup race: "check for pending signals, have a signal
come in, then call a blocking syscall/library (oblivious to the new
signal)."


> Gustavo, at some point you suggested making changes to Python so that
> all signals are blocked in all threads except for the main thread. I
> think I'd be more inclined to give that the green light than the patch
> using pipes for all signal handling, as long as we can make sure that
> this blocking of all signals isn't inherited by fork()'ed children --
> we had serious problems with that in 2.4 where child processes were
> unkillable (except for SIGKILL). I'd also be OK with a patch that
> leaves the existing signal handling code intact but *adds* a way to
> have a signal handler written in C that writes one byte to one end of
> a pipe -- where the pipe is provided by Python code.

I'm not sure this helps anything (without a dedicated signal-handler
thread).  It doesn't avoid interrupting random syscalls, and the
current code should already ensure only the main thread does any real
processing.

The "if (getpid() == main_pid) {" could use a more precise comment
though.  If I understand correctly, POSIX requires that to always
evaluate to true.  The only time it returns false is on LinuxThreads
(pre-NPTL), where it cancels out another bug of sending the signal to
ALL threads (rather than picking one at random.)


> Does any of this make sense still?
>
> Anyway, I would still like to discuss this on #python-dev Monday.
> Adam, in what time zone are you? (I'm PST.) Who else is interested?

MST.

-- 
Adam Olsen, aka Rhamphoryncus

From gjcarneiro at gmail.com  Sun Dec  9 02:30:34 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Sun, 9 Dec 2007 01:30:34 +0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
Message-ID: <a467ca4f0712081730u14f674bi6eafa268b4a8b88e@mail.gmail.com>

On 09/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 8, 2007 3:57 PM, Adam Olsen <rhamph at gmail.com> wrote:
> >
> > On Dec 8, 2007 4:28 PM, Guido van Rossum <guido at python.org> wrote:
> > >
> > > On Dec 8, 2007 2:36 PM, Adam Olsen <rhamph at gmail.com> wrote:
> > > > On Dec 8, 2007 2:56 PM,  <glyph at divmod.com> wrote:
> > > > > On 05:20 pm, guido at python.org wrote:
> > > > > >The best solution I can think of is to add a new API that takes a
> > > > > >signal and a file descriptor and registers a C-level handler for
> that
> > > > > >signal which writes a byte to the file descriptor. You can then
> create
> > > > > >a pipe, connect the signal handler to the write end, and add the
> read
> > > > > >end to your list of file descriptors passed to select() or
> poll(). The
> > > > > >handler must be written in C in order to avoid the race condition
> > > > > >referred to by Glyph (signals arriving after the signal check in
> the
> > > > > >VM main loop but before the select()/poll() system call is
> entered
> > > > > >will not be noticed until the select()/poll() call completes).
> > > > >
> > > > > This paragraph jogged my memory.  I remember this exact solution
> being
> > > > > discussed now, a year ago when I was last talking about these
> issues.
> > > > >
> > > > > There's another benefit to implementing a write-a-byte C signal
> handler.
> > > > > Without this feature, it wouldn't make sense to have passed the
> > > > > SA_RESTART flag to sigaction, because and GUIs written in Python
> could
> > > > > have spent an indefinite amount of time waiting to deliver their
> signal
> > > > > to Python code.  So, if you had to handle SIGCHLD in Python, for
> > > > > example, calls like file().write() would suddenly start raising a
> new
> > > > > exception (EINTR).  With it, you could avoid a whole class of
> subtle
> > > > > error-handling code in Twisted programs.
> > > >
> > > > SA_RESTART still isn't useful.  The low-level poll call (not write!)
> > > > must stop and call back into python.  If that doesn't indicate an
> > > > error you can safely restart your poll call though, and follow it
> with
> > > > a (probably non-blocking) write.
> > >
> > > Can't say I understand all of this, but it does reiterate that there
> > > are more problems with signals than just the issue that Gustavo is
> > > trying to squash. The possibility of having *any* I/O interrupted is
> > > indeed a big worry. Though perhaps this could be alleviated by rigging
> > > things so that signals get delivered (at the C level) to the main
> > > thread and the rest of the code runs in a non-main thread?
> >
> > That's the approach my threading patch will take, although reversed
> > (signals are handled by a background thread, leaving the main thread
> > as the *main* thread.)
>
> Hm... Does this mean you're *always* creating an extra thread to handle
> signals?
>
> > I share your concern about interrupting whatever random syscalls (not
> > even limited to I/O!) that a library happens to use.
> >
> >
> > > > Note that the only reason to use C for a low-level handler here is
> > > > give access to sigatomic_t and avoid needing locks.  If you ran the
> > > > signal handler in a background thread (using sigwait to trigger
> them)
> > > > you could use a python handler.
> > >
> > > I haven't seen Gustavo's patch yet, but *my* reason for using a C
> > > handler was different -- it was because writing a byte to a pipe in
> > > Python would do nothing to fix Gustavo's issue.
> > >
> > > Looking at the man page for sigwait()  it could be an alternative
> > > solution, but I'm not sure how it would actually allow PyGTK to catch
> > > KeyboardInterrupt.
> >
> > My mail at [1] was referring to this.  Option 1 involved writing to a
> > pipe that gets polled while option 2 requires we generate a new signal
> > targeting the specific thread we want to interrupt.
> >
> > I'd like to propose an interim solution though: pygtk could install
> > their own SIGINT handler during the gtk mainloop (or all gtk code?),
> > have it write to a pipe monitored by gtk, and have gtk raise
> > KeyboardInterrupt if it gets used.  This won't allow custom SIGINT
> > handlers or any other signal handlers to run promptly, but it should
> > be good enough for OLPC's use case.
> >
> >
> > [1]
> http://mail.python.org/pipermail/python-dev/2007-December/075607.html
>
> Since OLPC has to use 2.5 they don't really have another choice
> besides this or making the timeout (perhaps much) larger -- I'm not
> going to risk a change as big as anything proposed here for 2.5.2, so
> nothing will change before 2.6.
>
> I've got to say that all the cross-referencing and asynchronous
> discussion here makes it *very* difficult to wrap my head around the
> various proposals. It also doesn't help that different participants
> appear to have different use cases in mind. E.g. do we care about
> threads started directly from C++ code? (These happen all the time at
> Google, but we don't care much about signals.) And what about
> restarting system calls (like Glyph brought up)?
>
> I've seen references to bug #1643738 which got a thumbs up from Tim
> Peters -- Adam, what do you think of that? I know it doesn't address
> Gustavo's issue but it seems useful in its own right.


That issue seems orthogonal.  Just fixes the current async handling code.
See how complicated and error prone the current code is?  It's so easy to
get race conditions... The pipe solution is much simpler IMHO, less error
prone... ;-)

But, well, maybe it's just me that thinks it's simpler and no one else.
That's life.. :|

Gustavo, at some point you suggested making changes to Python so that
> all signals are blocked in all threads except for the main thread. I
> think I'd be more inclined to give that the green light than the patch
> using pipes for all signal handling, as long as we can make sure that
> this blocking of all signals isn't inherited by fork()'ed children --
> we had serious problems with that in 2.4 where child processes were
> unkillable (except for SIGKILL).


I don't think that solution works after all.  We can only block signals for
certain threads inside the threads themselves.  But we do not control all
threads.  Some are created by C libraries, and these threads will not have
signals blocked by default, and also there is no 'thread creation hook' that
we can use.

More promising would be pthread_kill solution suggested by loewis.  It's not
a bad solution, but  it does nothing to improve the possible race conditions
in signal handling.  Also it is not "theoretically" safe to call in signal
handlers, as far as I can tell.  At least it's not on the list of safe
functions to call in async handlers:
http://www.opengroup.org/onlinepubs/009695399/functions/xsh_chap02_04.html#tag_02_04_03

I'd also be OK with a patch that
> leaves the existing signal handling code intact but *adds* a way to
> have a signal handler written in C that writes one byte to one end of
> a pipe -- where the pipe is provided by Python code.


I think this is most balanced approach of all.

Does any of this make sense still?
>
> Anyway, I would still like to discuss this on #python-dev Monday.
> Adam, in what time zone are you? (I'm PST.) Who else is interested?


I'll try to show up if I have time.

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071209/64e88e5a/attachment.htm 

From rhamph at gmail.com  Sun Dec  9 02:44:41 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 8 Dec 2007 18:44:41 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <a467ca4f0712081730u14f674bi6eafa268b4a8b88e@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com> <475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<a467ca4f0712081730u14f674bi6eafa268b4a8b88e@mail.gmail.com>
Message-ID: <aac2c7cb0712081744r483158e4ob9e62aa229630828@mail.gmail.com>

On Dec 8, 2007 6:30 PM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> On 09/12/2007, Guido van Rossum <guido at python.org> wrote:
> > Gustavo, at some point you suggested making changes to Python so that
> > all signals are blocked in all threads except for the main thread. I
> > think I'd be more inclined to give that the green light than the patch
> > using pipes for all signal handling, as long as we can make sure that
> > this blocking of all signals isn't inherited by fork()'ed children --
> > we had serious problems with that in 2.4 where child processes were
> > unkillable (except for SIGKILL).
>
> I don't think that solution works after all.  We can only block signals for
> certain threads inside the threads themselves.  But we do not control all
> threads.  Some are created by C libraries, and these threads will not have
> signals blocked by default, and also there is no 'thread creation hook' that
> we can use.

Note that new threads inherit signal masks from their creator.  It's
only threads created before loading python that are a problem.  For my
threading patch I plan to document that as simply something embedders
have to do.


> > I'd also be OK with a patch that
> > leaves the existing signal handling code intact but *adds* a way to
> > have a signal handler written in C that writes one byte to one end of
> > a pipe -- where the pipe is provided by Python code.
>
> I think this is most balanced approach of all.

Yeah, use the existing Handlers array to record which signals have
come in, rather using the byte passed through the pipe.

And I missed a problem in bug #1643738: Handlers[...].tripped should
be a sig_atomic_t, not int.


-- 
Adam Olsen, aka Rhamphoryncus

From guido at python.org  Sun Dec  9 02:45:58 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 17:45:58 -0800
Subject: [Python-Dev] python25 on "Windows 2003 Enterprise Server OS"
In-Reply-To: <3F7160719EE8BB4583803E15D600E2B602DE428B@hasmsx414.ger.corp.intel.com>
References: <3F7160719EE8BB4583803E15D600E2B602DE428B@hasmsx414.ger.corp.intel.com>
Message-ID: <ca471dc20712081745k500e4a9bm199249f5539a8ec2@mail.gmail.com>

Probably. Have you tried it?

On Dec 5, 2007 1:41 AM, Meerovich, Demitry <demitry.meerovich at intel.com> wrote:
>
>
>
>
>
> Hello,
>
>
>
> Does python25 supports on the "Windows 2003 Enterprise Server OS" ?
>
>
>
> Thanks
>
>
>
> Meerovich Dima,
>
> Software
>
> Intel
>
>   ---------------------------------------------------------------------
> Intel Israel (74) Limited
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Dec  9 02:54:31 2007
From: guido at python.org (Guido van Rossum)
Date: Sat, 8 Dec 2007 17:54:31 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <aac2c7cb0712081730q17f97000ofae248dd7ad49957@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com> <475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<aac2c7cb0712081730q17f97000ofae248dd7ad49957@mail.gmail.com>
Message-ID: <ca471dc20712081754k1f9266b2v3bfed62a32c40211@mail.gmail.com>

On Dec 8, 2007 5:30 PM, Adam Olsen <rhamph at gmail.com> wrote:
> On Dec 8, 2007 5:21 PM, Guido van Rossum <guido at python.org> wrote:
> > Hm... Does this mean you're *always* creating an extra thread to handle signals?
>
> Yup, Py_Initialize will do it.

That's unacceptable. It must be possible to build Python without
threads (and still support signals -- in fact one could argue that
signals make *more* sense when there are no threads :-).

[...]

> To summarize, there's two problems to be solved:
> 1) low-level corruption in the signal handlers as they record a new
> signal, such as in Py_AddPendingCalls

This is purely theoretical, right? Has anyone ever observed this?

> 2) high-level wakeup race: "check for pending signals, have a signal
> come in, then call a blocking syscall/library (oblivious to the new
> signal)."

Right. That's the race which really does happen, and for which the
current lame-y work-around is to use a short timeout.

[...]

> > Anyway, I would still like to discuss this on #python-dev Monday.
> > Adam, in what time zone are you? (I'm PST.) Who else is interested?
>
> MST.

Unfortunately I can't stay at work later than 5:30 or so which would
be too early for you I believe. I could try again after 8pm, your 9pm.
Would that work at all? Otherwise I'd rather try earlier in the day if
that works at all for you.


-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rhamph at gmail.com  Sun Dec  9 03:22:15 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Sat, 8 Dec 2007 19:22:15 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081754k1f9266b2v3bfed62a32c40211@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<aac2c7cb0712081730q17f97000ofae248dd7ad49957@mail.gmail.com>
	<ca471dc20712081754k1f9266b2v3bfed62a32c40211@mail.gmail.com>
Message-ID: <aac2c7cb0712081822o69b76688oc3c620b8963f96f9@mail.gmail.com>

On Dec 8, 2007 6:54 PM, Guido van Rossum <guido at python.org> wrote:
> On Dec 8, 2007 5:30 PM, Adam Olsen <rhamph at gmail.com> wrote:
> > On Dec 8, 2007 5:21 PM, Guido van Rossum <guido at python.org> wrote:
> > > Hm... Does this mean you're *always* creating an extra thread to handle signals?
> >
> > Yup, Py_Initialize will do it.
>
> That's unacceptable. It must be possible to build Python without
> threads (and still support signals -- in fact one could argue that
> signals make *more* sense when there are no threads :-).

For my patch it won't make much sense to disable threads, so I don't
mind taking liberties there.

>
> [...]
>
> > To summarize, there's two problems to be solved:
> > 1) low-level corruption in the signal handlers as they record a new
> > signal, such as in Py_AddPendingCalls
>
> This is purely theoretical, right? Has anyone ever observed this?

I've never heard of it happening.  If the compiler doesn't do much
reordering (the CPU isn't an issue as this is only called in the main
thread) then the most you might get is dropped calls.

It's fairly safe the way signal handlers use it, but they'd work just
as well (and easier to understand/verify) without the whole queue
aspect; just setting some flags and resetting _Py_Ticker.


> > 2) high-level wakeup race: "check for pending signals, have a signal
> > come in, then call a blocking syscall/library (oblivious to the new
> > signal)."
>
> Right. That's the race which really does happen, and for which the
> current lame-y work-around is to use a short timeout.
>
> [...]
>
> > > Anyway, I would still like to discuss this on #python-dev Monday.
> > > Adam, in what time zone are you? (I'm PST.) Who else is interested?
> >
> > MST.
>
> Unfortunately I can't stay at work later than 5:30 or so which would
> be too early for you I believe. I could try again after 8pm, your 9pm.
> Would that work at all? Otherwise I'd rather try earlier in the day if
> that works at all for you.

5:30 am or 5:30 pm?  Any time after 11 am MST (10 am PST) should be
fine for me.  (My previous email was a little naive about how late I
get up.)  I shouldn't be gone until around midnight MST.


-- 
Adam Olsen, aka Rhamphoryncus

From djarb at highenergymagic.org  Sun Dec  9 03:56:45 2007
From: djarb at highenergymagic.org (Daniel Arbuckle)
Date: Sat, 8 Dec 2007 18:56:45 -0800
Subject: [Python-Dev] Python-Dev Digest, Vol 53, Issue 23
In-Reply-To: <mailman.9156.1197152144.13604.python-dev@python.org>
References: <mailman.9156.1197152144.13604.python-dev@python.org>
Message-ID: <ddd408be0712081856g2a7bc9bayd16bb93d6b20d67@mail.gmail.com>

> "Josiah Carlson" <josiah.carlson at gmail.com> wrote:
> On Dec 5, 2007 9:19 AM, Guido van Rossum <guido at python.org> wrote:
> > The asyncore and asynchat modules are in a difficult position when it
> > comes to Python 3000. None of the core developers use it or
> > particularly care about it (AFAIK), and the API has problems because
> > it wasn't written to deal with bytes vs. unicode. E.g. in
> > http://bugs.python.org/issue1067, Thomas suggests that these modules
> > need to be rewritten to use bytes internally and have separate APIs to
> > handle (unicode) text as desired, similar to the way file I/O was
> > redesigned. Another alternative would be to make these modules deal
> > strictly in bytes, but that would probably vastly reduce their
> > usefulness (though I don't know -- as I said, I don't use them).

> I can look into it later this month, but for the short term, I'm a
> little squeezed for time (work, finishing school, etc.).  I am a bit
> curious, however, I could have sworn that bytes were to become,
> essentially, the 2.x str type (without some methods).  If that is the
> case, no changes should really be necessary, except for a few small
> changes to asynchat with regards to line terminators.

> Then again, I haven't really been keeping up in the p3k/pydev lists
> for the last 6-8 months, and only the subject line of this posting
> caught my eye (because I use asyncore/asynchat, and support users of
> asyncore/asynchat that contact me directly).

You're exactly right about the (lack of) problems and the correct way
to fix them. I've placed a patch in the bug tracker that takes that
very approach.

http://bugs.python.org/issue1563

From mithrandi-python-dev at mithrandi.za.net  Sun Dec  9 04:35:01 2007
From: mithrandi-python-dev at mithrandi.za.net (Tristan Seligmann)
Date: Sun, 9 Dec 2007 05:35:01 +0200
Subject: [Python-Dev] PEP Idea: Syntactic sugar for StopIteration.
In-Reply-To: <20071209012017.CDB913A4068@sparrow.telecommunity.com>
References: <796874fc0712081049j47190c21ma0850da195b8ee18@mail.gmail.com>
	<005701c839df$a4ff63b0$9200a8c0@RaymondLaptop1>
	<796874fc0712081413t6e9ef92fr227a906d283b8636@mail.gmail.com>
	<475B322B.8030801@canterbury.ac.nz>
	<796874fc0712081655h27461609l3ec9b88f4a7e41b9@mail.gmail.com>
	<20071209012017.CDB913A4068@sparrow.telecommunity.com>
Message-ID: <20071209033500.GA8324@mithrandi.za.net>

* Phillip J. Eby <pje at telecommunity.com> [2007-12-08 20:19:29 -0500]:

> At 07:55 PM 12/8/2007 -0500, Manuel Alejandro Cer?n Estrada wrote:
> >2007/12/8, Greg Ewing <greg.ewing at canterbury.ac.nz>:
> > > I would put it the other way around -- the problem
> > > that 'yield break' is meant to solve is already solved
> > > by 'return'. So there's no need for change.
> >
> >I have been re-thinking the problem and this is true. The only
> >exception would be empty generators, but they are rare.
> 
> Note that:
> 
>     def g():
>         return
>         yield

And this isn't a generator at all, but is almost the same:

    def h():
        return iter([])
-- 
mithrandi, i Ainil en-Balandor, a faer Ambar
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20071209/229b1868/attachment.pgp 

From gnewsg at gmail.com  Sun Dec  9 11:52:46 2007
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Sun, 9 Dec 2007 02:52:46 -0800 (PST)
Subject: [Python-Dev] Python-Dev Digest, Vol 53, Issue 23
In-Reply-To: <ddd408be0712081856g2a7bc9bayd16bb93d6b20d67@mail.gmail.com>
References: <mailman.9156.1197152144.13604.python-dev@python.org> 
	<ddd408be0712081856g2a7bc9bayd16bb93d6b20d67@mail.gmail.com>
Message-ID: <5286d63a-39ee-4b1d-a061-c3ca25c6415e@i29g2000prf.googlegroups.com>

Some things I would change in the docstrings:

> +    A dispatcher object handles a single socket, processing connect,
> +    accept, close, read and write events as defined by the child
> +    handle_connect, handle_accept, handle_close, handle_read and
> +    handle_write methods, respectively.  The child may also define
> +    handle_expt to handle exceptions raised during the communication
> +    process.

handle_expt is used for managing of OOB (Out Of Band) data.
The method called when an unhandled exception is raised is
dispatcher.handle_error().

> +        """Sets the socket.SO_REUSEADDR socket option, is possible

Typo ("IF possible").

>      def handle_error(self):
> +        """Internal method, do not override."""

I would change into: "Called when an exception is raised and not
otherwise handled. The default version prints a condensed traceback.",
the same thing reported in the doc.

>      def handle_expt(self):
> +        """Called to handle an exception event.
> +
> +        Children may override this method to implement exception
> +        processing.
> +
> +        """

Like said above, this is called when arrived some OOB data.
I would change this into something like: "Called when some OOB data
arrived."

From foom at fuhm.net  Sun Dec  9 17:35:10 2007
From: foom at fuhm.net (James Y Knight)
Date: Sun, 9 Dec 2007 11:35:10 -0500
Subject: [Python-Dev] Python-Dev Digest, Vol 53, Issue 23
In-Reply-To: <5286d63a-39ee-4b1d-a061-c3ca25c6415e@i29g2000prf.googlegroups.com>
References: <mailman.9156.1197152144.13604.python-dev@python.org>
	<ddd408be0712081856g2a7bc9bayd16bb93d6b20d67@mail.gmail.com>
	<5286d63a-39ee-4b1d-a061-c3ca25c6415e@i29g2000prf.googlegroups.com>
Message-ID: <37242516-52FD-466A-813C-2FBC11CB3F22@fuhm.net>


On Dec 9, 2007, at 5:52 AM, Giampaolo Rodola' wrote:
>>     def handle_expt(self):
>
> Like said above, this is called when arrived some OOB data.
> I would change this into something like: "Called when some OOB data
> arrived."

Of course, that's not actually true. It's called for whatever the exc  
bit from select indicates, which varies by platform. Oh, and if you're  
using the "poll2" implementation of asyncore, handle_expt is called  
only when there's an error on the socket, instead. Ah, wonderful  
abstraction.

James


From gnewsg at gmail.com  Sun Dec  9 17:45:04 2007
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Sun, 9 Dec 2007 08:45:04 -0800 (PST)
Subject: [Python-Dev] Python-Dev Digest, Vol 53, Issue 23
In-Reply-To: <37242516-52FD-466A-813C-2FBC11CB3F22@fuhm.net>
References: <mailman.9156.1197152144.13604.python-dev@python.org> 
	<ddd408be0712081856g2a7bc9bayd16bb93d6b20d67@mail.gmail.com> 
	<5286d63a-39ee-4b1d-a061-c3ca25c6415e@i29g2000prf.googlegroups.com> 
	<37242516-52FD-466A-813C-2FBC11CB3F22@fuhm.net>
Message-ID: <36730c63-f06a-4a42-bc92-9054f4ed51d7@o42g2000hsc.googlegroups.com>

Mmmmm...
This is what asyncore documentation says about handle_expt:

> Called when there is out of band (OOB) data for a socket connection.
> This will almost never happen, as OOB is tenuously supported and
> rarely used.

So, if you're right, the doc is wrong and should be rewritten.
Or maybe this is just a big mistake: could you please take a look at
Python issue #1541?


On 9 Dic, 17:35, James Y Knight <f... at fuhm.net> wrote:
> On Dec 9, 2007, at 5:52 AM, Giampaolo Rodola' wrote:
>
> >>     def handle_expt(self):
>
> > Like said above, this is called when arrived some OOB data.
> > I would change this into something like: "Called when some OOB data
> > arrived."
>
> Of course, that's not actually true. It's called for whatever the exc  
> bit from select indicates, which varies by platform. Oh, and if you're  
> using the "poll2" implementation of asyncore, handle_expt is called  
> only when there's an error on the socket, instead. Ah, wonderful  
> abstraction.
>
> James
>
> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From skip at pobox.com  Sun Dec  9 18:20:50 2007
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 9 Dec 2007 11:20:50 -0600
Subject: [Python-Dev] Missing _sqlite checkin on trunk?
Message-ID: <18268.9202.666896.755038@montanaro.dyndns.org>

On Nov 25 Gerhard Haering checked in a change to the release25-maint branch:

    Author: gerhard.haering
    Date: Sun Nov 25 18:40:35 2007
    New Revision: 59184

    Modified:
       python/branches/release25-maint/Modules/_sqlite/statement.c
       python/branches/release25-maint/Modules/_sqlite/util.c
    Log:
    - Backported a workaround for a bug in SQLite 3.2.x/3.3.x versions where a
      statement recompilation with no bound parameters lead to a segfault
    - Backported a fix necessary because of an SQLite API change in version 3.5.
      This prevents segfaults when executing empty queries, like our test suite
      does.

This bug is also present on the trunk, yet I saw no indication that he
checked in such a fix there.  Gerhard's patch applies cleanly.  I sent him
an email after I saw the checkin and verified that the patch worked on the
trunk, but have yet to hear back from him.

Is there some different method for getting sqlite changes into the trunk?

Thx,

Skip


From glyph at divmod.com  Sun Dec  9 19:16:36 2007
From: glyph at divmod.com (glyph at divmod.com)
Date: Sun, 09 Dec 2007 18:16:36 -0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
Message-ID: <20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>


On 12:21 am, guido at python.org wrote:
>Anyway, I would still like to discuss this on #python-dev Monday.
>Adam, in what time zone are you? (I'm PST.) Who else is interested?

I'm also interested.  I'm EST, but my schedule is very flexible (I'm on 
IRC pretty much all day for work anyway).  Just let me know when it is.

From facundobatista at gmail.com  Mon Dec 10 14:12:23 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Mon, 10 Dec 2007 10:12:23 -0300
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <475A69D4.9070204@ronadam.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>
	<e04bdf310709191341l289c034fjb4bf6d6b88ae4ef9@mail.gmail.com>
	<46F1B6FD.70007@ronadam.com>
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>
	<471F7881.1070605@ronadam.com>
	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>
	<4720E7FF.7080404@ronadam.com>
	<e04bdf310711011213u68d88d35t96b7e9c330fd9c53@mail.gmail.com>
	<e04bdf310712071227w12dd74e2l2b14809160e780f6@mail.gmail.com>
	<475A69D4.9070204@ronadam.com>
Message-ID: <e04bdf310712100512l76ee1b91tcf3b4a9992ef9378@mail.gmail.com>

2007/12/8, Ron Adam <rrr at ronadam.com>:

> Looks much improved!  :-)

Thanks!


> Maybe components and keywords could be combined together and use check
> boxes so more than one item at a time can be selected?

Regarding the combination, I don't think so: I'm just showing the info
from the Tracker differently, I don't want to change its concepts.

Regarding the check-more-than-one: It could be done, but I don't know
if it actually will prove useful (it's always nice to be in more
control of the filters, but overcomplicating the searchs does not help
anybody, there's a point there in the middle, :) ).


> Ideally the temporal bar could be more like a mini gant/status chart which
> indicates the status as well as th activity.  Maybe the background color
> can change when the status changes.

For this the concept of "process" need to be added to the Tracker.
Today there's no such mandatory process for the issues.

As I just show that info differently, I won't create an artificial process here.

Thanks for your different proposals!!

Regards,

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From amk at amk.ca  Mon Dec 10 14:28:05 2007
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 10 Dec 2007 08:28:05 -0500
Subject: [Python-Dev] Bug Day in January?
In-Reply-To: <fjcch2$mlt$1@ger.gmane.org>
References: <fjcch2$mlt$1@ger.gmane.org>
Message-ID: <20071210132805.GA8280@amk-desktop.matrixgroup.net>

On Fri, Dec 07, 2007 at 10:04:38PM +0100, Georg Brandl wrote:
> there wasn't much response to the bug day proposal, but I still think it's
> a good idea. I'd propose a date in January, when Christmas etc. is over.
> It would also be nice if someone did the organizing who hasn't got a
> daily batch of GHOP tasks to review and commit :)

I agree that a bug day would be a good idea, and am happy to help with
the organizing.  What needs to be done, once a date has been set and a
bunch of announcements have been sent out?

--amk

From gh at ghaering.de  Mon Dec 10 16:33:22 2007
From: gh at ghaering.de (=?ISO-8859-1?Q?Gerhard_H=E4ring?=)
Date: Mon, 10 Dec 2007 16:33:22 +0100
Subject: [Python-Dev] Missing _sqlite checkin on trunk?
In-Reply-To: <18268.9202.666896.755038@montanaro.dyndns.org>
References: <18268.9202.666896.755038@montanaro.dyndns.org>
Message-ID: <475D5C42.8010503@ghaering.de>

skip at pobox.com wrote:
> On Nov 25 Gerhard Haering checked in a change to the release25-maint branch:
> 
>     Author: gerhard.haering
>     Date: Sun Nov 25 18:40:35 2007
>     New Revision: 59184
> 
>     Modified:
>        python/branches/release25-maint/Modules/_sqlite/statement.c
>        python/branches/release25-maint/Modules/_sqlite/util.c
>     Log:
>     - Backported a workaround for a bug in SQLite 3.2.x/3.3.x versions where a
>       statement recompilation with no bound parameters lead to a segfault
>     - Backported a fix necessary because of an SQLite API change in version 3.5.
>       This prevents segfaults when executing empty queries, like our test suite
>       does.
> 
> This bug is also present on the trunk, yet I saw no indication that he
> checked in such a fix there.  Gerhard's patch applies cleanly.  I sent him
> an email after I saw the checkin and verified that the patch worked on the
> trunk, but have yet to hear back from him.

Yes, I remember. I postponed updating the trunk because I planned sync 
it with the lastest pysqlite release instead. I've updated my TODO list.

> Is there some different method for getting sqlite changes into the trunk?

?! From my point of view it's a module like all the others, except it's 
also maintained externally as 'pysqlite'.

-- Gerhard

From djarb at highenergymagic.org  Mon Dec 10 17:20:12 2007
From: djarb at highenergymagic.org (Daniel Arbuckle)
Date: Mon, 10 Dec 2007 08:20:12 -0800
Subject: [Python-Dev] patching asyncore and asynchat
Message-ID: <ddd408be0712100820j1504e4acs637e5f53cad32305@mail.gmail.com>

I've posted a new patch with the documentation for handle_expt and
handle_error adjusted, thanks to the feedback from Giampaolo Rodola
and James Y Knight.

http://bugs.python.org/issue1563

From rrr at ronadam.com  Mon Dec 10 19:41:43 2007
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 10 Dec 2007 12:41:43 -0600
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <e04bdf310712100512l76ee1b91tcf3b4a9992ef9378@mail.gmail.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>	
	<e04bdf310709191341l289c034fjb4bf6d6b88ae4ef9@mail.gmail.com>	
	<46F1B6FD.70007@ronadam.com>	
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>	
	<471F7881.1070605@ronadam.com>	
	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>	
	<4720E7FF.7080404@ronadam.com>	
	<e04bdf310711011213u68d88d35t96b7e9c330fd9c53@mail.gmail.com>	
	<e04bdf310712071227w12dd74e2l2b14809160e780f6@mail.gmail.com>	
	<475A69D4.9070204@ronadam.com>
	<e04bdf310712100512l76ee1b91tcf3b4a9992ef9378@mail.gmail.com>
Message-ID: <475D8867.8000607@ronadam.com>



Facundo Batista wrote:
 > 2007/12/8, Ron Adam <rrr at ronadam.com>:
 >
 >> Looks much improved!  :-)
 >
 > Thanks!
 >
 >
 >> Maybe components and keywords could be combined together and use check
 >> boxes so more than one item at a time can be selected?
 >
 > Regarding the combination, I don't think so: I'm just showing the info
 > from the Tracker differently, I don't want to change its concepts.
 >
 > Regarding the check-more-than-one: It could be done, but I don't know
 > if it actually will prove useful (it's always nice to be in more
 > control of the filters, but overcomplicating the searchs does not help
 > anybody, there's a point there in the middle, :) ).

Ok, looking at it again, I agree.


 >> Ideally the temporal bar could be more like a mini gant/status chart which
 >> indicates the status as well as th activity.  Maybe the background color
 >> can change when the status changes.
 >
 > For this the concept of "process" need to be added to the Tracker.
 > Today there's no such mandatory process for the issues.
 >
 > As I just show that info differently, I won't create an artificial 
process here.
 >
 > Thanks for your different proposals!!
 >
 > Regards,


This is from the search page of the tracker.

   <select name="resolution" id="resolution">
     <option value="">don't care</option>

     <option value="" disabled="disabled">------------</option>
     <option value="1">accepted</option>

     <option value="2">duplicate</option>
     <option value="3">fixed</option>
     <option value="4">invalid</option>
     <option value="5">later</option>
     <option value="6">out of date</option>
     <option value="7">postponed</option>

     <option value="8">rejected</option>
     <option value="9">remind</option>
     <option value="10">wont fix</option>
     <option value="11">works for me</option>
   </select>


There are items in the tracker that have a resolutions set, but are still 
open.  So the resolution field is the process status field.  I don't think 
it needs to be mandatory.  And the status field includes open/pending/close.

Do you have access to these and the times they are set?


Cheers,
   Ron


From facundobatista at gmail.com  Mon Dec 10 20:48:33 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Mon, 10 Dec 2007 16:48:33 -0300
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <475D8867.8000607@ronadam.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>
	<471F7881.1070605@ronadam.com>
	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>
	<4720E7FF.7080404@ronadam.com>
	<e04bdf310711011213u68d88d35t96b7e9c330fd9c53@mail.gmail.com>
	<e04bdf310712071227w12dd74e2l2b14809160e780f6@mail.gmail.com>
	<475A69D4.9070204@ronadam.com>
	<e04bdf310712100512l76ee1b91tcf3b4a9992ef9378@mail.gmail.com>
	<475D8867.8000607@ronadam.com>
Message-ID: <e04bdf310712101148w24b8e8a3kabeda0d05933fc2b@mail.gmail.com>

2007/12/10, Ron Adam <rrr at ronadam.com>:

> This is from the search page of the tracker.
>
>    <select name="resolution" id="resolution">
>      <option value="">don't care</option>
>
>      <option value="" disabled="disabled">------------</option>
>      <option value="1">accepted</option>
>
>      <option value="2">duplicate</option>
>      <option value="3">fixed</option>
>      <option value="4">invalid</option>
>      <option value="5">later</option>
>      <option value="6">out of date</option>
>      <option value="7">postponed</option>
>
>      <option value="8">rejected</option>
>      <option value="9">remind</option>
>      <option value="10">wont fix</option>
>      <option value="11">works for me</option>
>    </select>
>
> There are items in the tracker that have a resolutions set, but are still
> open.  So the resolution field is the process status field.  I don't think
> it needs to be mandatory.  And the status field includes open/pending/close.

Some "resolutions" imply that the issue is still open, and some imply
that the issue should be closed.

For example, I won't expect to see the issue still open if it's marked
as "won't fix", "duplicate", "fixed", "invalid", "out of date", or
"rejected".

OTOH, it should be still open if the resolution is "later", or
"remind". I'm not decided with "works for me".

Anyway, as I only show the open issues, the resolution should be one
of the later. Do you think that is useful the color to change if it is
in "later" or "remind"? What's the benefit of this?

Note that if we find open issues that are in the first group of
resolutions, something should be corrected there.

Furthermore, I remember that somewhere there was a discussion of these
values, and if we should keep all of them or not, but I can't find
that thread.

Thank you!

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From guido at python.org  Mon Dec 10 20:54:47 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Dec 2007 11:54:47 -0800
Subject: [Python-Dev] patching asyncore and asynchat
In-Reply-To: <ddd408be0712100820j1504e4acs637e5f53cad32305@mail.gmail.com>
References: <ddd408be0712100820j1504e4acs637e5f53cad32305@mail.gmail.com>
Message-ID: <ca471dc20712101154m43d97345t5af0661112810b2d@mail.gmail.com>

On Dec 10, 2007 8:20 AM, Daniel Arbuckle <djarb at highenergymagic.org> wrote:
> I've posted a new patch with the documentation for handle_expt and
> handle_error adjusted, thanks to the feedback from Giampaolo Rodola
> and James Y Knight.
>
> http://bugs.python.org/issue1563

Can someone else who understands asynchat please review this?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 10 21:02:02 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Dec 2007 12:02:02 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
References: <20071207045606.GA13636@tummy.com> <475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
Message-ID: <ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>

On Dec 9, 2007 10:16 AM,  <glyph at divmod.com> wrote:
> On 12:21 am, guido at python.org wrote:
> >Anyway, I would still like to discuss this on #python-dev Monday.
> >Adam, in what time zone are you? (I'm PST.) Who else is interested?
>
> I'm also interested.  I'm EST, but my schedule is very flexible (I'm on
> IRC pretty much all day for work anyway).  Just let me know when it is.

Adam & I are now on #python-dev. Can you join?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gnewsg at gmail.com  Mon Dec 10 21:09:37 2007
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Mon, 10 Dec 2007 12:09:37 -0800 (PST)
Subject: [Python-Dev] patching asyncore and asynchat
In-Reply-To: <ddd408be0712100820j1504e4acs637e5f53cad32305@mail.gmail.com>
References: <ddd408be0712100820j1504e4acs637e5f53cad32305@mail.gmail.com>
Message-ID: <18096f82-1fb3-47a4-8e48-0ff15ccf70a7@b40g2000prf.googlegroups.com>

I remembered right now that there's a patch pending which should be
included in the trunk before solving issues related to py3k and/or
applying other changes:
http://bugs.python.org/issue1736190
Since it solves a lot of older and newer asyncore/chat issues I guess
you should work on that one instead of using the current asyncore/chat
versions available in the trunk.

From guido at python.org  Tue Dec 11 00:26:09 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Dec 2007 15:26:09 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
	<ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>
Message-ID: <ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>

> Adam & I are now on #python-dev. Can you join?

I think we successfully resolved this. Adam Olsen will produce a patch
that allows one to specify a single file descriptor to which a zero
byte will be written by the C-level signal handler. Twisted and PyGTK
will have to coordinate about this file descriptor. Glyph believes
this is possible and sufficient.

(A preliminary version of the patch may be found here:
http://dpaste.com/27576/ )

We considered two alternatives:

(a) A patch by myself where the file descriptor would instead be
passed together with a signal handler. This was eventually rejected
because it places an extra burden on every piece of code that
registers a signal handler.

(b) A more elaborate patch by Adam which would allow many file
descriptors to be registered. This was rejected for being more code
and solving a problem that most likely doesn't exist (multiple
independent main loops running in different threads).

We also located the exact source of the 100 msec timeout in PyGTK:

http://svn.gnome.org/viewvc/pygtk/trunk/gtk/gtk.override?annotate=2926

line 1075: *timeout = 100;

The recommendation for the OLPC XO project is to remove this line or
make the timeout much larger, as the only reason why this was even
added to PyGTK is wanting a fast response to ^C from the console,
which doesn't represent a viable use case on the XO.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rhamph at gmail.com  Tue Dec 11 00:28:18 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Mon, 10 Dec 2007 16:28:18 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
	<ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>
	<ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
Message-ID: <aac2c7cb0712101528t68bd5f56w1b8625250747aa41@mail.gmail.com>

On Dec 10, 2007 4:26 PM, Guido van Rossum <guido at python.org> wrote:
> > Adam & I are now on #python-dev. Can you join?
>
> I think we successfully resolved this. Adam Olsen will produce a patch
> that allows one to specify a single file descriptor to which a zero
> byte will be written by the C-level signal handler. Twisted and PyGTK
> will have to coordinate about this file descriptor. Glyph believes
> this is possible and sufficient.

http://bugs.python.org/issue1583

>
> (A preliminary version of the patch may be found here:
> http://dpaste.com/27576/ )
>
> We considered two alternatives:
>
> (a) A patch by myself where the file descriptor would instead be
> passed together with a signal handler. This was eventually rejected
> because it places an extra burden on every piece of code that
> registers a signal handler.
>
> (b) A more elaborate patch by Adam which would allow many file
> descriptors to be registered. This was rejected for being more code
> and solving a problem that most likely doesn't exist (multiple
> independent main loops running in different threads).
>
> We also located the exact source of the 100 msec timeout in PyGTK:
>
> http://svn.gnome.org/viewvc/pygtk/trunk/gtk/gtk.override?annotate=2926
>
> line 1075: *timeout = 100;
>
> The recommendation for the OLPC XO project is to remove this line or
> make the timeout much larger, as the only reason why this was even
> added to PyGTK is wanting a fast response to ^C from the console,
> which doesn't represent a viable use case on the XO.
>
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/rhamph%40gmail.com
>



-- 
Adam Olsen, aka Rhamphoryncus

From handzisk at tkn.tu-berlin.de  Tue Dec 11 00:46:18 2007
From: handzisk at tkn.tu-berlin.de (Vlado Handziski)
Date: Tue, 11 Dec 2007 00:46:18 +0100
Subject: [Python-Dev] Non-portable address length calculation for AF_UNIX
	sockets in socketmodule.c
Message-ID: <697986ec0712101546v55f75861y7557f120c903dc84@mail.gmail.com>

Hi all. I recently stumbled upon an address length calculation problem in
socketmodule.c, for UNIX sockets and anonymous "linux" sockets, that is
triggered on some embedded platforms due to padding issues.

I have submitted a patch dealing with the issue about 4 months ago, but it
is still unassigned, and the bug remains even in 3.0:

http://bugs.python.org/issue1754489

I would be very grateful if some of the core developers can take a look at
the issue so this can be fixed in time for 2.6.

Vlado
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071211/7197ffb7/attachment.htm 

From gjcarneiro at gmail.com  Tue Dec 11 01:07:09 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Tue, 11 Dec 2007 00:07:09 +0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
	<ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>
	<ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
Message-ID: <a467ca4f0712101607j6df34f9cp978f551a0c7257ed@mail.gmail.com>

On 10/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> > Adam & I are now on #python-dev. Can you join?
>
> I think we successfully resolved this. Adam Olsen will produce a patch
> that allows one to specify a single file descriptor to which a zero
> byte will be written by the C-level signal handler. Twisted and PyGTK
> will have to coordinate about this file descriptor. Glyph believes
> this is possible and sufficient.


Great that a solution was found at last.  But to be honest I don't
understand why my patch (the second one) is not good.  Why is it preferable
for libraries to provide their own file descriptor?

Are people concerned about the overhead of always creating a pipe even it
may not be used?
Always creating a pipe would at least avoid the need for any "coordination"
between PyGTK and Twisted.

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071211/12ceaf00/attachment.htm 

From guido at python.org  Tue Dec 11 01:10:34 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 10 Dec 2007 16:10:34 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <a467ca4f0712101607j6df34f9cp978f551a0c7257ed@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
	<ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>
	<ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
	<a467ca4f0712101607j6df34f9cp978f551a0c7257ed@mail.gmail.com>
Message-ID: <ca471dc20712101610n19517d3bw5dfa3798b17cad7d@mail.gmail.com>

On Dec 10, 2007 4:07 PM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> On 10/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> > > Adam & I are now on #python-dev. Can you join?
> >
> > I think we successfully resolved this. Adam Olsen will produce a patch
> > that allows one to specify a single file descriptor to which a zero
> > byte will be written by the C-level signal handler. Twisted and PyGTK
> > will have to coordinate about this file descriptor. Glyph believes
> > this is possible and sufficient.
>
> Great that a solution was found at last.  But to be honest I don't
> understand why my patch (the second one) is not good.  Why is it preferable
> for libraries to provide their own file descriptor?

So the app is in charge of creating the pipe if it wants it.

We expect that if the pipe is always created, apps that assume only
fds 0, 1 and 2 are open may get confused or accidentally close it,
etc.

> Are people concerned about the overhead of always creating a pipe even it
> may not be used?  Always creating a pipe would at least avoid the need for
> any "coordination" between PyGTK and Twisted.

I don't think that anyone involved though the need for coordination
was an issue.

Please let this rest. We have a solution.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From gjcarneiro at gmail.com  Tue Dec 11 01:18:07 2007
From: gjcarneiro at gmail.com (Gustavo Carneiro)
Date: Tue, 11 Dec 2007 00:18:07 +0000
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712101610n19517d3bw5dfa3798b17cad7d@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
	<ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>
	<ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
	<a467ca4f0712101607j6df34f9cp978f551a0c7257ed@mail.gmail.com>
	<ca471dc20712101610n19517d3bw5dfa3798b17cad7d@mail.gmail.com>
Message-ID: <a467ca4f0712101618v2aecb8dfp5b2e51c450550124@mail.gmail.com>

On 11/12/2007, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 10, 2007 4:07 PM, Gustavo Carneiro <gjcarneiro at gmail.com> wrote:
> > On 10/12/2007, Guido van Rossum <guido at python.org> wrote:
> >
> > > > Adam & I are now on #python-dev. Can you join?
> > >
> > > I think we successfully resolved this. Adam Olsen will produce a patch
> > > that allows one to specify a single file descriptor to which a zero
> > > byte will be written by the C-level signal handler. Twisted and PyGTK
> > > will have to coordinate about this file descriptor. Glyph believes
> > > this is possible and sufficient.
> >
> > Great that a solution was found at last.  But to be honest I don't
> > understand why my patch (the second one) is not good.  Why is it
> preferable
> > for libraries to provide their own file descriptor?
>
> So the app is in charge of creating the pipe if it wants it.
>
> We expect that if the pipe is always created, apps that assume only
> fds 0, 1 and 2 are open may get confused or accidentally close it,
> etc.


It's the first time I hear about this problem, but sounds plausible.

> Are people concerned about the overhead of always creating a pipe even it
> > may not be used?  Always creating a pipe would at least avoid the need
> for
> > any "coordination" between PyGTK and Twisted.
>
> I don't think that anyone involved though the need for coordination
> was an issue.


Yeah.   Thinking again on this, I don't think this is a problem, not because
coordination between PyGTK and Twisted can be done, but because no such
coordination is needed.  PyGTK can have one FD, Twisted another FD.  Since
only one of the main loops can be running, there's no conflict.

Please let this rest. We have a solution.


Sure.  :-)

-- 
Gustavo J. A. M. Carneiro
INESC Porto, Telecommunications and Multimedia Unit
"The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071211/3f9f906c/attachment.htm 

From gnewsg at gmail.com  Tue Dec 11 02:34:46 2007
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Mon, 10 Dec 2007 17:34:46 -0800 (PST)
Subject: [Python-Dev] patching asyncore and asynchat
In-Reply-To: <18096f82-1fb3-47a4-8e48-0ff15ccf70a7@b40g2000prf.googlegroups.com>
References: <ddd408be0712100820j1504e4acs637e5f53cad32305@mail.gmail.com> 
	<18096f82-1fb3-47a4-8e48-0ff15ccf70a7@b40g2000prf.googlegroups.com>
Message-ID: <c6d4e84b-8edf-4447-bf22-fac42df270f3@p69g2000hsa.googlegroups.com>

I provided a patch for the last issue I mentioned in the ticket a
month ago:
http://bugs.python.org/issue1736190

From rrr at ronadam.com  Tue Dec 11 04:15:43 2007
From: rrr at ronadam.com (Ron Adam)
Date: Mon, 10 Dec 2007 21:15:43 -0600
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <e04bdf310712101148w24b8e8a3kabeda0d05933fc2b@mail.gmail.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>	
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>	
	<471F7881.1070605@ronadam.com>	
	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>	
	<4720E7FF.7080404@ronadam.com>	
	<e04bdf310711011213u68d88d35t96b7e9c330fd9c53@mail.gmail.com>	
	<e04bdf310712071227w12dd74e2l2b14809160e780f6@mail.gmail.com>	
	<475A69D4.9070204@ronadam.com>	
	<e04bdf310712100512l76ee1b91tcf3b4a9992ef9378@mail.gmail.com>	
	<475D8867.8000607@ronadam.com>
	<e04bdf310712101148w24b8e8a3kabeda0d05933fc2b@mail.gmail.com>
Message-ID: <475E00DF.3030600@ronadam.com>



Facundo Batista wrote:
> 2007/12/10, Ron Adam <rrr at ronadam.com>:
> 
>> This is from the search page of the tracker.
>>
>>    <select name="resolution" id="resolution">
>>      <option value="">don't care</option>
>>
>>      <option value="" disabled="disabled">------------</option>
>>      <option value="1">accepted</option>
>>
>>      <option value="2">duplicate</option>
>>      <option value="3">fixed</option>
>>      <option value="4">invalid</option>
>>      <option value="5">later</option>
>>      <option value="6">out of date</option>
>>      <option value="7">postponed</option>
>>
>>      <option value="8">rejected</option>
>>      <option value="9">remind</option>
>>      <option value="10">wont fix</option>
>>      <option value="11">works for me</option>
>>    </select>
>>
>> There are items in the tracker that have a resolutions set, but are still
>> open.  So the resolution field is the process status field.  I don't think
>> it needs to be mandatory.  And the status field includes open/pending/close.
> 
> Some "resolutions" imply that the issue is still open, and some imply
> that the issue should be closed.
> 
> For example, I won't expect to see the issue still open if it's marked
> as "won't fix", "duplicate", "fixed", "invalid", "out of date", or
> "rejected".

There are open items with those items set.

    won't fix      4
    duplicate      0
    fixed          3
    invalid        2
    out of date    3
    rejected       2

Some of these may have been closed and reopened at some point.


> OTOH, it should be still open if the resolution is "later", or
> "remind". I'm not decided with "works for me".

> Anyway, as I only show the open issues, 

Is it "Open" issues or "Not Closed" issues?  The later would include 
"pending" issues as well.


the resolution should be one
> of the later. Do you think that is useful the color to change if it is
> in "later" or "remind"? What's the benefit of this?
 >
> Note that if we find open issues that are in the first group of
> resolutions, something should be corrected there.

With 1,328 Open issues, anything that may help move things along would be good.



I was thinking of maybe 4 background colors + grey,

     (Trying to use the existing resolution/status terms.)

     light grey:
              no resolution yet
              works for me?  (can't reproduce?)

     Positive Progression points:  light green
              works for me?  (working patch/fix available?)
              accepted       (patch accepted)
              fixed          (patch applied)

     Don't close yet:  light yellow
             later
             remind
             postponed
             pending
             out of date?      (patch no longer works?)

     ready to close:  light red
             duplicate
             won't fix
             rejected
             invalid


With darker color vertical marks for status/resolution change points.


> Furthermore, I remember that somewhere there was a discussion of these
> values, and if we should keep all of them or not, but I can't find
> that thread.

Possibly Here ...

http://mail.python.org/pipermail/python-dev/2007-November/075160.html

It was pointed out that these are hold-overs for SourceForge's bug tracker, 
and redesigning the work flow triage is something that needs to be done.

It refers to ...

http://wiki.python.org/moin/TrackerDocs/



> Thank you!

You're Welcome!

Ron





From djarb at highenergymagic.org  Tue Dec 11 19:03:00 2007
From: djarb at highenergymagic.org (Daniel Arbuckle)
Date: Tue, 11 Dec 2007 10:03:00 -0800
Subject: [Python-Dev] patching asyncore and asynchat
Message-ID: <ddd408be0712111003r219efd0at80ab7a1617156ed1@mail.gmail.com>

> From: "Giampaolo Rodola'" <gnewsg at gmail.com>
> I remembered right now that there's a patch pending which should be
> included in the trunk before solving issues related to py3k and/or
> applying other changes:
> http://bugs.python.org/issue1736190
> Since it solves a lot of older and newer asyncore/chat issues I guess
> you should work on that one instead of using the current asyncore/chat
> versions available in the trunk.

Those patches can't be applied directly to the py3k branch, where
Guido asked me to work, so instead I've ported them and merged them
with my patch where appropriate. This merged patch should address both
py3k porting and the issues in raised in 1736190

http://bugs.python.org/issue1563

From josepharmbruster at gmail.com  Tue Dec 11 20:08:16 2007
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Tue, 11 Dec 2007 14:08:16 -0500
Subject: [Python-Dev] builtin_format function code-spacing in bltinmodule.c
Message-ID: <938f42d70712111108q2ffad13bw9014733a865725cf@mail.gmail.com>

All,

Not sure if this is significant or not but the spacing of the builtin_format
function is not consistent with the
rest of the bltinmodule.c file.

Joseph Armbruster
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071211/1627247f/attachment.htm 

From guido at python.org  Tue Dec 11 20:13:27 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 11 Dec 2007 11:13:27 -0800
Subject: [Python-Dev] builtin_format function code-spacing in
	bltinmodule.c
In-Reply-To: <938f42d70712111108q2ffad13bw9014733a865725cf@mail.gmail.com>
References: <938f42d70712111108q2ffad13bw9014733a865725cf@mail.gmail.com>
Message-ID: <ca471dc20712111113v4087bd6ah5e9518b934d31b16@mail.gmail.com>

It is important; care to submit a fix?

On Dec 11, 2007 11:08 AM, Joseph Armbruster <josepharmbruster at gmail.com> wrote:
> All,
>
> Not sure if this is significant or not but the spacing of the builtin_format
> function is not consistent with the
> rest of the bltinmodule.c file.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From lists at janc.be  Wed Dec 12 01:54:42 2007
From: lists at janc.be (Jan Claeys)
Date: Wed, 12 Dec 2007 01:54:42 +0100
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071207142642.GC13636@tummy.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
Message-ID: <1197420882.10215.380.camel@localhost>

Op vrijdag 07-12-2007 om 07:26 uur [tijdzone -0700], schreef Sean
Reifschneider:
> I would say that this is an optimization that helps a specific set of
> platforms, including one that I think we really care about, the OLPC
> which needs it for decreased battery use.

Almost every laptop user would benefit from it, and even some desktop or
server users might save on their electric power bill...


-- 
Jan Claeys


From guido at python.org  Wed Dec 12 02:03:47 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 11 Dec 2007 17:03:47 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <1197420882.10215.380.camel@localhost>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
Message-ID: <ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>

On Dec 11, 2007 4:54 PM, Jan Claeys <lists at janc.be> wrote:
> Op vrijdag 07-12-2007 om 07:26 uur [tijdzone -0700], schreef Sean
> Reifschneider:
> > I would say that this is an optimization that helps a specific set of
> > platforms, including one that I think we really care about, the OLPC
> > which needs it for decreased battery use.
>
> Almost every laptop user would benefit from it, and even some desktop or
> server users might save on their electric power bill...

Do you have data to support this claim?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg.ewing at canterbury.ac.nz  Wed Dec 12 03:01:18 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Wed, 12 Dec 2007 15:01:18 +1300
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
	<ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
Message-ID: <475F40EE.309@canterbury.ac.nz>

Guido van Rossum wrote:

> On Dec 11, 2007 4:54 PM, Jan Claeys <lists at janc.be> wrote:
> 
>>Almost every laptop user would benefit from it, and even some desktop or
>>server users might save on their electric power bill...
> 
> 
> Do you have data to support this claim?

Even if it doesn't save any power, using CPU unnecessarily
is a bad thing for any application to do on a multitasking
system.

-- 
Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | Carpe post meridiem!          	  |
Christchurch, New Zealand	   | (I'm not a morning person.)          |
greg.ewing at canterbury.ac.nz	   +--------------------------------------+

From josepharmbruster at gmail.com  Wed Dec 12 03:23:18 2007
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Tue, 11 Dec 2007 21:23:18 -0500
Subject: [Python-Dev] sliceobject Py_None step inquiry
Message-ID: <475F4616.20700@gmail.com>


I was playing around with sliceobject.c this evening and noticed the following 
behavior.  If you slice with a step 0, you receive a ValueError but when you 
slice with a step of None, the step is set to 1.  As an example, observe the 
following interactive session:

 >>> a = [1,2,3,4,5,6]
 >>> b = slice(0,5,None)
 >>> a[b]
[1, 2, 3, 4, 5]
 >>> b = slice(0,5,0)
 >>> a[b]
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ValueError: slice step cannot be zero
 >>>

Within the code, it looks like Py_None performs a step of 1.  Does it make 
sense to create a patch so that None and 0 behave the same in this respect?

 >>> a = [1,2,3,4,5,6]
 >>> b = slice(0,5,None)
 >>> a[b]
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ValueError: slice step cannot be None
 >>> b = slice(0,5,0)
 >>> a[b]
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
ValueError: slice step cannot be zero
 >>> b = slice(0,5)
 >>> a[b]
[1, 2, 3, 4, 5]
 >>>


Joe

From ironfroggy at socialserve.com  Wed Dec 12 03:43:13 2007
From: ironfroggy at socialserve.com (Calvin Spealman)
Date: Tue, 11 Dec 2007 21:43:13 -0500
Subject: [Python-Dev] sliceobject Py_None step inquiry
In-Reply-To: <475F4616.20700@gmail.com>
References: <475F4616.20700@gmail.com>
Message-ID: <C4CAC1E1-8AD7-4992-85FA-7566CFAA47F1@socialserve.com>

I think that should not change. None is different than 0. It makes  
sense to use it as a "use the default value" kind of place holder.  
Silently using 1 when you pass 0 is a very different thing.

Maybe the slice was calculated and the developer should know about it  
being 0, because in this case they really don't want a step of 1, or  
the calculation was broken. There are lots of reasons.

On Dec 11, 2007, at 9:23 PM, Joseph Armbruster wrote:

>
> I was playing around with sliceobject.c this evening and noticed  
> the following
> behavior.  If you slice with a step 0, you receive a ValueError but  
> when you
> slice with a step of None, the step is set to 1.  As an example,  
> observe the
> following interactive session:
>
>>>> a = [1,2,3,4,5,6]
>>>> b = slice(0,5,None)
>>>> a[b]
> [1, 2, 3, 4, 5]
>>>> b = slice(0,5,0)
>>>> a[b]
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> ValueError: slice step cannot be zero
>>>>
>
> Within the code, it looks like Py_None performs a step of 1.  Does  
> it make
> sense to create a patch so that None and 0 behave the same in this  
> respect?
>
>>>> a = [1,2,3,4,5,6]
>>>> b = slice(0,5,None)
>>>> a[b]
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> ValueError: slice step cannot be None
>>>> b = slice(0,5,0)
>>>> a[b]
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
> ValueError: slice step cannot be zero
>>>> b = slice(0,5)
>>>> a[b]
> [1, 2, 3, 4, 5]
>>>>
>
>
> Joe
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ironfroggy%40socialserve.com


From guido at python.org  Wed Dec 12 06:29:35 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 11 Dec 2007 21:29:35 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <475F40EE.309@canterbury.ac.nz>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
	<ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
	<475F40EE.309@canterbury.ac.nz>
Message-ID: <ca471dc20712112129m2025c118tc708c57a9b2246a@mail.gmail.com>

On Dec 11, 2007 6:01 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>
> > On Dec 11, 2007 4:54 PM, Jan Claeys <lists at janc.be> wrote:
> >
> >>Almost every laptop user would benefit from it, and even some desktop or
> >>server users might save on their electric power bill...
> >
> >
> > Do you have data to support this claim?
>
> Even if it doesn't save any power, using CPU unnecessarily
> is a bad thing for any application to do on a multitasking
> system.

Hm, Apple and Microsoft don't seem to think so. They go out of their
way to implement elaborate visual effects.

Again -- is there any data about the cost of PyGTK's waking up 10x/sec
on a typical laptop or server? (The XO is a special case because it
has very different power management abilities.)

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From andrew-pythondev at puzzling.org  Wed Dec 12 07:00:30 2007
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Wed, 12 Dec 2007 17:00:30 +1100
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
	<ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
Message-ID: <20071212060030.GF30806@steerpike.home.puzzling.org>

Guido van Rossum wrote:
> On Dec 11, 2007 4:54 PM, Jan Claeys <lists at janc.be> wrote:
> > Op vrijdag 07-12-2007 om 07:26 uur [tijdzone -0700], schreef Sean
> > Reifschneider:
> > > I would say that this is an optimization that helps a specific set of
> > > platforms, including one that I think we really care about, the OLPC
> > > which needs it for decreased battery use.
> >
> > Almost every laptop user would benefit from it, and even some desktop or
> > server users might save on their electric power bill...
> 
> Do you have data to support this claim?

http://www.lesswatts.org/projects/powertop/powertop.php

Some quotes plucked from that page:

?In the screenshot, the laptop isn't doing very well. Most of the time the
processor is in C2, and then only for an average of 4.4 milliseconds at a time.
If the laptop spent most of its time in C4 for at least 20 milliseconds, the
battery life would have been approximately one hour longer.?

?When running a full GNOME desktop, 3 wakeups per second is achievable.?

There's considerable effort being invested in the GNOME and Linux software stack
at the moment to get rid of unnecessary CPU wakeups, and people are reporting
significant improvements in laptop power consumption as a result of that work.

-Andrew.


From rhamph at gmail.com  Wed Dec 12 08:02:52 2007
From: rhamph at gmail.com (Adam Olsen)
Date: Wed, 12 Dec 2007 00:02:52 -0700
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071212060030.GF30806@steerpike.home.puzzling.org>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
	<ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
	<20071212060030.GF30806@steerpike.home.puzzling.org>
Message-ID: <aac2c7cb0712112302n2801433ag87e9eabcef93c156@mail.gmail.com>

On Dec 11, 2007 11:00 PM, Andrew Bennetts <andrew-pythondev at puzzling.org> wrote:
> Guido van Rossum wrote:
> > On Dec 11, 2007 4:54 PM, Jan Claeys <lists at janc.be> wrote:
> > > Op vrijdag 07-12-2007 om 07:26 uur [tijdzone -0700], schreef Sean
> > > Reifschneider:
> > > > I would say that this is an optimization that helps a specific set of
> > > > platforms, including one that I think we really care about, the OLPC
> > > > which needs it for decreased battery use.
> > >
> > > Almost every laptop user would benefit from it, and even some desktop or
> > > server users might save on their electric power bill...
> >
> > Do you have data to support this claim?
>
> http://www.lesswatts.org/projects/powertop/powertop.php
>
> Some quotes plucked from that page:
>
> "In the screenshot, the laptop isn't doing very well. Most of the time the
> processor is in C2, and then only for an average of 4.4 milliseconds at a time.
> If the laptop spent most of its time in C4 for at least 20 milliseconds, the
> battery life would have been approximately one hour longer."
>
> "When running a full GNOME desktop, 3 wakeups per second is achievable."
>
> There's considerable effort being invested in the GNOME and Linux software stack
> at the moment to get rid of unnecessary CPU wakeups, and people are reporting
> significant improvements in laptop power consumption as a result of that work.

There's a known issues page on there, on the bottom of which is
sealert, which used python, gtk, and threads.  It has since been
rewritten to not use threads, but it did exhibit the problem
set_wakeup_fd fixes (at least provides our half of the fix.)

https://bugzilla.redhat.com/show_bug.cgi?id=239893

-- 
Adam Olsen, aka Rhamphoryncus

From greg.ewing at canterbury.ac.nz  Wed Dec 12 23:42:52 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 13 Dec 2007 11:42:52 +1300
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712112129m2025c118tc708c57a9b2246a@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
	<ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
	<475F40EE.309@canterbury.ac.nz>
	<ca471dc20712112129m2025c118tc708c57a9b2246a@mail.gmail.com>
Message-ID: <476063EC.6000507@canterbury.ac.nz>

Guido van Rossum wrote:

> Hm, Apple and Microsoft don't seem to think so. They go out of their
> way to implement elaborate visual effects.

Well, that's a different issue -- assuming the visual
effect is something wanted, then the CPU required to
produce it isn't unnecessary.

But there's no excuse for using CPU when the application
truly isn't doing anything other than waiting for
something to happen.

--
Greg

From guido at python.org  Wed Dec 12 23:52:18 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 12 Dec 2007 14:52:18 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <476063EC.6000507@canterbury.ac.nz>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
	<ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
	<475F40EE.309@canterbury.ac.nz>
	<ca471dc20712112129m2025c118tc708c57a9b2246a@mail.gmail.com>
	<476063EC.6000507@canterbury.ac.nz>
Message-ID: <ca471dc20712121452i33b59b1ftff8a10168509062@mail.gmail.com>

On Dec 12, 2007 2:42 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> But there's no excuse for using CPU when the application
> truly isn't doing anything other than waiting for
> something to happen.

There are tons of situations where polling is quite reasonable as logn
as it involves a sleep.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From g.brandl at gmx.net  Thu Dec 13 00:44:49 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 13 Dec 2007 00:44:49 +0100
Subject: [Python-Dev] Bug Day in January?
In-Reply-To: <20071210132805.GA8280@amk-desktop.matrixgroup.net>
References: <fjcch2$mlt$1@ger.gmane.org>
	<20071210132805.GA8280@amk-desktop.matrixgroup.net>
Message-ID: <fjprlq$tkk$1@ger.gmane.org>

A.M. Kuchling schrieb:
> On Fri, Dec 07, 2007 at 10:04:38PM +0100, Georg Brandl wrote:
>> there wasn't much response to the bug day proposal, but I still think it's
>> a good idea. I'd propose a date in January, when Christmas etc. is over.
>> It would also be nice if someone did the organizing who hasn't got a
>> daily batch of GHOP tasks to review and commit :)
> 
> I agree that a bug day would be a good idea, and am happy to help with
> the organizing.  What needs to be done, once a date has been set and a
> bunch of announcements have been sent out?

Not too much, apart from making sure as many people as possible attend :)

Last time we've communicated over IRC  and the Wiki, perhaps next time
a Google spreadsheet could be used instead of the Wiki, so one would
need to setup IRC logging and the initial spreadsheet.

There are a couple of pages in the Wiki with infos for participants
which must be brought up to date and perhaps moved into the Google doc too.

A neat thing for newcomers is if someone goes through the bugs beforehand
and selects some that are easily tackleable in the time period of the
bug day.

Georg


From greg at krypto.org  Thu Dec 13 09:44:59 2007
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 13 Dec 2007 00:44:59 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712121452i33b59b1ftff8a10168509062@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
	<ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
	<475F40EE.309@canterbury.ac.nz>
	<ca471dc20712112129m2025c118tc708c57a9b2246a@mail.gmail.com>
	<476063EC.6000507@canterbury.ac.nz>
	<ca471dc20712121452i33b59b1ftff8a10168509062@mail.gmail.com>
Message-ID: <52dc1c820712130044m2a2e7d51s2eb4572fde650d9d@mail.gmail.com>

On 12/12/07, Guido van Rossum <guido at python.org> wrote:
>
> On Dec 12, 2007 2:42 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> > But there's no excuse for using CPU when the application
> > truly isn't doing anything other than waiting for
> > something to happen.
>
> There are tons of situations where polling is quite reasonable as logn
> as it involves a sleep.


Now that I have to disagree with, possibly because sleep is ambiguous as
stated.  Periodic time based polling means your APIs are broken (not that
one often has control over what APIs are available).  Blocking only to be
woken up when any of the events your interested in is always best.  If you
have periodic tasks that must be performed then obviously an event you're
interested in is "time T or X time has passed" but that is distinct from a
process waking up regularly to check the empty work queue length only to
sleep again.

Regardless this thread already resolved the issue with an acceptable
solution (yay!) so further discussion is merely a bike shed.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071213/ab17deba/attachment.htm 

From rrr at ronadam.com  Fri Dec 14 17:57:23 2007
From: rrr at ronadam.com (Ron Adam)
Date: Fri, 14 Dec 2007 10:57:23 -0600
Subject: [Python-Dev] Python tickets summary
In-Reply-To: <e04bdf310712101148w24b8e8a3kabeda0d05933fc2b@mail.gmail.com>
References: <e04bdf310709101325o1315adb0pebcb2f33d8313f35@mail.gmail.com>	
	<e04bdf310710020926k1d02f6efg72f9bcc3ddd95349@mail.gmail.com>	
	<471F7881.1070605@ronadam.com>	
	<e04bdf310710250611h6fc6eab2m1ffee69777e8d3d5@mail.gmail.com>	
	<4720E7FF.7080404@ronadam.com>	
	<e04bdf310711011213u68d88d35t96b7e9c330fd9c53@mail.gmail.com>	
	<e04bdf310712071227w12dd74e2l2b14809160e780f6@mail.gmail.com>	
	<475A69D4.9070204@ronadam.com>	
	<e04bdf310712100512l76ee1b91tcf3b4a9992ef9378@mail.gmail.com>	
	<475D8867.8000607@ronadam.com>
	<e04bdf310712101148w24b8e8a3kabeda0d05933fc2b@mail.gmail.com>
Message-ID: <4762B5F3.1040404@ronadam.com>


* This didn't show up when I sent it the other day, so I'm resending it.


Facundo Batista wrote:
> 2007/12/10, Ron Adam <rrr at ronadam.com>:
> 
>> This is from the search page of the tracker.
>>
>>    <select name="resolution" id="resolution">
>>      <option value="">don't care</option>
>>
>>      <option value="" disabled="disabled">------------</option>
>>      <option value="1">accepted</option>
>>
>>      <option value="2">duplicate</option>
>>      <option value="3">fixed</option>
>>      <option value="4">invalid</option>
>>      <option value="5">later</option>
>>      <option value="6">out of date</option>
>>      <option value="7">postponed</option>
>>
>>      <option value="8">rejected</option>
>>      <option value="9">remind</option>
>>      <option value="10">wont fix</option>
>>      <option value="11">works for me</option>
>>    </select>
>>
>> There are items in the tracker that have a resolutions set, but are still
>> open.  So the resolution field is the process status field.  I don't think
>> it needs to be mandatory.  And the status field includes open/pending/close.
> 
> Some "resolutions" imply that the issue is still open, and some imply
> that the issue should be closed.
> 
> For example, I won't expect to see the issue still open if it's marked
> as "won't fix", "duplicate", "fixed", "invalid", "out of date", or
> "rejected".

There are open items with those items set.

    won't fix      4
    duplicate      0
    fixed          3
    invalid        2
    out of date    3
    rejected       2

Some of these may have been closed and reopened at some point.


> OTOH, it should be still open if the resolution is "later", or
> "remind". I'm not decided with "works for me".

> Anyway, as I only show the open issues, 

Is it "Open" issues or "Not Closed" issues?  The later would include
"pending" issues as well.


 > the resolution should be one
> of the later. Do you think that is useful the color to change if it is
> in "later" or "remind"? What's the benefit of this?
>
> Note that if we find open issues that are in the first group of
> resolutions, something should be corrected there.

With 1,328 Open issues, anything that may help move things along would be good.



I was thinking of maybe 4 background colors + grey,

     (Trying to use the existing resolution terms.)

     light grey:
              no resolution yet
              works for me?  (can't reproduce?)

     Positive Progression points:  light green
              works for me?  (working patch/fix available?)
              accepted       (patch accepted)
              fixed          (patch applied)

     Don't close yet:  light yellow
             later
             remind
             postponed
             pending
             out of date?      (patch no longer works?)

     ready to close:  light red
             duplicate
             won't fix
             rejected
             invalid


With darker color vertical marks for status/resolution change points.


> Furthermore, I remember that somewhere there was a discussion of these
> values, and if we should keep all of them or not, but I can't find
> that thread.

Possibly Here ...

http://mail.python.org/pipermail/python-dev/2007-November/075160.html

It was pointed out that these are hold-overs for SourceForge's bug tracker,
and redesigning the work flow triage is something that needs to be done.

It refers to ...

http://wiki.python.org/moin/TrackerDocs/



> Thank you!

You're Welcome!

Ron






From ijmorlan at cs.uwaterloo.ca  Fri Dec 14 18:38:55 2007
From: ijmorlan at cs.uwaterloo.ca (Isaac Morland)
Date: Fri, 14 Dec 2007 12:38:55 -0500 (EST)
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
	<ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>
	<ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
Message-ID: <Pine.GSO.4.64.0712141236020.25631@cpu102.cs.uwaterloo.ca>

On Mon, 10 Dec 2007, Guido van Rossum wrote:

> I think we successfully resolved this. Adam Olsen will produce a patch
> that allows one to specify a single file descriptor to which a zero
> byte will be written by the C-level signal handler. Twisted and PyGTK

I'm surprised that the byte is zero, rather than being the signal number. 
This would reflect the way C-level signal handlers work, where they are I 
believe called with the signal number.  Is any discussion of this point 
conveniently available online?  I ask for my own understanding, and 
definitely not out of a desire to re-open any settled issues!

Isaac Morland			CSCF Web Guru
DC 2554C, x36650		WWW Software Specialist

From guido at python.org  Fri Dec 14 18:45:58 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 14 Dec 2007 09:45:58 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <Pine.GSO.4.64.0712141236020.25631@cpu102.cs.uwaterloo.ca>
References: <20071207045606.GA13636@tummy.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<aac2c7cb0712081436l514953bbxc9cb804d9307aa6e@mail.gmail.com>
	<ca471dc20712081528i58b60d20s84206907edd1985@mail.gmail.com>
	<aac2c7cb0712081557y6a08d050n80f19550c91199a2@mail.gmail.com>
	<ca471dc20712081621t1bd262dercd6619b44f52dd18@mail.gmail.com>
	<20071209181636.21558.2113815590.divmod.xquotient.2498@joule.divmod.com>
	<ca471dc20712101202n60a78abapd39008cbe1991fdc@mail.gmail.com>
	<ca471dc20712101526h6dd4f54eq8725d594d3da71d8@mail.gmail.com>
	<Pine.GSO.4.64.0712141236020.25631@cpu102.cs.uwaterloo.ca>
Message-ID: <ca471dc20712140945h66339641t5cc1e05769dda1b2@mail.gmail.com>

On Dec 14, 2007 9:38 AM, Isaac Morland <ijmorlan at cs.uwaterloo.ca> wrote:
> > I think we successfully resolved this. Adam Olsen will produce a patch
> > that allows one to specify a single file descriptor to which a zero
> > byte will be written by the C-level signal handler. Twisted and PyGTK
>
> I'm surprised that the byte is zero, rather than being the signal number.
> This would reflect the way C-level signal handlers work, where they are I
> believe called with the signal number.  Is any discussion of this point
> conveniently available online?  I ask for my own understanding, and
> definitely not out of a desire to re-open any settled issues!

Yes, this was discussed previously in this thread. Go find it on
mail.python.org/pipermail/python-dev/

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From status at bugs.python.org  Fri Dec 14 19:06:09 2007
From: status at bugs.python.org (Tracker)
Date: Fri, 14 Dec 2007 18:06:09 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071214180609.293FF78384@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (12/07/07 - 12/14/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1355 open (+41) / 11756 closed (+19) / 13111 total (+60)

Open issues with patches:   423

Average duration of open issues: 689 days.
Median duration of open issues: 901 days.

Open Issues Breakdown
   open  1347 (+40)
pending     8 ( +1)

Issues Created Or Reopened (61)
_______________________________

Add VS CRT redist to the MSI installer                           12/07/07
       http://bugs.python.org/issue1569    created  tiran                    
       py3k                                                                    

Backport sys.maxsize to Python 2.6                               12/07/07
       http://bugs.python.org/issue1570    created  tiran                    
                                                                               

Better description of 'L' repr removal in "What's New"           12/08/07
CLOSED http://bugs.python.org/issue1571    created  salty-horse              
       py3k                                                                    

404 report of SimpleXMLRPCServer is broken                       12/08/07
CLOSED http://bugs.python.org/issue1572    created  tiran                    
       py3k                                                                    

Improper use of the keyword-only syntax makes the parser crash   12/08/07
CLOSED http://bugs.python.org/issue1573    reopened amaury.forgeotdarc       
       py3k                                                                    

Touchpad 2 Finger scroll does not work in IDLE on Mac (But scrol 12/09/07
       http://bugs.python.org/issue1574    created  arorap                   
                                                                               

typo in README                                                   12/09/07
CLOSED http://bugs.python.org/issue1575    created  dalke                    
                                                                               

[Patch] Working post import hook and lazy modules                12/09/07
       http://bugs.python.org/issue1576    created  tiran                    
       py3k, patch                                                             

shutil.move() does not use os.rename() if dst is a directory     12/09/07
       http://bugs.python.org/issue1577    created  init                     
                                                                               

Problems in win_getpass                                          12/10/07
CLOSED http://bugs.python.org/issue1578    created  vizcayno                 
       py3k                                                                    

logging documentation is unclear                                 12/10/07
       http://bugs.python.org/issue1579    created  Kylotan                  
                                                                               

Use shorter float repr when possible                             12/11/07
       http://bugs.python.org/issue1580    reopened gvanrossum               
       py3k, patch                                                             

xmlrpclib.ServerProxy() doesn't use x509 data                    12/10/07
       http://bugs.python.org/issue1581    created  ahasenack                
                                                                               

Documentation patch for reversed() and __reversed__()            12/10/07
       http://bugs.python.org/issue1582    created  mark_t_russell           
                                                                               

Patch for signal.set_wakeup_fd                                   12/10/07
       http://bugs.python.org/issue1583    created  rhamphoryncus            
       patch                                                                   

Mac OS X: building with X11 Tkinter                              12/11/07
       http://bugs.python.org/issue1584    created  ceball                   
                                                                               

IDLE uses non-existent xrange() function (Py30a2)                12/11/07
CLOSED http://bugs.python.org/issue1585    created  mark                     
       py3k                                                                    

IDLE no longer shows colour syntax highlighting in the Shell (Py 12/11/07
CLOSED http://bugs.python.org/issue1586    created  mark                     
       py3k                                                                    

instancemethod wrapper for PyCFunctions                          12/11/07
CLOSED http://bugs.python.org/issue1587    created  tiran                    
       py3k, patch                                                             

str.format() wrongly formats complex() numbers (Py30a2)          12/11/07
       http://bugs.python.org/issue1588    created  mark                     
                                                                               

New SSL module doesn't seem to verify hostname against commonNam 12/11/07
       http://bugs.python.org/issue1589    created  ahasenack                
                                                                               

"make altinstall" installs pydoc, idle, smtpd.py                 12/11/07
       http://bugs.python.org/issue1590    created  dripton                  
                                                                               

popen2.Popen3 class (Unix) documentation misleading / confusing  12/11/07
       http://bugs.python.org/issue1591    created  mtomczak                 
                                                                               

operations on closed shelves fail cryptically                    12/11/07
       http://bugs.python.org/issue1592    created  erno                     
                                                                               

spacing of the builtin_format function is inconsistent           12/11/07
CLOSED http://bugs.python.org/issue1593    created  JosephArmbruster         
       py3k, patch                                                             

MacOS.GetCreatorAndType() and SetCreatorAndType() broken on inte 12/11/07
       http://bugs.python.org/issue1594    created  jvr                      
                                                                               

Probable extra semicolon in Py_LeaveRecursiveCall macro          12/11/07
       http://bugs.python.org/issue1595    created  amaury.forgeotdarc       
       py3k                                                                    

Broken pipes should be handled better in 2.x                     12/11/07
       http://bugs.python.org/issue1596    created  alexandre.vassalotti     
                                                                               

running test_ctypes 25 times in a row causes it to fail          12/11/07
CLOSED http://bugs.python.org/issue1597    created  gvanrossum               
                                                                               

unexpected response in imaplib                                   12/12/07
       http://bugs.python.org/issue1598    created  smoser                   
                                                                               

IDLE hangs if os.spwanv command is given                         12/12/07
       http://bugs.python.org/issue1599    created  Pooja                    
                                                                               

str.format() produces different output on different platforms (P 12/12/07
       http://bugs.python.org/issue1600    created  mark                     
                                                                               

IDLE not working correctly on Windows (Py30a2/IDLE30a1)          12/12/07
       http://bugs.python.org/issue1601    created  mark                     
       py3k                                                                    

windows console doesn't print utf8 (Py30a2)                      12/12/07
       http://bugs.python.org/issue1602    created  mark                     
                                                                               

Wanted behaviour ?                                               12/12/07
CLOSED http://bugs.python.org/issue1603    created  pythonmeister            
                                                                               

collections.deque.__init__ doesn't initialize                    12/12/07
CLOSED http://bugs.python.org/issue1604    created  Horpner                  
                                                                               

Semi autogenerated _types module                                 12/13/07
       http://bugs.python.org/issue1605    created  tiran                    
       py3k, patch                                                             

Doc: subprocess wait() may lead to dead lock                     12/13/07
       http://bugs.python.org/issue1606    created  tiran                    
                                                                               

Patch for TCL 8.5 support                                        12/13/07
       http://bugs.python.org/issue1607    created  tiran                    
       py3k, patch                                                             

test_str.py crashes                                              12/13/07
CLOSED http://bugs.python.org/issue1608    created  cartman                  
                                                                               

test_re.py fails                                                 12/13/07
       http://bugs.python.org/issue1609    created  cartman                  
                                                                               

test_socket.py fails                                             12/13/07
       http://bugs.python.org/issue1610    created  cartman                  
                                                                               

doctest.testmod gets noisy if called more than once per SystemEx 12/13/07
       http://bugs.python.org/issue1611    created  p.lavarre at ieee.org       
                                                                               

infinite recursion when using collections.Sequence in a recursiv 12/13/07
CLOSED http://bugs.python.org/issue1612    created  mark                     
                                                                               

Makefile's VPATH feature is broken                               12/13/07
CLOSED http://bugs.python.org/issue1613    created  tiran                    
       py3k                                                                    

bug in string method "title"                                     12/13/07
CLOSED http://bugs.python.org/issue1614    created  ElGordo                  
                                                                               

descriptor protocol bug                                          12/13/07
       http://bugs.python.org/issue1615    created  gangesmaster             
                                                                               

compiler warnings (gcc 2.96)                                     12/13/07
       http://bugs.python.org/issue1616    created  gvanrossum               
                                                                               

Rare exception in test_urllib2net                                12/13/07
       http://bugs.python.org/issue1617    created  gvanrossum               
                                                                               

locale.strxfrm can't handle non-ascii strings                    12/13/07
       http://bugs.python.org/issue1618    created  filip                    
                                                                               

Test                                                             12/13/07
CLOSED http://bugs.python.org/issue1619    created  loewis                   
                                                                               

New @spam.getter property syntax modifies the property in place  12/14/07
CLOSED http://bugs.python.org/issue1620    created  tiran                    
       py3k, patch                                                             

Python should compile with -Wstrict-overflow when using gcc      12/14/07
       http://bugs.python.org/issue1621    created  gregory.p.smith          
                                                                               

zipfile hangs on certain zip files                               12/14/07
       http://bugs.python.org/issue1622    created  ehuss                    
       patch                                                                   

Implement PEP-3141 for Decimal                                   12/14/07
       http://bugs.python.org/issue1623    created  jyasskin                 
       py3k, patch                                                             

Remove output comparison for test_pep277                         12/14/07
       http://bugs.python.org/issue1624    created  brett.cannon             
                                                                               

bz2.BZ2File doesn't support multiple streams                     12/14/07
       http://bugs.python.org/issue1625    created  therve                   
                                                                               

threading.Thread objects are not reusable after join()           12/14/07
CLOSED http://bugs.python.org/issue1626    created  dweeves                  
                                                                               

Problem with httplib and Content-Length: -1                      12/14/07
       http://bugs.python.org/issue1627    created  airadier                 
                                                                               

test_distutils unit test is failing rev:59499                    12/14/07
       http://bugs.python.org/issue1628    created  JosephArmbruster         
                                                                               

Py_signal_pipe                                                   12/08/07
CLOSED http://bugs.python.org/issue1564547 reopened gvanrossum               
       patch                                                                   



Issues Now Closed (32)
______________________

Problems with the msi installer - python-3.0a1.msi                 93 days
       http://bugs.python.org/issue1110    vbr                      
                                                                               

IDLE - minor FormatParagraph bug fix                               38 days
       http://bugs.python.org/issue1374    kbk                      
       patch                                                                   

help for file/open should state which is prefered.                 10 days
       http://bugs.python.org/issue1510    skip.montanaro           
                                                                               

doctest should return error if not all tests passed                10 days
       http://bugs.python.org/issue1530    tebeka                   
       patch                                                                   

Avoid string copy when split char doesn't match                     6 days
       http://bugs.python.org/issue1538    skip.montanaro           
       patch                                                                   

PATCH: Attribute lookup caching                                     5 days
       http://bugs.python.org/issue1560    rhettinger               
       patch                                                                   

The set implementation should special-case PyUnicode instead of     4 days
       http://bugs.python.org/issue1564    tiran                    
       py3k                                                                    

Better description of 'L' repr removal in "What's New"              1 days
       http://bugs.python.org/issue1571    georg.brandl             
       py3k                                                                    

404 report of SimpleXMLRPCServer is broken                          0 days
       http://bugs.python.org/issue1572    tiran                    
       py3k                                                                    

Improper use of the keyword-only syntax makes the parser crash      1 days
       http://bugs.python.org/issue1573    amaury.forgeotdarc       
       py3k                                                                    

typo in README                                                      0 days
       http://bugs.python.org/issue1575    georg.brandl             
                                                                               

Problems in win_getpass                                             1 days
       http://bugs.python.org/issue1578    tiran                    
       py3k                                                                    

IDLE uses non-existent xrange() function (Py30a2)                   0 days
       http://bugs.python.org/issue1585    tiran                    
       py3k                                                                    

IDLE no longer shows colour syntax highlighting in the Shell (Py    2 days
       http://bugs.python.org/issue1586    kbk                      
       py3k                                                                    

instancemethod wrapper for PyCFunctions                             0 days
       http://bugs.python.org/issue1587    tiran                    
       py3k, patch                                                             

spacing of the builtin_format function is inconsistent              0 days
       http://bugs.python.org/issue1593    tiran                    
       py3k, patch                                                             

running test_ctypes 25 times in a row causes it to fail             1 days
       http://bugs.python.org/issue1597    theller                  
                                                                               

Wanted behaviour ?                                                  0 days
       http://bugs.python.org/issue1603    amaury.forgeotdarc       
                                                                               

collections.deque.__init__ doesn't initialize                       1 days
       http://bugs.python.org/issue1604    rhettinger               
                                                                               

test_str.py crashes                                                 1 days
       http://bugs.python.org/issue1608    theller                  
                                                                               

infinite recursion when using collections.Sequence in a recursiv    0 days
       http://bugs.python.org/issue1612    gvanrossum               
                                                                               

Makefile's VPATH feature is broken                                  0 days
       http://bugs.python.org/issue1613    tiran                    
       py3k                                                                    

bug in string method "title"                                        0 days
       http://bugs.python.org/issue1614    gvanrossum               
                                                                               

Test                                                                0 days
       http://bugs.python.org/issue1619    loewis                   
                                                                               

New @spam.getter property syntax modifies the property in place     0 days
       http://bugs.python.org/issue1620    tiran                    
       py3k, patch                                                             

threading.Thread objects are not reusable after join()              0 days
       http://bugs.python.org/issue1626    loewis                   
                                                                               

xml.sax segfault on error                                        1367 days
       http://bugs.python.org/issue914148  facundobatista           
                                                                               

Add SSL certificate validation                                   1042 days
       http://bugs.python.org/issue1114345 ahasenack                
       patch                                                                   

urlparse "caches" parses regardless of encoding                   800 days
       http://bugs.python.org/issue1313119 alexandre.vassalotti     
                                                                               

Py_signal_pipe                                                      2 days
       http://bugs.python.org/issue1564547 gvanrossum               
       patch                                                                   

Enhanced tabbed pane widget                                       366 days
       http://bugs.python.org/issue1612746 kbk                      
       patch                                                                   

Problem with signals in a single-threaded application             320 days
       http://bugs.python.org/issue1643738 gvanrossum               
                                                                               



Top Issues Most Discussed (10)
______________________________

 37 Use shorter float repr when possible                               4 days
open    http://bugs.python.org/issue1580   

 33 test_str.py crashes                                                1 days
closed  http://bugs.python.org/issue1608   

 22 SSL tests leak memory                                             25 days
open    http://bugs.python.org/issue1469   

 11 Improper use of the keyword-only syntax makes the parser crash     1 days
closed  http://bugs.python.org/issue1573   

 10 The set implementation should special-case PyUnicode instead of    4 days
closed  http://bugs.python.org/issue1564   

  8 test_re.py fails                                                   1 days
open    http://bugs.python.org/issue1609   

  7 New SSL module doesn't seem to verify hostname against commonNa    3 days
open    http://bugs.python.org/issue1589   

  7 shutil.move() does not use os.rename() if dst is a directory       5 days
open    http://bugs.python.org/issue1577   

  6 Doc: subprocess wait() may lead to dead lock                       2 days
open    http://bugs.python.org/issue1606   

  6 asyncore and asynchat incompatible with Py3k str and bytes         8 days
open    http://bugs.python.org/issue1563   




From josepharmbruster at gmail.com  Fri Dec 14 20:34:15 2007
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Fri, 14 Dec 2007 14:34:15 -0500
Subject: [Python-Dev] Issues 1559298 and 1475 consolidation
Message-ID: <938f42d70712141134s76c1b9fby2c3afe126034fb2e@mail.gmail.com>

Should these two issues be consolidated?  It looks as if they are attacking
the same problem.

Thoughts?
Joseph Armbruster
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071214/4e8d376b/attachment.htm 

From facundobatista at gmail.com  Fri Dec 14 20:51:33 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Fri, 14 Dec 2007 16:51:33 -0300
Subject: [Python-Dev] Call for Project Participation in Development Sprints
	at PyCon 2008
Message-ID: <e04bdf310712141151g1a71ada4g12f9bd769cb24e11@mail.gmail.com>

Python-related projects: join the PyCon Development Sprints!

The development sprints are a key part of PyCon, a chance for the
contributors to open-source projects to get together face-to-face for
up to four days of intensive learning and development.  Newbies sit at
the same table as the gurus, go out for lunch and dinner together, and
have a great time while advancing their project.  At PyCon 2007 in
Dallas we must have had 20 projects sprinting.

If your project would like to sprint at PyCon, now is the time to let
us know.  We need to collect the info and publish it, so participants
will have time to make plans.  We need to get the word out early,
because no matter what we do during the conference, most people who
haven't already decided to sprint won't be able to stay, because they
have a planes to catch and no hotel rooms.

In the past, many people have been reluctant to commit to sprinting.
Some may not know what sprinting is all about; others may think that
they're not "qualified" to sprint.  We want to change that perception.

* We want to help promote your sprint.  The PyCon website, the PyCon
 blog, the PyCon podcast, and press releases will be there for you.

* PyCon attendees will be asked to commit to sprints on the
 registration form, which will include a list of sprints with links
 to further info.

* We will be featuring a "How To Sprint" session on Sunday afternoon,
 followed by sprint-related tutorials, all for free.

* Some sponsors are helping out with the sprints as well.

There's also cost.  Although the sprinting itself is free, sprints
have associated time and hotel costs.  We can't do anything about the
time cost, but we may have some complimentary rooms and funding
available for sprinters.  We will have more to say on financial aid
later.

Those who want to propose a sprint should send the following
information to pycon-organizers at python.org:

* Project/sprint name
* Project URL
* The name and contact info (email & telephone) for the sprint
 leader(s) and other contributors who will attend the sprint
* Instructions for accessing the project's code repository and
 documentation (or a URL)
* Pointers to new contributor information (setup, etc.)
* Any special requirements (projector? whiteboard? flux capacitor?)

We will add this information to the PyCon website and set up a wiki
page for you (or we can link to yours).  Projects need a list of goals
(bugs to fix, features to add, docs to write, etc.), especially some
goals for beginners, to attract new sprinters.  The more detail you
put there, the more prepared your sprinters will be, and the more
results you'll get.

In 2007 there were sprints for Python, Jython, Zope, Django,
TurboGears, Python in Education, SchoolTool, Trac, Docutils, the
Python Job Board, PyCon-Tech, and other projects.  We would like to
see all these and more!

The sprints will run from Monday, March 17 through Thursday, March
20, 2008.  You can find more details here:
http://us.pycon.org/2008/sprints/.

Thank you very much, and happy coding!

Facundo Batista, PyCon 2008 Sprint Coordinator
David Goodger, PyCon 2008 Chair

From brett at python.org  Fri Dec 14 21:32:05 2007
From: brett at python.org (Brett Cannon)
Date: Fri, 14 Dec 2007 12:32:05 -0800
Subject: [Python-Dev] [Python-3000] Call for Project Participation in
	Development Sprints at PyCon 2008
In-Reply-To: <e04bdf310712141151g1a71ada4g12f9bd769cb24e11@mail.gmail.com>
References: <e04bdf310712141151g1a71ada4g12f9bd769cb24e11@mail.gmail.com>
Message-ID: <bbaeab100712141232q301d3ab7u9effa8b9abaafa4a@mail.gmail.com>

On Dec 14, 2007 11:51 AM, Facundo Batista <facundobatista at gmail.com> wrote:
> Python-related projects: join the PyCon Development Sprints!
>
> The development sprints are a key part of PyCon, a chance for the
> contributors to open-source projects to get together face-to-face for
> up to four days of intensive learning and development.  Newbies sit at
> the same table as the gurus, go out for lunch and dinner together, and
> have a great time while advancing their project.  At PyCon 2007 in
> Dallas we must have had 20 projects sprinting.
>
> If your project would like to sprint at PyCon, now is the time to let
> us know.  We need to collect the info and publish it, so participants
> will have time to make plans.  We need to get the word out early,
> because no matter what we do during the conference, most people who
> haven't already decided to sprint won't be able to stay, because they
> have a planes to catch and no hotel rooms.
>

Just so people know, I have already been tapped to be the coach for
the core sprint again this year.  I have also been asked to give
Facundo some help (who needs to reply to my other email; nudge, nudge
=), so the core is definitely being taken care of.

-Brett

From aahz at pythoncraft.com  Fri Dec 14 23:12:58 2007
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 14 Dec 2007 14:12:58 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
References: <aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
Message-ID: <20071214221258.GA15875@panix.com>

On Sat, Dec 08, 2007, glyph at divmod.com wrote:
> On 05:20 pm, guido at python.org wrote:
>>
>>The best solution I can think of is to add a new API that takes a
>>signal and a file descriptor and registers a C-level handler for that
>>signal which writes a byte to the file descriptor. You can then create
>>a pipe, connect the signal handler to the write end, and add the read
>>end to your list of file descriptors passed to select() or poll(). The
>>handler must be written in C in order to avoid the race condition
>>referred to by Glyph (signals arriving after the signal check in the
>>VM main loop but before the select()/poll() system call is entered
>>will not be noticed until the select()/poll() call completes).
> 
> This paragraph jogged my memory.  I remember this exact solution being 
> discussed now, a year ago when I was last talking about these issues.

Ayup.  I am extremely far from an expert here, but anyone wanting to
have an informed opinion should really re-read the threads after these
posts:
http://mail.python.org/pipermail/python-dev/2006-September/068569.html
http://mail.python.org/pipermail/python-dev/2007-January/070772.html

I would recommend requesting review from either Nick Maclaren or Tim
Peters as well.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith

From guido at python.org  Fri Dec 14 23:25:49 2007
From: guido at python.org (Guido van Rossum)
Date: Fri, 14 Dec 2007 14:25:49 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <20071214221258.GA15875@panix.com>
References: <aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<20071214221258.GA15875@panix.com>
Message-ID: <ca471dc20712141425v62b38fa2k35684796fe93f225@mail.gmail.com>

On Dec 14, 2007 2:12 PM, Aahz <aahz at pythoncraft.com> wrote:
> On Sat, Dec 08, 2007, glyph at divmod.com wrote:
> > On 05:20 pm, guido at python.org wrote:
> >>
> >>The best solution I can think of is to add a new API that takes a
> >>signal and a file descriptor and registers a C-level handler for that
> >>signal which writes a byte to the file descriptor. You can then create
> >>a pipe, connect the signal handler to the write end, and add the read
> >>end to your list of file descriptors passed to select() or poll(). The
> >>handler must be written in C in order to avoid the race condition
> >>referred to by Glyph (signals arriving after the signal check in the
> >>VM main loop but before the select()/poll() system call is entered
> >>will not be noticed until the select()/poll() call completes).
> >
> > This paragraph jogged my memory.  I remember this exact solution being
> > discussed now, a year ago when I was last talking about these issues.
>
> Ayup.  I am extremely far from an expert here, but anyone wanting to
> have an informed opinion should really re-read the threads after these
> posts:
> http://mail.python.org/pipermail/python-dev/2006-September/068569.html
> http://mail.python.org/pipermail/python-dev/2007-January/070772.html
>
> I would recommend requesting review from either Nick Maclaren or Tim
> Peters as well.

Oh no, not Nick Maclaren!

Anyway, did you see that we resolved this and have a patch ready for review?

http://bugs.python.org/issue1583

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Sat Dec 15 00:24:14 2007
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 14 Dec 2007 15:24:14 -0800
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712141425v62b38fa2k35684796fe93f225@mail.gmail.com>
References: <a467ca4f0712070648u462b7b7dg51acc0ce8349c8d6@mail.gmail.com>
	<20071207213519.21558.1807633738.divmod.xquotient.2425@joule.divmod.com>
	<aac2c7cb0712071527u72e9709bv66489d22b705c489@mail.gmail.com>
	<ca471dc20712080201q574a5ff9i59ed515b6d5b1f41@mail.gmail.com>
	<475A6F39.4090905@gnome.org>
	<a467ca4f0712080358y42ef2da1g4ff5709b68496d68@mail.gmail.com>
	<ca471dc20712080920n4f685076i4c92696ff9d38d1@mail.gmail.com>
	<20071208215633.21558.1273727216.divmod.xquotient.2458@joule.divmod.com>
	<20071214221258.GA15875@panix.com>
	<ca471dc20712141425v62b38fa2k35684796fe93f225@mail.gmail.com>
Message-ID: <20071214232414.GA29753@panix.com>

[private]

On Fri, Dec 14, 2007, Guido van Rossum wrote:
> On Dec 14, 2007 2:12 PM, Aahz <aahz at pythoncraft.com> wrote:
>> On Sat, Dec 08, 2007, glyph at divmod.com wrote:
>>> On 05:20 pm, guido at python.org wrote:
>>>>
>>>>The best solution I can think of is to add a new API that takes a
>>>>signal and a file descriptor and registers a C-level handler for that
>>>>signal which writes a byte to the file descriptor. You can then create
>>>>a pipe, connect the signal handler to the write end, and add the read
>>>>end to your list of file descriptors passed to select() or poll(). The
>>>>handler must be written in C in order to avoid the race condition
>>>>referred to by Glyph (signals arriving after the signal check in the
>>>>VM main loop but before the select()/poll() system call is entered
>>>>will not be noticed until the select()/poll() call completes).
>>>
>>> This paragraph jogged my memory.  I remember this exact solution being
>>> discussed now, a year ago when I was last talking about these issues.
>>
>> Ayup.  I am extremely far from an expert here, but anyone wanting to
>> have an informed opinion should really re-read the threads after these
>> posts:
>> http://mail.python.org/pipermail/python-dev/2006-September/068569.html
>> http://mail.python.org/pipermail/python-dev/2007-January/070772.html
>>
>> I would recommend requesting review from either Nick Maclaren or Tim
>> Peters as well.
> 
> Oh no, not Nick Maclaren!

;-)  Note carefully that I did not say you should necessarily follow
Nick's advice.  Still, having an expert tell you what you're doing wrong
usually helps even if their advice isn't relevant.

> Anyway, did you see that we resolved this and have a patch ready for review?
> 
> http://bugs.python.org/issue1583

Yeah, I saw that after the fact, but I still think referring to the
previous threads should be useful for anyone wanting to review the patch.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith

From tomerfiliba at gmail.com  Sat Dec 15 10:04:34 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sat, 15 Dec 2007 01:04:34 -0800 (PST)
Subject: [Python-Dev] functional continuations
Message-ID: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>

i'm working on some minimalistic asynchronous framework for python,
somewhat like twisted or stackless, but for different purposes. i came
to the conclusion i want to be able to "freeze" functions, and resume
them later, when some condition is matched.

the idea i came up with is, using exceptions for functional
continuations: after all, the exception's traceback holds the entire
context... so with some help from black magic (i.e., ctypes), i can
resurrect dead stackframes. i will have a reactor in the background
which manages the execution flow, and blocking operations (I/O, etc.)
will be wrapped with a thin wrapper, such as

class socket2(socket):
    def recv(self, count = None):
        raise WaitFor(self, "r")
        return socket.recv(self, count)

now, when read()ing from file2 objects, first an exception will
propagate up to the reactor, which will catch it, create a
continuation object (which holds the traceback) and register it with
the desired event.

then, when the event is triggered, the reactor will resurrect the
continuation object to the point where it stopped (f_lasti + 2), and
execution would continue normally from there.

the idea itself is similar to generator objects, and requires some
messing-around with f_back and f_lasti which is quite ugly. on the
other hand, the beauty of it is, it doesn't require any special design
patterns (unlike twisted) or replacing the interpreter (unlike
stackless), so it can be transparently used with any third-party libs.
all it takes is wrapping I/O objects to "raise WaitFor(...)" prior to
doing blocking I/O.

moreover, unlike generators, i gain the ability to "propagate" up to
the reactor (like normal exceptions), so the code in between the I/O
and the reactor needn't be aware of anything (only avoiding
unconstrained try-except blocks).

i wanted to get some feedback on the issue (i tried c.l.p, but they
didn't understand me well enough):
* has it been tried before?
* do you suppose it will work? are there any drawbacks i didn't
anticipate?
* is it useful enough to be added to the core interpreter (like
generators)? of course if it is added, it could be implemented with a
dedicated mechanism, rather than abusing exceptions for that.


-tomer

From pje at telecommunity.com  Sat Dec 15 17:57:51 2007
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 15 Dec 2007 11:57:51 -0500
Subject: [Python-Dev] functional continuations
In-Reply-To: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegro
	ups.com>
References: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
Message-ID: <20071215165856.206063A40A4@sparrow.telecommunity.com>

At 01:04 AM 12/15/2007 -0800, tomer filiba wrote:
>* do you suppose it will work? are there any drawbacks i didn't
>anticipate?

Yes.  :)

Specifically, think about what happens when a C function is in the 
call stack, e.g.:

def f1():
     return map(f2, [1,2,3])

def f2(ob):
     raise WaitFor(something)
     return ob

print f1()

If I understand you correctly, when this program is run, it will 
print "1", rather than "[1, 2, 3]", because there is no way for you 
to keep track of the internal state of the 'map()' call.

And this isn't just a problem for map() -- even something as simple 
as a property access passes through C code whose state can get lost.

I don't think this approach is practical; you'd be better off using a 
co-routine stack and trampoline, since the nature of generators 
naturally forces all the C code out of the picture.


From tomerfiliba at gmail.com  Sat Dec 15 18:46:13 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sat, 15 Dec 2007 19:46:13 +0200
Subject: [Python-Dev] functional continuations
In-Reply-To: <20071215165856.206063A40A4@sparrow.telecommunity.com>
References: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
	<20071215165856.206063A40A4@sparrow.telecommunity.com>
Message-ID: <1d85506f0712150946r614cdf09jee9817f6d1f44b70@mail.gmail.com>

yeah, i did think native functions wouldn't fit well with it, but then
again, i don't plan to have any c-side functions invoking python
callbacks. i can live with that. but, alas, ceval::EvalFrameEx will
clear the execution stack upon returning [1], so this couldn't work
anyway [2].

[1]
while (STACK_LEVEL() > b->b_level) {
    v = POP();
    Py_XDECREF(v);
}

[2] hrrrmpff


-tomer

On Dec 15, 2007 6:57 PM, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 01:04 AM 12/15/2007 -0800, tomer filiba wrote:
> >* do you suppose it will work? are there any drawbacks i didn't
> >anticipate?
>
> Yes.  :)
>
> Specifically, think about what happens when a C function is in the
> call stack, e.g.:
>
> def f1():
>     return map(f2, [1,2,3])
>
> def f2(ob):
>     raise WaitFor(something)
>     return ob
>
> print f1()
>
> If I understand you correctly, when this program is run, it will
> print "1", rather than "[1, 2, 3]", because there is no way for you
> to keep track of the internal state of the 'map()' call.
>
> And this isn't just a problem for map() -- even something as simple
> as a property access passes through C code whose state can get lost.
>
> I don't think this approach is practical; you'd be better off using a
> co-routine stack and trampoline, since the nature of generators
> naturally forces all the C code out of the picture.
>
>



-- 
An NCO and a Gentleman

From aahz at pythoncraft.com  Sat Dec 15 19:22:09 2007
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 15 Dec 2007 10:22:09 -0800
Subject: [Python-Dev] functional continuations
In-Reply-To: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
References: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
Message-ID: <20071215182209.GA2458@panix.com>

On Sat, Dec 15, 2007, tomer filiba wrote:
>
> i wanted to get some feedback on the issue (i tried c.l.p, but they
> didn't understand me well enough):

python-ideas is the best place for topics like this.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith

From greg.ewing at canterbury.ac.nz  Sat Dec 15 23:54:10 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Dec 2007 11:54:10 +1300
Subject: [Python-Dev] functional continuations
In-Reply-To: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
References: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
Message-ID: <47645B12.9020008@canterbury.ac.nz>

tomer filiba wrote:
> the idea i came up with is, using exceptions for functional
> continuations: after all, the exception's traceback holds the entire
> context...

The problem with this is that, if the call chain has passed
through a C-implemented function at some point, the traceback
*doesn't* contain the entire context -- some of it is in the
C stack frames that got unwound during the propagation of
the exception.

So you may be able to make this work where all the code
involved is pure Python, but it can't work in general,
unless you redesign the whole interpreter the way the
original Stackless did.

Having said that, it might still be a useful thing to
have, as the pure-Python case is probably fairly common.
But I'd want to know what the failure mode is like when
the pure-Python case doesn't hold. Do you get a clear
indication of what is wrong, or does it misbehave in
some obscure way?

A test case for this would be to call map() with a
function that tries to suspend itself using this
mechanism. What happens when you try to resume it?

--
Greg

From greg.ewing at canterbury.ac.nz  Sat Dec 15 23:57:38 2007
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Sun, 16 Dec 2007 11:57:38 +1300
Subject: [Python-Dev] functional continuations
In-Reply-To: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
References: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
Message-ID: <47645BE2.4070803@canterbury.ac.nz>

By the way, I wouldn't call this a "continuation", as that
word implies a bit more (reusability). It's more like a
coroutine or lightweight thread.

--
Greg

From janssen at parc.com  Sun Dec 16 01:32:18 2007
From: janssen at parc.com (Bill Janssen)
Date: Sat, 15 Dec 2007 16:32:18 PST
Subject: [Python-Dev] need Windows compiles for SSL 1.13
Message-ID: <07Dec15.163221pst."58696"@synergy1.parc.xerox.com>

The latest version of the PyPI SSL module is 1.13, and it seems pretty
stable.  I'd appreciate it if one of you who've compiled it in the past
would do so again, and send me Windows binary dists to post to the PyPI
site.

http://pypi.python.org/pypi/ssl/

Thanks!

Bill

From tomerfiliba at gmail.com  Sun Dec 16 09:26:21 2007
From: tomerfiliba at gmail.com (tomer filiba)
Date: Sun, 16 Dec 2007 10:26:21 +0200
Subject: [Python-Dev] functional continuations
In-Reply-To: <47645B12.9020008@canterbury.ac.nz>
References: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>
	<47645B12.9020008@canterbury.ac.nz>
Message-ID: <1d85506f0712160026k559cc7fv77da3b690714b398@mail.gmail.com>

yes, well, as i suspected and as PJE pointed out, extension or builtin
functions would
make life difficult, but these are difficulties i would be willing to
accept. however, my attempts
are futile anyways, as the evaluation stack is cleared before the exception
propagates up
to the frames above... which would explain the NULL deref exceptions i was
getting :)


-tomer

On Dec 16, 2007 12:54 AM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> tomer filiba wrote:
> > the idea i came up with is, using exceptions for functional
> > continuations: after all, the exception's traceback holds the entire
> > context...
>
> The problem with this is that, if the call chain has passed
> through a C-implemented function at some point, the traceback
> *doesn't* contain the entire context -- some of it is in the
> C stack frames that got unwound during the propagation of
> the exception.
>
> So you may be able to make this work where all the code
> involved is pure Python, but it can't work in general,
> unless you redesign the whole interpreter the way the
> original Stackless did.
>
> Having said that, it might still be a useful thing to
> have, as the pure-Python case is probably fairly common.
> But I'd want to know what the failure mode is like when
> the pure-Python case doesn't hold. Do you get a clear
> indication of what is wrong, or does it misbehave in
> some obscure way?
>
> A test case for this would be to call map() with a
> function that tries to suspend itself using this
> mechanism. What happens when you try to resume it?
>
> --
> Greg
>



-- 
An NCO and a Gentleman
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071216/72490810/attachment.htm 

From martin at v.loewis.de  Sun Dec 16 11:51:57 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 16 Dec 2007 11:51:57 +0100
Subject: [Python-Dev] functional continuations
In-Reply-To: <1d85506f0712150946r614cdf09jee9817f6d1f44b70@mail.gmail.com>
References: <08145108-acda-43fb-9a8c-26c255cbb149@p69g2000hsa.googlegroups.com>	<20071215165856.206063A40A4@sparrow.telecommunity.com>
	<1d85506f0712150946r614cdf09jee9817f6d1f44b70@mail.gmail.com>
Message-ID: <4765034D.5060002@v.loewis.de>

> yeah, i did think native functions wouldn't fit well with it, but then
> again, i don't plan to have any c-side functions invoking python
> callbacks. i can live with that.

Just in case it isn't clear - even if you can live with it, it makes
it unsuitable for inclusion into the Python interpreter.

Regards,
Martin

From georgeps at xmission.com  Sat Dec 15 23:34:45 2007
From: georgeps at xmission.com (George Peter Staplin)
Date: Sat, 15 Dec 2007 15:34:45 -0700
Subject: [Python-Dev] Tkinter problems with Tcl/Tk 8.5
Message-ID: <20071215153445.szf1gwdlw4ssk484@webmail.xmission.com>

Hello,

I am a Tcl/Tk core developer.  I'm trying to resolve some bugs that  
have surfaced in Tkinter with Tcl/Tk 8.5.  8.5.0 is very near release,  
and I'm hoping we can determine where the problem is, and resolve it  
soon.

http://sourceforge.net/tracker/index.php?func=detail&aid=1851526&group_id=12997&atid=112997

It seems very peculiar how the text widget's bbox is returning a  
Python-like list and therefore breaking the Tcl callback.  I haven't  
thus far been able to determine which python method is causing that,  
or if it's something related to the hooks you have added.  The same  
problem doesn't occur with 8.4.

It also has been suggested by some on the Tcl core team (during  
discussions about this bug) that you probably shouldn't be using  
Tcl_GetObjType and relying on the registered Tcl_ObjTypes, however I'm  
not sure of a way to get what you need otherwise.

Tcl 8.5 now supports big integers, so I think some changes may be  
needed in _tkinter.c for that.

The list internal rep has changed as well, and I'm not sure if that  
will affect you.  Another developer also pointed out that the text  
widget is returning a Tcl_Obj with a list type, rather than a string  
for the bbox subcommand, so that could be related to this bug.

I hope that we can work together to resolve the issues you may have with Tk.


George
-- 
http://www.xmission.com/~georgeps/  http://whim.linuxsys.net
http://code.google.com/p/megapkg/   http://code.google.com/p/opennexx/


From martin at v.loewis.de  Sun Dec 16 12:13:02 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 16 Dec 2007 12:13:02 +0100
Subject: [Python-Dev] Tkinter problems with Tcl/Tk 8.5
In-Reply-To: <20071215153445.szf1gwdlw4ssk484@webmail.xmission.com>
References: <20071215153445.szf1gwdlw4ssk484@webmail.xmission.com>
Message-ID: <4765083E.5060102@v.loewis.de>

> It also has been suggested by some on the Tcl core team (during  
> discussions about this bug) that you probably shouldn't be using  
> Tcl_GetObjType and relying on the registered Tcl_ObjTypes, however I'm  
> not sure of a way to get what you need otherwise.

Hi George,

I hope I can find some time next week to look into this in more detail,
but please let me respond to this first.

In Python, objects have a fixed type, given to them at the point of
creation. Also, it is common in Python to use multiple types, not
just a single one (say, string). So to convert between Tcl objects
and Python objects, we would like to preserve type information as
much as possible.

To do that, _tkinter has two functions: AsObj (converting PyObject
to Tcl_Obj), and FromObj (converting Tcl_Obj to PyObj). We map
the well-known (registered) types 1:1 to appropriate Python types,
and have a default for the rest.

If Tcl_GetObjType was not available, I see no other way but to
convert everything through strings, which would put the burden
of typing things onto the Tkinter user (or perhaps on Tkinter,
to type the well-known commands).

Regards,
Martin



From g.brandl at gmx.net  Mon Dec 17 00:19:38 2007
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 17 Dec 2007 00:19:38 +0100
Subject: [Python-Dev] reStructuredText docs now printable
Message-ID: <fk4bq9$m8k$1@ger.gmane.org>

Hi,

just wanted to let you know that I've added a LaTeX builder to the doctools,
so that one can produce LaTeX source from the docs again. It uses the old
python.sty package, for which I feel grateful.

You should be able to build the PDFs with a "make latex" in Doc, and a
"make all-pdf" in Doc/build/latex afterwards.

Georg

-- 
Thus spake the Lord: Thou shalt indent with four spaces. No more, no less.
Four shall be the number of spaces thou shalt indent, and the number of thy
indenting shall be four. Eight shalt thou not indent, nor either indent thou
two, excepting that thou then proceed to four. Tabs are right out.


From ndbecker2 at gmail.com  Mon Dec 17 13:02:54 2007
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 17 Dec 2007 07:02:54 -0500
Subject: [Python-Dev] Improved dyn mod load debug
Message-ID: <fk5ohe$vpl$1@ger.gmane.org>

I had mistakenly installed a module (Qsci.so) into the wrong directory. 
Debugging this was harder than it needed to be (c-level debug of shared
lib).

Currently, the only debug info is from importdl.c:
        m = PyDict_GetItemString(PyImport_GetModuleDict(), name);
        if (m == NULL) {
                PyErr_SetString(PyExc_SystemError,
                                "dynamic module not initialized properly");
                return NULL;
        }

I wonder if it would be difficult to print out the name expected, and the
name actually loaded?  In this case, it would have said:

expected: PyQt4/Qsci
loaded: Qsci



From lists at cheimes.de  Mon Dec 17 13:50:07 2007
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 17 Dec 2007 13:50:07 +0100
Subject: [Python-Dev] Improved dyn mod load debug
In-Reply-To: <fk5ohe$vpl$1@ger.gmane.org>
References: <fk5ohe$vpl$1@ger.gmane.org>
Message-ID: <4766707F.3000507@cheimes.de>

Neal Becker wrote:
> I had mistakenly installed a module (Qsci.so) into the wrong directory. 
> Debugging this was harder than it needed to be (c-level debug of shared
> lib).
> 
> Currently, the only debug info is from importdl.c:
>         m = PyDict_GetItemString(PyImport_GetModuleDict(), name);
>         if (m == NULL) {
>                 PyErr_SetString(PyExc_SystemError,
>                                 "dynamic module not initialized properly");
>                 return NULL;
>         }
> 
> I wonder if it would be difficult to print out the name expected, and the
> name actually loaded?  In this case, it would have said:

How do you like:

        if (p == NULL) {
                PyErr_Format(PyExc_ImportError,
                   "dynamic module '%.400s' does not define init
function (init%.200s)",
                             pathname, shortname);
                return NULL;
        }

        m = PyDict_GetItemString(PyImport_GetModuleDict(), name);
        if (m == NULL) {
                PyErr_Format(PyExc_SystemError,
                                "dynamic module '%.400s' not initialized
properly",
			     pathname);
                return NULL;
        }

Christian

From lists at cheimes.de  Mon Dec 17 17:15:16 2007
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 17 Dec 2007 17:15:16 +0100
Subject: [Python-Dev] [patch] Improvements for float and math
Message-ID: <4766A094.8060600@cheimes.de>

I've implemented several small improvements for floats and the math
module. You can find a description and the patches in our bug tracker:

http://bugs.python.org/issue1635
http://bugs.python.org/issue1640

Summary:

* Full and platform independent round trip for +/-inf and nan. Before
the patch the round trip didn't work on Windows and Windows returned
different repr() for nan and inf (1.#INF and -1.#IND).

>>> 1e500
inf
>>> -1e500
-inf
>>> 1e500 * 0.
nan

>>> float("nan")
nan
>>> float("inf")
inf
>>> float("-inf")
-inf

* math.isnan() and math.isinf() functions. The Py_IS_NAN and
Py_IS_INFINITY macros now use isinf and isnan on platforms that support
the functions. Although they are C99 functions they are also available
on GNU89 (C89 + GNU extensions) and other platforms. We already use them
on Windows.

* math.sign() function

>>> math.sign(42)
1
>>> math.sign(-42)
-1
>>> math.sign(0)
0

On platforms with full IEEE 754 semantics and copysign() support:
>>> math.sign(0.0)
1
>>> math.sign(-0.0)
-1

* Py_MATH_PI and Py_MATH_E macros with long double precision for future use.

* 1st and 2nd kind Bessel functions (j0, j1. jn, y0, y1, yn), error
functions (erf, erfc) and gamma function (lgamma with sign). They are
available in most libm libraries.

Please review the patches and comment on them.

Christian

From gnewsg at gmail.com  Mon Dec 17 17:28:09 2007
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Mon, 17 Dec 2007 08:28:09 -0800 (PST)
Subject: [Python-Dev] asyncore delayed calls feature
Message-ID: <1a0ef451-aa43-4418-a106-df3b871a10a5@s12g2000prg.googlegroups.com>

I would ask if someone using asyncore could review this, please:
http://bugs.python.org/issue1641

From dalcinl at gmail.com  Mon Dec 17 22:19:57 2007
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Mon, 17 Dec 2007 18:19:57 -0300
Subject: [Python-Dev] PyTuple_Pack with NULL argument
Message-ID: <e7ba66e40712171319x7c3429ectbef522f442491f8@mail.gmail.com>

Currently, PyTuple_Pack() does not check for NULL arguments, so it is
going to segfault in this case. Is this intended for performance
reasons?


-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594

From lists at janc.be  Mon Dec 17 22:57:52 2007
From: lists at janc.be (Jan Claeys)
Date: Mon, 17 Dec 2007 22:57:52 +0100
Subject: [Python-Dev] Signals+Threads (PyGTK waking up 10x/sec).
In-Reply-To: <ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
References: <20071207045606.GA13636@tummy.com>
	<aac2c7cb0712062155l43161355o446df2bc4c7e78aa@mail.gmail.com>
	<20071207142642.GC13636@tummy.com>
	<1197420882.10215.380.camel@localhost>
	<ca471dc20712111703j1f013772ue47987672f886c93@mail.gmail.com>
Message-ID: <1197928672.10215.548.camel@localhost>

Op dinsdag 11-12-2007 om 17:03 uur [tijdzone -0800], schreef Guido van
Rossum:
> On Dec 11, 2007 4:54 PM, Jan Claeys <lists at janc.be> wrote:
> > Op vrijdag 07-12-2007 om 07:26 uur [tijdzone -0700], schreef Sean
> > Reifschneider:
> > > I would say that this is an optimization that helps a specific set of
> > > platforms, including one that I think we really care about, the OLPC
> > > which needs it for decreased battery use.
> >
> > Almost every laptop user would benefit from it, and even some desktop or
> > server users might save on their electric power bill...
> 
> Do you have data to support this claim?

Not PyGTK-specific data, but you can see power-saving by reducing
wakeups by playing with Intel's 'powertop' utility on linux (also see
the "lesswatts" site mentioned earlier).

One example was the interaction between PulseAudio & ALSA that caused a
lot of "wakeups", and thus prevented laptops from going into the more
"agressive" power saving modes; if PyGTK has a similar issue, that would
make it less useful on laptops...


-- 
Jan Claeys


From andreas.raab at gmx.de  Mon Dec 17 23:25:03 2007
From: andreas.raab at gmx.de (Andreas Raab)
Date: Mon, 17 Dec 2007 14:25:03 -0800
Subject: [Python-Dev] Deploying embedded Python
Message-ID: <4766F73F.5070102@gmx.de>

Hi -

[Apologies if this isn't the right list but I couldn't find any mailing 
list specifically dedicated to embedded Python. Does such a list exist?]

I'm currently looking into a few deployment issues with our embedded 
Python interpreter and I'm looking for any information about deploying 
embedded Python that people may have. Specifically, I'm looking for the 
following information:

1) How to define a useful subset of the stdlib that can serve as an 
initial basis for the installation but later allows upgrade to the 
"full" library if desirable. In other words, I'd like to deploy a small 
subset of the stdlib to begin with (simply because of size constraints) 
which may later be extended to a full stdlib if this is desirable. Has 
someone done this before? I'd love to have a small "Python.zip" 
cross-platform stdlib surrogate that just gets shipped with the product. 
If not, what is the right starting point for analyzing the dependencies 
inside the stdlib?

2) How to isolate the embedded interpreter from environmental effects. I 
have found that on occasion, the interpreter would pick up "stray" 
installations which can cause weird problems. Which environmental 
settings affect the startup of an embedded Python interpreter? How does 
one work around/remove those dependencies? Is there any information 
available about how exactly the startup works? What is being read/loaded 
in which order etc?

3) General advice about deploying embedded Python. Pointers to web 
sites, general experience (good or bad) etc. are all very welcome.

Thanks,
   - Andreas

From guido at python.org  Mon Dec 17 23:58:15 2007
From: guido at python.org (Guido van Rossum)
Date: Mon, 17 Dec 2007 14:58:15 -0800
Subject: [Python-Dev] PyTuple_Pack with NULL argument
In-Reply-To: <e7ba66e40712171319x7c3429ectbef522f442491f8@mail.gmail.com>
References: <e7ba66e40712171319x7c3429ectbef522f442491f8@mail.gmail.com>
Message-ID: <ca471dc20712171458t7d20384cqccc08463f4684882@mail.gmail.com>

Yes, a tuple containing NULL should never be exposed to Python code --
it should only ever be a temporary result in C code. The C code should
ensure all tuple items are non-NULL before passing the tuple off to
Python (or even to its caller, in most cases).

--Guido

On Dec 17, 2007 1:19 PM, Lisandro Dalcin <dalcinl at gmail.com> wrote:
> Currently, PyTuple_Pack() does not check for NULL arguments, so it is
> going to segfault in this case. Is this intended for performance
> reasons?
>
>
> --
> Lisandro Dalc?n
> ---------------
> Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
> Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
> Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
> PTLC - G?emes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Tue Dec 18 00:36:36 2007
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 17 Dec 2007 15:36:36 -0800
Subject: [Python-Dev] Deploying embedded Python
In-Reply-To: <4766F73F.5070102@gmx.de>
References: <4766F73F.5070102@gmx.de>
Message-ID: <20071217233635.GA2295@panix.com>

On Mon, Dec 17, 2007, Andreas Raab wrote:
> 
> [Apologies if this isn't the right list but I couldn't find any mailing 
> list specifically dedicated to embedded Python. Does such a list exist?]

There isn't, really, but python-dev is by definition NOT the correct list
because you're asking questions about using Python rather than about
improving the code in Python.  You should use either comp.lang.python or
capi-sig.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"Typing is cheap.  Thinking is expensive."  --Roy Smith

From andreas.raab at gmx.de  Tue Dec 18 00:51:33 2007
From: andreas.raab at gmx.de (Andreas Raab)
Date: Mon, 17 Dec 2007 15:51:33 -0800
Subject: [Python-Dev] Deploying embedded Python
In-Reply-To: <20071217233635.GA2295@panix.com>
References: <4766F73F.5070102@gmx.de> <20071217233635.GA2295@panix.com>
Message-ID: <47670B85.7000800@gmx.de>

Aahz wrote:
> On Mon, Dec 17, 2007, Andreas Raab wrote:
>> [Apologies if this isn't the right list but I couldn't find any mailing 
>> list specifically dedicated to embedded Python. Does such a list exist?]
> 
> There isn't, really, but python-dev is by definition NOT the correct list
> because you're asking questions about using Python rather than about
> improving the code in Python.  You should use either comp.lang.python or
> capi-sig.

Thanks for pointing this out. I wasn't aware that this list is 
exclusively about improving the code in Python - I thought that 
discussions about, for example, the contents and structure of the stdlib 
as well as the startup of the interpreter would be on-topic here (mainly 
because it's fairly specialized knowledge that I wouldn't expect to be 
widely available outside of the python-dev community). In any case, I'll 
repost on the lists you're mentioning.

Cheers,
   - Andreas

From albertito at gmail.com  Tue Dec 18 15:57:19 2007
From: albertito at gmail.com (Alberto Bertogli)
Date: Tue, 18 Dec 2007 11:57:19 -0300
Subject: [Python-Dev] Make socket support TIPC
Message-ID: <20071218145715.GC3522@gmail.com>


Hi!

I wrote a patch adding TIPC support to the socket module, you can find
it in http://bugs.python.org/issue1646.

TIPC (http://tipc.sf.net) is an open protocol designed for use in
clustered computer environments. It currently has an open source
implementation for Linux (>= 2.6.16), and VxWorks.

On #python-dev, a very friendly person (whose name/nick I can't recall,
I had a power cut and lost my IRC session) helped me with the submission
explained that the patch will probably get rejected in order to keep the
core slim, and suggested I should send an email to this mailing list so
it can be discussed.

I already wrote an external module to support TIPC
(http://auriga.wearlab.de/~alb/pytipc/), but given that the socket
module already supports Linux-specific protocols (AF_PACKET and
AF_NETLINK), and the code impact was (IMHO) small, I wrote a patch to
add TIPC support.


The patch consists of adding the TIPC constants, and modifying
makesockaddr(), getsockaddrarg() and getsockaddrlen() to handle the
TIPC addresses. No generic code is changed.

Compilation is completely conditional on having the TIPC headers, and
has no runtime dependencies.

The diff adds 156 new lines and changes 2, and on x86 increases code
size by about 2k:

   text    data     bss     dec     hex filename
  37158   11980      20   49158    c006 /tmp/so/wo-tipc.so
  39162   11980      20   51162    c7da /tmp/so/w-tipc.so


Thanks a lot,
		Alberto



From ironfroggy at socialserve.com  Tue Dec 18 16:31:31 2007
From: ironfroggy at socialserve.com (Calvin Spealman)
Date: Tue, 18 Dec 2007 10:31:31 -0500
Subject: [Python-Dev] 1-tuple trailing comma
Message-ID: <4F16ED7F-3FDD-494C-BB8B-942785EBD6F5@socialserve.com>

I just had an issue brought up by another developer who had a  
trailing comma on an assignment causing a 1-tuple he did not expect.  
We were talking about it and came to the conclusion that it is at  
least worth bringing up the idea of enforcing a SyntaxError in the  
case of this. 1-tuple syntax is already an odd-man-out, so requiring  
more explicitness about it would catch some errors and be more  
readable. I think we could agree that a tuple of 2 or more elements  
is much easier to read without parens than a 1-tuple without parens.  
Aside from assignment I can't think of a single place when one would  
construct a 1-tuple without parens.

This is the offending erroneous and hard-to-catch code:

if foo:
	bar = 3,
L = [1,
	2,
	bar]

Would there be any possibility in considering further refining the 1- 
tuple syntax to require parens because of its nature?

From jmstall at exchange.microsoft.com  Tue Dec 18 19:11:01 2007
From: jmstall at exchange.microsoft.com (Mike Stall)
Date: Tue, 18 Dec 2007 10:11:01 -0800
Subject: [Python-Dev] 'with' __exit__ doesn't set sys.exc_info()
Message-ID: <5F42453283142A448EB8AD5B34FCFC6D010DCEEE4D00@DF-GRTDANE-MSG.exchange.corp.microsoft.com>

Can somebody confirm the following behavior is expected: When a 'with' statement invokes __exit__(), it looks like sys.exc_info() is not actually set.

I know this may be a very pedantic detail,  but I'm working on IronPython and would like to make it consistent with CPython's behavior.

The PEP (http://www.python.org/dev/peps/pep-0343/ ) says that 'With' can be substituted as follows:

        mgr = (EXPR)
        exit = mgr.__exit__  # Not calling it yet
        value = mgr.__enter__()
        exc = True
        try:
            try:
                VAR = value  # Only if "as VAR" is present
                BLOCK
            except:
                # The exceptional case is handled here
                exc = False
                if not exit(*sys.exc_info()):
                    raise
                # The exception is swallowed if exit() returns true
        finally:
            # The normal and non-local-goto cases are handled here
            if exc:
                exit(None, None, None)

and that "The details of the above translation are intended to prescribe the exact semantics.". This implies that sys.exc_info() would be set when exit is invoked.

I'm finding in practice that sys.exc_info() is not set when __exit__() is invoked in the exceptional case.   I attached a simple repro (repro.py) to demonstrate exactly what I mean.

Can somebody confirm this is the expected behavior?

Thanks,
Mike
http://blogs.msdn.com/jmstall


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071218/98926379/attachment.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: repro.py
Type: application/octet-stream
Size: 1475 bytes
Desc: repro.py
Url : http://mail.python.org/pipermail/python-dev/attachments/20071218/98926379/attachment.obj 

From guido at python.org  Tue Dec 18 19:41:43 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 18 Dec 2007 10:41:43 -0800
Subject: [Python-Dev] 1-tuple trailing comma
In-Reply-To: <4F16ED7F-3FDD-494C-BB8B-942785EBD6F5@socialserve.com>
References: <4F16ED7F-3FDD-494C-BB8B-942785EBD6F5@socialserve.com>
Message-ID: <ca471dc20712181041g7a198433y377a5f86e54c8309@mail.gmail.com>

On Dec 18, 2007 7:31 AM, Calvin Spealman <ironfroggy at socialserve.com> wrote:
> I just had an issue brought up by another developer who had a
> trailing comma on an assignment causing a 1-tuple he did not expect.
> We were talking about it and came to the conclusion that it is at
> least worth bringing up the idea of enforcing a SyntaxError in the
> case of this. 1-tuple syntax is already an odd-man-out, so requiring
> more explicitness about it would catch some errors and be more
> readable. I think we could agree that a tuple of 2 or more elements
> is much easier to read without parens than a 1-tuple without parens.
> Aside from assignment I can't think of a single place when one would
> construct a 1-tuple without parens.
>
> This is the offending erroneous and hard-to-catch code:
>
> if foo:
>         bar = 3,
> L = [1,
>         2,
>         bar]
>
> Would there be any possibility in considering further refining the 1-
> tuple syntax to require parens because of its nature?

Why don't you try to come up with a patch to see how feasible this is?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Dec 18 19:44:17 2007
From: guido at python.org (Guido van Rossum)
Date: Tue, 18 Dec 2007 10:44:17 -0800
Subject: [Python-Dev] 'with' __exit__ doesn't set sys.exc_info()
In-Reply-To: <5F42453283142A448EB8AD5B34FCFC6D010DCEEE4D00@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
References: <5F42453283142A448EB8AD5B34FCFC6D010DCEEE4D00@DF-GRTDANE-MSG.exchange.corp.microsoft.com>
Message-ID: <ca471dc20712181044p2c279eb2n73a58893c0a31dc3@mail.gmail.com>

Since no actual except clause is used, this is reasonable, and I
wouldn't want it changed. I guess the PEP overstated the exactness of
the translation. Suggestions for better wording are accepted.

On Dec 18, 2007 10:11 AM, Mike Stall <jmstall at exchange.microsoft.com> wrote:
>
>
>
>
> Can somebody confirm the following behavior is expected: When a 'with'
> statement invokes __exit__(), it looks like sys.exc_info() is not actually
> set.
>
>
>
> I know this may be a very pedantic detail,  but I'm working on IronPython
> and would like to make it consistent with CPython's behavior.
>
>
>
> The PEP (http://www.python.org/dev/peps/pep-0343/ ) says that 'With' can be
> substituted as follows:
>
>
>
>         mgr = (EXPR)
>
>         exit = mgr.__exit__  # Not calling it yet
>
>         value = mgr.__enter__()
>
>         exc = True
>
>         try:
>
>             try:
>
>                 VAR = value  # Only if "as VAR" is present
>
>                 BLOCK
>
>             except:
>
>                 # The exceptional case is handled here
>
>                 exc = False
>
>                 if not exit(*sys.exc_info()):
>
>                     raise
>
>                 # The exception is swallowed if exit() returns true
>
>         finally:
>
>             # The normal and non-local-goto cases are handled here
>
>             if exc:
>
>                 exit(None, None, None)
>
>
>
> and that "The details of the above translation are intended to prescribe the
> exact semantics.". This implies that sys.exc_info() would be set when exit
> is invoked.
>
>
>
> I'm finding in practice that sys.exc_info() is not set when __exit__() is
> invoked in the exceptional case.   I attached a simple repro (repro.py) to
> demonstrate exactly what I mean.
>
>
>
> Can somebody confirm this is the expected behavior?
>
>
>
> Thanks,
>
> Mike
>
> http://blogs.msdn.com/jmstall
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Thu Dec 20 01:33:02 2007
From: python at rcn.com (Raymond Hettinger)
Date: Wed, 19 Dec 2007 19:33:02 -0500 (EST)
Subject: [Python-Dev] Spurious Buildbot Reports
Message-ID: <20071219193302.ADS15700@ms19.lnh.mail.rcn.net>

The bots are kicking-off so many false alarms that it is becoming difficult to tell whether a check-in genuinely broke a build.

At the root of the problem is a number of tests in the test suite that randomly blow-up.  I now tend to automatically dismiss failures in test_logging and test_threading for example.


Raymond

From josepharmbruster at gmail.com  Thu Dec 20 02:38:19 2007
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Wed, 19 Dec 2007 20:38:19 -0500
Subject: [Python-Dev] integer subclass range behavior
Message-ID: <4769C78B.50608@gmail.com>

All,

I posted this up to comp.lang.python earlier today and asked a few questions 
around IRC.  The general consensus appeared to be that this was a bug.  Before 
opening up an issue on it, I wanted to run it by this list first (just in case) 
  Here is a copy / paste from comp.lang.python:

URL:

http://groups.google.com/group/comp.lang.python/browse_frm/thread/801f227fceff4066#e0305608a5931af1

Post:

I was wondering what would happen, so I tried this out for the heck of it with:
Python 3.0a2 (py3k:59572M, Dec 19 2007, 15:54:07) [MSC v.1500 32 bit (Intel)] 
on win32

class a(int):
   def __new__(cls,number):
     return int.__new__(cls,number)

for x in range(0,a(5)):
   print(x)

Which resulted in a:

Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
   File "a.py", line 5, in <module>
     for x in range(0,a(5)):
SystemError: ..\Objects\longobject.c:400: bad argument to internal
function
[41030 refs]

It looks like the rangeobject performs a FitsInLong test on each of
the parameters to range, which uses the function
_PyLong_FitsInLong(PyObject *vv) within longobject.c.  In tern, this
performs a typecheck:  #define PyLong_CheckExact(op) (Py_TYPE(op) ==
&PyLong_Type) that fails.

Interesting!

From brett at python.org  Thu Dec 20 02:58:35 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 19 Dec 2007 17:58:35 -0800
Subject: [Python-Dev] Spurious Buildbot Reports
In-Reply-To: <20071219193302.ADS15700@ms19.lnh.mail.rcn.net>
References: <20071219193302.ADS15700@ms19.lnh.mail.rcn.net>
Message-ID: <bbaeab100712191758q419aa141gbd3a0a01674c2640@mail.gmail.com>

On Dec 19, 2007 4:33 PM, Raymond Hettinger <python at rcn.com> wrote:
> The bots are kicking-off so many false alarms that it is becoming difficult to tell whether a check-in genuinely broke a build.
>
> At the root of the problem is a number of tests in the test suite that randomly blow-up.  I now tend to automatically dismiss failures in test_logging and test_threading for example.

Yeah, certain tests need some TLC to make them more predictable and
less prone to throw a failure because of some touch race condition or
something on the machine was not available to make the test work.

As I have stated on my blog, once I am done with importlib and the
stdlib reorg I plan on working on dev docs and then attack the whole
structure of the unit tests.

But who knows when that will happen.  =)

-Brett

From titus at caltech.edu  Thu Dec 20 03:06:44 2007
From: titus at caltech.edu (Titus Brown)
Date: Wed, 19 Dec 2007 18:06:44 -0800
Subject: [Python-Dev] Spurious Buildbot Reports
In-Reply-To: <bbaeab100712191758q419aa141gbd3a0a01674c2640@mail.gmail.com>
References: <20071219193302.ADS15700@ms19.lnh.mail.rcn.net>
	<bbaeab100712191758q419aa141gbd3a0a01674c2640@mail.gmail.com>
Message-ID: <20071220020643.GB27861@caltech.edu>

On Wed, Dec 19, 2007 at 05:58:35PM -0800, Brett Cannon wrote:
-> On Dec 19, 2007 4:33 PM, Raymond Hettinger <python at rcn.com> wrote:
-> > The bots are kicking-off so many false alarms that it is becoming difficult to tell whether a check-in genuinely broke a build.
-> >
-> > At the root of the problem is a number of tests in the test suite that randomly blow-up.  I now tend to automatically dismiss failures in test_logging and test_threading for example.
-> 
-> Yeah, certain tests need some TLC to make them more predictable and
-> less prone to throw a failure because of some touch race condition or
-> something on the machine was not available to make the test work.
-> 
-> As I have stated on my blog, once I am done with importlib and the
-> stdlib reorg I plan on working on dev docs and then attack the whole
-> structure of the unit tests.
-> 
-> But who knows when that will happen.  =)

OK, but this isn't really a structural problem, right? This is
non-determinism in certain tests ;)

How broken are these tests?  Can we point a 17-yr-old at them and tell
them to fix 'em (yes think "GHOP")?  (If the problem is reproducible on a
1-in-10 basis then I think the answer is "yes".)

And are test_threading and test_logging the two that need the most
work?

Hmm, perhaps a good starting task would be to run the tests 10-100
times, in random order, on a single machine, to get a statistical
picture of of the problem.

cheers,
--titus
(always on the lookout for core => GHOP tasks)

From guido at python.org  Thu Dec 20 04:31:46 2007
From: guido at python.org (Guido van Rossum)
Date: Wed, 19 Dec 2007 19:31:46 -0800
Subject: [Python-Dev] integer subclass range behavior
In-Reply-To: <4769C78B.50608@gmail.com>
References: <4769C78B.50608@gmail.com>
Message-ID: <ca471dc20712191931k3659fa65t8f667f8db977613b@mail.gmail.com>

Can you submit a bug at bugs.python.org?

The minimal example I found:

>>> class a(int): pass
...
>>> for i in range(0, a(5)): pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
SystemError: Objects/longobject.c:400: bad argument to internal function
>>>


On Dec 19, 2007 5:38 PM, Joseph Armbruster <josepharmbruster at gmail.com> wrote:
> All,
>
> I posted this up to comp.lang.python earlier today and asked a few questions
> around IRC.  The general consensus appeared to be that this was a bug.  Before
> opening up an issue on it, I wanted to run it by this list first (just in case)
>   Here is a copy / paste from comp.lang.python:
>
> URL:
>
> http://groups.google.com/group/comp.lang.python/browse_frm/thread/801f227fceff4066#e0305608a5931af1
>
> Post:
>
> I was wondering what would happen, so I tried this out for the heck of it with:
> Python 3.0a2 (py3k:59572M, Dec 19 2007, 15:54:07) [MSC v.1500 32 bit (Intel)]
> on win32
>
> class a(int):
>    def __new__(cls,number):
>      return int.__new__(cls,number)
>
> for x in range(0,a(5)):
>    print(x)
>
> Which resulted in a:
>
> Traceback (most recent call last):
>    File "<stdin>", line 1, in <module>
>    File "a.py", line 5, in <module>
>      for x in range(0,a(5)):
> SystemError: ..\Objects\longobject.c:400: bad argument to internal
> function
> [41030 refs]
>
> It looks like the rangeobject performs a FitsInLong test on each of
> the parameters to range, which uses the function
> _PyLong_FitsInLong(PyObject *vv) within longobject.c.  In tern, this
> performs a typecheck:  #define PyLong_CheckExact(op) (Py_TYPE(op) ==
> &PyLong_Type) that fails.
>
> Interesting!
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Thu Dec 20 10:50:26 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 20 Dec 2007 19:50:26 +1000
Subject: [Python-Dev] Spurious Buildbot Reports
In-Reply-To: <20071219193302.ADS15700@ms19.lnh.mail.rcn.net>
References: <20071219193302.ADS15700@ms19.lnh.mail.rcn.net>
Message-ID: <476A3AE2.7020509@gmail.com>

Raymond Hettinger wrote:
> The bots are kicking-off so many false alarms that it is becoming
> difficult to tell whether a check-in genuinely broke a build.

It would also be nice if the checkins list only got spammed for actual 
compile or test failures. I'm not all that interested in getting an 
email just because a box got rebooted or lost its net connection for a 
while.

In terms of checking the buildbot status page itself, it would be a lot 
more convenient if the "current/last build" field on the buildbot slave 
details page was a hyperlink to that build's results page rather than a 
mere number as it is now.

> At the root of the problem is a number of tests in the test suite
> that randomly blow-up.  I now tend to automatically dismiss failures
> in test_logging and test_threading for example.

I haven't noticed that so much as the fact that a few of the buildbots 
have been pretty much permanently red for quite some time. Some examples:

alpha tru64:
   Checking the build results for this machine usually shows quite a few 
different errors, although the latest addition to the collection is a 
segfault in test_ctypes. Do we really support tru64 well enough to have 
it as a build slave?

ppc debian unstable:
   test_xmlrpc failing with connection reset by peer errors. I think 
I've seen that on other platforms as well, but this one seems to do it 
nearly constantly. (I don't know enough about the structure of that test 
to know if a GHOP student could figure out the problem without access to 
a machine where the test fails consistently - might be worth a try though)

MIPS debian:
   test_urllib2net FTP tests are failing, apparently complaining that 
SocketType expects two arguments rather than three. Probably a hard one 
to figure out without access to a machine where the test is actually 
failing (even my description of the problem involves some guesswork 
based on reading the code near where the failure occurs).

I think a few of the others (such as hppa Ubuntu) also have issues.

It's gotten to the point where I only pay any real attention to about 
half the trunk buildbots (looking at the status summary page, I would 
probably investigate further if sparc solaris, g4 osx 10, XP-3, XP-4 or 
x86 FreeBSD turned red after one of my checkins) and pretty much ignore 
the rest.

Would it be possible to distinguish the reliable buildbots from the 
problematic ones in the build master? If such a distinction can be made, 
it might then be possible to arrange for email to be sent to the checkin 
list only if one of the reliable buildbots fail.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From qgallet at gmail.com  Thu Dec 20 11:07:57 2007
From: qgallet at gmail.com (Quentin Gallet-Gilles)
Date: Thu, 20 Dec 2007 11:07:57 +0100
Subject: [Python-Dev] Possible bug related with Tkinter ?
Message-ID: <8b943f2b0712200207n5e4ec764vc9fdfc89fe90c5c2@mail.gmail.com>

While testing the tkFileDialog, I encountered a strange error :

~/dev/trunk$ ./python Lib/lib-tk/tkFileDialog.py
Traceback (most recent call last):
  File "Lib/lib-tk/tkFileDialog.py", line 202, in <module>
    openfilename=askopenfilename(filetypes=[("all files", "*")])
  File "Lib/lib-tk/tkFileDialog.py", line 125, in askopenfilename
    return Open(**options).show()
  File "/home/quentin/dev/trunk/Lib/lib-tk/tkCommonDialog.py", line 48, in show
    s = w.tk.call(self.command, *w._options(self.options))
_tkinter.TclError: expected floating-point number but got "0.0"

Investigating a little, I discovered this is a known issue for some
applications like Pyraf and that the following workaround is effective
: "env LC_NUMERIC=C ./python Lib/lib-tk/tkFileDialog.py".

Looking further, this seems to resolve an atof issue and the fact that
Python assumes a C locale while my Linux box actually uses
fr_FR.UTF-8, but I have a hard time determining if that a Python
issue, a TCL issue or something else.

Has someone already encountered this ?

Quentin

From ncoghlan at gmail.com  Thu Dec 20 13:47:56 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 20 Dec 2007 22:47:56 +1000
Subject: [Python-Dev] Possible bug related with Tkinter ?
In-Reply-To: <8b943f2b0712200207n5e4ec764vc9fdfc89fe90c5c2@mail.gmail.com>
References: <8b943f2b0712200207n5e4ec764vc9fdfc89fe90c5c2@mail.gmail.com>
Message-ID: <476A647C.3020206@gmail.com>

Quentin Gallet-Gilles wrote:
> Has someone already encountered this ?

Without any version numbers, it's hard to say. More recent versions of 
Python don't assume LC_NUMERIC is set to the C locale, but older 
versions did.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From lists at cheimes.de  Thu Dec 20 18:08:47 2007
From: lists at cheimes.de (Christian Heimes)
Date: Thu, 20 Dec 2007 18:08:47 +0100
Subject: [Python-Dev] epoll() and kqueue wrapper for the select module
Message-ID: <fke7j0$iqt$1@ger.gmane.org>

Linux Kernel 2.6+ and BSD (including Mac OS X) have two I/O event
notification systems similar but superior to select() and poll(). From
the manuals:

kqueue, kevent -- kernel event notification mechanism

The kqueue() system call provides a generic method of notifying the user
when an event happens or a condition holds, based on the results of
small pieces of kernel code termed filters.  A kevent is identified by
the (ident, filter) pair; there may only be one unique kevent per kqueue.


epoll - I/O event notification facility

epoll  is  a variant of poll(2) that can be used either as an
edge-triggered or a level-triggered interface and scales well to large
numbers of watched file descriptors.


I've written wrappers for both mechanisms. Both wrappers are inspired
from Twisted and select.poll()'s API. The interface is more Pythonic
than the available wrappers and it reduced the burden on the user. The
users don't have to deal with low level control file descriptors and the
fd is closed automatically when the object is collected.

epoll interface

>>> ep = select.epoll(1)
>>> ep.register(fd, select.EPOLL_IN | select.EPOLL_OUT)
>>> ep.modify(fd, select.EPOLL_OUT)
>>> events = ep.wait(1, 1000)
>>> ep.unregister(fd)

kqueue interface

The kqueue interface is more low level than the epoll interface. It has
too many options.
>>> kq = select.kqueue()
>>> ev = [select.kevent(fd, select.KQ_FILTER_WRITE,
                  select.KQ_EV_ONESHOT | select.KQ_EV_ADD)]
>>> kq.control(ev, 0, 0)
>>> events = kq.control([], 1, None)

I already talked to some Twisted guys and they really like it. A patch
is available at http://bugs.python.org/issue1657. The code needs more
unit tests and documentation updates.

Christian


From rcohen at snurgle.org  Thu Dec 20 19:39:26 2007
From: rcohen at snurgle.org (Ross Cohen)
Date: Thu, 20 Dec 2007 13:39:26 -0500
Subject: [Python-Dev] epoll() and kqueue wrapper for the select module
In-Reply-To: <fke7j0$iqt$1@ger.gmane.org>
References: <fke7j0$iqt$1@ger.gmane.org>
Message-ID: <20071220183926.GJ2951@snurgle.org>

On Thu, Dec 20, 2007 at 06:08:47PM +0100, Christian Heimes wrote:

> I've written wrappers for both mechanisms. Both wrappers are inspired
> from Twisted and select.poll()'s API. The interface is more Pythonic
> than the available wrappers and it reduced the burden on the user. The
> users don't have to deal with low level control file descriptors and the
> fd is closed automatically when the object is collected.

Did you look at the python-epoll module which has been in the Cheese
Shop for quite some time? There is no messing with a low level control
file descriptor and it presents an identical interface to select.poll().

I would claim this whole new interface:

> epoll interface
> 
> >>> ep = select.epoll(1)
> >>> ep.register(fd, select.EPOLL_IN | select.EPOLL_OUT)
> >>> ep.modify(fd, select.EPOLL_OUT)
> >>> events = ep.wait(1, 1000)
> >>> ep.unregister(fd)

Is a greater burden on the user than:

>>> import epoll as select

I would also claim that adding a new interface to accomplish the same
task is not more pythonic. But I did write the python-epoll module in
question, so I may be a bit biased.

Ross

From guido at python.org  Thu Dec 20 21:58:32 2007
From: guido at python.org (Guido van Rossum)
Date: Thu, 20 Dec 2007 12:58:32 -0800
Subject: [Python-Dev] test_sys failures
Message-ID: <ca471dc20712201258v29d82f93l36ce295c62603b3e@mail.gmail.com>

When I build from scratch and run most tests (regrtest.py -uall) I get
some strange failures with test_sys.py:

test test_sys failed -- Traceback (most recent call last):
  File "/usr/local/google/home/guido/python/py3kd/Lib/test/test_sys.py",
line 302, in test_43581
    self.assertEqual(sys.__stdout__.encoding, sys.__stderr__.encoding)
AssertionError: 'ascii' != 'ISO-8859-1'

The same test doesn't fail when run in isolation.

Interestingly, I saw this with 2.5 as well as 3.0, but not with 2.6!

Any ideas?

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)

From josepharmbruster at gmail.com  Thu Dec 20 22:33:03 2007
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Thu, 20 Dec 2007 16:33:03 -0500
Subject: [Python-Dev] test_sys failures
In-Reply-To: <ca471dc20712201258v29d82f93l36ce295c62603b3e@mail.gmail.com>
References: <ca471dc20712201258v29d82f93l36ce295c62603b3e@mail.gmail.com>
Message-ID: <938f42d70712201333j5c9caad4y504c61810a14fc8d@mail.gmail.com>

I am unable to reproduce this in windows with:

Python 3.0a2 (py3k:59584M, Dec 20 2007, 16:24:12) [MSC v.1500 32 bit
(Intel)] on win32

Running the test via rt and in isolation does not appear to fail.  In
isolation produces 16 OKs.

Joe

On Dec 20, 2007 3:58 PM, Guido van Rossum <guido at python.org> wrote:

> When I build from scratch and run most tests (regrtest.py -uall) I get
> some strange failures with test_sys.py:
>
> test test_sys failed -- Traceback (most recent call last):
>  File "/usr/local/google/home/guido/python/py3kd/Lib/test/test_sys.py",
> line 302, in test_43581
>    self.assertEqual(sys.__stdout__.encoding, sys.__stderr__.encoding)
> AssertionError: 'ascii' != 'ISO-8859-1'
>
> The same test doesn't fail when run in isolation.
>
> Interestingly, I saw this with 2.5 as well as 3.0, but not with 2.6!
>
> Any ideas?
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/<http://www.python.org/%7Eguido/>
> )
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/josepharmbruster%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071220/d61b6ac7/attachment.htm 

From greg at krypto.org  Fri Dec 21 01:18:08 2007
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 20 Dec 2007 16:18:08 -0800
Subject: [Python-Dev] epoll() and kqueue wrapper for the select module
In-Reply-To: <20071220183926.GJ2951@snurgle.org>
References: <fke7j0$iqt$1@ger.gmane.org> <20071220183926.GJ2951@snurgle.org>
Message-ID: <52dc1c820712201618p72168e8doba4984a5eae8811c@mail.gmail.com>

On 12/20/07, Ross Cohen <rcohen at snurgle.org> wrote:
>
> On Thu, Dec 20, 2007 at 06:08:47PM +0100, Christian Heimes wrote:
>
> > I've written wrappers for both mechanisms. Both wrappers are inspired
> > from Twisted and select.poll()'s API. The interface is more Pythonic
> > than the available wrappers and it reduced the burden on the user. The
> > users don't have to deal with low level control file descriptors and the
> > fd is closed automatically when the object is collected.
>
> Did you look at the python-epoll module which has been in the Cheese
> Shop for quite some time? There is no messing with a low level control
> file descriptor and it presents an identical interface to select.poll().
>
> I would claim this whole new interface:
>
> > epoll interface
> >
> > >>> ep = select.epoll(1)
> > >>> ep.register(fd, select.EPOLL_IN | select.EPOLL_OUT)
> > >>> ep.modify(fd, select.EPOLL_OUT)
> > >>> events = ep.wait(1, 1000)
> > >>> ep.unregister(fd)
>
> Is a greater burden on the user than:
>
> >>> import epoll as select
>
> I would also claim that adding a new interface to accomplish the same
> task is not more pythonic. But I did write the python-epoll module in
> question, so I may be a bit biased.
>
> Ross


Providing a compatibility layer that mirrors the select.select or
select.poll interface using the best underlying syscall may be nice but...
In order to gain the most benefit from any of these system calls they should
be exposed directly.  Do the compatibility wrapping in python but leave the
low level calls exposed for those that want them, they make a big difference
to what an application can do.  (i believe kqueue/kevent can handle signals,
etc)

I like this patch.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071220/117465c5/attachment.htm 

From ncoghlan at gmail.com  Fri Dec 21 09:05:44 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 21 Dec 2007 18:05:44 +1000
Subject: [Python-Dev] test_sys failures
In-Reply-To: <ca471dc20712201258v29d82f93l36ce295c62603b3e@mail.gmail.com>
References: <ca471dc20712201258v29d82f93l36ce295c62603b3e@mail.gmail.com>
Message-ID: <476B73D8.5010400@gmail.com>

Guido van Rossum wrote:
> When I build from scratch and run most tests (regrtest.py -uall) I get
> some strange failures with test_sys.py:
> 
> test test_sys failed -- Traceback (most recent call last):
>   File "/usr/local/google/home/guido/python/py3kd/Lib/test/test_sys.py",
> line 302, in test_43581
>     self.assertEqual(sys.__stdout__.encoding, sys.__stderr__.encoding)
> AssertionError: 'ascii' != 'ISO-8859-1'
> 
> The same test doesn't fail when run in isolation.
> 
> Interestingly, I saw this with 2.5 as well as 3.0, but not with 2.6!
> 
> Any ideas?

It looks like the chunk of code in TextIOWrapper might be getting 
different answers when trying to work out the encoding for stdin and 
stderr (not that I can see how that would happen...). Or maybe there is 
some way that test_sys could be seeing the temporary '__stderr__' entry 
that is put in place until the io module is available?

What do you get for stdin/stdout/stderr.encoding at the interactive 
prompt? On Ubuntu, I get UTF-8 for all of them in both 3.0a2 and 2.5.1, 
but I'm not seeing the problem, either.

Other values of possible interest:
   os.device_encoding(1)
   os.device_encoding(2)
   locale.getpreferredencoding()

Another possibility would be to throw some debugging code into io.py to 
print out the encoding name to stderr when file is 1 or 2.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From lists at cheimes.de  Fri Dec 21 11:28:55 2007
From: lists at cheimes.de (Christian Heimes)
Date: Fri, 21 Dec 2007 11:28:55 +0100
Subject: [Python-Dev] epoll() and kqueue wrapper for the select module
In-Reply-To: <20071220183926.GJ2951@snurgle.org>
References: <fke7j0$iqt$1@ger.gmane.org> <20071220183926.GJ2951@snurgle.org>
Message-ID: <476B9567.1000008@cheimes.de>

Ross Cohen wrote:
> Did you look at the python-epoll module which has been in the Cheese
> Shop for quite some time? There is no messing with a low level control
> file descriptor and it presents an identical interface to select.poll().

No, I didn't see the module. To be honest I didn't look at the Python
Package Index but used Google to search for epoll wrappers. I found the
wrapper in Twisted on my disk and two other wrappers. One was written in
pure C and the other one used ctypes.

Your wrapper is a good implementation. I even found an inconvenience in
my implementation when I studied your code. My wrapper raised an
exception when a closed fd was removed with EPOLL_CTL_DEL.

> I would also claim that adding a new interface to accomplish the same
> task is not more pythonic. But I did write the python-epoll module in
> question, so I may be a bit biased.

I agree with Gregory on that part of your answer.

Christian

From titus at caltech.edu  Fri Dec 21 12:05:37 2007
From: titus at caltech.edu (Titus Brown)
Date: Fri, 21 Dec 2007 03:05:37 -0800
Subject: [Python-Dev] Converting tests to unittest/doctest?
Message-ID: <20071221110537.GA11521@caltech.edu>

Hi all,

a bit of grep'ping and personal examination discovered the following
tests in trunk/ that could be converted to unittest or doctests.  Any
thoughts, pro or con?

(I understand from Brett that the goal is to eradicate "old-style"
tests, by which I think he means tests that do not use unittest or
doctest.  There are some good reasons to switch to using unittests, not
least of which is that you can use a variety of frameworks (nose,
py.test) to do value-added wrapping and management of such tests.)

Suggestions for additional things to test or tests to modify, clean up,
or extend and expand are welcome.

thanks,
--titus

---

test_select
test_contains
test_crypt
test_dbm
test_dummy_threading
test_errno
test_getargs
test_gdbm
test_pep247
test_strftime

test_thread

test_queue

test_fcntl

test_format

test_curses

test_descr

test_funcattrs

test_gdbm

test_socketserver

From gnewsg at gmail.com  Fri Dec 21 15:42:53 2007
From: gnewsg at gmail.com (Giampaolo Rodola')
Date: Fri, 21 Dec 2007 06:42:53 -0800 (PST)
Subject: [Python-Dev] epoll() and kqueue wrapper for the select module
In-Reply-To: <fke7j0$iqt$1@ger.gmane.org>
References: <fke7j0$iqt$1@ger.gmane.org>
Message-ID: <3451ee35-7429-4b8c-9674-6b85173e3289@t1g2000pra.googlegroups.com>

This makes me very happy.
I could try to work on integrating this stuff in asyncore, if someone
finds this of some interest.

On 20 Dic, 18:08, Christian Heimes <li... at cheimes.de> wrote:
> Linux Kernel 2.6+ and BSD (including Mac OS X) have two I/O event
> notification systems similar but superior to select() and poll(). From
> the manuals:
>
> kqueue, kevent -- kernel event notification mechanism
>
> The kqueue() system call provides a generic method of notifying the user
> when an event happens or a condition holds, based on the results of
> small pieces of kernel code termed filters. ?A kevent is identified by
> the (ident, filter) pair; there may only be one unique kevent per kqueue.
>
> epoll - I/O event notification facility
>
> epoll ?is ?a variant of poll(2) that can be used either as an
> edge-triggered or a level-triggered interface and scales well to large
> numbers of watched file descriptors.
>
> I've written wrappers for both mechanisms. Both wrappers are inspired
> from Twisted and select.poll()'s API. The interface is more Pythonic
> than the available wrappers and it reduced the burden on the user. The
> users don't have to deal with low level control file descriptors and the
> fd is closed automatically when the object is collected.
>
> epoll interface
>
> >>> ep = select.epoll(1)
> >>> ep.register(fd, select.EPOLL_IN | select.EPOLL_OUT)
> >>> ep.modify(fd, select.EPOLL_OUT)
> >>> events = ep.wait(1, 1000)
> >>> ep.unregister(fd)
>
> kqueue interface
>
> The kqueue interface is more low level than the epoll interface. It has
> too many options.>>> kq = select.kqueue()
> >>> ev = [select.kevent(fd, select.KQ_FILTER_WRITE,
>
> ? ? ? ? ? ? ? ? ? select.KQ_EV_ONESHOT | select.KQ_EV_ADD)]
>
> >>> kq.control(ev, 0, 0)
> >>> events = kq.control([], 1, None)
>
> I already talked to some Twisted guys and they really like it. A patch
> is available athttp://bugs.python.org/issue1657. The code needs more
> unit tests and documentation updates.
>
> Christian
>
> _______________________________________________
> Python-Dev mailing list
> Python-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From qgallet at gmail.com  Fri Dec 21 16:02:21 2007
From: qgallet at gmail.com (Quentin Gallet-Gilles)
Date: Fri, 21 Dec 2007 16:02:21 +0100
Subject: [Python-Dev] Converting tests to unittest/doctest?
In-Reply-To: <20071221110537.GA11521@caltech.edu>
References: <20071221110537.GA11521@caltech.edu>
Message-ID: <8b943f2b0712210702h69855a85qf76dbfb5cc2a939b@mail.gmail.com>

(oops, realized I didn't send it to the list, just to Titus)

I remember that it was one of the tasks at the Python Sprint at Google last
summer, so I guess this is a good idea (for GHOP, right ?)

>From what remains of the spreadsheet used during the Sprint
(http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA&pli=1),
I believe you can add the following tests to your list:

test_tokenize
test_cProfile
test_extcall
test_logging
test_profile
test_thread
(and maybe test_pep277 that seems to use both unittest and test.test_support
)

HTH,
Quentin

On Dec 21, 2007 12:05 PM, Titus Brown <titus at caltech.edu> wrote:

> Hi all,
>
> a bit of grep'ping and personal examination discovered the following
> tests in trunk/ that could be converted to unittest or doctests.  Any
> thoughts, pro or con?
>
> (I understand from Brett that the goal is to eradicate "old-style"
> tests, by which I think he means tests that do not use unittest or
> doctest.  There are some good reasons to switch to using unittests, not
> least of which is that you can use a variety of frameworks (nose,
> py.test) to do value-added wrapping and management of such tests.)
>
> Suggestions for additional things to test or tests to modify, clean up,
> or extend and expand are welcome.
>
> thanks,
> --titus
>
> ---
>
> test_select
> test_contains
> test_crypt
> test_dbm
> test_dummy_threading
> test_errno
> test_getargs
> test_gdbm
> test_pep247
> test_strftime
>
> test_thread
>
> test_queue
>
> test_fcntl
>
> test_format
>
> test_curses
>
> test_descr
>
> test_funcattrs
>
> test_gdbm
>
> test_socketserver
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/qgallet%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071221/aaba482e/attachment.htm 

From titus at caltech.edu  Fri Dec 21 18:32:13 2007
From: titus at caltech.edu (Titus Brown)
Date: Fri, 21 Dec 2007 09:32:13 -0800
Subject: [Python-Dev] Converting tests to unittest/doctest?
In-Reply-To: <8b943f2b0712210702h69855a85qf76dbfb5cc2a939b@mail.gmail.com>
References: <20071221110537.GA11521@caltech.edu>
	<8b943f2b0712210702h69855a85qf76dbfb5cc2a939b@mail.gmail.com>
Message-ID: <20071221173212.GC24032@caltech.edu>

On Fri, Dec 21, 2007 at 04:02:21PM +0100, Quentin Gallet-Gilles wrote:
-> (oops, realized I didn't send it to the list, just to Titus)
-> 
-> I remember that it was one of the tasks at the Python Sprint at Google last
-> summer, so I guess this is a good idea (for GHOP, right ?)

Yep!

-> >From what remains of the spreadsheet used during the Sprint
-> (http://spreadsheets.google.com/ccc?key=pBLWM8elhFAmKbrhhh0ApQA&pli=1),
-> I believe you can add the following tests to your list:
-> 
-> test_tokenize
-> test_cProfile
-> test_extcall
-> test_logging
-> test_profile
-> test_thread
-> (and maybe test_pep277 that seems to use both unittest and test.test_support
-> )

These were already done as part of GHOP, just not all checked in yet --
we're waiting for some contributor agreements.  But thanks, that's
exactly the sort of info I'm looking for!

cheers,
--titus

From status at bugs.python.org  Fri Dec 21 19:06:07 2007
From: status at bugs.python.org (Tracker)
Date: Fri, 21 Dec 2007 18:06:07 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071221180607.DB3F07839C@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (12/14/07 - 12/21/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1366 open (+27) / 11799 closed (+27) / 13165 total (+54)

Open issues with patches:   426

Average duration of open issues: 689 days.
Median duration of open issues: 926 days.

Open Issues Breakdown
   open  1358 (+27)
pending     8 ( +0)

Issues Created Or Reopened (54)
_______________________________

Py_Size() should be named Py_SIZE()                              12/14/07
CLOSED http://bugs.python.org/issue1629    created  rhettinger               
                                                                               

sys.maxint is documented but should not be                       12/15/07
CLOSED http://bugs.python.org/issue1630    created  blair                    
       py3k                                                                    

Send output from subprocess.Popen objects to any object with a w 12/15/07
CLOSED http://bugs.python.org/issue1631    created  ngrover                  
                                                                               

email cannot be imported                                         12/15/07
CLOSED http://bugs.python.org/issue1632    created  Wubbulous                
                                                                               

smtplib                                                          12/15/07
CLOSED http://bugs.python.org/issue1633    created  Wubbulous                
                                                                               

with Statement SyntaxError generated while following Tutorial    12/15/07
CLOSED http://bugs.python.org/issue1634    created  astral451                
                                                                               

Float patch for inf and nan on Windows (and other platforms)     12/15/07
CLOSED http://bugs.python.org/issue1635    created  tiran                    
       py3k, patch                                                             

Execfile unable to take arguments beyond 255!                    12/16/07
       http://bugs.python.org/issue1636    created  jgatkinsn                
                                                                               

urlparse.urlparse misparses URLs with query but no path          12/16/07
       http://bugs.python.org/issue1637    created  nagle                    
                                                                               

%zd configure test fails on Linux                                12/16/07
CLOSED http://bugs.python.org/issue1638    created  hniksic                  
                                                                               

Low ascii 12 characters found in source.                         12/17/07
CLOSED http://bugs.python.org/issue1639    created  JosephArmbruster         
                                                                               

Enhancements for mathmodule                                      12/17/07
       http://bugs.python.org/issue1640    created  tiran                    
       patch                                                                   

asyncore delayed calls feature                                   12/17/07
       http://bugs.python.org/issue1641    created  giampaolo.rodola         
                                                                               

segfault when deleting value member in ctypes types              12/17/07
CLOSED http://bugs.python.org/issue1642    created  cfbolz                   
                                                                               

Add group() to itertools                                         12/18/07
CLOSED http://bugs.python.org/issue1643    created  KirkMcDonald             
                                                                               

Patch submission guidelines outdated                             12/18/07
CLOSED http://bugs.python.org/issue1644    created  albertito                
                                                                               

Fix socketmodule.c:sock_recvfrom_guts() comment.                 12/18/07
CLOSED http://bugs.python.org/issue1645    created  albertito                
       patch                                                                   

Make socket support TIPC.                                        12/18/07
       http://bugs.python.org/issue1646    created  albertito                
       patch                                                                   

IDLE messes around with sys.exitfunc                             12/18/07
       http://bugs.python.org/issue1647    created  tiran                    
       py3k                                                                    

add new function, sys.gettrace                                   12/18/07
       http://bugs.python.org/issue1648    created  titus                    
       patch                                                                   

IDLE error: dictionary changed size during iteration             12/18/07
CLOSED http://bugs.python.org/issue1649    created  tiran                    
                                                                               

IDLE: help() displays output on the wrong shell                  12/18/07
       http://bugs.python.org/issue1650    created  tiran                    
                                                                               

Limit the max size of PyUnicodeObject->defenc?                   12/18/07
       http://bugs.python.org/issue1651    created  tiran                    
       py3k                                                                    

subprocess should have an option to restore SIGPIPE to default a 12/18/07
       http://bugs.python.org/issue1652    created  cjwatson                 
                                                                               

a couple of  documentation issues                                12/18/07
CLOSED http://bugs.python.org/issue1653    created  JosephArmbruster         
                                                                               

HTTPSConnection is not IPv6-capable                              12/18/07
CLOSED http://bugs.python.org/issue1654    created  dmorr                    
                                                                               

imaplib is not IPv6-capable                                      12/18/07
       http://bugs.python.org/issue1655    created  dmorr                    
       patch                                                                   

Make math.{floor,ceil}(float) return ints per PEP 3141           12/19/07
       http://bugs.python.org/issue1656    created  jyasskin                 
       patch                                                                   

[patch] epoll and kqueue wrappers for the select module          12/19/07
       http://bugs.python.org/issue1657    created  tiran                    
       py3k, patch                                                             

"RuntimeError: dictionary changed size during iteration" in Tkin 12/19/07
       http://bugs.python.org/issue1658    created  quentin.gallet-gilles    
       patch                                                                   

Tests needing network flag?                                      12/19/07
       http://bugs.python.org/issue1659    created  skip.montanaro           
                                                                               

Tests needing network flag?                                      12/19/07
CLOSED http://bugs.python.org/issue1660    created  skip.montanaro           
                                                                               

re.match ignores optional flags when receiving a compiled regex  12/19/07
CLOSED http://bugs.python.org/issue1661    created  bcorfman                 
                                                                               

[patch] assert tp_traverse in PyType_GenericAlloc()              12/19/07
       http://bugs.python.org/issue1662    created  bsilverthorn             
       patch                                                                   

Modification HTMLParser.py                                       12/19/07
CLOSED http://bugs.python.org/issue1663    created  diegorubenarias          
                                                                               

nntplib is not IPv6-capable                                      12/19/07
       http://bugs.python.org/issue1664    created  dmorr                    
       patch                                                                   

re.match.func_code.co_filename returns "re.py"                   12/19/07
CLOSED http://bugs.python.org/issue1665    created  thekorn                  
                                                                               

integer subclass range behavior                                  12/20/07
CLOSED http://bugs.python.org/issue1666    created  JosephArmbruster         
                                                                               

license() does not process keyboard input correctly              12/20/07
CLOSED http://bugs.python.org/issue1667    created  JosephArmbruster         
                                                                               

-E command line parameter intent                                 12/20/07
       http://bugs.python.org/issue1668    created  JosephArmbruster         
       patch                                                                   

shutil.rmtree fails on symlink, after deleting contents          12/20/07
       http://bugs.python.org/issue1669    created  Tesiph                   
                                                                               

Threading.Condition.wait is non-iteruptable                      12/20/07
CLOSED http://bugs.python.org/issue1670    created  ronaldoussoren           
                                                                               

"World" tool ported to py3k                                      12/20/07
CLOSED http://bugs.python.org/issue1671    created  quentin.gallet-gilles    
       py3k, patch                                                             

test_subprocess tempfile issue                                   12/20/07
CLOSED http://bugs.python.org/issue1672    created  JosephArmbruster         
       patch                                                                   

test_pep277 fails                                                12/20/07
CLOSED http://bugs.python.org/issue1673    created  JosephArmbruster         
                                                                               

pythonw.exe crashes when run in one particular account on Window 12/20/07
       http://bugs.python.org/issue1674    created  Mr.E                     
                                                                               

Race condition in os.makedirs                                    12/20/07
       http://bugs.python.org/issue1675    created  ijmorlan                 
                                                                               

Fork/exec issues with Tk 8.5/Python 2.5.1 on OS X                12/21/07
       http://bugs.python.org/issue1676    created  wordtech                 
                                                                               

Ctrl-C will exit out of Python interpreter in Windows            12/21/07
       http://bugs.python.org/issue1677    created  Dude-X                   
       py3k                                                                    

description of startswith is confused                            12/21/07
CLOSED http://bugs.python.org/issue1678    created  mft                      
                                                                               

tokenizer permits invalid hex integer                            12/21/07
       http://bugs.python.org/issue1679    created  MartinRinehart           
                                                                               

what is decimal.Context.get_manager()?                           12/21/07
       http://bugs.python.org/issue1680    created  mft                      
                                                                               

parameter name for optparse parse_args incorrect                 12/21/07
       http://bugs.python.org/issue1681    created  wichert                  
                                                                               

Move Demo/classes/Rat.py to Lib/rational.py and fix it up.       12/21/07
       http://bugs.python.org/issue1682    created  jyasskin                 
                                                                               



Issues Now Closed (44)
______________________

Tools/msi/msi.py does not work with PCBuild8                       53 days
       http://bugs.python.org/issue1337    tiran                    
                                                                               

test_import breaks on Linux                                        42 days
       http://bugs.python.org/issue1377    tiran                    
       py3k                                                                    

VS2008, quick hack for distutils.msvccompiler                      32 days
       http://bugs.python.org/issue1455    amaury.forgeotdarc       
       py3k, patch                                                             

building python 2.6 with VC Express 2008 Beta2                     31 days
       http://bugs.python.org/issue1465    tiran                    
       patch                                                                   

SSL tests leak memory                                              25 days
       http://bugs.python.org/issue1469    gvanrossum               
       py3k                                                                    

Patch to remove API to create new unbound methods                  26 days
       http://bugs.python.org/issue1497    georg.brandl             
       py3k, patch                                                             

Regression with __hash__ definition and rich comparison operator   17 days
       http://bugs.python.org/issue1549    therve                   
       patch                                                                   

Backport sys.maxsize to Python 2.6                                 13 days
       http://bugs.python.org/issue1570    tiran                    
                                                                               

Patch for signal.set_wakeup_fd                                      9 days
       http://bugs.python.org/issue1583    gvanrossum               
       patch                                                                   

test_re.py fails                                                    7 days
       http://bugs.python.org/issue1609    cartman                  
                                                                               

test_socket.py fails                                                2 days
       http://bugs.python.org/issue1610    tiran                    
                                                                               

Remove output comparison for test_pep277                            1 days
       http://bugs.python.org/issue1624    tiran                    
       patch                                                                   

test_distutils unit test is failing rev:59499                       0 days
       http://bugs.python.org/issue1628    tiran                    
       py3k                                                                    

Py_Size() should be named Py_SIZE()                                 5 days
       http://bugs.python.org/issue1629    gvanrossum               
                                                                               

sys.maxint is documented but should not be                          0 days
       http://bugs.python.org/issue1630    tiran                    
       py3k                                                                    

Send output from subprocess.Popen objects to any object with a w    4 days
       http://bugs.python.org/issue1631    gvanrossum               
                                                                               

email cannot be imported                                            4 days
       http://bugs.python.org/issue1632    gvanrossum               
                                                                               

smtplib                                                             0 days
       http://bugs.python.org/issue1633    brett.cannon             
                                                                               

with Statement SyntaxError generated while following Tutorial       0 days
       http://bugs.python.org/issue1634    georg.brandl             
                                                                               

Float patch for inf and nan on Windows (and other platforms)        4 days
       http://bugs.python.org/issue1635    tiran                    
       py3k, patch                                                             

%zd configure test fails on Linux                                   2 days
       http://bugs.python.org/issue1638    tiran                    
                                                                               

Low ascii 12 characters found in source.                            0 days
       http://bugs.python.org/issue1639    tiran                    
                                                                               

segfault when deleting value member in ctypes types                 1 days
       http://bugs.python.org/issue1642    theller                  
                                                                               

Add group() to itertools                                            0 days
       http://bugs.python.org/issue1643    rhettinger               
                                                                               

Patch submission guidelines outdated                                3 days
       http://bugs.python.org/issue1644    georg.brandl             
                                                                               

Fix socketmodule.c:sock_recvfrom_guts() comment.                    1 days
       http://bugs.python.org/issue1645    gvanrossum               
       patch                                                                   

IDLE error: dictionary changed size during iteration                1 days
       http://bugs.python.org/issue1649    tiran                    
                                                                               

a couple of  documentation issues                                   0 days
       http://bugs.python.org/issue1653    loewis                   
                                                                               

HTTPSConnection is not IPv6-capable                                 0 days
       http://bugs.python.org/issue1654    janssen                  
                                                                               

Tests needing network flag?                                         0 days
       http://bugs.python.org/issue1660    skip.montanaro           
                                                                               

re.match ignores optional flags when receiving a compiled regex     0 days
       http://bugs.python.org/issue1661    rhettinger               
                                                                               

Modification HTMLParser.py                                          1 days
       http://bugs.python.org/issue1663    facundobatista           
                                                                               

re.match.func_code.co_filename returns "re.py"                      1 days
       http://bugs.python.org/issue1665    thekorn                  
                                                                               

integer subclass range behavior                                     1 days
       http://bugs.python.org/issue1666    loewis                   
                                                                               

license() does not process keyboard input correctly                 1 days
       http://bugs.python.org/issue1667    JosephArmbruster         
                                                                               

Threading.Condition.wait is non-iteruptable                         0 days
       http://bugs.python.org/issue1670    gvanrossum               
                                                                               

"World" tool ported to py3k                                         0 days
       http://bugs.python.org/issue1671    quentin.gallet-gilles    
       py3k, patch                                                             

test_subprocess tempfile issue                                      0 days
       http://bugs.python.org/issue1672    gvanrossum               
       patch                                                                   

test_pep277 fails                                                   1 days
       http://bugs.python.org/issue1673    loewis                   
                                                                               

description of startswith is confused                               0 days
       http://bugs.python.org/issue1678    georg.brandl             
                                                                               

str.__iter__ and unicode.__iter__                                 515 days
       http://bugs.python.org/issue1526367 rhettinger               
       patch                                                                   

use LSB version information to detect a platform                  449 days
       http://bugs.python.org/issue1565037 tiran                    
       patch                                                                   

SyntaxWarning for backquotes                                      343 days
       http://bugs.python.org/issue1631035 tiran                    
       patch                                                                   

Epoll wrapper                                                     288 days
       http://bugs.python.org/issue1675118 tiran                    
       patch                                                                   



Top Issues Most Discussed (10)
______________________________

 25 test_re.py fails                                                   7 days
closed  http://bugs.python.org/issue1609   

 15 [patch] epoll and kqueue wrappers for the select module            2 days
open    http://bugs.python.org/issue1657   

 12 Use shorter float repr when possible                              11 days
open    http://bugs.python.org/issue1580   

 11 Float patch for inf and nan on Windows (and other platforms)       4 days
closed  http://bugs.python.org/issue1635   

 10 license() does not process keyboard input correctly                1 days
closed  http://bugs.python.org/issue1667   

 10 Implement PEP-3141 for Decimal                                     8 days
open    http://bugs.python.org/issue1623   

  9 re.match.func_code.co_filename returns "re.py"                     1 days
closed  http://bugs.python.org/issue1665   

  9 Enhancements for mathmodule                                        4 days
open    http://bugs.python.org/issue1640   

  9 Py_Size() should be named Py_SIZE()                                5 days
closed  http://bugs.python.org/issue1629   

  9 str.format() produces different output on different platforms (    9 days
open    http://bugs.python.org/issue1600   




From albertito at gmail.com  Fri Dec 21 21:08:42 2007
From: albertito at gmail.com (Alberto Bertogli)
Date: Fri, 21 Dec 2007 17:08:42 -0300
Subject: [Python-Dev] Make socket support TIPC
In-Reply-To: <20071218145715.GC3522@gmail.com>
References: <20071218145715.GC3522@gmail.com>
Message-ID: <20071221200842.GC5150@gmail.com>

On Tue, Dec 18, 2007 at 11:57:19AM -0300, Alberto Bertogli wrote:
> I wrote a patch adding TIPC support to the socket module, you can find
> it in http://bugs.python.org/issue1646.

Well, I'm supposed to "tickle the interest of one of the many folks with
commit privileges" about this, so here I am =)

Is anyone interested in this patch?
I have no idea what else is needed to get this in, so if anybody can
give me some hints, I'd be great.

Thanks a lot,                                                                                        
		Alberto



From rcohen at snurgle.org  Fri Dec 21 22:16:45 2007
From: rcohen at snurgle.org (Ross Cohen)
Date: Fri, 21 Dec 2007 16:16:45 -0500
Subject: [Python-Dev] epoll() and kqueue wrapper for the select module
Message-ID: <20071221211645.GM2951@snurgle.org>

On Fri, Dec 21, 2007 at 11:28:55AM +0100, Christian Heimes wrote:
> Your wrapper is a good implementation. I even found an inconvenience in
> my implementation when I studied your code. My wrapper raised an
> exception when a closed fd was removed with EPOLL_CTL_DEL.

It should be a good reference, it's gotten a lot of testing.

> > I would also claim that adding a new interface to accomplish the same
> > task is not more pythonic. But I did write the python-epoll module in
> > question, so I may be a bit biased.
> 
> I agree with Gregory on that part of your answer.

I think we can both be right. The select.poll() code does not really
map to the low level poll() call. In fact, it maps much better to the
epoll() call. It appears that your new API is really a superset of the
the select.poll() interface. The only thing which keeps a user from
just importing your module and using it with code which uses the
existing interface is that you've changed names. Why not change the
names so that it Just Works?

Also, everything but the edge-triggered functionality could be
incorporated back into the select.poll() interface (where "everything"
means the modify() call.)

Ross
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20071221/34af4cec/attachment.pgp 

From nnorwitz at gmail.com  Fri Dec 21 22:24:45 2007
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 21 Dec 2007 13:24:45 -0800
Subject: [Python-Dev] Converting tests to unittest/doctest?
In-Reply-To: <20071221110537.GA11521@caltech.edu>
References: <20071221110537.GA11521@caltech.edu>
Message-ID: <ee2a432c0712211324t6f961989udcc49c9a6a56ea3a@mail.gmail.com>

Hi Titus.  Great work on GHOP!

On Dec 21, 2007 3:05 AM, Titus Brown <titus at caltech.edu> wrote:
> Hi all,
>
> a bit of grep'ping and personal examination discovered the following
> tests in trunk/ that could be converted to unittest or doctests.  Any
> thoughts, pro or con?

Yes, it would be great to get rid of the old style tests.  I didn't
verify your list, but it looked good.  The most important ones to get
rid of are the ones that compare output.

It would also be great to fix the flaky tests.  Nearly all are related
to networking.  I think Raymond started a thread about some of these
issues.  (Hopefully I'll catch up on some mail over the holidays.)
The flaky tests are probably too large for GHOP, but feel free to try
fix them.  From memory the flakiest ones are:  test_xmlrpc,
test_urllib2, test_urllib2net

n

From titus at caltech.edu  Sat Dec 22 09:08:30 2007
From: titus at caltech.edu (Titus Brown)
Date: Sat, 22 Dec 2007 00:08:30 -0800
Subject: [Python-Dev] Updating descrintro
Message-ID: <20071222080829.GA23584@caltech.edu>

Hi all,

re

	http://www.python.org/download/releases/2.2.3/descrintro/
	
on "Unifying types and classes in Python 2.2",

we have a GHOP task to "fill in" the Additional Topics section of this
document.  'novanasa', the student who took this task, has written up a
nice set of doctest additions in reST:

http://code.google.com/p/google-highly-open-participation-psf/issues/attachment?aid=-5215187968540901644&name=types-classes.rst

and has asked for feedback.  My feedback at the moment is, "looks good,
and useful!"  I thought I'd ask on python-dev to see if anyone has any
additional comments or ways to make this more directly useful to Python.

You can just add comments on to the task directly, if you like, or send
them to me.

One set of possible additions is to go through and make annotations on
stuff that is going to change in Py3K.  Anything else?

cheers,
--titus

From josepharmbruster at gmail.com  Sat Dec 22 20:04:37 2007
From: josepharmbruster at gmail.com (Joseph Armbruster)
Date: Sat, 22 Dec 2007 14:04:37 -0500
Subject: [Python-Dev] svn.python.org ?
Message-ID: <476D5FC5.8010906@gmail.com>


Is svn.python.org ok?  I am unable to perform an update at the moment.

Joseph Armbruster

From steve at holdenweb.com  Sat Dec 22 22:56:48 2007
From: steve at holdenweb.com (Steve Holden)
Date: Sat, 22 Dec 2007 16:56:48 -0500
Subject: [Python-Dev] svn.python.org ?
In-Reply-To: <476D5FC5.8010906@gmail.com>
References: <476D5FC5.8010906@gmail.com>
Message-ID: <fkk16t$61c$1@ger.gmane.org>

Joseph Armbruster wrote:
> Is svn.python.org ok?  I am unable to perform an update at the moment.
> 
Looks like a transient or location-related problem - I am getting an 
update as I write.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/


From steve at holdenweb.com  Sun Dec 23 08:07:09 2007
From: steve at holdenweb.com (Steve Holden)
Date: Sun, 23 Dec 2007 02:07:09 -0500
Subject: [Python-Dev] svn.python.org ?
In-Reply-To: <60bb7ceb0712221457w62885503ude9759e1ff05247b@mail.gmail.com>
References: <476D5FC5.8010906@gmail.com> <fkk16t$61c$1@ger.gmane.org>
	<60bb7ceb0712221457w62885503ude9759e1ff05247b@mail.gmail.com>
Message-ID: <476E091D.5060503@holdenweb.com>

Skip Montanaro wrote:
> I'm unable to get the (apprently external?) 2to3 to update:
> 
>   % svn up
> 
>   Fetching external item into 'Tools/2to3'
>   svn: PROPFIND request failed on '/projects/sandbox/trunk/2to3'
>   svn: PROPFIND of '/projects/sandbox/trunk/2to3': could not connect
> to server (http://svn.python.org)
> 
> Something seems amiss.
> 
Guess my previous output didn't apply to the sandbox. Sorry.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/

From facundobatista at gmail.com  Mon Dec 24 15:20:16 2007
From: facundobatista at gmail.com (Facundo Batista)
Date: Mon, 24 Dec 2007 11:20:16 -0300
Subject: [Python-Dev] Type hierarchy for numbers for 2.6
Message-ID: <e04bdf310712240620r1d0518f2p11556826c314ebe4@mail.gmail.com>

Hi!

The issue 1689 [1] proposes a patch to backport the PEP 3141 [2], A
Type Hierarchy for Numbers, to the trunk.

The patch applies cleanly, and all tests pass ok (of course, even the
new ones that are specific to this).

The patch misses documentation and lines for the NEWS file, I'll ask
for them there. But I wanted to tell you that this is going on, just
in case you want to take a look to it.

Regards,

[1] http://bugs.python.org/issue1689
[2] http://www.python.org/dev/peps/pep-3141/

-- 
.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From orsenthil at gmail.com  Wed Dec 26 18:49:30 2007
From: orsenthil at gmail.com (O.R.Senthil Kumaran)
Date: Wed, 26 Dec 2007 23:19:30 +0530
Subject: [Python-Dev] viewvc at svn.python.org down?
Message-ID: <20071226174930.GE4111@gmail.com>

The viewvc interface of svn.python.org seems down.
svn is working properly though.
Anyone aware of the problem?


-- 
O.R.Senthil Kumaran
http://uthcode.sarovar.org

From brett at python.org  Wed Dec 26 20:25:34 2007
From: brett at python.org (Brett Cannon)
Date: Wed, 26 Dec 2007 11:25:34 -0800
Subject: [Python-Dev] viewvc at svn.python.org down?
In-Reply-To: <20071226174930.GE4111@gmail.com>
References: <20071226174930.GE4111@gmail.com>
Message-ID: <bbaeab100712261125i3dbeaae8i6d7d617889e4c69a@mail.gmail.com>

On Dec 26, 2007 9:49 AM, O.R.Senthil Kumaran <orsenthil at gmail.com> wrote:
> The viewvc interface of svn.python.org seems down.
> svn is working properly though.
> Anyone aware of the problem?
>

I believe the machine hosting the Subversion server was to be upgraded
today.  That might be the cause of this.

-Brett

>
> --
> O.R.Senthil Kumaran
> http://uthcode.sarovar.org
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From martin at v.loewis.de  Wed Dec 26 23:54:50 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 26 Dec 2007 23:54:50 +0100
Subject: [Python-Dev] Spurious Buildbot Reports
In-Reply-To: <476A3AE2.7020509@gmail.com>
References: <20071219193302.ADS15700@ms19.lnh.mail.rcn.net>
	<476A3AE2.7020509@gmail.com>
Message-ID: <4772DBBA.8060409@v.loewis.de>

> It would also be nice if the checkins list only got spammed for actual 
> compile or test failures. I'm not all that interested in getting an 
> email just because a box got rebooted or lost its net connection for a 
> while.
> 
> In terms of checking the buildbot status page itself, it would be a lot 
> more convenient if the "current/last build" field on the buildbot slave 
> details page was a hyperlink to that build's results page rather than a 
> mere number as it is now.

In short: contributions are welcome.

Regards,
Martin

From ncoghlan at gmail.com  Thu Dec 27 08:35:23 2007
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 27 Dec 2007 17:35:23 +1000
Subject: [Python-Dev] Spurious Buildbot Reports
In-Reply-To: <4772DBBA.8060409@v.loewis.de>
References: <20071219193302.ADS15700@ms19.lnh.mail.rcn.net>
	<476A3AE2.7020509@gmail.com> <4772DBBA.8060409@v.loewis.de>
Message-ID: <477355BB.6010806@gmail.com>

Martin v. L?wis wrote:
>> It would also be nice if the checkins list only got spammed for actual 
>> compile or test failures. I'm not all that interested in getting an 
>> email just because a box got rebooted or lost its net connection for a 
>> while.
>>
>> In terms of checking the buildbot status page itself, it would be a lot 
>> more convenient if the "current/last build" field on the buildbot slave 
>> details page was a hyperlink to that build's results page rather than a 
>> mere number as it is now.
> 
> In short: contributions are welcome.

Where does the buildbot HTML/email generation code live? I thought I'd 
found it in the Tools directory, but that turned out to be just the 
utility scripts for the buildbot clients.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From arigo at tunes.org  Thu Dec 27 11:22:30 2007
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 27 Dec 2007 11:22:30 +0100
Subject: [Python-Dev] PATCH: Fast globals/builtins lookups for 2.6
In-Reply-To: <475036CC.6020306@cs.byu.edu>
References: <474E93DD.2020605@cs.byu.edu>
	<ca471dc20711290929y5d696332ufa0db526912bf368@mail.gmail.com>
	<474F2A1C.8010105@cs.byu.edu>
	<ca471dc20711291405g52a38aado6ff18ab424d6f7e7@mail.gmail.com>
	<474FA2F0.6020305@cs.byu.edu> <fip4iv$6kd$1@ger.gmane.org>
	<475036CC.6020306@cs.byu.edu>
Message-ID: <20071227102230.GA13703@code0.codespeak.net>

Hi Neil,

On Fri, Nov 30, 2007 at 09:14:04AM -0700, Neil Toronto wrote:
> >> whether 64 bits is necessary. It takes an hour of concerted effort - 
> >> nothing but "module.d = 1; del module.d" for an hour straight - to 
> >> overflow a 32-bit version number. Is anybody going to actually get close 
> >> to doing that in a global namespace?
> >>
> > Of course not. And 640k is as much memory as anyone could reasonably 
> > need ...
> 
> Point taken - forget I asked.

How much time does it take if the loop is in a C extension module, e.g.
like the following?

    while (1) { PyDict_SetItem(d, k, v); PyDict_DelItem(d, k); }

How much time would it take the same loop to overflow even the 64-bit
version number?  And how much will *this* take in ten year's time?


A bientot,

Armin

From daniel at stutzbachenterprises.com  Thu Dec 27 17:46:58 2007
From: daniel at stutzbachenterprises.com (Daniel Stutzbach)
Date: Thu, 27 Dec 2007 10:46:58 -0600
Subject: [Python-Dev] PATCH: Fast globals/builtins lookups for 2.6
In-Reply-To: <20071227102230.GA13703@code0.codespeak.net>
References: <474E93DD.2020605@cs.byu.edu>
	<ca471dc20711290929y5d696332ufa0db526912bf368@mail.gmail.com>
	<474F2A1C.8010105@cs.byu.edu>
	<ca471dc20711291405g52a38aado6ff18ab424d6f7e7@mail.gmail.com>
	<474FA2F0.6020305@cs.byu.edu> <fip4iv$6kd$1@ger.gmane.org>
	<475036CC.6020306@cs.byu.edu>
	<20071227102230.GA13703@code0.codespeak.net>
Message-ID: <eae285400712270846m6a90789cq6b52a8657cd42e4b@mail.gmail.com>

On Dec 27, 2007 4:22 AM, Armin Rigo <arigo at tunes.org> wrote:
> How much time does it take if the loop is in a C extension module, e.g.
> like the following?
>
>     while (1) { PyDict_SetItem(d, k, v); PyDict_DelItem(d, k); }
>
> How much time would it take the same loop to overflow even the 64-bit
> version number?

On a modern computer, more than a century.

> And how much will *this* take in ten year's time?

My rule of thumb is that a tight loop performing 2**32 operations
takes around a minute on a modern computer.  The PyDict operations
presumably take somewhat longer than a few primitive operations, so
this is a good lower bound.

Assuming that processing speed doubles every 1.5 year, we have

(2**64 operations / 2**32 operations per minute / 2**(10 years / 1.5
years per speed doubling))
= 42275935 minutes
= 1.34 years of running that tight loop to get an overflow.

I think 64 bits is pretty safe :-)

-- 
Daniel Stutzbach, Ph.D.             President, Stutzbach Enterprises LLC

From status at bugs.python.org  Fri Dec 28 19:06:11 2007
From: status at bugs.python.org (Tracker)
Date: Fri, 28 Dec 2007 18:06:11 +0000 (UTC)
Subject: [Python-Dev] Summary of Tracker Issues
Message-ID: <20071228180611.6B5A17836D@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (12/21/07 - 12/28/07)
Tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue 
number.  Do NOT respond to this message.


 1381 open (+20) / 11808 closed ( +4) / 13189 total (+24)

Open issues with patches:   424

Average duration of open issues: 688 days.
Median duration of open issues: 948 days.

Open Issues Breakdown
   open  1373 (+20)
pending     8 ( +0)

Issues Created Or Reopened (24)
_______________________________

Thread local storage and PyGILState_* mucked up by os.fork()     12/21/07
       http://bugs.python.org/issue1683    created  roudkerk                 
                                                                               

CGIHTTPServer does not chdir prior to executing the CGI script   12/21/07
       http://bugs.python.org/issue1684    created  majid                    
                                                                               

linecache .updatecache fails on utf8 encoded files               12/22/07
       http://bugs.python.org/issue1685    created  pyscripter               
                                                                               

string.Template.safe_substitute fail when overriding  pattern at 12/22/07
       http://bugs.python.org/issue1686    created  hagaigold                
                                                                               

plistlib.py restricts <integer> to Python int when writing       12/22/07
       http://bugs.python.org/issue1687    created  lafcadio                 
                                                                               

Incorrectly displayed non ascii characters in prompt using "inpu 12/22/07
       http://bugs.python.org/issue1688    created  vbr                      
                                                                               

Backport PEP 3141 to 2.6                                         12/23/07
       http://bugs.python.org/issue1689    created  jyasskin                 
                                                                               

Crash on cancellation of windows install                         12/23/07
       http://bugs.python.org/issue1690    created  mtvernon                 
                                                                               

feature request: methods that modify instances should return ins 12/23/07
CLOSED http://bugs.python.org/issue1691    created  eukaryot                 
                                                                               

Syntax Error exception dosen't print string; not informative     12/23/07
       http://bugs.python.org/issue1692    created  Dude-X                   
                                                                               

Please check merge from trunk                                    12/24/07
       http://bugs.python.org/issue1693    created  tiran                    
       py3k                                                                    

floating point number round failures under Linux                 12/24/07
       http://bugs.python.org/issue1694    created  falk_steinhauer          
                                                                               

time.localtime() docstring has a wrong fieldname                 12/24/07
CLOSED http://bugs.python.org/issue1695    created  tapybugs                 
                                                                               

Distutils ignore dry-run flag at clean-up of distutils.command.b 12/25/07
       http://bugs.python.org/issue1696    created  ludvig.ericson           
                                                                               

Problems with float and "6" type                                 12/25/07
CLOSED http://bugs.python.org/issue1697    created  pedrocpneto              
                                                                               

urlparse and usernames containing @                              12/25/07
       http://bugs.python.org/issue1698    created  ocroquette               
                                                                               

unconditional definiton of _BSD_SOURCE breaks builds with g++-4. 12/25/07
       http://bugs.python.org/issue1699    created  doko                     
                                                                               

Regular Expression inline flags not handled correctly for some u 12/25/07
       http://bugs.python.org/issue1700    created  sonnq                    
                                                                               

svn checkout does not work                                       12/26/07
CLOSED http://bugs.python.org/issue1701    created  hjmjohnson               
                                                                               

Word "alias" used in confusing way to compare open() and file()  12/27/07
       http://bugs.python.org/issue1702    created  Devin Jeanpierre         
                                                                               

getpass broken in Py3k: must flush()                             12/28/07
       http://bugs.python.org/issue1703    created  pjenvey                  
                                                                               

possible bug in randint                                          12/28/07
       http://bugs.python.org/issue1704    created  cephalo                  
                                                                               

trace module does not annotate global statement                  12/28/07
       http://bugs.python.org/issue1705    created  calvin                   
                                                                               

Force WINVER 0x0500 (Windows 2000)                               12/28/07
       http://bugs.python.org/issue1706    created  tiran                    
                                                                               



Issues Now Closed (9)
_____________________

IDLE - configDialog - new layout for key config                    40 days
       http://bugs.python.org/issue1457    kbk                      
       patch                                                                   

Patch for TCL 8.5 support                                          15 days
       http://bugs.python.org/issue1607    tiran                    
       py3k, patch                                                             

IDLE: help() displays output on the wrong shell                    10 days
       http://bugs.python.org/issue1650    kbk                      
                                                                               

pythonw.exe crashes when run in one particular account on Window    4 days
       http://bugs.python.org/issue1674    kbk                      
                                                                               

feature request: methods that modify instances should return ins    0 days
       http://bugs.python.org/issue1691    tiran                    
                                                                               

time.localtime() docstring has a wrong fieldname                    0 days
       http://bugs.python.org/issue1695    brett.cannon             
                                                                               

Problems with float and "6" type                                    0 days
       http://bugs.python.org/issue1697    tiran                    
                                                                               

svn checkout does not work                                          0 days
       http://bugs.python.org/issue1701    loewis                   
                                                                               

IDLE invokes completion even when running code                    461 days
       http://bugs.python.org/issue1563981 kbk                      
                                                                               



Top Issues Most Discussed (4)
_____________________________

  6 urlparse and usernames containing @                                3 days
open    http://bugs.python.org/issue1698   

  5 [patch] epoll and kqueue wrappers for the select module            9 days
open    http://bugs.python.org/issue1657   

  4 CGIHTTPServer does not chdir prior to executing the CGI script     7 days
open    http://bugs.python.org/issue1684   

  3 Patch for TCL 8.5 support                                         15 days
closed  http://bugs.python.org/issue1607   




From jwdevel at gmail.com  Sun Dec 16 03:40:26 2007
From: jwdevel at gmail.com (j w)
Date: Sat, 15 Dec 2007 18:40:26 -0800
Subject: [Python-Dev] msvccompiler and .NET sdk version
Message-ID: <fa8771800712151840g79ccac53kfd27bd44fbb13cf2@mail.gmail.com>

Maybe this is fixed somewhere already, but just fyi:

I have the .NET sdk 2.0 installed (but not 1.1)
I'm running ActivePython 2.5.1

When I was building Mercurial from sources, I got this error:
    error: Python was built with Visual Studio 2003;
    extensions must be built with a compiler than can generate
compatible binaries.
    Visual Studio 2003 was not found on this system. If you have
Cygwin installed,
    you can try compiling with MingW32, by passing "-c mingw32" to setup.py.

This is in spite of the fact that I had DISTUTILS_USE_SDK and MSSDK
set properly.

The problem was that msvccompiler.py was looking for
sdkinstallrootv1.1, and bailing when it couldn't be found.
I made a small change and it works fine for me now.
In load_macros, I put this:

if version > 7.0:
    try:
        self.set_macro("FrameworkSDKDir", net, "sdkinstallrootv1.1")
    except KeyError, exc:
        self.set_macro("FrameworkSDKDir", net, "sdkinstallrootv2.0")

I had no problems with it - thought it might be useful for someone else.

I do however think that if I have DISTUTILS_USE_SDK set it shouldn't
be demanding registry entries of me in the first place.
But load_macros gets called in the constructor of MacroExpander before
the enclosing MSVCCompiler checks for the env vars, which seems not
quite right.

I don't use the VStuido installers so much, I just rearrange my
environment as needed so I'm used to being bitten by these registry
checks and such. For me, and other developers who need to switch
between toolsets (VC98, 2003, 2005, etc), running installers each time
that stomp on each others' registry entries and install different SDKs
is a major pain.
In this view of things, the original error message could be seen as
slightly misleading as well. I would say that Visual Studio 2003 could
in fact be found on my system, just not .NET sdk 1.1. If you ran the
installer there's no distinction, but in my case there is.

-John

From skip.montanaro at gmail.com  Sat Dec 22 23:57:59 2007
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Sat, 22 Dec 2007 16:57:59 -0600
Subject: [Python-Dev] svn.python.org ?
In-Reply-To: <fkk16t$61c$1@ger.gmane.org>
References: <476D5FC5.8010906@gmail.com> <fkk16t$61c$1@ger.gmane.org>
Message-ID: <60bb7ceb0712221457w62885503ude9759e1ff05247b@mail.gmail.com>

I'm unable to get the (apprently external?) 2to3 to update:

  % svn up

  Fetching external item into 'Tools/2to3'
  svn: PROPFIND request failed on '/projects/sandbox/trunk/2to3'
  svn: PROPFIND of '/projects/sandbox/trunk/2to3': could not connect
to server (http://svn.python.org)

Something seems amiss.

Skip

From el at prans.net  Sun Dec 23 00:51:44 2007
From: el at prans.net (Elvis Pranskevichus)
Date: Sat, 22 Dec 2007 18:51:44 -0500
Subject: [Python-Dev] svn.python.org ?
References: <476D5FC5.8010906@gmail.com>
Message-ID: <E1J6E8A-0001QL-5y@asgard.prans.org>

Joseph Armbruster wrote:

> 
> Is svn.python.org ok?  I am unable to perform an update at the moment.
> 
> Joseph Armbruster

Looks like the httpd on svn.python.org is down:

telnet svn.python.org 80
Trying 82.94.237.220...
telnet: connect to address 82.94.237.220: Connection refused

SSH is up, though, hence the difference for those using svn+ssh.

Thanks,

             Elvis


From aferdowsi at gmail.com  Sun Dec 23 13:37:49 2007
From: aferdowsi at gmail.com (arashf)
Date: Sun, 23 Dec 2007 04:37:49 -0800 (PST)
Subject: [Python-Dev] 2.5.2 is coming
In-Reply-To: <ee2a432c0710302319h31b45446l8672a617028f6fc1@mail.gmail.com>
References: <ee2a432c0710112232u658f0addl91c391c4f5b0c1d3@mail.gmail.com> 
	<e04bdf310710240722m2b1a5ea5p3168a5e4caa8aaf4@mail.gmail.com> 
	<ee2a432c0710302319h31b45446l8672a617028f6fc1@mail.gmail.com>
Message-ID: <6588377c-a596-4e7c-b6de-893124545ec5@s8g2000prg.googlegroups.com>

any updates here?

On Oct 30, 10:19 pm, "Neal Norwitz" <nnorw... at gmail.com> wrote:
> On Oct 24, 2007 7:22 AM, Facundo Batista <facundobati... at gmail.com> wrote:
>
> > 2007/10/12, Neal Norwitz <nnorw... at gmail.com>:
>
> > > The plan is cut the release candidate around Tuesday/Wednesday next
> > > week (Oct 16/17).  If all goes well,2.5.2final will follow a week
> > > later.
>
> > Hi Neal! Do you have any update of this schedule?
>
> Sorry, the various schedules of everyone involved didn't allow us to
> get a release out as planned.  We are hoping to get out a2.5.2
> release candidate in a couple of weeks.
>
> n
> _______________________________________________Python-Dev mailing listPython-... at python.orghttp://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:http://mail.python.org/mailman/options/python-dev/python-dev2-garchiv...

From lists at cheimes.de  Sun Dec 30 20:26:09 2007
From: lists at cheimes.de (Christian Heimes)
Date: Sun, 30 Dec 2007 20:26:09 +0100
Subject: [Python-Dev] My pending patches
Message-ID: <4777F0D1.6000305@cheimes.de>

Good evening!

I've three patches in the bug tracker which are ready but I like to get
 approval before I submit them.

http://bugs.python.org/issue1657
Wrapper for epoll and kqueue sys calls for Python 2.6 and 3.0. I've
worked together with therve from the Twisted team.

http://bugs.python.org/issue1640
Enhancements for the math module. The latest patch just adds isnan(),
isinf() and sign() to the math module.

http://bugs.python.org/issue1567
The patch adds a new API function _PyImport_ImportModuleNoLock(char
*name). It first tries to get the module from sys.modules and falls back
to PyImport_ImportModule unless the import lock is hold. In the latter
case it raises an exception to prevent a dead lock. It solves a problem
with imports from C code and threads.

Christian

From lists at cheimes.de  Mon Dec 31 14:03:47 2007
From: lists at cheimes.de (Christian Heimes)
Date: Mon, 31 Dec 2007 14:03:47 +0100
Subject: [Python-Dev] Fate of Windows build directories
Message-ID: <4778E8B3.3090004@cheimes.de>

Good afternoon fellow developers!

We have finished the support for VS 2008 more or less successfully.
Python 3.0a2 was build with VS 2008 and shipped out without major
issues. The Tcl/Tk 8.5 problem was solved by Amaury and Martin is
working on the integration of the C runtime library.

What should happen to the other build directories?

PC/VS6/
  build dir for Visual Studio 6. It's no longer maintained
  and probably doesn't work at all. I like to remove it.

PC/bdist_wininst/
  The README contains only XXX comments and the project files
  must be ported to VS 2005 and VS 2008. I guess the directory
  contains the executable for distutils' bdist_wininst.

PC/example_nt/
  Example project and C files for VS 2003. Needs an update or remove it.

PC/os2*/
  OS2 build dirs. The files are no longer maintained.

PCbuild/
  Build dir for VS 2003. Keep it or remove it? I'm +1 for removal
  from 3.0 and +0 for remove from 2.6.

PCBuild8/
  Build dir for VS 2005. This directory is a nested directory
  structure and it doesn't integrate all pre-build steps. The user
  weck has developed a small Python script to back port the PCbuild9
  directory to PCbuild8 (http://bugs.python.org/issue1455). The
  script would make maintenance of the PCbuild8 directory easier.

PCbuild9/
  The new build dir for VS 2008.

Christian

From martin at v.loewis.de  Mon Dec 31 14:41:14 2007
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 31 Dec 2007 14:41:14 +0100
Subject: [Python-Dev] Fate of Windows build directories
In-Reply-To: <4778E8B3.3090004@cheimes.de>
References: <4778E8B3.3090004@cheimes.de>
Message-ID: <4778F17A.7040909@v.loewis.de>

> What should happen to the other build directories?
> 
> PC/VS6/
>   build dir for Visual Studio 6. It's no longer maintained
>   and probably doesn't work at all. I like to remove it.

Please keep it. It's not just for VS6, but also for some version
of Embedded VS, and people occasionally contribute to it.

Does it hurt to keep it?

> PC/bdist_wininst/
>   The README contains only XXX comments and the project files
>   must be ported to VS 2005 and VS 2008. I guess the directory
>   contains the executable for distutils' bdist_wininst.

Yes, it's the source of the binaries being packaged into bdist_wininst.

That needs to be updated for VS 2008: projects files need to be created,
and it needs to be built; the build output needs to go into
Lib/distutils/command/wininst-9.exe.

> PC/example_nt/
>   Example project and C files for VS 2003. Needs an update or remove it.

Update.

> PC/os2*/
>   OS2 build dirs. The files are no longer maintained.

Not true, I think. Andrew McIntyre updates them; the last checkin was in
July 2007.

> PCbuild/
>   Build dir for VS 2003. Keep it or remove it? I'm +1 for removal
>   from 3.0 and +0 for remove from 2.6.

Please move it to PC/VS7.1 (or some such).

> PCBuild8/
>   Build dir for VS 2005. This directory is a nested directory
>   structure and it doesn't integrate all pre-build steps. The user
>   weck has developed a small Python script to back port the PCbuild9
>   directory to PCbuild8 (http://bugs.python.org/issue1455). The
>   script would make maintenance of the PCbuild8 directory easier.

As I said in the tracker: I have no opinion, but Kristjan may have one.

> PCbuild9/
>   The new build dir for VS 2008.

Please rename it to PCbuild (after moving the previous one to PC).

Regards,
Martin

From arigo at tunes.org  Mon Dec 31 17:54:05 2007
From: arigo at tunes.org (Armin Rigo)
Date: Mon, 31 Dec 2007 17:54:05 +0100
Subject: [Python-Dev] PyPy Leysin Winter Sprint (12-19th January 2008)
Message-ID: <20071231165405.GA11846@code0.codespeak.net>

Hi all,

The next PyPy sprint will be held in Leysin, Switzerland, for
the fifth time.  The overall idea of the sprint is to continue
working on making PyPy ready for general use.

The proposed topics are: ctypes, JIT, testing, LLVM.  This is
a fully public sprint, so newcomers and other topics are
welcome.  And like previous winters, the main side goal is to
have fun in winter sports :-)

See the sprint announcement for details:

http://codespeak.net/pypy/extradoc/sprintinfo/leysin-winter-2008/announcement.html


Armin

From jayk123 at hotmail.com  Sun Dec 30 00:21:00 2007
From: jayk123 at hotmail.com (Jay)
Date: Sat, 29 Dec 2007 23:21:00 +0000
Subject: [Python-Dev] complaints..
Message-ID: <BAY139-W16EF3F5F04CDC7F9A14488E6560@phx.gbl>

I'm sure you've heard most/all of this before but..it..just..seems..so..true...

Finally this week I've "written" (ported from sh) a bunch of Perl, 2000 sparse lines.
While it sure beats Perl, it has some glaring flaws, more glaring due to
its overall goodness.

I feel compelled to voice my opinion, as if we don't live in a benevolent dictatorship :),
and as if the weight of existing code was zero.
Much of what I dislike cannot be changed without massive breakage.

Below is what i get from:

jbook:/dev2/cm3/scripts/python/flaws jay$ ls -l
total 72
-rw-r--r--   1 jay  admin  834 Dec 29 16:02 1_not_lexically_scoped.py
-rw-r--r--   1 jay  admin  238 Dec 29 16:02 2_reads_scoped_writes_not.py
-rw-r--r--   1 jay  admin  593 Dec 29 16:02 3_lambda_is_neutered.py
-rw-r--r--   1 jay  admin  377 Dec 29 16:03 4_assignment_is_not_expression.py
-rw-r--r--   1 jay  admin  760 Dec 29 16:02 5_for_loop_off_by_one.py
-rw-r--r--   1 jay  admin  412 Dec 29 16:01 6_docs_good_but_a_complaint.txt
-rw-r--r--   1 jay  admin  254 Dec 29 16:02 7_print_is_wierd.py
-rw-r--r--   1 jay  admin  286 Dec 29 16:06 8_eval_vs_exec.py
-rw-r--r--   1 jay  admin  824 Dec 29 16:14 9_typo_on_read_error_but_write_ok.py

jbook:/dev2/cm3/scripts/python/flaws jay$ cat * > all.txt

jbook:/dev2/cm3/scripts/python/flaws jay$ edit all.txt

Each "---" seperates a file and they are executable Python.

 - Jay


#----------------------------------------------------------
# flaw #1
# not lexically scopied
# Functions have their own locals, but other blocks do not.
# This is true both for "normal" variables and lexically nested functions.
#

#
# This is probably largely an outcome of not declaring variables?
#

A = "A1:global"

def F1():
    A = "A1:in first F1"
    print "F1:global"

if True:
    A = "A1:in if"
    def F1():
        A = "A1:in if.F1" # This does the right thing.
        print "F1:in if"
    F1() # This does the right thing.

# This should go to "global" but it goes to "in if"
F1()

def F2():
    A = "A1:in F2"
    def F1():
        A = "A1:in F2.F1"
        print "F1:in F2"

# Given how if behaved, you'd expect this to go to F2.F1 but it does not.
F1()

# This should be "global" but is "in if".
print("A is " + A)

#----------------------------------------------------------
# flaw #2
#
# In functions, reads are scoped. Writes are not.
#

A = "1"

def F1():
    A = "2" # not an error

def F2():
    #B = A # error
    A = "3"
    
F1()
F2()
print(A)

#----------------------------------------------------------
# flaw #3:
#  lambda is neutered
#  It allows just one expression, and no statements
#

# This should work:

import os

#os.path.walk(
#    "..",
#    lambda(a, b, c):
#        print(b)
#        print(b)
#    None)

# Instead I must say:

def Callback(a, b, c):
    print(b)
    print(b)
    
os.path.walk(
    "..",
    Callback,
    None)

#
# Moving callbacks away from their point of use hurts readability.
# This is probably mitigated by lexical scoping of functions, but
# notice that other blocks are not lexically scoped.
#

#----------------------------------------------------------
# flaw #4:
#   assignment is not an expression
#

# This should work (seen in Perl code, nice idiom):

#A = [1,2]
#while (B = A.pop()):
#    print(B)

# instead I must say:

A = [1,2]
while (A):
    B = A.pop()
    print(B)

# Even if you reject popping an empty list, ok
# there are PLENTY of applications of this.

#----------------------------------------------------------
# flaw #5
#
# for loops suck
#
# It should be easy to iterate from 0 to n, not 0 to n - 1,
# thereby knowing why the loop terminated
#

#This should work:

# print the first even number, if there are any

A = [1, 3]
for i in range(0, len(A)):
    if ((A[i] % 2) == 0):
        print("even number found")
        break;
if (i == len(A)):
    print("no even numbers found")
 
# instead I must write:

i = 0
while (i != len(A)):
    if ((A[i] % 2) == 0):
        print("even number found")
        break;
    i += 1
if (i == len(A)):
    print("no even numbers found")

# with the attendent problem of not being able to "continue" ever, since
# the end of the loop body must be run in order to proceed

Flaw #6

The docs are very good.

However the reference doesn't give much in the way
of semantic description.

It is surprising that an experienced programmer MUST
depend on the tutorial or any examples or semantics.
Light on examples, ok for reference.

The language reference is little more than the grammar in parts.

There needs to be links from the reference back to the tutorial.

Perhaps merge the indices for Tutorial, Language Reference, Library Reference.
Or maybe that's what search is for.

On the other hand, way better than Perl.

#----------------------------------------------------------
# Flaw #7
#
# This is a compatibility issue.
# print is both a statement and a function, or something.
#

print() # should print just a newline, but prints two parens

# workaround:

print("")

#----------------------------------------------------------
# flaw #8
#
# Having to eval expressions but exec statements feels wrong.
# There should just be eval.
#

# eval("print(1)") # error
exec("print(1)") # not an error

exec("1 + 2") # not an error?
eval("1 + 2") # not an error

#----------------------------------------------------------
# flaw #9
#
# Python protects me, via early checking, from typos
# when I read, but not when I write. It should do both.
# That is, I should declare variables.
#

Correct = 1
# proposed typo is Corect

#A = Corect # error, good
Corect = 1 # no error, bad

# For classes, by default, same thing:

class Foo(object):
    def __init__(self):
        self.Correct =1

F = Foo()
# print(F.Corect) # error, good
F.Corect = 1 # no error, bad

# __slots__ fixes this, but is not the default

class Bar(object):
    __slots__ = ["Correct"]
    def __init__(self):
        self.Correct =1

B = Bar()
# print(B.Corect) # error, good
#B.Corect = 1 # error, good

# __slots__ should be the default and then some other syntax
# for "expandable" types, like __slots__ = [ "*" ]


_________________________________________________________________
Get the power of Windows + Web with the new Windows Live.
http://www.windowslive.com?ocid=TXT_TAGHM_Wave2_powerofwindows_122007
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071229/dca11e4e/attachment-0001.htm 

From jquinn at cs.oberlin.edu  Mon Dec 31 07:04:12 2007
From: jquinn at cs.oberlin.edu (Jameson "Chema" Quinn)
Date: Mon, 31 Dec 2007 00:04:12 -0600
Subject: [Python-Dev] Extend reST spec to allow automatic recognition of
	identifiers in comments?
Message-ID: <faf2c12b0712302204u1764801di767158fef7559bd2@mail.gmail.com>

This is a VERY VERY rough draft of a PEP. The idea is that there should be
some formal way that reST parsers can differentiate (in docstrings) between
variable/function names and identical English words, within comments.

PEP: XXX
Title: Catching unmarked identifiers in docstrings
Version: 0.0.0.0.1
Last-Modified: 23-Aug-2007
Author: Jameson Quinn <firstname dot lastname at gmail>
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 23-Aug-2007
Post-History: 30-Aug-2002


Abstract
========

This PEP makes explicit some additional ways to parse docstrings and
comments
for python identifiers. These are intended to be implementable on their own
or
as extensions to reST, and to make as many existing docstrings
as possible usable by tools that change the visible
representation of identifiers, such as translating (non-english) code
editors
or visual programming environments. Docstrings in widely-used modules are
encouraged to use \`explicit backquotes\` to mark identifiers which are not
caught by these cases.

THIS IS AN EARLY DRAFT OF THIS PEP FOR DISCUSSION PURPOSES ONLY. ALL LOGIC
IS
INTENTIONALLY DEFINED ONLY BY EXAMPLES AND THERE IS NO REFERENCE
IMPLEMENTATION
UNTIL A THERE ARE AT LEAST GLIMMERINGS OF CONSENSUS ON THE RULE SET.


Rationale
=========

Python, like most computer languages, is based on English. This can
represent a hurdle to those who do not speak English. Work is underway
on Bityi_, a code viewer/editor which translates code to another language
on load and save. Among the many design issues in Bityi is that of
identifiers in docstrings. A view which translates the identifiers in
code, but leaves the untranslated identifier in the docstrings, makes
the docstrings worse than useless, even if the programmer has a
rudimentary grasp of English. Yet if all identifiers in docstrings are
translated, there is the problem of overtranslation in either direction.
It is necessary to distinguish between the variable named "variable",
which should be translated, and the comment that something is "highly
variable", which should not.

.. _Bityi: http://wiki.laptop.org/go/Bityi

Note that this is just one use-case; syntax coloring and docstring
hyperlinks are another one. This PEP is not the place for a discussion of
all the pros
and cons of a translating viewer.

PEP 287 standardizes reST as an optional way to markup docstrings.
This includes the possibility of using \`backquotes\` to flag Python
identifiers. However, as this PEP is purely optional, there are many
cases of identifiers in docstrings which are not flagged as such.
Moreover, many of these unflagged cases could be caught programatically.
This would reduce the task of making a module internationally-viewable,
or hyperlinkable, considerably.

This syntax is kept relatively open to allow for reuse with
other programming languages.


Common cases of identifiers in docstrings
=========================================

The most common case is that of lists of argument or
method names. We call these "identifier lists"::

  def register(func, *targs, **kargs):
      """register a function to be executed someday

      func - function to be called
      targs - optional arguments to pass
      kargs - optional keyword arguments to pass
      """

      #func, targs, and kargs would be recognized as identifiers in the
above.

  class MyClass(object):
      """Just a silly demonstration, with some methods:

      thisword     : is a class method and you can call
          it - it may even return a value.

          As with reST, the associated text can have
          several paragraphs.

          BUT - you can't nest this construct, so BUT isn't counted.
      anothermethod: is another method.
      eventhis -- is counted as a method.

      anynumber --- of dashes are allowed in this syntax

      But consider: two words are NOT counted as an identifier.

      things(that,look,like,functions): are functions (see below)

        Also, the docstring may have explanatory text, below or by
      itself: so we have to deal with that.
        Thus, any paragraph which is NOT preceded by an empty line
      or another identifier list - like "itself" above - does not count
      as an identifier.
      """
      #thisword, anothermethod, eventhis, anynumber, and things would be
      #recognized  as identifiers in the above.

Another case is things which look like functions, lists, indexes, or
dicts::

    """
    afunction(is,a,word,with,parentheses)
    [a,list,is,a,bunch,of,words,in,brackets]
    anindex[is, like, a, cross, between, the, above]
    {adict:is,just:words,in:curly, brackets: likethis}
    """
    #all of the above would be recogniszed as identifiers.

The "syntax" of what goes inside these is very loose.
identifier_list ::= [<initial_word>]<opening_symbol> <content_word>
{<separator_symbol> <content_word>} <closing symbol>
, with no whitespace after initial_word, and where separator_symbol is the
set of symbols ".,<>{}[]+-*^%=|/()[]{}" MINUS closing_symbol. content_word
could maybe be a quoted string, too.
In the "function name", no whitespace
is allowed, but the symbols ".,*^=><-" are. Thus::

    """
    this.long=>function.*name(counts, and: so |do| these {so-called]
arguments)
    {but,you - cant|use[nested]brackets{so,these,are.identifiers
}but,these,arent}
    {heres.an.example.of."a string, no identifiers in here",but.out.here.yes
}
    { even.one.pair.of.words.with.no
symbols.means.nothing.here.is.an.identifier}
    Any of these structures that open on one line {but.close.on.
    the.next} are NOT counted as identifiers.
    """
    #in the above: lines 1,2,and the parts of 3 outside the quotes
    #would be recognized as identifiers

The above flexibility is intended to cover the various possibilities for
argument lists in a fair subset of other languages. Languages which use only
whitespace for argument separation are not covered by these rules.

The final case is words that are in some_kind of mixedCase. These are only
optionally counted as identifiers if they are also present as an identifier
OUTSIDE
the comments somewhere in the same file.

Doctest and preformatted reST sections should be considered as 100% python
code and treated as identifiers (or keywords).

Recommended use
===============

The rules above are designed to catch the large majority of identifiers
already present in docstrings, while applying only extremely rarely to words

that should properly be considered as natural language. However, they are
inevitably imperfect. All docstrings of modules intended for wide use should
manually fix all cases in which these rules fail. If the rules underapply,
you can use either \`back quotes\` or parentheses() to mark words as
identifiers; if they overapply and reformatting tricks don't fix the
problem, <SOME DIRECTIVE TO TURN OFF ALL THIS LOGIC FOR A STRING>

Optional use inside comments or non-docstring strings
=====================================================

Comments
--------

Comments or blocks of comments alone on consecutive lines should be able,
optionally, to use these same tricks to spotlight identifiers.

Other strings
-------------

I'm not sure yet what the rules should be here. One option I'm considering
is to be able to turn on all the above logic with some evil hack such
as '' 'a string like this, concatenated with an empty string'.


Copyright
=========

This document has been placed in the public domain.



..
   Local Variables:
   mode: indented-text
   indent-tabs-mode: nil
   sentence-end-double-space: t
   fill-column: 70
   coding: utf-8
   End:
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20071231/c9779675/attachment-0001.htm